• Some users have recently had their accounts hijacked. It seems that the now defunct EVGA forums might have compromised your password there and seems many are using the same PW here. We would suggest you UPDATE YOUR PASSWORD and TURN ON 2FA for your account here to further secure it. None of the compromised accounts had 2FA turned on.
    Once you have enabled 2FA, your account will be updated soon to show a badge, letting other members know that you use 2FA to protect your account. This should be beneficial for everyone that uses FSFT.

Token Monitor Tool

FrgMstr

FrgMstr

Just Plain Mean
Staff member
2FA
Joined
May 18, 1997
Messages
58,241
Been working on this. Sneak Peek.

1785660101450.png
 
interesting...

As someone who has had to do a PoC for Claude at work, and seeing their opentelemetry options are limiting, and some mentioning custom MCP servers to really track things...
 
MrGuvernment said:
interesting...

As someone who has had to do a PoC for Claude at work, and seeing their opentelemetry options are limiting, and some mentioning custom MCP servers to really track things...
Click to expand...
This is a tool I have been building for 5 or 6 weeks now. This sits between Hermes and LM Studio currently, but should be extendable to other environments. A bit from the readme.md

"Everything lives in this one folder. Zip it, copy it to a USB stick, hand it to someone, it runs wherever it lands. Python standard library only; nothing to `pip install`.

"What this is — and how it differs from the observability tools.

"Most prompt- and token-observability lives inside **SDK-based platforms** - LangSmith, Langfuse, Helicone, Phoenix, OpenLLMetry and the like. Those work by *instrumenting your application*: you import their SDK, wrap your framework calls, and they read the structured pieces your own code hands them. They know the system-vs-tools-vs-history split because your code told them at construction time.

"This tool works from the opposite side. It's a **transparent reverse proxy** between a closed client (Hermes) and a local server (LM Studio), reconstructing prompt composition from the raw request bodies on the wire — no SDK, no framework hooks, no cooperation from either end. Because LM Studio returns just one number for the whole prompt (`prompt_tokens`), the proxy has no tokenizer to divide that number with. So instead it **calibrates its own tokens-per-character rates, per model, by least-squares regression over your own captured traffic** - and it refuses to call a split "measured" until the data can actually support one. Until then it says so, and falls back to a clearly-badged character-share estimate."
 
FrgMstr said:
This is a tool I have been building for 5 or 6 weeks now. This sits between Hermes and LM Studio currently, but should be extendable to other environments. A bit from the readme.md

"Everything lives in this one folder. Zip it, copy it to a USB stick, hand it to someone, it runs wherever it lands. Python standard library only; nothing to `pip install`.

"What this is — and how it differs from the observability tools.

"Most prompt- and token-observability lives inside **SDK-based platforms** - LangSmith, Langfuse, Helicone, Phoenix, OpenLLMetry and the like. Those work by *instrumenting your application*: you import their SDK, wrap your framework calls, and they read the structured pieces your own code hands them. They know the system-vs-tools-vs-history split because your code told them at construction time.

"This tool works from the opposite side. It's a **transparent reverse proxy** between a closed client (Hermes) and a local server (LM Studio), reconstructing prompt composition from the raw request bodies on the wire — no SDK, no framework hooks, no cooperation from either end. Because LM Studio returns just one number for the whole prompt (`prompt_tokens`), the proxy has no tokenizer to divide that number with. So instead it **calibrates its own tokens-per-character rates, per model, by least-squares regression over your own captured traffic** - and it refuses to call a split "measured" until the data can actually support one. Until then it says so, and falls back to a clearly-badged character-share estimate."
Click to expand...
Nice idea and tool! It's often helpful to allow for display of token budget reset windows, and as always having the means to translate tokens to currency (where applicable) can be useful.
 
Interesting info!

I am still at a stage of "I need to find the time to go down this rabbit hole and get building"

I have a beefy system (HP Z4 G4 Xeon(R) Gold 6230 / 192GB DDR4 ECC) , but I need some GPUs, I wanted to get into a self-hosted home assistant for some basic things at home, lights, camera's et cetera and then sound systems, so was looking at LLM's that can offload to system ram vs just use VRAM only..
 
You must log in or register to reply here.
Back
Top