Local LLMs are great. They are very efficient, and with API costs rising and AI tools becoming more expensive to use, native LLMs are the way to go. I have been Use Ollama to run the Qwen 2.5 encoder and it’s pretty good. The model can build things, fix bugs, and understand my requests almost like a Claude model, but of course it has its limitations. This is limited by my hardware and the number of settings it has. It’s a 7 billion parameter model, and the context window I can run natively is even smaller.
Not very good at complex tasks and sometimes gets stuck and keeps trying the same thing over and over again without ever finding an answer. So I recently decided to help the little man. Now, sometime my local LLM stuck, he can call the most capable model out there, Claude Fable 5, and it allowed him to solve problems that he couldn’t solve even with my intervention.
Gives Gwen an escape hatch
Tale 5 worked wonders on my local setup
I could just replace Gwen with Fable 5, but that would defeat the purpose of running a native model. Qwen 2.5 Coder handles most of my regular coding tasks without requiring an API call, and I still want those tasks to stay local. Calling Fable for every request would increase my overhead, send more of my code to the cloud, and leave much of my on-premise hardware unused.
Instead, I kept Gwen as the base model and gave her access to Fable 5 as a tool. Gwen still receives the original request, checks out the project, edits files, and executes any commands. Fable enters the process only when Gwen tries to solve the problem and fails to make further progress. This allows the small model to return without handing the entire task over to Clod from the beginning.
Delivery is also more focused than starting a separate conversation with Fable. When Gwen needs help, she creates a short report that includes the problem, the relevant code, the approaches she’s already tried, and the current error output. Fable examines this information and returns a possible diagnosis or solution. Her response then returns to the same Ollama session, allowing Gwen to apply the recommendation and continue working with the context that already exists.
Establishing a connection between Being and Story 5
Olama now supports tool calling
Integrating both models turned out to be simpler than I originally expected because Ollama already supports tool calling. All I had to do was create a small Python function that made a request to Fable 5 via Anthropic’s API, then provide that function to Qwen as a tool. This requires an Anthropic API key.
I started by installing the Ollama and Anthropic Python libraries. The actual function receives four bits of information from Qwen—the problem it’s trying to solve, the approaches it’s already tried, the current error output, and any relevant code or project context. It combines everything into a focused request and sends it to Fable 5 using Anthropic’s Messages API.
import os
from anthropic import Anthropic
claude = Anthropic(api_key=os.environ("ANTHROPIC_API_KEY"))
def ask_fable(
problem: str,
attempts: str,
error_output: str,
relevant_context: str,
) -> str:
"""Ask Fable 5 for help with an unresolved coding problem."""
prompt = f"""
Problem:
{problem}
Attempts already made:
{attempts}
Current error:
{error_output}
Relevant context:
{relevant_context}
Identify the cause and suggest a concrete solution.
Do not repeat approaches that have already failed.
"""
response = claude.messages.create(
model="claude-fable-5",
max_tokens=4000,
messages=({"role": "user", "content": prompt}),
)
return "\n".join(
block.text for block in response.content
if block.type == "text"
)The final step was to pass this function to Gwen via Ollama’s tools parameter. Casting converts a function and its arguments into a tool definition that the model can understand.
from ollama import chat
response = chat(
model="qwen2.5-coder:7b",
messages=messages,
tools=(ask_fable),
)
Each time Gwen calls the tool, the Python script executes the function, sends the provided context to Fable, and adds its response to the ongoing Ollama conversation. Gwen can then use that response when continuing the original task locally. At this point the connection works, but Gwen still needs clear rules for deciding when a problem merits an API call.
Knowing when to call Fable is the hard part
I have added clear conditions to the system request
Giving Gwen access to Fable was easy, but deciding when she would use that access took more work. A smaller model doesn’t always accept being stuck. It can keep changing the same function, re-running the same test, and producing slightly different versions of the already failed approach. If I just tell him to ask Fable when he needs help, it can also increase the usual problems he can solve locally.
Instead of leaving the decision entirely up to Gwen, I added clear conditions to the system request. He must attempt the task first, and he can only invoke Fable after two materially different approaches have failed. If Gwen hasn’t really changed her approach, a single retry failure isn’t enough. A tool should become an option only when the model cannot explain the error, identify another reasonable solution, or make measurable progress.
Call ask_fable only when:
1. Two materially different solutions have failed.
2. The same error remains after both attempts.
3. You cannot identify another reasonable approach.
Before calling the tool, collect the exact error, relevant code,
failed approaches, and expected behaviour.
Information sent during an escalation is just as important as time. Shipping the entire project will consume more tokens, increase the API call cost, and expose unnecessary files to Fable. Gwen instead sends the relevant functions, the exact error output, a brief description of the expected result, and a summary of what she’s already tried. This gives Fable enough context to diagnose the blocker without loading the full repository.
Tale 5 is really something
Anthropic recently relaunched Fable 5 globally after considerable controversy when the US government asked the company to restrict access for non-US users. The new model is one of the best available, if not the best. OpenAI’s new Sol models are coming, and Moonshot AI’s latest model, Kimi 3, is also pretty good, though Fable 5 remains the best modelespecially for technical use cases.
