Question about LLM Local deploy with copilot harness...

Hi everyone, I don't know if anyone has encountered this...

So I put my local LLM into my existing VSCode... and it starts to load all the tools, skills, instructions... etc (about 160-ish tools) into system prompt and causing my local LLM takes about 30s/60K context to process them - Is there any way to let VSCode only send a list of tools and auto inject them when using them?

I'm running llama.cpp as backend.