Repository navigation
Replies: 2 comments
|
I’ve tried rule-based setups like this as well, and my experience is that they help a lot, but only as guardrails. The biggest improvement usually comes from making the rules very specific and keeping them close to the domain, not from trying to replace the docs entirely. If the model is still finding ways around the intended path, that usually means the rule is still a bit too broad or too easy to bypass. So I would treat rules as one layer in the stack, together with focused docs and a few explicit tool restrictions. |
|
I have incorporated rules as one of the lowest building blocks in my AI process as well. I also agree they help to defined specific guardrails for preferences in your process and help improve the overall outcome of the iteration. I also feel this is very specific to personal preference. I chose this granular "blocks" because it allows me to re-use them for multiple processes, skills, etc. It can get a bit tricky to update or manage if you don't build a process to keep them updated, iterate on them, etc. |
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
In my AI journey I've increasingly become frustrated with some decisions made by the agent without even bothering to ask or reference almost 110 pieces of documentation detailing our entire system.
I've peppered CLAUDE.md with anti-patterns and references to our documentation as "must read" including our coding standards, but no joy. It loves finding "workarounds" to language used in there an override it by literally saying things like "I knew enough from my training data to not have to reference your documentation"
It ended up bloat. And I had to use its memory feature (that works sometimes) to help overcome some things. Still frustration.
So I explored "rules" with paths. If it hits a certain path, a rule loads (like for my domain aggregates) that directly references my documentation and contains some anti-patterns.
But this wasn't the silver bullet. It used bash with Grep/read inside of it to "overcome" not getting the rule to trigger. So I wrote a pre Tool use hook to deny it access to do so for the folders that should trigger the rules and force the documentation into its context.
It also reason's around the LSP which I think is insane, new rule, new pre Tool use.
With some of the testing I've been doing, it's behaving way better than it did previously, but I'm just waiting for the next little workaround it does to sidestep my documentation or rules.
It might be overkill? Has anyone fiddled with rules yet? In terms of context bloat, it doesn't add much after killing its auto memory and CLAUDE.md file from what I can see.
I've even wiped some skills and went back to basics.
(For context, this is a dotnet solution with about 153 projects, not small by any means)
All reactions