Today I realized an issue on my website with the embedded widget from cal.com that I use for people contacting me and since I was not in the mood to research on my own, I asked Claude to debug it for me.
The assumption
Claude quickly assumed the CSP in place being the culprit. It asked me to povide some console messages, and from there, it looped into changing hostnames and some other code. It pointed to caching and got lost in details about too aggressive caching. None of this had any impact on the bug itself.
The real issue and the way to a solution
After spending lots of tokens on useless stuff, I stopped it and in a new context window asked it to check on the error (a server error 500 shown inside the iframe widget) again. This time, we made progress: It tried to verify the error and realized after about 5 minutes and dozens of tests that it couldn’t reproduce the error. This was interesting.
A couple of minutes later I found out why and the solution was near: AI only uses Chromium based tests and I was seeing the error in Firefox. AI admitted this after asking but didn’t make any attempt to reproduce this using Firefox (of course it’s technically possible).
So I found out it’s all about the Tracking Protection that blocked several features cal.com is using and because it was blocked, they’re throwing an error 500. In addition, they serve a console log message with completely useless, falsy information inside (wrong hostnames, urls) — this one confused me and the AI massively.
In Chrome, everything works as expected. In Firefox people were seeing a nasty server error. So now we needed to build a solution around it for when the widget content isn’t loading we show a fallback message to open the calendar in a new window instead.
It had nothing to do with CSP and nothing with caching. Time lost because of an opinionated AI research that didn’t manage to interpret console errors correctly in its context. This shouldn’t be a generic AI rant, it helped me a lot in other cases, but perfectly shows why AI has its own opinionated issues, and is not always better than humans in debugging and coding.