I've had a fairly thorough exploration of how I can give my agent a whiteboard so we can work on architectures together. To my surprise, current solutions (including Excalidraw) were not good enough and didn't deliver what I wanted. I ended up finding Mermaid to be the most agent-friendly medium and coded an Obsidian plugin for it. It works okay; we can work on the same doc while I bring my own AI agent, and we can brainstorm together.
https://community.obsidian.md/plugins/mermaid-relay

YMMV, but the value I get from producing a diagram is derived from the thinking. Thinking about what I'm trying to draw leads to understanding about where my/my team's knowledge is poorer, what assumptions we're making, etc.

Excalidraw offers their own open source first party MCP endpoint[0] and server[1]:[0]https://mcp.excalidraw.com[1]https://github.com/excalidraw/excalidraw-mcp

Might actually make my rough Excalidraw layouts useful for scaffolding. The live canvas interaction is a neat touch.