Points at and draws on your screenBeta
When you ask about something on your Mac screen, Speechy's Agent marks the real thing: a circle on a button, an arrow, a spotlight, or the answer pinned beside a shape. Move the pointer around what you mean while you talk and it knows what “this” is. Its drawings follow the page as you scroll.
How to use Points at and draws on your screen
Ask where or what
“Where is”, “what's this” or “help me find” all work.
It marks the real thing
A hand-drawn circle, box, arrow or underline lands on it.
Point at what you mean
Circle something with the pointer as you talk and it reads that part up close.
Answers pinned in place
Solve something and the answer sits beside its shape.
What you ask, what it shows
Xcode
You ask
“Help me find the Run button”
On your screen
A circle on the Run button itself, and one short sentence.
Chrome
You ask
“What's the answer for the red area?”
On your screen
“9.” The red triangle is shaded, the answer is pinned beside it, and one line says how.
Safari
You ask
“Circle a paragraph with the pointer and ask “what does this mean?””
On your screen
It reads that part up close and explains just that.
Questions
No. Pointing and drawing never click or type. It only uses your Mac when you say yes to it taking control.
Drawings are pinned to what they mark, so they move with the page and hide when it scrolls out of sight. This needs Accessibility permission.
Press ⌘ Esc. That ends the Agent and clears its marks and drawings.
speechyv2