Give your AI full control of your computer: OpenClaw + useHID in practice

Imagine telling an AI in plain English: "Open Chrome, search GitHub, then scroll to the bottom." — and watching it actually move your mouse and type. Not sci-fi. Today, with OpenClaw + useHID, this works.

What is useHID?

useHID is a cross-platform virtual HID (Human Interface Device) library built specifically for AI agents. It gives you:

• 🖱️ Virtual mouse — move, click, scroll
• ⌨️ Virtual keyboard — type, press, key combos
• 🎮 Virtual gamepad — sticks, buttons, triggers
• 🤖 Agent-friendly API — JSON in, action out, perfect for LLM tool calls

macOS, Linux, Windows. No special drivers.

Why it exists

Most AI automation tools (think browser-use, Playwright) can only drive a browser. But most real work happens in desktop apps:

• Opening Finder / a file manager
• Operating an IDE to write code
• Driving Photoshop to edit images
• Playing games (yes — AI can play games for you)

useHID fills that gap. The AI can operate anything on your computer, the same way a human does.

Quick start

1. Install

pip install usehid

2. Grant permissions (macOS)

useHID uses the macOS CGEvent API, which needs Accessibility permission:

1. System Settings → Privacy & Security → Accessibility
2. Add your terminal app (Terminal, iTerm2, etc.)

3. Smoke test

from usehid import AgentHID

agent = AgentHID()

# Move the mouse
agent.execute({"action": "mouse_move", "x": 100, "y": 50})

# Click
agent.execute({"action": "mouse_click", "button": "left"})

# Type
agent.execute({"action": "type", "text": "Hello, World!"})

# Combo: Cmd+S to save
agent.execute({"action": "key_combo", "modifiers": ["cmd"], "key": "s"})

Hooking it into OpenClaw

1. Copy the skill

cp -r usehid/skills/usehid ~/.openclaw/skills/

2. Register it

In ~/.openclaw/workspace/SKILLS.md:

## usehid

Control the computer's mouse, keyboard, and gamepad.

Use when the user wants to click, type, scroll, or press keys on the desktop.

Location: ~/.openclaw/skills/usehid/SKILL.md

3. Talk to your computer in plain English

You: Open Spotlight, search for Chrome and launch it.

AI: Got it — opening Chrome.
[Cmd+Space]
[types "Chrome"]
[Enter]
Chrome is open.

You: Focus the address bar, type github.com, hit enter.

AI: On it.
[Cmd+L]
[types "github.com"]
[Enter]
GitHub is loaded.

You: Scroll down a bit.

AI: [scrolls]
Done.

Real-world recipes

Recipe 1 — fill out a form

You: Fill this signup form. Username demo, email [email protected].

AI: Filling it in.
[clicks username field]
[types "demo"]
[Tab to email]
[types "[email protected]"]
Form filled. Want me to submit?

Recipe 2 — open a file

You: Open report.pdf on the desktop.

AI: Opening it.
[Cmd+Space]
[types "report.pdf"]
[Enter]
File open.

Recipe 3 — screenshot

You: Take a screenshot, save to desktop.

AI: [Cmd+Shift+3]
Saved.

Supported actions

Mouse

Action	JSON
Move	{"action": "mouse_move", "x": 100, "y": 50}
Left click	{"action": "mouse_click", "button": "left"}
Right click	{"action": "mouse_click", "button": "right"}
Double click	{"action": "mouse_double_click"}
Scroll	{"action": "mouse_scroll", "delta": -3}
Drag	mouse_down → mouse_move → mouse_up

Keyboard

Action	JSON
Type	{"action": "type", "text": "Hello"}
Press key	{"action": "key_press", "key": "enter"}
Combo	{"action": "key_combo", "modifiers": ["cmd"], "key": "s"}

Modifiers

• ctrl — Control
• shift — Shift
• alt — Alt / Option
• cmd / meta / win — Command (macOS) / Windows key

Special keys

enter, escape, backspace, tab, space, up, down, left, right, home, end, pageup, pagedown, delete, f1–f12

Safety first

⚠️ Letting an AI drive your computer is powerful and risky. A few rules:

1. Confirm destructive operations. Always confirm before deleting files or closing unsaved work.
2. Test in a sandbox first. Don't unleash this on production environments.
3. Set boundaries. Limit which apps the AI can touch.
4. Supervise early. Watch closely the first few weeks.

How it works under the hood

Different backends per platform:

Platform	Mouse/Keyboard	Gamepad
macOS	CGEvent API	IOHIDUserDevice (requires signing)
Linux	uhid (/dev/uhid)	uhid
Windows	SendInput API	ViGEmBus

macOS uses CGEvent rather than IOHIDUserDevice because the latter needs an Apple developer signature, while CGEvent only needs Accessibility permission — much easier for users.

Wrap-up

useHID + OpenClaw opens a new chapter of AI automation. From simple clicks to full multi-app workflows, your AI can now operate your machine the way you do.

Repos:

• useHID: https://github.com/jiusanzhou/usehid
• OpenClaw: https://github.com/openclaw/openclaw

Install:

# Rust
cargo add usehid

# Python
pip install usehid

# Node.js
npm install usehid

# Go
go get go.zoe.im/usehid@latest

Give it a try — your AI desktop assistant is one cargo add away. 🚀

---

Questions or ideas? Open an issue or PR on GitHub.