bdambrosio/Cognitive_workbench

Get the API version information from the OSWorld server. Returns API version, server version, and protocol.

Clear or swap equipped item. Returns success/failure

Flatten Collection to single Note

Count words in text. Use to determine length of a document, e.g. to determine if it needs to be summarized before further use

Evaluate complex text criteria on each item in a Collection and return a new Collection of matching items. Use to identify relevant items in a Collection

Apply operation to each item in Collection

Debug tool: Get raw local_grid from world_state

Visible item entities within radius R (default/max 7), filtered by forward cone and line-of-sight.

Generates zoomable HTML visualization of the spatial map showing walkability, hazards, resources, and observation confidence.

Open block-based UI (crafting table, chest, furnace). Returns success/failure

Labels a coordinate with a waypoint name for reasoning about spatial relationships. Use to mark important locations for navigation and planning

Extract single field value from each Note in Collection

Get the current observation from the OSWorld environment. Returns screenshot (base64 PNG) and accessibility tree (JSON).

Queries the persistent spatial map representing everything the agent currently knows about the environment (from past and present observations). Useful for reasoning, explanation, and planning.

Transform Note structure into Collection

Reset embodiment after death - lifecycle/recovery. Returns ok

Synchronous wait for 1 second using time.sleep. Useful for timing delays or pacing actions

Stop all movement - cancel embodied motion. Critical for safety, interrupts, reflection pauses

Evaluate complex text criteria on each item in a Collection and return a new Collection of matching items. Use to identify relevant items in a Collection

Reset the OSWorld environment. Soft mode clears step counter only. Hard mode calls DesktopEnv.reset() to revert to snapshot state.

Filter Collection by field conditions (SQL WHERE)

Generate new text or code content from scratch using natural language prompt via LLM. Use to collect thoughts or generate a coherent statement on a topic

Check if text matches regex pattern or contains substring

Search Obsidian notes using Obsidian Local REST API. Returns Collection of JSON Notes with fields text, metadata.uri (alias: source_url), metadata.domain, format, char_count

Attempt exactly ONE adjacent climb (+1Y) forward. Atomic navigation operation suitable for composition.

Generates interactive 3D HTML visualization of the session-local occupancy grid. Opens in browser.

Build / modify world - place blocks. ASYNCHRONOUS. Returns request acceptance status

Check if a number is positive

Extract markdown document structure and content from mixed/embedded text (NOT for converting plain text TO markdown)

Close currently open UI. Returns success/failure

Check if text contains a question

Fetch all text from URL or base64 PDF. Collection-aware (extracts first item if given Collection). Auto-detects format (PDF/HTML/MD/TXT) and extracts complete text content

Visible entities within radius R (default/max 7), filtered by view cone and line-of-sight.