903ed36715
Add Claude Sonnet 4.5 support and improve action handling ( #362 )
...
🤖 Generated with [Claude Code](https://claude.com/claude-code )
Co-authored-by: Claude <noreply@anthropic.com>
2025-11-14 13:54:32 +08:00
7418f5cf2f
chore: add traceback import for enhanced error handling
...
- Introduced the traceback module to improve error reporting and debugging capabilities.
- Ensured that existing code logic remains unchanged while preparing for future enhancements.
2025-08-12 05:15:54 +00:00
bd6efcfc4d
fix: enhance screenshot retrieval in PythonController
...
- Added a static method to validate image responses for PNG and JPEG formats using magic bytes.
- Improved error handling in the get_screenshot method to log invalid payloads and retry attempts.
- Updated the requests call to include a timeout for better reliability.
2025-08-10 14:40:18 +00:00
b968155757
CoACT initialize ( #292 )
2025-07-31 10:35:20 +08:00
6419d707bc
Support Docker VM manager and provider ( #75 )
...
* Add docker provider framework
* Update VM download link
* Add stop container
* Update docker manager & provider
* Update
* Update
* Update provider
2024-09-28 21:10:40 +08:00
fffa8f8da6
Refactoring VMware Integration and Implementing AWS Support ( #44 )
...
* Initailize aws support
* Add README for the VM server
* Refactor OSWorld for supporting more cloud services.
* Initialize vmware and aws implementation v1, waiting for verification
* Initlize files for azure, gcp and virtualbox support
* Debug on the VMware provider
* Fix on aws interface mapping
* Fix instance type
* Refactor
* Clean
* hk region; debug
* Fix lock
* Remove print
* Remove key_name requirements when allocating aws vm
* Clean README
---------
Co-authored-by: XinyuanWangCS <xywang626@gmail.com>
2024-06-15 20:52:29 +08:00
c121869219
fix a small bug in computer_13 action space
2024-06-11 14:22:31 +08:00
306dcbda71
Add Support for QWEN VL models from API (QWEN-VL-max, etc.); Improve on the robustness of getting observation/files, etc.
2024-05-21 21:08:22 +08:00
f9594e476e
Add Support for QWEN models from API (QWEN-max, etc.); Improve on the robustness of getting observation
2024-05-20 00:47:43 +08:00
7ca91ca8c9
Add action execution timeout for corner cases
2024-03-21 11:16:57 +08:00
15e01e7ccc
ver Mar20thv2
...
fixed bugs in server/main.py (_create_pywinauto_node and
get_screen_size)
finished migration of a few task configs to Windows
fixed bug in python.py
2024-03-20 22:22:57 +08:00
48aedb09a7
add wandb settings, remember to set WANDB_KEY
2024-03-17 22:30:29 +08:00
068c6f5769
122324154
2024-02-02 14:36:53 +08:00
7d2d8c855e
Merge main
2024-01-29 21:51:26 +08:00
284d6fb379
Add human operation time log
2024-01-29 21:42:16 +08:00
6952b45de4
Improve on agent and tasks configs
2024-01-26 23:30:04 +08:00
186bf2e97c
Implement heuristic cutting on the accessibility tree to get the important nodes; Finish accessibility tree text agent
2024-01-16 16:43:32 +08:00
1141232d80
Merge remote-tracking branch 'origin/main'
...
# Conflicts:
# desktop_env/controllers/setup.py
2024-01-15 13:51:11 +08:00
24169a65d0
Accomplish the exp scripts v1; Add video recording and trajectory recording of desktop agent; Fix minor bugs
2024-01-15 13:49:48 +08:00
69b0514f99
fix error in pyautogui.typewrite()
2024-01-14 23:53:31 +08:00
f153a4c253
Add 'WAIT', 'FAIL', 'DONE' to the action space; Debug basic prompting-based GPT-4 and Gemini agents; Initialize experiments script;
2024-01-14 23:36:19 +08:00
a1c3e4c294
Finish Chrome example loading v1
2024-01-13 22:56:50 +08:00
d4116458ff
1. fix quote and \ characters in execute_command ; 2. add terminal output text as extra observation ; 3. move get_vm_*() to reset()
2024-01-12 18:09:05 +08:00
5a93a32958
Update on Chrome examples; Refactor on logic of controlling
2024-01-12 17:24:47 +08:00
820579a5a2
Make up missing getters and metrics; Update VLC scripts; Start to work on Chrome, update examples instructions
2024-01-11 21:27:40 +08:00
287876affc
Merge remote-tracking branch 'origin/main'
...
# Conflicts:
# desktop_env/evaluators/getters/__init__.py
# desktop_env/evaluators/metrics/__init__.py
# requirements.txt
2024-01-10 23:20:49 +08:00
49ece15ac3
VLC v1 finished, improve on instructions, improve on infra
2024-01-10 23:18:30 +08:00
cebae4b183
Merge branch 'main' into zdy
2024-01-10 22:16:25 +08:00
cf5d480f44
ver Jan10th
...
new Thunderbird task config
2024-01-10 17:36:59 +08:00
abcafce750
VLC updates, and some infra bugs fix
2024-01-09 23:14:06 +08:00
26b7d9010d
Merge branch 'zdy'
2024-01-05 15:55:41 +08:00
eeb8a120d6
ver Jan5th
...
debugged
2024-01-05 15:20:47 +08:00
ab71ebb2ba
Initialize VLC getters and metrics, fix some bugs in infra logic, needs to be refactored later on
2024-01-04 17:05:17 +08:00
c628a4e0e8
debug dragTo
2023-12-13 00:05:43 +08:00
2ca36109b5
Initialize evaluation protocols and examples; Implement one kind of eval; Update requirements
2023-12-12 18:10:55 +08:00
8818779329
Update compressor for data annotation
2023-12-04 00:51:33 +08:00
9471de4768
Fix the implementation of action 13 of computer
2023-12-03 00:59:02 +08:00
487fb8005b
Improve: fix bugs; add back the cursor in screenshot; add pause in env.step
2023-12-02 22:14:50 +08:00
9b214b3d23
Action space thoughts
2023-12-02 18:02:06 +08:00
992d8f8fce
Refactor with pyautogui
2023-12-02 17:52:00 +08:00
ebb5f1cbc5
add mouse cursor to screenshot
2023-11-30 17:31:46 +08:00
3d0d9d7758
Run through gpt_4v agent pipeline
2023-11-29 20:21:57 +08:00
a8aebf5d15
mouse and keyboard controllers for windows and linux
2023-11-08 09:22:43 +08:00
Qichen Fu
Timothyxxx
Linxin Song
HappySix
Tianbao Xie
rhythmcao
Timothyxxx
David Chang
Jason Lee
BlankCheng
tsuky_chen
Jing Hua