root
/
OSWorld
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
1,219 Commits 33 Branches 2 Tags 159 MiB
Commit Graph

39 Commits

Author SHA1 Message Date
HappySix 6419d707bc
Support Docker VM manager and provider (#75) 
* Add docker provider framework

* Update VM download link

* Add stop container

* Update docker manager & provider

* Update

* Update

* Update provider
 2024-09-28 21:10:40 +08:00
Tianbao Xie fffa8f8da6
Refactoring VMware Integration and Implementing AWS Support (#44) 
* Initailize aws support

* Add README for the VM server

* Refactor OSWorld for supporting more cloud services.

* Initialize vmware and aws implementation v1, waiting for verification

* Initlize files for azure, gcp and virtualbox support

* Debug on the VMware provider

* Fix on aws interface mapping

* Fix instance type

* Refactor

* Clean

* hk region; debug

* Fix lock

* Remove print

* Remove key_name requirements when allocating aws vm

* Clean README

---------

Co-authored-by: XinyuanWangCS <xywang626@gmail.com>
 2024-06-15 20:52:29 +08:00
rhythmcao c121869219 fix a small bug in computer_13 action space 2024-06-11 14:22:31 +08:00
Timothyxxx 306dcbda71 Add Support for QWEN VL models from API (QWEN-VL-max, etc.); Improve on the robustness of getting observation/files, etc. 2024-05-21 21:08:22 +08:00
Timothyxxx f9594e476e Add Support for QWEN models from API (QWEN-max, etc.); Improve on the robustness of getting observation 2024-05-20 00:47:43 +08:00
Timothyxxx 7ca91ca8c9 Add action execution timeout for corner cases 2024-03-21 11:16:57 +08:00
David Chang 15e01e7ccc
ver Mar20thv2 
fixed bugs in server/main.py (_create_pywinauto_node and
  get_screen_size)
finished migration of a few task configs to Windows
fixed bug in python.py
 2024-03-20 22:22:57 +08:00
Jason Lee 48aedb09a7 add wandb settings, remember to set WANDB_KEY 2024-03-17 22:30:29 +08:00
Timothyxxx 068c6f5769 122324154 2024-02-02 14:36:53 +08:00
BlankCheng 7d2d8c855e Merge main 2024-01-29 21:51:26 +08:00
BlankCheng 284d6fb379 Add human operation time log 2024-01-29 21:42:16 +08:00
Timothyxxx 6952b45de4 Improve on agent and tasks configs 2024-01-26 23:30:04 +08:00
Timothyxxx 186bf2e97c Implement heuristic cutting on the accessibility tree to get the important nodes; Finish accessibility tree text agent 2024-01-16 16:43:32 +08:00
Timothyxxx 1141232d80 Merge remote-tracking branch 'origin/main' 
# Conflicts:
#	desktop_env/controllers/setup.py
 2024-01-15 13:51:11 +08:00
Timothyxxx 24169a65d0 Accomplish the exp scripts v1; Add video recording and trajectory recording of desktop agent; Fix minor bugs 2024-01-15 13:49:48 +08:00
rhythmcao 69b0514f99 fix error in pyautogui.typewrite() 2024-01-14 23:53:31 +08:00
Timothyxxx f153a4c253 Add 'WAIT', 'FAIL', 'DONE' to the action space; Debug basic prompting-based GPT-4 and Gemini agents; Initialize experiments script; 2024-01-14 23:36:19 +08:00
Timothyxxx a1c3e4c294 Finish Chrome example loading v1 2024-01-13 22:56:50 +08:00
rhythmcao d4116458ff 1. fix quote and \ characters in execute_command ; 2. add terminal output text as extra observation ; 3. move get_vm_*() to reset() 2024-01-12 18:09:05 +08:00
Timothyxxx 5a93a32958 Update on Chrome examples; Refactor on logic of controlling 2024-01-12 17:24:47 +08:00
Timothyxxx 820579a5a2 Make up missing getters and metrics; Update VLC scripts; Start to work on Chrome, update examples instructions 2024-01-11 21:27:40 +08:00
Timothyxxx 287876affc Merge remote-tracking branch 'origin/main' 
# Conflicts:
#	desktop_env/evaluators/getters/__init__.py
#	desktop_env/evaluators/metrics/__init__.py
#	requirements.txt
 2024-01-10 23:20:49 +08:00
Timothyxxx 49ece15ac3 VLC v1 finished, improve on instructions, improve on infra 2024-01-10 23:18:30 +08:00
David Chang cebae4b183
Merge branch 'main' into zdy 2024-01-10 22:16:25 +08:00
David Chang cf5d480f44
ver Jan10th 
new Thunderbird task config
 2024-01-10 17:36:59 +08:00
Timothyxxx abcafce750 VLC updates, and some infra bugs fix 2024-01-09 23:14:06 +08:00
David Chang 26b7d9010d
Merge branch 'zdy' 2024-01-05 15:55:41 +08:00
David Chang eeb8a120d6
ver Jan5th 
debugged
 2024-01-05 15:20:47 +08:00
Timothyxxx ab71ebb2ba Initialize VLC getters and metrics, fix some bugs in infra logic, needs to be refactored later on 2024-01-04 17:05:17 +08:00
tsuky_chen c628a4e0e8
debug dragTo 2023-12-13 00:05:43 +08:00
Timothyxxx 2ca36109b5 Initialize evaluation protocols and examples; Implement one kind of eval; Update requirements 2023-12-12 18:10:55 +08:00
Timothyxxx 8818779329 Update compressor for data annotation 2023-12-04 00:51:33 +08:00
Timothyxxx 9471de4768 Fix the implementation of action 13 of computer 2023-12-03 00:59:02 +08:00
Timothyxxx 487fb8005b Improve: fix bugs; add back the cursor in screenshot; add pause in env.step 2023-12-02 22:14:50 +08:00
Timothyxxx 9b214b3d23 Action space thoughts 2023-12-02 18:02:06 +08:00
Timothyxxx 992d8f8fce Refactor with pyautogui 2023-12-02 17:52:00 +08:00
Jing Hua ebb5f1cbc5 add mouse cursor to screenshot 2023-11-30 17:31:46 +08:00
Timothyxxx 3d0d9d7758 Run through gpt_4v agent pipeline 2023-11-29 20:21:57 +08:00
Jing Hua a8aebf5d15 mouse and keyboard controllers for windows and linux 2023-11-08 09:22:43 +08:00