> For AI agents: the complete documentation index is available at /llms.txt, the full documentation bundle is available at /llms-full.txt.
# Midscene MobileWorld Benchmark Report
This is Midscene's test report for the MobileWorld benchmark. In this run, Midscene evaluated only 117 `GUI-ONLY` tasks and achieved **Pass\@1 78.63% (92/117)**.
:::info About MobileWorld
[MobileWorld](https://github.com/Tongyi-MAI/MobileWorld) is an Android Agent benchmark designed for realistic mobile scenarios. It covers cross-app workflows, long-horizon tasks, user interaction, and tool-augmented tasks, with reproducible Android environments used to validate Agent execution results.
:::
## Run configuration
| Field | Value |
| --------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Test date | 2026-07-28 |
| Model Name | `Gemini-3.6-Flash` |
| Midscene version | `1.10.3` |
| Device | `DockerEmulator` |
| Task scope | `GUI-ONLY` tasks only |
| Number of tasks | 117 |
| `MIDSCENE_REPLANNING_CYCLE_LIMIT` | 50 |
| MobileWorld setup | MobileWorld used a Midscene benchmark adapter that delegates Agent execution through Midscene RPC while retaining MobileWorld task setup and validation. |
| Validation notes | A MobileWorld validator was aligned with the task intent. The affected case is listed below. |
## Validation Condition Updates
The following MobileWorld validation check was adjusted for this benchmark:
| Change | Affected case |
| -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------- |
| The previous validator required a full five-day interval between `created_at` and `expires_at`. For example, if the start date is the 16th, “five days later” is the 21st; however, differences in the converted time of day can make the interval shorter than 5 × 24 hours, causing the previous rule to fail. The validator now accepts intervals longer than 4 days and no longer than 5 days. | `MastodonNewFilterTask` |
## Report files
The detailed reports for all 117 `GUI-ONLY` tasks are listed below. Each report is a compressed, self-contained HTML file. Select “report” to open the corresponding execution trace in a new page.
Round 1 (117 reports · 92 PASS · 25 FAIL)
| # | Task | Status | Report |
| --- | ---------------------------------------- | -------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| 1 | AcceptMeetingTask | PASS | report |
| 2 | AdjustBrightnessMaximumTask | PASS | report |
| 3 | AdjustBrightnessMinimumTask | PASS | report |
| 4 | AdjustFontIconMaximumTask | PASS | report |
| 5 | AdjustFontIconMinimumTask | PASS | report |
| 6 | BidFileRenameTask | PASS | report |
| 7 | CVEmailTask | PASS | report |
| 8 | CancelMeetingTask | PASS | report |
| 9 | CartInfoNotificationTask | PASS | report |
| 10 | CartManagementTask | FAIL | report |
| 11 | ChangeWallpaperTask | PASS | report |
| 12 | CheckCartPriceTask | PASS | report |
| 13 | CheckConferenceAndSendSmsTask1 | PASS | report |
| 14 | CheckConferenceAndSendSmsTask2 | FAIL | report |
| 15 | CheckConferenceDurationTask | PASS | report |
| 16 | CheckConferenceLocationTask | FAIL | report |
| 17 | CheckDeduplicatedEventsTask | PASS | report |
| 18 | CheckDepartTimeTask | PASS | report |
| 19 | CheckEventTimeTask | PASS | report |
| 20 | CheckGithubInfoTask | PASS | report |
| 21 | CheckInterviewTimesTask | FAIL | report |
| 22 | CheckInvoiceTask1 | PASS | report |
| 23 | CheckInvoiceTask2 | PASS | report |
| 24 | CheckInvoiceTask3 | PASS | report |
| 25 | CheckPuchasedItem | PASS | report |
| 26 | CheckRegistrationTask | PASS | report |
| 27 | CheckSetMeetTimeTask | PASS | report |
| 28 | ChromeSearchBeijingWeatherTask | PASS | report |
| 29 | CloseFlightModeTask | PASS | report |
| 30 | CountFileLinesTask | PASS | report |
| 31 | DownloadSendReceiptTask | PASS | report |
| 32 | GoogleMapsAlibabaPhoneContactTask | PASS | report |
| 33 | GoogleMapsAlibabaSouthNeighborTask | PASS | report |
| 34 | GraduationMassEmailTask | PASS | report |
| 35 | InvoiceReceiptCopyAskUserTask | FAIL | report |
| 36 | InvoiceReceiptCopyTask | PASS | report |
| 37 | ItemCheckoutTask | PASS | report |
| 38 | LocalFileManagementTask | PASS | report |
| 39 | LocalFileManagementTask2 | PASS | report |
| 40 | MastodonAddBookmarkTask | PASS | report |
| 41 | MastodonAddFeaturedHashtagsTask | FAIL | report |
| 42 | MastodonAdjustTootsTask | PASS | report |
| 43 | MastodonCalendarMultiMemosTask | FAIL | report |
| 44 | MastodonChangeHeaderTask | PASS | report |
| 45 | MastodonChangeLanguageTask | FAIL | report |
| 46 | MastodonConditionalFavoTask | PASS | report |
| 47 | MastodonCreateListTask | PASS | report |
| 48 | MastodonCreateMemoTask | PASS | report |
| 49 | MastodonExportFollowsTask | PASS | report |
| 50 | MastodonFavoriteTootsTask | PASS | report |
| 51 | MastodonFilterLanguageTask | PASS | report |
| 52 | MastodonFollowTask | PASS | report |
| 53 | MastodonGetServerInfoTask | FAIL | report |
| 54 | MastodonImportMutedUsersTask | PASS | report |
| 55 | MastodonInviteTask | PASS | report |
| 56 | MastodonMallPurchaseCommodityTask | PASS | report |
| 57 | MastodonMallShareOrderTask | PASS | report |
| 58 | MastodonManageHashtagsTask | PASS | report |
| 59 | MastodonManageMultiListTask | PASS | report |
| 60 | MastodonMattermostPostNoticeTask | PASS | report |
| 61 | MastodonMultiInviteTask | PASS | report |
| 62 | MastodonNewFilterTask | FAIL | report |
| 63 | MastodonNewPostTask | PASS | report |
| 64 | MastodonOpenAutomatedDeletionTask | PASS | report |
| 65 | MastodonPinTootsTask | PASS | report |
| 66 | MastodonPostEditedPhotoTask | PASS | report |
| 67 | MastodonPostPollTask | PASS | report |
| 68 | MastodonRemoveBookmarkTask | PASS | report |
| 69 | MastodonReplyTask | PASS | report |
| 70 | MastodonReportTask | PASS | report |
| 71 | MastodonRevisePhotoAltTask | PASS | report |
| 72 | MastodonRevisePollTask | PASS | report |
| 73 | MastodonSavePhotosTask | PASS | report |
| 74 | MastodonServerInfoReportTask | FAIL | report |
| 75 | MastodonShareLocationTask | FAIL | report |
| 76 | MastodonUnfollowTask | PASS | report |
| 77 | MastodonUpdateContactsTask | PASS | report |
| 78 | MattermostBudgetApprovalPipelineTask | PASS | report |
| 79 | MattermostCreateChannelTask | PASS | report |
| 80 | MattermostCustomerFeedbackAnalysisTask | PASS | report |
| 81 | MattermostDeadlineReconciliationTask | PASS | report |
| 82 | MattermostEmailTask | PASS | report |
| 83 | MattermostIncidentEscalationTask | PASS | report |
| 84 | MattermostProjectHandoverTask | FAIL | report |
| 85 | MattermostProjectStatusReportTask | FAIL | report |
| 86 | MattermostReadingGroupTask | FAIL | report |
| 87 | MattermostReplyToMessageTask | PASS | report |
| 88 | MattermostResourceConflictResolutionTask | FAIL | report |
| 89 | MattermostSendFileTask | FAIL | report |
| 90 | MattermostShiftCoverageTask | PASS | report |
| 91 | MattermostTechnicalDebtTriageTask | PASS | report |
| 92 | MattermostVisualInstructionResponseTask | FAIL | report |
| 93 | OpenFlightModeTask | PASS | report |
| 94 | PhotoManagementTask | FAIL | report |
| 95 | ReadQwen3PaperTask1 | PASS | report |
| 96 | ReadQwen3PaperTask2 | FAIL | report |
| 97 | ReadQwen3PaperTask3 | FAIL | report |
| 98 | ReadQwen3PaperTask4 | PASS | report |
| 99 | ReadQwen3PaperTask5 | FAIL | report |
| 100 | RecentTotalExpenseTask | PASS | report |
| 101 | RequestCarpoolingTask | PASS | report |
| 102 | ReviewPaperEmailTask | FAIL | report |
| 103 | SMSManagement | PASS | report |
| 104 | ScheduleCoffeeTimeViaSmsTask | PASS | report |
| 105 | ScheduleLunchViaSmsTask | PASS | report |
| 106 | SearchItemAndCheckoutTask | PASS | report |
| 107 | SendFormsTask | PASS | report |
| 108 | SendInterviewEmailTask | PASS | report |
| 109 | SendInterviewInvitationTask | FAIL | report |
| 110 | SendWaiverTask | PASS | report |
| 111 | SetAlarmTask | PASS | report |
| 112 | SharePhotosTask | PASS | report |
| 113 | SuggestPaperTask | PASS | report |
| 114 | SumFileLinesTask | PASS | report |
| 115 | TakeSelfieTask | PASS | report |
| 116 | TextArrivalTimeTask | FAIL | report |
| 117 | ThanksgivingPrepTask | PASS | report |