Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Textview not accessible to UI tests #63

Closed
jamesbebbington opened this issue Jan 18, 2017 · 8 comments
Closed

Textview not accessible to UI tests #63

jamesbebbington opened this issue Jan 18, 2017 · 8 comments

Comments

@jamesbebbington
Copy link
Contributor

Hey @fphilipe, we have an app that makes much use of PHFComposeBarView and as we're aiming to translate it into a couple of dozen languages, we're very keen to make use of UI Tests to automate UI testing, screenshots and video walkthroughs for our translators. Unfortunately we're having problems entering text into the ComposeBar programatically.

It looks like the core of the issue is that the text view is not visible to the underlying accessibility framework that UI Tests depend on.

I've created a minimal test case using your Example app that exhibits the issue. While it's possible to tap the ComposeBar and give the textview focus, the test framework then can't find any textview to type text into.

screen shot 2017-01-18 at 15 36 32

I've added some acessibilityIdentifiers to help make addressing specific elements easier, but if you look at the application's element subtree that's dumped to the test log, you should be able to see that there's no sign of any textview:

2017-01-18 14:41:55.352 XCTRunner[26241:2566947] Running tests...
2017-01-18 14:41:56.955 XCTRunner[26241:2566947] Continuing to run tests in the background with task ID 1
Test Suite 'Selected tests' started at 2017-01-18 14:41:57.493
Test Suite 'PHFComposeBarViewExampleUITests.xctest' started at 2017-01-18 14:41:57.494
Test Suite 'PHFComposeBarViewExampleUITests' started at 2017-01-18 14:41:57.494
Test Case '-[PHFComposeBarViewExampleUITests.PHFComposeBarViewExampleUITests testComposeBar]' started.
    t =     0.00s     Start Test at 2017-01-18 14:41:57.495
    t =     0.00s     Set Up
    t =     0.01s     Launch com.philipefatio.PHFComposeBarViewExample
    t =     5.06s         Waiting for accessibility to load
    t =     6.75s         Wait for app to idle
    t =     8.19s     Tap Button
    t =     8.19s         Wait for app to idle
    t =     8.22s         Find the Button
    t =     8.22s             Snapshot accessibility hierarchy for com.philipefatio.PHFComposeBarViewExample
    t =     8.51s             Find: Descendants matching type Other
    t =     8.52s             Find: Elements matching predicate '"PHFComposeBar" IN identifiers'
    t =     8.52s             Find: Descendants matching type Button
    t =     8.52s             Find: Element at index 1
    t =     8.53s             Wait for app to idle
    t =     8.56s         Synthesize event
    t =     8.81s         Wait for app to idle
    t =     9.80s     Type 'Here is some sampl...' into Target Application 0x6080000b2780
    t =     9.80s         Wait for app to idle
    t =     9.86s         Find the Target Application 0x6080000b2780
    t =     9.86s             Snapshot accessibility hierarchy for com.philipefatio.PHFComposeBarViewExample
    t =    10.94s             Wait for app to idle
    t =    11.33s         Synthesize event
    t =    11.39s             Assertion Failure: PHFComposeBarViewExampleUITests.swift:24: UI Testing Failure - Neither element nor any descendant has keyboard focus. Element:
Attributes: Application 0x608000165a00: {{0.0, 0.0}, {320.0, 568.0}}, label: 'PHFComposeBarViewExample'
Element subtree:
 →Application 0x608000165a00: {{0.0, 0.0}, {320.0, 568.0}}, label: 'PHFComposeBarViewExample'
    Window 0x6080001651c0: Main Window, {{0.0, 0.0}, {320.0, 568.0}}
      Other 0x608000166c00: traits: 8589934592, {{0.0, 0.0}, {320.0, 568.0}}
        Other 0x608000164500: traits: 8589934592, {{0.0, 0.0}, {320.0, 315.0}}
          TextView 0x608000165c40: traits: 140746078289984, {{0.0, 20.0}, {320.0, 295.0}}, value: Welcome to the Dem...
          Other 0x608000166540: traits: 8589934592, {{0.0, 271.0}, {320.0, 44.0}}, identifier: 'PHFComposeBar'
            Other 0x6080001669c0: traits: 8589934592, {{0.0, 271.0}, {320.0, 0.5}}
            Toolbar 0x608000166840: traits: 35192962023424, {{0.0, 271.5}, {320.0, 44.0}}
            Button 0x6080001660c0: traits: 8589934593, {{8.0, 281.0}, {25.0, 25.0}}, identifier: 'PHFComposeBarUtilityButton', label: 'Camera'
            Button 0x608000166fc0: traits: 8589934593, {{41.0, 279.0}, {224.0, 28.0}}
            Button 0x608000165580: traits: 8589934849, {{267.0, 281.0}, {53.0, 26.0}}, identifier: 'PHFComposeBarButton', label: 'Send'
    Window 0x608000166e40: {{0.0, 0.0}, {320.0, 568.0}}
      Other 0x608000167080: traits: 8589934592, {{0.0, 0.0}, {320.0, 568.0}}
        Other 0x608000167200: traits: 8589934592, {{0.0, 315.0}, {320.0, 253.0}}
          Other 0x6080001675c0: traits: 8589934592, {{0.0, 315.0}, {320.0, 253.0}}
    Window 0x608000167440: {{0.0, 0.0}, {320.0, 568.0}}
      StatusBar 0x6080001678c0: {{0.0, 0.0}, {320.0, 20.0}}
        Other 0x608000167800: {{0.0, 0.0}, {320.0, 20.0}}
        Other 0x608000167980: {{0.0, 0.0}, {320.0, 20.0}}
          Other 0x608000167b00: traits: 8388608, {{6.0, 0.0}, {39.0, 20.0}}
          Other 0x608000167bc0: traits: 8388608, {{50.0, 0.0}, {13.0, 20.0}}, label: '3 of 3 Wi-Fi bars', value: SSID
          Other 0x608000167d40: traits: 8389120, {{137.0, 0.0}, {50.0, 20.0}}, label: '2:42 PM'
          Other 0x608000167c80: traits: 8388608, {{282.0, 0.0}, {33.0, 20.0}}, label: '-100% battery power'
    Window 0x608000167e00: {{-0.0, -0.0}, {320.0, 568.0}}
      Other 0x608000167ec0: traits: 8589934592, {{-0.0, -0.0}, {320.0, 568.0}}
        Other 0x608000167f80: traits: 8589934592, {{-0.0, 315.0}, {320.0, 253.0}}
          Other 0x608000168040: traits: 8589934592, {{-0.0, 315.0}, {320.0, 253.0}}
            Other 0x608000168100: traits: 8589934592, {{-0.0, 315.0}, {320.0, 253.0}}
              Other 0x6080001681c0: traits: 8589934592, {{-0.0, 315.0}, {320.0, 253.0}}
              Other 0x608000168280: traits: 8589934592, {{-0.0, 315.0}, {320.0, 253.0}}
              Other 0x608000168340: traits: 8589934592, {{-0.0, 315.0}, {320.0, 253.0}}
              Other 0x608000168580: traits: 8589934592, {{-0.0, 315.0}, {320.0, 253.0}}
              Other 0x608000168700: traits: 8589934592, {{-0.0, 315.0}, {320.0, 253.0}}
          Other 0x608000168a00: traits: 8589934592, {{-0.0, 315.0}, {320.0, 253.0}}
            Keyboard 0x608000168b80: traits: 8589934592, {{-0.0, 315.0}, {320.0, 253.0}}
              Other 0x608000168c40: traits: 8589934592, {{-0.0, 315.0}, {320.0, 253.0}}
                Other 0x608000168d00: traits: 8589934592, {{-0.0, 315.0}, {320.0, 38.0}}, label: 'Typing Predictions'
                  Other 0x608000168dc0: traits: 8589934592, {{-0.0, 315.0}, {320.0, 38.0}}
                    Image 0x608000169000: traits: 8589934596, {{-0.0, 315.0}, {320.0, 38.0}}
                  Other 0x608000168f40: traits: 9126805504, {{-0.0, 315.0}, {106.0, 38.0}}, label: 'I'
                  Other 0x6080001690c0: traits: 9126805504, {{107.0, 315.0}, {106.0, 38.0}}, label: 'The'
                  Other 0x608000169480: traits: 9126805504, {{214.0, 315.0}, {106.0, 38.0}}, label: 'I'm'
                Other 0x608000169540: traits: 35192962023424, {{-0.0, 352.0}, {320.0, 216.0}}
                  Key 0x608000169600: traits: 8589934640, {{-0.0, 363.0}, {32.0, 42.0}}, label: 'Q'
                  Key 0x6080001696c0: traits: 8589934640, {{32.0, 363.0}, {32.0, 42.0}}, label: 'W'
                  Key 0x608000169840: traits: 8589934640, {{64.0, 363.0}, {32.0, 42.0}}, label: 'E'
                  Key 0x608000169900: traits: 8589934640, {{96.0, 363.0}, {32.0, 42.0}}, label: 'R'
                  Key 0x6080001699c0: traits: 8589934640, {{128.0, 363.0}, {32.0, 42.0}}, label: 'T'
                  Key 0x608000169a80: traits: 8589934640, {{160.0, 363.0}, {32.0, 42.0}}, label: 'Y'
                  Key 0x608000169b40: traits: 8589934640, {{192.0, 363.0}, {32.0, 42.0}}, label: 'U'
                  Key 0x608000169c00: traits: 8589934640, {{224.0, 363.0}, {32.0, 42.0}}, label: 'I'
                  Key 0x608000169cc0: traits: 8589934640, {{256.0, 363.0}, {32.0, 42.0}}, label: 'O'
                  Key 0x608000169d80: traits: 8589934640, {{288.0, 363.0}, {32.0, 42.0}}, label: 'P'
                  Key 0x608000169e40: traits: 8589934640, {{16.0, 417.0}, {32.0, 42.0}}, label: 'A'
                  Key 0x608000169f00: traits: 8589934640, {{48.0, 417.0}, {32.0, 42.0}}, label: 'S'
                  Key 0x608000169fc0: traits: 8589934640, {{80.0, 417.0}, {32.0, 42.0}}, label: 'D'
                  Key 0x60800016a080: traits: 8589934640, {{112.0, 417.0}, {32.0, 42.0}}, label: 'F'
                  Key 0x60800016a140: traits: 8589934640, {{144.0, 417.0}, {32.0, 42.0}}, label: 'G'
                  Key 0x60800016a200: traits: 8589934640, {{176.0, 417.0}, {32.0, 42.0}}, label: 'H'
                  Key 0x60800016a2c0: traits: 8589934640, {{208.0, 417.0}, {32.0, 42.0}}, label: 'J'
                  Key 0x60800016a380: traits: 8589934640, {{240.0, 417.0}, {32.0, 42.0}}, label: 'K'
                  Key 0x60800016a440: traits: 8589934640, {{272.0, 417.0}, {32.0, 42.0}}, label: 'L'
                  Button 0x60800016a500: traits: 8589934649, {{-0.0, 471.0}, {42.0, 42.0}}, identifier: 'shift', label: 'shift'
                  Key 0x60800016a5c0: traits: 8589934640, {{48.0, 471.0}, {32.0, 42.0}}, label: 'Z'
                  Key 0x60800016a680: traits: 8589934640, {{80.0, 471.0}, {32.0, 42.0}}, label: 'X'
                  Key 0x60800016a740: traits: 8589934640, {{112.0, 471.0}, {32.0, 42.0}}, label: 'C'
                  Key 0x60800016a800: traits: 8589934640, {{144.0, 471.0}, {32.0, 42.0}}, label: 'V'
                  Key 0x60800016a8c0: traits: 8589934640, {{176.0, 471.0}, {32.0, 42.0}}, label: 'B'
                  Key 0x60800016a980: traits: 8589934640, {{208.0, 471.0}, {32.0, 42.0}}, label: 'N'
                  Key 0x60800016aa40: traits: 8589934640, {{240.0, 471.0}, {32.0, 42.0}}, label: 'M'
                  Key 0x60800016ab00: traits: 9663676464, {{278.0, 471.0}, {42.0, 42.0}}, identifier: 'delete', label: 'delete'
                  Key 0x60800016abc0: traits: 8589934640, {{-0.0, 525.0}, {40.0, 42.0}}, identifier: 'more', label: 'more, numbers'
                  Button 0x60800016ac80: traits: 8589934641, {{40.0, 525.0}, {40.0, 42.0}}, label: 'Next keyboard', value: Emoji
                  Button 0x60800016ad40: traits: 8589936689, {{80.0, 525.0}, {32.0, 42.0}}, identifier: 'dictation', label: 'Dictate'
                  Key 0x60800016ae00: traits: 8589934640, {{112.0, 525.0}, {128.0, 42.0}}, label: 'space'
                  Button 0x60800016aec0: traits: 8589934641, {{240.0, 525.0}, {80.0, 42.0}}, identifier: 'Return', label: 'return'
Path to element:
 →Application 0x608000165a00: {{0.0, 0.0}, {320.0, 568.0}}, label: 'PHFComposeBarViewExample'
Query chain:
 →Find: Target Application 0x6080000b2780
  Output: {
    Application 0x608000165a00: {{0.0, 0.0}, {320.0, 568.0}}, label: 'PHFComposeBarViewExample'
  }

    t =    11.42s     Tear Down
Test Case '-[PHFComposeBarViewExampleUITests.PHFComposeBarViewExampleUITests testComposeBar]' failed (11.761 seconds).
Test Suite 'PHFComposeBarViewExampleUITests' failed at 2017-01-18 14:42:09.257.
	 Executed 1 test, with 1 failure (0 unexpected) in 11.761 (11.763) seconds
Test Suite 'PHFComposeBarViewExampleUITests.xctest' failed at 2017-01-18 14:42:09.258.
	 Executed 1 test, with 1 failure (0 unexpected) in 11.761 (11.764) seconds
Test Suite 'Selected tests' failed at 2017-01-18 14:42:09.258.
	 Executed 1 test, with 1 failure (0 unexpected) in 11.761 (11.766) seconds


Test session log:
	/Users/james/Library/Developer/Xcode/DerivedData/PHFComposeBarViewExample-foxaqozujhvretgsawvddytmcqux/Logs/Test/6895D49B-2F01-4FC1-89F1-2D93011CBAB9/Session-PHFComposeBarViewExampleUITests-2017-01-18_144146-qXDaXR.log

I suspect that this may be related to the fact that the textView is a child of a button, as brought up in #36. I'm sure you had good reason to nest the textview in the button, I'd much appreciate it if you could give us some background so that we can fix or workaround this issue.

Many thanks.

@jamesbebbington
Copy link
Contributor Author

Ah, it looks like the iOS Accessibility Programming Guide explains (inc. sample code) how to make the contents of custom container views accessible. Unfortunately we don't have any Obj-C expertise in-house to try out this implementation.

@fphilipe
Copy link
Owner

Hey @jamesbebbington. Sorry, but I don't have the capacity to implement this change.

@fphilipe
Copy link
Owner

Hey @jamesbebbington, I've looked a bit into this and the UIAccessibilityContainer protocol. I changed the container that contains the text view from a button to a normal view. Would you mind testing with the improve-accessibility branch, specifically this commit? 28a3b5c

Thanks!

@fphilipe fphilipe reopened this Jan 27, 2017
@jamesbebbington
Copy link
Contributor Author

jamesbebbington commented Jan 30, 2017

Thanks @fphilipe, much appreciated.

UI tests can now address the text view, however I'm seeing some strange behaviour when programatically entering text. Given this UI Test case:

import XCTest

class PHFComposeBarViewExampleUITests: XCTestCase {

    override func setUp() {
        super.setUp()
        continueAfterFailure = false
    }

    func testComposeBar() {
        let app = XCUIApplication()
        app.launch()

        app.staticTexts["PHFComposeBarPlaceholderLabel"].tap()
        app.textViews["PHFComposeBarTextView"].typeText("Here is some sample text. It's a few lines long but not too long. I think that should do it.")
        app.buttons["PHFComposeBarUtilityButton"].tap()
        sleep(5)
    }

}

the entered text is being partially erased as it's entered:

Oh and a default accessibility identifier for the placeholder label would be helpful, so it's easier to address in tests when the app is localised.

EDIT: I forgot that I had set the accessibility identifiers in the the example app. Test code above updated and pushed in new ui-tests-2 branch.

Thanks!

@jamesbebbington
Copy link
Contributor Author

I've pushed a couple of updates to the ui-tests-2 branch, fixing the misnamed buttons and adding a (failing) assert that checks that the entered message was properly saved.

@fphilipe
Copy link
Owner

@jamesbebbington thanks! I've added a commit introducing a test (based on yours). I was not able to reproduce your erasing text problem you showed in the GIF.

I'm wondering if the accessibility identifiers should be set directly in PHFComposeBarView or if it should be up to the consumer of it to set it (as you did and I've taken over).

From my side this would be ready. Let me know if this works for you.

@jamesbebbington
Copy link
Contributor Author

Cheers @fphilipe, will take a look now.

I'm wondering if the accessibility identifiers should be set directly in PHFComposeBarView or if it should be up to the consumer of it to set it (as you did and I've taken over).

I don't think it matters too much either way, as long as consumers can set them. I think it's probably fine to leave them unset, as I'd imagine anyone who wanted to use them would want to rename them anyway.

@jamesbebbington
Copy link
Contributor Author

I was not able to reproduce your erasing text problem you showed in the GIF.

Yeah, it's an odd one, it seems a bit intermittent.

I integrated the improve-accessibility branch into our app and ran tests on iPhone 4s, 5, 6 & 6 Plus across four different languages and they were all fine, so not really sure what's going on with that.

Let me know if this works for you.

Yup, many thanks @fphilipe!

fphilipe added a commit that referenced this issue Feb 1, 2017
@fphilipe fphilipe closed this as completed Feb 1, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants