🤯💥 Unleash the power of UI automation on Chromium tools! 💻 TreeWalking with UIA
How to automate Chromium based tools / Apps with UI Automation. We start off with Tree Walker and go on. Make sure you stick around to the end to discover why you don't just take this approach on every UI Automation.
Learn more about UI Automation here: https://the-Automator.com/UIA
TLDR: Tree walkers, UI Automation, and Chromium-based tools can be used to automate UI elements and access joint elements in UI automation.
1. 00:00 🤔 Automating Chromium-based tools using tree walkers, activating Skype to get the focused element, and saving the output into the clipboard is a workaround to access joint elements in UI automation.
1.1 We're demonstrating a workaround to automate Chromium-based tools using tree walkers, and discussing the differences between this approach and others.
1.2 Get element from handle, get root element, get element from point, get focused element, and element from I accessible are methods to access joint elements in UI automation.
1.3 We are trying to get the handle to the window and get an element from it, but the handle does not allow us to list or dump all the elements, so we are checking the handle for Skype and activating the ACT.
1.4 Script activates Skype, brings it to the front, and gets the focused element.
1.5 Activating Skype and focusing on the element allows for the whole shebang to be outputted and saved into the clipboard.
1.6 For discovery purposes, it is fine to use the function, but once you know what you're looking for, don't use it repeatedly as it takes a long time to load.
2. 09:37 🤖 Automate your balance repurchase with Deskglotta's approach to Skype, using UiAutomation, tree walkers, and the UI Automation Tool to search and filter elements.
2.1 Using the 3.1 element, you can target and automate your balance to repurchase automatically.
2.2 Descolada approach to Skype is more efficient and should be used instead of the other approach.
2.3 To get the document element, we need to use the UiAutomation method to call the focused element.
2.4 Using a tree walker, you can loop through a list of elements in an efficient manner.
2.5 We can use the UI Automation Tool to search for elements and filter them with conditions, and get the parent of an element by searching linearly from the top of the tree.
2.6 Tree walkers allow you to move backwards in the tree to get the parent element, which is not done by default.
3. 15:51 🌳 Create two conditions separately and pass them both to create an "and" condition for a tree walker.
3.1 Think of ways to give information without having to redraw the thing.
3.2 Creating a Tree Walker is like creating a DOM element that allows you to move backwards and forwards through the structure of parents, siblings, and children.
3.3 Creating conditions for a tree walker involves passing a function or complex condition into the walker.
3.4 Create two conditions separately and pass them both to create an "and" condition.
3.5 You need conditions to create a tree walker, and in this case it was a simple one of control type being equal to document.
4. 21:01 🤔 Restarting the computer multiple times can allow access to the document, and chromium apps can be used to access individual elements within a web page.
4.1 For some reason, the computer is now allowing access to the document after restarting multiple times.
4.2 Chromium apps can cause pain when trying to identify elements, but this problem does not exist in other apps.
4.3 We can access individual elements within a web page using a chromium app and the DOM exposed by the script gives more information than what can be seen in the inspector.
5. 24:52 🤔 Creating a library requires deciding how to match functions and understanding user needs for training.
5.1 We used document control type as a condition to find the button, which worked differently than the expression with the equal sign.
5.2 Normalizing elements ensures the correct element is obtained by checking if the element itself matches the condition and going up the tree if it does not.
5.3 The library searches for a control type document given a starting element.
5.4 Tree walker normalizes the focused element to the closest parent or element that matches the condition.
5.5 If the focused element is not the document, it will throw an error if it cannot find it as a parent.
5.6 Creating a library requires deciding whether to match the library functions or sugar-coat them, and understanding that users may need to be trained to use the library.
6. 34:08 🔎 Use a TreeWalker with a condition to get an element from a specific location on the screen, and use relative notation to find the first child.
6.1 Get an element from a specific location on the screen by creating a tree walker with a condition and normalizing it.
7. 47:08 🔎 Tree walkers allow you to find elements without knowing their specific path.
8. 53:34 🤔 Use a handle instead of the element for better compatibility with Chromium elements/windows.