Haiku’s GUI is in principle entirely scriptable. You can change a window’s position and size and manipulate pretty much every widget in it. The tool to do this is
hey. It sends BMessages to an application, thus emulating what happens if the user clicks on a menu, checkbox, or other widgets.
Am I mistaken or this is the kind of behavior that Google is worried about in it’s accessibility API?
Hi,
I’m not entirely sure; but I don’t think so.
The main problem Google would be worried about is “man in the middle snooping” – things like keyloggers that intercept user’s input and send it to a third party so they can scrape out your passwords, etc.
Hiaku’s “Hey” scripting looks like it doesn’t intercept anything from the user (and can’t be used for “man in the middle snooping”). It’s more like (a GUI equivalent of) batch files in MSDOS, or shell scripts in Unix, or…
Also note that the “inability” to write scripts for GUI applications is one of the common complaints that command line users bring up in “command line vs. GUI” arguments; even though there’s been many approaches to GUI scripting for a very long time (dating back to things like AppleScript and VBScript in the 1990s).
– Brendan
so… you couldn’t create a new text box widget with hey, hover it over one for a password in an application, log passwords and pass them onto the original application… hmmm….
No, but you could script an application that requires a password to automatically input a password and initiate login without doing it yourself
This would be useful for automated tests of the user interface of an application.
However, I would not do this for daily use as the script would not necessarily be encrypted.
Haiku always gets appreciation from me.
That being said, the examples seem a bit…. clunky. I think AppleScript could have been shining the beacon w.r.t. GUI app automation – applications expose less a UI surface, more a UI, and recording a UI workflow really repeats the actions behind the scenes, not the UI events. It was the classic Mac OS’ alternative to shell-based scripting in an environment that had absolutely no concept of a command line.
However, the AppleScript language was weird, so it never got a lot love from programmers. Automator was a late addition to help encourage AS development, but didn’t help in the end, JS was added later on, but by then, Apple had gutted the division.
Nope, not only did Apple expand the Automation team in order to develop JavaScript for Automation (from 2 developers to 3), they even took the JavaScriptCore project that the JSCore/WebKit folks had announced at WWDC13 as an AS alternative and handed it to the Automation team instead. The fundamental failure was PEBKAC: specifically, Sal Soghoian, head of the Automation team, was a fucking terrible Project Manager (a talented Automation evangelist promoted way beyond his level of competence; see: Peter Principle) who in his last decade at Apple effectively ran Mac Automation into the ground.
Result, the Automation team shipped JXA half-baked and half-broken in 10.10, then not only did they not try to fix it but didn’t even bother to document, promote, or support it. Unsurprisingly, JXA completely failed to capture any market, despite having millions of sitting JS users right there for the taking, with an incredibly powerful, widespread Automation infrastructure that should by all rights be Utterly Irresistable Industrial-Strength Catnip For Geeks™. That incredible failure is what got Sal’s ass belatedly sacked and his team disbanded.
If you’re curious, this is what an Apple event bridge for JavaScript should look like:
https://bitbucket.org/hhas/nodeautomation
Documentation’s a bit shit, as it’s decade-old recycled mess that I’ve never bothered to rewrite. But at this point I only really write these things for shits and giggles, and to prove other people wrong. (If you’re really curious, here’s the “OSA-packaged JSCore†reference implementation I sent Sal before 10.10 shipped: https://sourceforge.net/projects/appscript/files/ – bit messy/buggy as it was done in a huge rush, but still better’n the crap they shipped, and even has a nifty automagical AS-to-JS translation tool for turning AS commands into their equivalent JS syntax which alone would’ve gone a long way to making it a popular success.)
But anyway, I digress…TL;DR, Apple event IPC = Awesome Automation infrastructure, designed right and still the benchmark by which all others should be judged, screwed by being shackled to a shit language-cum-brand name and squandered by a vast not-nearly-as-bright-as-you-think corporation that after 20 years is quietly preparing to throw it away for good. (If FOSS/Linux had an ounce of sense, they’d steal all its concepts and philosophy and bat it out the park with nominal effort…eh but who are we kidding.[sad])
So… it seems like you might be the local MacOS automation expert?
I have a related question…
Some time ago I got a Connectix QuickCam, the 1st webcam, connected to ~classic Macs through the ADB port. And… what else to do with it than to put it on the web!
Some caveats though…
I do not have the drivers/software, I must yet to hunt it down on the web. And, having little experience with classic MacOS, I do not know how OS-version specific it is – would the driver be “universal” across 68k and PPC, or across OS 7.x, 8.x, 9.x …or not?
Also, I do not yet have a required Macintosh. It would have to be a machine with ADB and Ethernet ports, so probably some PPC Mac running MacOS 8.x or 9.x…
The way I want to use it, is to take a photo every minute, give it a name in the YYYYMMDDHHMM format, then upload it to a server for the world to see …is learning AppleScript my only option on such setup?
(what resources would you recommend?)
https://homepages.cwi.nl/~jack/macpython/macpython-older.html
Oh joy, probably everything I could hope for (planning to get into Python anyway / it’s not a dead tech). Thank you.
Hi,
I think someone forgot to include a link to the article being mentioned here.
That link is probably: https://www.haiku-os.org/blog/humdinger/2017-11-05_scripting_the_gui…
– Brendan
Haiku’s GUI is aging, device-dependant, bitmap based, pixel-wide surface with à lack of color freedom.
The script API IS hard to read, closed to this solely plateform, ready to use from à monthly-basis usage course.
Thé design pattern used in this framework are litteraly from another age, based on IOT principeles, unhealthy procédures and hard-to-find documentation.
The running processus itself lack of nowadays graphism effects, made to enjoy framed Gray buttons.
I AM sure Haiku’s développent team can make better with nowadays tools and design principles. Receipes from web and mobile GUI. With Todays desktop like KDE we can play with colorfull widgets and graphism effects. Enjoy drag’n’drop collections, Device-independant relative surfaces and a meta-framework with interactive event-based patterns.
Thanks for the sake of Haïku, it is completely original and out of frame designed piece of art.
Edited 2017-11-16 12:13 UTC
And some of us don’t want such superfluous crap. All i want my window manager to do is manage windows. I don’t need them transparent, or rainbow coloured. Haiku’s Window manager does things right. All the “innovation” has been focused on making “managing windows” as powerful as possible, and not on crap like aesthetics. Sure, it looks like it came out of the 90’s, but how many other OSes allow you to stack and tile a whole combination of applications?
Edited 2017-11-17 11:00 UTC
Yes it is BeOS from the late nineties but back then it was an improvement on Windows and xwindows. The documentation was not bad, at least it was included instead of paying a lot of $ for the RKM’s
Isn’t a scriptable GUI also the goal of KDE Plasma? You have to use QML though.
Haiku has had Hey for years, I remember playing with it in the alpha 1/2 days. Maybe KDE should hurry the f*** up.
What?
KDE Plasma already exists and you can write stuff for it in QML.