104

Support recording Desktop Apps

  • Live

Avatar
Kyle

Support recording Desktop Apps so that automations can accomplish more.

More versions of this will continue to be released addressing any bugs. Please post any issues below related to installation/recording

A

Activity Newest / Oldest

Avatar

Rid Gh

hi Kyle, Many of us are looking forward for desktop automation. we should be able to record our office work on specific tasks (such as toggling between files in our downloads files, MS Outlook, Ms Excel and other Oracle applications that we have installed in our desktop. If this is achieved, this will save TONS of hours to a lot of Office workers..
Here is it mentionning the the option of desk recording is "live" while I understand from the community that this is still in the works.. can you clarify please?


  • A
  • C
A

Hi, Kyle. Could you please point me out to a documentation or video on how to use this feature? Thanks a lot!


  • A
  • C
Avatar

Kyle

Status changed to: Live

A

Ivan

Possible Bug with TaskMagic Desktop Function

Link to the video : www.skool.com/lazyempire/possible-bug-with-taskmagic-desktop-function?p=c94d2140

Hi everyone,

WIN11: I’m using TaskMagic to create an automation with the desktop function (first time using "DESKTOP"). As soon as I click on "DESKTOP", the side menu shows the first step saying "Step 1: enter text in Spotlight/Windows search", and I wrote “Photoshop”, which successfully opens Photoshop. However, when I try to set up the next step using any of the other actions (e.g., Click, Type, Scroll), I get this message:

“Just a moment, there may be a lot of iframes on this page causing it to take longer for us to be ready, hold on, we don't want to miss any steps.”

This message has been displayed for an hour (until I close TaskManager). Naturally, I had to force close TaskMagic because there was no progress. I tried restarting my computer and repeating the steps multiple times, but I get the same result each time.

I also tried to uninstall TM, disable WIN11 antivirus, reboot WIN11, and download and install TM with Admin rights. During the installation, a Windows message said, "Do you want to allow this app from an UNKNOWN publisher to make changes to your computer?" I had to say yes and click on "run anyway."

After the installation, I opened TM again and tried to create a new DESKTOP automation but encountered the same result as mentioned above: “Just a moment, there may be a lot of iframes on this page causing it to take longer for us to be ready, hold on, we don't want to miss any steps.”

While this message was ongoing, I also tried to add an action by clicking on "+” (not on the sidebar but in the middle of the screen). I selected "click", a popup appeared saying "click anywhere you want, and save", I did it but nothing happened. It looks like there is a bug, and TM DESKTOP doesn't interact with the machine.

I tried to create a WEB automation and had no problem.

Please let me know if you need any logs, videos, or extra actions from my side. I know your priority is web/apps, but please let me know if you have a temporary workaround for this issue until you release the new version.

Stay fresh and thanks for your help.

Thank you!


Avatar

trait Pichardo

My ideal thing would be self hosting solution we hook came in that software has a new update or security has a new update taskmagic opens terminal and updates the soft then taskmagic sends me an email or notification of what was updated


Avatar

Kyle

OCR for desktop automation + Misc Desktop recording improvements are in development.

The automation is proving to work in minimal cases without OCR and screen changes are causing the automations to fail to frequently.

Updates to the installation script will be released in version 1.3.4 - if you are seeing any issues with installation of the desktop plugin please post screenshots in this ticket: request.taskmagic.com/b/bugs/install-of-desktop-controlbeta/


P.S. Most desktop automations can be accomplished with web/apps


  • Avatar
R

Rey

this would be pretty powerfull if i would use this i think it would be possible to keep all my hosted apps updated using the terminal would that be possible kyle when this feature roles out and is the link you provided above to be a beta tester?


A

Kyle, if you pull this off, along with a legit download-on-demand automation template library, TaskMagic would be the undisputed leader in the niche, above ElectroNeek, RTILA, U-Xer, etc.


V

1. Open excel document
2. Select sheet
3. Read last row
4. Add new row with information based on last row
5. Save document
6. Close excel


  • Avatar
  • A
Avatar

Paul

Articulate Storyline/Rise


Avatar

zshon

1. Open a Zoom meeting link and maximize the window.
2. Then, launch OBS (Open Broadcaster Software).
3. Once OBS is open, start recording the session.


  • Avatar
  • A
  • A
Avatar

Jeremy

Replying to this request I just got via email: If anyone can comment ideas below of automations they want to explore with desktop recordings so that myself and other developers can test it as we build it would be helpful! First up is testing automating / perfecting iMessage :)

I don't have a specific use-case. Generally, I'd like to be able to execute any program in my Applications folder, instruct TaskMagic where to click, and then populate with/and or extract variables/data, for example, from/to a google sheet or other online source.

I use locally-installed research tools, that have no web-frontend - although they pull in data and functionality via their own API's etc. Example: Screaming Frog SEOspider - a) open it, b) populate it with a specific URL, d) load specific presets via the applications file-menu, e) grab and store certain result information, f) check completeness/loop thru next, g) exit.

That kind of thing :)


  • A
Avatar

Jeremy

whoop whoop!


  • Avatar
  • Avatar
Avatar

Kyle

Status changed to: In progress

Avatar

Jeremy

this will be awesome!


  • Avatar
R

Robert

Mac Automator please...


  • Avatar
  • A
Avatar

Jeremy

+1


M

Mike

Yes please. I can imagine all the Zimmwriter hours I'll get back!


  • C
Avatar

Marcel Dinescu

I know what you mean! Please do a video explaining the process


Avatar

SB

This would be fantastic, and if you can have it upload/download/save files as part of automations as well that would be great.


  • Avatar
Avatar

Kyle

If anyone can comment ideas below of automations they want to explore with desktop recordings so that myself and other developers can test it as we build it would be helpful!

First up is testing automating / perfecting iMessage :)


Avatar

Pascal Noyel

Automating with Mac Shortcuts (Automator) would be great


J

AppSumo

+1 on IMessage

WhatsApp desktop

Outlook

File Folder tasks (windows/mac) such as bulk moving by parameter or file, bulk renaming (set nomeclature and let it auto-add series), etc

Integration with a photo tool to do bulk shrink

Auto download from browser email and integration with a locally run LLM to perform OCR tasks like text extraction


N

Nick

The more we can have it just be able to operate like it does in browser (freely), the better.

#1 AI Team in Slack:
- Set up a Slack Automation where specific channels represent specific "AI Team Members".
- Set up Automation --> When message sent to channel - extract message, add to conversation history spreadsheet, with date and time.
- Message is sent to Desktop Automation --> PromptGeneratorAI goes to ChatGPT, uploads conversation history as context, considers my recent message, decides how to respond, adds it to the spreadsheet conversation history, and responds to the message I sent in a threaded reply. All follow up messages are continually updated as if we are having ongoing conversations in realtime.
- Triggered Automation: Every 24 hours, take the conversation history for the day, and run through series of prompt automations in PromptGeneratorAI to create a "Daily Journal" for each "AI Team Member". Allowing them to grow, learn, develop deeper context and nuanced understanding, etc.

1st Team Member Goal: Personal Assistant
After that, I have a ton of specialized assistants I will build out.
_______

OVERALL & for SEO / DIgital PR / Link Building
For me - the desktop automation is all about me being able to create automations that allow my team to access the desktop tools I have access to, directly in a spreadsheet OR the workflows that involve desktop tools that I have no longer require me to do the manual data uploading / downloading / merging:
1) Spreadsheet input --> Data sent to desktop workflow
2) That data and resulting data will be used in a number of ways for SEO / trends analysis / digital PR / etc. usecases with tools like SEOUtils.app, screamingfrog, URL Profiler, PromptGeneratorAI, etc.
- Downloading files / extracting data + combining all that data together in different ways + re-using it with the tools above
3) Data gets output to spreadsheets for team use


S

Stephan

Since we are already using a windows desktop version, it would be great to also allow for explorer recordings. E.g., open specific folders or files within a workflow.


  • J
  • Avatar
Avatar

Kyle

Merged with: Allow Explorer/Desktop recordings

Avatar

Kyle

Status changed to: Planned

Avatar

David Mead

Yes please
The whole point of a desktop app


  • Avatar
Avatar

Jay

I would like to see this work with any software/app online and offline (that's already installed on the Device).
MS Office, Corel, Libre Office, Photoshop, Visual Studio etc.


  • Avatar