Automatically run scheduled python code with playwright
Hi all,
\+++++ Background for context (can be skipped) +++++
the last year I had my unraid running for just basic "NAS" things. Like media-streaming etc. But recently I came into learning a bit of python.
Now I have a simple python code, that auto logs into my girlfriends workplace planning tool, scrapes the time table and saves the data. Then after that another script, that reads the input of the data and checks if our google calender is up to date and if not changes / creates new events. Now while this runs fine on my Windows machine, I want this to be fully automated. The biggest problem is the web scraping. It uses playwright. And in a lot of different things I tried this was the blocker.
* Running it directly on the unraid server (in general or on a virtual environment) * playwright can not be installed * Installing the playwright Docker called "browserless" or "browserless-v2" * The Docker itself works fine, I have it running fine with the "changedetection.io" Docker container * But I was not able to connect to the playwright with my PyCharm or VSCode * Using the VSCode Docker, that has the option of a scheduler-plugin * Again, I have no right to activate playwright * The install is working fine, but after that you have to do the "sudo playwright install-deps" > the Docker runs under the user "nobody" which has afaik no password, but it asks for a password * Windows VM * This obviously would work, but for my server that was just to ressource intensive, to have it running in the background all the time, just for a script to be executed every 24h * Linux VM * I tried as well, I am not much acquainted with Linux in general, but this also seems to be a little overkill for this.
\+++++ Now the important part +++++
* Using the PyCharm Docker "binhex-pycharm" * Here we get as close to a solution as I feel I got with anything I tried * I could install playwright > manually running them works perfectly * I even wrote a little py script, that uses the "schedule" library, which was kinda fully automated, but to be honest a little bit too janky * Restarting the Docker kills the execution * For every additional script, either another schedule script has to be written for every repo I want to run scripts from the specific venv or make one big repo and butcher everything together.
With the PyCharm Docker, I was able to open a terminal and run the py script from there. So in theory there should be a possibility to have some kind of script that schedules all these things and is automatically started on Docker start. Obviously I can not just alter the Docker container. The question now is. Is there an easy way to use the Docker Container ([apperently this is the Github](https://github.com/binhex/arch-pycharm)) add maybe a file explorer and a script scheduler (it seems to be some kind of linux distro in the background, it even has Firefox installed). And create my own Docker Container, where I can just run a few scripts in sequence and with a schedule. That's it.
Now, I went through like a few dozen of rabbit wholes. Am I on the completly wrong path here? And is this a viable solution for my problem? If so, can someone help me how to do that? I unfortunately have no idea how to handle Linux, Docker or similar, but are willing to learn.
That's what happens when you go from: "Hey it only took 2h to right this python code, that's cool" to "let's look into, if I get this from my local PyCharm running on my unRAID server ... 48h later ..."
Thank you so much in advance for reading / helping me! Have a nice weekend!