Your very informative and nicely composed videos helped me get a job in this tough market. Thank you very much, and keep up the good work!
The naming of the ICECREAM module is a clever play on the abbreviation "IC", which phonetically mirrors "I See" in English. This choice is fitting for a Python debugging tool, as it elegantly underscores its purpose: to illuminate the inner mechanisms of your code, much like peering through a transparent layer in a typically opaque Python environment.
A CLI app is always a good idea when automating boring tasks at work like building reports or extracting data to manipulate. Its quick to develop and fast to execute.
A deep dive video on SQLModel would be great. I like it. On my first pass at learning it, the foreign key constraints were a little confusing and I ended up refactoring some code once I fully understood it. But seeing a video might be a great way to know if there is a better way to define them. I was doing a light project with a SQLite-based database, but ultimately want to get to a project that uses PostgreSQL with geocoding. If you did a video like that I would find it very interesting.
Since I've started to learn rust I found the Result type (railway oriented programming) an incredible programming mindset. The basic idea is that many of the exceptions we throw are not really exceptions, but just different paths that our program can take, so we shouldn't treat them differently. If that idea is well integrated into the language, is very powerful and allows to prevent issues.
Great video, thanks! Here is a new entry: Ray is a nice framework with a Python API that facilitates code parallelization, whether it is on a single machine or within a designated cluster. Side note for people who might be interested: while Ray offers a user friendly experience, there is a certain degree of syntax overhead, making it challenging to run code in a serial manner without code modifications. To that end, I have build and published a wrapping library (still a work in progress), “ray-ease”, that allows to switch from parallel to serial without requiring code modifications.
Good to see a recommendation for argparse. Most of these modules are already used by me so I'm glad I'm on a similar track. For the question on command line programs, we create command line programs as the work I do is mainly backend and we try to automate as much of our repeatable and tedious processes as we can. Vendor software provides some functionality and our tools just enhance that to make maintenance seamless and fill the gaps not covered by the vendor.
One of my go-to libraries for adding 12-factor structured logging to my code is structlog. Works equally well as a replacement or enhancement of the standard logging library, and extremely flexible and extensible.
Thanks Arjan ! A full video on SQLModel will be very appreciated..
We add CLI to almost all of our projects for a couple reasons: 1) it's a quick and easy way to test out what is the proper interface/API to your applications before you invest into a more complex API tooling and 2) our applications are mostly pipelines (orchestrated ML lifecycles), not always-up applications. CLI is just a very robust and controllable entrypoint for that type of applications.
Not a library, but I wanted to mention this: Python Poetry saved me so many headaches at work the past 2 months. I have been been a single dev working on some new infrastructure as code stacks with TerraforCDK and was having a rough time with dependency management. It simplifies virtual envs, dependencies and packaing. Def worth checking it out!
Awesome collection. Please create a complete video about SQL Model + FastAPI :oops:. Thanks
I write CLIs all the time. I generally design everything like a library so it would be easy to attach any frontend to it, but CLIs are easy to crank out with libraries like Typer. At work, I write a lot of custom tooling for things like: managing/requesting TLS certificates, DNS management for customer domains, and other oft-repeated day to day tasks. At home, I wrote an application to automate organizing my media collections (with remote metadata retrieval). I'm usually in the terminal,and automating GUIs is a pain, so I almost exclusively write CLIs for my libraries and leave the GUI or web service frontend for others to implement.
I write quite a lot of CLI applications, and personally I find the docopt library really neat.
I really like click as an alternative for argparse. Much simpler to use and very powerful! Only downside is that it adds a dependency.
Thanks for the informative video... A video on SQLModel would be nice! What about a video about integrating between a REST API and a SOAP API using Zeep (Getting the data from both APIs, manipulating and comparing the data between the two to check if you should make a post, update or delete request to the REST API)? I think it should be interesting... It can probably just be 2 REST APIs as well...
If you do any calculations that requires units I will always recommend 'units' from 'astropy'. You can basically transform any unit to any unit (like nanometers to feets, if you need that for some reason) in an easy way. Pretty useful in equations where you are not sure if you have converted the units properly as well
I build command line interfaces all the time for data science / ETL utility scripts & processes, pulling data from APIs, roll-up aggregate jobs, interacting with cloud services, etc... argparse, tqdm, and pydantic are all super handy.
really nice list, thanks for sharing Arjan! i started using argparse recently in a data engineering context, to set up a main job executor and pass ETL jobs and params as arguments. between that and config libraries like omegaconf and hydra, it's completely changed my application design for the better.
@ArjanCodes