Notes
Requires Docker and at least 4 GB RAM (see comment section bellow for further details)
Not applicable for single app
W...
For further actions, you may consider blocking this person and/or reporting abuse
I'd disagree that version managers are obsolete. I have great success with nvm and brew, with no complications. When I use Docker, a lot of RAM is used. I have a 32GB Mac and yet it still slows things down.
I do use Docker for some things successfully, but I can't imagine running all these different containers to do my day-to-day work. Wherever possible, I run things locally on my machine as close to the metal as possible.
Thanks for the post, I'm curious what your day-to-day looks like with all of these different images running?
I have also a 32 GB RAM Mac and I assigned 12-14 GB RAM for Docker (+2 GB for Swap).
I agree that the Docker Virtual Machine (Mac/Windows) could take a lot RAM. In my case 12 RAM is used whether there is any container or not.
For Linux, It takes all of the available Memory. So if for some reason there is a High Memory or Out of Memory, the System crash. For Mac/Windows, only Docker crash and then restart.
Everyday, I run a fullstack containers (api, app, db, broker, etc ) with at least 10 containers.
I know that there is a limitation with the number of containers on the machine (depend on the memory and cpu) but If you want, you can control the memory and the cpu usage of your containers.
If these containers launch dev-servers (like API or APP) so it is the same to launch manually servers on the OS because it is not costly.
I noticed that containers take a lot of RAM when building the image. In my case, I build the image only once and then bind it with a volume.
If you have a single app with few dependencies, so docker could be useless.
My servers are mostly 1GB or 2GB, so I can't fit many containers there.
Laptop is 8GB and no Docker. I have everything on Windows or in WSL2.
I wish I can afford 32GB.
Are you sure about that? I have used docker in Ubuntu. It almost used around 500mb memory. Though I don't always use docker. But docker is very close to linux. Correct me if I am wrong. And linux has a way to kill the process if it's using too much ram.
You're indeed correct.
Sincerely,
Another Ubuntu Docker user
32 GB RAM!!! I have begun questioning my mere existence... I got a 3GB RAM junkbox 😒😒
Haha yeah. This was provided by work, I've never had so much RAM before!
Same for me
This! Docker is resource hungry and I constantly need to go back and cleanup storage.
I use Docker, but not for these cases.
agree
Docker is great for production deployment but painful for development.
You don't have to necessary need another container for your database.
You could also have it directly installed on your OS.
In my case, I prefer to containerised all of my stack (db, api, app, broker, etc,)
To quote:
You don't have to necessary need another container for your database..You could also have it directly installed on your OS.
In my opinion that's the WORST WAY to use Docker. I rather run the 10 containers Confluent provides for Kafka and have everything working and setup than waste time configuring Kafka for my OS (which most probably won't work if it's Windows).
For development, you can do theses things :
1) Keep the container awake with a non-stop command ( ex: CMD tail -f /dev/null)
2) Bind a volume on your container (with docker-compose.yml)
3) Use a development server with hot-reloader (runserver for Django, run for flask, react-scripts start for Reactjs, etc)
About running a database on another container I see that as an advantage as it helps isolate parts of your application. But if you want to run your db and app server on the same container, that can also be achieved.
I'd suggest checking out FreeBSD with Jails then. They are the containerization system that Linux modeled after (but IMO didn't do it anywhere near as well). Jails are functionally containers, but from a UX perspective are closer to full blown virtual machines. You can SSH into them, local console access into them, they have the full FreeBSD package manager, can install and run virtually anything that could run on the host. They also have their own independent network stack via VNET, which fully supports DHCP, IPv6, the whole array of networking protocols, or optionally share the host's networking stack instead. Additionally, you get native ZFS support accessible on the host, and optionally accessible inside the jail for each snapshotting/backup/replication/deployment.
Our entire development environment runs in Docker. To address your points.
I like the idea of Docker, but it does cause pain for local development. Others in the comments have already expressed the main issues, but for me it's mainly just the slowness (e.g. having to rebuild) and extra complexity for simple tasks (e.g. setting up git pre-commit hooks).
For simple Node.js-only projects (no dependency on Postgres or Redis), Node.js version managers like NVM work great and keeps setups very simple. It's not a complete replication of the production environment, but you should have that in a deployed development environment.
Conceptually, we should use containers for everything. In practice (at least in my own experience), it causes unnecessary problems for local development.
Ok I will mention it that version managers could be used for simple project
Or...perhaps just use what works best for your workflow, no matter how many articles say you should "Stop using that one thing that literally does the job for you".
Docker is a regular part of my workflow, but no, I'm not going to spin up a Docker container every single time I want to run a Python file with particular packages. I'm quite proficient with virtualenv, and it works very well.
Meanwhile,
nvmsaved my hide when testing a PR in an Electron app in an actual user environment, which Docker is not (without a lot of wasted time building it, at least.)To say "only use Docker images for building" is like saying "always rent a food truck when you need to cook supper, rather than wasting your time cleaning and setting up your home kitchen." It's overkill. There's a place for both, and one cannot just replace the other.
But then, "don't use X, use Y" is virtually always fad-chasing, rather than fact-based.
I use nvm and pyenv with huge success most of the times, but I also know that it can be a huge struggle when these tools fail what they promise.
That's why I don't use Docker for these tasks but I often use it for starting a local test database with one command and throwing it away when I don't need it anymore.
But I also use Docker when I don't want to install a certain tool in development. Then I quickly write a
docker-compose.ymland mount the development folder into the container. Then there is no need to rebuild the container for every change and I can run the tool with shell inside the container.I respect all developer point of views and theirs laptop configuration (memory, cpu)
It is a pleasure to have this kind of debate in dev.to (I couldn't do the same at the office).
I will notice your feedbacks at the top of my post.
Thanks for all replies.
You should add a disclaimer:
When you have tons of RAM and disk space.
Docker images consume a lot of disk space and running containers use a lot of RAM due to the need of a virtualization tool in the OS (in case of macos) or because of the containers themselves.
I really need to get comfortable with Docker and by extension Kubernetes.
I recommend Mumshad's course on Udemy. I used it when I prepared for the CKA exam and it was the best online course I have taken in a long time. I'm not affiliated with the man in any way so this is coming from my heart.. (:
Thanks for the recommandation.
I would definitely add security to the benefits of using Docker vs. virtualenvs.
PyPI repositories are a wild west and you could end up running all sorts of crap on your host OS if you don't contain it.
Of course you have to audit your dependencies nevertheless because eventually the code will be deployed somewhere. But still, it at least makes me sleep my nights better.
I totally agree with you.
There is the security layer.
And I would also add the famous quote of : "it doesn't work on my machine".
Now sharing images will help to build app on several similar environment (local, development, staging, production) and look like the same environement (almost)
I've used Docker on GKE for federated (multiregion) delivery, but I found LXD containers to be easy to manage, secure, and far less resource intensive for our dev env. It's really simple to spin up a cloned container using your favorite distro in seconds and work in a familiar linux server env. Each solution has it's upside and critters to deal with, but that's what works for us. Thanks for the good write up.
Man, I think I must be misunderstanding this docker craze. Like I once tried to implement a Vue.js development environment on docker, and it was such a mission to get it to run locally on docker. It took me half a day to put together from several different articles, and then it still required all sorts of hacks and tricks to run locally vs building for production, for instance. In the end I gave up, and installed node, npm and vue to my pc and carried on working like I always have, with zero headaches.
So I suppose, I'd like to know what it really means to replace dev environments with docker, if it doesn't mean that you're setting up a fresh project to run locally on a docker container, and then ultimately building to a docker container?
Nothing in our workflow requires docker for dev, but it is handy.
For example, we have a separate repository with branches that are environment names. Throw a war (or jar) into the right directory, and then
docker-compose up ....That way, I can checkout a different branch, and I get the config set for a different environment, spin up my container and my version of the application, is connected to a different environment, so I can triage issues or test that my new build doesn't break something in that environment.
Sure, I could manage that in other ways, but some of our stuff runs in Tomcat, and I'd rather not have to worry about configuring my machine to match the config of some other environment, just so I can test stuff, when it's less than 10% of my time to test stuff like that anyway.
I did know a guy, that ran his whole IDE in a container, and used it to produce images. I think he was a little bonkers though.
Coz it also seems to me that if you're successfully able to set up a containerized dev environment, you're willingly adding another vector for bugs and problems. Like if you run into complicated bug on your containerized dev project, how do you know it's a coding bug and not a issue with your docker setup? And how do you explain your setup to the kind folks over on stackoverflow? I don't want to first have to type a 6 page essay on the configuration of my project, every time I'm over there, asking a question.
I don't agree with your post. I use the Python from my system when developing and I use images for testing (sometimes) and for external services and software that I need (ElasticSearch, Mongo, PostgresSQL).
Docker is painful, slow and remote debugging is tricky.
I'd prefer mixing between Docker images and package manager. For daily use languages/ applications such as golang, python3.8 (create venv when needed), node, etc. I install them directly in the OS, for all other temporary components, which are used for specific purposes, I use docker and expose their ports to OS for comumication with my developing app.
Benefits from this way are it's easy for me to quickly run some command, python scripting, etc. no need to remember different syntax with docker, or turn docker on. Temporary docker containers and data could be easily destroyed as well after being used, which keeps my machine clean.
i just still not comfortable using Docker for Development purpose, especially when i coding and debugging, yes, because remote debugging not right choice for me.
BTW i always use Docker image to install things like database or service broker.
Full disclosure: I’ve never used docker much outside of when it’s been required.
I kinda fail to see what Docker does better than tools like py/jenv. I can already install and set up different versions with their own separate packages with these, and I don’t have to worry about another layer of config/setup. Since I’ve already got my system set up pretty much the way I want, it seems like an unnecessary hassle.
I can see a big use for these in my data science work though. Having entire separate system environments could really be useful for things pulling env variables and whatnot, I suppose. I’d love to see a reason to use docker more in my personal workflow, but I guess I just haven’t found that magic feature that makes it a must-have yet.
Another alternative is nix with nix-shell, which can result in just-as-reliable reproducible development environments (especially when combined with direnv and lorri).
I'd be curious to know how is your IDE integration experience?
Hi, I just used docker volumes, so it is easy to integrate with IDE (ex: node_modules)
I couldn't imagine running a container for everything going on, on a day to day basis. I think my mac would actually explode and burn my house down.
Stop using Docker images and Use Linux Containers, lol jk.