A Scalable Development Environment

Photo by Thom on Unsplash

Now lets come to the development setup. I use a MacBook Air with just 4 GB of ram and it eats up half of it by default. MacBook Air seemed to be good due to its portability, sleekness, and all other stuff except for programming, data analysis, gaming, etc due to memory and GPU constraints. I was forced to look for a better development setup. This problem can be generalized to anyone who wants to build stuff but suffers from hardware limitations.

To start with I did remove all unwanted running applications. Further, I also used a tool called MemoryDiag which automatically frees up RAM when it reaches certain limits. The biggest memory hog comes from browsers like Firefox and Chrome which try to take up any free space and thus suffocating the development servers. Even my preferred IDE Vscode takes up a lot of RAM as I have to run 4 different windows to get the linting and integrated terminal working properly.

I had the following options in front of me

  1. Run a lightweight os like Linux on Mac
  2. Learn VIM
  3. Switch to cloud IDE
  4. Vscode Remote SSH
  5. Get a new Desktop / Laptop

Dual Booting Mac OS

My initial attempts to dual boot Ubuntu through a harddisk partition failed after repeated attempts. The problem seems to be around Ubuntu Installer not being able to detect the latest NVMe hard-disks. Ironically I seem to be facing this issue only after upgrading to macOS ElCaptain and higher.

I then used an external harddisk to get it working. Ubuntu full install with the third party closed sources is the best way forward as it has drivers for display, Wifi, etc. Only the camera did not work. Apart from this

  1. Though the ram usage was lesser, there were significant lags
  2. Occasionally the entire OS simply freezes forcing me to restart whereby losing any unsaved work.
  3. Getting the inbuild keyboard mapping to work with ubuntu was a lot harder and slowed the work. For the ease of work needed an extra wired keyboard and mouse.

I thought the issue might be with the default GNOME and tried my luck with other lightweight alternatives like XFCE and LXDE but in vain. Even here though the RAM consumption was even lower the same problems persisted.

VIM

After several unsuccessful attempts to learn VIM, I found that I am hooked to using the mouse cursor to move around for editing code. Developing muscle memory is easier said than done. Apart from this, every VIM user seems to have their configuration and I had to plow through and experiment with these before I could get started. Apart from this, I was learning too many new things already, and learning VIM simply did not work out.

Cloud IDEs

These seemed to be highly promising as they also offer scalability. I checked out both AWS Cloud9 and Codeenvy.

Cloud9 runs on an EC2 instance. This means that you can add resources as per your requirement and have to pay only for the resources consumed.

Codeenvy based on opensource Eclipse-Che though better was based on fixed pricing. It’s free up to 3 GB ram and additional GB would cost about 10 $ each.

Back then I found the interfaces and deployment difficult especially there was overall less community support.

New Desktop/Laptop

Getting a new PC seemed to be far better than paying for the cloud IDE. At least I can use it for multiple purposes and on top of all at least I own something for the price paid. When it comes to selecting a desktop or laptop the former seems to be much better because of reconfigurability and maintenance. Even if one part gets damaged you only need to fix that. On the contrary, a laptop seems to be more portable and can be taken along with you in case you’re traveling thus protecting your work from any interruptions. Also when it comes to the desktop you can make incremental investments whereas in laptops all the investment is tied up the particular configuration. Making such an investment was a hard choice for me back then.

VSCode remote

Finally, there was a breakthrough. I was excited to know that VS Code had released the Remote Development Extension. With this, you can ssh into any cloud instance or even docker container and run your code from there. This also gives the freedom to scale your development environment. I experienced the following benefits

  1. No more VIM or Nano. I can log in via ssh and edit any file directly in the ide with a mouse. This was especially helpful for editing the Ngnix files.
  2. Keep my Mac book clean from installing a lot of libraries or software like Redis, RabbitMQ, etc.
  3. Both the development and production environments had the same OS.
  4. I don’t have to learn a new IDE.
  5. With port forwarding options you don’t have to expose the machine to the public internet and suffer from DNS attacks or hacks.
  6. Running this setup with Google or AWS meant that I only had to pay for the resources used. Google preemptible was the best option here when it comes to price and data persistence. The AWS spot instances though cheap will not retain the resources similar to a normal VPS also the availability of a particular instance type when you want might be less/null. With Google Cloud, an instance with 3.75 GB ram running for the whole month could cost only up to 8 $. Being a development server I usually shut it down and even in the worst case, it gets shut down automatically within 24 hours.
  7. Running data analytics work has never been faster and easier.
  8. Keep in mind to run from the VS Code stable version and not the Insiders which seem to consume a lot of bandwidth.

The above setup freed up a lot of resources tied up with running dockers, databases, development extensions for linting/formatting. You can even delete the environment and set up a new one within minutes. Periodically using Git also ensures that the code changes are saved and retrievable.

Previous Post:

Next Post :

Life depends on Your Interpretation of it! I am crafting an interesting story out of it :)