I'm currently building Miniature.io, you can have a look here.

Linux’izing your Windows PC into a dev machine – Part 2

February 20, 2018

(Hey, for Part 1 have a look here: https://cepa.io/2018/02/10/linuxizing-your-windows-pc-part1/)

 

Last week I presented how to setup basic Linux features on Windows through the Bash On Windows aka Windows Subsystem for Linux (WSL). You should be now able to run variety of Linux apps both command line and with GUI. Also, presented how to install Docker and Vagrant which are popular tools to isolate your host environment from development playground.

 

 

Let’s install essential tools for variety of modern software development technologies and get prepared for some hardcore coding battles!

Reading feedback from comments on various websites really helped to select what tools should I try to install and run in WSL. I’m not actively using all of the languages and technologies tested down below, so feel free to comment if something isn’t right or perhaps you have some suggestions.

The Part 2 of Linux’izing Windows will be focused mostly on Web, DevOps and fancy Machine Learning stuff. I have skipped the Desktop or Mobile development as it is already perfectly doable on native Windows via dedicated tools, languages and technologies. Ok, maybe with exception or Linux desktop development, but that should just work with the X Server regardless of using Qt, GTK, or any other toolkit. OSX or iOS? Well for that you’ll need a Mac anyways.

The following is the table of contents from Part 1 and Part 2, you can jump straight to your weapon of choice.

Part I

Part II

 

 

Sharing environment between Windows and WSL

Probably should have started by desrcibing this last week, anyway, here it goes.
So, you install the WSL on your Windows PC and then what?

 

Where is WSL installed?

Your Linux filesystem will be created in your user’s AppData directory, preciesly in C:\Users\your.username\AppData\Local\lxss. It is a hidden folder, inside you’ll find root, home and other standard Unix directories, you should avoid manual changes there from outside of WSL.

 

How to share files between Windows and WSL?

Guess, there are plenty of ways, I’ll show you how I do it.

By default WSL has access to your Windows drive, it is mounted in /mnt/c, for example your Windows home directory is C:\Users\your.username and in WSL this directory is available at /mnt/c/Users/your.username.

I keep all my projects in the projects folder in my Windows home directory C:\Users\cepa\projects. When you start bash you’ll get into your WSL home directory which is located in WSL filesystem, however you can create a symlink to your projects folder.

cd ~
ln -s /mnt/c/Users/your.username/projects projects
cd projects

 

Accessing other disks and network locations?

Normally on Linux you would directly mount a filesystem from a disk using the mount command and a Windows network location using Samba. In WSL it is a bit different though.

WSL filesystem uses DrvFs which seems to be a Microsoft’s invention to share filesystem between Windows host and WSL layer, and apparently, you can use DrvFs to mount other Windows drives and network locations.

You can have a look here for more details:
https://blogs.msdn.microsoft.com/wsl/2017/04/18/file-system-improvements-to-the-windows-subsystem-for-linux/

  • Quick how to. Let’s say you have a drive D: in your Windows PC and to mount it run:
    sudo mkdir /mnt/d
    mount -t drvfs D: /mnt/d -o metadata,uid=1000,gid=1000,umask=22,fmask=111

    Now you can cd /mnt/d and access files. That works also with network locations mounted in Windows as shared drives.

  • For network locations you have two options, first mentioned above, just mount it as a drive, second is mounting directly in WSL:
    sudo mkdir /mnt/share
    sudo mount -t drvfs
    
    
  • The df -h works too:

 

 

Getting basic Linux utilities and services

Lets get ready for upcoming coding battles!

First, you can install bunch of standard Linux tools which are often required while installing other software packages or are just handy.

sudo apt-get install -y \
  apt-transport-https \
  lsb-release \
  ca-certificates \
  build-essential \
  curl wget unzip zip tmux git

It will get you the GCC compiler, Git and bunch of other utilities. There are lots of other tools available and for most you can follow Ubuntu how to’s to install them.

 

Accessing your Windows PC over SSH

This one is pretty cool 😉

With WSL you not only can use SSH to access remote Linux servers but you can actually run OpenSSH server in your WSL to access your Windows PC from outside! Just like any regular Linux box.

So, OpenSSH server on WSL:

  1. Reinstall OpenSSH in WSL
    sudo dpkg-reconfigure openssh-server
    
    
  2. Edit SSH configuration, sudo nano /etc/ssh/sshd_config, and change the following parameters:
    Port 2222
    # ...
    UsePrivilegeSeparation no
    # ...
    PasswordAuthentication yes

    Please be aware that it allows to use password authentication which is not the best idea, if you need SSH server please read about SSH hardening and password-less authentication.

  3. Open Windows firwall for a custom port (2222)
    In Windows 10 you might have an SSH Server Broker Services running which occupy port 22, thus the change to port 2222. You’ll need to open Windows 10 firewall to allow incoming connections to your SSH server.
    – Type WF.msc in the Start menu
    – In the Windows Firewall with Advanced Security click on Inbound Rules
    – Add a new rule for TCP 2222 and allow the connection
  4. Restart SSH serwer in WSL
    sudo service ssh --full-restart
  5. SSH into your Windows PC over from a remote host
    ssh your.wsl.username@your.windows.pc.ip -p 2222

Done, should just work.

 

 

LAMP stack for PHP development

PHP is an amazingly popular technology for creating various web projects. Many developers share the love-hate relationship with it due to its low entry and loads of crap code around and projects that are total mess from code perspective. WordPress anyone?

Before WSL times, you had basically two options, either use Linux and develop in PHP on a production like environment or struggle with WAMP bundle which is an acronym for Windows Apache MySQL PHP.

With WSL you can have a regular LAMPLinux Apache MySQL PHP – setup on your Windows PC that is very similar to production Linux based environment where your PHP application is most likely to be hosted.

 

Apache HTTP Server with PHP 7

This is going to be a pretty standard setup, you can of course install Nginx+PHP FPM or other variations, up to you.

  1. Install Linux packages for Apache and PHP
    sudo apt-get install \
      apache2 \
      libapache2-mod-php7.0 \
      php7.0 \
      php7.0-cli \
      php7.0-common \
      php7.0-curl \
      php7.0-dev \
      php7.0-gd \
      php7.0-json \
      php7.0-mysql \
      php7.0-opcache \
      php7.0-xml \
      php7.0-bz2 \
      php7.0-intl \
      php7.0-mbstring \
      php7.0-mcrypt \
      php7.0-zip
  2. Enable additional Apache modules like mod_rewrite
    sudo a2enmod rewrite
  3. Restart Apache service
    sudo service apache2 restart
  4. Create sample PHP file, run:
    sudo nano /var/www/html/phpinfo.php

    and write:

    <?php phpinfo();

    Ctrl+X

  5. Open http://localhost/phpinfo.php, you should get the PHP Info page.

 

MySQL Server

On Windows PC you can either have a MySQL server installed as a regular Windows service or inside WSL, here goes the WSL way:

  1. Install Linux MySQL packages:
     sudo apt-get install mysql-server mysql-client
    
    
  2. Start MySQL service:
     sudo service mysql start
  3. Open MySQL command line client, run
     sudo mysql

    it should display a MySQL prompt, can type:

     show databases;

    which should print something like:

  4. You can use MySQL Workbench and manage your WSL’s MySQL using it, use localhost:3306 for the connection.

 

Composer

Composer is a de-facto standard tool for PHP dependency management. You can easily install it in WSL, for more details visit https://getcomposer.org/download/ here’s the short version:

  1. Install Composer
    php -r "copy('https://getcomposer.org/installer', 'composer-setup.php');"
    php -r "if (hash_file('SHA384', 'composer-setup.php') === '544e09ee996cdf60ece3804abc52599c22b1f40f4323403c44d44fdfdd586475ca9813a858088ffbc1f233e9b180f061') { echo 'Installer verified'; } else { echo 'Installer corrupt'; unlink('composer-setup.php'); } echo PHP_EOL;"
    php composer-setup.phpphp -r "unlink('composer-setup.php');"
  2. Make it global
    sudo mv composer.phar /usr/local/bin/composer
    chmod +x /usr/local/bin/composer
  3. Run it

 

Symfony Project

So, far so good however, a simple phpinfo.php isn’t really a PHP project.
Let’s try something heavier, a lot heavier…

Symfony is probably the most popular enterprise-grade PHP framework of today. Typical boilerplate project is made of around 2800 files and weights around 8MB, that’s quite a lot of files to process, which makes it… a bit slow 😉

Details are available here https://symfony.com/doc/current/best_practices/creating-the-project.html but for sake of this blog post let’s do it quick, you can create a fresh Symfony project using Composer mentioned above:

  1. Create a new Symfony project
     composer create-project symfony/skeleton sftest

    It will take a while to finish…

  2. Run the built-in PHP test server
    cd sftest
    php -S 127.0.0.1:8000 -t public
  3. Open http://localhost:8000/ in a browser on Windows, you should get Symfony project page.

 

 

MEAN stack for Node.js development

MEAN stands for MongoDB Express.js Angular Node.js.

JavaScript and Node.js are the backbone of modern frontend development. With WSL you can easily install Node.js ecosystem and use just like on Linux or MacOSX.

 

Node.js v9 and NPM

For details you can have a look here https://nodejs.org/en/download/package-manager/#debian-and-ubuntu-based-linux-distributions

  1. Install Node.js on WSL
    curl -sL https://deb.nodesource.com/setup_9.x | sudo -E bash -
    sudo apt-get install nodejs
  2. Avoid permission errors (EPERM) with NPM by setting NPM to use a folder owned by your user to install Node.js components and binaries
    mkdir ~/npm
    sudo npm config set prefix ~/npm
    sudo chown -R $USER:$(id -qn $USER) ~/.config
  3. Add new NPM location to your WSL’s PATH variable
    echo 'export PATH="~/npm/bin:$PATH"' >> ~/.bashrc
    source ~/.bashrc
  4. Update NPM packages
    npm install -g npm
  5. Install common Node.js components (optional)
    Once Node.js and NPM are installed you can install bunch of common Node.js utilities

    npm install -g typescript bower grunt gulp less sass yarn webpack

So, it looks good but again, let’s try something heavier.

 

Angular in WSL

NPM is there so you can try with a new boilerplate Angular project.

  1. Install Angular CLI
     npm install -g @angular/cli

    Well… it will download half of Internet of dependencies so be patient…

  2. Create a new Angular project
     ng new ngtest
  3. Build and run Angular in Node.js server
     cd ngtest
     ng serve --host 0.0.0.0 --disable-host-check
  4. Open http://localhost:4200/ on Windows, you should get a blank Angular 4 project page.

 

React in WSL

There seem to be a religious war between Angular and React devotees so I shouldn’t forget about the other pillar of frontend stack, again, with NPM it’s easy.

  1. Install React’s creator
     npm install -g create-react-app

    Wait… again… the other half of Internet of dependencies…

  2. Create a new React project
     create-react-app reacttest
  3. Build and run React in Node.js server
     cd reacttest
     npm start
  4. Open http://localhost:3000/, interestingly React in WSL has triggered my Windows default browser to automagically open that link for me.

 

MongoDB in WSL

Well, I admit, this one was a bit of pain…
Generally you can install and use MongoDB in WSL, there are some issues with the way the MongoDB service is being run in WSL/Linux.

The following is based on the officiall how to https://docs.mongodb.com/manual/tutorial/install-mongodb-on-ubuntu/

  1. Instal Mongo Community Edition 3.6
     sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 2930ADAE8CAF5059EE73BB4B58712A2291FA4AD5
     echo "deb [ arch=amd64,arm64 ] https://repo.mongodb.org/apt/ubuntu xenial/mongodb-org/3.6 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-3.6.list
     sudo apt-get update
     sudo apt-get install mongodb-org
  2. Unfortunately, Mongo won’t start as a service, seems to be an issue with Systemd/Dbus, but you can run is the other way. You can start mongod as mongodb user using su:
     sudo su - mongodb -s /bin/bash -c "/usr/bin/mongod --config /etc/mongod.conf"
  3. Connect to Mongo server using Mongo client:
  4. Optionally you can use external clients like Mongo Booster and connect to Mongo server from a Windows app.

 

 

Python, Django and Machine Learning

Python is a fabulous language. I hardly can think of any project that can’t be built with it, anything from web applications, through cloud, image processing to machine learning. You can do it all with Python and the language itself is super easy to learn.

Python has its versions available for Windows, however the power of this language lies in the thousands of libraries created by community. Many of these libraries are either dependant of native libraries available on Linux or require direct device access for some features and that seem to be an issue in many cases while using a native Python for Windows.

It seems WSL has solved some of these issues. Although, you won’t get direct access to devices like GPU for say Machine Learning – at least I’m not aware you can – however, with WSL you can use libraries that are difficult to install natively on Windows, including these for mentioned Machine Learning.

 

Python in WSL and Virtualenv

First, let’s install Python in WSL and additionally virtualenv to manage project specific environments. This way you can have multiple version of Python in your projects, like Python 2.7 and Python 3.

  • sudo apt-get install python python-dev python-setuptools
    sudo easy_install pip
    sudo pip install virtualenv

Now run Python to confirm it works:

  • python --version

Good, now can build some Python projects.

 

Django in WSL

As a starter, can try with a fresh Django project.

  1. Create a new virtualenv with Python 3 for a Django project
     virtualenv -p python3 djangotest
  2. Activate virtualenv
     cd djangotest
     source ./bin/activate
  3. Install Django
     pip install Django
  4. Create new Django project
     django-admin startproject myapp
  5. Start Django
     cd myapp
     python manage.py runserver
  6. Open http://localhost:8000/, should get the default Django app page.

 

Python Machine Learning in WSL

I’m neither Data Scientist nor Machine Learning expert so this section is based on information found in these two articles:

https://github.com/dzorlu/GADS/wiki/Guide-to-installing-machine-learning-libraries-in-python
https://machinelearningmastery.com/setup-python-environment-machine-learning-deep-learning-anaconda/

  1. Create and activate a new virtualenv
     virtualenv pyml
     cd pyml
     source ./bin/activate
  2. Install NumPy
     pip install numpy
  3. Install SciPy
     pip install scipy
  4. Install Matplotlib
     sudo apt-get install libfreetype6-dev
     pip install matplotlib
  5. Install Patsy
     pip install patsy
  6. Install StatsModels
     pip install statsmodels
  7. Install ggplot
     pip install ggplot
  8. Install Sciki-learn
     pip install scikit-learn
  9. Install Theano
     pip install theano
  10. Install Tensorflow
     pip install tensorflow
  11. Install Keras
     pip install keras

Seems to be all right.

Could have installed them in a single pip instal … command, but wanted to make sure each or these works and catch potential issues.

 

Java and JVM ecosystem

When it comes to Java and JVM I would suggest installing it both direclty on your Windows and separately in WSL. Why? Because you’ll need Java for example to run IDEs like Eclipse or NetBeans and you could have a different version of Java in WSL.

First, I installed Oracle JDK 8 for Windows on my Windows host to run various Windows programs.

Second, I installed Oracle JDK 9 for Linux in my WSL. You can follow this guide https://www.digitalocean.com/community/tutorials/how-to-install-java-with-apt-get-on-ubuntu-16-04

Short version:

  1. Add Webupd8 repo
     sudo add-apt-repository ppa:webupd8team/java
     sudo apt-get update
  2. Install Oracle Java 9
     sudo apt-get install oracle-java9-installer
  3. Run Java in WSL to check version
     java -version

 

Maven in WSL

Maven and Ant are common tools in Java development environment and installing it on WSL is straightforward.

sudo apt-get install maven
mvn --version

Ok, so Java seems to be working fine, let’s try to build some real Java project.


Spring Boot in WSL

As an example you can use one of the Spring Boot samples which is a very popular Java framework. I’ve used the one from https://spring.io/guides#gs

  1. Clone the project
     git clone https://github.com/spring-guides/gs-spring-boot
  2. Build it
     cd gs-spring-boot
     cd complete
     chmod +x mvnw
     ./mvnw spring-boot:run
  3. Open http://localhost:8080/, you should get a page saying “Greetings from Spring Boot!”

 

SDKMAN! in WSL

To install other components for JVM ecosystem you can use SDKMAN!
Installation is easy, follow the official documentation at http://sdkman.io/install.html

  1. Install SDKMAN!
     curl -s "https://get.sdkman.io" | bash
  2. Add SDKMAN to the PATH
     source "$HOME/.sdkman/bin/sdkman-init.sh"
  3. Run and check version
     sdk version

 

Scala in WSL


To get Scala in WSL simply run SDKMAN.

sdk install scala
scala -version

All good!


Gradle in WSL

Can try installing Gradle in WSL using SDKMAN.

sdk install gradle 4.5.1
gradle -v

All good!

 

Kotlin in WSL

Let’s try with Kotlin, again with SDKMAN.

sdk install kotlin
kotlin -version

All good!

 

DevOps tools for Amazon AWS

With WSL you can easily run DevOps tools to manage your cloud infrastructure.

Azure tools should work out of the box in Windows, right Microsoft?
So, let’s get AWS tools up and running.

 

AWS CLI in WSL

If you work with Amazon AWS Cloud you’ll definitely need the AWS CLI which is the official command line client to manage your infrastructure in AWS cloud.

You can easily install it with Python Pip, see the Python section first and install Python. When done just run the following.

sudo pip install awscli
aws configure

Follow the official documentation to configure it https://docs.aws.amazon.com/cli/latest/userguide/installing.html

You’ll need an AWS Account with Access Key and Secret Key which you can set up in IAM in the AWS Web Console.


Terraform and Packer in WSL

Ever heard of the Infrastructure as Code concept? Basically you use a meta language to describe how your infra should “look like” and that meta description is processed by some software that eventually will build up your servers, networking, storage, etc.

If you look for a new hobby in life you can use AWS CloudFormation and write ass long JSON objects to describe how you want your AWS infra to be.

But, it’s 2018 and there are better tools. Apparently, HashiCorp heard the moaning of DevOps hordes and inventend two tools Terraform and Packer to automate cloud operations. They aren’t perfect, but A LOT better than JSON and CloudFormation.

You have two options on Windows, you can use native Terraform and Packer builds for Windows or you can use Linux ones in WSL. It’s a blog post about WSL so I’ll use the Linux ones.

  1. Install Terraform from https://www.terraform.io/downloads.html
    Download the Linux 64bit version
    Move it to the bin directory:

     mv terraform /usr/local/bin
     chmod +x terraform /usr/local/bin
     terraform -v
  2. Install Packer from https://www.packer.io/downloads.html
    Download Linux 64bit version
    Move it to the bin directory:

     mv packer /usr/local/bin
     chmod +x packer /usr/local/bin
     packer -v

Sample output:

Looks good!

 

Things that suck

Oi! Microsoft, you reading this? 😉
Can you guys please improve a couple of things?

  • IO is slow, very slow!
    I haven’t done any real benchmark yet but generally using WSL feels sluggish, a lot slower than working on Linux. Subjectively the difference is 10x or so which is a serious issue.
    The problem is especially while working with technologies like PHP or Node.js where projects have many thousands of files and are processing them. On Linux that just flies, on WSL it’s often unbearably slow.
    Frankly, I’m working on two computers now. One laptop with WSL on it which is my primary Office/Mail/Remote work machine and a normal stationary workstation with multiple monitors and Linux on it. Whenever I have a heavier task to do I feel more comfortable working from my workstation. To be fair, it would be good to perform a benchmark of WSL on a real workstation rather than a low power laptop but still I bet the IO speed difference would be significant.
  • Networking
    Many sysadmin tools just don’t work. I guess that’s because the WSL kernel doesn’t have all the features and have to share networking with Windows. If you’re primary workload is development WSL is just all right, but if you mostly do network operations then you’d be way better on Linux or Linux VM.
  • Lack of Systemd/Dbus and system wide startup
    See the MongoDB bit of this article. Linux packages usually contains the software and the startup scripts. Often, the startup scripts depend on Systemd which is not supported(?) in WSL and that causes issues with starting Linux services. That being said, be aware that you need to keep at least one Bash on Windows instance running to keep your services like say Apache, MySQL or SSH alive.
  • Filesystem permissions
    Bash on Windows has its own filesystem embedded into Window, however it is not fully compatible with typical Linux filesystem. You might have issues while working with Node.js or some other toolkits that need setfacl to work properly for various users other than your own which owns the current bash process.
  • Mounting external filesystems
    DrvFs works but can’t mount things like NFS or even SSHFS via Fuse. That would be nice to have.
  • Direct device access
    When you go to /dev there are only some dummy devices, can’t access drives or other hardware. Not sure if that would be possible to implement though.

 

Hot or not?

Am I going to ditch Linux completely?

No, at least not yet.

WSL is a great improvement over working on either a raw Windows 10 or Windows with Cygwin however there are times that you need a real not a castrated Linux like WSL. I think the 80/20 rule applies to it as well. For a regular daily use you can perform most of your tasks within WSL, but whenever you need some more advanced networking, device access or performance having a real Linux workstation is just better.

Personally I think WSL shines by bringing long missing features to Windows, it allows you to get a flexible machine both for office and development workstation use. With release of the build 1709, Windows 10 has became a real alternative for Mac OS X as all-in-one environment for developers and perhaps in time if it gets event better it might be a serious alternative for Linux desktop workstations.

Time will tell.

 

Other links:

 

Feel free to post feedback in comments, social media, email, etc and thanks in advance for sharing this article.

 

Comments (6)

  1. Markus Kohler says:

    Check phoronix.com for I/O and other benchmarks. IIRC I/O is 100 times slower on WSL..

  2. dzek says:

    > Node.js where projects have many thousands of files and are processing them. On Linux that just flies, on WSL it’s often unbearably slow.

    well, i guess you shouldn’t expect anything nice, as node development natively on windows is slower than on linux (try installing dependencies of same big project on same machine – it’s around 3x slower on windows. it’s faster when Linux is put inside VM inside Windows, weird but true).

  3. BRUNO MOURA says:

    Awesome fantastic step by step tutorial

    Would be nice see this process automated by vagrant file definition.

    The best developer setup that I ever have seen!

    Also, I used this to setup my new clean VM image based on the Xubuntu Core lightweight distro.

  4. dev-jun says:

    Thanks for the guide, it help a lot.

    I actually prefer this than using VMware, it cut’s the system resources considerably.

    Yeah, creating a react App is really slow, but I probably can tolerate that, since it’s just on creating the App part, after that, the development doesn’t involved that much IO operation. Could this slow IO process has something to do with the anti virus in our systems, I have Windows Defender installed.

    Again, thank you for the step by step tutorial.

  5. zakius says:

    well, since most common tools just work natively on windows I don’t really see the purpose of WSL, I’d rather focus on bringing the missing ones in native form instead

    on my personal setup the only thing WSL does is running mosh (and sometimes first phase of testing some oh-so-great end-user app before I reboot, though still haven’t found anything except mosh that I miss on windows)

Drop a comment

Your email address will not be published.