Kushal Das: That missing paragraph

In my last blog post, I wrote about a missing paragraph. I did not keep that text anywhere, I just deleted it while reviewing the post. Later Jason asked me in the comments to actually post that paragraph too.

So, I will write about it. 2018 was an amazing year, all told;, good, great, and terrible moments all together. Things were certain highs , and a few really low moments. Some things take time to heal, some moments make a life long impact.

The second part of 2018 went downhill at a pretty alarming rate, personally. Just after coming back from PyCon US 2018, from the end of May to the beginning of December, within 6 months we lost 4 family members. On the night of 30th May, my uncle called, telling me that my dad was admitted to the hospital, and the doctor wanted to talk to me. He told me to come back home as soon as possible. There was a very real chance that I wouldn’t be able to talk to him again. Anwesha and I, managed to reach Durgapur by 9AM and dad passed away within a few hours. From the time of that phone call, my brain suddenly became quite detached, very calm and thinking about next steps. Things to be handled, official documents to be taken care of, what needs to be done next.

I felt a few times that I’dburst into tears, but, the next thing that sprang to mind was that if I started crying, that would affect my mother and rest of the family too. Somehow, I managed not to cry and every time I got emotionally overwhelmed, I started thinking about next logical steps. I actually made sure, I did not talk about the whole incident much, until recently after things settled down. I also spent time in my village and then in Kolkata.

In the next 4 months, there have been 3 more deaths. Every time the news came, I did not show any reaction, but, it hurt.

Our education system is what supposed to help us grow in life. But, I feel it is more likely, that school is just training for the society to work cohesively and to make sure that the machines are well oiled. Nothing prepares us to deal with real life incidents. Moreover, death is a taboo subject with most of us.

Coming back to the effect of these demises, for a moment it created a real panic in my brain. What if I just vanish tomorrow? In my mind, our physical bodies are some amazing complex robots / programs. When one fails, the rest of them try to cope , try to fill in the gaps. But, the nearby endpoints never stay the same. I am working as usual, but, somehow my behavior has changed. I know that I have a long lasting problem with emails, but, that has grown a little out of hand in the last 5 months. I am putting in a lot of extra effort to reply to the emails I actually managed to notice. Before that, I was opening the editor to reply, but my mind blanked, and I could not type anything.

I don’t quite know how to end the post. The lines above are almost like a stream of consciousness in my mind and I don’t even know if they make sense in the order I put them in. But, at the same time, it makes sense to write it down. At the end of the day, we are all human, we make mistakes, we all have emotions, and often times it is okay to let it out.

In a future post, I will surely write another post talking about the changes I am bringing in my life to cope.

Planet Python

Mike Driscoll: PyDev of the Week: Kushal Das

This week we welcome Kushal Das (@kushaldas) as our PyDev of the Week! Kushal is a core developer of the Python programming language and a co-author of PEP 582. You can learn more about Kushal by checking out his blog or his Github profile. Let’s take a few moments to get to know Kushal better!


Can you tell us a little about yourself (hobbies, education, etc):

I am a staff member of Freedom of the Press Foundation. We are a non-profit that protects, defends, and empowers public-interest journalism in the 21st century. We work on encryption tools for journalists and whistleblowers, documentation of attacks on the press, training newsrooms on digital security practices, and advocating for the the public’s right to know.

I am also part of various Free Software projects through out my life. I am a core developer of CPython, and a director of the Python Software Foundation. I am part of the core team of the Tor project. I am a regular contributor to Fedora Project for over a decade now.

I co-ordinate https://dgplug.org along with a large group of friends and fellow contributors in various projects. We spend time together in learning new things and helping out each other on the #dgplug IRC channel on Freenode server. Feel free to visit the channel and say “Hi” to us.

I try to write about the things I learn regularly on my blog.

Why did you start using Python?

I started learning Python at the end of 2005. I wanted to write code for my new Nokia phone and Sirtaj Singh Kang suggested me to start learning Python for the same. While doing so I found that I had to write much less number of lines of code and also it was much easier to understand. I started talking more with the wider Python community over Internet and that hooked me into it more. As Brett Cannon said: “Came for the language, stayed for the community.” is true for many of us.

What other programming languages do you know and which is your favorite?

Through out my programming life, I kept learning a new language in every 8 months to a year. Before I started writing Python, I used to write C/Java/PHP based on what I was working on. Around 2009 I started spending time with functional programming, and loved Lisp a lot. I spent around a year to keep writing more Lisp and was trying to figure out how to use the ideas from there in my daily Python programming life. From 2013 I started writing Go and I do have many projects written in Go.

But, lately I am writing more and more of Rust. I really like the community and also the compiler 🙂

Just in case anyone wants to know how much we love Python in the family, our daughter is named “Py” 🙂

What projects are you working on now?

In my day job, I maintain SecureDrop project along with an amazing team of maintainers and community. SecureDrop is an open source whistleblower submission system that media organizations can install to securely accept documents from anonymous sources. It was originally coded by the late Aaron Swartz and is now managed by Freedom of the Press Foundation.

I am also working on various Python projects which will enable us to have a new Desktop client for the journalists on Qubes OS. Qubes Ansible is another project where I am trying to make sure that we can use Ansible to maintain our Qubes systems.

Which Python libraries are your favorite (core or 3rd party)?

I think I use json module from stdlib and requests module as third party almost everywhere. IIRC my first ever Cpython patch was about adding tests for json module.

In the Python world there are many other amazing libraries which I use regularly, most of them are the product of our amazing community.

What top three things have you learned contributing to open source projects?

  • People are more important than any code.
  • Be nice to everyone.
  • Communication is the key tool for everything in this modern connection world. We have to do a lot more communication over writing than video/audio calls.

Is there anything else you’d like to say?

I would suggest new programmers to look into more number of upstream projects. We need help in various level in all of the projects, so there is a chance to contribute not only by code, but in many different ways.

Last, but least, I would love to mention my wife Anwesha, who is being from a complete different background, helped me to contribute more to the upstream projects and also herself started helping out projects as required.

Thanks for doing the interview, Kushal!

Planet Python

Kushal Das: 2018 blog review

Last year, I made sure that I spend more time in writing, mostly by waking up early before anyone else in the house. The total number of posts was 60, but, that number came down to 32 in 2018. The number of page views were though 88% of 2017.

I managed to wake up early in most of the days, but, I spent that time in reading and experimenting with various tools/projects. SecureDrop, Tor Project, Qubes OS were in top of that list. I am also spending more time with books, though now the big problem is to find space at home to keep those books properly.

I never wrote regularly through out the year. If you see the dates I published, you will find that sometimes I managed to publish regularly for a month and then again vanished for sometime.

There was a whole paragraph here about why I did not write and vanish, but, then I deleted the paragraph before posting.

You can read the last year’s post on the same topic here.

Planet Python

Kushal Das: Flatkpak application shortcuts on Qubes OS

In my last blog post, I wrote about Flatpak applications on Qubes OS AppVMs. Later, Alexander Larsson pointed out that running the actual application from the command line is still not user friendly, and Flatpak already solved it by providing proper desktop files for each of the application installed by Flatpak.

How to enable the Flatpak application shortcut in Qubes OS?

The Qubes documentation has detailed steps on how to add a shortcut only for a given AppVM or make it available from the template to all VMs. I decided to add it from the template, so that I can click on the Qubes Setting menu and add it for the exact AppVM. I did not want to modify the required files in dom0 by hand. The reason: just being lazy.

From my AppVM (where I have the Flatpak application installed), I copied the desktop file and also the icon to the tempplate (Fedora 29 in this case).

qvm-copy /var/lib/flatpak/app/io.github.Hexchat/current/active/export/share/applications/io.github.Hexchat.desktop qvm-copy /var/lib/flatpa/app/io.github.Hexchat/current/active/export/share/icons/hicolor/48x48/apps/io.github.Hexchat.png 

Then in the template, I moved the files to their correct locations. I also modified the desktop file to mark that this is a Flatpak application.

sudo cp ~/QubesIncoming/xchat/io.github.Hexchat.desktop /usr/share/applications/io.github.Hexchat.desktop sudo cp ~/QubesIncoming/xchat/io.github.Hexchat.png /usr/share/icons/hicolor/48x48/ 

After this, I refreshed, and then added the entry from the Qubes Settings, and, then the application is available in the menu.

Planet Python

Kushal Das: Building wheels and Debian packages for SecureDrop on Qubes OS

For the last couple of months, the SecureDrop team is working on a new set of applications + system for the journalists, which are based on Qubes OS, and desktop application written particularly for Qubes. A major portion of the work is on the Qubes OS part, where we are setting up the right templateVMs and AppVMs on top of those templateVMs, setting up the qrexec services and right configuration to allow/deny services as required.

The other major work was to develop a proxy service (on top of Qubes qrexec service) which will allow our desktop application (written in PyQt) to talk to a SecureDrop server. This part finally gets into two different Debian packages.

  1. The securedrop-proxy package: which contains only the proxy tool
  2. The securedrop-client: which contains the Python SDK (to talk to the server using proxy) and desktop client tool

The way to build SecureDrop server packages

The legacy way of building SecureDrop server side has many steps and also installs wheels into the main Python site-packages. Which is something we plan to remove in future. While discussing about this during PyCon this year, Donald Stufft suggested to use dh-virtualenv. It allows to package a virtualenv for the application along with the actual application code into a Debian pacakge.

The new way of building Debian packages for the SecureDrop on Qubes OS

Creating requirements.txt file for the projects

We use pipenv for the development of the projects. pipenv lock -r can create a requirements.txt, but, it does not content any sha256sums. We also wanted to make sure that doing these steps become much easier. We have added a makefile target in our new packaging repo, which will first create the standard requirements.txt and then it will try to find the corresponding binary wheel sha256sums from a list of wheels+sha256sums, and before anything else, it verifies the list (signed with developers’ gpg keys).

PKG_DIR=~/code/securedrop-proxy make requirements 

If it finds any missing wheels (say new dependency or updated package version), it informs the developer, the developer then can use another makefile target to build the new wheels, the new wheels+sources do get synced to our simple index hosted on s3. The hashes of the wheels+sources also get signed and committed into the repository. Then, the developer retries to create the requirements.txt for the project.

Building the package

We also have makefile targets to build the Debian package. It actually creates a directory structure (only in parts) like rpmbuild does in home directory, and then copies over the source tarball, untars, copies the debian directory from the packaging repository, and then reverifies each hashes in the project requirements file with the current signed (and also verified) list of hashes. If everything looks good, then it goes to build the final Debian package. This happens by the following environment variable exported in the above mention script.

DH_PIP_EXTRA_ARGS="--no-cache-dir --require-hashes" 

Our debian/rules files make sure that we use our own packaging index for building the Debian package.

#!/usr/bin/make -f  %: 	dh $  @ --with python-virtualenv --python /usr/bin/python3.5 --setuptools --index-url https://dev-bin.ops.securedrop.org/simple 

For example, the following command will build the package securedrop-proxy version 0.0.1.

PKG_PATH=~/code/securedrop-proxy/dist/securedrop-proxy-0.0.1.tar.gz PKG_VERSION=0.0.1 make securedrop-proxy 

The following image describes the whole process.

We would love to get your feedback and any suggestions to improve the whole process. Feel free to comment in this post, or by creating issues in the corresponding Github project.

Planet Python