Mike Driscoll: PyDev of the Week: Mariusz Felisiak

This week we welcome Mariusz Felisiak (@MariuszFelisiak) as our PyDev of the Week! Mariusz is a core developer of the Django web framework and a maintainer of the django-request package. You can follow Mariusz over on Github to see what he’s been up to. Let’s spend some time getting to know Mariusz!

Can you tell us a little about yourself (hobbies, education, etc):

Few facts from my life. I’m a software developer with over 11 years of experience in designing and implementing web applications. I graduated in Computer Science in 2010. I defended my PhD thesis related with numerical algorithms in 2018. I’ve done this after few years of coping in the same time with open source activities, daily job and PhD studies. It was a really intense time.

I’m always eager to contribute to open source projects which takes a large part of my free time, or at least it took before my daughter was born 🙂 I have been in love with Python and Django for 10 years. From 2017 I’m a member of the Django Core Team and the Django Software Foundations. In the middle of March I’m starting as a Django Fellow so Django will become my daily work! The funniest thing is that I should have more “free-time” 🙂 I’m a huge fan of open source and communities that stand behind them, that’s why I’m trying to be as involved as possible.

Outside of the tech world I’m a speedway fan who spends free-time on traveling and winter mountaineering.

Why did you start using Python?

I started my professional career from ancient Oracle’s tools e.g. Oracle Forms & Reports. In 2009, we decided to write all new apps in a more modern framework, finally Python and Django were chosen. I had a sketchy knowledge of Python, so I’ve learned Python and Django in the same time 🙂 and I’ve never looked back.

What other programming languages do you know and which is your favorite?

I know SQL, PL/SQL, Java, JavaScript, HTML, CSS, C and C++. I probably also know how to write “Hello World” in few other programming languages. C and C++ are my favorite since I used them heavily in my studies and PhD.

What projects are you working on now?

I spend most of my time on daily work and Django itself. I’m also maintaining django-request package. In my daily work I maintain and develop RPR platform that connects healthcare stakeholders with global payer network. In Django, I’m mainly involved in the Django ORM.

What non-Python open source projects do you enjoy using?

I would say that vim is in the first place because I use it for writing code. Probably I’m old fashioned but that’s all I need. I don’t use any IDE 🙂

How did you end up becoming a Django core developer?

I worked for almost 10 years with Oracle databases and in the same time I used Django heavily. It seems that is not a common combination. When in 2015 Django folks posted a request for help in maintaining of the Oracle back-end and related CI infrastructure, I decided that is the best moment to help.

I was really surprised with such a friendly welcome, as a person that nobody knows I started to be responsible for Oracle’s CI infrastructure. Contributing to Django became a part of my daily routine. Checking new tickets, finding bugs, optimizations, small clean-ups, reviewing PRs etc. In February 2017 Django devs invited me to the Core Team which was really unexpected. I’ve never taken into account that I could become part of such a smart collection of people. It wouldn’t be possible without help from the entire Django community and especially from Tim Graham, Shai Berger, and Simon Charette. Thanks!

Is there anything else you’d like to say?

Believe in your dreams and be involved!

Thanks for doing the interview, Mariusz!

Planet Python

Preppin’ Data Project: Week 1

Preppin Data project in Tableau

Note: A big thank you to Carl Allchin and Jonathan Allenby for initiating the Preppin’ Data project for our community.

Hunker down, folks. It’s time to prep some data. Welcome to my submission for the first week of the Preppin’ Data project. For beginners, the idea of preparing data can be intimidating. My hope is to provide insight into my process. There are certainly many ways to achieve the same data goals, but here’s where we started:

Preppin Data project in Tableau

Above: Instructions from Week 1 of the Preppin’ Data Project

Preppin Data project in Tableau

The Steps in My Data Prep Process

  • Input Excel: The data lives on a Google Drive with the ability to download as .xlsx. I initially thought it’d be cool to connect directly to a Google Sheet of the info, but, alas, we don’t have that as a connector yet. If you’d like the ability to connect to Google Sheets using Tableau Prep, upvote this idea. In this scenario, I downloaded the data from Google Drive and connected via Excel in Prep.
  • Explore Data: It’s a good practice to explore before making changes to your data to ensure you’re not duplicating efforts.
  • Clean Date: I used Tableau’s MAKEDATE function to combine the When Sold Year and When Sold Month. After validating that the new date was reporting correctly, I removed the original two date columns:

Preppin Data project in Tableau

  • Pivot Colors: A clear beacon for pivoting occurs when you notice multiple columns representing very similar values (red cars, silver cars, etc.). I chose to pivot the data to have one column representing the number of vehicles sold, while another column described the vehicle color.

Note: This is the part where I did not read the Preppin’ Data instructions (oops). My third-grade teacher would be disappointed in me. In hindsight, since the instructions asked to retain the car sales per color columns, pivoting overcomplicated the result:

Preppin Data project in Tableau

Tableau Prep Tip: Want to see what your current data structure looks like in Tableau Desktop? Right-click any step in the flow to preview in Tableau Desktop:

Preppin Data project in Tableau

  • Clean Color: Since I now had one column representing all car color types, I felt the word color included in each value was a bit repetitive. Using Prep’s Split functionality, I separated Blue from Blue Cars and then removed the original field.
  • Aggregate Data: To achieve Total Car Sales per Month/Per Car Dealership, I knew the data must be aggregated. At the time, one row of information in the source described a dealership’s sales of a certain car color in a certain month (i.e. Dealership A sold 377 black cars in January 2018). I took a bit of liberty and assumed that per month/per car dealership meant the month/year (i.e. Jan 2018) instead of the summation of all years in a particular month (i.e. all Januaries).
  • Preview After Aggregation: I like to perform a quick sanity check after any big data manipulation, and I’m glad to have noticed that Sale Date came through (to my surprise) as a datetime rather than date. This is significant because date was used later as a part of my join clause, so it’s important that the fields maintain the correct data type. Additionally, I recognized that the field created in the aggregate step was more aptly named Total Cars Sold than Cars Sold.
  • Maintain Original: I made a decision at this point that, while I wanted to know the total cars sold per dealership per month/year, I also wanted to maintain each row individually so I could utilize the Color dimension. The challenge is that if a field is not used in the Aggregate step (like Color), then Tableau drops it from the flow moving forward. A workaround to this scenario is to branch off a copy of your original data and then join it back up once you’ve created the aggregate. In a way, you’re seeing both the Per Month/Per Dealership info AND the Per Month/Dealership/Car Color in the same data result.
  • Join Flows: The common fields within the two flows were Dealership and Sale Date, so I knew those were going to be included in my join clause. An inner, left or right join would have given me the same exact outcome in this situation because the dealerships/sale dates came from the same original source.
  • Preview Results: Since a join always brings ALL columns from BOTH tables, this caused two redundant fields: Sale Date and Dealership. After a quick clean-up, I’d arrived at my final result.
  • Output: Oddly enough, I commonly forget to add an Output step. I guess I can relate it to climbing to the top of a difficult rock wall but forgetting to ring the bell at the end. I was curious to see how each dealership performed over time, as well as the breakdown in car colors over the same period. I added the Output step (CSV in my case), ran the flow to process the entire amount of data and then connected in Tableau Desktop to build this quick visualization:

Lessons Learned

  • Tableau Prep changed the data type of my date field from date to datetime. I’m still not 100% positive why it happened, but my assumption is that it’s due to what my selection is in the Group by level date hierarchy.
  • Read ALL of the instructions before tackling the problem. While I’m not disappointed with the opportunity to walk us through my approach, I recognize that my result is different than the solution. Additionally, the format of my result needs careful consideration when used in Tableau Desktop. Using the SUM aggregation when displaying the Total Cars Sold field could result in an inaccurate visualization.

The post Preppin’ Data Project: Week 1 appeared first on InterWorks.


Mike Driscoll: PyDev of the Week: Maria McKinley

This week we welcome Maria McKinley (@twiteness) as our PyDev of the Week. Maria is a Senior Software Engineer at the Walt Disney Company and will be a speaker at PyCascades 2019. She is also teaching the Python Certificate Program at the University of Washington Continuing Education. Let’s spend a few moments getting to know her better.


Can you tell us a little about yourself (hobbies, education, etc):

I have a BS in Physics from the University of Washington, and taught myself how to code while working in Neuroscience labs at the same University. I got to work on some amazing research projects, while discovering how much I enjoyed writing code. In October 2015 I started working at the Walt Disney Company as a Senior Software Engineer. I also teach Python, both at the University of Washington and within Disney. And I’m a mom. All of that keeps me pretty busy, but I also try to set aside time for reading, exercise, playing boardgames, and art.

Why did you start using Python?

While I was working in the Neuroscience labs, I often did both software development and system administration. A few years ago, I was running a mail server, and trying to automate some administrative task related to spam. It wasn’t a terribly important or frequent one, so every once in a while I would work on automating it with a Bash script. After a few months, someone told me Python was good for sysadmin, so I tried it. In five minutes, I was able to get a script running, doing the task that had alluded me for so long. I was hooked.

What other programming languages do you know and which is your favorite?

Python is my favorite, and I’ve ben using it pretty exclusively for the last five years or so. I have used Matlab, Javascript, C, C#, C++, Actionscript, Igor, and PHP enough to have a basic understanding. I wouldn’t say I know them all anymore, but I did at one point to varying degrees. Just like a foreign language, if you don’t use it you lose it. I can still read them all, though, so there’s that. I taught myself basic when I was young, and I still remember having to change a bunch of the goto statements when I wanted to add something in the middle. That was hell. Python is better. 🙂

What projects are you working on now?

At the moment, I’m mostly working on teaching, and preparing a talk for PyCascades. I have dreams of playing around with Blender more, but that is going to have to wait for a couple of months. I have a couple of projects at work, as well. Those are REST APIs for backend SysOps colleagues, mostly.

What non-Python open source projects do you enjoy using?

Linux. Okay, I guess I have to go with a Python one, Blender.

How did you end up teaching Python in a university?

While I was at the university, I figured out that since I was self-taught, I did not know what I did not know. So one of the things I did to start figuring out where the holes were was to started a programming group at the university for secluded-in-a-lab-somewhere software developers to get together to learn from each other. The people teaching Python at the university found out about it, and asked me to post about openings they had. So, I did, and then applied myself.

What do students struggle with the most when learning Python?

I wish there were one thing that students struggle with most that we could just fix. Part of it depends on whether they are new to programming or not. People who are new to programming have an entirely different set of problems from those coming from a different language, and honestly, when taught together they can both get frustrated. I would say one of the biggest issues for beginning programmers is mindset. They get frustrated because they can’t write code the first time or that works, not realizing that that is true for experienced programmers, as well. So much of programming is trial and error, and that is not emphasized enough. If you get frustrated when things don’t go right, you will be frustrated most of the time as a developer, especially in the first few years.

How can the Python community make this better?

I think every class or tutorial for beginners should begin by encouraging students to embrace troubleshooting as a fun challenge or game. It is all about learning from your mistakes, and realizing that making mistakes is actually one of the best ways to learn. Teaching is also a great way to learn, so help your fellow students!

Is there anything else you’d like to say?

Super looking forward to adaptation of Good Omens to a television series. That book has a special place in my heart.

Thanks for doing the interview, Maria!

Planet Python

codingdirectional: Return the day in a week with python

Hi, I just have a time to solve another python question today. In this example, we are going to develop a method which will receive a number from the user input and returns which day in a week is that numbers refer to. For example, 1 is Sunday and 2 is Monday. If the number is too large or too small then the program will return an error message. Below is the solution to this question, if you have a better solution don’t forget to leave your answer on the below tweet.

 def whatday(num):      index = num -1     if(index < 0 or index > 6):         return "Wrong, please enter a number between 1 and 7"     else:         return ["Sunday", "Monday", "Tuesday", "Wednesday", "Thursday", "Friday", "Saturday"][index]          

Like the below tweet or follow me on twitter if you will.

Planet Python

Mike Driscoll: PyDev of the Week: Lance Bragstad

This week we welcome Lance Bragstad (@LanceBragstad) as our PyDev of the Week! Lance is a core developer of the OpenStack project. You can find out more about his passions via his website or his Github profile. Let’s spend some time getting to know Lance!

Can you tell us a little about yourself (hobbies, education, etc):

In 2012, I graduated with a degree in Computer Science from North Dakota State University, located in Fargo (yup, like the movie). Since then I’ve become more and more passionate about open-source software. I spend most of my time in the OpenStack ecosystem.

Besides being passionate about open-source software, I’m an avid outdoorsman. My wife and I train for running events together. I also donate time as a volunteer firefighter for our community of about 700 people.

Why did you start using Python?

After I graduated college, I started working at IBM building an OpenStack distribution. Since OpenStack is written in Python, learning Python was a requirement, and that’s how I was introduced to the language. Despite being given the opportunity to use different languages in college, I never really experimented with Python. Using it in a new setting with a new job was an exciting learning experience.

What projects are you working on now?

Currently, I spend the majority of my time working within OpenStack’s authentication and authorization realm. There is a dedicated identity service, called keystone, along with several libraries that orchestrate authorization across OpenStack.

Since there are many ways to approach identity management, it’s interesting to work on the piece that handles all of that. Keystone can be used to manage users with MySQL. It can also be configured to use LDAP or even identity providers that issue SAML assertions or use OpenID Connect.

The other exciting part is that OpenStack services offer such a rich set of APIs to users. Since services consume authorization information from keystone, keystone has to support protecting all of those APIs, which presents an interesting set of problems to solve.

Which Python libraries are your favorite (core or 3rd party)?

One library that comes to mind is pyca/cryptography. We used this library in keystone to implement a form of authenticated encryption, solving a significant scale and operations issue at the time. We looked around at some other options, but pyca/cryptography was a near perfect fit.

Before formally pulling it in as a dependency, I went through and read the source code and tests. The authors did a great job organizing functionality, and the tests were easy to parse. Despite the complexity involved in a topic like cryptography, it was refreshing to go through the code and see things laid out cleanly. Being able to clearly understand the intricacies of the library made solving a tough problem a little easier.

How did you get started with OpenStack?

I started working on OpenStack in January of 2012, shortly after Diablo was released and the community was busy developing Essex. I was wrapping up my undergraduate degree at NDSU, and we were entering our final semester before graduation in May. I was one of four members working on a capstone project. We were asked to find use cases to deploy Infrastructure-as-a-Service, primarily looking for people who would consume a private OpenStack cloud if we built one.

By the mid-term, we had the Diablo release deployed on three physical machines using crowbar, chef, and juju (fun fact: only two of the hosts had virtualizations extensions). We spent finals week putting the finishing touches on a brand new Essex deployment (including some marginally better hardware) shortly after it was released (2012 April 5).

Our deployment was primarily used by faculty and graduate students, fulfilling HPC use cases for research.

Where is OpenStack going in the future?

This is my own opinion, but I see OpenStack continuing to stabilize. I think we, as the OpenStack community, are getting better at working towards common goals, which is a challenge given the breadth of projects under OpenStack governance. I think this is exciting because it provides a consistent look and feel to our users, and improves the experience they have with the software we write.

For years contributors have poured tons of effort into features and functionality, especially within specific domains like networking, storage, or compute infrastructure. I think now we’re getting to the point where continuing to move forward means smoothing out usability between components.

I see a lot of truth in Robert Martin’s view that software design is a constant dynamic tension instead of a utopian endstate that is written once*. If you look, you can see we are making choices to simplify software across OpenStack. In my opinion, the tension is shifting from intense feature development to stability and simplicity for users and maintainers. It’s exciting to see these transitions happen when you’ve been involved with a project for a long time, and what they bring to everyone involved in our community.

* Uncle Bob talks about this extensively in his book Clean Architecture, which is a great read.

Thanks for doing the interview, Lance!

Planet Python