Davide Moro: High quality automated docker hub push using Github, TravisCI and pyup for Python tool distributions

Let’s say you want to distribute a Python tool with docker using known good dependency versions ready to be used by end users… In this article you will see how to continuously keeping up to date a Docker Hub container with minimal managing effort (because I’m a lazy guy) using github, TravisCI and pyup.

The goal was to reduce as much as possible any manual activity for updates, check all works fine before pushing, minimize build times and keep docker container always secure and updated with a final high quality confidence.

As an example let’s see what happens under the hood behind every pytest-play Docker Hub update on the official container https://cloud.docker.com/u/davidemoro/repository/docker/davidemoro/pytest-play (by the way if you are a pytest-play user: did you know that you can use Docker for running pytest-play and that there is a docker container ready to be used on Docker Hub? See a complete and working example here https://davidemoro.blogspot.com/2019/02/api-rest-testing-pytest-play-yaml-chuck-norris.html)

Repositories

The docker build/publish stuff lives on another repository, so https://github.com/davidemoro/pytest-play-docker is the repository that implements the Docker releasing workflow for https://github.com/pytest-dev/pytest-play on Docker Hub (https://hub.docker.com/r/davidemoro/pytest-play).

Workflow

This is the highly automated workflow at this time of writing for the pytest-play publishing on Docker Hub:

All tests executions run against the docker build so there is a warranty that what is pushed to Docker Hub works fine (it doesn’t check only that the build was successful but it runs integration tests against the docker build), so no versions incompatibilities, no integration issues between all the integrated third party pytest-play plugins and no issues due to the operative system integration (e.g., I recently experienced an issue on alpine linux with a pip install psycopg2-binary that apparently worked fine but if you try to import psycopg2 inside your code you get an unexpected import error due to a recent issue reported here https://github.com/psycopg/psycopg2/issues/684).

So now every time you run a command like the following one (see a complete and working example here https://davidemoro.blogspot.com/2019/02/api-rest-testing-pytest-play-yaml-chuck-norris.html):

docker run –rm -v $ (pwd):/src davidemoro/pytest-play

you know what was the workflow for every automated docker push for pytest-play.

Acknowledgements

Many thanks to Andrea Ratto for the 10 minutes travis build speedup due to Docker cache, from ~11 minutes to ~1 minute is a huge improvement indeed! It was possible thanks to the docker pull davidemoro/pytest-play command, the build with the –cache-from davidemoro/pytest-play option and running the longest steps in a separate and cacheable step (e.g., the very very long cassandra-driver compilation moved to requirements_cassandra.txt will be executed only if necessary).

Relevant technical details about pytest-play-docker follows (some minor optimizations are still possible saving in terms of final size).

pytest-play-docker/.travis.yml

sudo: required
services:
– docker
– …

env:
  global:
  – IMAGE_NAME=davidemoro/pytest-play
  – secure: …
before_script:
– …

script:
– travis_wait docker pull python:3.7
– travis_wait docker pull “$ IMAGE_NAME:latest”
– travis_wait 25 docker build –cache-from “$ IMAGE_NAME:latest” -t “$ IMAGE_NAME” .
– docker run -i –rm -v $ (pwd)/tests:/src –network host -v /var/run/mysqld/mysqld.sock:/var/run/mysqld/mysqld.sock $ IMAGE_NAME –splinter-webdriver=remote
  –splinter-remote-url=$ REMOTE_URL
deploy:
  provider: script
  script: bash docker_push
  on:
    branch: master

pytest-play-docker/docker_push

#!/bin/bash
echo “$ DOCKER_PASSWORD” | docker login -u “$ DOCKER_USERNAME” –password-stdin
docker tag “$ IMAGE_NAME” “$ IMAGE_NAME:$ TRAVIS_COMMIT”
docker tag “$ IMAGE_NAME” “$ IMAGE_NAME:latest”
docker push “$ IMAGE_NAME”:”$ TRAVIS_COMMIT”
docker push “$ IMAGE_NAME”:latest

Feedback

Any feedback will be always appreciated.

Do you like the Docker hub push process for pytest-play? Let me know becoming a pytest-play stargazer! Star
Planet Python

How I Went from a High School English Teacher to a Content Intern

the InterWorks marketing team in Tulsa

“Words are, in my not-so-humble opinion, our most inexhaustible source of magic, capable of both inflicting injury and remedying it.” Albus Dumbledore via J.K. Rowling

I have been a lover of language and wielder of words (and advocate of alliteration … oof) my entire life. When I went to college, I decided to double major in two different languages—English and Spanish—because my passion for reading and writing was so intense. I’ve kept a journal since I was ten years old, I prefer to handwrite things instead of type them, and I have sworn to never purchase an e-reader, no matter how much room and trouble it may save me when traveling. And yet … I work for a tech company. Record scratch.

How did I get here?

[What] to Be or [What] Not to Be, That Is the Question

Hamlet by William Shakespeare
In elementary school, I wanted to grow up to be an actress or a singer. I pursued this path with diligence until I—a fifth grader—was snubbed for the lead in the elementary musical, it being given to a fourth grader instead. After suffering this indignity, I changed course and became an athlete. One only has so much time in a day, and from that point forward, most of my hours were dedicated to practices, scrimmages, games and tournaments. My involvement in softball rerouted my adolescence, and it was poised to determine my college destination as well.

some of my softball teammates and I after winning nationals

Above: Some of my best friends and I after winning softball nationals

Two Roads Diverged

“The Road Not Taken” by Robert Frost 
Nothing in my life has ever really gone exactly how I planned it. Up until the spring of my senior year in high school, I was intending to go to a university in Illinois to play softball and pursue medicine; I had gone on an official visit and had been volunteering at the hospital in preparation. Instead, I ended up at Oklahoma State University, studying two languages and not playing any D1 sport. I wasn’t disappointed with this turn of events, but I was certainly surprised.

As I looked beyond college graduation, I started seeking a job in publishing in the hopes of becoming an editor or a writer one day. However, my job prospects were bleak. At a career fair in the spring of my senior year at OSU (the most crucial juncture of time for me, evidently), I was recruited by Teach for America to join the fight against the social injustices perpetrated against America’s youth in the form of educational inequity. Needing a job and yearning for a noble purpose, I accepted the offer and once again found myself where I least expected: in the classroom as a teacher.

me in the Harry Potter costume I wore on Halloween my first year teaching

Above: The Gryffindor costume I wore to school on my first Halloween as a teacher

The Belly of the Beast

The last part of the Departure stage in the Hero’s Journey 
My first stint in the classroom was tough. I was engaged and planning a wedding, plus trying to learn how to be a teacher (not to mention an adult) for the very first time. I was working in education—an industry I had never studied or anticipated entering—and the only classroom experience I had was a month-long summer school program where I taught in the mornings and sat in on curriculum sessions in the afternoons. Plus, I was confronted daily with that most fearsome of creatures: the teenager.

“O brave new world, / That has such people in’t!” – William Shakespeare, The Tempest

Fortunately, I was teaching the subject I majored in during college, so that was a comfort. An added bonus was that my student population was predominantly Hispanic, so I got to utilize my Spanish degree as well. I was able to use those language skills with my students, some of whom had just moved to the US, and conduct parent-teacher conferences in Spanish to help facilitate understanding for my students’ families.

some of my Peru team and I at Machu Picchu

Above: A group photo at Machu Picchu during the summer I lived in Peru

After my time with TFA, I was more confident and comfortable as a teacher, so I relocated to my alma mater and began teaching English to high school freshmen. We waded together into the mysterious realm of adolescence and endeavored to improve our skills of expression and analysis. Some of our most significant guides were Atticus Finch and Bill Shakespeare. From Boo Radley and Tom Robinson, we learned not to judge others simply because they’re different from you. Fear and ignorance do not justify cruelty. From the ancient grudge in fair Verona, we learned the importance of not following impulse and always weighing the counsel of others. From a group of schoolboys stranded on an island, we learned that rules can be necessary, and selflessness is sacrificial. One of the most important things we’d studied together was a lesson I was about to learn firsthand: courage.

After five years in the classroom, I felt it was finally time for me to move on. I loved my students, I loved my subject and I loved fully immersing myself in literature every day. However, I just could not see myself staying in education forever. I wanted more, but I wasn’t sure what I was qualified for, or capable of, so I made the decision to leave my teaching job and find out.

Hope Is the Thing with Feathers

“Hope is the thing with feathers” by Emily Dickinson
While I felt certain leaving my teaching job was the right decision, there was still some doubt and fear hanging over me as I began my job search. What field but education would want to take on a seasoned teacher with no other experience? Furthermore, for someone who spent her days instructing others to identify what they want to say and then articulately express it, I could not for the life of me figure out what I really wanted. Having only ever worked in education, I couldn’t broaden my mind to consider what else was out there. My husband’s job had us firmly planted in Tulsa, so with that in mind, I began to comb through local job openings. I thought I could do well in an interview, but my resume wasn’t compelling enough to get me to that point with most companies.

Fortunately, I found InterWorks.

Here was a place that looked at a person’s character and focused on whether they’d be a good cultural fit, rather than just strictly considered their qualifications. This was great for me, but it’s also part of the unique essence of InterWorks. People want to work here because it’s fun, and we do high-quality work that we’re proud of. The culture is special, which explains the thorough interview process. It’s very intentional and reflects the careful consideration of an applicant as a whole person, not just who they appear to be in one half-hour interview.

After my third round of interviews, it became clear to me that I really wanted this job. I thought I would be the right match for the responsibilities of the position but also for the people. This was a company I wanted to be a part of, and more than that, these were people I wanted to befriend.

the InterWorks marketing team in Tulsa

Above: Some of the awesome marketing team I get to work with at InterWorks

I Mean to Astonish You All Someday

Little Women by Louisa May Alcott 
Thus far in my time with InterWorks, I have been challenged and humbled and questioned and encouraged. I am definitely being sharpened as a human and growing my skills and experience as a professional. In my position as a content intern, I’ve been able to read and edit lots of different content, from culture and other “How I Went from” stories to Tableau training and all sorts of data visualizations. I’ve read just about every blog that’s been posted to the website and have learned more about business intelligence and data analytics than I ever thought I would. But I’ve also been given numerous opportunities to show my value, expand my skillset and exercise my creative muscles. I’m able to work autonomously while being pushed toward excellence and supported with grace.

I still have big dreams for my future, but I know neither what that future will hold nor what I will actually be when I grow up (if that time ever truly comes). In the meantime, though, I’m happy to be at a great company that values its people and knows how to communicate that value to them. I never foresaw myself working so closely with data and software like Tableau, but rarely do the things I anticipate for myself pan out as expected. I’m just trusting the process and seeing where it takes me, and so far, it’s taken me to some pretty amazing places.

The post How I Went from a High School English Teacher to a Content Intern appeared first on InterWorks.

InterWorks