Weekly News Summary for Admins — 2019-02-15

Lots of interesting posts this week.

But most interesting for MacAdmins fighting with SecureToken, read Rich Trouton’s two posts which might help you get out of situations that were so far considered dead ends.

MacRumors points to the week of June 3 for WWDC this year (no big surprise) and Bloomberg (the other Mac rumors site) points to March 25 for a ‘subscription service’ event. Since MacADUK starts on March 26, this event could lead to some last minute slide shuffling.

If you would rather get the weekly newsletter by email, you can subscribe to the Scripting OS X Weekly Newsletter here!! (Same content, delivered to your Inbox once a week.)

#! On Scripting OS X

📰News and Opinion

🐦MacAdmins on Twitter

  • Kyle Crawford: “Why does Apple need a sysdiagnose for a feature enhancement? Don’t they know the features of their own software? startosinstall should warn if Internet connection is needed before running. How is a having a sysdiagnose going to add any value?”
  • Jordan Rose: “Command-line tip: sftp -a is a mostly drop-in replacement for scp that allows resuming an interrupted download.”
  • Mike Boylan: “TIL about the “HP Easy Admin” tool. If you’re a school, large org, or enterprise and are looking for the HP driver package for a printer v the home/single-use Easy Setup app, you can use Easy Admin to go get the pkg-based driver installer to be used at scale.”
  • Graham Pugh: “The clear and reliable world of DEP enrollment…” (click to see full tweet)
  • Graham Pugh: “Regex of Mojave-compatible Mac model identifiers: ^(iMac1[3-8]|iMacPro|MacBook[8-9]|MacBook10|MacBookAir[5-7]|MacBookPro9|MacBookPro1[0-5]|MacMini[6-7]|MacPro[5-6])

🐞Bugs and Security

🔨Support and HowTos

🤖Scripting and Automation

🍏Apple Support

♻Updates and Releases

📺To Watch

📚Support

There are no ads on my webpage or this newsletter. If you are enjoying what you are reading here, please spread the word and recommend it to another Mac Admin!

If you want to support me and this website even further, then consider buying one (or all) of my books. It’s like a subscription fee, but you also get a useful book or two extra!

Scripting OS X

Vasudev Ram: pprint.isrecursive: Check if object requires recursive representation


By Vasudev Ram – Online Python training / SQL training / Linux training


Tree image attribution

Hi, readers,

I was using the pprint module to pretty-print some Python data structures in a program I was writing. Then saw that it has a function called isrecursive.

The docstring for pprint.isrecursive says:

>>> print pprint.isrecursive.__doc__
Determine if object requires a recursive representation.

Here is a Python 3 shell session that shows what the isrecursive function does, with a list:

>>> import pprint
>>> print(pprint.pprint.__doc__)
Pretty-print a Python object to a stream [default is sys.stdout].
>>> a = []
>>> a
[]
>>> pprint.isrecursive(a)
False
>>>
>>> a.append(a)
>>> a
[[...]]
>>>
>>> pprint.isrecursive(a)
True

How about for a dict?

>>> b = {}
>>> pprint.isrecursive(b)
False
>>>
>>> b[1] = b
>>> b
{1: {...}}
>>> id(b) == id(b[1])
True
>>> pprint.isrecursive(b)
True

How about if an object is recursive, but not directly, like in the above two examples? Instead, it is recursive via a chain of objects:

>>> c = []
>>> d = []
>>> e = []
>>> c.append(d)
>>> d.append(e)
>>> c
[[[]]]
>>> pprint.isrecursive(c)
False
>>> e.append(c)
>>> c
[[[[...]]]]
>>> pprint.isrecursive(c)
True

So we can see that isrecursive is useful to detect some recursive Python object structures.
Interestingly, if I compare c with c[0] (after making c a recursive structure), I get:

>>> c == c[0]
Traceback (most recent call last):
File "", line 1, in
RecursionError: maximum recursion depth exceeded in comparison
>>>

In Python 2, I get:

RuntimeError: maximum recursion depth exceeded in cmp

The image at the top of the post is of a tree created in the LOGO programming language using recursion.

– Enjoy.


Vasudev Ram – Online Python training and consulting


I conduct online courses on Python programming, Unix / Linux commands and shell scripting and SQL programming and database design, with course material and personal coaching sessions.

The course details and testimonials are here.

Contact me for details of course content, terms and schedule.

Try FreshBooks: Create and send professional looking invoices in less than 30 seconds.

Getting a new web site or blog, and want to help preserve the environment at the same time? Check out GreenGeeks.com web hosting.

Sell your digital products via DPD: Digital Publishing for Ebooks and Downloads.

Learning Linux? Hit the ground running with my vi quickstart tutorial. I wrote it at the request of two Windows system administrator friends who were given additional charge of some Unix systems. They later told me that it helped them to quickly start using vi to edit text files on Unix. Of course, vi/vim is one of the most ubiquitous text editors around, and works on most other common operating systems and on some uncommon ones too, so the knowledge of how to use it will carry over to those systems too.

Check out WP Engine, powerful WordPress hosting.

Get a fast web site with A2 Hosting.

Creating online products for sale? Check out ConvertKit, email marketing for online creators.

Teachable: feature-packed course creation platform, with unlimited video, courses and students.

Posts about: Python * DLang * xtopdf

My ActiveState Code recipes

Follow me on:


Planet Python

Apple Two-Factor Authentication for a Secondary Apple ID

Apple sent an email to developers, stating that later this months, two-factor authentication will be required for Apple IDs used for developer accounts.

If you, like me, use separate Apple IDs for your personal iCloud and your developer accounts, this will pose some kind of challenge. There is a solution, however Apple does not document it very well.

Two-factor authentication for the primary account

Assumption: you have two-factor authentication (2FA) enabled on your primary, personal Apple ID, and are logged in to that account on your Mac(s) and iOS devices. If you haven’t done that yet, do it now. 2FA does increase your account security significantly.

You can enable 2FA on any device logged in to the account in the iCloud Settings or preference pane. As part of the setup you can provide one or more phone numbers as a fall back mechanism. If no devices can be prompted through Apple’s built-in 2FA, it will send an SMS to the trusted phone numbers. You can use the same phone number for multiple Apple IDs, but there seems to be some limit on how often you can do that.

Enable 2FA for the secondary account

Assumption: The secondary account is your developer Apple ID, you don’t use it for iCloud storage, device backups, mail etc. You use it to log in to developer.apple.com and iTunes Connect, and to get all the certificates and other resources you need as a developer.

The challenge here is that you can only enable 2FA on the first account logged in to iCloud on a device. You could log out of your primary iCloud account, and the log in with the secondary, but this will disrupt a lot of things on your device. I’d rather avoid that.

On a Mac, you can have a separate iCloud account for each local user. So, it is easiest to create a second user account, log out of your first account, log in to the new second account and set up iCloud and 2FA for the developer Apple ID on this second local account.

You can sign in to the secondary Apple ID enable 2FA in System Preferences -> iCloud -> Account Details -> Security as described in Apple’s Support Article.

Follow the prompts to set up 2FA, you can re-use the same phone number as a trusted number. (There seem to be limits to how often you use the same phone number, but two accounts works fine for me.)

Once 2FA is set up, we don’t need the second user account on the Mac any more. Sign out of iCloud, log out of the second account and back in to your normal user account.

If you are ok with using SMS authentication (Apple calls this ‘two-step authentication’, rather than ‘two-factor authentication’) then you are done. However, many will argue codes over SMS are not good enough for secondary authentication, so we want go to ‘full’ 2FA.

Use the secondary Apple ID

As it turns out, you can be logged in to multiple iCloud accounts on the same device or account. Certain services, such as iCloud storage, or the Photo Library, will only work with the primary iCloud account, but other services, including 2FA, will work for all iCloud accounts.

On your iOS device go to Settings > Passwords & Accounts > Add Account, and choose to add another iCloud account. You probably want to turn off all services, like Mail, Calendar, etc. secondary account.

Second iCloud Account on iPhone
Second iCloud Account on iPhone

On the Mac you can do the same in System Preferences > Internet Accounts. You can use both your Mac and iOS devices for 2FA.

Second iCloud Account on macOS 
Second iCloud Account on macOS

Now the secondary Apple ID will prompt the devices you are logged in as for 2FA.

2FA prompt on the iPhone 
2FA prompt on the iPhone

Scripting OS X

PyCharm: PyCharm 2019.1 EAP 4

Our fourth Early Access Program (EAP) version for PyCharm 2019.1 is now available on our website.

New in This Version

Parallel and concurrent testing with pytest

695a0367-eed2-45e1-96ed-46c70fc6067e

PyCharm makes it easy to run tests quickly using multiprocessing (parallelism) and multithreading (concurrency). All you need to do in order to run your pytest tests in parallel is to install the pytest-xdist plugin as a normal python package using the PyCharm’s package manager, specify pytest as the project testing framework, create a pytest run/debug configuration where you can specify the number of CPUs to run the tests on, and you’re good to go.

Read more about setting up and running pytest tests in parallel in our help

Further Improvements

  • Many JavaScript improvements: PyCharm Professional Edition bundles all JavaScript features from WebStorm. You can read more about the new JavaScript features on the WebStorm blog.
  • Many platform improvements: PyCharm bundles new features and bug fixes coming from the IntelliJ platform
  • And more, read the release notes for details.

Interested?

Download this EAP from our website. Alternatively, you can use the JetBrains Toolbox App to stay up to date throughout the entire EAP.

With PyCharm 2019.1 we’re moving to a new runtime environment: this EAP build already bundles the brand new JetBrains Runtime Environment (a customized version of JRE 11). Unfortunately, since this build uses the brand-new platform, the patch-update from previous versions is not available this time. Please use the full installation method instead.

If you’re on Ubuntu 16.04 or later, you can use snap to get PyCharm EAP, and stay up to date. You can find the installation instructions on our website.

PyCharm 2019.1 is in development during the EAP phase, therefore not all new features are already available. More features will be added in the coming weeks. As PyCharm 2019.1 is pre-release software, it is not as stable as the release versions. Furthermore, we may decide to change and/or drop certain features as the EAP progresses.

All EAP versions will ship with a built-in EAP license, which means that these versions are free to use for 30 days after the day that they are built. As EAPs are released weekly, you’ll be able to use PyCharm Professional Edition EAP for free for the duration of the EAP program, as long as you upgrade at least once every 30 days.


Planet Python

Understanding Managed Databases

Introduction

Secure, reliable data storage is a must for nearly every modern application. However, the infrastructure needed for a self-managed, on-premises database can be prohibitively expensive for many teams. Similarly, employees who have the skills and experience needed to maintain a production database effectively can be difficult to come by.

The spread of cloud computing services has lowered the barriers to entry associated with provisioning a database, but many developers still lack the time or expertise needed to manage and tune a database to suit their needs. For this reason, many businesses are turning to managed database services to help them build and scale their databases in line with their growth.

In this conceptual article, we will go over what managed databases are and how they can be beneficial to many organizations. We will also cover some practical considerations one should make before building their next application on top of a managed database solution.

Managed Databases in a Nutshell

A managed database is a cloud computing service in which the end user pays a cloud service provider for access to a database. Unlike a typical database, users don’t have to set up or maintain a managed database on their own; rather, it’s the provider’s responsibility to oversee the database’s infrastructure. This allows the user to focus on building their application instead of spending time configuring their database and keeping it up to date.

The process of provisioning a managed database varies by provider, but in general it’s similar to that of any other cloud-based service. After registering an account and logging in to the dashboard, the user reviews the available database options — such as the database engine and cluster size — and then chooses the setup that’s right for them. After you provision the managed database, you can connect to it through a GUI or client and can then begin loading data and and integrating the database with your application.

Managed data solutions simplify the process of provisioning and maintaining a database. Instead of running commands from a terminal to install and set one up, you can deploy a production-ready database with just a few clicks in your browser. By simplifying and automating database management, cloud providers make it easier for anyone, even novice database users, to build data-driven applications and websites. This was the result of a decades-long trend towards simplifying, automating, and abstracting various database management tasks, which was itself a response to pain points long felt by database administrators.

Pain Points of On-Premises and Self-Managed Databases

Prior to the rise of the cloud computing model, any organization in need of a data center had to supply all the time, space, and resources that went into setting one up. Once their database was up and running, they also had to maintain the hardware, keep its software updated, hire a team to manage the database, and train their employees on how to use it.

As cloud computing services grew in popularity in the 2000s, it became easier and more affordable to provision server infrastructure, since the hardware and the space required for it no longer had to be owned or managed by those using it. Likewise, setting up a database entirely within the cloud became far less difficult; a business or developer would just have to requisition a server, install and configure their chosen database management system, and begin storing data.

While cloud computing did make the process of setting up a traditional database easier, it didn’t address all of its problems. For instance, in the cloud it can still be difficult to pinpoint the ideal size of a database’s infrastructure footprint before it begins collecting data. This is important because cloud consumers are charged based on the resources they consume, and they risk paying for more than what they require if the server they provision is larger than necessary. Additionally, as with traditional on-premises databases, managing one’s database in the cloud can be a costly endeavor. Depending on your needs, you may still need to hire an experienced database administrator or spend a significant amount of time and money training your existing staff to manage your database effectively.

Many of these issues are compounded for smaller organizations and independent developers. While a large business can usually afford to hire employees with a deep knowledge of databases, smaller teams usually have fewer resources available, leaving them with only their existing institutional knowledge. This makes tasks like replication, migrations, and backups all the more difficult and time consuming, as they can require a great deal of on-the-job learning as well as trial and error.

Managed databases help to resolve these pain points with a host of benefits to businesses and developers. Let’s walk through some of these benefits and how they can impact development teams.

Benefits of Managed Databases

Managed database services can help to reduce many of the headaches associated with provisioning and managing a database. For one thing, developers build applications on top of managed database services to drastically speed up the process of provisioning a database server. With a self-managed solution, you must obtain a server (either on-premises or in the cloud), connect to it from a client or terminal, configure and secure it, and then install and set up the database management software before you can begin storing data. With a managed database, you only have to decide on the initial size of the database server, configure any additional provider-specific options, and you’ll have a new database ready to integrate with your app or website. This can usually be done in just a few minutes through the provider’s user interface.

Another appeal of managed databases is automation. Self-managed databases can consume a large amount of an organization’s resources because its employees have to perform every administrative task — from scaling to performing updates, running migrations, and creating backups — manually. With a managed database, however, these and other tasks are done either automatically or on-demand, which markedly reduces the risk of human error.

This relates to the fact that managed database services help to streamline the process of database scaling. Scaling a self-managed database can be very time- and resource-intensive. Whether you choose sharding, replication, load balancing, or something else as your scaling strategy, if you manage the infrastructure yourself then you’re responsible for ensuring that no data is lost in the process and that the application will continue to work properly. If you integrate your application with a managed database service, however, you can scale the database cluster on demand. Rather than having to work out the optimal server size or CPU usage beforehand, you can quickly provision more resources on-the-fly. This helps you avoid using unnecessary resources, meaning you also won’t pay for what you don’t need.

Managed solutions tend to have built-in high-availability. In the context of cloud computing, a service is said to be highly available if it is stable and likely to run without failure for long periods of time. Most reputable cloud providers’ products come with a service level agreement (SLA), a commitment between the provider and its customers that guarantees the availability and reliability of their services. A typical SLA will specify how much downtime the customer should expect, and many also define the compensation for customers if these service levels are not met. This provides assurance for the customer that their database won’t crash and, if it does, they can at least expect some kind of reparation from the provider.

In general, managed databases simplify the tasks associated with provisioning and maintaining a database. Depending on the provider, you or your team will still likely need some level of experience working with databases in order to provision a database and interact with it as you build and scale your application. Ultimately, though, the database-specific experience needed to administer a managed database will be much less than with self-managed solution.

Of course, managed databases aren’t able to solve every problem, and may prove to be a less-than-ideal choice for some. Next, we’ll go over a few of the potential drawbacks one should consider before provisioning a managed database.

Practical Considerations

A managed database service can ease the stress of deploying and maintaining a database, but there are still a few things to keep in mind before committing to one. Recall that a principal draw of managed databases is that they abstract away most of the more tedious aspects of database administration. To this end, a managed database provider aims to deliver a rudimentary database that will satisfy the most common use cases. Accordingly, their database offerings won’t feature tons of customization options or the unique features included in more specialized database software. Because of this, you won’t have as much freedom to tailor your database and you’ll be limited to what the cloud provider has to offer.

A managed database is almost always more expensive than a self-managed one. This makes sense, since you’re paying for the cloud provider to support you in managing the database, but it can be a cause for concern for teams with limited resources. Moreover, pricing for managed databases is usually based on how much storage and RAM the database uses, how many reads it handles, and how many backups of the database the user creates. Likewise, any application using a managed database service that handle large amounts of data or traffic will be more expensive than if it were to use a self-managed cloud database.

One should also reflect on the impact switching to a managed database will have on their internal workflows and whether or not they’ll be able to adjust to those changes. Every provider differs, and depending on their SLA they may shoulder responsibility for only some administration tasks, which would be problematic for developers looking for a full-service solution. On the other hand, some providers could have a prohibitively restrictive SLA or make the customer entirely dependent on the provider in question, a situation known as vendor lock-in.

Lastly, and perhaps most importantly, one should carefully consider whether or not any managed database service they’re considering using will meet their security needs. All databases, including on-premises databases, are prone to certain security threats, like SQL injection attacks or data leaks. However, the security dynamic is far different for databases hosted in the cloud. Managed database users can’t control the physical location of their data or who has access to it, nor can they ensure compliance with specific security standards. This can be especially problematic if your client has heightened security needs.

To illustrate, imagine that you’re hired by a bank to build an application where its clients can access financial records and make payments. The bank may stipulate that the app must have data at rest encryption and appropriately scoped user permissions, and that it must be compliant with certain regulatory standards like PCI DSS. Not all managed database providers adhere to the same regulatory standards or maintain the same security practices, and they’re unlikely to adopt new standards or practices for just one of their customers. For this reason, it’s critical that you ensure any managed database provider you rely on for such an application is able to meet your security needs as well as the needs of your clients.

Conclusion

Managed databases have many features that appeal to a wide variety of businesses and developers, but a managed database may not solve every problem or suit everyone’s needs. Some may find that a managed database’s limited feature set and configuration options, increased cost, and reduced flexibility outweigh any of its potential advantages. However, compelling benefits like ease of use, scalability, automated backups and upgrades, and high availability have led to increased adoption of managed database solutions in a variety of industries.

If you’re interested in learning more about DigitalOcean Managed Databases, we encourage you to check out our Managed Databases product documentation.

DigitalOcean Community Tutorials