Moving to zsh, part 2: Configuration Files

In part one I talked about Apple’s motivation to switch the default shell and urge existing users to change to zsh.

Since I am new to zsh as well, I am planning to document my process of transferring my personal bash setup and learning the odds and ends of zsh.

Many websites and tutorials leap straight to projects like oh-my-zsh or prezto where you can choose from hundreds of pre-customized and pre-configured themes.

While these projects are very impressive and certainly show off the flexibility and power of zsh customization, I feel this will actually prevent an understanding of how zsh works and how it differs from bash. So, I am planning to build my own configuration ‘by hand’ first.

At first, I actually took a look at my current bash_profile and cleaned it up. There were many aliases and functions which I do not use or broke in some macOS update. I the end, this is what I want to re-create in zsh:

Most of these should be fairly easy to transfer. Some might be… interesting.

But first, where do we put our custom zsh configuration?

zsh Configuration Files

bash has a list of possible files that it tries in predefined order. I have the description in my post on the bash_profile.

zsh also has a list of files it will execute at shell startup. The list of possible files is even longer, but somewhat more ordered.

all users user login shell interactive shell scripts Terminal.app
/etc/zshenv .zshenv
/etc/zprofile .zprofile x x
/etc/zshrc .zshrc x
/etc/zlogin .zlogin x x
/etc/zlogout .zlogout x x

The files in /etc/ will be launched (when present) for all users. The .z* files only for the individual user.

By default, zsh will look in the root of the home directory for the user .z* files, but this behavior can be changed by setting the ZDOTDIR environment variable to another directory (e.g. ~/.zsh/) where you can then group all user zsh configuration in one place.

On macOS you could set the ZDOTDIR to ~/Documents/zsh/ and then use iCloud syncing (or a different file sync service) to have the same files on all your Macs. (I prefer to use git.)

bash will either use .bash_profile for login shells, or .bashrc for interactive shells. That means, when you want to centralize configuration for all use cases, you need to source your .bashrc from .bash_profile or vice versa.

zsh behaves differently. zsh will run all of these files in the appropriate context (login shell, interactive shell) when they exist.

zsh will start with /etc/zshenv, then the user’s .zshenv. The zshenv files are always used when they exist, even for scripts with the #!/bin/zsh shebang. Since changes applied in the zshenv will affect zsh behavior in all contexts, you should you should be very cautious about changes applied here.

Next, when the shell is a login shell, zsh will run /etc/zprofile and .zprofile. Then for interactive shells (and login shells) /etc/zshrc and .zshrc. Then, again, for login shells /etc/zlogin and .zlogin. Why are there two files for login shells? The zprofile exists as an analog for bash’s and sh’s profile files, and zlogin as an analog for ksh login files.

Finally, there are zlogout files that can be used for cleanup, when a login shell exits. In this case, the user level .zlogout is read first, then the central /etc/zlogout. If the shell is terminated by an external process, these files might not be run.

Apple Provided Configuration Files

macOS Mojave (and earlier versions) includes /etc/zprofile and /etc/zshrc files. Both are very basic.

/etc/zprofile uses /usr/libexec/path_helper to set the default PATH. Then /etc/zshrc enables UTF–8 with setopt combiningchars.

Like /etc/bashrc there is a line in /etc/zshrc that would load /etc/zshrc_Apple_Terminal if it existed. This is interesting as /etc/bashrc_Apple_Terminal contains quite a lot of code to help bash to communicate with the Terminal application. In particular bash will send a signal to the Terminal on every new prompt to update the path and icon displayed in the Terminal window title bar, and provides other code relevant for saving and restoring Terminal sessions between application restarts.

However, there is no /etc/zshrc_Apple_Terminal and we will have to provide some of this functionality ourselves.

Note: As of this writing, /etc/zshrc in the macOS Catalina beta is different from the Mojave /etc/zshrc and provides more configuration. However, since Catalina is still beta, I will focus these articles on Mojave and earlier. Once Catalina is released, I may update these articles or write a new one for Catalina, if necessary.

Which File to use?

When you want to use the ZDOTDIR variable to change the location of the other zsh configuration files, setting that variable in ~/.zshenv seems like a good choice. Other than that, you probably want to avoid using the zshenv files, since it will change settings for all invocations of zsh, including scripts.

macOS Terminal considers every new shell to be a login shell and an interactive shell. So, in Terminal a new zsh will potentially run all configuration files.

For simplicity’s sake, you should use just one file. The common choice is .zshrc.

Most tools you can download to configure zsh, such as ‘prezto’ or ‘oh-my-zsh’, will override or re-configure your .zshrc. You could consider moving your code to .zlogin instead. Since .zlogin is sourced after .zshrc it can override settings from .zshrc. However, .zlogin is only called for login shells.

The most common situation where you do not get a login shell with macOS Terminal, is when you switch to zsh from another shell by typing the zsh command.

I would recommend to put your configuration in your .zshrc file and if you want to use any of the theme projects, read and follow their instructions closely as to how you can preserve your configurations together with theirs.

Managing the shell for Administrators

MacAdmins may have the need to manage certain shell settings for their users, usually environment variables to configure certain command line tool’s behaviors.

The most common need is to expand the PATH environment variable for third party tools. Often the third party tools in question will have elaborate postinstall scripts that attempt to modify the current user’s .bash_profile or .bashrc. Sometimes, these tools even consider that a user might have changed the default shell to something other than bash.

On macOS, system wide changes to the PATH should be done by adding files to /etc/paths.d.

As an administrator you should be on the lookout for scripts and installers that attempt to modify configuration files on the user level, disable the scripts during deployment, and manage the required changes centrally. This will allow you to keep control of the settings even as tools change, are added or removed from the system, while preserving the user’s custom configurations.

To manage environment variables other than PATH centrally, administrators should consider /etc/zshenv or adding to the existing /etc/zshrc. In these cases you should always monitor whether updates to macOS overwrite or change these files with new, modified files of their own.

Summary

There are many possible files where the zsh can load user configuration. You should use ~/zshrc for your personal configurations.

There are many tools and projects out there that will configure zsh for you. This is fine, but might keep you from really understanding how things work.

MacAdmins who need to manage these settings centrally, should use /etc/paths.d and similar technologies or consider /etc/zshenv or /etc/zshrc.

Apple’s built-in support for zsh in Terminal is not as detailed as it is for bash.

Next: Shell Options, Environment Variables, and Functions

Scripting OS X

Wingware Blog: Extending Wing with Python (Part One)

In this issue of Wing Tips we start to look at how to extend Wing’s functionality by writing Python code. Extension scripts can be used to automate custom editing tasks, access Wing’s source code analysis, control the debugger, and much more.

A Simple Example

Scripts written to extend Wing are regular Python files containing one or more function definitions at the top level of the file. Each of these functions creates a new IDE command that can be used just like Wing’s built-in command set.

The following example creates a new command hello-world in the IDE:

 import wingapi def hello_world():   wingapi.gApplication.ShowMessageDialog("Hello", "Hello world!") 

To try this, drop it into a file named test.py in the scripts directory inside the Settings Directory listed in Wing’s About box. Then select Reload All Scripts from the Edit menu so Wing scans for newly added scripts. Once that’s done, you don’t need to use Reload All Scripts again since Wing watches already-loaded script files and reloads them automatically when you save them to disk.

Now you can run your script with Command by Name from the Edit menu, by typing the command name hello-world into the mini-buffer that appears at the bottom of Wing’s window and then pressing Enter:

/images/blog/scripting-1/command-by-name.png

You can also use the User Interface > Keyboard > Custom Key Bindings preference to bind the command hello-world to any key, or add the command to the tool bar with the User Interface > Toolbar > Custom Items preference.

Whatever method you use to invoke hello-world, you should see a dialog that looks something like this:

/images/blog/scripting-1/hello-world.png

Adding Menus

Commands created by scripts may also be added to new menus in the menu bar, by setting a function attribute as follows:

 import wingapi def hello_world():   wingapi.gApplication.ShowMessageDialog("Hello", "Hello world!") hello_world.contexts = [wingapi.kContextNewMenu("Scripts")] 

After making this change in your copy of test.py, you should see the Scripts menu appear in the menu bar immediately after saving the file:

/images/blog/scripting-1/menu.png

Scripts can also add items to the editor context menu, project context menu, or the high-level configuration menu in the top right of Wing’s window. We’ll go over these in a future article in this series.

Enabling Auto-Completion and Documentation

Creating scripts is a lot easier if Wing offers auto-completion and documentation in the Source Assistant tool. To enable that, let’s use a bit of script-based automation to take care of the necessary project setup.

Start by appending the following to the test.py script file that you created earlier, and then save to disk:

 import wingapi import os def setup_script_completion():   app = wingapi.gApplication   proj = app.GetProject()   pypath = proj.GetEnvironment().get('PYTHONPATH', None)   dirname = os.path.join(app.GetWingHome(), 'src')   if pypath is None:     pypath = dirname   elif not dirname in pypath.split(os.pathsep):     pypath += os.pathsep + dirname   app.GetProject().SetEnvironment(None, 'startup', {'PYTHONPATH': pypath}) setup_script_completion.contexts = [wingapi.kContextNewMenu("Scripts")] 

Now you can configure any project for easier scripting by selecting Setup script completion from the Scripts menu. This adds the directory that contains Wing’s scripting API to the Python Path in your project’s Project Properties.

Once this is done, the auto-completer works on the contents of wingapi, documentation for the API appears in the Source Assistant as you type or navigate code, and Goto Definition will bring up the API’s implementation source:

/images/blog/scripting-1/complete-sassist.gif

You may of course want to apply this configuration only to a project you create for working on your Wing extension scripts, to avoid altering the Python Path used in other projects.

A More Complex Example

Here’s a more complex real world example that moves the caret to the start of the current block in Python code. You can try this by appending it to your copy of test.py, saving to disk, and then using“Goto Start of Block“ in the Scripts menu in any Python file:

 import wingapi from wingbase import textutils  def _indent_chars(line):   indent_chars = 0   for i, c in enumerate(line):     if c in ' \t':       indent_chars += 1     else:       break   return indent_chars  def start_of_block():   ed = wingapi.gApplication.GetActiveEditor()   doc = ed.GetDocument()   txt = doc.GetText()   pos, end = ed.GetSelection()    first_indent = None   lineno = doc.GetLineNumberFromPosition(pos)   while lineno >= 0:     line = txt[doc.GetLineStart(lineno):doc.GetLineEnd(lineno)]     indent = _indent_chars(line)     if first_indent is None:       if indent != 0:         first_indent = indent       lineno -= 1     elif indent >= first_indent:       lineno -= 1     elif not line.strip():       lineno -= 1     else:       break    target = doc.GetLineStart(lineno)   target += _indent_chars(line)    ed.SetSelection(target, target)  def _start_of_block_available():   ed = wingapi.gApplication.GetActiveEditor()   if ed is None:     return False   mime = wingapi.gApplication.GetMimeType(ed.GetDocument().GetFilename())   return mime == 'text/x-python'  start_of_block.available =_start_of_block_available start_of_block.label = "Goto Start of Block" start_of_block.contexts = [wingapi.kContextNewMenu('Scripts', 1)] 

This example introduces a few other concepts:

(1) You can place utility functions at the top of a script file by prefixing their names with _ (underscore). This prevents Wing from creating a new command out of that function.

(2) Setting the function attribute available on a script controls when the command created by it is available in the user interface; when it isn’t, menu items are greyed out.

(3) The label attribute replaces Wing’s automatically generated label for the command, when it is shown in menus.

(4) A group number can be included in kContextNewMenu to group script-created commands into separate sections of menus.

This is just a small sampling of the capabilities of extension scripts in Wing. We’ll go into more depth in the next few installments in this series. Or see Scripting and Extending Wing for detailed documentation.

That’s it for now! In the next Wing Tip we’ll look at how to use Wing to debug extension scripts, to make it much easier to develop more complex extensions.

Planet Python

Evennia: Creating Evscaperoom, part 1

Over the last month (April-May 2019) I have taken part in the Mud Coder’s Guild Game Jam “Enter the (Multi-User) Dungeon”. This year the theme for the jam was One Room.

The result was Evscaperoom, an text-based multi-player “escape-room” written in Python using the Evennia MU* creation system. You can play it from that link in your browser or MU*-client of choice. If you are so inclined, you can also vote for it here in the jam (don’t forget to check out the other entries while you’re at it).

This little series of (likely two) dev-blog entries will try to recount the planning and technical aspects of the Evscaperoom. This is also for myself – I’d better write stuff down now while it’s still fresh in my mind!

Update: The next part of this blog is here.

Inception 

When I first heard about the upcoming game-jam’s theme of One Room, an ‘escape room’ was the first thing that came to mind, not the least because I just recently got to solve my my own first real-world escape-room as a gift on my birthday. 

If you are not familiar with escape-rooms, the premise is simple – you are locked into a room and have to figure out a way to get out of it by solving practical puzzles and finding hidden clues in the room. 

While you could create such a thing in your own bedroom (and there are also some one-use board game variants), most escape-rooms are managed by companies selling this as an experience for small groups. You usually have one hour to escape and if you get stuck you can press a button (or similar) to get a hint.

I thought making a computer escape-room. Not only can you do things in the computer that you cannot do in the real world, restricting the game to a single room limits so that it’s conceivable to actually finish the damned thing in a month. 

A concern I had was that everyone else in the jam surely must have went for the same obvious idea. In the end that was not an issue at all though.

Basic premises
 
I was pretty confident that I would technically be able to create the game in time (not only is Python and Evennia perfect for this kind of fast experimentation and prototyping, I know the engine very well). But that’s not enough; I had to first decide on how the thing should actually play. Here are the questions I had to consider:

Room State 

 An escape room can be seen as going through multiple states as puzzles are solved. For example, you may open a cabinet and that may open up new puzzles to solve. This is fine in a single-player game, but how to handle it in a multi-player environment?

My first thought was that each object may have multiple states and that players could co-exist in the same room, seeing different states at the same time. I really started planning for this. It would certainly be possible to implement.

But in the end I considered how a real-world escape-room works – people in the same room solves it together. For there to be any meaning with multi-player, they must share the room state.

So what I went with was a solution where players can create their own room or join an existing one. Each such room is generated on the fly (and filled with objects etc) and will change as players solve it. Once complete and/or everyone leaves, the room is deleted along with all objects in it. Clean and tidy.

So how to describe these states? I pictured that these would be described as normal Python modules with a start- and end function that initialized each state and cleaned it up when a new state was started. In the beginning I pictured these states as being pretty small (like one state to change one thing in the room). In the end though, the entire Evscaperoom fits in 12 state modules. I’ll describe them in more detail in the second part of this post. 

Accessibility and “pixel-hunting” in text

When I first started writing descriptions I didn’t always note which objects where interactive. It’s a very simple and tempting puzzle to add – mention an object as part of a larger description and let the player figure out that it’s something they can interact with. This practice is sort-of equivalent to pixel-hunting in graphical games – sweeping with the mouse across the screen until you find that little spot on the screen that you can do something with.

Problem is, pixel-hunting’s not really fun. You easily get stuck and when you eventually find out what was blocking you, you don’t really feel clever but only frustrated. So I decided that I should clearly mark every object that people could interact with and focus puzzles on better things.

In fact, in the end I made it an option:

Option menu ('quit' to return)   1: ( ) No item markings (hard mode)  2: ( ) Items marked as item (with color)  3: (*) Items are marked as [item] (screenreader friendly)  4: ( ) Screenreader mode

As part of this I had to remind myself never to use colors only when marking important information: Visually impaired people with screen readers will simply miss that. Not to mention that some just disable colors in their clients.

So while I personally think option 2 above is the most visually pleasing, Evscaperoom defaults to the third option. It should should start everyone off on equal footing. Evennia has a screen-reader mode out of the box, but I moved it into the menu here for easy access.

Inventory and collaboration

In a puzzle-game, you often find objects and combine them with other things. Again, this is simple to do in a single-player game: Players just pick things up and use them later.

But in a multi-player game this offers a huge risk: players that pick up something important and then log off. The remaining players in that room would then be stuck in an unsolvable room – and it would be very hard for them to know this.

In principle you could try to ‘clean’ player inventories when they leave, but not only does it add complexity, there is another issue with players picking things up: It means that the person first to find/pick up the item is the only one that can use it and look at it. Others won’t have access until the first player gives it up. Trusting that to anonymous players online is not a good idea.

So in the end I arrived at the following conclusions:

  • As soon as an item/resource is discovered, everyone in the room must be able to access it immediately.
  • There can be no inventory. Nothing can ever be picked up and tied to a specific player.
  • As soon as a discovery is made, this must be echoed to the entire room (it must not be up to the finder to announce what they found to everyone else).  

As a side-effect of this I also set a limit to the kind of puzzles I would allow:

  • No puzzles must require more than one player to solve. While one could indeed create some very cool puzzles where people collaborate, it’s simply not feasible to do so with random strangers on the internet. At any moment the other guy may log off and leave you stuck. And that’s if you even find someone logged in at the same time in the first place! The room should always be possible to solve solo, from beginning to end.

Focusing on objects

So without inventory system, how do you interact with objects? A trademark of any puzzle is using one object with another and also to explore things closer to find clues. I turned to graphical adventure games for inspiration:

Hovering with mouse over lens object offers action
Secret of Monkey Island ©1990 LucasArts. Image from old-games.com

A common way to operate on an object in traditional adventure games is to hover the mouse over it and then select the action you want to apply to it. In later (3D) games you might even zoom in of the object and rotate it around with your mouse to see if there are some clues to be had.

While Evennia and modern UI clients may allow you to use the mouse to select objects, I wanted this to work the traditional MUD-way, by inserting commands. So I decided that you as a player would be in one of two states:

  • The ‘normal’ state: When you use look you see the room description.
  • The ‘focused’ state: You focus on a specific object with the examine <target> command (aliases are ex or just e). Now object-specific actions become available to you. Use examine again to “un-focus”. 
A small stone fireplace sits in the middle of the wall opposite the [door]. On the chimney hangs a small oil [painting] of a man. Hanging over the hearth is a black [cauldron]. The piles of [ashes] below are cold.  (It looks like fireplace may be suitable to [climb].)


In the example above, the fireplace points out other objects you could also focus on, whereas the last parenthesis includes one or more “actions” that you can perform on the fireplace only when you have it focused. 

This ends up pretty different from most traditional MUD-style inputs. When I first released this to the public, I found people logged off after their first examine. It turned out that they couldn’t figure out how to leave the focus mode. So they just assumed the thing was buggy and quit instead. Of course it’s mentioned if you care to write help, but this is clearly one step too many for such an important UI concept. 

So I ended up adding the header above that always reminds you. And since then I’ve not seen any confusion over how the focus mode works.

For making it easy to focus on things, I also decided that each room would only ever have one object named a particular thing. So there is for example only one single object in the game named “key” that you can focus on. 

Communication

I wanted players to co-exist in the same room so that they could collaborate on solving it. This meant communication must be possible. I pictured people would want to point things out and talk to each other.

In my first round of revisions I had a truckload of individual emotes; you could

      point at target

 for example. In the end I just limited it to  

     say/shout/whisper <message>

and 

     emote <whatever>

And seeing what people actually use, this is more than enough (say alone is probably 99% of what people need, really). I had a notion that the shout/whisper could be used in a puzzle later but in the end I decided that communication commands should be strictly between players and not have anything to do with the puzzles.

I removed all other interaction: There is no fighting and without an inventory or requirement to collaborate on puzzles, there is no need for other interactions than to communicate.

First version you didn’t even see what the others did, but eventually I added so that you at least saw what other players were focusing on at the moment (and of course if some major thing was solved/found).

In the end I don’t even list characters as objects in the room (you have to use the who command to see who’s in there with you).

Listing of commands available in the Evscaperoom (output of the help-command in game)
The main help command output.

Story

It’s very common for this type of game to have a dangerous or scary theme. Things like “get out before the bomb explodes”, “save the space ship before the engines overheat”, “flee the axe murderer before he comes back” etc). I’m no stranger to dark themes, but for this I wanted something friendlier and brighter, maybe with a some dark undercurrents here and there.

My Jester character is someone I’ve not only depicted in art, but she’s also an old RP character and literary protagonist of mine. Who else would find it funny to lock someone into a room only to provide crazy puzzles and hints for them to get out again? So my flimsy ‘premise’ was this: 

The village Jester wants to win the pie eating contest. You are one of her most dangerous opponents. She tricked you to her cabin and now you are locked in! If you don’t get out in time, she’ll get to eat all those pies on her own and surely win!


That’s it – this became the premise from which the entire game flowed. I quickly decided that it to be a very “small-scale” story: no life-or-death situation, no saving of the world. The drama takes place in a small village with an “adversary” that doesn’t really want to hurt you, but only to eat more pies than you.

From this, the way to offer hints came naturally – just eat a slice of “hintberry pie” the jester made (she even encourage you to eat it). It gives you a hint but is also very filling. So if you eat too much, how will you beat her in the contest later, even if you do get out?

To further the rustic and friendly tone I made sure the story took place on a warm summer day. Many descriptions describe sunshine, chirping birds and the smell of pie. I aimed at letting the text point out quirky and slightly comedic tone of the puzzles the Jester left behind. The player also sometimes gets teased by the game when doing things that does not make sense.

I won’t go into the story further here – it’s best if you experience it yourself. Let’s just say that the village has some old secrets. And and the Jester has her own ways of doing things and of telling a story. The game has multiple endings and so far people have drawn very different conclusions in the end.

Scoring

Most often in escape rooms, final score is determined by the time and the number of hints used. I do keep the latter – for every pie you eat, you get a penalty on your final score.

As for time – this background story would fit very well with a time limit (get out in X time, after which the pie-eating contest will start!). But from experience with other online text-based games I decided against this. Not only should a player be able to take a break, they may also want to wait for a friend to leave and come back etc. 

But more importantly, I want players to explore and read all my carefully crafted descriptions! So I’d much rather prefer they take their time and reward them for being thorough. 

So in the end I give specific scores for actions throughout the game instead. Most points are for doing things that drive the story forward, such as using something or solving a puzzle. But a significant portion of the score comes from turning every stone and trying everything out. The nice side-effect of this is that even if you know exactly how to solve everything and rush through the game you will still not end up with a perfect score. 

The final score, adjusted by hints is then used to determine if you make it in time to the contest and how you fare. This means that if you explore carefully you have a “buffer” of points so eating a few pies may still land you a good result in the end.
 

First sketch

I really entered the game ‘building’ aspect with no real notion of how the Jester’s cabin should look nor which puzzles should be in it. I tried to write things down beforehand but it didn’t really work for me. 

So in the end I decided “let’s just put a lot of interesting stuff in the room and then I’ll figure out how they interact with each other”. I’m sure this is different from game-maker to game-maker. But for me, this process worked perfectly. 

Scribbles on my notebook, sketching up the room's main items
My first, very rough, sketch of the Jester’s cabin


The above, first sketch ended up being what I used, although many of the objects mentioned never ended up in the final game and some things switched places. I did some other sketches too, but they’d be spoilers so I won’t show them here …


The actual game logic

The Evscaperoom principles outlined above deviate quite a bit from the traditional MU* style of game play. 

While Evennia provides everything for database management, in-game objects, commands, networking and other resources, the specifics of your game is something you need to make yourself – and you have the full power of Python to do it!

So for the first three days of the jam I used Evennia to build the custom game logic needed to provide the evscaperoom style of game play. I also made the tools I needed to quickly create the game content (which then took me the rest of the jam to make). 

In part 2 of this blog post I will cover the technical details of the Evscaperoom I built. I’ll also go through some issues I ran into and conclusions I drew. I’ll link to that from here when it’s available!

Continue to part 2.
Planet Python

Catalin George Festila: Python 3.7.3 : Using the win32com – part 004.

This is a source code I created a few days ago. import win32com.client strComputer = “.” objWMIService = win32com.client.Dispatch(“WbemScripting.SWbemLocator”) objSWbemServices = objWMIService.ConnectServer(strComputer,”root\cimv2″) colItems = objSWbemServices.ExecQuery(“Select * from Win32_ProgIDSpecification”) for objItem in colItems: print (“Caption: “, objItem.Caption) print (“Check
Planet Python

Catalin George Festila: Python 3.7.3 : Using the win32com – part 003.

This is another python script with win32com python module and python version 3.7.3 . The script shows us info about the drive: import win32com from win32com import client strComputer = “.” objWMIService = win32com.client.Dispatch(“WbemScripting.SWbemLocator”) objSWbemServices = objWMIService.ConnectServer(strComputer,”root\cimv2″) out_items = objSWbemServices.ExecQuery(“SELECT * FROM
Planet Python