The Python os Module: Dealing Directly With the System

python osThe Python os module may not seem that exciting (the official Python documentation refers to it as “miscellaneous operating system interfaces”), but it is valuable, because it gives you access to a variety of system-based functions, including many that are useful for file and directory manipulation.

(There’s a high demand for good Python programmers, by the way, and if you’re interested in learning Python, there are some very good classes available online.)

Using os

Before you use the os module, you first need to bring it up by means of the Python import command:

import os

This makes the os module available.  if you try to run os without importing it, you will see an error message:

NameError: name 'os' is not defined

You can test individual os commands in the Python shell by entering them on the Python shell command line.

System Dependent

Since the os module provides access to system features, the actual options that are available will depend on the underlying operating system.  Many of the os options are available for all operating systems, but there are several which are available for UNIX (and most UNIX-based systems, including Linux and the Mac OS), but not for Windows, and some which are available only for certain flavors of UNIX.

Since the available commands are system-dependent, when you create a Python program that uses os commands, you will probably want to include a check to find out which os module you’re dealing with, using the os.name command:

this_sys = os.name
if this_sys == "nt":
    print ("Windows")
elif this_sys == "posix":
    print ("POSIX")
elif this_sys == "os2":
    print ("Really??!")
...

Getting Around

So what is the os module good for?  Quite a lot, as it turns out.

For one thing, if includes several commands for directory navigation and directory manipulation.  Suppose for example that you’ve determined that Python is running on a Windows system, and you want to change the current working directory to C:\temp.  You can use the following command:

os.chdir("C:\\temp")

Notice that two backslashes in C:\\temp  —  the first one’s there to tell Python that the second backslash is part of the string, and not part of some formatting code.

The Case of the Missing Directory

But what if C:\temp isn’t there?  Look what happens if we try to change to some nonexistent directory:

 os.chdir("C:\\bangsnaggle")
Traceback (most recent call last):
  File "<pyshell#77>", line 1, in <module>
    os.chdir("C:\\bangsnaggle")
FileNotFoundError: [WinError 2] The system cannot find the file specified: 'C:\\bangsnaggle'

How do you avoid an error?

Check First

The os module has a way to check  —  the os.access() method:

os.access("C:\\bangsnaggle",os.F_OK)
False

The first argument for 0s.access() is the path, and the second argument is the mode  —  one of four codes indicating the kind of access that you’re checking for.  The mode we’re using (os.F_OK) just checks for the existence of the path.  If it finds the path, it returns “True”; if not, it returns “False”.

(The other modes are os.R_OK for “readable,” os.W_OK for “writable,” and os.X_OK for “executable.”)

Make it Happen

Once you know whether or not the directory exists, you can create it if its not there using the os.mkdir() method:

os.mkdir("C:\\bangsnaggle")

Now check for access again:

os.access("C:\\bangsnaggle",os.F_OK)
True

But what if you want to create a directory with a subdirectory?

os.mkdir("C:\\bangsnaggle\\subsnag")
Traceback (most recent call last):
  File "<pyshell#90>", line 1, in <module>
    os.mkdir("C:\\bangsnaggle\\subsnag")
FileNotFoundError: [WinError 3] The system cannot find the path specified: 'C:\\bangsnaggle\\subsnag'

It didn’t work!  What do you do?

Create a Path

You could, of course, create the parent directory, then create the subdirectory afterward:

os.mkdir("C:\\bangsnaggle")
os.mkdir("C:\\bangsnaggle\\subsnag")

But there’s an easier way  —  you can create an entire path recursively with the os.makedirs() method:

os.makedirs("C:\\bangsnaggle\\subsnag")

There are equivalent methods for removing directories and paths:

os.rmdir("C:\\bangsnaggle\\subsnag")

removes the \subsnag subdirectory, but leaves C:\bangsnaggle\ in place:

os.removedirs("C:\\bangsnaggle\\subsnag")

will remove the entire path  —    C:\bangsnaggle\ and the \subsnag subdirectory.

Doing Something With It

Now let’s do something constructive with the os module.  Remember the C:\temp directory?  Maybe we can use it now:

import os
is_temp = os.access("C:\\temp\\python",os.F_OK)
if is_temp == False:
     os.makedirs("C:\\temp\\python")
os.chdir("C:\\temp\\python")
py_tmp = os.open( "py_tmp.txt", os.O_RDWR|os.O_CREAT )
os.write(py_tmp, b"This is a Python scratchpad text file.\n")
os.close(py_tmp)

Check and Create

Using Python 3.4.0 running on Windows 7 Professional with admin privileges, this will check to see if C:\temp\python exists.  If it doesn’t exist, it will create it, using the os.makedirs() method.  If C:\tmp already exists, makedirs() won’t worry about the part that’s already there, it will just create the \python subdirectory in C:\temp.  If C:\temp doesn’t exist, it will create the entire C:\temp\python path.

It will then change the working directory to C:\temp\python using os.chdir().  Python needed to check the path using os.access before using os.makedirs() or os.chdir() because if the path had already existed, os.makedirs() would have generated an error, while if it hadn’t existed, os.chdir() would have produced an error.

Open or Create

Next, we open a file named “py_tmp.txt” in C:\temp\python using os.open().  But what if the file doesn’t exist?  As it turns out, os.open() requires at least one flag, which can be combined using the | “or” operator.  We use two flags:  os.O_RDWR, which opens a file as read/write, and os.O_CREAT, which creates a file if none exists.

Since the file doesn’t exist the first time that we run the program, we automatically create it.

Write and Close

Next, we write some text to the file, then close it.  Notice the “b” before the text string.  it’s necessary in this case because Python 3 handles strings differently than Python 2.  If we tried to run the code without the added “b”, it would generate an error in Python 3:

os.write(py_tmp, "This is a Python scratchpad text file.\n")

with this error:

TypeError: 'str' does not support the buffer interface

The problem is that in Python 3, str assumes that the string is Unicode, and when it encounters an ASCII string, it doesn’t handle it automatically.  So we use the initial “b” to tell Python to handle the string as bytes, rather than double-byte Unicode.

Much More

Needless to say, there’s much, much more that you can do with the Python os module, and with Python in general.  Among other things, the os module includes this handy method:  os.system().  You can use it to run most native system commands, which you enter as strings:

os.system("systeminfo > infofile.txt")

“systeminfo > infofile.txt” is just a standard Windows command-line string; it runs the Windows SYSTEMINFO command and sends the output to a file named infofile.txt in the working directory, creating the file if it doesn’t already exist.

Note that os.system() doesn’t capture the output from SYSTEMINFO itself; all it returns in Python is 0 from the cmd.exe shell.  You could open infofile.txt and read the contents, but if you want to return values to Python, it’s better to run Python os commands rather than native system commands.  What os.system() is good for is taking care of tasks that only the native system can handle.

UNIX, Linus, and the MAC OS

As we mentioned earlier, on POSIX systems, such as UNIX, Linux, the Macintosh OS, or Android, a variety of UNIX-based commands are available as part of the os module (as opposed to using them by means of os.system()).  The actual commands that are available depend on the system.  In general, the UNIX-based os.commands give you access to a large subset of the file, directory, and system capabilities which are available at the root level.  This includes such commands as chroot(), chown(), and chmod().

As you can see, if you’re planning on making more than just minor use of the os module, you need to take into account the nature and the capabilities of the systems on which it will be running, and to make allowance for the differences between systems.  But if you do keep system differences in mind, you will find that the os module can provide you with a set of tools which can be of considerable practical use when you start to put Python to work.

And given how versatile Python is, how important it has become, and how great a demand for Python programmers there is and almost certainly will continue to be, that says a lot.