Determining Multiple CPUs with Python

Just a quick entry for something i learned about that's pretty nifty. So if you wanted to take advantage of multiple cpus in python, it used to be you'd have to do a bit of OS detection first. This post has the source i repeat here:

def detectCPUs():
"""
Detects the number of CPUs on a system. Cribbed from pp.
"""
# Linux, Unix and MacOS:
if hasattr(os, "sysconf"):
  if os.sysconf_names.has_key("SC_NPROCESSORS_ONLN"):
      # Linux & Unix:
      ncpus = os.sysconf("SC_NPROCESSORS_ONLN")
      if isinstance(ncpus, int) and ncpus > 0:
          return ncpus
  else: # OSX:
      return int(os.popen2("sysctl -n hw.ncpu")[1].read())
# Windows:
if os.environ.has_key("NUMBER_OF_PROCESSORS"):
      ncpus = int(os.environ["NUMBER_OF_PROCESSORS"]);
      if ncpus > 0:
          return ncpus
return 1 # Default

That is a bit of a pain to remember all the time. Thankfully since Python 2.6, you can use the multiprocessing library to handle this for you. All the details of how it determines the number of cpus are now abstracted away from me. I don't really care, just do it =)

import multiprocessing
numCPUs = multiprocessing.cpu_count()

Then you can go from there. For instance, if i have 1 cpu, the program could decide to use the threading library. If i have 2+ cpus then it may benefit from using the full on multiprocessor library. Depends on your application.