Atomic file update (replace)

Useful for:

  • Saving documents to ensure there is always a non-broken version
  • git refs operations

APIs:

  • Posix: rename
  • Windows: ReplaceFile

Cache a computation using Numpy

try:
    example = np.load("example.npy")
except FileNotFoundError:
    example = compute()
    np.save("example.npy", example)

Cache a computation using Pickle + gzip

import gzip
import pickle

try:
    with gzip.open("example.pickle.gz", "rb") as f:
        a, b, c = pickle.load(f)
except FileNotFoundError:
    a, b, c = compute()

    with gzip.open("example.pickle.gz", "wb") as f:
        pickle.dump((a, b, c), f)

Check if an output needs to be rebuild from source based on modification timestamp

def need_to_rebuild(output_path, source_path):
    try:
        output_mtime = os.path.getmtime(output_path)
    except FileNotFoundError:
        return True

    source_mtime = os.path.getmtime(source_path)

    return output_mtime < source_mtime