Multithreading vs Multiprocessing vs Distributed Systems

• multi-threading: multiple threads running within a single process, sharing memory and resources.
• multi-processing: multiple processes running on a single machine, using multiple central processing units which cooperate and communicate among themselves using shared memory, priority-based task scheduling, and preemptive interrupt handling.
• Distributed systems: multiple processes running on different machines; the processes communicate by passing messages over the network.

For Distributed systems, since the processes are running on different machines:

• The machines can fail independently
• Communication between processes is unreliable
• Reasoning about the behavior of distributed systems is difficult, as they're prone to complex emergent behaviours