Let's consider a simple CPU architecture. As shown, a CPU consists of three main parts:

• Registers
  
• Logic blocks
  We have two shown: The ALU and the Instruction Decode.
• Busses
  We have two shown: Bus0 and Bus1. Busses are communication lines which connect registers together. That is how we can move data from, for instance, the Accumulator to Register0. And it is how we move any data into the ALU.

There is another part of the CPU that is not shown. This is the clock unit. Each operation in the CPU happens in a clock cycle. That is how the time is allocated when moving data from place to place inside the CPU.

• PC -- Program Counter
  Each machine instruction is loaded from the address in main memory which is pointed to by the PC. After each instruction the PC is incremented. If the instruction includes data words, the PC will be incremented multiple times past those words. If the instruction is a jump, the PC will be loaded with that new address.
• IR -- Instruction Register
  Each instruction to be executed is loaded into the IR.
• Instruction Decode
  The Instruction Decode logic views the contents of the IR. It sends out control signals to the ALU [shown] and each register [not shown], telling each what it will do during this instruction.
• ALU -- Arithmetic/Logic Unit
  Performs math and other operations on its two inputs. Those inputs are the Bus0 and Bus1.
• Accumulator
  All outputs from the ALU are stored in the Accumulator. From there they may be moved to other registers or to main memory.
• Memory Data Out
  Main memory is typically slower than the CPU. This register holds data destined for the main memory until that data is safely stored.
• Memory Address
  Each memory address to be read or written is pointed to by this register.
• Memory Data In
  Data from main memory is held in this register before being passed to Bus0.
• Main Memory -- RAM
  The main memory is not actually part of the CPU, but is shown here for clarity. In this architecture, I/O devices such as the serial and parallel ports will be memory mapped. This is to say that they will appear to the CPU as tho they are actually in the main memory, even tho they are not actually RAM.
• Register Bank
  These are general purpose registers [Register0 thru RegisterN] available for temporary storage during computation.
• Bus0
  The main data bus within the CPU.
• Bus1
  Secondary data bus feeding into the ALU. This allows us to perform arithmetic upon any of the registers combined with either the Accumulator or one of the registers in the Register Bank.

A different simple architecture is described here: http://www.dgp.toronto.edu/~ajr/258/notes/microcode.html That is the case of a single bus CPU. Yet another simple architecture is described here: http://www.zetetics.com/bj/papers/piscedu2.htm