Showing posts with label Apollo. Show all posts
Showing posts with label Apollo. Show all posts

Reverse engineering the mysterious Up-Data Link Test Set from Apollo

Back in 2021, a collector friend of ours was visiting a dusty warehouse in search of Apollo-era communications equipment. A box with NASA-style lights caught his eye—the "AGC Confirm" light suggested a connection with the Apollo Guidance Computer. Disappointingly, the box was just an empty chassis and the circuit boards were all missing. He continued to poke around the warehouse when, to his surprise, he found a bag on the other side of the warehouse that contained the missing boards! After reuniting the box with its wayward circuit cards, he brought it to us: could we make this undocumented unit work?

The Up-Data Link Confidence Test Set, powered up.

The Up-Data Link Confidence Test Set, powered up.

A label on the back indicated that it is an "Up-Data Link Confidence Test Set", built by Motorola. As the name suggests, the box was designed to test Apollo's Up-Data Link (UDL), a system that allowed digital commands to be sent up to the spacecraft. As I'll explain in detail below, these commands allowed ground stations to switch spacecraft circuits on or off, interact with the Apollo Guidance Computer, or set the spacecraft's clock. The Up-Data Link needed to be tested on the ground to ensure that its functions operated correctly. Generating the test signals for the Up-Data Link and verifying its outputs was the responsibility of the Up-Data Link Confidence Test Set (which I'll call the Test Set for short)

The Test Set illustrates how, before integrated circuits, complicated devices could be constructed from thumb-sized encapsulated modules. Since I couldn't uncover any documentation on these modules, I had to reverse-engineer them, discovering that different modules implemented everything from flip-flops and logic gates to opto-isolators and analog circuits. With the help of a Lumafield 3-dimensional X-ray scanner, we looked inside the modules and examined the discrete transistors, resistors, diodes, and other components mounted inside.

Four of the 13-pin Motorola modules. These implement logic gates (2/2G & 2/1G), lamp drivers (LD), more logic gates (2P/3G), and a flip-flop (LP FF). The modules have 13 staggered pins, ensuring that they can't be plugged in backward.

Four of the 13-pin Motorola modules. These implement logic gates (2/2G & 2/1G), lamp drivers (LD), more logic gates (2P/3G), and a flip-flop (LP FF). The modules have 13 staggered pins, ensuring that they can't be plugged in backward.

Reverse-engineering this system—from the undocumented modules to the mess of wiring—was a challenge. Mike found one NASA document that mentioned the Test Set, but the document was remarkably uninformative.1 Moreover, key components of the box were missing, probably removed for salvage years ago. In this article, I'll describe how we learned the system's functionality, uncovered the secrets of the encapsulated modules, built a system to automatically trace the wiring, and used the UDL Test Set in a large-scale re-creation of the Apollo communications system.

The Apollo Up-Data Link

Before describing the Up-Data Link Test Set, I'll explain the Up-Data Link (UDL) itself. The Up-Data Link provided a mechanism for the Apollo spacecraft to receive digital commands from ground stations. These commands allowed ground stations to control the Apollo Guidance Computer, turn equipment on or off, or update the spacecraft's clock. Physically, the Up-Data Link is a light blue metal box with an irregular L shape, weighing almost 20 pounds.

The Up-Data Link box.

The Up-Data Link box.

The Apollo Command Module was crammed with boxes of electronics, from communication and navigation to power and sequencing. The Up-Data Link was mounted above the AC power inverters, below the Apollo Guidance Computer, and to the left of the waste management system and urine bags.

The lower equipment bay of the Apollo Command Module. The Up-Data Link is highlighted in yellow. Click this image (or any other) for a larger version. From Command/Service Module Systems Handbook p212.

The lower equipment bay of the Apollo Command Module. The Up-Data Link is highlighted in yellow. Click this image (or any other) for a larger version. From Command/Service Module Systems Handbook p212.

Up-Data Link Messages

The Up-Data Link supported four types of messages:

  • Mission Control had direct access to the Apollo Guidance Computer (AGC) through the UDL, controlling the computer, keypress by keypress. That is, each message caused the UDL to simulate a keypress on the Display/Keyboard (DSKY), the astronaut's interface to the computer.

  • The spacecraft had a clock, called the Central Timing Equipment or CTE, that tracked the elapsed time of the mission, from days to seconds. A CTE message could set the clock to a specified time.

  • A system called Real Time Control (RTC) allowed the UDL to turn relays on or off, so some spacecraft systems to be controlled from the ground.2 These 32 relays, mounted inside the Up-Data Link box, could do everything from illuminating an Abort light—indicating that Mission Control says to abort—to controlling the data tape recorder or the S-band radio.

  • Finally, the UDL supported two test messages to "exercise all process, transfer and program control logic" in the UDL.

The diagram below shows the format of messages to the Up-Data Link. Each message consisted of 12 to 30 bits, depending on the message type. The first three bits, the Vehicle Address, selected which spacecraft should receive the message. (This allowed messages to be directed to the Saturn V booster, the Command Module, or the Lunar Module.3) Next, three System Address bits specified the spacecraft system to receive the message, corresponding to the four message types above. The remaining bits supplied the message text.

Format of the messages to the Up-Data Link. From Telecommunication Systems Study Guide.
Note that the vehicle access code uses a different sub-bit pattern from the rest of the message.
This diagram shows an earlier sub-bit encoding, not the encoding used by the Test Set.

Format of the messages to the Up-Data Link. From Telecommunication Systems Study Guide. Note that the vehicle access code uses a different sub-bit pattern from the rest of the message. This diagram shows an earlier sub-bit encoding, not the encoding used by the Test Set.

The contents of the message text depended on the message type. A Real Time Control (RTC) message had a six-bit value specifying the relay number as well as whether it should be turned off or on. An Apollo Guidance Computer (AGC) message had a five-bit value specifying a key on the Display/Keyboard (DSKY). For reliability, the message was encoded in 16 bits: the message, the message inverted, the message again, and a padding bit; any mismatching bits would trigger an error. A CTE message set the clock using four 6-bit values indicating seconds, minutes, hours, and days. The UDL processed the message by resetting the clock and then advancing the time by issuing the specified number of pulses to the CTE to advance the seconds, minutes, hours, and days. (This is similar to setting a digital alarm clock by advancing the digits one at a time.) Finally, the two self test messages consisted of 24-bit patterns that would exercise the UDL's internal circuitry. The results of the test were sent back to Earth via Apollo's telemetry system.

For reliability, each bit transmitted to the UDL was replaced by five "sub-bits": each "1" bit was replaced with the sub-bit sequence "01011", and each "0" bit was replaced with the complement, "10100".4 The purpose of the sub-bits was that any corrupted data would result in an invalid sub-bit code so corrupted messages could be rejected. The Up-Data Link performed this validation by matching the input data stream against "01011" or "10100". (The vehicle address at the start of a message used a different sub-bit code, ensuring that the start of the message was properly identified.) By modern standards, sub-bits are an inefficient way of providing redundancy, since the message becomes five times larger. As a consequence, the effective transmission rate was low: 200 bits per second.

There was no security in the Up-Data Link messages, apart from the need for a large transmitter. Of the systems on Apollo, only the rocket destruct system—euphemistically called the Propellant Dispersion System—was cryptographically secure.5

Since the Apollo radio system was analog, the digital sub-bits couldn't be transmitted from ground to space directly. Instead, a technique called phase-shift keying (PSK) converted the data into an audio signal. This audio signal consists of a sine wave that is inverted to indicate a 0 bit versus a 1 bit; in other words, its phase is shifted by 180 degrees for a 0 bit. The Up-Data Link box takes this audio signal as input and demodulates it to extract the digital message data. (Transmitting this audio signal from ground to the Up-Data Link required more steps that aren't relevant to the Test Set, so I'll describe them in a footnote.6)

The Up-Data Link Test Set

Now that I've explained the Up-Data Link, I can describe the Test Set in more detail. The purpose of the UDL Test Set is to test the Up-Data Link system. It sends a message—as an audio signal—to the Up-Data Link box, implementing the message formatting, sub-bit encoding, and phase shift keying described above. Then it verifies the outputs from the UDL to ensure that the UDL performed the correct action.

Perhaps the most visible feature of the Test Set is the paper tape reader on the front panel: this reader is how the Test Set obtains messages to transmit. Messages are punched onto strips of paper tape, encoded as a sequence of 13 octal digits.7 After a message is read from paper tape, it is shown on the 13-digit display. The first three digits are an arbitrary message number, while the remaining 10 octal digits denote the 30-bit message to send to the UDL. Based on the type of message, specified by the System Address digit, the Test Set validates the UDL's response and indicates success or errors on the panel lights.

I created the block diagram below to explain the architecture and construction of the Test Set (click for a larger view). The system has 25 circuit boards, labeled A1 through A25;8 for the most part, they correspond to functional blocks in the diagram.

My block diagram of the Up-Data Link Test Set. (Click for a larger image.)

My block diagram of the Up-Data Link Test Set. (Click for a larger image.)

The Test Set's front panel is dominated by its display of 13 large digits. It turns out that the storage of these digits is the heart of the Test Set. This storage (A3-A9) assembles the digits as they are read from the paper tape, circulates the bits for transmission, and provides digits to the other circuits to select the message type and validate the results. To accomplish this, the 13 digit circuits are configured as a 39-bit shift register. As the message is read from the paper tape, its bits are shifted into the digit storage, right to left, and the message is shown on the display. To send the message, the shift register is reconfigured so the 10 digits form a loop, excluding the message number. As the bits cycle through the loop, the leftmost bit is encoded and transmitted. At the end of the transmission, the digits have cycled back to their original positions, so the message can be transmitted again if desired. Thus, the shift-register mechanism both deserializes the message when it is read and serializes the message for transmission.

The Test Set uses three boards (A15, A2, and A1) to expand the message with sub-bits and to encode the message into audio. The first board converts each bit into five sub-bits. The second board applies phase-shift keying (PSK) modulation, and the third board has filters to produce clean sine waves from the digital signals.

On the input side, the Test Set receives signals from the Up-Data Link (UDL) box through round military-style connectors. These input signals are buffered by boards A25, A22, A23, A10, and A24. Board 15 verifies the input sub-bits by comparing them with the transmitted sub-bits. For an AGC message, the computer signals are verified by board A14. The timing (CTE) signals are verified by boards A20 and A21. The UDL status (validity) signals are processed by board A12. Board A11 implements a switching power supply to power the interface boards.

You can see from the block diagram that the Test Set is complex and implements multiple functions. On the other hand, the block diagram also shows that it takes a lot of 1960s circuitry to implement anything. For instance, one board can only handle two digits, so the digit display alone requires seven boards. Another example is the inputs, requiring a full board for two or three input bits.

Encapsulated modules

The box is built from modules that are somewhat like integrated circuits but contain discrete components. Modules like these were used in the early 1960s before ICs caught on. Each module implements a simple function such as a flip-flop or buffer. They were more convenient than individual components, since a module provided a ready-made function. They were also compact, since the components were tightly packaged inside the module.

Physically, each module has 13 pins: a row of 7 on one side and a row of 6 offset on the other side. This arrangement ensures that a module cannot be plugged in backward.

A Motorola "LP FF" module. This module implements a J-K flip-flop. "LP" could indicate low performance, low power, or low propagation; the system also uses "HP FF" modules, which could be high performance.

A Motorola "LP FF" module. This module implements a J-K flip-flop. "LP" could indicate low performance, low power, or low propagation; the system also uses "HP FF" modules, which could be high performance.

Reverse engineering these modules was difficult since they were encapsulated in plastic and the components were inaccessible. The text printed on each module hinted at its function. For example, the J-K flip-flop module above is labeled "LP FF". The "2/2G & 2/1G" module turned out to contain two NAND gates and two inverters (the 2G and 1G gates). A "2P/3G" module contains two pull-up resistors and two three-input NAND gates. Other modules provided special-purpose analog functions for the PSK modulation.

I reverse-engineered the functions of the modules by applying signals and observing the results. Conveniently, the pins are on 0.200" spacing so I could plug modules into a standard breadboard. The functions of the logic modules were generally straightforward to determine. The analog modules were more difficult; for instance, the "-3.9V" module contains a -3.9-volt Zener diode, six resistors, and three capacitors in complicated arrangements.

To determine how the modules are constructed internally, we had a module X-rayed by John McMaster and another module X-rayed in three dimensions by Lumafield. The X-rays revealed that modules were built with "cordwood construction", a common technique in the 1960s. That is, cylindrical components were mounted between two boards, stacked parallel similar to a pile of wood logs. Instead of using printed-circuit boards, the leads of the components were welded to metal strips to provide the interconnections.

A 3-D scan of the module showing the circuitry inside the compact package, courtesy of Lumafield. Two transistors are visible near the center.

A 3-D scan of the module showing the circuitry inside the compact package, courtesy of Lumafield. Two transistors are visible near the center.

For more information on these modules, see my articles Reverse-engineering a 1960s cordwood flip-flop module with X-ray CT scans and X-ray reverse-engineering a hybrid module. You can interact with the scan here.

The boards

In this section, I'll describe some of the circuit boards and point out their interesting features. A typical board has up to 15 modules, arranged as five rows of three. The modules are carefully spaced so that two boards can be meshed with the components on one board fitting into the gaps on the other board. Thus, a pair of boards forms a dense block.

This photo shows how the modules of the two circuit boards are arranged so the boards can be packed together tightly.

This photo shows how the modules of the two circuit boards are arranged so the boards can be packed together tightly.

Each pair of boards is attached to side rails and a mounting bracket, forming a unit.8 The bracket has ejectors to remove the board unit, since the backplane connectors grip the boards tightly. Finally, each bracket is labeled with the board numbers, the test point numbers, and the Motorola logo. The complexity of this mechanical assembly suggests that Motorola had developed an integrated prototyping system around the circuit modules, prior to the Test Set.

Digit driver boards

The photo below shows a typical board, the digit driver board. At the left, a 47-pin plug provides the connection between the board and the Test Set's backplane. At the right, 15 test connections allow the board to be probed and tested while it is installed. The board itself is a two-sided printed circuit board with gold plating. Boards are powered with +6V, -6V, and ground; the two red capacitors in the lower left filter the two voltages.

Boards A4 through A9 are identical digit driver boards.

Boards A4 through A9 are identical digit driver boards.

The digit driver is the most common board in the system, appearing six times.9 Each board stores two octal digits in a shift register and drives two digit displays on the front panel. Since the digits are octal, each digit requires three bits of storage, implemented with three flip-flop modules connected as a shift register. If you look closely, you can spot the six flip-flop modules, labeled "LP FF".

The digits are displayed through an unusual technology: an edge-lit lightguide display.10 From a distance, it resembles a Nixie tube, but it uses 10 lightbulbs, one for each number value, with a plastic layer for each digit. Each plastic sheet has numerous dots etched in the shape of the corresponding number. One sheet is illuminated from the edge, causing the dots in the sheet to light up and display that number. In the photo below, you can see both the illuminated and the unilluminated dots. The displays take 14 volts, but the box runs at 28 volts, so a board full of resistors on the front panel drops the voltage from 28 to 14, giving off noticeable heat in the process.

A close-up of a digit in the Test Set, showing the structure of the edge-lit lightguide display.

A close-up of a digit in the Test Set, showing the structure of the edge-lit lightguide display.

For each digit position, the driver board provides eight drive signals, one for each bulb. The drivers are implemented in "LD" modules. Since each LD module contains two drive transistors controlled by 4-input AND gates, a module supports two bulbs. Thus, a driver board holds eight LD modules in total. The LD modules are also used on other boards to drive the lights on the front panel.

Ring counters

The Test Set contains multiple counters to count bits, sub-bits, digits, states, and so forth. While a modern design would use binary counters, the Test Set is implemented with a circuit called a ring counter that optimizes the hardware.

For instance, to count to ten, five flip-flops are arranged as a shift register so each flip-flop sends its output to the next one. However, the last flip-flop sends its inverted output to the first. The result is that the counter will proceed: 10000, 11000, 11100, 11110, 11111 as 1 bits are shifted in at the left. But after a 1 reaches the last bit, 0 bits will be shifted in at the left: 01111, 00111, 00011, 00001, and finally 0000. Thus, the counter moves through ten states.

Why not use a 4-bit binary counter and save a flip-flop? First, the binary counter requires additional logic to go from 9 back to 0. Moreover, acting on a particular binary value requires a 4-input gate to check the four bits. But a particular value of a ring counter can be detected with a smaller 2-input gate by checking the bits on either side of the 0/1 boundary. For instance, to detect a count of 3 (11100), only the two highlighted bits need to be tested. Thus, the decoding logic is much simpler for a ring counter, which is important when each gate comes in an expensive module.

Another use of the ring counter is in the sub-state generator, counting out the five states. Since this ring counter uses three flip-flops, you might expect it to count to six. However, the first flip-flop gets one of its inputs from the second flip-flop, resulting in five states: 000, 100, 110, 011, and 001, with the 111 state skipped.11 This illustrates the flexibility of ring counters to generate arbitrary numbers of states.

The PSK boards

Digital data could not be broadcast directly to the spacecraft, so the data was turned into an audio signal using phase-shift keying (PSK). The Test Set uses two boards (A1 and A2) to produce this signal. These boards are interesting and unusual because they are analog, unlike the other boards in the Test Set.

The idea behind phase-shift keying is to change the phase of a sine wave depending on the bit (i.e., sub-bit) value. Specifically, a 2 kHz sine wave indicated a one bit, while the sine wave was inverted for a zero bit. That is, a phase shift of 180º indicated a 0 bit. But how do you tell which sine wave is original and which is flipped? The solution was to combine the information signal with a 1 kHz reference signal that indicates the start and phase of each bit. The diagram below shows how the bits 1-0-1 are encoded into the composite audio signal that is decoded by the Up-Data Link box.

The phase-shift keying modulation process. This encoded digital data into an audio signal for transmission to the Up-DataLink. Note that "1 kc" is 1 kilocycle, or 1 kilohertz in modern usage. From Apollo Digital Up-Data Link Description.

The phase-shift keying modulation process. This encoded digital data into an audio signal for transmission to the Up-DataLink. Note that "1 kc" is 1 kilocycle, or 1 kilohertz in modern usage. From Apollo Digital Up-Data Link Description.

The core of the PSK modulation circuit is a transformer with a split input winding. The 2 kHz sine wave is applied to the winding's center tap. One side of the winding is grounded (by the "ø DET" module) for a 0 bit, but the other side of the winding is grounded for a 1 bit. This causes the signal to go through the winding in one direction for a 1 bit and the opposite direction for a 0 bit. The transformer's output winding thus receives an inverted signal for a 0 bit, giving the 180º phase shift seen in the second waveform above. Finally, the board produces the composite audio signal by mixing in the reference signal through a potentiometer and the "SUM" module.12

Board A2 is the heart of the PSK encoding. The black transformer selects the phase shift, controlled by the "ø DET" and "ø DET D" modules in front of it. The two central potentiometers  balance the components of the output signal.

Board A2 is the heart of the PSK encoding. The black transformer selects the phase shift, controlled by the "ø DET" and "ø DET D" modules in front of it. The two central potentiometers balance the components of the output signal.

Inconveniently, some key components of the Test Set were missing; probably the most valuable components were salvaged when the box was scrapped. The missing components included the power supplies and amplifiers on the back of the box, as well as parts from PSK board A1. This board had ten white wires that had been cut, going to missing components labeled MP1, R2, L1, and L2. By studying the circuitry, I determined that MP1 had been a 4-kHz oscillator that provided the master clock for the Test Set. R2 was simply a potentiometer to adjust signal levels.

Marc added circuitry to board A1 to replace the two missing filters and the missing oscillator. (The oscillator was used earlier to drive a clock from Soyuz.)

Marc added circuitry to board A1 to replace the two missing filters and the missing oscillator. (The oscillator was used earlier to drive a clock from Soyuz.)

But L1 and L2 were more difficult. It took a lot of reverse-engineering before we determined that L1 and L2 were resonant filters to convert the digital waveforms to the sine waves needed for the PSK output. Marc used a combination of theory and trial-and-error to determine the inductor and capacitor values that produced a clean signal. The photo above shows our substitute filters, along with a replacement oscillator.

Input boards

The Test Set receives signals from the Up-Data Link box under test and verifies that these signals are correct. The Test Set has five input boards (A22 through A25) to buffer the input signals and convert them to digital levels. The input boards also provide electrical isolation between the input signals and the Test Set, avoiding problems caused by ground loops or different voltage levels.

A typical input board is A22, which receives two input signals, supplied through coaxial cables. The board buffers the signals with op-amps, and then produces a digital signal for use by the box. The op-amp outputs go into "1 SS" isolation modules that pass the signal through to the box while ensuring isolation. These modules are optocouplers, using an LED and a phototransistor to provide isolation.13 The op-amps are powered by an isolated power supply.

Board A22 handles two input signals. It has two op-amps and associated circuitry. Note the empty module positions; board A23 has these positions populated so it supports three inputs.

Board A22 handles two input signals. It has two op-amps and associated circuitry. Note the empty module positions; board A23 has these positions populated so it supports three inputs.

Each op-amp module is a Burr-Brown Model 1506 module,14 encapsulating a transistorized op-amp into a convenient 8-pin module. The module is similar to an integrated-circuit op-amp, except it has discrete components inside and is considerably larger than an integrated circuit. Burr-Brown is said to have created the first solid-state op-amp in 1957, and started making op-amp modules around 1962.

Board A24 is also an isolated input board, but uses different circuitry. It has two modules that each contain four Schmitt triggers, circuits to sharpen up a noisy input. These modules have the puzzling label "-12+6LC". Each output goes through a "1 SS" isolation module, as with the previous input boards. This board receives the 8-bit "validity" signal from the Up-Data Link.

The switching power supply board

Board A11 is interesting: instead of sealed modules, it has a large green cube with numerous wires attached. This board turned out to be a switching power supply that implements six dual-voltage power supplies. The green cube is a transformer with 14 center-tapped windings connected to 42 pins. The transformer ensures that the power supply's outputs are isolated. This allows the op-amps on the input boards to remain electrically isolated from the rest of the Test Set.

The switching power supply board is dominated by a large green transformer with many windings. The two black power transistors are at the front.

The switching power supply board is dominated by a large green transformer with many windings. The two black power transistors are at the front.

The power supply uses a design known as a Royer Converter; the two transistors drive the transformer in a push-pull configuration. The transistors are turned on alternately at high frequency, driven by a feedback winding. The transformer has multiple windings, one for each output. Each center-tapped winding uses two diodes to produce a DC output, filtered by the large capacitors. In total, the power supply has four ±7V outputs and two ±14V outputs to supply the input boards.

This switching power supply is independent from the power supplies for the rest of the Test Set. On the back of the box, we could see where power supplies and amplifiers had been removed. Determining the voltages of the missing power supplies would have been a challenge. Fortunately, the front of the box had test points with labels for the various voltages: -6, +6, and +28, so we knew what voltages were required.

The front panel

The front panel reveals many of the features of the Test Set. At the top, lights indicate the success or failure of various tests. "Sub-bit agree/error" indicates if the sub-bits read back into the Test Set match the values sent. "AGC confirm/error" shows the results of an Apollo Guidance Computer message, while "CTE confirm/error" shows the results of a Central Timing Equipment message. "Verif confirm/error" indicates if the verification message from the UDL matches the expected value for a test message. At the right, lights indicate the status of the UDL: standby, active, or powered off.

A close-up of the Test Set's front panel.

A close-up of the Test Set's front panel.

In the middle, toggle switches control the UDL operation. The "Sub-bit spoil" switch causes sub-bits to be occasionally corrupted for testing purposes. "Sub-bit compare/override" enables or disables sub-bit verification. The four switches on the right control the paper tape reader. The "Program start" switch is the important one: it causes the UDL to send one message (in "Single" mode) or multiple messages (in "Serial" mode). The Test Set can stop or continue when an error occurs ("Stop on error" / "Bypass error"). Finally, "Tape advance" causes messages to be read from paper tape, while "Tape stop" causes the UDL to re-use the current message rather than loading a new one.

The UDL provides a verification code that indicates its status. The "Verification Return" knob selects the source of this verification code: the "Direct" position uses a 4-bit verification code, while "Remote" uses an 8-bit verification code.15

At the bottom, "PSK high/low" selects the output level for the PSK signal from the Test Set. (Since the amplifier was removed from our Test Set, this switch has no effect. Likewise, the "Power On / Off" switch has no effect since the power supplies were removed. We power the Test Set with an external lab supply.) In the middle, 15 test points allow access to various signals inside the Test Set. The round elapsed time indicator shows how many hours the Test Set has been running (apparently over 12 months of continuous operation).

Reverse-engineering the backplane

Once I figured out the circuitry on each board, the next problem was determining how the boards were connected. The backplane consists of rows of 47-pin sockets, one for each board. Dense white wiring runs between the sockets as well as to switches, displays, and connectors. I started beeping out the connections with a multimeter, picking a wire and then trying to find the other end. Some wires were easy since I could see both ends, but many wires disappeared into a bundle. I soon realized that manually tracing the wiring was impractically slow: with 25 boards and 47 connections per board, brute-force testing of every pair of connections would require hundreds of thousands of checks.

The backplane wiring of the Test Set consisted of bundles of white wires, as shown in this view of the underside of the Test Set.

The backplane wiring of the Test Set consisted of bundles of white wires, as shown in this view of the underside of the Test Set.

To automate the beeping-out of connections, I built a system that I call Beep-o-matic. The idea behind Beep-o-matic is to automatically find all the connections between two motherboard slots by plugging two special boards into the slots. By energizing all the pins on the first board in sequence, a microcontroller can detect connected pins on the second board, revealing the wiring between the two slots.

This system worked better than I expected, rapidly generating a list of connections. I still had to plug the Beep-o-matic boards into each pair of slots (about 300 combinations in total), but each scan took just a few seconds, so a full scan was practical. To find the wiring to the switches and connectors, I used a variant of the process. I plugged a board into a slot and used a program to continuously monitor the pins for changes. I went through the various switch positions and applied signals to the connectors to find the associated connections.

Conclusions

I started reverse-engineering the Test Set out of curiosity: given an undocumented box made from mystery modules and missing key components, could we understand it? Could we at least get the paper tape reader to run and the lights to flash? It was a tricky puzzle to figure out the modules and the circuitry, but eventually we could read a paper tape and see the results on the display.

But the box turned out to be useful. Marc has amassed a large and operational collection of Apollo communications hardware. We use the UDL Test Set to generate realistic signals that we feed into Apollo's S-band communication system. We haven't transmitted these signals to the Moon, but we have transmitted signals between antennas a few feet apart, receiving them with a box called the S-band Transponder. Moreover, we have used the Test Set to control an Up-Data Link box, a CTE clock, and a simulated Apollo Guidance Computer, reading commands from the paper tape and sending them through the complete communication path. Ironically, the one thing we haven't done with the Test Set is use it to test the Up-Data Link in the way it is intended: connecting the UDL's outputs to the Test Set and checking the panel lights.

From a wider perspective, the Test Set provides a glimpse of the vast scope of the Apollo program. This complicated box was just one part of the test apparatus for one small part of Apollo's electronics. Think of the many different electronic systems in the Apollo spacecraft, and consider the enormous effort to test them all. And electronics was just a small part of Apollo alongside the engines, mechanical structures, fuel cells, and life support systems. With all this complexity, it's not surprising that the Apollo program employed 400,000 people.

For more information, the footnotes include a list of UDL documentation16 and CuriousMarc's videos17. Follow me on Bluesky (@righto.com), Mastodon (@[email protected]), or RSS. (I've given up on Twitter.) I worked on this project with CuriousMarc, Mike Stewart, and Eric Schlapfer. Thanks to John McMaster for X-rays, thanks to Lumafield for the CT scans, and thanks to Marcel for providing the box.

Notes and references

  1. Mike found a NASA document Functional Integrated System Schematics that includes "Up Data Link GSE/SC Integrated Schematic Diagram". Unfortunately, this was not very helpful since the diagram merely shows the Test Set as a rectangle with one wire in and one wire out. The remainder of the diagram (omitted) shows that the output line passes through a dozen boxes (modulators, switches, amplifiers, and so forth) and then enters the UDL onboard the Spacecraft Command Module. At least we could confirm that the Test Set was part of the functional integrated testing of the UDL.

    Detail from "Up Data Link GSE/SC Integrated Schematic Diagram", page GT3.

    Detail from "Up Data Link GSE/SC Integrated Schematic Diagram", page GT3.

    Notably, this diagram has the Up-Data Link Confidence Test Set denoted with "2A17". If you examine the photo of the Test Set at the top of the article, you can see that the physical box has a Dymo label "2A17", confirming that this is the same box. 

  2. The table below lists the functions that could be performed by sending a "realtime command" to the Up-Data Link to activate a relay. The crew could reset any of the relays except for K1-K5 (Abort Light A and Crew Alarm).

    The functions controlled by the relays. Adapted from Command/Service Module Systems Handbook.

    The functions controlled by the relays. Adapted from Command/Service Module Systems Handbook.

    A message selected one of 32 relays and specified if the relay should be turned on or off. The relays were magnetic latching relays, so they stayed in the selected position even when de-energized. The relay control also supported "salvo reset": four commands to reset a bank of relays at once. 

  3. The Saturn V booster had a system for receiving commands from the ground, closely related to the Up-Data Link, but with some differences. The Saturn V system used the same Phase-Shift Keying (PSK) and 70 kHz subcarrier as the Up-Data Link, but the frequency of the S-band signal was different for Saturn V (2101.8 MHz). (Since the Command Module and the booster use separate frequencies, the use of different addresses in the up-data messages was somewhat redundant.) Both systems used sub-bit encoding. Both systems used three bits for the vehicle address, but the remainder of the Saturn message was different, consisting of 14 bits for the decoder address, and 18 bits for message data. A typical message for the Launch Vehicle Digital Computer (LVDC) includes a 7-bit command followed by the 7 bits inverted for error detection. The command system for the Saturn V was located in the Instrument Unit, the ring containing most of the electronic systems that was mounted at the top of the rocket, below the Lunar Module. The command system is described in Astrionics System Handbook section 6.2.

    The Saturn Command Decoder. From Saturn IB/V Instrument Unit System Description and Component Data.

    The Saturn Command Decoder. From Saturn IB/V Instrument Unit System Description and Component Data.

    The Lunar Module also had an Up-Data system, called the Digital Up-link Assembly (DUA) and built with integrated circuits. The Digital Up-link Assembly was similar to the Command Module's Up-Data Link and allowed ground stations to control the Lunar Guidance Computer. The DUA also controlled relays to arm the ascent engine. The DUA messages consisted of three vehicle address bits, three system address bits, and 16 information bits. Unlike the Command Module's UDL, the DUA includes the 70-kHz discriminator to demodulate the sub-band. The DUA also provided a redundant up-link voice path, using the data subcarrier to transmit audio. (The Command Module had a similar redundant voice path, but the demodulation was performed in the Premodulation Processor.) The DUA was based on the Digital-Command Assembly (DCA) that received up-link commands on the development vehicles. See Lunar Module Communication System and LM10 Handbook 2.7.4.2.2. 

  4. Unexpectedly, we found three different sets of sub-bit codes in different documents. The Telecommunications Study Guide says that the first digit (the Vehicle Address) encodes a one bit with the sub-bits 11011; for the remaining digits, a one bit is encoded by 10101. Apollo Digital Command System says that the first digit uses 11001 and the remainder use 10001. The schematic in Apollo Digital Up-Data Link Description shows that the first digit uses 11000 and the remainder use 01011. This encoding matches our Up-Data Link and the Test Set, although the Test Set flipped the phase in the PSK signal. (In all cases, a zero bit is encoded by inverting all five sub-bits.) 

  5. To provide range safety if the rocket went off course, the Saturn V booster had a destruct system. This system used detonating fuses along the RP-1 and LOX tanks to split the tanks open. As this happened, the escape tower at the top of the rocket would pull the astronauts to safety, away from the booster. The destruct system was controlled by the Digital Range Safety Command System (DRSCS), which used a cryptographic plug to prevent a malevolent actor from blowing up the rocket.

    The DRSCS—used on both the Saturn and Skylab programs—received a message consisting of a 9-character "Address" word and a 2-character "Command" word. Each character was composed of two audio-frequency tones from an "alphabet" of seven tones, reminiscent of the Dual-Tone Multi-Frequency (DTMF) signals used by Touch-Tone phones. The commands could arm the destruct circuitry, shut off propellants, disperse propellants, or switch the DRSCS off.

    To make this system secure, a "code plug" was carefully installed in the rocket shortly before launch. This code plug provided the "key-of-the-day" by shuffling the mapping between tone pairs and characters. With 21 characters, there were 21! (factorial) possible keys, so the chances of spoofing a message were astronomically small. Moreover, as the System Handbook writes with understatement: "Much attention has been given to preventing execution of a catastrophic command should one component fail during flight."

    For details of the range safety system, see Saturn Launch Vehicle Systems Handbook, Astrionics System Handbook (schematic in section 6.3), Apollo Spacecraft & Saturn V Launch Vehicle Pyrotechnics / Explosive Devices, The Evolution of Electronic Tracking, Optical, Telemetry, and Command Systems at the Kennedy Space Center, and Saturn V Stage I (S-IC) Overview

  6. I explained above how the Up-Data Link message was encoded into an audio signal using phase-shift keying. However, more steps were required before this signal could be transmitted over Apollo's complicated S-band radio system. Rather than using a separate communication link for each subsystem, Apollo unified most communication over a high-frequency S-band link, calling this the "Unified S-Band". Apollo had many communication streams—voice, control data, scientific data, ranging, telemetry, television—so cramming them onto a single radio link required multiple layers of modulation, like nested Russian Matryoshka dolls with a message inside.

    For the Up-Data Link, the analog PSK signal was modulated onto a subcarrier using frequency modulation. It was combined with the voice signal from ground and the pseudo-random ranging signal, and the combined signal was phase-modulated at 2106.40625 MHz and transmitted to the spacecraft through an enormous dish antenna at a ground station.

    The spectrum of the S-band signal to the Command Module. The Up-Data is transmitted on the 70 kHz subcarrier. Note the very wide spectrum of the pseudo-random ranging signal.

    The spectrum of the S-band signal to the Command Module. The Up-Data is transmitted on the 70 kHz subcarrier. Note the very wide spectrum of the pseudo-random ranging signal.

    Thus, the initial message was wrapped in several layers of modulation before transmission: the binary message was expanded to five times its length by the sub-bits, modulated with Phase-Shift Keying, modulated with frequency modulation, and modulated with phase modulation.

    On the spacecraft, the signal went through corresponding layers of demodulation to extract the message. A box called the Unified S-band Transceiver demodulated the phase-modulated signal and sent the data and voice signals to the pre-modulation processor (PMP). The PMP split out the voice and data subcarriers and demodulated the signals with FM discriminators. It sent the data signal (now a 2-kHz audio signal) to the Up-Data Link, where a phase-shift keying demodulator produced a binary output. Finally, each group of five sub-bits was converted to a single bit, revealing the message. 

  7. The Test Set uses eight-bit paper tape, but the encoding is unusual. Each character of the paper tape consists of a three-bit octal digit, the same digit inverted, and two control bits. Because of this redundancy, the Test Set could detect errors while reading the tape.

    One puzzling aspect of the paper tape reader was that we got it working, but when we tilted the Test Set on its side, the reader completely stopped working. It turned out that the reader's motor was controlled by a mercury-wetted relay, a high-current relay that uses mercury for the switch. Since mercury is a liquid, the relay would only work in the proper orientation; when we tilted the box, the mercury rolled away from the contacts. 

  8. This view of the Test Set from the top shows the positions of the 25 circuit boards, A1 through A25. Most of the boards are mounted in pairs, although A1, A2, and A15 are mounted singly. Because boards A1 and A11 have larger components, they have empty slots next to them; these are not missing boards. Each board unit has two ejector levers to remove it, along with two metal tabs to lock the unit into position. The 15 numbered holes allow access to the test points for each board. (I don't know the meaning of the text "CTS" on each board unit.) The thirteen digit display modules are at the bottom, with their dropping resistors at the bottom right.

    Top view of the Test Set.

    Top view of the Test Set.

     

  9. There are seven driver boards: A3 through A9. Board A3 is different from the others because it implements one digit instead of two. Instead, board A3 includes validation logic for the paper tape data. 

  10. Here is the datasheet for the digit displays in the Test Set: "Numerik Indicator IND-0300". In current dollars, they cost over $200 each! The cutaway diagram shows how the bent plastic sheets are stacked and illuminated.

    Datasheet from General Radio Catalog, 1963.

    Datasheet from General Radio Catalog, 1963.

    For amazing photos that show the internal structure of the displays, see this article. Fran Blanche's video discusses a similar display. Wikipedia has a page on lightguide displays.

    While restoring the Test Set, we discovered that a few of the light bulbs were burnt out. Since displaying an octal digit only uses eight of the ten bulbs, we figured that we could swap the failed bulbs with unused bulbs from "8" or "9". It turned out that we weren't the first people to think of this—many of the "unused" bulbs were burnt out. 

  11. I'll give more details on the count-to-five ring counter. The first flip-flop gets its J input from the Q' output of the last flip-flop as expected, but it gets its K input from the Q output of the second flip-flop, not the last flip-flop. If you examine the states, this causes the transition from 110 to 011 (a toggle instead of a set to 111), resulting in five states instead of six. 

  12. To explain the phase-shift keying circuitry in a bit more detail, board A1 produces a 4 kHz clock signal. Board A2 divides the clock, producing a 2 kHz signal and a 1 kHz signal. The 2 kHz signal is fed into the transformer to be phase-shifted. Then the 1 kHz reference signal is mixed in to form the PSK output. Resonant filters on board A1 convert the square-wave clock signals to smooth sine waves. 

  13. I was surprised to find LED opto-isolators in a device from the mid-1960s. I expected that the Test Set isolator used a light bulb, but testing showed that it switches on at 550 mV (like a diode) and operated successfully at over 100 kHz, impossible with a light bulb or photoresistor. It turns out that Texas Instruments filed a patent for an LED-based opto-isolator in 1963 and turned this into a product in 1964. The "PEX 3002" used a gallium-arsenide LED and a silicon phototransistor. Strangely, TI called this product a "molecular multiplex switch/chopper". Nowadays, an opto-isolator costs pennies, but at the time, these devices were absurdly expensive: TI's device sold for $275 (almost $3000 in current dollars). For more, see The Optical Link: A New Circuit Tool, 1965. 

  14. For more information on the Burr-Brown 1506 op amp module, see Burr-Brown Handbook of Operational Amplifier RC Networks. Other documents are Burr-Brown Handbook of Operational Amplifier Applications, Op-Amp History, Operational Amplifier Milestones, and an ad for the Burr-Brown 130 op amp. 

  15. I'm not sure of the meaning of the Direct versus Remote verification codes. The Block I (earlier) UDL had an 8-bit code, while the Block II (flight) UDL had a 4-bit code. The Direct code presumably comes from the UDL itself, while the Remote code is perhaps supplied through telemetry? 

  16. The block diagram below shows the structure of the Up-Data Link (UDL). It uses the sub-bit decoder and a 24-stage register to deserialize the message. Based on the message, the UDL triggers relays (RTC), outputs data to the Apollo Guidance Computer (called the CMC, Command Module Computer here), sends pulses to the CTE clock, or sends validity signals back to Earth.

    UDL block diagram, from Apollo Operations Handbook, page 31

    UDL block diagram, from Apollo Operations Handbook, page 31

    For details of the Apollo Up-Data system, see the diagram below (click it for a very large image). This diagram is from the Command/Service Module Systems Handbook (PDF page 64); see page 80 for written specifications of the UDL.

    This diagram of the Apollo Updata system specifies the message formats, relay usages, and internal structure of the UDL.

    This diagram of the Apollo Updata system specifies the message formats, relay usages, and internal structure of the UDL.

    Other important sources of information: Apollo Digital Up-Data Link Description contains schematics and a detailed description of the UDL. Telecommunication Systems Study Guide describes the earlier UDL that included a 450 MHz FM receiver. 

  17. The following CuriousMarc videos describe the Up-Data Link and the Test Set, so smash that Like button and subscribe :-)

     

Reverse-engineering the Apollo spacecraft's FM radio

How did NASA communicate with the Apollo astronauts, hundreds of thousands of miles from Earth? The premodulation processor1 (below) was the heart of the communication system onboard the Apollo spacecraft. Its multiple functions included an FM radio for communication to the astronauts, implemented by the Voice Detector, the module second from the top. In this blog post, I reverse-engineer the circuitry for that module and explain how it worked.

With the case of the premodulation processor removed, its internal modules are visible, as well as the wiring harnesses.

With the case of the premodulation processor removed, its internal modules are visible, as well as the wiring harnesses.

The Apollo communication system was complex and full of redundancy. Most communication took place over a high-frequency radio link that supported audio, telemetry, scientific data, and television images.2 NASA's massive 85-foot dish antennas transmitted signals to the spacecraft at 2106.4 megahertz, an S-band frequency, giving the system the name "Unified S-Band". These radio signals were encoded using phase modulation;3 onboard the spacecraft, a complex box called the transponder received the S-band signal and demodulated it.4

The voice and data signals from Earth were combined through a second layer of modulation: voice was frequency-modulated (FM) onto a 30-kilohertz subcarrier while data was on a 70-kilohertz subcarrier, so the two signals wouldn't conflict.5 One of the tasks of the premodulation processor was to extract the voice and data signals from the transponder's output. These voice signals went to yet another box, the Audio Center Equipment, so the astronauts could hear the messages from the ground. The data signals were decoded by the Up-Data Link, allowing NASA to send commands to the Apollo Guidance Computer, control onboard relays, or set the spacecraft's clock.

Many systems worked together for communication, but I'm focusing on a single module: the voice detector inside the premodulation processor that performed the FM demodulation. The block diagram below shows the operation of the voice detector; I've grayed out the data detector.6 The input contains the voice signal and the data signal at different frequencies; a band-pass filter (green) separates out the voice signal at 30 kilohertz. Next, the blue triangle7 demodulates the FM signal using a "clipper discriminator" circuit. The cyan triangle is an amplifier, producing the "up voice" output signal (red), so-called because it had been transmitted "up" from a ground station. I'll explain this circuitry in detail below.

Block diagram of the data and voice detectors, with the data detector grayed out. Each "Q" indicates a transistor in the circuit. Click this image (or any other) for a larger version. Based on Command/Service Module Systems Handbook p63.

Block diagram of the data and voice detectors, with the data detector grayed out. Each "Q" indicates a transistor in the circuit. Click this image (or any other) for a larger version. Based on Command/Service Module Systems Handbook p63.

The photo below shows the premodulation processor in its case.1 The premodulation processor (PMP) weighs 14.5 pounds and measures 4.7"×6"×10.5". It used 8.5 watts of power, supplied at 28 volts DC from the spacecraft's hydrogen/oxygen fuel cells or silver-oxide zinc batteries. The PMP was mounted in the Command Module's equipment bay, along with most of the electronic equipment.8 It was fastened to a "cold plate", cooled by water-glycol loops that removed heat through radiators and water evaporators.

The premodulation processor is a bluish box with four round connectors on top.

The premodulation processor is a bluish box with four round connectors on top.

Construction

The modules in the premodulation processor don't use printed-circuit boards, but instead are built from components that are soldered to metal pegs, forming a messy jumble of wiring. The circular transistors are mounted upside down with color-coded wiring: yellow for the emitter, green for the base, and blue for the collector. Capacitors are silver cylinders or gray squares, while the orange striped cylinder is a diode. The resistors have colored stripes, indicating their values. Point-to-point wiring provided additional connections, a mixture of color-coded insulated wires, bare wires, and wires in clear sleeves.

A closeup of the wiring in the premodulation processor. These connections are soldered, but others are spot-welded.

A closeup of the wiring in the premodulation processor. These connections are soldered, but others are spot-welded.

Since the components and wiring are visible, it seemed like these modules should be easy to reverse-engineer, but it's trickier than it seems. The components are liberally covered in what looks like hot glue but is probably silicone. (I suspect that this was only used in equipment for ground testing, while modules for spaceflight were fully encapsulated to prevent short circuits.) Much of the wiring is obscured, so I had to beep out many of the connections with a multimeter. As a result, my reverse-engineering probably has a few errors.

The modules have circuitry on both sides, which increased the density. The photo below shows the top side of the module. This module includes a few larger components that are mounted directly to the chassis, rather than to the circuit board. The large metal box at the top is the bandpass filter, built by Bulova Electronics, a division of the watch company that produced quartz crystals, oscillators, filters, servo amplifiers, and other components. This bandpass filter was built for Collins Radio, the manufacturer of the premodulation processor, and presumably contains a quartz filter, selecting the voice sub-band at 30 kilohertz and rejecting other frequencies. At the lower right is a smaller black box, an electromechanical relay that switched signals. The two grayish boxes are audio transformers to couple the module's output signals. The connector at the left has its wiring completely covered in silicone, inconvenient for reverse engineering.

Top view of the voice detector module.

Top view of the voice detector module.

The circuitry on the bottom side of the board is arranged in orderly columns, unlike the other side. I don't know why the design styles of the two sides are so different. However, in a few places they put components under other components, so the circuitry isn't as orderly as it appears. The back of the bandpass filter is visible at the bottom. The two sides of the module are connected by a few gray wires.

Bottom view of the voice detector module.

Bottom view of the voice detector module.

How the FM demodulator works

FM radio has a tragic history. It was invented in 1933 by Edwin Armstrong, a prolific inventor of radio technologies (no relation to Neil Armstrong, first to walk on the Moon). FM was a superior alternative to AM (amplitude modulation), an earlier radio transmission system. Unfortunately, RCA (Radio Corporation of America) had invested heavily in AM radio and lobbied to block the introduction of FM. Armstrong spent years battling RCA in court with little success, resulting in his suicide in 1954. Almost a year later, his wife obtained a million-dollar settlement from RCA (about $8 million in current dollars), followed by successful patent litigation leading to recognition of Armstrong as the inventor of FM. Eventually, in the 1960s, FM radio achieved commercial success, as well as its use in the space program.

In an FM signal, the frequency of a carrier signal is changed (modulated) depending on an input signal. That is, the varying frequency of the transmitted signal indicates the level of the input. An FM demodulation circuit undoes this process, converting a varying frequency input into the corresponding voltage to recover the original signal. Many FM receiver designs have been used with tradeoffs of linearity, noise rejection, and circuit complexity.9 The simple but inaccurate slope detector uses a high pass filter to produce more output at higher frequencies. Several techniques use a circuit tuned to the carrier frequency so the output increases with frequency deviation: the vintage Foster-Seely discriminator dating back to 1936, the ratio detector, and the simple and popular quadrature detector. The complex phase-locked loop (PLL) approach keeps an oscillator locked onto the input frequency while producing the corresponding voltage.

The premodulation processor, however, used a pulse-averaging discriminator, a high-quality but expensive demodulator used in wideband applications such as telemetry. The diagram below illustrates the FM modulation and demodulation process. The red line at the top shows an audio input signal, which modulates the purple signal: when the input is higher, the purple signal has a higher frequency. The FM purple signal is transmitted to the spacecraft. To demodulate the signal, the premodulation processor first amplifies and clips the signal (green). Next, it produces short, fixed-width pulses (gray), triggered by each green pulse; as the input frequency increases, these pulses will be closer together. Applying a low-pass filter smooths out the pulses, resulting in a higher output level when the pulses are close together.10 The result (red, bottom) matches the input.

The FM signal at various stages of processing: input, FM-modulated signal, clipped signal, fixed-width pulses, and output.

The FM signal at various stages of processing: input, FM-modulated signal, clipped signal, fixed-width pulses, and output.

Looking at the premodulation processor's circuitry, the first step is to amplify and clip the input signal, turning the input into square waves. This step removes any variations in the input signal level, reduces the effect of noise, and creates a clean signal for the next phase. Clipping is done with a pair of diodes. A diode will turn on at about 0.6 volts, so the result is that the signal is limited to -0.6 volts to 0.6 volts. You can think of clipping as cutting the peaks off the sine waves and amplifying, so you end up with sharp transitions rather than a smooth wave.11 The schematic below shows one of the two clipping stages. The transistor amplifies the input, using a basic NPN transistor amplifier circuit. The two diodes in the middle clip the signal. The capacitors block the DC component of the signal, ensuring that it is centered around 0 for symmetrical clipping.

Schematic of one stage of the clipper.

Schematic of one stage of the clipper.

The clipper is followed by a two-transistor pulse generator, a "single-shot monostable multivibrator". Each input pulse discharges its capacitor, which then recharges through a resistor. This resistor-capacitor delay creates a fixed-width pulse, and then the circuit waits for the next input pulse. The next stage is a two-transistor low-pass filter that turns the pulses into a smooth output, using a handful of capacitors. It is followed by a transistor amplifier (that can be turned off as needed). This feeds two audio transformers to produce the voice outputs that go to the Audio Center, and thus the astronauts.

More features

In the basic configuration, the voice detector extracts the voice signal, while the data detector extracts the data signal, using a similar circuit. However, in case the voice communication circuitry failed, the system provided an "up voice backup" circuitry, a redundant way for ground stations to send voice to the spacecraft. The backup path transmitted voice over the data sub-band, and the data detector performed the FM demodulation. By flipping a switch, the data detector's output was routed to the voice circuitry, providing a voice path for emergencies. (Backup voice was completely analog, even though it used the data detector module.) During the Apollo 13 incident, the astronauts used backup voice to conserve electricity, which was running critically low.12

If voice communication failed entirely, the astronauts could switch to "emergency key" mode and transmit Morse code using the push-to-talk button on their umbilical cable. Most of the emergency key circuitry is elsewhere, but the voice detector has an input for the emergency key tone to get mixed into the audio that the astronauts heard.

The XMIT button on the astronaut's umbilical could be used as a key to transmit Morse code in an emergency.
From Apollo operations handbook.

The XMIT button on the astronaut's umbilical could be used as a key to transmit Morse code in an emergency. From Apollo operations handbook.

The communication system also included "squelch", a feature that silenced the audio if the signal strength dropped too low. ("Squelch" is a curious word, an onomatopeic 17th-century word meaning "to crush", later used figuratively as "suppress", and then in the 1930s as a radio circuit to suppress noise.) The S-band radio originally didn't include squelch but NASA soon found that a loss of the carrier signal created high noise levels that could disrupt other audio channels. To avoid this problem, a squelch feature was added to the radio before the Moon landings.

The squelch circuit detected the level of the carrier signal much like an AM radio, using a diode to rectify the sine wave and track the peaks. It used two transistors to amplify the signal and a third transistor to disable the audio, triggering squelch if the carrier level fell too low. A squelch disable switch was provided to ensure that, if necessary, voice could be used even at low signal levels. Moreover, some astronauts liked disabling squelch so they could use the channel noise to determine the channel's status.

Another important feature for redundancy was relay support. If, for instance, the S-band radio failed on the Lunar Module, the Command Module could relay communication to and from the ground, using the VHF radio, as shown below. The circuit to relay communication from the ground uses a clever implementation trick. By flipping a switch, the ground up-voice signal replaced the Command Module pilot's microphone (#2, center seat), so ground communication could be transmitted just like the astronaut's speech, sending it over the VHF radio to the Lunar Module, for instance. The diagram below illustrates two scenarios: from ground to an extra-vehicular activity or from ground to the Lunar Module, relayed through the Command Module.

Illustrations of how relay mode works from the ground (MSFN, Manned Space Flight Network) to an extra-vehicular activity (EVA), as well as to the Lunar Module (LM). Adapted from Apollo CSM Logistics Training.

Illustrations of how relay mode works from the ground (MSFN, Manned Space Flight Network) to an extra-vehicular activity (EVA), as well as to the Lunar Module (LM). Adapted from Apollo CSM Logistics Training.

The voice relay circuit was implemented with an electromechanical relay in the voice decoder module—don't be confused with the two completely different meanings of "relay" in this system. Flipping a switch caused the electromechanical relay to replace the microphone signal with the up voice signal.13

The astronauts sat in front of a complex control panel full of switches and gauges. The controls for the premodulation processor were grouped in the lower-right corner of the console with other communications switches. The diagram below shows the switches that controlled the voice detector features: squelch, backup voice, and voice relay, highlighted in yellow.

The Command Module's control panel with relevant switches highlighted.
  Diagram based on from Command/Service Module Systems Handbook p208.

The Command Module's control panel with relevant switches highlighted. Diagram based on from Command/Service Module Systems Handbook p208.

Schematic

The detailed block diagram shows the construction of the voice detector and data detector modules. Each triangle corresponds to a transistor. I've grayed out the data detector and colored external switching circuitry in blue; these switches match the ones above. You can see how the backup up-voice comes from the data detector module, and then is merged into the voice detector's output, under the control of the "data / up voice BU" switch. At the bottom, the relay switches the voice signal in place of the microphone #2 signal when relaying voice from Earth. ("AC" is the Audio Console, the audio system connected to the astronaut's headphones and microphones.)

Detailed block diagram of the voice detector and data detector modules, data detector grayed out. Based on Apollo Telecommunication System training.

Detailed block diagram of the voice detector and data detector modules, data detector grayed out. Based on Apollo Telecommunication System training.

After tracing out the module's circuitry, I generated the schematic below. You can match the schematic against the block diagram to see how the functional blocks are implemented,14 using relatively simple circuits with one or two transistors per function.

My reverse-engineered schematic of the voice detector module. Expect a few errors. Click for a larger version.

My reverse-engineered schematic of the voice detector module. Expect a few errors. Click for a larger version.

The photos below show how the circuitry maps onto the physical layout of the boards in the voice detector module. Signal processing starts on the right with the FM circuitry (clippers and pulse generator), and the squelch circuit. The low-pass filter and output circuitry is on the left board.

The voice detector with the main functional blocks labeled.

The voice detector with the main functional blocks labeled.

Conclusion

From the outside, the premodulation processor is a mysterious blue box. Opening it up reveals relatively straightforward transistor circuits, implemented with a surprisingly haphazard construction technique.

Although I reverse-engineered this module partly from curiosity, the main motivation was to uncover a pin that was missing from our documentation, specifically the pin to control squelch, missing since squelch was added relatively late in the design. We plan to wire up the premodulation processor, using an elaborate "breakout board" that Eric is designing. We can then use the premodulation processor as it would have operated during a mission, hooking it up to the transponder and giving it radio signals. I announce my latest blog posts on Twitter, so follow me @kenshirriff for updates. I also have an RSS feed.

For an overview of the premodulation processor, see my previous blog post. Also see Curious Marc's video where the premodulation processor is disassembled (below). Thanks to Mike Stewart, Curious Marc, and Eric Schlaepfer for their roles in the premodulation processor investigation. Thanks to Marcel for providing the premodulation processor.

Notes and references

  1. For detailed specifications of the premodulation processor, see Command/Service Module Systems Handbook p73. 

  2. The design standard for the Apollo audio system was 90% word intelligibility for the main links and 70% for the backup links. This standard seems surprisingly poor, with one out of 10 words unintelligible, but achieving this standard was challenging due to the extreme distance to the Moon. For detailed information on the voice communication system, see Apollo Experience Report - Voice Communications Techniques and Performances. It discusses the performance requirements for the Apollo communications system and how the system was designed to achieve the intelligibility requirements. 

  3. Phase modulation (PM) varies the phase of the carrier signal, rather than varying the frequency as in frequency modulation (FM). The techniques are very similar since increasing the phase compresses the waveform, increasing the frequency. Specifically, phase modulation of an input is the same as frequency modulation of the input's derivative. Apollo used phase modulation for the overall signal because it keeps the frequency (mostly) constant so doppler ranging could be used to measure the spacecraft's speed. 

  4. The transponder got its name because it also sent the signals back to Earth after shifting the frequency, so the distance to the spacecraft could be accurately determined; see my discussion here

  5. The data signal from Earth had a third layer of modulation: the binary data was modulated with phase-shift keying at 2 kilohertz to produce an audio signal for transmission. Another box, the Up-Data Link demodulated and decoded this signal after the premodulation processor had demodulated the FM layer. I have another blog post that describes this. 

  6. I'm not covering the data detector in this blog post, but since it's so closely tied to the voice detector, I'll give an overview. Its circuitry is similar to the voice detector, but simpler, since it doesn't have squelch or the relay. It has a similar bandpass filter module, but at 70 kilohertz rather than 30 kilohertz, reflecting the data subcarrier frequency.

    The data detector module.

    The data detector module.

     

  7. In case anyone is studying the block diagram carefully, I'll explain the labels such as "Q1-6V". This indicates transistors 1 through 6 in the voice module. "Q8D", on the other hand, indicates transistor 8 in the data module. 

  8. The premodulation processor was one of many boxes of electronic circuitry packed into the spacecraft and linked by thick cables. The diagram below highlights where it was mounted in the lower equipment bay of the Apollo Command Module.

    The premodulation processor was one of many electronic boxes in the Command Module's lower equipment bay. Diagram from Command/Service Module Systems Handbook p212.

    The premodulation processor was one of many electronic boxes in the Command Module's lower equipment bay. Diagram from Command/Service Module Systems Handbook p212.

     

  9. Various FM detector circuits are described here

  10. The technique of using varying digital pulses to generate an analog signal is similar to the PWM (pulse-width modulation) technique used for analog outputs on the Arduino. The difference is that the Arduino uses pulses with a fixed frequency and varying width, while the FM discriminator uses pulses with a varying frequency and fixed width. 

  11. The clipping process preserves the "zero-crossings", the points where the waveform's voltage crosses zero. This throws away amplitude fluctuations and most of the noise that may be in the signal. 

  12. The idea of backup voice was to provide a voice channel for emergencies that used less power, at the cost of garbling up to 30% of the words. After the explosion, Apollo 13 used the backup voice system so they could turn off the Lunar Module's power amplifier and conserve electrical power. (See Apollo 13 Mission Operations Report pages N-2 and N-7, as well as the transcript.) Backup voice was also used at times during Apollo 16 due to a failure of the Lunar Module's steerable S-band antenna; see Apollo 16 Mission Report page 7-3, which calls this mode "down voice backup". (I should point out that these backup voice incidents involved the Lunar Module, so the Command Module's premodulation processor didn't take part.) 

  13. The relay circuitry was a bit more complicated than I expected. Its main task is to switch between the microphone input and the voice signal. However, it also switches a 50Ω resistor across the transformer if the voice signal is not used, presumably so the impedance remains unchanged and Audio Console level doesn't jump. In other words, the resistor gives the unused voice signal somewhere to go. 

  14. The major difference between the block diagram and my schematic is that the block diagram shows the transformers connected to ground, while I found that they are connected to +18V.