How are the exact symbols (C C# D D# E F F# G G# A A# B) of 12 tone equal temperament with octave (2x) and (3/2) found?
What is the math or formula used to find those specific symbols?
Imagine a tuning that is equal temperament but instead of 12 tones has 3 tones, it still has octave but has 1.618 instead of the usual (3/2), what would be the symbols of this tuning?
You seem to have several confusions.
First of all, 12-tone equal temperament does not, and cannot have perfect fifths defined as exactly 3/2. It is impossible by definition. In 12TET, all half steps must be the same size, and there must be twelve of them in an octave. This means that whatever factor (x) you multiply a frequency by to raise it a half step must equal exactly 2 when you multiply it 12 times. Or in other words: x12 = 2, which means that x = 21/12.
In 12TET, a perfect fifth is defined as exactly seven of these half steps, so it can be found by multiplying the lower frequency by 27/12 (~1.49830707). You'll note that this is an irrational number, and specifically, is not equal to 3/2 (=1.50).
Secondly, unlike the frequencies, the letter symbols are not exactly found by any kind of formula, much less one based on 12TET (the letters predate the adoption of 12TET). Rather, they are simply the result of western music being based on a seven note scale, and naming those seven notes after the first 7 letters of the alphabet. See: Why are notes named the way they are?
That said, there is a pattern to the letters, if you go by the Line of Fifths (illustrated here), in that the letters always repeat in the same order when you go by fifths (F, C, G, D, A, E, B). Continuing the pater, these seven notes are followed by seven sharp notes with the same letter order (F♯, C♯,...), and those are followed by double-sharps in the same letter order. Working backwards, F is preceded by flats in the same order as well (reversed since we're going backwards, B♭, E♭,...), and these are preceded by double-flats, once again, with the same letter order.
Thirdly, if you come up with some kind of scale of your own (like three equally spaced notes), you really shouldn't redefine what the letters mean (like naively naming them A, B, C, or something). That will cause needless confusion. You should just call them by the nearest actual pitch. As Bob mentioned in a comment, in the case of three equally-spaced notes, they will just be major thirds apart. Exactly which major thirds will depend on which note you start on.
You already know some of the ratios. An octave is 2:1. A just/pure fifth is 3:2. (A perfect fifth may not have this exact ratio, by the way. More on that later.)
If you divide an octave into twelve equal parts, you need a number that, when multiplied by itself 12 times, equals two. That is 
. (Read it as "the twelfth root of two.") It's approximately equal to 1.059463094359295.
To determine the frequencies of equal temperament, you need a starting frequency. That is usually defined as A4 = 440 Hz, which is basically arbitrary.
To get A♯/B♭, multiply 440 by the twelfth root of two. To get B, multiply that number by the twelfth root of two. By the time you've done this 12 times, you'll reach A5, which is 880.
All symbols, including the use of letters for notes (as well as the letters themselves), are ultimately arbitrary; so the symbols of your imagined tuning would be whatever you defined them to be. The symbols used in Western music are not universal. They aren't even consistent within Western music. (What English speakers call "B" is sometimes called "H" or "Si" in other Western countries, for example.) The reason why there are twelve notes is explained here: What is the theory behind scales?
A perfect fifth and a just/pure fifth are not necessarily the same, and, in equal temperament, they never are. The perfect fifth of A is E. Following the ratio of 3:2, you'd expect to find E at 660 Hz. That is a pure fifth. But, in equal temperament, E ≈ 659.25511382574. It's still a perfect fifth—because it's E—but it is no longer pure/just. Other temperaments have perfect fifths which are pure and others which are not, or fifths which are sometimes closer to pure/just than others. In equal temperament, the fifths are all the same distance apart; hence "equal".
In Pythagorean tuning, the pitches are generated by compounding perfect (just intonated) fifths, one way to do this is to go symmetrically outward from a central pitch, octave reducing.
Note that Gf, Fs generated this way are very close in pitch (enharmonic). The reason why is happens is a numerical coincidence (3/2)**12 is approximately equal to 2**7, i.e. going out enough fifths (approximately) gets you some number of octaves above your starting pitch. This makes 12 notes a "natural" stopping point in considering the spine of fifths. Also note that the 5-note pentatonic and 7 note diatonic (B-flat major in this case) scales, appear in the sequence of scales generated in this way.
In many cases one says Gf and Fs are "close enough" to be considered equivalent, one of them is dropped, and you have a 12 note chromatic scale. Note that one can start at a different starting point (e.g. start on B) and get a 12 note scale where some of the notes are replaced with their enharmonic equivalents.
If you generate 12 tones by extending along (or outwards) the spine of fifths, you can generate 12 notes that might serve as a workable scale (I'm not sure to what extent pure Pythagorean tuning has been used in practice -- this theory is more applicable to 7 note diatonic scales), and the note names associated with those pitches will be in the sensible order.
Although rarely considered, one can continue extending the "line-of-fifths" to gneerate, e.g. G-sharp, or A-flat-flat and so on; it is formally it is infinite.
There is no reason why one couldn't start at a given pitch, and then grow outwards using an alternative "generator", e.g. the golden ratio as alluded to in the question to generate
and so on (I just made up symbols for these notes); aside from some numerical coincidences that one could look for, there is no reason to think that the pitches generated in this way will have any particular relationships to the pitches generated via Pythagorean tuning; this even includes the idea of getting an approximate octave-equivalence that would allow one to close the "circle-of-phi" and thus have a workable number of scale degrees.
The following Python script demonstrates how to generate a table of relative pitches given a single generating interval.
import math
def octave_reduce(f):
while f > 2:
f=f/2
while f < 0.5:
f=f*2
return f
p_steps=xrange(12) # generate a 12 note scale
p_basis=3.0/2.0 # pythagorean is generated by compounding just intonated fifthe
p_frequencies=[ octave_reduce( p_basis**p ) for p in p_steps]
p_note_names=['Df', 'Af','Ef','Bf','F','C','G','D','A','E','B','Fs']
# output the Pythagorean scale in of pitch, which is just their scale order
print [x[1] for x in sorted( zip(p_frequencies, p_names))]
# one could make a golden ratio based scale
# in essentially the same way:
g_basis=0.5*(1+math.sqrt(5))
g_frequencies=[ octave_reduce( g_basis**p) for p in p_steps]
