Telex is still alive and well
I-Telex - Internet Connected Teleprinter Network.
www.i-telex.net and
www.telexforum.de
Please do not be put off by the German language - a lot of members speak very good English and are very helpful.
Going back to the original question, the audio in will be expecting AFSK (Audio Frequency Shift Keying)
The output of a soundcard RTTY program will work, but you will need to faff around with the audio levels, and possible matching, though this is probably not critical.
The line out will go to your soundcard input - again levels will need adjusting.
Beware ground loops, and potential oddball things like the radio ground not actually being at ground at all.....