Connecting with the Internet

by:

Paul Lynch
P & L Systems
paul@plsys.co.uk
paul@seer.demon.co.uk
+44 1494 432422
+44 1494 432478 (fax)

I have just recently (June 1992) been through the exercise of hooking up my NeXT at home to a full Internet feed, and while it is still fresh in my mind, here is how I did it.

My system is a standalone NeXT, with a data modem. The instructions given here apply only to similar configurations, although sections are included which deal with other common configurations. In particular, SLIP_920904A only runs on black NeXTs. Another free SLIP (PNI), from the same author, is also available, which runs on black and Intel NeXTs, but the setup instructions are slightly different (I will update this accordingly at some point). PPP2.2 is also attractive to use as an Internet package. Both of the latter lack d idle timeout and dial on demand.

You will need before you start:

later on you will need the following, and it may be more convenient to get them through your Internet service when you are operational:

and for news:

But first of all, why bother with the Internet?

Most of us have some kind of on-line system that we use to handle our mail, and maybe cover some useful files and chat. Some of us might have a uucp feed to get Usenet news and mail. What else does the Internet have to offer? The Internet proper ("Internet" is often used incorrectly to refer to mail and Usenet) is a high speed network running (mostly) on Unix systems around the world. These systems that form the Internet run TCP/IP networking between sites, which allows the standard TCP/IP tools of telnet, ftp, rlogin and rcp to be used, as well as a number of specialised tools that have evolved around the Internet; a few of them are listed above. Any user of any Internet machine can login at any other machine on the Internet, and can copy files. Most systems have a special area for public files, and allow Internet users to login to access those files. There are quite a few Internet machines that offer interactive game playing, information services and on-line chat.

Once you have a Internet connection you can use all of this.

I was attracted to a new service set up in the UK, where Usenet news is censored by the main hubs. This service offered a dial up connection to the Internet, with full, uncensored Usenet news, mail, and ftp, all for £10 per month (about $15), plus your phone bill. No other charges.

First Steps

This service is called Demon Internet Services, also known as "tenner a month", for obvious reasons. They can be reached (by voice) on 0181-371-1010, which is the Internet support line, or on 0181-371-1234 (office hours only). For people with Internet mail access already, they can be reached as internet@demon.net.

From DIS I was given a phone number to dial (0181-343-4848), a site name (seer.demon.co.uk), and the Internet name (demon-du.demon.co.uk) and Internet address (158.152.1.222) of their gateway host. I was able to suggest the site name.

The first step was to dial in, using tip (which comes with your machine. This requires a custom remote setup (see below). I saw a standard Unix 'login:' prompt, to which I gave my site name (seer). Then I had to set a password. Once I was logged in, it told me my Internet node number (158.152.8.45). Nothing else to do, just hang-up.

One word of warning: Demon requires 8 bit clean input; if you use tip from a NeXT, you won't be able to login without some changes. In order to use tip, you will need to edit the /etc/remote file to include the keyword pa=none. For example:

demon|Dial-out on cua to Demon:\
	:dv=/dev/cua:br#9600:el=^C^S^Q^U^D:ie=%$:oe=^D:\
	:pa=zero:

Once you have created this entry in /etc/remote, type tip demon into a Terminal window to start a connection with the modem. Then type atdt01813434848 to dial in to Demon, and follow a similar dialogue to that given below.

seer% tip demon
connected
atdt0181 343 4848
CONNECT 14400
Demon Internet Services; London Point of Presence.
Unauthorised use is prosecutable under the Computer Misuse Act.
gate login: seer
Password: <type your password>
seer: IP Address: 158.152.8.45  Running: SLIP on tty3D
*** LOWER PRICES ***
We now have stock of HST DS with FAX at #500 and the V32bis with FAX #400
For more information call the Sales Line 081-349 0063 or mail orders@demon.
1  mail messages waiting
HELLO
<type ~.>
seer% 

A new user would be invited as part of the dialog to set their new password. The point to note is the line following the password, which gives your IP address. The final line with HELLO should also be noted, just to confirm that it arrives correctly. After that you will only see line noise, which looks like random characters appearing on the screen, and you should disconnect the line straight away.

A Small Network Change

For installation run HostManager, and use the Local menu item to configure the NetInfo domain binding to 'local domain only' and give your machine a hostname, the same as your Demon id. If, for some strange reason, it won't let you set the hostname, then resort to brute force and load /etc/hostconfig in Edit (while logged in as root) and change the relevant line to HOSTNAME=seer. The IP address only needs to be set if your machine is part of a network.

SLIPing (TransSys SLIP)

I then had to get a copy of SLIP. I chose to use the free port to the NeXT written by Louis Mamakos. Instructions for installing this depend on the version; I have the 920904A release, which comes as a pkg file. This has to be installed as root; instructions are given in the Manual.wn file, from page 19 onwards. There may be more recent versions of SLIP; keep a look out. You might want to install MorningStar PPP instead (this is a commercially supported package). For the adventurous, there is a free PPP implementation for NeXTs.

Then I had to set up the SLIP configuration files in /usr/dialupip/config. This starts by copying the contents of the SAMPLES directory back into config. I made the following changes:

config.slip: enter the addresses for the SLIP0 interface (note: use your own IP address for SLIP0LOCAL)

SLIP0LOCAL=158.152.8.45
SLIP0REMOTE=158.152.1.65

diald.conf: set the login instructions

slip0:gate.demon.co.uk:cufa#38400:slip-srv.script 0181-343-4848 seer xxxxxxxxx :/usr/dialupip/log/trans@3:slipsrv.access

This involved editing the supplied config file to include the new telephone number, the gateway host address, the port and speed used, the phone number, and the login id and password (not the real one :-). Remember to remove the # comment marker from the start of the line. Once the system is working to your satisfaction, you can remove the /usr/dialupip/log/trans@3, which causes details to be logged to the trans file.

slip-srv.script.tcl: I just changed this to use the Unix login script, and refer to the correct dialer script; these lines are the last two source statements in the file, you may not need to change them:

source "/usr/dialupip/config/dial-modem.tcl"

...

source "/usr/dialupip/config/login-unix.tcl"

dial-modem.tcl: this must be changed to work with your modem. For my system, using a Miracom/US Robotics Courier V.32bis, I changed the final set timeout 30 line to say 40 (you might want to use 60 or even 90 as this delay value if you have a pulse dial exchange). For safety, you should also add a line saying: parity zero, at the start of the file. The final change is the most modem specific, and should not be used for non-Courier modems; alter the line that says: xmit {ATZ\r} to say: xmit {ATZB0&B1\r}. This line should contain any important modem initialisation commands.

login-unix.tcl: I changed this to reflect the new prompts that were being given, and to add in extra delays that were needed to synchronise the script properly. This involved adding sleep 1 before the xmit's for $userid and $passwd, and changing the connection complete string:

expect timeout {error "waiting for SLIP startup"} "*HELLO*"

You will need to add a couple of lines to allow for setting the protocol to SLIP. To accomodate this, add the following lines after xmit "$passwd\r":

expect timeout {error "waiting for protocol"} "*rotocol:*"
sleep 1
xmit "SLIP\r"

slipsrv.access: there is one vital change to be made in here. Once you have a SLIP connection running, it will stay up until either end (you or Demon) decide to drop the line. At one time (until recently) Demon dropped the line after about four minutes of inactivity, but there are no guarantees that this will continue. The last line of this file should be updated to read:

inactivity 300

which will keep the link up for five minutes after the last packet goes through. So far, so good. Now to install SLIP and test it. Change /etc/rc.local to include

/bin/sh /usr/dialupip/config/rc.slip	>/dev/console 2>&1

at the end of the file. You must also edit /etc/syslog.conf to route SLIP messages to the appropriate file. Add a line at the end of the file:

local3.info	/usr/dialupip/log/syslog

If you need more detailed debugging information to help solve a SLIP problem, change the .info to .debug.

Now reboot, and you will have a (barely) usable SLIP system.

If you can, boot using the console, so that you get all of the boot messages. You should see a few messages about "loading kernel device driver", and "installing major device number xx". Test it by forcing a connection, by launching Terminal and typing:

/usr/dialupip/bin/duioctl slip0 BRINGUP

Then try ping 158.152.1.222. Double check with ping seer (your host name). First connections made with just a ping (without the duioctl command) are possible, but they often timeout before a connection is established. If there are any problems, look in /usr/dialupip/log to find out details (syslog and trans are the files to examine). At this stage, your system should be able to automatically login in to Demon; the modem lights will show that a connection is maintained. If you do nothing else, the modem should keep the connection for just over five minutes.

Assuming that you can ping both yourself and your gateway (use Control C to finish the ping test), the next stage is to give some names to the non-local hosts. Don't worry if ping reports that some packets were lost. Over a dial-up connection packet transmission takes time; I would expect to see one packet lost out of a batch many more times than not.

After a test, assuming that you were successful, the following command will terminate the SLIP connection:

/usr/dialupip/bin/duioctl slip0 BRINGDOWN

SLIPing (TransSys PNI)

This has a couple of limitations compared to TransSys SLIP, and one big advantage. The advantage is that it runs on NeXTSTEP/Intel as well as Motorola. The disadvantages are that it doesn't support inactivity timeout (at least, in the free version), and it doesn't support dial on demand; so you will always have to manually start the connection.

In general, configuration is very similar to TransSys SLIP. Modify the scripts in /etc/pni/support in exactly the same was as described above for dial-modem.tcl and login-unix.tcl. In addition, you will need to set up a /etc/pni/config/pni0.config file. copy /etc/pni/SAMPLES/config.slip to /etc/pni/config/pni0.config, and edit:

MODEMTYPE: modem, SERVERTYPE unix, ADDRESS: 158.152.x.x, REMOTEADDRESS: 158.152.1.222, NUMBER: 0181-343-4848; replace SECRETFILE with:

set Config(SlipServerName:USERNAME)	seer
set Config(SlipServerName:PASSWORD)	<password>

Name Servers and ftp

This can best be done by running HostManager; use the Host->New option to create the new host. Fill in Hostname and Internet Address as given; ignore the Ethernet Address and other fields.

Do the same to create an entry with your local site name and IP address; for example, seer, with an alias of seer.demon.co.uk and an IP address of 192.42.172.1 (i.e. not your Demon IP address).

You also have to create a file called /etc/resolv.conf:

nameserver 158.152.1.65
nameserver 158.152.1.193

With all of this done, you have given names to your local domain, localhost and SLIP server.

Now try:

ping gate.demon.co.uk
ping ftp.uu.net

The first will be resolved locally, the last requires the BIND name server running on the gateway host to operate. With this much functioning, you will now be able to ftp to anywhere.

Before you attempt this, please remember that the anonymous ftp service is a courtesy that costs the supplying site some effort, a lot of disk space, and a significant network and CPU load. So be equally polite when using it. Do not attempt an ftp connect at the sites local working hours, normally between 8am and 7pm. Initially, I tried 'ftp ftp.demon.co.uk', and used the anonymous ftp convention to gain access. This means that at the user prompt, you should type in 'anonymous', and for the password, type in your email address (for example, 'paul@seer.demon.co.uk'). Inside ftp, 'dir' or 'ls' will list files, 'cd xxx' will change directory into xxx, and 'bye' will terminate the connection. Use 'get filename' to transfer any files that you need.

You can now use GatorFTP or Archie (or RBrowser or YFtp or ncftp, all ftp frontends); if you don't already have this, you might try to retrieve it by ftp from ftp.cs.orst.edu.

At this level, you can also read Usenet news. Run NewsGrazer (which you can also get from orst), and configure it in Preferences to use nntp from your host (news.demon.co.uk, in my case).

This isn't the most efficient means of access; NewsGrazer will download a full list of news groups (100K or so) before giving you access, and it will retrieve individual news articles one at a time, requiring you to be on-line all of the time that you are reading news.

Mail

The next stage is to arrange for your outgoing and incoming mail queues to be handed over to the server. Using the standard sendmail configuration (where /etc/sendmail/sendmail.cf is linked to /etc/sendmail/sendmail.subsidiary.cf) you have to define a mailhost in NetInfo.

Run NetinfoManager, and create a new subdirectory of /locations. Open the new directory, and set the name property to sendmail. Create two new properties, and call them sendmail.cf and mailhost. Give a value of /etc/sendmail/sendmail.subsidiary.cf to the sendmail.cf property, and a value of post.demon.co.uk to mailhost.

Now, sending mail to a non-local address will immediately start up the SLIP connection, and transfer the mail to your server; you can change this in sendmail.cf if you wish.

Queued outgoing mail will only be sent if the SLIP connection is running when sendmail "wakes up" to look for incoming mail.

Incoming mail is sent by the mailhost (post.demon.co.uk) from their SMTP server whenever they detect that your connection is running. Keeping the connection up for a couple of minutes will accomplish this. If you are concerned that mail isn't getting through to you, try the following command: finger seer@post.demon.co.uk. This will tell you if any mail is waiting to be delivered. If there are any problems, call the Demon support line and ask for a manual delivery.

Test the mail by sending mail to yourself at another address. If this is a problem, try sending mail to yourself via the server machine; using an address like remotehost!localhost!userid (for example post.demon.co.uk!seer.demon.co.uk!paul).

You might also need to make a few changes to sendmail.cf. Do so from a copy of sendmail.subsidiary.cf, not through the link, and make sure that you update the name in NetInfoManager. Here are the changes that I made:

First of all, I changed the Dj macro to generate a proper hostname to:

Djseer.demon.co.uk

This isn't strictly neccessary for a stand-alone system, but will be for a networked system.

I also uncommented the 'copy errors to Postmaster' line (this is useful for any user):

OPPostmaster

As far as I can see, there were no other changes to sendmail, although there are some possibilities for people who want to connect with a machine that uses the sendmail.mailhost.cf configuration file. It might be worth looking in NeXTanswers, or in the Admin manual for more sendmail tricks. This is a very complex subject, and there is a lot of ill-informed twaddle going around about configuring sendmail on NeXTs. If in doubt, don't mess with it.

What NeXT?

We can now make full use of the Internet link, receive and send mail, ftp and read news. What remains to be done? The mail link isn't tidy; you want to control exactly when a link is made, and ensure that the SLIP connection is automatically started when required (this is normal), and shut down when you are done. See the section on automation later. It is possible to configure SLIP to queue up mail sent at peak times to be sent when telephone costs are lower; this is done in a similar way.

You will probably also want to install news software to arrange proper collection of interesting news groups.

You may also want to investigate irc (Internet Relay Chat), or allow incoming SLIP connections.

Usenet

If you want to handle news more efficiently, and read it on your machine while the SLIP connection is down, you will need to install cnews, and nntp. Get one of the cnews packages and install it.

Once again, installation instructions seem to vary, so I won't repeat them here. Much the same goes for nntp. As a stand alone machine, install just the client module, and also the slurp and nntpxmit add-ons.

The current nntp version is 1.5.11; I installed this with only minor changes to common/dist.h, but I also found an upgraded version of nntpxfer to use. There are a few of these available; they are all intended to support batching of incoming news (instead of processing each article as it is received, before fetching the next one).

If you use the CNews.pkg listed above, then this includes the necessary precompiled binaries for nntp; but you will still need to install slurp.

Most of what I had to do was changing the directories that are used to store various news and configuration files; my version of cnews put files into a different place from "standard" BSD, so I suspect that I had to make more changes than a normal installation might require.

If you aren't comfortable with typical Unix installations of source code applications, don't attempt this. Both cnews and nntp come as source code, and you will need to set up configuration files that require fairly detailed knowledge of the system.

Files that require local changes include: active (probably), mailname, mailpaths, organization, sys and whoami.

sys gave the biggest problems, as the comments in the file describe how to change it, and they don't match what I had to do. There should be two lines in the file; one describes the groups you receive, and is easy to set up (hint: ME:all/all:f); the other controls your outgoing posts, and was my problem. The third field describes what should be sent; I used Ln. 'L' means pass on local posts only (you would leave this out if you wanted to act as a news feed for that site), and 'n' generates a batch file with message ids in it.

An alternative to the cnews/nntp combination is INN. I am told that this is easily installed on NeXTs, but I haven't tried it.

Change NewsGrazer to use flat files: you can start it up in nntp mode and wait for ten minutes before killing it; or you can cheat and use DefaultMgr to change the NewsGrazer->NewsGrazerUseNNTP value from 0 to 1.

The next step is to create a set of directories to store news, by running /usr/local/news/bin/maint/adddirs, which creates the spool files from your active file. Also create: /usr/spool/news/in.coming, /usr/spool/news/in.coming/bad, and /usr/spool/news/out.going. To get your first batch of news, start up SLIP, and type in:

/usr/local/bin/slurp news "demon.*" 950329 000000

where demon.* requests all of the local newsgroups, and 950329 is a date a couple of days ago in YYMMDD format. You should make sure that your active file (/usr/local/news/lib/active) contains suitable entries for these groups. If you have alredy run NewsGrazer in nntp mode, it will have been automatically created. If we are successful, you will have a lot of new files appearing in /usr/spool/news/in.coming. Next, we have to load them into news, so run:

/usr/local/bin/news/input/newsrun

After a short while, start up NewsGrazer (check that the Preferences point to the right flat files), and tell it to look for new news (Tools menu). If this all works, you should create a file /usr/spool/news/nntp.news, which contains the list of groups you want to receive in the same format as the nntpxfer command line, with groups separated by commas (and no white space). The dates and times of incoming batches will be updated automatically by nntp. You can still use the command line parameters for when you want to re-request certain groups, or to fill in a new news group. A sample file follows:

demon.*,uk.*,comp.sys.next*,misc.jobs.*,news.* 920720 050623 

Note that the local news group (demon.*), and the news.* hierarchy should be considered obligatory for all Usenet followers. The final two fields are the date and time; again, choose a date a couple of days ago.

Your cnews installation documents will tell you to put some extra lines in /etc/crontab; do this, but don't change /etc/crontab; the correct file to change is /etc/crontab.local, and be prepared to change the intervals that it uses to run newsrun in particular. If you didn't have a /etc/crontab.local file initially, then you will have to reboot before any commands it contains are executed. Doing this now, even though we haven't automated the connection, allows us more flexibility for getting news from other sources in the future. This is from my /etc/crontab.local file:

0,15,30,45 * * * * 	news -c '/usr/local/bin/news/input/newsrun'
00 00 * * * 	news -c '/usr/local/bin/news/input/newsrunning on'
00 01 * * * 	news -c '/usr/local/bin/news/batch/sendbatches'
00 04 * * * 	news -c '/usr/local/bin/news/expire/doexpire'
00 05 * * * 	news -c '/usr/local/bin/news/maint/newsdaily'
30 05 * * * 	news -c '/usr/local/bin/news/maint/newswatch'

With the changes made to the cnews configuration files (sys in particular), posting should just work. First of all; check NewsGrazer's configuration Preferences are set correctly. Secondly, processing a post takes an incredibly long time (due to a cascade of scripts that are called). You should find that your post is created as a file in the appropriate news directory, and a header line is added to the out.going subdirectory, in a file called togo. To process the postings, run nntpxmit:

/usr/local/bin/nntpxmit news:/usr/spool/news/out.going/gate.demon.co.uk/togo

Test with a post in a local newsgroup (demon.test), and ask for a friendly soul to make a reply to your post. If you find that you have a double signature, then just ask NewsGrazer in Preferences to not add a signature, as inews already handles that. To have your very own signature, put it into a file in your home directory and call it .signature; a useful guideline is to use no more than four lines.

If you point the inews preference in NewsGrazer to the mini-inews that comes with nntp, then you can only make postings while on-line to DIS (but this method is very reliable). You can read news with flat files, and post through the nntp inews without any problems. If you have any difficulty getting batched postings to work, here are a few debugging tips.

Try running the posting mechanism manually. NewsGrazer uses the command:

/usr/local/bin/inews -h
Subject: Rubbish
Newsgroups: demon.test

This is a test post; please ignore
^D

Try the same manually, and see what error messages you get. The commonest will be permission and ownership problems (remember that all news files should be owned by news and belong to group news). The relaynews program called by inews may also object if your /usr/local/news/lib/bin/config file is different from the one it was compiled with.

Automation

If you aren't using a fax modem, this is reasonably straightforward. If you are, then you have to manually use PrintManager to mark the fax modem as 'unavailable' before you can bring up SLIP. This isn't too bad, partly because there aren't any fax modems fast enough for SLIP use and supported by NeXT, and partly because, if the modem is unavailable, the automated script just doesn't run. There is a program available (in the Compuserve NeXTForum libraries, and also supplied with MicroPhone Pro) that will switch your fax modem between fax and data. This can be put into /etc/crontab.local, or into your SLIP scripts. The NXFax driver handles this automatically.

There are several different levels of automation that you can aim for, and differing parts of the process to automate. The questions you need to ask are divided into four: outgoing mail, incoming mail, outgoing news posts, and incoming news.

Outgoing mail can either be sent on demand, or queued. In the config.slip file, you can postpone all outgoing mail until a connection is deliberately made by putting DISABLECALL on the SLIPxCONFIG line. If you decide that you only want dial on demand at certain times, then take a look at the slipsrv.access file. Your outgoing mail will be sent when a connection is up, and sendmail is running. As the default sendmail queue processing interval in /etc/rc is one hour, it is a good idea to include /usr/lib/sendmail -q in your script. Incoming mail is always processed whenever you make a link (see later).

Incoming and outgoing news is processed as described above.

Here is an example connection script:

#!/bin/sh
#
# script run from /etc/crontab to open a slip connection, and grab
# a batch of news.  Will also force a check for mail.  Relies on
# SLIP dropping the line after five minutes.
#
if [ "$1x" != "-fx" ] ; then
    if [ -f /usr/spool/uucp/LCK/LCK..cua -o -f /usr/spool/uucp/LCK/LCK..cufa ] ;
    then
	exit 0
    fi
fi
# 
until (/usr/dialupip/bin/duioctl slip0 GSOFTFLAGS | grep -s '*LACTIVE')
do
	/usr/dialupip/bin/duioctl slip0 BRINGUP
	sleep 40
done

echo "Send Mail Queue"
/usr/lib/sendmail -q &
echo "Get new News"
(until /usr/local/bin/slurp news
  do
  :
  done
  echo "News Collected"; /usr/local/news/bin/input/newsrun) &
#
# now kick off outgoing news:
#
echo "Checking for outgoing News"
if [ -f /usr/spool/news/out.going/news.demon.co.uk/togo ]
 then
	echo "News to go"
	for batch in /usr/spool/news/out.going/news.demon.co.uk/togo*
	do
 		su news -c "/usr/local/bin/nntpxmit news:$batch"
	done
fi
#
# and finally, synchronise clocks
#
/usr/etc/ntp -s ntp.demon.co.uk

To use this script with PNI SLIP, replace the five key lines (the loop involving BRINGUP) with:

until (/etc/pni/bin/pnistat | grep -s 'PNI*')
do
	/etc/pni/pnirun pni0 &
	sleep 40
done

To run this automatically, you need to add a few lines to /etc/crontab.local:

10 00,06,18,22 * * 1-5 root /bin/sh -c /usr/local/bin/SLIPup >/dev/console 2>&1
0 00,06,11,18,22 * * 6,7 root /bin/sh -c /usr/local/bin/SLIPup >/dev/console 2>&1

These two lines run the script (called SLIPup) at one frequency for weekdays, and another at weekends. Make sure that this schedule meshes with the one installed for processing incoming news (the newsrun script), which you installed into /etc/crontab when you installed cnews.

Installing on a NeXT Network

Installing SLIP on a NeXT network is just a simple variation on the process already described. You should install the connection on the mailhost machine, which should have a hostname the same as your Demon nodename. Define gate.demon.co.uk as mail-relay, instead of mailhost, and use the sendmail.mailhost.cf configuration file for sendmail., with suitable changes as described above.

The standard sendmail file creates invalid from addresses for the client machines; to resolve this, you can either set the Reply-To preference in Mail, or modify sendmail ruleset 6. Comment out the line, shortly following S6, that has square brackets in it ([]).

There are two lines at the end of sendmail.mailhost.cf that need to be modified. A comment in the file explains that you have to uncomment one and comment out the other.

Created by Paul Lynch

Using WebPages 1.7

Last updated: Oct 17, 1995

P & L Systems
Paul's Home Page