[Top][Contents][Prev][Next][Last]Search

Getting MAX TNT Core Dumps

This appendix contains the following sections:
What is a core dump?
Before you begin
The Ascendump daemon
Coredump command
Examples
Troubleshooting core dumps

What is a core dump?

A MAX TNT core dump is a snapshot of MAX TNT shelf controller or slot card memory. An Ascend representative might ask you to obtain a core dump to help diagnose a problem. To get a core dump from the MAX TNT, you must use the Coredump command on the MAX TNT and the Ascendump utility on a local UNIX workstation.

The Coredump command controls how the MAX TNT generates core dumps. Ascendump controls how the MAX TNT core dumps are written to disk. You can specify that the core dump be collected whenever there is a fatal error, or you can get the core dump at any time from the server running Ascendump or from the MAX TNT itself.

The core-dump server can be connected through any LAN or WAN interface, and may be multiple hops away. The only restriction is that the data path from a crashing shelf or card must pass through shelves or cards that are still alive. The only exception is that a crashing shelf can dump through its own Ethernet port, and a crashing Ethernet card can dump through one of its own Ethernet ports.

The MAX TNT uses UDP to write core dumps over the Ethernet.

Caution: Do not use core dumps unless specifically requested to by an Ascend representative.

Before you begin

Before installing and using the Ascendump utility, make sure you:

To obtain a core dump from the MAX TNT, both the daemon and the MAX TNT must be configured to allow it.

The Ascendump daemon

Ascendump has the following syntax:

ascendump [-v -r -c -u -p] [-n email-recipient] [-s slot] [-d 
directory] [host]  






 Option



 Explanation




 -v



 Verbose mode, report detailed progress of core dump. With this option, Ascendump stops after a single dump. Without this option, Ascendump will accept multiple core dumps. 




 -r



 Reset the MAX TNT after the core dump. This is the default in daemon mode.




 -c



 Do not reset the box after the core dump. This is the default in client mode.




 -p



 Print diagnostics to the terminal screen instead of Syslog. By default the server mode uses Syslog and the client mode prints to the terminal. Running Ascendump with this option does not allow multiple core-dump clients to dump at the same time. 




 -s slot



 Dump the memory of the card in slot number slot. Network traffic will be forwarded through the shelf controller. In a multishelf system, a slot card can only dump through its own shelf controller.




 -u



 Store files uncompressed. By default files are compressed with gzip.




 -n email-recipient



Send an email notification to the specified email recipient. You can use this option more than once to designate multiple recipients. You can also use mail aliases. 




 -d directory



  The directory path for writing the core dumps. The default is /usr/ascendumps.




 host



 The name of the MAX TNT from which to get the core dump. This is known as client mode.

Coredump command

The Coredump command's syntax provides the following valid entries:

Core dump naming conventions and file characteristics

 The core-dump files use the following naming convention:

hostname-[shelf, slot]-loadname-swversion-YYMMDD-HH:MM.gz

where:

For example:

tnt10.abc.com-1,3-tntmdm56k-1.3Ap22-980101-13:42.gz

When transferring the core-dump files via FTP, use binary mode.

Trigger events

The events that normally trigger a core dump are system or slot-card resets. These usually show up in the fatal error log either as "Fatal Errors" or "Operator Resets." You cannot specify the types of events that trigger core dumps.

UDP port numbers

The MAX TNT listens for core dumps on the UDP port given by the following formula:

10,000 + (shelf-number *100) + slot-number

For example, for a card on shelf 1, slot 5, the UDP port for the core dump is 10105. For the shelf controller (slot number 17) on shelf 1, the UDP port for the core dump is 10117. Similarly, the shelf controller on shelf 8 uses UDP port 10817.

Examples

This section uses examples to show how to get core dumps from the MAX TNT.

Enabling Ascendump

To start the Ascendump daemon, proceed as in the following example:

This example runs the daemon in verbose mode and will write the core dumps in uncompressed format to /usr/ascendumps.

Enabling core dumps on the MAX TNT

In the following example, the MAX TNT writes the core dump to the host at 172.31.4.34 whenever there is a fatal error:

Pulling a core dump from the MAX TNT

 In the following example, the MAX TNT enables the Ascendump daemon to solicit a dump from the MAX TNT. The Ascendump daemon is operating in client mode, and the MAX TNT core-dump facility is operating in server mode.

Once remote core dumps are enabled on the MAX TNT, an administrator can "pull" a core dump as in the following example:

where /usr/ascendumps is the directory on the Ascendump server and tnt10 is the name of the MAX TNT from which to get the core dump.

Initiating an immediate core dump

In the next example, an administrator forces an immediate core dump:

Getting core dumps from slot cards

 You can configure the Ascendump daemon to request a core dump from a particular MAX TNT slot. In the following example the modem card in slot 4 of the MAX TNT named tnt10 will write to the Ascendump server when it crashes:

  1. After opening a session with the card, execute Coredump with the remote option:

  2. Start the Ascendump daemon in slot mode:

Disabling core dumps

 To disable core dumps on the MAX TNT:

admin> coredump disable

coredump over UDP is disabled

Fatal error log and core dumps

 The fatal-error log lists the pseudouser coredump as the responsible user when the master shelf controller resets after a core dump. For example:

        OPERATOR RESET:  Index:  99  Revision: 1.3Ap8  Shelf 1 (tntsr)

                Date: 09/12/1997.       Time: 15:52:43

                Reset from unknown, user profile coredump.

Troubleshooting core dumps

Take the following steps if you have difficulty setting up the Ascend core dumps:

  1. If you have previously installed Ascendump in inetd.conf, temporarily disable it now, by commenting out the Ascendump line, then, logged in as root, send the SIGHUP command to inetd.

  2. Change to a writable directory, and enter ascendump -p -v -d

    Performing initial tests in this manner saves time by making failures immediately diagnosable.

  3. On the MAX TNT, enable core dumps to the server machine that is running Ascendump.

  4. Look for old debug profiles by entering, dir debug from the master shelf controller.

    The only reason to have a debug profile on a card other than the master shelf controller is to override the settings for the master shelf controller. Unless you want to do that, you should define a single debug profile for the master shelf controller and delete all other debug profiles. You can edit Debug profiles by using the Read, Set, and Write commands, or by using the coredump local server command, where server is the IP address of the core-dump server.

  5. Test slot-card dumps by opening a session with a slot card. You should perform a test dump first on the T1 or E1 card, if present, because these cards have smaller memories, and are quick to reboot.

  6. From the session on the card, enter coredump to check the status of core dump. The resulting output should report that core dump is enabled and that dumps will be directed to the server you specified in step 3.

  7. Force a core dump with the following command:

    coredump now

    Ascendump should print something like this:

Occasionally, core dump fails because gzip is not installed or not in the user's path. If this is the case, you should download gzip-1.2.4.tar.gz from any GNU FTP mirror site, then compile and install it, or use the -u (uncompressed) option in the Ascendump command line.

If you still have unexplained failures, run tcpdump or snoop or a packet sniffer on the Ethernet segment attached to the MAX TNT that is in the route to the dump server. Do the same on the Ethernet segment attached to the dump server in the route to the MAX TNT.

Ascend Coredump uses UDP, so filter UDP packets. If there's too much UDP traffic, you might want to filter on port-number ranges as well. For information about the UDP port core dump uses, see UDP port numbers.

Proceed to testing more cards by opening CLI channels to them and using the coredump now command. Finish by testing Coredump from the master shelf controller.

Once you have established that core dump works, reinstate your inetd.conf entry, if present, or add one if necessary. Be sure that the entry points to the same Ascendump binary that you just tested.

Here is a sample inetd.conf entry:

ascendump dgram udp nowait root /usr/local/bin/ascendump ascendump -n 
dump-notify
The -n dump-notify argument tells Ascendump to send email to the email alias dump-notify whenever a core dump is captured.


[Top][Contents][Prev][Next][Last]Search

techpubs@eng.ascend.com

Copyright © 1997, Ascend Communications, Inc. All rights reserved.