tar : tape archive, typical command to put many files into a single archive file cpio : equiv to tar, different format. pax : dump : backup dir, originally intented to go to tape, but can be to a file now. ufsdump : solaris version of dump command ufsrestore : cpio -i [opts] [pattern] # IN mode = extract (think tar xf) # pattern should be quoted to avoid being interpreted by shell # "*" would extract all files? cpio -o [opts] # OUT mode = create a backup file (think tar cf) cpio opts: -B # block input -c # use ASCII for header, more portable -d # create dir as needed -m # retain file modification time -u # unconditional copy, overwrite old files -v # print a list of filenames. Think of tar tvf. ?? -t # display a toc of input file. resemble ls -l (not tar tvf??) find . -depth -print | cpio -padm /newTree # -p copy create a tree strucutre, not creating files. zip myfile.zip filename1.txt filename2.png zip -9 Smelly.zip -r Smelly/ -9 for best compression -r for recursive. the folder name Smelly/ will also be present in the zip file when this option is used
Solaris Backup
Solaris tar command eg (usable from solaris 8)
GNU tar is better. Solaris will not replace absolute path starting as /usr and replace it as ./usr ...
tar cfv test.tar $HOME
+----+
| |
VV V
tar cfX test.tar exclude.list -I include.list
^ ^ ^^ ^
| | | |
+-------------+ +-----+
cf = create tar file (called test.tar)
v = verbose, list files as they are tar-ed up
X = exclude list, followed by a filename (exclude.list), which
contain one file/dir per line
XX = list TWO exclude file list (according to man page)
-I = include list, followed by the filename (include.list).
eg of exclude.list ::
/tmp/logs
/tmp/cores
/var/cores
eg of include.list ::
/home
/var
/tmp
whatever that is in both include and excluce, it will be excluded.
For Linux, GNU TAR support all of the above, and more.
-M for multi-volume support
tar cf -M ... ?
Solaris quick and dirty backup to tape
[UBR p 75]
ufsdump 0ubdsf 64 1000000 1000000 /dev/rmt/0cbn /
0 = level 0 dump
u = update /etc/dumpdates (make sure it exist or else nothing will be done)
b = blocking factor, default is 64 (or 126)
d = density, just use large number
s = size in ft, just use large number
f = file/device to place the dump, eg /dev/rmt/0n /dev/rmt/0cbn
0 = tape drive number 0
c = use hardware compression (omit = no compression)
b = Berkeley style (no adv to next rec when reaching
end of current rec. w/o b means
ATT style, which will auto adv)
n = non rewinding (omit = rewind)
last param are the filesys to backup, this eg it is the root fs /.
Note that this is only the / fs, any other fs, eg /var or /opt, will need
to be backed up separately. ufsdump will not take multiple FS in a single
command!!
Note, ufsdump support remote devices, but need to be able to rsh to that
remote host where the tape is. Can turn on /.rhost for root, finish
backup, and remove it... but it does need rsh to be on, which is more
involved to turn off...
solaris restore command
ufsrestore tbfy 64 /dev/rmt/0n
t = table of content
b = blocking size, usually it is 64
f = file/device
y = yes on try to continue after error, alwyas use this
other options:
s = seek, eg dump created by hostdump.sh req restore tsbfy 2, ie, seek to 2nd record (first one by hostdump.sh is a lable)
x = extract only listed files (provide at end of cmd), if none specified, extract all.
i = interactive restore, provide subshell that allows ls, pwd, add, extract, etc cmds. For interactive restore, volume number asked by restore starts with 1 for the first volume (not 0) [USAH p195].
v = verbose: list file as they are extracted.
NOTE: I experienced some issue with tape backuped up by ufsdump from sol7
and then sol8 restore complained about it.
Linux Backup
dump -0 -b 64 -f /mnt/vfat/dump/etc /etc
dump -0 -b 64 -M -f /mnt/vfat/dump/home /home
-0 = level 0 dump, may want to use
-0u= level 0, update dump date, for when doing incremental backup with tapes
-b = block, 64 is good for disk xfer
-f = output file to dump to, could be a tape device
-M = create multi volume. Filename is treated as prefix, 2GB file for ext2, 4GB for vfat.
For multi-volume, when end of tape of max FS limit reached,
dump will present two prompts like:
DUMP: Do you want to rewrite this volume?: ("yes" or "no")
DUMP: Do you want to start the next tape?: ("yes" or "no")
Answering no to the first, and yes to the second question will
make dump create a new volume, incrementing the file sequence number as needed.
Note:
- dump by default will use bzip level 2 compression, ie -j2
- dump will generate relative path, so restore won't be forced to overwrite fs/dir.
- dump will NOT cross fs
- Source path consideration:
If the source path is a mount point, eg /boot, then the dir name "boot"
itself won't appear on the dump volume.
On the other hand, if the source path is a dir that is not an actual mount point,
eg /etc, then the dir name "etc/" itself will be in the dump volume.
So, when restoring, watch out where files will be placed.
restore -i first to check things our would be a good idea.
See below for more details.
restore -i -f /mnt/vfat/dump/etc
-i = enter interactive restore,
-f = source file to use for restore to read data from
The restored file will be placed in the current dir
restore -t -M -V -f /mnt/vfat/dump/home > home_restore.txt
-t = generate table of content, but really list of all files dump
expected to copy, which may not be actual list in the tape/volume,
as data maybe in subsequent tape/volume that is lost and unretrievable.
It is somewhat similar to tar tvf, but don't offer a real test.
-M = expect filename from -f as prefix, add 001, 002, 003, etc.
Copying whole fs using dump
ufsdump of a filesystem create file of the form ./path/filename, thus
ufsrestore can just cd into a desired directory and restore w/o
overwriting existing files. note that the relative path means other
mechanism need to be provided to know what fs dump is a given image.
eg for Solaris:
cd /newhd
ufsdump 0ubdsf 64 1000000 1000000 - / | ufsrestore xbfvy 64 -
cd /newhd/export/home
ufsdump 0ubdsf 64 1000000 1000000 - /export/home | ufsrestore xbfvy 64 -
eg for Linux:
mount /mnt/newhd
dump -0 -f - / | (cd /mnt/newhd ; restore rf - )
mount /mnt/newhd_boot
dump -0 -f - /boot | (cd /mnt/newhd_boot ; restore rf - )
# Be careful that source path itself is a mount point,
# or restore place files in undesired location.
# Say eg if /opt is a separate fs on source, but just dir in new hd, then do:
mkdir /mnt/newhd/opt
# cd-ing of this dir is important!
# or else opt/* will be dumpted to the dst dir
dump -0 -f - /opt | (cd /mnt/newhd/opt ; restore rf - )
# eg2, say just want to copy /home dir (part of / FS),
# then, there is no need to create the home dir,
# as dump will have it.
# The following will place a copy of /home /mnt/newhd/home
dump -0 -f - /home | (cd /mnt/newhd ; restore rf - )
solaris magnetic tape tool
solaris magnetic tape tool, for moving around inside a tape among various
volume (separate dump, tar, etc instace on the tape) [unix Backup and
Recovery book, page 84]:
mt -f /dev/rmt/0n
cmd:
rewind = rewind the tape
fsf x = seek forward to next x dump image (eg use x = 1 to forward to next record)
bsf x = seek backward x (to backup to previous image, use x=2, then check where pointer is...)
status = show status of tape driv. file 0 is first image, file 1 is 2nd, etc.
Make sure to use device with 'n' option to specify non rewinding device!
NOTE: mt and restore use of record number is offset by 1. [UBR p100]
mt, move to 2nd record when tape is newly inserted, use mt fsf 1
while for restore, use (s)eek of 2.
NOTE:
- make sure use non rewind device. or else whatever tape positioning will
be gone as soon as the cmd is finished!
- Be very careful with use of (b)erkeley or att style access to tape. b =
won't advance to next image on tape, useful when just dd-ing a block,
having restore command retrieve list of files, etc. omitting the b (thus
access tape a la att style), it will advance to next image when current
command is done, this is the case even when asking for status! use of
this is useful when one is at last block of current image and just want to
advance enough to the next image. then just use mt -f /dev/rmt/0cn
status. it will generate report, and advance tape just 1 block to the
beginning of new image.
So, use mt -f /dev/rmt/0cbn status to see current pos in tape.
If at the end of the record, and just want to advance to the begining of
the record, do this ONCE: mt -f /dev/rmt/0cn
In general, 'b'erkeley mode is good in that it doesn't waste time for tape
to reposition.
hostdump.sh
[UBR] p86, its eg use /dev/rmt/0cbn too..
hostdump.sh 0 /dev/rmt/0cbn backup.log host1 host2
0 = level 0 dump
/dev/rmt/0cbn is the device to use
backup.log is the log file, they seems to suggest use of full path for this
host1 host2 are the host to do the backup
hostdump.sh creates 3 files in the tape (and it eject when done, so expect
only 1 run of hostdump.sh on a tape, i guess it will even check lable and
complain if try to append or something...)
record 1 is a tar archieve, having a BACKUP.LABEL on details of the tape.
record 2 is the dump archieve, restrore as per instruction in label
record 3 is another tar archieve, containing the stdout and stderr of the
backup process done by hostdump.sh, it contain a list of files backed up.
retrieve as mt fsf 4 (not sure why this number, in theory it should be
2... but last try had 3 + 1)
Record 1 extraction method:
cd
tar xvf /dev/rmt/0cbn
To find out what is the block size of the tape. Note the bs=1024k, if
omited, get weired error such as not enough space!. UBR suggested size of
128k, but larger size for this test is just safer).
dd if=/dev/rmt/0cbn of=/space/junk2 count=1024 bs=1024k
for the device, can use /dev/rmt/0n too, it doesn't seems to matter in
this case
---
http://docs.sun.com/ab2/coll.47.11/SYSADV1/@Ab2PageView/idmatch(BKUPTAPEDEVICE-42512)?Ab2Lang=C&Ab2Enc=iso-8859-1#BKUPTAPEDEVICE-42512
describe in more detail the solaris backup device naming convention
l = low
m = medium
h = high
u = ultra
c = compress
if unspecified density, device use its preference. most scsi tape sense
this correctly.
Veritas VxFS backup
Veritas VxFS need their own backup commands. hostdump.sh should have been
able to take care of this, but well... vxdump is a command similar to
ufsdump. Note that in both cases, only 1 fs can be specified at a time
for the backup!
eg
nohup /usr/sbin/vxdump 0ubf 64 mis10:/dev/rmt/0cbn /disks/bigdump >> /ufsdump-staging01.log 2>&1
- i backed up the /dev/vx/dsk/rootvol using ufsdump, it was mounted as ufs
too, althought it was veritas encapsulated...
restoring files from vxdump
vxrestore tbfy 64 mis10:/dev/rmt/0cbn = list files in the dump
---
note: ufsdump to restore device use the remote shell stuff, which seems to
use port 514, 513, 512, plus new port that it fork off for the real
continuous talk. read up on firewall how to allow rsh to remote machine
with command. but right now it is becoming a pain for the qa net. :( will
use bb2...
Backup thru rsh and dd
< !--
(This is quite tricky to get everything working correctly,
error message are usually lost and/or cryptic).
MACHINE='172.17.17.11'
DUMP="/usr/sbin/ufsdump 0ubdsf 64 1000000 1000000 -"
TAPE='/dev/rmt/0cbn'
rsh $MACHINE "$DUMP /" | dd obs=64k of=$TAPE
restore as:
dd if=/dev/rmt/0cbn obs=64k | ufsrestore tbfy 64 - > dump-qa01.fil3list
dd if=/dev/rmt/0cbn obs=64k | rsh $MACHINE "ufsrestore tbfy 64 - > dump-qa01.fil3list"
from tinman prompt, this worked (listed content of file 1 (2nd dump image) of tape
dd if=/dev/rmt/0cbn obs=64k of=/space/qa01.fil1dd
ufsrestore tvbf 64 qa01.fil1dd
hmm, not sure if i was right above, another test, after copy the dd file
over, still need dd to read the file and pipe to ufsrestore, i guess it
has to deal with the blocking factor dd write the stream. so, used :
ssh mis02.dmz2 "$DUMP /" | dd obs=64k of=$TAPE
mt rewind
dd if=/dev/rmt/0cbn of=test03a
dd if=test03a | ufsrestore tbfy 64 - > /tmp/list2
and that produced a list of files
seems to have managed to do it w/o first copy to disk. the trick was
apperently specify ibs=64k instead of just bs=64k... default block size of
dd is apperently not 64k, which was what was used as input during the
backup...
dd if=/dev/rmt/0cbn ibs=64k | ufsrestore tvbfy 64 - > /tmp/qa01.fil2.list
Issues about backup and restore
ufsrestore sometime give the error message (especially when i try to use
dd first, but i think i got the req command now, so it is really a bug on
their part): "xtrmap: too many map entries" read from some amanda posting
that it is cuz of bug in ufsrestore, it can't handle the really large
number of files that was created by ufsdump... they say it was just a bug
in giving file listing, but apperently no red panic that the files could
not be restored. look into patching ufsrestore when needing to extract
these archives.
if the backup was dumped to the tape using dd, ufsrestore may not get a
chance to complain that the tape if full. not sure what will happen then.
did a dd to disk, copy part of the dd image to diff file, and did
ufsrestre tvbfy and it generated the table of content w/o complain. not
sure which one is the last file physically on tape, does not necessarily
be the last file in the toc generated bu ufsdump...
etc bit stuff.
dd and solaris floppy drive.
first, use volcheck, the it will setup the links for use w/ dd.
/vol/dev/diskette0/
/vol/dev/rdiskette0/
If there are no floppy accessible, those dirs show no entries.
If there is a floppy, then there will be a volume name in there.
ls -l /vol/dev/rdiskette0/
total 0
crw-rw-rw- 1 nobody nobody 91, 12 Dec 11 11:28 unnamed_floppy#1
copy raw image to floppy:
dd if=bootnet.img of=/vol/dev/rdiskette0/volume_1
dd if=pcmciadd.img of=/vol/dev/rdiskette0/unnamed_floppy#1
etc, depending on what the floppy volume got named by vold.
seems to take rather long to finish. at time i end it with ^C...
use eject floppy before manually taking out disk, or vold will get rather
confused about it.
tar stuff to floppy drive as raw device (floppy will be a stream device
like tape then):
tar cf /vol/dev/rdiskette0/unlabeled prog.c prog.C prog.f
----
2004/02
restoring oracle backup from outsource, on oa11i as user oracle:
mt -f /dev/rmt/0cbn status
dd if=/dev/rmt/0cbn of=./tape_rec1.dd count=1024 bs=1024k
file tape_rec1.dd
tar tvf tape_rec1.dd # ensure extract to desireble place
mt -f /dev/rmt/0cbn rewind
tar xvf /dev/rmt/0cbn
mt -f /dev/rmt/0cbn fsf 1
dd if=/dev/rmt/0cbn of=./tape_rec2.dd count=1024 bs=1024k # chk no 2nd rec.
---
hardware
Sun DDS4
really HP C5683A (firmware C005, Ultra2-SE)
device tape cap in gb/compressed speed/@comp access time in sec
DDS-3 DDS-3 12 / 24 1 / 2 40
DDS-4 DDS-4 20 / 40 1.5 / 3 50
DLT2 20 / 40 ??
DLT3 35 / 70 ??
DLT4 40 / 80 ?
SuperDLT1 160/320
The DDS-4 drive can read and write DDS-2, DDS-3, and DDS-4 tapes.
tape latch right, see white blocker =
left, see hole =
open = write protected (read-only)
close = write ok
cost dds-3 dds-4
drive 1300
tape (ea,sun) 45
tape (ea,web) 10
----
/dev/st0 = scsi tape, similar to solaris /dev/rmt/0
----
restoring in linux via tar
tar -b 40 -x -f /dev/st0 # codetree tape
tar -b 40 -x -f /dev/st1 # db data file tape #2
--
gnu tar will automatically convert apsolute path to relative,
but dropping the leading / when creating the tape archive.
-M multi volume
172.27.7.208:1 knoppix tmp
172.27.5.76 unixbackup
linux dd from file created on solaris gnu tar, bs=1024 result in unable to
allocate memory error!
dd if=/dev/st1 of=data_tape_2.dd bs=2048k
Sync Sort
sync sort backup express
/opt/BackEx/bin/bexps
to see if backup is running
root 9537 1 0 10:38:37 pts/1 0:00 cmagent
root 9536 1 0 10:38:37 ? 0:00 ./nibbler
cmagent is the daemon that always run,
nibbler is usually a zombie process
/etc/rc2.d/S95BackEx stop often cannot kill nibbler
at that point, nuke process with kill cmd
if rc script starts afresh and nibbler comes on, bad sign!
hoti1
bofh1
hoti1
bofh1