How to use the find command on Unix/Linux systems

Here is how to list the regular files (indicated by -type f) files generated the last 30 minutes in a particular directory on a Linux/Unix server:

find . -mmin -30 -type f -exec ls -l {} +

Another example, listing files in my audit_file_dest older than 10 days:

find /u01/admin/proddb01/adump/ -name '*.aud' -mtime +10 -exec ls -latrh {} \;

Count all files with extension ".trm" generated one day or less ago:

find . -name '*.trm' -mtime -1 -type f -exec ls -l {} + | wc -l

Count all files with extension ".trm" generated more than 30 days ago:

find . -name '*.trm' -mtime +30 -type f -exec ls -l {} + | wc -l

Find all files with timestamp new than April 15th, 16:14 in the afternoon:

export TIMENOW=`date +"%a %b %e %H:%M"`
echo $TIMENOW
Thu Apr 15 16:14
find . -type f -newermt "$TIMENOW" -exec ls -la {} \;

Find and delete can be done in different ways, the recommended way is to use the -delete option. In the command below, I want to find and delete all *.trc and *.trm files older than 10 days:

find /u01/oracle/diag/rdbms/mydb/mydb/trace -type f -name 'mydb*.tr*' -mtime +10 -delete

Delete all *.trm files in the current directory generated earlier than 24 hrs ago:

find *.trm -mmin +1440 -type f -delete

If the -delete option is not available, you can use a different method:

find /u01/oracle/diag/rdbms/mydb/mydb/trace -type f -name 'mydb*.tr*' -mtime +10 -exec rm -f {} \;

Or, using xargs, finding files older than 2 days:

find /u01/oracle/diag/rdbms/mydb/mydb/trace -type f -name 'mydb*.tr*' -mtime +2 -print0 | xargs -I {} -0 rm -v "{}"

Pipe to wc to get the number of files deleted:

find /u01/oracle/diag/rdbms/mydb/mydb/trace -type f -name 'mydb*.tr*' -mtime +2 -print0 | xargs -I {} -0 rm -v "{}" | wc -l

Some good external sources:

How To Format Date For Display or Use In a Shell Script
Linux / Unix: Find And Remove Files With One Command On Fly
Linux Find Command with Practical Examples

How to to use the script-command to log everything a script does

The utility "script" will make a typescript of a terminal session.
Very useful if you have a script which produces lots of output, and you need to search for errors after the script has run to completion:

script -c /tmp/install_oracle.sh

Unless you add the name of the output file, the resulting file will be called typescript.

How to delete a specific line from a file, or all characters following a specific character using sed

If you want to search a file for a particular string, and then delete that line, use sed, like this:

sed -i "/_use_column_stats_for_function/d" init.ora

In bash, using the double quotation marks was the only way to make this work when using a variable instead of a fixed string.

To delete a specific line number, in this case line number 7:

 sed -i '7d' myfile.txt
 

To delete lines 7 and 8:

 sed -i '7,8d' insert_data.sql

To delete everything after a specific character, for example #:

sed -i "s/#.*//g" /tmp/autoupgrade.ora

Possible solution to unix error "X11 connection rejected because of wrong authentication"

I was trying to start an X window Application on my server as user Oracle, but received the error

oracle@myserver:[testdb01]# xclock
X11 connection rejected because of wrong authentication.
Error: Can't open display: localhost:10.0

The DISPLAY variable was set to localhost:10.0.

The solution was to logon as root and execute

root@myserver~]# xauth list $DISPLAY

The result from this was

myserver.mydomain.com/unix:10  MIT-MAGIC-COOKIE-1  83f9f76d03106db6fc0880edfb005607

Then change user back to oracle and add the cookie to oracle's authorization file:

[root@myserver ~]# su - oracle
oracle@myserver:[testdb01]# xauth add :10 MIT-MAGIC-COOKIE-1 83f9f76d03106db6fc0880edfb005607

If you get the error:

xauth:  file /home/oracle/.Xauthority does not exist

create the file first:

touch /home/oracle/.Xauthority

Then retry the "xauth add" command above.
Verify with:

oracle@myserver:[testdb01]# xauth list $DISPLAY
myserver.mydomain.com/unix:10  MIT-MAGIC-COOKIE-1  83f9f76d03106db6fc0880edfb005607
oracle@myserver:[testdb01]# echo $DISPLAY
localhost:10.0

I could now execute X Applications:

oracle@myserver:[testdb01]# xclock

How to check the current kernel settings

sysctl -a 

or, to limit the search to kernel parameters that starts with "sem":

sysctl -a |grep shm

Example output:

[root@myserver ~]# sysctl -a | grep sem
kernel.sem = 250        32000   100     128

How to use yum to list installed packages

yum list package_name

For example:

[root@myserver ~]# yum list compat-libcap1
Loaded plugins: product-id, rhnplugin, search-disabled-repos, security
This system is receiving updates from RHN Classic or RHN Satellite.
Installed Packages
compat-libcap1.x86_64                                                   1.10-1                                                   @test-rhel-x86_64-server-6
Available Packages
compat-libcap1.i686                                                     1.10-1    

See also "yum command cheat sheet" from RH for a good overview.

How to clear your bash shell's history

history -c && history -w

How to tar and compress a directory

You want to compress the folder "setup", located in /u01/admin/scripts, with all its contents including subfolders:

cd /u01/admin/scripts
ls -latr 
drwxr-xr-x  8 oracle dba  8192 Mar 13 09:17 .
drwxrwxr-x  2 oracle dba  4096 Mar 13 09:40 setup

tar -cvzf setup.tar.gz setup

How to use gzip and tar together

To compress and tar a directory in one command, use the following syntax:

# tar -czvf mytarfile.tar.gz mydir

which will tar and compress the directory "mydir" with its contents.

To extract and untar a .tar.gz file in one command, use the following syntax:

# tar -zxvf mytarfile.tar.gz

How to manually set the date and time on an AIX server

Old, incorrect date:

[root@myserver:/] date
Wed Jul 12 10:11:36 CEST 2017

Change it to the current date:

[root@myserver:/] date 010510152017
Thu Jan  5 10:15:31 CET 2017

Where

01=month
05=day
1015=time and minute
2017=year


Let's check it, immediately after we have set it:

[root@myserver:/] date
Thu Jan  5 10:16:27 CET 2017

The ntpd deamon has already adjusted the time slightly.
Note that ntpd will only adjust the server time if the local time of the server is within 1000 sec (16 minutes) of the current time. From the xntpd man pages:


Note: When operating in a client mode, the xntpd daemon will exit with an error if no configured servers are within 1000 seconds of local system time. Use the date or ntpdate command to set the time of a bad skewed system before starting xntpd.

How to check status and start/stop daemons on the AIX platform

For example, let's check the status of the ntpd service:

[root@myserver:/] lssrc -a|grep ntpd
 xntpd            tcpip                         inoperative

It is inoperative, meaning not running.

Start it:

[root@myserver:/] startsrc -s xntpd
0513-059 The xntpd Subsystem has been started. Subsystem PID is 11141174.

If you want to stop it, use

[root@myserver:/] stopsrc -s xntpd

How to expand the /tmp file system on a Linux server

Problem: /tmp needs more space.
It is placed in the root directory of the server:

[root@myserver ~]# df -h /
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/vg_system-Root
               5.8G  4.0G  1.5G  73% /

Use vgdisplay to check how much free space exist in the volume group by looking at Free PE / Size:

[root@myserver]# vgdisplay
  --- Volume group ---
  VG Name               vg_system
  System ID
  Format                lvm2
  Metadata Areas        1
  Metadata Sequence No  11
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                5
  Open LV               5
  Max PV                0
  Cur PV                1
  Act PV                1
  VG Size               24.00 GiB <-- total size of VG
  PE Size               4.00 MiB
  Total PE              6143
  Alloc PE / Size       5631 / 22.00 GiB
  Free  PE / Size       512 / 4.00 GiB <-- 4 GB free
  VG UUID               d0OgTw-stMx-GfyT-N1zA-cEv8-1OGY-2qZ8D4

Increase size of logical volume by 2GB:

lvextend -L +2G /dev/vg_system/Root 

 Size of logical volume vg_system/Root changed from 8.00 GiB (2047 extents) to 10.00 GiB (2559 extents).
  Logical volume Root successfully resized.

Finally increase size of file system:

resize2fs /dev/mapper/vg_system-Root 

resize2fs 1.41.12 (17-May-2010)
Filesystem at /dev/mapper/vg_system-Root is mounted on /; on-line resizing required
old desc_blocks = 1, new_desc_blocks = 1
Performing an on-line resize of /dev/mapper/vg_system-Root to 2620416 (4k) blocks.
The filesystem on /dev/mapper/vg_system-Root is now 2620416 blocks long.

In this example:

vg_system = volume group name

/dev/vg_system/Root = Logical volume Path

/dev/mapper/vg_system-Root = file system name

The file system name you will find in /etc/fstab:

/dev/mapper/vg_system-Root      /       ext4    defaults        1       1

Is it possible to temporarily change the system time on my server, without the Oracle instance complaining?

Yes, it's possible. Oracle doesn't care about the wall-clock time, and will keep the redo log stream intact, regardless of what time the hosting server is set to.

The whole procedure is quite simple:

1. shutdown the database and listener

sqlplus / as sysdba
shutdown immediate
exit
lsnrctl stop

2. as root, change the system time. In my example, I am setting it to 10 days into the future:

Thu Dec 15 12:27:00 CET 2016
  [root@myserver ~]# date
  Thu Dec 15 12:27:01 CET 2016

3. Disable NTP på serveren:

  service ntpd stop

4. Open the instance and the listener

sqlplus / as sysdba
startup
exit
lsnrctl start

5. Verify that the database has picked up the new system time

SQL> select startup_time from v$instance;

STARTUP_TIME
-------------------
15.12.2016 12:28:17

6. Have a look at the archived logs:

select name,sequence#, archived,applied, status, completion_time from v$archived_log;

NAME                                                                      SEQUENCE# ARCHIVED  APPLIED    STA COMPLETION_TIME
------------------------------------------------------------------------- ---------- --------- ---------- --- -------------------
/u04/fra/mydb/archivelog/2016_12_05/o1_mf_1_12538__1480935048084626_.arc  12538 YES       NO         A   05.12.2016 11:50:48
/u04/fra/mydb/archivelog/2016_12_05/o1_mf_1_12539__1480935059227231_.arc  12539 YES       NO         A   05.12.2016 11:50:59

Force a log switch and note the time stamp of the last log:

alter system archive log current;

NAME                                                                      SEQUENCE# ARCHIVED  APPLIED    STA COMPLETION_TIME
------------------------------------------------------------------------- ---------- --------- ---------- --- -------------------
/u04/fra/mydb/archivelog/2016_12_05/o1_mf_1_12538__1480935048084626_.arc  12538 YES       NO         A   05.12.2016 11:50:48
/u04/fra/mydb/archivelog/2016_12_05/o1_mf_1_12539__1480935059227231_.arc  12539 YES       NO         A   05.12.2016 11:50:59
/u04/fra/mydb/archivelog/2016_12_15/o1_mf_1_12540__1481801585872720_.arc  12540 YES       NO         A   15.12.2016 12:33:06 

Change back again:

7. shutdown the database and the listener

sqlplus / as sysdba
shutdown immediate
exit
lsnrctl stop

8. enable ntp to have the server syncronized with the current time:

service ntpd start

9. Open the database

sqlplus / as sysdba
startup
exit
lsnrctl start

10. Check the archived logs again. Note that the log sequence doesn't care what the registered time was at the time of the log switch:

select name,sequence#, archived,applied, status, completion_time from v$archived_log;

NAME                                                                      SEQUENCE# ARCHIVED  APPLIED    STA COMPLETION
------------------------------------------------------------------------- ---------- --------- ---------- --- ----------
/u04/fra/mydb/archivelog/2016_12_05/o1_mf_1_12538__1480935048084626_.arc  12538 YES       NO         A   05.12.2016
/u04/fra/mydb/archivelog/2016_12_05/o1_mf_1_12539__1480935059227231_.arc  12539 YES       NO         A   05.12.2016
/u04/fra/mydb/archivelog/2016_12_15/o1_mf_1_12540__1481801585872720_.arc  12540 YES       NO         A   15.12.2016

alter system archive log current;

select name,sequence#, archived,applied, status, completion_time from v$archived_log;

NAME                                                                      SEQUENCE# ARCHIVED  APPLIED    STA COMPLETION
------------------------------------------------------------------------- ---------- --------- ---------- --- ----------
/u04/fra/mydb/archivelog/2016_12_05/o1_mf_1_12538__1480935048084626_.arc  12538 YES       NO         A   05.12.2016
/u04/fra/mydb/archivelog/2016_12_05/o1_mf_1_12539__1480935059227231_.arc  12539 YES       NO         A   05.12.2016
/u04/fra/mydb/archivelog/2016_12_15/o1_mf_1_12540__1481801585872720_.arc  12540 YES       NO         A   15.12.2016 
/u04/fra/mydb/archivelog/2016_12_05/o1_mf_1_12541__1480937880624137_.arc  12541 YES       NO         A   05.12.2016 

How to find the page size for a Linux installation

In order to set one of the mandatory kernel parameters in a Red Hat flavored Linux distribution, you need to know the page size.
To find it:

[root@myserver]$ getconf PAGESIZE
4096

Simple sed-command for exchanging a string in multiple files

sed -i "s/oldstring/newstring/g" 

For example

sed -i "s/11204/12102/g" /etc/oratab

How to find information about CPUs on an AIX server

lsconf | grep Processor
Processor Type: PowerPC_POWER7
Processor Implementation Mode: POWER 7
Processor Version: PV_7_Compat
Number Of Processors: 5
Processor Clock Speed: 3000 MHz
  Model Implementation: Multiple Processor, PCI bus
+ proc0                                                                          Processor
+ proc4                                                                          Processor
+ proc8                                                                          Processor
+ proc12                                                                         Processor
+ proc16                                                                         Processor    

pmcycles -m
CPU 0 runs at 3000 MHz
CPU 1 runs at 3000 MHz
CPU 2 runs at 3000 MHz
CPU 3 runs at 3000 MHz
CPU 4 runs at 3000 MHz
CPU 5 runs at 3000 MHz
CPU 6 runs at 3000 MHz
CPU 7 runs at 3000 MHz
CPU 8 runs at 3000 MHz
CPU 9 runs at 3000 MHz
CPU 10 runs at 3000 MHz
CPU 11 runs at 3000 MHz
CPU 12 runs at 3000 MHz
CPU 13 runs at 3000 MHz
CPU 14 runs at 3000 MHz
CPU 15 runs at 3000 MHz
CPU 16 runs at 3000 MHz
CPU 17 runs at 3000 MHz
CPU 18 runs at 3000 MHz
CPU 19 runs at 3000 MHz

lparstat -i
Node Name                                  : myhost
Partition Name                             : myhost
Partition Number                           : 25
Type                                       : Shared-SMT-4
Mode                                       : Uncapped
Entitled Capacity                          : 0.50
Partition Group-ID                         : 32793
Shared Pool ID                             : 0
Online Virtual CPUs                        : 5
Maximum Virtual CPUs                       : 16
Minimum Virtual CPUs                       : 1
Online Memory                              : 8192 MB
Maximum Memory                             : 12288 MB
Minimum Memory                             : 1024 MB
Variable Capacity Weight                   : 128
Minimum Capacity                           : 0.10
Maximum Capacity                           : 16.00
Capacity Increment                         : 0.01
Maximum Physical CPUs in system            : 16
Active Physical CPUs in system             : 16
Active CPUs in Pool                        : 16
Shared Physical CPUs in system             : 16
Maximum Capacity of Pool                   : 1600
Entitled Capacity of Pool                  : 1300
Unallocated Capacity                       : 0.00
Physical CPU Percentage                    : 10.00%
Unallocated Weight                         : 0
Memory Mode                                : Shared
Total I/O Memory Entitlement               : 411.000 MB
Variable Memory Capacity Weight            : 128
Memory Pool ID                             : 0
Physical Memory in the Pool                : 224.000 GB
Hypervisor Page Size                       : 4K
Unallocated Variable Memory Capacity Weight: 0
Unallocated I/O Memory entitlement         : 0.000 MB
Memory Group ID of LPAR                    : 32793
Desired Virtual CPUs                       : 5
Desired Memory                             : 8192 MB
Desired Variable Capacity Weight           : 128
Desired Capacity                           : 0.50
Target Memory Expansion Factor             : -
Target Memory Expansion Size               : -
Power Saving Mode                          : Disabled
Sub Processor Mode                         : -

Find number of cores per processors (Oracle consider threads to be cores, when estimating for example cpu_count)

smtctl

This system is SMT capable.
This system supports up to 4 SMT threads per processor.
SMT is currently enabled.
SMT boot mode is not set.
SMT threads are bound to the same virtual processor.

proc0 has 4 SMT threads.
Bind processor 0 is bound with proc0
Bind processor 1 is bound with proc0
Bind processor 2 is bound with proc0
Bind processor 3 is bound with proc0


proc4 has 4 SMT threads.
Bind processor 4 is bound with proc4
Bind processor 5 is bound with proc4
Bind processor 6 is bound with proc4
Bind processor 7 is bound with proc4


proc8 has 4 SMT threads.
Bind processor 8 is bound with proc8
Bind processor 9 is bound with proc8
Bind processor 10 is bound with proc8
Bind processor 11 is bound with proc8


proc12 has 4 SMT threads.
Bind processor 12 is bound with proc12
Bind processor 13 is bound with proc12
Bind processor 14 is bound with proc12
Bind processor 15 is bound with proc12


proc16 has 4 SMT threads.
Bind processor 16 is bound with proc16
Bind processor 17 is bound with proc16
Bind processor 18 is bound with proc16
Bind processor 19 is bound with proc16

On a Linux system, what do the parameters shmall, shmmax and shmmni define?

shmall
* indicates the total amount of shared memory that the system can use at one time (measured in pages)

shmmax
* defines the maximum size in bytes of a single shared memory segment that a Linux process can allocate in its virtual address space, in bytes.
Oracle recommends that more than half of the physical memory is assigned to shmmax

shmmni
* defines the system wide maximum number of shared memory segments

How to check memory consumption and swapping on a Linux server

The easiest way to check for memory consumption on Linux is in my opinion the "free" utility:

oracle@prodserver1:[proddb01]# free -m
             total       used       free     shared    buffers     cached
Mem:         16056       9027       7029          0        233       1093
-/+ buffers/cache:       7700       8356
Swap:         8191       2674       5517

Add totals with the -t flag:

 free -t -m
             total       used       free     shared    buffers     cached
Mem:         16056       9029       7027          0        233       1093
-/+ buffers/cache:       7701       8354
Swap:         8191       2674       5517
Total:       24248      11703      12545

So in this case I am using 7701 MB of memory, while 8354 MB is free for use.

To get a quick glance, you can have a look at the file /proc/meminfo, grep for the highlights, like this:

 egrep  'Mem|Cache|Swap' /proc/meminfo
MemTotal:       16442312 kB
MemFree:         7255224 kB
Cached:          1120380 kB
SwapCached:        45632 kB
SwapTotal:       8388600 kB
SwapFree:        5650100 kB

A good option for finding out if a server is swapping, is by using vmstat:

oracle@myserver# vmstat -w 10 4
procs -----------------------memory---------------------- ---swap-- -----io---- -system-- --------cpu--------
 r  b         swpd         free         buff        cache   si   so    bi    bo   in   cs  us  sy  id  wa  st
 3  0      2772764       659396        20340     10764836    4    4    19     9    3    4  13   6  81   0   0
 6  0      2771740       735076        20364     10766444   82    0    82     7 68360 52089  57  30  13   0   0
10  0      2771484       767816        20376     10766628    8    0    12     4 67349 52155  55  31  15   0   0
 7  0      2771228       747480        22832     10768888   34    0   495    12 67799 52119  57  30  13   0   0

where * The flag -w indicates wide output * Sample every 10 second * 4 iterations Columns: * si: Amount of memory swapped in from disk (/s) * so: Amount of memory swapped to disk (/s) When a lot of negative swapping occurs the value of "so" (swap out) is increasing.

How to append text at beginning and at end of each line in vi (unix)

Go to top of line with :0

Append text to the end of each line, globally:

:%s/$/text/g

Add text to the beginning of each line, globally:

:%s/^/text/g

How to check your Operating System's code page for a particular character

I have been involved in several migration projects where we had to deal with Western European characters, which have invalid representation in the database, and needs to be converted.

Like I showed in my post about usage of the chr function, you sometimes need to find the decimal and/or hexadecimal value for a particular character.

But how can you verify that your operating system actually agrees; how would your Operating System translate a character passed from a terminal, e.g. putty?

First, make sure your putty terminal has its translation-settings set to the character set of the server you are logging into: Right-click your putty terminal upper frame, and pick "Change Settings" from the menu. Go to the "translation" settings and then select the correct character set from the drop-down menu "Remote character set".

On Linux platforms, a simple way to check how a character would be translated would be to use the hexdump utility. Thanks to Gray Watson for demonstrating this!

man hexdump
The hexdump utility is a filter which displays the specified files, or the standard input, if no files are specified, in a user specified format.

Let's try to hexdump the character ø, and see what the internal hexadecimal representation is:

echo "ø" | hexdump -v -e '"x" 1/1 "%02X" " "'
xC3 xB8 x0A 

The prefix x in front of the values represents hexadecimal values, so the important part here is "C3 and B8" - in a multibyte character set this represent the Scandinavian character ø ( I must admit, I never figured out what the 0A represents. Anyone?)

Another way is to use the "od" utility:

man od

od - dump files in octal and other formats
       -x     same as -t x2, select hexadecimal 2-byte units

Using the -x flag straight off will give the hexadecimal value in 2-byte units:

echo "ø" | od -x
0000000 b8c3 000a
0000003

This time, the values are cast around, and should be read backwards for meaning. I have not found an explanation to why od does this. Anyone?

However, if you use the -t x notation instead,:

echo "ø" | od -t x1
0000000 c3 b8 0a
0000003

The values come out as a DBA expects; c3b8 corresponds to decimal value 50104 which in turn represent the Scandinavian letter ø.

( And again, I never figured out what the 0a represents. Anyone?)