Lokeshaggarwal's DBA

Live the life you love. Love the life you live

ORA-27154: post/wait create failed ORA-27300: OS system dependent operation:semget failed with status: 28


In the below problem, I was trying to startup my database, but the system throws an error ” ORA-27300, ORA-27300″.So, below are some of the steps.

[oracle@rac1 ~]$ sqlplus “/as sysdba”

SQL*Plus: Release 11.2.0.1.0 Production on Wed Nov 7 13:59:52 2012

Copyright (c) 1982, 2009, Oracle. All rights reserved.

Connected to an idle instance.

SQL> startup
ORA-27154: post/wait create failed
ORA-27300: OS system dependent operation:semget failed with status: 28
ORA-27301: OS failure message: No space left on device
ORA-27302: failure occurred at: sskgpsemsper
SQL> exit
Disconnected

Now, wen we and see the above error, we can easily understand that these error is basically realted to some OS error.

So, I had tried to execute df -kh, but the filesystem had sufficient space.

Next step I tried to find out the memory usage:

[oracle@rac1 ~]$ top

top – 14:01:43 up 11 min, 2 users, load average: 0.06, 0.45, 0.45
Tasks: 164 total, 2 running, 162 sleeping, 0 stopped, 0 zombie
Cpu(s): 50.0%us, 0.0%sy, 0.0%ni, 50.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 2075560k total, 1540896k used, 534664k free, 43604k buffers
Swap: 2096440k total, 0k used, 2096440k free, 687916k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
7086 oracle 15 0 2196 1052 804 R 213.7 0.1 0:00.04 top
1 root 15 0 2064 628 536 S 0.0 0.0 0:00.55 init
2 root RT -5 0 0 0 S 0.0 0.0 0:00.06 migration/0
3 root 34 19 0 0 0 R 0.0 0.0 0:00.00 ksoftirqd/0
4 root RT -5 0 0 0 S 0.0 0.0 0:00.00 watchdog/0
5 root RT -5 0 0 0 S 0.0 0.0 0:00.09 migration/1
6 root 34 19 0 0 0 S 0.0 0.0 0:00.00 ksoftirqd/1
7 root RT -5 0 0 0 S 0.0 0.0 0:00.00 watchdog/1
8 root 10 -5 0 0 0 S 0.0 0.0 0:00.02 events/0
9 root 10 -5 0 0 0 S 0.0 0.0 0:00.03 events/1
10 root 12 -5 0 0 0 S 0.0 0.0 0:00.00 khelper
11 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 kthread
15 root 10 -5 0 0 0 S 0.0 0.0 0:00.01 kblockd/0
16 root 10 -5 0 0 0 S 0.0 0.0 0:00.11 kblockd/1
17 root 15 -5 0 0 0 S 0.0 0.0 0:00.00 kacpid
186 root 13 -5 0 0 0 S 0.0 0.0 0:00.00 cqueue/0
187 root 14 -5 0 0 0 S 0.0 0.0 0:00.00 cqueue/1
[oracle@rac1 ~]$ free
total used free shared buffers cached
Mem: 2075560 1540904 534656 0 43604 687916
-/+ buffers/cache: 809384 1266176

Swap: 2096440 0 2096440

[oracle@rac1 ~]$ vmstat 5 5
procs ———–memory———- —swap– —–io—- –system– —–cpu——
r b swpd free buff cache si so bi bo in cs us sy id wa st
0 0 0 534496 43660 687928 0 0 575 49 596 977 2 5 83 10 0
0 0 0 534480 43668 687952 0 0 0 14 1025 1719 0 7 92 0 0
0 0 0 534248 43684 687956 0 0 1 20 1028 1844 0 2 98 0 0
0 0 0 534240 43684 687956 0 0 0 0 1021 1989 0 1 99 0 0
1 0 0 534240 43692 687956 0 0 1 11 1028 1888 0 2 98 0 0
[oracle@rac1 ~]$ cat /proc/swaps
Filename Type Size Used Priority
/dev/sda5 partition 2096440 0 -1

So, from the above command i.e top and vmstat we got to know that system was not lacking physical memory.

[oracle@rac1 ~]$ ipcs -la

—— Shared Memory Limits ——–
max number of segments = 4096
max seg size (kbytes) = 4194303
max total shared memory (kbytes) = 1073741824
min seg size (bytes) = 1

—— Semaphore Limits ——–
max number of arrays = 128
max semaphores per array = 250
max semaphores system wide = 100
max ops per semop call = 32
semaphore max value = 32767

—— Messages: Limits ——–
max queues system wide = 16
max size of message (bytes) = 65536
default max size of queue (bytes) = 65536

[oracle@rac1 ~]$ cat /etc/sysctl.conf
# Kernel sysctl configuration file for Red Hat Linux
#
# For binary values, 0 is disabled, 1 is enabled. See sysctl(8) and
# sysctl.conf(5) for more details.

# Controls IP packet forwarding
net.ipv4.ip_forward = 0

# Controls source route verification
net.ipv4.conf.default.rp_filter = 1

# Do not accept source routing
net.ipv4.conf.default.accept_source_route = 0

# Controls the System Request debugging functionality of the kernel
kernel.sysrq = 0

# Controls whether core dumps will append the PID to the core filename
# Useful for debugging multi-threaded applications
kernel.core_uses_pid = 1

# Controls the use of TCP syncookies
net.ipv4.tcp_syncookies = 1

# Controls the maximum size of a message, in bytes
kernel.msgmnb = 65536

# Controls the default maxmimum size of a mesage queue
kernel.msgmax = 65536

# Controls the maximum shared segment size, in bytes
kernel.shmmax = 4294967295

# Controls the maximum number of shared memory segments, in pages
kernel.shmall = 268435456
fs.file-max = 6815744
net.ipv4.ip_local_port_range = 9000 65500
net.core.rmem_default = 262144
net.core.wmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_max = 1048576
fs.aio-max-nr = 1048576
kernel.sem = 250 100
[oracle@rac1 ~]$ vi /etc/sys
sysconfig/ sysctl.conf sysctl.confe syslog.conf
[oracle@rac1 ~]$ vi /etc/sysctl.conf
[oracle@rac1 ~]$ exit

logout

As per the Oracle ,the value for the semaphore should be set to :

kernel.sem = 250 32000 100 200

[root@rac1 raczone]# vi /etc/sysctl.conf
[root@rac1 raczone]# sysctl -p
net.ipv4.ip_forward = 0
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.default.accept_source_route = 0
kernel.sysrq = 0
kernel.core_uses_pid = 1
net.ipv4.tcp_syncookies = 1
kernel.msgmnb = 65536
kernel.msgmax = 65536
kernel.shmmax = 4294967295
kernel.shmall = 268435456
fs.file-max = 6815744
net.ipv4.ip_local_port_range = 9000 65500
net.core.rmem_default = 262144
net.core.wmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_max = 1048576
fs.aio-max-nr = 1048576
kernel.sem = 250 32000 100 128
[root@rac1 raczone]# su – oracle
[oracle@rac1 ~]$ export ORACLE_BASE=/raczone/11.2.0
[oracle@rac1 ~]$ . oraenv
ORACLE_SID = [oracle] ? orcl
The Oracle base for ORACLE_HOME=/raczone/11.2.0/product/11.2.0/dbhome_1 is /raczone/11.2.0
[oracle@rac1 ~]$ cat /proc/swapinfo
cat: /proc/swapinfo: No such file or directory
[oracle@rac1 ~]$ sqlplus “/as sysdba”

SQL*Plus: Release 11.2.0.1.0 Production on Wed Nov 7 14:06:56 2012

Copyright (c) 1982, 2009, Oracle. All rights reserved.

Connected to an idle instance.

SQL> startup
ORACLE instance started.

Total System Global Area 389189632 bytes
Fixed Size 1336736 bytes
Variable Size 268438112 bytes
Database Buffers 113246208 bytes
Redo Buffers 6168576 bytes
Database mounted.
Database opened.
SQL>

Advertisements

November 7, 2012 - Posted by | Troubleshooting

No comments yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: