Access Server: Issues caused due to Insufficient Disk Space

Description:

When OpenVPN Access Server has insufficient disk space available, this may lead to issues such as:

• The Admin Web UI doesn't work.
• The Admin Web UI works, but crashes after you enter your username and password.
• You can sign into the Admin Web UI, but it crashes when you click any configuration tabs.
• You can sign into the Admin Web UI and make configuration changes, but it crashes when you click save.
• The Client Web UI crashes when you try to download a client profile.
• You can't download a client profile via "Import" from OpenVPN Connect.
• The OpenVPN Services are all down.
• Your VPN Connection is intermittent or doesn't work at all.
• You can't install a fixed license.
• You experience general slowness.

Known error messages:

Admin Web UI error messages:

1. This site can't be reached.
2. The connection was reset.
3. ERR_CONNECTION_RESET

You may encounter any of the above messages due to insufficient disk space while accessing the Admin Web UI. It can occur when you first open the address for your admin site, after signing in, clicking on any configuration tabs, or trying to save configuration changes.

Client Web UI error messages:

1. Internal server error.
2. CPage error in CMain/locateChild.
3. Sorry, an internal server error has occurred.

You may encounter any of the above messages due to insufficient disk space while accessing the Client Web UI. It can occur when you first open the address for the client site, after signing in, or clicking to download a client app or profile.

Fixed license error message:

LicenseActivate: <Fault 9000: 'exceptions.IOError: ActivateKey post error: [Errno 28] No space left on device'>

You may encounter the above message due to insufficient disk space while applying a fixed license.

Resolution:

The following troubleshooting steps can help you resolve error messages related to insufficient disk space.

1. Check disk utilization:

Enter the following command:

df -TH

The output looks like this example:

Filesystem     Type      Size  Used Avail Use% Mounted on
udev           devtmpfs  2.1G     0  2.1G   0% /dev
tmpfs          tmpfs     410M   40M  370M  10% /run
/dev/sda1      ext4      8.4G  8.1G     0 100% /         >>> Here is the High Disk Utilization
tmpfs          tmpfs     2.1G     0  2.1G   0% /dev/shm
tmpfs          tmpfs     5.3M     0  5.3M   0% /run/lock
tmpfs          tmpfs     2.1G     0  2.1G   0% /sys/fs/cgroup
tmpfs          tmpfs     410M     0  410M   0% /run/user/0

This is a simple virtual machine example.  Your server may have many more filesystems.  Any time the root filesystem (it is "/" in the "Mounted on" column above) is full, you experience issues.

2. Delete unnecessary/unused packages logs and the download cache:

For Ubuntu/Debian:

sudo su 
apt-get autoremove 
apt-get clean

For Red Hat/CentOS

sudo su
dnf autoremove
dnf clean all

Note: Older Red Hat and similar systems had a command yum(1) with similar usage.

3. In case of the "CMain/locateChild" error, check the Access Server logs using the below command:

Enter the following command:

grep "CMain/locateChild" /var/log/openvpnas.log

The output looks like this example:

2022-04-04T13:34:55-0700 [stdout#info] [WEB] OUT: '2022-04-04T13:34:55-0700 [stdout#info] *** CWS Exception (getChild):  [Errno 28] No space left on device
2022-04-04T13:34:55-0700 [stdout#info] [WEB] OUT: "2022-04-04T13:34:55-0700 [UDSProxyQueryProtocol,client] expired session b'3c245c..."
2022-04-04T13:34:55-0700 [stdout#info] [WEB] OUT: '2022-04-04T13:34:55-0700 [stdout#info] CPage exception in CMain/locateChild: [Errno 28] No space left on device: cws/cmain:93,internet/defer:1418,cws/cmain:227,conf/instgen:26,conf/dmggen:30,conf/genbase:36,python3.6/shutil:122,python3.6/shutil:82 (OSError)'

4. Check the /var/log directory:

Often, the files in /var/log cause high disk utilization. Check in the /var/log directory with these commands:

cd /var/log
ls -lsh --sort=size

The contents of the directory display in descending size order, biggest to smallest.

Or you can run these commands:

cd /var/log
du -sh * | sort -rh

Any of the two above show you the files consuming more disk space.

5. Check the log.db file:

Another log you can check is the Access Server's log.db file.  This file contains more consumable information such as how long a person was connected, who it was, how much bandwidth was used, and more.  Check this log file with these commands:

cd /usr/local/openvpn_as/etc/db
ls -lsh --sort=size

Suppose the log.db file consumes a lot of space. In that case, use the below command to wipe it:

sudo /usr/local/openvpn_as/scripts/logdba --wipe
sqlite3 /usr/local/openvpn_as/etc/db/log.db 'VACUUM;'

or

service openvpnas stop
cd  /usr/local/openvpn_as/etc/db/
mv log.db log.db.backup
sqlite3 log.db.backup .dump | sqlite3 log.db
rm log.db.backup
service openvpnas start

6. Increase your hard disk size:

We recommend at least 16GB of disk space. If your hard disk has 4GB or 8GB, increase it to 16GB. Refer to our Access Server requirement guidance for more.

If you have additional questions, submit a ticket.