Android System Packet Capture with HttpCanary Tutorial!

1. Introduction
Before writing crawlers, we need to perform packet capture on target applications, analyze the data, and then proceed to the script writing phase. For iPhone users, daily packet capture is quite straightforward. PC tools like Charles and Fiddler are fully capable, and “Stream” is a powerful network packet capture application for iOS with a clean interface and robust features.

However, for users of high-version Android systems, packet capture isn’t as convenient! Due to enhanced security policies in newer Android versions, CA certificates must be placed in the system certificate directory for normal packet capture. User-installed certificates are defaulted to user certificates, causing many apps to fail normal packet capture.

This article will explain the operational steps for packet capture on high-version Android systems using common scenarios.

2. Non-Rooted Devices
Android also has a powerful network packet capture tool: “HttpCanary” (Little Yellow Bird). The specific operational steps are as follows:

2.1 Export HttpCanary Root Certificate
After installing the HttpCanary app, go to the settings page and export the HttpCanary root certificate. Select “System Trusted(.0)” as the type. This saves the HttpCanary root certificate to the “Internal Storage/HttpCanary/cert/…0” directory.

2.2 Install APKPure and VMOS Pro Applications
VMOS PRO download address: https://apkpure.com/cn/vmos-pro/com.vmos.ggp
PS: Since VMOS PRO application format is XAPK, it’s recommended to install XAPK format applications through APKPure.

2.3 Import RE App, Target App, and HttpCanary Root Certificate into VMOS
Open the VMOS Pro application and import the Root Explorer app, target application, and HttpCanary root certificate file. This way, VMOS contains the target application, RE file management application, and the HttpCanary root certificate file is default saved to “VMOSfiletransferstatio/”.

2.4 Import Certificate to System Certificate Directory
In VMOS Pro, use the RE application to move the HttpCanary root certificate file to the system certificate directory.
System certificate directory: /system/etc/security/cacerts/

2.5 Start Packet Capture
Open the HttpCanary app, set the target application to “VMSO” in settings, then enable the capture switch on the main interface. Finally, operate the target application in VMOS. The target application’s network requests will be displayed in HttpCanary’s main interface list.

It’s worth noting that HttpCanary app more recommends using “Parallel Space” for packet capture, but actual usage found issues with crashes and plugin installation problems, so it’s not recommended.

3. Rooted Devices
If the phone is already rooted, we only need to move third-party certificates (like HttpCanary, Charles, etc.) to the system certificate directory. Here we explain using HttpCanary app and Charles; Fiddler is similar.

3.1 HttpCanary App Packet Capture
Operation steps:

1. Unlock and root the phone
2. Install HttpCanary app and export HttpCanary root certificate with the same format as above
3. Copy CA certificate to PC via data cable
4. Download adb on PC and configure environment variables
5. Use the following adb commands to PUSH the certificate to the system certificate directory
6. Open HttpCanary app and set the target application
7. Click the capture button on HttpCanary main interface to start capturing target application packets

bash

# Grant adb root permissions
adb root
# Disable system verification
adb disable-verity 
# Restart phone
adb reboot

# Grant adb root permissions
adb root

# Before pushing files to '/system' folder, must enter command 'adb remount'
adb remount

# Copy certificate to /system/etc/security/cacerts/
# adb push 87bc3517.0 /system/etc/security/cacerts/

# Restart
adb reroot

# Check if imported CA certificate is included
adb root
adb shell
cd /system/etc/security/cacerts/
ls

3.2 Charles Packet Capture
Operation steps:

1. Download certificate in Charles (e.g., CER certificate) from Help section, copy to phone via data cable
2. Find this certificate in file manager and install manually
3. By default installs to user certificates
4. Installation directory: /data/misc/user/0/cacerts-added/
5. Use following adb commands to enable read/write permissions for phone system directory
6. Install RE file manager app and grant Root permissions, move the above certificate from user certificate directory to system certificate directory
7. System certificate directory: /system/etc/security/cacerts/
8. Restart phone
9. Packet capture test: Check PC’s IP address, keep phone on same LAN, then set to manual proxy, finally test packet capture

bash

# Execute with root permissions
adb root
# Disable system verification
adb disable-verity
# Restart phone
adb reboot
# Run with root permissions
adb root
# Remount
adb remount

4. Conclusion
The above explains packet capture processes for various scenarios on high-version Android systems based on whether the phone is rooted. Besides the above methods, there are many alternative solutions. For rooted devices, we can install Magisk and use movecert module for packet capture; or we can use EdXposed framework + trustmealredy module for packet capture. In actual work, we can choose the method that suits our needs based on requirements.