Exploit Dev

Reverse Engineering Network Protocols

2022
bordergate

Many applications implement bespoke binary protocols for network communications. Being able to reverse engineer how a protocol works will allow you to target parts of the application that would otherwise be inaccessible. In this article, we’re going to look at reverse engineering a very simple protocol. Source code for the application is available at the end of the article.

First, let’s start by debugging the application to find which part of the code is responsible for parsing network traffic. Load the application in WinDBG Preview, and use the lm command to find the currently loaded modules;

1
2
3
4
5
6
7
8
9
10
11
12
0:004> lm
start    end        module name
00a80000 00aa1000   SimpleTCPServer C (private pdb symbols)  C:\ProgramData\Dbg\sym\SimpleTCPServer.pdb\6D32CFA3C1604A66BE332ABE75C39FD11\SimpleTCPServer.pdb
561b0000 56326000   ucrtbased   (deferred)            
56330000 5634e000   VCRUNTIME140D   (deferred)            
72ff0000 7308f000   apphelp    (deferred)            
73380000 733d2000   mswsock    (deferred)            
760a0000 76103000   WS2_32     (deferred)            
76370000 76589000   KERNELBASE   (deferred)            
766d0000 7678e000   RPCRT4     (deferred)            
772a0000 77390000   KERNEL32   (export symbols)       C:\WINDOWS\System32\KERNEL32.DLL
77520000 776c4000   ntdll      (pdb symbols)          C:\ProgramData\Dbg\sym\wntdll.pdb\57A5FC91763644189F8C0088D7B2DBFC1\wntdll.pdb

Since it appears the standard WinSock (WS2_32) library is being used, we can set a breakpoint on it’s receive function.

1
2
3
4
5
6
7
8
0:004> bp ws2_32!recv
0:004> g
Breakpoint 0 hit
eax=00000200 ebx=00c20000 ecx=00eff5bc edx=00000110 esi=00eff344 edi=00eff9c0
eip=760b23a0 esp=00eff330 ebp=00eff9c0 iopl=0         nv up ei pl zr na pe nc
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000246
WS2_32!recv:
760b23a0 8bff            mov     edi,edi

We can then generate some network traffic using ncat.

1
2
3
4
ncat 127.0.0.1  2600 -v
Ncat: Version 7.92 ( https://nmap.org/ncat )
Ncat: Connected to 127.0.0.1:2600.
AAAAA

When the breakpoint triggers, we can use the dd esp command to view the parameters supplied to the recv function.

1
2
3
4
5
6
0:000> dd esp L4
00eff330  00a91d63 00000110 00eff5bc 00000200
0:000> da 00eff5bc
00eff5bc  "AAAA."
0:000> ? 00000200
Evaluate expression: 512 = 00000200

The above output shows the memory address 0x00eff5bc is being used to store characters sent from the client. The value next to that (0x200) is the number of bytes that will be accepted by the socket in one go.

We know this is the order of the functions arguments based on the documented function parameters:

1
2
3
4
5
6
int recv(
  [in]  SOCKET s,
  [out] char   *buf,
  [in]  int    len,
  [in]  int    flags
);

Stepping through execution with the p command we end up in the main server code.

1
2
3
4
5
6
0:000> p
eax=00000005 ebx=00c20000 ecx=00000002 edx=00000000 esi=00eff344 edi=00eff9c0
eip=00a91d63 esp=00eff344 ebp=00eff9c0 iopl=0         nv up ei pl zr na pe nc
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000246
SimpleTCPServer!main+0x3a3:
00a91d63 3bf4            cmp     esi,esp

At this point, we know the memory address of the main module that is going to be performing processing on the data sent (0x00a91d63). To examine the programs logic in IDA Pro we will need to syncronise IDA’s memory addresses with the currently running server being debugged using WinDBG.

1
2
3
4
lm m SimpleTCPServer
Browse full module list
start    end        module name
00a80000 00aa1000   SimpleTCPServer C (private pdb symbols)  C:\ProgramData\Dbg\sym\SimpleTCPServer.pdb\6D32CFA3C1604A66BE332ABE75C39FD11\SimpleTCPServer.pdb

In IDA Pro, select Edit > Segments > Rebase Program, and add the start address from the above output.

Hitting the g key in IDA will then allow us to jump to the same memory address were currently looking at in WinDBG.

From reviewing the IDA output, we can see a decision appears to take place based on the data being sent;

By adding a breakpoint just before this block, we can see that our first byte of input is being read and the number 0x41 is being subtracted from it. The result is zero, which then branches execution to the left. This is a common assembly construct for a switch statement being implemented in a higher level language.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
0:000> g
Breakpoint 2 hit
eax=00000041 ebx=00c20000 ecx=00000041 edx=56335a84 esi=00eff344 edi=00eff9c0
eip=00a91e1b esp=00eff344 ebp=00eff9c0 iopl=0         nv up ei pl nz ac pe nc
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000216
SimpleTCPServer!main+0x45b:
00a91e1b 83e941          sub     ecx,41h
0:000> r ecx
ecx=00000041
0:000> p
Breakpoint 1 hit
eax=00000041 ebx=00c20000 ecx=00000000 edx=56335a84 esi=00eff344 edi=00eff9c0
eip=00a91e2d esp=00eff344 ebp=00eff9c0 iopl=0         nv up ei ng nz ac po cy
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000293
SimpleTCPServer!main+0x46d:
00a91e2d 8b9590f9ffff    mov     edx,dword ptr [ebp-670h] ss:002b:00eff350=00000000

We know that 0x41 (or the capital letter A in ASCII) leads us down the first branch of execution. The first, second and third branches do not appear to do anything interesting. However, the last branch takes a size input, and executes an additional function. As such it warrents further investigation. Based on the previous analysis, we can reach this area of code by entering 0x44 as our first input character.

Examining the function we have routed to, we can see a memcpy operation being performed:

We set a breakpoint in the application before the memcpy call, and send the data “DCBA”, we can see the size parameter seems to be populated based on the second byte of data being sent:

1
2
3
4
5
6
7
8
9
10
0:004> bp 00A91859
0:004> g
Breakpoint 0 hit
eax=00000043 ebx=00d21000 ecx=00baf498 edx=00baf1f4 esi=00baf220 edi=00baf210
eip=00a91859 esp=00baf118 ebp=00baf210 iopl=0         nv up ei pl nz ac pe nc
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000216
SimpleTCPServer!vulnFunction+0x59:
00a91859 e8b8faffff      call    SimpleTCPServer!ILT+785(_memcpy) (00a91316)
0:000> r eax
eax=00000043

So we have learnt the following:

  • The application takes upto 512 bytes of user input.
  • By entering 0x44 (‘D’ in ASCII) as the first character we can route execution to a memcpy function.
  • Based on the previous output, the second byte is supplied as the size parameter to the memcpy function.

Due to these conditions, the application seems suseptible to a stack based buffer overflow. I’ll leave exploiting that as an exercise for the reader 🙂

Closing Thoughts

The above example shows simple network protocol analysis. I’ve provided the C++ vulnerable server code below if you would like to follow along with this tutorial.

C++ Server Code

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
#undef UNICODE
 
#define WIN32_LEAN_AND_MEAN
 
#include <windows.h>
#include <winsock2.h>
#include <ws2tcpip.h>
#include <stdlib.h>
#include <stdio.h>
#include <string>
#pragma comment (lib, "Ws2_32.lib")
 
#define DEFAULT_BUFLEN 512
#define DEFAULT_PORT "2600"
 
void vulnFunction(char sourcebuffer[], int length)
{
    char output_buffer[20];
    printf("In vulnerable function!\n");
    printf("Number of bytes to copy: %i\n", length);
    memcpy(output_buffer,sourcebuffer, length);
}
 
int __cdecl main(void)
{
    WSADATA wsaData;
    int iResult;
 
    SOCKET ListenSocket = INVALID_SOCKET;
    SOCKET ClientSocket = INVALID_SOCKET;
 
    struct addrinfo* result = NULL;
    struct addrinfo hints;
 
    int iSendResult;
    char recvbuf[DEFAULT_BUFLEN];
    int recvbuflen = DEFAULT_BUFLEN;
 
    iResult = WSAStartup(MAKEWORD(2, 2), &wsaData);
    if (iResult != 0) {
        printf("WSAStartup failed with error: %d\n", iResult);
        return 1;
    }
 
    ZeroMemory(&hints, sizeof(hints));
    hints.ai_family = AF_INET;
    hints.ai_socktype = SOCK_STREAM;
    hints.ai_protocol = IPPROTO_TCP;
    hints.ai_flags = AI_PASSIVE;
 
    iResult = getaddrinfo(NULL, DEFAULT_PORT, &hints, &result);
    if (iResult != 0) {
        printf("getaddrinfo failed with error: %d\n", iResult);
        WSACleanup();
        return 1;
    }
 
    ListenSocket = socket(result->ai_family, result->ai_socktype, result->ai_protocol);
    if (ListenSocket == INVALID_SOCKET) {
        printf("socket failed with error: %ld\n", WSAGetLastError());
        freeaddrinfo(result);
        WSACleanup();
        return 1;
    }
 
    iResult = bind(ListenSocket, result->ai_addr, (int)result->ai_addrlen);
    if (iResult == SOCKET_ERROR) {
        printf("bind failed with error: %d\n", WSAGetLastError());
        freeaddrinfo(result);
        closesocket(ListenSocket);
        WSACleanup();
        return 1;
    }
 
    freeaddrinfo(result);
 
    iResult = listen(ListenSocket, SOMAXCONN);
    if (iResult == SOCKET_ERROR) {
        printf("listen failed with error: %d\n", WSAGetLastError());
        closesocket(ListenSocket);
        WSACleanup();
        return 1;
    }
 
    ClientSocket = accept(ListenSocket, NULL, NULL);
    if (ClientSocket == INVALID_SOCKET) {
        printf("accept failed with error: %d\n", WSAGetLastError());
        closesocket(ListenSocket);
        WSACleanup();
        return 1;
    }
 
    printf("Listening...\n");
 
    closesocket(ListenSocket);
 
    do {
 
        iResult = recv(ClientSocket, recvbuf, recvbuflen, 0);
        if (iResult > 0) {
            printf("----------------------------------\n");
            printf("Number of bytes received: %d\n", iResult);
            // Add NULL byte to we can process client input as string
            recvbuf[iResult] = '\00';
            printf("Recieved: %s\n", recvbuf);
 
            int opcode = recvbuf[0];
            printf("Opcode: 0x%X\n", opcode);
 
            switch (opcode)
            {
            case 0x41:
                printf("Opcode 1 called\n");
                break;
            case 0x42:
                printf("Opcode 2 called\n");
                break;
            case 0x43:
                printf("Opcode 3 called\n");
                break;
            case 0x44:
                printf("Opcode 4 called\n");
                vulnFunction(recvbuf, recvbuf[1]);
                break;
            }
 
            iSendResult = send(ClientSocket, recvbuf, iResult, 0);
            printf("Bytes sent: %s\n", recvbuf);
 
            if (iSendResult == SOCKET_ERROR) {
                printf("send failed with error: %d\n", WSAGetLastError());
                closesocket(ClientSocket);
                WSACleanup();
                return 1;
            }
            printf("Bytes sent: %d\n", iSendResult);
        }
        else if (iResult == 0)
            printf("Connection closing...\n");
        else {
            printf("recv failed with error: %d\n", WSAGetLastError());
            closesocket(ClientSocket);
            WSACleanup();
            return 1;
        }
 
    } while (iResult > 0);
 
    iResult = shutdown(ClientSocket, SD_SEND);
    if (iResult == SOCKET_ERROR) {
        printf("shutdown failed with error: %d\n", WSAGetLastError());
        closesocket(ClientSocket);
        WSACleanup();
        return 1;
    }
 
    closesocket(ClientSocket);
    WSACleanup();
    return 0;
}

Related Posts

Exploit Dev

x64 GOT Pointer Leakage

2025
bordergate
Exploit Dev

Source Code Auditing

2025
bordergate
Exploit Dev

Directory Junctions

2025
bordergate
Exploit Dev

Integer Security

2025
bordergate