I participated in this years Metasploit's community ctf with Snadoteam we finished all but 2 of the challenges finishing fifth overall.

One of the most interesting challenges was definitely 10 of Diamonds being the "hardest" RE challenge in the ctf.

We found the 10_of_diamonds.exe in the home directory of the administrator on the windows box. Running file on it reveals that it is a MS-DOS executable, no surprise there.

Therefore i moved it over to my reverse engineering vm, Windows Defender sadly didn't enjoy the new file on the filesystem and promptly deleted it. Because of this I decided to upload it to virustotal to see if it recognized the "malware".

As we can see most of the hits are based on heuristics so the program is probably just doing some funky memory stuff. Quite curious I opened it in Radare

Huh

As we can see there is definitively something funky going on. A lot of functions are of the same length and there is one huge function at offset 0x00401337. The flag format for this ctf where png images so I made a educated guess that function did something related to the image and that the image was embedded into the executable somehow

Based on that i ran binwalk without any success on the file.

Since the only function called before the "main" function is the function.00491b63 I decided to take a look at it.

:S 

As we can see this pops the function address of from the stack after pushing it (call) and inserts that into eax then it ads another constant to that.

This overflows eax leaving us with a function that returns 0x491b63 any god fearing compiler would never have done some thing like this therefore I decided to move the file back into my vm and check if DIE recognized any type of packer/linker

DIE only recognized that there was some kind of linker which is something I already suspected. Based on this I decided to take a look at entry0 which is the entry point of the binary

            ;-- eip:
/ (fcn) entry0 462
|   entry0 ();
|           ; var int local_38h @ ebp-0x38
|           ; var int local_34h @ ebp-0x34
|           ; var int argc @ ebp-0x30
|           ; var int local_2ch @ ebp-0x2c
|           ; var int local_28h @ ebp-0x28
|           ; var int local_24h @ ebp-0x24
|           ; var unsigned int local_20h @ ebp-0x20
|           ; var int local_1ch @ ebp-0x1c
|           ; var int local_18h @ ebp-0x18
|           ; var unsigned int local_14h @ ebp-0x14
|           ; var int local_10h @ ebp-0x10
|           ; var int local_ch @ ebp-0xc
|           ; var int local_8h @ ebp-0x8
|           ; var int local_4h @ ebp-0x4
|           0x00491995      55             push ebp
|           0x00491996      89e5           mov ebp, esp
|           0x00491998      83ec3c         sub esp, 0x3c               ; '<'
|           0x0049199b      56             push esi
|           0x0049199c      e8c2010000     call fcn.00491b63
|           0x004919a1      89c6           mov esi, eax
|           0x004919a3      8d86867d0600   lea eax, [esi + 0x67d86]
|           0x004919a9      8945f4         mov dword [local_ch], eax
|           0x004919ac      c745f8000000.  mov dword [local_8h], 0
|           0x004919b3      8d8e877d0600   lea ecx, [esi + 0x67d87]
|           0x004919b9      894dfc         mov dword [local_4h], ecx
|           0x004919bc      c745f09563ca.  mov dword [local_10h], 0xca6395
|           0x004919c3      8b55f0         mov edx, dword [local_10h]
|           0x004919c6      85d2           test edx, edx
|       ,=< 0x004919c8      0f8407000000   je 0x4919d5
|       |   0x004919ce      c745f0198420.  mov dword [local_10h], 0x3208419
|       |   ; CODE XREF from entry0 (0x4919c8)
|       `-> 0x004919d5      8d45f4         lea eax, [local_ch]
|           0x004919d8      50             push eax
|           0x004919d9      e880fdffff     call fcn.0049175e
|           0x004919de      83c404         add esp, 4
|           0x004919e1      8945ec         mov dword [local_14h], eax
|           0x004919e4      837dec00       cmp dword [local_14h], 0
|       ,=< 0x004919e8      0f841d000000   je 0x491a0b
|       |   0x004919ee      e870010000     call fcn.00491b63
|       |   0x004919f3      89c6           mov esi, eax
|       |   0x004919f5      8d86887d0600   lea eax, [esi + 0x67d88]
|       |   0x004919fb      50             push eax                    ; int argc
|       |   0x004919fc      e8d2010000     call main                   ; int main(int argc, char **argv, char **envp)
|       |   0x00491a01      b801000000     mov eax, 1
|      ,==< 0x00491a06      e953010000     jmp 0x491b5e
|      ||   ; CODE XREF from entry0 (0x4919e8)
|      |`-> 0x00491a0b      c745e8fffa04.  mov dword [local_18h], 0x4faff
|      |    0x00491a12      c745e40f59c1.  mov dword [local_1ch], 0x3c1590f
|      |    0x00491a19      8b45e4         mov eax, dword [local_1ch]
|      |    0x00491a1c      8945e0         mov dword [local_20h], eax
|      |    0x00491a1f      817de0bd5390.  cmp dword [local_20h], 0x19053bd
|      |,=< 0x00491a26      0f8405000000   je 0x491a31
|     ,===< 0x00491a2c      e90c000000     jmp 0x491a3d
|     |||   ; CODE XREF from entry0 (0x491a26)
|     ||`-> 0x00491a31      c745e4760177.  mov dword [local_1ch], 0x770176
|     ||,=< 0x00491a38      e900000000     jmp 0x491a3d
|     |||   ; CODE XREFS from entry0 (0x491a2c, 0x491a38)
|     `-`-> 0x00491a3d      6a04           push 4                      ; 4
|      |    0x00491a3f      6800100000     push 0x1000
|      |    0x00491a44      ff75e8         push dword [local_18h]
|      |    0x00491a47      6a00           push 0
|      |    0x00491a49      e89b010000     call fcn.00491be9
|      |    0x00491a4e      8945dc         mov dword [local_24h], eax
|      |    0x00491a51      c745d894b53b.  mov dword [local_28h], 0x13bb594
|      |    0x00491a58      ff75e8         push dword [local_18h]
|      |    0x00491a5b      6a00           push 0
|      |    0x00491a5d      ff75dc         push dword [local_24h]
|      |    0x00491a60      e858010000     call fcn.00491bbd
|      |    0x00491a65      83c40c         add esp, 0xc
|      |    0x00491a68      ff75dc         push dword [local_24h]
|      |    0x00491a6b      e8c7f8f6ff     call fcn.00401337
|      |    0x00491a70      83c404         add esp, 4
|      |    0x00491a73      e8eb000000     call fcn.00491b63
|      |    0x00491a78      89c6           mov esi, eax
|      |    0x00491a7a      8d86a97d0600   lea eax, [esi + 0x67da9]
|      |    0x00491a80      50             push eax
|      |    0x00491a81      ff75fc         push dword [local_4h]
|      |    0x00491a84      e813010000     call fcn.00491b9c
|      |    0x00491a89      83c408         add esp, 8
|      |    0x00491a8c      83f800         cmp eax, 0
|      |,=< 0x00491a8f      0f8422000000   je 0x491ab7
|      ||   0x00491a95      8d86ca7d0600   lea eax, [esi + 0x67dca]
|      ||   0x00491a9b      50             push eax                    ; int argc
|      ||   0x00491a9c      e832010000     call main                   ; int main(int argc, char **argv, char **envp)
|      ||   0x00491aa1      8d86f97d0600   lea eax, [esi + 0x67df9]
|      ||   0x00491aa7      50             push eax                    ; int argc
|      ||   0x00491aa8      e826010000     call main                   ; int main(int argc, char **argv, char **envp)
|      ||   0x00491aad      b8ffffffff     mov eax, 0xffffffff         ; -1
|     ,===< 0x00491ab2      e9a7000000     jmp 0x491b5e
|     |||   ; CODE XREF from entry0 (0x491a8f)
|     ||`-> 0x00491ab7      6a04           push 4                      ; 4
|     ||    0x00491ab9      6800100000     push 0x1000
|     ||    0x00491abe      ff75e8         push dword [local_18h]
|     ||    0x00491ac1      6a00           push 0
|     ||    0x00491ac3      e821010000     call fcn.00491be9
|     ||    0x00491ac8      8945d4         mov dword [local_2ch], eax
|     ||    0x00491acb      ff75e8         push dword [local_18h]
|     ||    0x00491ace      6a00           push 0
|     ||    0x00491ad0      ff75d4         push dword [local_2ch]
|     ||    0x00491ad3      e8e5000000     call fcn.00491bbd
|     ||    0x00491ad8      83c40c         add esp, 0xc
|     ||    0x00491adb      68fffa0400     push 0x4faff
|     ||    0x00491ae0      ff75d4         push dword [local_2ch]
|     ||    0x00491ae3      ff75dc         push dword [local_24h]
|     ||    0x00491ae6      ff75fc         push dword [local_4h]
|     ||    0x00491ae9      e8dbf7f6ff     call fcn.004012c9
|     ||    0x00491aee      83c410         add esp, 0x10
|     ||    0x00491af1      e86d000000     call fcn.00491b63
|     ||    0x00491af6      89c6           mov esi, eax
|     ||    0x00491af8      8d861a7e0600   lea eax, [esi + 0x67e1a]
|     ||    0x00491afe      8945d0         mov dword [argc], eax
|     ||    0x00491b01      ff75d0         push dword [argc]           ; int argc
|     ||    0x00491b04      e8ca000000     call main                   ; int main(int argc, char **argv, char **envp)
|     ||    0x00491b09      6a00           push 0
|     ||    0x00491b0b      6880000000     push 0x80                   ; 128
|     ||    0x00491b10      6a02           push 2                      ; 2
|     ||    0x00491b12      6a00           push 0
|     ||    0x00491b14      6a00           push 0
|     ||    0x00491b16      6800000040     push 0x40000000
|     ||    0x00491b1b      8d863b7e0600   lea eax, [esi + 0x67e3b]
|     ||    0x00491b21      50             push eax
|     ||    0x00491b22      e8d8000000     call fcn.00491bff
|     ||    0x00491b27      8945c8         mov dword [local_38h], eax
|     ||    0x00491b2a      6a00           push 0
|     ||    0x00491b2c      8d45cc         lea eax, [local_34h]
|     ||    0x00491b2f      50             push eax
|     ||    0x00491b30      68fffa0400     push 0x4faff
|     ||    0x00491b35      ff75d4         push dword [local_2ch]
|     ||    0x00491b38      ff75c8         push dword [local_38h]
|     ||    0x00491b3b      e8d5000000     call fcn.00491c15
|     ||    0x00491b40      e883000000     call fcn.00491bc8
|     ||    0x00491b45      ff75c8         push dword [local_38h]
|     ||    0x00491b48      e8bd000000     call fcn.00491c0a
|     ||    0x00491b4d      8d86447e0600   lea eax, [esi + 0x67e44]
|     ||    0x00491b53      50             push eax                    ; int argc
|     ||    0x00491b54      e87a000000     call main                   ; int main(int argc, char **argv, char **envp)
|     ||    0x00491b59      b800000000     mov eax, 0
|     ||    ; CODE XREFS from entry0 (0x491a06, 0x491ab2)
|     ``--> 0x00491b5e      5e             pop esi
|           0x00491b5f      89ec           mov esp, ebp
|           0x00491b61      5d             pop ebp
\           0x00491b62      c3             ret

main is being called multiple times which isn't exactly normal I therefore decided that radare wasn't the best tool for the job and opened it up in ida

signed int start()
{
  signed int v0; // esi
  signed int v1; // esi
  signed int result; // eax
  signed int v3; // esi
  signed int v4; // eax
  signed int v5; // esi
  int v6; // ST24_4
  char v7; // [esp+Ch] [ebp-34h]
  int v8; // [esp+10h] [ebp-30h]
  int v9; // [esp+14h] [ebp-2Ch]
  int v10; // [esp+18h] [ebp-28h]
  int v11; // [esp+1Ch] [ebp-24h]
  int v12; // [esp+20h] [ebp-20h]
  int v13; // [esp+24h] [ebp-1Ch]
  int v14; // [esp+28h] [ebp-18h]
  int v15; // [esp+2Ch] [ebp-14h]
  int v16; // [esp+30h] [ebp-10h]
  int v17; // [esp+34h] [ebp-Ch]
  int v18; // [esp+38h] [ebp-8h]
  int v19; // [esp+3Ch] [ebp-4h]

  v0 = sub_491B63();
  v17 = v0 + 425350;
  v18 = 0;
  v19 = v0 + 425351;
  v16 = 52462617;
  v15 = sub_49175E(&v17);
  if ( v15 )
  {
    v1 = sub_491B63();
    sub_491BD3(v1 + 425352);
    result = 1;
  }
  else
  {
    v14 = 326399;
    v13 = 63002895;
    v12 = 63002895;
    v11 = sub_491BE9(0, 326399, 4096, 4);
    v10 = 20690324;
    sub_491BBD(v11, 0, 326399);
    sub_401337(v11);
    v3 = sub_491B63();
    if ( sub_491B9C(v19, v3 + 425385) )
    {
      sub_491BD3(v3 + 425418);
      sub_491BD3(v3 + 425465);
      result = -1;
    }
    else
    {
      v9 = sub_491BE9(0, v14, 4096, 4);
      sub_491BBD(v9, 0, v14);
      sub_4012C9(v19, v11, v9, 326399);
      v4 = sub_491B63();
      v5 = v4;
      v8 = v4 + 425498;
      sub_491BD3(v4 + 425498);
      v6 = sub_491BFF(v5 + 425531, 0x40000000, 0, 0, 2, 128, 0);
      sub_491C15(v6, v9, 326399, &v7, 0);
      sub_491BC8();
      sub_491C0A(v6);
      sub_491BD3(v5 + 425540);
      result = 0;
    }
  }
  return result;
}

as we can se the deassembled code doesn't make much sense which is expected as ida isn't able to recognize exactly what is being called. Fortunately it knows what is getting imported

2

Does those offsets seem familiar ? Based on this I decided to try to rename all of the function calls programatically, this ended up not working as I wanted to so I decided to go for a manual dynamic  approach.

I also decided to rename the function which returned the constant to ret_[constant] and to rename the big function to gold so that i could keep track of them. After that I sorted the functions based on function size adding a break point at the jmp of each of the tiny functions

Then I ignored ida's warning and ran the program

At each breakpoint I executed the jmp with the execute each instruction so that i jmped into the function being "called"

After viewing the function name i pressed the return to the calling function and renamed the function call above eip to something similar to the function name

After steeping trough another breakpoint this message was displayed

this happens in the following function called by start

signed int __cdecl sub_48175E(int a1)
{
  signed int v1; // eax
  signed int result; // eax
  int v3; // ST3C_4
  int v4; // [esp+24h] [ebp-14h]
  int v5; // [esp+28h] [ebp-10h]
  int v6; // [esp+2Ch] [ebp-Ch]
  int v7; // [esp+30h] [ebp-8h]
  int v8; // [esp+34h] [ebp-4h]

  v8 = 28856048;
  v7 = 28856048;
  v1 = ret_4791139();
  v6 = createFile(v1 + 425108, 2147483648, 0, 0, 3, 128, 0);
  if ( v6 == -1 )
  {
    v4 = 86806304;
    v5 = 5364207;
    ret_4791139();
    MessageBox();
    result = -1;
  }
  else
  {
    v5 = 21823640;
    v3 = ret_4791139() + 425161;
    sub_481BC8();
    sub_481BC8();
    if ( sub_481644(v6, &v4) == -1 )
    {
      sub_481B7B();
      ret_4791139();
      MessageBox();
      sub_481BD3();
      sub_481C0A(v6);
      sub_481BC8();
      sub_481BC8();
      result = -1;
    }
    else if ( sub_481311(a1, v4) == -1 )
    {
      ret_4791139();
      MessageBox();
      sub_481C0A(v6);
      result = -1;
    }
    else
    {
      ret_4791139();
      sub_481BD3();
      sub_481C0A(v6);
      sub_481BD3();
      result = 0;
    }
  }
  return result;
}

after creating a 10_of_diamonds.key file in the same dir as the program it spat out another message

luckily this brought me trough more of my breakpoints allowing me to rename more of the functions.

Based on this functions behavior and it's position in start, which is basically the main of this program, I decided to rename it to fileini I then sat a breakpoint right before a conditional jump based on the return value of the function.

Since this function returns 0 only in case of an error while parsing the .key file I knew that the green branch was the one i was intrested in. I therefore forced IDA to follow the green path by right klicking on the first instruction in the greenpath and forcefully setting EIP to that instruction.

After this i returned execution to the program again, it covered some more small functions which I then renamed. Sadly nothing else seemed to happen and the program exited normally. After taking a look at start function again i noticed that there was a conditional jump based on a string compare and a filecreate in case the strings where equal I therefore decided to force the execution to take the green branch as I did before.

I also decided to inspect the local variable which was used in the strcmp and the constant, the local variable was just garbage so nothing of interest, but the constant was 73d3a702db472629f27b06ac8f056476. I kept this in the back of my head. Continuing execution in the way written above

This sadly made the program perform a divide by zero

 

This divide by zero happened in a function that took the local variable as an argument

Based on this i guessed that the local variable was somehow a seed for what was apparently a decoding function. Because of this I decided to break at the push before the strcmp and set it's value to that of eax which contained the constant string

this made the program take the right branch after the strcmp and after some micro seconds the program exited. I looked excitedly after any new files and lo and behold in the program's directory there is a new png called flag.png

I want to thank Rapid7 for hosting this ctf, as I poped my first shell on the first metasploitable box I feel a particular connection to ctf's hosted by them.

I also want to thank snadoteam for allowing me to participate with them