2012年01月13日
點擊數: 23687 字體: 大 中 小 
一鍵關注匯訊
這份報告對開源工程HttpTunnel的工作原理及源程序運行流程進行分析，重點分析HttpTunnel對HTTP協議的封裝與實現。
1 工作原理概述
HttpTunnel通過HTTP請求方式，提供了一個雙向的、虛擬的數據通路，可以通過http proxy來使用（不是必需）。
HttpTunnel編譯連接之后得到兩個可執行的文件，hts和htc，hts是服務器端程序，用于要連接的外部主機之上，htc是客戶端程序，用于本地主機之上。這兩個部分連接用于產生一個虛擬的數據通道（tunnel）。
Http Tunnel利用HTPP的POST與GET兩個命令建立了兩個連接，分別用于客戶端向服務器發送和接收數據，而且考慮到了HTTP數據的合法性，會隨時檢查所接收或發出的數據是否超過content-length規定的長度，如果是則填充完后重新開連接進行處理新的數據，保證了數據始終是合法的HTTP數據，完全等價于客戶端在同時用HTTP協議在上傳和下載一個大文件，文件數據中不需要任何的HTTP命令或HTML語言標記。
2 HttpTunnel程序流程分析
2.1 服務器端程序流程分析
hts是服務器端,安裝在外部網絡一側，也就是沒有防火墻的一端。
hts源文件包括hts.c, common.c, tunnel.c, http.c這幾個文件和port目錄下的庫。
流程如下：
main() (hts.c)
parse_arguments (argc, argv, &arg),解析命令行參數;
調用tunnel = tunnel_new_server (arg.port, arg.content_length)創建新的tunnel的服務端;
初始化tunnel, in_fd, out_fd都為-1
在tunnel->server_socket監聽server_socket (tunnel->dest.host_port, 1), backlog=1, 注意用了SO_REUSEADDR選項; àcommon.c
通道的一些選項:strict_content_length, keep_alive, max_connection_age, 這些在HTTP頭說明中會用到;
寫PID文件
進入無限循環
如果定義輸入輸出的終端設備(arg.device),打開設備fd = open_device (arg.device),common.c;
tunnel_accept接受外部連接 àtunnel.c
accept()連接
解析HTTP協議請求數據: http_parse_request (s, &request), http.c
POST或PUT命令, 將此socket設置為tunnel的in_fd, 用于接收客戶端的數據;
GET命令, 將此socket設置為tunnel的out_fd, 用于發送去往客戶端的數據;
如果設置了轉發端口(arg.forward_port != -1), 調用do_connect連接到這個端口,描述符為fd, 注意這個連接是本機的
自己連自己的內部連接, hts起到代理轉發作用, 目標端口也就是真正的被HTTP包裹的協議端口;
進入下一個循環
poll: fd 和tunnel->server_socket, 也就是網絡通道數據和內部連接數據對倒;
handle_input()
handle_device_input()處理fd輸入信息; common.c
handle_input()
handle_tunnel_input()處理來自通道的信息; common.c
服務器端接收了客戶端的兩個連接, POST命令對應的連接服務器接收數據, GET命令對應的連接服務器發送數據。
2.2 客戶端程序流程分析
htc是客戶端,安裝在防火墻內部網絡一側，也即客戶瀏覽器一端。
htc源文件包括htc.c, common.c, tunnel.c, http.c, base64.c這幾個文件和port目錄下的庫程序
流程如下：
main.c (htc.c)
parse_arguments (argc, argv, &arg); 解析命令行參數, àhtc.c
如果轉發端口(arg.forward_port != -1),在此端口打開socket監聽s = server_socket
(arg.forward_port, 0),
backlog=0, 其實不收連接, 注意用了SO_REUSEADDR選項, àcommon.c
進入無限循環for (;;);
定義輸入輸出的終端設備(arg.device),打開設備fd = open_device (arg.device),common.c;
否則如果定義了轉發端口(arg.forward_port != -1),輸入輸出通過連接的socket來進行fd = wait_for_connection_on_socket (s),就是accept() , à htc.c
打開一個新的通道 tunnel = tunnel_new_client (arg.host_name, arg.host_port,
arg.proxy_name, arg.proxy_port,
arg.content_length); tunnel.c
注意tunnel結構中的in_fd, out_fd都初始化為-1,表示沒有連接, 缺省的content_length是100K字節.
設置通道的一些選項:strict_content_length, keep_alive, max_connection_age, user_agent這些在HTTP頭說明中會用到, 支持代理;
如果要進行代理認證(arg.proxy_authorization != NULL),將認證參數進行base64編碼,作為通道的proxy_authorization參數;
通道連接對方tunnel_connect (tunnel), tunnel.c, 建立http tunnel,主要是調用函數tunnel_write_request (tunnel, TUNNEL_OPEN, auth_data, sizeof auth_data)
如果要寫入和已經寫入的數據長度超過content_length, 對tuenel進行填充;
對于客戶端已經連接好的tunnel,超時時進行斷開;
對于斷開(或第一次連接)的客戶端, 調用tunnel_out_connect (tunnel)發起連接
調用do_connect()連接服務器(第一個連接), socket描述符為tunnel->out_fd.
設置該socket的一些選項;
調用shutdown(out_fd, 0), 不接收數據;
調用http_post()函數向服務器發送HTTP的POST命令; http.c
調用tunnel_write_data (tunnel, &request, sizeof request)函數向服務器寫要執行的請求(此時為TUNNEL_OPEN)
繼續寫data部分,先寫長度,然后是數據(此時為dummy的auth_data=42, length=1),(即HTTP的POST命令向服務器寫TUNNEL_OPEN命令和一個dummy數據),然后進行數據長度判斷是否在此連接中數據寫多了.
進入函數tunnel_in_connect (tunnel) , 數據進入的連接
調用do_connect()連接服務器(第2個連接),socket描述符為tunnel->in_fd.
設置該socket的一些選項;
調用http_get()函數向服務器發送HTTP的GET命令; à http.c
調用shutdown(in_fd, 1), 不再寫數據;
調用http_parse_response (tunnel->in_fd, &response)解析HTTP服務器返回數據;
處理統計信息
此時tunnel_connect完成
進入下一個循環
poll選擇用戶的輸入設備fd和網絡tunnel的in_fd(第2條連接)數據,也就是用戶的輸入信息和服務器返回的信息進行對倒:
fd輸入->tunnel->out_fd輸出; tunnel->in_fd輸入->fd輸出.
handle_input()
handle_device_input()處理用戶輸入信息, àcommon.c
handle_input()
handle_tunnel_input()處理來自通道的信息, àcommon.c
客戶端一共向服務器發起了兩個連接, 第一個連接用于發送數據,第2個連接用于接收數據
3 HttpTunnel源碼相關分析
3.1 HTTP頭的封裝與實現
3.1.1 HTTP頭結構
在HttpTunnel中，將HTTP頭實現為名值對。是一個遞歸的結構。
typedef struct http_header Http_header;
struct http_header
{
const char *name;
const char *value;
Http_header *next; /* FIXME: this is ugly; need cons cell. */
};
3.1.2 創建HTTP頭
static inline Http_header *
http_alloc_header (const char *name, const char *value)
{
Http_header *header;
header = malloc (sizeof (Http_header));
if (header == NULL)
return NULL;
header->name = header->value = NULL;
header->name = strdup (name);
header->value = strdup (value);
if (name == NULL || value == NULL)
{
if (name == NULL)
free ((char *)name);
if (value == NULL)
free ((char *)value);
free (header);
return NULL;
}
return header;
}
3.1.3 添加HTTP頭
Http_header *
http_add_header (Http_header **header, const char *name, const char *value)
{
Http_header *new_header;
new_header = http_alloc_header (name, value);
if (new_header == NULL)
return NULL;
new_header->next = NULL;
while (*header)
header = &(*header)->next;
*header = new_header;
return new_header;
}
3.1.4 解析HTTP頭
static ssize_t
parse_header (int fd, Http_header **header)
{
unsigned char buf[2];
unsigned char *data;
Http_header *h;
size_t len;
ssize_t n;
*header = NULL;
n = read_all (fd, buf, 2);
if (n <= 0)
return n;
if (buf[0] == ' ' && buf[1] == ' ')
return n;
h = malloc (sizeof (Http_header));
if (h == NULL)
{
log_error ("parse_header: malloc failed");
return -1;
}
*header = h;
h->name = NULL;
h->value = NULL;
n = read_until (fd, ':', &data);
if (n <= 0)
return n;
data = realloc (data, n + 2);
if (data == NULL)
{
log_error ("parse_header: realloc failed");
return -1;
}
memmove (data + 2, data, n);
memcpy (data, buf, 2);
n += 2;
data[n - 1] = 0;
h->name = data;
len = n;
n = read_until (fd, ' ', &data);
if (n <= 0)
return n;
data[n - 1] = 0;
h->value = data;
len += n;
n = read_until (fd, ' ', &data);
if (n <= 0)
return n;
free (data);
if (n != 1)
{
log_error ("parse_header: invalid line ending");
return -1;
}
len += n;
log_verbose ("parse_header: %s:%s", h->name, h->value);
n = parse_header (fd, &h->next);
if (n <= 0)
return n;
len += n;
return len;
}
3.1.5 寫HTTP頭
static ssize_t
http_write_header (int fd, Http_header *header)
{
ssize_t n = 0, m;
if (header == NULL)
return write_all (fd, " ", 2);
m = write_all (fd, (void *)header->name, strlen (header->name));
if (m == -1)
{
return -1;
}
n += m;
m = write_all (fd, ": ", 2);
if (m == -1)
{
return -1;
}
n += m;
m = write_all (fd, (void *)header->value, strlen (header->value));
if (m == -1)
{
return -1;
}
n += m;
m = write_all (fd, " ", 2);
if (m == -1)
{
return -1;
}
n += m;
m = http_write_header (fd, header->next);
if (m == -1)
{
return -1;
}
n += m;
return n;
}
3.1.6 查找HTTP頭
static Http_header *
http_header_find (Http_header *header, const char *name)
{
if (header == NULL)
return NULL;
if (strcmp (header->name, name) == 0)
return header;
return http_header_find (header->next, name);
}
3.1.7 獲取HTTP頭
const char *
http_header_get (Http_header *header, const char *name)
{
Http_header *h;
h = http_header_find (header, name);
if (h == NULL)
return NULL;
return h->value;
}
3.1.8 銷毀HTTP頭
static void
http_destroy_header (Http_header *header)
{
if (header == NULL)
return;
http_destroy_header (header->next);
if (header->name)
free ((char *)header->name);
if (header->value)
free ((char *)header->value);
free (header);
}
3.2 HTTP方法的封裝與實現
3.2.1 HTTP方法簡介
OPTIONS :
GET :
HEAD :
POST :
PUT :
DELETE:
TRACE :
CONNECT:
3.2.2 HTTP方法枚舉
在程序中，定義以下的枚舉類型。
typedef enum
{
HTTP_GET,
HTTP_PUT,
HTTP_POST,
HTTP_OPTIONS,
HTTP_HEAD,
HTTP_DELETE,
HTTP_TRACE
} Http_method;
3.2.3 HTTP方法的通用實現
在HttpTunnel里面，首先定義了一個通用的實現HTTP方法的函數，即。
static inline ssize_t
http_method (int fd, Http_destination *dest,
Http_method method, ssize_t length)
{
char str[1024]; /* FIXME: possible buffer overflow */
Http_request *request;
ssize_t n;
if (fd == -1)
{
log_error ("http_method: fd == -1");
return -1;
}
n = 0;
if (dest->proxy_name != NULL)
n = sprintf (str, "http://%s:%d", dest->host_name, dest->host_port);
sprintf (str + n, "/index.html?crap=%ld", time (NULL));
request = http_create_request (method, str, 1, 1);
if (request == NULL)
return -1;
sprintf (str, "%s:%d", dest->host_name, dest->host_port);
http_add_header (&request->header, "Host", str);
if (length >= 0)
{
sprintf (str, "%d", length);
http_add_header (&request->header, "Content-Length", str);
}
http_add_header (&request->header, "Connection", "close");
if (dest->proxy_authorization)
{
http_add_header (&request->header,
"Proxy-Authorization",
dest->proxy_authorization);
}
if (dest->user_agent)
{
http_add_header (&request->header,
"User-Agent",
dest->user_agent);
}
n = http_write_request (fd, request);
http_destroy_request (request);
return n;
}
該函數工作流程分析如下：
3.2.4 GET方法的實現
ssize_t
http_get (int fd, Http_destination *dest)
{
return http_method (fd, dest, HTTP_GET, -1);
}
3.2.5 POST方法的實現
http_post (int fd, Http_destination *dest, size_t length)
{
return http_method (fd, dest, HTTP_POST, (ssize_t)length);
}
3.3 HTTP請求的封裝與實現
3.3.1 HTTP請求結構
typedef struct
{
Http_method method;
const char *uri;
int major_version;
int minor_version;
Http_header *header;
} Http_request;
3.3.2 為HTTP請求分配內存空間
static inline Http_request *
http_allocate_request (const char *uri)
{
Http_request *request;
request = malloc (sizeof (Http_request));
if (request == NULL)
return NULL;
request->uri = strdup (uri);
if (request->uri == NULL)
{
free (request);
return NULL;
}
return request;
}
3.3.3 創建HTTP請求
Http_request *
http_create_request (Http_method method,
const char *uri,
int major_version,
int minor_version)
{
Http_request *request;
request = http_allocate_request (uri);
if (request == NULL)
return NULL;
request->method = method;
request->major_version = major_version;
request->minor_version = minor_version;
request->header = NULL;
return request;
}
3.3.4 解析HTTP請求
ssize_t
http_parse_request (int fd, Http_request **request_)
{
Http_request *request;
unsigned char *data;
size_t len;
ssize_t n;
*request_ = NULL;
request = malloc (sizeof (Http_request));
if (request == NULL)
{
log_error ("http_parse_request: out of memory");
return -1;
}
request->method = -1;
request->uri = NULL;
request->major_version = -1;
request->minor_version = -1;
request->header = NULL;
n = read_until (fd, ' ', &data);
if (n <= 0)
{
free (request);
return n;
}
request->method = http_string_to_method (data, n - 1);
if (request->method == -1)
{
log_error ("http_parse_request: expected an HTTP method");
free (data);
free (request);
return -1;
}
data[n - 1] = 0;
log_verbose ("http_parse_request: method = "%s"", data);
free (data);
len = n;
n = read_until (fd, ' ', &data);
if (n <= 0)
{
free (request);
return n;
}
data[n - 1] = 0;
request->uri = data;
len += n;
log_verbose ("http_parse_request: uri = "%s"", request->uri);
n = read_until (fd, '/', &data);
if (n <= 0)
{
http_destroy_request (request);
return n;
}
else if (n != 5 || memcmp (data, "HTTP", 4) != 0)
{
log_error ("http_parse_request: expected "HTTP"");
free (data);
http_destroy_request (request);
return -1;
}
free (data);
len = n;
n = read_until (fd, '.', &data);
if (n <= 0)
{
http_destroy_request (request);
return n;
}
data[n - 1] = 0;
request->major_version = atoi (data);
log_verbose ("http_parse_request: major version = %d",
request->major_version);
free (data);
len += n;
n = read_until (fd, ' ', &data);
if (n <= 0)
{
http_destroy_request (request);
return n;
}
data[n - 1] = 0;
request->minor_version = atoi (data);
log_verbose ("http_parse_request: minor version = %d",
request->minor_version);
free (data);
len += n;
n = read_until (fd, ' ', &data);
if (n <= 0)
{
http_destroy_request (request);
return n;
}
free (data);
if (n != 1)
{
log_error ("http_parse_request: invalid line ending");
http_destroy_request (request);
return -1;
}
len += n;
n = parse_header (fd, &request->header);
if (n <= 0)
{
http_destroy_request (request);
return n;
}
len += n;
*request_ = request;
return len;
}
3.3.5 銷毀HTTP請求
void
http_destroy_request (Http_request *request)
{
if (request->uri)
free ((char *)request->uri);
http_destroy_header (request->header);
free (request);
}
3.4 HTTP響應的封裝與實現
3.4.1 HTTP響應結構
typedef struct
{
int major_version;
int minor_version;
int status_code;
const char *status_message;
Http_header *header;
} Http_response;
3.4.2 為HTTP響應分配內存空間
static inline Http_response *
http_allocate_response (const char *status_message)
{
Http_response *response;
response = malloc (sizeof (Http_response));
if (response == NULL)
return NULL;
response->status_message = strdup (status_message);
if (response->status_message == NULL)
{
free (response);
return NULL;
}
return response;
}
3.4.3 創建HTTP響應
Http_response *
http_create_response (int major_version,
int minor_version,
int status_code,
const char *status_message)
{
Http_response *response;
response = http_allocate_response (status_message);
if (response == NULL)
return NULL;
response->major_version = major_version;
response->minor_version = minor_version;
response->status_code = status_code;
response->header = NULL;
return response;
}
3.4.4 解析HTTP響應
ssize_t
http_parse_response (int fd, Http_response **response_)
{
Http_response *response;
unsigned char *data;
size_t len;
ssize_t n;
*response_ = NULL;
response = malloc (sizeof (Http_response));
if (response == NULL)
{
log_error ("http_parse_response: out of memory");
return -1;
}
response->major_version = -1;
response->minor_version = -1;
response->status_code = -1;
response->status_message = NULL;
response->header = NULL;
n = read_until (fd, '/', &data);
if (n <= 0)
{
free (response);
return n;
}
else if (n != 5 || memcmp (data, "HTTP", 4) != 0)
{
log_error ("http_parse_response: expected "HTTP"");
free (data);
free (response);
return -1;
}
free (data);
len = n;
n = read_until (fd, '.', &data);
if (n <= 0)
{
free (response);
return n;
}
data[n - 1] = 0;
response->major_version = atoi (data);
log_verbose ("http_parse_response: major version = %d",
response->major_version);
free (data);
len += n;
n = read_until (fd, ' ', &data);
if (n <= 0)
{
free (response);
return n;
}
data[n - 1] = 0;
response->minor_version = atoi (data);
log_verbose ("http_parse_response: minor version = %d",
response->minor_version);
free (data);
len += n;
n = read_until (fd, ' ', &data);
if (n <= 0)
{
free (response);
return n;
}
data[n - 1] = 0;
response->status_code = atoi (data);
log_verbose ("http_parse_response: status code = %d",
response->status_code);
free (data);
len += n;
n = read_until (fd, ' ', &data);
if (n <= 0)
{
free (response);
return n;
}
data[n - 1] = 0;
response->status_message = data;
log_verbose ("http_parse_response: status message = "%s"",
response->status_message);
len += n;
n = read_until (fd, ' ', &data);
if (n <= 0)
{
http_destroy_response (response);
return n;
}
free (data);
if (n != 1)
{
log_error ("http_parse_request: invalid line ending");
http_destroy_response (response);
return -1;
}
len += n;
n = parse_header (fd, &response->header);
if (n <= 0)
{
http_destroy_response (response);
return n;
}
len += n;
*response_ = response;
return len;
}
3.5 HTTP目標的封裝
typedef struct
{
const char *host_name;
int host_port;
const char *proxy_name;
int proxy_port;
const char *proxy_authorization;
const char *user_agent;
} Http_destination;
還有一些其他相關函數可以參考源程序。
