工具集

本节归类了一些在编程中可能用到的工具和编程技术. libev 的帮助页面 page 也介绍了一些有用的代码模式, 只需要简单地修改 API 的调用方式, libev 中的某些模式也可以应用到 libuv中.

定时器(Timers)

定时器在启动之后经过事先设置好的某一时间间隔就会调用回调函数. libuv 的定时器也可以被设置为按照某一时间间隔重复调用回调函数, 而不是仅调用一次.

Libuv 定时器的使用非常简单, 如下代码所示, 初始化一个定时器, 然后启动它, 在你启动定时器的同时可以设置超时时间 timeout 和可选的 repeat 参数.

定时器可以在任何时候停止.

uv_timer_t timer_req;

uv_timer_init(loop, &timer_req);
uv_timer_start(&timer_req, callback, 5000, 2000);

将启动一个定期重复触发的定时器, 该定时器将会在执行了 uv_timer_start 5 秒(timeout) 后执行回调函数, 然后每隔 2 秒(repeat)执行一次.调用

uv_timer_stop(&timer_req);

来停掉定时器. 该函数也可以在回调函数中调用.

定时器的重复时间间隔可以在任何时候通过函数设置:

uv_timer_set_repeat(uv_timer_t *timer, int64_t repeat);

如果在回调函数中调用了该函数, 则意味着:

  • 如果定时器是一次性的, 定时器已经停止, 则需要调用 uv_timer_start.
  • 如果定时器不是一次性的(not repeatedly), 并且下一次超时设置还未生效, 那么旧的超时间隔还会被使用一次, 此后新的超时间隔才会生效.

辅助函数:

int uv_timer_again(uv_timer_t *)

只对 repeating 定时器(repeating timers) 有效, 该函数和先停掉定时器然后再将 timeoutrepeat 参数设置为原始值并重启该定时器的效果一样. 如果定时器事先没有启动则该函数会出错(错误码 UV_EINVAL) 并返回 -1.

下面是一个定时器的实际例子 reference count section.

事件循环引用计数(Event loop reference count)

只有存在活动的监视器(active watchers), 事件循环就会一直运行. libuv 在事件循环启动时会让每个监视器增加它的引用计数器, 并在其退出时减少引用计数器. 也可以通过下面的函数手动修改事件循环的引用计数:

void uv_ref(uv_handle_t*);
void uv_unref(uv_handle_t*);

上述两个函数也可以是的事件循环退出执行, 即使监视器此时还是活动的(active), 也可以使用自定义对象让事件循环活着(alive).

前者可用于定时器。你可能需要每隔X秒进行GC,或者你的网络服务需要周期地发送心跳,但是你不想在GC完成或错误发生时停止它们,或者你希望你的程序在所有其他的监视器都结束了才退出。在这种情况下,在创建定时器之后直接调用unref,如果它是当前唯一运行的监视器,``un_run``仍然将会退出。

后者用于在node.js中一些libuv的方法上升到JS API中。``uv_handle_t``(所有的watcher的父类)被创建于每一个JS对象中,并且可以被增加或者减少计数(ref/unrefed)。

ref-timer/main.c

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
uv_loop_t *loop;
uv_timer_t gc_req;
uv_timer_t fake_job_req;

int main() {
    loop = uv_default_loop();

    uv_timer_init(loop, &gc_req);
    uv_unref((uv_handle_t*) &gc_req);

    uv_timer_start(&gc_req, gc, 0, 2000);

    // could actually be a TCP download or something
    uv_timer_init(loop, &fake_job_req);
    uv_timer_start(&fake_job_req, fake_job, 9000, 0);
    return uv_run(loop, UV_RUN_DEFAULT);
}

我们初始化GC定时器时,立即调用了``unref``。注意到9秒后,当测试任务完成时,程序自动退出了,即便GC仍然在运行。

空闲监视器模式(Idle watcher pattern)

空闲监视器的回调函数只会在事件循环队列中没有其他事件的情况下才会被调用。在这种情况下他们在每次循环迭代的都是都会被调用一次。空闲回调函数被用于执行一些非常低优先级的任务。比如,你可以为开发者分派每日程序性能的分析摘要在空转的周期,或者可以用户进行SETI计算:)(SETI是一项利用全球联网的计算机共同搜寻地外文明的科学实验计划)。空闲监视器对于有用户界面的程序也十分有用。如果你在使用事件循环在下载一个文件,如果TCP套接字正在建立连接并且没有其他的事件被送达,事件循环就会被停止(block),这就意味着你的进度条会卡住不动,用户就会认为程序已经挂掉了。这种情况,排队等待和空闲监视器会响应你的UI操作。

idle-compute/main.c

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
uv_loop_t *loop;
uv_fs_t stdin_watcher;
uv_idle_t idler;
char buffer[1024];

int main() {
    loop = uv_default_loop();

    uv_idle_init(loop, &idler);

    uv_fs_read(loop, &stdin_watcher, 1, buffer, 1024, -1, on_type);
    uv_idle_start(&idler, crunch_away);
    return uv_run(loop, UV_RUN_DEFAULT);
}

这里,我们初始化了空闲监视器并且把入队了一个真实的感兴趣的事件。``crunch_away``将会被重复的调用直到用户随意输入一些内容并按下回车。随后他会被短暂的中断,循环队列回去处理输入数据,之后又会重新调用空闲回调函数。

idle-compute/main.c

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
void crunch_away(uv_idle_t* handle, int status) {
    // Compute extra-terrestrial life
    // fold proteins
    // computer another digit of PI
    // or similar
    fprintf(stderr, "Computing PI...\n");
    // just to avoid overwhelming your terminal emulator
    uv_idle_stop(handle);
}

向工作者线程传递数据(Passing data to worker thread)

当使用 uv_queue_work 你通常需要传递一个复杂的数据结构向工作线程。解决方案是使用``struct``并令``uv_work_t.data``指向它。一个轻微的变化是``uv_work_t``自己会被最为结构体的第一个成员变量(被称为指挥棒[#]_)。它允许通过调用清理函数,清除有所的工作请求和数据。

1
2
3
4
5
6
7
struct ftp_baton {
    uv_work_t req;
    char *host;
    int port;
    char *username;
    char *password;
}
1
2
3
4
5
6
7
ftp_baton *baton = (ftp_baton*) malloc(sizeof(ftp_baton));
baton->req.data = (void*) baton;
baton->host = strdup("my.webhost.com");
baton->port = 21;
// ...

uv_queue_work(loop, &baton->req, ftp_session, ftp_cleanup);

这里我们创建了指挥棒和队列任务 此时任务函数就可以提取出它需要的数据了:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
void ftp_session(uv_work_t *req) {
    ftp_baton *baton = (ftp_baton*) req->data;

    fprintf(stderr, "Connecting to %s\n", baton->host);
}

void ftp_cleanup(uv_work_t *req) {
    ftp_baton *baton = (ftp_baton*) req->data;

    free(baton->host);
    // ...
    free(baton);
}

随后我们释放了指挥棒同时也释放了监视器。

轮询方式下的外部 I/O(External I/O with polling)

Usually third-party libraries will handle their own I/O, and keep track of their sockets and other files internally. In this case it isn’t possible to use the standard stream I/O operations, but the library can still be integrated into the libuv event loop. All that is required is that the library allow you to access the underlying file descriptors and provide functions that process tasks in small increments as decided by your application. Some libraries though will not allow such access, providing only a standard blocking function which will perform the entire I/O transaction and only then return. It is unwise to use these in the event loop thread, use the libuv 工作队列 instead. Of course this will also mean losing granular control on the library.

The uv_poll section of libuv simply watches file descriptors using the operating system notification mechanism. In some sense, all the I/O operations that libuv implements itself are also backed by uv_poll like code. Whenever the OS notices a change of state in file descriptors being polled, libuv will invoke the associated callback.

Here we will walk through a simple download manager that will use libcurl to download files. Rather than give all control to libcurl, we’ll instead be using the libuv event loop, and use the non-blocking, async multi interface to progress with the download whenever libuv notifies of I/O readiness.

uvwget/main.c - The setup

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
#include <stdio.h>
#include <stdlib.h>
#include <sys/select.h>
#include <uv.h>
#include <curl/curl.h>

uv_loop_t *loop;
CURLM *curl_handle;
uv_timer_t timeout;

int main(int argc, char **argv) {
    loop = uv_default_loop();

    if (argc <= 1)
        return 0;

    if (curl_global_init(CURL_GLOBAL_ALL)) {
        fprintf(stderr, "Could not init cURL\n");
        return 1;
    }

    uv_timer_init(loop, &timeout);

    curl_handle = curl_multi_init();
    curl_multi_setopt(curl_handle, CURLMOPT_SOCKETFUNCTION, handle_socket);
    curl_multi_setopt(curl_handle, CURLMOPT_TIMERFUNCTION, start_timeout);

    while (argc-- > 1) {
        add_download(argv[argc], argc);
    }

    uv_run(loop, UV_RUN_DEFAULT);
    curl_multi_cleanup(curl_handle);
    return 0;
}

The way each library is integrated with libuv will vary. In the case of libcurl, we can register two callbacks. The socket callback handle_socket is invoked whenever the state of a socket changes and we have to start polling it. start_timeout is called by libcurl to notify us of the next timeout interval, after which we should drive libcurl forward regardless of I/O status. This is so that libcurl can handle errors or do whatever else is required to get the download moving.

Our downloader is to be invoked as:

$ ./uvwget [url1] [url2] ...

So we add each argument as an URL

uvwget/main.c - Adding urls

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
void add_download(const char *url, int num) {
    char filename[50];
    sprintf(filename, "%d.download", num);
    FILE *file;

    file = fopen(filename, "w");
    if (file == NULL) {
        fprintf(stderr, "Error opening %s\n", filename);
        return;
    }

    CURL *handle = curl_easy_init();
    curl_easy_setopt(handle, CURLOPT_WRITEDATA, file);
    curl_easy_setopt(handle, CURLOPT_URL, url);
    curl_multi_add_handle(curl_handle, handle);
    fprintf(stderr, "Added download %s -> %s\n", url, filename);
}

We let libcurl directly write the data to a file, but much more is possible if you so desire.

start_timeout will be called immediately the first time by libcurl, so things are set in motion. This simply starts a libuv timer which drives curl_multi_socket_action with CURL_SOCKET_TIMEOUT whenever it times out. curl_multi_socket_action is what drives libcurl, and what we call whenever sockets change state. But before we go into that, we need to poll on sockets whenever handle_socket is called.

uvwget/main.c - Setting up polling

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
int handle_socket(CURL *easy, curl_socket_t s, int action, void *userp, void *socketp) {
    uv_poll_t *poll_fd;
    if (action == CURL_POLL_IN || action == CURL_POLL_OUT) {
        if (socketp) {
            poll_fd = (uv_poll_t*) socketp;
        }
        else {
            poll_fd = (uv_poll_t*) malloc(sizeof(uv_poll_t));
            uv_poll_init(loop, poll_fd, s);
        }
        curl_multi_assign(curl_handle, s, (void *) poll_fd);
    }

    switch (action) {
        case CURL_POLL_IN:
            uv_poll_start(poll_fd, UV_READABLE, curl_perform);
            break;
        case CURL_POLL_OUT:
            uv_poll_start(poll_fd, UV_WRITABLE, curl_perform);
            break;
        case CURL_POLL_REMOVE:
            if (socketp) {
                uv_poll_stop((uv_poll_t*) socketp);
                uv_close((uv_handle_t*) socketp, (uv_close_cb) free);
                curl_multi_assign(curl_handle, s, NULL);
            }
            break;
        default:
            abort();
    }

    return 0;
}

We are interested in the socket fd s, and the action. For every socket we create a uv_poll_t handle if it doesn’t exist, and associate it with the socket using curl_multi_assign. This way socketp points to it whenever the callback is invoked.

In the case that the download is done or fails, libcurl requests removal of the poll. So we stop and free the poll handle.

Depending on what events libcurl wishes to watch for, we start polling with UV_READABLE or UV_WRITABLE. Now libuv will invoke the poll callback whenever the socket is ready for reading or writing. Calling uv_poll_start multiple times on the same handle is acceptable, it will just update the events mask with the new value. curl_perform is the crux of this program.

uvwget/main.c - Setting up polling

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
void curl_perform(uv_poll_t *req, int status, int events) {
    uv_timer_stop(&timeout);
    int running_handles;
    int flags = 0;
    if (events & UV_READABLE) flags |= CURL_CSELECT_IN;
    if (events & UV_WRITABLE) flags |= CURL_CSELECT_OUT;

    curl_multi_socket_action(curl_handle, req->io_watcher.fd, flags, &running_handles);

    char *done_url;

    CURLMsg *message;
    int pending;
    while ((message = curl_multi_info_read(curl_handle, &pending))) {
        switch (message->msg) {
            case CURLMSG_DONE:
                curl_easy_getinfo(message->easy_handle, CURLINFO_EFFECTIVE_URL, &done_url);
                printf("%s DONE\n", done_url);

                curl_multi_remove_handle(curl_handle, message->easy_handle);
                curl_easy_cleanup(message->easy_handle);

                break;
            default:
                fprintf(stderr, "CURLMSG default\n");
                abort();
        }
    }
}

The first thing we do is to stop the timer, since there has been some progress in the interval. Then depending on what event triggered the callback, we inform libcurl of the same. Then we call curl_multi_socket_action with the socket that progressed and the flags informing about what events happened. At this point libcurl does all of its internal tasks in small increments, and will attempt to return as fast as possible, which is exactly what an evented program wants in its main thread. libcurl keeps queueing messages into its own queue about transfer progress. In our case we are only interested in transfers that are completed. So we extract these messages, and clean up handles whose transfers are done.

检查并预备监视器(Check & Prepare watchers)

TODO

库的加载(Loading libraries)

libuv provides a cross platform API to dynamically load shared libraries. This can be used to implement your own plugin/extension/module system and is used by node.js to implement require() support for bindings. The usage is quite simple as long as your library exports the right symbols. Be careful with sanity and security checks when loading third party code, otherwise your program will behave unpredicatably. This example implements a very simple plugin system which does nothing except print the name of the plugin.

Let us first look at the interface provided to plugin authors.

plugin/plugin.h

1
2
3
4
5
6
#ifndef UVBOOK_PLUGIN_SYSTEM
#define UVBOOK_PLUGIN_SYSTEM

void mfp_register(const char *name);

#endif

plugin/plugin.c

1
2
3
4
5
#include <stdio.h>

void mfp_register(const char *name) {
    fprintf(stderr, "Registered plugin \"%s\"\n", name);
}

You can similarly add more functions that plugin authors can use to do useful things in your application [1]. A sample plugin using this API is:

plugin/hello.c

1
2
3
4
5
#include "plugin.h"

void initialize() {
    mfp_register("Hello World!");
}

Our interface defines that all plugins should have an initialize function which will be called by the application. This plugin is compiled as a shared library and can be loaded by running our application:

$ ./plugin libhello.dylib
Loading libhello.dylib
Registered plugin "Hello World!"

This is done by using uv_dlopen to first load the shared library libhello.dylib. Then we get access to the initialize function using uv_dlsym and invoke it.

plugin/main.c

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
#include "plugin.h"

typedef void (*init_plugin_function)();

int main(int argc, char **argv) {
    if (argc == 1) {
        fprintf(stderr, "Usage: %s [plugin1] [plugin2] ...\n", argv[0]);
        return 0;
    }

    uv_lib_t *lib = (uv_lib_t*) malloc(sizeof(uv_lib_t));
    while (--argc) {
        fprintf(stderr, "Loading %s\n", argv[argc]);
        if (uv_dlopen(argv[argc], lib)) {
            fprintf(stderr, "Error: %s\n", uv_dlerror(lib));
            continue;
        }

        init_plugin_function init_plugin;
        if (uv_dlsym(lib, "initialize", (void **) &init_plugin)) {
            fprintf(stderr, "dlsym error: %s\n", uv_dlerror(lib));
            continue;
        }

        init_plugin();
    }

    return 0;
}

uv_dlopen expects a path to the shared library and sets the opaque uv_lib_t pointer. It returns 0 on success, -1 on error. Use uv_dlerror to get the error message.

uv_dlsym stores a pointer to the symbol in the second argument in the third argument. init_plugin_function is a function pointer to the sort of function we are looking for in the application’s plugins.

TTY

文本终端一直以来都都过 pretty standardised 来支持基本的格式化. 文本终端的格式化可以改善终端输出的可读性, 例如, grep --colour. libuv 提供了 uv_tty_t 结构(流)和相关的函数来实现跨平台的 ANSI 字符转义, 即 libuv 可以将 ANSI 码转换为与 Windows 环境下向匹配的编码, 另外 libuv 也提供了获取终端信息的函数.

首先需要初始化 uv_tty_t 结构, 传入的第三个参数为需要读/写的文件描述符:

int uv_tty_init(uv_loop_t*, uv_tty_t*, uv_file fd, int readable)

如果 readable 为 false, 后续 uv_write 调用将会被 阻塞. blocking.

最好使用 uv_tty_set_mode 函数来设置终端模式为 normal (0), 该模式下允许 TTY 的格式化, 控制流和其他设置. libuv 也支持 raw 模式.

Remember to call uv_tty_reset_mode when your program exits to restore the state of the terminal. Just good manners. Another set of good manners is to be aware of redirection. If the user redirects the output of your command to a file, control sequences should not be written as they impede readability and grep. To check if the file descriptor is indeed a TTY, call uv_guess_handle with the file descriptor and compare the return value with UV_TTY.

Here is a simple example which prints white text on a red background:

tty/main.c

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
#include <stdio.h>
#include <string.h>
#include <unistd.h>
#include <uv.h>

uv_loop_t *loop;
uv_tty_t tty;
int main() {
    loop = uv_default_loop();

    uv_tty_init(loop, &tty, 1, 0);
    uv_tty_set_mode(&tty, 0);
    
    if (uv_guess_handle(1) == UV_TTY) {
        uv_write_t req;
        uv_buf_t buf;
        buf.base = "\033[41;37m";
        buf.len = strlen(buf.base);
        uv_write(&req, (uv_stream_t*) &tty, &buf, 1, NULL);
    }

    uv_write_t req;
    uv_buf_t buf;
    buf.base = "Hello TTY\n";
    buf.len = strlen(buf.base);
    uv_write(&req, (uv_stream_t*) &tty, &buf, 1, NULL);
    uv_tty_reset_mode();
    return uv_run(loop, UV_RUN_DEFAULT);
}

The final TTY helper is uv_tty_get_winsize() which is used to get the width and height of the terminal and returns 0 on success. Here is a small program which does some animation using the function and character position escape codes.

tty-gravity/main.c

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
#include <stdio.h>
#include <string.h>
#include <unistd.h>
#include <uv.h>

uv_loop_t *loop;
uv_tty_t tty;
uv_timer_t tick;
uv_write_t write_req;
int width, height;
int pos = 0;
char *message = "  Hello TTY  ";

void update(uv_timer_t *req, int status) {
    char data[500];

    uv_buf_t buf;
    buf.base = data;
    buf.len = sprintf(data, "\033[2J\033[H\033[%dB\033[%luC\033[42;37m%s",
                            pos,
                            (unsigned long) (width-strlen(message))/2,
                            message);
    uv_write(&write_req, (uv_stream_t*) &tty, &buf, 1, NULL);

    pos++;
    if (pos > height) {
        uv_tty_reset_mode();
        uv_timer_stop(&tick);
    }
}

int main() {
    loop = uv_default_loop();

    uv_tty_init(loop, &tty, 1, 0);
    uv_tty_set_mode(&tty, 0);
    
    if (uv_tty_get_winsize(&tty, &width, &height)) {
        fprintf(stderr, "Could not get TTY information\n");
        uv_tty_reset_mode();
        return 1;
    }

    fprintf(stderr, "Width %d, height %d\n", width, height);
    uv_timer_init(loop, &tick);
    uv_timer_start(&tick, update, 200, 200);
    return uv_run(loop, UV_RUN_DEFAULT);
}

The escape codes are:

Code Meaning
2 J Clear part of the screen, 2 is entire screen
H Moves cursor to certain position, default top-left
n B Moves cursor down by n lines
n C Moves cursor right by n columns
m Obeys string of display settings, in this case green background (40+2), white text (30+7)

As you can see this is very useful to produce nicely formatted output, or even console based arcade games if that tickles your fancy. For fancier control you can try ncurses.


[1]mfp is My Fancy Plugin
[2]I was first introduced to the term baton in this context, in Konstantin Käfer’s excellent slides on writing node.js bindings – http://kkaefer.github.com/node-cpp-modules/#baton