Thursday, June 11, 2015

Global proxy with Node.js

It would be nice if applications written in node.js respected system proxy setting (i.e. http_proxy and https_proxy). But this doesn't seem to be the case.
If you make http(s) requests from your node.js application, the system setting is ignored unless you take extra care to do the right thing.
So how does one even use a proxy with http and https modules? It turns out the answer is: use an agent (the same for https). Agents can do many things and they can also implement proxying. The most popular agents for proxying seem to be http-proxy-agent and https-proxy-agent but there's also proxying-agent which can do both HTTP and HTTPS.

This is cool and dandy but it means you have to make sure the agent is passed to each and every http and https request. This is especially unpleasant if the actual requests originate somewhere in a library of a library of a library you're using and you have no reasonable way of pushing you agent through all the layers.
There is an io.js ticket for global proxy support but the general attitude seems to be a resounding meh.
At first I thought what I need to do is:
http.globalAgent = httpProxyAgent;
https.globalAgent = httpsProxyAgent;
but it turns out that doesn't do anything. The global agent is pretty much read only. It is only useful for changing the properties of the default instance that is created by http and https. (See the source of the http and https modules to see why nothing happens.)
After some discussion in freenode's #node.js someone suggested to monkey patch http.request and https.request. It's ugly, but it works. Here's what I ended up using:
var http = require('http');
var https = require('https');
var proxying = require('proxying-agent');

function monkeyPatch(module, functionName, newFunction) {
    module[functionName] = newFunction.bind(undefined, module[functionName]);
}

var httpProxy = process.env.http_proxy;
if (httpProxy) {
    console.log("Using HTTP proxy: " + httpProxy);

    var httpProxyingAgent = new proxying.ProxyingAgent({
        proxy: httpProxy
    });

    monkeyPatch(http, "request", function(originalRequest, options, callback) {
        // the change to options propagates to the caller, but it doesn't matter
        if (!options.agent)
            options.agent = httpProxyingAgent;
        return originalRequest(options, callback);
    });
}

var httpsProxy = process.env.https_proxy;

if (httpsProxy) {

    console.log("Using HTTPS proxy: " + httpsProxy);

    var httpsProxyingAgent = new proxying.ProxyingAgent({
        proxy: httpsProxy,
        tunnel: true
    });

    monkeyPatch(https, "request", function(originalRequest, options, callback) {
        // the change to options propagates to the caller, but it doesn't matter
        if (!options.agent)
            options.agent = httpsProxyingAgent;
        return originalRequest(options, callback);
    });
}

Note that if a request with an explicit agent is created, the proxy is not added. In other words: this works as long as custom agents for requests are not used.

2 comments:

  1. This is quite an old post, but it was one of the first things on Google searching for global Node.js proxy. Node.js v12 and above you can use https://github.com/gajus/global-agent. For older version of Node.js you can use https://www.npmjs.com/package/global-tunnel-ng.

    ReplyDelete
    Replies
    1. Thank you for update.

      But I still find it sad that rather than follow the convention of http_proxy and https_proxy by default (and let you override it when you want/need to), one has to pay extra attention to handle it. The result is that unless the author needed proxy, very likely the resulting sw will not support it.

      Delete