Undici proxyAgent with an authenticated HTTP proxy
Undici is a modern http framework for node, and is used under the hood in many libraries like discord.js.
Sometimes we need to do requests through a proxy to prevent IP-blocks, rate-limits or geographical restriction.
The nice thing about HTTP proxies is that they are standardized. You can find a phletora of different cheap proxy providers, whereas you go with a solution like ScrapingBee, you will be locked into that system.
Undici without proxy
import { request } from "undici"
const { body } = await request("https://httpbin.org/ip")
console.log(await body.json()) // => {origin: "YOUR_IP_ADDRESS"}Undici with proxy
import { request, ProxyAgent } from "undici"
const proxyAgent = new ProxyAgent("my.proxy.com")
const { body } = await request("https://httpbin.org/ip", {
dispatcher: proxyAgent,
})
console.log(await body.json()) // => {origin: "PROXY_IP"}Undici with ð authenticated proxy
Most proxy urls you find will be authenticated. The ones from Apify have this shape: http://<username>:<password>@proxy.apify.com:8000
If you try this with undici, it will give you this error:
Proxy response !== 200 when HTTP TunnelingTurns out we need to add a Proxy-Authorization header, because Undici does not do this for us. We can add this by setting the token parameter of ProxyAgent.
The resulting code that works with both authenticated proxies and not, is:
import { request, ProxyAgent } from "undici"
import url from "url"
const proxyUrl = "http://myuser:mypwd>@proxy.apify.com:8000"
const proxyUrlParsed = url.parse(proxyUrl)
const proxyAgent = new ProxyAgent({
uri: proxyUrl,
token: proxyUrlParsed.auth
? `Basic ${Buffer.from(proxyUrlParsed.auth).toString("base64")}`
: undefined,
})
const { body } = await request("https://httpbin.org/ip", {
dispatcher: proxyAgent,
})
console.log(await body.json()) // => {origin: "PROXY_IP"}