Cloudflare Proxied DNS + visitor real IP (Nginx Ingress Controller tweaks)

Questions and Answers
1

Hey Qovery Team!

We were trying to configure the nginx ingress controller to forward the real visitor IP address from a CloudFlare proxied DNS and we managed to achieve it by adding the following parameters in the nginx-ingress-ingress-nginx-controller configmap. However, this deployment is managed by Qovery and probably it can get overwritten.
Do you have any suggestions on how to implement it in a permanent way as it could directly affect our customers?

  allow-snippet-annotations: "true"
  forwarded-for-header: X-Forwarded-For
  proxy-body-size: 100m
  server-snippet: |
    real_ip_header X-Forwarded-For;
    set_real_ip_from 10.42.0.0/16;
    set_real_ip_from 173.245.48.0/20;
    set_real_ip_from 103.21.244.0/22;
    set_real_ip_from 103.22.200.0/22;
    set_real_ip_from 103.31.4.0/22;
    set_real_ip_from 141.101.64.0/18;
    set_real_ip_from 108.162.192.0/18;
    set_real_ip_from 190.93.240.0/20;
    set_real_ip_from 188.114.96.0/20;
    set_real_ip_from 197.234.240.0/22;
    set_real_ip_from 198.41.128.0/17;
    set_real_ip_from 162.158.0.0/15;
    set_real_ip_from 104.16.0.0/13;
    set_real_ip_from 104.24.0.0/14;
    set_real_ip_from 172.64.0.0/13;
    set_real_ip_from 131.0.72.0/22;
    set_real_ip_from 2400:cb00::/32;
    set_real_ip_from 2606:4700::/32;
    set_real_ip_from 2803:f800::/32;
    set_real_ip_from 2405:b500::/32;
    set_real_ip_from 2405:8100::/32;
    set_real_ip_from 2a06:98c0::/29;
    set_real_ip_from 2c0f:f248::/32;
  server-tokens: "false"
  use-forwarded-headers: "true"
1 Like
2

Hey @rafael-blueskyelearn

X-Forwarded-For should already be set. Can you check your application HTTP headers? IP address should be the real one already. Let me know if it’s not.

Cheers

3

Hey @bchastanier,

thanks for replying!

I’ve exhaustively tested it in my Staging environment and once I’ve turned on Cloudflare Proxied DNS, my application started to receive one of Cloudflare’s IPs Range. The only way that we have found to get the visitor real IP was adding up these changes as recommended by Cloudflare. Restoring original visitor IPs · Cloudflare Support docs

Where exactly is the X-Forwarded-For set?

4

Hey !

Can you share your Qovery console URL for this app please? Also, if possible, removing those extra settings so I can have a look?

For example, here’s logs for an application of mine:

0.0.130.51 - - [20/Mar/2024 10:55:37] "GET / HTTP/1.1" 200 -
ERROR:root:Host: p8000-z5ca19b4e-z936fc018-gtw.z33c3c94d.jvm.world
X-Request-ID: fa385bd75d0b48212b88536dd5e28727
X-Real-IP: 92.154.45.101
X-Forwarded-For: 92.154.45.101
X-Forwarded-Host: p8000-z5ca19b4e-z936fc018-gtw.z33c3c94d.jvm.world
X-Forwarded-Port: 443
X-Forwarded-Proto: https
X-Forwarded-Scheme: https
X-Scheme: https
user-agent: Mozilla/5.0 (X11; Linux x86_64; rv:122.0) Gecko/20100101 Firefox/122.0
accept: image/avif,image/webp,*/*
accept-language: en-US,en;q=0.5
accept-encoding: gzip, deflate, br
dnt: 1
referer: https://p8000-z5ca19b4e-z936fc018-gtw.z33c3c94d.jvm.world/
sec-fetch-dest: image
sec-fetch-mode: no-cors
sec-fetch-site: same-origin

The header has X-Forwarded-For with the proper IP.
Can you describe your setup and how / what your app is checking?

Thanks !

5

Hey,

https://console.qovery.com/organization/0a4f3c2b-cb5b-4f8c-807e-8d9b23b983c4/project/fe8fa189-570f-4c84-afe3-ab965050cd88/environment/7dc1c002-7bf8-4985-8f8c-d32f80cd10dd/application/a7940b43-d840-4256-a1d8-c7edfdaff4f7/general

I’ve removed the changes. You can test using the following urls staging.pathlms.io or pathlms.ai

6

Hey !

Are you able to log raw HTTP headers from your app?

Also can you share your CF configuration for the domain, especially the transform part, is there any chance the mask client IP option is checked out?

CleanShot 2024-03-20 at 23.25.20
CleanShot 2024-03-20 at 23.25.201925Ă—645 93.4 KB

7

Hello!

The “Remove visitor IP headers” rule is disabled in Cloudflare. We don’t have the headers logged in our app but I managed to get it using tcpdump inside the pod. So when we don’t have my Nginx Ingress Controller configmap changes, we get this header:

X-Frame-Options: ALLOWALL
P3P: CP="NOI ADM DEV COM NAV OUR STP"
Location: https://pathlms.ai/admin/sign_in
Content-Type: text/html; charset=utf-8
Cache-Control: no-cache
X-Request-Id: 2d18e3d3f9ab16b98ec604ad40bf391d
X-Runtime: 0.333381
Transfer-Encoding: chunked

But once I add those settings in the configmap, we receive these headers:

Host: pathlms.ai
X-Request-ID: 148c9ca619a69b04fae7db84ea3884da
X-Real-IP: 91.193.131.107
X-Forwarded-For: 91.193.131.107
X-Forwarded-Host: pathlms.ai
X-Forwarded-Port: 443
X-Forwarded-Proto: https
X-Forwarded-Scheme: https
X-Scheme: https
X-Original-Forwarded-For: 91.193.131.107
cdn-loop: cloudflare
cf-ipcountry: UA
accept-encoding: gzip, br
cf-ray: 867ed3cdb8295b8d-VIE
cf-visitor: {"scheme":"https"}
user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:123.0) Gecko/20100101 Firefox/123.0
accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8
accept-language: en-US,en;q=0.5
referer: https://pathlms.ai/
upgrade-insecure-requests: 1
sec-fetch-dest: document
sec-fetch-mode: navigate
sec-fetch-site: same-origin
sec-fetch-user: ?1
pragma: no-cache
cache-control: no-cache
cf-connecting-ip: 91.193.131.107
8

Hey @rafael-blueskyelearn !

Interesting.
And what happens if you deactivate the proxied option on CF and remove extra nginx options?

Cheers

9

Hello @bchastanier,

then we will get the visitor real IP. The point is that we need to keep the proxied option, and we enable it we get just the proxy IP from the cloudflare IP range list.

10

Hey @rafael-blueskyelearn,

Ok, what we can try is to add an advanced settings for nginx ConfigMap - Ingress-Nginx Controller so you can set it to true.
I am gonna test on my end if that would be working, feel free to try it as well if you can.
If it looks good then we will provide this option as advanced settings.

Update: this solo param use-forwarded-headers is not working, I am looking for the minimal setup to make this work especially without having to specify all CF IPs if any

Cheers

11

Update 2: enable-real-ip seems to do the trick ConfigMap - Ingress-Nginx Controller

Did a test setting enable-real-ip: "true" in the config map and I am able to see the real client IP.

I use this debug container Docker, you can have a try BEFORE / AFTER altering the config-map, X-Forwarded-For IP value should be the client one.

If everything is ok, I will add this advanced settings.

I am also not sure why your app doesn’t get al the headers, hence, can you try with the test container I shared?

Thanks !

12

Hey @bchastanier

the enable-real-ip: “true” property did the trick, we managed to get the visitor real IP. We would really appreciate if we could add this in advanced settings.

Thank you very much!

13

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.