1:1 stateless NAT is useful for static IPs. Since all your addresses are otherwise global, if you need to switch providers or give up your /64, then you’ll need to re-address your static addresses. Instead, you can give your machines static private IPs, and just translate the prefix when going through NAT. It’s a lot less horrible than IPv4 NAT since there’s no connection tracking needed.
This is something I probably should have done setting up my home Kubernetes cluster. My current IPv6 prefix is from Hurricane Electric, and if my ISP ever gives me a real IPv6 prefix, I will have to delete the entire cluster and recreate it with the new prefix.
It should only be needed if your ISP is brain-dead and only gives you a /64 instead of what they should be doing and also giving you a /56 or /48 with prefix delegation (I.e it should be getting both a 64 for the wan interface, and a delegation for routing)
You router should be using that prefix and sticking just a /64 on the lan interface which it advertises appropriately (and you can route the others as you please)
Internal ipv6 should be using site-local ipv6, and if they have internet access they would have both addresses.
Site-local ipv6 would work here as well, true. But then my containers wouldnt have internet access. Kubernetes containers use Ipam with a single subnet, they can’t use SLAAC.
Point is, you should be able to have them have both. Or stick a reverse proxy in front that can translate. Unless they’re somehow meant to be directly internet reachable the public addresses could be autogenerated
Full disclosure though I don’t know anything about kubernetes.
I haven’t really looked into it, but it doesn’t seem like it.
Heres the documentation about having multiple cidr pools in one cluster with the Cilium network driver, and it seems to imply that each Pod only gets one IP.
There’s something called Multus that I haven’t looked into, but even then it looks like that is for multiple interfaces per Pod, not multiple IPS per interface.
Containers are just network namespaces on Linux, and all the routing is done in iptables or ebpf, so it’s theoretically possible to have multiple IP addresses, but doesn’t look like anybody has started implementing it. There’s actually a lot of Kubernetes clusters that just use stateful IPv6 NAT for the internal Pod network, unfortunately.
Weird! Though I guess a lot of these would be sitting behind load balancers / reverse proxies anyway (so ipv4 is fine) and unlikely to up and change isps very often? Lol
I have static IPs for my Kubernetes nodes, and I actually use DHCPv6 for dynamic dns so I can reach any device with a hostname, even though most of my devices don’t have static IPs.
The issue is those static IPs are tied to my current ISP, preventing me from changing ISPs without deleting my entire Kubernetes cluster.
My ISP does this right (provides a /56 for routing), but unfortunately both are dynamic and change periodically. Every time I disconnect and reconnect from the internet, I get a different prefix.
I ended up needing to have ULAs for devices where I need to know the IPv6 address on my network (e.g. my internal DNS servers).
I use openwrt on my home network which uses dnsmasq for dhcp. It can give a static suffix which just works with the global prefix on the interface and the site local / ula prefix it uses
1:1 stateless NAT is useful for static IPs. Since all your addresses are otherwise global, if you need to switch providers or give up your /64, then you’ll need to re-address your static addresses. Instead, you can give your machines static private IPs, and just translate the prefix when going through NAT. It’s a lot less horrible than IPv4 NAT since there’s no connection tracking needed.
This is something I probably should have done setting up my home Kubernetes cluster. My current IPv6 prefix is from Hurricane Electric, and if my ISP ever gives me a real IPv6 prefix, I will have to delete the entire cluster and recreate it with the new prefix.
It should only be needed if your ISP is brain-dead and only gives you a /64 instead of what they should be doing and also giving you a /56 or /48 with prefix delegation (I.e it should be getting both a 64 for the wan interface, and a delegation for routing)
You router should be using that prefix and sticking just a /64 on the lan interface which it advertises appropriately (and you can route the others as you please)
Internal ipv6 should be using site-local ipv6, and if they have internet access they would have both addresses.
Hurricane Electric gives me a /48.
Site-local ipv6 would work here as well, true. But then my containers wouldnt have internet access. Kubernetes containers use Ipam with a single subnet, they can’t use SLAAC.
Point is, you should be able to have them have both. Or stick a reverse proxy in front that can translate. Unless they’re somehow meant to be directly internet reachable the public addresses could be autogenerated
Full disclosure though I don’t know anything about kubernetes.
Yeah, I wonder if there’s any proposals to allow for multiple IPV6 addresses in Kubernetes, it would be a much better solution than NAT.
As far as I know, it’s currently not possible. Every container/Pod receives a single IPv4 and/or IPv6 address on creation from the networking driver.
Even if you give it multiple ip6 cidrs to allocate?
I haven’t really looked into it, but it doesn’t seem like it.
Heres the documentation about having multiple cidr pools in one cluster with the Cilium network driver, and it seems to imply that each Pod only gets one IP.
https://docs.cilium.io/en/stable/network/concepts/ipam/multi-pool/
There’s something called Multus that I haven’t looked into, but even then it looks like that is for multiple interfaces per Pod, not multiple IPS per interface.
https://github.com/k8snetworkplumbingwg/multus-cni
Containers are just network namespaces on Linux, and all the routing is done in iptables or ebpf, so it’s theoretically possible to have multiple IP addresses, but doesn’t look like anybody has started implementing it. There’s actually a lot of Kubernetes clusters that just use stateful IPv6 NAT for the internal Pod network, unfortunately.
Weird! Though I guess a lot of these would be sitting behind load balancers / reverse proxies anyway (so ipv4 is fine) and unlikely to up and change isps very often? Lol
If I had to, I’d be trying to add an extra cidr to one of the options listed at https://kubernetes.io/docs/concepts/services-networking/dual-stack/#configure-ipv4-ipv6-dual-stack But as you say, probably doesn’t work!
And if you want static ips either use dhcp6 or disable the randomisation of eui64 addresses
I have static IPs for my Kubernetes nodes, and I actually use DHCPv6 for dynamic dns so I can reach any device with a hostname, even though most of my devices don’t have static IPs.
The issue is those static IPs are tied to my current ISP, preventing me from changing ISPs without deleting my entire Kubernetes cluster.
It certainly looks like you can give it a prefix to assign out, possibly even multiples
Nitpicking, but the address for the wan interface wouldn’t have a prefix, so the host would just set it as a /128 (point-to-point)
Ehh, I’ve seen both. Perhaps not in a home router context though, never really bothered to check
My ISP does this right (provides a /56 for routing), but unfortunately both are dynamic and change periodically. Every time I disconnect and reconnect from the internet, I get a different prefix.
I ended up needing to have ULAs for devices where I need to know the IPv6 address on my network (e.g. my internal DNS servers).
Indeed, that’s correct ula usage, but shouldn’t need nat rewriting. The global prefixes just need to be advertised by RA packets
Yeah I’m not using NAT, sorry for the confusion.
My router doesn’t support RAs for a ULA range though, so I’m running radvd on my home server.
I use openwrt on my home network which uses dnsmasq for dhcp. It can give a static suffix which just works with the global prefix on the interface and the site local / ula prefix it uses
Note that Android doesn’t support DHCPv6, just in case you have Android devices and ever have to debug IPv6 on them.
Yup indeed. That’s why it advertises both dhcp and slaac