0 Replies Latest reply on Aug 20, 2016 12:41 AM by 1663643

    Multiple default routes or Source/Policy based routing

    1663643

      Hi,

       

      I have a Solaris 10U11 1/13 system with the latest CPU installed (kernel patch 150400-38) and a gigaswift (ce) quad ethernet card. The interfaces are connected as such:

       

      ce0 100.1.1.1/29 - ISP #1 (gw: 100.1.1.254)

      ce1 10.1.1.1/21 - LAN

      ce2 192.168.1.1/29 - DMZ

      ce3 200.1.1.1/23 - ISP #2 (gw: 200.1.1.254)

       

      Originally, all DMZ & LAN traffic were going out ISP #1 and my ipf & ipnat rules were working great. I recently added ISP #2 and now I'd like all traffic from the DMZ/ce2 to go out ISP#1/ce0 and all traffic from the LAN/ce1 to go out ISP #2/ce3.

       

      My updated ipf/ipnat rules seem to be working fine but the routing is not. Since I technically now have two default routes, Solaris seems to be round-robining between them from both LAN/ce1 & DMZ/ce2 . Here's my routing table:

       

      Routing Table: IPv4

      Destination          Gateway              Flags Ref   Use    Interface

      -------------------- -------------------- ----- ---- ------- ---------

      default              200.1.1.254          UG       1   25108 ce3

      default              100.1.1.254          UG       1    2316 ce0

      10.1.0.0             10.1.1.1             U        1    4717 ce1

      200.1.1.0            200.1.1.1            U        1       1 ce3

      100.1.1.0            100.1.1.1            U        1      88 ce0

      192.168.1.0          192.168.1.1          U        1     141 ce2

      224.0.0.0            10.1.1.1             U        1       0 ce1

      127.0.0.1            127.0.0.1            UH       2      75 lo0

       

      If I remove the 200.1.1.254 default route, all traffic to/from the DMZ/ce2 works great. If I remove the 100.1.1.254 default route, all traffic to/from the LAN/ce1 works great. With neither, nothing works and with both, I see lots of dropped packets (ping tests) because the ICMP packets are going o ut the wrong interface.

       

      From what I've read, I need some sort of source-based or policy-based routing solution. I've also read on MOS that some users use ipfilter as a workaround. This is the part I'm trying to figure out but not having much luck.

       

      I tried adding these rules:

       

      pass out log quick on ce0 to ce3:200.1.1.254 from 10.1.1.0/21 to any

      pass out log quick on ce3 to ce0:100.1.1.254 from 192.168.1.0/29 to any

       

      And since I wasn't sure if ipnat rules applied before or after ipf, I added these:

       

      pass out log quick on ce0 to ce3:200.1.1.254 from 200.1.1.1 to any

      pass out log quick on ce3 to ce0:100.1.1.254 from 100.1.1.1 to any

       

      This didn't help. I was still getting dropped ICMP packets from the respective LAN/DMZ interfaces. I then added:

       

      block out log quick on ce3 to ce0:100.1.1.254 from 10.1.1.0/21 to any

      block out log quick on ce0 to ce3:200.1.1.254 from 192.168.1.0/29 to any

       

      And that helped a little. Now, I don't get dropped packets within a single ping command but subsequent ping commands don't always use the correct outgoing so zero packets are returned.

       

      Someone also mentioned populating /etc/gateways using routed but that didn't seem to help. However, I took a page from routed and also added these network routes:

       

      route add 10.1.1.0 200.1.1.254

      route add 192.168.1.0 100.1.1.254

       

      I had these routes in place while testing the above ipf rules but I'm not sure if they were required.

       

      At this point, I feel like I'm chasing my tail and would appreciate any ideas or if I'm doing something wrong.

       

      Thanks!