In case of attacks, you can fiddle up with the following values:

net.inet.tcp.msl (on my machine, 10000. default is 30000)
net.inet.tcp.keepidle (default 10000)
net.inet.tcp.keepintvl (default 7500)
net.inet.tcp/udp.blackhole (turn on for DoS)
net.inet.tcp.tcbhashsize (push up to a reasonable value)


--------------------------------------------------------------------------------

For a better performance, you should mess up with:

kern.ipc.somaxconn (my machine = 4096, default. 128)
kern.ipc.maxsockets
net.inet.ip.intr_queue_maxlen
kern.maxfiles (65535, 16424 as default)
vfs.vmiodirenable (set to 1, 0 is the default)
net.inet.tcp.sendspace (see tuning(7) for more explanations)
net.inet.tcp.recvspace
option NMBCLUSTERS in the kernel (check how many clusters you use with netstat -mb -- don't overtune it, on my busiest webservers the number of used clusters never went above 2256, so 8192 should be enough for all servers), and, of course, maxusers. Well, that era has ended. Right now I discovered a
2827/16384/16384 mbuf clusters in use (current/peak/max)
on a webserver, so I'm gonna bump it to 32k. Also, I'm going to reduce the net.inet.tcp.sendspace from 32k to 16k, because the web traffic means a lot of small files. In case you find out you're just about to run out of NMBCLUSTERS (ex. 6301/8100/8704 mbuf clusters in use (current/peak/max) -- on one of my servers), in case of a webserver, and cannot recompile a new kernel with a bumped NMBCLUSTERS, then set KeepAlive to off in your Apache, and this will save you some clusters by removing many FIN_WAIT_2 connections (More here).
kern.ipc.shm_use_phys -- turn to 1 if the main application of your server uses shared memory, it has improved the activity of my web server.
netstat -f inet can give you valuable informations. For example, a lot of connections with Send-Q != 0 means that your server is storing datas into the mbufs, because the uplink is saturated or (more often) the clients are too 'slow' to receive datas. Non-null values for Recv-Q, on the other hand, means that your server is too slow in serving the requests, which increase the number of mbufs for incoming connections.
net.inet.tcp.msl -- take it down from the default 30,000 to something like 10,000 or even less if you notice too many TIME_WAIT connections in netstat -f inet
net.inet.tcp.inflight_enable -- for bandwidth delay limiting (TCP connections). Read more about in tuning(7).


--------------------------------------------------------------------------------

Firewalling with IPF gave me quite some problems, mostly related to the state table. The customers experienced some broke connections (browser hanging forever when loading up a page), even though the channel was not full (bandwidth-wise). What you can do in a situation like this is play with:

net.inet.ipf.fr_tcpidletimeout=7200 (I like to leave this higher, though, because it kills my idle ssh sessions on the servers as well -- and I hate logging in each other hour or so).
net.inet.ipf.fr_tcpclosewait=120
net.inet.ipf.fr_tcplastack=120
net.inet.ipf.fr_tcptimeout=240
net.inet.ipf.fr_tcpclosed=60
net.inet.ipf.fr_tcphalfclosed=300
net.inet.ipf.fr_udptimeout=90
net.inet.ipf.fr_icmptimeout=35
More about this values here.
You can check how many states are active by looking into the output of ipfstat -s (active). I experienced values growing from 0 to approx. 4000 and then 0 again, which meant the state table got full, and was resetted. You can also carefully increment the number of states in '/usr/include/netinet/ip_state.h' (IPSTATE_SIZE and IPSTATE_MAX -- in my case IPSTATE_MAX was set to 4013, which made the table reset like I said before, at a value approx. equal to 4000). You have to set these two values at resonable values (not too high, don't overtune!), they need to be prime values, and IPSTATE_MAX should be approx. 70% of IPSTATE_SIZE. More infos can be found here.
One other thing that one might do is remove any unnecessary 'keep state's from the firewall configuration. For example, Apache communicates with the clients on port 80 exclusively, so if you 'pass all from any to $my_host port = 80', then you don't need keep states.
As about the NAT using IPFilter, you might consider defining LARGE_NAT in src/contrib/ipfilter/ip_nat.h and src/sys/contrib/ipfilter/netinet/ip_nat.h.
One other problem that I had, having the same source, was that FTP transfers of a zillion+ files would just stall from time to time (after about 100 transferred files). Reason was that the state table was becoming full, and was 'cleaned up' by the kernel, which meant lost state. Freeing the unneeded 'keep states' from the firewall rules, and twaking the parameters above made this problem dissapear as well.
A very nice feature of FreeBSD's kernel is also the DEVICE_POLLING kernel option, which basically means that the system will not treat any interrupt coming from the network cards independently, but rather 'poll' the devices at certain intervals of times. That saves a lot of system activity. You might also consider tweaking with the "option HZ", and also enabling 'kern.polling.enable' and 'kenr.polling.user_frac'. Unfortunately, DEVICE_POLLING works only with certain NICs, but I've experienced very good results with the Intel EtherExpress (fxp). You can see the performance on some snapshots of my firewall here.


--------------------------------------------------------------------------------

Special settings:

In order to run ipf and ipfw on the same machine (ipf for firewall, ipfw for traffic shaping), you can do the following:
ipf -f a_file, where a_file contains something like: "pass out quick proto tcp from x.x.x.x to y.y.y.y port = z flags S keep state"
ipfw add pipe 10 ip from x.x.x.x to y.y.y.y
ipfw pipe 10 config bw 10Kbit/s queue 50KBytes


http://www.nsrc.org/freebsd-tips.html