MACsec on Linux
Starting with kernel 4.6, support for MACsec has been added in Linux so it won’t be needed to use a release candidate to test this feature.
There are two ways to implement MACsec:
- manually configure secure channel(SC), security association(SA) and the keys(this is what we are going to see)
- use dot1x with MACsec extensions that allows dynamic discovery of MACsec peers, SA and SC setup, key generation and distribution
This is the topology that is being used to demonstrate most of the implementation of MACsec on Linux and the purpose is to have connectivity between the two hosts using MACsec.
Between the two hosts there is a L2VPN that is provided by the QFX10K switches.
I won’t discuss how to set up the L2VPN as we already did this several times, one example being L2circuit for L2 protocol tunneling.
On top of this, we want to have additional security at Layer 2 between the two Linux hosts, hence MACsec is the suitable option here.
There are few prerequisites for running MACsec on Linux. I won’t mention here that you need a kernel that supports MACsec:
- add the macsec module in kernel
- install the latest version of iproute2
This is how you perform these two operations
git clone git://git.kernel.org/pub/scm/linux/kernel/git/shemminger/iproute2.git cd iproute2/ ./configure make make install modprobe macsec
So let’s move further with the configuration.
The required steps to configure MACsec are the following:
- create a MACsec device on the physical link over the traffic will be received and sent
- configure a secure association on the MACsec device
- configure a receive channel(you will need to use the peer MAC address as parameter)
- configure a receive association(you will need to use the peer MAC address as parameter)
First we need to know the MAC addresses of the two hosts between which MACsec will be configured. Each host needs to know from what MAC address will receive protected traffic.
This is UBUNTU-1:
root@UBUNTU-1:~# ifconfig eth1 eth1 Link encap:Ethernet HWaddr 56:68:a6:6f:08:d1 inet6 addr: fe80::5468:a6ff:fe6f:8d1/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:114 errors:2 dropped:91 overruns:0 frame:2 TX packets:158 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:30957 (30.9 KB) TX bytes:26724 (26.7 KB) root@UBUNTU-1:~#
And this is UBUNTU-2:
root@UBUNTU-2:~# ifconfig eth1 eth1 Link encap:Ethernet HWaddr 56:68:a6:6f:08:d6 inet6 addr: fe80::5468:a6ff:fe6f:8d6/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:63 errors:2 dropped:36 overruns:0 frame:2 TX packets:163 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:15079 (15.0 KB) TX bytes:27392 (27.3 KB) root@UBUNTU-2:~#
Let’s see the configuration for UBUNTU-1(the last two commands are also adding an IP address on the newly created interface and bring it up so we can test later on the IP reachability between the hosts):
ip link add link eth1 macsec0 type macsec
Creates the MACsec device on eth1 interface
ip macsec add macsec0 tx sa 0 pn 1 on key 01 12345678901234567890123456789012
Configure the transmit secure association, the packet number used as the start ID for the packets sent through this SA and the key.
ip macsec add macsec0 rx address 56:68:a5:c2:37:76 port 1 ip macsec add macsec0 rx address 56:68:a5:c2:37:76 port 1 sa 0 pn 1 on key 02 09876543210987654321098765432109
Configure the receive channel and receive association based on the peer MAC address, the port number, the first packet number expected and the key.
ip link set dev macsec0 up ifconfig macsec0 10.10.12.1/24
These two bring up the interface and configure an IP address on macsec0 interface.
Remember that the transmit SA key has to match the peer’s receive SA key and the other way around.
And this is the configuration for UBUNTU-2:
ip link add link eth1 macsec0 type macsec ip macsec add macsec0 tx sa 0 pn 1 on key 02 09876543210987654321098765432109 ip macsec add macsec0 rx address 56:68:a5:c2:4c:14 port 1 ip macsec add macsec0 rx address 56:68:a5:c2:4c:14 port 1 sa 0 pn 1 on key 01 12345678901234567890123456789012 ip link set dev macsec0 up ifconfig macsec0 10.10.12.2/24
Once the configuration is applied on both sides, you can check the MACsec configuration:
root@UBUNTU-1:~# ip macsec show 8: macsec0: protect on validate strict sc off sa off encrypt off send_sci on end_station off scb off replay off cipher suite: GCM-AES-128, using ICV length 16 TXSC: 5668a5c24c140001 on SA 0 0: PN 12, state on, key 12345678901234567890123456789012 RXSC: 5668a5c237760001, state on 0: PN 12, state on, key 09876543210987654321098765432109 root@UBUNTU-1:~#
As you can see the traffic is authenticated and encrypted by default using AES-GCM-128.
From the above output, some packets protected by MACsec exited and entered this device(“PN 12” shows this, we started at 1).
Let’s send some packets between the two hosts:
root@UBUNTU-1:~# ping 10.10.12.2 -c 3 PING 10.10.12.2 (10.10.12.2) 56(84) bytes of data. 64 bytes from 10.10.12.2: icmp_seq=1 ttl=64 time=24.3 ms 64 bytes from 10.10.12.2: icmp_seq=2 ttl=64 time=20.8 ms 64 bytes from 10.10.12.2: icmp_seq=3 ttl=64 time=19.3 ms --- 10.10.12.2 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2003ms rtt min/avg/max/mdev = 19.367/21.533/24.388/2.106 ms root@UBUNTU-1:~# ip macsec show 8: macsec0: protect on validate strict sc off sa off encrypt off send_sci on end_station off scb off replay off cipher suite: GCM-AES-128, using ICV length 16 TXSC: 5668a5c24c140001 on SA 0 0: PN 15, state on, key 12345678901234567890123456789012 RXSC: 5668a5c237760001, state on 0: PN 15, state on, key 09876543210987654321098765432109 root@UBUNTU-1:~#
As you can see, the packet number increased.
You can also check detailed statistics about the MACsec traffic like this:
root@UBUNTU-1:~# ip -s macsec show 8: macsec0: protect on validate strict sc off sa off encrypt off send_sci on end_station off scb off replay off cipher suite: GCM-AES-128, using ICV length 16 TXSC: 5668a5c24c140001 on SA 0 stats: OutPktsUntagged InPktsUntagged OutPktsTooLong InPktsNoTag InPktsBadTag InPktsUnknownSCI InPktsNoSCI InPktsOverrun 0 0 0 9 0 0 0 0 stats: OutOctetsProtected OutOctetsEncrypted OutPktsProtected OutPktsEncrypted 14 0 1572 0 0: PN 15, state on, key 12345678901234567890123456789012 OutPktsProtected OutPktsEncrypted 14 0 RXSC: 5668a5c237760001, state on stats: InOctetsValidated InOctetsDecrypted InPktsUnchecked InPktsDelayed InPktsOK InPktsInvalid InPktsLate InPktsNotValid InPktsNotUsingSA InPktsUnusedSA 668 0 0 0 6 0 0 0 0 0 0: PN 15, state on, key 09876543210987654321098765432109 InPktsOK InPktsInvalid InPktsNotValid InPktsNotUsingSA InPktsUnusedSA 6 0 0 0 0 root@UBUNTU-1:~#
Two optional features that increase the security on MACsec traffic are encryption and replay protection.
- Encryption – The original payload is encrypted and authenticated
- Replay protection – The packet number of each packet that crossed the MACsec secured link is checked. If there is any packet that arrived out of sequence and the difference between the packet numbers is higher than the replay protection window size, the packet is dropped.
Let’s see how these are configured.
First the encryption:
ip link set macsec0 type macsec encrypt on
Remember that we were at PN 15. Let’s send another 3 packets using ping and then check the statistics:
root@UBUNTU-1:~# ip macsec show 8: macsec0: protect on validate strict sc off sa off encrypt on send_sci on end_station off scb off replay off cipher suite: GCM-AES-128, using ICV length 16 TXSC: 5668a5c24c140001 on SA 0 0: PN 19, state on, key 12345678901234567890123456789012 RXSC: 5668a5c237760001, state on 0: PN 19, state on, key 09876543210987654321098765432109 root@UBUNTU-1:~# ip -s macsec show 8: macsec0: protect on validate strict sc off sa off encrypt on send_sci on end_station off scb off replay off cipher suite: GCM-AES-128, using ICV length 16 TXSC: 5668a5c24c140001 on SA 0 stats: OutPktsUntagged InPktsUntagged OutPktsTooLong InPktsNoTag InPktsBadTag InPktsUnknownSCI InPktsNoSCI InPktsOverrun 0 0 0 23 0 0 0 0 stats: OutOctetsProtected OutOctetsEncrypted OutPktsProtected OutPktsEncrypted 14 4 1572 464 0: PN 19, state on, key 12345678901234567890123456789012 OutPktsProtected OutPktsEncrypted 14 4 RXSC: 5668a5c237760001, state on stats: InOctetsValidated InOctetsDecrypted InPktsUnchecked InPktsDelayed InPktsOK InPktsInvalid InPktsLate InPktsNotValid InPktsNotUsingSA InPktsUnusedSA 668 464 0 0 10 0 0 0 0 0 0: PN 19, state on, key 09876543210987654321098765432109 InPktsOK InPktsInvalid InPktsNotValid InPktsNotUsingSA InPktsUnusedSA 10 0 0 0 0 root@UBUNTU-1:~#
As you can see, we are now at PN 19, which means that actually there were 4 packets that were sent.
Three of them were the ICMP packets and one of them was the ARP Request.
The 4 packets have a total size of 464B. Let’s decompose the ICMP Request packet:
IP – 20B
ICMP – 64B
ICV – 16B
SecTag – 16B
Ethernet – 14
So a total of 130B and this means that 3 ICMP Request packets are 390B, which leave us 74B for the ARP Request packet which is broken down like this:
ARP – 28B
ICV – 16B
SecTag – 16B
Ethernet – 14
Actually doing a tcpdump on UBUNTU-2 while an ICMP Request/Reply was received/sent, you can see that the size is 130B:
root@UBUNTU-2:~# tcpdump -i eth1 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth1, link-type EN10MB (Ethernet), capture size 262144 bytes 07:51:20.389014 56:68:a5:c2:4c:14 (oui Unknown) > 56:68:a5:c2:37:76 (oui Unknown), ethertype Unknown (0x88e5), length 130: 07:51:20.389190 56:68:a5:c2:37:76 (oui Unknown) > 56:68:a5:c2:4c:14 (oui Unknown), ethertype Unknown (0x88e5), length 130: ^C 2 packets captured 2 packets received by filter 0 packets dropped by kernel root@UBUNTU-2:~#
You can also see the MACsec ether-type, 0x88e5.
This is how you can enable the replay protection:
ip link set macsec0 type macsec replay on window 128
You can see that encryption and replay protection are enabled by checking the MACsec configuration:
root@UBUNTU-1:~# ip macsec show 8: macsec0: protect on validate strict sc off sa off encrypt on send_sci on end_station off scb off replay on window 128 cipher suite: GCM-AES-128, using ICV length 16 TXSC: 5668a5c24c140001 on SA 0 0: PN 40, state on, key 12345678901234567890123456789012 RXSC: 5668a5c237760001, state on 0: PN 40, state on, key 09876543210987654321098765432109 root@UBUNTU-1:~#
And this would be the basic configuration that you need to enable MACsec to protect the Layer 2 traffic.
I also tried to enable MACsec on bond links(aggregated interfaces or port-channels how they are named in networking vendors terminology), but I wasn’t able to do it.
In case of bond interfaces, the macsec devices are enslaved instead of the physical links and the macsec devices are created on the physical links. However, I wasn’t allowed to enslave the macsec devices in the bond for some reason.
I hope you found this post useful.
References:
Paris ARAU
Latest posts by Paris ARAU (see all)
- Junos Fusion – Part IV – Satellite policies and uplink failure detection - 30 July 2018
- Junos Fusion – Part III – Satellite commands and traffic forwarding - 16 July 2018
- Junos Fusion – Part II – Configuration, Administration and Operation - 16 July 2018
- Junos Fusion – Part I – Overview, Components, Ports and Software - 11 July 2018
- Vagrant – Part IV – Network topology using Juniper and Cumulus - 26 April 2018
Thank for your guideline about MACsec. I had use MACsec, but it run normal in 2-3 days, after that tunnel not working until I restart. I checked kernel log but there nothing here. Do you have any ideal about this situation ?.
Hi,
What do you mean by restart? Reboot the device? Flap the interface where MACsec is configured? Deactivate/activate the MACsec configuration?
Thanks,
Paris
I think the configuration commands for the RX ports are incorrect – the commands you show don’t refer to the MAC addresses of the transmitting devices at the other end of the link.
Or perhaps I am misunderstanding something.
Otherwise I found this useful in helping to figure out how to configure MACSec on Ubuntu. I was able to get it working between two VM’s on virtualbox.
Hi Eric,
I am glad that you found this useful.
As for the configuration “rx address 56:68:a5:c2:37:76” should reference the oposite device MAC address.
Thanks,
Paris
Hi Paris,
The above example is a great one.
I am currently working on making a Switch port connect to the Linux box.
How does the configuration work between a Macsec capable switch and Linux with Dot1x authentication?
Could you help me with the configuration on the linux?
Thanks,
DhananjaY
Hi Dhananjay,
Unfortunately, I did not try this scenario.
I played some time back with dot1x on Linux, but I do not recall the details.
Thanks,
Paris
Hi,
Is it possible to install MACsec in containers? Can I implement the same in lxc container?
When I run this command, I am getting the “Operation not permitted” error in my container:
>ip macsec add macsec0 tx sa 0 pn 1 on key 01 12345678901234567890123456789012
>RTNETLINK answers: Operation not permitted
Any insights will help me to move forward in my work!
Thanks.
Hi,
No, I did not try this in container.
Thanks,
Paris
Thanks for the quick response.
Can you redirect me to some of the sites or sources in the internet. This would be of great help, if you share.
To use MACsec in a LXC container, you have to configure the lxc profile to use security.priviledged: “true”
For example, run
lxc profile edit default
and set
config:
security.privileged: “true”
Hi,
Is it possible to implement the above MACsec in an LXC container?
When I issue the below command in my LXC container, I am getting an error as “Operation not permitted”:
>root@lxc-container:~# ip macsec add macsec0 tx sa 0 pn 1 on key 01 12345678901234567890123456789012
>RTNETLINK answers: Operation not permitted
Also I am not able to do ‘modprobe macsec’ which also results an error as “FATAL: Module macsec not found in directory”.
What am I doing wrong? Any insights on this will be helpful for me to proceed forward.
Thanks.
This is a great example of usage of MACsec. I am curious, does Ubuntu already supports the offload to hardware supporting MACsec?
Hi,
Thanks for this wonderful and helpful tutorial, i was able to connect two VMs and send macsec traffic from macsec interfaces.
I observed we did not do insmod of macsec driver anywhere on the above tutorial and still it worked, whereas i see linux macsec driver is built as a module and is not loaded on bootup. Could you please let me know how it works? does ip tool takes care of inserting the module?
Thanks,
Kshitij
Outstanding! of the few posts I’ve found on MACSec for Linux, this is the only one to have worked for me “out of the box”.
How is the window size calculated as 128 in the command to enable replay protection?
ip link set macsec0 type macsec replay on window 128