DPDK is written in C and it is expected to be used in C environments. Yes I know, there are exports ready for C++ usage all over the lib, but if you ever tried to include DPDK code into your C++ application then you’ve already discovered that’s actually quite tricky to build DPDK with C++ toolchains. I mean don’t get me wrong, I’m not saying that’s not possible but only that’s very annoying.
So, what’ I’m going present you here is a solution which do not require too much mixing with the makefiles (or MakeLists) and still allow you to benefit of this powerful packet processing library in you C++ code. I’ll need to make things extremely simple for the sake of this explanation, but don’t worry the simple example can scale quite well to big projects… (Yeah, sure..).
Alright, let’s say we have those functions in our DPDK LIB:
nic_port nic_ports[MAX_NUMBER_OF_NIC_PORT];
uint8_t init_dpdk_capture(
int argc,
char** argv
)
{
UNUSED( argc );
int ret = rte_eal_init( 1, argv );
if( ret < 0 ) {
err( "Error while initializing EAL, error: \'%s\'",
rte_strerror( rte_errno ) );
return 0;
}
uint8_t nic_count = rte_eth_dev_count();
vlog( "Discovered %d NIC ports.",
nic_count );
uint8_t nic = 0;
for( nic = 0 ; nic < nic_count ; ++nic ) {
collect_nic_port_data( nic );
}
return nic_count;
}
p_nic_port collect_nic_port_data( const uint8_t port_id )
{
if( rte_eth_dev_is_valid_port( port_id ) == 0 ) {
err( "NIC \'%"PRIu8"\' is not a valid port!", port_id );
return NULL;
}
p_nic_port nic = &nic_ports[port_id];
rte_eth_dev_info_get( port_id, &nic->dev_info );
rte_eth_dev_get_name_by_port( port_id, nic->pci_dev_name );
if( nic->dev_info.if_index != 0 ) {
if_indextoname( nic->dev_info.if_index,
nic->if_name );
}
vlog( "Discovered NIC: PCI device name:\'%s\', driver name:\'%s\', if name:\'%s\'",
nic->pci_dev_name,
nic->dev_info.driver_name,
nic->if_name );
return nic;
}
p_nic_port get_nic_data( const uint8_t port_id )
{
if( port_id >= MAX_NUMBER_OF_NIC_PORT ) {
return NULL;
}
return &nic_ports[port_id];
}
The struct nic_port is just a small structure that supposedly is going to contain all the NIC port informations, again, this is just a banal example please don’t blame me for writing useless code, probably I should have avoided error checking at all in this piece to avoid confusion.. Oh man, confusion is the worst thing ever when writing code.. Whatever… This is the nic_port structure:
typedef struct nic_port_t {
uint8_t port_id;
struct rte_eth_dev_info dev_info;
char pci_dev_name[RTE_ETH_NAME_MAX_LEN];
char if_name[IF_NAMESIZE];
} nic_port, *p_nic_port;
Cool. This code can be compiled using DPDK makefiles quite easily, I can add a main entry and test this functionality (by calling init_dpdk_capture, in case you can’t guess it youself :), on my machine the output is:
EAL: Detected 8 lcore(s) EAL: Multi-process socket /var/run/.rte_unix EAL: Probing VFIO support... EAL: PCI device 0000:06:00.0 on NUMA socket -1 EAL: Invalid NUMA socket, default to 0 EAL: probe driver: 15b3:1015 net_mlx5 PMD: net_mlx5: PCI information matches, using device "mlx5_0" (SR-IOV: false) PMD: net_mlx5: 1 port(s) detected PMD: net_mlx5: MPS is enabled PMD: net_mlx5: Reserved UAR address space: 0x7f6780000000 PMD: net_mlx5: port 1 MAC address is cc:ae:95:51:49:2a EAL: PCI device 0000:06:00.1 on NUMA socket -1 EAL: Invalid NUMA socket, default to 0 EAL: probe driver: 15b3:1015 net_mlx5 PMD: net_mlx5: PCI information matches, using device "mlx5_1" (SR-IOV: false) PMD: net_mlx5: 1 port(s) detected PMD: net_mlx5: MPS is enabled PMD: net_mlx5: port 1 MAC address is cc:ae:d0:ee:7d:fa OUT(0): Discovered 2 NIC ports. OUT(0): Discovered NIC: PCI device name:'0000:06:00.0 port 0', driver name:'net_mlx5', if name:'enp6s0f0' OUT(0): Discovered NIC: PCI device name:'0000:06:00.1 port 0', driver name:'net_mlx5', if name:'enp6s0f1'
Which is the basic EAL and PMD init output plus a couple of additional prints. To build this code as a lib you should use the makefile /mk/rte.lib.mk from your DPDK installation directory, there’s nothing really fancy here just read THIS and you should good to go. well.. no.. The point of all this post is that the instruction on the DPDK web page are not completely honest on the amount of stuff to do in order to make this thing work, let’s say we have the following C++ application:
extern "C" {
#include "your_dpdk_code.h"
}
#include <iostream>
int main( int argc, char** argv)
{
int nic_count = init_dpdk_capture( argc, argv );
std::cout<<"DPDK initialized, number of NICs "<<nic_count<<std::endl;
for( int nic{0} ; nic < nic_count ; ++nic ){
auto nic_data = get_nic_data(nic);
std::cout<<"Found "<if_name <<" mapped to port "<<unsigned(nic_data->port_id)<<std::endl;
}
return 0;
}
And, your bare bone makefile would look probably like this:
all: g++ -std=c++11 -I../dpdk-18.02/x86_64-native-linuxapp-gcc/include -L../build/lib -L../dpdk-18.02/x86_64-native-linuxapp-gcc/lib main.cpp -lpthread -ldl -lnuma -lrte_eal -lyour_dpdk_lib -ldpdk -o test
To make it work you should use some proper paths, also here I’m assuming that the name of the LIB created using the DPDK toolkit is your_dpdk_lib. Now, this makefile is perfectly fine and can compile our powerful C++ application, sadly the output of it’s execution is a bit disappointing:
EAL: Detected 8 lcore(s) EAL: Multi-process socket /var/run/.rte_unix EAL: Probing VFIO support... OUT(0): Discovered 0 NIC ports. DPDK initialized, number of NICs 0
Where’s all the EAL and PMD stuff? No NIC was discovered! The problem is banal, but also very painful to debug if you don’t read my blog, as probably you’ll be spending quite some time asking yourself why the included DPDK lib does not initialize properly EAL and PMD.
The issue here is that DPDK use a bunch of constructor functions to initialize all it’s stuff before main is even executed, those constructors are not referenced directly in any of the code above so the linker assume that those are not needed and remove them from the binary. Shit storm, right?
Long story short, you need to include in your hand made makefile the linker flags that will force the inclusion of all the DPDK stuff:
-Wl,--whole-archive -ldpdk -Wl,--no-whole-archive -lmlx5 -libverbs
Which will include all of the DPDK stuff into your binary, most probably you might want to fine tune the inclusion to only the piece of the lib that you’re actually using; Also, I’m adding mlx5 and libverbs that are needed to compile all the code above.
With the new makefile, the output of ./test is correct:
DPDK initialized, number of NICs 2 Found enp6s0f0 mapped to port 0 Found enp6s0f1 mapped to port 1
I’m removing from the snipped all the prints from EAL and PMD which I have already copy-pasted earlier.
Cool, now you can include the DPDK libs into your C++ application with really no effort, just a simple modification of your Makefile should work.