Linux netmap memory allocation

[Charlie Smurthwaite]

Hi,

To answer my own question, I have resolved this, there were seemingly 3
errors in my use of the API:

1) I did not initialize the req struct. Some uninitialized data were
being passed to the ioctl. I feel silly.
2) Values greater then 1 do not seem to be accepted by nr_arg2, setting
this to 1, or to 0 (which resolves to 1) works.
3) When fixing the above, the memory size is always 343019520, plenty
for my requirement.

Thanks,

Charlie



On 28/12/17 17:34, Charlie Smurthwaite wrote:
> Hi,
>
> I'm just starting to use netmap and it is my intention to do zero-copy
> forwarding of frames between a large number of NICs. I am using Intel
> i350 (igb) on Linux. I therefore require a large memory area for rings
> and buffers.
>
> My calculation:
> 32 NICs * 2 rings (TX+RX) * 256 frames * 2048 bytes = 32MB
>
> I am currently having a problem (or perhaps just a misunderstanding)
> regarding allocation of this memory. I am attempting to use the
> following code:
>
> void thread_main(int thread_id) {
>   struct nmreq req; // A struct for the netmap request
>   int fd;           // File descriptor for netmap socket
>   void * mem;       // Pointer to allocated memory area
>
>   fd = open("/dev/netmap", 0);     // Open a generic netmap socket
>   strcpy(req.nr_name, "enp8s0f0"); // Copy NIC name into request
>   req.nr_version = NETMAP_API;     // Set version number
>   req.nr_flags = NR_REG_ONE_NIC;   // We will be using a single hw ring
>
>   // Select ring 0, disable TX on poll
>   req.nr_ringid = NETMAP_NO_TX_POLL | NETMAP_HW_RING | 0;
>
>   // Ask for 64 additional rings to be allocated (32 * (TX+RX))
>   req.nr_arg1 = 64;
>
>   // Allocate a separate memory area for each thread
>   req.nr_arg2 = 10 + thread_id;
>
>   // Ask for additional buffers (256 per ring)
>   req.nr_arg3 = 64*256;
>
>   // Initialize port
>   ioctl(fd, NIOCREGIF, &req);
>
>   // Check the allocated memory size
>   printf("memsize: %u\n", req.nr_memsize);
>   // Check the allocated memory area
>   printf("nr_arg2: %u\n", req.nr_arg2);
> }
>
> The output is as follows:
>
> memsize: 4206859
> nr_arg2: 10
>
> This is far short of the amount of memory I am hoping to be allocated.
> Am I doing something wrong, or is this simply an indication that the
> driver is unwilling to allocate more than 4MB?
>
> A secondary (related) problem is that if I don't set arg1,arg2,arg3 in
> my code (ie they will be zero), then I get varying output (it varies
> between each of the following):
>
> memsize: 4206843
> nr_arg2: 0
>
> memsize: 343019520
> nr_arg2: 1
>
> Any pointers would be appreciated. Thanks!
>
> Charlie



Charlie Smurthwaite
Technical Director

tel.  email. charlie at atech.media<mailto:charlie at atech.media> web. https://atech.media

This e-mail has been sent by aTech Media Limited (or one of its assoicated group companys, Dial 9 Communications Limited or Viaduct Hosting Limited). Its contents are confidential therefore if you have received this message in error, we would appreciate it if you could let us know and delete the message. aTech Media Limited is a UK limited company, registration number 5523199. Dial 9 Communications Limited is a UK limited company, registration number 7740921. Viaduct Hosting Limited is a UK limited company, registration number 8514362. All companies are registered at Unit 9 Winchester Place, North Street, Poole, Dorset, BH15 1NX.