Friday, August 28, 2015

Failed to open ethX: pfring_open error


This is a blogpost about getting around the following error when using Suricata with pfring:

(source-pfring.c:444) <Error> (ReceivePfringThreadInit) -- [ERRCODE: SC_ERR_PF_RING_OPEN(34)] - Failed to open eth2: pfring_open error. Check if eth2 exists and pf_ring module is loaded.
(tmqh-packetpool.c:394) <Info> (PacketPoolInit) -- preallocated 65534 packets. Total memory 230679680
pfring_set_channel_id() failed: -1

However in my case eth2 existed, was up and running and the pfring module was loaded. So what happened in a bit more detail below :

I experienced this after a git pull update/upgrade of Suricata (latest at the moment of this writing) and after I re compiled pfring (using the latest pfring from git (https://github.com/ntop/PF_RING.git).

My set up (linux Debian/Ubuntu like systems):

root@suricata:/var/data/log/suricata# ifconfig eth2
eth2      Link encap:Ethernet  HWaddr 00:e0:ed:19:e3:e0
          inet6 addr: fe80::2e0:edff:fe19:e3e0/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:2962266192 errors:0 dropped:5527381 overruns:0 frame:0
          TX packets:19 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:2867936692537 (2.8 TB)  TX bytes:3345 (3.3 KB)
The pfring set up I had was configured like this below:
root@suricata:/var/data/log/suricata# modprobe pf_ring transparent_mode=0 min_num_slots=65534

A regular check reveals nothing abnormal:
root@suricata:/var/data/log/suricata# modinfo pf_ring && cat /proc/net/pf_ring/info
filename:       /lib/modules/3.14.0-031400-generic/kernel/net/pf_ring/pf_ring.ko
alias:          net-pf-27
description:    Packet capture acceleration and analysis
author:         ntop.org
license:        GPL
srcversion:     E344EB01757B55E97A93D0C
depends:     
vermagic:       3.14.0-031400-generic SMP mod_unload modversions
parm:           min_num_slots:Min number of ring slots (uint)
parm:           perfect_rules_hash_size:Perfect rules hash size (uint)
parm:           transparent_mode:(deprecated) (uint)
parm:           enable_debug:Set to 1 to enable PF_RING debug tracing into the syslog (uint)
parm:           enable_tx_capture:Set to 1 to capture outgoing packets (uint)
parm:           enable_frag_coherence:Set to 1 to handle fragments (flow coherence) in clusters (uint)
parm:           enable_ip_defrag:Set to 1 to enable IP defragmentation(only rx traffic is defragmentead) (uint)
parm:           quick_mode:Set to 1 to run at full speed but with upto one socket per interface (uint)
PF_RING Version          : 6.1.1 (dev:250a67fe1082121ac511a19ebc3fe1fc5f494bfe)
Total rings              : 0

Standard (non DNA/ZC) Options
Ring slots               : 65534
Slot version             : 16
Capture TX               : Yes [RX+TX]
IP Defragment            : No
Socket Mode              : Standard
Total plugins            : 0
Cluster Fragment Queue   : 0
Cluster Fragment Discard : 0
Suricata and pfring have been installed as explained here - on the Suricata redmine wiki.
root@suricata:~# ldd /usr/local/bin/suricata
    linux-vdso.so.1 =>  (0x00007fff419fe000)
    libhtp-0.5.17.so.1 => /usr/local/lib/libhtp-0.5.17.so.1 (0x00007f32af5a1000)
    libGeoIP.so.1 => /usr/lib/x86_64-linux-gnu/libGeoIP.so.1 (0x00007f32af372000)
    libluajit-5.1.so.2 => /usr/local/lib/libluajit-5.1.so.2 (0x00007f32af103000)
    libmagic.so.1 => /usr/lib/x86_64-linux-gnu/libmagic.so.1 (0x00007f32aeee7000)
    libcap-ng.so.0 => /usr/local/lib/libcap-ng.so.0 (0x00007f32aece2000)
    libpfring.so => /usr/local/lib/libpfring.so (0x00007f32aeaa3000)
    libpcap.so.1 => /usr/local/pfring/lib/libpcap.so.1 (0x00007f32ae80e000)
    libnet.so.1 => /usr/lib/x86_64-linux-gnu/libnet.so.1 (0x00007f32ae5f5000)
    libjansson.so.4 => /usr/lib/x86_64-linux-gnu/libjansson.so.4 (0x00007f32ae3e8000)
    libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f32ae1ca000)
    libyaml-0.so.2 => /usr/lib/x86_64-linux-gnu/libyaml-0.so.2 (0x00007f32adfaa000)
    libpcre.so.3 => /lib/x86_64-linux-gnu/libpcre.so.3 (0x00007f32add6b000)
    libnss3.so => /usr/lib/x86_64-linux-gnu/libnss3.so (0x00007f32ada31000)
    libnspr4.so => /usr/lib/x86_64-linux-gnu/libnspr4.so (0x00007f32ad7f4000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f32ad42e000)
    libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f32ad215000)
    libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f32acf0f000)
    libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f32acd0a000)
    libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f32acaf4000)
    /lib64/ld-linux-x86-64.so.2 (0x00007f32af7d4000)
    libnuma.so.1 => /usr/lib/x86_64-linux-gnu/libnuma.so.1 (0x00007f32ac8e9000)
    librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f32ac6e0000)
    libnssutil3.so => /usr/lib/x86_64-linux-gnu/libnssutil3.so (0x00007f32ac4b5000)
    libplc4.so => /usr/lib/x86_64-linux-gnu/libplc4.so (0x00007f32ac2b0000)
    libplds4.so => /usr/lib/x86_64-linux-gnu/libplds4.so (0x00007f32ac0ab000)


Further more my Suricata start line was like this:

suricata --pfring-int=eth2 --pfring-cluster-id=99 --pfring-cluster-type=cluster_flow -c /etc/suricata/peter-yaml/suricata-pfring.yaml --pidfile /var/run/suricata.pid -v

Even though everything seems fine - I could  not start Suricata with pfring:

[31591] 5/8/2015 -- 17:10:31 - (tmqh-packetpool.c:394) <Info> (PacketPoolInit) -- preallocated 65534 packets. Total memory 230679680
pfring_set_channel_id() failed: -1
[31591] 5/8/2015 -- 17:10:31 - (source-pfring.c:444) <Error> (ReceivePfringThreadInit) -- [ERRCODE: SC_ERR_PF_RING_OPEN(34)] - Failed to open eth2: pfring_open error. Check if eth2 exists and pf_ring module is loaded.
[31592] 5/8/2015 -- 17:10:31 - (tmqh-packetpool.c:394) <Info> (PacketPoolInit) -- preallocated 65534 packets. Total memory 230679680
pfring_set_channel_id() failed: -1
[31592] 5/8/2015 -- 17:10:31 - (source-pfring.c:444) <Error> (ReceivePfringThreadInit) -- [ERRCODE: SC_ERR_PF_RING_OPEN(34)] - Failed to open eth2: pfring_open error. Check if eth2 exists and pf_ring module is loaded.
[31593] 5/8/2015 -- 17:10:32 - (tmqh-packetpool.c:394) <Info> (PacketPoolInit) -- preallocated 65534 packets. Total memory 230679680
pfring_set_channel_id() failed: -1
[31593] 5/8/2015 -- 17:10:32 - (source-pfring.c:444) <Error> (ReceivePfringThreadInit) -- [ERRCODE: SC_ERR_PF_RING_OPEN(34)] - Failed to open eth2: pfring_open error. Check if eth2 exists and pf_ring module is loaded.
[31594] 5/8/2015 -- 17:10:32 - (tmqh-packetpool.c:394) <Info> (PacketPoolInit) -- preallocated 65534 packets. Total memory 230679680
pfring_set_channel_id() failed: -1
[31594] 5/8/2015 -- 17:10:32 - (source-pfring.c:444) <Error> (ReceivePfringThreadInit) -- [ERRCODE: SC_ERR_PF_RING_OPEN(34)] - Failed to open eth2: pfring_open error. Check if eth2 exists and pf_ring module is loaded.
....

I was getting that error even though I reloaded the pfring module:
rmmod pr_ring
modprobe pf_ring transparent_mode=0 min_num_slots=65534
the way I usually do...

In short - this is the fix:

LD_LIBRARY_PATH=/usr/local/pfring/lib suricata --pfring-int=eth2  --pfring-cluster-id=99 --pfring-cluster-type=cluster_flow  -c /etc/suricata/peter-yaml/suricata-pfring.yaml --pidfile /var/run/suricata.pid -v

Notice the use of:
LD_LIBRARY_PATH=/usr/local/pfring/lib suricata 

More information about what is LD_LIBRARY_PATH     

To get rid of LD_LIBRARY_PATH you can create a pfring.conf file in /etc/ld.so.conf.d/ containing:
/usr/local/pfring/lib
and run
sudo ldconfig




7 comments:

  1. I have do it ,but it still get pfring_open error. can you help me.

    31/3/2016 -- 14:55:49 - - stats output device (regular) initialized: stats.log
    31/3/2016 -- 14:55:49 - - Using flow cluster mode for PF_RING (iface p5p1@0)
    31/3/2016 -- 14:55:49 - - Going to use 1 thread(s)
    31/3/2016 -- 14:55:49 - - preallocated 1024 packets. Total memory 3606528
    31/3/2016 -- 14:55:49 - - Enabling zero-copy for p5p1@0
    dna_init() failed
    31/3/2016 -- 14:55:49 - - [ERRCODE: SC_ERR_PF_RING_OPEN(34)] - Failed to open p5p1@0: pfring_open error. Check if p5p1@0 exists and pf_ring module is loaded.
    31/3/2016 -- 14:55:49 - - Using flow cluster mode for PF_RING (iface p5p1@1)
    31/3/2016 -- 14:55:49 - - Going to use 1 thread(s)
    31/3/2016 -- 14:55:49 - - preallocated 1024 packets. Total memory 3606528
    31/3/2016 -- 14:55:49 - - Enabling zero-copy for p5p1@1
    dna_init() failed

    ReplyDelete
  2. it's good work with pfcount

    [root@localhost examples]# ./pfcount -i p5p1
    Using PF_RING v.6.2.0
    Capturing from p5p1 [00:1B:21:90:F6:54][ifIndex: 12]
    # Device RX channels: 32
    # Polling threads: 1
    Dumping statistics on /proc/net/pf_ring/stats/2926-p5p1.90
    =========================
    Absolute Stats: [2'186 pkts total][0 pkts dropped][0.0% dropped]
    [2'186 pkts rcvd][790'053 bytes rcvd]
    =========================

    =========================
    Absolute Stats: [4'657 pkts total][0 pkts dropped][0.0% dropped]
    [4'657 pkts rcvd][1'611'315 bytes rcvd][4'656.45 pkt/sec][12.89 Mbit/sec]
    =========================
    Actual Stats: [2'471 pkts rcvd][1'000.11 ms][2'470.71 pps][0.01 Gbps]

    ReplyDelete
  3. Do you use pfring zc ? If so I think you need to specify that in the suricata.yaml pfring section as so - "zc:p5p1@0"..."zc:p5p1@1"...etc (per thread)

    ReplyDelete
    Replies
    1. yeah, I have try,but it does't work.
      my pfring is 6.2 DNA.
      NetworkInterfaceCard is Intel 82599,
      but suricata identify the dirver is [Intel 1 Gbit e1000e family].
      pfcount identify is ixgbe 82599-based.


      1/4/2016 -- 14:57:14 - - Using flow cluster mode for PF_RING (iface p5p1)
      1/4/2016 -- 14:57:14 - - Going to use 1 thread(s)
      1/4/2016 -- 14:57:14 - - preallocated 1024 packets. Total memory 3606528
      1/4/2016 -- 14:57:14 - - Enabling zero-copy for p5p1
      ###################################################
      # ERROR: You do not seem to have a valid DNA license for p5p1 [Intel 1 Gbit e1000e family].
      # We're now working in demo mode with packet capture
      # and transmission limited to 0 day(s) 00:05:00
      ###################################################
      dna_init() failed
      1/4/2016 -- 14:57:14 - - [ERRCODE: SC_ERR_PF_RING_OPEN(34)] - Failed to open p5p1: pfring_open error. Check if p5p1 exists and pf_ring module is loaded.
      1/4/2016 -- 14:57:14 - - Using flow cluster

      Delete
  4. Seems dont have a license. If you try the non ZC pfring and it works ok - then that could be the reason.

    ReplyDelete
    Replies
    1. Thx,I roll back the pf_ring from 6.2 to 6.0.2,the pfring_open error was Solution,but can't set PFRING Cluster-id . At last, I change the OS from centos 6.5 to ubuntu 14.04. all the thing is ok.

      Delete
  5. You might want to report a reproducible case for that to the pfring/ntop support channels/mailing list(if you consider).

    ReplyDelete