u3g Sierra Wireless LTE device disconnects ppp after 2-3MB of outgoing traffic

[Adrian Gschwend]

Hi everyone,

I brought this up on opnsense forum yesterday
(https://forum.opnsense.org/index.php?topic=15187.0) but I start to
suspect that it's related to mpd or the FreeBSD driver of my LTE Sierra
Wireless card so allow me to ask here as well.

What I describe here I tried with the latest version of OPNsense
(19.7.7) and PFsense (2.4.4). IIRC that is FreeBSD 12.1 or 11.3.

I have a "Sierra Wireless MC7430 Qualcomm Snapdragon X7 LTE-A" card in
my apu3b4 board with 2 sim card slots. The card is recognized just fine
and I can do AT commands on /dev/cuaU0.2.

I've set up PPP for my provider and I can connect without problems, I
get a private V4 address and no IPv6 address.

Download is ok, between 20-30MB/sec, pretty much what I get with my
Netgear AirCard 810S Mobile Hotspot as well using the same SIM card.

However, there seems to be a problem with outgoing traffic. It starts
all fine but at one point upload stalls and the connection dies, also
for incoming traffic. Took me a while to figure out a pattern but I'm
pretty sure it depends on how much outgoing traffic I generate on the
PPP interface. From monitoring the interface it looks like after 2-3MB
of traffic, the connection dies. When I wait long enough it recovers and
I can reproduce the behavior, every 2-3MB of outgoing traffic.

In ppp.log I see this the moment the link stalls:
https://pastebin.com/tA6kystN

system.log:

Dec  3 01:28:12 bidul opnsense: /usr/local/etc/rc.newwanip: IP renewal
is starting on 'ppp0'
Dec  3 01:28:12 bidul opnsense: /usr/local/etc/rc.newwanip: On (IP
address: 10.131.180.114) (interface: WAN[wan]) (real interface: ppp0).
Dec  3 01:28:12 bidul opnsense: plugins_configure hosts ()
Dec  3 01:28:12 bidul opnsense: /usr/local/etc/rc.newwanip: ROUTING:
entering configure using 'wan'
Dec  3 01:28:12 bidul opnsense: /usr/local/etc/rc.newwanip: ROUTING:
IPv4 default gateway set to wan
Dec  3 01:28:12 bidul opnsense: /usr/local/etc/rc.newwanip: ROUTING:
setting IPv4 default route to 10.64.64.0
Dec  3 01:28:12 bidul opnsense: /usr/local/etc/rc.newwanip: ROUTING:
keeping current default gateway '10.64.64.0'
Dec  3 01:28:12 bidul opnsense: /usr/local/etc/rc.newwanip: ROUTING:
IPv6 default gateway set to wan
Dec  3 01:28:12 bidul opnsense: /usr/local/etc/rc.newwanip: ROUTING:
skipping IPv6 default route
Dec  3 01:28:12 bidul opnsense: plugins_configure monitor ()
Dec  3 01:28:12 bidul opnsense: /usr/local/etc/rc.newwanip: The
WAN_DHCP6 monitor address is empty, skipping.
Dec  3 01:28:12 bidul opnsense: /usr/local/etc/rc.newwanip: The WAN_PPP
monitor address is empty, skipping.

As a side-note, ICMP behaves really strange on both opnsense and pfsense
with this interface: Sometimes it works towards the Internet, sometimes
it doesn't. If it did work, it is most probably broken after I had the
first disconnect and won't come back. TCP does not seem to be affected
though, even without being able to ping I can use http just fine.

I'm a bit lost on how to debug that, any hints are welcome. Is this a
PPP problem or more likely an issue with the u3g driver?

FWIW I've also updated the firmware on my LTE card, it was almost 3
years old. But same behavior with the latest release. LTE Signal is in
general very good, MIMO antenna attached to it. All values are in Good
or Excellent range (tx & rx).

thanks

Adrian