diff options
| author | Linus Torvalds <torvalds@linux-foundation.org> | 2022-10-04 13:38:03 -0700 |
|---|---|---|
| committer | Linus Torvalds <torvalds@linux-foundation.org> | 2022-10-04 13:38:03 -0700 |
| commit | 0326074ff4652329f2a1a9c8685104576bd8d131 (patch) | |
| tree | 9a7574c7ccb05bf4c7cb34fc5a65457bb8f495cb /drivers | |
| parent | 522667b24f08009591c90e75bfe2ffb67f555498 (diff) | |
| parent | 681bf011b9b5989c6e9db6beb64494918aab9a43 (diff) | |
| download | linux-0326074ff4652329f2a1a9c8685104576bd8d131.tar.gz | |
Merge tag 'net-next-6.1' of git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next
Pull networking updates from Jakub Kicinski:
"Core:
- Introduce and use a single page frag cache for allocating small skb
heads, clawing back the 10-20% performance regression in UDP flood
test from previous fixes.
- Run packets which already went thru HW coalescing thru SW GRO. This
significantly improves TCP segment coalescing and simplifies
deployments as different workloads benefit from HW or SW GRO.
- Shrink the size of the base zero-copy send structure.
- Move TCP init under a new slow / sleepable version of DO_ONCE().
BPF:
- Add BPF-specific, any-context-safe memory allocator.
- Add helpers/kfuncs for PKCS#7 signature verification from BPF
programs.
- Define a new map type and related helpers for user space -> kernel
communication over a ring buffer (BPF_MAP_TYPE_USER_RINGBUF).
- Allow targeting BPF iterators to loop through resources of one
task/thread.
- Add ability to call selected destructive functions. Expose
crash_kexec() to allow BPF to trigger a kernel dump. Use
CAP_SYS_BOOT check on the loading process to judge permissions.
- Enable BPF to collect custom hierarchical cgroup stats efficiently
by integrating with the rstat framework.
- Support struct arguments for trampoline based programs. Only
structs with size <= 16B and x86 are supported.
- Invoke cgroup/connect{4,6} programs for unprivileged ICMP ping
sockets (instead of just TCP and UDP sockets).
- Add a helper for accessing CLOCK_TAI for time sensitive network
related programs.
- Support accessing network tunnel metadata's flags.
- Make TCP SYN ACK RTO tunable by BPF programs with TCP Fast Open.
- Add support for writing to Netfilter's nf_conn:mark.
Protocols:
- WiFi: more Extremely High Throughput (EHT) and Multi-Link Operation
(MLO) work (802.11be, WiFi 7).
- vsock: improve support for SO_RCVLOWAT.
- SMC: support SO_REUSEPORT.
- Netlink: define and document how to use netlink in a "modern" way.
Support reporting missing attributes via extended ACK.
- IPSec: support collect metadata mode for xfrm interfaces.
- TCPv6: send consistent autoflowlabel in SYN_RECV state and RST
packets.
- TCP: introduce optional per-netns connection hash table to allow
better isolation between namespaces (opt-in, at the cost of memory
and cache pressure).
- MPTCP: support TCP_FASTOPEN_CONNECT.
- Add NEXT-C-SID support in Segment Routing (SRv6) End behavior.
- Adjust IP_UNICAST_IF sockopt behavior for connected UDP sockets.
- Open vSwitch:
- Allow specifying ifindex of new interfaces.
- Allow conntrack and metering in non-initial user namespace.
- TLS: support the Korean ARIA-GCM crypto algorithm.
- Remove DECnet support.
Driver API:
- Allow selecting the conduit interface used by each port in DSA
switches, at runtime.
- Ethernet Power Sourcing Equipment and Power Device support.
- Add tc-taprio support for queueMaxSDU parameter, i.e. setting per
traffic class max frame size for time-based packet schedules.
- Support PHY rate matching - adapting between differing host-side
and link-side speeds.
- Introduce QUSGMII PHY mode and 1000BASE-KX interface mode.
- Validate OF (device tree) nodes for DSA shared ports; make
phylink-related properties mandatory on DSA and CPU ports.
Enforcing more uniformity should allow transitioning to phylink.
- Require that flash component name used during update matches one of
the components for which version is reported by info_get().
- Remove "weight" argument from driver-facing NAPI API as much as
possible. It's one of those magic knobs which seemed like a good
idea at the time but is too indirect to use in practice.
- Support offload of TLS connections with 256 bit keys.
New hardware / drivers:
- Ethernet:
- Microchip KSZ9896 6-port Gigabit Ethernet Switch
- Renesas Ethernet AVB (EtherAVB-IF) Gen4 SoCs
- Analog Devices ADIN1110 and ADIN2111 industrial single pair
Ethernet (10BASE-T1L) MAC+PHY.
- Rockchip RV1126 Gigabit Ethernet (a version of stmmac IP).
- Ethernet SFPs / modules:
- RollBall / Hilink / Turris 10G copper SFPs
- HALNy GPON module
- WiFi:
- CYW43439 SDIO chipset (brcmfmac)
- CYW89459 PCIe chipset (brcmfmac)
- BCM4378 on Apple platforms (brcmfmac)
Drivers:
- CAN:
- gs_usb: HW timestamp support
- Ethernet PHYs:
- lan8814: cable diagnostics
- Ethernet NICs:
- Intel (100G):
- implement control of FCS/CRC stripping
- port splitting via devlink
- L2TPv3 filtering offload
- nVidia/Mellanox:
- tunnel offload for sub-functions
- MACSec offload, w/ Extended packet number and replay window
offload
- significantly restructure, and optimize the AF_XDP support,
align the behavior with other vendors
- Huawei:
- configuring DSCP map for traffic class selection
- querying standard FEC statistics
- querying SerDes lane number via ethtool
- Marvell/Cavium:
- egress priority flow control
- MACSec offload
- AMD/SolarFlare:
- PTP over IPv6 and raw Ethernet
- small / embedded:
- ax88772: convert to phylink (to support SFP cages)
- altera: tse: convert to phylink
- ftgmac100: support fixed link
- enetc: standard Ethtool counters
- macb: ZynqMP SGMII dynamic configuration support
- tsnep: support multi-queue and use page pool
- lan743x: Rx IP & TCP checksum offload
- igc: add xdp frags support to ndo_xdp_xmit
- Ethernet high-speed switches:
- Marvell (prestera):
- support SPAN port features (traffic mirroring)
- nexthop object offloading
- Microchip (sparx5):
- multicast forwarding offload
- QoS queuing offload (tc-mqprio, tc-tbf, tc-ets)
- Ethernet embedded switches:
- Marvell (mv88e6xxx):
- support RGMII cmode
- NXP (felix):
- standardized ethtool counters
- Microchip (lan966x):
- QoS queuing offload (tc-mqprio, tc-tbf, tc-cbs, tc-ets)
- traffic policing and mirroring
- link aggregation / bonding offload
- QUSGMII PHY mode support
- Qualcomm 802.11ax WiFi (ath11k):
- cold boot calibration support on WCN6750
- support to connect to a non-transmit MBSSID AP profile
- enable remain-on-channel support on WCN6750
- Wake-on-WLAN support for WCN6750
- support to provide transmit power from firmware via nl80211
- support to get power save duration for each client
- spectral scan support for 160 MHz
- MediaTek WiFi (mt76):
- WiFi-to-Ethernet bridging offload for MT7986 chips
- RealTek WiFi (rtw89):
- P2P support"
* tag 'net-next-6.1' of git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next: (1864 commits)
eth: pse: add missing static inlines
once: rename _SLOW to _SLEEPABLE
net: pse-pd: add regulator based PSE driver
dt-bindings: net: pse-dt: add bindings for regulator based PoDL PSE controller
ethtool: add interface to interact with Ethernet Power Equipment
net: mdiobus: search for PSE nodes by parsing PHY nodes.
net: mdiobus: fwnode_mdiobus_register_phy() rework error handling
net: add framework to support Ethernet PSE and PDs devices
dt-bindings: net: phy: add PoDL PSE property
net: marvell: prestera: Propagate nh state from hw to kernel
net: marvell: prestera: Add neighbour cache accounting
net: marvell: prestera: add stub handler neighbour events
net: marvell: prestera: Add heplers to interact with fib_notifier_info
net: marvell: prestera: Add length macros for prestera_ip_addr
net: marvell: prestera: add delayed wq and flush wq on deinit
net: marvell: prestera: Add strict cleanup of fib arbiter
net: marvell: prestera: Add cleanup of allocated fib_nodes
net: marvell: prestera: Add router nexthops ABI
eth: octeon: fix build after netif_napi_add() changes
net/mlx5: E-Switch, Return EBUSY if can't get mode lock
...
Diffstat (limited to 'drivers')
1191 files changed, 81040 insertions, 23628 deletions
diff --git a/drivers/bcma/driver_mips.c b/drivers/bcma/driver_mips.c index 12aca34e8db0..4f01e6b17bb9 100644 --- a/drivers/bcma/driver_mips.c +++ b/drivers/bcma/driver_mips.c @@ -30,7 +30,7 @@ enum bcma_boot_dev { BCMA_BOOT_DEV_NAND, }; -/* The 47162a0 hangs when reading MIPS DMP registers registers */ +/* The 47162a0 hangs when reading MIPS DMP registers */ static inline bool bcma_core_mips_bcm47162a0_quirk(struct bcma_device *dev) { return dev->bus->chipinfo.id == BCMA_CHIP_ID_BCM47162 && diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c index 2a709daefbc4..6cec9ce23fd3 100644 --- a/drivers/block/nbd.c +++ b/drivers/block/nbd.c @@ -2322,6 +2322,7 @@ static struct genl_family nbd_genl_family __ro_after_init = { .module = THIS_MODULE, .small_ops = nbd_connect_genl_ops, .n_small_ops = ARRAY_SIZE(nbd_connect_genl_ops), + .resv_start_op = NBD_CMD_STATUS + 1, .maxattr = NBD_ATTR_MAX, .policy = nbd_attr_policy, .mcgrps = nbd_mcast_grps, diff --git a/drivers/bluetooth/btintel.c b/drivers/bluetooth/btintel.c index 818681c89db8..a657e9a3e96a 100644 --- a/drivers/bluetooth/btintel.c +++ b/drivers/bluetooth/btintel.c @@ -449,6 +449,7 @@ static int btintel_version_info_tlv(struct hci_dev *hdev, case 0x17: /* TyP */ case 0x18: /* Slr */ case 0x19: /* Slr-F */ + case 0x1b: /* Mgr */ break; default: bt_dev_err(hdev, "Unsupported Intel hardware variant (0x%x)", @@ -2330,6 +2331,7 @@ static void btintel_set_msft_opcode(struct hci_dev *hdev, u8 hw_variant) case 0x17: case 0x18: case 0x19: + case 0x1b: hci_set_msft_opcode(hdev, 0xFC1E); break; default: @@ -2439,15 +2441,20 @@ static int btintel_setup_combined(struct hci_dev *hdev) INTEL_ROM_LEGACY_NO_WBS_SUPPORT)) set_bit(HCI_QUIRK_WIDEBAND_SPEECH_SUPPORTED, &hdev->quirks); + if (ver.hw_variant == 0x08 && ver.fw_variant == 0x22) + set_bit(HCI_QUIRK_VALID_LE_STATES, + &hdev->quirks); err = btintel_legacy_rom_setup(hdev, &ver); break; case 0x0b: /* SfP */ - case 0x0c: /* WsP */ case 0x11: /* JfP */ case 0x12: /* ThP */ case 0x13: /* HrP */ case 0x14: /* CcP */ + set_bit(HCI_QUIRK_VALID_LE_STATES, &hdev->quirks); + fallthrough; + case 0x0c: /* WsP */ /* Apply the device specific HCI quirks * * All Legacy bootloader devices support WBS @@ -2455,11 +2462,6 @@ static int btintel_setup_combined(struct hci_dev *hdev) set_bit(HCI_QUIRK_WIDEBAND_SPEECH_SUPPORTED, &hdev->quirks); - /* Valid LE States quirk for JfP/ThP familiy */ - if (ver.hw_variant == 0x11 || ver.hw_variant == 0x12) - set_bit(HCI_QUIRK_VALID_LE_STATES, - &hdev->quirks); - /* Setup MSFT Extension support */ btintel_set_msft_opcode(hdev, ver.hw_variant); @@ -2530,9 +2532,8 @@ static int btintel_setup_combined(struct hci_dev *hdev) */ set_bit(HCI_QUIRK_WIDEBAND_SPEECH_SUPPORTED, &hdev->quirks); - /* Valid LE States quirk for JfP/ThP familiy */ - if (ver.hw_variant == 0x11 || ver.hw_variant == 0x12) - set_bit(HCI_QUIRK_VALID_LE_STATES, &hdev->quirks); + /* Set Valid LE States quirk */ + set_bit(HCI_QUIRK_VALID_LE_STATES, &hdev->quirks); /* Setup MSFT Extension support */ btintel_set_msft_opcode(hdev, ver.hw_variant); @@ -2542,6 +2543,7 @@ static int btintel_setup_combined(struct hci_dev *hdev) case 0x17: case 0x18: case 0x19: + case 0x1b: /* Display version information of TLV type */ btintel_version_info_tlv(hdev, &ver_tlv); diff --git a/drivers/bluetooth/btusb.c b/drivers/bluetooth/btusb.c index 15caa6469538..271963805a38 100644 --- a/drivers/bluetooth/btusb.c +++ b/drivers/bluetooth/btusb.c @@ -426,6 +426,8 @@ static const struct usb_device_id blacklist_table[] = { BTUSB_WIDEBAND_SPEECH }, { USB_DEVICE(0x04ca, 0x4006), .driver_info = BTUSB_REALTEK | BTUSB_WIDEBAND_SPEECH }, + { USB_DEVICE(0x0cb8, 0xc549), .driver_info = BTUSB_REALTEK | + BTUSB_WIDEBAND_SPEECH }, /* Realtek 8852CE Bluetooth devices */ { USB_DEVICE(0x04ca, 0x4007), .driver_info = BTUSB_REALTEK | @@ -438,6 +440,8 @@ static const struct usb_device_id blacklist_table[] = { BTUSB_WIDEBAND_SPEECH }, { USB_DEVICE(0x13d3, 0x3586), .driver_info = BTUSB_REALTEK | BTUSB_WIDEBAND_SPEECH }, + { USB_DEVICE(0x13d3, 0x3592), .driver_info = BTUSB_REALTEK | + BTUSB_WIDEBAND_SPEECH }, /* Realtek Bluetooth devices */ { USB_VENDOR_AND_INTERFACE_INFO(0x0bda, 0xe0, 0x01, 0x01), @@ -466,6 +470,9 @@ static const struct usb_device_id blacklist_table[] = { { USB_DEVICE(0x0489, 0xe0c8), .driver_info = BTUSB_MEDIATEK | BTUSB_WIDEBAND_SPEECH | BTUSB_VALID_LE_STATES }, + { USB_DEVICE(0x0489, 0xe0e0), .driver_info = BTUSB_MEDIATEK | + BTUSB_WIDEBAND_SPEECH | + BTUSB_VALID_LE_STATES }, { USB_DEVICE(0x04ca, 0x3802), .driver_info = BTUSB_MEDIATEK | BTUSB_WIDEBAND_SPEECH | BTUSB_VALID_LE_STATES }, @@ -478,9 +485,18 @@ static const struct usb_device_id blacklist_table[] = { { USB_DEVICE(0x13d3, 0x3567), .driver_info = BTUSB_MEDIATEK | BTUSB_WIDEBAND_SPEECH | BTUSB_VALID_LE_STATES }, + { USB_DEVICE(0x13d3, 0x3578), .driver_info = BTUSB_MEDIATEK | + BTUSB_WIDEBAND_SPEECH | + BTUSB_VALID_LE_STATES }, + { USB_DEVICE(0x13d3, 0x3583), .driver_info = BTUSB_MEDIATEK | + BTUSB_WIDEBAND_SPEECH | + BTUSB_VALID_LE_STATES }, { USB_DEVICE(0x0489, 0xe0cd), .driver_info = BTUSB_MEDIATEK | BTUSB_WIDEBAND_SPEECH | BTUSB_VALID_LE_STATES }, + { USB_DEVICE(0x0e8d, 0x0608), .driver_info = BTUSB_MEDIATEK | + BTUSB_WIDEBAND_SPEECH | + BTUSB_VALID_LE_STATES }, /* MediaTek MT7922A Bluetooth devices */ { USB_DEVICE(0x0489, 0xe0d8), .driver_info = BTUSB_MEDIATEK | @@ -516,19 +532,17 @@ static const struct usb_device_id blacklist_table[] = { { USB_DEVICE(0x0bda, 0xb009), .driver_info = BTUSB_REALTEK }, { USB_DEVICE(0x2ff8, 0xb011), .driver_info = BTUSB_REALTEK }, - /* Additional Realtek 8761B Bluetooth devices */ + /* Additional Realtek 8761BUV Bluetooth devices */ { USB_DEVICE(0x2357, 0x0604), .driver_info = BTUSB_REALTEK | BTUSB_WIDEBAND_SPEECH }, - - /* Additional Realtek 8761BU Bluetooth devices */ { USB_DEVICE(0x0b05, 0x190e), .driver_info = BTUSB_REALTEK | BTUSB_WIDEBAND_SPEECH }, { USB_DEVICE(0x2550, 0x8761), .driver_info = BTUSB_REALTEK | BTUSB_WIDEBAND_SPEECH }, - - /* Additional Realtek 8761BUV Bluetooth devices */ { USB_DEVICE(0x0bda, 0x8771), .driver_info = BTUSB_REALTEK | BTUSB_WIDEBAND_SPEECH }, + { USB_DEVICE(0x7392, 0xc611), .driver_info = BTUSB_REALTEK | + BTUSB_WIDEBAND_SPEECH }, /* Additional Realtek 8821AE Bluetooth devices */ { USB_DEVICE(0x0b05, 0x17dc), .driver_info = BTUSB_REALTEK }, @@ -2477,15 +2491,29 @@ static int btusb_mtk_hci_wmt_sync(struct hci_dev *hdev, set_bit(BTUSB_TX_WAIT_VND_EVT, &data->flags); + /* WMT cmd/event doesn't follow up the generic HCI cmd/event handling, + * it needs constantly polling control pipe until the host received the + * WMT event, thus, we should require to specifically acquire PM counter + * on the USB to prevent the interface from entering auto suspended + * while WMT cmd/event in progress. + */ + err = usb_autopm_get_interface(data->intf); + if (err < 0) + goto err_free_wc; + err = __hci_cmd_send(hdev, 0xfc6f, hlen, wc); if (err < 0) { clear_bit(BTUSB_TX_WAIT_VND_EVT, &data->flags); + usb_autopm_put_interface(data->intf); goto err_free_wc; } /* Submit control IN URB on demand to process the WMT event */ err = btusb_mtk_submit_wmt_recv_urb(hdev); + + usb_autopm_put_interface(data->intf); + if (err < 0) goto err_free_wc; diff --git a/drivers/bluetooth/hci_ldisc.c b/drivers/bluetooth/hci_ldisc.c index f537673ede17..865112e96ff9 100644 --- a/drivers/bluetooth/hci_ldisc.c +++ b/drivers/bluetooth/hci_ldisc.c @@ -493,6 +493,11 @@ static int hci_uart_tty_open(struct tty_struct *tty) BT_ERR("Can't allocate control structure"); return -ENFILE; } + if (percpu_init_rwsem(&hu->proto_lock)) { + BT_ERR("Can't allocate semaphore structure"); + kfree(hu); + return -ENOMEM; + } tty->disc_data = hu; hu->tty = tty; @@ -505,8 +510,6 @@ static int hci_uart_tty_open(struct tty_struct *tty) INIT_WORK(&hu->init_ready, hci_uart_init_work); INIT_WORK(&hu->write_work, hci_uart_write_work); - percpu_init_rwsem(&hu->proto_lock); - /* Flush any pending characters in the driver */ tty_driver_flush_buffer(tty); diff --git a/drivers/bluetooth/hci_serdev.c b/drivers/bluetooth/hci_serdev.c index c0e5f42ec6b7..f16fd79bc02b 100644 --- a/drivers/bluetooth/hci_serdev.c +++ b/drivers/bluetooth/hci_serdev.c @@ -310,11 +310,12 @@ int hci_uart_register_device(struct hci_uart *hu, serdev_device_set_client_ops(hu->serdev, &hci_serdev_client_ops); + if (percpu_init_rwsem(&hu->proto_lock)) + return -ENOMEM; + err = serdev_device_open(hu->serdev); if (err) - return err; - - percpu_init_rwsem(&hu->proto_lock); + goto err_rwsem; err = p->open(hu); if (err) @@ -389,6 +390,8 @@ err_alloc: p->close(hu); err_open: serdev_device_close(hu->serdev); +err_rwsem: + percpu_free_rwsem(&hu->proto_lock); return err; } EXPORT_SYMBOL_GPL(hci_uart_register_device); @@ -410,5 +413,6 @@ void hci_uart_unregister_device(struct hci_uart *hu) clear_bit(HCI_UART_PROTO_READY, &hu->flags); serdev_device_close(hu->serdev); } + percpu_free_rwsem(&hu->proto_lock); } EXPORT_SYMBOL_GPL(hci_uart_unregister_device); diff --git a/drivers/bus/mhi/host/pci_generic.c b/drivers/bus/mhi/host/pci_generic.c index 9e545f2a5a26..fa2246da63c1 100644 --- a/drivers/bus/mhi/host/pci_generic.c +++ b/drivers/bus/mhi/host/pci_generic.c @@ -507,6 +507,8 @@ static const struct mhi_channel_config mhi_telit_fn990_channels[] = { MHI_CHANNEL_CONFIG_DL(13, "MBIM", 32, 0), MHI_CHANNEL_CONFIG_UL(32, "DUN", 32, 0), MHI_CHANNEL_CONFIG_DL(33, "DUN", 32, 0), + MHI_CHANNEL_CONFIG_UL(92, "DUN2", 32, 1), + MHI_CHANNEL_CONFIG_DL(93, "DUN2", 32, 1), MHI_CHANNEL_CONFIG_HW_UL(100, "IP_HW0_MBIM", 128, 2), MHI_CHANNEL_CONFIG_HW_DL(101, "IP_HW0_MBIM", 128, 3), }; diff --git a/drivers/firmware/xilinx/zynqmp.c b/drivers/firmware/xilinx/zynqmp.c index d1f652802181..ff5cabe70a2b 100644 --- a/drivers/firmware/xilinx/zynqmp.c +++ b/drivers/firmware/xilinx/zynqmp.c @@ -1312,6 +1312,37 @@ int zynqmp_pm_get_feature_config(enum pm_feature_config_id id, } /** + * zynqmp_pm_set_sd_config - PM call to set value of SD config registers + * @node: SD node ID + * @config: The config type of SD registers + * @value: Value to be set + * + * Return: Returns 0 on success or error value on failure. + */ +int zynqmp_pm_set_sd_config(u32 node, enum pm_sd_config_type config, u32 value) +{ + return zynqmp_pm_invoke_fn(PM_IOCTL, node, IOCTL_SET_SD_CONFIG, + config, value, NULL); +} +EXPORT_SYMBOL_GPL(zynqmp_pm_set_sd_config); + +/** + * zynqmp_pm_set_gem_config - PM call to set value of GEM config registers + * @node: GEM node ID + * @config: The config type of GEM registers + * @value: Value to be set + * + * Return: Returns 0 on success or error value on failure. + */ +int zynqmp_pm_set_gem_config(u32 node, enum pm_gem_config_type config, + u32 value) +{ + return zynqmp_pm_invoke_fn(PM_IOCTL, node, IOCTL_SET_GEM_CONFIG, + config, value, NULL); +} +EXPORT_SYMBOL_GPL(zynqmp_pm_set_gem_config); + +/** * struct zynqmp_pm_shutdown_scope - Struct for shutdown scope * @subtype: Shutdown subtype * @name: Matching string for scope argument diff --git a/drivers/infiniband/hw/mlx5/mad.c b/drivers/infiniband/hw/mlx5/mad.c index b4dc52392275..9c8a7b206dcf 100644 --- a/drivers/infiniband/hw/mlx5/mad.c +++ b/drivers/infiniband/hw/mlx5/mad.c @@ -147,6 +147,28 @@ static void pma_cnt_assign(struct ib_pma_portcounters *pma_cnt, vl_15_dropped); } +static int query_ib_ppcnt(struct mlx5_core_dev *dev, u8 port_num, void *out, + size_t sz) +{ + u32 *in; + int err; + + in = kvzalloc(sz, GFP_KERNEL); + if (!in) { + err = -ENOMEM; + return err; + } + + MLX5_SET(ppcnt_reg, in, local_port, port_num); + + MLX5_SET(ppcnt_reg, in, grp, MLX5_INFINIBAND_PORT_COUNTERS_GROUP); + err = mlx5_core_access_reg(dev, in, sz, out, + sz, MLX5_REG_PPCNT, 0, 0); + + kvfree(in); + return err; +} + static int process_pma_cmd(struct mlx5_ib_dev *dev, u32 port_num, const struct ib_mad *in_mad, struct ib_mad *out_mad) { @@ -208,8 +230,7 @@ static int process_pma_cmd(struct mlx5_ib_dev *dev, u32 port_num, goto done; } - err = mlx5_core_query_ib_ppcnt(mdev, mdev_port_num, - out_cnt, sz); + err = query_ib_ppcnt(mdev, mdev_port_num, out_cnt, sz); if (!err) pma_cnt_assign(pma_cnt, out_cnt); } diff --git a/drivers/infiniband/hw/mlx5/mlx5_ib.h b/drivers/infiniband/hw/mlx5/mlx5_ib.h index e66bf72f1f04..62338f44a30e 100644 --- a/drivers/infiniband/hw/mlx5/mlx5_ib.h +++ b/drivers/infiniband/hw/mlx5/mlx5_ib.h @@ -1541,6 +1541,18 @@ int mlx5_ib_test_wc(struct mlx5_ib_dev *dev); static inline bool mlx5_ib_lag_should_assign_affinity(struct mlx5_ib_dev *dev) { + /* + * If the driver is in hash mode and the port_select_flow_table_bypass cap + * is supported, it means that the driver no longer needs to assign the port + * affinity by default. If a user wants to set the port affinity explicitly, + * the user has a dedicated API to do that, so there is no need to assign + * the port affinity by default. + */ + if (dev->lag_active && + mlx5_lag_mode_is_hash(dev->mdev) && + MLX5_CAP_PORT_SELECTION(dev->mdev, port_select_flow_table_bypass)) + return 0; + return dev->lag_active || (MLX5_CAP_GEN(dev->mdev, num_lag_ports) > 1 && MLX5_CAP_GEN(dev->mdev, lag_tx_port_affinity)); diff --git a/drivers/isdn/capi/kcapi.c b/drivers/isdn/capi/kcapi.c index e69c4bf557bf..ae24848af233 100644 --- a/drivers/isdn/capi/kcapi.c +++ b/drivers/isdn/capi/kcapi.c @@ -798,7 +798,7 @@ u16 capi20_get_serial(u32 contr, u8 serial[CAPI_SERIAL_LEN]) u16 ret; if (contr == 0) { - strlcpy(serial, driver_serial, CAPI_SERIAL_LEN); + strscpy(serial, driver_serial, CAPI_SERIAL_LEN); return CAPI_NOERROR; } @@ -806,7 +806,7 @@ u16 capi20_get_serial(u32 contr, u8 serial[CAPI_SERIAL_LEN]) ctr = get_capi_ctr_by_nr(contr); if (ctr && ctr->state == CAPI_CTR_RUNNING) { - strlcpy(serial, ctr->serial, CAPI_SERIAL_LEN); + strscpy(serial, ctr->serial, CAPI_SERIAL_LEN); ret = CAPI_NOERROR; } else ret = CAPI_REGNOTINSTALLED; diff --git a/drivers/isdn/mISDN/l1oip.h b/drivers/isdn/mISDN/l1oip.h index 7ea10db20e3a..48133d022812 100644 --- a/drivers/isdn/mISDN/l1oip.h +++ b/drivers/isdn/mISDN/l1oip.h @@ -59,6 +59,7 @@ struct l1oip { int bundle; /* bundle channels in one frm */ int codec; /* codec to use for transmis. */ int limit; /* limit number of bchannels */ + bool shutdown; /* if card is released */ /* timer */ struct timer_list keep_tl; diff --git a/drivers/isdn/mISDN/l1oip_core.c b/drivers/isdn/mISDN/l1oip_core.c index 2c40412466e6..a77195e378b7 100644 --- a/drivers/isdn/mISDN/l1oip_core.c +++ b/drivers/isdn/mISDN/l1oip_core.c @@ -275,7 +275,7 @@ l1oip_socket_send(struct l1oip *hc, u8 localcodec, u8 channel, u32 chanmask, p = frame; /* restart timer */ - if (time_before(hc->keep_tl.expires, jiffies + 5 * HZ)) + if (time_before(hc->keep_tl.expires, jiffies + 5 * HZ) && !hc->shutdown) mod_timer(&hc->keep_tl, jiffies + L1OIP_KEEPALIVE * HZ); else hc->keep_tl.expires = jiffies + L1OIP_KEEPALIVE * HZ; @@ -601,7 +601,9 @@ multiframe: goto multiframe; /* restart timer */ - if (time_before(hc->timeout_tl.expires, jiffies + 5 * HZ) || !hc->timeout_on) { + if ((time_before(hc->timeout_tl.expires, jiffies + 5 * HZ) || + !hc->timeout_on) && + !hc->shutdown) { hc->timeout_on = 1; mod_timer(&hc->timeout_tl, jiffies + L1OIP_TIMEOUT * HZ); } else /* only adjust timer */ @@ -1232,11 +1234,10 @@ release_card(struct l1oip *hc) { int ch; - if (timer_pending(&hc->keep_tl)) - del_timer(&hc->keep_tl); + hc->shutdown = true; - if (timer_pending(&hc->timeout_tl)) - del_timer(&hc->timeout_tl); + del_timer_sync(&hc->keep_tl); + del_timer_sync(&hc->timeout_tl); cancel_work_sync(&hc->workq); diff --git a/drivers/mfd/Kconfig b/drivers/mfd/Kconfig index abb58ab1a1a4..c3dd1fe8d8c9 100644 --- a/drivers/mfd/Kconfig +++ b/drivers/mfd/Kconfig @@ -963,6 +963,27 @@ config MFD_MENF21BMC This driver can also be built as a module. If so the module will be called menf21bmc. +config MFD_OCELOT + tristate "Microsemi Ocelot External Control Support" + depends on SPI_MASTER + select MFD_CORE + select REGMAP_SPI + help + Ocelot is a family of networking chips that support multiple ethernet + and fibre interfaces. In addition to networking, they contain several + other functions, including pinctrl, MDIO, and communication with + external chips. While some chips have an internal processor capable of + running an OS, others don't. All chips can be controlled externally + through different interfaces, including SPI, I2C, and PCIe. + + Say yes here to add support for Ocelot chips (VSC7511, VSC7512, + VSC7513, VSC7514) controlled externally. + + To compile this driver as a module, choose M here: the module will be + called ocelot-soc. + + If unsure, say N. + config EZX_PCAP bool "Motorola EZXPCAP Support" depends on SPI_MASTER diff --git a/drivers/mfd/Makefile b/drivers/mfd/Makefile index 858cacf659d6..0004b7e86220 100644 --- a/drivers/mfd/Makefile +++ b/drivers/mfd/Makefile @@ -120,6 +120,9 @@ obj-$(CONFIG_MFD_MC13XXX_I2C) += mc13xxx-i2c.o obj-$(CONFIG_MFD_CORE) += mfd-core.o +ocelot-soc-objs := ocelot-core.o ocelot-spi.o +obj-$(CONFIG_MFD_OCELOT) += ocelot-soc.o + obj-$(CONFIG_EZX_PCAP) += ezx-pcap.o obj-$(CONFIG_MFD_CPCAP) += motorola-cpcap.o diff --git a/drivers/mfd/ocelot-core.c b/drivers/mfd/ocelot-core.c new file mode 100644 index 000000000000..1816d52c65c5 --- /dev/null +++ b/drivers/mfd/ocelot-core.c @@ -0,0 +1,161 @@ +// SPDX-License-Identifier: (GPL-2.0 OR MIT) +/* + * Core driver for the Ocelot chip family. + * + * The VSC7511, 7512, 7513, and 7514 can be controlled internally via an + * on-chip MIPS processor, or externally via SPI, I2C, PCIe. This core driver is + * intended to be the bus-agnostic glue between, for example, the SPI bus and + * the child devices. + * + * Copyright 2021-2022 Innovative Advantage Inc. + * + * Author: Colin Foster <colin.foster@in-advantage.com> + */ + +#include <linux/bits.h> +#include <linux/device.h> +#include <linux/export.h> +#include <linux/iopoll.h> +#include <linux/ioport.h> +#include <linux/kernel.h> +#include <linux/mfd/core.h> +#include <linux/mfd/ocelot.h> +#include <linux/module.h> +#include <linux/regmap.h> +#include <linux/types.h> + +#include <soc/mscc/ocelot.h> + +#include "ocelot.h" + +#define REG_GCB_SOFT_RST 0x0008 + +#define BIT_SOFT_CHIP_RST BIT(0) + +#define VSC7512_MIIM0_RES_START 0x7107009c +#define VSC7512_MIIM1_RES_START 0x710700c0 +#define VSC7512_MIIM_RES_SIZE 0x024 + +#define VSC7512_PHY_RES_START 0x710700f0 +#define VSC7512_PHY_RES_SIZE 0x004 + +#define VSC7512_GPIO_RES_START 0x71070034 +#define VSC7512_GPIO_RES_SIZE 0x06c + +#define VSC7512_SIO_CTRL_RES_START 0x710700f8 +#define VSC7512_SIO_CTRL_RES_SIZE 0x100 + +#define VSC7512_GCB_RST_SLEEP_US 100 +#define VSC7512_GCB_RST_TIMEOUT_US 100000 + +static int ocelot_gcb_chip_rst_status(struct ocelot_ddata *ddata) +{ + int val, err; + + err = regmap_read(ddata->gcb_regmap, REG_GCB_SOFT_RST, &val); + if (err) + return err; + + return val; +} + +int ocelot_chip_reset(struct device *dev) +{ + struct ocelot_ddata *ddata = dev_get_drvdata(dev); + int ret, val; + + /* + * Reset the entire chip here to put it into a completely known state. + * Other drivers may want to reset their own subsystems. The register + * self-clears, so one write is all that is needed and wait for it to + * clear. + */ + ret = regmap_write(ddata->gcb_regmap, REG_GCB_SOFT_RST, BIT_SOFT_CHIP_RST); + if (ret) + return ret; + + return readx_poll_timeout(ocelot_gcb_chip_rst_status, ddata, val, !val, + VSC7512_GCB_RST_SLEEP_US, VSC7512_GCB_RST_TIMEOUT_US); +} +EXPORT_SYMBOL_NS(ocelot_chip_reset, MFD_OCELOT); + +static const struct resource vsc7512_miim0_resources[] = { + DEFINE_RES_REG_NAMED(VSC7512_MIIM0_RES_START, VSC7512_MIIM_RES_SIZE, "gcb_miim0"), + DEFINE_RES_REG_NAMED(VSC7512_PHY_RES_START, VSC7512_PHY_RES_SIZE, "gcb_phy"), +}; + +static const struct resource vsc7512_miim1_resources[] = { + DEFINE_RES_REG_NAMED(VSC7512_MIIM1_RES_START, VSC7512_MIIM_RES_SIZE, "gcb_miim1"), +}; + +static const struct resource vsc7512_pinctrl_resources[] = { + DEFINE_RES_REG_NAMED(VSC7512_GPIO_RES_START, VSC7512_GPIO_RES_SIZE, "gcb_gpio"), +}; + +static const struct resource vsc7512_sgpio_resources[] = { + DEFINE_RES_REG_NAMED(VSC7512_SIO_CTRL_RES_START, VSC7512_SIO_CTRL_RES_SIZE, "gcb_sio"), +}; + +static const struct mfd_cell vsc7512_devs[] = { + { + .name = "ocelot-pinctrl", + .of_compatible = "mscc,ocelot-pinctrl", + .num_resources = ARRAY_SIZE(vsc7512_pinctrl_resources), + .resources = vsc7512_pinctrl_resources, + }, { + .name = "ocelot-sgpio", + .of_compatible = "mscc,ocelot-sgpio", + .num_resources = ARRAY_SIZE(vsc7512_sgpio_resources), + .resources = vsc7512_sgpio_resources, + }, { + .name = "ocelot-miim0", + .of_compatible = "mscc,ocelot-miim", + .of_reg = VSC7512_MIIM0_RES_START, + .use_of_reg = true, + .num_resources = ARRAY_SIZE(vsc7512_miim0_resources), + .resources = vsc7512_miim0_resources, + }, { + .name = "ocelot-miim1", + .of_compatible = "mscc,ocelot-miim", + .of_reg = VSC7512_MIIM1_RES_START, + .use_of_reg = true, + .num_resources = ARRAY_SIZE(vsc7512_miim1_resources), + .resources = vsc7512_miim1_resources, + }, +}; + +static void ocelot_core_try_add_regmap(struct device *dev, + const struct resource *res) +{ + if (dev_get_regmap(dev, res->name)) + return; + + ocelot_spi_init_regmap(dev, res); +} + +static void ocelot_core_try_add_regmaps(struct device *dev, + const struct mfd_cell *cell) +{ + int i; + + for (i = 0; i < cell->num_resources; i++) + ocelot_core_try_add_regmap(dev, &cell->resources[i]); +} + +int ocelot_core_init(struct device *dev) +{ + int i, ndevs; + + ndevs = ARRAY_SIZE(vsc7512_devs); + + for (i = 0; i < ndevs; i++) + ocelot_core_try_add_regmaps(dev, &vsc7512_devs[i]); + + return devm_mfd_add_devices(dev, PLATFORM_DEVID_AUTO, vsc7512_devs, ndevs, NULL, 0, NULL); +} +EXPORT_SYMBOL_NS(ocelot_core_init, MFD_OCELOT); + +MODULE_DESCRIPTION("Externally Controlled Ocelot Chip Driver"); +MODULE_AUTHOR("Colin Foster <colin.foster@in-advantage.com>"); +MODULE_LICENSE("GPL"); +MODULE_IMPORT_NS(MFD_OCELOT_SPI); diff --git a/drivers/mfd/ocelot-spi.c b/drivers/mfd/ocelot-spi.c new file mode 100644 index 000000000000..0f097f4829d1 --- /dev/null +++ b/drivers/mfd/ocelot-spi.c @@ -0,0 +1,299 @@ +// SPDX-License-Identifier: (GPL-2.0 OR MIT) +/* + * SPI core driver for the Ocelot chip family. + * + * This driver will handle everything necessary to allow for communication over + * SPI to the VSC7511, VSC7512, VSC7513 and VSC7514 chips. The main functions + * are to prepare the chip's SPI interface for a specific bus speed, and a host + * processor's endianness. This will create and distribute regmaps for any + * children. + * + * Copyright 2021-2022 Innovative Advantage Inc. + * + * Author: Colin Foster <colin.foster@in-advantage.com> + */ + +#include <linux/device.h> +#include <linux/err.h> +#include <linux/errno.h> +#include <linux/export.h> +#include <linux/ioport.h> +#include <linux/mod_devicetable.h> +#include <linux/module.h> +#include <linux/regmap.h> +#include <linux/spi/spi.h> +#include <linux/types.h> +#include <linux/units.h> + +#include "ocelot.h" + +#define REG_DEV_CPUORG_IF_CTRL 0x0000 +#define REG_DEV_CPUORG_IF_CFGSTAT 0x0004 + +#define CFGSTAT_IF_NUM_VCORE (0 << 24) +#define CFGSTAT_IF_NUM_VRAP (1 << 24) +#define CFGSTAT_IF_NUM_SI (2 << 24) +#define CFGSTAT_IF_NUM_MIIM (3 << 24) + +#define VSC7512_DEVCPU_ORG_RES_START 0x71000000 +#define VSC7512_DEVCPU_ORG_RES_SIZE 0x38 + +#define VSC7512_CHIP_REGS_RES_START 0x71070000 +#define VSC7512_CHIP_REGS_RES_SIZE 0x14 + +static const struct resource vsc7512_dev_cpuorg_resource = + DEFINE_RES_REG_NAMED(VSC7512_DEVCPU_ORG_RES_START, + VSC7512_DEVCPU_ORG_RES_SIZE, + "devcpu_org"); + +static const struct resource vsc7512_gcb_resource = + DEFINE_RES_REG_NAMED(VSC7512_CHIP_REGS_RES_START, + VSC7512_CHIP_REGS_RES_SIZE, + "devcpu_gcb_chip_regs"); + +static int ocelot_spi_initialize(struct device *dev) +{ + struct ocelot_ddata *ddata = dev_get_drvdata(dev); + u32 val, check; + int err; + + val = OCELOT_SPI_BYTE_ORDER; + + /* + * The SPI address must be big-endian, but we want the payload to match + * our CPU. These are two bits (0 and 1) but they're repeated such that + * the write from any configuration will be valid. The four + * configurations are: + * + * 0b00: little-endian, MSB first + * | 111111 | 22221111 | 33222222 | + * | 76543210 | 54321098 | 32109876 | 10987654 | + * + * 0b01: big-endian, MSB first + * | 33222222 | 22221111 | 111111 | | + * | 10987654 | 32109876 | 54321098 | 76543210 | + * + * 0b10: little-endian, LSB first + * | 111111 | 11112222 | 22222233 | + * | 01234567 | 89012345 | 67890123 | 45678901 | + * + * 0b11: big-endian, LSB first + * | 22222233 | 11112222 | 111111 | | + * | 45678901 | 67890123 | 89012345 | 01234567 | + */ + err = regmap_write(ddata->cpuorg_regmap, REG_DEV_CPUORG_IF_CTRL, val); + if (err) + return err; + + /* + * Apply the number of padding bytes between a read request and the data + * payload. Some registers have access times of up to 1us, so if the + * first payload bit is shifted out too quickly, the read will fail. + */ + val = ddata->spi_padding_bytes; + err = regmap_write(ddata->cpuorg_regmap, REG_DEV_CPUORG_IF_CFGSTAT, val); + if (err) + return err; + + /* + * After we write the interface configuration, read it back here. This + * will verify several different things. The first is that the number of + * padding bytes actually got written correctly. These are found in bits + * 0:3. + * + * The second is that bit 16 is cleared. Bit 16 is IF_CFGSTAT:IF_STAT, + * and will be set if the register access is too fast. This would be in + * the condition that the number of padding bytes is insufficient for + * the SPI bus frequency. + * + * The last check is for bits 31:24, which define the interface by which + * the registers are being accessed. Since we're accessing them via the + * serial interface, it must return IF_NUM_SI. + */ + check = val | CFGSTAT_IF_NUM_SI; + + err = regmap_read(ddata->cpuorg_regmap, REG_DEV_CPUORG_IF_CFGSTAT, &val); + if (err) + return err; + + if (check != val) + return -ENODEV; + + return 0; +} + +static const struct regmap_config ocelot_spi_regmap_config = { + .reg_bits = 24, + .reg_stride = 4, + .reg_downshift = 2, + .val_bits = 32, + + .write_flag_mask = 0x80, + + .use_single_write = true, + .can_multi_write = false, + + .reg_format_endian = REGMAP_ENDIAN_BIG, + .val_format_endian = REGMAP_ENDIAN_NATIVE, +}; + +static int ocelot_spi_regmap_bus_read(void *context, const void *reg, size_t reg_size, + void *val, size_t val_size) +{ + struct spi_transfer xfers[3] = {0}; + struct device *dev = context; + struct ocelot_ddata *ddata; + struct spi_device *spi; + struct spi_message msg; + unsigned int index = 0; + + ddata = dev_get_drvdata(dev); + spi = to_spi_device(dev); + + xfers[index].tx_buf = reg; + xfers[index].len = reg_size; + index++; + + if (ddata->spi_padding_bytes) { + xfers[index].len = ddata->spi_padding_bytes; + xfers[index].tx_buf = ddata->dummy_buf; + xfers[index].dummy_data = 1; + index++; + } + + xfers[index].rx_buf = val; + xfers[index].len = val_size; + index++; + + spi_message_init_with_transfers(&msg, xfers, index); + + return spi_sync(spi, &msg); +} + +static int ocelot_spi_regmap_bus_write(void *context, const void *data, size_t count) +{ + struct device *dev = context; + struct spi_device *spi = to_spi_device(dev); + + return spi_write(spi, data, count); +} + +static const struct regmap_bus ocelot_spi_regmap_bus = { + .write = ocelot_spi_regmap_bus_write, + .read = ocelot_spi_regmap_bus_read, +}; + +struct regmap *ocelot_spi_init_regmap(struct device *dev, const struct resource *res) +{ + struct regmap_config regmap_config; + + memcpy(®map_config, &ocelot_spi_regmap_config, sizeof(regmap_config)); + + regmap_config.name = res->name; + regmap_config.max_register = resource_size(res) - 1; + regmap_config.reg_base = res->start; + + return devm_regmap_init(dev, &ocelot_spi_regmap_bus, dev, ®map_config); +} +EXPORT_SYMBOL_NS(ocelot_spi_init_regmap, MFD_OCELOT_SPI); + +static int ocelot_spi_probe(struct spi_device *spi) +{ + struct device *dev = &spi->dev; + struct ocelot_ddata *ddata; + struct regmap *r; + int err; + + ddata = devm_kzalloc(dev, sizeof(*ddata), GFP_KERNEL); + if (!ddata) + return -ENOMEM; + + spi_set_drvdata(spi, ddata); + + if (spi->max_speed_hz <= 500000) { + ddata->spi_padding_bytes = 0; + } else { + /* + * Calculation taken from the manual for IF_CFGSTAT:IF_CFG. + * Register access time is 1us, so we need to configure and send + * out enough padding bytes between the read request and data + * transmission that lasts at least 1 microsecond. + */ + ddata->spi_padding_bytes = 1 + (spi->max_speed_hz / HZ_PER_MHZ + 2) / 8; + + ddata->dummy_buf = devm_kzalloc(dev, ddata->spi_padding_bytes, GFP_KERNEL); + if (!ddata->dummy_buf) + return -ENOMEM; + } + + spi->bits_per_word = 8; + + err = spi_setup(spi); + if (err) + return dev_err_probe(&spi->dev, err, "Error performing SPI setup\n"); + + r = ocelot_spi_init_regmap(dev, &vsc7512_dev_cpuorg_resource); + if (IS_ERR(r)) + return PTR_ERR(r); + + ddata->cpuorg_regmap = r; + + r = ocelot_spi_init_regmap(dev, &vsc7512_gcb_resource); + if (IS_ERR(r)) + return PTR_ERR(r); + + ddata->gcb_regmap = r; + + /* + * The chip must be set up for SPI before it gets initialized and reset. + * This must be done before calling init, and after a chip reset is + * performed. + */ + err = ocelot_spi_initialize(dev); + if (err) + return dev_err_probe(dev, err, "Error initializing SPI bus\n"); + + err = ocelot_chip_reset(dev); + if (err) + return dev_err_probe(dev, err, "Error resetting device\n"); + + /* + * A chip reset will clear the SPI configuration, so it needs to be done + * again before we can access any registers. + */ + err = ocelot_spi_initialize(dev); + if (err) + return dev_err_probe(dev, err, "Error initializing SPI bus after reset\n"); + + err = ocelot_core_init(dev); + if (err) + return dev_err_probe(dev, err, "Error initializing Ocelot core\n"); + + return 0; +} + +static const struct spi_device_id ocelot_spi_ids[] = { + { "vsc7512", 0 }, + { } +}; + +static const struct of_device_id ocelot_spi_of_match[] = { + { .compatible = "mscc,vsc7512" }, + { } +}; +MODULE_DEVICE_TABLE(of, ocelot_spi_of_match); + +static struct spi_driver ocelot_spi_driver = { + .driver = { + .name = "ocelot-soc", + .of_match_table = ocelot_spi_of_match, + }, + .id_table = ocelot_spi_ids, + .probe = ocelot_spi_probe, +}; +module_spi_driver(ocelot_spi_driver); + +MODULE_DESCRIPTION("SPI Controlled Ocelot Chip Driver"); +MODULE_AUTHOR("Colin Foster <colin.foster@in-advantage.com>"); +MODULE_LICENSE("Dual MIT/GPL"); +MODULE_IMPORT_NS(MFD_OCELOT); diff --git a/drivers/mfd/ocelot.h b/drivers/mfd/ocelot.h new file mode 100644 index 000000000000..b8bc2f1486e2 --- /dev/null +++ b/drivers/mfd/ocelot.h @@ -0,0 +1,49 @@ +/* SPDX-License-Identifier: GPL-2.0 OR MIT */ +/* Copyright 2021, 2022 Innovative Advantage Inc. */ + +#ifndef _MFD_OCELOT_H +#define _MFD_OCELOT_H + +#include <linux/kconfig.h> + +struct device; +struct regmap; +struct resource; + +/** + * struct ocelot_ddata - Private data for an external Ocelot chip + * @gcb_regmap: General Configuration Block regmap. Used for + * operations like chip reset. + * @cpuorg_regmap: CPU Device Origin Block regmap. Used for operations + * like SPI bus configuration. + * @spi_padding_bytes: Number of padding bytes that must be thrown out before + * read data gets returned. This is calculated during + * initialization based on bus speed. + * @dummy_buf: Zero-filled buffer of spi_padding_bytes size. The dummy + * bytes that will be sent out between the address and + * data of a SPI read operation. + */ +struct ocelot_ddata { + struct regmap *gcb_regmap; + struct regmap *cpuorg_regmap; + int spi_padding_bytes; + void *dummy_buf; +}; + +int ocelot_chip_reset(struct device *dev); +int ocelot_core_init(struct device *dev); + +/* SPI-specific routines that won't be necessary for other interfaces */ +struct regmap *ocelot_spi_init_regmap(struct device *dev, + const struct resource *res); + +#define OCELOT_SPI_BYTE_ORDER_LE 0x00000000 +#define OCELOT_SPI_BYTE_ORDER_BE 0x81818181 + +#ifdef __LITTLE_ENDIAN +#define OCELOT_SPI_BYTE_ORDER OCELOT_SPI_BYTE_ORDER_LE +#else +#define OCELOT_SPI_BYTE_ORDER OCELOT_SPI_BYTE_ORDER_BE +#endif + +#endif diff --git a/drivers/net/Kconfig b/drivers/net/Kconfig index 94c889802566..15d4a38b1351 100644 --- a/drivers/net/Kconfig +++ b/drivers/net/Kconfig @@ -500,6 +500,8 @@ config NET_SB1000 source "drivers/net/phy/Kconfig" +source "drivers/net/pse-pd/Kconfig" + source "drivers/net/can/Kconfig" source "drivers/net/mctp/Kconfig" diff --git a/drivers/net/Makefile b/drivers/net/Makefile index 3f1192d3c52d..6ce076462dbf 100644 --- a/drivers/net/Makefile +++ b/drivers/net/Makefile @@ -23,6 +23,7 @@ obj-$(CONFIG_NET) += loopback.o obj-$(CONFIG_NETDEV_LEGACY_INIT) += Space.o obj-$(CONFIG_NETCONSOLE) += netconsole.o obj-y += phy/ +obj-y += pse-pd/ obj-y += mdio/ obj-y += pcs/ obj-$(CONFIG_RIONET) += rionet.o diff --git a/drivers/net/Space.c b/drivers/net/Space.c index f475eef14390..83214e2e70ab 100644 --- a/drivers/net/Space.c +++ b/drivers/net/Space.c @@ -68,7 +68,7 @@ static int netdev_boot_setup_add(char *name, struct ifmap *map) for (i = 0; i < NETDEV_BOOT_SETUP_MAX; i++) { if (s[i].name[0] == '\0' || s[i].name[0] == ' ') { memset(s[i].name, 0, sizeof(s[i].name)); - strlcpy(s[i].name, name, IFNAMSIZ); + strscpy(s[i].name, name, IFNAMSIZ); memcpy(&s[i].map, map, sizeof(s[i].map)); break; } diff --git a/drivers/net/amt.c b/drivers/net/amt.c index 9a247eb7679c..2d20be6ffb7e 100644 --- a/drivers/net/amt.c +++ b/drivers/net/amt.c @@ -2894,8 +2894,7 @@ static void amt_event_work(struct work_struct *work) amt_event_send_request(amt); break; default: - if (skb) - kfree_skb(skb); + kfree_skb(skb); break; } } @@ -3033,8 +3032,7 @@ static int amt_dev_stop(struct net_device *dev) cancel_work_sync(&amt->event_wq); for (i = 0; i < AMT_MAX_EVENTS; i++) { skb = amt->events[i].skb; - if (skb) - kfree_skb(skb); + kfree_skb(skb); amt->events[i].event = AMT_EVENT_NONE; amt->events[i].skb = NULL; } diff --git a/drivers/net/bonding/bond_main.c b/drivers/net/bonding/bond_main.c index 86d42306aa5e..24bb50dfd362 100644 --- a/drivers/net/bonding/bond_main.c +++ b/drivers/net/bonding/bond_main.c @@ -5650,7 +5650,7 @@ static int bond_ethtool_get_link_ksettings(struct net_device *bond_dev, static void bond_ethtool_get_drvinfo(struct net_device *bond_dev, struct ethtool_drvinfo *drvinfo) { - strlcpy(drvinfo->driver, DRV_NAME, sizeof(drvinfo->driver)); + strscpy(drvinfo->driver, DRV_NAME, sizeof(drvinfo->driver)); snprintf(drvinfo->fw_version, sizeof(drvinfo->fw_version), "%d", BOND_ABI_VERSION); } diff --git a/drivers/net/bonding/bond_sysfs.c b/drivers/net/bonding/bond_sysfs.c index 9b5a5df23d21..8996bd0a194a 100644 --- a/drivers/net/bonding/bond_sysfs.c +++ b/drivers/net/bonding/bond_sysfs.c @@ -47,10 +47,10 @@ static ssize_t bonding_show_bonds(struct class *cls, /* not enough space for another interface name */ if ((PAGE_SIZE - res) > 10) res = PAGE_SIZE - 10; - res += sprintf(buf + res, "++more++ "); + res += sysfs_emit_at(buf, res, "++more++ "); break; } - res += sprintf(buf + res, "%s ", bond->dev->name); + res += sysfs_emit_at(buf, res, "%s ", bond->dev->name); } if (res) buf[res-1] = '\n'; /* eat the leftover space */ @@ -178,10 +178,10 @@ static ssize_t bonding_show_slaves(struct device *d, /* not enough space for another interface name */ if ((PAGE_SIZE - res) > 10) res = PAGE_SIZE - 10; - res += sprintf(buf + res, "++more++ "); + res += sysfs_emit_at(buf, res, "++more++ "); break; } - res += sprintf(buf + res, "%s ", slave->dev->name); + res += sysfs_emit_at(buf, res, "%s ", slave->dev->name); } rtnl_unlock(); @@ -203,7 +203,7 @@ static ssize_t bonding_show_mode(struct device *d, val = bond_opt_get_val(BOND_OPT_MODE, BOND_MODE(bond)); - return sprintf(buf, "%s %d\n", val->string, BOND_MODE(bond)); + return sysfs_emit(buf, "%s %d\n", val->string, BOND_MODE(bond)); } static DEVICE_ATTR(mode, 0644, bonding_show_mode, bonding_sysfs_store_option); @@ -217,7 +217,7 @@ static ssize_t bonding_show_xmit_hash(struct device *d, val = bond_opt_get_val(BOND_OPT_XMIT_HASH, bond->params.xmit_policy); - return sprintf(buf, "%s %d\n", val->string, bond->params.xmit_policy); + return sysfs_emit(buf, "%s %d\n", val->string, bond->params.xmit_policy); } static DEVICE_ATTR(xmit_hash_policy, 0644, bonding_show_xmit_hash, bonding_sysfs_store_option); @@ -233,7 +233,7 @@ static ssize_t bonding_show_arp_validate(struct device *d, val = bond_opt_get_val(BOND_OPT_ARP_VALIDATE, bond->params.arp_validate); - return sprintf(buf, "%s %d\n", val->string, bond->params.arp_validate); + return sysfs_emit(buf, "%s %d\n", val->string, bond->params.arp_validate); } static DEVICE_ATTR(arp_validate, 0644, bonding_show_arp_validate, bonding_sysfs_store_option); @@ -248,7 +248,7 @@ static ssize_t bonding_show_arp_all_targets(struct device *d, val = bond_opt_get_val(BOND_OPT_ARP_ALL_TARGETS, bond->params.arp_all_targets); - return sprintf(buf, "%s %d\n", + return sysfs_emit(buf, "%s %d\n", val->string, bond->params.arp_all_targets); } static DEVICE_ATTR(arp_all_targets, 0644, @@ -265,7 +265,7 @@ static ssize_t bonding_show_fail_over_mac(struct device *d, val = bond_opt_get_val(BOND_OPT_FAIL_OVER_MAC, bond->params.fail_over_mac); - return sprintf(buf, "%s %d\n", val->string, bond->params.fail_over_mac); + return sysfs_emit(buf, "%s %d\n", val->string, bond->params.fail_over_mac); } static DEVICE_ATTR(fail_over_mac, 0644, bonding_show_fail_over_mac, bonding_sysfs_store_option); @@ -277,7 +277,7 @@ static ssize_t bonding_show_arp_interval(struct device *d, { struct bonding *bond = to_bond(d); - return sprintf(buf, "%d\n", bond->params.arp_interval); + return sysfs_emit(buf, "%d\n", bond->params.arp_interval); } static DEVICE_ATTR(arp_interval, 0644, bonding_show_arp_interval, bonding_sysfs_store_option); @@ -292,8 +292,8 @@ static ssize_t bonding_show_arp_targets(struct device *d, for (i = 0; i < BOND_MAX_ARP_TARGETS; i++) { if (bond->params.arp_targets[i]) - res += sprintf(buf + res, "%pI4 ", - &bond->params.arp_targets[i]); + res += sysfs_emit_at(buf, res, "%pI4 ", + &bond->params.arp_targets[i]); } if (res) buf[res-1] = '\n'; /* eat the leftover space */ @@ -310,7 +310,7 @@ static ssize_t bonding_show_missed_max(struct device *d, { struct bonding *bond = to_bond(d); - return sprintf(buf, "%u\n", bond->params.missed_max); + return sysfs_emit(buf, "%u\n", bond->params.missed_max); } static DEVICE_ATTR(arp_missed_max, 0644, bonding_show_missed_max, bonding_sysfs_store_option); @@ -322,7 +322,7 @@ static ssize_t bonding_show_downdelay(struct device *d, { struct bonding *bond = to_bond(d); - return sprintf(buf, "%d\n", bond->params.downdelay * bond->params.miimon); + return sysfs_emit(buf, "%d\n", bond->params.downdelay * bond->params.miimon); } static DEVICE_ATTR(downdelay, 0644, bonding_show_downdelay, bonding_sysfs_store_option); @@ -333,7 +333,7 @@ static ssize_t bonding_show_updelay(struct device *d, { struct bonding *bond = to_bond(d); - return sprintf(buf, "%d\n", bond->params.updelay * bond->params.miimon); + return sysfs_emit(buf, "%d\n", bond->params.updelay * bond->params.miimon); } static DEVICE_ATTR(updelay, 0644, @@ -345,8 +345,8 @@ static ssize_t bonding_show_peer_notif_delay(struct device *d, { struct bonding *bond = to_bond(d); - return sprintf(buf, "%d\n", - bond->params.peer_notif_delay * bond->params.miimon); + return sysfs_emit(buf, "%d\n", + bond->params.peer_notif_delay * bond->params.miimon); } static DEVICE_ATTR(peer_notif_delay, 0644, bonding_show_peer_notif_delay, bonding_sysfs_store_option); @@ -361,7 +361,7 @@ static ssize_t bonding_show_lacp_active(struct device *d, val = bond_opt_get_val(BOND_OPT_LACP_ACTIVE, bond->params.lacp_active); - return sprintf(buf, "%s %d\n", val->string, bond->params.lacp_active); + return sysfs_emit(buf, "%s %d\n", val->string, bond->params.lacp_active); } static DEVICE_ATTR(lacp_active, 0644, bonding_show_lacp_active, bonding_sysfs_store_option); @@ -375,7 +375,7 @@ static ssize_t bonding_show_lacp_rate(struct device *d, val = bond_opt_get_val(BOND_OPT_LACP_RATE, bond->params.lacp_fast); - return sprintf(buf, "%s %d\n", val->string, bond->params.lacp_fast); + return sysfs_emit(buf, "%s %d\n", val->string, bond->params.lacp_fast); } static DEVICE_ATTR(lacp_rate, 0644, bonding_show_lacp_rate, bonding_sysfs_store_option); @@ -386,7 +386,7 @@ static ssize_t bonding_show_min_links(struct device *d, { struct bonding *bond = to_bond(d); - return sprintf(buf, "%u\n", bond->params.min_links); + return sysfs_emit(buf, "%u\n", bond->params.min_links); } static DEVICE_ATTR(min_links, 0644, bonding_show_min_links, bonding_sysfs_store_option); @@ -400,7 +400,7 @@ static ssize_t bonding_show_ad_select(struct device *d, val = bond_opt_get_val(BOND_OPT_AD_SELECT, bond->params.ad_select); - return sprintf(buf, "%s %d\n", val->string, bond->params.ad_select); + return sysfs_emit(buf, "%s %d\n", val->string, bond->params.ad_select); } static DEVICE_ATTR(ad_select, 0644, bonding_show_ad_select, bonding_sysfs_store_option); @@ -412,7 +412,7 @@ static ssize_t bonding_show_num_peer_notif(struct device *d, { struct bonding *bond = to_bond(d); - return sprintf(buf, "%d\n", bond->params.num_peer_notif); + return sysfs_emit(buf, "%d\n", bond->params.num_peer_notif); } static DEVICE_ATTR(num_grat_arp, 0644, bonding_show_num_peer_notif, bonding_sysfs_store_option); @@ -426,7 +426,7 @@ static ssize_t bonding_show_miimon(struct device *d, { struct bonding *bond = to_bond(d); - return sprintf(buf, "%d\n", bond->params.miimon); + return sysfs_emit(buf, "%d\n", bond->params.miimon); } static DEVICE_ATTR(miimon, 0644, bonding_show_miimon, bonding_sysfs_store_option); @@ -443,7 +443,7 @@ static ssize_t bonding_show_primary(struct device *d, rcu_read_lock(); primary = rcu_dereference(bond->primary_slave); if (primary) - count = sprintf(buf, "%s\n", primary->dev->name); + count = sysfs_emit(buf, "%s\n", primary->dev->name); rcu_read_unlock(); return count; @@ -462,8 +462,8 @@ static ssize_t bonding_show_primary_reselect(struct device *d, val = bond_opt_get_val(BOND_OPT_PRIMARY_RESELECT, bond->params.primary_reselect); - return sprintf(buf, "%s %d\n", - val->string, bond->params.primary_reselect); + return sysfs_emit(buf, "%s %d\n", + val->string, bond->params.primary_reselect); } static DEVICE_ATTR(primary_reselect, 0644, bonding_show_primary_reselect, bonding_sysfs_store_option); @@ -475,7 +475,7 @@ static ssize_t bonding_show_carrier(struct device *d, { struct bonding *bond = to_bond(d); - return sprintf(buf, "%d\n", bond->params.use_carrier); + return sysfs_emit(buf, "%d\n", bond->params.use_carrier); } static DEVICE_ATTR(use_carrier, 0644, bonding_show_carrier, bonding_sysfs_store_option); @@ -493,7 +493,7 @@ static ssize_t bonding_show_active_slave(struct device *d, rcu_read_lock(); slave_dev = bond_option_active_slave_get_rcu(bond); if (slave_dev) - count = sprintf(buf, "%s\n", slave_dev->name); + count = sysfs_emit(buf, "%s\n", slave_dev->name); rcu_read_unlock(); return count; @@ -509,7 +509,7 @@ static ssize_t bonding_show_mii_status(struct device *d, struct bonding *bond = to_bond(d); bool active = netif_carrier_ok(bond->dev); - return sprintf(buf, "%s\n", active ? "up" : "down"); + return sysfs_emit(buf, "%s\n", active ? "up" : "down"); } static DEVICE_ATTR(mii_status, 0444, bonding_show_mii_status, NULL); @@ -524,9 +524,9 @@ static ssize_t bonding_show_ad_aggregator(struct device *d, if (BOND_MODE(bond) == BOND_MODE_8023AD) { struct ad_info ad_info; - count = sprintf(buf, "%d\n", - bond_3ad_get_active_agg_info(bond, &ad_info) - ? 0 : ad_info.aggregator_id); + count = sysfs_emit(buf, "%d\n", + bond_3ad_get_active_agg_info(bond, &ad_info) + ? 0 : ad_info.aggregator_id); } return count; @@ -545,9 +545,9 @@ static ssize_t bonding_show_ad_num_ports(struct device *d, if (BOND_MODE(bond) == BOND_MODE_8023AD) { struct ad_info ad_info; - count = sprintf(buf, "%d\n", - bond_3ad_get_active_agg_info(bond, &ad_info) - ? 0 : ad_info.ports); + count = sysfs_emit(buf, "%d\n", + bond_3ad_get_active_agg_info(bond, &ad_info) + ? 0 : ad_info.ports); } return count; @@ -566,9 +566,9 @@ static ssize_t bonding_show_ad_actor_key(struct device *d, if (BOND_MODE(bond) == BOND_MODE_8023AD && capable(CAP_NET_ADMIN)) { struct ad_info ad_info; - count = sprintf(buf, "%d\n", - bond_3ad_get_active_agg_info(bond, &ad_info) - ? 0 : ad_info.actor_key); + count = sysfs_emit(buf, "%d\n", + bond_3ad_get_active_agg_info(bond, &ad_info) + ? 0 : ad_info.actor_key); } return count; @@ -587,9 +587,9 @@ static ssize_t bonding_show_ad_partner_key(struct device *d, if (BOND_MODE(bond) == BOND_MODE_8023AD && capable(CAP_NET_ADMIN)) { struct ad_info ad_info; - count = sprintf(buf, "%d\n", - bond_3ad_get_active_agg_info(bond, &ad_info) - ? 0 : ad_info.partner_key); + count = sysfs_emit(buf, "%d\n", + bond_3ad_get_active_agg_info(bond, &ad_info) + ? 0 : ad_info.partner_key); } return count; @@ -609,7 +609,7 @@ static ssize_t bonding_show_ad_partner_mac(struct device *d, struct ad_info ad_info; if (!bond_3ad_get_active_agg_info(bond, &ad_info)) - count = sprintf(buf, "%pM\n", ad_info.partner_system); + count = sysfs_emit(buf, "%pM\n", ad_info.partner_system); } return count; @@ -634,11 +634,11 @@ static ssize_t bonding_show_queue_id(struct device *d, /* not enough space for another interface_name:queue_id pair */ if ((PAGE_SIZE - res) > 10) res = PAGE_SIZE - 10; - res += sprintf(buf + res, "++more++ "); + res += sysfs_emit_at(buf, res, "++more++ "); break; } - res += sprintf(buf + res, "%s:%d ", - slave->dev->name, slave->queue_id); + res += sysfs_emit_at(buf, res, "%s:%d ", + slave->dev->name, slave->queue_id); } if (res) buf[res-1] = '\n'; /* eat the leftover space */ @@ -658,7 +658,7 @@ static ssize_t bonding_show_slaves_active(struct device *d, { struct bonding *bond = to_bond(d); - return sprintf(buf, "%d\n", bond->params.all_slaves_active); + return sysfs_emit(buf, "%d\n", bond->params.all_slaves_active); } static DEVICE_ATTR(all_slaves_active, 0644, bonding_show_slaves_active, bonding_sysfs_store_option); @@ -670,7 +670,7 @@ static ssize_t bonding_show_resend_igmp(struct device *d, { struct bonding *bond = to_bond(d); - return sprintf(buf, "%d\n", bond->params.resend_igmp); + return sysfs_emit(buf, "%d\n", bond->params.resend_igmp); } static DEVICE_ATTR(resend_igmp, 0644, bonding_show_resend_igmp, bonding_sysfs_store_option); @@ -682,7 +682,7 @@ static ssize_t bonding_show_lp_interval(struct device *d, { struct bonding *bond = to_bond(d); - return sprintf(buf, "%d\n", bond->params.lp_interval); + return sysfs_emit(buf, "%d\n", bond->params.lp_interval); } static DEVICE_ATTR(lp_interval, 0644, bonding_show_lp_interval, bonding_sysfs_store_option); @@ -693,7 +693,7 @@ static ssize_t bonding_show_tlb_dynamic_lb(struct device *d, { struct bonding *bond = to_bond(d); - return sprintf(buf, "%d\n", bond->params.tlb_dynamic_lb); + return sysfs_emit(buf, "%d\n", bond->params.tlb_dynamic_lb); } static DEVICE_ATTR(tlb_dynamic_lb, 0644, bonding_show_tlb_dynamic_lb, bonding_sysfs_store_option); @@ -705,7 +705,7 @@ static ssize_t bonding_show_packets_per_slave(struct device *d, struct bonding *bond = to_bond(d); unsigned int packets_per_slave = bond->params.packets_per_slave; - return sprintf(buf, "%u\n", packets_per_slave); + return sysfs_emit(buf, "%u\n", packets_per_slave); } static DEVICE_ATTR(packets_per_slave, 0644, bonding_show_packets_per_slave, bonding_sysfs_store_option); @@ -717,7 +717,7 @@ static ssize_t bonding_show_ad_actor_sys_prio(struct device *d, struct bonding *bond = to_bond(d); if (BOND_MODE(bond) == BOND_MODE_8023AD && capable(CAP_NET_ADMIN)) - return sprintf(buf, "%hu\n", bond->params.ad_actor_sys_prio); + return sysfs_emit(buf, "%hu\n", bond->params.ad_actor_sys_prio); return 0; } @@ -731,7 +731,7 @@ static ssize_t bonding_show_ad_actor_system(struct device *d, struct bonding *bond = to_bond(d); if (BOND_MODE(bond) == BOND_MODE_8023AD && capable(CAP_NET_ADMIN)) - return sprintf(buf, "%pM\n", bond->params.ad_actor_system); + return sysfs_emit(buf, "%pM\n", bond->params.ad_actor_system); return 0; } @@ -746,7 +746,7 @@ static ssize_t bonding_show_ad_user_port_key(struct device *d, struct bonding *bond = to_bond(d); if (BOND_MODE(bond) == BOND_MODE_8023AD && capable(CAP_NET_ADMIN)) - return sprintf(buf, "%hu\n", bond->params.ad_user_port_key); + return sysfs_emit(buf, "%hu\n", bond->params.ad_user_port_key); return 0; } diff --git a/drivers/net/bonding/bond_sysfs_slave.c b/drivers/net/bonding/bond_sysfs_slave.c index 69b0a3751dff..313866f2c0e4 100644 --- a/drivers/net/bonding/bond_sysfs_slave.c +++ b/drivers/net/bonding/bond_sysfs_slave.c @@ -22,30 +22,30 @@ static ssize_t state_show(struct slave *slave, char *buf) { switch (bond_slave_state(slave)) { case BOND_STATE_ACTIVE: - return sprintf(buf, "active\n"); + return sysfs_emit(buf, "active\n"); case BOND_STATE_BACKUP: - return sprintf(buf, "backup\n"); + return sysfs_emit(buf, "backup\n"); default: - return sprintf(buf, "UNKNOWN\n"); + return sysfs_emit(buf, "UNKNOWN\n"); } } static SLAVE_ATTR_RO(state); static ssize_t mii_status_show(struct slave *slave, char *buf) { - return sprintf(buf, "%s\n", bond_slave_link_status(slave->link)); + return sysfs_emit(buf, "%s\n", bond_slave_link_status(slave->link)); } static SLAVE_ATTR_RO(mii_status); static ssize_t link_failure_count_show(struct slave *slave, char *buf) { - return sprintf(buf, "%d\n", slave->link_failure_count); + return sysfs_emit(buf, "%d\n", slave->link_failure_count); } static SLAVE_ATTR_RO(link_failure_count); static ssize_t perm_hwaddr_show(struct slave *slave, char *buf) { - return sprintf(buf, "%*phC\n", + return sysfs_emit(buf, "%*phC\n", slave->dev->addr_len, slave->perm_hwaddr); } @@ -53,7 +53,7 @@ static SLAVE_ATTR_RO(perm_hwaddr); static ssize_t queue_id_show(struct slave *slave, char *buf) { - return sprintf(buf, "%d\n", slave->queue_id); + return sysfs_emit(buf, "%d\n", slave->queue_id); } static SLAVE_ATTR_RO(queue_id); @@ -64,11 +64,11 @@ static ssize_t ad_aggregator_id_show(struct slave *slave, char *buf) if (BOND_MODE(slave->bond) == BOND_MODE_8023AD) { agg = SLAVE_AD_INFO(slave)->port.aggregator; if (agg) - return sprintf(buf, "%d\n", - agg->aggregator_identifier); + return sysfs_emit(buf, "%d\n", + agg->aggregator_identifier); } - return sprintf(buf, "N/A\n"); + return sysfs_emit(buf, "N/A\n"); } static SLAVE_ATTR_RO(ad_aggregator_id); @@ -79,11 +79,11 @@ static ssize_t ad_actor_oper_port_state_show(struct slave *slave, char *buf) if (BOND_MODE(slave->bond) == BOND_MODE_8023AD) { ad_port = &SLAVE_AD_INFO(slave)->port; if (ad_port->aggregator) - return sprintf(buf, "%u\n", + return sysfs_emit(buf, "%u\n", ad_port->actor_oper_port_state); } - return sprintf(buf, "N/A\n"); + return sysfs_emit(buf, "N/A\n"); } static SLAVE_ATTR_RO(ad_actor_oper_port_state); @@ -94,11 +94,11 @@ static ssize_t ad_partner_oper_port_state_show(struct slave *slave, char *buf) if (BOND_MODE(slave->bond) == BOND_MODE_8023AD) { ad_port = &SLAVE_AD_INFO(slave)->port; if (ad_port->aggregator) - return sprintf(buf, "%u\n", + return sysfs_emit(buf, "%u\n", ad_port->partner_oper.port_state); } - return sprintf(buf, "N/A\n"); + return sysfs_emit(buf, "N/A\n"); } static SLAVE_ATTR_RO(ad_partner_oper_port_state); diff --git a/drivers/net/can/ctucanfd/ctucanfd_base.c b/drivers/net/can/ctucanfd/ctucanfd_base.c index 3c18d028bd8c..b8da15ea6ad9 100644 --- a/drivers/net/can/ctucanfd/ctucanfd_base.c +++ b/drivers/net/can/ctucanfd/ctucanfd_base.c @@ -657,7 +657,6 @@ static void ctucan_read_rx_frame(struct ctucan_priv *priv, struct canfd_frame *c cf->can_id = (idw >> 18) & CAN_SFF_MASK; /* BRS, ESI, RTR Flags */ - cf->flags = 0; if (FIELD_GET(REG_FRAME_FORMAT_W_FDF, ffw)) { if (FIELD_GET(REG_FRAME_FORMAT_W_BRS, ffw)) cf->flags |= CANFD_BRS; @@ -1425,7 +1424,7 @@ int ctucan_probe_common(struct device *dev, void __iomem *addr, int irq, unsigne priv->can.clock.freq = can_clk_rate; - netif_napi_add(ndev, &priv->napi, ctucan_rx_poll, NAPI_POLL_WEIGHT); + netif_napi_add(ndev, &priv->napi, ctucan_rx_poll); ret = register_candev(ndev); if (ret) { diff --git a/drivers/net/can/ctucanfd/ctucanfd_platform.c b/drivers/net/can/ctucanfd/ctucanfd_platform.c index 89d54c2151e1..f83684f006ea 100644 --- a/drivers/net/can/ctucanfd/ctucanfd_platform.c +++ b/drivers/net/can/ctucanfd/ctucanfd_platform.c @@ -58,7 +58,6 @@ static int ctucan_platform_probe(struct platform_device *pdev) res = platform_get_resource(pdev, IORESOURCE_MEM, 0); addr = devm_ioremap_resource(dev, res); if (IS_ERR(addr)) { - dev_err(dev, "Cannot remap address.\n"); ret = PTR_ERR(addr); goto err; } diff --git a/drivers/net/can/dev/rx-offload.c b/drivers/net/can/dev/rx-offload.c index a32a01c172d4..81ebf0562c89 100644 --- a/drivers/net/can/dev/rx-offload.c +++ b/drivers/net/can/dev/rx-offload.c @@ -247,7 +247,7 @@ unsigned int can_rx_offload_get_echo_skb(struct can_rx_offload *offload, struct net_device *dev = offload->dev; struct net_device_stats *stats = &dev->stats; struct sk_buff *skb; - u8 len; + unsigned int len; int err; skb = __can_get_echo_skb(dev, idx, &len, frame_len_ptr); @@ -329,7 +329,7 @@ static int can_rx_offload_init_queue(struct net_device *dev, { offload->dev = dev; - /* Limit queue len to 4x the weight (rounted to next power of two) */ + /* Limit queue len to 4x the weight (rounded to next power of two) */ offload->skb_queue_len_max = 2 << fls(weight); offload->skb_queue_len_max *= 4; skb_queue_head_init(&offload->skb_queue); diff --git a/drivers/net/can/dev/skb.c b/drivers/net/can/dev/skb.c index 07e0feac8629..791a51e2f5d6 100644 --- a/drivers/net/can/dev/skb.c +++ b/drivers/net/can/dev/skb.c @@ -91,8 +91,8 @@ int can_put_echo_skb(struct sk_buff *skb, struct net_device *dev, EXPORT_SYMBOL_GPL(can_put_echo_skb); struct sk_buff * -__can_get_echo_skb(struct net_device *dev, unsigned int idx, u8 *len_ptr, - unsigned int *frame_len_ptr) +__can_get_echo_skb(struct net_device *dev, unsigned int idx, + unsigned int *len_ptr, unsigned int *frame_len_ptr) { struct can_priv *priv = netdev_priv(dev); @@ -108,16 +108,12 @@ __can_get_echo_skb(struct net_device *dev, unsigned int idx, u8 *len_ptr, */ struct sk_buff *skb = priv->echo_skb[idx]; struct can_skb_priv *can_skb_priv = can_skb_prv(skb); - struct canfd_frame *cf = (struct canfd_frame *)skb->data; if (skb_shinfo(skb)->tx_flags & SKBTX_IN_PROGRESS) skb_tstamp_tx(skb, skb_hwtstamps(skb)); /* get the real payload length for netdev statistics */ - if (cf->can_id & CAN_RTR_FLAG) - *len_ptr = 0; - else - *len_ptr = cf->len; + *len_ptr = can_skb_get_data_len(skb); if (frame_len_ptr) *frame_len_ptr = can_skb_priv->frame_len; @@ -147,7 +143,7 @@ unsigned int can_get_echo_skb(struct net_device *dev, unsigned int idx, unsigned int *frame_len_ptr) { struct sk_buff *skb; - u8 len; + unsigned int len; skb = __can_get_echo_skb(dev, idx, &len, frame_len_ptr); if (!skb) @@ -191,6 +187,20 @@ void can_free_echo_skb(struct net_device *dev, unsigned int idx, } EXPORT_SYMBOL_GPL(can_free_echo_skb); +/* fill common values for CAN sk_buffs */ +static void init_can_skb_reserve(struct sk_buff *skb) +{ + skb->pkt_type = PACKET_BROADCAST; + skb->ip_summed = CHECKSUM_UNNECESSARY; + + skb_reset_mac_header(skb); + skb_reset_network_header(skb); + skb_reset_transport_header(skb); + + can_skb_reserve(skb); + can_skb_prv(skb)->skbcnt = 0; +} + struct sk_buff *alloc_can_skb(struct net_device *dev, struct can_frame **cf) { struct sk_buff *skb; @@ -204,16 +214,8 @@ struct sk_buff *alloc_can_skb(struct net_device *dev, struct can_frame **cf) } skb->protocol = htons(ETH_P_CAN); - skb->pkt_type = PACKET_BROADCAST; - skb->ip_summed = CHECKSUM_UNNECESSARY; - - skb_reset_mac_header(skb); - skb_reset_network_header(skb); - skb_reset_transport_header(skb); - - can_skb_reserve(skb); + init_can_skb_reserve(skb); can_skb_prv(skb)->ifindex = dev->ifindex; - can_skb_prv(skb)->skbcnt = 0; *cf = skb_put_zero(skb, sizeof(struct can_frame)); @@ -235,23 +237,51 @@ struct sk_buff *alloc_canfd_skb(struct net_device *dev, } skb->protocol = htons(ETH_P_CANFD); - skb->pkt_type = PACKET_BROADCAST; - skb->ip_summed = CHECKSUM_UNNECESSARY; - - skb_reset_mac_header(skb); - skb_reset_network_header(skb); - skb_reset_transport_header(skb); - - can_skb_reserve(skb); + init_can_skb_reserve(skb); can_skb_prv(skb)->ifindex = dev->ifindex; - can_skb_prv(skb)->skbcnt = 0; *cfd = skb_put_zero(skb, sizeof(struct canfd_frame)); + /* set CAN FD flag by default */ + (*cfd)->flags = CANFD_FDF; + return skb; } EXPORT_SYMBOL_GPL(alloc_canfd_skb); +struct sk_buff *alloc_canxl_skb(struct net_device *dev, + struct canxl_frame **cxl, + unsigned int data_len) +{ + struct sk_buff *skb; + + if (data_len < CANXL_MIN_DLEN || data_len > CANXL_MAX_DLEN) + goto out_error; + + skb = netdev_alloc_skb(dev, sizeof(struct can_skb_priv) + + CANXL_HDR_SIZE + data_len); + if (unlikely(!skb)) + goto out_error; + + skb->protocol = htons(ETH_P_CANXL); + init_can_skb_reserve(skb); + can_skb_prv(skb)->ifindex = dev->ifindex; + + *cxl = skb_put_zero(skb, CANXL_HDR_SIZE + data_len); + + /* set CAN XL flag and length information by default */ + (*cxl)->flags = CANXL_XLF; + (*cxl)->len = data_len; + + return skb; + +out_error: + *cxl = NULL; + + return NULL; +} +EXPORT_SYMBOL_GPL(alloc_canxl_skb); + struct sk_buff *alloc_can_err_skb(struct net_device *dev, struct can_frame **cf) { struct sk_buff *skb; @@ -291,6 +321,14 @@ static bool can_skb_headroom_valid(struct net_device *dev, struct sk_buff *skb) skb_reset_mac_header(skb); skb_reset_network_header(skb); skb_reset_transport_header(skb); + + /* set CANFD_FDF flag for CAN FD frames */ + if (can_is_canfd_skb(skb)) { + struct canfd_frame *cfd; + + cfd = (struct canfd_frame *)skb->data; + cfd->flags |= CANFD_FDF; + } } return true; @@ -299,18 +337,25 @@ static bool can_skb_headroom_valid(struct net_device *dev, struct sk_buff *skb) /* Drop a given socketbuffer if it does not contain a valid CAN frame. */ bool can_dropped_invalid_skb(struct net_device *dev, struct sk_buff *skb) { - const struct canfd_frame *cfd = (struct canfd_frame *)skb->data; struct can_priv *priv = netdev_priv(dev); - if (skb->protocol == htons(ETH_P_CAN)) { - if (unlikely(skb->len != CAN_MTU || - cfd->len > CAN_MAX_DLEN)) + switch (ntohs(skb->protocol)) { + case ETH_P_CAN: + if (!can_is_can_skb(skb)) goto inval_skb; - } else if (skb->protocol == htons(ETH_P_CANFD)) { - if (unlikely(skb->len != CANFD_MTU || - cfd->len > CANFD_MAX_DLEN)) + break; + + case ETH_P_CANFD: + if (!can_is_canfd_skb(skb)) goto inval_skb; - } else { + break; + + case ETH_P_CANXL: + if (!can_is_canxl_skb(skb)) + goto inval_skb; + break; + + default: goto inval_skb; } diff --git a/drivers/net/can/flexcan/flexcan-core.c b/drivers/net/can/flexcan/flexcan-core.c index ccb438eca517..5ee38e586fd8 100644 --- a/drivers/net/can/flexcan/flexcan-core.c +++ b/drivers/net/can/flexcan/flexcan-core.c @@ -295,45 +295,45 @@ static_assert(sizeof(struct flexcan_regs) == 0x4 * 18 + 0xfb8); static const struct flexcan_devtype_data fsl_mcf5441x_devtype_data = { .quirks = FLEXCAN_QUIRK_BROKEN_PERR_STATE | FLEXCAN_QUIRK_NR_IRQ_3 | FLEXCAN_QUIRK_NR_MB_16 | - FLEXCAN_QUIRK_SUPPPORT_RX_MAILBOX | - FLEXCAN_QUIRK_SUPPPORT_RX_FIFO, + FLEXCAN_QUIRK_SUPPORT_RX_MAILBOX | + FLEXCAN_QUIRK_SUPPORT_RX_FIFO, }; static const struct flexcan_devtype_data fsl_p1010_devtype_data = { .quirks = FLEXCAN_QUIRK_BROKEN_WERR_STATE | FLEXCAN_QUIRK_BROKEN_PERR_STATE | FLEXCAN_QUIRK_DEFAULT_BIG_ENDIAN | - FLEXCAN_QUIRK_SUPPPORT_RX_MAILBOX | - FLEXCAN_QUIRK_SUPPPORT_RX_FIFO, + FLEXCAN_QUIRK_SUPPORT_RX_MAILBOX | + FLEXCAN_QUIRK_SUPPORT_RX_FIFO, }; static const struct flexcan_devtype_data fsl_imx25_devtype_data = { .quirks = FLEXCAN_QUIRK_BROKEN_WERR_STATE | FLEXCAN_QUIRK_BROKEN_PERR_STATE | - FLEXCAN_QUIRK_SUPPPORT_RX_MAILBOX | - FLEXCAN_QUIRK_SUPPPORT_RX_FIFO, + FLEXCAN_QUIRK_SUPPORT_RX_MAILBOX | + FLEXCAN_QUIRK_SUPPORT_RX_FIFO, }; static const struct flexcan_devtype_data fsl_imx28_devtype_data = { .quirks = FLEXCAN_QUIRK_BROKEN_PERR_STATE | - FLEXCAN_QUIRK_SUPPPORT_RX_MAILBOX | - FLEXCAN_QUIRK_SUPPPORT_RX_FIFO, + FLEXCAN_QUIRK_SUPPORT_RX_MAILBOX | + FLEXCAN_QUIRK_SUPPORT_RX_FIFO, }; static const struct flexcan_devtype_data fsl_imx6q_devtype_data = { .quirks = FLEXCAN_QUIRK_DISABLE_RXFG | FLEXCAN_QUIRK_ENABLE_EACEN_RRS | FLEXCAN_QUIRK_USE_RX_MAILBOX | FLEXCAN_QUIRK_BROKEN_PERR_STATE | FLEXCAN_QUIRK_SETUP_STOP_MODE_GPR | - FLEXCAN_QUIRK_SUPPPORT_RX_MAILBOX | - FLEXCAN_QUIRK_SUPPPORT_RX_MAILBOX_RTR, + FLEXCAN_QUIRK_SUPPORT_RX_MAILBOX | + FLEXCAN_QUIRK_SUPPORT_RX_MAILBOX_RTR, }; static const struct flexcan_devtype_data fsl_imx8qm_devtype_data = { .quirks = FLEXCAN_QUIRK_DISABLE_RXFG | FLEXCAN_QUIRK_ENABLE_EACEN_RRS | FLEXCAN_QUIRK_USE_RX_MAILBOX | FLEXCAN_QUIRK_BROKEN_PERR_STATE | FLEXCAN_QUIRK_SUPPORT_FD | FLEXCAN_QUIRK_SETUP_STOP_MODE_SCFW | - FLEXCAN_QUIRK_SUPPPORT_RX_MAILBOX | - FLEXCAN_QUIRK_SUPPPORT_RX_MAILBOX_RTR, + FLEXCAN_QUIRK_SUPPORT_RX_MAILBOX | + FLEXCAN_QUIRK_SUPPORT_RX_MAILBOX_RTR, }; static struct flexcan_devtype_data fsl_imx8mp_devtype_data = { @@ -341,23 +341,23 @@ static struct flexcan_devtype_data fsl_imx8mp_devtype_data = { FLEXCAN_QUIRK_DISABLE_MECR | FLEXCAN_QUIRK_USE_RX_MAILBOX | FLEXCAN_QUIRK_BROKEN_PERR_STATE | FLEXCAN_QUIRK_SETUP_STOP_MODE_GPR | FLEXCAN_QUIRK_SUPPORT_FD | FLEXCAN_QUIRK_SUPPORT_ECC | - FLEXCAN_QUIRK_SUPPPORT_RX_MAILBOX | - FLEXCAN_QUIRK_SUPPPORT_RX_MAILBOX_RTR, + FLEXCAN_QUIRK_SUPPORT_RX_MAILBOX | + FLEXCAN_QUIRK_SUPPORT_RX_MAILBOX_RTR, }; static const struct flexcan_devtype_data fsl_vf610_devtype_data = { .quirks = FLEXCAN_QUIRK_DISABLE_RXFG | FLEXCAN_QUIRK_ENABLE_EACEN_RRS | FLEXCAN_QUIRK_DISABLE_MECR | FLEXCAN_QUIRK_USE_RX_MAILBOX | FLEXCAN_QUIRK_BROKEN_PERR_STATE | FLEXCAN_QUIRK_SUPPORT_ECC | - FLEXCAN_QUIRK_SUPPPORT_RX_MAILBOX | - FLEXCAN_QUIRK_SUPPPORT_RX_MAILBOX_RTR, + FLEXCAN_QUIRK_SUPPORT_RX_MAILBOX | + FLEXCAN_QUIRK_SUPPORT_RX_MAILBOX_RTR, }; static const struct flexcan_devtype_data fsl_ls1021a_r2_devtype_data = { .quirks = FLEXCAN_QUIRK_DISABLE_RXFG | FLEXCAN_QUIRK_ENABLE_EACEN_RRS | FLEXCAN_QUIRK_BROKEN_PERR_STATE | FLEXCAN_QUIRK_USE_RX_MAILBOX | - FLEXCAN_QUIRK_SUPPPORT_RX_MAILBOX | - FLEXCAN_QUIRK_SUPPPORT_RX_MAILBOX_RTR, + FLEXCAN_QUIRK_SUPPORT_RX_MAILBOX | + FLEXCAN_QUIRK_SUPPORT_RX_MAILBOX_RTR, }; static const struct flexcan_devtype_data fsl_lx2160a_r1_devtype_data = { @@ -365,8 +365,8 @@ static const struct flexcan_devtype_data fsl_lx2160a_r1_devtype_data = { FLEXCAN_QUIRK_DISABLE_MECR | FLEXCAN_QUIRK_BROKEN_PERR_STATE | FLEXCAN_QUIRK_USE_RX_MAILBOX | FLEXCAN_QUIRK_SUPPORT_FD | FLEXCAN_QUIRK_SUPPORT_ECC | - FLEXCAN_QUIRK_SUPPPORT_RX_MAILBOX | - FLEXCAN_QUIRK_SUPPPORT_RX_MAILBOX_RTR, + FLEXCAN_QUIRK_SUPPORT_RX_MAILBOX | + FLEXCAN_QUIRK_SUPPORT_RX_MAILBOX_RTR, }; static const struct can_bittiming_const flexcan_bittiming_const = { @@ -2085,20 +2085,20 @@ static int flexcan_probe(struct platform_device *pdev) if ((devtype_data->quirks & FLEXCAN_QUIRK_SUPPORT_FD) && !((devtype_data->quirks & (FLEXCAN_QUIRK_USE_RX_MAILBOX | - FLEXCAN_QUIRK_SUPPPORT_RX_MAILBOX | - FLEXCAN_QUIRK_SUPPPORT_RX_MAILBOX_RTR | - FLEXCAN_QUIRK_SUPPPORT_RX_FIFO)) == + FLEXCAN_QUIRK_SUPPORT_RX_MAILBOX | + FLEXCAN_QUIRK_SUPPORT_RX_MAILBOX_RTR | + FLEXCAN_QUIRK_SUPPORT_RX_FIFO)) == (FLEXCAN_QUIRK_USE_RX_MAILBOX | - FLEXCAN_QUIRK_SUPPPORT_RX_MAILBOX | - FLEXCAN_QUIRK_SUPPPORT_RX_MAILBOX_RTR))) { + FLEXCAN_QUIRK_SUPPORT_RX_MAILBOX | + FLEXCAN_QUIRK_SUPPORT_RX_MAILBOX_RTR))) { dev_err(&pdev->dev, "CAN-FD mode doesn't work in RX-FIFO mode!\n"); return -EINVAL; } if ((devtype_data->quirks & - (FLEXCAN_QUIRK_SUPPPORT_RX_MAILBOX | - FLEXCAN_QUIRK_SUPPPORT_RX_MAILBOX_RTR)) == - FLEXCAN_QUIRK_SUPPPORT_RX_MAILBOX_RTR) { + (FLEXCAN_QUIRK_SUPPORT_RX_MAILBOX | + FLEXCAN_QUIRK_SUPPORT_RX_MAILBOX_RTR)) == + FLEXCAN_QUIRK_SUPPORT_RX_MAILBOX_RTR) { dev_err(&pdev->dev, "Quirks (0x%08x) inconsistent: RX_MAILBOX_RX supported but not RX_MAILBOX\n", devtype_data->quirks); @@ -2177,8 +2177,7 @@ static int flexcan_probe(struct platform_device *pdev) err = flexcan_setup_stop_mode(pdev); if (err < 0) { - if (err != -EPROBE_DEFER) - dev_err(&pdev->dev, "setup stop mode failed\n"); + dev_err_probe(&pdev->dev, err, "setup stop mode failed\n"); goto failed_setup_stop_mode; } diff --git a/drivers/net/can/flexcan/flexcan.h b/drivers/net/can/flexcan/flexcan.h index 8621a8ea1dea..025c3417031f 100644 --- a/drivers/net/can/flexcan/flexcan.h +++ b/drivers/net/can/flexcan/flexcan.h @@ -63,11 +63,11 @@ /* Setup 16 mailboxes */ #define FLEXCAN_QUIRK_NR_MB_16 BIT(13) /* Device supports RX via mailboxes */ -#define FLEXCAN_QUIRK_SUPPPORT_RX_MAILBOX BIT(14) +#define FLEXCAN_QUIRK_SUPPORT_RX_MAILBOX BIT(14) /* Device supports RTR reception via mailboxes */ -#define FLEXCAN_QUIRK_SUPPPORT_RX_MAILBOX_RTR BIT(15) +#define FLEXCAN_QUIRK_SUPPORT_RX_MAILBOX_RTR BIT(15) /* Device supports RX via FIFO */ -#define FLEXCAN_QUIRK_SUPPPORT_RX_FIFO BIT(16) +#define FLEXCAN_QUIRK_SUPPORT_RX_FIFO BIT(16) struct flexcan_devtype_data { u32 quirks; /* quirks needed for different IP cores */ @@ -121,7 +121,7 @@ flexcan_supports_rx_mailbox(const struct flexcan_priv *priv) { const u32 quirks = priv->devtype_data.quirks; - return quirks & FLEXCAN_QUIRK_SUPPPORT_RX_MAILBOX; + return quirks & FLEXCAN_QUIRK_SUPPORT_RX_MAILBOX; } static inline bool @@ -129,10 +129,10 @@ flexcan_supports_rx_mailbox_rtr(const struct flexcan_priv *priv) { const u32 quirks = priv->devtype_data.quirks; - return (quirks & (FLEXCAN_QUIRK_SUPPPORT_RX_MAILBOX | - FLEXCAN_QUIRK_SUPPPORT_RX_MAILBOX_RTR)) == - (FLEXCAN_QUIRK_SUPPPORT_RX_MAILBOX | - FLEXCAN_QUIRK_SUPPPORT_RX_MAILBOX_RTR); + return (quirks & (FLEXCAN_QUIRK_SUPPORT_RX_MAILBOX | + FLEXCAN_QUIRK_SUPPORT_RX_MAILBOX_RTR)) == + (FLEXCAN_QUIRK_SUPPORT_RX_MAILBOX | + FLEXCAN_QUIRK_SUPPORT_RX_MAILBOX_RTR); } static inline bool @@ -140,7 +140,7 @@ flexcan_supports_rx_fifo(const struct flexcan_priv *priv) { const u32 quirks = priv->devtype_data.quirks; - return quirks & FLEXCAN_QUIRK_SUPPPORT_RX_FIFO; + return quirks & FLEXCAN_QUIRK_SUPPORT_RX_FIFO; } static inline bool @@ -149,7 +149,7 @@ flexcan_active_rx_rtr(const struct flexcan_priv *priv) const u32 quirks = priv->devtype_data.quirks; if (quirks & FLEXCAN_QUIRK_USE_RX_MAILBOX) { - if (quirks & FLEXCAN_QUIRK_SUPPPORT_RX_MAILBOX_RTR) + if (quirks & FLEXCAN_QUIRK_SUPPORT_RX_MAILBOX_RTR) return true; } else { /* RX-FIFO is always RTR capable */ diff --git a/drivers/net/can/ifi_canfd/ifi_canfd.c b/drivers/net/can/ifi_canfd/ifi_canfd.c index ad7a89b95da7..8d42b7e6661f 100644 --- a/drivers/net/can/ifi_canfd/ifi_canfd.c +++ b/drivers/net/can/ifi_canfd/ifi_canfd.c @@ -973,7 +973,7 @@ static int ifi_canfd_plat_probe(struct platform_device *pdev) priv->ndev = ndev; priv->base = addr; - netif_napi_add(ndev, &priv->napi, ifi_canfd_poll, 64); + netif_napi_add(ndev, &priv->napi, ifi_canfd_poll); priv->can.state = CAN_STATE_STOPPED; diff --git a/drivers/net/can/kvaser_pciefd.c b/drivers/net/can/kvaser_pciefd.c index ed54c0b3c7d4..4e9680c8eb34 100644 --- a/drivers/net/can/kvaser_pciefd.c +++ b/drivers/net/can/kvaser_pciefd.c @@ -329,12 +329,9 @@ MODULE_DEVICE_TABLE(pci, kvaser_pciefd_id_table); static int kvaser_pciefd_spi_wait_loop(struct kvaser_pciefd *pcie, int msk) { u32 res; - int ret; - - ret = readl_poll_timeout(pcie->reg_base + KVASER_PCIEFD_SPI_STATUS_REG, - res, res & msk, 0, 10); - return ret; + return readl_poll_timeout(pcie->reg_base + KVASER_PCIEFD_SPI_STATUS_REG, + res, res & msk, 0, 10); } static int kvaser_pciefd_spi_cmd(struct kvaser_pciefd *pcie, const u8 *tx, diff --git a/drivers/net/can/m_can/m_can.c b/drivers/net/can/m_can/m_can.c index 4709c012b1dc..dcb582563d5e 100644 --- a/drivers/net/can/m_can/m_can.c +++ b/drivers/net/can/m_can/m_can.c @@ -1467,8 +1467,7 @@ static int m_can_dev_setup(struct m_can_classdev *cdev) } if (!cdev->is_peripheral) - netif_napi_add(dev, &cdev->napi, - m_can_poll, NAPI_POLL_WEIGHT); + netif_napi_add(dev, &cdev->napi, m_can_poll); /* Shared properties of all M_CAN versions */ cdev->version = m_can_version; diff --git a/drivers/net/can/rcar/rcar_canfd.c b/drivers/net/can/rcar/rcar_canfd.c index 27085b796e75..567620d215f8 100644 --- a/drivers/net/can/rcar/rcar_canfd.c +++ b/drivers/net/can/rcar/rcar_canfd.c @@ -1880,10 +1880,9 @@ static int rcar_canfd_probe(struct platform_device *pdev) /* Global controller context */ gpriv = devm_kzalloc(&pdev->dev, sizeof(*gpriv), GFP_KERNEL); - if (!gpriv) { - err = -ENOMEM; - goto fail_dev; - } + if (!gpriv) + return -ENOMEM; + gpriv->pdev = pdev; gpriv->channels_mask = channels_mask; gpriv->fdmode = fdmode; @@ -1904,12 +1903,9 @@ static int rcar_canfd_probe(struct platform_device *pdev) /* Peripheral clock */ gpriv->clkp = devm_clk_get(&pdev->dev, "fck"); - if (IS_ERR(gpriv->clkp)) { - err = PTR_ERR(gpriv->clkp); - dev_err(&pdev->dev, "cannot get peripheral clock, error %d\n", - err); - goto fail_dev; - } + if (IS_ERR(gpriv->clkp)) + return dev_err_probe(&pdev->dev, PTR_ERR(gpriv->clkp), + "cannot get peripheral clock\n"); /* fCAN clock: Pick External clock. If not available fallback to * CANFD clock @@ -1917,12 +1913,10 @@ static int rcar_canfd_probe(struct platform_device *pdev) gpriv->can_clk = devm_clk_get(&pdev->dev, "can_clk"); if (IS_ERR(gpriv->can_clk) || (clk_get_rate(gpriv->can_clk) == 0)) { gpriv->can_clk = devm_clk_get(&pdev->dev, "canfd"); - if (IS_ERR(gpriv->can_clk)) { - err = PTR_ERR(gpriv->can_clk); - dev_err(&pdev->dev, - "cannot get canfd clock, error %d\n", err); - goto fail_dev; - } + if (IS_ERR(gpriv->can_clk)) + return dev_err_probe(&pdev->dev, PTR_ERR(gpriv->can_clk), + "cannot get canfd clock\n"); + gpriv->fcan = RCANFD_CANFDCLK; } else { diff --git a/drivers/net/can/sja1000/peak_pcmcia.c b/drivers/net/can/sja1000/peak_pcmcia.c index 131a084c3535..ebd5941c3f53 100644 --- a/drivers/net/can/sja1000/peak_pcmcia.c +++ b/drivers/net/can/sja1000/peak_pcmcia.c @@ -478,7 +478,7 @@ static void pcan_free_channels(struct pcan_pccard *card) if (!netdev) continue; - strlcpy(name, netdev->name, IFNAMSIZ); + strscpy(name, netdev->name, IFNAMSIZ); unregister_sja1000dev(netdev); diff --git a/drivers/net/can/sja1000/sja1000.c b/drivers/net/can/sja1000/sja1000.c index 98dfd5f295a7..1bb1129b0450 100644 --- a/drivers/net/can/sja1000/sja1000.c +++ b/drivers/net/can/sja1000/sja1000.c @@ -661,8 +661,6 @@ static const struct ethtool_ops sja1000_ethtool_ops = { int register_sja1000dev(struct net_device *dev) { - int ret; - if (!sja1000_probe_chip(dev)) return -ENODEV; @@ -673,9 +671,7 @@ int register_sja1000dev(struct net_device *dev) set_reset_mode(dev); chipset_init(dev); - ret = register_candev(dev); - - return ret; + return register_candev(dev); } EXPORT_SYMBOL_GPL(register_sja1000dev); diff --git a/drivers/net/can/sja1000/sja1000_platform.c b/drivers/net/can/sja1000/sja1000_platform.c index 81bc741905fd..6779d5357069 100644 --- a/drivers/net/can/sja1000/sja1000_platform.c +++ b/drivers/net/can/sja1000/sja1000_platform.c @@ -14,6 +14,7 @@ #include <linux/irq.h> #include <linux/can/dev.h> #include <linux/can/platform/sja1000.h> +#include <linux/clk.h> #include <linux/io.h> #include <linux/of.h> #include <linux/of_device.h> @@ -103,6 +104,11 @@ static void sp_technologic_init(struct sja1000_priv *priv, struct device_node *o spin_lock_init(&tp->io_lock); } +static void sp_rzn1_init(struct sja1000_priv *priv, struct device_node *of) +{ + priv->flags = SJA1000_QUIRK_NO_CDR_REG; +} + static void sp_populate(struct sja1000_priv *priv, struct sja1000_platform_data *pdata, unsigned long resource_mem_flags) @@ -153,11 +159,13 @@ static void sp_populate_of(struct sja1000_priv *priv, struct device_node *of) priv->write_reg = sp_write_reg8; } - err = of_property_read_u32(of, "nxp,external-clock-frequency", &prop); - if (!err) - priv->can.clock.freq = prop / 2; - else - priv->can.clock.freq = SP_CAN_CLOCK; /* default */ + if (!priv->can.clock.freq) { + err = of_property_read_u32(of, "nxp,external-clock-frequency", &prop); + if (!err) + priv->can.clock.freq = prop / 2; + else + priv->can.clock.freq = SP_CAN_CLOCK; /* default */ + } err = of_property_read_u32(of, "nxp,tx-output-mode", &prop); if (!err) @@ -192,8 +200,13 @@ static struct sja1000_of_data technologic_data = { .init = sp_technologic_init, }; +static struct sja1000_of_data renesas_data = { + .init = sp_rzn1_init, +}; + static const struct of_device_id sp_of_table[] = { { .compatible = "nxp,sja1000", .data = NULL, }, + { .compatible = "renesas,rzn1-sja1000", .data = &renesas_data, }, { .compatible = "technologic,sja1000", .data = &technologic_data, }, { /* sentinel */ }, }; @@ -210,6 +223,7 @@ static int sp_probe(struct platform_device *pdev) struct device_node *of = pdev->dev.of_node; const struct sja1000_of_data *of_data = NULL; size_t priv_sz = 0; + struct clk *clk; pdata = dev_get_platdata(&pdev->dev); if (!pdata && !of) { @@ -234,6 +248,11 @@ static int sp_probe(struct platform_device *pdev) irq = platform_get_irq(pdev, 0); if (irq < 0) return irq; + + clk = devm_clk_get_optional_enabled(&pdev->dev, NULL); + if (IS_ERR(clk)) + return dev_err_probe(&pdev->dev, PTR_ERR(clk), + "CAN clk operation failed"); } else { res_irq = platform_get_resource(pdev, IORESOURCE_IRQ, 0); if (!res_irq) @@ -262,6 +281,15 @@ static int sp_probe(struct platform_device *pdev) priv->reg_base = addr; if (of) { + if (clk) { + priv->can.clock.freq = clk_get_rate(clk) / 2; + if (!priv->can.clock.freq) { + err = -EINVAL; + dev_err(&pdev->dev, "Zero CAN clk rate"); + goto exit_free; + } + } + sp_populate_of(priv, of); if (of_data && of_data->init) diff --git a/drivers/net/can/usb/etas_es58x/es58x_core.h b/drivers/net/can/usb/etas_es58x/es58x_core.h index d769bdf740b7..640fe0a1df63 100644 --- a/drivers/net/can/usb/etas_es58x/es58x_core.h +++ b/drivers/net/can/usb/etas_es58x/es58x_core.h @@ -222,7 +222,7 @@ union es58x_urb_cmd { u8 cmd_type; u8 cmd_id; } __packed; - u8 raw_cmd[0]; + DECLARE_FLEX_ARRAY(u8, raw_cmd); }; /** diff --git a/drivers/net/can/usb/gs_usb.c b/drivers/net/can/usb/gs_usb.c index c1ff3c046d62..f0065d40eb24 100644 --- a/drivers/net/can/usb/gs_usb.c +++ b/drivers/net/can/usb/gs_usb.c @@ -10,20 +10,24 @@ */ #include <linux/bitfield.h> +#include <linux/clocksource.h> #include <linux/ethtool.h> #include <linux/init.h> #include <linux/module.h> #include <linux/netdevice.h> #include <linux/signal.h> +#include <linux/timecounter.h> +#include <linux/units.h> #include <linux/usb.h> +#include <linux/workqueue.h> #include <linux/can.h> #include <linux/can/dev.h> #include <linux/can/error.h> /* Device specific constants */ -#define USB_GSUSB_1_VENDOR_ID 0x1d50 -#define USB_GSUSB_1_PRODUCT_ID 0x606f +#define USB_GS_USB_1_VENDOR_ID 0x1d50 +#define USB_GS_USB_1_PRODUCT_ID 0x606f #define USB_CANDLELIGHT_VENDOR_ID 0x1209 #define USB_CANDLELIGHT_PRODUCT_ID 0x2323 @@ -34,8 +38,16 @@ #define USB_ABE_CANDEBUGGER_FD_VENDOR_ID 0x16d0 #define USB_ABE_CANDEBUGGER_FD_PRODUCT_ID 0x10b8 -#define GSUSB_ENDPOINT_IN 1 -#define GSUSB_ENDPOINT_OUT 2 +#define GS_USB_ENDPOINT_IN 1 +#define GS_USB_ENDPOINT_OUT 2 + +/* Timestamp 32 bit timer runs at 1 MHz (1 µs tick). Worker accounts + * for timer overflow (will be after ~71 minutes) + */ +#define GS_USB_TIMESTAMP_TIMER_HZ (1 * HZ_PER_MHZ) +#define GS_USB_TIMESTAMP_WORK_DELAY_SEC 1800 +static_assert(GS_USB_TIMESTAMP_WORK_DELAY_SEC < + CYCLECOUNTER_MASK(32) / GS_USB_TIMESTAMP_TIMER_HZ / 2); /* Device specific constants */ enum gs_usb_breq { @@ -52,6 +64,8 @@ enum gs_usb_breq { GS_USB_BREQ_SET_USER_ID, GS_USB_BREQ_DATA_BITTIMING, GS_USB_BREQ_BT_CONST_EXT, + GS_USB_BREQ_SET_TERMINATION, + GS_USB_BREQ_GET_TERMINATION, }; enum gs_can_mode { @@ -75,6 +89,14 @@ enum gs_can_identify_mode { GS_CAN_IDENTIFY_ON }; +enum gs_can_termination_state { + GS_CAN_TERMINATION_STATE_OFF = 0, + GS_CAN_TERMINATION_STATE_ON +}; + +#define GS_USB_TERMINATION_DISABLED CAN_TERMINATION_DISABLED +#define GS_USB_TERMINATION_ENABLED 120 + /* data types passed between host and device */ /* The firmware on the original USB2CAN by Geschwister Schneider @@ -111,6 +133,7 @@ struct gs_device_config { #define GS_CAN_MODE_FD BIT(8) /* GS_CAN_FEATURE_REQ_USB_QUIRK_LPC546XX BIT(9) */ /* GS_CAN_FEATURE_BT_CONST_EXT BIT(10) */ +/* GS_CAN_FEATURE_TERMINATION BIT(11) */ struct gs_device_mode { __le32 mode; @@ -135,6 +158,10 @@ struct gs_identify_mode { __le32 mode; } __packed; +struct gs_device_termination_state { + __le32 state; +} __packed; + #define GS_CAN_FEATURE_LISTEN_ONLY BIT(0) #define GS_CAN_FEATURE_LOOP_BACK BIT(1) #define GS_CAN_FEATURE_TRIPLE_SAMPLE BIT(2) @@ -146,7 +173,8 @@ struct gs_identify_mode { #define GS_CAN_FEATURE_FD BIT(8) #define GS_CAN_FEATURE_REQ_USB_QUIRK_LPC546XX BIT(9) #define GS_CAN_FEATURE_BT_CONST_EXT BIT(10) -#define GS_CAN_FEATURE_MASK GENMASK(10, 0) +#define GS_CAN_FEATURE_TERMINATION BIT(11) +#define GS_CAN_FEATURE_MASK GENMASK(11, 0) /* internal quirks - keep in GS_CAN_FEATURE space for now */ @@ -199,6 +227,11 @@ struct classic_can { u8 data[8]; } __packed; +struct classic_can_ts { + u8 data[8]; + __le32 timestamp_us; +} __packed; + struct classic_can_quirk { u8 data[8]; u8 quirk; @@ -208,6 +241,11 @@ struct canfd { u8 data[64]; } __packed; +struct canfd_ts { + u8 data[64]; + __le32 timestamp_us; +} __packed; + struct canfd_quirk { u8 data[64]; u8 quirk; @@ -224,8 +262,10 @@ struct gs_host_frame { union { DECLARE_FLEX_ARRAY(struct classic_can, classic_can); + DECLARE_FLEX_ARRAY(struct classic_can_ts, classic_can_ts); DECLARE_FLEX_ARRAY(struct classic_can_quirk, classic_can_quirk); DECLARE_FLEX_ARRAY(struct canfd, canfd); + DECLARE_FLEX_ARRAY(struct canfd_ts, canfd_ts); DECLARE_FLEX_ARRAY(struct canfd_quirk, canfd_quirk); }; } __packed; @@ -259,6 +299,12 @@ struct gs_can { struct can_bittiming_const bt_const, data_bt_const; unsigned int channel; /* channel number */ + /* time counter for hardware timestamps */ + struct cyclecounter cc; + struct timecounter tc; + spinlock_t tc_lock; /* spinlock to guard access tc->cycle_last */ + struct delayed_work timestamp; + u32 feature; unsigned int hf_size_tx; @@ -268,8 +314,6 @@ struct gs_can { struct usb_anchor tx_submitted; atomic_t active_tx_urbs; - void *rxbuf[GS_MAX_RX_URBS]; - dma_addr_t rxbuf_dma[GS_MAX_RX_URBS]; }; /* usb interface struct */ @@ -328,27 +372,109 @@ static struct gs_tx_context *gs_get_tx_context(struct gs_can *dev, return NULL; } -static int gs_cmd_reset(struct gs_can *gsdev) +static int gs_cmd_reset(struct gs_can *dev) +{ + struct gs_device_mode dm = { + .mode = GS_CAN_MODE_RESET, + }; + + return usb_control_msg_send(interface_to_usbdev(dev->iface), 0, + GS_USB_BREQ_MODE, + USB_DIR_OUT | USB_TYPE_VENDOR | USB_RECIP_INTERFACE, + dev->channel, 0, &dm, sizeof(dm), 1000, + GFP_KERNEL); +} + +static inline int gs_usb_get_timestamp(const struct gs_can *dev, + u32 *timestamp_p) { - struct gs_device_mode *dm; - struct usb_interface *intf = gsdev->iface; + __le32 timestamp; int rc; - dm = kzalloc(sizeof(*dm), GFP_KERNEL); - if (!dm) - return -ENOMEM; + rc = usb_control_msg_recv(interface_to_usbdev(dev->iface), 0, + GS_USB_BREQ_TIMESTAMP, + USB_DIR_IN | USB_TYPE_VENDOR | USB_RECIP_INTERFACE, + dev->channel, 0, + ×tamp, sizeof(timestamp), + USB_CTRL_GET_TIMEOUT, + GFP_KERNEL); + if (rc) + return rc; + + *timestamp_p = le32_to_cpu(timestamp); - dm->mode = GS_CAN_MODE_RESET; + return 0; +} - rc = usb_control_msg(interface_to_usbdev(intf), - usb_sndctrlpipe(interface_to_usbdev(intf), 0), - GS_USB_BREQ_MODE, - USB_DIR_OUT | USB_TYPE_VENDOR | USB_RECIP_INTERFACE, - gsdev->channel, 0, dm, sizeof(*dm), 1000); +static u64 gs_usb_timestamp_read(const struct cyclecounter *cc) __must_hold(&dev->tc_lock) +{ + struct gs_can *dev = container_of(cc, struct gs_can, cc); + u32 timestamp = 0; + int err; + + lockdep_assert_held(&dev->tc_lock); + + /* drop lock for synchronous USB transfer */ + spin_unlock_bh(&dev->tc_lock); + err = gs_usb_get_timestamp(dev, ×tamp); + spin_lock_bh(&dev->tc_lock); + if (err) + netdev_err(dev->netdev, + "Error %d while reading timestamp. HW timestamps may be inaccurate.", + err); + + return timestamp; +} - kfree(dm); +static void gs_usb_timestamp_work(struct work_struct *work) +{ + struct delayed_work *delayed_work = to_delayed_work(work); + struct gs_can *dev; - return rc; + dev = container_of(delayed_work, struct gs_can, timestamp); + spin_lock_bh(&dev->tc_lock); + timecounter_read(&dev->tc); + spin_unlock_bh(&dev->tc_lock); + + schedule_delayed_work(&dev->timestamp, + GS_USB_TIMESTAMP_WORK_DELAY_SEC * HZ); +} + +static void gs_usb_skb_set_timestamp(struct gs_can *dev, + struct sk_buff *skb, u32 timestamp) +{ + struct skb_shared_hwtstamps *hwtstamps = skb_hwtstamps(skb); + u64 ns; + + spin_lock_bh(&dev->tc_lock); + ns = timecounter_cyc2time(&dev->tc, timestamp); + spin_unlock_bh(&dev->tc_lock); + + hwtstamps->hwtstamp = ns_to_ktime(ns); +} + +static void gs_usb_timestamp_init(struct gs_can *dev) +{ + struct cyclecounter *cc = &dev->cc; + + cc->read = gs_usb_timestamp_read; + cc->mask = CYCLECOUNTER_MASK(32); + cc->shift = 32 - bits_per(NSEC_PER_SEC / GS_USB_TIMESTAMP_TIMER_HZ); + cc->mult = clocksource_hz2mult(GS_USB_TIMESTAMP_TIMER_HZ, cc->shift); + + spin_lock_init(&dev->tc_lock); + spin_lock_bh(&dev->tc_lock); + timecounter_init(&dev->tc, &dev->cc, ktime_get_real_ns()); + spin_unlock_bh(&dev->tc_lock); + + INIT_DELAYED_WORK(&dev->timestamp, gs_usb_timestamp_work); + schedule_delayed_work(&dev->timestamp, + GS_USB_TIMESTAMP_WORK_DELAY_SEC * HZ); +} + +static void gs_usb_timestamp_stop(struct gs_can *dev) +{ + cancel_delayed_work_sync(&dev->timestamp); } static void gs_update_state(struct gs_can *dev, struct can_frame *cf) @@ -376,6 +502,24 @@ static void gs_update_state(struct gs_can *dev, struct can_frame *cf) } } +static void gs_usb_set_timestamp(struct gs_can *dev, struct sk_buff *skb, + const struct gs_host_frame *hf) +{ + u32 timestamp; + + if (!(dev->feature & GS_CAN_FEATURE_HW_TIMESTAMP)) + return; + + if (hf->flags & GS_CAN_FLAG_FD) + timestamp = le32_to_cpu(hf->canfd_ts->timestamp_us); + else + timestamp = le32_to_cpu(hf->classic_can_ts->timestamp_us); + + gs_usb_skb_set_timestamp(dev, skb, timestamp); + + return; +} + static void gs_usb_receive_bulk_callback(struct urb *urb) { struct gs_usb *usbcan = urb->context; @@ -443,6 +587,8 @@ static void gs_usb_receive_bulk_callback(struct urb *urb) gs_update_state(dev, cf); } + gs_usb_set_timestamp(dev, skb, hf); + netdev->stats.rx_packets++; netdev->stats.rx_bytes += hf->can_dlc; @@ -465,6 +611,9 @@ static void gs_usb_receive_bulk_callback(struct urb *urb) goto resubmit_urb; } + skb = dev->can.echo_skb[hf->echo_id]; + gs_usb_set_timestamp(dev, skb, hf); + netdev->stats.tx_packets++; netdev->stats.tx_bytes += can_get_echo_skb(netdev, hf->echo_id, NULL); @@ -491,7 +640,7 @@ static void gs_usb_receive_bulk_callback(struct urb *urb) resubmit_urb: usb_fill_bulk_urb(urb, usbcan->udev, - usb_rcvbulkpipe(usbcan->udev, GSUSB_ENDPOINT_IN), + usb_rcvbulkpipe(usbcan->udev, GS_USB_ENDPOINT_IN), hf, dev->parent->hf_size_rx, gs_usb_receive_bulk_callback, usbcan); @@ -511,72 +660,44 @@ static int gs_usb_set_bittiming(struct net_device *netdev) { struct gs_can *dev = netdev_priv(netdev); struct can_bittiming *bt = &dev->can.bittiming; - struct usb_interface *intf = dev->iface; - int rc; - struct gs_device_bittiming *dbt; - - dbt = kmalloc(sizeof(*dbt), GFP_KERNEL); - if (!dbt) - return -ENOMEM; - - dbt->prop_seg = cpu_to_le32(bt->prop_seg); - dbt->phase_seg1 = cpu_to_le32(bt->phase_seg1); - dbt->phase_seg2 = cpu_to_le32(bt->phase_seg2); - dbt->sjw = cpu_to_le32(bt->sjw); - dbt->brp = cpu_to_le32(bt->brp); + struct gs_device_bittiming dbt = { + .prop_seg = cpu_to_le32(bt->prop_seg), + .phase_seg1 = cpu_to_le32(bt->phase_seg1), + .phase_seg2 = cpu_to_le32(bt->phase_seg2), + .sjw = cpu_to_le32(bt->sjw), + .brp = cpu_to_le32(bt->brp), + }; /* request bit timings */ - rc = usb_control_msg(interface_to_usbdev(intf), - usb_sndctrlpipe(interface_to_usbdev(intf), 0), - GS_USB_BREQ_BITTIMING, - USB_DIR_OUT | USB_TYPE_VENDOR | USB_RECIP_INTERFACE, - dev->channel, 0, dbt, sizeof(*dbt), 1000); - - kfree(dbt); - - if (rc < 0) - dev_err(netdev->dev.parent, "Couldn't set bittimings (err=%d)", - rc); - - return (rc > 0) ? 0 : rc; + return usb_control_msg_send(interface_to_usbdev(dev->iface), 0, + GS_USB_BREQ_BITTIMING, + USB_DIR_OUT | USB_TYPE_VENDOR | USB_RECIP_INTERFACE, + dev->channel, 0, &dbt, sizeof(dbt), 1000, + GFP_KERNEL); } static int gs_usb_set_data_bittiming(struct net_device *netdev) { struct gs_can *dev = netdev_priv(netdev); struct can_bittiming *bt = &dev->can.data_bittiming; - struct usb_interface *intf = dev->iface; - struct gs_device_bittiming *dbt; + struct gs_device_bittiming dbt = { + .prop_seg = cpu_to_le32(bt->prop_seg), + .phase_seg1 = cpu_to_le32(bt->phase_seg1), + .phase_seg2 = cpu_to_le32(bt->phase_seg2), + .sjw = cpu_to_le32(bt->sjw), + .brp = cpu_to_le32(bt->brp), + }; u8 request = GS_USB_BREQ_DATA_BITTIMING; - int rc; - - dbt = kmalloc(sizeof(*dbt), GFP_KERNEL); - if (!dbt) - return -ENOMEM; - - dbt->prop_seg = cpu_to_le32(bt->prop_seg); - dbt->phase_seg1 = cpu_to_le32(bt->phase_seg1); - dbt->phase_seg2 = cpu_to_le32(bt->phase_seg2); - dbt->sjw = cpu_to_le32(bt->sjw); - dbt->brp = cpu_to_le32(bt->brp); if (dev->feature & GS_CAN_FEATURE_QUIRK_BREQ_CANTACT_PRO) request = GS_USB_BREQ_QUIRK_CANTACT_PRO_DATA_BITTIMING; - /* request bit timings */ - rc = usb_control_msg(interface_to_usbdev(intf), - usb_sndctrlpipe(interface_to_usbdev(intf), 0), - request, - USB_DIR_OUT | USB_TYPE_VENDOR | USB_RECIP_INTERFACE, - dev->channel, 0, dbt, sizeof(*dbt), 1000); - - kfree(dbt); - - if (rc < 0) - dev_err(netdev->dev.parent, - "Couldn't set data bittimings (err=%d)", rc); - - return (rc > 0) ? 0 : rc; + /* request data bit timings */ + return usb_control_msg_send(interface_to_usbdev(dev->iface), 0, + request, + USB_DIR_OUT | USB_TYPE_VENDOR | USB_RECIP_INTERFACE, + dev->channel, 0, &dbt, sizeof(dbt), 1000, + GFP_KERNEL); } static void gs_usb_xmit_callback(struct urb *urb) @@ -587,9 +708,6 @@ static void gs_usb_xmit_callback(struct urb *urb) if (urb->status) netdev_info(netdev, "usb xmit fail %u\n", txc->echo_id); - - usb_free_coherent(urb->dev, urb->transfer_buffer_length, - urb->transfer_buffer, urb->transfer_dma); } static netdev_tx_t gs_can_start_xmit(struct sk_buff *skb, @@ -618,8 +736,7 @@ static netdev_tx_t gs_can_start_xmit(struct sk_buff *skb, if (!urb) goto nomem_urb; - hf = usb_alloc_coherent(dev->udev, dev->hf_size_tx, GFP_ATOMIC, - &urb->transfer_dma); + hf = kmalloc(dev->hf_size_tx, GFP_ATOMIC); if (!hf) { netdev_err(netdev, "No memory left for USB buffer\n"); goto nomem_hf; @@ -659,11 +776,11 @@ static netdev_tx_t gs_can_start_xmit(struct sk_buff *skb, } usb_fill_bulk_urb(urb, dev->udev, - usb_sndbulkpipe(dev->udev, GSUSB_ENDPOINT_OUT), + usb_sndbulkpipe(dev->udev, GS_USB_ENDPOINT_OUT), hf, dev->hf_size_tx, gs_usb_xmit_callback, txc); - urb->transfer_flags |= URB_NO_TRANSFER_DMA_MAP; + urb->transfer_flags |= URB_FREE_BUFFER; usb_anchor_urb(urb, &dev->tx_submitted); can_put_echo_skb(skb, netdev, idx, 0); @@ -678,8 +795,6 @@ static netdev_tx_t gs_can_start_xmit(struct sk_buff *skb, gs_free_tx_context(txc); usb_unanchor_urb(urb); - usb_free_coherent(dev->udev, urb->transfer_buffer_length, - urb->transfer_buffer, urb->transfer_dma); if (rc == -ENODEV) { netif_device_detach(netdev); @@ -699,8 +814,7 @@ static netdev_tx_t gs_can_start_xmit(struct sk_buff *skb, return NETDEV_TX_OK; badidx: - usb_free_coherent(dev->udev, urb->transfer_buffer_length, - urb->transfer_buffer, urb->transfer_dma); + kfree(hf); nomem_hf: usb_free_urb(urb); @@ -715,11 +829,13 @@ static int gs_can_open(struct net_device *netdev) { struct gs_can *dev = netdev_priv(netdev); struct gs_usb *parent = dev->parent; - int rc, i; - struct gs_device_mode *dm; + struct gs_device_mode dm = { + .mode = cpu_to_le32(GS_CAN_MODE_START), + }; struct gs_host_frame *hf; u32 ctrlmode; u32 flags = 0; + int rc, i; rc = open_candev(netdev); if (rc) @@ -744,7 +860,6 @@ static int gs_can_open(struct net_device *netdev) for (i = 0; i < GS_MAX_RX_URBS; i++) { struct urb *urb; u8 *buf; - dma_addr_t buf_dma; /* alloc rx urb */ urb = usb_alloc_urb(0, GFP_KERNEL); @@ -752,10 +867,8 @@ static int gs_can_open(struct net_device *netdev) return -ENOMEM; /* alloc rx buffer */ - buf = usb_alloc_coherent(dev->udev, - dev->parent->hf_size_rx, - GFP_KERNEL, - &buf_dma); + buf = kmalloc(dev->parent->hf_size_rx, + GFP_KERNEL); if (!buf) { netdev_err(netdev, "No memory left for USB buffer\n"); @@ -763,17 +876,15 @@ static int gs_can_open(struct net_device *netdev) return -ENOMEM; } - urb->transfer_dma = buf_dma; - /* fill, anchor, and submit rx urb */ usb_fill_bulk_urb(urb, dev->udev, usb_rcvbulkpipe(dev->udev, - GSUSB_ENDPOINT_IN), + GS_USB_ENDPOINT_IN), buf, dev->parent->hf_size_rx, gs_usb_receive_bulk_callback, parent); - urb->transfer_flags |= URB_NO_TRANSFER_DMA_MAP; + urb->transfer_flags |= URB_FREE_BUFFER; usb_anchor_urb(urb, &parent->rx_submitted); @@ -786,17 +897,10 @@ static int gs_can_open(struct net_device *netdev) "usb_submit failed (err=%d)\n", rc); usb_unanchor_urb(urb); - usb_free_coherent(dev->udev, - sizeof(struct gs_host_frame), - buf, - buf_dma); usb_free_urb(urb); break; } - dev->rxbuf[i] = buf; - dev->rxbuf_dma[i] = buf_dma; - /* Drop reference, * USB core will take care of freeing it */ @@ -804,10 +908,6 @@ static int gs_can_open(struct net_device *netdev) } } - dm = kmalloc(sizeof(*dm), GFP_KERNEL); - if (!dm) - return -ENOMEM; - /* flags */ if (ctrlmode & CAN_CTRLMODE_LOOPBACK) flags |= GS_CAN_MODE_LOOP_BACK; @@ -823,25 +923,30 @@ static int gs_can_open(struct net_device *netdev) if (ctrlmode & CAN_CTRLMODE_3_SAMPLES) flags |= GS_CAN_MODE_TRIPLE_SAMPLE; + /* if hardware supports timestamps, enable it */ + if (dev->feature & GS_CAN_FEATURE_HW_TIMESTAMP) + flags |= GS_CAN_MODE_HW_TIMESTAMP; + + /* start polling timestamp */ + if (dev->feature & GS_CAN_FEATURE_HW_TIMESTAMP) + gs_usb_timestamp_init(dev); + /* finally start device */ dev->can.state = CAN_STATE_ERROR_ACTIVE; - dm->mode = cpu_to_le32(GS_CAN_MODE_START); - dm->flags = cpu_to_le32(flags); - rc = usb_control_msg(interface_to_usbdev(dev->iface), - usb_sndctrlpipe(interface_to_usbdev(dev->iface), 0), - GS_USB_BREQ_MODE, - USB_DIR_OUT | USB_TYPE_VENDOR | USB_RECIP_INTERFACE, - dev->channel, 0, dm, sizeof(*dm), 1000); - - if (rc < 0) { + dm.flags = cpu_to_le32(flags); + rc = usb_control_msg_send(interface_to_usbdev(dev->iface), 0, + GS_USB_BREQ_MODE, + USB_DIR_OUT | USB_TYPE_VENDOR | USB_RECIP_INTERFACE, + dev->channel, 0, &dm, sizeof(dm), 1000, + GFP_KERNEL); + if (rc) { netdev_err(netdev, "Couldn't start device (err=%d)\n", rc); - kfree(dm); + if (dev->feature & GS_CAN_FEATURE_HW_TIMESTAMP) + gs_usb_timestamp_stop(dev); dev->can.state = CAN_STATE_STOPPED; return rc; } - kfree(dm); - parent->active_channels++; if (!(dev->can.ctrlmode & CAN_CTRLMODE_LISTENONLY)) netif_start_queue(netdev); @@ -854,19 +959,17 @@ static int gs_can_close(struct net_device *netdev) int rc; struct gs_can *dev = netdev_priv(netdev); struct gs_usb *parent = dev->parent; - unsigned int i; netif_stop_queue(netdev); + /* stop polling timestamp */ + if (dev->feature & GS_CAN_FEATURE_HW_TIMESTAMP) + gs_usb_timestamp_stop(dev); + /* Stop polling */ parent->active_channels--; if (!parent->active_channels) { usb_kill_anchored_urbs(&parent->rx_submitted); - for (i = 0; i < GS_MAX_RX_URBS; i++) - usb_free_coherent(dev->udev, - sizeof(struct gs_host_frame), - dev->rxbuf[i], - dev->rxbuf_dma[i]); } /* Stop sending URBs */ @@ -890,38 +993,39 @@ static int gs_can_close(struct net_device *netdev) return 0; } +static int gs_can_eth_ioctl(struct net_device *netdev, struct ifreq *ifr, int cmd) +{ + const struct gs_can *dev = netdev_priv(netdev); + + if (dev->feature & GS_CAN_FEATURE_HW_TIMESTAMP) + return can_eth_ioctl_hwts(netdev, ifr, cmd); + + return -EOPNOTSUPP; +} + static const struct net_device_ops gs_usb_netdev_ops = { .ndo_open = gs_can_open, .ndo_stop = gs_can_close, .ndo_start_xmit = gs_can_start_xmit, .ndo_change_mtu = can_change_mtu, + .ndo_eth_ioctl = gs_can_eth_ioctl, }; static int gs_usb_set_identify(struct net_device *netdev, bool do_identify) { struct gs_can *dev = netdev_priv(netdev); - struct gs_identify_mode *imode; - int rc; - - imode = kmalloc(sizeof(*imode), GFP_KERNEL); - - if (!imode) - return -ENOMEM; + struct gs_identify_mode imode; if (do_identify) - imode->mode = cpu_to_le32(GS_CAN_IDENTIFY_ON); + imode.mode = cpu_to_le32(GS_CAN_IDENTIFY_ON); else - imode->mode = cpu_to_le32(GS_CAN_IDENTIFY_OFF); - - rc = usb_control_msg(interface_to_usbdev(dev->iface), - usb_sndctrlpipe(interface_to_usbdev(dev->iface), 0), - GS_USB_BREQ_IDENTIFY, - USB_DIR_OUT | USB_TYPE_VENDOR | USB_RECIP_INTERFACE, - dev->channel, 0, imode, sizeof(*imode), 100); + imode.mode = cpu_to_le32(GS_CAN_IDENTIFY_OFF); - kfree(imode); - - return (rc > 0) ? 0 : rc; + return usb_control_msg_send(interface_to_usbdev(dev->iface), 0, + GS_USB_BREQ_IDENTIFY, + USB_DIR_OUT | USB_TYPE_VENDOR | USB_RECIP_INTERFACE, + dev->channel, 0, &imode, sizeof(imode), 100, + GFP_KERNEL); } /* blink LED's for finding the this interface */ @@ -948,9 +1052,67 @@ static int gs_usb_set_phys_id(struct net_device *netdev, return rc; } +static int gs_usb_get_ts_info(struct net_device *netdev, + struct ethtool_ts_info *info) +{ + struct gs_can *dev = netdev_priv(netdev); + + /* report if device supports HW timestamps */ + if (dev->feature & GS_CAN_FEATURE_HW_TIMESTAMP) + return can_ethtool_op_get_ts_info_hwts(netdev, info); + + return ethtool_op_get_ts_info(netdev, info); +} + static const struct ethtool_ops gs_usb_ethtool_ops = { .set_phys_id = gs_usb_set_phys_id, - .get_ts_info = ethtool_op_get_ts_info, + .get_ts_info = gs_usb_get_ts_info, +}; + +static int gs_usb_get_termination(struct net_device *netdev, u16 *term) +{ + struct gs_can *dev = netdev_priv(netdev); + struct gs_device_termination_state term_state; + int rc; + + rc = usb_control_msg_recv(interface_to_usbdev(dev->iface), 0, + GS_USB_BREQ_GET_TERMINATION, + USB_DIR_IN | USB_TYPE_VENDOR | USB_RECIP_INTERFACE, + dev->channel, 0, + &term_state, sizeof(term_state), 1000, + GFP_KERNEL); + if (rc) + return rc; + + if (term_state.state == cpu_to_le32(GS_CAN_TERMINATION_STATE_ON)) + *term = GS_USB_TERMINATION_ENABLED; + else + *term = GS_USB_TERMINATION_DISABLED; + + return 0; +} + +static int gs_usb_set_termination(struct net_device *netdev, u16 term) +{ + struct gs_can *dev = netdev_priv(netdev); + struct gs_device_termination_state term_state; + + if (term == GS_USB_TERMINATION_ENABLED) + term_state.state = cpu_to_le32(GS_CAN_TERMINATION_STATE_ON); + else + term_state.state = cpu_to_le32(GS_CAN_TERMINATION_STATE_OFF); + + return usb_control_msg_send(interface_to_usbdev(dev->iface), 0, + GS_USB_BREQ_SET_TERMINATION, + USB_DIR_OUT | USB_TYPE_VENDOR | USB_RECIP_INTERFACE, + dev->channel, 0, + &term_state, sizeof(term_state), 1000, + GFP_KERNEL); +} + +static const u16 gs_usb_termination_const[] = { + GS_USB_TERMINATION_DISABLED, + GS_USB_TERMINATION_ENABLED }; static struct gs_can *gs_make_candev(unsigned int channel, @@ -960,26 +1122,21 @@ static struct gs_can *gs_make_candev(unsigned int channel, struct gs_can *dev; struct net_device *netdev; int rc; - struct gs_device_bt_const *bt_const; - struct gs_device_bt_const_extended *bt_const_extended; + struct gs_device_bt_const_extended bt_const_extended; + struct gs_device_bt_const bt_const; u32 feature; - bt_const = kmalloc(sizeof(*bt_const), GFP_KERNEL); - if (!bt_const) - return ERR_PTR(-ENOMEM); - /* fetch bit timing constants */ - rc = usb_control_msg(interface_to_usbdev(intf), - usb_rcvctrlpipe(interface_to_usbdev(intf), 0), - GS_USB_BREQ_BT_CONST, - USB_DIR_IN | USB_TYPE_VENDOR | USB_RECIP_INTERFACE, - channel, 0, bt_const, sizeof(*bt_const), 1000); + rc = usb_control_msg_recv(interface_to_usbdev(intf), 0, + GS_USB_BREQ_BT_CONST, + USB_DIR_IN | USB_TYPE_VENDOR | USB_RECIP_INTERFACE, + channel, 0, &bt_const, sizeof(bt_const), 1000, + GFP_KERNEL); - if (rc < 0) { + if (rc) { dev_err(&intf->dev, - "Couldn't get bit timing const for channel (err=%d)\n", - rc); - kfree(bt_const); + "Couldn't get bit timing const for channel %d (%pe)\n", + channel, ERR_PTR(rc)); return ERR_PTR(rc); } @@ -987,7 +1144,6 @@ static struct gs_can *gs_make_candev(unsigned int channel, netdev = alloc_candev(sizeof(struct gs_can), GS_MAX_TX_URBS); if (!netdev) { dev_err(&intf->dev, "Couldn't allocate candev\n"); - kfree(bt_const); return ERR_PTR(-ENOMEM); } @@ -1000,14 +1156,14 @@ static struct gs_can *gs_make_candev(unsigned int channel, /* dev setup */ strcpy(dev->bt_const.name, KBUILD_MODNAME); - dev->bt_const.tseg1_min = le32_to_cpu(bt_const->tseg1_min); - dev->bt_const.tseg1_max = le32_to_cpu(bt_const->tseg1_max); - dev->bt_const.tseg2_min = le32_to_cpu(bt_const->tseg2_min); - dev->bt_const.tseg2_max = le32_to_cpu(bt_const->tseg2_max); - dev->bt_const.sjw_max = le32_to_cpu(bt_const->sjw_max); - dev->bt_const.brp_min = le32_to_cpu(bt_const->brp_min); - dev->bt_const.brp_max = le32_to_cpu(bt_const->brp_max); - dev->bt_const.brp_inc = le32_to_cpu(bt_const->brp_inc); + dev->bt_const.tseg1_min = le32_to_cpu(bt_const.tseg1_min); + dev->bt_const.tseg1_max = le32_to_cpu(bt_const.tseg1_max); + dev->bt_const.tseg2_min = le32_to_cpu(bt_const.tseg2_min); + dev->bt_const.tseg2_max = le32_to_cpu(bt_const.tseg2_max); + dev->bt_const.sjw_max = le32_to_cpu(bt_const.sjw_max); + dev->bt_const.brp_min = le32_to_cpu(bt_const.brp_min); + dev->bt_const.brp_max = le32_to_cpu(bt_const.brp_max); + dev->bt_const.brp_inc = le32_to_cpu(bt_const.brp_inc); dev->udev = interface_to_usbdev(intf); dev->iface = intf; @@ -1024,13 +1180,13 @@ static struct gs_can *gs_make_candev(unsigned int channel, /* can setup */ dev->can.state = CAN_STATE_STOPPED; - dev->can.clock.freq = le32_to_cpu(bt_const->fclk_can); + dev->can.clock.freq = le32_to_cpu(bt_const.fclk_can); dev->can.bittiming_const = &dev->bt_const; dev->can.do_set_bittiming = gs_usb_set_bittiming; dev->can.ctrlmode_supported = CAN_CTRLMODE_CC_LEN8_DLC; - feature = le32_to_cpu(bt_const->feature); + feature = le32_to_cpu(bt_const.feature); dev->feature = FIELD_GET(GS_CAN_FEATURE_MASK, feature); if (feature & GS_CAN_FEATURE_LISTEN_ONLY) dev->can.ctrlmode_supported |= CAN_CTRLMODE_LISTENONLY; @@ -1053,6 +1209,21 @@ static struct gs_can *gs_make_candev(unsigned int channel, dev->can.do_set_data_bittiming = gs_usb_set_data_bittiming; } + if (feature & GS_CAN_FEATURE_TERMINATION) { + rc = gs_usb_get_termination(netdev, &dev->can.termination); + if (rc) { + dev->feature &= ~GS_CAN_FEATURE_TERMINATION; + + dev_info(&intf->dev, + "Disabling termination support for channel %d (%pe)\n", + channel, ERR_PTR(rc)); + } else { + dev->can.termination_const = gs_usb_termination_const; + dev->can.termination_const_cnt = ARRAY_SIZE(gs_usb_termination_const); + dev->can.do_set_termination = gs_usb_set_termination; + } + } + /* The CANtact Pro from LinkLayer Labs is based on the * LPC54616 µC, which is affected by the NXP LPC USB transfer * erratum. However, the current firmware (version 2) doesn't @@ -1067,8 +1238,8 @@ static struct gs_can *gs_make_candev(unsigned int channel, * GS_CAN_FEATURE_QUIRK_BREQ_CANTACT_PRO to workaround this * issue. */ - if (dev->udev->descriptor.idVendor == cpu_to_le16(USB_GSUSB_1_VENDOR_ID) && - dev->udev->descriptor.idProduct == cpu_to_le16(USB_GSUSB_1_PRODUCT_ID) && + if (dev->udev->descriptor.idVendor == cpu_to_le16(USB_GS_USB_1_VENDOR_ID) && + dev->udev->descriptor.idProduct == cpu_to_le16(USB_GS_USB_1_PRODUCT_ID) && dev->udev->manufacturer && dev->udev->product && !strcmp(dev->udev->manufacturer, "LinkLayer Labs") && !strcmp(dev->udev->product, "CANtact Pro") && @@ -1081,57 +1252,52 @@ static struct gs_can *gs_make_candev(unsigned int channel, feature & GS_CAN_FEATURE_IDENTIFY)) dev->feature &= ~GS_CAN_FEATURE_IDENTIFY; - kfree(bt_const); - /* fetch extended bit timing constants if device has feature * GS_CAN_FEATURE_FD and GS_CAN_FEATURE_BT_CONST_EXT */ if (feature & GS_CAN_FEATURE_FD && feature & GS_CAN_FEATURE_BT_CONST_EXT) { - bt_const_extended = kmalloc(sizeof(*bt_const_extended), GFP_KERNEL); - if (!bt_const_extended) - return ERR_PTR(-ENOMEM); - - rc = usb_control_msg(interface_to_usbdev(intf), - usb_rcvctrlpipe(interface_to_usbdev(intf), 0), - GS_USB_BREQ_BT_CONST_EXT, - USB_DIR_IN | USB_TYPE_VENDOR | USB_RECIP_INTERFACE, - channel, 0, bt_const_extended, - sizeof(*bt_const_extended), - 1000); - if (rc < 0) { + rc = usb_control_msg_recv(interface_to_usbdev(intf), 0, + GS_USB_BREQ_BT_CONST_EXT, + USB_DIR_IN | USB_TYPE_VENDOR | USB_RECIP_INTERFACE, + channel, 0, &bt_const_extended, + sizeof(bt_const_extended), + 1000, GFP_KERNEL); + if (rc) { dev_err(&intf->dev, - "Couldn't get extended bit timing const for channel (err=%d)\n", - rc); - kfree(bt_const_extended); - return ERR_PTR(rc); + "Couldn't get extended bit timing const for channel %d (%pe)\n", + channel, ERR_PTR(rc)); + goto out_free_candev; } strcpy(dev->data_bt_const.name, KBUILD_MODNAME); - dev->data_bt_const.tseg1_min = le32_to_cpu(bt_const_extended->dtseg1_min); - dev->data_bt_const.tseg1_max = le32_to_cpu(bt_const_extended->dtseg1_max); - dev->data_bt_const.tseg2_min = le32_to_cpu(bt_const_extended->dtseg2_min); - dev->data_bt_const.tseg2_max = le32_to_cpu(bt_const_extended->dtseg2_max); - dev->data_bt_const.sjw_max = le32_to_cpu(bt_const_extended->dsjw_max); - dev->data_bt_const.brp_min = le32_to_cpu(bt_const_extended->dbrp_min); - dev->data_bt_const.brp_max = le32_to_cpu(bt_const_extended->dbrp_max); - dev->data_bt_const.brp_inc = le32_to_cpu(bt_const_extended->dbrp_inc); + dev->data_bt_const.tseg1_min = le32_to_cpu(bt_const_extended.dtseg1_min); + dev->data_bt_const.tseg1_max = le32_to_cpu(bt_const_extended.dtseg1_max); + dev->data_bt_const.tseg2_min = le32_to_cpu(bt_const_extended.dtseg2_min); + dev->data_bt_const.tseg2_max = le32_to_cpu(bt_const_extended.dtseg2_max); + dev->data_bt_const.sjw_max = le32_to_cpu(bt_const_extended.dsjw_max); + dev->data_bt_const.brp_min = le32_to_cpu(bt_const_extended.dbrp_min); + dev->data_bt_const.brp_max = le32_to_cpu(bt_const_extended.dbrp_max); + dev->data_bt_const.brp_inc = le32_to_cpu(bt_const_extended.dbrp_inc); dev->can.data_bittiming_const = &dev->data_bt_const; - - kfree(bt_const_extended); } SET_NETDEV_DEV(netdev, &intf->dev); rc = register_candev(dev->netdev); if (rc) { - free_candev(dev->netdev); - dev_err(&intf->dev, "Couldn't register candev (err=%d)\n", rc); - return ERR_PTR(rc); + dev_err(&intf->dev, + "Couldn't register candev for channel %d (%pe)\n", + channel, ERR_PTR(rc)); + goto out_free_candev; } return dev; + + out_free_candev: + free_candev(dev->netdev); + return ERR_PTR(rc); } static void gs_destroy_candev(struct gs_can *dev) @@ -1147,76 +1313,61 @@ static int gs_usb_probe(struct usb_interface *intf, struct usb_device *udev = interface_to_usbdev(intf); struct gs_host_frame *hf; struct gs_usb *dev; - int rc = -ENOMEM; + struct gs_host_config hconf = { + .byte_order = cpu_to_le32(0x0000beef), + }; + struct gs_device_config dconf; unsigned int icount, i; - struct gs_host_config *hconf; - struct gs_device_config *dconf; - - hconf = kmalloc(sizeof(*hconf), GFP_KERNEL); - if (!hconf) - return -ENOMEM; - - hconf->byte_order = cpu_to_le32(0x0000beef); + int rc; /* send host config */ - rc = usb_control_msg(udev, usb_sndctrlpipe(udev, 0), - GS_USB_BREQ_HOST_FORMAT, - USB_DIR_OUT | USB_TYPE_VENDOR | USB_RECIP_INTERFACE, - 1, intf->cur_altsetting->desc.bInterfaceNumber, - hconf, sizeof(*hconf), 1000); - - kfree(hconf); - - if (rc < 0) { + rc = usb_control_msg_send(udev, 0, + GS_USB_BREQ_HOST_FORMAT, + USB_DIR_OUT | USB_TYPE_VENDOR | USB_RECIP_INTERFACE, + 1, intf->cur_altsetting->desc.bInterfaceNumber, + &hconf, sizeof(hconf), 1000, + GFP_KERNEL); + if (rc) { dev_err(&intf->dev, "Couldn't send data format (err=%d)\n", rc); return rc; } - dconf = kmalloc(sizeof(*dconf), GFP_KERNEL); - if (!dconf) - return -ENOMEM; - /* read device config */ - rc = usb_control_msg(udev, usb_rcvctrlpipe(udev, 0), - GS_USB_BREQ_DEVICE_CONFIG, - USB_DIR_IN | USB_TYPE_VENDOR | USB_RECIP_INTERFACE, - 1, intf->cur_altsetting->desc.bInterfaceNumber, - dconf, sizeof(*dconf), 1000); - if (rc < 0) { + rc = usb_control_msg_recv(udev, 0, + GS_USB_BREQ_DEVICE_CONFIG, + USB_DIR_IN | USB_TYPE_VENDOR | USB_RECIP_INTERFACE, + 1, intf->cur_altsetting->desc.bInterfaceNumber, + &dconf, sizeof(dconf), 1000, + GFP_KERNEL); + if (rc) { dev_err(&intf->dev, "Couldn't get device config: (err=%d)\n", rc); - kfree(dconf); return rc; } - icount = dconf->icount + 1; + icount = dconf.icount + 1; dev_info(&intf->dev, "Configuring for %u interfaces\n", icount); if (icount > GS_MAX_INTF) { dev_err(&intf->dev, "Driver cannot handle more that %u CAN interfaces\n", GS_MAX_INTF); - kfree(dconf); return -EINVAL; } dev = kzalloc(sizeof(*dev), GFP_KERNEL); - if (!dev) { - kfree(dconf); + if (!dev) return -ENOMEM; - } init_usb_anchor(&dev->rx_submitted); - /* default to classic CAN, switch to CAN-FD if at least one of - * our channels support CAN-FD. - */ - dev->hf_size_rx = struct_size(hf, classic_can, 1); usb_set_intfdata(intf, dev); dev->udev = udev; for (i = 0; i < icount; i++) { - dev->canch[i] = gs_make_candev(i, intf, dconf); + unsigned int hf_size_rx = 0; + + dev->canch[i] = gs_make_candev(i, intf, &dconf); if (IS_ERR_OR_NULL(dev->canch[i])) { /* save error code to return later */ rc = PTR_ERR(dev->canch[i]); @@ -1227,18 +1378,28 @@ static int gs_usb_probe(struct usb_interface *intf, gs_destroy_candev(dev->canch[i]); usb_kill_anchored_urbs(&dev->rx_submitted); - kfree(dconf); kfree(dev); return rc; } dev->canch[i]->parent = dev; - if (dev->canch[i]->can.ctrlmode_supported & CAN_CTRLMODE_FD) - dev->hf_size_rx = struct_size(hf, canfd, 1); + /* set RX packet size based on FD and if hardware + * timestamps are supported. + */ + if (dev->canch[i]->can.ctrlmode_supported & CAN_CTRLMODE_FD) { + if (dev->canch[i]->feature & GS_CAN_FEATURE_HW_TIMESTAMP) + hf_size_rx = struct_size(hf, canfd_ts, 1); + else + hf_size_rx = struct_size(hf, canfd, 1); + } else { + if (dev->canch[i]->feature & GS_CAN_FEATURE_HW_TIMESTAMP) + hf_size_rx = struct_size(hf, classic_can_ts, 1); + else + hf_size_rx = struct_size(hf, classic_can, 1); + } + dev->hf_size_rx = max(dev->hf_size_rx, hf_size_rx); } - kfree(dconf); - return 0; } @@ -1263,8 +1424,8 @@ static void gs_usb_disconnect(struct usb_interface *intf) } static const struct usb_device_id gs_usb_table[] = { - { USB_DEVICE_INTERFACE_NUMBER(USB_GSUSB_1_VENDOR_ID, - USB_GSUSB_1_PRODUCT_ID, 0) }, + { USB_DEVICE_INTERFACE_NUMBER(USB_GS_USB_1_VENDOR_ID, + USB_GS_USB_1_PRODUCT_ID, 0) }, { USB_DEVICE_INTERFACE_NUMBER(USB_CANDLELIGHT_VENDOR_ID, USB_CANDLELIGHT_PRODUCT_ID, 0) }, { USB_DEVICE_INTERFACE_NUMBER(USB_CES_CANEXT_FD_VENDOR_ID, diff --git a/drivers/net/can/usb/kvaser_usb/kvaser_usb_hydra.c b/drivers/net/can/usb/kvaser_usb/kvaser_usb_hydra.c index dd65c101bfb8..6871d474dabf 100644 --- a/drivers/net/can/usb/kvaser_usb/kvaser_usb_hydra.c +++ b/drivers/net/can/usb/kvaser_usb/kvaser_usb_hydra.c @@ -534,7 +534,7 @@ static int kvaser_usb_hydra_send_simple_cmd(struct kvaser_usb *dev, struct kvaser_cmd *cmd; int err; - cmd = kcalloc(1, sizeof(struct kvaser_cmd), GFP_KERNEL); + cmd = kzalloc(sizeof(*cmd), GFP_KERNEL); if (!cmd) return -ENOMEM; @@ -573,7 +573,7 @@ kvaser_usb_hydra_send_simple_cmd_async(struct kvaser_usb_net_priv *priv, struct kvaser_usb *dev = priv->dev; int err; - cmd = kcalloc(1, sizeof(struct kvaser_cmd), GFP_ATOMIC); + cmd = kzalloc(sizeof(*cmd), GFP_ATOMIC); if (!cmd) return -ENOMEM; @@ -694,7 +694,7 @@ static int kvaser_usb_hydra_map_channel(struct kvaser_usb *dev, u16 transid, struct kvaser_cmd *cmd; int err; - cmd = kcalloc(1, sizeof(struct kvaser_cmd), GFP_KERNEL); + cmd = kzalloc(sizeof(*cmd), GFP_KERNEL); if (!cmd) return -ENOMEM; @@ -735,7 +735,7 @@ static int kvaser_usb_hydra_get_single_capability(struct kvaser_usb *dev, int err; int i; - cmd = kcalloc(1, sizeof(struct kvaser_cmd), GFP_KERNEL); + cmd = kzalloc(sizeof(*cmd), GFP_KERNEL); if (!cmd) return -ENOMEM; @@ -1394,7 +1394,7 @@ kvaser_usb_hydra_frame_to_cmd_ext(const struct kvaser_usb_net_priv *priv, u32 kcan_id; u32 kcan_header; - cmd = kcalloc(1, sizeof(struct kvaser_cmd_ext), GFP_ATOMIC); + cmd = kzalloc(sizeof(*cmd), GFP_ATOMIC); if (!cmd) return NULL; @@ -1468,7 +1468,7 @@ kvaser_usb_hydra_frame_to_cmd_std(const struct kvaser_usb_net_priv *priv, u32 flags; u32 id; - cmd = kcalloc(1, sizeof(struct kvaser_cmd), GFP_ATOMIC); + cmd = kzalloc(sizeof(*cmd), GFP_ATOMIC); if (!cmd) return NULL; @@ -1533,7 +1533,7 @@ static int kvaser_usb_hydra_set_bittiming(struct net_device *netdev) int sjw = bt->sjw; int err; - cmd = kcalloc(1, sizeof(struct kvaser_cmd), GFP_KERNEL); + cmd = kzalloc(sizeof(*cmd), GFP_KERNEL); if (!cmd) return -ENOMEM; @@ -1567,7 +1567,7 @@ static int kvaser_usb_hydra_set_data_bittiming(struct net_device *netdev) int sjw = dbt->sjw; int err; - cmd = kcalloc(1, sizeof(struct kvaser_cmd), GFP_KERNEL); + cmd = kzalloc(sizeof(*cmd), GFP_KERNEL); if (!cmd) return -ENOMEM; @@ -1711,7 +1711,7 @@ static int kvaser_usb_hydra_get_software_details(struct kvaser_usb *dev) u32 flags; struct kvaser_usb_dev_card_data *card_data = &dev->card_data; - cmd = kcalloc(1, sizeof(struct kvaser_cmd), GFP_KERNEL); + cmd = kzalloc(sizeof(*cmd), GFP_KERNEL); if (!cmd) return -ENOMEM; @@ -1851,7 +1851,7 @@ static int kvaser_usb_hydra_set_opt_mode(const struct kvaser_usb_net_priv *priv) return -EINVAL; } - cmd = kcalloc(1, sizeof(struct kvaser_cmd), GFP_KERNEL); + cmd = kzalloc(sizeof(*cmd), GFP_KERNEL); if (!cmd) return -ENOMEM; diff --git a/drivers/net/can/usb/peak_usb/pcan_usb_core.c b/drivers/net/can/usb/peak_usb/pcan_usb_core.c index 8c9d53f6e24c..225697d70a9a 100644 --- a/drivers/net/can/usb/peak_usb/pcan_usb_core.c +++ b/drivers/net/can/usb/peak_usb/pcan_usb_core.c @@ -962,7 +962,7 @@ static void peak_usb_disconnect(struct usb_interface *intf) dev_prev_siblings = dev->prev_siblings; dev->state &= ~PCAN_USB_STATE_CONNECTED; - strlcpy(name, netdev->name, IFNAMSIZ); + strscpy(name, netdev->name, IFNAMSIZ); unregister_netdev(netdev); diff --git a/drivers/net/can/vcan.c b/drivers/net/can/vcan.c index 36b6310a2e5b..285635c23443 100644 --- a/drivers/net/can/vcan.c +++ b/drivers/net/can/vcan.c @@ -71,11 +71,10 @@ MODULE_PARM_DESC(echo, "Echo sent frames (for testing). Default: 0 (Off)"); static void vcan_rx(struct sk_buff *skb, struct net_device *dev) { - struct canfd_frame *cfd = (struct canfd_frame *)skb->data; struct net_device_stats *stats = &dev->stats; stats->rx_packets++; - stats->rx_bytes += cfd->len; + stats->rx_bytes += can_skb_get_data_len(skb); skb->pkt_type = PACKET_BROADCAST; skb->dev = dev; @@ -86,14 +85,14 @@ static void vcan_rx(struct sk_buff *skb, struct net_device *dev) static netdev_tx_t vcan_tx(struct sk_buff *skb, struct net_device *dev) { - struct canfd_frame *cfd = (struct canfd_frame *)skb->data; struct net_device_stats *stats = &dev->stats; - int loop, len; + unsigned int len; + int loop; if (can_dropped_invalid_skb(dev, skb)) return NETDEV_TX_OK; - len = cfd->can_id & CAN_RTR_FLAG ? 0 : cfd->len; + len = can_skb_get_data_len(skb); stats->tx_packets++; stats->tx_bytes += len; @@ -137,7 +136,8 @@ static int vcan_change_mtu(struct net_device *dev, int new_mtu) if (dev->flags & IFF_UP) return -EBUSY; - if (new_mtu != CAN_MTU && new_mtu != CANFD_MTU) + if (new_mtu != CAN_MTU && new_mtu != CANFD_MTU && + !can_is_canxl_dev_mtu(new_mtu)) return -EINVAL; dev->mtu = new_mtu; diff --git a/drivers/net/can/vxcan.c b/drivers/net/can/vxcan.c index cffd107d8b28..26a472d2ea58 100644 --- a/drivers/net/can/vxcan.c +++ b/drivers/net/can/vxcan.c @@ -38,10 +38,9 @@ static netdev_tx_t vxcan_xmit(struct sk_buff *oskb, struct net_device *dev) { struct vxcan_priv *priv = netdev_priv(dev); struct net_device *peer; - struct canfd_frame *cfd = (struct canfd_frame *)oskb->data; struct net_device_stats *peerstats, *srcstats = &dev->stats; struct sk_buff *skb; - u8 len; + unsigned int len; if (can_dropped_invalid_skb(dev, oskb)) return NETDEV_TX_OK; @@ -70,7 +69,7 @@ static netdev_tx_t vxcan_xmit(struct sk_buff *oskb, struct net_device *dev) skb->dev = peer; skb->ip_summed = CHECKSUM_UNNECESSARY; - len = cfd->can_id & CAN_RTR_FLAG ? 0 : cfd->len; + len = can_skb_get_data_len(skb); if (netif_rx(skb) == NET_RX_SUCCESS) { srcstats->tx_packets++; srcstats->tx_bytes += len; @@ -132,7 +131,8 @@ static int vxcan_change_mtu(struct net_device *dev, int new_mtu) if (dev->flags & IFF_UP) return -EBUSY; - if (new_mtu != CAN_MTU && new_mtu != CANFD_MTU) + if (new_mtu != CAN_MTU && new_mtu != CANFD_MTU && + !can_is_canxl_dev_mtu(new_mtu)) return -EINVAL; dev->mtu = new_mtu; diff --git a/drivers/net/dsa/Kconfig b/drivers/net/dsa/Kconfig index d8ae0e8af2a0..07507b4820d7 100644 --- a/drivers/net/dsa/Kconfig +++ b/drivers/net/dsa/Kconfig @@ -76,7 +76,7 @@ config NET_DSA_SMSC_LAN9303 select NET_DSA_TAG_LAN9303 select REGMAP help - This enables support for the SMSC/Microchip LAN9303 3 port ethernet + This enables support for the Microchip LAN9303/LAN9354 3 port ethernet switch chips. config NET_DSA_SMSC_LAN9303_I2C @@ -90,11 +90,11 @@ config NET_DSA_SMSC_LAN9303_I2C for I2C managed mode. config NET_DSA_SMSC_LAN9303_MDIO - tristate "SMSC/Microchip LAN9303 3-ports 10/100 ethernet switch in MDIO managed mode" + tristate "Microchip LAN9303/LAN9354 3-ports 10/100 ethernet switch in MDIO managed mode" select NET_DSA_SMSC_LAN9303 depends on VLAN_8021Q || VLAN_8021Q=n help - Enable access functions if the SMSC/Microchip LAN9303 is configured + Enable access functions if the Microchip LAN9303/LAN9354 is configured for MDIO managed mode. config NET_DSA_VITESSE_VSC73XX diff --git a/drivers/net/dsa/b53/b53_common.c b/drivers/net/dsa/b53/b53_common.c index 48cf344750ff..59cdfc51ce06 100644 --- a/drivers/net/dsa/b53/b53_common.c +++ b/drivers/net/dsa/b53/b53_common.c @@ -972,7 +972,7 @@ void b53_get_strings(struct dsa_switch *ds, int port, u32 stringset, if (stringset == ETH_SS_STATS) { for (i = 0; i < mib_size; i++) - strlcpy(data + i * ETH_GSTRING_LEN, + strscpy(data + i * ETH_GSTRING_LEN, mibs[i].name, ETH_GSTRING_LEN); } else if (stringset == ETH_SS_PHY_STATS) { phydev = b53_get_phy_device(ds, port); diff --git a/drivers/net/dsa/b53/b53_mdio.c b/drivers/net/dsa/b53/b53_mdio.c index a7aeb3c132c9..6ddc03b58b28 100644 --- a/drivers/net/dsa/b53/b53_mdio.c +++ b/drivers/net/dsa/b53/b53_mdio.c @@ -356,8 +356,6 @@ static void b53_mdio_remove(struct mdio_device *mdiodev) return; b53_switch_remove(dev); - - dev_set_drvdata(&mdiodev->dev, NULL); } static void b53_mdio_shutdown(struct mdio_device *mdiodev) diff --git a/drivers/net/dsa/b53/b53_mmap.c b/drivers/net/dsa/b53/b53_mmap.c index ae4c79d39bc0..e968322dfbf0 100644 --- a/drivers/net/dsa/b53/b53_mmap.c +++ b/drivers/net/dsa/b53/b53_mmap.c @@ -316,8 +316,6 @@ static int b53_mmap_remove(struct platform_device *pdev) if (dev) b53_switch_remove(dev); - platform_set_drvdata(pdev, NULL); - return 0; } diff --git a/drivers/net/dsa/b53/b53_srab.c b/drivers/net/dsa/b53/b53_srab.c index da0b889880f6..bcb44034404d 100644 --- a/drivers/net/dsa/b53/b53_srab.c +++ b/drivers/net/dsa/b53/b53_srab.c @@ -667,8 +667,6 @@ static int b53_srab_remove(struct platform_device *pdev) b53_srab_intr_set(dev->priv, false); b53_switch_remove(dev); - platform_set_drvdata(pdev, NULL); - return 0; } diff --git a/drivers/net/dsa/bcm_sf2.c b/drivers/net/dsa/bcm_sf2.c index be0edfa093d0..cde253d27bd0 100644 --- a/drivers/net/dsa/bcm_sf2.c +++ b/drivers/net/dsa/bcm_sf2.c @@ -94,6 +94,24 @@ static u16 bcm_sf2_reg_led_base(struct bcm_sf2_priv *priv, int port) return REG_SWITCH_STATUS; } +static u32 bcm_sf2_port_override_offset(struct bcm_sf2_priv *priv, int port) +{ + switch (priv->type) { + case BCM4908_DEVICE_ID: + case BCM7445_DEVICE_ID: + return port == 8 ? CORE_STS_OVERRIDE_IMP : + CORE_STS_OVERRIDE_GMIIP_PORT(port); + case BCM7278_DEVICE_ID: + return port == 8 ? CORE_STS_OVERRIDE_IMP2 : + CORE_STS_OVERRIDE_GMIIP2_PORT(port); + default: + WARN_ONCE(1, "Unsupported device: %d\n", priv->type); + } + + /* RO fallback register */ + return REG_SWITCH_STATUS; +} + /* Return the number of active ports, not counting the IMP (CPU) port */ static unsigned int bcm_sf2_num_active_ports(struct dsa_switch *ds) { @@ -141,7 +159,7 @@ static void bcm_sf2_imp_setup(struct dsa_switch *ds, int port) { struct bcm_sf2_priv *priv = bcm_sf2_to_priv(ds); unsigned int i; - u32 reg, offset; + u32 reg; /* Enable the port memories */ reg = core_readl(priv, CORE_MEM_PSM_VDD_CTRL); @@ -167,21 +185,6 @@ static void bcm_sf2_imp_setup(struct dsa_switch *ds, int port) b53_brcm_hdr_setup(ds, port); if (port == 8) { - if (priv->type == BCM4908_DEVICE_ID || - priv->type == BCM7445_DEVICE_ID) - offset = CORE_STS_OVERRIDE_IMP; - else - offset = CORE_STS_OVERRIDE_IMP2; - - /* Force link status for IMP port */ - reg = core_readl(priv, offset); - reg |= (MII_SW_OR | LINK_STS); - if (priv->type == BCM4908_DEVICE_ID) - reg |= GMII_SPEED_UP_2G; - else - reg &= ~GMII_SPEED_UP_2G; - core_writel(priv, reg, offset); - /* Enable Broadcast, Multicast, Unicast forwarding to IMP port */ reg = core_readl(priv, CORE_IMP_CTL); reg |= (RX_BCST_EN | RX_MCST_EN | RX_UCST_EN); @@ -812,17 +815,10 @@ static void bcm_sf2_sw_mac_link_down(struct dsa_switch *ds, int port, if (priv->wol_ports_mask & BIT(port)) return; - if (port != core_readl(priv, CORE_IMP0_PRT_ID)) { - if (priv->type == BCM4908_DEVICE_ID || - priv->type == BCM7445_DEVICE_ID) - offset = CORE_STS_OVERRIDE_GMIIP_PORT(port); - else - offset = CORE_STS_OVERRIDE_GMIIP2_PORT(port); - - reg = core_readl(priv, offset); - reg &= ~LINK_STS; - core_writel(priv, reg, offset); - } + offset = bcm_sf2_port_override_offset(priv, port); + reg = core_readl(priv, offset); + reg &= ~LINK_STS; + core_writel(priv, reg, offset); bcm_sf2_sw_mac_link_set(ds, port, interface, false); } @@ -836,56 +832,56 @@ static void bcm_sf2_sw_mac_link_up(struct dsa_switch *ds, int port, { struct bcm_sf2_priv *priv = bcm_sf2_to_priv(ds); struct ethtool_eee *p = &priv->dev->ports[port].eee; + u32 reg_rgmii_ctrl = 0; + u32 reg, offset; bcm_sf2_sw_mac_link_set(ds, port, interface, true); - if (port != core_readl(priv, CORE_IMP0_PRT_ID)) { - u32 reg_rgmii_ctrl = 0; - u32 reg, offset; + offset = bcm_sf2_port_override_offset(priv, port); - if (priv->type == BCM4908_DEVICE_ID || - priv->type == BCM7445_DEVICE_ID) - offset = CORE_STS_OVERRIDE_GMIIP_PORT(port); - else - offset = CORE_STS_OVERRIDE_GMIIP2_PORT(port); - - if (interface == PHY_INTERFACE_MODE_RGMII || - interface == PHY_INTERFACE_MODE_RGMII_TXID || - interface == PHY_INTERFACE_MODE_MII || - interface == PHY_INTERFACE_MODE_REVMII) { - reg_rgmii_ctrl = bcm_sf2_reg_rgmii_cntrl(priv, port); - reg = reg_readl(priv, reg_rgmii_ctrl); - reg &= ~(RX_PAUSE_EN | TX_PAUSE_EN); - - if (tx_pause) - reg |= TX_PAUSE_EN; - if (rx_pause) - reg |= RX_PAUSE_EN; - - reg_writel(priv, reg, reg_rgmii_ctrl); - } - - reg = SW_OVERRIDE | LINK_STS; - switch (speed) { - case SPEED_1000: - reg |= SPDSTS_1000 << SPEED_SHIFT; - break; - case SPEED_100: - reg |= SPDSTS_100 << SPEED_SHIFT; - break; - } - - if (duplex == DUPLEX_FULL) - reg |= DUPLX_MODE; + if (phy_interface_mode_is_rgmii(interface) || + interface == PHY_INTERFACE_MODE_MII || + interface == PHY_INTERFACE_MODE_REVMII) { + reg_rgmii_ctrl = bcm_sf2_reg_rgmii_cntrl(priv, port); + reg = reg_readl(priv, reg_rgmii_ctrl); + reg &= ~(RX_PAUSE_EN | TX_PAUSE_EN); if (tx_pause) - reg |= TXFLOW_CNTL; + reg |= TX_PAUSE_EN; if (rx_pause) - reg |= RXFLOW_CNTL; + reg |= RX_PAUSE_EN; - core_writel(priv, reg, offset); + reg_writel(priv, reg, reg_rgmii_ctrl); } + reg = LINK_STS; + if (port == 8) { + if (priv->type == BCM4908_DEVICE_ID) + reg |= GMII_SPEED_UP_2G; + reg |= MII_SW_OR; + } else { + reg |= SW_OVERRIDE; + } + + switch (speed) { + case SPEED_1000: + reg |= SPDSTS_1000 << SPEED_SHIFT; + break; + case SPEED_100: + reg |= SPDSTS_100 << SPEED_SHIFT; + break; + } + + if (duplex == DUPLEX_FULL) + reg |= DUPLX_MODE; + + if (tx_pause) + reg |= TXFLOW_CNTL; + if (rx_pause) + reg |= RXFLOW_CNTL; + + core_writel(priv, reg, offset); + if (mode == MLO_AN_PHY && phydev) p->eee_enabled = b53_eee_init(ds, port, phydev); } @@ -987,7 +983,7 @@ static int bcm_sf2_sw_resume(struct dsa_switch *ds) static void bcm_sf2_sw_get_wol(struct dsa_switch *ds, int port, struct ethtool_wolinfo *wol) { - struct net_device *p = dsa_to_port(ds, port)->cpu_dp->master; + struct net_device *p = dsa_port_to_master(dsa_to_port(ds, port)); struct bcm_sf2_priv *priv = bcm_sf2_to_priv(ds); struct ethtool_wolinfo pwol = { }; @@ -1011,7 +1007,7 @@ static void bcm_sf2_sw_get_wol(struct dsa_switch *ds, int port, static int bcm_sf2_sw_set_wol(struct dsa_switch *ds, int port, struct ethtool_wolinfo *wol) { - struct net_device *p = dsa_to_port(ds, port)->cpu_dp->master; + struct net_device *p = dsa_port_to_master(dsa_to_port(ds, port)); struct bcm_sf2_priv *priv = bcm_sf2_to_priv(ds); s8 cpu_port = dsa_to_port(ds, port)->cpu_dp->index; struct ethtool_wolinfo pwol = { }; @@ -1555,8 +1551,6 @@ static int bcm_sf2_sw_remove(struct platform_device *pdev) if (priv->type == BCM7278_DEVICE_ID) reset_control_assert(priv->rcdev); - platform_set_drvdata(pdev, NULL); - return 0; } diff --git a/drivers/net/dsa/bcm_sf2_cfp.c b/drivers/net/dsa/bcm_sf2_cfp.c index edbe5e7f1cb6..c4010b7bf089 100644 --- a/drivers/net/dsa/bcm_sf2_cfp.c +++ b/drivers/net/dsa/bcm_sf2_cfp.c @@ -1102,7 +1102,7 @@ static int bcm_sf2_cfp_rule_get_all(struct bcm_sf2_priv *priv, int bcm_sf2_get_rxnfc(struct dsa_switch *ds, int port, struct ethtool_rxnfc *nfc, u32 *rule_locs) { - struct net_device *p = dsa_to_port(ds, port)->cpu_dp->master; + struct net_device *p = dsa_port_to_master(dsa_to_port(ds, port)); struct bcm_sf2_priv *priv = bcm_sf2_to_priv(ds); int ret = 0; @@ -1145,7 +1145,7 @@ int bcm_sf2_get_rxnfc(struct dsa_switch *ds, int port, int bcm_sf2_set_rxnfc(struct dsa_switch *ds, int port, struct ethtool_rxnfc *nfc) { - struct net_device *p = dsa_to_port(ds, port)->cpu_dp->master; + struct net_device *p = dsa_port_to_master(dsa_to_port(ds, port)); struct bcm_sf2_priv *priv = bcm_sf2_to_priv(ds); int ret = 0; @@ -1296,7 +1296,7 @@ void bcm_sf2_cfp_get_strings(struct dsa_switch *ds, int port, "CFP%03d_%sCntr", i, bcm_sf2_cfp_stats[j].name); iter = (i - 1) * s + j; - strlcpy(data + iter * ETH_GSTRING_LEN, + strscpy(data + iter * ETH_GSTRING_LEN, buf, ETH_GSTRING_LEN); } } diff --git a/drivers/net/dsa/dsa_loop.c b/drivers/net/dsa/dsa_loop.c index 263e41191c29..b9107fe40023 100644 --- a/drivers/net/dsa/dsa_loop.c +++ b/drivers/net/dsa/dsa_loop.c @@ -351,8 +351,6 @@ static void dsa_loop_drv_remove(struct mdio_device *mdiodev) dsa_unregister_switch(ds); dev_put(ps->netdev); - - dev_set_drvdata(&mdiodev->dev, NULL); } static void dsa_loop_drv_shutdown(struct mdio_device *mdiodev) diff --git a/drivers/net/dsa/hirschmann/hellcreek.c b/drivers/net/dsa/hirschmann/hellcreek.c index 01f90994dedd..951f7935c872 100644 --- a/drivers/net/dsa/hirschmann/hellcreek.c +++ b/drivers/net/dsa/hirschmann/hellcreek.c @@ -128,6 +128,16 @@ static void hellcreek_select_prio(struct hellcreek *hellcreek, int prio) hellcreek_write(hellcreek, val, HR_PSEL); } +static void hellcreek_select_port_prio(struct hellcreek *hellcreek, int port, + int prio) +{ + u16 val = port << HR_PSEL_PTWSEL_SHIFT; + + val |= prio << HR_PSEL_PRTCWSEL_SHIFT; + + hellcreek_write(hellcreek, val, HR_PSEL); +} + static void hellcreek_select_counter(struct hellcreek *hellcreek, int counter) { u16 val = counter << HR_CSEL_SHIFT; @@ -288,7 +298,7 @@ static void hellcreek_get_strings(struct dsa_switch *ds, int port, for (i = 0; i < ARRAY_SIZE(hellcreek_counter); ++i) { const struct hellcreek_counter *counter = &hellcreek_counter[i]; - strlcpy(data + i * ETH_GSTRING_LEN, + strscpy(data + i * ETH_GSTRING_LEN, counter->name, ETH_GSTRING_LEN); } } @@ -1537,6 +1547,45 @@ out: return ret; } +static void hellcreek_setup_maxsdu(struct hellcreek *hellcreek, int port, + const struct tc_taprio_qopt_offload *schedule) +{ + int tc; + + for (tc = 0; tc < 8; ++tc) { + u32 max_sdu = schedule->max_sdu[tc] + VLAN_ETH_HLEN - ETH_FCS_LEN; + u16 val; + + if (!schedule->max_sdu[tc]) + continue; + + dev_dbg(hellcreek->dev, "Configure max-sdu %u for tc %d on port %d\n", + max_sdu, tc, port); + + hellcreek_select_port_prio(hellcreek, port, tc); + + val = (max_sdu & HR_PTPRTCCFG_MAXSDU_MASK) << HR_PTPRTCCFG_MAXSDU_SHIFT; + + hellcreek_write(hellcreek, val, HR_PTPRTCCFG); + } +} + +static void hellcreek_reset_maxsdu(struct hellcreek *hellcreek, int port) +{ + int tc; + + for (tc = 0; tc < 8; ++tc) { + u16 val; + + hellcreek_select_port_prio(hellcreek, port, tc); + + val = (HELLCREEK_DEFAULT_MAX_SDU & HR_PTPRTCCFG_MAXSDU_MASK) + << HR_PTPRTCCFG_MAXSDU_SHIFT; + + hellcreek_write(hellcreek, val, HR_PTPRTCCFG); + } +} + static void hellcreek_setup_gcl(struct hellcreek *hellcreek, int port, const struct tc_taprio_qopt_offload *schedule) { @@ -1720,7 +1769,10 @@ static int hellcreek_port_set_schedule(struct dsa_switch *ds, int port, } hellcreek_port->current_schedule = taprio_offload_get(taprio); - /* Then select port */ + /* Configure max sdu */ + hellcreek_setup_maxsdu(hellcreek, port, hellcreek_port->current_schedule); + + /* Select tdg */ hellcreek_select_tgd(hellcreek, port); /* Enable gating and keep defaults */ @@ -1772,7 +1824,10 @@ static int hellcreek_port_del_schedule(struct dsa_switch *ds, int port) hellcreek_port->current_schedule = NULL; } - /* Then select port */ + /* Reset max sdu */ + hellcreek_reset_maxsdu(hellcreek, port); + + /* Select tgd */ hellcreek_select_tgd(hellcreek, port); /* Disable gating and return to regular switching flow */ @@ -1809,22 +1864,43 @@ static bool hellcreek_validate_schedule(struct hellcreek *hellcreek, return true; } +static int hellcreek_tc_query_caps(struct tc_query_caps_base *base) +{ + switch (base->type) { + case TC_SETUP_QDISC_TAPRIO: { + struct tc_taprio_caps *caps = base->caps; + + caps->supports_queue_max_sdu = true; + + return 0; + } + default: + return -EOPNOTSUPP; + } +} + static int hellcreek_port_setup_tc(struct dsa_switch *ds, int port, enum tc_setup_type type, void *type_data) { - struct tc_taprio_qopt_offload *taprio = type_data; struct hellcreek *hellcreek = ds->priv; - if (type != TC_SETUP_QDISC_TAPRIO) - return -EOPNOTSUPP; + switch (type) { + case TC_QUERY_CAPS: + return hellcreek_tc_query_caps(type_data); + case TC_SETUP_QDISC_TAPRIO: { + struct tc_taprio_qopt_offload *taprio = type_data; - if (!hellcreek_validate_schedule(hellcreek, taprio)) - return -EOPNOTSUPP; + if (!hellcreek_validate_schedule(hellcreek, taprio)) + return -EOPNOTSUPP; - if (taprio->enable) - return hellcreek_port_set_schedule(ds, port, taprio); + if (taprio->enable) + return hellcreek_port_set_schedule(ds, port, taprio); - return hellcreek_port_del_schedule(ds, port); + return hellcreek_port_del_schedule(ds, port); + } + default: + return -EOPNOTSUPP; + } } static const struct dsa_switch_ops hellcreek_ds_ops = { @@ -1996,7 +2072,6 @@ static int hellcreek_remove(struct platform_device *pdev) hellcreek_hwtstamp_free(hellcreek); hellcreek_ptp_free(hellcreek); dsa_unregister_switch(hellcreek->ds); - platform_set_drvdata(pdev, NULL); return 0; } diff --git a/drivers/net/dsa/hirschmann/hellcreek.h b/drivers/net/dsa/hirschmann/hellcreek.h index 9e303b8ab13c..4a678f7d61ae 100644 --- a/drivers/net/dsa/hirschmann/hellcreek.h +++ b/drivers/net/dsa/hirschmann/hellcreek.h @@ -37,6 +37,7 @@ #define HELLCREEK_VLAN_UNTAGGED_MEMBER 0x1 #define HELLCREEK_VLAN_TAGGED_MEMBER 0x3 #define HELLCREEK_NUM_EGRESS_QUEUES 8 +#define HELLCREEK_DEFAULT_MAX_SDU 1536 /* Register definitions */ #define HR_MODID_C (0 * 2) @@ -72,6 +73,12 @@ #define HR_PRTCCFG_PCP_TC_MAP_SHIFT 0 #define HR_PRTCCFG_PCP_TC_MAP_MASK GENMASK(2, 0) +#define HR_PTPRTCCFG (0xa9 * 2) +#define HR_PTPRTCCFG_SET_QTRACK BIT(15) +#define HR_PTPRTCCFG_REJECT BIT(14) +#define HR_PTPRTCCFG_MAXSDU_SHIFT 0 +#define HR_PTPRTCCFG_MAXSDU_MASK GENMASK(10, 0) + #define HR_CSEL (0x8d * 2) #define HR_CSEL_SHIFT 0 #define HR_CSEL_MASK GENMASK(7, 0) diff --git a/drivers/net/dsa/lan9303-core.c b/drivers/net/dsa/lan9303-core.c index e03ff1f267bb..438e46af03e9 100644 --- a/drivers/net/dsa/lan9303-core.c +++ b/drivers/net/dsa/lan9303-core.c @@ -22,6 +22,10 @@ */ #define LAN9303_CHIP_REV 0x14 # define LAN9303_CHIP_ID 0x9303 +# define LAN9352_CHIP_ID 0x9352 +# define LAN9353_CHIP_ID 0x9353 +# define LAN9354_CHIP_ID 0x9354 +# define LAN9355_CHIP_ID 0x9355 #define LAN9303_IRQ_CFG 0x15 # define LAN9303_IRQ_CFG_IRQ_ENABLE BIT(8) # define LAN9303_IRQ_CFG_IRQ_POL BIT(4) @@ -32,6 +36,7 @@ #define LAN9303_INT_EN 0x17 # define LAN9303_INT_EN_PHY_INT2_EN BIT(27) # define LAN9303_INT_EN_PHY_INT1_EN BIT(26) +#define LAN9303_BYTE_ORDER 0x19 #define LAN9303_HW_CFG 0x1D # define LAN9303_HW_CFG_READY BIT(27) # define LAN9303_HW_CFG_AMDX_EN_PORT2 BIT(26) @@ -851,15 +856,12 @@ static int lan9303_check_device(struct lan9303 *chip) if (ret) { dev_err(chip->dev, "failed to read chip revision register: %d\n", ret); - if (!chip->reset_gpio) { - dev_dbg(chip->dev, - "hint: maybe failed due to missing reset GPIO\n"); - } return ret; } - if ((reg >> 16) != LAN9303_CHIP_ID) { - dev_err(chip->dev, "expecting LAN9303 chip, but found: %X\n", + if (((reg >> 16) != LAN9303_CHIP_ID) && + ((reg >> 16) != LAN9354_CHIP_ID)) { + dev_err(chip->dev, "unexpected device found: LAN%4.4X\n", reg >> 16); return -ENODEV; } @@ -875,7 +877,7 @@ static int lan9303_check_device(struct lan9303 *chip) if (ret) dev_warn(chip->dev, "failed to disable switching %d\n", ret); - dev_info(chip->dev, "Found LAN9303 rev. %u\n", reg & 0xffff); + dev_info(chip->dev, "Found LAN%4.4X rev. %u\n", (reg >> 16), reg & 0xffff); ret = lan9303_detect_phy_setup(chip); if (ret) { @@ -1090,7 +1092,7 @@ static int lan9303_port_enable(struct dsa_switch *ds, int port, if (!dsa_port_is_user(dp)) return 0; - vlan_vid_add(dp->cpu_dp->master, htons(ETH_P_8021Q), port); + vlan_vid_add(dsa_port_to_master(dp), htons(ETH_P_8021Q), port); return lan9303_enable_processing_port(chip, port); } @@ -1103,7 +1105,7 @@ static void lan9303_port_disable(struct dsa_switch *ds, int port) if (!dsa_port_is_user(dp)) return; - vlan_vid_del(dp->cpu_dp->master, htons(ETH_P_8021Q), port); + vlan_vid_del(dsa_port_to_master(dp), htons(ETH_P_8021Q), port); lan9303_disable_processing_port(chip, port); lan9303_phy_write(ds, chip->phy_addr_base + port, MII_BMCR, BMCR_PDOWN); @@ -1349,6 +1351,7 @@ static int lan9303_probe_reset_gpio(struct lan9303 *chip, int lan9303_probe(struct lan9303 *chip, struct device_node *np) { int ret; + u32 reg; mutex_init(&chip->indirect_mutex); mutex_init(&chip->alr_mutex); @@ -1359,6 +1362,19 @@ int lan9303_probe(struct lan9303 *chip, struct device_node *np) lan9303_handle_reset(chip); + /* First read to the device. This is a Dummy read to ensure MDIO */ + /* access is in 32-bit sync. */ + ret = lan9303_read(chip->regmap, LAN9303_BYTE_ORDER, ®); + if (ret) { + dev_err(chip->dev, "failed to access the device: %d\n", + ret); + if (!chip->reset_gpio) { + dev_dbg(chip->dev, + "hint: maybe failed due to missing reset GPIO\n"); + } + return ret; + } + ret = lan9303_check_device(chip); if (ret) return ret; diff --git a/drivers/net/dsa/lan9303_i2c.c b/drivers/net/dsa/lan9303_i2c.c index 8ca4713310fa..79be5fc044bd 100644 --- a/drivers/net/dsa/lan9303_i2c.c +++ b/drivers/net/dsa/lan9303_i2c.c @@ -74,8 +74,6 @@ static int lan9303_i2c_remove(struct i2c_client *client) lan9303_remove(&sw_dev->chip); - i2c_set_clientdata(client, NULL); - return 0; } diff --git a/drivers/net/dsa/lan9303_mdio.c b/drivers/net/dsa/lan9303_mdio.c index bbb7032409ba..4f33369a2de5 100644 --- a/drivers/net/dsa/lan9303_mdio.c +++ b/drivers/net/dsa/lan9303_mdio.c @@ -138,8 +138,6 @@ static void lan9303_mdio_remove(struct mdio_device *mdiodev) return; lan9303_remove(&sw_dev->chip); - - dev_set_drvdata(&mdiodev->dev, NULL); } static void lan9303_mdio_shutdown(struct mdio_device *mdiodev) @@ -158,6 +156,7 @@ static void lan9303_mdio_shutdown(struct mdio_device *mdiodev) static const struct of_device_id lan9303_mdio_of_match[] = { { .compatible = "smsc,lan9303-mdio" }, + { .compatible = "microchip,lan9354-mdio" }, { /* sentinel */ }, }; MODULE_DEVICE_TABLE(of, lan9303_mdio_of_match); diff --git a/drivers/net/dsa/lantiq_gswip.c b/drivers/net/dsa/lantiq_gswip.c index e531b93f3cb2..05ecaa007ab1 100644 --- a/drivers/net/dsa/lantiq_gswip.c +++ b/drivers/net/dsa/lantiq_gswip.c @@ -1989,11 +1989,9 @@ static int gswip_gphy_fw_probe(struct gswip_priv *priv, } gphy_fw->reset = of_reset_control_array_get_exclusive(gphy_fw_np); - if (IS_ERR(gphy_fw->reset)) { - if (PTR_ERR(gphy_fw->reset) != -EPROBE_DEFER) - dev_err(dev, "Failed to lookup gphy reset\n"); - return PTR_ERR(gphy_fw->reset); - } + if (IS_ERR(gphy_fw->reset)) + return dev_err_probe(dev, PTR_ERR(gphy_fw->reset), + "Failed to lookup gphy reset\n"); return gswip_gphy_fw_load(priv, gphy_fw); } @@ -2231,8 +2229,6 @@ static int gswip_remove(struct platform_device *pdev) for (i = 0; i < priv->num_gphy_fw; i++) gswip_gphy_fw_remove(priv, &priv->gphy_fw[i]); - platform_set_drvdata(pdev, NULL); - return 0; } diff --git a/drivers/net/dsa/microchip/ksz8.h b/drivers/net/dsa/microchip/ksz8.h index 42c50cc4d853..8582b4b67d98 100644 --- a/drivers/net/dsa/microchip/ksz8.h +++ b/drivers/net/dsa/microchip/ksz8.h @@ -17,8 +17,8 @@ u32 ksz8_get_port_addr(int port, int offset); void ksz8_cfg_port_member(struct ksz_device *dev, int port, u8 member); void ksz8_flush_dyn_mac_table(struct ksz_device *dev, int port); void ksz8_port_setup(struct ksz_device *dev, int port, bool cpu_port); -void ksz8_r_phy(struct ksz_device *dev, u16 phy, u16 reg, u16 *val); -void ksz8_w_phy(struct ksz_device *dev, u16 phy, u16 reg, u16 val); +int ksz8_r_phy(struct ksz_device *dev, u16 phy, u16 reg, u16 *val); +int ksz8_w_phy(struct ksz_device *dev, u16 phy, u16 reg, u16 val); int ksz8_r_dyn_mac_table(struct ksz_device *dev, u16 addr, u8 *mac_addr, u8 *fid, u8 *src_port, u8 *timestamp, u16 *entries); int ksz8_r_sta_mac_table(struct ksz_device *dev, u16 addr, diff --git a/drivers/net/dsa/microchip/ksz8795.c b/drivers/net/dsa/microchip/ksz8795.c index c79a5128235f..bd3b133e7085 100644 --- a/drivers/net/dsa/microchip/ksz8795.c +++ b/drivers/net/dsa/microchip/ksz8795.c @@ -552,7 +552,7 @@ static void ksz8_w_vlan_table(struct ksz_device *dev, u16 vid, u16 vlan) ksz8_w_table(dev, TABLE_VLAN, addr, buf); } -void ksz8_r_phy(struct ksz_device *dev, u16 phy, u16 reg, u16 *val) +int ksz8_r_phy(struct ksz_device *dev, u16 phy, u16 reg, u16 *val) { u8 restart, speed, ctrl, link; int processed = true; @@ -560,14 +560,24 @@ void ksz8_r_phy(struct ksz_device *dev, u16 phy, u16 reg, u16 *val) u8 val1, val2; u16 data = 0; u8 p = phy; + int ret; regs = dev->info->regs; switch (reg) { case MII_BMCR: - ksz_pread8(dev, p, regs[P_NEG_RESTART_CTRL], &restart); - ksz_pread8(dev, p, regs[P_SPEED_STATUS], &speed); - ksz_pread8(dev, p, regs[P_FORCE_CTRL], &ctrl); + ret = ksz_pread8(dev, p, regs[P_NEG_RESTART_CTRL], &restart); + if (ret) + return ret; + + ret = ksz_pread8(dev, p, regs[P_SPEED_STATUS], &speed); + if (ret) + return ret; + + ret = ksz_pread8(dev, p, regs[P_FORCE_CTRL], &ctrl); + if (ret) + return ret; + if (restart & PORT_PHY_LOOPBACK) data |= BMCR_LOOPBACK; if (ctrl & PORT_FORCE_100_MBIT) @@ -597,7 +607,10 @@ void ksz8_r_phy(struct ksz_device *dev, u16 phy, u16 reg, u16 *val) data |= KSZ886X_BMCR_DISABLE_LED; break; case MII_BMSR: - ksz_pread8(dev, p, regs[P_LINK_STATUS], &link); + ret = ksz_pread8(dev, p, regs[P_LINK_STATUS], &link); + if (ret) + return ret; + data = BMSR_100FULL | BMSR_100HALF | BMSR_10FULL | @@ -618,7 +631,10 @@ void ksz8_r_phy(struct ksz_device *dev, u16 phy, u16 reg, u16 *val) data = KSZ8795_ID_LO; break; case MII_ADVERTISE: - ksz_pread8(dev, p, regs[P_LOCAL_CTRL], &ctrl); + ret = ksz_pread8(dev, p, regs[P_LOCAL_CTRL], &ctrl); + if (ret) + return ret; + data = ADVERTISE_CSMA; if (ctrl & PORT_AUTO_NEG_SYM_PAUSE) data |= ADVERTISE_PAUSE_CAP; @@ -632,7 +648,10 @@ void ksz8_r_phy(struct ksz_device *dev, u16 phy, u16 reg, u16 *val) data |= ADVERTISE_10HALF; break; case MII_LPA: - ksz_pread8(dev, p, regs[P_REMOTE_STATUS], &link); + ret = ksz_pread8(dev, p, regs[P_REMOTE_STATUS], &link); + if (ret) + return ret; + data = LPA_SLCT; if (link & PORT_REMOTE_SYM_PAUSE) data |= LPA_PAUSE_CAP; @@ -648,8 +667,14 @@ void ksz8_r_phy(struct ksz_device *dev, u16 phy, u16 reg, u16 *val) data |= LPA_LPACK; break; case PHY_REG_LINK_MD: - ksz_pread8(dev, p, REG_PORT_LINK_MD_CTRL, &val1); - ksz_pread8(dev, p, REG_PORT_LINK_MD_RESULT, &val2); + ret = ksz_pread8(dev, p, REG_PORT_LINK_MD_CTRL, &val1); + if (ret) + return ret; + + ret = ksz_pread8(dev, p, REG_PORT_LINK_MD_RESULT, &val2); + if (ret) + return ret; + if (val1 & PORT_START_CABLE_DIAG) data |= PHY_START_CABLE_DIAG; @@ -664,7 +689,10 @@ void ksz8_r_phy(struct ksz_device *dev, u16 phy, u16 reg, u16 *val) FIELD_GET(PORT_CABLE_FAULT_COUNTER_L, val2)); break; case PHY_REG_PHY_CTRL: - ksz_pread8(dev, p, regs[P_LINK_STATUS], &link); + ret = ksz_pread8(dev, p, regs[P_LINK_STATUS], &link); + if (ret) + return ret; + if (link & PORT_MDIX_STATUS) data |= KSZ886X_CTRL_MDIX_STAT; break; @@ -674,13 +702,16 @@ void ksz8_r_phy(struct ksz_device *dev, u16 phy, u16 reg, u16 *val) } if (processed) *val = data; + + return 0; } -void ksz8_w_phy(struct ksz_device *dev, u16 phy, u16 reg, u16 val) +int ksz8_w_phy(struct ksz_device *dev, u16 phy, u16 reg, u16 val) { u8 restart, speed, ctrl, data; const u16 *regs; u8 p = phy; + int ret; regs = dev->info->regs; @@ -690,15 +721,26 @@ void ksz8_w_phy(struct ksz_device *dev, u16 phy, u16 reg, u16 val) /* Do not support PHY reset function. */ if (val & BMCR_RESET) break; - ksz_pread8(dev, p, regs[P_SPEED_STATUS], &speed); + ret = ksz_pread8(dev, p, regs[P_SPEED_STATUS], &speed); + if (ret) + return ret; + data = speed; if (val & KSZ886X_BMCR_HP_MDIX) data |= PORT_HP_MDIX; else data &= ~PORT_HP_MDIX; - if (data != speed) - ksz_pwrite8(dev, p, regs[P_SPEED_STATUS], data); - ksz_pread8(dev, p, regs[P_FORCE_CTRL], &ctrl); + + if (data != speed) { + ret = ksz_pwrite8(dev, p, regs[P_SPEED_STATUS], data); + if (ret) + return ret; + } + + ret = ksz_pread8(dev, p, regs[P_FORCE_CTRL], &ctrl); + if (ret) + return ret; + data = ctrl; if (ksz_is_ksz88x3(dev)) { if ((val & BMCR_ANENABLE)) @@ -724,9 +766,17 @@ void ksz8_w_phy(struct ksz_device *dev, u16 phy, u16 reg, u16 val) data |= PORT_FORCE_FULL_DUPLEX; else data &= ~PORT_FORCE_FULL_DUPLEX; - if (data != ctrl) - ksz_pwrite8(dev, p, regs[P_FORCE_CTRL], data); - ksz_pread8(dev, p, regs[P_NEG_RESTART_CTRL], &restart); + + if (data != ctrl) { + ret = ksz_pwrite8(dev, p, regs[P_FORCE_CTRL], data); + if (ret) + return ret; + } + + ret = ksz_pread8(dev, p, regs[P_NEG_RESTART_CTRL], &restart); + if (ret) + return ret; + data = restart; if (val & KSZ886X_BMCR_DISABLE_LED) data |= PORT_LED_OFF; @@ -756,11 +806,19 @@ void ksz8_w_phy(struct ksz_device *dev, u16 phy, u16 reg, u16 val) data |= PORT_PHY_LOOPBACK; else data &= ~PORT_PHY_LOOPBACK; - if (data != restart) - ksz_pwrite8(dev, p, regs[P_NEG_RESTART_CTRL], data); + + if (data != restart) { + ret = ksz_pwrite8(dev, p, regs[P_NEG_RESTART_CTRL], + data); + if (ret) + return ret; + } break; case MII_ADVERTISE: - ksz_pread8(dev, p, regs[P_LOCAL_CTRL], &ctrl); + ret = ksz_pread8(dev, p, regs[P_LOCAL_CTRL], &ctrl); + if (ret) + return ret; + data = ctrl; data &= ~(PORT_AUTO_NEG_SYM_PAUSE | PORT_AUTO_NEG_100BTX_FD | @@ -777,8 +835,12 @@ void ksz8_w_phy(struct ksz_device *dev, u16 phy, u16 reg, u16 val) data |= PORT_AUTO_NEG_10BT_FD; if (val & ADVERTISE_10HALF) data |= PORT_AUTO_NEG_10BT; - if (data != ctrl) - ksz_pwrite8(dev, p, regs[P_LOCAL_CTRL], data); + + if (data != ctrl) { + ret = ksz_pwrite8(dev, p, regs[P_LOCAL_CTRL], data); + if (ret) + return ret; + } break; case PHY_REG_LINK_MD: if (val & PHY_START_CABLE_DIAG) @@ -787,6 +849,8 @@ void ksz8_w_phy(struct ksz_device *dev, u16 phy, u16 reg, u16 val) default: break; } + + return 0; } void ksz8_cfg_port_member(struct ksz_device *dev, int port, u8 member) @@ -1187,7 +1251,6 @@ void ksz8_config_cpu_port(struct dsa_switch *ds) if (i == dev->phy_port_cnt) break; p->on = 1; - p->phy = 1; } for (i = 0; i < dev->phy_port_cnt; i++) { p = &dev->ports[i]; diff --git a/drivers/net/dsa/microchip/ksz8863_smi.c b/drivers/net/dsa/microchip/ksz8863_smi.c index 5247fdfb964d..ddb40838181e 100644 --- a/drivers/net/dsa/microchip/ksz8863_smi.c +++ b/drivers/net/dsa/microchip/ksz8863_smi.c @@ -180,8 +180,6 @@ static void ksz8863_smi_remove(struct mdio_device *mdiodev) if (dev) ksz_switch_remove(dev); - - dev_set_drvdata(&mdiodev->dev, NULL); } static void ksz8863_smi_shutdown(struct mdio_device *mdiodev) diff --git a/drivers/net/dsa/microchip/ksz9477.c b/drivers/net/dsa/microchip/ksz9477.c index e4f446db0ca1..a6a0321a8931 100644 --- a/drivers/net/dsa/microchip/ksz9477.c +++ b/drivers/net/dsa/microchip/ksz9477.c @@ -193,6 +193,11 @@ int ksz9477_reset_switch(struct ksz_device *dev) ksz_write32(dev, REG_SW_PORT_INT_MASK__4, 0x7F); ksz_read32(dev, REG_SW_PORT_INT_STATUS__4, &data32); + /* KSZ9893 compatible chips do not support refclk configuration */ + if (dev->chip_id == KSZ9893_CHIP_ID || + dev->chip_id == KSZ8563_CHIP_ID) + return 0; + data8 = SW_ENABLE_REFCLKO; if (dev->synclko_disable) data8 = 0; @@ -264,9 +269,20 @@ void ksz9477_port_init_cnt(struct ksz_device *dev, int port) mutex_unlock(&mib->cnt_mutex); } -void ksz9477_r_phy(struct ksz_device *dev, u16 addr, u16 reg, u16 *data) +static void ksz9477_r_phy_quirks(struct ksz_device *dev, u16 addr, u16 reg, + u16 *data) +{ + /* KSZ8563R do not have extended registers but BMSR_ESTATEN and + * BMSR_ERCAP bits are set. + */ + if (dev->chip_id == KSZ8563_CHIP_ID && reg == MII_BMSR) + *data &= ~(BMSR_ESTATEN | BMSR_ERCAP); +} + +int ksz9477_r_phy(struct ksz_device *dev, u16 addr, u16 reg, u16 *data) { u16 val = 0xffff; + int ret; /* No real PHY after this. Simulate the PHY. * A fixed PHY can be setup in the device tree, but this function is @@ -274,7 +290,7 @@ void ksz9477_r_phy(struct ksz_device *dev, u16 addr, u16 reg, u16 *data) * For RGMII PHY there is no way to access it so the fixed PHY should * be used. For SGMII PHY the supporting code will be added later. */ - if (addr >= dev->phy_port_cnt) { + if (!dev->info->internal_phy[addr]) { struct ksz_port *p = &dev->ports[addr]; switch (reg) { @@ -307,23 +323,25 @@ void ksz9477_r_phy(struct ksz_device *dev, u16 addr, u16 reg, u16 *data) break; } } else { - ksz_pread16(dev, addr, 0x100 + (reg << 1), &val); + ret = ksz_pread16(dev, addr, 0x100 + (reg << 1), &val); + if (ret) + return ret; + + ksz9477_r_phy_quirks(dev, addr, reg, &val); } *data = val; + + return 0; } -void ksz9477_w_phy(struct ksz_device *dev, u16 addr, u16 reg, u16 val) +int ksz9477_w_phy(struct ksz_device *dev, u16 addr, u16 reg, u16 val) { /* No real PHY after this. */ - if (addr >= dev->phy_port_cnt) - return; + if (!dev->info->internal_phy[addr]) + return 0; - /* No gigabit support. Do not write to this register. */ - if (!(dev->features & GBIT_SUPPORT) && reg == MII_CTRL1000) - return; - - ksz_pwrite16(dev, addr, 0x100 + (reg << 1), val); + return ksz_pwrite16(dev, addr, 0x100 + (reg << 1), val); } void ksz9477_cfg_port_member(struct ksz_device *dev, int port, u8 member) @@ -869,7 +887,7 @@ static phy_interface_t ksz9477_get_interface(struct ksz_device *dev, int port) phy_interface_t interface; bool gbit; - if (port < dev->phy_port_cnt) + if (dev->info->internal_phy[port]) return PHY_INTERFACE_MODE_NA; gbit = ksz_get_gbit(dev, port); @@ -914,7 +932,7 @@ static void ksz9477_phy_errata_setup(struct ksz_device *dev, int port) /* Energy Efficient Ethernet (EEE) feature select must * be manually disabled (except on KSZ8565 which is 100Mbit) */ - if (dev->features & GBIT_SUPPORT) + if (dev->info->gbit_capable[port]) ksz9477_port_mmd_write(dev, port, 0x07, 0x3c, 0x0000); /* Register settings are required to meet data sheet @@ -941,10 +959,35 @@ void ksz9477_get_caps(struct ksz_device *dev, int port, config->mac_capabilities = MAC_10 | MAC_100 | MAC_ASYM_PAUSE | MAC_SYM_PAUSE; - if (dev->features & GBIT_SUPPORT) + if (dev->info->gbit_capable[port]) config->mac_capabilities |= MAC_1000FD; } +int ksz9477_set_ageing_time(struct ksz_device *dev, unsigned int msecs) +{ + u32 secs = msecs / 1000; + u8 value; + u8 data; + int ret; + + value = FIELD_GET(SW_AGE_PERIOD_7_0_M, secs); + + ret = ksz_write8(dev, REG_SW_LUE_CTRL_3, value); + if (ret < 0) + return ret; + + data = FIELD_GET(SW_AGE_PERIOD_10_8_M, secs); + + ret = ksz_read8(dev, REG_SW_LUE_CTRL_0, &value); + if (ret < 0) + return ret; + + value &= ~SW_AGE_CNT_M; + value |= FIELD_PREP(SW_AGE_CNT_M, data); + + return ksz_write8(dev, REG_SW_LUE_CTRL_0, value); +} + void ksz9477_port_setup(struct ksz_device *dev, int port, bool cpu_port) { struct dsa_switch *ds = dev->ds; @@ -976,7 +1019,7 @@ void ksz9477_port_setup(struct ksz_device *dev, int port, bool cpu_port) /* enable 802.1p priority */ ksz_port_cfg(dev, port, P_PRIO_CTRL, PORT_802_1P_PRIO_ENABLE, true); - if (port < dev->phy_port_cnt) { + if (dev->info->internal_phy[port]) { /* do not force flow control */ ksz_port_cfg(dev, port, REG_PORT_CTRL_0, PORT_FORCE_TX_FLOW_CTRL | PORT_FORCE_RX_FLOW_CTRL, @@ -999,7 +1042,7 @@ void ksz9477_port_setup(struct ksz_device *dev, int port, bool cpu_port) ksz9477_cfg_port_member(dev, port, member); /* clear pending interrupts */ - if (port < dev->phy_port_cnt) + if (dev->info->internal_phy[port]) ksz_pread16(dev, port, REG_PORT_PHY_INT_ENABLE, &data16); } @@ -1051,25 +1094,13 @@ void ksz9477_config_cpu_port(struct dsa_switch *ds) /* enable cpu port */ ksz9477_port_setup(dev, i, true); - p->on = 1; } } for (i = 0; i < dev->info->port_cnt; i++) { if (i == dev->cpu_port) continue; - p = &dev->ports[i]; - ksz_port_stp_state_set(ds, i, BR_STATE_DISABLED); - p->on = 1; - if (i < dev->phy_port_cnt) - p->phy = 1; - if (dev->chip_id == 0x00947700 && i == 6) { - p->sgmii = 1; - - /* SGMII PHY detection code is not implemented yet. */ - p->phy = 0; - } } } @@ -1158,29 +1189,6 @@ int ksz9477_switch_init(struct ksz_device *dev) if (ret) return ret; - ret = ksz_read8(dev, REG_GLOBAL_OPTIONS, &data8); - if (ret) - return ret; - - /* Number of ports can be reduced depending on chip. */ - dev->phy_port_cnt = 5; - - /* Default capability is gigabit capable. */ - dev->features = GBIT_SUPPORT; - - if (dev->chip_id == KSZ9893_CHIP_ID) { - dev->features |= IS_9893; - - /* Chip does not support gigabit. */ - if (data8 & SW_QW_ABLE) - dev->features &= ~GBIT_SUPPORT; - dev->phy_port_cnt = 2; - } else { - /* Chip does not support gigabit. */ - if (!(data8 & SW_GIGABIT_ABLE)) - dev->features &= ~GBIT_SUPPORT; - } - return 0; } diff --git a/drivers/net/dsa/microchip/ksz9477.h b/drivers/net/dsa/microchip/ksz9477.h index cd278b307b3c..00862c4cfb7f 100644 --- a/drivers/net/dsa/microchip/ksz9477.h +++ b/drivers/net/dsa/microchip/ksz9477.h @@ -16,8 +16,9 @@ u32 ksz9477_get_port_addr(int port, int offset); void ksz9477_cfg_port_member(struct ksz_device *dev, int port, u8 member); void ksz9477_flush_dyn_mac_table(struct ksz_device *dev, int port); void ksz9477_port_setup(struct ksz_device *dev, int port, bool cpu_port); -void ksz9477_r_phy(struct ksz_device *dev, u16 addr, u16 reg, u16 *data); -void ksz9477_w_phy(struct ksz_device *dev, u16 addr, u16 reg, u16 val); +int ksz9477_set_ageing_time(struct ksz_device *dev, unsigned int msecs); +int ksz9477_r_phy(struct ksz_device *dev, u16 addr, u16 reg, u16 *data); +int ksz9477_w_phy(struct ksz_device *dev, u16 addr, u16 reg, u16 val); void ksz9477_r_mib_cnt(struct ksz_device *dev, int port, u16 addr, u64 *cnt); void ksz9477_r_mib_pkt(struct ksz_device *dev, int port, u16 addr, u64 *dropped, u64 *cnt); diff --git a/drivers/net/dsa/microchip/ksz9477_i2c.c b/drivers/net/dsa/microchip/ksz9477_i2c.c index 99966514d444..e111756f6473 100644 --- a/drivers/net/dsa/microchip/ksz9477_i2c.c +++ b/drivers/net/dsa/microchip/ksz9477_i2c.c @@ -59,8 +59,6 @@ static int ksz9477_i2c_remove(struct i2c_client *i2c) if (dev) ksz_switch_remove(dev); - i2c_set_clientdata(i2c, NULL); - return 0; } @@ -92,6 +90,10 @@ static const struct of_device_id ksz9477_dt_ids[] = { .data = &ksz_switch_chips[KSZ9477] }, { + .compatible = "microchip,ksz9896", + .data = &ksz_switch_chips[KSZ9896] + }, + { .compatible = "microchip,ksz9897", .data = &ksz_switch_chips[KSZ9897] }, diff --git a/drivers/net/dsa/microchip/ksz9477_reg.h b/drivers/net/dsa/microchip/ksz9477_reg.h index ddf99d1e4bbd..53c68d286dd3 100644 --- a/drivers/net/dsa/microchip/ksz9477_reg.h +++ b/drivers/net/dsa/microchip/ksz9477_reg.h @@ -189,8 +189,9 @@ #define SW_VLAN_ENABLE BIT(7) #define SW_DROP_INVALID_VID BIT(6) -#define SW_AGE_CNT_M 0x7 +#define SW_AGE_CNT_M GENMASK(5, 3) #define SW_AGE_CNT_S 3 +#define SW_AGE_PERIOD_10_8_M GENMASK(10, 8) #define SW_RESV_MCAST_ENABLE BIT(2) #define SW_HASH_OPTION_M 0x03 #define SW_HASH_OPTION_CRC 1 @@ -225,6 +226,7 @@ #define SW_PRIO_LOWEST_DA_SA 3 #define REG_SW_LUE_CTRL_3 0x0313 +#define SW_AGE_PERIOD_7_0_M GENMASK(7, 0) #define REG_SW_LUE_INT_STATUS 0x0314 #define REG_SW_LUE_INT_ENABLE 0x0315 diff --git a/drivers/net/dsa/microchip/ksz_common.c b/drivers/net/dsa/microchip/ksz_common.c index 872aba63e7d4..d612181b3226 100644 --- a/drivers/net/dsa/microchip/ksz_common.c +++ b/drivers/net/dsa/microchip/ksz_common.c @@ -14,6 +14,9 @@ #include <linux/phy.h> #include <linux/etherdevice.h> #include <linux/if_bridge.h> +#include <linux/irq.h> +#include <linux/irqdomain.h> +#include <linux/of_mdio.h> #include <linux/of_device.h> #include <linux/of_net.h> #include <linux/micrel_phy.h> @@ -183,6 +186,7 @@ static const struct ksz_dev_ops ksz9477_dev_ops = { .cfg_port_member = ksz9477_cfg_port_member, .flush_dyn_mac_table = ksz9477_flush_dyn_mac_table, .port_setup = ksz9477_port_setup, + .set_ageing_time = ksz9477_set_ageing_time, .r_phy = ksz9477_r_phy, .w_phy = ksz9477_w_phy, .r_mib_cnt = ksz9477_r_mib_cnt, @@ -213,10 +217,12 @@ static const struct ksz_dev_ops ksz9477_dev_ops = { static const struct ksz_dev_ops lan937x_dev_ops = { .setup = lan937x_setup, + .teardown = lan937x_teardown, .get_port_addr = ksz9477_get_port_addr, .cfg_port_member = ksz9477_cfg_port_member, .flush_dyn_mac_table = ksz9477_flush_dyn_mac_table, .port_setup = lan937x_port_setup, + .set_ageing_time = lan937x_set_ageing_time, .r_phy = lan937x_r_phy, .w_phy = lan937x_w_phy, .r_mib_cnt = ksz9477_r_mib_cnt, @@ -421,7 +427,636 @@ static const u8 lan937x_shifts[] = { [ALU_STAT_INDEX] = 8, }; +static const struct regmap_range ksz8563_valid_regs[] = { + regmap_reg_range(0x0000, 0x0003), + regmap_reg_range(0x0006, 0x0006), + regmap_reg_range(0x000f, 0x001f), + regmap_reg_range(0x0100, 0x0100), + regmap_reg_range(0x0104, 0x0107), + regmap_reg_range(0x010d, 0x010d), + regmap_reg_range(0x0110, 0x0113), + regmap_reg_range(0x0120, 0x012b), + regmap_reg_range(0x0201, 0x0201), + regmap_reg_range(0x0210, 0x0213), + regmap_reg_range(0x0300, 0x0300), + regmap_reg_range(0x0302, 0x031b), + regmap_reg_range(0x0320, 0x032b), + regmap_reg_range(0x0330, 0x0336), + regmap_reg_range(0x0338, 0x033e), + regmap_reg_range(0x0340, 0x035f), + regmap_reg_range(0x0370, 0x0370), + regmap_reg_range(0x0378, 0x0378), + regmap_reg_range(0x037c, 0x037d), + regmap_reg_range(0x0390, 0x0393), + regmap_reg_range(0x0400, 0x040e), + regmap_reg_range(0x0410, 0x042f), + regmap_reg_range(0x0500, 0x0519), + regmap_reg_range(0x0520, 0x054b), + regmap_reg_range(0x0550, 0x05b3), + + /* port 1 */ + regmap_reg_range(0x1000, 0x1001), + regmap_reg_range(0x1004, 0x100b), + regmap_reg_range(0x1013, 0x1013), + regmap_reg_range(0x1017, 0x1017), + regmap_reg_range(0x101b, 0x101b), + regmap_reg_range(0x101f, 0x1021), + regmap_reg_range(0x1030, 0x1030), + regmap_reg_range(0x1100, 0x1111), + regmap_reg_range(0x111a, 0x111d), + regmap_reg_range(0x1122, 0x1127), + regmap_reg_range(0x112a, 0x112b), + regmap_reg_range(0x1136, 0x1139), + regmap_reg_range(0x113e, 0x113f), + regmap_reg_range(0x1400, 0x1401), + regmap_reg_range(0x1403, 0x1403), + regmap_reg_range(0x1410, 0x1417), + regmap_reg_range(0x1420, 0x1423), + regmap_reg_range(0x1500, 0x1507), + regmap_reg_range(0x1600, 0x1612), + regmap_reg_range(0x1800, 0x180f), + regmap_reg_range(0x1900, 0x1907), + regmap_reg_range(0x1914, 0x191b), + regmap_reg_range(0x1a00, 0x1a03), + regmap_reg_range(0x1a04, 0x1a08), + regmap_reg_range(0x1b00, 0x1b01), + regmap_reg_range(0x1b04, 0x1b04), + regmap_reg_range(0x1c00, 0x1c05), + regmap_reg_range(0x1c08, 0x1c1b), + + /* port 2 */ + regmap_reg_range(0x2000, 0x2001), + regmap_reg_range(0x2004, 0x200b), + regmap_reg_range(0x2013, 0x2013), + regmap_reg_range(0x2017, 0x2017), + regmap_reg_range(0x201b, 0x201b), + regmap_reg_range(0x201f, 0x2021), + regmap_reg_range(0x2030, 0x2030), + regmap_reg_range(0x2100, 0x2111), + regmap_reg_range(0x211a, 0x211d), + regmap_reg_range(0x2122, 0x2127), + regmap_reg_range(0x212a, 0x212b), + regmap_reg_range(0x2136, 0x2139), + regmap_reg_range(0x213e, 0x213f), + regmap_reg_range(0x2400, 0x2401), + regmap_reg_range(0x2403, 0x2403), + regmap_reg_range(0x2410, 0x2417), + regmap_reg_range(0x2420, 0x2423), + regmap_reg_range(0x2500, 0x2507), + regmap_reg_range(0x2600, 0x2612), + regmap_reg_range(0x2800, 0x280f), + regmap_reg_range(0x2900, 0x2907), + regmap_reg_range(0x2914, 0x291b), + regmap_reg_range(0x2a00, 0x2a03), + regmap_reg_range(0x2a04, 0x2a08), + regmap_reg_range(0x2b00, 0x2b01), + regmap_reg_range(0x2b04, 0x2b04), + regmap_reg_range(0x2c00, 0x2c05), + regmap_reg_range(0x2c08, 0x2c1b), + + /* port 3 */ + regmap_reg_range(0x3000, 0x3001), + regmap_reg_range(0x3004, 0x300b), + regmap_reg_range(0x3013, 0x3013), + regmap_reg_range(0x3017, 0x3017), + regmap_reg_range(0x301b, 0x301b), + regmap_reg_range(0x301f, 0x3021), + regmap_reg_range(0x3030, 0x3030), + regmap_reg_range(0x3300, 0x3301), + regmap_reg_range(0x3303, 0x3303), + regmap_reg_range(0x3400, 0x3401), + regmap_reg_range(0x3403, 0x3403), + regmap_reg_range(0x3410, 0x3417), + regmap_reg_range(0x3420, 0x3423), + regmap_reg_range(0x3500, 0x3507), + regmap_reg_range(0x3600, 0x3612), + regmap_reg_range(0x3800, 0x380f), + regmap_reg_range(0x3900, 0x3907), + regmap_reg_range(0x3914, 0x391b), + regmap_reg_range(0x3a00, 0x3a03), + regmap_reg_range(0x3a04, 0x3a08), + regmap_reg_range(0x3b00, 0x3b01), + regmap_reg_range(0x3b04, 0x3b04), + regmap_reg_range(0x3c00, 0x3c05), + regmap_reg_range(0x3c08, 0x3c1b), +}; + +static const struct regmap_access_table ksz8563_register_set = { + .yes_ranges = ksz8563_valid_regs, + .n_yes_ranges = ARRAY_SIZE(ksz8563_valid_regs), +}; + +static const struct regmap_range ksz9477_valid_regs[] = { + regmap_reg_range(0x0000, 0x0003), + regmap_reg_range(0x0006, 0x0006), + regmap_reg_range(0x0010, 0x001f), + regmap_reg_range(0x0100, 0x0100), + regmap_reg_range(0x0103, 0x0107), + regmap_reg_range(0x010d, 0x010d), + regmap_reg_range(0x0110, 0x0113), + regmap_reg_range(0x0120, 0x012b), + regmap_reg_range(0x0201, 0x0201), + regmap_reg_range(0x0210, 0x0213), + regmap_reg_range(0x0300, 0x0300), + regmap_reg_range(0x0302, 0x031b), + regmap_reg_range(0x0320, 0x032b), + regmap_reg_range(0x0330, 0x0336), + regmap_reg_range(0x0338, 0x033b), + regmap_reg_range(0x033e, 0x033e), + regmap_reg_range(0x0340, 0x035f), + regmap_reg_range(0x0370, 0x0370), + regmap_reg_range(0x0378, 0x0378), + regmap_reg_range(0x037c, 0x037d), + regmap_reg_range(0x0390, 0x0393), + regmap_reg_range(0x0400, 0x040e), + regmap_reg_range(0x0410, 0x042f), + regmap_reg_range(0x0444, 0x044b), + regmap_reg_range(0x0450, 0x046f), + regmap_reg_range(0x0500, 0x0519), + regmap_reg_range(0x0520, 0x054b), + regmap_reg_range(0x0550, 0x05b3), + regmap_reg_range(0x0604, 0x060b), + regmap_reg_range(0x0610, 0x0612), + regmap_reg_range(0x0614, 0x062c), + regmap_reg_range(0x0640, 0x0645), + regmap_reg_range(0x0648, 0x064d), + + /* port 1 */ + regmap_reg_range(0x1000, 0x1001), + regmap_reg_range(0x1013, 0x1013), + regmap_reg_range(0x1017, 0x1017), + regmap_reg_range(0x101b, 0x101b), + regmap_reg_range(0x101f, 0x1020), + regmap_reg_range(0x1030, 0x1030), + regmap_reg_range(0x1100, 0x1115), + regmap_reg_range(0x111a, 0x111f), + regmap_reg_range(0x1122, 0x1127), + regmap_reg_range(0x112a, 0x112b), + regmap_reg_range(0x1136, 0x1139), + regmap_reg_range(0x113e, 0x113f), + regmap_reg_range(0x1400, 0x1401), + regmap_reg_range(0x1403, 0x1403), + regmap_reg_range(0x1410, 0x1417), + regmap_reg_range(0x1420, 0x1423), + regmap_reg_range(0x1500, 0x1507), + regmap_reg_range(0x1600, 0x1613), + regmap_reg_range(0x1800, 0x180f), + regmap_reg_range(0x1820, 0x1827), + regmap_reg_range(0x1830, 0x1837), + regmap_reg_range(0x1840, 0x184b), + regmap_reg_range(0x1900, 0x1907), + regmap_reg_range(0x1914, 0x191b), + regmap_reg_range(0x1920, 0x1920), + regmap_reg_range(0x1923, 0x1927), + regmap_reg_range(0x1a00, 0x1a03), + regmap_reg_range(0x1a04, 0x1a07), + regmap_reg_range(0x1b00, 0x1b01), + regmap_reg_range(0x1b04, 0x1b04), + regmap_reg_range(0x1c00, 0x1c05), + regmap_reg_range(0x1c08, 0x1c1b), + + /* port 2 */ + regmap_reg_range(0x2000, 0x2001), + regmap_reg_range(0x2013, 0x2013), + regmap_reg_range(0x2017, 0x2017), + regmap_reg_range(0x201b, 0x201b), + regmap_reg_range(0x201f, 0x2020), + regmap_reg_range(0x2030, 0x2030), + regmap_reg_range(0x2100, 0x2115), + regmap_reg_range(0x211a, 0x211f), + regmap_reg_range(0x2122, 0x2127), + regmap_reg_range(0x212a, 0x212b), + regmap_reg_range(0x2136, 0x2139), + regmap_reg_range(0x213e, 0x213f), + regmap_reg_range(0x2400, 0x2401), + regmap_reg_range(0x2403, 0x2403), + regmap_reg_range(0x2410, 0x2417), + regmap_reg_range(0x2420, 0x2423), + regmap_reg_range(0x2500, 0x2507), + regmap_reg_range(0x2600, 0x2613), + regmap_reg_range(0x2800, 0x280f), + regmap_reg_range(0x2820, 0x2827), + regmap_reg_range(0x2830, 0x2837), + regmap_reg_range(0x2840, 0x284b), + regmap_reg_range(0x2900, 0x2907), + regmap_reg_range(0x2914, 0x291b), + regmap_reg_range(0x2920, 0x2920), + regmap_reg_range(0x2923, 0x2927), + regmap_reg_range(0x2a00, 0x2a03), + regmap_reg_range(0x2a04, 0x2a07), + regmap_reg_range(0x2b00, 0x2b01), + regmap_reg_range(0x2b04, 0x2b04), + regmap_reg_range(0x2c00, 0x2c05), + regmap_reg_range(0x2c08, 0x2c1b), + + /* port 3 */ + regmap_reg_range(0x3000, 0x3001), + regmap_reg_range(0x3013, 0x3013), + regmap_reg_range(0x3017, 0x3017), + regmap_reg_range(0x301b, 0x301b), + regmap_reg_range(0x301f, 0x3020), + regmap_reg_range(0x3030, 0x3030), + regmap_reg_range(0x3100, 0x3115), + regmap_reg_range(0x311a, 0x311f), + regmap_reg_range(0x3122, 0x3127), + regmap_reg_range(0x312a, 0x312b), + regmap_reg_range(0x3136, 0x3139), + regmap_reg_range(0x313e, 0x313f), + regmap_reg_range(0x3400, 0x3401), + regmap_reg_range(0x3403, 0x3403), + regmap_reg_range(0x3410, 0x3417), + regmap_reg_range(0x3420, 0x3423), + regmap_reg_range(0x3500, 0x3507), + regmap_reg_range(0x3600, 0x3613), + regmap_reg_range(0x3800, 0x380f), + regmap_reg_range(0x3820, 0x3827), + regmap_reg_range(0x3830, 0x3837), + regmap_reg_range(0x3840, 0x384b), + regmap_reg_range(0x3900, 0x3907), + regmap_reg_range(0x3914, 0x391b), + regmap_reg_range(0x3920, 0x3920), + regmap_reg_range(0x3923, 0x3927), + regmap_reg_range(0x3a00, 0x3a03), + regmap_reg_range(0x3a04, 0x3a07), + regmap_reg_range(0x3b00, 0x3b01), + regmap_reg_range(0x3b04, 0x3b04), + regmap_reg_range(0x3c00, 0x3c05), + regmap_reg_range(0x3c08, 0x3c1b), + + /* port 4 */ + regmap_reg_range(0x4000, 0x4001), + regmap_reg_range(0x4013, 0x4013), + regmap_reg_range(0x4017, 0x4017), + regmap_reg_range(0x401b, 0x401b), + regmap_reg_range(0x401f, 0x4020), + regmap_reg_range(0x4030, 0x4030), + regmap_reg_range(0x4100, 0x4115), + regmap_reg_range(0x411a, 0x411f), + regmap_reg_range(0x4122, 0x4127), + regmap_reg_range(0x412a, 0x412b), + regmap_reg_range(0x4136, 0x4139), + regmap_reg_range(0x413e, 0x413f), + regmap_reg_range(0x4400, 0x4401), + regmap_reg_range(0x4403, 0x4403), + regmap_reg_range(0x4410, 0x4417), + regmap_reg_range(0x4420, 0x4423), + regmap_reg_range(0x4500, 0x4507), + regmap_reg_range(0x4600, 0x4613), + regmap_reg_range(0x4800, 0x480f), + regmap_reg_range(0x4820, 0x4827), + regmap_reg_range(0x4830, 0x4837), + regmap_reg_range(0x4840, 0x484b), + regmap_reg_range(0x4900, 0x4907), + regmap_reg_range(0x4914, 0x491b), + regmap_reg_range(0x4920, 0x4920), + regmap_reg_range(0x4923, 0x4927), + regmap_reg_range(0x4a00, 0x4a03), + regmap_reg_range(0x4a04, 0x4a07), + regmap_reg_range(0x4b00, 0x4b01), + regmap_reg_range(0x4b04, 0x4b04), + regmap_reg_range(0x4c00, 0x4c05), + regmap_reg_range(0x4c08, 0x4c1b), + + /* port 5 */ + regmap_reg_range(0x5000, 0x5001), + regmap_reg_range(0x5013, 0x5013), + regmap_reg_range(0x5017, 0x5017), + regmap_reg_range(0x501b, 0x501b), + regmap_reg_range(0x501f, 0x5020), + regmap_reg_range(0x5030, 0x5030), + regmap_reg_range(0x5100, 0x5115), + regmap_reg_range(0x511a, 0x511f), + regmap_reg_range(0x5122, 0x5127), + regmap_reg_range(0x512a, 0x512b), + regmap_reg_range(0x5136, 0x5139), + regmap_reg_range(0x513e, 0x513f), + regmap_reg_range(0x5400, 0x5401), + regmap_reg_range(0x5403, 0x5403), + regmap_reg_range(0x5410, 0x5417), + regmap_reg_range(0x5420, 0x5423), + regmap_reg_range(0x5500, 0x5507), + regmap_reg_range(0x5600, 0x5613), + regmap_reg_range(0x5800, 0x580f), + regmap_reg_range(0x5820, 0x5827), + regmap_reg_range(0x5830, 0x5837), + regmap_reg_range(0x5840, 0x584b), + regmap_reg_range(0x5900, 0x5907), + regmap_reg_range(0x5914, 0x591b), + regmap_reg_range(0x5920, 0x5920), + regmap_reg_range(0x5923, 0x5927), + regmap_reg_range(0x5a00, 0x5a03), + regmap_reg_range(0x5a04, 0x5a07), + regmap_reg_range(0x5b00, 0x5b01), + regmap_reg_range(0x5b04, 0x5b04), + regmap_reg_range(0x5c00, 0x5c05), + regmap_reg_range(0x5c08, 0x5c1b), + + /* port 6 */ + regmap_reg_range(0x6000, 0x6001), + regmap_reg_range(0x6013, 0x6013), + regmap_reg_range(0x6017, 0x6017), + regmap_reg_range(0x601b, 0x601b), + regmap_reg_range(0x601f, 0x6020), + regmap_reg_range(0x6030, 0x6030), + regmap_reg_range(0x6300, 0x6301), + regmap_reg_range(0x6400, 0x6401), + regmap_reg_range(0x6403, 0x6403), + regmap_reg_range(0x6410, 0x6417), + regmap_reg_range(0x6420, 0x6423), + regmap_reg_range(0x6500, 0x6507), + regmap_reg_range(0x6600, 0x6613), + regmap_reg_range(0x6800, 0x680f), + regmap_reg_range(0x6820, 0x6827), + regmap_reg_range(0x6830, 0x6837), + regmap_reg_range(0x6840, 0x684b), + regmap_reg_range(0x6900, 0x6907), + regmap_reg_range(0x6914, 0x691b), + regmap_reg_range(0x6920, 0x6920), + regmap_reg_range(0x6923, 0x6927), + regmap_reg_range(0x6a00, 0x6a03), + regmap_reg_range(0x6a04, 0x6a07), + regmap_reg_range(0x6b00, 0x6b01), + regmap_reg_range(0x6b04, 0x6b04), + regmap_reg_range(0x6c00, 0x6c05), + regmap_reg_range(0x6c08, 0x6c1b), + + /* port 7 */ + regmap_reg_range(0x7000, 0x7001), + regmap_reg_range(0x7013, 0x7013), + regmap_reg_range(0x7017, 0x7017), + regmap_reg_range(0x701b, 0x701b), + regmap_reg_range(0x701f, 0x7020), + regmap_reg_range(0x7030, 0x7030), + regmap_reg_range(0x7200, 0x7203), + regmap_reg_range(0x7206, 0x7207), + regmap_reg_range(0x7300, 0x7301), + regmap_reg_range(0x7400, 0x7401), + regmap_reg_range(0x7403, 0x7403), + regmap_reg_range(0x7410, 0x7417), + regmap_reg_range(0x7420, 0x7423), + regmap_reg_range(0x7500, 0x7507), + regmap_reg_range(0x7600, 0x7613), + regmap_reg_range(0x7800, 0x780f), + regmap_reg_range(0x7820, 0x7827), + regmap_reg_range(0x7830, 0x7837), + regmap_reg_range(0x7840, 0x784b), + regmap_reg_range(0x7900, 0x7907), + regmap_reg_range(0x7914, 0x791b), + regmap_reg_range(0x7920, 0x7920), + regmap_reg_range(0x7923, 0x7927), + regmap_reg_range(0x7a00, 0x7a03), + regmap_reg_range(0x7a04, 0x7a07), + regmap_reg_range(0x7b00, 0x7b01), + regmap_reg_range(0x7b04, 0x7b04), + regmap_reg_range(0x7c00, 0x7c05), + regmap_reg_range(0x7c08, 0x7c1b), +}; + +static const struct regmap_access_table ksz9477_register_set = { + .yes_ranges = ksz9477_valid_regs, + .n_yes_ranges = ARRAY_SIZE(ksz9477_valid_regs), +}; + +static const struct regmap_range ksz9896_valid_regs[] = { + regmap_reg_range(0x0000, 0x0003), + regmap_reg_range(0x0006, 0x0006), + regmap_reg_range(0x0010, 0x001f), + regmap_reg_range(0x0100, 0x0100), + regmap_reg_range(0x0103, 0x0107), + regmap_reg_range(0x010d, 0x010d), + regmap_reg_range(0x0110, 0x0113), + regmap_reg_range(0x0120, 0x0127), + regmap_reg_range(0x0201, 0x0201), + regmap_reg_range(0x0210, 0x0213), + regmap_reg_range(0x0300, 0x0300), + regmap_reg_range(0x0302, 0x030b), + regmap_reg_range(0x0310, 0x031b), + regmap_reg_range(0x0320, 0x032b), + regmap_reg_range(0x0330, 0x0336), + regmap_reg_range(0x0338, 0x033b), + regmap_reg_range(0x033e, 0x033e), + regmap_reg_range(0x0340, 0x035f), + regmap_reg_range(0x0370, 0x0370), + regmap_reg_range(0x0378, 0x0378), + regmap_reg_range(0x037c, 0x037d), + regmap_reg_range(0x0390, 0x0393), + regmap_reg_range(0x0400, 0x040e), + regmap_reg_range(0x0410, 0x042f), + + /* port 1 */ + regmap_reg_range(0x1000, 0x1001), + regmap_reg_range(0x1013, 0x1013), + regmap_reg_range(0x1017, 0x1017), + regmap_reg_range(0x101b, 0x101b), + regmap_reg_range(0x101f, 0x1020), + regmap_reg_range(0x1030, 0x1030), + regmap_reg_range(0x1100, 0x1115), + regmap_reg_range(0x111a, 0x111f), + regmap_reg_range(0x1122, 0x1127), + regmap_reg_range(0x112a, 0x112b), + regmap_reg_range(0x1136, 0x1139), + regmap_reg_range(0x113e, 0x113f), + regmap_reg_range(0x1400, 0x1401), + regmap_reg_range(0x1403, 0x1403), + regmap_reg_range(0x1410, 0x1417), + regmap_reg_range(0x1420, 0x1423), + regmap_reg_range(0x1500, 0x1507), + regmap_reg_range(0x1600, 0x1612), + regmap_reg_range(0x1800, 0x180f), + regmap_reg_range(0x1820, 0x1827), + regmap_reg_range(0x1830, 0x1837), + regmap_reg_range(0x1840, 0x184b), + regmap_reg_range(0x1900, 0x1907), + regmap_reg_range(0x1914, 0x1915), + regmap_reg_range(0x1a00, 0x1a03), + regmap_reg_range(0x1a04, 0x1a07), + regmap_reg_range(0x1b00, 0x1b01), + regmap_reg_range(0x1b04, 0x1b04), + + /* port 2 */ + regmap_reg_range(0x2000, 0x2001), + regmap_reg_range(0x2013, 0x2013), + regmap_reg_range(0x2017, 0x2017), + regmap_reg_range(0x201b, 0x201b), + regmap_reg_range(0x201f, 0x2020), + regmap_reg_range(0x2030, 0x2030), + regmap_reg_range(0x2100, 0x2115), + regmap_reg_range(0x211a, 0x211f), + regmap_reg_range(0x2122, 0x2127), + regmap_reg_range(0x212a, 0x212b), + regmap_reg_range(0x2136, 0x2139), + regmap_reg_range(0x213e, 0x213f), + regmap_reg_range(0x2400, 0x2401), + regmap_reg_range(0x2403, 0x2403), + regmap_reg_range(0x2410, 0x2417), + regmap_reg_range(0x2420, 0x2423), + regmap_reg_range(0x2500, 0x2507), + regmap_reg_range(0x2600, 0x2612), + regmap_reg_range(0x2800, 0x280f), + regmap_reg_range(0x2820, 0x2827), + regmap_reg_range(0x2830, 0x2837), + regmap_reg_range(0x2840, 0x284b), + regmap_reg_range(0x2900, 0x2907), + regmap_reg_range(0x2914, 0x2915), + regmap_reg_range(0x2a00, 0x2a03), + regmap_reg_range(0x2a04, 0x2a07), + regmap_reg_range(0x2b00, 0x2b01), + regmap_reg_range(0x2b04, 0x2b04), + + /* port 3 */ + regmap_reg_range(0x3000, 0x3001), + regmap_reg_range(0x3013, 0x3013), + regmap_reg_range(0x3017, 0x3017), + regmap_reg_range(0x301b, 0x301b), + regmap_reg_range(0x301f, 0x3020), + regmap_reg_range(0x3030, 0x3030), + regmap_reg_range(0x3100, 0x3115), + regmap_reg_range(0x311a, 0x311f), + regmap_reg_range(0x3122, 0x3127), + regmap_reg_range(0x312a, 0x312b), + regmap_reg_range(0x3136, 0x3139), + regmap_reg_range(0x313e, 0x313f), + regmap_reg_range(0x3400, 0x3401), + regmap_reg_range(0x3403, 0x3403), + regmap_reg_range(0x3410, 0x3417), + regmap_reg_range(0x3420, 0x3423), + regmap_reg_range(0x3500, 0x3507), + regmap_reg_range(0x3600, 0x3612), + regmap_reg_range(0x3800, 0x380f), + regmap_reg_range(0x3820, 0x3827), + regmap_reg_range(0x3830, 0x3837), + regmap_reg_range(0x3840, 0x384b), + regmap_reg_range(0x3900, 0x3907), + regmap_reg_range(0x3914, 0x3915), + regmap_reg_range(0x3a00, 0x3a03), + regmap_reg_range(0x3a04, 0x3a07), + regmap_reg_range(0x3b00, 0x3b01), + regmap_reg_range(0x3b04, 0x3b04), + + /* port 4 */ + regmap_reg_range(0x4000, 0x4001), + regmap_reg_range(0x4013, 0x4013), + regmap_reg_range(0x4017, 0x4017), + regmap_reg_range(0x401b, 0x401b), + regmap_reg_range(0x401f, 0x4020), + regmap_reg_range(0x4030, 0x4030), + regmap_reg_range(0x4100, 0x4115), + regmap_reg_range(0x411a, 0x411f), + regmap_reg_range(0x4122, 0x4127), + regmap_reg_range(0x412a, 0x412b), + regmap_reg_range(0x4136, 0x4139), + regmap_reg_range(0x413e, 0x413f), + regmap_reg_range(0x4400, 0x4401), + regmap_reg_range(0x4403, 0x4403), + regmap_reg_range(0x4410, 0x4417), + regmap_reg_range(0x4420, 0x4423), + regmap_reg_range(0x4500, 0x4507), + regmap_reg_range(0x4600, 0x4612), + regmap_reg_range(0x4800, 0x480f), + regmap_reg_range(0x4820, 0x4827), + regmap_reg_range(0x4830, 0x4837), + regmap_reg_range(0x4840, 0x484b), + regmap_reg_range(0x4900, 0x4907), + regmap_reg_range(0x4914, 0x4915), + regmap_reg_range(0x4a00, 0x4a03), + regmap_reg_range(0x4a04, 0x4a07), + regmap_reg_range(0x4b00, 0x4b01), + regmap_reg_range(0x4b04, 0x4b04), + + /* port 5 */ + regmap_reg_range(0x5000, 0x5001), + regmap_reg_range(0x5013, 0x5013), + regmap_reg_range(0x5017, 0x5017), + regmap_reg_range(0x501b, 0x501b), + regmap_reg_range(0x501f, 0x5020), + regmap_reg_range(0x5030, 0x5030), + regmap_reg_range(0x5100, 0x5115), + regmap_reg_range(0x511a, 0x511f), + regmap_reg_range(0x5122, 0x5127), + regmap_reg_range(0x512a, 0x512b), + regmap_reg_range(0x5136, 0x5139), + regmap_reg_range(0x513e, 0x513f), + regmap_reg_range(0x5400, 0x5401), + regmap_reg_range(0x5403, 0x5403), + regmap_reg_range(0x5410, 0x5417), + regmap_reg_range(0x5420, 0x5423), + regmap_reg_range(0x5500, 0x5507), + regmap_reg_range(0x5600, 0x5612), + regmap_reg_range(0x5800, 0x580f), + regmap_reg_range(0x5820, 0x5827), + regmap_reg_range(0x5830, 0x5837), + regmap_reg_range(0x5840, 0x584b), + regmap_reg_range(0x5900, 0x5907), + regmap_reg_range(0x5914, 0x5915), + regmap_reg_range(0x5a00, 0x5a03), + regmap_reg_range(0x5a04, 0x5a07), + regmap_reg_range(0x5b00, 0x5b01), + regmap_reg_range(0x5b04, 0x5b04), + + /* port 6 */ + regmap_reg_range(0x6000, 0x6001), + regmap_reg_range(0x6013, 0x6013), + regmap_reg_range(0x6017, 0x6017), + regmap_reg_range(0x601b, 0x601b), + regmap_reg_range(0x601f, 0x6020), + regmap_reg_range(0x6030, 0x6030), + regmap_reg_range(0x6100, 0x6115), + regmap_reg_range(0x611a, 0x611f), + regmap_reg_range(0x6122, 0x6127), + regmap_reg_range(0x612a, 0x612b), + regmap_reg_range(0x6136, 0x6139), + regmap_reg_range(0x613e, 0x613f), + regmap_reg_range(0x6300, 0x6301), + regmap_reg_range(0x6400, 0x6401), + regmap_reg_range(0x6403, 0x6403), + regmap_reg_range(0x6410, 0x6417), + regmap_reg_range(0x6420, 0x6423), + regmap_reg_range(0x6500, 0x6507), + regmap_reg_range(0x6600, 0x6612), + regmap_reg_range(0x6800, 0x680f), + regmap_reg_range(0x6820, 0x6827), + regmap_reg_range(0x6830, 0x6837), + regmap_reg_range(0x6840, 0x684b), + regmap_reg_range(0x6900, 0x6907), + regmap_reg_range(0x6914, 0x6915), + regmap_reg_range(0x6a00, 0x6a03), + regmap_reg_range(0x6a04, 0x6a07), + regmap_reg_range(0x6b00, 0x6b01), + regmap_reg_range(0x6b04, 0x6b04), +}; + +static const struct regmap_access_table ksz9896_register_set = { + .yes_ranges = ksz9896_valid_regs, + .n_yes_ranges = ARRAY_SIZE(ksz9896_valid_regs), +}; + const struct ksz_chip_data ksz_switch_chips[] = { + [KSZ8563] = { + .chip_id = KSZ8563_CHIP_ID, + .dev_name = "KSZ8563", + .num_vlans = 4096, + .num_alus = 4096, + .num_statics = 16, + .cpu_ports = 0x07, /* can be configured as cpu port */ + .port_cnt = 3, /* total port count */ + .ops = &ksz9477_dev_ops, + .mib_names = ksz9477_mib_names, + .mib_cnt = ARRAY_SIZE(ksz9477_mib_names), + .reg_mib_cnt = MIB_COUNTER_NUM, + .regs = ksz9477_regs, + .masks = ksz9477_masks, + .shifts = ksz9477_shifts, + .xmii_ctrl0 = ksz9477_xmii_ctrl0, + .xmii_ctrl1 = ksz8795_xmii_ctrl1, /* Same as ksz8795 */ + .supports_mii = {false, false, true}, + .supports_rmii = {false, false, true}, + .supports_rgmii = {false, false, true}, + .internal_phy = {true, true, false}, + .gbit_capable = {false, false, true}, + .wr_table = &ksz8563_register_set, + .rd_table = &ksz8563_register_set, + }, + [KSZ8795] = { .chip_id = KSZ8795_CHIP_ID, .dev_name = "KSZ8795", @@ -536,6 +1171,7 @@ const struct ksz_chip_data ksz_switch_chips[] = { .num_statics = 16, .cpu_ports = 0x7F, /* can be configured as cpu port */ .port_cnt = 7, /* total physical port count */ + .port_nirqs = 4, .ops = &ksz9477_dev_ops, .phy_errata_9477 = true, .mib_names = ksz9477_mib_names, @@ -554,6 +1190,41 @@ const struct ksz_chip_data ksz_switch_chips[] = { false, true, false}, .internal_phy = {true, true, true, true, true, false, false}, + .gbit_capable = {true, true, true, true, true, true, true}, + .wr_table = &ksz9477_register_set, + .rd_table = &ksz9477_register_set, + }, + + [KSZ9896] = { + .chip_id = KSZ9896_CHIP_ID, + .dev_name = "KSZ9896", + .num_vlans = 4096, + .num_alus = 4096, + .num_statics = 16, + .cpu_ports = 0x3F, /* can be configured as cpu port */ + .port_cnt = 6, /* total physical port count */ + .port_nirqs = 2, + .ops = &ksz9477_dev_ops, + .phy_errata_9477 = true, + .mib_names = ksz9477_mib_names, + .mib_cnt = ARRAY_SIZE(ksz9477_mib_names), + .reg_mib_cnt = MIB_COUNTER_NUM, + .regs = ksz9477_regs, + .masks = ksz9477_masks, + .shifts = ksz9477_shifts, + .xmii_ctrl0 = ksz9477_xmii_ctrl0, + .xmii_ctrl1 = ksz9477_xmii_ctrl1, + .supports_mii = {false, false, false, false, + false, true}, + .supports_rmii = {false, false, false, false, + false, true}, + .supports_rgmii = {false, false, false, false, + false, true}, + .internal_phy = {true, true, true, true, + true, false}, + .gbit_capable = {true, true, true, true, true, true}, + .wr_table = &ksz9896_register_set, + .rd_table = &ksz9896_register_set, }, [KSZ9897] = { @@ -564,6 +1235,7 @@ const struct ksz_chip_data ksz_switch_chips[] = { .num_statics = 16, .cpu_ports = 0x7F, /* can be configured as cpu port */ .port_cnt = 7, /* total physical port count */ + .port_nirqs = 2, .ops = &ksz9477_dev_ops, .phy_errata_9477 = true, .mib_names = ksz9477_mib_names, @@ -582,6 +1254,7 @@ const struct ksz_chip_data ksz_switch_chips[] = { false, true, true}, .internal_phy = {true, true, true, true, true, false, false}, + .gbit_capable = {true, true, true, true, true, true, true}, }, [KSZ9893] = { @@ -592,6 +1265,7 @@ const struct ksz_chip_data ksz_switch_chips[] = { .num_statics = 16, .cpu_ports = 0x07, /* can be configured as cpu port */ .port_cnt = 3, /* total port count */ + .port_nirqs = 2, .ops = &ksz9477_dev_ops, .mib_names = ksz9477_mib_names, .mib_cnt = ARRAY_SIZE(ksz9477_mib_names), @@ -605,6 +1279,7 @@ const struct ksz_chip_data ksz_switch_chips[] = { .supports_rmii = {false, false, true}, .supports_rgmii = {false, false, true}, .internal_phy = {true, true, false}, + .gbit_capable = {true, true, true}, }, [KSZ9567] = { @@ -615,6 +1290,7 @@ const struct ksz_chip_data ksz_switch_chips[] = { .num_statics = 16, .cpu_ports = 0x7F, /* can be configured as cpu port */ .port_cnt = 7, /* total physical port count */ + .port_nirqs = 3, .ops = &ksz9477_dev_ops, .phy_errata_9477 = true, .mib_names = ksz9477_mib_names, @@ -633,6 +1309,7 @@ const struct ksz_chip_data ksz_switch_chips[] = { false, true, true}, .internal_phy = {true, true, true, true, true, false, false}, + .gbit_capable = {true, true, true, true, true, true, true}, }, [LAN9370] = { @@ -643,6 +1320,7 @@ const struct ksz_chip_data ksz_switch_chips[] = { .num_statics = 256, .cpu_ports = 0x10, /* can be configured as cpu port */ .port_cnt = 5, /* total physical port count */ + .port_nirqs = 6, .ops = &lan937x_dev_ops, .mib_names = ksz9477_mib_names, .mib_cnt = ARRAY_SIZE(ksz9477_mib_names), @@ -666,6 +1344,7 @@ const struct ksz_chip_data ksz_switch_chips[] = { .num_statics = 256, .cpu_ports = 0x30, /* can be configured as cpu port */ .port_cnt = 6, /* total physical port count */ + .port_nirqs = 6, .ops = &lan937x_dev_ops, .mib_names = ksz9477_mib_names, .mib_cnt = ARRAY_SIZE(ksz9477_mib_names), @@ -689,6 +1368,7 @@ const struct ksz_chip_data ksz_switch_chips[] = { .num_statics = 256, .cpu_ports = 0x30, /* can be configured as cpu port */ .port_cnt = 8, /* total physical port count */ + .port_nirqs = 6, .ops = &lan937x_dev_ops, .mib_names = ksz9477_mib_names, .mib_cnt = ARRAY_SIZE(ksz9477_mib_names), @@ -716,6 +1396,7 @@ const struct ksz_chip_data ksz_switch_chips[] = { .num_statics = 256, .cpu_ports = 0x38, /* can be configured as cpu port */ .port_cnt = 5, /* total physical port count */ + .port_nirqs = 6, .ops = &lan937x_dev_ops, .mib_names = ksz9477_mib_names, .mib_cnt = ARRAY_SIZE(ksz9477_mib_names), @@ -743,6 +1424,7 @@ const struct ksz_chip_data ksz_switch_chips[] = { .num_statics = 256, .cpu_ports = 0x30, /* can be configured as cpu port */ .port_cnt = 8, /* total physical port count */ + .port_nirqs = 6, .ops = &lan937x_dev_ops, .mib_names = ksz9477_mib_names, .mib_cnt = ARRAY_SIZE(ksz9477_mib_names), @@ -974,9 +1656,280 @@ static void ksz_update_port_member(struct ksz_device *dev, int port) dev->dev_ops->cfg_port_member(dev, port, port_member | cpu_port); } +static int ksz_sw_mdio_read(struct mii_bus *bus, int addr, int regnum) +{ + struct ksz_device *dev = bus->priv; + u16 val; + int ret; + + if (regnum & MII_ADDR_C45) + return -EOPNOTSUPP; + + ret = dev->dev_ops->r_phy(dev, addr, regnum, &val); + if (ret < 0) + return ret; + + return val; +} + +static int ksz_sw_mdio_write(struct mii_bus *bus, int addr, int regnum, + u16 val) +{ + struct ksz_device *dev = bus->priv; + + if (regnum & MII_ADDR_C45) + return -EOPNOTSUPP; + + return dev->dev_ops->w_phy(dev, addr, regnum, val); +} + +static int ksz_irq_phy_setup(struct ksz_device *dev) +{ + struct dsa_switch *ds = dev->ds; + int phy; + int irq; + int ret; + + for (phy = 0; phy < KSZ_MAX_NUM_PORTS; phy++) { + if (BIT(phy) & ds->phys_mii_mask) { + irq = irq_find_mapping(dev->ports[phy].pirq.domain, + PORT_SRC_PHY_INT); + if (irq < 0) { + ret = irq; + goto out; + } + ds->slave_mii_bus->irq[phy] = irq; + } + } + return 0; +out: + while (phy--) + if (BIT(phy) & ds->phys_mii_mask) + irq_dispose_mapping(ds->slave_mii_bus->irq[phy]); + + return ret; +} + +static void ksz_irq_phy_free(struct ksz_device *dev) +{ + struct dsa_switch *ds = dev->ds; + int phy; + + for (phy = 0; phy < KSZ_MAX_NUM_PORTS; phy++) + if (BIT(phy) & ds->phys_mii_mask) + irq_dispose_mapping(ds->slave_mii_bus->irq[phy]); +} + +static int ksz_mdio_register(struct ksz_device *dev) +{ + struct dsa_switch *ds = dev->ds; + struct device_node *mdio_np; + struct mii_bus *bus; + int ret; + + mdio_np = of_get_child_by_name(dev->dev->of_node, "mdio"); + if (!mdio_np) + return 0; + + bus = devm_mdiobus_alloc(ds->dev); + if (!bus) { + of_node_put(mdio_np); + return -ENOMEM; + } + + bus->priv = dev; + bus->read = ksz_sw_mdio_read; + bus->write = ksz_sw_mdio_write; + bus->name = "ksz slave smi"; + snprintf(bus->id, MII_BUS_ID_SIZE, "SMI-%d", ds->index); + bus->parent = ds->dev; + bus->phy_mask = ~ds->phys_mii_mask; + + ds->slave_mii_bus = bus; + + if (dev->irq > 0) { + ret = ksz_irq_phy_setup(dev); + if (ret) { + of_node_put(mdio_np); + return ret; + } + } + + ret = devm_of_mdiobus_register(ds->dev, bus, mdio_np); + if (ret) { + dev_err(ds->dev, "unable to register MDIO bus %s\n", + bus->id); + if (dev->irq > 0) + ksz_irq_phy_free(dev); + } + + of_node_put(mdio_np); + + return ret; +} + +static void ksz_irq_mask(struct irq_data *d) +{ + struct ksz_irq *kirq = irq_data_get_irq_chip_data(d); + + kirq->masked |= BIT(d->hwirq); +} + +static void ksz_irq_unmask(struct irq_data *d) +{ + struct ksz_irq *kirq = irq_data_get_irq_chip_data(d); + + kirq->masked &= ~BIT(d->hwirq); +} + +static void ksz_irq_bus_lock(struct irq_data *d) +{ + struct ksz_irq *kirq = irq_data_get_irq_chip_data(d); + + mutex_lock(&kirq->dev->lock_irq); +} + +static void ksz_irq_bus_sync_unlock(struct irq_data *d) +{ + struct ksz_irq *kirq = irq_data_get_irq_chip_data(d); + struct ksz_device *dev = kirq->dev; + int ret; + + ret = ksz_write32(dev, kirq->reg_mask, kirq->masked); + if (ret) + dev_err(dev->dev, "failed to change IRQ mask\n"); + + mutex_unlock(&dev->lock_irq); +} + +static const struct irq_chip ksz_irq_chip = { + .name = "ksz-irq", + .irq_mask = ksz_irq_mask, + .irq_unmask = ksz_irq_unmask, + .irq_bus_lock = ksz_irq_bus_lock, + .irq_bus_sync_unlock = ksz_irq_bus_sync_unlock, +}; + +static int ksz_irq_domain_map(struct irq_domain *d, + unsigned int irq, irq_hw_number_t hwirq) +{ + irq_set_chip_data(irq, d->host_data); + irq_set_chip_and_handler(irq, &ksz_irq_chip, handle_level_irq); + irq_set_noprobe(irq); + + return 0; +} + +static const struct irq_domain_ops ksz_irq_domain_ops = { + .map = ksz_irq_domain_map, + .xlate = irq_domain_xlate_twocell, +}; + +static void ksz_irq_free(struct ksz_irq *kirq) +{ + int irq, virq; + + free_irq(kirq->irq_num, kirq); + + for (irq = 0; irq < kirq->nirqs; irq++) { + virq = irq_find_mapping(kirq->domain, irq); + irq_dispose_mapping(virq); + } + + irq_domain_remove(kirq->domain); +} + +static irqreturn_t ksz_irq_thread_fn(int irq, void *dev_id) +{ + struct ksz_irq *kirq = dev_id; + unsigned int nhandled = 0; + struct ksz_device *dev; + unsigned int sub_irq; + u8 data; + int ret; + u8 n; + + dev = kirq->dev; + + /* Read interrupt status register */ + ret = ksz_read8(dev, kirq->reg_status, &data); + if (ret) + goto out; + + for (n = 0; n < kirq->nirqs; ++n) { + if (data & BIT(n)) { + sub_irq = irq_find_mapping(kirq->domain, n); + handle_nested_irq(sub_irq); + ++nhandled; + } + } +out: + return (nhandled > 0 ? IRQ_HANDLED : IRQ_NONE); +} + +static int ksz_irq_common_setup(struct ksz_device *dev, struct ksz_irq *kirq) +{ + int ret, n; + + kirq->dev = dev; + kirq->masked = ~0; + + kirq->domain = irq_domain_add_simple(dev->dev->of_node, kirq->nirqs, 0, + &ksz_irq_domain_ops, kirq); + if (!kirq->domain) + return -ENOMEM; + + for (n = 0; n < kirq->nirqs; n++) + irq_create_mapping(kirq->domain, n); + + ret = request_threaded_irq(kirq->irq_num, NULL, ksz_irq_thread_fn, + IRQF_ONESHOT | IRQF_TRIGGER_FALLING, + kirq->name, kirq); + if (ret) + goto out; + + return 0; + +out: + ksz_irq_free(kirq); + + return ret; +} + +static int ksz_girq_setup(struct ksz_device *dev) +{ + struct ksz_irq *girq = &dev->girq; + + girq->nirqs = dev->info->port_cnt; + girq->reg_mask = REG_SW_PORT_INT_MASK__1; + girq->reg_status = REG_SW_PORT_INT_STATUS__1; + snprintf(girq->name, sizeof(girq->name), "global_port_irq"); + + girq->irq_num = dev->irq; + + return ksz_irq_common_setup(dev, girq); +} + +static int ksz_pirq_setup(struct ksz_device *dev, u8 p) +{ + struct ksz_irq *pirq = &dev->ports[p].pirq; + + pirq->nirqs = dev->info->port_nirqs; + pirq->reg_mask = dev->dev_ops->get_port_addr(p, REG_PORT_INT_MASK); + pirq->reg_status = dev->dev_ops->get_port_addr(p, REG_PORT_INT_STATUS); + snprintf(pirq->name, sizeof(pirq->name), "port_irq-%d", p); + + pirq->irq_num = irq_find_mapping(dev->girq.domain, p); + if (pirq->irq_num < 0) + return pirq->irq_num; + + return ksz_irq_common_setup(dev, pirq); +} + static int ksz_setup(struct dsa_switch *ds) { struct ksz_device *dev = ds->priv; + struct dsa_port *dp; struct ksz_port *p; const u16 *regs; int ret; @@ -1025,11 +1978,55 @@ static int ksz_setup(struct dsa_switch *ds) p = &dev->ports[dev->cpu_port]; p->learning = true; + if (dev->irq > 0) { + ret = ksz_girq_setup(dev); + if (ret) + return ret; + + dsa_switch_for_each_user_port(dp, dev->ds) { + ret = ksz_pirq_setup(dev, dp->index); + if (ret) + goto out_girq; + } + } + + ret = ksz_mdio_register(dev); + if (ret < 0) { + dev_err(dev->dev, "failed to register the mdio"); + goto out_pirq; + } + /* start switch */ regmap_update_bits(dev->regmap[0], regs[S_START_CTRL], SW_START, SW_START); return 0; + +out_pirq: + if (dev->irq > 0) + dsa_switch_for_each_user_port(dp, dev->ds) + ksz_irq_free(&dev->ports[dp->index].pirq); +out_girq: + if (dev->irq > 0) + ksz_irq_free(&dev->girq); + + return ret; +} + +static void ksz_teardown(struct dsa_switch *ds) +{ + struct ksz_device *dev = ds->priv; + struct dsa_port *dp; + + if (dev->irq > 0) { + dsa_switch_for_each_user_port(dp, dev->ds) + ksz_irq_free(&dev->ports[dp->index].pirq); + + ksz_irq_free(&dev->girq); + } + + if (dev->dev_ops->teardown) + dev->dev_ops->teardown(ds); } static void port_r_cnt(struct ksz_device *dev, int port) @@ -1113,8 +2110,11 @@ static int ksz_phy_read16(struct dsa_switch *ds, int addr, int reg) { struct ksz_device *dev = ds->priv; u16 val = 0xffff; + int ret; - dev->dev_ops->r_phy(dev, addr, reg, &val); + ret = dev->dev_ops->r_phy(dev, addr, reg, &val); + if (ret) + return ret; return val; } @@ -1122,8 +2122,11 @@ static int ksz_phy_read16(struct dsa_switch *ds, int addr, int reg) static int ksz_phy_write16(struct dsa_switch *ds, int addr, int reg, u16 val) { struct ksz_device *dev = ds->priv; + int ret; - dev->dev_ops->w_phy(dev, addr, reg, val); + ret = dev->dev_ops->w_phy(dev, addr, reg, val); + if (ret) + return ret; return 0; } @@ -1212,6 +2215,16 @@ static void ksz_port_fast_age(struct dsa_switch *ds, int port) dev->dev_ops->flush_dyn_mac_table(dev, port); } +static int ksz_set_ageing_time(struct dsa_switch *ds, unsigned int msecs) +{ + struct ksz_device *dev = ds->priv; + + if (!dev->dev_ops->set_ageing_time) + return -EOPNOTSUPP; + + return dev->dev_ops->set_ageing_time(dev, msecs); +} + static int ksz_port_fdb_add(struct dsa_switch *ds, int port, const unsigned char *addr, u16 vid, struct dsa_db db) @@ -1375,10 +2388,12 @@ static enum dsa_tag_protocol ksz_get_tag_protocol(struct dsa_switch *ds, proto = DSA_TAG_PROTO_KSZ8795; if (dev->chip_id == KSZ8830_CHIP_ID || + dev->chip_id == KSZ8563_CHIP_ID || dev->chip_id == KSZ9893_CHIP_ID) proto = DSA_TAG_PROTO_KSZ9893; if (dev->chip_id == KSZ9477_CHIP_ID || + dev->chip_id == KSZ9896_CHIP_ID || dev->chip_id == KSZ9897_CHIP_ID || dev->chip_id == KSZ9567_CHIP_ID) proto = DSA_TAG_PROTO_KSZ9477; @@ -1493,7 +2508,8 @@ static void ksz_set_xmii(struct ksz_device *dev, int port, case PHY_INTERFACE_MODE_RGMII_RXID: data8 |= bitval[P_RGMII_SEL]; /* On KSZ9893, disable RGMII in-band status support */ - if (dev->features & IS_9893) + if (dev->chip_id == KSZ9893_CHIP_ID || + dev->chip_id == KSZ8563_CHIP_ID) data8 &= ~P_MII_MAC_MODE; break; default: @@ -1703,7 +2719,7 @@ static void ksz_phylink_mac_link_up(struct dsa_switch *ds, int port, static int ksz_switch_detect(struct ksz_device *dev) { - u8 id1, id2; + u8 id1, id2, id4; u16 id16; u32 id32; int ret; @@ -1748,8 +2764,8 @@ static int ksz_switch_detect(struct ksz_device *dev) switch (id32) { case KSZ9477_CHIP_ID: + case KSZ9896_CHIP_ID: case KSZ9897_CHIP_ID: - case KSZ9893_CHIP_ID: case KSZ9567_CHIP_ID: case LAN9370_CHIP_ID: case LAN9371_CHIP_ID: @@ -1758,6 +2774,18 @@ static int ksz_switch_detect(struct ksz_device *dev) case LAN9374_CHIP_ID: dev->chip_id = id32; break; + case KSZ9893_CHIP_ID: + ret = ksz_read8(dev, REG_CHIP_ID4, + &id4); + if (ret) + return ret; + + if (id4 == SKU_ID_KSZ8563) + dev->chip_id = KSZ8563_CHIP_ID; + else + dev->chip_id = KSZ9893_CHIP_ID; + + break; default: dev_err(dev->dev, "unsupported switch detected %x)\n", id32); @@ -1771,6 +2799,7 @@ static const struct dsa_switch_ops ksz_switch_ops = { .get_tag_protocol = ksz_get_tag_protocol, .get_phy_flags = ksz_get_phy_flags, .setup = ksz_setup, + .teardown = ksz_teardown, .phy_read = ksz_phy_read16, .phy_write = ksz_phy_write16, .phylink_get_caps = ksz_phylink_get_caps, @@ -1778,6 +2807,7 @@ static const struct dsa_switch_ops ksz_switch_ops = { .phylink_mac_link_up = ksz_phylink_mac_link_up, .phylink_mac_link_down = ksz_mac_link_down, .port_enable = ksz_enable_port, + .set_ageing_time = ksz_set_ageing_time, .get_strings = ksz_get_strings, .get_ethtool_stats = ksz_get_ethtool_stats, .get_sset_count = ksz_sset_count, @@ -1935,6 +2965,9 @@ int ksz_switch_register(struct ksz_device *dev) GFP_KERNEL); if (!dev->ports[i].mib.counters) return -ENOMEM; + + dev->ports[i].ksz_dev = dev; + dev->ports[i].num = i; } /* set the real number of ports */ diff --git a/drivers/net/dsa/microchip/ksz_common.h b/drivers/net/dsa/microchip/ksz_common.h index 0d9520dc6d2d..9cfa179575ce 100644 --- a/drivers/net/dsa/microchip/ksz_common.h +++ b/drivers/net/dsa/microchip/ksz_common.h @@ -13,9 +13,12 @@ #include <linux/phy.h> #include <linux/regmap.h> #include <net/dsa.h> +#include <linux/irq.h> #define KSZ_MAX_NUM_PORTS 8 +struct ksz_device; + struct vlan_table { u32 table[3]; }; @@ -42,6 +45,7 @@ struct ksz_chip_data { int num_statics; int cpu_ports; int port_cnt; + u8 port_nirqs; const struct ksz_dev_ops *ops; bool phy_errata_9477; bool ksz87xx_eee_link_erratum; @@ -61,6 +65,20 @@ struct ksz_chip_data { bool supports_rmii[KSZ_MAX_NUM_PORTS]; bool supports_rgmii[KSZ_MAX_NUM_PORTS]; bool internal_phy[KSZ_MAX_NUM_PORTS]; + bool gbit_capable[KSZ_MAX_NUM_PORTS]; + const struct regmap_access_table *wr_table; + const struct regmap_access_table *rd_table; +}; + +struct ksz_irq { + u16 masked; + u16 reg_mask; + u16 reg_status; + struct irq_domain *domain; + int nirqs; + int irq_num; + char name[16]; + struct ksz_device *dev; }; struct ksz_port { @@ -70,9 +88,7 @@ struct ksz_port { struct phy_device phydev; u32 on:1; /* port is not disabled by hardware */ - u32 phy:1; /* port has a PHY */ u32 fiber:1; /* port is fiber */ - u32 sgmii:1; /* port is SGMII */ u32 force:1; u32 read:1; /* read MIB counters in background */ u32 freeze:1; /* MIB counter freeze is enabled */ @@ -82,6 +98,9 @@ struct ksz_port { u16 max_frame; u32 rgmii_tx_val; u32 rgmii_rx_val; + struct ksz_device *ksz_dev; + struct ksz_irq pirq; + u8 num; }; struct ksz_device { @@ -99,6 +118,7 @@ struct ksz_device { struct regmap *regmap[3]; void *priv; + int irq; struct gpio_desc *reset_gpio; /* Optional reset GPIO */ @@ -118,17 +138,20 @@ struct ksz_device { unsigned long mib_read_interval; u16 mirror_rx; u16 mirror_tx; - u32 features; /* chip specific features */ u16 port_mask; + struct mutex lock_irq; /* IRQ Access */ + struct ksz_irq girq; }; /* List of supported models */ enum ksz_model { + KSZ8563, KSZ8795, KSZ8794, KSZ8765, KSZ8830, KSZ9477, + KSZ9896, KSZ9897, KSZ9893, KSZ9567, @@ -140,11 +163,13 @@ enum ksz_model { }; enum ksz_chip_id { + KSZ8563_CHIP_ID = 0x8563, KSZ8795_CHIP_ID = 0x8795, KSZ8794_CHIP_ID = 0x8794, KSZ8765_CHIP_ID = 0x8765, KSZ8830_CHIP_ID = 0x8830, KSZ9477_CHIP_ID = 0x00947700, + KSZ9896_CHIP_ID = 0x00989600, KSZ9897_CHIP_ID = 0x00989700, KSZ9893_CHIP_ID = 0x00989300, KSZ9567_CHIP_ID = 0x00956700, @@ -254,13 +279,15 @@ struct alu_struct { struct ksz_dev_ops { int (*setup)(struct dsa_switch *ds); + void (*teardown)(struct dsa_switch *ds); u32 (*get_port_addr)(int port, int offset); void (*cfg_port_member)(struct ksz_device *dev, int port, u8 member); void (*flush_dyn_mac_table)(struct ksz_device *dev, int port); void (*port_cleanup)(struct ksz_device *dev, int port); void (*port_setup)(struct ksz_device *dev, int port, bool cpu_port); - void (*r_phy)(struct ksz_device *dev, u16 phy, u16 reg, u16 *val); - void (*w_phy)(struct ksz_device *dev, u16 phy, u16 reg, u16 val); + int (*set_ageing_time)(struct ksz_device *dev, unsigned int msecs); + int (*r_phy)(struct ksz_device *dev, u16 phy, u16 reg, u16 *val); + int (*w_phy)(struct ksz_device *dev, u16 phy, u16 reg, u16 val); void (*r_mib_cnt)(struct ksz_device *dev, int port, u16 addr, u64 *cnt); void (*r_mib_pkt)(struct ksz_device *dev, int port, u16 addr, @@ -330,6 +357,10 @@ static inline int ksz_read8(struct ksz_device *dev, u32 reg, u8 *val) unsigned int value; int ret = regmap_read(dev->regmap[0], reg, &value); + if (ret) + dev_err(dev->dev, "can't read 8bit reg: 0x%x %pe\n", reg, + ERR_PTR(ret)); + *val = value; return ret; } @@ -339,6 +370,10 @@ static inline int ksz_read16(struct ksz_device *dev, u32 reg, u16 *val) unsigned int value; int ret = regmap_read(dev->regmap[1], reg, &value); + if (ret) + dev_err(dev->dev, "can't read 16bit reg: 0x%x %pe\n", reg, + ERR_PTR(ret)); + *val = value; return ret; } @@ -348,6 +383,10 @@ static inline int ksz_read32(struct ksz_device *dev, u32 reg, u32 *val) unsigned int value; int ret = regmap_read(dev->regmap[2], reg, &value); + if (ret) + dev_err(dev->dev, "can't read 32bit reg: 0x%x %pe\n", reg, + ERR_PTR(ret)); + *val = value; return ret; } @@ -358,7 +397,10 @@ static inline int ksz_read64(struct ksz_device *dev, u32 reg, u64 *val) int ret; ret = regmap_bulk_read(dev->regmap[2], reg, value, 2); - if (!ret) + if (ret) + dev_err(dev->dev, "can't read 64bit reg: 0x%x %pe\n", reg, + ERR_PTR(ret)); + else *val = (u64)value[0] << 32 | value[1]; return ret; @@ -366,17 +408,38 @@ static inline int ksz_read64(struct ksz_device *dev, u32 reg, u64 *val) static inline int ksz_write8(struct ksz_device *dev, u32 reg, u8 value) { - return regmap_write(dev->regmap[0], reg, value); + int ret; + + ret = regmap_write(dev->regmap[0], reg, value); + if (ret) + dev_err(dev->dev, "can't write 8bit reg: 0x%x %pe\n", reg, + ERR_PTR(ret)); + + return ret; } static inline int ksz_write16(struct ksz_device *dev, u32 reg, u16 value) { - return regmap_write(dev->regmap[1], reg, value); + int ret; + + ret = regmap_write(dev->regmap[1], reg, value); + if (ret) + dev_err(dev->dev, "can't write 16bit reg: 0x%x %pe\n", reg, + ERR_PTR(ret)); + + return ret; } static inline int ksz_write32(struct ksz_device *dev, u32 reg, u32 value) { - return regmap_write(dev->regmap[2], reg, value); + int ret; + + ret = regmap_write(dev->regmap[2], reg, value); + if (ret) + dev_err(dev->dev, "can't write 32bit reg: 0x%x %pe\n", reg, + ERR_PTR(ret)); + + return ret; } static inline int ksz_write64(struct ksz_device *dev, u32 reg, u64 value) @@ -391,40 +454,42 @@ static inline int ksz_write64(struct ksz_device *dev, u32 reg, u64 value) return regmap_bulk_write(dev->regmap[2], reg, val, 2); } -static inline void ksz_pread8(struct ksz_device *dev, int port, int offset, - u8 *data) +static inline int ksz_pread8(struct ksz_device *dev, int port, int offset, + u8 *data) { - ksz_read8(dev, dev->dev_ops->get_port_addr(port, offset), data); + return ksz_read8(dev, dev->dev_ops->get_port_addr(port, offset), data); } -static inline void ksz_pread16(struct ksz_device *dev, int port, int offset, - u16 *data) +static inline int ksz_pread16(struct ksz_device *dev, int port, int offset, + u16 *data) { - ksz_read16(dev, dev->dev_ops->get_port_addr(port, offset), data); + return ksz_read16(dev, dev->dev_ops->get_port_addr(port, offset), data); } -static inline void ksz_pread32(struct ksz_device *dev, int port, int offset, - u32 *data) +static inline int ksz_pread32(struct ksz_device *dev, int port, int offset, + u32 *data) { - ksz_read32(dev, dev->dev_ops->get_port_addr(port, offset), data); + return ksz_read32(dev, dev->dev_ops->get_port_addr(port, offset), data); } -static inline void ksz_pwrite8(struct ksz_device *dev, int port, int offset, - u8 data) +static inline int ksz_pwrite8(struct ksz_device *dev, int port, int offset, + u8 data) { - ksz_write8(dev, dev->dev_ops->get_port_addr(port, offset), data); + return ksz_write8(dev, dev->dev_ops->get_port_addr(port, offset), data); } -static inline void ksz_pwrite16(struct ksz_device *dev, int port, int offset, - u16 data) +static inline int ksz_pwrite16(struct ksz_device *dev, int port, int offset, + u16 data) { - ksz_write16(dev, dev->dev_ops->get_port_addr(port, offset), data); + return ksz_write16(dev, dev->dev_ops->get_port_addr(port, offset), + data); } -static inline void ksz_pwrite32(struct ksz_device *dev, int port, int offset, - u32 data) +static inline int ksz_pwrite32(struct ksz_device *dev, int port, int offset, + u32 data) { - ksz_write32(dev, dev->dev_ops->get_port_addr(port, offset), data); + return ksz_write32(dev, dev->dev_ops->get_port_addr(port, offset), + data); } static inline void ksz_prmw8(struct ksz_device *dev, int port, int offset, @@ -483,6 +548,10 @@ static inline int is_lan937x(struct ksz_device *dev) #define SW_REV_ID_M GENMASK(7, 4) +/* KSZ9893, KSZ9563, KSZ8563 specific register */ +#define REG_CHIP_ID4 0x0f +#define SKU_ID_KSZ8563 0x3c + /* Driver set switch broadcast storm protection at 10% rate. */ #define BROADCAST_STORM_PROT_RATE 10 @@ -497,10 +566,6 @@ static inline int is_lan937x(struct ksz_device *dev) #define SW_START 0x01 -/* Used with variable features to indicate capabilities. */ -#define GBIT_SUPPORT BIT(0) -#define IS_9893 BIT(2) - /* xMII configuration */ #define P_MII_DUPLEX_M BIT(6) #define P_MII_100MBIT_M BIT(4) @@ -511,6 +576,15 @@ static inline int is_lan937x(struct ksz_device *dev) #define P_MII_MAC_MODE BIT(2) #define P_MII_SEL_M 0x3 +/* Interrupt */ +#define REG_SW_PORT_INT_STATUS__1 0x001B +#define REG_SW_PORT_INT_MASK__1 0x001F + +#define REG_PORT_INT_STATUS 0x001B +#define REG_PORT_INT_MASK 0x001F + +#define PORT_SRC_PHY_INT 1 + /* Regmap tables generation */ #define KSZ_SPI_OP_RD 3 #define KSZ_SPI_OP_WR 2 diff --git a/drivers/net/dsa/microchip/ksz_spi.c b/drivers/net/dsa/microchip/ksz_spi.c index 05bd089795f8..1b6ab891b986 100644 --- a/drivers/net/dsa/microchip/ksz_spi.c +++ b/drivers/net/dsa/microchip/ksz_spi.c @@ -66,7 +66,10 @@ static int ksz_spi_probe(struct spi_device *spi) for (i = 0; i < ARRAY_SIZE(ksz8795_regmap_config); i++) { rc = regmap_config[i]; rc.lock_arg = &dev->regmap_mutex; + rc.wr_table = chip->wr_table; + rc.rd_table = chip->rd_table; dev->regmap[i] = devm_regmap_init_spi(spi, &rc); + if (IS_ERR(dev->regmap[i])) { ret = PTR_ERR(dev->regmap[i]); dev_err(&spi->dev, @@ -85,6 +88,8 @@ static int ksz_spi_probe(struct spi_device *spi) if (ret) return ret; + dev->irq = spi->irq; + ret = ksz_switch_register(dev); /* Main DSA driver may not be started yet. */ @@ -102,8 +107,6 @@ static void ksz_spi_remove(struct spi_device *spi) if (dev) ksz_switch_remove(dev); - - spi_set_drvdata(spi, NULL); } static void ksz_spi_shutdown(struct spi_device *spi) @@ -147,6 +150,10 @@ static const struct of_device_id ksz_dt_ids[] = { .data = &ksz_switch_chips[KSZ9477] }, { + .compatible = "microchip,ksz9896", + .data = &ksz_switch_chips[KSZ9896] + }, + { .compatible = "microchip,ksz9897", .data = &ksz_switch_chips[KSZ9897] }, @@ -160,7 +167,7 @@ static const struct of_device_id ksz_dt_ids[] = { }, { .compatible = "microchip,ksz8563", - .data = &ksz_switch_chips[KSZ9893] + .data = &ksz_switch_chips[KSZ8563] }, { .compatible = "microchip,ksz9567", @@ -197,6 +204,7 @@ static const struct spi_device_id ksz_spi_ids[] = { { "ksz8863" }, { "ksz8873" }, { "ksz9477" }, + { "ksz9896" }, { "ksz9897" }, { "ksz9893" }, { "ksz9563" }, @@ -226,6 +234,7 @@ static struct spi_driver ksz_spi_driver = { module_spi_driver(ksz_spi_driver); MODULE_ALIAS("spi:ksz9477"); +MODULE_ALIAS("spi:ksz9896"); MODULE_ALIAS("spi:ksz9897"); MODULE_ALIAS("spi:ksz9893"); MODULE_ALIAS("spi:ksz9563"); diff --git a/drivers/net/dsa/microchip/lan937x.h b/drivers/net/dsa/microchip/lan937x.h index 4e0b1dccec27..8e9e66d6728d 100644 --- a/drivers/net/dsa/microchip/lan937x.h +++ b/drivers/net/dsa/microchip/lan937x.h @@ -8,14 +8,16 @@ int lan937x_reset_switch(struct ksz_device *dev); int lan937x_setup(struct dsa_switch *ds); +void lan937x_teardown(struct dsa_switch *ds); void lan937x_port_setup(struct ksz_device *dev, int port, bool cpu_port); void lan937x_config_cpu_port(struct dsa_switch *ds); int lan937x_switch_init(struct ksz_device *dev); void lan937x_switch_exit(struct ksz_device *dev); -void lan937x_r_phy(struct ksz_device *dev, u16 addr, u16 reg, u16 *data); -void lan937x_w_phy(struct ksz_device *dev, u16 addr, u16 reg, u16 val); +int lan937x_r_phy(struct ksz_device *dev, u16 addr, u16 reg, u16 *data); +int lan937x_w_phy(struct ksz_device *dev, u16 addr, u16 reg, u16 val); int lan937x_change_mtu(struct ksz_device *dev, int port, int new_mtu); void lan937x_phylink_get_caps(struct ksz_device *dev, int port, struct phylink_config *config); void lan937x_setup_rgmii_delay(struct ksz_device *dev, int port); +int lan937x_set_ageing_time(struct ksz_device *dev, unsigned int msecs); #endif diff --git a/drivers/net/dsa/microchip/lan937x_main.c b/drivers/net/dsa/microchip/lan937x_main.c index 5579644e8fde..7e4f307a0387 100644 --- a/drivers/net/dsa/microchip/lan937x_main.c +++ b/drivers/net/dsa/microchip/lan937x_main.c @@ -7,7 +7,6 @@ #include <linux/iopoll.h> #include <linux/phy.h> #include <linux/of_net.h> -#include <linux/of_mdio.h> #include <linux/if_bridge.h> #include <linux/if_vlan.h> #include <linux/math.h> @@ -128,81 +127,14 @@ static int lan937x_internal_phy_read(struct ksz_device *dev, int addr, int reg, return ksz_read16(dev, REG_VPHY_IND_DATA__2, val); } -void lan937x_r_phy(struct ksz_device *dev, u16 addr, u16 reg, u16 *data) +int lan937x_r_phy(struct ksz_device *dev, u16 addr, u16 reg, u16 *data) { - lan937x_internal_phy_read(dev, addr, reg, data); + return lan937x_internal_phy_read(dev, addr, reg, data); } -void lan937x_w_phy(struct ksz_device *dev, u16 addr, u16 reg, u16 val) +int lan937x_w_phy(struct ksz_device *dev, u16 addr, u16 reg, u16 val) { - lan937x_internal_phy_write(dev, addr, reg, val); -} - -static int lan937x_sw_mdio_read(struct mii_bus *bus, int addr, int regnum) -{ - struct ksz_device *dev = bus->priv; - u16 val; - int ret; - - if (regnum & MII_ADDR_C45) - return -EOPNOTSUPP; - - ret = lan937x_internal_phy_read(dev, addr, regnum, &val); - if (ret < 0) - return ret; - - return val; -} - -static int lan937x_sw_mdio_write(struct mii_bus *bus, int addr, int regnum, - u16 val) -{ - struct ksz_device *dev = bus->priv; - - if (regnum & MII_ADDR_C45) - return -EOPNOTSUPP; - - return lan937x_internal_phy_write(dev, addr, regnum, val); -} - -static int lan937x_mdio_register(struct ksz_device *dev) -{ - struct dsa_switch *ds = dev->ds; - struct device_node *mdio_np; - struct mii_bus *bus; - int ret; - - mdio_np = of_get_child_by_name(dev->dev->of_node, "mdio"); - if (!mdio_np) { - dev_err(ds->dev, "no MDIO bus node\n"); - return -ENODEV; - } - - bus = devm_mdiobus_alloc(ds->dev); - if (!bus) { - of_node_put(mdio_np); - return -ENOMEM; - } - - bus->priv = dev; - bus->read = lan937x_sw_mdio_read; - bus->write = lan937x_sw_mdio_write; - bus->name = "lan937x slave smi"; - snprintf(bus->id, MII_BUS_ID_SIZE, "SMI-%d", ds->index); - bus->parent = ds->dev; - bus->phy_mask = ~ds->phys_mii_mask; - - ds->slave_mii_bus = bus; - - ret = devm_of_mdiobus_register(ds->dev, bus, mdio_np); - if (ret) { - dev_err(ds->dev, "unable to register MDIO bus %s\n", - bus->id); - } - - of_node_put(mdio_np); - - return ret; + return lan937x_internal_phy_write(dev, addr, reg, val); } int lan937x_reset_switch(struct ksz_device *dev) @@ -225,6 +157,10 @@ int lan937x_reset_switch(struct ksz_device *dev) if (ret < 0) return ret; + ret = ksz_write32(dev, REG_SW_INT_STATUS__4, POR_READY_INT); + if (ret < 0) + return ret; + ret = ksz_write32(dev, REG_SW_PORT_INT_MASK__4, 0xFF); if (ret < 0) return ret; @@ -311,6 +247,23 @@ int lan937x_change_mtu(struct ksz_device *dev, int port, int new_mtu) return 0; } +int lan937x_set_ageing_time(struct ksz_device *dev, unsigned int msecs) +{ + u32 secs = msecs / 1000; + u32 value; + int ret; + + value = FIELD_GET(SW_AGE_PERIOD_7_0_M, secs); + + ret = ksz_write8(dev, REG_SW_AGE_PERIOD__1, value); + if (ret < 0) + return ret; + + value = FIELD_GET(SW_AGE_PERIOD_19_8_M, secs); + + return ksz_write16(dev, REG_SW_AGE_PERIOD__2, value); +} + static void lan937x_set_tune_adj(struct ksz_device *dev, int port, u16 reg, u8 val) { @@ -379,6 +332,13 @@ void lan937x_setup_rgmii_delay(struct ksz_device *dev, int port) } } +int lan937x_switch_init(struct ksz_device *dev) +{ + dev->port_mask = (1 << dev->info->port_cnt) - 1; + + return 0; +} + int lan937x_setup(struct dsa_switch *ds) { struct ksz_device *dev = ds->priv; @@ -391,12 +351,6 @@ int lan937x_setup(struct dsa_switch *ds) return ret; } - ret = lan937x_mdio_register(dev); - if (ret < 0) { - dev_err(dev->dev, "failed to register the mdio"); - return ret; - } - /* The VLAN aware is a global setting. Mixed vlan * filterings are not supported. */ @@ -422,11 +376,9 @@ int lan937x_setup(struct dsa_switch *ds) return 0; } -int lan937x_switch_init(struct ksz_device *dev) +void lan937x_teardown(struct dsa_switch *ds) { - dev->port_mask = (1 << dev->info->port_cnt) - 1; - return 0; } void lan937x_switch_exit(struct ksz_device *dev) diff --git a/drivers/net/dsa/microchip/lan937x_reg.h b/drivers/net/dsa/microchip/lan937x_reg.h index ba4adaddb3ec..5bc16a4c4441 100644 --- a/drivers/net/dsa/microchip/lan937x_reg.h +++ b/drivers/net/dsa/microchip/lan937x_reg.h @@ -62,6 +62,12 @@ #define SW_FAST_AGING BIT(1) #define SW_LINK_AUTO_AGING BIT(0) +#define REG_SW_AGE_PERIOD__1 0x0313 +#define SW_AGE_PERIOD_7_0_M GENMASK(7, 0) + +#define REG_SW_AGE_PERIOD__2 0x0320 +#define SW_AGE_PERIOD_19_8_M GENMASK(19, 8) + #define REG_SW_MAC_CTRL_0 0x0330 #define SW_NEW_BACKOFF BIT(7) #define SW_PAUSE_UNH_MODE BIT(1) @@ -118,6 +124,18 @@ /* Port Registers */ /* 0 - Operation */ +#define REG_PORT_INT_STATUS 0x001B +#define REG_PORT_INT_MASK 0x001F + +#define PORT_TAS_INT BIT(5) +#define PORT_QCI_INT BIT(4) +#define PORT_SGMII_INT BIT(3) +#define PORT_PTP_INT BIT(2) +#define PORT_PHY_INT BIT(1) +#define PORT_ACL_INT BIT(0) + +#define PORT_SRC_PHY_INT 1 + #define REG_PORT_CTRL_0 0x0020 #define PORT_MAC_LOOPBACK BIT(7) diff --git a/drivers/net/dsa/mt7530.c b/drivers/net/dsa/mt7530.c index 409d5c3d76ea..e74c6b406172 100644 --- a/drivers/net/dsa/mt7530.c +++ b/drivers/net/dsa/mt7530.c @@ -2708,9 +2708,6 @@ mt7531_mac_config(struct dsa_switch *ds, int port, unsigned int mode, case PHY_INTERFACE_MODE_NA: case PHY_INTERFACE_MODE_1000BASEX: case PHY_INTERFACE_MODE_2500BASEX: - if (phylink_autoneg_inband(mode)) - return -EINVAL; - return mt7531_sgmii_setup_mode_force(priv, port, interface); default: return -EINVAL; @@ -2786,13 +2783,6 @@ unsupported: return; } - if (phylink_autoneg_inband(mode) && - state->interface != PHY_INTERFACE_MODE_SGMII) { - dev_err(ds->dev, "%s: in-band negotiation unsupported\n", - __func__); - return; - } - mcr_cur = mt7530_read(priv, MT7530_PMCR_P(port)); mcr_new = mcr_cur; mcr_new &= ~PMCR_LINK_SETTINGS_MASK; @@ -2929,6 +2919,9 @@ static void mt753x_phylink_get_caps(struct dsa_switch *ds, int port, config->mac_capabilities = MAC_ASYM_PAUSE | MAC_SYM_PAUSE | MAC_10 | MAC_100 | MAC_1000FD; + if ((priv->id == ID_MT7531) && mt753x_is_mac_port(port)) + config->mac_capabilities |= MAC_2500FD; + /* This driver does not make use of the speed, duplex, pause or the * advertisement in its mac_config, so it is safe to mark this driver * as non-legacy. @@ -2994,6 +2987,7 @@ mt7531_sgmii_pcs_get_state_an(struct mt7530_priv *priv, int port, status = mt7530_read(priv, MT7531_PCS_CONTROL_1(port)); state->link = !!(status & MT7531_SGMII_LINK_STATUS); + state->an_complete = !!(status & MT7531_SGMII_AN_COMPLETE); if (state->interface == PHY_INTERFACE_MODE_SGMII && (status & MT7531_SGMII_AN_ENABLE)) { val = mt7530_read(priv, MT7531_PCS_SPEED_ABILITY(port)); @@ -3024,16 +3018,44 @@ mt7531_sgmii_pcs_get_state_an(struct mt7530_priv *priv, int port, return 0; } +static void +mt7531_sgmii_pcs_get_state_inband(struct mt7530_priv *priv, int port, + struct phylink_link_state *state) +{ + unsigned int val; + + val = mt7530_read(priv, MT7531_PCS_CONTROL_1(port)); + state->link = !!(val & MT7531_SGMII_LINK_STATUS); + if (!state->link) + return; + + state->an_complete = state->link; + + if (state->interface == PHY_INTERFACE_MODE_2500BASEX) + state->speed = SPEED_2500; + else + state->speed = SPEED_1000; + + state->duplex = DUPLEX_FULL; + state->pause = MLO_PAUSE_NONE; +} + static void mt7531_pcs_get_state(struct phylink_pcs *pcs, struct phylink_link_state *state) { struct mt7530_priv *priv = pcs_to_mt753x_pcs(pcs)->priv; int port = pcs_to_mt753x_pcs(pcs)->port; - if (state->interface == PHY_INTERFACE_MODE_SGMII) + if (state->interface == PHY_INTERFACE_MODE_SGMII) { mt7531_sgmii_pcs_get_state_an(priv, port, state); - else - state->link = false; + return; + } else if ((state->interface == PHY_INTERFACE_MODE_1000BASEX) || + (state->interface == PHY_INTERFACE_MODE_2500BASEX)) { + mt7531_sgmii_pcs_get_state_inband(priv, port, state); + return; + } + + state->link = false; } static int mt753x_pcs_config(struct phylink_pcs *pcs, unsigned int mode, @@ -3074,6 +3096,8 @@ mt753x_setup(struct dsa_switch *ds) priv->pcs[i].pcs.ops = priv->info->pcs_ops; priv->pcs[i].priv = priv; priv->pcs[i].port = i; + if (mt753x_is_mac_port(i)) + priv->pcs[i].pcs.poll = 1; } ret = priv->info->sw_setup(ds); @@ -3307,8 +3331,6 @@ mt7530_remove(struct mdio_device *mdiodev) dsa_unregister_switch(priv->ds); mutex_destroy(&priv->reg_mutex); - - dev_set_drvdata(&mdiodev->dev, NULL); } static void mt7530_shutdown(struct mdio_device *mdiodev) diff --git a/drivers/net/dsa/mt7530.h b/drivers/net/dsa/mt7530.h index e509af95c354..e8d966435350 100644 --- a/drivers/net/dsa/mt7530.h +++ b/drivers/net/dsa/mt7530.h @@ -373,6 +373,7 @@ enum mt7530_vlan_port_acc_frm { #define MT7531_SGMII_LINK_STATUS BIT(18) #define MT7531_SGMII_AN_ENABLE BIT(12) #define MT7531_SGMII_AN_RESTART BIT(9) +#define MT7531_SGMII_AN_COMPLETE BIT(21) /* Register for SGMII PCS_SPPED_ABILITY */ #define MT7531_PCS_SPEED_ABILITY(p) MT7531_SGMII_REG(p, 0x08) diff --git a/drivers/net/dsa/mv88e6060.c b/drivers/net/dsa/mv88e6060.c index 83dca9179aa0..fdda62d6eb16 100644 --- a/drivers/net/dsa/mv88e6060.c +++ b/drivers/net/dsa/mv88e6060.c @@ -297,8 +297,6 @@ static void mv88e6060_remove(struct mdio_device *mdiodev) return; dsa_unregister_switch(ds); - - dev_set_drvdata(&mdiodev->dev, NULL); } static void mv88e6060_shutdown(struct mdio_device *mdiodev) diff --git a/drivers/net/dsa/mv88e6xxx/chip.c b/drivers/net/dsa/mv88e6xxx/chip.c index 07e9a4da924c..2479be3a1e35 100644 --- a/drivers/net/dsa/mv88e6xxx/chip.c +++ b/drivers/net/dsa/mv88e6xxx/chip.c @@ -816,6 +816,14 @@ static void mv88e6393x_phylink_get_caps(struct mv88e6xxx_chip *chip, int port, MAC_10000FD; } } + + if (port == 0) { + __set_bit(PHY_INTERFACE_MODE_RMII, supported); + __set_bit(PHY_INTERFACE_MODE_RGMII, supported); + __set_bit(PHY_INTERFACE_MODE_RGMII_ID, supported); + __set_bit(PHY_INTERFACE_MODE_RGMII_RXID, supported); + __set_bit(PHY_INTERFACE_MODE_RGMII_TXID, supported); + } } static void mv88e6xxx_get_caps(struct dsa_switch *ds, int port, @@ -1128,7 +1136,7 @@ static void mv88e6xxx_atu_vtu_get_strings(uint8_t *data) unsigned int i; for (i = 0; i < ARRAY_SIZE(mv88e6xxx_atu_vtu_stats_strings); i++) - strlcpy(data + i * ETH_GSTRING_LEN, + strscpy(data + i * ETH_GSTRING_LEN, mv88e6xxx_atu_vtu_stats_strings[i], ETH_GSTRING_LEN); } @@ -6585,14 +6593,17 @@ out: static bool mv88e6xxx_lag_can_offload(struct dsa_switch *ds, struct dsa_lag lag, - struct netdev_lag_upper_info *info) + struct netdev_lag_upper_info *info, + struct netlink_ext_ack *extack) { struct mv88e6xxx_chip *chip = ds->priv; struct dsa_port *dp; int members = 0; - if (!mv88e6xxx_has_lag(chip)) + if (!mv88e6xxx_has_lag(chip)) { + NL_SET_ERR_MSG_MOD(extack, "Chip does not support LAG offload"); return false; + } if (!lag.id) return false; @@ -6601,14 +6612,20 @@ static bool mv88e6xxx_lag_can_offload(struct dsa_switch *ds, /* Includes the port joining the LAG */ members++; - if (members > 8) + if (members > 8) { + NL_SET_ERR_MSG_MOD(extack, + "Cannot offload more than 8 LAG ports"); return false; + } /* We could potentially relax this to include active * backup in the future. */ - if (info->tx_type != NETDEV_LAG_TX_TYPE_HASH) + if (info->tx_type != NETDEV_LAG_TX_TYPE_HASH) { + NL_SET_ERR_MSG_MOD(extack, + "Can only offload LAG using hash TX type"); return false; + } /* Ideally we would also validate that the hash type matches * the hardware. Alas, this is always set to unknown on team @@ -6761,12 +6778,13 @@ static int mv88e6xxx_port_lag_change(struct dsa_switch *ds, int port) static int mv88e6xxx_port_lag_join(struct dsa_switch *ds, int port, struct dsa_lag lag, - struct netdev_lag_upper_info *info) + struct netdev_lag_upper_info *info, + struct netlink_ext_ack *extack) { struct mv88e6xxx_chip *chip = ds->priv; int err, id; - if (!mv88e6xxx_lag_can_offload(ds, lag, info)) + if (!mv88e6xxx_lag_can_offload(ds, lag, info, extack)) return -EOPNOTSUPP; /* DSA LAG IDs are one-based */ @@ -6819,12 +6837,13 @@ static int mv88e6xxx_crosschip_lag_change(struct dsa_switch *ds, int sw_index, static int mv88e6xxx_crosschip_lag_join(struct dsa_switch *ds, int sw_index, int port, struct dsa_lag lag, - struct netdev_lag_upper_info *info) + struct netdev_lag_upper_info *info, + struct netlink_ext_ack *extack) { struct mv88e6xxx_chip *chip = ds->priv; int err; - if (!mv88e6xxx_lag_can_offload(ds, lag, info)) + if (!mv88e6xxx_lag_can_offload(ds, lag, info, extack)) return -EOPNOTSUPP; mv88e6xxx_reg_lock(chip); @@ -7166,8 +7185,6 @@ static void mv88e6xxx_remove(struct mdio_device *mdiodev) mv88e6xxx_g1_irq_free(chip); else mv88e6xxx_irq_poll_free(chip); - - dev_set_drvdata(&mdiodev->dev, NULL); } static void mv88e6xxx_shutdown(struct mdio_device *mdiodev) diff --git a/drivers/net/dsa/mv88e6xxx/global2.h b/drivers/net/dsa/mv88e6xxx/global2.h index 807aeaad9830..7536b8b0ad01 100644 --- a/drivers/net/dsa/mv88e6xxx/global2.h +++ b/drivers/net/dsa/mv88e6xxx/global2.h @@ -298,7 +298,7 @@ #define MV88E6352_G2_SCRATCH_CONFIG_DATA1 0x71 #define MV88E6352_G2_SCRATCH_CONFIG_DATA1_NO_CPU BIT(2) #define MV88E6352_G2_SCRATCH_CONFIG_DATA2 0x72 -#define MV88E6352_G2_SCRATCH_CONFIG_DATA2_P0_MODE_MASK 0x3 +#define MV88E6352_G2_SCRATCH_CONFIG_DATA2_P0_MODE_MASK 0xf #define MV88E6352_G2_SCRATCH_CONFIG_DATA3 0x73 #define MV88E6352_G2_SCRATCH_CONFIG_DATA3_S_SEL BIT(1) diff --git a/drivers/net/dsa/mv88e6xxx/port.c b/drivers/net/dsa/mv88e6xxx/port.c index 90c55f23b7c9..5c4195c635b0 100644 --- a/drivers/net/dsa/mv88e6xxx/port.c +++ b/drivers/net/dsa/mv88e6xxx/port.c @@ -517,6 +517,12 @@ static int mv88e6xxx_port_set_cmode(struct mv88e6xxx_chip *chip, int port, case PHY_INTERFACE_MODE_RMII: cmode = MV88E6XXX_PORT_STS_CMODE_RMII; break; + case PHY_INTERFACE_MODE_RGMII: + case PHY_INTERFACE_MODE_RGMII_ID: + case PHY_INTERFACE_MODE_RGMII_RXID: + case PHY_INTERFACE_MODE_RGMII_TXID: + cmode = MV88E6XXX_PORT_STS_CMODE_RGMII; + break; case PHY_INTERFACE_MODE_1000BASEX: cmode = MV88E6XXX_PORT_STS_CMODE_1000BASEX; break; @@ -634,6 +640,19 @@ int mv88e6393x_port_set_cmode(struct mv88e6xxx_chip *chip, int port, if (port != 0 && port != 9 && port != 10) return -EOPNOTSUPP; + if (port == 9 || port == 10) { + switch (mode) { + case PHY_INTERFACE_MODE_RMII: + case PHY_INTERFACE_MODE_RGMII: + case PHY_INTERFACE_MODE_RGMII_ID: + case PHY_INTERFACE_MODE_RGMII_RXID: + case PHY_INTERFACE_MODE_RGMII_TXID: + return -EINVAL; + default: + break; + } + } + /* mv88e6393x errata 4.5: EEE should be disabled on SERDES ports */ err = mv88e6xxx_port_read(chip, port, MV88E6XXX_PORT_MAC_CTL, ®); if (err) diff --git a/drivers/net/dsa/ocelot/felix.c b/drivers/net/dsa/ocelot/felix.c index aadb0bd7c24f..dd3a18cc89dd 100644 --- a/drivers/net/dsa/ocelot/felix.c +++ b/drivers/net/dsa/ocelot/felix.c @@ -42,6 +42,25 @@ static struct net_device *felix_classify_db(struct dsa_db db) } } +static int felix_cpu_port_for_master(struct dsa_switch *ds, + struct net_device *master) +{ + struct ocelot *ocelot = ds->priv; + struct dsa_port *cpu_dp; + int lag; + + if (netif_is_lag_master(master)) { + mutex_lock(&ocelot->fwd_domain_lock); + lag = ocelot_bond_get_id(ocelot, master); + mutex_unlock(&ocelot->fwd_domain_lock); + + return lag; + } + + cpu_dp = master->dsa_ptr; + return cpu_dp->index; +} + /* Set up VCAP ES0 rules for pushing a tag_8021q VLAN towards the CPU such that * the tagger can perform RX source port identification. */ @@ -422,6 +441,40 @@ static unsigned long felix_tag_npi_get_host_fwd_mask(struct dsa_switch *ds) return BIT(ocelot->num_phys_ports); } +static int felix_tag_npi_change_master(struct dsa_switch *ds, int port, + struct net_device *master, + struct netlink_ext_ack *extack) +{ + struct dsa_port *dp = dsa_to_port(ds, port), *other_dp; + struct ocelot *ocelot = ds->priv; + + if (netif_is_lag_master(master)) { + NL_SET_ERR_MSG_MOD(extack, + "LAG DSA master only supported using ocelot-8021q"); + return -EOPNOTSUPP; + } + + /* Changing the NPI port breaks user ports still assigned to the old + * one, so only allow it while they're down, and don't allow them to + * come back up until they're all changed to the new one. + */ + dsa_switch_for_each_user_port(other_dp, ds) { + struct net_device *slave = other_dp->slave; + + if (other_dp != dp && (slave->flags & IFF_UP) && + dsa_port_to_master(other_dp) != master) { + NL_SET_ERR_MSG_MOD(extack, + "Cannot change while old master still has users"); + return -EOPNOTSUPP; + } + } + + felix_npi_port_deinit(ocelot, ocelot->npi); + felix_npi_port_init(ocelot, felix_cpu_port_for_master(ds, master)); + + return 0; +} + /* Alternatively to using the NPI functionality, that same hardware MAC * connected internally to the enetc or fman DSA master can be configured to * use the software-defined tag_8021q frame format. As far as the hardware is @@ -433,6 +486,7 @@ static const struct felix_tag_proto_ops felix_tag_npi_proto_ops = { .setup = felix_tag_npi_setup, .teardown = felix_tag_npi_teardown, .get_host_fwd_mask = felix_tag_npi_get_host_fwd_mask, + .change_master = felix_tag_npi_change_master, }; static int felix_tag_8021q_setup(struct dsa_switch *ds) @@ -445,6 +499,9 @@ static int felix_tag_8021q_setup(struct dsa_switch *ds) if (err) return err; + dsa_switch_for_each_cpu_port(dp, ds) + ocelot_port_setup_dsa_8021q_cpu(ocelot, dp->index); + dsa_switch_for_each_user_port(dp, ds) ocelot_port_assign_dsa_8021q_cpu(ocelot, dp->index, dp->cpu_dp->index); @@ -493,6 +550,9 @@ static void felix_tag_8021q_teardown(struct dsa_switch *ds) dsa_switch_for_each_user_port(dp, ds) ocelot_port_unassign_dsa_8021q_cpu(ocelot, dp->index); + dsa_switch_for_each_cpu_port(dp, ds) + ocelot_port_teardown_dsa_8021q_cpu(ocelot, dp->index); + dsa_tag_8021q_unregister(ds); } @@ -501,10 +561,24 @@ static unsigned long felix_tag_8021q_get_host_fwd_mask(struct dsa_switch *ds) return dsa_cpu_ports(ds); } +static int felix_tag_8021q_change_master(struct dsa_switch *ds, int port, + struct net_device *master, + struct netlink_ext_ack *extack) +{ + int cpu = felix_cpu_port_for_master(ds, master); + struct ocelot *ocelot = ds->priv; + + ocelot_port_unassign_dsa_8021q_cpu(ocelot, port); + ocelot_port_assign_dsa_8021q_cpu(ocelot, port, cpu); + + return felix_update_trapping_destinations(ds, true); +} + static const struct felix_tag_proto_ops felix_tag_8021q_proto_ops = { .setup = felix_tag_8021q_setup, .teardown = felix_tag_8021q_teardown, .get_host_fwd_mask = felix_tag_8021q_get_host_fwd_mask, + .change_master = felix_tag_8021q_change_master, }; static void felix_set_host_flood(struct dsa_switch *ds, unsigned long mask, @@ -667,6 +741,16 @@ static void felix_port_set_host_flood(struct dsa_switch *ds, int port, !!felix->host_flood_mc_mask, true); } +static int felix_port_change_master(struct dsa_switch *ds, int port, + struct net_device *master, + struct netlink_ext_ack *extack) +{ + struct ocelot *ocelot = ds->priv; + struct felix *felix = ocelot_to_felix(ocelot); + + return felix->tag_proto_ops->change_master(ds, port, master, extack); +} + static int felix_set_ageing_time(struct dsa_switch *ds, unsigned int ageing_time) { @@ -855,11 +939,21 @@ static void felix_bridge_leave(struct dsa_switch *ds, int port, static int felix_lag_join(struct dsa_switch *ds, int port, struct dsa_lag lag, - struct netdev_lag_upper_info *info) + struct netdev_lag_upper_info *info, + struct netlink_ext_ack *extack) { struct ocelot *ocelot = ds->priv; + int err; + + err = ocelot_port_lag_join(ocelot, port, lag.dev, info, extack); + if (err) + return err; + + /* Update the logical LAG port that serves as tag_8021q CPU port */ + if (!dsa_is_cpu_port(ds, port)) + return 0; - return ocelot_port_lag_join(ocelot, port, lag.dev, info); + return felix_port_change_master(ds, port, lag.dev, extack); } static int felix_lag_leave(struct dsa_switch *ds, int port, @@ -869,7 +963,11 @@ static int felix_lag_leave(struct dsa_switch *ds, int port, ocelot_port_lag_leave(ocelot, port, lag.dev); - return 0; + /* Update the logical LAG port that serves as tag_8021q CPU port */ + if (!dsa_is_cpu_port(ds, port)) + return 0; + + return felix_port_change_master(ds, port, lag.dev, NULL); } static int felix_lag_change(struct dsa_switch *ds, int port) @@ -1007,6 +1105,27 @@ static void felix_phylink_mac_link_up(struct dsa_switch *ds, int port, felix->info->port_sched_speed_set(ocelot, port, speed); } +static int felix_port_enable(struct dsa_switch *ds, int port, + struct phy_device *phydev) +{ + struct dsa_port *dp = dsa_to_port(ds, port); + struct ocelot *ocelot = ds->priv; + + if (!dsa_port_is_user(dp)) + return 0; + + if (ocelot->npi >= 0) { + struct net_device *master = dsa_port_to_master(dp); + + if (felix_cpu_port_for_master(ds, master) != ocelot->npi) { + dev_err(ds->dev, "Multiple masters are not allowed\n"); + return -EINVAL; + } + } + + return 0; +} + static void felix_port_qos_map_init(struct ocelot *ocelot, int port) { int i; @@ -1028,6 +1147,55 @@ static void felix_port_qos_map_init(struct ocelot *ocelot, int port) } } +static void felix_get_stats64(struct dsa_switch *ds, int port, + struct rtnl_link_stats64 *stats) +{ + struct ocelot *ocelot = ds->priv; + + ocelot_port_get_stats64(ocelot, port, stats); +} + +static void felix_get_pause_stats(struct dsa_switch *ds, int port, + struct ethtool_pause_stats *pause_stats) +{ + struct ocelot *ocelot = ds->priv; + + ocelot_port_get_pause_stats(ocelot, port, pause_stats); +} + +static void felix_get_rmon_stats(struct dsa_switch *ds, int port, + struct ethtool_rmon_stats *rmon_stats, + const struct ethtool_rmon_hist_range **ranges) +{ + struct ocelot *ocelot = ds->priv; + + ocelot_port_get_rmon_stats(ocelot, port, rmon_stats, ranges); +} + +static void felix_get_eth_ctrl_stats(struct dsa_switch *ds, int port, + struct ethtool_eth_ctrl_stats *ctrl_stats) +{ + struct ocelot *ocelot = ds->priv; + + ocelot_port_get_eth_ctrl_stats(ocelot, port, ctrl_stats); +} + +static void felix_get_eth_mac_stats(struct dsa_switch *ds, int port, + struct ethtool_eth_mac_stats *mac_stats) +{ + struct ocelot *ocelot = ds->priv; + + ocelot_port_get_eth_mac_stats(ocelot, port, mac_stats); +} + +static void felix_get_eth_phy_stats(struct dsa_switch *ds, int port, + struct ethtool_eth_phy_stats *phy_stats) +{ + struct ocelot *ocelot = ds->priv; + + ocelot_port_get_eth_phy_stats(ocelot, port, phy_stats); +} + static void felix_get_strings(struct dsa_switch *ds, int port, u32 stringset, u8 *data) { @@ -1144,11 +1312,55 @@ static int felix_parse_dt(struct felix *felix, phy_interface_t *port_phy_modes) return err; } +static struct regmap *felix_request_regmap_by_name(struct felix *felix, + const char *resource_name) +{ + struct ocelot *ocelot = &felix->ocelot; + struct resource res; + int i; + + for (i = 0; i < felix->info->num_resources; i++) { + if (strcmp(resource_name, felix->info->resources[i].name)) + continue; + + memcpy(&res, &felix->info->resources[i], sizeof(res)); + res.start += felix->switch_base; + res.end += felix->switch_base; + + return ocelot_regmap_init(ocelot, &res); + } + + return ERR_PTR(-ENOENT); +} + +static struct regmap *felix_request_regmap(struct felix *felix, + enum ocelot_target target) +{ + const char *resource_name = felix->info->resource_names[target]; + + /* If the driver didn't provide a resource name for the target, + * the resource is optional. + */ + if (!resource_name) + return NULL; + + return felix_request_regmap_by_name(felix, resource_name); +} + +static struct regmap *felix_request_port_regmap(struct felix *felix, int port) +{ + char resource_name[32]; + + sprintf(resource_name, "port%d", port); + + return felix_request_regmap_by_name(felix, resource_name); +} + static int felix_init_structs(struct felix *felix, int num_phys_ports) { struct ocelot *ocelot = &felix->ocelot; phy_interface_t *port_phy_modes; - struct resource res; + struct regmap *target; int port, i, err; ocelot->num_phys_ports = num_phys_ports; @@ -1182,20 +1394,11 @@ static int felix_init_structs(struct felix *felix, int num_phys_ports) } for (i = 0; i < TARGET_MAX; i++) { - struct regmap *target; - - if (!felix->info->target_io_res[i].name) - continue; - - memcpy(&res, &felix->info->target_io_res[i], sizeof(res)); - res.flags = IORESOURCE_MEM; - res.start += felix->switch_base; - res.end += felix->switch_base; - - target = felix->info->init_regmap(ocelot, &res); + target = felix_request_regmap(felix, i); if (IS_ERR(target)) { dev_err(ocelot->dev, - "Failed to map device memory space\n"); + "Failed to map device memory space: %pe\n", + target); kfree(port_phy_modes); return PTR_ERR(target); } @@ -1212,7 +1415,6 @@ static int felix_init_structs(struct felix *felix, int num_phys_ports) for (port = 0; port < num_phys_ports; port++) { struct ocelot_port *ocelot_port; - struct regmap *target; ocelot_port = devm_kzalloc(ocelot->dev, sizeof(struct ocelot_port), @@ -1224,16 +1426,11 @@ static int felix_init_structs(struct felix *felix, int num_phys_ports) return -ENOMEM; } - memcpy(&res, &felix->info->port_io_res[port], sizeof(res)); - res.flags = IORESOURCE_MEM; - res.start += felix->switch_base; - res.end += felix->switch_base; - - target = felix->info->init_regmap(ocelot, &res); + target = felix_request_port_regmap(felix, port); if (IS_ERR(target)) { dev_err(ocelot->dev, - "Failed to map memory space for port %d\n", - port); + "Failed to map memory space for port %d: %pe\n", + port, target); kfree(port_phy_modes); return PTR_ERR(target); } @@ -1842,6 +2039,12 @@ const struct dsa_switch_ops felix_switch_ops = { .setup = felix_setup, .teardown = felix_teardown, .set_ageing_time = felix_set_ageing_time, + .get_stats64 = felix_get_stats64, + .get_pause_stats = felix_get_pause_stats, + .get_rmon_stats = felix_get_rmon_stats, + .get_eth_ctrl_stats = felix_get_eth_ctrl_stats, + .get_eth_mac_stats = felix_get_eth_mac_stats, + .get_eth_phy_stats = felix_get_eth_phy_stats, .get_strings = felix_get_strings, .get_ethtool_stats = felix_get_ethtool_stats, .get_sset_count = felix_get_sset_count, @@ -1851,6 +2054,7 @@ const struct dsa_switch_ops felix_switch_ops = { .phylink_mac_select_pcs = felix_phylink_mac_select_pcs, .phylink_mac_link_down = felix_phylink_mac_link_down, .phylink_mac_link_up = felix_phylink_mac_link_up, + .port_enable = felix_port_enable, .port_fast_age = felix_port_fast_age, .port_fdb_dump = felix_fdb_dump, .port_fdb_add = felix_fdb_add, @@ -1906,6 +2110,7 @@ const struct dsa_switch_ops felix_switch_ops = { .port_add_dscp_prio = felix_port_add_dscp_prio, .port_del_dscp_prio = felix_port_del_dscp_prio, .port_set_host_flood = felix_port_set_host_flood, + .port_change_master = felix_port_change_master, }; struct net_device *felix_port_to_netdev(struct ocelot *ocelot, int port) diff --git a/drivers/net/dsa/ocelot/felix.h b/drivers/net/dsa/ocelot/felix.h index deb8dde1fc19..c9c29999c336 100644 --- a/drivers/net/dsa/ocelot/felix.h +++ b/drivers/net/dsa/ocelot/felix.h @@ -16,9 +16,13 @@ /* Platform-specific information */ struct felix_info { - const struct resource *target_io_res; - const struct resource *port_io_res; - const struct resource *imdio_res; + /* Hardcoded resources provided by the hardware instantiation. */ + const struct resource *resources; + size_t num_resources; + /* Names of the mandatory resources that will be requested during + * probe. Must have TARGET_MAX elements, since it is indexed by target. + */ + const char *const *resource_names; const struct reg_field *regfields; const u32 *const *map; const struct ocelot_ops *ops; @@ -56,8 +60,6 @@ struct felix_info { void (*tas_guard_bands_update)(struct ocelot *ocelot, int port); void (*port_sched_speed_set)(struct ocelot *ocelot, int port, u32 speed); - struct regmap *(*init_regmap)(struct ocelot *ocelot, - struct resource *res); }; /* Methods for initializing the hardware resources specific to a tagging @@ -71,6 +73,9 @@ struct felix_tag_proto_ops { int (*setup)(struct dsa_switch *ds); void (*teardown)(struct dsa_switch *ds); unsigned long (*get_host_fwd_mask)(struct dsa_switch *ds); + int (*change_master)(struct dsa_switch *ds, int port, + struct net_device *master, + struct netlink_ext_ack *extack); }; extern const struct dsa_switch_ops felix_switch_ops; @@ -83,7 +88,6 @@ struct felix { struct mii_bus *imdio; struct phylink_pcs **pcs; resource_size_t switch_base; - resource_size_t imdio_base; enum dsa_tag_protocol tag_proto; const struct felix_tag_proto_ops *tag_proto_ops; struct kthread_worker *xmit_worker; diff --git a/drivers/net/dsa/ocelot/felix_vsc9959.c b/drivers/net/dsa/ocelot/felix_vsc9959.c index f8f19a85744c..26a35ae322d1 100644 --- a/drivers/net/dsa/ocelot/felix_vsc9959.c +++ b/drivers/net/dsa/ocelot/felix_vsc9959.c @@ -348,7 +348,7 @@ static const u32 vsc9959_sys_regmap[] = { REG(SYS_COUNT_TX_GREEN_PRIO_5, 0x00026c), REG(SYS_COUNT_TX_GREEN_PRIO_6, 0x000270), REG(SYS_COUNT_TX_GREEN_PRIO_7, 0x000274), - REG(SYS_COUNT_TX_AGING, 0x000278), + REG(SYS_COUNT_TX_AGED, 0x000278), REG(SYS_COUNT_DROP_LOCAL, 0x000400), REG(SYS_COUNT_DROP_TAIL, 0x000404), REG(SYS_COUNT_DROP_YELLOW_PRIO_0, 0x000408), @@ -367,6 +367,10 @@ static const u32 vsc9959_sys_regmap[] = { REG(SYS_COUNT_DROP_GREEN_PRIO_5, 0x00043c), REG(SYS_COUNT_DROP_GREEN_PRIO_6, 0x000440), REG(SYS_COUNT_DROP_GREEN_PRIO_7, 0x000444), + REG(SYS_COUNT_SF_MATCHING_FRAMES, 0x000800), + REG(SYS_COUNT_SF_NOT_PASSING_FRAMES, 0x000804), + REG(SYS_COUNT_SF_NOT_PASSING_SDU, 0x000808), + REG(SYS_COUNT_SF_RED_FRAMES, 0x00080c), REG(SYS_RESET_CFG, 0x000e00), REG(SYS_SR_ETYPE_CFG, 0x000e04), REG(SYS_VLAN_ETYPE_CFG, 0x000e08), @@ -388,7 +392,6 @@ static const u32 vsc9959_sys_regmap[] = { REG_RESERVED(SYS_MMGT_FAST), REG_RESERVED(SYS_EVENTS_DIF), REG_RESERVED(SYS_EVENTS_CORE), - REG(SYS_CNT, 0x000000), REG(SYS_PTP_STATUS, 0x000f14), REG(SYS_PTP_TXSTAMP, 0x000f18), REG(SYS_PTP_NXT, 0x000f1c), @@ -474,100 +477,43 @@ static const u32 *vsc9959_regmap[TARGET_MAX] = { }; /* Addresses are relative to the PCI device's base address */ -static const struct resource vsc9959_target_io_res[TARGET_MAX] = { - [ANA] = { - .start = 0x0280000, - .end = 0x028ffff, - .name = "ana", - }, - [QS] = { - .start = 0x0080000, - .end = 0x00800ff, - .name = "qs", - }, - [QSYS] = { - .start = 0x0200000, - .end = 0x021ffff, - .name = "qsys", - }, - [REW] = { - .start = 0x0030000, - .end = 0x003ffff, - .name = "rew", - }, - [SYS] = { - .start = 0x0010000, - .end = 0x001ffff, - .name = "sys", - }, - [S0] = { - .start = 0x0040000, - .end = 0x00403ff, - .name = "s0", - }, - [S1] = { - .start = 0x0050000, - .end = 0x00503ff, - .name = "s1", - }, - [S2] = { - .start = 0x0060000, - .end = 0x00603ff, - .name = "s2", - }, - [PTP] = { - .start = 0x0090000, - .end = 0x00900cb, - .name = "ptp", - }, - [GCB] = { - .start = 0x0070000, - .end = 0x00701ff, - .name = "devcpu_gcb", - }, +static const struct resource vsc9959_resources[] = { + DEFINE_RES_MEM_NAMED(0x0010000, 0x0010000, "sys"), + DEFINE_RES_MEM_NAMED(0x0030000, 0x0010000, "rew"), + DEFINE_RES_MEM_NAMED(0x0040000, 0x0000400, "s0"), + DEFINE_RES_MEM_NAMED(0x0050000, 0x0000400, "s1"), + DEFINE_RES_MEM_NAMED(0x0060000, 0x0000400, "s2"), + DEFINE_RES_MEM_NAMED(0x0070000, 0x0000200, "devcpu_gcb"), + DEFINE_RES_MEM_NAMED(0x0080000, 0x0000100, "qs"), + DEFINE_RES_MEM_NAMED(0x0090000, 0x00000cc, "ptp"), + DEFINE_RES_MEM_NAMED(0x0100000, 0x0010000, "port0"), + DEFINE_RES_MEM_NAMED(0x0110000, 0x0010000, "port1"), + DEFINE_RES_MEM_NAMED(0x0120000, 0x0010000, "port2"), + DEFINE_RES_MEM_NAMED(0x0130000, 0x0010000, "port3"), + DEFINE_RES_MEM_NAMED(0x0140000, 0x0010000, "port4"), + DEFINE_RES_MEM_NAMED(0x0150000, 0x0010000, "port5"), + DEFINE_RES_MEM_NAMED(0x0200000, 0x0020000, "qsys"), + DEFINE_RES_MEM_NAMED(0x0280000, 0x0010000, "ana"), }; -static const struct resource vsc9959_port_io_res[] = { - { - .start = 0x0100000, - .end = 0x010ffff, - .name = "port0", - }, - { - .start = 0x0110000, - .end = 0x011ffff, - .name = "port1", - }, - { - .start = 0x0120000, - .end = 0x012ffff, - .name = "port2", - }, - { - .start = 0x0130000, - .end = 0x013ffff, - .name = "port3", - }, - { - .start = 0x0140000, - .end = 0x014ffff, - .name = "port4", - }, - { - .start = 0x0150000, - .end = 0x015ffff, - .name = "port5", - }, +static const char * const vsc9959_resource_names[TARGET_MAX] = { + [SYS] = "sys", + [REW] = "rew", + [S0] = "s0", + [S1] = "s1", + [S2] = "s2", + [GCB] = "devcpu_gcb", + [QS] = "qs", + [PTP] = "ptp", + [QSYS] = "qsys", + [ANA] = "ana", }; /* Port MAC 0 Internal MDIO bus through which the SerDes acting as an * SGMII/QSGMII MAC PCS can be found. */ -static const struct resource vsc9959_imdio_res = { - .start = 0x8030, - .end = 0x8040, - .name = "imdio", -}; +static const struct resource vsc9959_imdio_res = + DEFINE_RES_MEM_NAMED(0x8030, 0x8040, "imdio"); static const struct reg_field vsc9959_regfields[REGFIELD_MAX] = { [ANA_ADVLEARN_VLAN_CHK] = REG_FIELD(ANA_ADVLEARN, 6, 6), @@ -620,378 +566,7 @@ static const struct reg_field vsc9959_regfields[REGFIELD_MAX] = { }; static const struct ocelot_stat_layout vsc9959_stats_layout[OCELOT_NUM_STATS] = { - [OCELOT_STAT_RX_OCTETS] = { - .name = "rx_octets", - .reg = SYS_COUNT_RX_OCTETS, - }, - [OCELOT_STAT_RX_UNICAST] = { - .name = "rx_unicast", - .reg = SYS_COUNT_RX_UNICAST, - }, - [OCELOT_STAT_RX_MULTICAST] = { - .name = "rx_multicast", - .reg = SYS_COUNT_RX_MULTICAST, - }, - [OCELOT_STAT_RX_BROADCAST] = { - .name = "rx_broadcast", - .reg = SYS_COUNT_RX_BROADCAST, - }, - [OCELOT_STAT_RX_SHORTS] = { - .name = "rx_shorts", - .reg = SYS_COUNT_RX_SHORTS, - }, - [OCELOT_STAT_RX_FRAGMENTS] = { - .name = "rx_fragments", - .reg = SYS_COUNT_RX_FRAGMENTS, - }, - [OCELOT_STAT_RX_JABBERS] = { - .name = "rx_jabbers", - .reg = SYS_COUNT_RX_JABBERS, - }, - [OCELOT_STAT_RX_CRC_ALIGN_ERRS] = { - .name = "rx_crc_align_errs", - .reg = SYS_COUNT_RX_CRC_ALIGN_ERRS, - }, - [OCELOT_STAT_RX_SYM_ERRS] = { - .name = "rx_sym_errs", - .reg = SYS_COUNT_RX_SYM_ERRS, - }, - [OCELOT_STAT_RX_64] = { - .name = "rx_frames_below_65_octets", - .reg = SYS_COUNT_RX_64, - }, - [OCELOT_STAT_RX_65_127] = { - .name = "rx_frames_65_to_127_octets", - .reg = SYS_COUNT_RX_65_127, - }, - [OCELOT_STAT_RX_128_255] = { - .name = "rx_frames_128_to_255_octets", - .reg = SYS_COUNT_RX_128_255, - }, - [OCELOT_STAT_RX_256_511] = { - .name = "rx_frames_256_to_511_octets", - .reg = SYS_COUNT_RX_256_511, - }, - [OCELOT_STAT_RX_512_1023] = { - .name = "rx_frames_512_to_1023_octets", - .reg = SYS_COUNT_RX_512_1023, - }, - [OCELOT_STAT_RX_1024_1526] = { - .name = "rx_frames_1024_to_1526_octets", - .reg = SYS_COUNT_RX_1024_1526, - }, - [OCELOT_STAT_RX_1527_MAX] = { - .name = "rx_frames_over_1526_octets", - .reg = SYS_COUNT_RX_1527_MAX, - }, - [OCELOT_STAT_RX_PAUSE] = { - .name = "rx_pause", - .reg = SYS_COUNT_RX_PAUSE, - }, - [OCELOT_STAT_RX_CONTROL] = { - .name = "rx_control", - .reg = SYS_COUNT_RX_CONTROL, - }, - [OCELOT_STAT_RX_LONGS] = { - .name = "rx_longs", - .reg = SYS_COUNT_RX_LONGS, - }, - [OCELOT_STAT_RX_CLASSIFIED_DROPS] = { - .name = "rx_classified_drops", - .reg = SYS_COUNT_RX_CLASSIFIED_DROPS, - }, - [OCELOT_STAT_RX_RED_PRIO_0] = { - .name = "rx_red_prio_0", - .reg = SYS_COUNT_RX_RED_PRIO_0, - }, - [OCELOT_STAT_RX_RED_PRIO_1] = { - .name = "rx_red_prio_1", - .reg = SYS_COUNT_RX_RED_PRIO_1, - }, - [OCELOT_STAT_RX_RED_PRIO_2] = { - .name = "rx_red_prio_2", - .reg = SYS_COUNT_RX_RED_PRIO_2, - }, - [OCELOT_STAT_RX_RED_PRIO_3] = { - .name = "rx_red_prio_3", - .reg = SYS_COUNT_RX_RED_PRIO_3, - }, - [OCELOT_STAT_RX_RED_PRIO_4] = { - .name = "rx_red_prio_4", - .reg = SYS_COUNT_RX_RED_PRIO_4, - }, - [OCELOT_STAT_RX_RED_PRIO_5] = { - .name = "rx_red_prio_5", - .reg = SYS_COUNT_RX_RED_PRIO_5, - }, - [OCELOT_STAT_RX_RED_PRIO_6] = { - .name = "rx_red_prio_6", - .reg = SYS_COUNT_RX_RED_PRIO_6, - }, - [OCELOT_STAT_RX_RED_PRIO_7] = { - .name = "rx_red_prio_7", - .reg = SYS_COUNT_RX_RED_PRIO_7, - }, - [OCELOT_STAT_RX_YELLOW_PRIO_0] = { - .name = "rx_yellow_prio_0", - .reg = SYS_COUNT_RX_YELLOW_PRIO_0, - }, - [OCELOT_STAT_RX_YELLOW_PRIO_1] = { - .name = "rx_yellow_prio_1", - .reg = SYS_COUNT_RX_YELLOW_PRIO_1, - }, - [OCELOT_STAT_RX_YELLOW_PRIO_2] = { - .name = "rx_yellow_prio_2", - .reg = SYS_COUNT_RX_YELLOW_PRIO_2, - }, - [OCELOT_STAT_RX_YELLOW_PRIO_3] = { - .name = "rx_yellow_prio_3", - .reg = SYS_COUNT_RX_YELLOW_PRIO_3, - }, - [OCELOT_STAT_RX_YELLOW_PRIO_4] = { - .name = "rx_yellow_prio_4", - .reg = SYS_COUNT_RX_YELLOW_PRIO_4, - }, - [OCELOT_STAT_RX_YELLOW_PRIO_5] = { - .name = "rx_yellow_prio_5", - .reg = SYS_COUNT_RX_YELLOW_PRIO_5, - }, - [OCELOT_STAT_RX_YELLOW_PRIO_6] = { - .name = "rx_yellow_prio_6", - .reg = SYS_COUNT_RX_YELLOW_PRIO_6, - }, - [OCELOT_STAT_RX_YELLOW_PRIO_7] = { - .name = "rx_yellow_prio_7", - .reg = SYS_COUNT_RX_YELLOW_PRIO_7, - }, - [OCELOT_STAT_RX_GREEN_PRIO_0] = { - .name = "rx_green_prio_0", - .reg = SYS_COUNT_RX_GREEN_PRIO_0, - }, - [OCELOT_STAT_RX_GREEN_PRIO_1] = { - .name = "rx_green_prio_1", - .reg = SYS_COUNT_RX_GREEN_PRIO_1, - }, - [OCELOT_STAT_RX_GREEN_PRIO_2] = { - .name = "rx_green_prio_2", - .reg = SYS_COUNT_RX_GREEN_PRIO_2, - }, - [OCELOT_STAT_RX_GREEN_PRIO_3] = { - .name = "rx_green_prio_3", - .reg = SYS_COUNT_RX_GREEN_PRIO_3, - }, - [OCELOT_STAT_RX_GREEN_PRIO_4] = { - .name = "rx_green_prio_4", - .reg = SYS_COUNT_RX_GREEN_PRIO_4, - }, - [OCELOT_STAT_RX_GREEN_PRIO_5] = { - .name = "rx_green_prio_5", - .reg = SYS_COUNT_RX_GREEN_PRIO_5, - }, - [OCELOT_STAT_RX_GREEN_PRIO_6] = { - .name = "rx_green_prio_6", - .reg = SYS_COUNT_RX_GREEN_PRIO_6, - }, - [OCELOT_STAT_RX_GREEN_PRIO_7] = { - .name = "rx_green_prio_7", - .reg = SYS_COUNT_RX_GREEN_PRIO_7, - }, - [OCELOT_STAT_TX_OCTETS] = { - .name = "tx_octets", - .reg = SYS_COUNT_TX_OCTETS, - }, - [OCELOT_STAT_TX_UNICAST] = { - .name = "tx_unicast", - .reg = SYS_COUNT_TX_UNICAST, - }, - [OCELOT_STAT_TX_MULTICAST] = { - .name = "tx_multicast", - .reg = SYS_COUNT_TX_MULTICAST, - }, - [OCELOT_STAT_TX_BROADCAST] = { - .name = "tx_broadcast", - .reg = SYS_COUNT_TX_BROADCAST, - }, - [OCELOT_STAT_TX_COLLISION] = { - .name = "tx_collision", - .reg = SYS_COUNT_TX_COLLISION, - }, - [OCELOT_STAT_TX_DROPS] = { - .name = "tx_drops", - .reg = SYS_COUNT_TX_DROPS, - }, - [OCELOT_STAT_TX_PAUSE] = { - .name = "tx_pause", - .reg = SYS_COUNT_TX_PAUSE, - }, - [OCELOT_STAT_TX_64] = { - .name = "tx_frames_below_65_octets", - .reg = SYS_COUNT_TX_64, - }, - [OCELOT_STAT_TX_65_127] = { - .name = "tx_frames_65_to_127_octets", - .reg = SYS_COUNT_TX_65_127, - }, - [OCELOT_STAT_TX_128_255] = { - .name = "tx_frames_128_255_octets", - .reg = SYS_COUNT_TX_128_255, - }, - [OCELOT_STAT_TX_256_511] = { - .name = "tx_frames_256_511_octets", - .reg = SYS_COUNT_TX_256_511, - }, - [OCELOT_STAT_TX_512_1023] = { - .name = "tx_frames_512_1023_octets", - .reg = SYS_COUNT_TX_512_1023, - }, - [OCELOT_STAT_TX_1024_1526] = { - .name = "tx_frames_1024_1526_octets", - .reg = SYS_COUNT_TX_1024_1526, - }, - [OCELOT_STAT_TX_1527_MAX] = { - .name = "tx_frames_over_1526_octets", - .reg = SYS_COUNT_TX_1527_MAX, - }, - [OCELOT_STAT_TX_YELLOW_PRIO_0] = { - .name = "tx_yellow_prio_0", - .reg = SYS_COUNT_TX_YELLOW_PRIO_0, - }, - [OCELOT_STAT_TX_YELLOW_PRIO_1] = { - .name = "tx_yellow_prio_1", - .reg = SYS_COUNT_TX_YELLOW_PRIO_1, - }, - [OCELOT_STAT_TX_YELLOW_PRIO_2] = { - .name = "tx_yellow_prio_2", - .reg = SYS_COUNT_TX_YELLOW_PRIO_2, - }, - [OCELOT_STAT_TX_YELLOW_PRIO_3] = { - .name = "tx_yellow_prio_3", - .reg = SYS_COUNT_TX_YELLOW_PRIO_3, - }, - [OCELOT_STAT_TX_YELLOW_PRIO_4] = { - .name = "tx_yellow_prio_4", - .reg = SYS_COUNT_TX_YELLOW_PRIO_4, - }, - [OCELOT_STAT_TX_YELLOW_PRIO_5] = { - .name = "tx_yellow_prio_5", - .reg = SYS_COUNT_TX_YELLOW_PRIO_5, - }, - [OCELOT_STAT_TX_YELLOW_PRIO_6] = { - .name = "tx_yellow_prio_6", - .reg = SYS_COUNT_TX_YELLOW_PRIO_6, - }, - [OCELOT_STAT_TX_YELLOW_PRIO_7] = { - .name = "tx_yellow_prio_7", - .reg = SYS_COUNT_TX_YELLOW_PRIO_7, - }, - [OCELOT_STAT_TX_GREEN_PRIO_0] = { - .name = "tx_green_prio_0", - .reg = SYS_COUNT_TX_GREEN_PRIO_0, - }, - [OCELOT_STAT_TX_GREEN_PRIO_1] = { - .name = "tx_green_prio_1", - .reg = SYS_COUNT_TX_GREEN_PRIO_1, - }, - [OCELOT_STAT_TX_GREEN_PRIO_2] = { - .name = "tx_green_prio_2", - .reg = SYS_COUNT_TX_GREEN_PRIO_2, - }, - [OCELOT_STAT_TX_GREEN_PRIO_3] = { - .name = "tx_green_prio_3", - .reg = SYS_COUNT_TX_GREEN_PRIO_3, - }, - [OCELOT_STAT_TX_GREEN_PRIO_4] = { - .name = "tx_green_prio_4", - .reg = SYS_COUNT_TX_GREEN_PRIO_4, - }, - [OCELOT_STAT_TX_GREEN_PRIO_5] = { - .name = "tx_green_prio_5", - .reg = SYS_COUNT_TX_GREEN_PRIO_5, - }, - [OCELOT_STAT_TX_GREEN_PRIO_6] = { - .name = "tx_green_prio_6", - .reg = SYS_COUNT_TX_GREEN_PRIO_6, - }, - [OCELOT_STAT_TX_GREEN_PRIO_7] = { - .name = "tx_green_prio_7", - .reg = SYS_COUNT_TX_GREEN_PRIO_7, - }, - [OCELOT_STAT_TX_AGED] = { - .name = "tx_aged", - .reg = SYS_COUNT_TX_AGING, - }, - [OCELOT_STAT_DROP_LOCAL] = { - .name = "drop_local", - .reg = SYS_COUNT_DROP_LOCAL, - }, - [OCELOT_STAT_DROP_TAIL] = { - .name = "drop_tail", - .reg = SYS_COUNT_DROP_TAIL, - }, - [OCELOT_STAT_DROP_YELLOW_PRIO_0] = { - .name = "drop_yellow_prio_0", - .reg = SYS_COUNT_DROP_YELLOW_PRIO_0, - }, - [OCELOT_STAT_DROP_YELLOW_PRIO_1] = { - .name = "drop_yellow_prio_1", - .reg = SYS_COUNT_DROP_YELLOW_PRIO_1, - }, - [OCELOT_STAT_DROP_YELLOW_PRIO_2] = { - .name = "drop_yellow_prio_2", - .reg = SYS_COUNT_DROP_YELLOW_PRIO_2, - }, - [OCELOT_STAT_DROP_YELLOW_PRIO_3] = { - .name = "drop_yellow_prio_3", - .reg = SYS_COUNT_DROP_YELLOW_PRIO_3, - }, - [OCELOT_STAT_DROP_YELLOW_PRIO_4] = { - .name = "drop_yellow_prio_4", - .reg = SYS_COUNT_DROP_YELLOW_PRIO_4, - }, - [OCELOT_STAT_DROP_YELLOW_PRIO_5] = { - .name = "drop_yellow_prio_5", - .reg = SYS_COUNT_DROP_YELLOW_PRIO_5, - }, - [OCELOT_STAT_DROP_YELLOW_PRIO_6] = { - .name = "drop_yellow_prio_6", - .reg = SYS_COUNT_DROP_YELLOW_PRIO_6, - }, - [OCELOT_STAT_DROP_YELLOW_PRIO_7] = { - .name = "drop_yellow_prio_7", - .reg = SYS_COUNT_DROP_YELLOW_PRIO_7, - }, - [OCELOT_STAT_DROP_GREEN_PRIO_0] = { - .name = "drop_green_prio_0", - .reg = SYS_COUNT_DROP_GREEN_PRIO_0, - }, - [OCELOT_STAT_DROP_GREEN_PRIO_1] = { - .name = "drop_green_prio_1", - .reg = SYS_COUNT_DROP_GREEN_PRIO_1, - }, - [OCELOT_STAT_DROP_GREEN_PRIO_2] = { - .name = "drop_green_prio_2", - .reg = SYS_COUNT_DROP_GREEN_PRIO_2, - }, - [OCELOT_STAT_DROP_GREEN_PRIO_3] = { - .name = "drop_green_prio_3", - .reg = SYS_COUNT_DROP_GREEN_PRIO_3, - }, - [OCELOT_STAT_DROP_GREEN_PRIO_4] = { - .name = "drop_green_prio_4", - .reg = SYS_COUNT_DROP_GREEN_PRIO_4, - }, - [OCELOT_STAT_DROP_GREEN_PRIO_5] = { - .name = "drop_green_prio_5", - .reg = SYS_COUNT_DROP_GREEN_PRIO_5, - }, - [OCELOT_STAT_DROP_GREEN_PRIO_6] = { - .name = "drop_green_prio_6", - .reg = SYS_COUNT_DROP_GREEN_PRIO_6, - }, - [OCELOT_STAT_DROP_GREEN_PRIO_7] = { - .name = "drop_green_prio_7", - .reg = SYS_COUNT_DROP_GREEN_PRIO_7, - }, + OCELOT_COMMON_STATS, }; static const struct vcap_field vsc9959_vcap_es0_keys[] = { @@ -1371,9 +946,11 @@ static void vsc9959_wm_stat(u32 val, u32 *inuse, u32 *maxuse) static int vsc9959_mdio_bus_alloc(struct ocelot *ocelot) { + struct pci_dev *pdev = to_pci_dev(ocelot->dev); struct felix *felix = ocelot_to_felix(ocelot); struct enetc_mdio_priv *mdio_priv; struct device *dev = ocelot->dev; + resource_size_t imdio_base; void __iomem *imdio_regs; struct resource res; struct enetc_hw *hw; @@ -1389,10 +966,11 @@ static int vsc9959_mdio_bus_alloc(struct ocelot *ocelot) return -ENOMEM; } - memcpy(&res, felix->info->imdio_res, sizeof(res)); - res.flags = IORESOURCE_MEM; - res.start += felix->imdio_base; - res.end += felix->imdio_base; + imdio_base = pci_resource_start(pdev, VSC9959_IMDIO_PCI_BAR); + + memcpy(&res, &vsc9959_imdio_res, sizeof(res)); + res.start += imdio_base; + res.end += imdio_base; imdio_regs = devm_ioremap_resource(dev, &res); if (IS_ERR(imdio_regs)) @@ -1616,6 +1194,14 @@ static u32 vsc9959_port_qmaxsdu_get(struct ocelot *ocelot, int port, int tc) } } +static u32 vsc9959_tas_tc_max_sdu(struct tc_taprio_qopt_offload *taprio, int tc) +{ + if (!taprio || !taprio->max_sdu[tc]) + return 0; + + return taprio->max_sdu[tc] + ETH_HLEN + 2 * VLAN_HLEN + ETH_FCS_LEN; +} + /* Update QSYS_PORT_MAX_SDU to make sure the static guard bands added by the * switch (see the ALWAYS_GUARD_BAND_SCH_Q comment) are correct at all MTU * values (the default value is 1518). Also, for traffic class windows smaller @@ -1625,6 +1211,7 @@ static u32 vsc9959_port_qmaxsdu_get(struct ocelot *ocelot, int port, int tc) static void vsc9959_tas_guard_bands_update(struct ocelot *ocelot, int port) { struct ocelot_port *ocelot_port = ocelot->ports[port]; + struct tc_taprio_qopt_offload *taprio; u64 min_gate_len[OCELOT_NUM_TC]; int speed, picos_per_byte; u64 needed_bit_time_ps; @@ -1634,6 +1221,8 @@ static void vsc9959_tas_guard_bands_update(struct ocelot *ocelot, int port) lockdep_assert_held(&ocelot->tas_lock); + taprio = ocelot_port->taprio; + val = ocelot_read_rix(ocelot, QSYS_TAG_CONFIG, port); tas_speed = QSYS_TAG_CONFIG_LINK_SPEED_X(val); @@ -1670,11 +1259,12 @@ static void vsc9959_tas_guard_bands_update(struct ocelot *ocelot, int port) "port %d: max frame size %d needs %llu ps at speed %d\n", port, maxlen, needed_bit_time_ps, speed); - vsc9959_tas_min_gate_lengths(ocelot_port->taprio, min_gate_len); + vsc9959_tas_min_gate_lengths(taprio, min_gate_len); mutex_lock(&ocelot->fwd_domain_lock); for (tc = 0; tc < OCELOT_NUM_TC; tc++) { + u32 requested_max_sdu = vsc9959_tas_tc_max_sdu(taprio, tc); u64 remaining_gate_len_ps; u32 max_sdu; @@ -1685,7 +1275,7 @@ static void vsc9959_tas_guard_bands_update(struct ocelot *ocelot, int port) /* Setting QMAXSDU_CFG to 0 disables oversized frame * dropping. */ - max_sdu = 0; + max_sdu = requested_max_sdu; dev_dbg(ocelot->dev, "port %d tc %d min gate len %llu" ", sending all frames\n", @@ -1716,6 +1306,10 @@ static void vsc9959_tas_guard_bands_update(struct ocelot *ocelot, int port) */ if (max_sdu > 20) max_sdu -= 20; + + if (requested_max_sdu && requested_max_sdu < max_sdu) + max_sdu = requested_max_sdu; + dev_info(ocelot->dev, "port %d tc %d min gate length %llu" " ns not enough for max frame size %d at %d" @@ -2005,6 +1599,21 @@ static int vsc9959_qos_port_cbs_set(struct dsa_switch *ds, int port, return 0; } +static int vsc9959_qos_query_caps(struct tc_query_caps_base *base) +{ + switch (base->type) { + case TC_SETUP_QDISC_TAPRIO: { + struct tc_taprio_caps *caps = base->caps; + + caps->supports_queue_max_sdu = true; + + return 0; + } + default: + return -EOPNOTSUPP; + } +} + static int vsc9959_port_setup_tc(struct dsa_switch *ds, int port, enum tc_setup_type type, void *type_data) @@ -2012,6 +1621,8 @@ static int vsc9959_port_setup_tc(struct dsa_switch *ds, int port, struct ocelot *ocelot = ds->priv; switch (type) { + case TC_QUERY_CAPS: + return vsc9959_qos_query_caps(type_data); case TC_SETUP_QDISC_TAPRIO: return vsc9959_qos_port_tas_set(ocelot, port, type_data); case TC_SETUP_QDISC_CBS: @@ -2043,7 +1654,15 @@ struct felix_stream { u32 ssid; }; +struct felix_stream_filter_counters { + u64 match; + u64 not_pass_gate; + u64 not_pass_sdu; + u64 red; +}; + struct felix_stream_filter { + struct felix_stream_filter_counters stats; struct list_head list; refcount_t refcount; u32 index; @@ -2058,13 +1677,6 @@ struct felix_stream_filter { u32 maxsdu; }; -struct felix_stream_filter_counters { - u32 match; - u32 not_pass_gate; - u32 not_pass_sdu; - u32 red; -}; - struct felix_stream_gate { u32 index; u8 enable; @@ -2568,29 +2180,6 @@ static void vsc9959_psfp_sgi_table_del(struct ocelot *ocelot, } } -static void vsc9959_psfp_counters_get(struct ocelot *ocelot, u32 index, - struct felix_stream_filter_counters *counters) -{ - spin_lock(&ocelot->stats_lock); - - ocelot_rmw(ocelot, SYS_STAT_CFG_STAT_VIEW(index), - SYS_STAT_CFG_STAT_VIEW_M, - SYS_STAT_CFG); - - counters->match = ocelot_read_gix(ocelot, SYS_CNT, 0x200); - counters->not_pass_gate = ocelot_read_gix(ocelot, SYS_CNT, 0x201); - counters->not_pass_sdu = ocelot_read_gix(ocelot, SYS_CNT, 0x202); - counters->red = ocelot_read_gix(ocelot, SYS_CNT, 0x203); - - /* Clear the PSFP counter. */ - ocelot_write(ocelot, - SYS_STAT_CFG_STAT_VIEW(index) | - SYS_STAT_CFG_STAT_CLEAR_SHOT(0x10), - SYS_STAT_CFG); - - spin_unlock(&ocelot->stats_lock); -} - static int vsc9959_psfp_filter_add(struct ocelot *ocelot, int port, struct flow_cls_offload *f) { @@ -2615,6 +2204,8 @@ static int vsc9959_psfp_filter_add(struct ocelot *ocelot, int port, return ret; } + mutex_lock(&psfp->lock); + flow_action_for_each(i, a, &f->rule->action) { switch (a->id) { case FLOW_ACTION_GATE: @@ -2656,6 +2247,7 @@ static int vsc9959_psfp_filter_add(struct ocelot *ocelot, int port, sfi.maxsdu = a->police.mtu; break; default: + mutex_unlock(&psfp->lock); return -EOPNOTSUPP; } } @@ -2725,6 +2317,8 @@ static int vsc9959_psfp_filter_add(struct ocelot *ocelot, int port, goto err; } + mutex_unlock(&psfp->lock); + return 0; err: @@ -2734,6 +2328,8 @@ err: if (sfi.fm_valid) ocelot_vcap_policer_del(ocelot, sfi.fmid); + mutex_unlock(&psfp->lock); + return ret; } @@ -2741,18 +2337,22 @@ static int vsc9959_psfp_filter_del(struct ocelot *ocelot, struct flow_cls_offload *f) { struct felix_stream *stream, tmp, *stream_entry; + struct ocelot_psfp_list *psfp = &ocelot->psfp; static struct felix_stream_filter *sfi; - struct ocelot_psfp_list *psfp; - psfp = &ocelot->psfp; + mutex_lock(&psfp->lock); stream = vsc9959_stream_table_get(&psfp->stream_list, f->cookie); - if (!stream) + if (!stream) { + mutex_unlock(&psfp->lock); return -ENOMEM; + } sfi = vsc9959_psfp_sfi_table_get(&psfp->sfi_list, stream->sfid); - if (!sfi) + if (!sfi) { + mutex_unlock(&psfp->lock); return -ENOMEM; + } if (sfi->sg_valid) vsc9959_psfp_sgi_table_del(ocelot, sfi->sgid); @@ -2778,27 +2378,83 @@ static int vsc9959_psfp_filter_del(struct ocelot *ocelot, stream_entry->ports); } + mutex_unlock(&psfp->lock); + return 0; } +static void vsc9959_update_sfid_stats(struct ocelot *ocelot, + struct felix_stream_filter *sfi) +{ + struct felix_stream_filter_counters *s = &sfi->stats; + u32 match, not_pass_gate, not_pass_sdu, red; + u32 sfid = sfi->index; + + lockdep_assert_held(&ocelot->stat_view_lock); + + ocelot_rmw(ocelot, SYS_STAT_CFG_STAT_VIEW(sfid), + SYS_STAT_CFG_STAT_VIEW_M, + SYS_STAT_CFG); + + match = ocelot_read(ocelot, SYS_COUNT_SF_MATCHING_FRAMES); + not_pass_gate = ocelot_read(ocelot, SYS_COUNT_SF_NOT_PASSING_FRAMES); + not_pass_sdu = ocelot_read(ocelot, SYS_COUNT_SF_NOT_PASSING_SDU); + red = ocelot_read(ocelot, SYS_COUNT_SF_RED_FRAMES); + + /* Clear the PSFP counter. */ + ocelot_write(ocelot, + SYS_STAT_CFG_STAT_VIEW(sfid) | + SYS_STAT_CFG_STAT_CLEAR_SHOT(0x10), + SYS_STAT_CFG); + + s->match += match; + s->not_pass_gate += not_pass_gate; + s->not_pass_sdu += not_pass_sdu; + s->red += red; +} + +/* Caller must hold &ocelot->stat_view_lock */ +static void vsc9959_update_stats(struct ocelot *ocelot) +{ + struct ocelot_psfp_list *psfp = &ocelot->psfp; + struct felix_stream_filter *sfi; + + mutex_lock(&psfp->lock); + + list_for_each_entry(sfi, &psfp->sfi_list, list) + vsc9959_update_sfid_stats(ocelot, sfi); + + mutex_unlock(&psfp->lock); +} + static int vsc9959_psfp_stats_get(struct ocelot *ocelot, struct flow_cls_offload *f, struct flow_stats *stats) { - struct felix_stream_filter_counters counters; - struct ocelot_psfp_list *psfp; + struct ocelot_psfp_list *psfp = &ocelot->psfp; + struct felix_stream_filter_counters *s; + static struct felix_stream_filter *sfi; struct felix_stream *stream; - psfp = &ocelot->psfp; stream = vsc9959_stream_table_get(&psfp->stream_list, f->cookie); if (!stream) return -ENOMEM; - vsc9959_psfp_counters_get(ocelot, stream->sfid, &counters); + sfi = vsc9959_psfp_sfi_table_get(&psfp->sfi_list, stream->sfid); + if (!sfi) + return -EINVAL; + + mutex_lock(&ocelot->stat_view_lock); + + vsc9959_update_sfid_stats(ocelot, sfi); + + s = &sfi->stats; + stats->pkts = s->match; + stats->drops = s->not_pass_gate + s->not_pass_sdu + s->red; - stats->pkts = counters.match; - stats->drops = counters.not_pass_gate + counters.not_pass_sdu + - counters.red; + memset(s, 0, sizeof(*s)); + + mutex_unlock(&ocelot->stat_view_lock); return 0; } @@ -2810,6 +2466,7 @@ static void vsc9959_psfp_init(struct ocelot *ocelot) INIT_LIST_HEAD(&psfp->stream_list); INIT_LIST_HEAD(&psfp->sfi_list); INIT_LIST_HEAD(&psfp->sgi_list); + mutex_init(&psfp->lock); } /* When using cut-through forwarding and the egress port runs at a higher data @@ -2908,12 +2565,13 @@ static const struct ocelot_ops vsc9959_ops = { .psfp_stats_get = vsc9959_psfp_stats_get, .cut_through_fwd = vsc9959_cut_through_fwd, .tas_clock_adjust = vsc9959_tas_clock_adjust, + .update_stats = vsc9959_update_stats, }; static const struct felix_info felix_info_vsc9959 = { - .target_io_res = vsc9959_target_io_res, - .port_io_res = vsc9959_port_io_res, - .imdio_res = &vsc9959_imdio_res, + .resources = vsc9959_resources, + .num_resources = ARRAY_SIZE(vsc9959_resources), + .resource_names = vsc9959_resource_names, .regfields = vsc9959_regfields, .map = vsc9959_regmap, .ops = &vsc9959_ops, @@ -2935,7 +2593,6 @@ static const struct felix_info felix_info_vsc9959 = { .port_setup_tc = vsc9959_port_setup_tc, .port_sched_speed_set = vsc9959_sched_speed_set, .tas_guard_bands_update = vsc9959_tas_guard_bands_update, - .init_regmap = ocelot_regmap_init, }; static irqreturn_t felix_irq_handler(int irq, void *data) @@ -2987,7 +2644,6 @@ static int felix_pci_probe(struct pci_dev *pdev, ocelot->num_flooding_pgids = OCELOT_NUM_TC; felix->info = &felix_info_vsc9959; felix->switch_base = pci_resource_start(pdev, VSC9959_SWITCH_PCI_BAR); - felix->imdio_base = pci_resource_start(pdev, VSC9959_IMDIO_PCI_BAR); pci_set_master(pdev); @@ -3048,8 +2704,6 @@ static void felix_pci_remove(struct pci_dev *pdev) kfree(felix); pci_disable_device(pdev); - - pci_set_drvdata(pdev, NULL); } static void felix_pci_shutdown(struct pci_dev *pdev) diff --git a/drivers/net/dsa/ocelot/seville_vsc9953.c b/drivers/net/dsa/ocelot/seville_vsc9953.c index b34f4cdfe814..7af33b2c685d 100644 --- a/drivers/net/dsa/ocelot/seville_vsc9953.c +++ b/drivers/net/dsa/ocelot/seville_vsc9953.c @@ -343,7 +343,7 @@ static const u32 vsc9953_sys_regmap[] = { REG(SYS_COUNT_TX_GREEN_PRIO_5, 0x00016c), REG(SYS_COUNT_TX_GREEN_PRIO_6, 0x000170), REG(SYS_COUNT_TX_GREEN_PRIO_7, 0x000174), - REG(SYS_COUNT_TX_AGING, 0x000178), + REG(SYS_COUNT_TX_AGED, 0x000178), REG(SYS_COUNT_DROP_LOCAL, 0x000200), REG(SYS_COUNT_DROP_TAIL, 0x000204), REG(SYS_COUNT_DROP_YELLOW_PRIO_0, 0x000208), @@ -383,7 +383,6 @@ static const u32 vsc9953_sys_regmap[] = { REG_RESERVED(SYS_MMGT_FAST), REG_RESERVED(SYS_EVENTS_DIF), REG_RESERVED(SYS_EVENTS_CORE), - REG_RESERVED(SYS_CNT), REG_RESERVED(SYS_PTP_STATUS), REG_RESERVED(SYS_PTP_TXSTAMP), REG_RESERVED(SYS_PTP_NXT), @@ -459,110 +458,40 @@ static const u32 *vsc9953_regmap[TARGET_MAX] = { }; /* Addresses are relative to the device's base address */ -static const struct resource vsc9953_target_io_res[TARGET_MAX] = { - [ANA] = { - .start = 0x0280000, - .end = 0x028ffff, - .name = "ana", - }, - [QS] = { - .start = 0x0080000, - .end = 0x00800ff, - .name = "qs", - }, - [QSYS] = { - .start = 0x0200000, - .end = 0x021ffff, - .name = "qsys", - }, - [REW] = { - .start = 0x0030000, - .end = 0x003ffff, - .name = "rew", - }, - [SYS] = { - .start = 0x0010000, - .end = 0x001ffff, - .name = "sys", - }, - [S0] = { - .start = 0x0040000, - .end = 0x00403ff, - .name = "s0", - }, - [S1] = { - .start = 0x0050000, - .end = 0x00503ff, - .name = "s1", - }, - [S2] = { - .start = 0x0060000, - .end = 0x00603ff, - .name = "s2", - }, - [PTP] = { - .start = 0x0090000, - .end = 0x00900cb, - .name = "ptp", - }, - [GCB] = { - .start = 0x0070000, - .end = 0x00701ff, - .name = "devcpu_gcb", - }, +static const struct resource vsc9953_resources[] = { + DEFINE_RES_MEM_NAMED(0x0010000, 0x0010000, "sys"), + DEFINE_RES_MEM_NAMED(0x0030000, 0x0010000, "rew"), + DEFINE_RES_MEM_NAMED(0x0040000, 0x0000400, "s0"), + DEFINE_RES_MEM_NAMED(0x0050000, 0x0000400, "s1"), + DEFINE_RES_MEM_NAMED(0x0060000, 0x0000400, "s2"), + DEFINE_RES_MEM_NAMED(0x0070000, 0x0000200, "devcpu_gcb"), + DEFINE_RES_MEM_NAMED(0x0080000, 0x0000100, "qs"), + DEFINE_RES_MEM_NAMED(0x0090000, 0x00000cc, "ptp"), + DEFINE_RES_MEM_NAMED(0x0100000, 0x0010000, "port0"), + DEFINE_RES_MEM_NAMED(0x0110000, 0x0010000, "port1"), + DEFINE_RES_MEM_NAMED(0x0120000, 0x0010000, "port2"), + DEFINE_RES_MEM_NAMED(0x0130000, 0x0010000, "port3"), + DEFINE_RES_MEM_NAMED(0x0140000, 0x0010000, "port4"), + DEFINE_RES_MEM_NAMED(0x0150000, 0x0010000, "port5"), + DEFINE_RES_MEM_NAMED(0x0160000, 0x0010000, "port6"), + DEFINE_RES_MEM_NAMED(0x0170000, 0x0010000, "port7"), + DEFINE_RES_MEM_NAMED(0x0180000, 0x0010000, "port8"), + DEFINE_RES_MEM_NAMED(0x0190000, 0x0010000, "port9"), + DEFINE_RES_MEM_NAMED(0x0200000, 0x0020000, "qsys"), + DEFINE_RES_MEM_NAMED(0x0280000, 0x0010000, "ana"), }; -static const struct resource vsc9953_port_io_res[] = { - { - .start = 0x0100000, - .end = 0x010ffff, - .name = "port0", - }, - { - .start = 0x0110000, - .end = 0x011ffff, - .name = "port1", - }, - { - .start = 0x0120000, - .end = 0x012ffff, - .name = "port2", - }, - { - .start = 0x0130000, - .end = 0x013ffff, - .name = "port3", - }, - { - .start = 0x0140000, - .end = 0x014ffff, - .name = "port4", - }, - { - .start = 0x0150000, - .end = 0x015ffff, - .name = "port5", - }, - { - .start = 0x0160000, - .end = 0x016ffff, - .name = "port6", - }, - { - .start = 0x0170000, - .end = 0x017ffff, - .name = "port7", - }, - { - .start = 0x0180000, - .end = 0x018ffff, - .name = "port8", - }, - { - .start = 0x0190000, - .end = 0x019ffff, - .name = "port9", - }, +static const char * const vsc9953_resource_names[TARGET_MAX] = { + [SYS] = "sys", + [REW] = "rew", + [S0] = "s0", + [S1] = "s1", + [S2] = "s2", + [GCB] = "devcpu_gcb", + [QS] = "qs", + [PTP] = "ptp", + [QSYS] = "qsys", + [ANA] = "ana", }; static const struct reg_field vsc9953_regfields[REGFIELD_MAX] = { @@ -615,378 +544,7 @@ static const struct reg_field vsc9953_regfields[REGFIELD_MAX] = { }; static const struct ocelot_stat_layout vsc9953_stats_layout[OCELOT_NUM_STATS] = { - [OCELOT_STAT_RX_OCTETS] = { - .name = "rx_octets", - .reg = SYS_COUNT_RX_OCTETS, - }, - [OCELOT_STAT_RX_UNICAST] = { - .name = "rx_unicast", - .reg = SYS_COUNT_RX_UNICAST, - }, - [OCELOT_STAT_RX_MULTICAST] = { - .name = "rx_multicast", - .reg = SYS_COUNT_RX_MULTICAST, - }, - [OCELOT_STAT_RX_BROADCAST] = { - .name = "rx_broadcast", - .reg = SYS_COUNT_RX_BROADCAST, - }, - [OCELOT_STAT_RX_SHORTS] = { - .name = "rx_shorts", - .reg = SYS_COUNT_RX_SHORTS, - }, - [OCELOT_STAT_RX_FRAGMENTS] = { - .name = "rx_fragments", - .reg = SYS_COUNT_RX_FRAGMENTS, - }, - [OCELOT_STAT_RX_JABBERS] = { - .name = "rx_jabbers", - .reg = SYS_COUNT_RX_JABBERS, - }, - [OCELOT_STAT_RX_CRC_ALIGN_ERRS] = { - .name = "rx_crc_align_errs", - .reg = SYS_COUNT_RX_CRC_ALIGN_ERRS, - }, - [OCELOT_STAT_RX_SYM_ERRS] = { - .name = "rx_sym_errs", - .reg = SYS_COUNT_RX_SYM_ERRS, - }, - [OCELOT_STAT_RX_64] = { - .name = "rx_frames_below_65_octets", - .reg = SYS_COUNT_RX_64, - }, - [OCELOT_STAT_RX_65_127] = { - .name = "rx_frames_65_to_127_octets", - .reg = SYS_COUNT_RX_65_127, - }, - [OCELOT_STAT_RX_128_255] = { - .name = "rx_frames_128_to_255_octets", - .reg = SYS_COUNT_RX_128_255, - }, - [OCELOT_STAT_RX_256_511] = { - .name = "rx_frames_256_to_511_octets", - .reg = SYS_COUNT_RX_256_511, - }, - [OCELOT_STAT_RX_512_1023] = { - .name = "rx_frames_512_to_1023_octets", - .reg = SYS_COUNT_RX_512_1023, - }, - [OCELOT_STAT_RX_1024_1526] = { - .name = "rx_frames_1024_to_1526_octets", - .reg = SYS_COUNT_RX_1024_1526, - }, - [OCELOT_STAT_RX_1527_MAX] = { - .name = "rx_frames_over_1526_octets", - .reg = SYS_COUNT_RX_1527_MAX, - }, - [OCELOT_STAT_RX_PAUSE] = { - .name = "rx_pause", - .reg = SYS_COUNT_RX_PAUSE, - }, - [OCELOT_STAT_RX_CONTROL] = { - .name = "rx_control", - .reg = SYS_COUNT_RX_CONTROL, - }, - [OCELOT_STAT_RX_LONGS] = { - .name = "rx_longs", - .reg = SYS_COUNT_RX_LONGS, - }, - [OCELOT_STAT_RX_CLASSIFIED_DROPS] = { - .name = "rx_classified_drops", - .reg = SYS_COUNT_RX_CLASSIFIED_DROPS, - }, - [OCELOT_STAT_RX_RED_PRIO_0] = { - .name = "rx_red_prio_0", - .reg = SYS_COUNT_RX_RED_PRIO_0, - }, - [OCELOT_STAT_RX_RED_PRIO_1] = { - .name = "rx_red_prio_1", - .reg = SYS_COUNT_RX_RED_PRIO_1, - }, - [OCELOT_STAT_RX_RED_PRIO_2] = { - .name = "rx_red_prio_2", - .reg = SYS_COUNT_RX_RED_PRIO_2, - }, - [OCELOT_STAT_RX_RED_PRIO_3] = { - .name = "rx_red_prio_3", - .reg = SYS_COUNT_RX_RED_PRIO_3, - }, - [OCELOT_STAT_RX_RED_PRIO_4] = { - .name = "rx_red_prio_4", - .reg = SYS_COUNT_RX_RED_PRIO_4, - }, - [OCELOT_STAT_RX_RED_PRIO_5] = { - .name = "rx_red_prio_5", - .reg = SYS_COUNT_RX_RED_PRIO_5, - }, - [OCELOT_STAT_RX_RED_PRIO_6] = { - .name = "rx_red_prio_6", - .reg = SYS_COUNT_RX_RED_PRIO_6, - }, - [OCELOT_STAT_RX_RED_PRIO_7] = { - .name = "rx_red_prio_7", - .reg = SYS_COUNT_RX_RED_PRIO_7, - }, - [OCELOT_STAT_RX_YELLOW_PRIO_0] = { - .name = "rx_yellow_prio_0", - .reg = SYS_COUNT_RX_YELLOW_PRIO_0, - }, - [OCELOT_STAT_RX_YELLOW_PRIO_1] = { - .name = "rx_yellow_prio_1", - .reg = SYS_COUNT_RX_YELLOW_PRIO_1, - }, - [OCELOT_STAT_RX_YELLOW_PRIO_2] = { - .name = "rx_yellow_prio_2", - .reg = SYS_COUNT_RX_YELLOW_PRIO_2, - }, - [OCELOT_STAT_RX_YELLOW_PRIO_3] = { - .name = "rx_yellow_prio_3", - .reg = SYS_COUNT_RX_YELLOW_PRIO_3, - }, - [OCELOT_STAT_RX_YELLOW_PRIO_4] = { - .name = "rx_yellow_prio_4", - .reg = SYS_COUNT_RX_YELLOW_PRIO_4, - }, - [OCELOT_STAT_RX_YELLOW_PRIO_5] = { - .name = "rx_yellow_prio_5", - .reg = SYS_COUNT_RX_YELLOW_PRIO_5, - }, - [OCELOT_STAT_RX_YELLOW_PRIO_6] = { - .name = "rx_yellow_prio_6", - .reg = SYS_COUNT_RX_YELLOW_PRIO_6, - }, - [OCELOT_STAT_RX_YELLOW_PRIO_7] = { - .name = "rx_yellow_prio_7", - .reg = SYS_COUNT_RX_YELLOW_PRIO_7, - }, - [OCELOT_STAT_RX_GREEN_PRIO_0] = { - .name = "rx_green_prio_0", - .reg = SYS_COUNT_RX_GREEN_PRIO_0, - }, - [OCELOT_STAT_RX_GREEN_PRIO_1] = { - .name = "rx_green_prio_1", - .reg = SYS_COUNT_RX_GREEN_PRIO_1, - }, - [OCELOT_STAT_RX_GREEN_PRIO_2] = { - .name = "rx_green_prio_2", - .reg = SYS_COUNT_RX_GREEN_PRIO_2, - }, - [OCELOT_STAT_RX_GREEN_PRIO_3] = { - .name = "rx_green_prio_3", - .reg = SYS_COUNT_RX_GREEN_PRIO_3, - }, - [OCELOT_STAT_RX_GREEN_PRIO_4] = { - .name = "rx_green_prio_4", - .reg = SYS_COUNT_RX_GREEN_PRIO_4, - }, - [OCELOT_STAT_RX_GREEN_PRIO_5] = { - .name = "rx_green_prio_5", - .reg = SYS_COUNT_RX_GREEN_PRIO_5, - }, - [OCELOT_STAT_RX_GREEN_PRIO_6] = { - .name = "rx_green_prio_6", - .reg = SYS_COUNT_RX_GREEN_PRIO_6, - }, - [OCELOT_STAT_RX_GREEN_PRIO_7] = { - .name = "rx_green_prio_7", - .reg = SYS_COUNT_RX_GREEN_PRIO_7, - }, - [OCELOT_STAT_TX_OCTETS] = { - .name = "tx_octets", - .reg = SYS_COUNT_TX_OCTETS, - }, - [OCELOT_STAT_TX_UNICAST] = { - .name = "tx_unicast", - .reg = SYS_COUNT_TX_UNICAST, - }, - [OCELOT_STAT_TX_MULTICAST] = { - .name = "tx_multicast", - .reg = SYS_COUNT_TX_MULTICAST, - }, - [OCELOT_STAT_TX_BROADCAST] = { - .name = "tx_broadcast", - .reg = SYS_COUNT_TX_BROADCAST, - }, - [OCELOT_STAT_TX_COLLISION] = { - .name = "tx_collision", - .reg = SYS_COUNT_TX_COLLISION, - }, - [OCELOT_STAT_TX_DROPS] = { - .name = "tx_drops", - .reg = SYS_COUNT_TX_DROPS, - }, - [OCELOT_STAT_TX_PAUSE] = { - .name = "tx_pause", - .reg = SYS_COUNT_TX_PAUSE, - }, - [OCELOT_STAT_TX_64] = { - .name = "tx_frames_below_65_octets", - .reg = SYS_COUNT_TX_64, - }, - [OCELOT_STAT_TX_65_127] = { - .name = "tx_frames_65_to_127_octets", - .reg = SYS_COUNT_TX_65_127, - }, - [OCELOT_STAT_TX_128_255] = { - .name = "tx_frames_128_255_octets", - .reg = SYS_COUNT_TX_128_255, - }, - [OCELOT_STAT_TX_256_511] = { - .name = "tx_frames_256_511_octets", - .reg = SYS_COUNT_TX_256_511, - }, - [OCELOT_STAT_TX_512_1023] = { - .name = "tx_frames_512_1023_octets", - .reg = SYS_COUNT_TX_512_1023, - }, - [OCELOT_STAT_TX_1024_1526] = { - .name = "tx_frames_1024_1526_octets", - .reg = SYS_COUNT_TX_1024_1526, - }, - [OCELOT_STAT_TX_1527_MAX] = { - .name = "tx_frames_over_1526_octets", - .reg = SYS_COUNT_TX_1527_MAX, - }, - [OCELOT_STAT_TX_YELLOW_PRIO_0] = { - .name = "tx_yellow_prio_0", - .reg = SYS_COUNT_TX_YELLOW_PRIO_0, - }, - [OCELOT_STAT_TX_YELLOW_PRIO_1] = { - .name = "tx_yellow_prio_1", - .reg = SYS_COUNT_TX_YELLOW_PRIO_1, - }, - [OCELOT_STAT_TX_YELLOW_PRIO_2] = { - .name = "tx_yellow_prio_2", - .reg = SYS_COUNT_TX_YELLOW_PRIO_2, - }, - [OCELOT_STAT_TX_YELLOW_PRIO_3] = { - .name = "tx_yellow_prio_3", - .reg = SYS_COUNT_TX_YELLOW_PRIO_3, - }, - [OCELOT_STAT_TX_YELLOW_PRIO_4] = { - .name = "tx_yellow_prio_4", - .reg = SYS_COUNT_TX_YELLOW_PRIO_4, - }, - [OCELOT_STAT_TX_YELLOW_PRIO_5] = { - .name = "tx_yellow_prio_5", - .reg = SYS_COUNT_TX_YELLOW_PRIO_5, - }, - [OCELOT_STAT_TX_YELLOW_PRIO_6] = { - .name = "tx_yellow_prio_6", - .reg = SYS_COUNT_TX_YELLOW_PRIO_6, - }, - [OCELOT_STAT_TX_YELLOW_PRIO_7] = { - .name = "tx_yellow_prio_7", - .reg = SYS_COUNT_TX_YELLOW_PRIO_7, - }, - [OCELOT_STAT_TX_GREEN_PRIO_0] = { - .name = "tx_green_prio_0", - .reg = SYS_COUNT_TX_GREEN_PRIO_0, - }, - [OCELOT_STAT_TX_GREEN_PRIO_1] = { - .name = "tx_green_prio_1", - .reg = SYS_COUNT_TX_GREEN_PRIO_1, - }, - [OCELOT_STAT_TX_GREEN_PRIO_2] = { - .name = "tx_green_prio_2", - .reg = SYS_COUNT_TX_GREEN_PRIO_2, - }, - [OCELOT_STAT_TX_GREEN_PRIO_3] = { - .name = "tx_green_prio_3", - .reg = SYS_COUNT_TX_GREEN_PRIO_3, - }, - [OCELOT_STAT_TX_GREEN_PRIO_4] = { - .name = "tx_green_prio_4", - .reg = SYS_COUNT_TX_GREEN_PRIO_4, - }, - [OCELOT_STAT_TX_GREEN_PRIO_5] = { - .name = "tx_green_prio_5", - .reg = SYS_COUNT_TX_GREEN_PRIO_5, - }, - [OCELOT_STAT_TX_GREEN_PRIO_6] = { - .name = "tx_green_prio_6", - .reg = SYS_COUNT_TX_GREEN_PRIO_6, - }, - [OCELOT_STAT_TX_GREEN_PRIO_7] = { - .name = "tx_green_prio_7", - .reg = SYS_COUNT_TX_GREEN_PRIO_7, - }, - [OCELOT_STAT_TX_AGED] = { - .name = "tx_aged", - .reg = SYS_COUNT_TX_AGING, - }, - [OCELOT_STAT_DROP_LOCAL] = { - .name = "drop_local", - .reg = SYS_COUNT_DROP_LOCAL, - }, - [OCELOT_STAT_DROP_TAIL] = { - .name = "drop_tail", - .reg = SYS_COUNT_DROP_TAIL, - }, - [OCELOT_STAT_DROP_YELLOW_PRIO_0] = { - .name = "drop_yellow_prio_0", - .reg = SYS_COUNT_DROP_YELLOW_PRIO_0, - }, - [OCELOT_STAT_DROP_YELLOW_PRIO_1] = { - .name = "drop_yellow_prio_1", - .reg = SYS_COUNT_DROP_YELLOW_PRIO_1, - }, - [OCELOT_STAT_DROP_YELLOW_PRIO_2] = { - .name = "drop_yellow_prio_2", - .reg = SYS_COUNT_DROP_YELLOW_PRIO_2, - }, - [OCELOT_STAT_DROP_YELLOW_PRIO_3] = { - .name = "drop_yellow_prio_3", - .reg = SYS_COUNT_DROP_YELLOW_PRIO_3, - }, - [OCELOT_STAT_DROP_YELLOW_PRIO_4] = { - .name = "drop_yellow_prio_4", - .reg = SYS_COUNT_DROP_YELLOW_PRIO_4, - }, - [OCELOT_STAT_DROP_YELLOW_PRIO_5] = { - .name = "drop_yellow_prio_5", - .reg = SYS_COUNT_DROP_YELLOW_PRIO_5, - }, - [OCELOT_STAT_DROP_YELLOW_PRIO_6] = { - .name = "drop_yellow_prio_6", - .reg = SYS_COUNT_DROP_YELLOW_PRIO_6, - }, - [OCELOT_STAT_DROP_YELLOW_PRIO_7] = { - .name = "drop_yellow_prio_7", - .reg = SYS_COUNT_DROP_YELLOW_PRIO_7, - }, - [OCELOT_STAT_DROP_GREEN_PRIO_0] = { - .name = "drop_green_prio_0", - .reg = SYS_COUNT_DROP_GREEN_PRIO_0, - }, - [OCELOT_STAT_DROP_GREEN_PRIO_1] = { - .name = "drop_green_prio_1", - .reg = SYS_COUNT_DROP_GREEN_PRIO_1, - }, - [OCELOT_STAT_DROP_GREEN_PRIO_2] = { - .name = "drop_green_prio_2", - .reg = SYS_COUNT_DROP_GREEN_PRIO_2, - }, - [OCELOT_STAT_DROP_GREEN_PRIO_3] = { - .name = "drop_green_prio_3", - .reg = SYS_COUNT_DROP_GREEN_PRIO_3, - }, - [OCELOT_STAT_DROP_GREEN_PRIO_4] = { - .name = "drop_green_prio_4", - .reg = SYS_COUNT_DROP_GREEN_PRIO_4, - }, - [OCELOT_STAT_DROP_GREEN_PRIO_5] = { - .name = "drop_green_prio_5", - .reg = SYS_COUNT_DROP_GREEN_PRIO_5, - }, - [OCELOT_STAT_DROP_GREEN_PRIO_6] = { - .name = "drop_green_prio_6", - .reg = SYS_COUNT_DROP_GREEN_PRIO_6, - }, - [OCELOT_STAT_DROP_GREEN_PRIO_7] = { - .name = "drop_green_prio_7", - .reg = SYS_COUNT_DROP_GREEN_PRIO_7, - }, + OCELOT_COMMON_STATS, }; static const struct vcap_field vsc9953_vcap_es0_keys[] = { @@ -1432,8 +990,9 @@ static void vsc9953_mdio_bus_free(struct ocelot *ocelot) } static const struct felix_info seville_info_vsc9953 = { - .target_io_res = vsc9953_target_io_res, - .port_io_res = vsc9953_port_io_res, + .resources = vsc9953_resources, + .num_resources = ARRAY_SIZE(vsc9953_resources), + .resource_names = vsc9953_resource_names, .regfields = vsc9953_regfields, .map = vsc9953_regmap, .ops = &vsc9953_ops, @@ -1450,7 +1009,6 @@ static const struct felix_info seville_info_vsc9953 = { .mdio_bus_free = vsc9953_mdio_bus_free, .phylink_validate = vsc9953_phylink_validate, .port_modes = vsc9953_port_modes, - .init_regmap = ocelot_regmap_init, }; static int seville_probe(struct platform_device *pdev) @@ -1525,8 +1083,6 @@ static int seville_remove(struct platform_device *pdev) kfree(felix->ds); kfree(felix); - platform_set_drvdata(pdev, NULL); - return 0; } diff --git a/drivers/net/dsa/qca/ar9331.c b/drivers/net/dsa/qca/ar9331.c index 0796b7cf8cae..e7b98b864fa1 100644 --- a/drivers/net/dsa/qca/ar9331.c +++ b/drivers/net/dsa/qca/ar9331.c @@ -1099,8 +1099,6 @@ static void ar9331_sw_remove(struct mdio_device *mdiodev) dsa_unregister_switch(&priv->ds); reset_control_assert(priv->sw_reset); - - dev_set_drvdata(&mdiodev->dev, NULL); } static void ar9331_sw_shutdown(struct mdio_device *mdiodev) diff --git a/drivers/net/dsa/qca/qca8k-8xxx.c b/drivers/net/dsa/qca/qca8k-8xxx.c index c181346388a4..5669c92c93f7 100644 --- a/drivers/net/dsa/qca/qca8k-8xxx.c +++ b/drivers/net/dsa/qca/qca8k-8xxx.c @@ -1957,8 +1957,6 @@ qca8k_sw_remove(struct mdio_device *mdiodev) qca8k_port_set_status(priv, i, 0); dsa_unregister_switch(priv->ds); - - dev_set_drvdata(&mdiodev->dev, NULL); } static void qca8k_sw_shutdown(struct mdio_device *mdiodev) diff --git a/drivers/net/dsa/qca/qca8k-common.c b/drivers/net/dsa/qca/qca8k-common.c index bba95613e218..fb45b598847b 100644 --- a/drivers/net/dsa/qca/qca8k-common.c +++ b/drivers/net/dsa/qca/qca8k-common.c @@ -1017,7 +1017,8 @@ int qca8k_port_vlan_del(struct dsa_switch *ds, int port, static bool qca8k_lag_can_offload(struct dsa_switch *ds, struct dsa_lag lag, - struct netdev_lag_upper_info *info) + struct netdev_lag_upper_info *info, + struct netlink_ext_ack *extack) { struct dsa_port *dp; int members = 0; @@ -1029,15 +1030,24 @@ static bool qca8k_lag_can_offload(struct dsa_switch *ds, /* Includes the port joining the LAG */ members++; - if (members > QCA8K_NUM_PORTS_FOR_LAG) + if (members > QCA8K_NUM_PORTS_FOR_LAG) { + NL_SET_ERR_MSG_MOD(extack, + "Cannot offload more than 4 LAG ports"); return false; + } - if (info->tx_type != NETDEV_LAG_TX_TYPE_HASH) + if (info->tx_type != NETDEV_LAG_TX_TYPE_HASH) { + NL_SET_ERR_MSG_MOD(extack, + "Can only offload LAG using hash TX type"); return false; + } if (info->hash_type != NETDEV_LAG_HASH_L2 && - info->hash_type != NETDEV_LAG_HASH_L23) + info->hash_type != NETDEV_LAG_HASH_L23) { + NL_SET_ERR_MSG_MOD(extack, + "Can only offload L2 or L2+L3 TX hash"); return false; + } return true; } @@ -1160,11 +1170,12 @@ static int qca8k_lag_refresh_portmap(struct dsa_switch *ds, int port, } int qca8k_port_lag_join(struct dsa_switch *ds, int port, struct dsa_lag lag, - struct netdev_lag_upper_info *info) + struct netdev_lag_upper_info *info, + struct netlink_ext_ack *extack) { int ret; - if (!qca8k_lag_can_offload(ds, lag, info)) + if (!qca8k_lag_can_offload(ds, lag, info, extack)) return -EOPNOTSUPP; ret = qca8k_lag_setup_hash(ds, lag, info); diff --git a/drivers/net/dsa/qca/qca8k.h b/drivers/net/dsa/qca/qca8k.h index e36ecc9777f4..0b7a5cb12321 100644 --- a/drivers/net/dsa/qca/qca8k.h +++ b/drivers/net/dsa/qca/qca8k.h @@ -512,7 +512,8 @@ int qca8k_port_vlan_del(struct dsa_switch *ds, int port, /* Common port LAG function */ int qca8k_port_lag_join(struct dsa_switch *ds, int port, struct dsa_lag lag, - struct netdev_lag_upper_info *info); + struct netdev_lag_upper_info *info, + struct netlink_ext_ack *extack); int qca8k_port_lag_leave(struct dsa_switch *ds, int port, struct dsa_lag lag); diff --git a/drivers/net/dsa/realtek/realtek-mdio.c b/drivers/net/dsa/realtek/realtek-mdio.c index c58f49d558d2..3e54fac5f902 100644 --- a/drivers/net/dsa/realtek/realtek-mdio.c +++ b/drivers/net/dsa/realtek/realtek-mdio.c @@ -245,8 +245,6 @@ static void realtek_mdio_remove(struct mdio_device *mdiodev) /* leave the device reset asserted */ if (priv->reset) gpiod_set_value(priv->reset, 1); - - dev_set_drvdata(&mdiodev->dev, NULL); } static void realtek_mdio_shutdown(struct mdio_device *mdiodev) diff --git a/drivers/net/dsa/realtek/realtek-smi.c b/drivers/net/dsa/realtek/realtek-smi.c index 45992f79ec8d..1b447d96b9c4 100644 --- a/drivers/net/dsa/realtek/realtek-smi.c +++ b/drivers/net/dsa/realtek/realtek-smi.c @@ -522,8 +522,6 @@ static int realtek_smi_remove(struct platform_device *pdev) if (priv->reset) gpiod_set_value(priv->reset, 1); - platform_set_drvdata(pdev, NULL); - return 0; } diff --git a/drivers/net/dsa/rzn1_a5psw.c b/drivers/net/dsa/rzn1_a5psw.c index 0744e8162e1d..ed413d555bec 100644 --- a/drivers/net/dsa/rzn1_a5psw.c +++ b/drivers/net/dsa/rzn1_a5psw.c @@ -1025,8 +1025,6 @@ static int a5psw_remove(struct platform_device *pdev) clk_disable_unprepare(a5psw->hclk); clk_disable_unprepare(a5psw->clk); - platform_set_drvdata(pdev, NULL); - return 0; } diff --git a/drivers/net/dsa/sja1105/sja1105_main.c b/drivers/net/dsa/sja1105/sja1105_main.c index b03d0d0c3dbf..412666111b0c 100644 --- a/drivers/net/dsa/sja1105/sja1105_main.c +++ b/drivers/net/dsa/sja1105/sja1105_main.c @@ -3351,8 +3351,6 @@ static void sja1105_remove(struct spi_device *spi) return; dsa_unregister_switch(priv->ds); - - spi_set_drvdata(spi, NULL); } static void sja1105_shutdown(struct spi_device *spi) diff --git a/drivers/net/dsa/vitesse-vsc73xx-platform.c b/drivers/net/dsa/vitesse-vsc73xx-platform.c index fe4b154a0a57..bd4206e8f9af 100644 --- a/drivers/net/dsa/vitesse-vsc73xx-platform.c +++ b/drivers/net/dsa/vitesse-vsc73xx-platform.c @@ -121,8 +121,6 @@ static int vsc73xx_platform_remove(struct platform_device *pdev) vsc73xx_remove(&vsc_platform->vsc); - platform_set_drvdata(pdev, NULL); - return 0; } diff --git a/drivers/net/dsa/vitesse-vsc73xx-spi.c b/drivers/net/dsa/vitesse-vsc73xx-spi.c index 97a92e6da60d..85b9a0f51dd8 100644 --- a/drivers/net/dsa/vitesse-vsc73xx-spi.c +++ b/drivers/net/dsa/vitesse-vsc73xx-spi.c @@ -167,8 +167,6 @@ static void vsc73xx_spi_remove(struct spi_device *spi) return; vsc73xx_remove(&vsc_spi->vsc); - - spi_set_drvdata(spi, NULL); } static void vsc73xx_spi_shutdown(struct spi_device *spi) diff --git a/drivers/net/dsa/xrs700x/xrs700x_i2c.c b/drivers/net/dsa/xrs700x/xrs700x_i2c.c index 6deae388a0d6..cd533b9e17ec 100644 --- a/drivers/net/dsa/xrs700x/xrs700x_i2c.c +++ b/drivers/net/dsa/xrs700x/xrs700x_i2c.c @@ -114,8 +114,6 @@ static int xrs700x_i2c_remove(struct i2c_client *i2c) xrs700x_switch_remove(priv); - i2c_set_clientdata(i2c, NULL); - return 0; } diff --git a/drivers/net/dsa/xrs700x/xrs700x_mdio.c b/drivers/net/dsa/xrs700x/xrs700x_mdio.c index 127a677d1f39..5f7d344b5d73 100644 --- a/drivers/net/dsa/xrs700x/xrs700x_mdio.c +++ b/drivers/net/dsa/xrs700x/xrs700x_mdio.c @@ -140,8 +140,6 @@ static void xrs700x_mdio_remove(struct mdio_device *mdiodev) return; xrs700x_switch_remove(priv); - - dev_set_drvdata(&mdiodev->dev, NULL); } static void xrs700x_mdio_shutdown(struct mdio_device *mdiodev) diff --git a/drivers/net/dummy.c b/drivers/net/dummy.c index f82ad7419508..aa0fc00faecb 100644 --- a/drivers/net/dummy.c +++ b/drivers/net/dummy.c @@ -102,7 +102,7 @@ static const struct net_device_ops dummy_netdev_ops = { static void dummy_get_drvinfo(struct net_device *dev, struct ethtool_drvinfo *info) { - strlcpy(info->driver, DRV_NAME, sizeof(info->driver)); + strscpy(info->driver, DRV_NAME, sizeof(info->driver)); } static const struct ethtool_ops dummy_ethtool_ops = { diff --git a/drivers/net/ethernet/3com/3c509.c b/drivers/net/ethernet/3com/3c509.c index 846fa3af4504..fb68339e1511 100644 --- a/drivers/net/ethernet/3com/3c509.c +++ b/drivers/net/ethernet/3com/3c509.c @@ -1135,7 +1135,7 @@ el3_netdev_set_ecmd(struct net_device *dev, static void el3_get_drvinfo(struct net_device *dev, struct ethtool_drvinfo *info) { - strlcpy(info->driver, DRV_NAME, sizeof(info->driver)); + strscpy(info->driver, DRV_NAME, sizeof(info->driver)); } static int el3_get_link_ksettings(struct net_device *dev, diff --git a/drivers/net/ethernet/3com/3c515.c b/drivers/net/ethernet/3com/3c515.c index 1d124b0f65e7..d2f4358cc550 100644 --- a/drivers/net/ethernet/3com/3c515.c +++ b/drivers/net/ethernet/3com/3c515.c @@ -1527,7 +1527,7 @@ static void set_rx_mode(struct net_device *dev) static void netdev_get_drvinfo(struct net_device *dev, struct ethtool_drvinfo *info) { - strlcpy(info->driver, DRV_NAME, sizeof(info->driver)); + strscpy(info->driver, DRV_NAME, sizeof(info->driver)); snprintf(info->bus_info, sizeof(info->bus_info), "ISA 0x%lx", dev->base_addr); } diff --git a/drivers/net/ethernet/3com/3c589_cs.c b/drivers/net/ethernet/3com/3c589_cs.c index 4673bc1604e7..82f94b1635bf 100644 --- a/drivers/net/ethernet/3com/3c589_cs.c +++ b/drivers/net/ethernet/3com/3c589_cs.c @@ -480,7 +480,7 @@ static void tc589_reset(struct net_device *dev) static void netdev_get_drvinfo(struct net_device *dev, struct ethtool_drvinfo *info) { - strlcpy(info->driver, DRV_NAME, sizeof(info->driver)); + strscpy(info->driver, DRV_NAME, sizeof(info->driver)); snprintf(info->bus_info, sizeof(info->bus_info), "PCMCIA 0x%lx", dev->base_addr); } diff --git a/drivers/net/ethernet/3com/3c59x.c b/drivers/net/ethernet/3com/3c59x.c index ccf07667aa5e..082388bb6169 100644 --- a/drivers/net/ethernet/3com/3c59x.c +++ b/drivers/net/ethernet/3com/3c59x.c @@ -2959,13 +2959,13 @@ static void vortex_get_drvinfo(struct net_device *dev, { struct vortex_private *vp = netdev_priv(dev); - strlcpy(info->driver, DRV_NAME, sizeof(info->driver)); + strscpy(info->driver, DRV_NAME, sizeof(info->driver)); if (VORTEX_PCI(vp)) { - strlcpy(info->bus_info, pci_name(VORTEX_PCI(vp)), + strscpy(info->bus_info, pci_name(VORTEX_PCI(vp)), sizeof(info->bus_info)); } else { if (VORTEX_EISA(vp)) - strlcpy(info->bus_info, dev_name(vp->gendev), + strscpy(info->bus_info, dev_name(vp->gendev), sizeof(info->bus_info)); else snprintf(info->bus_info, sizeof(info->bus_info), diff --git a/drivers/net/ethernet/3com/typhoon.c b/drivers/net/ethernet/3com/typhoon.c index cad4f354cc76..aaaff3ba43ef 100644 --- a/drivers/net/ethernet/3com/typhoon.c +++ b/drivers/net/ethernet/3com/typhoon.c @@ -969,12 +969,12 @@ typhoon_get_drvinfo(struct net_device *dev, struct ethtool_drvinfo *info) smp_rmb(); if (tp->card_state == Sleeping) { - strlcpy(info->fw_version, "Sleep image", + strscpy(info->fw_version, "Sleep image", sizeof(info->fw_version)); } else { INIT_COMMAND_WITH_RESPONSE(&xp_cmd, TYPHOON_CMD_READ_VERSIONS); if (typhoon_issue_command(tp, 1, &xp_cmd, 3, xp_resp) < 0) { - strlcpy(info->fw_version, "Unknown runtime", + strscpy(info->fw_version, "Unknown runtime", sizeof(info->fw_version)); } else { u32 sleep_ver = le32_to_cpu(xp_resp[0].parm2); @@ -984,8 +984,8 @@ typhoon_get_drvinfo(struct net_device *dev, struct ethtool_drvinfo *info) } } - strlcpy(info->driver, KBUILD_MODNAME, sizeof(info->driver)); - strlcpy(info->bus_info, pci_name(pci_dev), sizeof(info->bus_info)); + strscpy(info->driver, KBUILD_MODNAME, sizeof(info->driver)); + strscpy(info->bus_info, pci_name(pci_dev), sizeof(info->bus_info)); } static int diff --git a/drivers/net/ethernet/8390/ax88796.c b/drivers/net/ethernet/8390/ax88796.c index 1f8acbba5b6b..af603256b724 100644 --- a/drivers/net/ethernet/8390/ax88796.c +++ b/drivers/net/ethernet/8390/ax88796.c @@ -579,9 +579,9 @@ static void ax_get_drvinfo(struct net_device *dev, { struct platform_device *pdev = to_platform_device(dev->dev.parent); - strlcpy(info->driver, DRV_NAME, sizeof(info->driver)); - strlcpy(info->version, DRV_VERSION, sizeof(info->version)); - strlcpy(info->bus_info, pdev->name, sizeof(info->bus_info)); + strscpy(info->driver, DRV_NAME, sizeof(info->driver)); + strscpy(info->version, DRV_VERSION, sizeof(info->version)); + strscpy(info->bus_info, pdev->name, sizeof(info->bus_info)); } static u32 ax_get_msglevel(struct net_device *dev) diff --git a/drivers/net/ethernet/8390/etherh.c b/drivers/net/ethernet/8390/etherh.c index e7b879123bb1..05d39ecb97ff 100644 --- a/drivers/net/ethernet/8390/etherh.c +++ b/drivers/net/ethernet/8390/etherh.c @@ -555,9 +555,9 @@ static int __init etherm_addr(char *addr) static void etherh_get_drvinfo(struct net_device *dev, struct ethtool_drvinfo *info) { - strlcpy(info->driver, DRV_NAME, sizeof(info->driver)); - strlcpy(info->version, DRV_VERSION, sizeof(info->version)); - strlcpy(info->bus_info, dev_name(dev->dev.parent), + strscpy(info->driver, DRV_NAME, sizeof(info->driver)); + strscpy(info->version, DRV_VERSION, sizeof(info->version)); + strscpy(info->bus_info, dev_name(dev->dev.parent), sizeof(info->bus_info)); } diff --git a/drivers/net/ethernet/8390/mcf8390.c b/drivers/net/ethernet/8390/mcf8390.c index 21047ae1bc3d..8a7918d33419 100644 --- a/drivers/net/ethernet/8390/mcf8390.c +++ b/drivers/net/ethernet/8390/mcf8390.c @@ -450,8 +450,7 @@ static int mcf8390_remove(struct platform_device *pdev) unregister_netdev(dev); mem = platform_get_resource(pdev, IORESOURCE_MEM, 0); - if (mem) - release_mem_region(mem->start, resource_size(mem)); + release_mem_region(mem->start, resource_size(mem)); free_netdev(dev); return 0; } diff --git a/drivers/net/ethernet/Kconfig b/drivers/net/ethernet/Kconfig index 9a55c1d5a0a1..1917da784191 100644 --- a/drivers/net/ethernet/Kconfig +++ b/drivers/net/ethernet/Kconfig @@ -121,6 +121,7 @@ config LANTIQ_XRX200 Support for the PMAC of the Gigabit switch (GSWIP) inside the Lantiq / Intel VRX200 VDSL SoC +source "drivers/net/ethernet/adi/Kconfig" source "drivers/net/ethernet/litex/Kconfig" source "drivers/net/ethernet/marvell/Kconfig" source "drivers/net/ethernet/mediatek/Kconfig" diff --git a/drivers/net/ethernet/Makefile b/drivers/net/ethernet/Makefile index c06e75ed4231..0d872d4efcd1 100644 --- a/drivers/net/ethernet/Makefile +++ b/drivers/net/ethernet/Makefile @@ -8,6 +8,7 @@ obj-$(CONFIG_NET_VENDOR_8390) += 8390/ obj-$(CONFIG_NET_VENDOR_ACTIONS) += actions/ obj-$(CONFIG_NET_VENDOR_ADAPTEC) += adaptec/ obj-$(CONFIG_GRETH) += aeroflex/ +obj-$(CONFIG_NET_VENDOR_ADI) += adi/ obj-$(CONFIG_NET_VENDOR_AGERE) += agere/ obj-$(CONFIG_NET_VENDOR_ALACRITECH) += alacritech/ obj-$(CONFIG_NET_VENDOR_ALLWINNER) += allwinner/ diff --git a/drivers/net/ethernet/actions/owl-emac.c b/drivers/net/ethernet/actions/owl-emac.c index 1cfdd01b4c2e..cd4d71b83c33 100644 --- a/drivers/net/ethernet/actions/owl-emac.c +++ b/drivers/net/ethernet/actions/owl-emac.c @@ -1576,7 +1576,7 @@ static int owl_emac_probe(struct platform_device *pdev) netdev->watchdog_timeo = OWL_EMAC_TX_TIMEOUT; netdev->netdev_ops = &owl_emac_netdev_ops; netdev->ethtool_ops = &owl_emac_ethtool_ops; - netif_napi_add(netdev, &priv->napi, owl_emac_poll, NAPI_POLL_WEIGHT); + netif_napi_add(netdev, &priv->napi, owl_emac_poll); ret = devm_register_netdev(dev, netdev); if (ret) { diff --git a/drivers/net/ethernet/adaptec/starfire.c b/drivers/net/ethernet/adaptec/starfire.c index 8f0a6b9c518e..857361c74f5d 100644 --- a/drivers/net/ethernet/adaptec/starfire.c +++ b/drivers/net/ethernet/adaptec/starfire.c @@ -1844,8 +1844,8 @@ static int check_if_running(struct net_device *dev) static void get_drvinfo(struct net_device *dev, struct ethtool_drvinfo *info) { struct netdev_private *np = netdev_priv(dev); - strlcpy(info->driver, DRV_NAME, sizeof(info->driver)); - strlcpy(info->bus_info, pci_name(np->pci_dev), sizeof(info->bus_info)); + strscpy(info->driver, DRV_NAME, sizeof(info->driver)); + strscpy(info->bus_info, pci_name(np->pci_dev), sizeof(info->bus_info)); } static int get_link_ksettings(struct net_device *dev, diff --git a/drivers/net/ethernet/adi/Kconfig b/drivers/net/ethernet/adi/Kconfig new file mode 100644 index 000000000000..da3bdd302502 --- /dev/null +++ b/drivers/net/ethernet/adi/Kconfig @@ -0,0 +1,28 @@ +# SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause +# +# Analog Devices device configuration +# + +config NET_VENDOR_ADI + bool "Analog Devices devices" + default y + depends on SPI + help + If you have a network (Ethernet) card belonging to this class, say Y. + + Note that the answer to this question doesn't directly affect the + kernel: saying N will just cause the configurator to skip all + the questions about ADI devices. If you say Y, you will be asked + for your specific card in the following questions. + +if NET_VENDOR_ADI + +config ADIN1110 + tristate "Analog Devices ADIN1110 MAC-PHY" + depends on SPI && NET_SWITCHDEV + select CRC8 + help + Say yes here to build support for Analog Devices ADIN1110 + Low Power 10BASE-T1L Ethernet MAC-PHY. + +endif # NET_VENDOR_ADI diff --git a/drivers/net/ethernet/adi/Makefile b/drivers/net/ethernet/adi/Makefile new file mode 100644 index 000000000000..d0383d94303c --- /dev/null +++ b/drivers/net/ethernet/adi/Makefile @@ -0,0 +1,6 @@ +# SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause +# +# Makefile for the Analog Devices network device drivers. +# + +obj-$(CONFIG_ADIN1110) += adin1110.o diff --git a/drivers/net/ethernet/adi/adin1110.c b/drivers/net/ethernet/adi/adin1110.c new file mode 100644 index 000000000000..aaee7c4248e6 --- /dev/null +++ b/drivers/net/ethernet/adi/adin1110.c @@ -0,0 +1,1697 @@ +// SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause +/* ADIN1110 Low Power 10BASE-T1L Ethernet MAC-PHY + * ADIN2111 2-Port Ethernet Switch with Integrated 10BASE-T1L PHY + * + * Copyright 2021 Analog Devices Inc. + */ + +#include <linux/bitfield.h> +#include <linux/bits.h> +#include <linux/cache.h> +#include <linux/crc8.h> +#include <linux/etherdevice.h> +#include <linux/ethtool.h> +#include <linux/if_bridge.h> +#include <linux/interrupt.h> +#include <linux/iopoll.h> +#include <linux/gpio.h> +#include <linux/kernel.h> +#include <linux/mii.h> +#include <linux/module.h> +#include <linux/netdevice.h> +#include <linux/regulator/consumer.h> +#include <linux/phy.h> +#include <linux/property.h> +#include <linux/spi/spi.h> + +#include <net/switchdev.h> + +#include <asm/unaligned.h> + +#define ADIN1110_PHY_ID 0x1 + +#define ADIN1110_RESET 0x03 +#define ADIN1110_SWRESET BIT(0) + +#define ADIN1110_CONFIG1 0x04 +#define ADIN1110_CONFIG1_SYNC BIT(15) + +#define ADIN1110_CONFIG2 0x06 +#define ADIN2111_P2_FWD_UNK2HOST BIT(12) +#define ADIN2111_PORT_CUT_THRU_EN BIT(11) +#define ADIN1110_CRC_APPEND BIT(5) +#define ADIN1110_FWD_UNK2HOST BIT(2) + +#define ADIN1110_STATUS0 0x08 + +#define ADIN1110_STATUS1 0x09 +#define ADIN2111_P2_RX_RDY BIT(17) +#define ADIN1110_SPI_ERR BIT(10) +#define ADIN1110_RX_RDY BIT(4) + +#define ADIN1110_IMASK1 0x0D +#define ADIN2111_RX_RDY_IRQ BIT(17) +#define ADIN1110_SPI_ERR_IRQ BIT(10) +#define ADIN1110_RX_RDY_IRQ BIT(4) +#define ADIN1110_TX_RDY_IRQ BIT(3) + +#define ADIN1110_MDIOACC 0x20 +#define ADIN1110_MDIO_TRDONE BIT(31) +#define ADIN1110_MDIO_ST GENMASK(29, 28) +#define ADIN1110_MDIO_OP GENMASK(27, 26) +#define ADIN1110_MDIO_PRTAD GENMASK(25, 21) +#define ADIN1110_MDIO_DEVAD GENMASK(20, 16) +#define ADIN1110_MDIO_DATA GENMASK(15, 0) + +#define ADIN1110_TX_FSIZE 0x30 +#define ADIN1110_TX 0x31 +#define ADIN1110_TX_SPACE 0x32 + +#define ADIN1110_MAC_ADDR_FILTER_UPR 0x50 +#define ADIN2111_MAC_ADDR_APPLY2PORT2 BIT(31) +#define ADIN1110_MAC_ADDR_APPLY2PORT BIT(30) +#define ADIN2111_MAC_ADDR_TO_OTHER_PORT BIT(17) +#define ADIN1110_MAC_ADDR_TO_HOST BIT(16) + +#define ADIN1110_MAC_ADDR_FILTER_LWR 0x51 + +#define ADIN1110_MAC_ADDR_MASK_UPR 0x70 +#define ADIN1110_MAC_ADDR_MASK_LWR 0x71 + +#define ADIN1110_RX_FSIZE 0x90 +#define ADIN1110_RX 0x91 + +#define ADIN2111_RX_P2_FSIZE 0xC0 +#define ADIN2111_RX_P2 0xC1 + +#define ADIN1110_CLEAR_STATUS0 0xFFF + +/* MDIO_OP codes */ +#define ADIN1110_MDIO_OP_WR 0x1 +#define ADIN1110_MDIO_OP_RD 0x3 + +#define ADIN1110_CD BIT(7) +#define ADIN1110_WRITE BIT(5) + +#define ADIN1110_MAX_BUFF 2048 +#define ADIN1110_MAX_FRAMES_READ 64 +#define ADIN1110_WR_HEADER_LEN 2 +#define ADIN1110_FRAME_HEADER_LEN 2 +#define ADIN1110_INTERNAL_SIZE_HEADER_LEN 2 +#define ADIN1110_RD_HEADER_LEN 3 +#define ADIN1110_REG_LEN 4 +#define ADIN1110_FEC_LEN 4 + +#define ADIN1110_PHY_ID_VAL 0x0283BC91 +#define ADIN2111_PHY_ID_VAL 0x0283BCA1 + +#define ADIN_MAC_MAX_PORTS 2 +#define ADIN_MAC_MAX_ADDR_SLOTS 16 + +#define ADIN_MAC_MULTICAST_ADDR_SLOT 0 +#define ADIN_MAC_BROADCAST_ADDR_SLOT 1 +#define ADIN_MAC_P1_ADDR_SLOT 2 +#define ADIN_MAC_P2_ADDR_SLOT 3 +#define ADIN_MAC_FDB_ADDR_SLOT 4 + +DECLARE_CRC8_TABLE(adin1110_crc_table); + +enum adin1110_chips_id { + ADIN1110_MAC = 0, + ADIN2111_MAC, +}; + +struct adin1110_cfg { + enum adin1110_chips_id id; + char name[MDIO_NAME_SIZE]; + u32 phy_ids[PHY_MAX_ADDR]; + u32 ports_nr; + u32 phy_id_val; +}; + +struct adin1110_port_priv { + struct adin1110_priv *priv; + struct net_device *netdev; + struct net_device *bridge; + struct phy_device *phydev; + struct work_struct tx_work; + u64 rx_packets; + u64 tx_packets; + u64 rx_bytes; + u64 tx_bytes; + struct work_struct rx_mode_work; + u32 flags; + struct sk_buff_head txq; + u32 nr; + u32 state; + struct adin1110_cfg *cfg; +}; + +struct adin1110_priv { + struct mutex lock; /* protect spi */ + spinlock_t state_lock; /* protect RX mode */ + struct mii_bus *mii_bus; + struct spi_device *spidev; + bool append_crc; + struct adin1110_cfg *cfg; + u32 tx_space; + u32 irq_mask; + bool forwarding; + int irq; + struct adin1110_port_priv *ports[ADIN_MAC_MAX_PORTS]; + char mii_bus_name[MII_BUS_ID_SIZE]; + u8 data[ADIN1110_MAX_BUFF] ____cacheline_aligned; +}; + +struct adin1110_switchdev_event_work { + struct work_struct work; + struct switchdev_notifier_fdb_info fdb_info; + struct adin1110_port_priv *port_priv; + unsigned long event; +}; + +static struct adin1110_cfg adin1110_cfgs[] = { + { + .id = ADIN1110_MAC, + .name = "adin1110", + .phy_ids = {1}, + .ports_nr = 1, + .phy_id_val = ADIN1110_PHY_ID_VAL, + }, + { + .id = ADIN2111_MAC, + .name = "adin2111", + .phy_ids = {1, 2}, + .ports_nr = 2, + .phy_id_val = ADIN2111_PHY_ID_VAL, + }, +}; + +static u8 adin1110_crc_data(u8 *data, u32 len) +{ + return crc8(adin1110_crc_table, data, len, 0); +} + +static int adin1110_read_reg(struct adin1110_priv *priv, u16 reg, u32 *val) +{ + u32 header_len = ADIN1110_RD_HEADER_LEN; + u32 read_len = ADIN1110_REG_LEN; + struct spi_transfer t[2] = {0}; + int ret; + + priv->data[0] = ADIN1110_CD | FIELD_GET(GENMASK(12, 8), reg); + priv->data[1] = FIELD_GET(GENMASK(7, 0), reg); + priv->data[2] = 0x00; + + if (priv->append_crc) { + priv->data[2] = adin1110_crc_data(&priv->data[0], 2); + priv->data[3] = 0x00; + header_len++; + } + + t[0].tx_buf = &priv->data[0]; + t[0].len = header_len; + + if (priv->append_crc) + read_len++; + + memset(&priv->data[header_len], 0, read_len); + t[1].rx_buf = &priv->data[header_len]; + t[1].len = read_len; + + ret = spi_sync_transfer(priv->spidev, t, 2); + if (ret) + return ret; + + if (priv->append_crc) { + u8 recv_crc; + u8 crc; + + crc = adin1110_crc_data(&priv->data[header_len], + ADIN1110_REG_LEN); + recv_crc = priv->data[header_len + ADIN1110_REG_LEN]; + + if (crc != recv_crc) { + dev_err_ratelimited(&priv->spidev->dev, "CRC error."); + return -EBADMSG; + } + } + + *val = get_unaligned_be32(&priv->data[header_len]); + + return ret; +} + +static int adin1110_write_reg(struct adin1110_priv *priv, u16 reg, u32 val) +{ + u32 header_len = ADIN1110_WR_HEADER_LEN; + u32 write_len = ADIN1110_REG_LEN; + + priv->data[0] = ADIN1110_CD | ADIN1110_WRITE | FIELD_GET(GENMASK(12, 8), reg); + priv->data[1] = FIELD_GET(GENMASK(7, 0), reg); + + if (priv->append_crc) { + priv->data[2] = adin1110_crc_data(&priv->data[0], header_len); + header_len++; + } + + put_unaligned_be32(val, &priv->data[header_len]); + if (priv->append_crc) { + priv->data[header_len + write_len] = adin1110_crc_data(&priv->data[header_len], + write_len); + write_len++; + } + + return spi_write(priv->spidev, &priv->data[0], header_len + write_len); +} + +static int adin1110_set_bits(struct adin1110_priv *priv, u16 reg, + unsigned long mask, unsigned long val) +{ + u32 write_val; + int ret; + + ret = adin1110_read_reg(priv, reg, &write_val); + if (ret < 0) + return ret; + + set_mask_bits(&write_val, mask, val); + + return adin1110_write_reg(priv, reg, write_val); +} + +static int adin1110_round_len(int len) +{ + /* can read/write only mutiples of 4 bytes of payload */ + len = ALIGN(len, 4); + + /* NOTE: ADIN1110_WR_HEADER_LEN should be used for write ops. */ + if (len + ADIN1110_RD_HEADER_LEN > ADIN1110_MAX_BUFF) + return -EINVAL; + + return len; +} + +static int adin1110_read_fifo(struct adin1110_port_priv *port_priv) +{ + struct adin1110_priv *priv = port_priv->priv; + u32 header_len = ADIN1110_RD_HEADER_LEN; + struct spi_transfer t[2] = {0}; + u32 frame_size_no_fcs; + struct sk_buff *rxb; + u32 frame_size; + int round_len; + u16 reg; + int ret; + + if (!port_priv->nr) { + reg = ADIN1110_RX; + ret = adin1110_read_reg(priv, ADIN1110_RX_FSIZE, &frame_size); + } else { + reg = ADIN2111_RX_P2; + ret = adin1110_read_reg(priv, ADIN2111_RX_P2_FSIZE, + &frame_size); + } + + if (ret < 0) + return ret; + + /* The read frame size includes the extra 2 bytes + * from the ADIN1110 frame header. + */ + if (frame_size < ADIN1110_FRAME_HEADER_LEN + ADIN1110_FEC_LEN) + return ret; + + round_len = adin1110_round_len(frame_size); + if (round_len < 0) + return ret; + + frame_size_no_fcs = frame_size - ADIN1110_FRAME_HEADER_LEN - ADIN1110_FEC_LEN; + + rxb = netdev_alloc_skb(port_priv->netdev, round_len); + if (!rxb) + return -ENOMEM; + + memset(priv->data, 0, round_len + ADIN1110_RD_HEADER_LEN); + + priv->data[0] = ADIN1110_CD | FIELD_GET(GENMASK(12, 8), reg); + priv->data[1] = FIELD_GET(GENMASK(7, 0), reg); + + if (priv->append_crc) { + priv->data[2] = adin1110_crc_data(&priv->data[0], 2); + header_len++; + } + + skb_put(rxb, frame_size_no_fcs + ADIN1110_FRAME_HEADER_LEN); + + t[0].tx_buf = &priv->data[0]; + t[0].len = header_len; + + t[1].rx_buf = &rxb->data[0]; + t[1].len = round_len; + + ret = spi_sync_transfer(priv->spidev, t, 2); + if (ret) { + kfree_skb(rxb); + return ret; + } + + skb_pull(rxb, ADIN1110_FRAME_HEADER_LEN); + rxb->protocol = eth_type_trans(rxb, port_priv->netdev); + + if ((port_priv->flags & IFF_ALLMULTI && rxb->pkt_type == PACKET_MULTICAST) || + (port_priv->flags & IFF_BROADCAST && rxb->pkt_type == PACKET_BROADCAST)) + rxb->offload_fwd_mark = 1; + + netif_rx(rxb); + + port_priv->rx_bytes += frame_size - ADIN1110_FRAME_HEADER_LEN; + port_priv->rx_packets++; + + return 0; +} + +static int adin1110_write_fifo(struct adin1110_port_priv *port_priv, + struct sk_buff *txb) +{ + struct adin1110_priv *priv = port_priv->priv; + u32 header_len = ADIN1110_WR_HEADER_LEN; + __be16 frame_header; + int padding = 0; + int padded_len; + int round_len; + int ret; + + /* Pad frame to 64 byte length, + * MAC nor PHY will otherwise add the + * required padding. + * The FEC will be added by the MAC internally. + */ + if (txb->len + ADIN1110_FEC_LEN < 64) + padding = 64 - (txb->len + ADIN1110_FEC_LEN); + + padded_len = txb->len + padding + ADIN1110_FRAME_HEADER_LEN; + + round_len = adin1110_round_len(padded_len); + if (round_len < 0) + return round_len; + + ret = adin1110_write_reg(priv, ADIN1110_TX_FSIZE, padded_len); + if (ret < 0) + return ret; + + memset(priv->data, 0, round_len + ADIN1110_WR_HEADER_LEN); + + priv->data[0] = ADIN1110_CD | ADIN1110_WRITE; + priv->data[0] |= FIELD_GET(GENMASK(12, 8), ADIN1110_TX); + priv->data[1] = FIELD_GET(GENMASK(7, 0), ADIN1110_TX); + if (priv->append_crc) { + priv->data[2] = adin1110_crc_data(&priv->data[0], 2); + header_len++; + } + + /* mention the port on which to send the frame in the frame header */ + frame_header = cpu_to_be16(port_priv->nr); + memcpy(&priv->data[header_len], &frame_header, + ADIN1110_FRAME_HEADER_LEN); + + memcpy(&priv->data[header_len + ADIN1110_FRAME_HEADER_LEN], + txb->data, txb->len); + + ret = spi_write(priv->spidev, &priv->data[0], round_len + header_len); + if (ret < 0) + return ret; + + port_priv->tx_bytes += txb->len; + port_priv->tx_packets++; + + return 0; +} + +static int adin1110_read_mdio_acc(struct adin1110_priv *priv) +{ + u32 val; + int ret; + + mutex_lock(&priv->lock); + ret = adin1110_read_reg(priv, ADIN1110_MDIOACC, &val); + mutex_unlock(&priv->lock); + if (ret < 0) + return 0; + + return val; +} + +static int adin1110_mdio_read(struct mii_bus *bus, int phy_id, int reg) +{ + struct adin1110_priv *priv = bus->priv; + u32 val = 0; + int ret; + + if (mdio_phy_id_is_c45(phy_id)) + return -EOPNOTSUPP; + + val |= FIELD_PREP(ADIN1110_MDIO_OP, ADIN1110_MDIO_OP_RD); + val |= FIELD_PREP(ADIN1110_MDIO_ST, 0x1); + val |= FIELD_PREP(ADIN1110_MDIO_PRTAD, phy_id); + val |= FIELD_PREP(ADIN1110_MDIO_DEVAD, reg); + + /* write the clause 22 read command to the chip */ + mutex_lock(&priv->lock); + ret = adin1110_write_reg(priv, ADIN1110_MDIOACC, val); + mutex_unlock(&priv->lock); + if (ret < 0) + return ret; + + /* ADIN1110_MDIO_TRDONE BIT of the ADIN1110_MDIOACC + * register is set when the read is done. + * After the transaction is done, ADIN1110_MDIO_DATA + * bitfield of ADIN1110_MDIOACC register will contain + * the requested register value. + */ + ret = readx_poll_timeout(adin1110_read_mdio_acc, priv, val, + (val & ADIN1110_MDIO_TRDONE), 10000, 30000); + if (ret < 0) + return ret; + + return (val & ADIN1110_MDIO_DATA); +} + +static int adin1110_mdio_write(struct mii_bus *bus, int phy_id, + int reg, u16 reg_val) +{ + struct adin1110_priv *priv = bus->priv; + u32 val = 0; + int ret; + + if (mdio_phy_id_is_c45(phy_id)) + return -EOPNOTSUPP; + + val |= FIELD_PREP(ADIN1110_MDIO_OP, ADIN1110_MDIO_OP_WR); + val |= FIELD_PREP(ADIN1110_MDIO_ST, 0x1); + val |= FIELD_PREP(ADIN1110_MDIO_PRTAD, phy_id); + val |= FIELD_PREP(ADIN1110_MDIO_DEVAD, reg); + val |= FIELD_PREP(ADIN1110_MDIO_DATA, reg_val); + + /* write the clause 22 write command to the chip */ + mutex_lock(&priv->lock); + ret = adin1110_write_reg(priv, ADIN1110_MDIOACC, val); + mutex_unlock(&priv->lock); + if (ret < 0) + return ret; + + return readx_poll_timeout(adin1110_read_mdio_acc, priv, val, + (val & ADIN1110_MDIO_TRDONE), 10000, 30000); +} + +/* ADIN1110 MAC-PHY contains an ADIN1100 PHY. + * ADIN2111 MAC-PHY contains two ADIN1100 PHYs. + * By registering a new MDIO bus we allow the PAL to discover + * the encapsulated PHY and probe the ADIN1100 driver. + */ +static int adin1110_register_mdiobus(struct adin1110_priv *priv, + struct device *dev) +{ + struct mii_bus *mii_bus; + int ret; + + mii_bus = devm_mdiobus_alloc(dev); + if (!mii_bus) + return -ENOMEM; + + snprintf(priv->mii_bus_name, MII_BUS_ID_SIZE, "%s-%u", + priv->cfg->name, priv->spidev->chip_select); + + mii_bus->name = priv->mii_bus_name; + mii_bus->read = adin1110_mdio_read; + mii_bus->write = adin1110_mdio_write; + mii_bus->priv = priv; + mii_bus->parent = dev; + mii_bus->phy_mask = ~((u32)GENMASK(2, 0)); + mii_bus->probe_capabilities = MDIOBUS_C22; + snprintf(mii_bus->id, MII_BUS_ID_SIZE, "%s", dev_name(dev)); + + ret = devm_mdiobus_register(dev, mii_bus); + if (ret) + return ret; + + priv->mii_bus = mii_bus; + + return 0; +} + +static bool adin1110_port_rx_ready(struct adin1110_port_priv *port_priv, + u32 status) +{ + if (!netif_oper_up(port_priv->netdev)) + return false; + + if (!port_priv->nr) + return !!(status & ADIN1110_RX_RDY); + else + return !!(status & ADIN2111_P2_RX_RDY); +} + +static void adin1110_read_frames(struct adin1110_port_priv *port_priv, + unsigned int budget) +{ + struct adin1110_priv *priv = port_priv->priv; + u32 status1; + int ret; + + while (budget) { + ret = adin1110_read_reg(priv, ADIN1110_STATUS1, &status1); + if (ret < 0) + return; + + if (!adin1110_port_rx_ready(port_priv, status1)) + break; + + ret = adin1110_read_fifo(port_priv); + if (ret < 0) + return; + + budget--; + } +} + +static void adin1110_wake_queues(struct adin1110_priv *priv) +{ + int i; + + for (i = 0; i < priv->cfg->ports_nr; i++) + netif_wake_queue(priv->ports[i]->netdev); +} + +static irqreturn_t adin1110_irq(int irq, void *p) +{ + struct adin1110_priv *priv = p; + u32 status1; + u32 val; + int ret; + int i; + + mutex_lock(&priv->lock); + + ret = adin1110_read_reg(priv, ADIN1110_STATUS1, &status1); + if (ret < 0) + goto out; + + if (priv->append_crc && (status1 & ADIN1110_SPI_ERR)) + dev_warn_ratelimited(&priv->spidev->dev, + "SPI CRC error on write.\n"); + + ret = adin1110_read_reg(priv, ADIN1110_TX_SPACE, &val); + if (ret < 0) + goto out; + + /* TX FIFO space is expressed in half-words */ + priv->tx_space = 2 * val; + + for (i = 0; i < priv->cfg->ports_nr; i++) { + if (adin1110_port_rx_ready(priv->ports[i], status1)) + adin1110_read_frames(priv->ports[i], + ADIN1110_MAX_FRAMES_READ); + } + + /* clear IRQ sources */ + adin1110_write_reg(priv, ADIN1110_STATUS0, ADIN1110_CLEAR_STATUS0); + adin1110_write_reg(priv, ADIN1110_STATUS1, priv->irq_mask); + +out: + mutex_unlock(&priv->lock); + + if (priv->tx_space > 0 && ret >= 0) + adin1110_wake_queues(priv); + + return IRQ_HANDLED; +} + +/* ADIN1110 can filter up to 16 MAC addresses, mac_nr here is the slot used */ +static int adin1110_write_mac_address(struct adin1110_port_priv *port_priv, + int mac_nr, const u8 *addr, + u8 *mask, u32 port_rules) +{ + struct adin1110_priv *priv = port_priv->priv; + u32 offset = mac_nr * 2; + u32 port_rules_mask; + int ret; + u32 val; + + if (!port_priv->nr) + port_rules_mask = ADIN1110_MAC_ADDR_APPLY2PORT; + else + port_rules_mask = ADIN2111_MAC_ADDR_APPLY2PORT2; + + if (port_rules & port_rules_mask) + port_rules_mask |= ADIN1110_MAC_ADDR_TO_HOST | ADIN2111_MAC_ADDR_TO_OTHER_PORT; + + port_rules_mask |= GENMASK(15, 0); + val = port_rules | get_unaligned_be16(&addr[0]); + ret = adin1110_set_bits(priv, ADIN1110_MAC_ADDR_FILTER_UPR + offset, + port_rules_mask, val); + if (ret < 0) + return ret; + + val = get_unaligned_be32(&addr[2]); + ret = adin1110_write_reg(priv, + ADIN1110_MAC_ADDR_FILTER_LWR + offset, val); + if (ret < 0) + return ret; + + /* Only the first two MAC address slots support masking. */ + if (mac_nr < ADIN_MAC_P1_ADDR_SLOT) { + val = get_unaligned_be16(&mask[0]); + ret = adin1110_write_reg(priv, + ADIN1110_MAC_ADDR_MASK_UPR + offset, + val); + if (ret < 0) + return ret; + + val = get_unaligned_be32(&mask[2]); + return adin1110_write_reg(priv, + ADIN1110_MAC_ADDR_MASK_LWR + offset, + val); + } + + return 0; +} + +static int adin1110_clear_mac_address(struct adin1110_priv *priv, int mac_nr) +{ + u32 offset = mac_nr * 2; + int ret; + + ret = adin1110_write_reg(priv, ADIN1110_MAC_ADDR_FILTER_UPR + offset, 0); + if (ret < 0) + return ret; + + ret = adin1110_write_reg(priv, ADIN1110_MAC_ADDR_FILTER_LWR + offset, 0); + if (ret < 0) + return ret; + + /* only the first two MAC address slots are maskable */ + if (mac_nr <= 1) { + ret = adin1110_write_reg(priv, ADIN1110_MAC_ADDR_MASK_UPR + offset, 0); + if (ret < 0) + return ret; + + ret = adin1110_write_reg(priv, ADIN1110_MAC_ADDR_MASK_LWR + offset, 0); + } + + return ret; +} + +static u32 adin1110_port_rules(struct adin1110_port_priv *port_priv, + bool fw_to_host, + bool fw_to_other_port) +{ + u32 port_rules = 0; + + if (!port_priv->nr) + port_rules |= ADIN1110_MAC_ADDR_APPLY2PORT; + else + port_rules |= ADIN2111_MAC_ADDR_APPLY2PORT2; + + if (fw_to_host) + port_rules |= ADIN1110_MAC_ADDR_TO_HOST; + + if (fw_to_other_port && port_priv->priv->forwarding) + port_rules |= ADIN2111_MAC_ADDR_TO_OTHER_PORT; + + return port_rules; +} + +static int adin1110_multicast_filter(struct adin1110_port_priv *port_priv, + int mac_nr, bool accept_multicast) +{ + u8 mask[ETH_ALEN] = {0}; + u8 mac[ETH_ALEN] = {0}; + u32 port_rules = 0; + + mask[0] = BIT(0); + mac[0] = BIT(0); + + if (accept_multicast && port_priv->state == BR_STATE_FORWARDING) + port_rules = adin1110_port_rules(port_priv, true, true); + + return adin1110_write_mac_address(port_priv, mac_nr, mac, + mask, port_rules); +} + +static int adin1110_broadcasts_filter(struct adin1110_port_priv *port_priv, + int mac_nr, bool accept_broadcast) +{ + u32 port_rules = 0; + u8 mask[ETH_ALEN]; + + memset(mask, 0xFF, ETH_ALEN); + + if (accept_broadcast && port_priv->state == BR_STATE_FORWARDING) + port_rules = adin1110_port_rules(port_priv, true, true); + + return adin1110_write_mac_address(port_priv, mac_nr, mask, + mask, port_rules); +} + +static int adin1110_set_mac_address(struct net_device *netdev, + const unsigned char *dev_addr) +{ + struct adin1110_port_priv *port_priv = netdev_priv(netdev); + u8 mask[ETH_ALEN]; + u32 port_rules; + u32 mac_slot; + + if (!is_valid_ether_addr(dev_addr)) + return -EADDRNOTAVAIL; + + eth_hw_addr_set(netdev, dev_addr); + memset(mask, 0xFF, ETH_ALEN); + + mac_slot = (!port_priv->nr) ? ADIN_MAC_P1_ADDR_SLOT : ADIN_MAC_P2_ADDR_SLOT; + port_rules = adin1110_port_rules(port_priv, true, false); + + return adin1110_write_mac_address(port_priv, mac_slot, netdev->dev_addr, + mask, port_rules); +} + +static int adin1110_ndo_set_mac_address(struct net_device *netdev, void *addr) +{ + struct sockaddr *sa = addr; + int ret; + + ret = eth_prepare_mac_addr_change(netdev, addr); + if (ret < 0) + return ret; + + return adin1110_set_mac_address(netdev, sa->sa_data); +} + +static int adin1110_ioctl(struct net_device *netdev, struct ifreq *rq, int cmd) +{ + if (!netif_running(netdev)) + return -EINVAL; + + return phy_do_ioctl(netdev, rq, cmd); +} + +static int adin1110_set_promisc_mode(struct adin1110_port_priv *port_priv, + bool promisc) +{ + struct adin1110_priv *priv = port_priv->priv; + u32 mask; + + if (port_priv->state != BR_STATE_FORWARDING) + promisc = false; + + if (!port_priv->nr) + mask = ADIN1110_FWD_UNK2HOST; + else + mask = ADIN2111_P2_FWD_UNK2HOST; + + return adin1110_set_bits(priv, ADIN1110_CONFIG2, + mask, promisc ? mask : 0); +} + +static int adin1110_setup_rx_mode(struct adin1110_port_priv *port_priv) +{ + int ret; + + ret = adin1110_set_promisc_mode(port_priv, + !!(port_priv->flags & IFF_PROMISC)); + if (ret < 0) + return ret; + + ret = adin1110_multicast_filter(port_priv, ADIN_MAC_MULTICAST_ADDR_SLOT, + !!(port_priv->flags & IFF_ALLMULTI)); + if (ret < 0) + return ret; + + ret = adin1110_broadcasts_filter(port_priv, + ADIN_MAC_BROADCAST_ADDR_SLOT, + !!(port_priv->flags & IFF_BROADCAST)); + if (ret < 0) + return ret; + + return adin1110_set_bits(port_priv->priv, ADIN1110_CONFIG1, + ADIN1110_CONFIG1_SYNC, ADIN1110_CONFIG1_SYNC); +} + +static bool adin1110_can_offload_forwarding(struct adin1110_priv *priv) +{ + int i; + + if (priv->cfg->id != ADIN2111_MAC) + return false; + + /* Can't enable forwarding if ports do not belong to the same bridge */ + if (priv->ports[0]->bridge != priv->ports[1]->bridge || !priv->ports[0]->bridge) + return false; + + /* Can't enable forwarding if there is a port + * that has been blocked by STP. + */ + for (i = 0; i < priv->cfg->ports_nr; i++) { + if (priv->ports[i]->state != BR_STATE_FORWARDING) + return false; + } + + return true; +} + +static void adin1110_rx_mode_work(struct work_struct *work) +{ + struct adin1110_port_priv *port_priv; + struct adin1110_priv *priv; + + port_priv = container_of(work, struct adin1110_port_priv, rx_mode_work); + priv = port_priv->priv; + + mutex_lock(&priv->lock); + adin1110_setup_rx_mode(port_priv); + mutex_unlock(&priv->lock); +} + +static void adin1110_set_rx_mode(struct net_device *dev) +{ + struct adin1110_port_priv *port_priv = netdev_priv(dev); + struct adin1110_priv *priv = port_priv->priv; + + spin_lock(&priv->state_lock); + + port_priv->flags = dev->flags; + schedule_work(&port_priv->rx_mode_work); + + spin_unlock(&priv->state_lock); +} + +static int adin1110_net_open(struct net_device *net_dev) +{ + struct adin1110_port_priv *port_priv = netdev_priv(net_dev); + struct adin1110_priv *priv = port_priv->priv; + u32 val; + int ret; + + mutex_lock(&priv->lock); + + /* Configure MAC to compute and append the FCS itself. */ + ret = adin1110_write_reg(priv, ADIN1110_CONFIG2, ADIN1110_CRC_APPEND); + if (ret < 0) + goto out; + + val = ADIN1110_TX_RDY_IRQ | ADIN1110_RX_RDY_IRQ | ADIN1110_SPI_ERR_IRQ; + if (priv->cfg->id == ADIN2111_MAC) + val |= ADIN2111_RX_RDY_IRQ; + + priv->irq_mask = val; + ret = adin1110_write_reg(priv, ADIN1110_IMASK1, ~val); + if (ret < 0) { + netdev_err(net_dev, "Failed to enable chip IRQs: %d\n", ret); + goto out; + } + + ret = adin1110_read_reg(priv, ADIN1110_TX_SPACE, &val); + if (ret < 0) { + netdev_err(net_dev, "Failed to read TX FIFO space: %d\n", ret); + goto out; + } + + priv->tx_space = 2 * val; + + port_priv->state = BR_STATE_FORWARDING; + ret = adin1110_set_mac_address(net_dev, net_dev->dev_addr); + if (ret < 0) { + netdev_err(net_dev, "Could not set MAC address: %pM, %d\n", + net_dev->dev_addr, ret); + goto out; + } + + ret = adin1110_set_bits(priv, ADIN1110_CONFIG1, ADIN1110_CONFIG1_SYNC, + ADIN1110_CONFIG1_SYNC); + +out: + mutex_unlock(&priv->lock); + + if (ret < 0) + return ret; + + phy_start(port_priv->phydev); + + netif_start_queue(net_dev); + + return 0; +} + +static int adin1110_net_stop(struct net_device *net_dev) +{ + struct adin1110_port_priv *port_priv = netdev_priv(net_dev); + struct adin1110_priv *priv = port_priv->priv; + u32 mask; + int ret; + + mask = !port_priv->nr ? ADIN2111_RX_RDY_IRQ : ADIN1110_RX_RDY_IRQ; + + /* Disable RX RDY IRQs */ + mutex_lock(&priv->lock); + ret = adin1110_set_bits(priv, ADIN1110_IMASK1, mask, mask); + mutex_unlock(&priv->lock); + if (ret < 0) + return ret; + + netif_stop_queue(port_priv->netdev); + flush_work(&port_priv->tx_work); + phy_stop(port_priv->phydev); + + return 0; +} + +static void adin1110_tx_work(struct work_struct *work) +{ + struct adin1110_port_priv *port_priv; + struct adin1110_priv *priv; + struct sk_buff *txb; + int ret; + + port_priv = container_of(work, struct adin1110_port_priv, tx_work); + priv = port_priv->priv; + + mutex_lock(&priv->lock); + + while ((txb = skb_dequeue(&port_priv->txq))) { + ret = adin1110_write_fifo(port_priv, txb); + if (ret < 0) + dev_err_ratelimited(&priv->spidev->dev, + "Frame write error: %d\n", ret); + + dev_kfree_skb(txb); + } + + mutex_unlock(&priv->lock); +} + +static netdev_tx_t adin1110_start_xmit(struct sk_buff *skb, struct net_device *dev) +{ + struct adin1110_port_priv *port_priv = netdev_priv(dev); + struct adin1110_priv *priv = port_priv->priv; + netdev_tx_t netdev_ret = NETDEV_TX_OK; + u32 tx_space_needed; + + tx_space_needed = skb->len + ADIN1110_FRAME_HEADER_LEN + ADIN1110_INTERNAL_SIZE_HEADER_LEN; + if (tx_space_needed > priv->tx_space) { + netif_stop_queue(dev); + netdev_ret = NETDEV_TX_BUSY; + } else { + priv->tx_space -= tx_space_needed; + skb_queue_tail(&port_priv->txq, skb); + } + + schedule_work(&port_priv->tx_work); + + return netdev_ret; +} + +static void adin1110_ndo_get_stats64(struct net_device *dev, + struct rtnl_link_stats64 *storage) +{ + struct adin1110_port_priv *port_priv = netdev_priv(dev); + + storage->rx_packets = port_priv->rx_packets; + storage->tx_packets = port_priv->tx_packets; + + storage->rx_bytes = port_priv->rx_bytes; + storage->tx_bytes = port_priv->tx_bytes; +} + +static int adin1110_port_get_port_parent_id(struct net_device *dev, + struct netdev_phys_item_id *ppid) +{ + struct adin1110_port_priv *port_priv = netdev_priv(dev); + struct adin1110_priv *priv = port_priv->priv; + + ppid->id_len = strnlen(priv->mii_bus_name, MAX_PHYS_ITEM_ID_LEN); + memcpy(ppid->id, priv->mii_bus_name, ppid->id_len); + + return 0; +} + +static int adin1110_ndo_get_phys_port_name(struct net_device *dev, + char *name, size_t len) +{ + struct adin1110_port_priv *port_priv = netdev_priv(dev); + int err; + + err = snprintf(name, len, "p%d", port_priv->nr); + if (err >= len) + return -EINVAL; + + return 0; +} + +static const struct net_device_ops adin1110_netdev_ops = { + .ndo_open = adin1110_net_open, + .ndo_stop = adin1110_net_stop, + .ndo_eth_ioctl = adin1110_ioctl, + .ndo_start_xmit = adin1110_start_xmit, + .ndo_set_mac_address = adin1110_ndo_set_mac_address, + .ndo_set_rx_mode = adin1110_set_rx_mode, + .ndo_validate_addr = eth_validate_addr, + .ndo_get_stats64 = adin1110_ndo_get_stats64, + .ndo_get_port_parent_id = adin1110_port_get_port_parent_id, + .ndo_get_phys_port_name = adin1110_ndo_get_phys_port_name, +}; + +static void adin1110_get_drvinfo(struct net_device *dev, + struct ethtool_drvinfo *di) +{ + strscpy(di->driver, "ADIN1110", sizeof(di->driver)); + strscpy(di->bus_info, dev_name(dev->dev.parent), sizeof(di->bus_info)); +} + +static const struct ethtool_ops adin1110_ethtool_ops = { + .get_drvinfo = adin1110_get_drvinfo, + .get_link = ethtool_op_get_link, + .get_link_ksettings = phy_ethtool_get_link_ksettings, + .set_link_ksettings = phy_ethtool_set_link_ksettings, +}; + +static void adin1110_adjust_link(struct net_device *dev) +{ + struct phy_device *phydev = dev->phydev; + + if (!phydev->link) + phy_print_status(phydev); +} + +/* PHY ID is stored in the MAC registers too, + * check spi connection by reading it. + */ +static int adin1110_check_spi(struct adin1110_priv *priv) +{ + int ret; + u32 val; + + ret = adin1110_read_reg(priv, ADIN1110_PHY_ID, &val); + if (ret < 0) + return ret; + + if (val != priv->cfg->phy_id_val) { + dev_err(&priv->spidev->dev, "PHY ID expected: %x, read: %x\n", + priv->cfg->phy_id_val, val); + return -EIO; + } + + return 0; +} + +static int adin1110_hw_forwarding(struct adin1110_priv *priv, bool enable) +{ + int ret; + int i; + + priv->forwarding = enable; + + if (!priv->forwarding) { + for (i = ADIN_MAC_FDB_ADDR_SLOT; i < ADIN_MAC_MAX_ADDR_SLOTS; i++) { + ret = adin1110_clear_mac_address(priv, i); + if (ret < 0) + return ret; + } + } + + /* Forwarding is optimised when MAC runs in Cut Through mode. */ + ret = adin1110_set_bits(priv, ADIN1110_CONFIG2, + ADIN2111_PORT_CUT_THRU_EN, + priv->forwarding ? ADIN2111_PORT_CUT_THRU_EN : 0); + if (ret < 0) + return ret; + + for (i = 0; i < priv->cfg->ports_nr; i++) { + ret = adin1110_setup_rx_mode(priv->ports[i]); + if (ret < 0) + return ret; + } + + return ret; +} + +static int adin1110_port_bridge_join(struct adin1110_port_priv *port_priv, + struct net_device *bridge) +{ + struct adin1110_priv *priv = port_priv->priv; + int ret; + + port_priv->bridge = bridge; + + if (adin1110_can_offload_forwarding(priv)) { + mutex_lock(&priv->lock); + ret = adin1110_hw_forwarding(priv, true); + mutex_unlock(&priv->lock); + + if (ret < 0) + return ret; + } + + return adin1110_set_mac_address(port_priv->netdev, bridge->dev_addr); +} + +static int adin1110_port_bridge_leave(struct adin1110_port_priv *port_priv, + struct net_device *bridge) +{ + struct adin1110_priv *priv = port_priv->priv; + int ret; + + port_priv->bridge = NULL; + + mutex_lock(&priv->lock); + ret = adin1110_hw_forwarding(priv, false); + mutex_unlock(&priv->lock); + + return ret; +} + +static int adin1110_netdevice_event(struct notifier_block *unused, + unsigned long event, void *ptr) +{ + struct net_device *dev = netdev_notifier_info_to_dev(ptr); + struct adin1110_port_priv *port_priv = netdev_priv(dev); + struct netdev_notifier_changeupper_info *info = ptr; + int ret = 0; + + switch (event) { + case NETDEV_CHANGEUPPER: + if (netif_is_bridge_master(info->upper_dev)) { + if (info->linking) + ret = adin1110_port_bridge_join(port_priv, info->upper_dev); + else + ret = adin1110_port_bridge_leave(port_priv, info->upper_dev); + } + break; + default: + break; + } + + return notifier_from_errno(ret); +} + +static struct notifier_block adin1110_netdevice_nb = { + .notifier_call = adin1110_netdevice_event, +}; + +static void adin1110_disconnect_phy(void *data) +{ + phy_disconnect(data); +} + +static bool adin1110_port_dev_check(const struct net_device *dev) +{ + return dev->netdev_ops == &adin1110_netdev_ops; +} + +static int adin1110_port_set_forwarding_state(struct adin1110_port_priv *port_priv) +{ + struct adin1110_priv *priv = port_priv->priv; + int ret; + + port_priv->state = BR_STATE_FORWARDING; + + mutex_lock(&priv->lock); + ret = adin1110_set_mac_address(port_priv->netdev, + port_priv->netdev->dev_addr); + if (ret < 0) + goto out; + + if (adin1110_can_offload_forwarding(priv)) + ret = adin1110_hw_forwarding(priv, true); + else + ret = adin1110_setup_rx_mode(port_priv); +out: + mutex_unlock(&priv->lock); + + return ret; +} + +static int adin1110_port_set_blocking_state(struct adin1110_port_priv *port_priv) +{ + u8 mac[ETH_ALEN] = {0x01, 0x80, 0xC2, 0x00, 0x00, 0x00}; + struct adin1110_priv *priv = port_priv->priv; + u8 mask[ETH_ALEN]; + u32 port_rules; + int mac_slot; + int ret; + + port_priv->state = BR_STATE_BLOCKING; + + mutex_lock(&priv->lock); + + mac_slot = (!port_priv->nr) ? ADIN_MAC_P1_ADDR_SLOT : ADIN_MAC_P2_ADDR_SLOT; + ret = adin1110_clear_mac_address(priv, mac_slot); + if (ret < 0) + goto out; + + ret = adin1110_hw_forwarding(priv, false); + if (ret < 0) + goto out; + + /* Allow only BPDUs to be passed to the CPU */ + memset(mask, 0xFF, ETH_ALEN); + port_rules = adin1110_port_rules(port_priv, true, false); + ret = adin1110_write_mac_address(port_priv, mac_slot, mac, + mask, port_rules); +out: + mutex_unlock(&priv->lock); + + return ret; +} + +/* ADIN1110/2111 does not have any native STP support. + * Listen for bridge core state changes and + * allow all frames to pass or only the BPDUs. + */ +static int adin1110_port_attr_stp_state_set(struct adin1110_port_priv *port_priv, + u8 state) +{ + switch (state) { + case BR_STATE_FORWARDING: + return adin1110_port_set_forwarding_state(port_priv); + case BR_STATE_LEARNING: + case BR_STATE_LISTENING: + case BR_STATE_DISABLED: + case BR_STATE_BLOCKING: + return adin1110_port_set_blocking_state(port_priv); + default: + return -EINVAL; + } +} + +static int adin1110_port_attr_set(struct net_device *dev, const void *ctx, + const struct switchdev_attr *attr, + struct netlink_ext_ack *extack) +{ + struct adin1110_port_priv *port_priv = netdev_priv(dev); + + switch (attr->id) { + case SWITCHDEV_ATTR_ID_PORT_STP_STATE: + return adin1110_port_attr_stp_state_set(port_priv, + attr->u.stp_state); + default: + return -EOPNOTSUPP; + } +} + +static int adin1110_switchdev_blocking_event(struct notifier_block *unused, + unsigned long event, + void *ptr) +{ + struct net_device *netdev = switchdev_notifier_info_to_dev(ptr); + int ret; + + if (event == SWITCHDEV_PORT_ATTR_SET) { + ret = switchdev_handle_port_attr_set(netdev, ptr, + adin1110_port_dev_check, + adin1110_port_attr_set); + + return notifier_from_errno(ret); + } + + return NOTIFY_DONE; +} + +static struct notifier_block adin1110_switchdev_blocking_notifier = { + .notifier_call = adin1110_switchdev_blocking_event, +}; + +static void adin1110_fdb_offload_notify(struct net_device *netdev, + struct switchdev_notifier_fdb_info *rcv) +{ + struct switchdev_notifier_fdb_info info = {}; + + info.addr = rcv->addr; + info.vid = rcv->vid; + info.offloaded = true; + call_switchdev_notifiers(SWITCHDEV_FDB_OFFLOADED, + netdev, &info.info, NULL); +} + +static int adin1110_fdb_add(struct adin1110_port_priv *port_priv, + struct switchdev_notifier_fdb_info *fdb) +{ + struct adin1110_priv *priv = port_priv->priv; + struct adin1110_port_priv *other_port; + u8 mask[ETH_ALEN]; + u32 port_rules; + int mac_nr; + u32 val; + int ret; + + netdev_dbg(port_priv->netdev, + "DEBUG: %s: MACID = %pM vid = %u flags = %u %u -- port %d\n", + __func__, fdb->addr, fdb->vid, fdb->added_by_user, + fdb->offloaded, port_priv->nr); + + if (!priv->forwarding) + return 0; + + if (fdb->is_local) + return -EINVAL; + + /* Find free FDB slot on device. */ + for (mac_nr = ADIN_MAC_FDB_ADDR_SLOT; mac_nr < ADIN_MAC_MAX_ADDR_SLOTS; mac_nr++) { + ret = adin1110_read_reg(priv, ADIN1110_MAC_ADDR_FILTER_UPR + (mac_nr * 2), &val); + if (ret < 0) + return ret; + if (!val) + break; + } + + if (mac_nr == ADIN_MAC_MAX_ADDR_SLOTS) + return -ENOMEM; + + other_port = priv->ports[!port_priv->nr]; + port_rules = adin1110_port_rules(port_priv, false, true); + memset(mask, 0xFF, ETH_ALEN); + + return adin1110_write_mac_address(other_port, mac_nr, (u8 *)fdb->addr, + mask, port_rules); +} + +static int adin1110_read_mac(struct adin1110_priv *priv, int mac_nr, u8 *addr) +{ + u32 val; + int ret; + + ret = adin1110_read_reg(priv, ADIN1110_MAC_ADDR_FILTER_UPR + (mac_nr * 2), &val); + if (ret < 0) + return ret; + + put_unaligned_be16(val, addr); + + ret = adin1110_read_reg(priv, ADIN1110_MAC_ADDR_FILTER_LWR + (mac_nr * 2), &val); + if (ret < 0) + return ret; + + put_unaligned_be32(val, addr + 2); + + return 0; +} + +static int adin1110_fdb_del(struct adin1110_port_priv *port_priv, + struct switchdev_notifier_fdb_info *fdb) +{ + struct adin1110_priv *priv = port_priv->priv; + u8 addr[ETH_ALEN]; + int mac_nr; + int ret; + + netdev_dbg(port_priv->netdev, + "DEBUG: %s: MACID = %pM vid = %u flags = %u %u -- port %d\n", + __func__, fdb->addr, fdb->vid, fdb->added_by_user, + fdb->offloaded, port_priv->nr); + + if (fdb->is_local) + return -EINVAL; + + for (mac_nr = ADIN_MAC_FDB_ADDR_SLOT; mac_nr < ADIN_MAC_MAX_ADDR_SLOTS; mac_nr++) { + ret = adin1110_read_mac(priv, mac_nr, addr); + if (ret < 0) + return ret; + + if (ether_addr_equal(addr, fdb->addr)) { + ret = adin1110_clear_mac_address(priv, mac_nr); + if (ret < 0) + return ret; + } + } + + return 0; +} + +static void adin1110_switchdev_event_work(struct work_struct *work) +{ + struct adin1110_switchdev_event_work *switchdev_work; + struct adin1110_port_priv *port_priv; + int ret; + + switchdev_work = container_of(work, struct adin1110_switchdev_event_work, work); + port_priv = switchdev_work->port_priv; + + mutex_lock(&port_priv->priv->lock); + + switch (switchdev_work->event) { + case SWITCHDEV_FDB_ADD_TO_DEVICE: + ret = adin1110_fdb_add(port_priv, &switchdev_work->fdb_info); + if (!ret) + adin1110_fdb_offload_notify(port_priv->netdev, + &switchdev_work->fdb_info); + break; + case SWITCHDEV_FDB_DEL_TO_DEVICE: + adin1110_fdb_del(port_priv, &switchdev_work->fdb_info); + break; + default: + break; + } + + mutex_unlock(&port_priv->priv->lock); + + kfree(switchdev_work->fdb_info.addr); + kfree(switchdev_work); + dev_put(port_priv->netdev); +} + +/* called under rcu_read_lock() */ +static int adin1110_switchdev_event(struct notifier_block *unused, + unsigned long event, void *ptr) +{ + struct net_device *netdev = switchdev_notifier_info_to_dev(ptr); + struct adin1110_port_priv *port_priv = netdev_priv(netdev); + struct adin1110_switchdev_event_work *switchdev_work; + struct switchdev_notifier_fdb_info *fdb_info = ptr; + + if (!adin1110_port_dev_check(netdev)) + return NOTIFY_DONE; + + switchdev_work = kzalloc(sizeof(*switchdev_work), GFP_ATOMIC); + if (WARN_ON(!switchdev_work)) + return NOTIFY_BAD; + + INIT_WORK(&switchdev_work->work, adin1110_switchdev_event_work); + switchdev_work->port_priv = port_priv; + switchdev_work->event = event; + + switch (event) { + case SWITCHDEV_FDB_ADD_TO_DEVICE: + case SWITCHDEV_FDB_DEL_TO_DEVICE: + memcpy(&switchdev_work->fdb_info, ptr, + sizeof(switchdev_work->fdb_info)); + switchdev_work->fdb_info.addr = kzalloc(ETH_ALEN, GFP_ATOMIC); + + if (!switchdev_work->fdb_info.addr) + goto err_addr_alloc; + + ether_addr_copy((u8 *)switchdev_work->fdb_info.addr, + fdb_info->addr); + dev_hold(netdev); + break; + default: + kfree(switchdev_work); + return NOTIFY_DONE; + } + + queue_work(system_long_wq, &switchdev_work->work); + + return NOTIFY_DONE; + +err_addr_alloc: + kfree(switchdev_work); + return NOTIFY_BAD; +} + +static struct notifier_block adin1110_switchdev_notifier = { + .notifier_call = adin1110_switchdev_event, +}; + +static void adin1110_unregister_notifiers(void *data) +{ + unregister_switchdev_blocking_notifier(&adin1110_switchdev_blocking_notifier); + unregister_switchdev_notifier(&adin1110_switchdev_notifier); + unregister_netdevice_notifier(&adin1110_netdevice_nb); +} + +static int adin1110_setup_notifiers(struct adin1110_priv *priv) +{ + struct device *dev = &priv->spidev->dev; + int ret; + + ret = register_netdevice_notifier(&adin1110_netdevice_nb); + if (ret < 0) + return ret; + + ret = register_switchdev_notifier(&adin1110_switchdev_notifier); + if (ret < 0) + goto err_netdev; + + ret = register_switchdev_blocking_notifier(&adin1110_switchdev_blocking_notifier); + if (ret < 0) + goto err_sdev; + + return devm_add_action_or_reset(dev, adin1110_unregister_notifiers, NULL); + +err_sdev: + unregister_switchdev_notifier(&adin1110_switchdev_notifier); + +err_netdev: + unregister_netdevice_notifier(&adin1110_netdevice_nb); + return ret; +} + +static int adin1110_probe_netdevs(struct adin1110_priv *priv) +{ + struct device *dev = &priv->spidev->dev; + struct adin1110_port_priv *port_priv; + struct net_device *netdev; + int ret; + int i; + + for (i = 0; i < priv->cfg->ports_nr; i++) { + netdev = devm_alloc_etherdev(dev, sizeof(*port_priv)); + if (!netdev) + return -ENOMEM; + + port_priv = netdev_priv(netdev); + port_priv->netdev = netdev; + port_priv->priv = priv; + port_priv->cfg = priv->cfg; + port_priv->nr = i; + priv->ports[i] = port_priv; + SET_NETDEV_DEV(netdev, dev); + + ret = device_get_ethdev_address(dev, netdev); + if (ret < 0) + return ret; + + netdev->irq = priv->spidev->irq; + INIT_WORK(&port_priv->tx_work, adin1110_tx_work); + INIT_WORK(&port_priv->rx_mode_work, adin1110_rx_mode_work); + skb_queue_head_init(&port_priv->txq); + + netif_carrier_off(netdev); + + netdev->if_port = IF_PORT_10BASET; + netdev->netdev_ops = &adin1110_netdev_ops; + netdev->ethtool_ops = &adin1110_ethtool_ops; + netdev->priv_flags |= IFF_UNICAST_FLT; + netdev->features |= NETIF_F_NETNS_LOCAL; + + port_priv->phydev = get_phy_device(priv->mii_bus, i + 1, false); + if (IS_ERR(port_priv->phydev)) { + netdev_err(netdev, "Could not find PHY with device address: %d.\n", i); + return PTR_ERR(port_priv->phydev); + } + + port_priv->phydev = phy_connect(netdev, + phydev_name(port_priv->phydev), + adin1110_adjust_link, + PHY_INTERFACE_MODE_INTERNAL); + if (IS_ERR(port_priv->phydev)) { + netdev_err(netdev, "Could not connect PHY with device address: %d.\n", i); + return PTR_ERR(port_priv->phydev); + } + + ret = devm_add_action_or_reset(dev, adin1110_disconnect_phy, + port_priv->phydev); + if (ret < 0) + return ret; + } + + /* ADIN1110 INT_N pin will be used to signal the host */ + ret = devm_request_threaded_irq(dev, priv->spidev->irq, NULL, + adin1110_irq, + IRQF_TRIGGER_LOW | IRQF_ONESHOT, + dev_name(dev), priv); + if (ret < 0) + return ret; + + ret = adin1110_setup_notifiers(priv); + if (ret < 0) + return ret; + + for (i = 0; i < priv->cfg->ports_nr; i++) { + ret = devm_register_netdev(dev, priv->ports[i]->netdev); + if (ret < 0) { + dev_err(dev, "Failed to register network device.\n"); + return ret; + } + } + + return 0; +} + +static int adin1110_probe(struct spi_device *spi) +{ + const struct spi_device_id *dev_id = spi_get_device_id(spi); + struct device *dev = &spi->dev; + struct adin1110_priv *priv; + int ret; + + priv = devm_kzalloc(dev, sizeof(struct adin1110_priv), GFP_KERNEL); + if (!priv) + return -ENOMEM; + + priv->spidev = spi; + priv->cfg = &adin1110_cfgs[dev_id->driver_data]; + spi->bits_per_word = 8; + spi->mode = SPI_MODE_0; + + mutex_init(&priv->lock); + spin_lock_init(&priv->state_lock); + + /* use of CRC on control and data transactions is pin dependent */ + priv->append_crc = device_property_read_bool(dev, "adi,spi-crc"); + if (priv->append_crc) + crc8_populate_msb(adin1110_crc_table, 0x7); + + ret = adin1110_check_spi(priv); + if (ret < 0) { + dev_err(dev, "Probe SPI Read check failed: %d\n", ret); + return ret; + } + + ret = adin1110_write_reg(priv, ADIN1110_RESET, ADIN1110_SWRESET); + if (ret < 0) + return ret; + + ret = adin1110_register_mdiobus(priv, dev); + if (ret < 0) { + dev_err(dev, "Could not register MDIO bus %d\n", ret); + return ret; + } + + return adin1110_probe_netdevs(priv); +} + +static const struct of_device_id adin1110_match_table[] = { + { .compatible = "adi,adin1110" }, + { .compatible = "adi,adin2111" }, + { } +}; +MODULE_DEVICE_TABLE(of, adin1110_match_table); + +static const struct spi_device_id adin1110_spi_id[] = { + { .name = "adin1110", .driver_data = ADIN1110_MAC }, + { .name = "adin2111", .driver_data = ADIN2111_MAC }, + { } +}; +MODULE_DEVICE_TABLE(spi, adin1110_spi_id); + +static struct spi_driver adin1110_driver = { + .driver = { + .name = "adin1110", + .of_match_table = adin1110_match_table, + }, + .probe = adin1110_probe, + .id_table = adin1110_spi_id, +}; +module_spi_driver(adin1110_driver); + +MODULE_DESCRIPTION("ADIN1110 Network driver"); +MODULE_AUTHOR("Alexandru Tachici <alexandru.tachici@analog.com>"); +MODULE_LICENSE("Dual BSD/GPL"); diff --git a/drivers/net/ethernet/aeroflex/greth.c b/drivers/net/ethernet/aeroflex/greth.c index 447dc64a17e5..e104fb02817d 100644 --- a/drivers/net/ethernet/aeroflex/greth.c +++ b/drivers/net/ethernet/aeroflex/greth.c @@ -1112,9 +1112,9 @@ static void greth_get_drvinfo(struct net_device *dev, struct ethtool_drvinfo *in { struct greth_private *greth = netdev_priv(dev); - strlcpy(info->driver, dev_driver_string(greth->dev), + strscpy(info->driver, dev_driver_string(greth->dev), sizeof(info->driver)); - strlcpy(info->bus_info, greth->dev->bus->name, sizeof(info->bus_info)); + strscpy(info->bus_info, greth->dev->bus->name, sizeof(info->bus_info)); } static void greth_get_regs(struct net_device *dev, struct ethtool_regs *regs, void *p) @@ -1507,7 +1507,7 @@ static int greth_of_probe(struct platform_device *ofdev) } /* setup NAPI */ - netif_napi_add(dev, &greth->napi, greth_poll, 64); + netif_napi_add(dev, &greth->napi, greth_poll); return 0; diff --git a/drivers/net/ethernet/agere/et131x.c b/drivers/net/ethernet/agere/et131x.c index d19d1579c415..5fab589b3ddf 100644 --- a/drivers/net/ethernet/agere/et131x.c +++ b/drivers/net/ethernet/agere/et131x.c @@ -2952,8 +2952,8 @@ static void et131x_get_drvinfo(struct net_device *netdev, { struct et131x_adapter *adapter = netdev_priv(netdev); - strlcpy(info->driver, DRIVER_NAME, sizeof(info->driver)); - strlcpy(info->bus_info, pci_name(adapter->pdev), + strscpy(info->driver, DRIVER_NAME, sizeof(info->driver)); + strscpy(info->bus_info, pci_name(adapter->pdev), sizeof(info->bus_info)); } @@ -3969,7 +3969,7 @@ static int et131x_pci_setup(struct pci_dev *pdev, et131x_init_send(adapter); - netif_napi_add(netdev, &adapter->napi, et131x_poll, 64); + netif_napi_add(netdev, &adapter->napi, et131x_poll); eth_hw_addr_set(netdev, adapter->addr); diff --git a/drivers/net/ethernet/alacritech/slicoss.c b/drivers/net/ethernet/alacritech/slicoss.c index ce353b0c02a3..a30d0f172986 100644 --- a/drivers/net/ethernet/alacritech/slicoss.c +++ b/drivers/net/ethernet/alacritech/slicoss.c @@ -1531,8 +1531,8 @@ static void slic_get_drvinfo(struct net_device *dev, { struct slic_device *sdev = netdev_priv(dev); - strlcpy(info->driver, DRV_NAME, sizeof(info->driver)); - strlcpy(info->bus_info, pci_name(sdev->pdev), sizeof(info->bus_info)); + strscpy(info->driver, DRV_NAME, sizeof(info->driver)); + strscpy(info->bus_info, pci_name(sdev->pdev), sizeof(info->bus_info)); } static const struct ethtool_ops slic_ethtool_ops = { @@ -1803,7 +1803,7 @@ static int slic_probe(struct pci_dev *pdev, const struct pci_device_id *ent) goto unmap; } - netif_napi_add(dev, &sdev->napi, slic_poll, NAPI_POLL_WEIGHT); + netif_napi_add(dev, &sdev->napi, slic_poll); netif_carrier_off(dev); err = register_netdev(dev); diff --git a/drivers/net/ethernet/allwinner/sun4i-emac.c b/drivers/net/ethernet/allwinner/sun4i-emac.c index 621ce742ad21..a94c62956eed 100644 --- a/drivers/net/ethernet/allwinner/sun4i-emac.c +++ b/drivers/net/ethernet/allwinner/sun4i-emac.c @@ -331,8 +331,8 @@ prepare_err: static void emac_get_drvinfo(struct net_device *dev, struct ethtool_drvinfo *info) { - strlcpy(info->driver, DRV_NAME, sizeof(info->driver)); - strlcpy(info->bus_info, dev_name(&dev->dev), sizeof(info->bus_info)); + strscpy(info->driver, DRV_NAME, sizeof(info->driver)); + strscpy(info->bus_info, dev_name(&dev->dev), sizeof(info->bus_info)); } static u32 emac_get_msglevel(struct net_device *dev) diff --git a/drivers/net/ethernet/alteon/acenic.c b/drivers/net/ethernet/alteon/acenic.c index 22fe98555b24..d7762da8b2c0 100644 --- a/drivers/net/ethernet/alteon/acenic.c +++ b/drivers/net/ethernet/alteon/acenic.c @@ -2691,12 +2691,12 @@ static void ace_get_drvinfo(struct net_device *dev, { struct ace_private *ap = netdev_priv(dev); - strlcpy(info->driver, "acenic", sizeof(info->driver)); + strscpy(info->driver, "acenic", sizeof(info->driver)); snprintf(info->fw_version, sizeof(info->version), "%i.%i.%i", ap->firmware_major, ap->firmware_minor, ap->firmware_fix); if (ap->pdev) - strlcpy(info->bus_info, pci_name(ap->pdev), + strscpy(info->bus_info, pci_name(ap->pdev), sizeof(info->bus_info)); } diff --git a/drivers/net/ethernet/altera/Kconfig b/drivers/net/ethernet/altera/Kconfig index 914e56b91467..dd7fd41ccde5 100644 --- a/drivers/net/ethernet/altera/Kconfig +++ b/drivers/net/ethernet/altera/Kconfig @@ -3,6 +3,8 @@ config ALTERA_TSE tristate "Altera Triple-Speed Ethernet MAC support" depends on HAS_DMA select PHYLIB + select PHYLINK + select PCS_ALTERA_TSE help This driver supports the Altera Triple-Speed (TSE) Ethernet MAC. diff --git a/drivers/net/ethernet/altera/altera_tse.h b/drivers/net/ethernet/altera/altera_tse.h index f17acfb579a0..db5eed06e92d 100644 --- a/drivers/net/ethernet/altera/altera_tse.h +++ b/drivers/net/ethernet/altera/altera_tse.h @@ -27,6 +27,7 @@ #include <linux/list.h> #include <linux/netdevice.h> #include <linux/phy.h> +#include <linux/phylink.h> #define ALTERA_TSE_SW_RESET_WATCHDOG_CNTR 10000 #define ALTERA_TSE_MAC_FIFO_WIDTH 4 /* TX/RX FIFO width in @@ -109,17 +110,6 @@ #define MAC_CMDCFG_DISABLE_READ_TIMEOUT_GET(v) GET_BIT_VALUE(v, 27) #define MAC_CMDCFG_CNT_RESET_GET(v) GET_BIT_VALUE(v, 31) -/* SGMII PCS register addresses - */ -#define SGMII_PCS_SCRATCH 0x10 -#define SGMII_PCS_REV 0x11 -#define SGMII_PCS_LINK_TIMER_0 0x12 -#define SGMII_PCS_LINK_TIMER_1 0x13 -#define SGMII_PCS_IF_MODE 0x14 -#define SGMII_PCS_DIS_READ_TO 0x15 -#define SGMII_PCS_READ_TO 0x16 -#define SGMII_PCS_SW_RESET_TIMEOUT 100 /* usecs */ - /* MDIO registers within MAC register Space */ struct altera_tse_mdio { @@ -423,6 +413,9 @@ struct altera_tse_private { void __iomem *tx_dma_csr; void __iomem *tx_dma_desc; + /* SGMII PCS address space */ + void __iomem *pcs_base; + /* Rx buffers queue */ struct tse_buffer *rx_ring; u32 rx_cons; @@ -480,6 +473,10 @@ struct altera_tse_private { u32 msg_enable; struct altera_dmaops *dmaops; + + struct phylink *phylink; + struct phylink_config phylink_config; + struct phylink_pcs *pcs; }; /* Function prototypes diff --git a/drivers/net/ethernet/altera/altera_tse_ethtool.c b/drivers/net/ethernet/altera/altera_tse_ethtool.c index 4299f1301149..81313c85833e 100644 --- a/drivers/net/ethernet/altera/altera_tse_ethtool.c +++ b/drivers/net/ethernet/altera/altera_tse_ethtool.c @@ -199,9 +199,9 @@ static int tse_reglen(struct net_device *dev) static void tse_get_regs(struct net_device *dev, struct ethtool_regs *regs, void *regbuf) { - int i; struct altera_tse_private *priv = netdev_priv(dev); u32 *buf = regbuf; + int i; /* Set version to a known value, so ethtool knows * how to do any special formatting of this data. @@ -221,6 +221,22 @@ static void tse_get_regs(struct net_device *dev, struct ethtool_regs *regs, buf[i] = csrrd32(priv->mac_dev, i * 4); } +static int tse_ethtool_set_link_ksettings(struct net_device *dev, + const struct ethtool_link_ksettings *cmd) +{ + struct altera_tse_private *priv = netdev_priv(dev); + + return phylink_ethtool_ksettings_set(priv->phylink, cmd); +} + +static int tse_ethtool_get_link_ksettings(struct net_device *dev, + struct ethtool_link_ksettings *cmd) +{ + struct altera_tse_private *priv = netdev_priv(dev); + + return phylink_ethtool_ksettings_get(priv->phylink, cmd); +} + static const struct ethtool_ops tse_ethtool_ops = { .get_drvinfo = tse_get_drvinfo, .get_regs_len = tse_reglen, @@ -231,8 +247,9 @@ static const struct ethtool_ops tse_ethtool_ops = { .get_ethtool_stats = tse_fill_stats, .get_msglevel = tse_get_msglevel, .set_msglevel = tse_set_msglevel, - .get_link_ksettings = phy_ethtool_get_link_ksettings, - .set_link_ksettings = phy_ethtool_set_link_ksettings, + .get_link_ksettings = tse_ethtool_get_link_ksettings, + .set_link_ksettings = tse_ethtool_set_link_ksettings, + .get_ts_info = ethtool_op_get_ts_info, }; void altera_tse_set_ethtool_ops(struct net_device *netdev) diff --git a/drivers/net/ethernet/altera/altera_tse_main.c b/drivers/net/ethernet/altera/altera_tse_main.c index 8c5828582c21..7633b227b2ca 100644 --- a/drivers/net/ethernet/altera/altera_tse_main.c +++ b/drivers/net/ethernet/altera/altera_tse_main.c @@ -32,6 +32,7 @@ #include <linux/of_mdio.h> #include <linux/of_net.h> #include <linux/of_platform.h> +#include <linux/pcs-altera-tse.h> #include <linux/phy.h> #include <linux/platform_device.h> #include <linux/skbuff.h> @@ -86,27 +87,6 @@ static inline u32 tse_tx_avail(struct altera_tse_private *priv) return priv->tx_cons + priv->tx_ring_size - priv->tx_prod - 1; } -/* PCS Register read/write functions - */ -static u16 sgmii_pcs_read(struct altera_tse_private *priv, int regnum) -{ - return csrrd32(priv->mac_dev, - tse_csroffs(mdio_phy0) + regnum * 4) & 0xffff; -} - -static void sgmii_pcs_write(struct altera_tse_private *priv, int regnum, - u16 value) -{ - csrwr32(value, priv->mac_dev, tse_csroffs(mdio_phy0) + regnum * 4); -} - -/* Check PCS scratch memory */ -static int sgmii_pcs_scratch_test(struct altera_tse_private *priv, u16 value) -{ - sgmii_pcs_write(priv, SGMII_PCS_SCRATCH, value); - return (sgmii_pcs_read(priv, SGMII_PCS_SCRATCH) == value); -} - /* MDIO specific functions */ static int altera_tse_mdio_read(struct mii_bus *bus, int mii_id, int regnum) @@ -141,10 +121,10 @@ static int altera_tse_mdio_write(struct mii_bus *bus, int mii_id, int regnum, static int altera_tse_mdio_create(struct net_device *dev, unsigned int id) { struct altera_tse_private *priv = netdev_priv(dev); - int ret; struct device_node *mdio_node = NULL; - struct mii_bus *mdio = NULL; struct device_node *child_node = NULL; + struct mii_bus *mdio = NULL; + int ret; for_each_child_of_node(priv->device->of_node, child_node) { if (of_device_is_compatible(child_node, "altr,tse-mdio")) { @@ -236,8 +216,8 @@ static int tse_init_rx_buffer(struct altera_tse_private *priv, static void tse_free_rx_buffer(struct altera_tse_private *priv, struct tse_buffer *rxbuffer) { - struct sk_buff *skb = rxbuffer->skb; dma_addr_t dma_addr = rxbuffer->dma_addr; + struct sk_buff *skb = rxbuffer->skb; if (skb != NULL) { if (dma_addr) @@ -358,6 +338,7 @@ static inline void tse_rx_vlan(struct net_device *dev, struct sk_buff *skb) { struct ethhdr *eth_hdr; u16 vid; + if ((dev->features & NETIF_F_HW_VLAN_CTAG_RX) && !__vlan_get_tag(skb, &vid)) { eth_hdr = (struct ethhdr *)skb->data; @@ -371,10 +352,10 @@ static inline void tse_rx_vlan(struct net_device *dev, struct sk_buff *skb) */ static int tse_rx(struct altera_tse_private *priv, int limit) { - unsigned int count = 0; + unsigned int entry = priv->rx_cons % priv->rx_ring_size; unsigned int next_entry; + unsigned int count = 0; struct sk_buff *skb; - unsigned int entry = priv->rx_cons % priv->rx_ring_size; u32 rxstatus; u16 pktlength; u16 pktstatus; @@ -448,10 +429,10 @@ static int tse_rx(struct altera_tse_private *priv, int limit) static int tse_tx_complete(struct altera_tse_private *priv) { unsigned int txsize = priv->tx_ring_size; - u32 ready; - unsigned int entry; struct tse_buffer *tx_buff; + unsigned int entry; int txcomplete = 0; + u32 ready; spin_lock(&priv->tx_lock); @@ -497,8 +478,8 @@ static int tse_poll(struct napi_struct *napi, int budget) { struct altera_tse_private *priv = container_of(napi, struct altera_tse_private, napi); - int rxcomplete = 0; unsigned long int flags; + int rxcomplete = 0; tse_tx_complete(priv); @@ -561,13 +542,13 @@ static irqreturn_t altera_isr(int irq, void *dev_id) static netdev_tx_t tse_start_xmit(struct sk_buff *skb, struct net_device *dev) { struct altera_tse_private *priv = netdev_priv(dev); + unsigned int nopaged_len = skb_headlen(skb); unsigned int txsize = priv->tx_ring_size; - unsigned int entry; - struct tse_buffer *buffer = NULL; int nfrags = skb_shinfo(skb)->nr_frags; - unsigned int nopaged_len = skb_headlen(skb); + struct tse_buffer *buffer = NULL; netdev_tx_t ret = NETDEV_TX_OK; dma_addr_t dma_addr; + unsigned int entry; spin_lock_bh(&priv->tx_lock); @@ -619,117 +600,6 @@ out: return ret; } -/* Called every time the controller might need to be made - * aware of new link state. The PHY code conveys this - * information through variables in the phydev structure, and this - * function converts those variables into the appropriate - * register values, and can bring down the device if needed. - */ -static void altera_tse_adjust_link(struct net_device *dev) -{ - struct altera_tse_private *priv = netdev_priv(dev); - struct phy_device *phydev = dev->phydev; - int new_state = 0; - - /* only change config if there is a link */ - spin_lock(&priv->mac_cfg_lock); - if (phydev->link) { - /* Read old config */ - u32 cfg_reg = ioread32(&priv->mac_dev->command_config); - - /* Check duplex */ - if (phydev->duplex != priv->oldduplex) { - new_state = 1; - if (!(phydev->duplex)) - cfg_reg |= MAC_CMDCFG_HD_ENA; - else - cfg_reg &= ~MAC_CMDCFG_HD_ENA; - - netdev_dbg(priv->dev, "%s: Link duplex = 0x%x\n", - dev->name, phydev->duplex); - - priv->oldduplex = phydev->duplex; - } - - /* Check speed */ - if (phydev->speed != priv->oldspeed) { - new_state = 1; - switch (phydev->speed) { - case 1000: - cfg_reg |= MAC_CMDCFG_ETH_SPEED; - cfg_reg &= ~MAC_CMDCFG_ENA_10; - break; - case 100: - cfg_reg &= ~MAC_CMDCFG_ETH_SPEED; - cfg_reg &= ~MAC_CMDCFG_ENA_10; - break; - case 10: - cfg_reg &= ~MAC_CMDCFG_ETH_SPEED; - cfg_reg |= MAC_CMDCFG_ENA_10; - break; - default: - if (netif_msg_link(priv)) - netdev_warn(dev, "Speed (%d) is not 10/100/1000!\n", - phydev->speed); - break; - } - priv->oldspeed = phydev->speed; - } - iowrite32(cfg_reg, &priv->mac_dev->command_config); - - if (!priv->oldlink) { - new_state = 1; - priv->oldlink = 1; - } - } else if (priv->oldlink) { - new_state = 1; - priv->oldlink = 0; - priv->oldspeed = 0; - priv->oldduplex = -1; - } - - if (new_state && netif_msg_link(priv)) - phy_print_status(phydev); - - spin_unlock(&priv->mac_cfg_lock); -} -static struct phy_device *connect_local_phy(struct net_device *dev) -{ - struct altera_tse_private *priv = netdev_priv(dev); - struct phy_device *phydev = NULL; - char phy_id_fmt[MII_BUS_ID_SIZE + 3]; - - if (priv->phy_addr != POLL_PHY) { - snprintf(phy_id_fmt, MII_BUS_ID_SIZE + 3, PHY_ID_FMT, - priv->mdio->id, priv->phy_addr); - - netdev_dbg(dev, "trying to attach to %s\n", phy_id_fmt); - - phydev = phy_connect(dev, phy_id_fmt, &altera_tse_adjust_link, - priv->phy_iface); - if (IS_ERR(phydev)) { - netdev_err(dev, "Could not attach to PHY\n"); - phydev = NULL; - } - - } else { - int ret; - phydev = phy_find_first(priv->mdio); - if (phydev == NULL) { - netdev_err(dev, "No PHY found\n"); - return phydev; - } - - ret = phy_connect_direct(dev, phydev, &altera_tse_adjust_link, - priv->phy_iface); - if (ret != 0) { - netdev_err(dev, "Could not attach to PHY\n"); - phydev = NULL; - } - } - return phydev; -} - static int altera_tse_phy_get_addr_mdio_create(struct net_device *dev) { struct altera_tse_private *priv = netdev_priv(dev); @@ -768,91 +638,6 @@ static int altera_tse_phy_get_addr_mdio_create(struct net_device *dev) return 0; } -/* Initialize driver's PHY state, and attach to the PHY - */ -static int init_phy(struct net_device *dev) -{ - struct altera_tse_private *priv = netdev_priv(dev); - struct phy_device *phydev; - struct device_node *phynode; - bool fixed_link = false; - int rc = 0; - - /* Avoid init phy in case of no phy present */ - if (!priv->phy_iface) - return 0; - - priv->oldlink = 0; - priv->oldspeed = 0; - priv->oldduplex = -1; - - phynode = of_parse_phandle(priv->device->of_node, "phy-handle", 0); - - if (!phynode) { - /* check if a fixed-link is defined in device-tree */ - if (of_phy_is_fixed_link(priv->device->of_node)) { - rc = of_phy_register_fixed_link(priv->device->of_node); - if (rc < 0) { - netdev_err(dev, "cannot register fixed PHY\n"); - return rc; - } - - /* In the case of a fixed PHY, the DT node associated - * to the PHY is the Ethernet MAC DT node. - */ - phynode = of_node_get(priv->device->of_node); - fixed_link = true; - - netdev_dbg(dev, "fixed-link detected\n"); - phydev = of_phy_connect(dev, phynode, - &altera_tse_adjust_link, - 0, priv->phy_iface); - } else { - netdev_dbg(dev, "no phy-handle found\n"); - if (!priv->mdio) { - netdev_err(dev, "No phy-handle nor local mdio specified\n"); - return -ENODEV; - } - phydev = connect_local_phy(dev); - } - } else { - netdev_dbg(dev, "phy-handle found\n"); - phydev = of_phy_connect(dev, phynode, - &altera_tse_adjust_link, 0, priv->phy_iface); - } - of_node_put(phynode); - - if (!phydev) { - netdev_err(dev, "Could not find the PHY\n"); - if (fixed_link) - of_phy_deregister_fixed_link(priv->device->of_node); - return -ENODEV; - } - - /* Stop Advertising 1000BASE Capability if interface is not GMII - */ - if ((priv->phy_iface == PHY_INTERFACE_MODE_MII) || - (priv->phy_iface == PHY_INTERFACE_MODE_RMII)) - phy_set_max_speed(phydev, SPEED_100); - - /* Broken HW is sometimes missing the pull-up resistor on the - * MDIO line, which results in reads to non-existent devices returning - * 0 rather than 0xffff. Catch this here and treat 0 as a non-existent - * device as well. If a fixed-link is used the phy_id is always 0. - * Note: phydev->phy_id is the result of reading the UID PHY registers. - */ - if ((phydev->phy_id == 0) && !fixed_link) { - netdev_err(dev, "Bad PHY UID 0x%08x\n", phydev->phy_id); - phy_disconnect(phydev); - return -ENODEV; - } - - netdev_dbg(dev, "attached to PHY %d UID 0x%08x Link = %d\n", - phydev->mdio.addr, phydev->phy_id, phydev->link); - - return 0; -} - static void tse_update_mac_addr(struct altera_tse_private *priv, const u8 *addr) { u32 msb; @@ -1012,8 +797,8 @@ static int tse_change_mtu(struct net_device *dev, int new_mtu) static void altera_tse_set_mcfilter(struct net_device *dev) { struct altera_tse_private *priv = netdev_priv(dev); - int i; struct netdev_hw_addr *ha; + int i; /* clear the hash filter */ for (i = 0; i < 64; i++) @@ -1087,74 +872,14 @@ static void tse_set_rx_mode(struct net_device *dev) spin_unlock(&priv->mac_cfg_lock); } -/* Initialise (if necessary) the SGMII PCS component - */ -static int init_sgmii_pcs(struct net_device *dev) -{ - struct altera_tse_private *priv = netdev_priv(dev); - int n; - unsigned int tmp_reg = 0; - - if (priv->phy_iface != PHY_INTERFACE_MODE_SGMII) - return 0; /* Nothing to do, not in SGMII mode */ - - /* The TSE SGMII PCS block looks a little like a PHY, it is - * mapped into the zeroth MDIO space of the MAC and it has - * ID registers like a PHY would. Sadly this is often - * configured to zeroes, so don't be surprised if it does - * show 0x00000000. - */ - - if (sgmii_pcs_scratch_test(priv, 0x0000) && - sgmii_pcs_scratch_test(priv, 0xffff) && - sgmii_pcs_scratch_test(priv, 0xa5a5) && - sgmii_pcs_scratch_test(priv, 0x5a5a)) { - netdev_info(dev, "PCS PHY ID: 0x%04x%04x\n", - sgmii_pcs_read(priv, MII_PHYSID1), - sgmii_pcs_read(priv, MII_PHYSID2)); - } else { - netdev_err(dev, "SGMII PCS Scratch memory test failed.\n"); - return -ENOMEM; - } - - /* Starting on page 5-29 of the MegaCore Function User Guide - * Set SGMII Link timer to 1.6ms - */ - sgmii_pcs_write(priv, SGMII_PCS_LINK_TIMER_0, 0x0D40); - sgmii_pcs_write(priv, SGMII_PCS_LINK_TIMER_1, 0x03); - - /* Enable SGMII Interface and Enable SGMII Auto Negotiation */ - sgmii_pcs_write(priv, SGMII_PCS_IF_MODE, 0x3); - - /* Enable Autonegotiation */ - tmp_reg = sgmii_pcs_read(priv, MII_BMCR); - tmp_reg |= (BMCR_SPEED1000 | BMCR_FULLDPLX | BMCR_ANENABLE); - sgmii_pcs_write(priv, MII_BMCR, tmp_reg); - - /* Reset PCS block */ - tmp_reg |= BMCR_RESET; - sgmii_pcs_write(priv, MII_BMCR, tmp_reg); - for (n = 0; n < SGMII_PCS_SW_RESET_TIMEOUT; n++) { - if (!(sgmii_pcs_read(priv, MII_BMCR) & BMCR_RESET)) { - netdev_info(dev, "SGMII PCS block initialised OK\n"); - return 0; - } - udelay(1); - } - - /* We failed to reset the block, return a timeout */ - netdev_err(dev, "SGMII PCS block reset failed.\n"); - return -ETIMEDOUT; -} - /* Open and initialize the interface */ static int tse_open(struct net_device *dev) { struct altera_tse_private *priv = netdev_priv(dev); + unsigned long flags; int ret = 0; int i; - unsigned long int flags; /* Reset and configure TSE MAC and probe associated PHY */ ret = priv->dmaops->init_dma(priv); @@ -1171,14 +896,6 @@ static int tse_open(struct net_device *dev) netdev_warn(dev, "TSE revision %x\n", priv->revision); spin_lock(&priv->mac_cfg_lock); - /* no-op if MAC not operating in SGMII mode*/ - ret = init_sgmii_pcs(dev); - if (ret) { - netdev_err(dev, - "Cannot init the SGMII PCS (error: %d)\n", ret); - spin_unlock(&priv->mac_cfg_lock); - goto phy_error; - } ret = reset_mac(priv); /* Note that reset_mac will fail if the clocks are gated by the PHY @@ -1236,8 +953,12 @@ static int tse_open(struct net_device *dev) spin_unlock_irqrestore(&priv->rxdma_irq_lock, flags); - if (dev->phydev) - phy_start(dev->phydev); + ret = phylink_of_phy_connect(priv->phylink, priv->device->of_node, 0); + if (ret) { + netdev_err(dev, "could not connect phylink (%d)\n", ret); + goto tx_request_irq_error; + } + phylink_start(priv->phylink); napi_enable(&priv->napi); netif_start_queue(dev); @@ -1265,13 +986,10 @@ phy_error: static int tse_shutdown(struct net_device *dev) { struct altera_tse_private *priv = netdev_priv(dev); - int ret; unsigned long int flags; + int ret; - /* Stop the PHY */ - if (dev->phydev) - phy_stop(dev->phydev); - + phylink_stop(priv->phylink); netif_stop_queue(dev); napi_disable(&priv->napi); @@ -1317,11 +1035,79 @@ static struct net_device_ops altera_tse_netdev_ops = { .ndo_validate_addr = eth_validate_addr, }; +static void alt_tse_mac_an_restart(struct phylink_config *config) +{ +} + +static void alt_tse_mac_config(struct phylink_config *config, unsigned int mode, + const struct phylink_link_state *state) +{ + struct net_device *ndev = to_net_dev(config->dev); + struct altera_tse_private *priv = netdev_priv(ndev); + + spin_lock(&priv->mac_cfg_lock); + reset_mac(priv); + tse_set_mac(priv, true); + spin_unlock(&priv->mac_cfg_lock); +} + +static void alt_tse_mac_link_down(struct phylink_config *config, + unsigned int mode, phy_interface_t interface) +{ +} + +static void alt_tse_mac_link_up(struct phylink_config *config, + struct phy_device *phy, unsigned int mode, + phy_interface_t interface, int speed, + int duplex, bool tx_pause, bool rx_pause) +{ + struct net_device *ndev = to_net_dev(config->dev); + struct altera_tse_private *priv = netdev_priv(ndev); + u32 ctrl; + + ctrl = csrrd32(priv->mac_dev, tse_csroffs(command_config)); + ctrl &= ~(MAC_CMDCFG_ENA_10 | MAC_CMDCFG_ETH_SPEED | MAC_CMDCFG_HD_ENA); + + if (duplex == DUPLEX_HALF) + ctrl |= MAC_CMDCFG_HD_ENA; + + if (speed == SPEED_1000) + ctrl |= MAC_CMDCFG_ETH_SPEED; + else if (speed == SPEED_10) + ctrl |= MAC_CMDCFG_ENA_10; + + spin_lock(&priv->mac_cfg_lock); + csrwr32(ctrl, priv->mac_dev, tse_csroffs(command_config)); + spin_unlock(&priv->mac_cfg_lock); +} + +static struct phylink_pcs *alt_tse_select_pcs(struct phylink_config *config, + phy_interface_t interface) +{ + struct net_device *ndev = to_net_dev(config->dev); + struct altera_tse_private *priv = netdev_priv(ndev); + + if (interface == PHY_INTERFACE_MODE_SGMII || + interface == PHY_INTERFACE_MODE_1000BASEX) + return priv->pcs; + else + return NULL; +} + +static const struct phylink_mac_ops alt_tse_phylink_ops = { + .validate = phylink_generic_validate, + .mac_an_restart = alt_tse_mac_an_restart, + .mac_config = alt_tse_mac_config, + .mac_link_down = alt_tse_mac_link_down, + .mac_link_up = alt_tse_mac_link_up, + .mac_select_pcs = alt_tse_select_pcs, +}; + static int request_and_map(struct platform_device *pdev, const char *name, struct resource **res, void __iomem **ptr) { - struct resource *region; struct device *device = &pdev->dev; + struct resource *region; *res = platform_get_resource_byname(pdev, IORESOURCE_MEM, name); if (*res == NULL) { @@ -1350,13 +1136,15 @@ static int request_and_map(struct platform_device *pdev, const char *name, */ static int altera_tse_probe(struct platform_device *pdev) { - struct net_device *ndev; - int ret = -ENODEV; + const struct of_device_id *of_id = NULL; + struct altera_tse_private *priv; struct resource *control_port; struct resource *dma_res; - struct altera_tse_private *priv; + struct resource *pcs_res; + struct net_device *ndev; void __iomem *descmap; - const struct of_device_id *of_id = NULL; + int pcs_reg_width = 2; + int ret = -ENODEV; ndev = alloc_etherdev(sizeof(struct altera_tse_private)); if (!ndev) { @@ -1467,6 +1255,17 @@ static int altera_tse_probe(struct platform_device *pdev) if (ret) goto err_free_netdev; + /* SGMII PCS address space. The location can vary depending on how the + * IP is integrated. We can have a resource dedicated to it at a specific + * address space, but if it's not the case, we fallback to the mdiophy0 + * from the MAC's address space + */ + ret = request_and_map(pdev, "pcs", &pcs_res, + &priv->pcs_base); + if (ret) { + priv->pcs_base = priv->mac_dev + tse_csroffs(mdio_phy0); + pcs_reg_width = 4; + } /* Rx IRQ */ priv->rx_irq = platform_get_irq_byname(pdev, "rx_irq"); @@ -1566,7 +1365,7 @@ static int altera_tse_probe(struct platform_device *pdev) ndev->features |= NETIF_F_HW_VLAN_CTAG_RX; /* setup NAPI interface */ - netif_napi_add(ndev, &priv->napi, tse_poll, NAPI_POLL_WEIGHT); + netif_napi_add(ndev, &priv->napi, tse_poll); spin_lock_init(&priv->mac_cfg_lock); spin_lock_init(&priv->tx_lock); @@ -1590,11 +1389,32 @@ static int altera_tse_probe(struct platform_device *pdev) (unsigned long) control_port->start, priv->rx_irq, priv->tx_irq); - ret = init_phy(ndev); - if (ret != 0) { - netdev_err(ndev, "Cannot attach to PHY (error: %d)\n", ret); + priv->pcs = alt_tse_pcs_create(ndev, priv->pcs_base, pcs_reg_width); + + priv->phylink_config.dev = &ndev->dev; + priv->phylink_config.type = PHYLINK_NETDEV; + priv->phylink_config.mac_capabilities = MAC_SYM_PAUSE | MAC_10 | + MAC_100 | MAC_1000FD; + + phy_interface_set_rgmii(priv->phylink_config.supported_interfaces); + __set_bit(PHY_INTERFACE_MODE_MII, + priv->phylink_config.supported_interfaces); + __set_bit(PHY_INTERFACE_MODE_GMII, + priv->phylink_config.supported_interfaces); + __set_bit(PHY_INTERFACE_MODE_SGMII, + priv->phylink_config.supported_interfaces); + __set_bit(PHY_INTERFACE_MODE_1000BASEX, + priv->phylink_config.supported_interfaces); + + priv->phylink = phylink_create(&priv->phylink_config, + of_fwnode_handle(priv->device->of_node), + priv->phy_iface, &alt_tse_phylink_ops); + if (IS_ERR(priv->phylink)) { + dev_err(&pdev->dev, "failed to create phylink\n"); + ret = PTR_ERR(priv->phylink); goto err_init_phy; } + return 0; err_init_phy: @@ -1614,16 +1434,10 @@ static int altera_tse_remove(struct platform_device *pdev) struct net_device *ndev = platform_get_drvdata(pdev); struct altera_tse_private *priv = netdev_priv(ndev); - if (ndev->phydev) { - phy_disconnect(ndev->phydev); - - if (of_phy_is_fixed_link(priv->device->of_node)) - of_phy_deregister_fixed_link(priv->device->of_node); - } - platform_set_drvdata(pdev, NULL); altera_tse_mdio_destroy(ndev); unregister_netdev(ndev); + phylink_destroy(priv->phylink); free_netdev(ndev); return 0; diff --git a/drivers/net/ethernet/amazon/ena/ena_ethtool.c b/drivers/net/ethernet/amazon/ena/ena_ethtool.c index 39242c5a1729..98d6386b7f39 100644 --- a/drivers/net/ethernet/amazon/ena/ena_ethtool.c +++ b/drivers/net/ethernet/amazon/ena/ena_ethtool.c @@ -462,8 +462,8 @@ static void ena_get_drvinfo(struct net_device *dev, { struct ena_adapter *adapter = netdev_priv(dev); - strlcpy(info->driver, DRV_MODULE_NAME, sizeof(info->driver)); - strlcpy(info->bus_info, pci_name(adapter->pdev), + strscpy(info->driver, DRV_MODULE_NAME, sizeof(info->driver)); + strscpy(info->bus_info, pci_name(adapter->pdev), sizeof(info->bus_info)); } diff --git a/drivers/net/ethernet/amazon/ena/ena_netdev.c b/drivers/net/ethernet/amazon/ena/ena_netdev.c index 6a356a6cee15..d350eeec8bad 100644 --- a/drivers/net/ethernet/amazon/ena/ena_netdev.c +++ b/drivers/net/ethernet/amazon/ena/ena_netdev.c @@ -2265,10 +2265,8 @@ static void ena_init_napi_in_range(struct ena_adapter *adapter, for (i = first_index; i < first_index + count; i++) { struct ena_napi *napi = &adapter->ena_napi[i]; - netif_napi_add(adapter->netdev, - &napi->napi, - ENA_IS_XDP_INDEX(adapter, i) ? ena_xdp_io_poll : ena_io_poll, - NAPI_POLL_WEIGHT); + netif_napi_add(adapter->netdev, &napi->napi, + ENA_IS_XDP_INDEX(adapter, i) ? ena_xdp_io_poll : ena_io_poll); if (!ENA_IS_XDP_INDEX(adapter, i)) { napi->rx_ring = &adapter->rx_ring[i]; @@ -3166,7 +3164,7 @@ static void ena_config_host_info(struct ena_com_dev *ena_dev, struct pci_dev *pd host_info->bdf = (pdev->bus->number << 8) | pdev->devfn; host_info->os_type = ENA_ADMIN_OS_LINUX; host_info->kernel_ver = LINUX_VERSION_CODE; - strlcpy(host_info->kernel_ver_str, utsname()->version, + strscpy(host_info->kernel_ver_str, utsname()->version, sizeof(host_info->kernel_ver_str) - 1); host_info->os_dist = 0; strncpy(host_info->os_dist_str, utsname()->release, diff --git a/drivers/net/ethernet/amd/a2065.c b/drivers/net/ethernet/amd/a2065.c index 3a351d3396bf..68983b717145 100644 --- a/drivers/net/ethernet/amd/a2065.c +++ b/drivers/net/ethernet/amd/a2065.c @@ -695,7 +695,7 @@ static int a2065_init_one(struct zorro_dev *z, } dev = alloc_etherdev(sizeof(struct lance_private)); - if (dev == NULL) { + if (!dev) { release_mem_region(base_addr, sizeof(struct lance_regs)); release_mem_region(mem_start, A2065_RAM_SIZE); return -ENOMEM; diff --git a/drivers/net/ethernet/amd/amd8111e.c b/drivers/net/ethernet/amd/amd8111e.c index 5d1baa01360f..ea6cfc2095e1 100644 --- a/drivers/net/ethernet/amd/amd8111e.c +++ b/drivers/net/ethernet/amd/amd8111e.c @@ -43,7 +43,7 @@ Revision History: 3.0.4 12/09/2003 1. Added set_mac_address routine for bonding driver support. 2. Tested the driver for bonding support - 3. Bug fix: Fixed mismach in actual receive buffer lenth and lenth + 3. Bug fix: Fixed mismach in actual receive buffer length and length indicated to the h/w. 4. Modified amd8111e_rx() routine to receive all the received packets in the first interrupt. @@ -185,24 +185,23 @@ static void amd8111e_set_ext_phy(struct net_device *dev) advert = amd8111e_mdio_read(dev, lp->ext_phy_addr, MII_ADVERTISE); tmp = advert & ~(ADVERTISE_ALL | ADVERTISE_100BASE4); switch (lp->ext_phy_option) { - - default: - case SPEED_AUTONEG: /* advertise all values */ - tmp |= (ADVERTISE_10HALF | ADVERTISE_10FULL | - ADVERTISE_100HALF | ADVERTISE_100FULL); - break; - case SPEED10_HALF: - tmp |= ADVERTISE_10HALF; - break; - case SPEED10_FULL: - tmp |= ADVERTISE_10FULL; - break; - case SPEED100_HALF: - tmp |= ADVERTISE_100HALF; - break; - case SPEED100_FULL: - tmp |= ADVERTISE_100FULL; - break; + default: + case SPEED_AUTONEG: /* advertise all values */ + tmp |= (ADVERTISE_10HALF | ADVERTISE_10FULL | + ADVERTISE_100HALF | ADVERTISE_100FULL); + break; + case SPEED10_HALF: + tmp |= ADVERTISE_10HALF; + break; + case SPEED10_FULL: + tmp |= ADVERTISE_10FULL; + break; + case SPEED100_HALF: + tmp |= ADVERTISE_100HALF; + break; + case SPEED100_FULL: + tmp |= ADVERTISE_100FULL; + break; } if(advert != tmp) @@ -237,7 +236,7 @@ static int amd8111e_free_skbs(struct net_device *dev) /* Freeing previously allocated receive buffers */ for (i = 0; i < NUM_RX_BUFFERS; i++) { rx_skbuff = lp->rx_skbuff[i]; - if (rx_skbuff != NULL) { + if (rx_skbuff) { dma_unmap_single(&lp->pci_dev->dev, lp->rx_dma_addr[i], lp->rx_buff_len - 2, DMA_FROM_DEVICE); @@ -1084,7 +1083,7 @@ static irqreturn_t amd8111e_interrupt(int irq, void *dev_id) unsigned int intr0, intren0; unsigned int handled = 1; - if (unlikely(dev == NULL)) + if (unlikely(!dev)) return IRQ_NONE; spin_lock(&lp->lock); @@ -1109,7 +1108,7 @@ static irqreturn_t amd8111e_interrupt(int irq, void *dev_id) /* Check if Receive Interrupt has occurred. */ if (intr0 & RINT0) { if (napi_schedule_prep(&lp->napi)) { - /* Disable receive interupts */ + /* Disable receive interrupts */ writel(RINTEN0, mmio + INTEN0); /* Schedule a polling routine */ __napi_schedule(&lp->napi); @@ -1364,10 +1363,10 @@ static void amd8111e_get_drvinfo(struct net_device *dev, { struct amd8111e_priv *lp = netdev_priv(dev); struct pci_dev *pci_dev = lp->pci_dev; - strlcpy(info->driver, MODULE_NAME, sizeof(info->driver)); + strscpy(info->driver, MODULE_NAME, sizeof(info->driver)); snprintf(info->fw_version, sizeof(info->fw_version), "%u", chip_version); - strlcpy(info->bus_info, pci_name(pci_dev), sizeof(info->bus_info)); + strscpy(info->bus_info, pci_name(pci_dev), sizeof(info->bus_info)); } static int amd8111e_get_regs_len(struct net_device *dev) @@ -1554,7 +1553,7 @@ static int amd8111e_enable_magicpkt(struct amd8111e_priv *lp) static int amd8111e_enable_link_change(struct amd8111e_priv *lp) { - /* Adapter is already stoped/suspended/interrupt-disabled */ + /* Adapter is already stopped/suspended/interrupt-disabled */ writel(VAL0 | LCMODE_SW, lp->mmio + CMD7); /* To eliminate PCI posting bug */ diff --git a/drivers/net/ethernet/amd/amd8111e.h b/drivers/net/ethernet/amd/amd8111e.h index 37da79da5f5e..9d570adb295b 100644 --- a/drivers/net/ethernet/amd/amd8111e.h +++ b/drivers/net/ethernet/amd/amd8111e.h @@ -600,7 +600,7 @@ typedef enum { #define CSTATE 1 #define SSTATE 2 -/* Assume contoller gets data 10 times the maximum processing time */ +/* Assume controller gets data 10 times the maximum processing time */ #define REPEAT_CNT 10 /* amd8111e descriptor flag definitions */ diff --git a/drivers/net/ethernet/amd/ariadne.c b/drivers/net/ethernet/amd/ariadne.c index 4ea7b9f3c424..38153e633231 100644 --- a/drivers/net/ethernet/amd/ariadne.c +++ b/drivers/net/ethernet/amd/ariadne.c @@ -193,7 +193,7 @@ static int ariadne_rx(struct net_device *dev) struct sk_buff *skb; skb = netdev_alloc_skb(dev, pkt_len + 2); - if (skb == NULL) { + if (!skb) { for (i = 0; i < RX_RING_SIZE; i++) if (lowb(priv->rx_ring[(entry + i) % RX_RING_SIZE]->RMD1) & RF_OWN) break; @@ -731,7 +731,7 @@ static int ariadne_init_one(struct zorro_dev *z, } dev = alloc_etherdev(sizeof(struct ariadne_private)); - if (dev == NULL) { + if (!dev) { release_mem_region(base_addr, sizeof(struct Am79C960)); release_mem_region(mem_start, ARIADNE_RAM_SIZE); return -ENOMEM; diff --git a/drivers/net/ethernet/amd/atarilance.c b/drivers/net/ethernet/amd/atarilance.c index 27869164c6e6..3222c48ce6ae 100644 --- a/drivers/net/ethernet/amd/atarilance.c +++ b/drivers/net/ethernet/amd/atarilance.c @@ -581,15 +581,15 @@ static unsigned long __init lance_probe1( struct net_device *dev, /* Get the ethernet address */ switch( lp->cardtype ) { - case OLD_RIEBL: + case OLD_RIEBL: /* No ethernet address! (Set some default address) */ eth_hw_addr_set(dev, OldRieblDefHwaddr); break; - case NEW_RIEBL: + case NEW_RIEBL: lp->memcpy_f(addr, RIEBL_HWADDR_ADDR, ETH_ALEN); eth_hw_addr_set(dev, addr); break; - case PAM_CARD: + case PAM_CARD: i = IO->eeprom; for( i = 0; i < 6; ++i ) addr[i] = @@ -854,7 +854,7 @@ static irqreturn_t lance_interrupt( int irq, void *dev_id ) int csr0, boguscnt = 10; int handled = 0; - if (dev == NULL) { + if (!dev) { DPRINTK( 1, ( "lance_interrupt(): interrupt for unknown device.\n" )); return IRQ_NONE; } @@ -995,7 +995,7 @@ static int lance_rx( struct net_device *dev ) } else { skb = netdev_alloc_skb(dev, pkt_len + 2); - if (skb == NULL) { + if (!skb) { for( i = 0; i < RX_RING_SIZE; i++ ) if (MEM->rx_head[(entry+i) & RX_RING_MOD_MASK].flag & RMD1_OWN_CHIP) diff --git a/drivers/net/ethernet/amd/au1000_eth.c b/drivers/net/ethernet/amd/au1000_eth.c index d5f2c6989221..c5cec4e79489 100644 --- a/drivers/net/ethernet/amd/au1000_eth.c +++ b/drivers/net/ethernet/amd/au1000_eth.c @@ -650,7 +650,7 @@ au1000_get_drvinfo(struct net_device *dev, struct ethtool_drvinfo *info) { struct au1000_private *aup = netdev_priv(dev); - strlcpy(info->driver, DRV_NAME, sizeof(info->driver)); + strscpy(info->driver, DRV_NAME, sizeof(info->driver)); snprintf(info->bus_info, sizeof(info->bus_info), "%s %d", DRV_NAME, aup->mac_id); } @@ -786,7 +786,7 @@ static int au1000_rx(struct net_device *dev) frmlen = (status & RX_FRAME_LEN_MASK); frmlen -= 4; /* Remove FCS */ skb = netdev_alloc_skb(dev, frmlen + 2); - if (skb == NULL) { + if (!skb) { dev->stats.rx_dropped++; continue; } @@ -1199,7 +1199,7 @@ static int au1000_probe(struct platform_device *pdev) } aup->mii_bus = mdiobus_alloc(); - if (aup->mii_bus == NULL) { + if (!aup->mii_bus) { dev_err(&pdev->dev, "failed to allocate mdiobus structure\n"); err = -ENOMEM; goto err_mdiobus_alloc; @@ -1284,7 +1284,7 @@ static int au1000_probe(struct platform_device *pdev) return 0; err_out: - if (aup->mii_bus != NULL) + if (aup->mii_bus) mdiobus_unregister(aup->mii_bus); /* here we should have a valid dev plus aup-> register addresses diff --git a/drivers/net/ethernet/amd/lance.c b/drivers/net/ethernet/amd/lance.c index 462016666752..fb8686214a32 100644 --- a/drivers/net/ethernet/amd/lance.c +++ b/drivers/net/ethernet/amd/lance.c @@ -880,7 +880,7 @@ lance_init_ring(struct net_device *dev, gfp_t gfp) rx_buff = skb->data; else rx_buff = kmalloc(PKT_BUF_SZ, GFP_DMA | gfp); - if (rx_buff == NULL) + if (!rx_buff) lp->rx_ring[i].base = 0; else lp->rx_ring[i].base = (u32)isa_virt_to_bus(rx_buff) | 0x80000000; @@ -1186,7 +1186,7 @@ lance_rx(struct net_device *dev) else { skb = dev_alloc_skb(pkt_len+2); - if (skb == NULL) + if (!skb) { printk("%s: Memory squeeze, deferring packet.\n", dev->name); for (i=0; i < RX_RING_SIZE; i++) diff --git a/drivers/net/ethernet/amd/nmclan_cs.c b/drivers/net/ethernet/amd/nmclan_cs.c index 30ee5329bd7c..823a329a921f 100644 --- a/drivers/net/ethernet/amd/nmclan_cs.c +++ b/drivers/net/ethernet/amd/nmclan_cs.c @@ -485,10 +485,10 @@ static int mace_read(mace_private *lp, unsigned int ioaddr, int reg) unsigned long flags; switch (reg >> 4) { - case 0: /* register 0-15 */ + case 0: /* register 0-15 */ data = inb(ioaddr + AM2150_MACE_BASE + reg); break; - case 1: /* register 16-31 */ + case 1: /* register 16-31 */ spin_lock_irqsave(&lp->bank_lock, flags); MACEBANK(1); data = inb(ioaddr + AM2150_MACE_BASE + (reg & 0x0F)); @@ -512,10 +512,10 @@ static void mace_write(mace_private *lp, unsigned int ioaddr, int reg, unsigned long flags; switch (reg >> 4) { - case 0: /* register 0-15 */ + case 0: /* register 0-15 */ outb(data & 0xFF, ioaddr + AM2150_MACE_BASE + reg); break; - case 1: /* register 16-31 */ + case 1: /* register 16-31 */ spin_lock_irqsave(&lp->bank_lock, flags); MACEBANK(1); outb(data & 0xFF, ioaddr + AM2150_MACE_BASE + (reg & 0x0F)); @@ -567,13 +567,13 @@ static int mace_init(mace_private *lp, unsigned int ioaddr, * Or just set ASEL in PHYCC below! */ switch (if_port) { - case 1: + case 1: mace_write(lp, ioaddr, MACE_PLSCC, 0x02); break; - case 2: + case 2: mace_write(lp, ioaddr, MACE_PLSCC, 0x00); break; - default: + default: mace_write(lp, ioaddr, MACE_PHYCC, /* ASEL */ 4); /* ASEL Auto Select. When set, the PORTSEL[1-0] bits are overridden, and the MACE device will automatically select the operating media @@ -815,7 +815,7 @@ static int mace_close(struct net_device *dev) static void netdev_get_drvinfo(struct net_device *dev, struct ethtool_drvinfo *info) { - strlcpy(info->driver, DRV_NAME, sizeof(info->driver)); + strscpy(info->driver, DRV_NAME, sizeof(info->driver)); snprintf(info->bus_info, sizeof(info->bus_info), "PCMCIA 0x%lx", dev->base_addr); } @@ -918,7 +918,7 @@ static irqreturn_t mace_interrupt(int irq, void *dev_id) int status; int IntrCnt = MACE_MAX_IR_ITERATIONS; - if (dev == NULL) { + if (!dev) { pr_debug("mace_interrupt(): irq 0x%X for unknown device.\n", irq); return IRQ_NONE; @@ -1102,7 +1102,7 @@ static int mace_rx(struct net_device *dev, unsigned char RxCnt) skb = netdev_alloc_skb(dev, pkt_len + 2); - if (skb != NULL) { + if (skb) { skb_reserve(skb, 2); insw(ioaddr + AM2150_RCV, skb_put(skb, pkt_len), pkt_len>>1); if (pkt_len & 1) diff --git a/drivers/net/ethernet/amd/pcnet32.c b/drivers/net/ethernet/amd/pcnet32.c index b5ff47283cfe..72db9f9e7bee 100644 --- a/drivers/net/ethernet/amd/pcnet32.c +++ b/drivers/net/ethernet/amd/pcnet32.c @@ -488,7 +488,7 @@ static void pcnet32_realloc_tx_ring(struct net_device *dev, dma_alloc_coherent(&lp->pci_dev->dev, sizeof(struct pcnet32_tx_head) * entries, &new_ring_dma_addr, GFP_ATOMIC); - if (new_tx_ring == NULL) + if (!new_tx_ring) return; new_dma_addr_list = kcalloc(entries, sizeof(dma_addr_t), GFP_ATOMIC); @@ -547,7 +547,7 @@ static void pcnet32_realloc_rx_ring(struct net_device *dev, dma_alloc_coherent(&lp->pci_dev->dev, sizeof(struct pcnet32_rx_head) * entries, &new_ring_dma_addr, GFP_ATOMIC); - if (new_rx_ring == NULL) + if (!new_rx_ring) return; new_dma_addr_list = kcalloc(entries, sizeof(dma_addr_t), GFP_ATOMIC); @@ -797,9 +797,9 @@ static void pcnet32_get_drvinfo(struct net_device *dev, { struct pcnet32_private *lp = netdev_priv(dev); - strlcpy(info->driver, DRV_NAME, sizeof(info->driver)); + strscpy(info->driver, DRV_NAME, sizeof(info->driver)); if (lp->pci_dev) - strlcpy(info->bus_info, pci_name(lp->pci_dev), + strscpy(info->bus_info, pci_name(lp->pci_dev), sizeof(info->bus_info)); else snprintf(info->bus_info, sizeof(info->bus_info), @@ -1249,7 +1249,7 @@ static void pcnet32_rx_entry(struct net_device *dev, } else skb = netdev_alloc_skb(dev, pkt_len + NET_IP_ALIGN); - if (skb == NULL) { + if (!skb) { dev->stats.rx_dropped++; return; } @@ -2018,7 +2018,7 @@ static int pcnet32_alloc_ring(struct net_device *dev, const char *name) lp->tx_ring = dma_alloc_coherent(&lp->pci_dev->dev, sizeof(struct pcnet32_tx_head) * lp->tx_ring_size, &lp->tx_ring_dma_addr, GFP_KERNEL); - if (lp->tx_ring == NULL) { + if (!lp->tx_ring) { netif_err(lp, drv, dev, "Coherent memory allocation failed\n"); return -ENOMEM; } @@ -2026,7 +2026,7 @@ static int pcnet32_alloc_ring(struct net_device *dev, const char *name) lp->rx_ring = dma_alloc_coherent(&lp->pci_dev->dev, sizeof(struct pcnet32_rx_head) * lp->rx_ring_size, &lp->rx_ring_dma_addr, GFP_KERNEL); - if (lp->rx_ring == NULL) { + if (!lp->rx_ring) { netif_err(lp, drv, dev, "Coherent memory allocation failed\n"); return -ENOMEM; } @@ -2365,7 +2365,7 @@ static int pcnet32_init_ring(struct net_device *dev) for (i = 0; i < lp->rx_ring_size; i++) { struct sk_buff *rx_skbuff = lp->rx_skbuff[i]; - if (rx_skbuff == NULL) { + if (!rx_skbuff) { lp->rx_skbuff[i] = netdev_alloc_skb(dev, PKT_BUF_SKB); rx_skbuff = lp->rx_skbuff[i]; if (!rx_skbuff) { diff --git a/drivers/net/ethernet/amd/sun3lance.c b/drivers/net/ethernet/amd/sun3lance.c index 007bd7787291..246f34c43765 100644 --- a/drivers/net/ethernet/amd/sun3lance.c +++ b/drivers/net/ethernet/amd/sun3lance.c @@ -341,7 +341,7 @@ static int __init lance_probe( struct net_device *dev) /* XXX - leak? */ MEM = dvma_malloc_align(sizeof(struct lance_memory), 0x10000); - if (MEM == NULL) { + if (!MEM) { #ifdef CONFIG_SUN3 iounmap((void __iomem *)ioaddr); #endif @@ -796,7 +796,7 @@ static int lance_rx( struct net_device *dev ) } else { skb = netdev_alloc_skb(dev, pkt_len + 2); - if (skb == NULL) { + if (!skb) { dev->stats.rx_dropped++; head->msg_length = 0; head->flag |= RMD1_OWN_CHIP; diff --git a/drivers/net/ethernet/amd/sunlance.c b/drivers/net/ethernet/amd/sunlance.c index 22d609563af8..68ca1225eedc 100644 --- a/drivers/net/ethernet/amd/sunlance.c +++ b/drivers/net/ethernet/amd/sunlance.c @@ -530,7 +530,7 @@ static void lance_rx_dvma(struct net_device *dev) len = (rd->mblength & 0xfff) - 4; skb = netdev_alloc_skb(dev, len + 2); - if (skb == NULL) { + if (!skb) { dev->stats.rx_dropped++; rd->mblength = 0; rd->rmd1_bits = LE_R1_OWN; @@ -700,7 +700,7 @@ static void lance_rx_pio(struct net_device *dev) len = (sbus_readw(&rd->mblength) & 0xfff) - 4; skb = netdev_alloc_skb(dev, len + 2); - if (skb == NULL) { + if (!skb) { dev->stats.rx_dropped++; sbus_writew(0, &rd->mblength); sbus_writeb(LE_R1_OWN, &rd->rmd1_bits); @@ -1276,7 +1276,7 @@ static void lance_free_hwresources(struct lance_private *lp) /* Ethtool support... */ static void sparc_lance_get_drvinfo(struct net_device *dev, struct ethtool_drvinfo *info) { - strlcpy(info->driver, "sunlance", sizeof(info->driver)); + strscpy(info->driver, "sunlance", sizeof(info->driver)); } static const struct ethtool_ops sparc_lance_ethtool_ops = { diff --git a/drivers/net/ethernet/amd/xgbe/xgbe-drv.c b/drivers/net/ethernet/amd/xgbe/xgbe-drv.c index f342bb853189..7b666106feee 100644 --- a/drivers/net/ethernet/amd/xgbe/xgbe-drv.c +++ b/drivers/net/ethernet/amd/xgbe/xgbe-drv.c @@ -952,14 +952,14 @@ static void xgbe_napi_enable(struct xgbe_prv_data *pdata, unsigned int add) channel = pdata->channel[i]; if (add) netif_napi_add(pdata->netdev, &channel->napi, - xgbe_one_poll, NAPI_POLL_WEIGHT); + xgbe_one_poll); napi_enable(&channel->napi); } } else { if (add) netif_napi_add(pdata->netdev, &pdata->napi, - xgbe_all_poll, NAPI_POLL_WEIGHT); + xgbe_all_poll); napi_enable(&pdata->napi); } diff --git a/drivers/net/ethernet/amd/xgbe/xgbe-ethtool.c b/drivers/net/ethernet/amd/xgbe/xgbe-ethtool.c index 6ceb1cdf6eba..6e83ff59172a 100644 --- a/drivers/net/ethernet/amd/xgbe/xgbe-ethtool.c +++ b/drivers/net/ethernet/amd/xgbe/xgbe-ethtool.c @@ -402,8 +402,8 @@ static void xgbe_get_drvinfo(struct net_device *netdev, struct xgbe_prv_data *pdata = netdev_priv(netdev); struct xgbe_hw_features *hw_feat = &pdata->hw_feat; - strlcpy(drvinfo->driver, XGBE_DRV_NAME, sizeof(drvinfo->driver)); - strlcpy(drvinfo->bus_info, dev_name(pdata->dev), + strscpy(drvinfo->driver, XGBE_DRV_NAME, sizeof(drvinfo->driver)); + strscpy(drvinfo->bus_info, dev_name(pdata->dev), sizeof(drvinfo->bus_info)); snprintf(drvinfo->fw_version, sizeof(drvinfo->fw_version), "%d.%d.%d", XGMAC_GET_BITS(hw_feat->version, MAC_VR, USERVER), diff --git a/drivers/net/ethernet/apm/xgene-v2/main.c b/drivers/net/ethernet/apm/xgene-v2/main.c index d022b6db9e06..379d19d18dbe 100644 --- a/drivers/net/ethernet/apm/xgene-v2/main.c +++ b/drivers/net/ethernet/apm/xgene-v2/main.c @@ -672,7 +672,7 @@ static int xge_probe(struct platform_device *pdev) if (ret) goto err; - netif_napi_add(ndev, &pdata->napi, xge_napi, NAPI_POLL_WEIGHT); + netif_napi_add(ndev, &pdata->napi, xge_napi); ret = register_netdev(ndev); if (ret) { diff --git a/drivers/net/ethernet/apm/xgene/xgene_enet_main.c b/drivers/net/ethernet/apm/xgene/xgene_enet_main.c index 53dc8d5fede8..d6cfea65a714 100644 --- a/drivers/net/ethernet/apm/xgene/xgene_enet_main.c +++ b/drivers/net/ethernet/apm/xgene/xgene_enet_main.c @@ -1977,14 +1977,12 @@ static void xgene_enet_napi_add(struct xgene_enet_pdata *pdata) for (i = 0; i < pdata->rxq_cnt; i++) { napi = &pdata->rx_ring[i]->napi; - netif_napi_add(pdata->ndev, napi, xgene_enet_napi, - NAPI_POLL_WEIGHT); + netif_napi_add(pdata->ndev, napi, xgene_enet_napi); } for (i = 0; i < pdata->cq_cnt; i++) { napi = &pdata->tx_ring[i]->cp_ring->napi; - netif_napi_add(pdata->ndev, napi, xgene_enet_napi, - NAPI_POLL_WEIGHT); + netif_napi_add(pdata->ndev, napi, xgene_enet_napi); } } diff --git a/drivers/net/ethernet/aquantia/atlantic/aq_ethtool.c b/drivers/net/ethernet/aquantia/atlantic/aq_ethtool.c index 1daecd483b8d..a08f221e30d4 100644 --- a/drivers/net/ethernet/aquantia/atlantic/aq_ethtool.c +++ b/drivers/net/ethernet/aquantia/atlantic/aq_ethtool.c @@ -238,7 +238,7 @@ static void aq_ethtool_get_drvinfo(struct net_device *ndev, "%u.%u.%u", firmware_version >> 24, (firmware_version >> 16) & 0xFFU, firmware_version & 0xFFFFU); - strlcpy(drvinfo->bus_info, pdev ? pci_name(pdev) : "", + strscpy(drvinfo->bus_info, pdev ? pci_name(pdev) : "", sizeof(drvinfo->bus_info)); drvinfo->n_stats = aq_ethtool_n_stats(ndev); drvinfo->testinfo_len = 0; diff --git a/drivers/net/ethernet/aquantia/atlantic/aq_macsec.c b/drivers/net/ethernet/aquantia/atlantic/aq_macsec.c index 02058fe79f52..3d0e16791e1c 100644 --- a/drivers/net/ethernet/aquantia/atlantic/aq_macsec.c +++ b/drivers/net/ethernet/aquantia/atlantic/aq_macsec.c @@ -292,9 +292,6 @@ static int aq_mdo_dev_open(struct macsec_context *ctx) struct aq_nic_s *nic = netdev_priv(ctx->netdev); int ret = 0; - if (ctx->prepare) - return 0; - if (netif_carrier_ok(nic->ndev)) ret = aq_apply_secy_cfg(nic, ctx->secy); @@ -306,9 +303,6 @@ static int aq_mdo_dev_stop(struct macsec_context *ctx) struct aq_nic_s *nic = netdev_priv(ctx->netdev); int i; - if (ctx->prepare) - return 0; - for (i = 0; i < AQ_MACSEC_MAX_SC; i++) { if (nic->macsec_cfg->txsc_idx_busy & BIT(i)) aq_clear_secy(nic, nic->macsec_cfg->aq_txsc[i].sw_secy, @@ -466,9 +460,6 @@ static int aq_mdo_add_secy(struct macsec_context *ctx) if (txsc_idx == AQ_MACSEC_MAX_SC) return -ENOSPC; - if (ctx->prepare) - return 0; - cfg->sc_sa = sc_sa; cfg->aq_txsc[txsc_idx].hw_sc_idx = aq_to_hw_sc_idx(txsc_idx, sc_sa); cfg->aq_txsc[txsc_idx].sw_secy = secy; @@ -492,9 +483,6 @@ static int aq_mdo_upd_secy(struct macsec_context *ctx) if (txsc_idx < 0) return -ENOENT; - if (ctx->prepare) - return 0; - if (netif_carrier_ok(nic->ndev) && netif_running(secy->netdev)) ret = aq_set_txsc(nic, txsc_idx); @@ -543,9 +531,6 @@ static int aq_mdo_del_secy(struct macsec_context *ctx) struct aq_nic_s *nic = netdev_priv(ctx->netdev); int ret = 0; - if (ctx->prepare) - return 0; - if (!nic->macsec_cfg) return 0; @@ -601,9 +586,6 @@ static int aq_mdo_add_txsa(struct macsec_context *ctx) if (txsc_idx < 0) return -EINVAL; - if (ctx->prepare) - return 0; - aq_txsc = &cfg->aq_txsc[txsc_idx]; set_bit(ctx->sa.assoc_num, &aq_txsc->tx_sa_idx_busy); @@ -631,9 +613,6 @@ static int aq_mdo_upd_txsa(struct macsec_context *ctx) if (txsc_idx < 0) return -EINVAL; - if (ctx->prepare) - return 0; - aq_txsc = &cfg->aq_txsc[txsc_idx]; if (netif_carrier_ok(nic->ndev) && netif_running(secy->netdev)) ret = aq_update_txsa(nic, aq_txsc->hw_sc_idx, secy, @@ -681,9 +660,6 @@ static int aq_mdo_del_txsa(struct macsec_context *ctx) if (txsc_idx < 0) return -EINVAL; - if (ctx->prepare) - return 0; - ret = aq_clear_txsa(nic, &cfg->aq_txsc[txsc_idx], ctx->sa.assoc_num, AQ_CLEAR_ALL); @@ -780,9 +756,6 @@ static int aq_mdo_add_rxsc(struct macsec_context *ctx) if (rxsc_idx >= rxsc_idx_max) return -ENOSPC; - if (ctx->prepare) - return 0; - cfg->aq_rxsc[rxsc_idx].hw_sc_idx = aq_to_hw_sc_idx(rxsc_idx, cfg->sc_sa); cfg->aq_rxsc[rxsc_idx].sw_secy = ctx->secy; @@ -809,9 +782,6 @@ static int aq_mdo_upd_rxsc(struct macsec_context *ctx) if (rxsc_idx < 0) return -ENOENT; - if (ctx->prepare) - return 0; - if (netif_carrier_ok(nic->ndev) && netif_running(ctx->secy->netdev)) ret = aq_set_rxsc(nic, rxsc_idx); @@ -876,9 +846,6 @@ static int aq_mdo_del_rxsc(struct macsec_context *ctx) if (rxsc_idx < 0) return -ENOENT; - if (ctx->prepare) - return 0; - if (netif_carrier_ok(nic->ndev)) clear_type = AQ_CLEAR_ALL; @@ -948,9 +915,6 @@ static int aq_mdo_add_rxsa(struct macsec_context *ctx) if (rxsc_idx < 0) return -EINVAL; - if (ctx->prepare) - return 0; - aq_rxsc = &nic->macsec_cfg->aq_rxsc[rxsc_idx]; set_bit(ctx->sa.assoc_num, &aq_rxsc->rx_sa_idx_busy); @@ -978,9 +942,6 @@ static int aq_mdo_upd_rxsa(struct macsec_context *ctx) if (rxsc_idx < 0) return -EINVAL; - if (ctx->prepare) - return 0; - if (netif_carrier_ok(nic->ndev) && netif_running(secy->netdev)) ret = aq_update_rxsa(nic, cfg->aq_rxsc[rxsc_idx].hw_sc_idx, secy, ctx->sa.rx_sa, NULL, @@ -1029,9 +990,6 @@ static int aq_mdo_del_rxsa(struct macsec_context *ctx) if (rxsc_idx < 0) return -EINVAL; - if (ctx->prepare) - return 0; - ret = aq_clear_rxsa(nic, &cfg->aq_rxsc[rxsc_idx], ctx->sa.assoc_num, AQ_CLEAR_ALL); @@ -1044,9 +1002,6 @@ static int aq_mdo_get_dev_stats(struct macsec_context *ctx) struct aq_macsec_common_stats *stats = &nic->macsec_cfg->stats; struct aq_hw_s *hw = nic->aq_hw; - if (ctx->prepare) - return 0; - aq_get_macsec_common_stats(hw, stats); ctx->stats.dev_stats->OutPktsUntagged = stats->out.untagged_pkts; @@ -1073,9 +1028,6 @@ static int aq_mdo_get_tx_sc_stats(struct macsec_context *ctx) if (txsc_idx < 0) return -ENOENT; - if (ctx->prepare) - return 0; - aq_txsc = &nic->macsec_cfg->aq_txsc[txsc_idx]; stats = &aq_txsc->stats; aq_get_txsc_stats(hw, aq_txsc->hw_sc_idx, stats); @@ -1106,9 +1058,6 @@ static int aq_mdo_get_tx_sa_stats(struct macsec_context *ctx) if (txsc_idx < 0) return -EINVAL; - if (ctx->prepare) - return 0; - aq_txsc = &cfg->aq_txsc[txsc_idx]; sa_idx = aq_txsc->hw_sc_idx | ctx->sa.assoc_num; stats = &aq_txsc->tx_sa_stats[ctx->sa.assoc_num]; @@ -1147,9 +1096,6 @@ static int aq_mdo_get_rx_sc_stats(struct macsec_context *ctx) if (rxsc_idx < 0) return -ENOENT; - if (ctx->prepare) - return 0; - aq_rxsc = &cfg->aq_rxsc[rxsc_idx]; for (i = 0; i < MACSEC_NUM_AN; i++) { if (!test_bit(i, &aq_rxsc->rx_sa_idx_busy)) @@ -1196,9 +1142,6 @@ static int aq_mdo_get_rx_sa_stats(struct macsec_context *ctx) if (rxsc_idx < 0) return -EINVAL; - if (ctx->prepare) - return 0; - aq_rxsc = &cfg->aq_rxsc[rxsc_idx]; stats = &aq_rxsc->rx_sa_stats[ctx->sa.assoc_num]; sa_idx = aq_rxsc->hw_sc_idx | ctx->sa.assoc_num; diff --git a/drivers/net/ethernet/aquantia/atlantic/aq_ptp.c b/drivers/net/ethernet/aquantia/atlantic/aq_ptp.c index 275324c9e51e..80b44043e6c5 100644 --- a/drivers/net/ethernet/aquantia/atlantic/aq_ptp.c +++ b/drivers/net/ethernet/aquantia/atlantic/aq_ptp.c @@ -1217,8 +1217,7 @@ int aq_ptp_init(struct aq_nic_s *aq_nic, unsigned int idx_vec) atomic_set(&aq_ptp->offset_egress, 0); atomic_set(&aq_ptp->offset_ingress, 0); - netif_napi_add(aq_nic_get_ndev(aq_nic), &aq_ptp->napi, - aq_ptp_poll, NAPI_POLL_WEIGHT); + netif_napi_add(aq_nic_get_ndev(aq_nic), &aq_ptp->napi, aq_ptp_poll); aq_ptp->idx_vector = idx_vec; diff --git a/drivers/net/ethernet/aquantia/atlantic/aq_vec.c b/drivers/net/ethernet/aquantia/atlantic/aq_vec.c index f0fdf20f01c1..f5db1c44e9b9 100644 --- a/drivers/net/ethernet/aquantia/atlantic/aq_vec.c +++ b/drivers/net/ethernet/aquantia/atlantic/aq_vec.c @@ -119,8 +119,7 @@ struct aq_vec_s *aq_vec_alloc(struct aq_nic_s *aq_nic, unsigned int idx, self->tx_rings = 0; self->rx_rings = 0; - netif_napi_add(aq_nic_get_ndev(aq_nic), &self->napi, - aq_vec_poll, NAPI_POLL_WEIGHT); + netif_napi_add(aq_nic_get_ndev(aq_nic), &self->napi, aq_vec_poll); err_exit: return self; diff --git a/drivers/net/ethernet/arc/emac_main.c b/drivers/net/ethernet/arc/emac_main.c index 288e2961823e..ba0646b3b122 100644 --- a/drivers/net/ethernet/arc/emac_main.c +++ b/drivers/net/ethernet/arc/emac_main.c @@ -91,7 +91,7 @@ static void arc_emac_get_drvinfo(struct net_device *ndev, { struct arc_emac_priv *priv = netdev_priv(ndev); - strlcpy(info->driver, priv->drv_name, sizeof(info->driver)); + strscpy(info->driver, priv->drv_name, sizeof(info->driver)); } static const struct ethtool_ops arc_emac_ethtool_ops = { diff --git a/drivers/net/ethernet/asix/ax88796c_main.c b/drivers/net/ethernet/asix/ax88796c_main.c index 6ba5b024a7be..8b7cdf015a16 100644 --- a/drivers/net/ethernet/asix/ax88796c_main.c +++ b/drivers/net/ethernet/asix/ax88796c_main.c @@ -293,7 +293,7 @@ ax88796c_tx_fixup(struct net_device *ndev, struct sk_buff_head *q) skb_put(skb, padlen); /* EOP header */ - memcpy(skb_put(skb, TX_EOP_SIZE), &info.eop, TX_EOP_SIZE); + skb_put_data(skb, &info.eop, TX_EOP_SIZE); skb_unlink(skb, q); @@ -381,7 +381,7 @@ static int ax88796c_hard_xmit(struct ax88796c_device *ax_local) return 1; } -static int +static netdev_tx_t ax88796c_start_xmit(struct sk_buff *skb, struct net_device *ndev) { struct ax88796c_device *ax_local = to_ax88796c_device(ndev); diff --git a/drivers/net/ethernet/atheros/ag71xx.c b/drivers/net/ethernet/atheros/ag71xx.c index e461f4764066..cc932b3cf873 100644 --- a/drivers/net/ethernet/atheros/ag71xx.c +++ b/drivers/net/ethernet/atheros/ag71xx.c @@ -451,8 +451,8 @@ static void ag71xx_get_drvinfo(struct net_device *ndev, { struct ag71xx *ag = netdev_priv(ndev); - strlcpy(info->driver, "ag71xx", sizeof(info->driver)); - strlcpy(info->bus_info, of_node_full_name(ag->pdev->dev.of_node), + strscpy(info->driver, "ag71xx", sizeof(info->driver)); + strscpy(info->bus_info, of_node_full_name(ag->pdev->dev.of_node), sizeof(info->bus_info)); } diff --git a/drivers/net/ethernet/atheros/alx/main.c b/drivers/net/ethernet/atheros/alx/main.c index a89b93cb4e26..d30d11872719 100644 --- a/drivers/net/ethernet/atheros/alx/main.c +++ b/drivers/net/ethernet/atheros/alx/main.c @@ -752,7 +752,7 @@ static int alx_alloc_napis(struct alx_priv *alx) goto err_out; np->alx = alx; - netif_napi_add(alx->dev, &np->napi, alx_poll, 64); + netif_napi_add(alx->dev, &np->napi, alx_poll); alx->qnapi[i] = np; } @@ -1912,11 +1912,14 @@ static int alx_suspend(struct device *dev) if (!netif_running(alx->dev)) return 0; + + rtnl_lock(); netif_device_detach(alx->dev); mutex_lock(&alx->mtx); __alx_stop(alx); mutex_unlock(&alx->mtx); + rtnl_unlock(); return 0; } @@ -1927,6 +1930,7 @@ static int alx_resume(struct device *dev) struct alx_hw *hw = &alx->hw; int err; + rtnl_lock(); mutex_lock(&alx->mtx); alx_reset_phy(hw); @@ -1943,6 +1947,7 @@ static int alx_resume(struct device *dev) unlock: mutex_unlock(&alx->mtx); + rtnl_unlock(); return err; } diff --git a/drivers/net/ethernet/atheros/atl1c/atl1c_ethtool.c b/drivers/net/ethernet/atheros/atl1c/atl1c_ethtool.c index e2eb7b8c63a0..0bce122c68f1 100644 --- a/drivers/net/ethernet/atheros/atl1c/atl1c_ethtool.c +++ b/drivers/net/ethernet/atheros/atl1c/atl1c_ethtool.c @@ -220,8 +220,8 @@ static void atl1c_get_drvinfo(struct net_device *netdev, { struct atl1c_adapter *adapter = netdev_priv(netdev); - strlcpy(drvinfo->driver, atl1c_driver_name, sizeof(drvinfo->driver)); - strlcpy(drvinfo->bus_info, pci_name(adapter->pdev), + strscpy(drvinfo->driver, atl1c_driver_name, sizeof(drvinfo->driver)); + strscpy(drvinfo->bus_info, pci_name(adapter->pdev), sizeof(drvinfo->bus_info)); } diff --git a/drivers/net/ethernet/atheros/atl1c/atl1c_main.c b/drivers/net/ethernet/atheros/atl1c/atl1c_main.c index be4b1f8eef29..40c781695d58 100644 --- a/drivers/net/ethernet/atheros/atl1c/atl1c_main.c +++ b/drivers/net/ethernet/atheros/atl1c/atl1c_main.c @@ -2732,7 +2732,7 @@ static int atl1c_probe(struct pci_dev *pdev, const struct pci_device_id *ent) dev_set_threaded(netdev, true); for (i = 0; i < adapter->rx_queue_count; ++i) netif_napi_add(netdev, &adapter->rrd_ring[i].napi, - atl1c_clean_rx, 64); + atl1c_clean_rx); for (i = 0; i < adapter->tx_queue_count; ++i) netif_napi_add_tx(netdev, &adapter->tpd_ring[i].napi, atl1c_clean_tx); diff --git a/drivers/net/ethernet/atheros/atl1e/atl1e_ethtool.c b/drivers/net/ethernet/atheros/atl1e/atl1e_ethtool.c index 0cbde352d1ba..68f1832a198d 100644 --- a/drivers/net/ethernet/atheros/atl1e/atl1e_ethtool.c +++ b/drivers/net/ethernet/atheros/atl1e/atl1e_ethtool.c @@ -306,9 +306,9 @@ static void atl1e_get_drvinfo(struct net_device *netdev, { struct atl1e_adapter *adapter = netdev_priv(netdev); - strlcpy(drvinfo->driver, atl1e_driver_name, sizeof(drvinfo->driver)); - strlcpy(drvinfo->fw_version, "L1e", sizeof(drvinfo->fw_version)); - strlcpy(drvinfo->bus_info, pci_name(adapter->pdev), + strscpy(drvinfo->driver, atl1e_driver_name, sizeof(drvinfo->driver)); + strscpy(drvinfo->fw_version, "L1e", sizeof(drvinfo->fw_version)); + strscpy(drvinfo->bus_info, pci_name(adapter->pdev), sizeof(drvinfo->bus_info)); } diff --git a/drivers/net/ethernet/atheros/atl1e/atl1e_main.c b/drivers/net/ethernet/atheros/atl1e/atl1e_main.c index 57a51fb7746c..5db0f3495a32 100644 --- a/drivers/net/ethernet/atheros/atl1e/atl1e_main.c +++ b/drivers/net/ethernet/atheros/atl1e/atl1e_main.c @@ -2354,7 +2354,7 @@ static int atl1e_probe(struct pci_dev *pdev, const struct pci_device_id *ent) adapter->mii.phy_id_mask = 0x1f; adapter->mii.reg_num_mask = MDIO_REG_ADDR_MASK; - netif_napi_add(netdev, &adapter->napi, atl1e_clean, 64); + netif_napi_add(netdev, &adapter->napi, atl1e_clean); timer_setup(&adapter->phy_config_timer, atl1e_phy_config, 0); diff --git a/drivers/net/ethernet/atheros/atlx/atl1.c b/drivers/net/ethernet/atheros/atlx/atl1.c index ff1fe09abf9f..c8444bcdf527 100644 --- a/drivers/net/ethernet/atheros/atlx/atl1.c +++ b/drivers/net/ethernet/atheros/atlx/atl1.c @@ -2977,7 +2977,7 @@ static int atl1_probe(struct pci_dev *pdev, const struct pci_device_id *ent) netdev->netdev_ops = &atl1_netdev_ops; netdev->watchdog_timeo = 5 * HZ; - netif_napi_add(netdev, &adapter->napi, atl1_rings_clean, 64); + netif_napi_add(netdev, &adapter->napi, atl1_rings_clean); netdev->ethtool_ops = &atl1_ethtool_ops; adapter->bd_number = cards_found; @@ -3340,8 +3340,8 @@ static void atl1_get_drvinfo(struct net_device *netdev, { struct atl1_adapter *adapter = netdev_priv(netdev); - strlcpy(drvinfo->driver, ATLX_DRIVER_NAME, sizeof(drvinfo->driver)); - strlcpy(drvinfo->bus_info, pci_name(adapter->pdev), + strscpy(drvinfo->driver, ATLX_DRIVER_NAME, sizeof(drvinfo->driver)); + strscpy(drvinfo->bus_info, pci_name(adapter->pdev), sizeof(drvinfo->bus_info)); } diff --git a/drivers/net/ethernet/atheros/atlx/atl2.c b/drivers/net/ethernet/atheros/atlx/atl2.c index bbc4d7b08a49..1b487c071cb6 100644 --- a/drivers/net/ethernet/atheros/atlx/atl2.c +++ b/drivers/net/ethernet/atheros/atlx/atl2.c @@ -1980,9 +1980,9 @@ static void atl2_get_drvinfo(struct net_device *netdev, { struct atl2_adapter *adapter = netdev_priv(netdev); - strlcpy(drvinfo->driver, atl2_driver_name, sizeof(drvinfo->driver)); - strlcpy(drvinfo->fw_version, "L2", sizeof(drvinfo->fw_version)); - strlcpy(drvinfo->bus_info, pci_name(adapter->pdev), + strscpy(drvinfo->driver, atl2_driver_name, sizeof(drvinfo->driver)); + strscpy(drvinfo->fw_version, "L2", sizeof(drvinfo->fw_version)); + strscpy(drvinfo->bus_info, pci_name(adapter->pdev), sizeof(drvinfo->bus_info)); } diff --git a/drivers/net/ethernet/broadcom/b44.c b/drivers/net/ethernet/broadcom/b44.c index e5857e88c207..7f876721596c 100644 --- a/drivers/net/ethernet/broadcom/b44.c +++ b/drivers/net/ethernet/broadcom/b44.c @@ -1790,13 +1790,13 @@ static void b44_get_drvinfo (struct net_device *dev, struct ethtool_drvinfo *inf struct b44 *bp = netdev_priv(dev); struct ssb_bus *bus = bp->sdev->bus; - strlcpy(info->driver, DRV_MODULE_NAME, sizeof(info->driver)); + strscpy(info->driver, DRV_MODULE_NAME, sizeof(info->driver)); switch (bus->bustype) { case SSB_BUSTYPE_PCI: - strlcpy(info->bus_info, pci_name(bus->host_pci), sizeof(info->bus_info)); + strscpy(info->bus_info, pci_name(bus->host_pci), sizeof(info->bus_info)); break; case SSB_BUSTYPE_SSB: - strlcpy(info->bus_info, "SSB", sizeof(info->bus_info)); + strscpy(info->bus_info, "SSB", sizeof(info->bus_info)); break; case SSB_BUSTYPE_PCMCIA: case SSB_BUSTYPE_SDIO: @@ -2375,7 +2375,7 @@ static int b44_init_one(struct ssb_device *sdev, bp->tx_pending = B44_DEF_TX_RING_PENDING; dev->netdev_ops = &b44_netdev_ops; - netif_napi_add(dev, &bp->napi, b44_poll, 64); + netif_napi_add(dev, &bp->napi, b44_poll); dev->watchdog_timeo = B44_TX_TIMEOUT; dev->min_mtu = B44_MIN_MTU; dev->max_mtu = B44_MAX_MTU; diff --git a/drivers/net/ethernet/broadcom/bcm4908_enet.c b/drivers/net/ethernet/broadcom/bcm4908_enet.c index c131d8118489..93ccf549e2ed 100644 --- a/drivers/net/ethernet/broadcom/bcm4908_enet.c +++ b/drivers/net/ethernet/broadcom/bcm4908_enet.c @@ -507,7 +507,7 @@ static int bcm4908_enet_stop(struct net_device *netdev) return 0; } -static int bcm4908_enet_start_xmit(struct sk_buff *skb, struct net_device *netdev) +static netdev_tx_t bcm4908_enet_start_xmit(struct sk_buff *skb, struct net_device *netdev) { struct bcm4908_enet *enet = netdev_priv(netdev); struct bcm4908_enet_dma_ring *ring = &enet->tx_ring; @@ -716,6 +716,8 @@ static int bcm4908_enet_probe(struct platform_device *pdev) SET_NETDEV_DEV(netdev, &pdev->dev); err = of_get_ethdev_address(dev->of_node, netdev); + if (err == -EPROBE_DEFER) + goto err_dma_free; if (err) eth_hw_addr_random(netdev); netdev->netdev_ops = &bcm4908_enet_netdev_ops; @@ -723,17 +725,20 @@ static int bcm4908_enet_probe(struct platform_device *pdev) netdev->mtu = ETH_DATA_LEN; netdev->max_mtu = ENET_MTU_MAX; netif_napi_add_tx(netdev, &enet->tx_ring.napi, bcm4908_enet_poll_tx); - netif_napi_add(netdev, &enet->rx_ring.napi, bcm4908_enet_poll_rx, NAPI_POLL_WEIGHT); + netif_napi_add(netdev, &enet->rx_ring.napi, bcm4908_enet_poll_rx); err = register_netdev(netdev); - if (err) { - bcm4908_enet_dma_free(enet); - return err; - } + if (err) + goto err_dma_free; platform_set_drvdata(pdev, enet); return 0; + +err_dma_free: + bcm4908_enet_dma_free(enet); + + return err; } static int bcm4908_enet_remove(struct platform_device *pdev) diff --git a/drivers/net/ethernet/broadcom/bcm63xx_enet.c b/drivers/net/ethernet/broadcom/bcm63xx_enet.c index 1c6aea12db72..d91fdb0c2649 100644 --- a/drivers/net/ethernet/broadcom/bcm63xx_enet.c +++ b/drivers/net/ethernet/broadcom/bcm63xx_enet.c @@ -1321,8 +1321,8 @@ static const u32 unused_mib_regs[] = { static void bcm_enet_get_drvinfo(struct net_device *netdev, struct ethtool_drvinfo *drvinfo) { - strlcpy(drvinfo->driver, bcm_enet_driver_name, sizeof(drvinfo->driver)); - strlcpy(drvinfo->bus_info, "bcm63xx", sizeof(drvinfo->bus_info)); + strscpy(drvinfo->driver, bcm_enet_driver_name, sizeof(drvinfo->driver)); + strscpy(drvinfo->bus_info, "bcm63xx", sizeof(drvinfo->bus_info)); } static int bcm_enet_get_sset_count(struct net_device *netdev, diff --git a/drivers/net/ethernet/broadcom/bcmsysport.c b/drivers/net/ethernet/broadcom/bcmsysport.c index 47fc8e6963d5..867f14c30e09 100644 --- a/drivers/net/ethernet/broadcom/bcmsysport.c +++ b/drivers/net/ethernet/broadcom/bcmsysport.c @@ -308,8 +308,8 @@ static const struct bcm_sysport_stats bcm_sysport_gstrings_stats[] = { static void bcm_sysport_get_drvinfo(struct net_device *dev, struct ethtool_drvinfo *info) { - strlcpy(info->driver, KBUILD_MODNAME, sizeof(info->driver)); - strlcpy(info->bus_info, "platform", sizeof(info->bus_info)); + strscpy(info->driver, KBUILD_MODNAME, sizeof(info->driver)); + strscpy(info->bus_info, "platform", sizeof(info->bus_info)); } static u32 bcm_sysport_get_msglvl(struct net_device *dev) @@ -2564,7 +2564,7 @@ static int bcm_sysport_probe(struct platform_device *pdev) dev_set_drvdata(&pdev->dev, dev); dev->ethtool_ops = &bcm_sysport_ethtool_ops; dev->netdev_ops = &bcm_sysport_netdev_ops; - netif_napi_add(dev, &priv->napi, bcm_sysport_poll, 64); + netif_napi_add(dev, &priv->napi, bcm_sysport_poll); dev->features |= NETIF_F_RXCSUM | NETIF_F_HIGHDMA | NETIF_F_IP_CSUM | NETIF_F_IPV6_CSUM | diff --git a/drivers/net/ethernet/broadcom/bgmac.c b/drivers/net/ethernet/broadcom/bgmac.c index 93580484a3f4..5fb3af5670ec 100644 --- a/drivers/net/ethernet/broadcom/bgmac.c +++ b/drivers/net/ethernet/broadcom/bgmac.c @@ -1367,7 +1367,7 @@ static void bgmac_get_strings(struct net_device *dev, u32 stringset, return; for (i = 0; i < BGMAC_STATS_LEN; i++) - strlcpy(data + i * ETH_GSTRING_LEN, + strscpy(data + i * ETH_GSTRING_LEN, bgmac_get_strings_stats[i].name, ETH_GSTRING_LEN); } @@ -1395,8 +1395,8 @@ static void bgmac_get_ethtool_stats(struct net_device *dev, static void bgmac_get_drvinfo(struct net_device *net_dev, struct ethtool_drvinfo *info) { - strlcpy(info->driver, KBUILD_MODNAME, sizeof(info->driver)); - strlcpy(info->bus_info, "AXI", sizeof(info->bus_info)); + strscpy(info->driver, KBUILD_MODNAME, sizeof(info->driver)); + strscpy(info->bus_info, "AXI", sizeof(info->bus_info)); } static const struct ethtool_ops bgmac_ethtool_ops = { @@ -1527,7 +1527,7 @@ int bgmac_enet_probe(struct bgmac *bgmac) if (bcm47xx_nvram_getenv("et0_no_txint", NULL, 0) == 0) bgmac->int_mask &= ~BGMAC_IS_TX_MASK; - netif_napi_add(net_dev, &bgmac->napi, bgmac_poll, NAPI_POLL_WEIGHT); + netif_napi_add(net_dev, &bgmac->napi, bgmac_poll); err = bgmac_phy_connect(bgmac); if (err) { diff --git a/drivers/net/ethernet/broadcom/bnx2.c b/drivers/net/ethernet/broadcom/bnx2.c index b97ed9b5f685..fec57f1982c8 100644 --- a/drivers/net/ethernet/broadcom/bnx2.c +++ b/drivers/net/ethernet/broadcom/bnx2.c @@ -176,12 +176,12 @@ static const struct flash_spec flash_table[] = {0x19000002, 0x5b808201, 0x000500db, 0x03840253, 0xaf020406, NONBUFFERED_FLAGS, ST_MICRO_FLASH_PAGE_BITS, ST_MICRO_FLASH_PAGE_SIZE, ST_MICRO_FLASH_BYTE_ADDR_MASK, ST_MICRO_FLASH_BASE_TOTAL_SIZE*2, - "Entry 0101: ST M45PE10 (128kB non-bufferred)"}, + "Entry 0101: ST M45PE10 (128kB non-buffered)"}, /* Entry 0110: ST M45PE20 (non-buffered flash)*/ {0x15000001, 0x57808201, 0x000500db, 0x03840253, 0xaf020406, NONBUFFERED_FLAGS, ST_MICRO_FLASH_PAGE_BITS, ST_MICRO_FLASH_PAGE_SIZE, ST_MICRO_FLASH_BYTE_ADDR_MASK, ST_MICRO_FLASH_BASE_TOTAL_SIZE*4, - "Entry 0110: ST M45PE20 (256kB non-bufferred)"}, + "Entry 0110: ST M45PE20 (256kB non-buffered)"}, /* Saifun SA25F005 (non-buffered flash) */ /* strap, cfg1, & write1 need updates */ {0x1d000003, 0x5f808201, 0x00050081, 0x03840253, 0xaf020406, @@ -7042,9 +7042,9 @@ bnx2_get_drvinfo(struct net_device *dev, struct ethtool_drvinfo *info) { struct bnx2 *bp = netdev_priv(dev); - strlcpy(info->driver, DRV_MODULE_NAME, sizeof(info->driver)); - strlcpy(info->bus_info, pci_name(bp->pdev), sizeof(info->bus_info)); - strlcpy(info->fw_version, bp->fw_version, sizeof(info->fw_version)); + strscpy(info->driver, DRV_MODULE_NAME, sizeof(info->driver)); + strscpy(info->bus_info, pci_name(bp->pdev), sizeof(info->bus_info)); + strscpy(info->fw_version, bp->fw_version, sizeof(info->fw_version)); } #define BNX2_REGDUMP_LEN (32 * 1024) @@ -8522,7 +8522,7 @@ bnx2_init_napi(struct bnx2 *bp) else poll = bnx2_poll_msix; - netif_napi_add(bp->dev, &bp->bnx2_napi[i].napi, poll, 64); + netif_napi_add(bp->dev, &bp->bnx2_napi[i].napi, poll); bnapi->bp = bp; } } diff --git a/drivers/net/ethernet/broadcom/bnx2x/bnx2x_cmn.c b/drivers/net/ethernet/broadcom/bnx2x/bnx2x_cmn.c index 712b5595bc39..16c490692f42 100644 --- a/drivers/net/ethernet/broadcom/bnx2x/bnx2x_cmn.c +++ b/drivers/net/ethernet/broadcom/bnx2x/bnx2x_cmn.c @@ -44,8 +44,7 @@ static void bnx2x_add_all_napi_cnic(struct bnx2x *bp) /* Add NAPI objects */ for_each_rx_queue_cnic(bp, i) { - netif_napi_add(bp->dev, &bnx2x_fp(bp, i, napi), - bnx2x_poll, NAPI_POLL_WEIGHT); + netif_napi_add(bp->dev, &bnx2x_fp(bp, i, napi), bnx2x_poll); } } @@ -55,8 +54,7 @@ static void bnx2x_add_all_napi(struct bnx2x *bp) /* Add NAPI objects */ for_each_eth_queue(bp, i) { - netif_napi_add(bp->dev, &bnx2x_fp(bp, i, napi), - bnx2x_poll, NAPI_POLL_WEIGHT); + netif_napi_add(bp->dev, &bnx2x_fp(bp, i, napi), bnx2x_poll); } } @@ -150,7 +148,7 @@ void bnx2x_fill_fw_str(struct bnx2x *bp, char *buf, size_t buf_len) phy_fw_ver[0] = '\0'; bnx2x_get_ext_phy_fw_version(&bp->link_params, phy_fw_ver, PHY_FW_VER_LEN); - strlcpy(buf, bp->fw_ver, buf_len); + strscpy(buf, bp->fw_ver, buf_len); snprintf(buf + strlen(bp->fw_ver), 32 - strlen(bp->fw_ver), "bc %d.%d.%d%s%s", (bp->common.bc_ver & 0xff0000) >> 16, @@ -789,6 +787,7 @@ static void bnx2x_tpa_stop(struct bnx2x *bp, struct bnx2x_fastpath *fp, BNX2X_ERR("skb_put is about to fail... pad %d len %d rx_buf_size %d\n", pad, len, fp->rx_buf_size); bnx2x_panic(); + bnx2x_frag_free(fp, new_data); return; } #endif diff --git a/drivers/net/ethernet/broadcom/bnx2x/bnx2x_ethtool.c b/drivers/net/ethernet/broadcom/bnx2x/bnx2x_ethtool.c index 0e319ac7799f..bda3ccc28eca 100644 --- a/drivers/net/ethernet/broadcom/bnx2x/bnx2x_ethtool.c +++ b/drivers/net/ethernet/broadcom/bnx2x/bnx2x_ethtool.c @@ -1112,7 +1112,7 @@ static void bnx2x_get_drvinfo(struct net_device *dev, int ext_dev_info_offset; u32 mbi; - strlcpy(info->driver, DRV_MODULE_NAME, sizeof(info->driver)); + strscpy(info->driver, DRV_MODULE_NAME, sizeof(info->driver)); if (SHMEM2_HAS(bp, extended_dev_info_shared_addr)) { ext_dev_info_offset = SHMEM2_RD(bp, @@ -1126,7 +1126,7 @@ static void bnx2x_get_drvinfo(struct net_device *dev, (mbi & 0xff000000) >> 24, (mbi & 0x00ff0000) >> 16, (mbi & 0x0000ff00) >> 8); - strlcpy(info->fw_version, version, + strscpy(info->fw_version, version, sizeof(info->fw_version)); } } @@ -1135,7 +1135,7 @@ static void bnx2x_get_drvinfo(struct net_device *dev, bnx2x_fill_fw_str(bp, version, ETHTOOL_FWVERS_LEN); strlcat(info->fw_version, version, sizeof(info->fw_version)); - strlcpy(info->bus_info, pci_name(bp->pdev), sizeof(info->bus_info)); + strscpy(info->bus_info, pci_name(bp->pdev), sizeof(info->bus_info)); } static void bnx2x_get_wol(struct net_device *dev, struct ethtool_wolinfo *wol) diff --git a/drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c b/drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c index 962253db25b8..51b1690fd045 100644 --- a/drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c +++ b/drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c @@ -3385,7 +3385,7 @@ static void bnx2x_drv_info_ether_stat(struct bnx2x *bp) &bp->sp_objs->mac_obj; int i; - strlcpy(ether_stat->version, DRV_MODULE_VERSION, + strscpy(ether_stat->version, DRV_MODULE_VERSION, ETH_STAT_INFO_VERSION_LEN); /* get DRV_INFO_ETH_STAT_NUM_MACS_REQUIRED macs, placing them in the diff --git a/drivers/net/ethernet/broadcom/bnx2x/bnx2x_sriov.h b/drivers/net/ethernet/broadcom/bnx2x/bnx2x_sriov.h index 2dac704dc346..02a4e557e176 100644 --- a/drivers/net/ethernet/broadcom/bnx2x/bnx2x_sriov.h +++ b/drivers/net/ethernet/broadcom/bnx2x/bnx2x_sriov.h @@ -518,7 +518,7 @@ int bnx2x_vfpf_storm_rx_mode(struct bnx2x *bp); static inline void bnx2x_vf_fill_fw_str(struct bnx2x *bp, char *buf, size_t buf_len) { - strlcpy(buf, bp->acquire_resp.pfdev_info.fw_ver, buf_len); + strscpy(buf, bp->acquire_resp.pfdev_info.fw_ver, buf_len); } static inline int bnx2x_vf_ustorm_prods_offset(struct bnx2x *bp, diff --git a/drivers/net/ethernet/broadcom/bnx2x/bnx2x_vfpf.c b/drivers/net/ethernet/broadcom/bnx2x/bnx2x_vfpf.c index c9129b9ba446..0657a0f5170f 100644 --- a/drivers/net/ethernet/broadcom/bnx2x/bnx2x_vfpf.c +++ b/drivers/net/ethernet/broadcom/bnx2x/bnx2x_vfpf.c @@ -380,7 +380,7 @@ int bnx2x_vfpf_acquire(struct bnx2x *bp, u8 tx_count, u8 rx_count) bp->igu_base_sb = bp->acquire_resp.resc.hw_sbs[0].hw_sb_id; bp->vlan_credit = bp->acquire_resp.resc.num_vlan_filters; - strlcpy(bp->fw_ver, bp->acquire_resp.pfdev_info.fw_ver, + strscpy(bp->fw_ver, bp->acquire_resp.pfdev_info.fw_ver, sizeof(bp->fw_ver)); if (is_valid_ether_addr(bp->acquire_resp.resc.current_mac_addr)) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c index 96da0ba3d507..eed98c10ca9d 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c @@ -9366,16 +9366,16 @@ static void bnxt_init_napi(struct bnxt *bp) cp_nr_rings--; for (i = 0; i < cp_nr_rings; i++) { bnapi = bp->bnapi[i]; - netif_napi_add(bp->dev, &bnapi->napi, poll_fn, 64); + netif_napi_add(bp->dev, &bnapi->napi, poll_fn); } if (BNXT_CHIP_TYPE_NITRO_A0(bp)) { bnapi = bp->bnapi[cp_nr_rings]; netif_napi_add(bp->dev, &bnapi->napi, - bnxt_poll_nitroa0, 64); + bnxt_poll_nitroa0); } } else { bnapi = bp->bnapi[0]; - netif_napi_add(bp->dev, &bnapi->napi, bnxt_poll, 64); + netif_napi_add(bp->dev, &bnapi->napi, bnxt_poll); } } diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c index 87eb5362ad70..f57e524c7e30 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c @@ -1371,9 +1371,9 @@ static void bnxt_get_drvinfo(struct net_device *dev, { struct bnxt *bp = netdev_priv(dev); - strlcpy(info->driver, DRV_MODULE_NAME, sizeof(info->driver)); - strlcpy(info->fw_version, bp->fw_ver_str, sizeof(info->fw_version)); - strlcpy(info->bus_info, pci_name(bp->pdev), sizeof(info->bus_info)); + strscpy(info->driver, DRV_MODULE_NAME, sizeof(info->driver)); + strscpy(info->fw_version, bp->fw_ver_str, sizeof(info->fw_version)); + strscpy(info->bus_info, pci_name(bp->pdev), sizeof(info->bus_info)); info->n_stats = bnxt_get_num_stats(bp); info->testinfo_len = bp->num_tests; /* TODO CHIMP_FW: eeprom dump details */ @@ -3876,7 +3876,7 @@ void bnxt_ethtool_init(struct bnxt *bp) } else if (i == BNXT_IRQ_TEST_IDX) { strcpy(str, "Interrupt_test (offline)"); } else { - strlcpy(str, fw_str, ETH_GSTRING_LEN); + strscpy(str, fw_str, ETH_GSTRING_LEN); strncat(str, " test", ETH_GSTRING_LEN - strlen(str)); if (test_info->offline_mask & (1 << i)) strncat(str, " (offline)", diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_ptp.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_ptp.c index 8e316367f6ce..2132ce63193c 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt_ptp.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_ptp.c @@ -505,9 +505,13 @@ static int bnxt_hwrm_ptp_cfg(struct bnxt *bp) ptp->tstamp_filters = flags; if (netif_running(bp->dev)) { - rc = bnxt_close_nic(bp, false, false); - if (!rc) - rc = bnxt_open_nic(bp, false, false); + if (ptp->rx_filter == HWTSTAMP_FILTER_ALL) { + rc = bnxt_close_nic(bp, false, false); + if (!rc) + rc = bnxt_open_nic(bp, false, false); + } else { + bnxt_ptp_cfg_tstamp_filters(bp); + } if (!rc && !ptp->tstamp_filters) rc = -EIO; } diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_vfr.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_vfr.c index eb4803b11c0e..fcc65890820a 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt_vfr.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_vfr.c @@ -222,7 +222,7 @@ static int bnxt_vf_rep_get_phys_port_name(struct net_device *dev, char *buf, static void bnxt_vf_rep_get_drvinfo(struct net_device *dev, struct ethtool_drvinfo *info) { - strlcpy(info->driver, DRV_MODULE_NAME, sizeof(info->driver)); + strscpy(info->driver, DRV_MODULE_NAME, sizeof(info->driver)); } static int bnxt_vf_rep_get_port_parent_id(struct net_device *dev, diff --git a/drivers/net/ethernet/broadcom/genet/bcmgenet.c b/drivers/net/ethernet/broadcom/genet/bcmgenet.c index 8309fb993cdb..25c450606985 100644 --- a/drivers/net/ethernet/broadcom/genet/bcmgenet.c +++ b/drivers/net/ethernet/broadcom/genet/bcmgenet.c @@ -1146,7 +1146,7 @@ static const struct bcmgenet_stats bcmgenet_gstrings_stats[] = { static void bcmgenet_get_drvinfo(struct net_device *dev, struct ethtool_drvinfo *info) { - strlcpy(info->driver, "bcmgenet", sizeof(info->driver)); + strscpy(info->driver, "bcmgenet", sizeof(info->driver)); } static int bcmgenet_get_sset_count(struct net_device *dev, int string_set) @@ -2707,8 +2707,7 @@ static int bcmgenet_init_rx_ring(struct bcmgenet_priv *priv, bcmgenet_init_rx_coalesce(ring); /* Initialize Rx NAPI */ - netif_napi_add(priv->dev, &ring->napi, bcmgenet_rx_poll, - NAPI_POLL_WEIGHT); + netif_napi_add(priv->dev, &ring->napi, bcmgenet_rx_poll); bcmgenet_rdma_ring_writel(priv, index, 0, RDMA_PROD_INDEX); bcmgenet_rdma_ring_writel(priv, index, 0, RDMA_CONS_INDEX); diff --git a/drivers/net/ethernet/broadcom/tg3.c b/drivers/net/ethernet/broadcom/tg3.c index 89889d8150da..4179a12fc881 100644 --- a/drivers/net/ethernet/broadcom/tg3.c +++ b/drivers/net/ethernet/broadcom/tg3.c @@ -7380,9 +7380,9 @@ static void tg3_napi_init(struct tg3 *tp) { int i; - netif_napi_add(tp->dev, &tp->napi[0].napi, tg3_poll, 64); + netif_napi_add(tp->dev, &tp->napi[0].napi, tg3_poll); for (i = 1; i < tp->irq_cnt; i++) - netif_napi_add(tp->dev, &tp->napi[i].napi, tg3_poll_msix, 64); + netif_napi_add(tp->dev, &tp->napi[i].napi, tg3_poll_msix); } static void tg3_napi_fini(struct tg3 *tp) @@ -12302,9 +12302,9 @@ static void tg3_get_drvinfo(struct net_device *dev, struct ethtool_drvinfo *info { struct tg3 *tp = netdev_priv(dev); - strlcpy(info->driver, DRV_MODULE_NAME, sizeof(info->driver)); - strlcpy(info->fw_version, tp->fw_ver, sizeof(info->fw_version)); - strlcpy(info->bus_info, pci_name(tp->pdev), sizeof(info->bus_info)); + strscpy(info->driver, DRV_MODULE_NAME, sizeof(info->driver)); + strscpy(info->fw_version, tp->fw_ver, sizeof(info->fw_version)); + strscpy(info->bus_info, pci_name(tp->pdev), sizeof(info->bus_info)); } static void tg3_get_wol(struct net_device *dev, struct ethtool_wolinfo *wol) diff --git a/drivers/net/ethernet/brocade/bna/bnad.c b/drivers/net/ethernet/brocade/bna/bnad.c index 29dd0f93d6c0..d6d90f9722a7 100644 --- a/drivers/net/ethernet/brocade/bna/bnad.c +++ b/drivers/net/ethernet/brocade/bna/bnad.c @@ -1891,7 +1891,7 @@ bnad_napi_add(struct bnad *bnad, u32 rx_id) for (i = 0; i < bnad->num_rxp_per_rx; i++) { rx_ctrl = &bnad->rx_info[rx_id].rx_ctrl[i]; netif_napi_add(bnad->netdev, &rx_ctrl->napi, - bnad_napi_poll_rx, NAPI_POLL_WEIGHT); + bnad_napi_poll_rx); } } diff --git a/drivers/net/ethernet/brocade/bna/bnad_ethtool.c b/drivers/net/ethernet/brocade/bna/bnad_ethtool.c index 8aca768571b2..df10edff5603 100644 --- a/drivers/net/ethernet/brocade/bna/bnad_ethtool.c +++ b/drivers/net/ethernet/brocade/bna/bnad_ethtool.c @@ -114,7 +114,7 @@ static const char *bnad_net_stats_strings[] = { "mac_tx_deferral", "mac_tx_excessive_deferral", "mac_tx_single_collision", - "mac_tx_muliple_collision", + "mac_tx_multiple_collision", "mac_tx_late_collision", "mac_tx_excessive_collision", "mac_tx_total_collision", @@ -283,7 +283,7 @@ bnad_get_drvinfo(struct net_device *netdev, struct ethtool_drvinfo *drvinfo) struct bfa_ioc_attr *ioc_attr; unsigned long flags; - strlcpy(drvinfo->driver, BNAD_NAME, sizeof(drvinfo->driver)); + strscpy(drvinfo->driver, BNAD_NAME, sizeof(drvinfo->driver)); ioc_attr = kzalloc(sizeof(*ioc_attr), GFP_KERNEL); if (ioc_attr) { @@ -291,12 +291,12 @@ bnad_get_drvinfo(struct net_device *netdev, struct ethtool_drvinfo *drvinfo) bfa_nw_ioc_get_attr(&bnad->bna.ioceth.ioc, ioc_attr); spin_unlock_irqrestore(&bnad->bna_lock, flags); - strlcpy(drvinfo->fw_version, ioc_attr->adapter_attr.fw_ver, + strscpy(drvinfo->fw_version, ioc_attr->adapter_attr.fw_ver, sizeof(drvinfo->fw_version)); kfree(ioc_attr); } - strlcpy(drvinfo->bus_info, pci_name(bnad->pcidev), + strscpy(drvinfo->bus_info, pci_name(bnad->pcidev), sizeof(drvinfo->bus_info)); } diff --git a/drivers/net/ethernet/cadence/macb_main.c b/drivers/net/ethernet/cadence/macb_main.c index a2897549f9c4..51c9fd6f68a4 100644 --- a/drivers/net/ethernet/cadence/macb_main.c +++ b/drivers/net/ethernet/cadence/macb_main.c @@ -38,6 +38,7 @@ #include <linux/pm_runtime.h> #include <linux/ptp_classify.h> #include <linux/reset.h> +#include <linux/firmware/xlnx-zynqmp.h> #include "macb.h" /* This structure is only used for MACB on SiFive FU540 devices */ @@ -3977,8 +3978,8 @@ static int macb_init(struct platform_device *pdev) queue = &bp->queues[q]; queue->bp = bp; spin_lock_init(&queue->tx_ptr_lock); - netif_napi_add(dev, &queue->napi_rx, macb_rx_poll, NAPI_POLL_WEIGHT); - netif_napi_add(dev, &queue->napi_tx, macb_tx_poll, NAPI_POLL_WEIGHT); + netif_napi_add(dev, &queue->napi_rx, macb_rx_poll); + netif_napi_add(dev, &queue->napi_tx, macb_tx_poll); if (hw_q) { queue->ISR = GEM_ISR(hw_q - 1); queue->IER = GEM_IER(hw_q - 1); @@ -4621,6 +4622,25 @@ static int init_reset_optional(struct platform_device *pdev) "failed to init SGMII PHY\n"); } + ret = zynqmp_pm_is_function_supported(PM_IOCTL, IOCTL_SET_GEM_CONFIG); + if (!ret) { + u32 pm_info[2]; + + ret = of_property_read_u32_array(pdev->dev.of_node, "power-domains", + pm_info, ARRAY_SIZE(pm_info)); + if (ret) { + dev_err(&pdev->dev, "Failed to read power management information\n"); + goto err_out_phy_exit; + } + ret = zynqmp_pm_set_gem_config(pm_info[1], GEM_CONFIG_FIXED, 0); + if (ret) + goto err_out_phy_exit; + + ret = zynqmp_pm_set_gem_config(pm_info[1], GEM_CONFIG_SGMII_MODE, 1); + if (ret) + goto err_out_phy_exit; + } + /* Fully reset controller at hardware level if mapped in device tree */ ret = device_reset_optional(&pdev->dev); if (ret) { @@ -4629,6 +4649,8 @@ static int init_reset_optional(struct platform_device *pdev) } ret = macb_init(pdev); + +err_out_phy_exit: if (ret) phy_exit(bp->sgmii_phy); diff --git a/drivers/net/ethernet/calxeda/xgmac.c b/drivers/net/ethernet/calxeda/xgmac.c index 1281d1565ef8..f4f87dfa9687 100644 --- a/drivers/net/ethernet/calxeda/xgmac.c +++ b/drivers/net/ethernet/calxeda/xgmac.c @@ -1792,7 +1792,7 @@ static int xgmac_probe(struct platform_device *pdev) netdev_warn(ndev, "MAC address %pM not valid", ndev->dev_addr); - netif_napi_add(ndev, &priv->napi, xgmac_poll, 64); + netif_napi_add(ndev, &priv->napi, xgmac_poll); ret = register_netdev(ndev); if (ret) goto err_reg; diff --git a/drivers/net/ethernet/cavium/liquidio/cn23xx_pf_regs.h b/drivers/net/ethernet/cavium/liquidio/cn23xx_pf_regs.h index 3f1c189646f4..a0fd32476225 100644 --- a/drivers/net/ethernet/cavium/liquidio/cn23xx_pf_regs.h +++ b/drivers/net/ethernet/cavium/liquidio/cn23xx_pf_regs.h @@ -87,8 +87,8 @@ */ #define CN23XX_SLI_PKT_IN_JABBER 0x29170 /* The input jabber is used to determine the TSO max size. - * Due to H/W limitation, this need to be reduced to 60000 - * in order to to H/W TSO and avoid the WQE malfarmation + * Due to H/W limitation, this needs to be reduced to 60000 + * in order to use H/W TSO and avoid the WQE malformation * PKO_BUG_24989_WQE_LEN */ #define CN23XX_DEFAULT_INPUT_JABBER 0xEA60 /*60000*/ diff --git a/drivers/net/ethernet/cavium/liquidio/cn23xx_vf_regs.h b/drivers/net/ethernet/cavium/liquidio/cn23xx_vf_regs.h index d33dd8f4226f..e956109415cd 100644 --- a/drivers/net/ethernet/cavium/liquidio/cn23xx_vf_regs.h +++ b/drivers/net/ethernet/cavium/liquidio/cn23xx_vf_regs.h @@ -36,8 +36,8 @@ #define CN23XX_CONFIG_PCIE_FLTMSK 0x720 /* The input jabber is used to determine the TSO max size. - * Due to H/W limitation, this need to be reduced to 60000 - * in order to to H/W TSO and avoid the WQE malfarmation + * Due to H/W limitation, this needs to be reduced to 60000 + * in order to use H/W TSO and avoid the WQE malformation * PKO_BUG_24989_WQE_LEN */ #define CN23XX_DEFAULT_INPUT_JABBER 0xEA60 /*60000*/ diff --git a/drivers/net/ethernet/cavium/liquidio/lio_core.c b/drivers/net/ethernet/cavium/liquidio/lio_core.c index 73cb03266549..882b2be06ea0 100644 --- a/drivers/net/ethernet/cavium/liquidio/lio_core.c +++ b/drivers/net/ethernet/cavium/liquidio/lio_core.c @@ -851,7 +851,7 @@ int liquidio_setup_io_queues(struct octeon_device *octeon_dev, int ifidx, napi = &droq->napi; dev_dbg(&octeon_dev->pci_dev->dev, "netif_napi_add netdev:%llx oct:%llx\n", (u64)netdev, (u64)octeon_dev); - netif_napi_add(netdev, napi, liquidio_napi_poll, 64); + netif_napi_add(netdev, napi, liquidio_napi_poll); /* designate a CPU for this droq */ droq->cpu_id = cpu_id; diff --git a/drivers/net/ethernet/cavium/liquidio/lio_main.c b/drivers/net/ethernet/cavium/liquidio/lio_main.c index bee35ce60171..d312bd594935 100644 --- a/drivers/net/ethernet/cavium/liquidio/lio_main.c +++ b/drivers/net/ethernet/cavium/liquidio/lio_main.c @@ -92,11 +92,6 @@ static int octeon_console_debug_enabled(u32 console) /* time to wait for possible in-flight requests in milliseconds */ #define WAIT_INFLIGHT_REQUEST msecs_to_jiffies(1000) -struct lio_trusted_vf_ctx { - struct completion complete; - int status; -}; - struct oct_link_status_resp { u64 rh; struct oct_link_info link_info; diff --git a/drivers/net/ethernet/cavium/octeon/octeon_mgmt.c b/drivers/net/ethernet/cavium/octeon/octeon_mgmt.c index 103591dcea1c..edde0b8fa49c 100644 --- a/drivers/net/ethernet/cavium/octeon/octeon_mgmt.c +++ b/drivers/net/ethernet/cavium/octeon/octeon_mgmt.c @@ -1342,7 +1342,7 @@ static void octeon_mgmt_poll_controller(struct net_device *netdev) static void octeon_mgmt_get_drvinfo(struct net_device *netdev, struct ethtool_drvinfo *info) { - strlcpy(info->driver, DRV_NAME, sizeof(info->driver)); + strscpy(info->driver, DRV_NAME, sizeof(info->driver)); } static int octeon_mgmt_nway_reset(struct net_device *dev) @@ -1396,8 +1396,8 @@ static int octeon_mgmt_probe(struct platform_device *pdev) platform_set_drvdata(pdev, netdev); p = netdev_priv(netdev); - netif_napi_add(netdev, &p->napi, octeon_mgmt_napi_poll, - OCTEON_MGMT_NAPI_WEIGHT); + netif_napi_add_weight(netdev, &p->napi, octeon_mgmt_napi_poll, + OCTEON_MGMT_NAPI_WEIGHT); p->netdev = netdev; p->dev = &pdev->dev; diff --git a/drivers/net/ethernet/cavium/thunder/nicvf_ethtool.c b/drivers/net/ethernet/cavium/thunder/nicvf_ethtool.c index 5a9fad61e9ea..e5c71f907852 100644 --- a/drivers/net/ethernet/cavium/thunder/nicvf_ethtool.c +++ b/drivers/net/ethernet/cavium/thunder/nicvf_ethtool.c @@ -191,8 +191,8 @@ static void nicvf_get_drvinfo(struct net_device *netdev, { struct nicvf *nic = netdev_priv(netdev); - strlcpy(info->driver, DRV_NAME, sizeof(info->driver)); - strlcpy(info->bus_info, pci_name(nic->pdev), sizeof(info->bus_info)); + strscpy(info->driver, DRV_NAME, sizeof(info->driver)); + strscpy(info->bus_info, pci_name(nic->pdev), sizeof(info->bus_info)); } static u32 nicvf_get_msglevel(struct net_device *netdev) diff --git a/drivers/net/ethernet/cavium/thunder/nicvf_main.c b/drivers/net/ethernet/cavium/thunder/nicvf_main.c index 768ea426d49f..98f3dc460ca7 100644 --- a/drivers/net/ethernet/cavium/thunder/nicvf_main.c +++ b/drivers/net/ethernet/cavium/thunder/nicvf_main.c @@ -1472,8 +1472,7 @@ int nicvf_open(struct net_device *netdev) } cq_poll->cq_idx = qidx; cq_poll->nicvf = nic; - netif_napi_add(netdev, &cq_poll->napi, nicvf_poll, - NAPI_POLL_WEIGHT); + netif_napi_add(netdev, &cq_poll->napi, nicvf_poll); napi_enable(&cq_poll->napi); nic->napi[qidx] = cq_poll; } diff --git a/drivers/net/ethernet/chelsio/cxgb/cxgb2.c b/drivers/net/ethernet/chelsio/cxgb/cxgb2.c index f4054d2553ea..d2286adf09fe 100644 --- a/drivers/net/ethernet/chelsio/cxgb/cxgb2.c +++ b/drivers/net/ethernet/chelsio/cxgb/cxgb2.c @@ -429,8 +429,8 @@ static void get_drvinfo(struct net_device *dev, struct ethtool_drvinfo *info) { struct adapter *adapter = dev->ml_priv; - strlcpy(info->driver, DRV_NAME, sizeof(info->driver)); - strlcpy(info->bus_info, pci_name(adapter->pdev), + strscpy(info->driver, DRV_NAME, sizeof(info->driver)); + strscpy(info->bus_info, pci_name(adapter->pdev), sizeof(info->bus_info)); } @@ -1053,7 +1053,7 @@ static int init_one(struct pci_dev *pdev, const struct pci_device_id *ent) netdev->hard_header_len += (netdev->hw_features & NETIF_F_TSO) ? sizeof(struct cpl_tx_pkt_lso) : sizeof(struct cpl_tx_pkt); - netif_napi_add(netdev, &adapter->napi, t1_poll, 64); + netif_napi_add(netdev, &adapter->napi, t1_poll); netdev->ethtool_ops = &t1_ethtool_ops; diff --git a/drivers/net/ethernet/chelsio/cxgb3/cxgb3_main.c b/drivers/net/ethernet/chelsio/cxgb3/cxgb3_main.c index 174b1e156669..a52e6b6e2876 100644 --- a/drivers/net/ethernet/chelsio/cxgb3/cxgb3_main.c +++ b/drivers/net/ethernet/chelsio/cxgb3/cxgb3_main.c @@ -609,8 +609,7 @@ static void init_napi(struct adapter *adap) struct sge_qset *qs = &adap->sge.qs[i]; if (qs->adap) - netif_napi_add(qs->netdev, &qs->napi, qs->napi.poll, - 64); + netif_napi_add(qs->netdev, &qs->napi, qs->napi.poll); } /* @@ -1627,8 +1626,8 @@ static void get_drvinfo(struct net_device *dev, struct ethtool_drvinfo *info) t3_get_tp_version(adapter, &tp_vers); spin_unlock(&adapter->stats_lock); - strlcpy(info->driver, DRV_NAME, sizeof(info->driver)); - strlcpy(info->bus_info, pci_name(adapter->pdev), + strscpy(info->driver, DRV_NAME, sizeof(info->driver)); + strscpy(info->bus_info, pci_name(adapter->pdev), sizeof(info->bus_info)); if (fw_vers) snprintf(info->fw_version, sizeof(info->fw_version), diff --git a/drivers/net/ethernet/chelsio/cxgb4/cxgb4_ethtool.c b/drivers/net/ethernet/chelsio/cxgb4/cxgb4_ethtool.c index 77897edd2bc0..8477a93cee6b 100644 --- a/drivers/net/ethernet/chelsio/cxgb4/cxgb4_ethtool.c +++ b/drivers/net/ethernet/chelsio/cxgb4/cxgb4_ethtool.c @@ -199,8 +199,8 @@ static void get_drvinfo(struct net_device *dev, struct ethtool_drvinfo *info) struct adapter *adapter = netdev2adap(dev); u32 exprom_vers; - strlcpy(info->driver, cxgb4_driver_name, sizeof(info->driver)); - strlcpy(info->bus_info, pci_name(adapter->pdev), + strscpy(info->driver, cxgb4_driver_name, sizeof(info->driver)); + strscpy(info->bus_info, pci_name(adapter->pdev), sizeof(info->bus_info)); info->regdump_len = get_regs_len(dev); diff --git a/drivers/net/ethernet/chelsio/cxgb4/cxgb4_main.c b/drivers/net/ethernet/chelsio/cxgb4/cxgb4_main.c index d0061921529f..9cbce1faab26 100644 --- a/drivers/net/ethernet/chelsio/cxgb4/cxgb4_main.c +++ b/drivers/net/ethernet/chelsio/cxgb4/cxgb4_main.c @@ -3903,8 +3903,8 @@ static void cxgb4_mgmt_get_drvinfo(struct net_device *dev, { struct adapter *adapter = netdev2adap(dev); - strlcpy(info->driver, cxgb4_driver_name, sizeof(info->driver)); - strlcpy(info->bus_info, pci_name(adapter->pdev), + strscpy(info->driver, cxgb4_driver_name, sizeof(info->driver)); + strscpy(info->bus_info, pci_name(adapter->pdev), sizeof(info->bus_info)); } diff --git a/drivers/net/ethernet/chelsio/cxgb4/sge.c b/drivers/net/ethernet/chelsio/cxgb4/sge.c index ee52e3b1d74f..46809e2d94ee 100644 --- a/drivers/net/ethernet/chelsio/cxgb4/sge.c +++ b/drivers/net/ethernet/chelsio/cxgb4/sge.c @@ -4467,7 +4467,7 @@ int t4_sge_alloc_rxq(struct adapter *adap, struct sge_rspq *iq, bool fwevtq, if (ret) goto err; - netif_napi_add(dev, &iq->napi, napi_rx_handler, 64); + netif_napi_add(dev, &iq->napi, napi_rx_handler); iq->cur_desc = iq->desc; iq->cidx = 0; iq->gen = 1; diff --git a/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf_main.c b/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf_main.c index c2822e635f89..54db79f4dcfe 100644 --- a/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf_main.c +++ b/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf_main.c @@ -1553,8 +1553,8 @@ static void cxgb4vf_get_drvinfo(struct net_device *dev, { struct adapter *adapter = netdev2adap(dev); - strlcpy(drvinfo->driver, KBUILD_MODNAME, sizeof(drvinfo->driver)); - strlcpy(drvinfo->bus_info, pci_name(to_pci_dev(dev->dev.parent)), + strscpy(drvinfo->driver, KBUILD_MODNAME, sizeof(drvinfo->driver)); + strscpy(drvinfo->bus_info, pci_name(to_pci_dev(dev->dev.parent)), sizeof(drvinfo->bus_info)); snprintf(drvinfo->fw_version, sizeof(drvinfo->fw_version), "%u.%u.%u.%u, TP %u.%u.%u.%u", diff --git a/drivers/net/ethernet/chelsio/cxgb4vf/sge.c b/drivers/net/ethernet/chelsio/cxgb4vf/sge.c index 43b2ceb6aa32..2d0cf76fb3c5 100644 --- a/drivers/net/ethernet/chelsio/cxgb4vf/sge.c +++ b/drivers/net/ethernet/chelsio/cxgb4vf/sge.c @@ -2336,7 +2336,7 @@ int t4vf_sge_alloc_rxq(struct adapter *adapter, struct sge_rspq *rspq, if (ret) goto err; - netif_napi_add(dev, &rspq->napi, napi_rx_handler, 64); + netif_napi_add(dev, &rspq->napi, napi_rx_handler); rspq->cur_desc = rspq->desc; rspq->cidx = 0; rspq->gen = 1; diff --git a/drivers/net/ethernet/chelsio/inline_crypto/chtls/chtls_cm.c b/drivers/net/ethernet/chelsio/inline_crypto/chtls/chtls_cm.c index ddfe9208529a..f90bfba4b303 100644 --- a/drivers/net/ethernet/chelsio/inline_crypto/chtls/chtls_cm.c +++ b/drivers/net/ethernet/chelsio/inline_crypto/chtls/chtls_cm.c @@ -1069,8 +1069,7 @@ static void chtls_pass_accept_rpl(struct sk_buff *skb, cxgb4_l2t_send(csk->egress_dev, skb, csk->l2t_entry); } -static void inet_inherit_port(struct inet_hashinfo *hash_info, - struct sock *lsk, struct sock *newsk) +static void inet_inherit_port(struct sock *lsk, struct sock *newsk) { local_bh_disable(); __inet_inherit_port(lsk, newsk); @@ -1240,7 +1239,7 @@ static struct sock *chtls_recv_sock(struct sock *lsk, ipv4.sysctl_tcp_window_scaling), tp->window_clamp); neigh_release(n); - inet_inherit_port(&tcp_hashinfo, lsk, newsk); + inet_inherit_port(lsk, newsk); csk_set_flag(csk, CSK_CONN_INLINE); bh_unlock_sock(newsk); /* tcp_create_openreq_child ->sk_clone_lock */ diff --git a/drivers/net/ethernet/chelsio/inline_crypto/chtls/chtls_main.c b/drivers/net/ethernet/chelsio/inline_crypto/chtls/chtls_main.c index 9098b3eed4da..1e55b12fee51 100644 --- a/drivers/net/ethernet/chelsio/inline_crypto/chtls/chtls_main.c +++ b/drivers/net/ethernet/chelsio/inline_crypto/chtls/chtls_main.c @@ -193,7 +193,7 @@ static void chtls_register_dev(struct chtls_dev *cdev) { struct tls_toe_device *tlsdev = &cdev->tlsdev; - strlcpy(tlsdev->name, "chtls", TLS_TOE_DEVICE_NAME_MAX); + strscpy(tlsdev->name, "chtls", TLS_TOE_DEVICE_NAME_MAX); strlcat(tlsdev->name, cdev->lldi->ports[0]->name, TLS_TOE_DEVICE_NAME_MAX); tlsdev->feature = chtls_inline_feature; diff --git a/drivers/net/ethernet/cirrus/ep93xx_eth.c b/drivers/net/ethernet/cirrus/ep93xx_eth.c index 21ba6e893072..8627ab19d470 100644 --- a/drivers/net/ethernet/cirrus/ep93xx_eth.c +++ b/drivers/net/ethernet/cirrus/ep93xx_eth.c @@ -689,7 +689,7 @@ static int ep93xx_ioctl(struct net_device *dev, struct ifreq *ifr, int cmd) static void ep93xx_get_drvinfo(struct net_device *dev, struct ethtool_drvinfo *info) { - strlcpy(info->driver, DRV_MODULE_NAME, sizeof(info->driver)); + strscpy(info->driver, DRV_MODULE_NAME, sizeof(info->driver)); } static int ep93xx_get_link_ksettings(struct net_device *dev, @@ -812,7 +812,7 @@ static int ep93xx_eth_probe(struct platform_device *pdev) ep = netdev_priv(dev); ep->dev = dev; SET_NETDEV_DEV(dev, &pdev->dev); - netif_napi_add(dev, &ep->napi, ep93xx_poll, 64); + netif_napi_add(dev, &ep->napi, ep93xx_poll); platform_set_drvdata(pdev, dev); diff --git a/drivers/net/ethernet/cisco/enic/enic_ethtool.c b/drivers/net/ethernet/cisco/enic/enic_ethtool.c index 60d8c0fbc037..08b7cc0a1809 100644 --- a/drivers/net/ethernet/cisco/enic/enic_ethtool.c +++ b/drivers/net/ethernet/cisco/enic/enic_ethtool.c @@ -131,10 +131,10 @@ static void enic_get_drvinfo(struct net_device *netdev, if (err == -ENOMEM) return; - strlcpy(drvinfo->driver, DRV_NAME, sizeof(drvinfo->driver)); - strlcpy(drvinfo->fw_version, fw_info->fw_version, + strscpy(drvinfo->driver, DRV_NAME, sizeof(drvinfo->driver)); + strscpy(drvinfo->fw_version, fw_info->fw_version, sizeof(drvinfo->fw_version)); - strlcpy(drvinfo->bus_info, pci_name(enic->pdev), + strscpy(drvinfo->bus_info, pci_name(enic->pdev), sizeof(drvinfo->bus_info)); } diff --git a/drivers/net/ethernet/cisco/enic/enic_main.c b/drivers/net/ethernet/cisco/enic/enic_main.c index 372fb7b3a282..29500d32e362 100644 --- a/drivers/net/ethernet/cisco/enic/enic_main.c +++ b/drivers/net/ethernet/cisco/enic/enic_main.c @@ -2633,16 +2633,17 @@ static int enic_dev_init(struct enic *enic) switch (vnic_dev_get_intr_mode(enic->vdev)) { default: - netif_napi_add(netdev, &enic->napi[0], enic_poll, 64); + netif_napi_add(netdev, &enic->napi[0], enic_poll); break; case VNIC_DEV_INTR_MODE_MSIX: for (i = 0; i < enic->rq_count; i++) { netif_napi_add(netdev, &enic->napi[i], - enic_poll_msix_rq, NAPI_POLL_WEIGHT); + enic_poll_msix_rq); } for (i = 0; i < enic->wq_count; i++) - netif_napi_add(netdev, &enic->napi[enic_cq_wq(enic, i)], - enic_poll_msix_wq, NAPI_POLL_WEIGHT); + netif_napi_add(netdev, + &enic->napi[enic_cq_wq(enic, i)], + enic_poll_msix_wq); break; } diff --git a/drivers/net/ethernet/cortina/gemini.c b/drivers/net/ethernet/cortina/gemini.c index 6dae768671e3..fdf10318758b 100644 --- a/drivers/net/ethernet/cortina/gemini.c +++ b/drivers/net/ethernet/cortina/gemini.c @@ -2471,7 +2471,7 @@ static int gemini_ethernet_port_probe(struct platform_device *pdev) netdev->max_mtu = 10236 - VLAN_ETH_HLEN; port->freeq_refill = 0; - netif_napi_add(netdev, &port->napi, gmac_napi_poll, NAPI_POLL_WEIGHT); + netif_napi_add(netdev, &port->napi, gmac_napi_poll); ret = of_get_mac_address(np, mac); if (!ret) { diff --git a/drivers/net/ethernet/davicom/dm9000.c b/drivers/net/ethernet/davicom/dm9000.c index 0985ab216566..b21e56de6167 100644 --- a/drivers/net/ethernet/davicom/dm9000.c +++ b/drivers/net/ethernet/davicom/dm9000.c @@ -28,8 +28,7 @@ #include <linux/irq.h> #include <linux/slab.h> #include <linux/regulator/consumer.h> -#include <linux/gpio.h> -#include <linux/of_gpio.h> +#include <linux/gpio/consumer.h> #include <asm/delay.h> #include <asm/irq.h> @@ -540,8 +539,8 @@ static void dm9000_get_drvinfo(struct net_device *dev, { struct board_info *dm = to_dm9000_board(dev); - strlcpy(info->driver, CARDNAME, sizeof(info->driver)); - strlcpy(info->bus_info, to_platform_device(dm->dev)->name, + strscpy(info->driver, CARDNAME, sizeof(info->driver)); + strscpy(info->bus_info, to_platform_device(dm->dev)->name, sizeof(info->bus_info)); } @@ -1012,7 +1011,7 @@ static void dm9000_send_packet(struct net_device *dev, * Hardware start transmission. * Send a packet to media from the upper layer. */ -static int +static netdev_tx_t dm9000_start_xmit(struct sk_buff *skb, struct net_device *dev) { unsigned long flags; @@ -1421,8 +1420,7 @@ dm9000_probe(struct platform_device *pdev) int iosize; int i; u32 id_val; - int reset_gpios; - enum of_gpio_flags flags; + struct gpio_desc *reset_gpio; struct regulator *power; bool inv_mac_addr = false; u8 addr[ETH_ALEN]; @@ -1442,20 +1440,24 @@ dm9000_probe(struct platform_device *pdev) dev_dbg(dev, "regulator enabled\n"); } - reset_gpios = of_get_named_gpio_flags(dev->of_node, "reset-gpios", 0, - &flags); - if (gpio_is_valid(reset_gpios)) { - ret = devm_gpio_request_one(dev, reset_gpios, flags, - "dm9000_reset"); + reset_gpio = devm_gpiod_get_optional(dev, "reset", GPIOD_OUT_HIGH); + ret = PTR_ERR_OR_ZERO(reset_gpio); + if (ret) { + dev_err(dev, "failed to request reset gpio: %d\n", ret); + goto out_regulator_disable; + } + + if (reset_gpio) { + ret = gpiod_set_consumer_name(reset_gpio, "dm9000_reset"); if (ret) { - dev_err(dev, "failed to request reset gpio %d: %d\n", - reset_gpios, ret); + dev_err(dev, "failed to set reset gpio name: %d\n", + ret); goto out_regulator_disable; } /* According to manual PWRST# Low Period Min 1ms */ msleep(2); - gpio_set_value(reset_gpios, 1); + gpiod_set_value_cansleep(reset_gpio, 0); /* Needs 3ms to read eeprom when PWRST is deasserted */ msleep(4); } diff --git a/drivers/net/ethernet/dec/tulip/de2104x.c b/drivers/net/ethernet/dec/tulip/de2104x.c index d51b3d24a0c8..cd3dc4b89518 100644 --- a/drivers/net/ethernet/dec/tulip/de2104x.c +++ b/drivers/net/ethernet/dec/tulip/de2104x.c @@ -1606,8 +1606,8 @@ static void de_get_drvinfo (struct net_device *dev,struct ethtool_drvinfo *info) { struct de_private *de = netdev_priv(dev); - strlcpy(info->driver, DRV_NAME, sizeof(info->driver)); - strlcpy(info->bus_info, pci_name(de->pdev), sizeof(info->bus_info)); + strscpy(info->driver, DRV_NAME, sizeof(info->driver)); + strscpy(info->bus_info, pci_name(de->pdev), sizeof(info->bus_info)); } static int de_get_regs_len(struct net_device *dev) diff --git a/drivers/net/ethernet/dec/tulip/dmfe.c b/drivers/net/ethernet/dec/tulip/dmfe.c index 83f1727d1423..3188ba7b450f 100644 --- a/drivers/net/ethernet/dec/tulip/dmfe.c +++ b/drivers/net/ethernet/dec/tulip/dmfe.c @@ -1074,8 +1074,8 @@ static void dmfe_ethtool_get_drvinfo(struct net_device *dev, { struct dmfe_board_info *np = netdev_priv(dev); - strlcpy(info->driver, DRV_NAME, sizeof(info->driver)); - strlcpy(info->bus_info, pci_name(np->pdev), sizeof(info->bus_info)); + strscpy(info->driver, DRV_NAME, sizeof(info->driver)); + strscpy(info->bus_info, pci_name(np->pdev), sizeof(info->bus_info)); } static int dmfe_ethtool_set_wol(struct net_device *dev, diff --git a/drivers/net/ethernet/dec/tulip/tulip_core.c b/drivers/net/ethernet/dec/tulip/tulip_core.c index b8e46c4849ef..ecfad43df45a 100644 --- a/drivers/net/ethernet/dec/tulip/tulip_core.c +++ b/drivers/net/ethernet/dec/tulip/tulip_core.c @@ -858,8 +858,8 @@ static struct net_device_stats *tulip_get_stats(struct net_device *dev) static void tulip_get_drvinfo(struct net_device *dev, struct ethtool_drvinfo *info) { struct tulip_private *np = netdev_priv(dev); - strlcpy(info->driver, DRV_NAME, sizeof(info->driver)); - strlcpy(info->bus_info, pci_name(np->pdev), sizeof(info->bus_info)); + strscpy(info->driver, DRV_NAME, sizeof(info->driver)); + strscpy(info->bus_info, pci_name(np->pdev), sizeof(info->bus_info)); } diff --git a/drivers/net/ethernet/dec/tulip/uli526x.c b/drivers/net/ethernet/dec/tulip/uli526x.c index 77d9058431e3..ff080ab0f116 100644 --- a/drivers/net/ethernet/dec/tulip/uli526x.c +++ b/drivers/net/ethernet/dec/tulip/uli526x.c @@ -971,8 +971,8 @@ static void netdev_get_drvinfo(struct net_device *dev, { struct uli526x_board_info *np = netdev_priv(dev); - strlcpy(info->driver, DRV_NAME, sizeof(info->driver)); - strlcpy(info->bus_info, pci_name(np->pdev), sizeof(info->bus_info)); + strscpy(info->driver, DRV_NAME, sizeof(info->driver)); + strscpy(info->bus_info, pci_name(np->pdev), sizeof(info->bus_info)); } static int netdev_get_link_ksettings(struct net_device *dev, diff --git a/drivers/net/ethernet/dec/tulip/winbond-840.c b/drivers/net/ethernet/dec/tulip/winbond-840.c index 1db19463fd46..37fba39c0056 100644 --- a/drivers/net/ethernet/dec/tulip/winbond-840.c +++ b/drivers/net/ethernet/dec/tulip/winbond-840.c @@ -1374,8 +1374,8 @@ static void netdev_get_drvinfo (struct net_device *dev, struct ethtool_drvinfo * { struct netdev_private *np = netdev_priv(dev); - strlcpy(info->driver, DRV_NAME, sizeof(info->driver)); - strlcpy(info->bus_info, pci_name(np->pci_dev), sizeof(info->bus_info)); + strscpy(info->driver, DRV_NAME, sizeof(info->driver)); + strscpy(info->bus_info, pci_name(np->pci_dev), sizeof(info->bus_info)); } static int netdev_get_link_ksettings(struct net_device *dev, diff --git a/drivers/net/ethernet/dlink/dl2k.c b/drivers/net/ethernet/dlink/dl2k.c index a301f7e6a440..2c67a857a42f 100644 --- a/drivers/net/ethernet/dlink/dl2k.c +++ b/drivers/net/ethernet/dlink/dl2k.c @@ -1235,8 +1235,8 @@ static void rio_get_drvinfo(struct net_device *dev, struct ethtool_drvinfo *info { struct netdev_private *np = netdev_priv(dev); - strlcpy(info->driver, "dl2k", sizeof(info->driver)); - strlcpy(info->bus_info, pci_name(np->pdev), sizeof(info->bus_info)); + strscpy(info->driver, "dl2k", sizeof(info->driver)); + strscpy(info->bus_info, pci_name(np->pdev), sizeof(info->bus_info)); } static int rio_get_link_ksettings(struct net_device *dev, diff --git a/drivers/net/ethernet/dlink/sundance.c b/drivers/net/ethernet/dlink/sundance.c index 8dd7bf9014ec..43def191f26f 100644 --- a/drivers/net/ethernet/dlink/sundance.c +++ b/drivers/net/ethernet/dlink/sundance.c @@ -1644,8 +1644,8 @@ static int check_if_running(struct net_device *dev) static void get_drvinfo(struct net_device *dev, struct ethtool_drvinfo *info) { struct netdev_private *np = netdev_priv(dev); - strlcpy(info->driver, DRV_NAME, sizeof(info->driver)); - strlcpy(info->bus_info, pci_name(np->pci_dev), sizeof(info->bus_info)); + strscpy(info->driver, DRV_NAME, sizeof(info->driver)); + strscpy(info->bus_info, pci_name(np->pci_dev), sizeof(info->bus_info)); } static int get_link_ksettings(struct net_device *dev, diff --git a/drivers/net/ethernet/dnet.c b/drivers/net/ethernet/dnet.c index 92462ed87bc4..08184f20f510 100644 --- a/drivers/net/ethernet/dnet.c +++ b/drivers/net/ethernet/dnet.c @@ -725,8 +725,8 @@ static struct net_device_stats *dnet_get_stats(struct net_device *dev) static void dnet_get_drvinfo(struct net_device *dev, struct ethtool_drvinfo *info) { - strlcpy(info->driver, DRV_NAME, sizeof(info->driver)); - strlcpy(info->bus_info, "0", sizeof(info->bus_info)); + strscpy(info->driver, DRV_NAME, sizeof(info->driver)); + strscpy(info->bus_info, "0", sizeof(info->bus_info)); } static const struct ethtool_ops dnet_ethtool_ops = { @@ -788,7 +788,7 @@ static int dnet_probe(struct platform_device *pdev) } dev->netdev_ops = &dnet_netdev_ops; - netif_napi_add(dev, &bp->napi, dnet_poll, 64); + netif_napi_add(dev, &bp->napi, dnet_poll); dev->ethtool_ops = &dnet_ethtool_ops; dev->base_addr = (unsigned long)bp->regs; diff --git a/drivers/net/ethernet/emulex/benet/be_cmds.c b/drivers/net/ethernet/emulex/benet/be_cmds.c index b4f5e57d0285..08ec84cd21c0 100644 --- a/drivers/net/ethernet/emulex/benet/be_cmds.c +++ b/drivers/net/ethernet/emulex/benet/be_cmds.c @@ -1878,9 +1878,9 @@ int be_cmd_get_fw_ver(struct be_adapter *adapter) if (!status) { struct be_cmd_resp_get_fw_version *resp = embedded_payload(wrb); - strlcpy(adapter->fw_ver, resp->firmware_version_string, + strscpy(adapter->fw_ver, resp->firmware_version_string, sizeof(adapter->fw_ver)); - strlcpy(adapter->fw_on_flash, resp->fw_on_flash_version_string, + strscpy(adapter->fw_on_flash, resp->fw_on_flash_version_string, sizeof(adapter->fw_on_flash)); } err: @@ -2373,7 +2373,7 @@ static int lancer_cmd_write_object(struct be_adapter *adapter, be_dws_cpu_to_le(ctxt, sizeof(req->context)); req->write_offset = cpu_to_le32(data_offset); - strlcpy(req->object_name, obj_name, sizeof(req->object_name)); + strscpy(req->object_name, obj_name, sizeof(req->object_name)); req->descriptor_count = cpu_to_le32(1); req->buf_len = cpu_to_le32(data_size); req->addr_low = cpu_to_le32((cmd->dma + @@ -2442,9 +2442,9 @@ int be_cmd_query_sfp_info(struct be_adapter *adapter) status = be_cmd_read_port_transceiver_data(adapter, TR_PAGE_A0, 0, PAGE_DATA_LEN, page_data); if (!status) { - strlcpy(adapter->phy.vendor_name, page_data + + strscpy(adapter->phy.vendor_name, page_data + SFP_VENDOR_NAME_OFFSET, SFP_VENDOR_NAME_LEN - 1); - strlcpy(adapter->phy.vendor_pn, + strscpy(adapter->phy.vendor_pn, page_data + SFP_VENDOR_PN_OFFSET, SFP_VENDOR_NAME_LEN - 1); } @@ -2473,7 +2473,7 @@ static int lancer_cmd_delete_object(struct be_adapter *adapter, OPCODE_COMMON_DELETE_OBJECT, sizeof(*req), wrb, NULL); - strlcpy(req->object_name, obj_name, sizeof(req->object_name)); + strscpy(req->object_name, obj_name, sizeof(req->object_name)); status = be_mcc_notify_wait(adapter); err: diff --git a/drivers/net/ethernet/emulex/benet/be_ethtool.c b/drivers/net/ethernet/emulex/benet/be_ethtool.c index bd0df189d871..77edc3d9b505 100644 --- a/drivers/net/ethernet/emulex/benet/be_ethtool.c +++ b/drivers/net/ethernet/emulex/benet/be_ethtool.c @@ -220,15 +220,15 @@ static void be_get_drvinfo(struct net_device *netdev, { struct be_adapter *adapter = netdev_priv(netdev); - strlcpy(drvinfo->driver, DRV_NAME, sizeof(drvinfo->driver)); + strscpy(drvinfo->driver, DRV_NAME, sizeof(drvinfo->driver)); if (!memcmp(adapter->fw_ver, adapter->fw_on_flash, FW_VER_LEN)) - strlcpy(drvinfo->fw_version, adapter->fw_ver, + strscpy(drvinfo->fw_version, adapter->fw_ver, sizeof(drvinfo->fw_version)); else snprintf(drvinfo->fw_version, sizeof(drvinfo->fw_version), "%s [%s]", adapter->fw_ver, adapter->fw_on_flash); - strlcpy(drvinfo->bus_info, pci_name(adapter->pdev), + strscpy(drvinfo->bus_info, pci_name(adapter->pdev), sizeof(drvinfo->bus_info)); } diff --git a/drivers/net/ethernet/emulex/benet/be_main.c b/drivers/net/ethernet/emulex/benet/be_main.c index 414362febbb9..a92a74761546 100644 --- a/drivers/net/ethernet/emulex/benet/be_main.c +++ b/drivers/net/ethernet/emulex/benet/be_main.c @@ -2982,8 +2982,7 @@ static int be_evt_queues_create(struct be_adapter *adapter) return -ENOMEM; cpumask_set_cpu(cpumask_local_spread(i, numa_node), eqo->affinity_mask); - netif_napi_add(adapter->netdev, &eqo->napi, be_poll, - NAPI_POLL_WEIGHT); + netif_napi_add(adapter->netdev, &eqo->napi, be_poll); } return 0; } diff --git a/drivers/net/ethernet/engleder/Kconfig b/drivers/net/ethernet/engleder/Kconfig index f4e2b1102d8f..3df6bf476ae7 100644 --- a/drivers/net/ethernet/engleder/Kconfig +++ b/drivers/net/ethernet/engleder/Kconfig @@ -21,6 +21,7 @@ config TSNEP depends on HAS_IOMEM && HAS_DMA depends on PTP_1588_CLOCK_OPTIONAL select PHYLIB + select PAGE_POOL help Support for the Engleder TSN endpoint Ethernet MAC IP Core. diff --git a/drivers/net/ethernet/engleder/Makefile b/drivers/net/ethernet/engleder/Makefile index cce2191cb889..b6e3b16623de 100644 --- a/drivers/net/ethernet/engleder/Makefile +++ b/drivers/net/ethernet/engleder/Makefile @@ -6,5 +6,5 @@ obj-$(CONFIG_TSNEP) += tsnep.o tsnep-objs := tsnep_main.o tsnep_ethtool.o tsnep_ptp.o tsnep_tc.o \ - $(tsnep-y) + tsnep_rxnfc.o $(tsnep-y) tsnep-$(CONFIG_TSNEP_SELFTESTS) += tsnep_selftests.o diff --git a/drivers/net/ethernet/engleder/tsnep.h b/drivers/net/ethernet/engleder/tsnep.h index 23bbece6b7de..09a723b827c7 100644 --- a/drivers/net/ethernet/engleder/tsnep.h +++ b/drivers/net/ethernet/engleder/tsnep.h @@ -21,8 +21,6 @@ #define TSNEP_RING_ENTRIES_PER_PAGE (PAGE_SIZE / TSNEP_DESC_SIZE) #define TSNEP_RING_PAGE_COUNT (TSNEP_RING_SIZE / TSNEP_RING_ENTRIES_PER_PAGE) -#define TSNEP_QUEUES 1 - struct tsnep_gcl { void __iomem *addr; @@ -39,6 +37,24 @@ struct tsnep_gcl { bool change; }; +enum tsnep_rxnfc_filter_type { + TSNEP_RXNFC_ETHER_TYPE, +}; + +struct tsnep_rxnfc_filter { + enum tsnep_rxnfc_filter_type type; + union { + u16 ether_type; + }; +}; + +struct tsnep_rxnfc_rule { + struct list_head list; + struct tsnep_rxnfc_filter filter; + int queue_index; + int location; +}; + struct tsnep_tx_entry { struct tsnep_tx_desc *desc; struct tsnep_tx_desc_wb *desc_wb; @@ -55,6 +71,7 @@ struct tsnep_tx_entry { struct tsnep_tx { struct tsnep_adapter *adapter; void __iomem *addr; + int queue_index; void *page[TSNEP_RING_PAGE_COUNT]; dma_addr_t page_dma[TSNEP_RING_PAGE_COUNT]; @@ -79,14 +96,15 @@ struct tsnep_rx_entry { u32 properties; - struct sk_buff *skb; + struct page *page; size_t len; - DEFINE_DMA_UNMAP_ADDR(dma); + dma_addr_t dma; }; struct tsnep_rx { struct tsnep_adapter *adapter; void __iomem *addr; + int queue_index; void *page[TSNEP_RING_PAGE_COUNT]; dma_addr_t page_dma[TSNEP_RING_PAGE_COUNT]; @@ -95,6 +113,7 @@ struct tsnep_rx { int read; u32 owner_counter; int increment_owner_counter; + struct page_pool *page_pool; u32 packets; u32 bytes; @@ -104,12 +123,14 @@ struct tsnep_rx { struct tsnep_queue { struct tsnep_adapter *adapter; + char name[IFNAMSIZ + 9]; struct tsnep_tx *tx; struct tsnep_rx *rx; struct napi_struct napi; + int irq; u32 irq_mask; }; @@ -125,7 +146,6 @@ struct tsnep_adapter { struct platform_device *pdev; struct device *dmadev; void __iomem *addr; - int irq; bool gate_control; /* gate control lock */ @@ -140,6 +160,12 @@ struct tsnep_adapter { /* ptp clock lock */ spinlock_t ptp_lock; + /* RX flow classification rules lock */ + struct mutex rxnfc_lock; + struct list_head rxnfc_rules; + int rxnfc_count; + int rxnfc_max; + int num_tx_queues; struct tsnep_tx tx[TSNEP_MAX_QUEUES]; int num_rx_queues; @@ -160,6 +186,18 @@ void tsnep_tc_cleanup(struct tsnep_adapter *adapter); int tsnep_tc_setup(struct net_device *netdev, enum tc_setup_type type, void *type_data); +int tsnep_rxnfc_init(struct tsnep_adapter *adapter); +void tsnep_rxnfc_cleanup(struct tsnep_adapter *adapter); +int tsnep_rxnfc_get_rule(struct tsnep_adapter *adapter, + struct ethtool_rxnfc *cmd); +int tsnep_rxnfc_get_all(struct tsnep_adapter *adapter, + struct ethtool_rxnfc *cmd, + u32 *rule_locs); +int tsnep_rxnfc_add_rule(struct tsnep_adapter *adapter, + struct ethtool_rxnfc *cmd); +int tsnep_rxnfc_del_rule(struct tsnep_adapter *adapter, + struct ethtool_rxnfc *cmd); + #if IS_ENABLED(CONFIG_TSNEP_SELFTESTS) int tsnep_ethtool_get_test_count(void); void tsnep_ethtool_get_test_strings(u8 *data); diff --git a/drivers/net/ethernet/engleder/tsnep_ethtool.c b/drivers/net/ethernet/engleder/tsnep_ethtool.c index e6760dc68ddd..a713a126b227 100644 --- a/drivers/net/ethernet/engleder/tsnep_ethtool.c +++ b/drivers/net/ethernet/engleder/tsnep_ethtool.c @@ -250,6 +250,44 @@ static int tsnep_ethtool_get_sset_count(struct net_device *netdev, int sset) } } +static int tsnep_ethtool_get_rxnfc(struct net_device *dev, + struct ethtool_rxnfc *cmd, u32 *rule_locs) +{ + struct tsnep_adapter *adapter = netdev_priv(dev); + + switch (cmd->cmd) { + case ETHTOOL_GRXRINGS: + cmd->data = adapter->num_rx_queues; + return 0; + case ETHTOOL_GRXCLSRLCNT: + cmd->rule_cnt = adapter->rxnfc_count; + cmd->data = adapter->rxnfc_max; + cmd->data |= RX_CLS_LOC_SPECIAL; + return 0; + case ETHTOOL_GRXCLSRULE: + return tsnep_rxnfc_get_rule(adapter, cmd); + case ETHTOOL_GRXCLSRLALL: + return tsnep_rxnfc_get_all(adapter, cmd, rule_locs); + default: + return -EOPNOTSUPP; + } +} + +static int tsnep_ethtool_set_rxnfc(struct net_device *dev, + struct ethtool_rxnfc *cmd) +{ + struct tsnep_adapter *adapter = netdev_priv(dev); + + switch (cmd->cmd) { + case ETHTOOL_SRXCLSRLINS: + return tsnep_rxnfc_add_rule(adapter, cmd); + case ETHTOOL_SRXCLSRLDEL: + return tsnep_rxnfc_del_rule(adapter, cmd); + default: + return -EOPNOTSUPP; + } +} + static int tsnep_ethtool_get_ts_info(struct net_device *dev, struct ethtool_ts_info *info) { @@ -287,6 +325,8 @@ const struct ethtool_ops tsnep_ethtool_ops = { .get_strings = tsnep_ethtool_get_strings, .get_ethtool_stats = tsnep_ethtool_get_ethtool_stats, .get_sset_count = tsnep_ethtool_get_sset_count, + .get_rxnfc = tsnep_ethtool_get_rxnfc, + .set_rxnfc = tsnep_ethtool_set_rxnfc, .get_ts_info = tsnep_ethtool_get_ts_info, .get_link_ksettings = phy_ethtool_get_link_ksettings, .set_link_ksettings = phy_ethtool_set_link_ksettings, diff --git a/drivers/net/ethernet/engleder/tsnep_hw.h b/drivers/net/ethernet/engleder/tsnep_hw.h index 916ceac3ada2..315dada75323 100644 --- a/drivers/net/ethernet/engleder/tsnep_hw.h +++ b/drivers/net/ethernet/engleder/tsnep_hw.h @@ -34,6 +34,7 @@ #define ECM_INT_LINK 0x00000020 #define ECM_INT_TX_0 0x00000100 #define ECM_INT_RX_0 0x00000200 +#define ECM_INT_TXRX_SHIFT 2 #define ECM_INT_ALL 0x7FFFFFFF #define ECM_INT_DISABLE 0x80000000 @@ -92,8 +93,7 @@ /* tsnep register */ #define TSNEP_INFO 0x0100 -#define TSNEP_INFO_RX_ASSIGN 0x00010000 -#define TSNEP_INFO_TX_TIME 0x00020000 +#define TSNEP_INFO_TX_TIME 0x00010000 #define TSNEP_CONTROL 0x0108 #define TSNEP_CONTROL_TX_RESET 0x00000001 #define TSNEP_CONTROL_TX_ENABLE 0x00000002 @@ -122,10 +122,6 @@ #define TSNEP_RX_STATISTIC_BUFFER_TOO_SMALL 0x0191 #define TSNEP_RX_STATISTIC_FIFO_OVERFLOW 0x0192 #define TSNEP_RX_STATISTIC_INVALID_FRAME 0x0193 -#define TSNEP_RX_ASSIGN 0x01A0 -#define TSNEP_RX_ASSIGN_ETHER_TYPE_ACTIVE 0x00000001 -#define TSNEP_RX_ASSIGN_ETHER_TYPE_MASK 0xFFFF0000 -#define TSNEP_RX_ASSIGN_ETHER_TYPE_SHIFT 16 #define TSNEP_MAC_ADDRESS_LOW 0x0800 #define TSNEP_MAC_ADDRESS_HIGH 0x0804 #define TSNEP_RX_FILTER 0x0806 @@ -152,6 +148,14 @@ #define TSNEP_GCL_A 0x2000 #define TSNEP_GCL_B 0x2800 #define TSNEP_GCL_SIZE SZ_2K +#define TSNEP_RX_ASSIGN 0x0840 +#define TSNEP_RX_ASSIGN_ACTIVE 0x00000001 +#define TSNEP_RX_ASSIGN_QUEUE_MASK 0x00000006 +#define TSNEP_RX_ASSIGN_QUEUE_SHIFT 1 +#define TSNEP_RX_ASSIGN_OFFSET 1 +#define TSNEP_RX_ASSIGN_ETHER_TYPE 0x0880 +#define TSNEP_RX_ASSIGN_ETHER_TYPE_OFFSET 2 +#define TSNEP_RX_ASSIGN_ETHER_TYPE_COUNT 2 /* tsnep gate control list operation */ struct tsnep_gcl_operation { diff --git a/drivers/net/ethernet/engleder/tsnep_main.c b/drivers/net/ethernet/engleder/tsnep_main.c index a5f7152a1716..48fb391951dd 100644 --- a/drivers/net/ethernet/engleder/tsnep_main.c +++ b/drivers/net/ethernet/engleder/tsnep_main.c @@ -27,10 +27,10 @@ #include <linux/phy.h> #include <linux/iopoll.h> -#define RX_SKB_LENGTH (round_up(TSNEP_RX_INLINE_METADATA_SIZE + ETH_HLEN + \ - TSNEP_MAX_FRAME_SIZE + ETH_FCS_LEN, 4)) -#define RX_SKB_RESERVE ((16 - TSNEP_RX_INLINE_METADATA_SIZE) + NET_IP_ALIGN) -#define RX_SKB_ALLOC_LENGTH (RX_SKB_RESERVE + RX_SKB_LENGTH) +#define TSNEP_SKB_PAD (NET_SKB_PAD + NET_IP_ALIGN) +#define TSNEP_HEADROOM ALIGN(TSNEP_SKB_PAD, 4) +#define TSNEP_MAX_RX_BUF_SIZE (PAGE_SIZE - TSNEP_HEADROOM - \ + SKB_DATA_ALIGN(sizeof(struct skb_shared_info))) #ifdef CONFIG_ARCH_DMA_ADDR_T_64BIT #define DMA_ADDR_HIGH(dma_addr) ((u32)(((dma_addr) >> 32) & 0xFFFFFFFF)) @@ -60,22 +60,29 @@ static irqreturn_t tsnep_irq(int irq, void *arg) iowrite32(active, adapter->addr + ECM_INT_ACKNOWLEDGE); /* handle link interrupt */ - if ((active & ECM_INT_LINK) != 0) { - if (adapter->netdev->phydev) - phy_mac_interrupt(adapter->netdev->phydev); - } + if ((active & ECM_INT_LINK) != 0) + phy_mac_interrupt(adapter->netdev->phydev); /* handle TX/RX queue 0 interrupt */ if ((active & adapter->queue[0].irq_mask) != 0) { - if (adapter->netdev) { - tsnep_disable_irq(adapter, adapter->queue[0].irq_mask); - napi_schedule(&adapter->queue[0].napi); - } + tsnep_disable_irq(adapter, adapter->queue[0].irq_mask); + napi_schedule(&adapter->queue[0].napi); } return IRQ_HANDLED; } +static irqreturn_t tsnep_irq_txrx(int irq, void *arg) +{ + struct tsnep_queue *queue = arg; + + /* handle TX/RX queue interrupt */ + tsnep_disable_irq(queue->adapter, queue->irq_mask); + napi_schedule(&queue->napi); + + return IRQ_HANDLED; +} + static int tsnep_mdiobus_read(struct mii_bus *bus, int addr, int regnum) { struct tsnep_adapter *adapter = bus->priv; @@ -124,30 +131,51 @@ static int tsnep_mdiobus_write(struct mii_bus *bus, int addr, int regnum, return 0; } +static void tsnep_set_link_mode(struct tsnep_adapter *adapter) +{ + u32 mode; + + switch (adapter->phydev->speed) { + case SPEED_100: + mode = ECM_LINK_MODE_100; + break; + case SPEED_1000: + mode = ECM_LINK_MODE_1000; + break; + default: + mode = ECM_LINK_MODE_OFF; + break; + } + iowrite32(mode, adapter->addr + ECM_STATUS); +} + static void tsnep_phy_link_status_change(struct net_device *netdev) { struct tsnep_adapter *adapter = netdev_priv(netdev); struct phy_device *phydev = netdev->phydev; - u32 mode; - if (phydev->link) { - switch (phydev->speed) { - case SPEED_100: - mode = ECM_LINK_MODE_100; - break; - case SPEED_1000: - mode = ECM_LINK_MODE_1000; - break; - default: - mode = ECM_LINK_MODE_OFF; - break; - } - iowrite32(mode, adapter->addr + ECM_STATUS); - } + if (phydev->link) + tsnep_set_link_mode(adapter); phy_print_status(netdev->phydev); } +static int tsnep_phy_loopback(struct tsnep_adapter *adapter, bool enable) +{ + int retval; + + retval = phy_loopback(adapter->phydev, enable); + + /* PHY link state change is not signaled if loopback is enabled, it + * would delay a working loopback anyway, let's ensure that loopback + * is working immediately by setting link mode directly + */ + if (!retval && enable) + tsnep_set_link_mode(adapter); + + return retval; +} + static int tsnep_phy_open(struct tsnep_adapter *adapter) { struct phy_device *phydev; @@ -241,14 +269,14 @@ alloc_failed: return retval; } -static void tsnep_tx_activate(struct tsnep_tx *tx, int index, bool last) +static void tsnep_tx_activate(struct tsnep_tx *tx, int index, int length, + bool last) { struct tsnep_tx_entry *entry = &tx->entry[index]; entry->properties = 0; if (entry->skb) { - entry->properties = - skb_pagelen(entry->skb) & TSNEP_DESC_LENGTH_MASK; + entry->properties = length & TSNEP_DESC_LENGTH_MASK; entry->properties |= TSNEP_DESC_INTERRUPT_FLAG; if (skb_shinfo(entry->skb)->tx_flags & SKBTX_IN_PROGRESS) entry->properties |= TSNEP_DESC_EXTENDED_WRITEBACK_FLAG; @@ -313,6 +341,7 @@ static int tsnep_tx_map(struct sk_buff *skb, struct tsnep_tx *tx, int count) struct tsnep_tx_entry *entry; unsigned int len; dma_addr_t dma; + int map_len = 0; int i; for (i = 0; i < count; i++) { @@ -335,15 +364,18 @@ static int tsnep_tx_map(struct sk_buff *skb, struct tsnep_tx *tx, int count) dma_unmap_addr_set(entry, dma, dma); entry->desc->tx = __cpu_to_le64(dma); + + map_len += len; } - return 0; + return map_len; } -static void tsnep_tx_unmap(struct tsnep_tx *tx, int index, int count) +static int tsnep_tx_unmap(struct tsnep_tx *tx, int index, int count) { struct device *dmadev = tx->adapter->dmadev; struct tsnep_tx_entry *entry; + int map_len = 0; int i; for (i = 0; i < count; i++) { @@ -360,9 +392,12 @@ static void tsnep_tx_unmap(struct tsnep_tx *tx, int index, int count) dma_unmap_addr(entry, dma), dma_unmap_len(entry, len), DMA_TO_DEVICE); + map_len += entry->len; entry->len = 0; } } + + return map_len; } static netdev_tx_t tsnep_xmit_frame_ring(struct sk_buff *skb, @@ -371,6 +406,7 @@ static netdev_tx_t tsnep_xmit_frame_ring(struct sk_buff *skb, unsigned long flags; int count = 1; struct tsnep_tx_entry *entry; + int length; int i; int retval; @@ -394,7 +430,7 @@ static netdev_tx_t tsnep_xmit_frame_ring(struct sk_buff *skb, entry->skb = skb; retval = tsnep_tx_map(skb, tx, count); - if (retval != 0) { + if (retval < 0) { tsnep_tx_unmap(tx, tx->write, count); dev_kfree_skb_any(entry->skb); entry->skb = NULL; @@ -407,12 +443,13 @@ static netdev_tx_t tsnep_xmit_frame_ring(struct sk_buff *skb, return NETDEV_TX_OK; } + length = retval; if (skb_shinfo(skb)->tx_flags & SKBTX_HW_TSTAMP) skb_shinfo(skb)->tx_flags |= SKBTX_IN_PROGRESS; for (i = 0; i < count; i++) - tsnep_tx_activate(tx, (tx->write + i) % TSNEP_RING_SIZE, + tsnep_tx_activate(tx, (tx->write + i) % TSNEP_RING_SIZE, length, i == (count - 1)); tx->write = (tx->write + count) % TSNEP_RING_SIZE; @@ -428,9 +465,6 @@ static netdev_tx_t tsnep_xmit_frame_ring(struct sk_buff *skb, netif_stop_queue(tx->adapter->netdev); } - tx->packets++; - tx->bytes += skb_pagelen(entry->skb) + ETH_FCS_LEN; - spin_unlock_irqrestore(&tx->lock, flags); return NETDEV_TX_OK; @@ -442,6 +476,7 @@ static bool tsnep_tx_poll(struct tsnep_tx *tx, int napi_budget) int budget = 128; struct tsnep_tx_entry *entry; int count; + int length; spin_lock_irqsave(&tx->lock, flags); @@ -464,7 +499,7 @@ static bool tsnep_tx_poll(struct tsnep_tx *tx, int napi_budget) if (skb_shinfo(entry->skb)->nr_frags > 0) count += skb_shinfo(entry->skb)->nr_frags; - tsnep_tx_unmap(tx, tx->read, count); + length = tsnep_tx_unmap(tx, tx->read, count); if ((skb_shinfo(entry->skb)->tx_flags & SKBTX_IN_PROGRESS) && (__le32_to_cpu(entry->desc_wb->properties) & @@ -491,6 +526,9 @@ static bool tsnep_tx_poll(struct tsnep_tx *tx, int napi_budget) tx->read = (tx->read + count) % TSNEP_RING_SIZE; + tx->packets++; + tx->bytes += length + ETH_FCS_LEN; + budget--; } while (likely(budget)); @@ -505,7 +543,7 @@ static bool tsnep_tx_poll(struct tsnep_tx *tx, int napi_budget) } static int tsnep_tx_open(struct tsnep_adapter *adapter, void __iomem *addr, - struct tsnep_tx *tx) + int queue_index, struct tsnep_tx *tx) { dma_addr_t dma; int retval; @@ -513,6 +551,7 @@ static int tsnep_tx_open(struct tsnep_adapter *adapter, void __iomem *addr, memset(tx, 0, sizeof(*tx)); tx->adapter = adapter; tx->addr = addr; + tx->queue_index = queue_index; retval = tsnep_tx_ring_init(tx); if (retval) @@ -548,14 +587,15 @@ static void tsnep_rx_ring_cleanup(struct tsnep_rx *rx) for (i = 0; i < TSNEP_RING_SIZE; i++) { entry = &rx->entry[i]; - if (dma_unmap_addr(entry, dma)) - dma_unmap_single(dmadev, dma_unmap_addr(entry, dma), - dma_unmap_len(entry, len), - DMA_FROM_DEVICE); - if (entry->skb) - dev_kfree_skb(entry->skb); + if (entry->page) + page_pool_put_full_page(rx->page_pool, entry->page, + false); + entry->page = NULL; } + if (rx->page_pool) + page_pool_destroy(rx->page_pool); + memset(rx->entry, 0, sizeof(rx->entry)); for (i = 0; i < TSNEP_RING_PAGE_COUNT; i++) { @@ -568,31 +608,19 @@ static void tsnep_rx_ring_cleanup(struct tsnep_rx *rx) } } -static int tsnep_rx_alloc_and_map_skb(struct tsnep_rx *rx, - struct tsnep_rx_entry *entry) +static int tsnep_rx_alloc_buffer(struct tsnep_rx *rx, + struct tsnep_rx_entry *entry) { - struct device *dmadev = rx->adapter->dmadev; - struct sk_buff *skb; - dma_addr_t dma; + struct page *page; - skb = __netdev_alloc_skb(rx->adapter->netdev, RX_SKB_ALLOC_LENGTH, - GFP_ATOMIC | GFP_DMA); - if (!skb) + page = page_pool_dev_alloc_pages(rx->page_pool); + if (unlikely(!page)) return -ENOMEM; - skb_reserve(skb, RX_SKB_RESERVE); - - dma = dma_map_single(dmadev, skb->data, RX_SKB_LENGTH, - DMA_FROM_DEVICE); - if (dma_mapping_error(dmadev, dma)) { - dev_kfree_skb(skb); - return -ENOMEM; - } - - entry->skb = skb; - entry->len = RX_SKB_LENGTH; - dma_unmap_addr_set(entry, dma, dma); - entry->desc->rx = __cpu_to_le64(dma); + entry->page = page; + entry->len = TSNEP_MAX_RX_BUF_SIZE; + entry->dma = page_pool_get_dma_addr(entry->page); + entry->desc->rx = __cpu_to_le64(entry->dma + TSNEP_SKB_PAD); return 0; } @@ -601,6 +629,7 @@ static int tsnep_rx_ring_init(struct tsnep_rx *rx) { struct device *dmadev = rx->adapter->dmadev; struct tsnep_rx_entry *entry; + struct page_pool_params pp_params = { 0 }; struct tsnep_rx_entry *next_entry; int i, j; int retval; @@ -622,12 +651,28 @@ static int tsnep_rx_ring_init(struct tsnep_rx *rx) entry->desc_dma = rx->page_dma[i] + TSNEP_DESC_SIZE * j; } } + + pp_params.flags = PP_FLAG_DMA_MAP | PP_FLAG_DMA_SYNC_DEV; + pp_params.order = 0; + pp_params.pool_size = TSNEP_RING_SIZE; + pp_params.nid = dev_to_node(dmadev); + pp_params.dev = dmadev; + pp_params.dma_dir = DMA_FROM_DEVICE; + pp_params.max_len = TSNEP_MAX_RX_BUF_SIZE; + pp_params.offset = TSNEP_SKB_PAD; + rx->page_pool = page_pool_create(&pp_params); + if (IS_ERR(rx->page_pool)) { + retval = PTR_ERR(rx->page_pool); + rx->page_pool = NULL; + goto failed; + } + for (i = 0; i < TSNEP_RING_SIZE; i++) { entry = &rx->entry[i]; next_entry = &rx->entry[(i + 1) % TSNEP_RING_SIZE]; entry->desc->next = __cpu_to_le64(next_entry->desc_dma); - retval = tsnep_rx_alloc_and_map_skb(rx, entry); + retval = tsnep_rx_alloc_buffer(rx, entry); if (retval) goto failed; } @@ -643,7 +688,7 @@ static void tsnep_rx_activate(struct tsnep_rx *rx, int index) { struct tsnep_rx_entry *entry = &rx->entry[index]; - /* RX_SKB_LENGTH is a multiple of 4 */ + /* TSNEP_MAX_RX_BUF_SIZE is a multiple of 4 */ entry->properties = entry->len & TSNEP_DESC_LENGTH_MASK; entry->properties |= TSNEP_DESC_INTERRUPT_FLAG; if (index == rx->increment_owner_counter) { @@ -666,19 +711,52 @@ static void tsnep_rx_activate(struct tsnep_rx *rx, int index) entry->desc->properties = __cpu_to_le32(entry->properties); } +static struct sk_buff *tsnep_build_skb(struct tsnep_rx *rx, struct page *page, + int length) +{ + struct sk_buff *skb; + + skb = napi_build_skb(page_address(page), PAGE_SIZE); + if (unlikely(!skb)) + return NULL; + + /* update pointers within the skb to store the data */ + skb_reserve(skb, TSNEP_SKB_PAD + TSNEP_RX_INLINE_METADATA_SIZE); + __skb_put(skb, length - TSNEP_RX_INLINE_METADATA_SIZE - ETH_FCS_LEN); + + if (rx->adapter->hwtstamp_config.rx_filter == HWTSTAMP_FILTER_ALL) { + struct skb_shared_hwtstamps *hwtstamps = skb_hwtstamps(skb); + struct tsnep_rx_inline *rx_inline = + (struct tsnep_rx_inline *)(page_address(page) + + TSNEP_SKB_PAD); + + skb_shinfo(skb)->tx_flags |= + SKBTX_HW_TSTAMP_NETDEV; + memset(hwtstamps, 0, sizeof(*hwtstamps)); + hwtstamps->netdev_data = rx_inline; + } + + skb_record_rx_queue(skb, rx->queue_index); + skb->protocol = eth_type_trans(skb, rx->adapter->netdev); + + return skb; +} + static int tsnep_rx_poll(struct tsnep_rx *rx, struct napi_struct *napi, int budget) { struct device *dmadev = rx->adapter->dmadev; int done = 0; + enum dma_data_direction dma_dir; struct tsnep_rx_entry *entry; + struct page *page; struct sk_buff *skb; - size_t len; - dma_addr_t dma; int length; bool enable = false; int retval; + dma_dir = page_pool_get_dma_dir(rx->page_pool); + while (likely(done < budget)) { entry = &rx->entry[rx->read]; if ((__le32_to_cpu(entry->desc_wb->properties) & @@ -691,42 +769,34 @@ static int tsnep_rx_poll(struct tsnep_rx *rx, struct napi_struct *napi, */ dma_rmb(); - skb = entry->skb; - len = dma_unmap_len(entry, len); - dma = dma_unmap_addr(entry, dma); + prefetch(page_address(entry->page) + TSNEP_SKB_PAD); + length = __le32_to_cpu(entry->desc_wb->properties) & + TSNEP_DESC_LENGTH_MASK; + dma_sync_single_range_for_cpu(dmadev, entry->dma, TSNEP_SKB_PAD, + length, dma_dir); + page = entry->page; /* forward skb only if allocation is successful, otherwise - * skb is reused and frame dropped + * page is reused and frame dropped */ - retval = tsnep_rx_alloc_and_map_skb(rx, entry); + retval = tsnep_rx_alloc_buffer(rx, entry); if (!retval) { - dma_unmap_single(dmadev, dma, len, DMA_FROM_DEVICE); - - length = __le32_to_cpu(entry->desc_wb->properties) & - TSNEP_DESC_LENGTH_MASK; - skb_put(skb, length - ETH_FCS_LEN); - if (rx->adapter->hwtstamp_config.rx_filter == - HWTSTAMP_FILTER_ALL) { - struct skb_shared_hwtstamps *hwtstamps = - skb_hwtstamps(skb); - struct tsnep_rx_inline *rx_inline = - (struct tsnep_rx_inline *)skb->data; - - skb_shinfo(skb)->tx_flags |= - SKBTX_HW_TSTAMP_NETDEV; - memset(hwtstamps, 0, sizeof(*hwtstamps)); - hwtstamps->netdev_data = rx_inline; - } - skb_pull(skb, TSNEP_RX_INLINE_METADATA_SIZE); - skb->protocol = eth_type_trans(skb, - rx->adapter->netdev); + skb = tsnep_build_skb(rx, page, length); + if (skb) { + page_pool_release_page(rx->page_pool, page); - rx->packets++; - rx->bytes += length - TSNEP_RX_INLINE_METADATA_SIZE; - if (skb->pkt_type == PACKET_MULTICAST) - rx->multicast++; + rx->packets++; + rx->bytes += length - + TSNEP_RX_INLINE_METADATA_SIZE; + if (skb->pkt_type == PACKET_MULTICAST) + rx->multicast++; - napi_gro_receive(napi, skb); + napi_gro_receive(napi, skb); + } else { + page_pool_recycle_direct(rx->page_pool, page); + + rx->dropped++; + } done++; } else { rx->dropped++; @@ -752,7 +822,7 @@ static int tsnep_rx_poll(struct tsnep_rx *rx, struct napi_struct *napi, } static int tsnep_rx_open(struct tsnep_adapter *adapter, void __iomem *addr, - struct tsnep_rx *rx) + int queue_index, struct tsnep_rx *rx) { dma_addr_t dma; int i; @@ -761,6 +831,7 @@ static int tsnep_rx_open(struct tsnep_adapter *adapter, void __iomem *addr, memset(rx, 0, sizeof(*rx)); rx->adapter = adapter; rx->addr = addr; + rx->queue_index = queue_index; retval = tsnep_rx_ring_init(rx); if (retval) @@ -821,6 +892,56 @@ static int tsnep_poll(struct napi_struct *napi, int budget) return min(done, budget - 1); } +static int tsnep_request_irq(struct tsnep_queue *queue, bool first) +{ + const char *name = netdev_name(queue->adapter->netdev); + irq_handler_t handler; + void *dev; + int retval; + + if (first) { + sprintf(queue->name, "%s-mac", name); + handler = tsnep_irq; + dev = queue->adapter; + } else { + if (queue->tx && queue->rx) + sprintf(queue->name, "%s-txrx-%d", name, + queue->rx->queue_index); + else if (queue->tx) + sprintf(queue->name, "%s-tx-%d", name, + queue->tx->queue_index); + else + sprintf(queue->name, "%s-rx-%d", name, + queue->rx->queue_index); + handler = tsnep_irq_txrx; + dev = queue; + } + + retval = request_irq(queue->irq, handler, 0, queue->name, dev); + if (retval) { + /* if name is empty, then interrupt won't be freed */ + memset(queue->name, 0, sizeof(queue->name)); + } + + return retval; +} + +static void tsnep_free_irq(struct tsnep_queue *queue, bool first) +{ + void *dev; + + if (!strlen(queue->name)) + return; + + if (first) + dev = queue->adapter; + else + dev = queue; + + free_irq(queue->irq, dev); + memset(queue->name, 0, sizeof(queue->name)); +} + static int tsnep_netdev_open(struct net_device *netdev) { struct tsnep_adapter *adapter = netdev_priv(netdev); @@ -830,15 +951,11 @@ static int tsnep_netdev_open(struct net_device *netdev) int rx_queue_index = 0; int retval; - retval = tsnep_phy_open(adapter); - if (retval) - return retval; - for (i = 0; i < adapter->num_queues; i++) { adapter->queue[i].adapter = adapter; if (adapter->queue[i].tx) { addr = adapter->addr + TSNEP_QUEUE(tx_queue_index); - retval = tsnep_tx_open(adapter, addr, + retval = tsnep_tx_open(adapter, addr, tx_queue_index, adapter->queue[i].tx); if (retval) goto failed; @@ -847,11 +964,20 @@ static int tsnep_netdev_open(struct net_device *netdev) if (adapter->queue[i].rx) { addr = adapter->addr + TSNEP_QUEUE(rx_queue_index); retval = tsnep_rx_open(adapter, addr, + rx_queue_index, adapter->queue[i].rx); if (retval) goto failed; rx_queue_index++; } + + retval = tsnep_request_irq(&adapter->queue[i], i == 0); + if (retval) { + netif_err(adapter, drv, adapter->netdev, + "can't get assigned irq %d.\n", + adapter->queue[i].irq); + goto failed; + } } retval = netif_set_real_num_tx_queues(adapter->netdev, @@ -863,9 +989,14 @@ static int tsnep_netdev_open(struct net_device *netdev) if (retval) goto failed; + tsnep_enable_irq(adapter, ECM_INT_LINK); + retval = tsnep_phy_open(adapter); + if (retval) + goto phy_failed; + for (i = 0; i < adapter->num_queues; i++) { netif_napi_add(adapter->netdev, &adapter->queue[i].napi, - tsnep_poll, 64); + tsnep_poll); napi_enable(&adapter->queue[i].napi); tsnep_enable_irq(adapter, adapter->queue[i].irq_mask); @@ -873,14 +1004,18 @@ static int tsnep_netdev_open(struct net_device *netdev) return 0; +phy_failed: + tsnep_disable_irq(adapter, ECM_INT_LINK); + tsnep_phy_close(adapter); failed: for (i = 0; i < adapter->num_queues; i++) { + tsnep_free_irq(&adapter->queue[i], i == 0); + if (adapter->queue[i].rx) tsnep_rx_close(adapter->queue[i].rx); if (adapter->queue[i].tx) tsnep_tx_close(adapter->queue[i].tx); } - tsnep_phy_close(adapter); return retval; } @@ -889,20 +1024,23 @@ static int tsnep_netdev_close(struct net_device *netdev) struct tsnep_adapter *adapter = netdev_priv(netdev); int i; + tsnep_disable_irq(adapter, ECM_INT_LINK); + tsnep_phy_close(adapter); + for (i = 0; i < adapter->num_queues; i++) { tsnep_disable_irq(adapter, adapter->queue[i].irq_mask); napi_disable(&adapter->queue[i].napi); netif_napi_del(&adapter->queue[i].napi); + tsnep_free_irq(&adapter->queue[i], i == 0); + if (adapter->queue[i].rx) tsnep_rx_close(adapter->queue[i].rx); if (adapter->queue[i].tx) tsnep_tx_close(adapter->queue[i].tx); } - tsnep_phy_close(adapter); - return 0; } @@ -1017,6 +1155,22 @@ static int tsnep_netdev_set_mac_address(struct net_device *netdev, void *addr) return 0; } +static int tsnep_netdev_set_features(struct net_device *netdev, + netdev_features_t features) +{ + struct tsnep_adapter *adapter = netdev_priv(netdev); + netdev_features_t changed = netdev->features ^ features; + bool enable; + int retval = 0; + + if (changed & NETIF_F_LOOPBACK) { + enable = !!(features & NETIF_F_LOOPBACK); + retval = tsnep_phy_loopback(adapter, enable); + } + + return retval; +} + static ktime_t tsnep_netdev_get_tstamp(struct net_device *netdev, const struct skb_shared_hwtstamps *hwtstamps, bool cycles) @@ -1038,9 +1192,9 @@ static const struct net_device_ops tsnep_netdev_ops = { .ndo_start_xmit = tsnep_netdev_xmit_frame, .ndo_eth_ioctl = tsnep_netdev_ioctl, .ndo_set_rx_mode = tsnep_netdev_set_multicast, - .ndo_get_stats64 = tsnep_netdev_get_stats64, .ndo_set_mac_address = tsnep_netdev_set_mac_address, + .ndo_set_features = tsnep_netdev_set_features, .ndo_get_tstamp = tsnep_netdev_get_tstamp, .ndo_setup_tc = tsnep_tc_setup, }; @@ -1141,6 +1295,52 @@ static int tsnep_phy_init(struct tsnep_adapter *adapter) return 0; } +static int tsnep_queue_init(struct tsnep_adapter *adapter, int queue_count) +{ + u32 irq_mask = ECM_INT_TX_0 | ECM_INT_RX_0; + char name[8]; + int i; + int retval; + + /* one TX/RX queue pair for netdev is mandatory */ + if (platform_irq_count(adapter->pdev) == 1) + retval = platform_get_irq(adapter->pdev, 0); + else + retval = platform_get_irq_byname(adapter->pdev, "mac"); + if (retval < 0) + return retval; + adapter->num_tx_queues = 1; + adapter->num_rx_queues = 1; + adapter->num_queues = 1; + adapter->queue[0].irq = retval; + adapter->queue[0].tx = &adapter->tx[0]; + adapter->queue[0].rx = &adapter->rx[0]; + adapter->queue[0].irq_mask = irq_mask; + + adapter->netdev->irq = adapter->queue[0].irq; + + /* add additional TX/RX queue pairs only if dedicated interrupt is + * available + */ + for (i = 1; i < queue_count; i++) { + sprintf(name, "txrx-%d", i); + retval = platform_get_irq_byname_optional(adapter->pdev, name); + if (retval < 0) + break; + + adapter->num_tx_queues++; + adapter->num_rx_queues++; + adapter->num_queues++; + adapter->queue[i].irq = retval; + adapter->queue[i].tx = &adapter->tx[i]; + adapter->queue[i].rx = &adapter->rx[i]; + adapter->queue[i].irq_mask = + irq_mask << (ECM_INT_TXRX_SHIFT * i); + } + + return 0; +} + static int tsnep_probe(struct platform_device *pdev) { struct tsnep_adapter *adapter; @@ -1149,6 +1349,7 @@ static int tsnep_probe(struct platform_device *pdev) u32 type; int revision; int version; + int queue_count; int retval; netdev = devm_alloc_etherdev_mqs(&pdev->dev, @@ -1170,41 +1371,39 @@ static int tsnep_probe(struct platform_device *pdev) netdev->max_mtu = TSNEP_MAX_FRAME_SIZE; mutex_init(&adapter->gate_control_lock); + mutex_init(&adapter->rxnfc_lock); + INIT_LIST_HEAD(&adapter->rxnfc_rules); io = platform_get_resource(pdev, IORESOURCE_MEM, 0); adapter->addr = devm_ioremap_resource(&pdev->dev, io); if (IS_ERR(adapter->addr)) return PTR_ERR(adapter->addr); - adapter->irq = platform_get_irq(pdev, 0); netdev->mem_start = io->start; netdev->mem_end = io->end; - netdev->irq = adapter->irq; type = ioread32(adapter->addr + ECM_TYPE); revision = (type & ECM_REVISION_MASK) >> ECM_REVISION_SHIFT; version = (type & ECM_VERSION_MASK) >> ECM_VERSION_SHIFT; + queue_count = (type & ECM_QUEUE_COUNT_MASK) >> ECM_QUEUE_COUNT_SHIFT; adapter->gate_control = type & ECM_GATE_CONTROL; - - adapter->num_tx_queues = TSNEP_QUEUES; - adapter->num_rx_queues = TSNEP_QUEUES; - adapter->num_queues = TSNEP_QUEUES; - adapter->queue[0].tx = &adapter->tx[0]; - adapter->queue[0].rx = &adapter->rx[0]; - adapter->queue[0].irq_mask = ECM_INT_TX_0 | ECM_INT_RX_0; + adapter->rxnfc_max = TSNEP_RX_ASSIGN_ETHER_TYPE_COUNT; tsnep_disable_irq(adapter, ECM_INT_ALL); - retval = devm_request_irq(&adapter->pdev->dev, adapter->irq, tsnep_irq, - 0, TSNEP, adapter); - if (retval != 0) { - dev_err(&adapter->pdev->dev, "can't get assigned irq %d.\n", - adapter->irq); + + retval = tsnep_queue_init(adapter, queue_count); + if (retval) + return retval; + + retval = dma_set_mask_and_coherent(&adapter->pdev->dev, + DMA_BIT_MASK(64)); + if (retval) { + dev_err(&adapter->pdev->dev, "no usable DMA configuration.\n"); return retval; } - tsnep_enable_irq(adapter, ECM_INT_LINK); retval = tsnep_mac_init(adapter); if (retval) - goto mac_init_failed; + return retval; retval = tsnep_mdio_init(adapter); if (retval) @@ -1222,10 +1421,14 @@ static int tsnep_probe(struct platform_device *pdev) if (retval) goto tc_init_failed; + retval = tsnep_rxnfc_init(adapter); + if (retval) + goto rxnfc_init_failed; + netdev->netdev_ops = &tsnep_netdev_ops; netdev->ethtool_ops = &tsnep_ethtool_ops; netdev->features = NETIF_F_SG; - netdev->hw_features = netdev->features; + netdev->hw_features = netdev->features | NETIF_F_LOOPBACK; /* carrier off reporting is important to ethtool even BEFORE open */ netif_carrier_off(netdev); @@ -1242,6 +1445,8 @@ static int tsnep_probe(struct platform_device *pdev) return 0; register_failed: + tsnep_rxnfc_cleanup(adapter); +rxnfc_init_failed: tsnep_tc_cleanup(adapter); tc_init_failed: tsnep_ptp_cleanup(adapter); @@ -1250,8 +1455,6 @@ phy_init_failed: if (adapter->mdiobus) mdiobus_unregister(adapter->mdiobus); mdio_init_failed: -mac_init_failed: - tsnep_disable_irq(adapter, ECM_INT_ALL); return retval; } @@ -1261,6 +1464,8 @@ static int tsnep_remove(struct platform_device *pdev) unregister_netdev(adapter->netdev); + tsnep_rxnfc_cleanup(adapter); + tsnep_tc_cleanup(adapter); tsnep_ptp_cleanup(adapter); diff --git a/drivers/net/ethernet/engleder/tsnep_rxnfc.c b/drivers/net/ethernet/engleder/tsnep_rxnfc.c new file mode 100644 index 000000000000..9ac2a0cf3833 --- /dev/null +++ b/drivers/net/ethernet/engleder/tsnep_rxnfc.c @@ -0,0 +1,307 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (C) 2022 Gerhard Engleder <gerhard@engleder-embedded.com> */ + +#include "tsnep.h" + +#define ETHER_TYPE_FULL_MASK ((__force __be16)~0) + +static void tsnep_enable_rule(struct tsnep_adapter *adapter, + struct tsnep_rxnfc_rule *rule) +{ + u8 rx_assign; + void __iomem *addr; + + rx_assign = TSNEP_RX_ASSIGN_ACTIVE; + rx_assign |= (rule->queue_index << TSNEP_RX_ASSIGN_QUEUE_SHIFT) & + TSNEP_RX_ASSIGN_QUEUE_MASK; + + addr = adapter->addr + TSNEP_RX_ASSIGN_ETHER_TYPE + + TSNEP_RX_ASSIGN_ETHER_TYPE_OFFSET * rule->location; + iowrite16(rule->filter.ether_type, addr); + + /* enable rule after all settings are done */ + addr = adapter->addr + TSNEP_RX_ASSIGN + + TSNEP_RX_ASSIGN_OFFSET * rule->location; + iowrite8(rx_assign, addr); +} + +static void tsnep_disable_rule(struct tsnep_adapter *adapter, + struct tsnep_rxnfc_rule *rule) +{ + void __iomem *addr; + + addr = adapter->addr + TSNEP_RX_ASSIGN + + TSNEP_RX_ASSIGN_OFFSET * rule->location; + iowrite8(0, addr); +} + +static struct tsnep_rxnfc_rule *tsnep_get_rule(struct tsnep_adapter *adapter, + int location) +{ + struct tsnep_rxnfc_rule *rule; + + list_for_each_entry(rule, &adapter->rxnfc_rules, list) { + if (rule->location == location) + return rule; + if (rule->location > location) + break; + } + + return NULL; +} + +static void tsnep_add_rule(struct tsnep_adapter *adapter, + struct tsnep_rxnfc_rule *rule) +{ + struct tsnep_rxnfc_rule *pred, *cur; + + tsnep_enable_rule(adapter, rule); + + pred = NULL; + list_for_each_entry(cur, &adapter->rxnfc_rules, list) { + if (cur->location >= rule->location) + break; + pred = cur; + } + + list_add(&rule->list, pred ? &pred->list : &adapter->rxnfc_rules); + adapter->rxnfc_count++; +} + +static void tsnep_delete_rule(struct tsnep_adapter *adapter, + struct tsnep_rxnfc_rule *rule) +{ + tsnep_disable_rule(adapter, rule); + + list_del(&rule->list); + adapter->rxnfc_count--; + + kfree(rule); +} + +static void tsnep_flush_rules(struct tsnep_adapter *adapter) +{ + struct tsnep_rxnfc_rule *rule, *tmp; + + mutex_lock(&adapter->rxnfc_lock); + + list_for_each_entry_safe(rule, tmp, &adapter->rxnfc_rules, list) + tsnep_delete_rule(adapter, rule); + + mutex_unlock(&adapter->rxnfc_lock); +} + +int tsnep_rxnfc_get_rule(struct tsnep_adapter *adapter, + struct ethtool_rxnfc *cmd) +{ + struct ethtool_rx_flow_spec *fsp = &cmd->fs; + struct tsnep_rxnfc_rule *rule = NULL; + + cmd->data = adapter->rxnfc_max; + + mutex_lock(&adapter->rxnfc_lock); + + rule = tsnep_get_rule(adapter, fsp->location); + if (!rule) { + mutex_unlock(&adapter->rxnfc_lock); + + return -ENOENT; + } + + fsp->flow_type = ETHER_FLOW; + fsp->ring_cookie = rule->queue_index; + + if (rule->filter.type == TSNEP_RXNFC_ETHER_TYPE) { + fsp->h_u.ether_spec.h_proto = htons(rule->filter.ether_type); + fsp->m_u.ether_spec.h_proto = ETHER_TYPE_FULL_MASK; + } + + mutex_unlock(&adapter->rxnfc_lock); + + return 0; +} + +int tsnep_rxnfc_get_all(struct tsnep_adapter *adapter, + struct ethtool_rxnfc *cmd, + u32 *rule_locs) +{ + struct tsnep_rxnfc_rule *rule; + int count = 0; + + cmd->data = adapter->rxnfc_max; + + mutex_lock(&adapter->rxnfc_lock); + + list_for_each_entry(rule, &adapter->rxnfc_rules, list) { + if (count == cmd->rule_cnt) { + mutex_unlock(&adapter->rxnfc_lock); + + return -EMSGSIZE; + } + + rule_locs[count] = rule->location; + count++; + } + + mutex_unlock(&adapter->rxnfc_lock); + + cmd->rule_cnt = count; + + return 0; +} + +static int tsnep_rxnfc_find_location(struct tsnep_adapter *adapter) +{ + struct tsnep_rxnfc_rule *tmp; + int location = 0; + + list_for_each_entry(tmp, &adapter->rxnfc_rules, list) { + if (tmp->location == location) + location++; + else + return location; + } + + if (location >= adapter->rxnfc_max) + return -ENOSPC; + + return location; +} + +static void tsnep_rxnfc_init_rule(struct tsnep_rxnfc_rule *rule, + const struct ethtool_rx_flow_spec *fsp) +{ + INIT_LIST_HEAD(&rule->list); + + rule->queue_index = fsp->ring_cookie; + rule->location = fsp->location; + + rule->filter.type = TSNEP_RXNFC_ETHER_TYPE; + rule->filter.ether_type = ntohs(fsp->h_u.ether_spec.h_proto); +} + +static int tsnep_rxnfc_check_rule(struct tsnep_adapter *adapter, + struct tsnep_rxnfc_rule *rule) +{ + struct net_device *dev = adapter->netdev; + struct tsnep_rxnfc_rule *tmp; + + list_for_each_entry(tmp, &adapter->rxnfc_rules, list) { + if (!memcmp(&rule->filter, &tmp->filter, sizeof(rule->filter)) && + tmp->location != rule->location) { + netdev_dbg(dev, "rule already exists\n"); + + return -EEXIST; + } + } + + return 0; +} + +int tsnep_rxnfc_add_rule(struct tsnep_adapter *adapter, + struct ethtool_rxnfc *cmd) +{ + struct net_device *netdev = adapter->netdev; + struct ethtool_rx_flow_spec *fsp = + (struct ethtool_rx_flow_spec *)&cmd->fs; + struct tsnep_rxnfc_rule *rule, *old_rule; + int retval; + + /* only EtherType is supported */ + if (fsp->flow_type != ETHER_FLOW || + !is_zero_ether_addr(fsp->m_u.ether_spec.h_dest) || + !is_zero_ether_addr(fsp->m_u.ether_spec.h_source) || + fsp->m_u.ether_spec.h_proto != ETHER_TYPE_FULL_MASK) { + netdev_dbg(netdev, "only ethernet protocol is supported\n"); + + return -EOPNOTSUPP; + } + + if (fsp->ring_cookie > + (TSNEP_RX_ASSIGN_QUEUE_MASK >> TSNEP_RX_ASSIGN_QUEUE_SHIFT)) { + netdev_dbg(netdev, "invalid action\n"); + + return -EINVAL; + } + + if (fsp->location != RX_CLS_LOC_ANY && + fsp->location >= adapter->rxnfc_max) { + netdev_dbg(netdev, "invalid location\n"); + + return -EINVAL; + } + + rule = kzalloc(sizeof(*rule), GFP_KERNEL); + if (!rule) + return -ENOMEM; + + mutex_lock(&adapter->rxnfc_lock); + + if (fsp->location == RX_CLS_LOC_ANY) { + retval = tsnep_rxnfc_find_location(adapter); + if (retval < 0) + goto failed; + fsp->location = retval; + } + + tsnep_rxnfc_init_rule(rule, fsp); + + retval = tsnep_rxnfc_check_rule(adapter, rule); + if (retval) + goto failed; + + old_rule = tsnep_get_rule(adapter, fsp->location); + if (old_rule) + tsnep_delete_rule(adapter, old_rule); + + tsnep_add_rule(adapter, rule); + + mutex_unlock(&adapter->rxnfc_lock); + + return 0; + +failed: + mutex_unlock(&adapter->rxnfc_lock); + kfree(rule); + return retval; +} + +int tsnep_rxnfc_del_rule(struct tsnep_adapter *adapter, + struct ethtool_rxnfc *cmd) +{ + struct ethtool_rx_flow_spec *fsp = + (struct ethtool_rx_flow_spec *)&cmd->fs; + struct tsnep_rxnfc_rule *rule; + + mutex_lock(&adapter->rxnfc_lock); + + rule = tsnep_get_rule(adapter, fsp->location); + if (!rule) { + mutex_unlock(&adapter->rxnfc_lock); + + return -ENOENT; + } + + tsnep_delete_rule(adapter, rule); + + mutex_unlock(&adapter->rxnfc_lock); + + return 0; +} + +int tsnep_rxnfc_init(struct tsnep_adapter *adapter) +{ + int i; + + /* disable all rules */ + for (i = 0; i < adapter->rxnfc_max; + i += sizeof(u32) / TSNEP_RX_ASSIGN_OFFSET) + iowrite32(0, adapter->addr + TSNEP_RX_ASSIGN + i); + + return 0; +} + +void tsnep_rxnfc_cleanup(struct tsnep_adapter *adapter) +{ + tsnep_flush_rules(adapter); +} diff --git a/drivers/net/ethernet/ethoc.c b/drivers/net/ethernet/ethoc.c index 437c5acfe222..95cbad198b4b 100644 --- a/drivers/net/ethernet/ethoc.c +++ b/drivers/net/ethernet/ethoc.c @@ -1224,7 +1224,7 @@ static int ethoc_probe(struct platform_device *pdev) netdev->ethtool_ops = ðoc_ethtool_ops; /* setup NAPI */ - netif_napi_add(netdev, &priv->napi, ethoc_poll, 64); + netif_napi_add(netdev, &priv->napi, ethoc_poll); spin_lock_init(&priv->lock); diff --git a/drivers/net/ethernet/faraday/ftgmac100.c b/drivers/net/ethernet/faraday/ftgmac100.c index c03663785a8d..a03879a27b04 100644 --- a/drivers/net/ethernet/faraday/ftgmac100.c +++ b/drivers/net/ethernet/faraday/ftgmac100.c @@ -1063,8 +1063,8 @@ static int ftgmac100_mdiobus_write(struct mii_bus *bus, int phy_addr, static void ftgmac100_get_drvinfo(struct net_device *netdev, struct ethtool_drvinfo *info) { - strlcpy(info->driver, DRV_NAME, sizeof(info->driver)); - strlcpy(info->bus_info, dev_name(&netdev->dev), sizeof(info->bus_info)); + strscpy(info->driver, DRV_NAME, sizeof(info->driver)); + strscpy(info->bus_info, dev_name(&netdev->dev), sizeof(info->bus_info)); } static void @@ -1506,7 +1506,7 @@ static int ftgmac100_open(struct net_device *netdev) goto err_hw; /* Initialize NAPI */ - netif_napi_add(netdev, &priv->napi, ftgmac100_poll, 64); + netif_napi_add(netdev, &priv->napi, ftgmac100_poll); /* Grab our interrupt */ err = request_irq(netdev->irq, ftgmac100_interrupt, 0, netdev->name, netdev); @@ -1701,10 +1701,14 @@ err_register_mdiobus: static void ftgmac100_phy_disconnect(struct net_device *netdev) { + struct ftgmac100 *priv = netdev_priv(netdev); + if (!netdev->phydev) return; phy_disconnect(netdev->phydev); + if (of_phy_is_fixed_link(priv->dev->of_node)) + of_phy_deregister_fixed_link(priv->dev->of_node); } static void ftgmac100_destroy_mdio(struct net_device *netdev) @@ -1867,6 +1871,26 @@ static int ftgmac100_probe(struct platform_device *pdev) err = -EINVAL; goto err_phy_connect; } + } else if (np && of_phy_is_fixed_link(np)) { + struct phy_device *phy; + + err = of_phy_register_fixed_link(np); + if (err) { + dev_err(&pdev->dev, "Failed to register fixed PHY\n"); + goto err_phy_connect; + } + + phy = of_phy_get_and_connect(priv->netdev, np, + &ftgmac100_adjust_link); + if (!phy) { + dev_err(&pdev->dev, "Failed to connect to fixed PHY\n"); + of_phy_deregister_fixed_link(np); + err = -EINVAL; + goto err_phy_connect; + } + + /* Display what we found */ + phy_attached_info(phy); } else if (np && of_get_property(np, "phy-handle", NULL)) { struct phy_device *phy; diff --git a/drivers/net/ethernet/faraday/ftmac100.c b/drivers/net/ethernet/faraday/ftmac100.c index 8a341e2d5833..d95d78230828 100644 --- a/drivers/net/ethernet/faraday/ftmac100.c +++ b/drivers/net/ethernet/faraday/ftmac100.c @@ -807,8 +807,8 @@ static void ftmac100_mdio_write(struct net_device *netdev, int phy_id, int reg, static void ftmac100_get_drvinfo(struct net_device *netdev, struct ethtool_drvinfo *info) { - strlcpy(info->driver, DRV_NAME, sizeof(info->driver)); - strlcpy(info->bus_info, dev_name(&netdev->dev), sizeof(info->bus_info)); + strscpy(info->driver, DRV_NAME, sizeof(info->driver)); + strscpy(info->bus_info, dev_name(&netdev->dev), sizeof(info->bus_info)); } static int ftmac100_get_link_ksettings(struct net_device *netdev, @@ -1075,6 +1075,11 @@ static int ftmac100_probe(struct platform_device *pdev) SET_NETDEV_DEV(netdev, &pdev->dev); netdev->ethtool_ops = &ftmac100_ethtool_ops; netdev->netdev_ops = &ftmac100_netdev_ops; + netdev->max_mtu = MAX_PKT_SIZE; + + err = platform_get_ethdev_address(&pdev->dev, netdev); + if (err == -EPROBE_DEFER) + goto defer_get_mac; platform_set_drvdata(pdev, netdev); @@ -1086,7 +1091,7 @@ static int ftmac100_probe(struct platform_device *pdev) spin_lock_init(&priv->tx_lock); /* initialize NAPI */ - netif_napi_add(netdev, &priv->napi, ftmac100_poll, 64); + netif_napi_add(netdev, &priv->napi, ftmac100_poll); /* map io memory */ priv->res = request_mem_region(res->start, resource_size(res), @@ -1137,6 +1142,7 @@ err_ioremap: release_resource(priv->res); err_req_mem: netif_napi_del(&priv->napi); +defer_get_mac: free_netdev(netdev); err_alloc_etherdev: return err; diff --git a/drivers/net/ethernet/faraday/ftmac100.h b/drivers/net/ethernet/faraday/ftmac100.h index fe986f1673fc..8af32f9070f4 100644 --- a/drivers/net/ethernet/faraday/ftmac100.h +++ b/drivers/net/ethernet/faraday/ftmac100.h @@ -122,9 +122,9 @@ * Transmit descriptor, aligned to 16 bytes */ struct ftmac100_txdes { - unsigned int txdes0; - unsigned int txdes1; - unsigned int txdes2; /* TXBUF_BADR */ + __le32 txdes0; + __le32 txdes1; + __le32 txdes2; /* TXBUF_BADR */ unsigned int txdes3; /* not used by HW */ } __attribute__ ((aligned(16))); @@ -143,9 +143,9 @@ struct ftmac100_txdes { * Receive descriptor, aligned to 16 bytes */ struct ftmac100_rxdes { - unsigned int rxdes0; - unsigned int rxdes1; - unsigned int rxdes2; /* RXBUF_BADR */ + __le32 rxdes0; + __le32 rxdes1; + __le32 rxdes2; /* RXBUF_BADR */ unsigned int rxdes3; /* not used by HW */ } __attribute__ ((aligned(16))); diff --git a/drivers/net/ethernet/fealnx.c b/drivers/net/ethernet/fealnx.c index b3939a5f7b03..ed18450fd2cc 100644 --- a/drivers/net/ethernet/fealnx.c +++ b/drivers/net/ethernet/fealnx.c @@ -1809,8 +1809,8 @@ static void netdev_get_drvinfo(struct net_device *dev, struct ethtool_drvinfo *i { struct netdev_private *np = netdev_priv(dev); - strlcpy(info->driver, DRV_NAME, sizeof(info->driver)); - strlcpy(info->bus_info, pci_name(np->pci_dev), sizeof(info->bus_info)); + strscpy(info->driver, DRV_NAME, sizeof(info->driver)); + strscpy(info->bus_info, pci_name(np->pci_dev), sizeof(info->bus_info)); } static int netdev_get_link_ksettings(struct net_device *dev, diff --git a/drivers/net/ethernet/freescale/Kconfig b/drivers/net/ethernet/freescale/Kconfig index e04e1c5cb013..ce866ae3df03 100644 --- a/drivers/net/ethernet/freescale/Kconfig +++ b/drivers/net/ethernet/freescale/Kconfig @@ -9,7 +9,7 @@ config NET_VENDOR_FREESCALE depends on FSL_SOC || QUICC_ENGINE || CPM1 || CPM2 || PPC_MPC512x || \ M523x || M527x || M5272 || M528x || M520x || M532x || \ ARCH_MXC || ARCH_MXS || (PPC_MPC52xx && PPC_BESTCOMM) || \ - ARCH_LAYERSCAPE || COMPILE_TEST + ARCH_LAYERSCAPE || ARCH_S32 || COMPILE_TEST help If you have a network (Ethernet) card belonging to this class, say Y. @@ -23,15 +23,16 @@ if NET_VENDOR_FREESCALE config FEC tristate "FEC ethernet controller (of ColdFire and some i.MX CPUs)" depends on (M523x || M527x || M5272 || M528x || M520x || M532x || \ - ARCH_MXC || SOC_IMX28 || COMPILE_TEST) + ARCH_MXC || ARCH_S32 || SOC_IMX28 || COMPILE_TEST) default ARCH_MXC || SOC_IMX28 if ARM depends on PTP_1588_CLOCK_OPTIONAL select CRC32 select PHYLIB + select PAGE_POOL imply NET_SELFTESTS help Say Y here if you want to use the built-in 10/100 Fast ethernet - controller on some Motorola ColdFire and Freescale i.MX processors. + controller on some Motorola ColdFire and Freescale i.MX/S32 processors. config FEC_MPC52xx tristate "FEC MPC52xx driver" diff --git a/drivers/net/ethernet/freescale/dpaa/dpaa_eth.c b/drivers/net/ethernet/freescale/dpaa/dpaa_eth.c index a770bab4d1ed..31cfa121333d 100644 --- a/drivers/net/ethernet/freescale/dpaa/dpaa_eth.c +++ b/drivers/net/ethernet/freescale/dpaa/dpaa_eth.c @@ -197,12 +197,15 @@ static int dpaa_rx_extra_headroom; #define dpaa_get_max_mtu() \ (dpaa_max_frm - (VLAN_ETH_HLEN + ETH_FCS_LEN)) +static void dpaa_eth_cgr_set_speed(struct mac_device *mac_dev, int speed); + static int dpaa_netdev_init(struct net_device *net_dev, const struct net_device_ops *dpaa_ops, u16 tx_timeout) { struct dpaa_priv *priv = netdev_priv(net_dev); struct device *dev = net_dev->dev.parent; + struct mac_device *mac_dev = priv->mac_dev; struct dpaa_percpu_priv *percpu_priv; const u8 *mac_addr; int i, err; @@ -216,10 +219,10 @@ static int dpaa_netdev_init(struct net_device *net_dev, } net_dev->netdev_ops = dpaa_ops; - mac_addr = priv->mac_dev->addr; + mac_addr = mac_dev->addr; - net_dev->mem_start = priv->mac_dev->res->start; - net_dev->mem_end = priv->mac_dev->res->end; + net_dev->mem_start = (unsigned long)mac_dev->vaddr; + net_dev->mem_end = (unsigned long)mac_dev->vaddr_end; net_dev->min_mtu = ETH_MIN_MTU; net_dev->max_mtu = dpaa_get_max_mtu(); @@ -246,7 +249,7 @@ static int dpaa_netdev_init(struct net_device *net_dev, eth_hw_addr_set(net_dev, mac_addr); } else { eth_hw_addr_random(net_dev); - err = priv->mac_dev->change_addr(priv->mac_dev->fman_mac, + err = mac_dev->change_addr(mac_dev->fman_mac, (const enet_addr_t *)net_dev->dev_addr); if (err) { dev_err(dev, "Failed to set random MAC address\n"); @@ -261,6 +264,9 @@ static int dpaa_netdev_init(struct net_device *net_dev, net_dev->needed_headroom = priv->tx_headroom; net_dev->watchdog_timeo = msecs_to_jiffies(tx_timeout); + mac_dev->net_dev = net_dev; + mac_dev->update_speed = dpaa_eth_cgr_set_speed; + /* start without the RUNNING flag, phylib controls it later */ netif_carrier_off(net_dev); @@ -288,10 +294,9 @@ static int dpaa_stop(struct net_device *net_dev) */ msleep(200); - err = mac_dev->stop(mac_dev); - if (err < 0) - netif_err(priv, ifdown, net_dev, "mac_dev->stop() = %d\n", - err); + if (mac_dev->phy_dev) + phy_stop(mac_dev->phy_dev); + mac_dev->disable(mac_dev->fman_mac); for (i = 0; i < ARRAY_SIZE(mac_dev->port); i++) { error = fman_port_disable(mac_dev->port[i]); @@ -826,10 +831,10 @@ static int dpaa_eth_cgr_init(struct dpaa_priv *priv) initcgr.we_mask = cpu_to_be16(QM_CGR_WE_CSCN_EN | QM_CGR_WE_CS_THRES); initcgr.cgr.cscn_en = QM_CGR_EN; - /* Set different thresholds based on the MAC speed. - * This may turn suboptimal if the MAC is reconfigured at a speed - * lower than its max, e.g. if a dTSEC later negotiates a 100Mbps link. - * In such cases, we ought to reconfigure the threshold, too. + /* Set different thresholds based on the configured MAC speed. + * This may turn suboptimal if the MAC is reconfigured at another + * speed, so MACs must call dpaa_eth_cgr_set_speed in their adjust_link + * callback. */ if (priv->mac_dev->if_support & SUPPORTED_10000baseT_Full) cs_th = DPAA_CS_THRESHOLD_10G; @@ -858,6 +863,31 @@ out_error: return err; } +static void dpaa_eth_cgr_set_speed(struct mac_device *mac_dev, int speed) +{ + struct net_device *net_dev = mac_dev->net_dev; + struct dpaa_priv *priv = netdev_priv(net_dev); + struct qm_mcc_initcgr opts = { }; + u32 cs_th; + int err; + + opts.we_mask = cpu_to_be16(QM_CGR_WE_CS_THRES); + switch (speed) { + case SPEED_10000: + cs_th = DPAA_CS_THRESHOLD_10G; + break; + case SPEED_1000: + default: + cs_th = DPAA_CS_THRESHOLD_1G; + break; + } + qm_cgr_cs_thres_set64(&opts.cgr.cs_thres, cs_th, 1); + + err = qman_update_cgr_safe(&priv->cgr_data.cgr, &opts); + if (err) + netdev_err(net_dev, "could not update speed: %d\n", err); +} + static inline void dpaa_setup_ingress(const struct dpaa_priv *priv, struct dpaa_fq *fq, const struct qman_fq *template) @@ -2946,11 +2976,12 @@ static int dpaa_open(struct net_device *net_dev) goto mac_start_failed; } - err = priv->mac_dev->start(mac_dev); + err = priv->mac_dev->enable(mac_dev->fman_mac); if (err < 0) { - netif_err(priv, ifup, net_dev, "mac_dev->start() = %d\n", err); + netif_err(priv, ifup, net_dev, "mac_dev->enable() = %d\n", err); goto mac_start_failed; } + phy_start(priv->mac_dev->phy_dev); netif_tx_start_all_queues(net_dev); @@ -3152,8 +3183,7 @@ static int dpaa_napi_add(struct net_device *net_dev) for_each_possible_cpu(cpu) { percpu_priv = per_cpu_ptr(priv->percpu_priv, cpu); - netif_napi_add(net_dev, &percpu_priv->np.napi, - dpaa_eth_poll, NAPI_POLL_WEIGHT); + netif_napi_add(net_dev, &percpu_priv->np.napi, dpaa_eth_poll); } return 0; diff --git a/drivers/net/ethernet/freescale/dpaa/dpaa_eth_sysfs.c b/drivers/net/ethernet/freescale/dpaa/dpaa_eth_sysfs.c index 4fee74c024bd..258eb6c8f4c0 100644 --- a/drivers/net/ethernet/freescale/dpaa/dpaa_eth_sysfs.c +++ b/drivers/net/ethernet/freescale/dpaa/dpaa_eth_sysfs.c @@ -18,7 +18,7 @@ static ssize_t dpaa_eth_show_addr(struct device *dev, if (mac_dev) return sprintf(buf, "%llx", - (unsigned long long)mac_dev->res->start); + (unsigned long long)mac_dev->vaddr); else return sprintf(buf, "none"); } diff --git a/drivers/net/ethernet/freescale/dpaa/dpaa_ethtool.c b/drivers/net/ethernet/freescale/dpaa/dpaa_ethtool.c index 73f07881ce2d..769e936a263c 100644 --- a/drivers/net/ethernet/freescale/dpaa/dpaa_ethtool.c +++ b/drivers/net/ethernet/freescale/dpaa/dpaa_ethtool.c @@ -80,9 +80,9 @@ static int dpaa_set_link_ksettings(struct net_device *net_dev, static void dpaa_get_drvinfo(struct net_device *net_dev, struct ethtool_drvinfo *drvinfo) { - strlcpy(drvinfo->driver, KBUILD_MODNAME, + strscpy(drvinfo->driver, KBUILD_MODNAME, sizeof(drvinfo->driver)); - strlcpy(drvinfo->bus_info, dev_name(net_dev->dev.parent->parent), + strscpy(drvinfo->bus_info, dev_name(net_dev->dev.parent->parent), sizeof(drvinfo->bus_info)); } diff --git a/drivers/net/ethernet/freescale/dpaa2/dpaa2-eth.c b/drivers/net/ethernet/freescale/dpaa2/dpaa2-eth.c index 75d51572693d..8d029addddad 100644 --- a/drivers/net/ethernet/freescale/dpaa2/dpaa2-eth.c +++ b/drivers/net/ethernet/freescale/dpaa2/dpaa2-eth.c @@ -4565,8 +4565,7 @@ static void dpaa2_eth_add_ch_napi(struct dpaa2_eth_priv *priv) for (i = 0; i < priv->num_channels; i++) { ch = priv->channel[i]; /* NAPI weight *MUST* be a multiple of DPAA2_ETH_STORE_SIZE */ - netif_napi_add(priv->net_dev, &ch->napi, dpaa2_eth_poll, - NAPI_POLL_WEIGHT); + netif_napi_add(priv->net_dev, &ch->napi, dpaa2_eth_poll); } } diff --git a/drivers/net/ethernet/freescale/dpaa2/dpaa2-mac.c b/drivers/net/ethernet/freescale/dpaa2/dpaa2-mac.c index c9bee9a0c9b2..49ff85633783 100644 --- a/drivers/net/ethernet/freescale/dpaa2/dpaa2-mac.c +++ b/drivers/net/ethernet/freescale/dpaa2/dpaa2-mac.c @@ -549,7 +549,7 @@ void dpaa2_mac_get_strings(u8 *data) int i; for (i = 0; i < DPAA2_MAC_NUM_STATS; i++) { - strlcpy(p, dpaa2_mac_ethtool_stats[i], ETH_GSTRING_LEN); + strscpy(p, dpaa2_mac_ethtool_stats[i], ETH_GSTRING_LEN); p += ETH_GSTRING_LEN; } } diff --git a/drivers/net/ethernet/freescale/dpaa2/dpaa2-switch.c b/drivers/net/ethernet/freescale/dpaa2/dpaa2-switch.c index e507e9065214..2b5909fa93cf 100644 --- a/drivers/net/ethernet/freescale/dpaa2/dpaa2-switch.c +++ b/drivers/net/ethernet/freescale/dpaa2/dpaa2-switch.c @@ -3373,9 +3373,8 @@ static int dpaa2_switch_probe(struct fsl_mc_device *sw_dev) * different queues for each switch ports. */ for (i = 0; i < DPAA2_SWITCH_RX_NUM_FQS; i++) - netif_napi_add(ethsw->ports[0]->netdev, - ðsw->fq[i].napi, dpaa2_switch_poll, - NAPI_POLL_WEIGHT); + netif_napi_add(ethsw->ports[0]->netdev, ðsw->fq[i].napi, + dpaa2_switch_poll); /* Setup IRQs */ err = dpaa2_switch_setup_irqs(sw_dev); diff --git a/drivers/net/ethernet/freescale/enetc/enetc.c b/drivers/net/ethernet/freescale/enetc/enetc.c index 9f5b921039bd..54bc92fc6bf0 100644 --- a/drivers/net/ethernet/freescale/enetc/enetc.c +++ b/drivers/net/ethernet/freescale/enetc/enetc.c @@ -2116,13 +2116,14 @@ static void enetc_setup_rxbdr(struct enetc_hw *hw, struct enetc_bdr *rx_ring) static void enetc_setup_bdrs(struct enetc_ndev_priv *priv) { + struct enetc_hw *hw = &priv->si->hw; int i; for (i = 0; i < priv->num_tx_rings; i++) - enetc_setup_txbdr(&priv->si->hw, priv->tx_ring[i]); + enetc_setup_txbdr(hw, priv->tx_ring[i]); for (i = 0; i < priv->num_rx_rings; i++) - enetc_setup_rxbdr(&priv->si->hw, priv->rx_ring[i]); + enetc_setup_rxbdr(hw, priv->rx_ring[i]); } static void enetc_clear_rxbdr(struct enetc_hw *hw, struct enetc_bdr *rx_ring) @@ -2155,13 +2156,14 @@ static void enetc_clear_txbdr(struct enetc_hw *hw, struct enetc_bdr *tx_ring) static void enetc_clear_bdrs(struct enetc_ndev_priv *priv) { + struct enetc_hw *hw = &priv->si->hw; int i; for (i = 0; i < priv->num_tx_rings; i++) - enetc_clear_txbdr(&priv->si->hw, priv->tx_ring[i]); + enetc_clear_txbdr(hw, priv->tx_ring[i]); for (i = 0; i < priv->num_rx_rings; i++) - enetc_clear_rxbdr(&priv->si->hw, priv->rx_ring[i]); + enetc_clear_rxbdr(hw, priv->rx_ring[i]); udelay(1); } @@ -2169,13 +2171,13 @@ static void enetc_clear_bdrs(struct enetc_ndev_priv *priv) static int enetc_setup_irqs(struct enetc_ndev_priv *priv) { struct pci_dev *pdev = priv->si->pdev; + struct enetc_hw *hw = &priv->si->hw; int i, j, err; for (i = 0; i < priv->bdr_int_num; i++) { int irq = pci_irq_vector(pdev, ENETC_BDR_INT_BASE_IDX + i); struct enetc_int_vector *v = priv->int_vector[i]; int entry = ENETC_BDR_INT_BASE_IDX + i; - struct enetc_hw *hw = &priv->si->hw; snprintf(v->name, sizeof(v->name), "%s-rxtx%d", priv->ndev->name, i); @@ -2263,13 +2265,14 @@ static void enetc_setup_interrupts(struct enetc_ndev_priv *priv) static void enetc_clear_interrupts(struct enetc_ndev_priv *priv) { + struct enetc_hw *hw = &priv->si->hw; int i; for (i = 0; i < priv->num_tx_rings; i++) - enetc_txbdr_wr(&priv->si->hw, i, ENETC_TBIER, 0); + enetc_txbdr_wr(hw, i, ENETC_TBIER, 0); for (i = 0; i < priv->num_rx_rings; i++) - enetc_rxbdr_wr(&priv->si->hw, i, ENETC_RBIER, 0); + enetc_rxbdr_wr(hw, i, ENETC_RBIER, 0); } static int enetc_phylink_connect(struct net_device *ndev) @@ -2436,6 +2439,7 @@ int enetc_setup_tc_mqprio(struct net_device *ndev, void *type_data) { struct enetc_ndev_priv *priv = netdev_priv(ndev); struct tc_mqprio_qopt *mqprio = type_data; + struct enetc_hw *hw = &priv->si->hw; struct enetc_bdr *tx_ring; int num_stack_tx_queues; u8 num_tc; @@ -2452,7 +2456,7 @@ int enetc_setup_tc_mqprio(struct net_device *ndev, void *type_data) /* Reset all ring priorities to 0 */ for (i = 0; i < priv->num_tx_rings; i++) { tx_ring = priv->tx_ring[i]; - enetc_set_bdr_prio(&priv->si->hw, tx_ring->index, 0); + enetc_set_bdr_prio(hw, tx_ring->index, 0); } return 0; @@ -2471,7 +2475,7 @@ int enetc_setup_tc_mqprio(struct net_device *ndev, void *type_data) */ for (i = 0; i < num_tc; i++) { tx_ring = priv->tx_ring[i]; - enetc_set_bdr_prio(&priv->si->hw, tx_ring->index, i); + enetc_set_bdr_prio(hw, tx_ring->index, i); } /* Reset the number of netdev queues based on the TC count */ @@ -2584,19 +2588,21 @@ static int enetc_set_rss(struct net_device *ndev, int en) static void enetc_enable_rxvlan(struct net_device *ndev, bool en) { struct enetc_ndev_priv *priv = netdev_priv(ndev); + struct enetc_hw *hw = &priv->si->hw; int i; for (i = 0; i < priv->num_rx_rings; i++) - enetc_bdr_enable_rxvlan(&priv->si->hw, i, en); + enetc_bdr_enable_rxvlan(hw, i, en); } static void enetc_enable_txvlan(struct net_device *ndev, bool en) { struct enetc_ndev_priv *priv = netdev_priv(ndev); + struct enetc_hw *hw = &priv->si->hw; int i; for (i = 0; i < priv->num_tx_rings; i++) - enetc_bdr_enable_txvlan(&priv->si->hw, i, en); + enetc_bdr_enable_txvlan(hw, i, en); } void enetc_set_features(struct net_device *ndev, netdev_features_t features) @@ -2759,8 +2765,7 @@ int enetc_alloc_msix(struct enetc_ndev_priv *priv) v->rx_dim_en = true; } INIT_WORK(&v->rx_dim.work, enetc_rx_dim_work); - netif_napi_add(priv->ndev, &v->napi, enetc_poll, - NAPI_POLL_WEIGHT); + netif_napi_add(priv->ndev, &v->napi, enetc_poll); v->count_tx_rings = v_tx_rings; for (j = 0; j < v_tx_rings; j++) { diff --git a/drivers/net/ethernet/freescale/enetc/enetc.h b/drivers/net/ethernet/freescale/enetc/enetc.h index 2cfe6944ebd3..161930a65f61 100644 --- a/drivers/net/ethernet/freescale/enetc/enetc.h +++ b/drivers/net/ethernet/freescale/enetc/enetc.h @@ -453,7 +453,11 @@ static inline void enetc_cbd_free_data_mem(struct enetc_si *si, int size, data, *dma); } +void enetc_reset_ptcmsdur(struct enetc_hw *hw); +void enetc_set_ptcmsdur(struct enetc_hw *hw, u32 *queue_max_sdu); + #ifdef CONFIG_FSL_ENETC_QOS +int enetc_qos_query_caps(struct net_device *ndev, void *type_data); int enetc_setup_tc_taprio(struct net_device *ndev, void *type_data); void enetc_sched_speed_set(struct enetc_ndev_priv *priv, int speed); int enetc_setup_tc_cbs(struct net_device *ndev, void *type_data); @@ -467,19 +471,20 @@ int enetc_set_psfp(struct net_device *ndev, bool en); static inline void enetc_get_max_cap(struct enetc_ndev_priv *priv) { + struct enetc_hw *hw = &priv->si->hw; u32 reg; - reg = enetc_port_rd(&priv->si->hw, ENETC_PSIDCAPR); + reg = enetc_port_rd(hw, ENETC_PSIDCAPR); priv->psfp_cap.max_streamid = reg & ENETC_PSIDCAPR_MSK; /* Port stream filter capability */ - reg = enetc_port_rd(&priv->si->hw, ENETC_PSFCAPR); + reg = enetc_port_rd(hw, ENETC_PSFCAPR); priv->psfp_cap.max_psfp_filter = reg & ENETC_PSFCAPR_MSK; /* Port stream gate capability */ - reg = enetc_port_rd(&priv->si->hw, ENETC_PSGCAPR); + reg = enetc_port_rd(hw, ENETC_PSGCAPR); priv->psfp_cap.max_psfp_gate = (reg & ENETC_PSGCAPR_SGIT_MSK); priv->psfp_cap.max_psfp_gatelist = (reg & ENETC_PSGCAPR_GCL_MSK) >> 16; /* Port flow meter capability */ - reg = enetc_port_rd(&priv->si->hw, ENETC_PFMCAPR); + reg = enetc_port_rd(hw, ENETC_PFMCAPR); priv->psfp_cap.max_psfp_meter = reg & ENETC_PFMCAPR_MSK; } @@ -520,6 +525,7 @@ static inline int enetc_psfp_disable(struct enetc_ndev_priv *priv) } #else +#define enetc_qos_query_caps(ndev, type_data) -EOPNOTSUPP #define enetc_setup_tc_taprio(ndev, type_data) -EOPNOTSUPP #define enetc_sched_speed_set(priv, speed) (void)0 #define enetc_setup_tc_cbs(ndev, type_data) -EOPNOTSUPP diff --git a/drivers/net/ethernet/freescale/enetc/enetc_ethtool.c b/drivers/net/ethernet/freescale/enetc/enetc_ethtool.c index ff872e40ce85..c8369e3752b0 100644 --- a/drivers/net/ethernet/freescale/enetc/enetc_ethtool.c +++ b/drivers/net/ethernet/freescale/enetc/enetc_ethtool.c @@ -125,68 +125,68 @@ static const struct { int reg; char name[ETH_GSTRING_LEN]; } enetc_port_counters[] = { - { ENETC_PM0_REOCT, "MAC rx ethernet octets" }, - { ENETC_PM0_RALN, "MAC rx alignment errors" }, - { ENETC_PM0_RXPF, "MAC rx valid pause frames" }, - { ENETC_PM0_RFRM, "MAC rx valid frames" }, - { ENETC_PM0_RFCS, "MAC rx fcs errors" }, - { ENETC_PM0_RVLAN, "MAC rx VLAN frames" }, - { ENETC_PM0_RERR, "MAC rx frame errors" }, - { ENETC_PM0_RUCA, "MAC rx unicast frames" }, - { ENETC_PM0_RMCA, "MAC rx multicast frames" }, - { ENETC_PM0_RBCA, "MAC rx broadcast frames" }, - { ENETC_PM0_RDRP, "MAC rx dropped packets" }, - { ENETC_PM0_RPKT, "MAC rx packets" }, - { ENETC_PM0_RUND, "MAC rx undersized packets" }, - { ENETC_PM0_R64, "MAC rx 64 byte packets" }, - { ENETC_PM0_R127, "MAC rx 65-127 byte packets" }, - { ENETC_PM0_R255, "MAC rx 128-255 byte packets" }, - { ENETC_PM0_R511, "MAC rx 256-511 byte packets" }, - { ENETC_PM0_R1023, "MAC rx 512-1023 byte packets" }, - { ENETC_PM0_R1522, "MAC rx 1024-1522 byte packets" }, - { ENETC_PM0_R1523X, "MAC rx 1523 to max-octet packets" }, - { ENETC_PM0_ROVR, "MAC rx oversized packets" }, - { ENETC_PM0_RJBR, "MAC rx jabber packets" }, - { ENETC_PM0_RFRG, "MAC rx fragment packets" }, - { ENETC_PM0_RCNP, "MAC rx control packets" }, - { ENETC_PM0_RDRNTP, "MAC rx fifo drop" }, - { ENETC_PM0_TEOCT, "MAC tx ethernet octets" }, - { ENETC_PM0_TOCT, "MAC tx octets" }, - { ENETC_PM0_TCRSE, "MAC tx carrier sense errors" }, - { ENETC_PM0_TXPF, "MAC tx valid pause frames" }, - { ENETC_PM0_TFRM, "MAC tx frames" }, - { ENETC_PM0_TFCS, "MAC tx fcs errors" }, - { ENETC_PM0_TVLAN, "MAC tx VLAN frames" }, - { ENETC_PM0_TERR, "MAC tx frame errors" }, - { ENETC_PM0_TUCA, "MAC tx unicast frames" }, - { ENETC_PM0_TMCA, "MAC tx multicast frames" }, - { ENETC_PM0_TBCA, "MAC tx broadcast frames" }, - { ENETC_PM0_TPKT, "MAC tx packets" }, - { ENETC_PM0_TUND, "MAC tx undersized packets" }, - { ENETC_PM0_T64, "MAC tx 64 byte packets" }, - { ENETC_PM0_T127, "MAC tx 65-127 byte packets" }, - { ENETC_PM0_T255, "MAC tx 128-255 byte packets" }, - { ENETC_PM0_T511, "MAC tx 256-511 byte packets" }, - { ENETC_PM0_T1023, "MAC tx 512-1023 byte packets" }, - { ENETC_PM0_T1522, "MAC tx 1024-1522 byte packets" }, - { ENETC_PM0_T1523X, "MAC tx 1523 to max-octet packets" }, - { ENETC_PM0_TCNP, "MAC tx control packets" }, - { ENETC_PM0_TDFR, "MAC tx deferred packets" }, - { ENETC_PM0_TMCOL, "MAC tx multiple collisions" }, - { ENETC_PM0_TSCOL, "MAC tx single collisions" }, - { ENETC_PM0_TLCOL, "MAC tx late collisions" }, - { ENETC_PM0_TECOL, "MAC tx excessive collisions" }, - { ENETC_UFDMF, "SI MAC nomatch u-cast discards" }, - { ENETC_MFDMF, "SI MAC nomatch m-cast discards" }, - { ENETC_PBFDSIR, "SI MAC nomatch b-cast discards" }, - { ENETC_PUFDVFR, "SI VLAN nomatch u-cast discards" }, - { ENETC_PMFDVFR, "SI VLAN nomatch m-cast discards" }, - { ENETC_PBFDVFR, "SI VLAN nomatch b-cast discards" }, - { ENETC_PFDMSAPR, "SI pruning discarded frames" }, - { ENETC_PICDR(0), "ICM DR0 discarded frames" }, - { ENETC_PICDR(1), "ICM DR1 discarded frames" }, - { ENETC_PICDR(2), "ICM DR2 discarded frames" }, - { ENETC_PICDR(3), "ICM DR3 discarded frames" }, + { ENETC_PM_REOCT(0), "MAC rx ethernet octets" }, + { ENETC_PM_RALN(0), "MAC rx alignment errors" }, + { ENETC_PM_RXPF(0), "MAC rx valid pause frames" }, + { ENETC_PM_RFRM(0), "MAC rx valid frames" }, + { ENETC_PM_RFCS(0), "MAC rx fcs errors" }, + { ENETC_PM_RVLAN(0), "MAC rx VLAN frames" }, + { ENETC_PM_RERR(0), "MAC rx frame errors" }, + { ENETC_PM_RUCA(0), "MAC rx unicast frames" }, + { ENETC_PM_RMCA(0), "MAC rx multicast frames" }, + { ENETC_PM_RBCA(0), "MAC rx broadcast frames" }, + { ENETC_PM_RDRP(0), "MAC rx dropped packets" }, + { ENETC_PM_RPKT(0), "MAC rx packets" }, + { ENETC_PM_RUND(0), "MAC rx undersized packets" }, + { ENETC_PM_R64(0), "MAC rx 64 byte packets" }, + { ENETC_PM_R127(0), "MAC rx 65-127 byte packets" }, + { ENETC_PM_R255(0), "MAC rx 128-255 byte packets" }, + { ENETC_PM_R511(0), "MAC rx 256-511 byte packets" }, + { ENETC_PM_R1023(0), "MAC rx 512-1023 byte packets" }, + { ENETC_PM_R1522(0), "MAC rx 1024-1522 byte packets" }, + { ENETC_PM_R1523X(0), "MAC rx 1523 to max-octet packets" }, + { ENETC_PM_ROVR(0), "MAC rx oversized packets" }, + { ENETC_PM_RJBR(0), "MAC rx jabber packets" }, + { ENETC_PM_RFRG(0), "MAC rx fragment packets" }, + { ENETC_PM_RCNP(0), "MAC rx control packets" }, + { ENETC_PM_RDRNTP(0), "MAC rx fifo drop" }, + { ENETC_PM_TEOCT(0), "MAC tx ethernet octets" }, + { ENETC_PM_TOCT(0), "MAC tx octets" }, + { ENETC_PM_TCRSE(0), "MAC tx carrier sense errors" }, + { ENETC_PM_TXPF(0), "MAC tx valid pause frames" }, + { ENETC_PM_TFRM(0), "MAC tx frames" }, + { ENETC_PM_TFCS(0), "MAC tx fcs errors" }, + { ENETC_PM_TVLAN(0), "MAC tx VLAN frames" }, + { ENETC_PM_TERR(0), "MAC tx frame errors" }, + { ENETC_PM_TUCA(0), "MAC tx unicast frames" }, + { ENETC_PM_TMCA(0), "MAC tx multicast frames" }, + { ENETC_PM_TBCA(0), "MAC tx broadcast frames" }, + { ENETC_PM_TPKT(0), "MAC tx packets" }, + { ENETC_PM_TUND(0), "MAC tx undersized packets" }, + { ENETC_PM_T64(0), "MAC tx 64 byte packets" }, + { ENETC_PM_T127(0), "MAC tx 65-127 byte packets" }, + { ENETC_PM_T255(0), "MAC tx 128-255 byte packets" }, + { ENETC_PM_T511(0), "MAC tx 256-511 byte packets" }, + { ENETC_PM_T1023(0), "MAC tx 512-1023 byte packets" }, + { ENETC_PM_T1522(0), "MAC tx 1024-1522 byte packets" }, + { ENETC_PM_T1523X(0), "MAC tx 1523 to max-octet packets" }, + { ENETC_PM_TCNP(0), "MAC tx control packets" }, + { ENETC_PM_TDFR(0), "MAC tx deferred packets" }, + { ENETC_PM_TMCOL(0), "MAC tx multiple collisions" }, + { ENETC_PM_TSCOL(0), "MAC tx single collisions" }, + { ENETC_PM_TLCOL(0), "MAC tx late collisions" }, + { ENETC_PM_TECOL(0), "MAC tx excessive collisions" }, + { ENETC_UFDMF, "SI MAC nomatch u-cast discards" }, + { ENETC_MFDMF, "SI MAC nomatch m-cast discards" }, + { ENETC_PBFDSIR, "SI MAC nomatch b-cast discards" }, + { ENETC_PUFDVFR, "SI VLAN nomatch u-cast discards" }, + { ENETC_PMFDVFR, "SI VLAN nomatch m-cast discards" }, + { ENETC_PBFDVFR, "SI VLAN nomatch b-cast discards" }, + { ENETC_PFDMSAPR, "SI pruning discarded frames" }, + { ENETC_PICDR(0), "ICM DR0 discarded frames" }, + { ENETC_PICDR(1), "ICM DR1 discarded frames" }, + { ENETC_PICDR(2), "ICM DR2 discarded frames" }, + { ENETC_PICDR(3), "ICM DR3 discarded frames" }, }; static const char rx_ring_stats[][ETH_GSTRING_LEN] = { @@ -236,7 +236,7 @@ static void enetc_get_strings(struct net_device *ndev, u32 stringset, u8 *data) switch (stringset) { case ETH_SS_STATS: for (i = 0; i < ARRAY_SIZE(enetc_si_counters); i++) { - strlcpy(p, enetc_si_counters[i].name, ETH_GSTRING_LEN); + strscpy(p, enetc_si_counters[i].name, ETH_GSTRING_LEN); p += ETH_GSTRING_LEN; } for (i = 0; i < priv->num_tx_rings; i++) { @@ -258,7 +258,7 @@ static void enetc_get_strings(struct net_device *ndev, u32 stringset, u8 *data) break; for (i = 0; i < ARRAY_SIZE(enetc_port_counters); i++) { - strlcpy(p, enetc_port_counters[i].name, + strscpy(p, enetc_port_counters[i].name, ETH_GSTRING_LEN); p += ETH_GSTRING_LEN; } @@ -301,6 +301,113 @@ static void enetc_get_ethtool_stats(struct net_device *ndev, data[o++] = enetc_port_rd(hw, enetc_port_counters[i].reg); } +static void enetc_get_pause_stats(struct net_device *ndev, + struct ethtool_pause_stats *pause_stats) +{ + struct enetc_ndev_priv *priv = netdev_priv(ndev); + struct enetc_hw *hw = &priv->si->hw; + + pause_stats->tx_pause_frames = enetc_port_rd(hw, ENETC_PM_TXPF(0)); + pause_stats->rx_pause_frames = enetc_port_rd(hw, ENETC_PM_RXPF(0)); +} + +static void enetc_mac_stats(struct enetc_hw *hw, int mac, + struct ethtool_eth_mac_stats *s) +{ + s->FramesTransmittedOK = enetc_port_rd(hw, ENETC_PM_TFRM(mac)); + s->SingleCollisionFrames = enetc_port_rd(hw, ENETC_PM_TSCOL(mac)); + s->MultipleCollisionFrames = enetc_port_rd(hw, ENETC_PM_TMCOL(mac)); + s->FramesReceivedOK = enetc_port_rd(hw, ENETC_PM_RFRM(mac)); + s->FrameCheckSequenceErrors = enetc_port_rd(hw, ENETC_PM_RFCS(mac)); + s->AlignmentErrors = enetc_port_rd(hw, ENETC_PM_RALN(mac)); + s->OctetsTransmittedOK = enetc_port_rd(hw, ENETC_PM_TEOCT(mac)); + s->FramesWithDeferredXmissions = enetc_port_rd(hw, ENETC_PM_TDFR(mac)); + s->LateCollisions = enetc_port_rd(hw, ENETC_PM_TLCOL(mac)); + s->FramesAbortedDueToXSColls = enetc_port_rd(hw, ENETC_PM_TECOL(mac)); + s->FramesLostDueToIntMACXmitError = enetc_port_rd(hw, ENETC_PM_TERR(mac)); + s->CarrierSenseErrors = enetc_port_rd(hw, ENETC_PM_TCRSE(mac)); + s->OctetsReceivedOK = enetc_port_rd(hw, ENETC_PM_REOCT(mac)); + s->FramesLostDueToIntMACRcvError = enetc_port_rd(hw, ENETC_PM_RDRNTP(mac)); + s->MulticastFramesXmittedOK = enetc_port_rd(hw, ENETC_PM_TMCA(mac)); + s->BroadcastFramesXmittedOK = enetc_port_rd(hw, ENETC_PM_TBCA(mac)); + s->MulticastFramesReceivedOK = enetc_port_rd(hw, ENETC_PM_RMCA(mac)); + s->BroadcastFramesReceivedOK = enetc_port_rd(hw, ENETC_PM_RBCA(mac)); +} + +static void enetc_ctrl_stats(struct enetc_hw *hw, int mac, + struct ethtool_eth_ctrl_stats *s) +{ + s->MACControlFramesTransmitted = enetc_port_rd(hw, ENETC_PM_TCNP(mac)); + s->MACControlFramesReceived = enetc_port_rd(hw, ENETC_PM_RCNP(mac)); +} + +static const struct ethtool_rmon_hist_range enetc_rmon_ranges[] = { + { 64, 64 }, + { 65, 127 }, + { 128, 255 }, + { 256, 511 }, + { 512, 1023 }, + { 1024, 1522 }, + { 1523, ENETC_MAC_MAXFRM_SIZE }, + {}, +}; + +static void enetc_rmon_stats(struct enetc_hw *hw, int mac, + struct ethtool_rmon_stats *s, + const struct ethtool_rmon_hist_range **ranges) +{ + s->undersize_pkts = enetc_port_rd(hw, ENETC_PM_RUND(mac)); + s->oversize_pkts = enetc_port_rd(hw, ENETC_PM_ROVR(mac)); + s->fragments = enetc_port_rd(hw, ENETC_PM_RFRG(mac)); + s->jabbers = enetc_port_rd(hw, ENETC_PM_RJBR(mac)); + + s->hist[0] = enetc_port_rd(hw, ENETC_PM_R64(mac)); + s->hist[1] = enetc_port_rd(hw, ENETC_PM_R127(mac)); + s->hist[2] = enetc_port_rd(hw, ENETC_PM_R255(mac)); + s->hist[3] = enetc_port_rd(hw, ENETC_PM_R511(mac)); + s->hist[4] = enetc_port_rd(hw, ENETC_PM_R1023(mac)); + s->hist[5] = enetc_port_rd(hw, ENETC_PM_R1522(mac)); + s->hist[6] = enetc_port_rd(hw, ENETC_PM_R1523X(mac)); + + s->hist_tx[0] = enetc_port_rd(hw, ENETC_PM_T64(mac)); + s->hist_tx[1] = enetc_port_rd(hw, ENETC_PM_T127(mac)); + s->hist_tx[2] = enetc_port_rd(hw, ENETC_PM_T255(mac)); + s->hist_tx[3] = enetc_port_rd(hw, ENETC_PM_T511(mac)); + s->hist_tx[4] = enetc_port_rd(hw, ENETC_PM_T1023(mac)); + s->hist_tx[5] = enetc_port_rd(hw, ENETC_PM_T1522(mac)); + s->hist_tx[6] = enetc_port_rd(hw, ENETC_PM_T1523X(mac)); + + *ranges = enetc_rmon_ranges; +} + +static void enetc_get_eth_mac_stats(struct net_device *ndev, + struct ethtool_eth_mac_stats *mac_stats) +{ + struct enetc_ndev_priv *priv = netdev_priv(ndev); + struct enetc_hw *hw = &priv->si->hw; + + enetc_mac_stats(hw, 0, mac_stats); +} + +static void enetc_get_eth_ctrl_stats(struct net_device *ndev, + struct ethtool_eth_ctrl_stats *ctrl_stats) +{ + struct enetc_ndev_priv *priv = netdev_priv(ndev); + struct enetc_hw *hw = &priv->si->hw; + + enetc_ctrl_stats(hw, 0, ctrl_stats); +} + +static void enetc_get_rmon_stats(struct net_device *ndev, + struct ethtool_rmon_stats *rmon_stats, + const struct ethtool_rmon_hist_range **ranges) +{ + struct enetc_ndev_priv *priv = netdev_priv(ndev); + struct enetc_hw *hw = &priv->si->hw; + + enetc_rmon_stats(hw, 0, rmon_stats, ranges); +} + #define ENETC_RSSHASH_L3 (RXH_L2DA | RXH_VLAN | RXH_L3_PROTO | RXH_IP_SRC | \ RXH_IP_DST) #define ENETC_RSSHASH_L4 (ENETC_RSSHASH_L3 | RXH_L4_B_0_1 | RXH_L4_B_2_3) @@ -766,6 +873,10 @@ static const struct ethtool_ops enetc_pf_ethtool_ops = { .get_sset_count = enetc_get_sset_count, .get_strings = enetc_get_strings, .get_ethtool_stats = enetc_get_ethtool_stats, + .get_pause_stats = enetc_get_pause_stats, + .get_rmon_stats = enetc_get_rmon_stats, + .get_eth_ctrl_stats = enetc_get_eth_ctrl_stats, + .get_eth_mac_stats = enetc_get_eth_mac_stats, .get_rxnfc = enetc_get_rxnfc, .set_rxnfc = enetc_set_rxnfc, .get_rxfh_key_size = enetc_get_rxfh_key_size, diff --git a/drivers/net/ethernet/freescale/enetc/enetc_hw.h b/drivers/net/ethernet/freescale/enetc/enetc_hw.h index 647c87f73bf7..18ca1f42b1f7 100644 --- a/drivers/net/ethernet/freescale/enetc/enetc_hw.h +++ b/drivers/net/ethernet/freescale/enetc/enetc_hw.h @@ -276,58 +276,60 @@ enum enetc_bdr_type {TX, RX}; #define ENETC_PFMCAPR 0x1b38 #define ENETC_PFMCAPR_MSK GENMASK(15, 0) -/* MAC counters */ -#define ENETC_PM0_REOCT 0x8100 -#define ENETC_PM0_RALN 0x8110 -#define ENETC_PM0_RXPF 0x8118 -#define ENETC_PM0_RFRM 0x8120 -#define ENETC_PM0_RFCS 0x8128 -#define ENETC_PM0_RVLAN 0x8130 -#define ENETC_PM0_RERR 0x8138 -#define ENETC_PM0_RUCA 0x8140 -#define ENETC_PM0_RMCA 0x8148 -#define ENETC_PM0_RBCA 0x8150 -#define ENETC_PM0_RDRP 0x8158 -#define ENETC_PM0_RPKT 0x8160 -#define ENETC_PM0_RUND 0x8168 -#define ENETC_PM0_R64 0x8170 -#define ENETC_PM0_R127 0x8178 -#define ENETC_PM0_R255 0x8180 -#define ENETC_PM0_R511 0x8188 -#define ENETC_PM0_R1023 0x8190 -#define ENETC_PM0_R1522 0x8198 -#define ENETC_PM0_R1523X 0x81A0 -#define ENETC_PM0_ROVR 0x81A8 -#define ENETC_PM0_RJBR 0x81B0 -#define ENETC_PM0_RFRG 0x81B8 -#define ENETC_PM0_RCNP 0x81C0 -#define ENETC_PM0_RDRNTP 0x81C8 -#define ENETC_PM0_TEOCT 0x8200 -#define ENETC_PM0_TOCT 0x8208 -#define ENETC_PM0_TCRSE 0x8210 -#define ENETC_PM0_TXPF 0x8218 -#define ENETC_PM0_TFRM 0x8220 -#define ENETC_PM0_TFCS 0x8228 -#define ENETC_PM0_TVLAN 0x8230 -#define ENETC_PM0_TERR 0x8238 -#define ENETC_PM0_TUCA 0x8240 -#define ENETC_PM0_TMCA 0x8248 -#define ENETC_PM0_TBCA 0x8250 -#define ENETC_PM0_TPKT 0x8260 -#define ENETC_PM0_TUND 0x8268 -#define ENETC_PM0_T64 0x8270 -#define ENETC_PM0_T127 0x8278 -#define ENETC_PM0_T255 0x8280 -#define ENETC_PM0_T511 0x8288 -#define ENETC_PM0_T1023 0x8290 -#define ENETC_PM0_T1522 0x8298 -#define ENETC_PM0_T1523X 0x82A0 -#define ENETC_PM0_TCNP 0x82C0 -#define ENETC_PM0_TDFR 0x82D0 -#define ENETC_PM0_TMCOL 0x82D8 -#define ENETC_PM0_TSCOL 0x82E0 -#define ENETC_PM0_TLCOL 0x82E8 -#define ENETC_PM0_TECOL 0x82F0 +/* Port MAC counters: Port MAC 0 corresponds to the eMAC and + * Port MAC 1 to the pMAC. + */ +#define ENETC_PM_REOCT(mac) (0x8100 + 0x1000 * (mac)) +#define ENETC_PM_RALN(mac) (0x8110 + 0x1000 * (mac)) +#define ENETC_PM_RXPF(mac) (0x8118 + 0x1000 * (mac)) +#define ENETC_PM_RFRM(mac) (0x8120 + 0x1000 * (mac)) +#define ENETC_PM_RFCS(mac) (0x8128 + 0x1000 * (mac)) +#define ENETC_PM_RVLAN(mac) (0x8130 + 0x1000 * (mac)) +#define ENETC_PM_RERR(mac) (0x8138 + 0x1000 * (mac)) +#define ENETC_PM_RUCA(mac) (0x8140 + 0x1000 * (mac)) +#define ENETC_PM_RMCA(mac) (0x8148 + 0x1000 * (mac)) +#define ENETC_PM_RBCA(mac) (0x8150 + 0x1000 * (mac)) +#define ENETC_PM_RDRP(mac) (0x8158 + 0x1000 * (mac)) +#define ENETC_PM_RPKT(mac) (0x8160 + 0x1000 * (mac)) +#define ENETC_PM_RUND(mac) (0x8168 + 0x1000 * (mac)) +#define ENETC_PM_R64(mac) (0x8170 + 0x1000 * (mac)) +#define ENETC_PM_R127(mac) (0x8178 + 0x1000 * (mac)) +#define ENETC_PM_R255(mac) (0x8180 + 0x1000 * (mac)) +#define ENETC_PM_R511(mac) (0x8188 + 0x1000 * (mac)) +#define ENETC_PM_R1023(mac) (0x8190 + 0x1000 * (mac)) +#define ENETC_PM_R1522(mac) (0x8198 + 0x1000 * (mac)) +#define ENETC_PM_R1523X(mac) (0x81A0 + 0x1000 * (mac)) +#define ENETC_PM_ROVR(mac) (0x81A8 + 0x1000 * (mac)) +#define ENETC_PM_RJBR(mac) (0x81B0 + 0x1000 * (mac)) +#define ENETC_PM_RFRG(mac) (0x81B8 + 0x1000 * (mac)) +#define ENETC_PM_RCNP(mac) (0x81C0 + 0x1000 * (mac)) +#define ENETC_PM_RDRNTP(mac) (0x81C8 + 0x1000 * (mac)) +#define ENETC_PM_TEOCT(mac) (0x8200 + 0x1000 * (mac)) +#define ENETC_PM_TOCT(mac) (0x8208 + 0x1000 * (mac)) +#define ENETC_PM_TCRSE(mac) (0x8210 + 0x1000 * (mac)) +#define ENETC_PM_TXPF(mac) (0x8218 + 0x1000 * (mac)) +#define ENETC_PM_TFRM(mac) (0x8220 + 0x1000 * (mac)) +#define ENETC_PM_TFCS(mac) (0x8228 + 0x1000 * (mac)) +#define ENETC_PM_TVLAN(mac) (0x8230 + 0x1000 * (mac)) +#define ENETC_PM_TERR(mac) (0x8238 + 0x1000 * (mac)) +#define ENETC_PM_TUCA(mac) (0x8240 + 0x1000 * (mac)) +#define ENETC_PM_TMCA(mac) (0x8248 + 0x1000 * (mac)) +#define ENETC_PM_TBCA(mac) (0x8250 + 0x1000 * (mac)) +#define ENETC_PM_TPKT(mac) (0x8260 + 0x1000 * (mac)) +#define ENETC_PM_TUND(mac) (0x8268 + 0x1000 * (mac)) +#define ENETC_PM_T64(mac) (0x8270 + 0x1000 * (mac)) +#define ENETC_PM_T127(mac) (0x8278 + 0x1000 * (mac)) +#define ENETC_PM_T255(mac) (0x8280 + 0x1000 * (mac)) +#define ENETC_PM_T511(mac) (0x8288 + 0x1000 * (mac)) +#define ENETC_PM_T1023(mac) (0x8290 + 0x1000 * (mac)) +#define ENETC_PM_T1522(mac) (0x8298 + 0x1000 * (mac)) +#define ENETC_PM_T1523X(mac) (0x82A0 + 0x1000 * (mac)) +#define ENETC_PM_TCNP(mac) (0x82C0 + 0x1000 * (mac)) +#define ENETC_PM_TDFR(mac) (0x82D0 + 0x1000 * (mac)) +#define ENETC_PM_TMCOL(mac) (0x82D8 + 0x1000 * (mac)) +#define ENETC_PM_TSCOL(mac) (0x82E0 + 0x1000 * (mac)) +#define ENETC_PM_TLCOL(mac) (0x82E8 + 0x1000 * (mac)) +#define ENETC_PM_TECOL(mac) (0x82F0 + 0x1000 * (mac)) /* Port counters */ #define ENETC_PICDR(n) (0x0700 + (n) * 8) /* n = [0..3] */ @@ -943,13 +945,13 @@ static inline u32 enetc_usecs_to_cycles(u32 usecs) } /* port time gating control register */ -#define ENETC_QBV_PTGCR_OFFSET 0x11a00 -#define ENETC_QBV_TGE BIT(31) -#define ENETC_QBV_TGPE BIT(30) +#define ENETC_PTGCR 0x11a00 +#define ENETC_PTGCR_TGE BIT(31) +#define ENETC_PTGCR_TGPE BIT(30) /* Port time gating capability register */ -#define ENETC_QBV_PTGCAPR_OFFSET 0x11a08 -#define ENETC_QBV_MAX_GCL_LEN_MASK GENMASK(15, 0) +#define ENETC_PTGCAPR 0x11a08 +#define ENETC_PTGCAPR_MAX_GCL_LEN_MASK GENMASK(15, 0) /* Port time specific departure */ #define ENETC_PTCTSDR(n) (0x1210 + 4 * (n)) diff --git a/drivers/net/ethernet/freescale/enetc/enetc_pf.c b/drivers/net/ethernet/freescale/enetc/enetc_pf.c index bb7750222691..bdf94335ee99 100644 --- a/drivers/net/ethernet/freescale/enetc/enetc_pf.c +++ b/drivers/net/ethernet/freescale/enetc/enetc_pf.c @@ -516,15 +516,34 @@ static void enetc_port_si_configure(struct enetc_si *si) enetc_port_wr(hw, ENETC_PSIVLANFMR, ENETC_PSIVLANFMR_VS); } -static void enetc_configure_port_mac(struct enetc_hw *hw) +void enetc_set_ptcmsdur(struct enetc_hw *hw, u32 *max_sdu) { int tc; - enetc_port_wr(hw, ENETC_PM0_MAXFRM, - ENETC_SET_MAXFRM(ENETC_RX_MAXFRM_SIZE)); + for (tc = 0; tc < 8; tc++) { + u32 val = ENETC_MAC_MAXFRM_SIZE; + + if (max_sdu[tc]) + val = max_sdu[tc] + VLAN_ETH_HLEN; + + enetc_port_wr(hw, ENETC_PTCMSDUR(tc), val); + } +} + +void enetc_reset_ptcmsdur(struct enetc_hw *hw) +{ + int tc; for (tc = 0; tc < 8; tc++) enetc_port_wr(hw, ENETC_PTCMSDUR(tc), ENETC_MAC_MAXFRM_SIZE); +} + +static void enetc_configure_port_mac(struct enetc_hw *hw) +{ + enetc_port_wr(hw, ENETC_PM0_MAXFRM, + ENETC_SET_MAXFRM(ENETC_RX_MAXFRM_SIZE)); + + enetc_reset_ptcmsdur(hw); enetc_port_wr(hw, ENETC_PM0_CMD_CFG, ENETC_PM0_CMD_PHY_TX_EN | ENETC_PM0_CMD_TXP | ENETC_PM0_PROMISC); @@ -738,6 +757,8 @@ static int enetc_pf_setup_tc(struct net_device *ndev, enum tc_setup_type type, void *type_data) { switch (type) { + case TC_QUERY_CAPS: + return enetc_qos_query_caps(ndev, type_data); case TC_SETUP_QDISC_MQPRIO: return enetc_setup_tc_mqprio(ndev, type_data); case TC_SETUP_QDISC_TAPRIO: diff --git a/drivers/net/ethernet/freescale/enetc/enetc_qos.c b/drivers/net/ethernet/freescale/enetc/enetc_qos.c index f8a2f02ce22d..e6416332ec79 100644 --- a/drivers/net/ethernet/freescale/enetc/enetc_qos.c +++ b/drivers/net/ethernet/freescale/enetc/enetc_qos.c @@ -7,18 +7,19 @@ #include <linux/math64.h> #include <linux/refcount.h> #include <net/pkt_cls.h> +#include <net/pkt_sched.h> #include <net/tc_act/tc_gate.h> static u16 enetc_get_max_gcl_len(struct enetc_hw *hw) { - return enetc_rd(hw, ENETC_QBV_PTGCAPR_OFFSET) - & ENETC_QBV_MAX_GCL_LEN_MASK; + return enetc_rd(hw, ENETC_PTGCAPR) & ENETC_PTGCAPR_MAX_GCL_LEN_MASK; } void enetc_sched_speed_set(struct enetc_ndev_priv *priv, int speed) { + struct enetc_hw *hw = &priv->si->hw; u32 old_speed = priv->speed; - u32 pspeed; + u32 pspeed, tmp; if (speed == old_speed) return; @@ -39,16 +40,15 @@ void enetc_sched_speed_set(struct enetc_ndev_priv *priv, int speed) } priv->speed = speed; - enetc_port_wr(&priv->si->hw, ENETC_PMR, - (enetc_port_rd(&priv->si->hw, ENETC_PMR) - & (~ENETC_PMR_PSPEED_MASK)) - | pspeed); + tmp = enetc_port_rd(hw, ENETC_PMR); + enetc_port_wr(hw, ENETC_PMR, (tmp & ~ENETC_PMR_PSPEED_MASK) | pspeed); } static int enetc_setup_taprio(struct net_device *ndev, struct tc_taprio_qopt_offload *admin_conf) { struct enetc_ndev_priv *priv = netdev_priv(ndev); + struct enetc_hw *hw = &priv->si->hw; struct enetc_cbd cbd = {.cmd = 0}; struct tgs_gcl_conf *gcl_config; struct tgs_gcl_data *gcl_data; @@ -61,15 +61,14 @@ static int enetc_setup_taprio(struct net_device *ndev, int err; int i; - if (admin_conf->num_entries > enetc_get_max_gcl_len(&priv->si->hw)) + if (admin_conf->num_entries > enetc_get_max_gcl_len(hw)) return -EINVAL; gcl_len = admin_conf->num_entries; - tge = enetc_rd(&priv->si->hw, ENETC_QBV_PTGCR_OFFSET); + tge = enetc_rd(hw, ENETC_PTGCR); if (!admin_conf->enable) { - enetc_wr(&priv->si->hw, - ENETC_QBV_PTGCR_OFFSET, - tge & (~ENETC_QBV_TGE)); + enetc_wr(hw, ENETC_PTGCR, tge & ~ENETC_PTGCR_TGE); + enetc_reset_ptcmsdur(hw); priv->active_offloads &= ~ENETC_F_QBV; @@ -117,27 +116,28 @@ static int enetc_setup_taprio(struct net_device *ndev, cbd.cls = BDCR_CMD_PORT_GCL; cbd.status_flags = 0; - enetc_wr(&priv->si->hw, ENETC_QBV_PTGCR_OFFSET, - tge | ENETC_QBV_TGE); + enetc_wr(hw, ENETC_PTGCR, tge | ENETC_PTGCR_TGE); err = enetc_send_cmd(priv->si, &cbd); if (err) - enetc_wr(&priv->si->hw, - ENETC_QBV_PTGCR_OFFSET, - tge & (~ENETC_QBV_TGE)); + enetc_wr(hw, ENETC_PTGCR, tge & ~ENETC_PTGCR_TGE); enetc_cbd_free_data_mem(priv->si, data_size, tmp, &dma); - if (!err) - priv->active_offloads |= ENETC_F_QBV; + if (err) + return err; - return err; + enetc_set_ptcmsdur(hw, admin_conf->max_sdu); + priv->active_offloads |= ENETC_F_QBV; + + return 0; } int enetc_setup_tc_taprio(struct net_device *ndev, void *type_data) { struct tc_taprio_qopt_offload *taprio = type_data; struct enetc_ndev_priv *priv = netdev_priv(ndev); + struct enetc_hw *hw = &priv->si->hw; int err; int i; @@ -147,16 +147,14 @@ int enetc_setup_tc_taprio(struct net_device *ndev, void *type_data) return -EBUSY; for (i = 0; i < priv->num_tx_rings; i++) - enetc_set_bdr_prio(&priv->si->hw, - priv->tx_ring[i]->index, + enetc_set_bdr_prio(hw, priv->tx_ring[i]->index, taprio->enable ? i : 0); err = enetc_setup_taprio(ndev, taprio); if (err) for (i = 0; i < priv->num_tx_rings; i++) - enetc_set_bdr_prio(&priv->si->hw, - priv->tx_ring[i]->index, + enetc_set_bdr_prio(hw, priv->tx_ring[i]->index, taprio->enable ? 0 : i); return err; @@ -178,7 +176,7 @@ int enetc_setup_tc_cbs(struct net_device *ndev, void *type_data) struct tc_cbs_qopt_offload *cbs = type_data; u32 port_transmit_rate = priv->speed; u8 tc_nums = netdev_get_num_tc(ndev); - struct enetc_si *si = priv->si; + struct enetc_hw *hw = &priv->si->hw; u32 hi_credit_bit, hi_credit_reg; u32 max_interference_size; u32 port_frame_max_size; @@ -199,15 +197,15 @@ int enetc_setup_tc_cbs(struct net_device *ndev, void *type_data) * lower than this TC have been disabled. */ if (tc == prio_top && - enetc_get_cbs_enable(&si->hw, prio_next)) { + enetc_get_cbs_enable(hw, prio_next)) { dev_err(&ndev->dev, "Disable TC%d before disable TC%d\n", prio_next, tc); return -EINVAL; } - enetc_port_wr(&si->hw, ENETC_PTCCBSR1(tc), 0); - enetc_port_wr(&si->hw, ENETC_PTCCBSR0(tc), 0); + enetc_port_wr(hw, ENETC_PTCCBSR1(tc), 0); + enetc_port_wr(hw, ENETC_PTCCBSR0(tc), 0); return 0; } @@ -224,13 +222,13 @@ int enetc_setup_tc_cbs(struct net_device *ndev, void *type_data) * higher than this TC have been enabled. */ if (tc == prio_next) { - if (!enetc_get_cbs_enable(&si->hw, prio_top)) { + if (!enetc_get_cbs_enable(hw, prio_top)) { dev_err(&ndev->dev, "Enable TC%d first before enable TC%d\n", prio_top, prio_next); return -EINVAL; } - bw_sum += enetc_get_cbs_bw(&si->hw, prio_top); + bw_sum += enetc_get_cbs_bw(hw, prio_top); } if (bw_sum + bw >= 100) { @@ -239,7 +237,7 @@ int enetc_setup_tc_cbs(struct net_device *ndev, void *type_data) return -EINVAL; } - enetc_port_rd(&si->hw, ENETC_PTCMSDUR(tc)); + enetc_port_rd(hw, ENETC_PTCMSDUR(tc)); /* For top prio TC, the max_interfrence_size is maxSizedFrame. * @@ -259,8 +257,8 @@ int enetc_setup_tc_cbs(struct net_device *ndev, void *type_data) u32 m0, ma, r0, ra; m0 = port_frame_max_size * 8; - ma = enetc_port_rd(&si->hw, ENETC_PTCMSDUR(prio_top)) * 8; - ra = enetc_get_cbs_bw(&si->hw, prio_top) * + ma = enetc_port_rd(hw, ENETC_PTCMSDUR(prio_top)) * 8; + ra = enetc_get_cbs_bw(hw, prio_top) * port_transmit_rate * 10000ULL; r0 = port_transmit_rate * 1000000ULL; max_interference_size = m0 + ma + @@ -280,10 +278,10 @@ int enetc_setup_tc_cbs(struct net_device *ndev, void *type_data) hi_credit_reg = (u32)div_u64((ENETC_CLK * 100ULL) * hi_credit_bit, port_transmit_rate * 1000000ULL); - enetc_port_wr(&si->hw, ENETC_PTCCBSR1(tc), hi_credit_reg); + enetc_port_wr(hw, ENETC_PTCCBSR1(tc), hi_credit_reg); /* Set bw register and enable this traffic class */ - enetc_port_wr(&si->hw, ENETC_PTCCBSR0(tc), bw | ENETC_CBSE); + enetc_port_wr(hw, ENETC_PTCCBSR0(tc), bw | ENETC_CBSE); return 0; } @@ -293,6 +291,7 @@ int enetc_setup_tc_txtime(struct net_device *ndev, void *type_data) struct enetc_ndev_priv *priv = netdev_priv(ndev); struct tc_etf_qopt_offload *qopt = type_data; u8 tc_nums = netdev_get_num_tc(ndev); + struct enetc_hw *hw = &priv->si->hw; int tc; if (!tc_nums) @@ -304,12 +303,11 @@ int enetc_setup_tc_txtime(struct net_device *ndev, void *type_data) return -EINVAL; /* TSD and Qbv are mutually exclusive in hardware */ - if (enetc_rd(&priv->si->hw, ENETC_QBV_PTGCR_OFFSET) & ENETC_QBV_TGE) + if (enetc_rd(hw, ENETC_PTGCR) & ENETC_PTGCR_TGE) return -EBUSY; priv->tx_ring[tc]->tsd_enable = qopt->enable; - enetc_port_wr(&priv->si->hw, ENETC_PTCTSDR(tc), - qopt->enable ? ENETC_TSDE : 0); + enetc_port_wr(hw, ENETC_PTCTSDR(tc), qopt->enable ? ENETC_TSDE : 0); return 0; } @@ -1601,3 +1599,23 @@ int enetc_setup_tc_psfp(struct net_device *ndev, void *type_data) return 0; } + +int enetc_qos_query_caps(struct net_device *ndev, void *type_data) +{ + struct enetc_ndev_priv *priv = netdev_priv(ndev); + struct tc_query_caps_base *base = type_data; + struct enetc_si *si = priv->si; + + switch (base->type) { + case TC_SETUP_QDISC_TAPRIO: { + struct tc_taprio_caps *caps = base->caps; + + if (si->hw_features & ENETC_SI_F_QBV) + caps->supports_queue_max_sdu = true; + + return 0; + } + default: + return -EOPNOTSUPP; + } +} diff --git a/drivers/net/ethernet/freescale/fec.h b/drivers/net/ethernet/freescale/fec.h index a5fed00cb971..33f84a30e167 100644 --- a/drivers/net/ethernet/freescale/fec.h +++ b/drivers/net/ethernet/freescale/fec.h @@ -17,8 +17,11 @@ #include <linux/clocksource.h> #include <linux/net_tstamp.h> #include <linux/pm_qos.h> +#include <linux/bpf.h> #include <linux/ptp_clock_kernel.h> #include <linux/timecounter.h> +#include <dt-bindings/firmware/imx/rsrc.h> +#include <linux/firmware/imx/sci.h> #if defined(CONFIG_M523x) || defined(CONFIG_M527x) || defined(CONFIG_M528x) || \ defined(CONFIG_M520x) || defined(CONFIG_M532x) || defined(CONFIG_ARM) || \ @@ -344,8 +347,11 @@ struct bufdesc_ex { * the skbuffer directly. */ +#define FEC_ENET_XDP_HEADROOM (XDP_PACKET_HEADROOM) + #define FEC_ENET_RX_PAGES 256 -#define FEC_ENET_RX_FRSIZE 2048 +#define FEC_ENET_RX_FRSIZE (PAGE_SIZE - FEC_ENET_XDP_HEADROOM \ + - SKB_DATA_ALIGN(sizeof(struct skb_shared_info))) #define FEC_ENET_RX_FRPPG (PAGE_SIZE / FEC_ENET_RX_FRSIZE) #define RX_RING_SIZE (FEC_ENET_RX_FRPPG * FEC_ENET_RX_PAGES) #define FEC_ENET_TX_FRSIZE 2048 @@ -515,6 +521,12 @@ struct bufdesc_prop { unsigned char dsize_log2; }; +struct fec_enet_priv_txrx_info { + int offset; + struct page *page; + struct sk_buff *skb; +}; + struct fec_enet_priv_tx_q { struct bufdesc_prop bd; unsigned char *tx_bounce[TX_RING_SIZE]; @@ -530,7 +542,14 @@ struct fec_enet_priv_tx_q { struct fec_enet_priv_rx_q { struct bufdesc_prop bd; - struct sk_buff *rx_skbuff[RX_RING_SIZE]; + struct fec_enet_priv_txrx_info rx_skb_info[RX_RING_SIZE]; + + /* page_pool */ + struct page_pool *page_pool; + struct xdp_rxq_info xdp_rxq; + + /* rx queue number, in the range 0-7 */ + u8 id; }; struct fec_stop_mode_gpr { @@ -583,6 +602,7 @@ struct fec_enet_private { struct device_node *phy_node; bool rgmii_txc_dly; bool rgmii_rxc_dly; + bool rpm_active; int link; int full_duplex; int speed; @@ -639,6 +659,8 @@ struct fec_enet_private { int pps_enable; unsigned int next_counter; + struct imx_sc_ipc *ipc_handle; + u64 ethtool_stats[]; }; diff --git a/drivers/net/ethernet/freescale/fec_main.c b/drivers/net/ethernet/freescale/fec_main.c index 92c55e1a5507..98d5cd313fdd 100644 --- a/drivers/net/ethernet/freescale/fec_main.c +++ b/drivers/net/ethernet/freescale/fec_main.c @@ -66,6 +66,8 @@ #include <linux/mfd/syscon.h> #include <linux/regmap.h> #include <soc/imx/cpuidle.h> +#include <linux/filter.h> +#include <linux/bpf.h> #include <asm/cacheflush.h> @@ -156,6 +158,13 @@ static const struct fec_devinfo fec_imx8qm_info = { FEC_QUIRK_DELAYED_CLKS_SUPPORT, }; +static const struct fec_devinfo fec_s32v234_info = { + .quirks = FEC_QUIRK_ENET_MAC | FEC_QUIRK_HAS_GBIT | + FEC_QUIRK_HAS_BUFDESC_EX | FEC_QUIRK_HAS_CSUM | + FEC_QUIRK_HAS_VLAN | FEC_QUIRK_HAS_AVB | + FEC_QUIRK_ERR007885 | FEC_QUIRK_BUG_CAPTURE, +}; + static struct platform_device_id fec_devtype[] = { { /* keep it for coldfire */ @@ -189,6 +198,9 @@ static struct platform_device_id fec_devtype[] = { .name = "imx8qm-fec", .driver_data = (kernel_ulong_t)&fec_imx8qm_info, }, { + .name = "s32v234-fec", + .driver_data = (kernel_ulong_t)&fec_s32v234_info, + }, { /* sentinel */ } }; @@ -204,6 +216,7 @@ enum imx_fec_type { IMX6UL_FEC, IMX8MQ_FEC, IMX8QM_FEC, + S32V234_FEC, }; static const struct of_device_id fec_dt_ids[] = { @@ -216,6 +229,7 @@ static const struct of_device_id fec_dt_ids[] = { { .compatible = "fsl,imx6ul-fec", .data = &fec_devtype[IMX6UL_FEC], }, { .compatible = "fsl,imx8mq-fec", .data = &fec_devtype[IMX8MQ_FEC], }, { .compatible = "fsl,imx8qm-fec", .data = &fec_devtype[IMX8QM_FEC], }, + { .compatible = "fsl,s32v234-fec", .data = &fec_devtype[S32V234_FEC], }, { /* sentinel */ } }; MODULE_DEVICE_TABLE(of, fec_dt_ids); @@ -410,6 +424,48 @@ fec_enet_clear_csum(struct sk_buff *skb, struct net_device *ndev) return 0; } +static int +fec_enet_create_page_pool(struct fec_enet_private *fep, + struct fec_enet_priv_rx_q *rxq, int size) +{ + struct page_pool_params pp_params = { + .order = 0, + .flags = PP_FLAG_DMA_MAP | PP_FLAG_DMA_SYNC_DEV, + .pool_size = size, + .nid = dev_to_node(&fep->pdev->dev), + .dev = &fep->pdev->dev, + .dma_dir = DMA_FROM_DEVICE, + .offset = FEC_ENET_XDP_HEADROOM, + .max_len = FEC_ENET_RX_FRSIZE, + }; + int err; + + rxq->page_pool = page_pool_create(&pp_params); + if (IS_ERR(rxq->page_pool)) { + err = PTR_ERR(rxq->page_pool); + rxq->page_pool = NULL; + return err; + } + + err = xdp_rxq_info_reg(&rxq->xdp_rxq, fep->netdev, rxq->id, 0); + if (err < 0) + goto err_free_pp; + + err = xdp_rxq_info_reg_mem_model(&rxq->xdp_rxq, MEM_TYPE_PAGE_POOL, + rxq->page_pool); + if (err) + goto err_unregister_rxq; + + return 0; + +err_unregister_rxq: + xdp_rxq_info_unreg(&rxq->xdp_rxq); +err_free_pp: + page_pool_destroy(rxq->page_pool); + rxq->page_pool = NULL; + return err; +} + static struct bufdesc * fec_enet_txq_submit_frag_skb(struct fec_enet_priv_tx_q *txq, struct sk_buff *skb, @@ -1168,6 +1224,34 @@ fec_restart(struct net_device *ndev) } +static int fec_enet_ipc_handle_init(struct fec_enet_private *fep) +{ + if (!(of_machine_is_compatible("fsl,imx8qm") || + of_machine_is_compatible("fsl,imx8qxp") || + of_machine_is_compatible("fsl,imx8dxl"))) + return 0; + + return imx_scu_get_handle(&fep->ipc_handle); +} + +static void fec_enet_ipg_stop_set(struct fec_enet_private *fep, bool enabled) +{ + struct device_node *np = fep->pdev->dev.of_node; + u32 rsrc_id, val; + int idx; + + if (!np || !fep->ipc_handle) + return; + + idx = of_alias_get_id(np, "ethernet"); + if (idx < 0) + idx = 0; + rsrc_id = idx ? IMX_SC_R_ENET_1 : IMX_SC_R_ENET_0; + + val = enabled ? 1 : 0; + imx_sc_misc_set_control(fep->ipc_handle, rsrc_id, IMX_SC_C_IPG_STOP, val); +} + static void fec_enet_stop_mode(struct fec_enet_private *fep, bool enabled) { struct fec_platform_data *pdata = fep->pdev->dev.platform_data; @@ -1183,6 +1267,8 @@ static void fec_enet_stop_mode(struct fec_enet_private *fep, bool enabled) BIT(stop_gpr->bit), 0); } else if (pdata && pdata->sleep_mode_enable) { pdata->sleep_mode_enable(enabled); + } else { + fec_enet_ipg_stop_set(fep, enabled); } } @@ -1408,7 +1494,7 @@ static void fec_enet_tx(struct net_device *ndev) fec_enet_tx_queue(ndev, i); } -static int +static int __maybe_unused fec_enet_new_rxbdp(struct net_device *ndev, struct bufdesc *bdp, struct sk_buff *skb) { struct fec_enet_private *fep = netdev_priv(ndev); @@ -1428,8 +1514,9 @@ fec_enet_new_rxbdp(struct net_device *ndev, struct bufdesc *bdp, struct sk_buff return 0; } -static bool fec_enet_copybreak(struct net_device *ndev, struct sk_buff **skb, - struct bufdesc *bdp, u32 length, bool swap) +static bool __maybe_unused +fec_enet_copybreak(struct net_device *ndev, struct sk_buff **skb, + struct bufdesc *bdp, u32 length, bool swap) { struct fec_enet_private *fep = netdev_priv(ndev); struct sk_buff *new_skb; @@ -1454,6 +1541,21 @@ static bool fec_enet_copybreak(struct net_device *ndev, struct sk_buff **skb, return true; } +static void fec_enet_update_cbd(struct fec_enet_priv_rx_q *rxq, + struct bufdesc *bdp, int index) +{ + struct page *new_page; + dma_addr_t phys_addr; + + new_page = page_pool_dev_alloc_pages(rxq->page_pool); + WARN_ON(!new_page); + rxq->rx_skb_info[index].page = new_page; + + rxq->rx_skb_info[index].offset = FEC_ENET_XDP_HEADROOM; + phys_addr = page_pool_get_dma_addr(new_page) + FEC_ENET_XDP_HEADROOM; + bdp->cbd_bufaddr = cpu_to_fec32(phys_addr); +} + /* During a receive, the bd_rx.cur points to the current incoming buffer. * When we update through the ring, if the next incoming buffer has * not been given to the system, we just set the empty indicator, @@ -1466,7 +1568,6 @@ fec_enet_rx_queue(struct net_device *ndev, int budget, u16 queue_id) struct fec_enet_priv_rx_q *rxq; struct bufdesc *bdp; unsigned short status; - struct sk_buff *skb_new = NULL; struct sk_buff *skb; ushort pkt_len; __u8 *data; @@ -1475,8 +1576,8 @@ fec_enet_rx_queue(struct net_device *ndev, int budget, u16 queue_id) bool vlan_packet_rcvd = false; u16 vlan_tag; int index = 0; - bool is_copybreak; bool need_swap = fep->quirks & FEC_QUIRK_SWAP_FRAME; + struct page *page; #ifdef CONFIG_M532x flush_cache_all(); @@ -1528,31 +1629,25 @@ fec_enet_rx_queue(struct net_device *ndev, int budget, u16 queue_id) ndev->stats.rx_bytes += pkt_len; index = fec_enet_get_bd_index(bdp, &rxq->bd); - skb = rxq->rx_skbuff[index]; + page = rxq->rx_skb_info[index].page; + dma_sync_single_for_cpu(&fep->pdev->dev, + fec32_to_cpu(bdp->cbd_bufaddr), + pkt_len, + DMA_FROM_DEVICE); + prefetch(page_address(page)); + fec_enet_update_cbd(rxq, bdp, index); /* The packet length includes FCS, but we don't want to * include that when passing upstream as it messes up * bridging applications. */ - is_copybreak = fec_enet_copybreak(ndev, &skb, bdp, pkt_len - 4, - need_swap); - if (!is_copybreak) { - skb_new = netdev_alloc_skb(ndev, FEC_ENET_RX_FRSIZE); - if (unlikely(!skb_new)) { - ndev->stats.rx_dropped++; - goto rx_processing_done; - } - dma_unmap_single(&fep->pdev->dev, - fec32_to_cpu(bdp->cbd_bufaddr), - FEC_ENET_RX_FRSIZE - fep->rx_align, - DMA_FROM_DEVICE); - } - - prefetch(skb->data - NET_IP_ALIGN); + skb = build_skb(page_address(page), PAGE_SIZE); + skb_reserve(skb, FEC_ENET_XDP_HEADROOM); skb_put(skb, pkt_len - 4); + skb_mark_for_recycle(skb); data = skb->data; - if (!is_copybreak && need_swap) + if (need_swap) swap_buffer(data, pkt_len); #if !defined(CONFIG_M5272) @@ -1607,16 +1702,6 @@ fec_enet_rx_queue(struct net_device *ndev, int budget, u16 queue_id) skb_record_rx_queue(skb, queue_id); napi_gro_receive(&fep->napi, skb); - if (is_copybreak) { - dma_sync_single_for_device(&fep->pdev->dev, - fec32_to_cpu(bdp->cbd_bufaddr), - FEC_ENET_RX_FRSIZE - fep->rx_align, - DMA_FROM_DEVICE); - } else { - rxq->rx_skbuff[index] = skb_new; - fec_enet_new_rxbdp(ndev, bdp, skb_new); - } - rx_processing_done: /* Clear the status flags for this buffer */ status &= ~BD_ENET_RX_STATS; @@ -2105,13 +2190,13 @@ static int fec_enet_mii_probe(struct net_device *ndev) continue; if (dev_id--) continue; - strlcpy(mdio_bus_id, fep->mii_bus->id, MII_BUS_ID_SIZE); + strscpy(mdio_bus_id, fep->mii_bus->id, MII_BUS_ID_SIZE); break; } if (phy_id >= PHY_MAX_ADDR) { netdev_info(ndev, "no PHY, assuming direct connection to switch\n"); - strlcpy(mdio_bus_id, "fixed-0", MII_BUS_ID_SIZE); + strscpy(mdio_bus_id, "fixed-0", MII_BUS_ID_SIZE); phy_id = 0; } @@ -2295,9 +2380,9 @@ static void fec_enet_get_drvinfo(struct net_device *ndev, { struct fec_enet_private *fep = netdev_priv(ndev); - strlcpy(info->driver, fep->pdev->dev.driver->name, + strscpy(info->driver, fep->pdev->dev.driver->name, sizeof(info->driver)); - strlcpy(info->bus_info, dev_name(&ndev->dev), sizeof(info->bus_info)); + strscpy(info->bus_info, dev_name(&ndev->dev), sizeof(info->bus_info)); } static int fec_enet_get_regs_len(struct net_device *ndev) @@ -2960,26 +3045,19 @@ static void fec_enet_free_buffers(struct net_device *ndev) struct fec_enet_private *fep = netdev_priv(ndev); unsigned int i; struct sk_buff *skb; - struct bufdesc *bdp; struct fec_enet_priv_tx_q *txq; struct fec_enet_priv_rx_q *rxq; unsigned int q; for (q = 0; q < fep->num_rx_queues; q++) { rxq = fep->rx_queue[q]; - bdp = rxq->bd.base; - for (i = 0; i < rxq->bd.ring_size; i++) { - skb = rxq->rx_skbuff[i]; - rxq->rx_skbuff[i] = NULL; - if (skb) { - dma_unmap_single(&fep->pdev->dev, - fec32_to_cpu(bdp->cbd_bufaddr), - FEC_ENET_RX_FRSIZE - fep->rx_align, - DMA_FROM_DEVICE); - dev_kfree_skb(skb); - } - bdp = fec_enet_get_nextdesc(bdp, &rxq->bd); - } + for (i = 0; i < rxq->bd.ring_size; i++) + page_pool_release_page(rxq->page_pool, rxq->rx_skb_info[i].page); + + if (xdp_rxq_info_is_reg(&rxq->xdp_rxq)) + xdp_rxq_info_unreg(&rxq->xdp_rxq); + page_pool_destroy(rxq->page_pool); + rxq->page_pool = NULL; } for (q = 0; q < fep->num_tx_queues; q++) { @@ -3069,24 +3147,31 @@ static int fec_enet_alloc_rxq_buffers(struct net_device *ndev, unsigned int queue) { struct fec_enet_private *fep = netdev_priv(ndev); - unsigned int i; - struct sk_buff *skb; - struct bufdesc *bdp; struct fec_enet_priv_rx_q *rxq; + dma_addr_t phys_addr; + struct bufdesc *bdp; + struct page *page; + int i, err; rxq = fep->rx_queue[queue]; bdp = rxq->bd.base; + + err = fec_enet_create_page_pool(fep, rxq, rxq->bd.ring_size); + if (err < 0) { + netdev_err(ndev, "%s failed queue %d (%d)\n", __func__, queue, err); + return err; + } + for (i = 0; i < rxq->bd.ring_size; i++) { - skb = __netdev_alloc_skb(ndev, FEC_ENET_RX_FRSIZE, GFP_KERNEL); - if (!skb) + page = page_pool_dev_alloc_pages(rxq->page_pool); + if (!page) goto err_alloc; - if (fec_enet_new_rxbdp(ndev, bdp, skb)) { - dev_kfree_skb(skb); - goto err_alloc; - } + phys_addr = page_pool_get_dma_addr(page) + FEC_ENET_XDP_HEADROOM; + bdp->cbd_bufaddr = cpu_to_fec32(phys_addr); - rxq->rx_skbuff[i] = skb; + rxq->rx_skb_info[i].page = page; + rxq->rx_skb_info[i].offset = FEC_ENET_XDP_HEADROOM; bdp->cbd_sc = cpu_to_fec16(BD_ENET_RX_EMPTY); if (fep->bufdesc_ex) { @@ -3566,7 +3651,7 @@ static int fec_enet_init(struct net_device *ndev) ndev->ethtool_ops = &fec_enet_ethtool_ops; writel(FEC_RX_DISABLED_IMASK, fep->hwp + FEC_IMASK); - netif_napi_add(ndev, &fep->napi, fec_enet_rx_napi, NAPI_POLL_WEIGHT); + netif_napi_add(ndev, &fep->napi, fec_enet_rx_napi); if (fep->quirks & FEC_QUIRK_HAS_VLAN) /* enable hw VLAN support */ @@ -3824,6 +3909,10 @@ fec_probe(struct platform_device *pdev) !of_property_read_bool(np, "fsl,err006687-workaround-present")) fep->quirks |= FEC_QUIRK_ERR006687; + ret = fec_enet_ipc_handle_init(fep); + if (ret) + goto failed_ipc_init; + if (of_get_property(np, "fsl,magic-packet", NULL)) fep->wol_flag |= FEC_WOL_HAS_MAGIC_PACKET; @@ -4021,6 +4110,7 @@ failed_rgmii_delay: of_phy_deregister_fixed_link(np); of_node_put(phy_node); failed_stop_mode: +failed_ipc_init: failed_phy: dev_id--; failed_ioremap: @@ -4065,6 +4155,7 @@ static int __maybe_unused fec_suspend(struct device *dev) { struct net_device *ndev = dev_get_drvdata(dev); struct fec_enet_private *fep = netdev_priv(ndev); + int ret; rtnl_lock(); if (netif_running(ndev)) { @@ -4089,6 +4180,15 @@ static int __maybe_unused fec_suspend(struct device *dev) } /* It's safe to disable clocks since interrupts are masked */ fec_enet_clk_enable(ndev, false); + + fep->rpm_active = !pm_runtime_status_suspended(dev); + if (fep->rpm_active) { + ret = pm_runtime_force_suspend(dev); + if (ret < 0) { + rtnl_unlock(); + return ret; + } + } } rtnl_unlock(); @@ -4119,6 +4219,9 @@ static int __maybe_unused fec_resume(struct device *dev) rtnl_lock(); if (netif_running(ndev)) { + if (fep->rpm_active) + pm_runtime_force_resume(dev); + ret = fec_enet_clk_enable(ndev, true); if (ret) { rtnl_unlock(); diff --git a/drivers/net/ethernet/freescale/fec_ptp.c b/drivers/net/ethernet/freescale/fec_ptp.c index 3dc3c0b626c2..cffd9ad499dd 100644 --- a/drivers/net/ethernet/freescale/fec_ptp.c +++ b/drivers/net/ethernet/freescale/fec_ptp.c @@ -578,7 +578,7 @@ void fec_ptp_init(struct platform_device *pdev, int irq_idx) int ret; fep->ptp_caps.owner = THIS_MODULE; - strlcpy(fep->ptp_caps.name, "fec ptp", sizeof(fep->ptp_caps.name)); + strscpy(fep->ptp_caps.name, "fec ptp", sizeof(fep->ptp_caps.name)); fep->ptp_caps.max_adj = 250000000; fep->ptp_caps.n_alarm = 0; diff --git a/drivers/net/ethernet/freescale/fman/fman.c b/drivers/net/ethernet/freescale/fman/fman.c index 8f0db61cb1f6..9d85fb136e34 100644 --- a/drivers/net/ethernet/freescale/fman/fman.c +++ b/drivers/net/ethernet/freescale/fman/fman.c @@ -1,34 +1,7 @@ +// SPDX-License-Identifier: BSD-3-Clause OR GPL-2.0-or-later /* - * Copyright 2008-2015 Freescale Semiconductor Inc. + * Copyright 2008 - 2015 Freescale Semiconductor Inc. * Copyright 2020 NXP - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * * Redistributions of source code must retain the above copyright - * notice, this list of conditions and the following disclaimer. - * * Redistributions in binary form must reproduce the above copyright - * notice, this list of conditions and the following disclaimer in the - * documentation and/or other materials provided with the distribution. - * * Neither the name of Freescale Semiconductor nor the - * names of its contributors may be used to endorse or promote products - * derived from this software without specific prior written permission. - * - * - * ALTERNATIVELY, this software may be distributed under the terms of the - * GNU General Public License ("GPL") as published by the Free Software - * Foundation, either version 2 of that License or (at your option) any - * later version. - * - * THIS SOFTWARE IS PROVIDED BY Freescale Semiconductor ``AS IS'' AND ANY - * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL Freescale Semiconductor BE LIABLE FOR ANY - * DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES - * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; - * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND - * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT - * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS - * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt diff --git a/drivers/net/ethernet/freescale/fman/fman.h b/drivers/net/ethernet/freescale/fman/fman.h index f2ede1360f03..2ea575a46675 100644 --- a/drivers/net/ethernet/freescale/fman/fman.h +++ b/drivers/net/ethernet/freescale/fman/fman.h @@ -1,34 +1,7 @@ +/* SPDX-License-Identifier: BSD-3-Clause OR GPL-2.0-or-later */ /* - * Copyright 2008-2015 Freescale Semiconductor Inc. + * Copyright 2008 - 2015 Freescale Semiconductor Inc. * Copyright 2020 NXP - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * * Redistributions of source code must retain the above copyright - * notice, this list of conditions and the following disclaimer. - * * Redistributions in binary form must reproduce the above copyright - * notice, this list of conditions and the following disclaimer in the - * documentation and/or other materials provided with the distribution. - * * Neither the name of Freescale Semiconductor nor the - * names of its contributors may be used to endorse or promote products - * derived from this software without specific prior written permission. - * - * - * ALTERNATIVELY, this software may be distributed under the terms of the - * GNU General Public License ("GPL") as published by the Free Software - * Foundation, either version 2 of that License or (at your option) any - * later version. - * - * THIS SOFTWARE IS PROVIDED BY Freescale Semiconductor ``AS IS'' AND ANY - * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL Freescale Semiconductor BE LIABLE FOR ANY - * DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES - * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; - * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND - * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT - * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS - * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ #ifndef __FM_H diff --git a/drivers/net/ethernet/freescale/fman/fman_dtsec.c b/drivers/net/ethernet/freescale/fman/fman_dtsec.c index 1950a8936bc0..6617932fd3fd 100644 --- a/drivers/net/ethernet/freescale/fman/fman_dtsec.c +++ b/drivers/net/ethernet/freescale/fman/fman_dtsec.c @@ -1,39 +1,13 @@ +// SPDX-License-Identifier: BSD-3-Clause OR GPL-2.0-or-later /* - * Copyright 2008-2015 Freescale Semiconductor Inc. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * * Redistributions of source code must retain the above copyright - * notice, this list of conditions and the following disclaimer. - * * Redistributions in binary form must reproduce the above copyright - * notice, this list of conditions and the following disclaimer in the - * documentation and/or other materials provided with the distribution. - * * Neither the name of Freescale Semiconductor nor the - * names of its contributors may be used to endorse or promote products - * derived from this software without specific prior written permission. - * - * - * ALTERNATIVELY, this software may be distributed under the terms of the - * GNU General Public License ("GPL") as published by the Free Software - * Foundation, either version 2 of that License or (at your option) any - * later version. - * - * THIS SOFTWARE IS PROVIDED BY Freescale Semiconductor ``AS IS'' AND ANY - * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL Freescale Semiconductor BE LIABLE FOR ANY - * DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES - * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; - * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND - * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT - * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS - * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + * Copyright 2008 - 2015 Freescale Semiconductor Inc. */ #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt #include "fman_dtsec.h" #include "fman.h" +#include "mac.h" #include <linux/slab.h> #include <linux/bitrev.h> @@ -327,7 +301,7 @@ struct fman_mac { /* Ethernet physical interface */ phy_interface_t phy_if; u16 max_speed; - void *dev_id; /* device cookie used by the exception cbs */ + struct mac_device *dev_id; /* device cookie used by the exception cbs */ fman_mac_exception_cb *exception_cb; fman_mac_exception_cb *event_cb; /* Number of individual addresses in registers for this station */ @@ -840,73 +814,45 @@ static void free_init_resources(struct fman_mac *dtsec) dtsec->unicast_addr_hash = NULL; } -int dtsec_cfg_max_frame_len(struct fman_mac *dtsec, u16 new_val) -{ - if (is_init_done(dtsec->dtsec_drv_param)) - return -EINVAL; - - dtsec->dtsec_drv_param->maximum_frame = new_val; - - return 0; -} - -int dtsec_cfg_pad_and_crc(struct fman_mac *dtsec, bool new_val) -{ - if (is_init_done(dtsec->dtsec_drv_param)) - return -EINVAL; - - dtsec->dtsec_drv_param->tx_pad_crc = new_val; - - return 0; -} - -static void graceful_start(struct fman_mac *dtsec, enum comm_mode mode) +static void graceful_start(struct fman_mac *dtsec) { struct dtsec_regs __iomem *regs = dtsec->regs; - if (mode & COMM_MODE_TX) - iowrite32be(ioread32be(®s->tctrl) & - ~TCTRL_GTS, ®s->tctrl); - if (mode & COMM_MODE_RX) - iowrite32be(ioread32be(®s->rctrl) & - ~RCTRL_GRS, ®s->rctrl); + iowrite32be(ioread32be(®s->tctrl) & ~TCTRL_GTS, ®s->tctrl); + iowrite32be(ioread32be(®s->rctrl) & ~RCTRL_GRS, ®s->rctrl); } -static void graceful_stop(struct fman_mac *dtsec, enum comm_mode mode) +static void graceful_stop(struct fman_mac *dtsec) { struct dtsec_regs __iomem *regs = dtsec->regs; u32 tmp; /* Graceful stop - Assert the graceful Rx stop bit */ - if (mode & COMM_MODE_RX) { - tmp = ioread32be(®s->rctrl) | RCTRL_GRS; - iowrite32be(tmp, ®s->rctrl); + tmp = ioread32be(®s->rctrl) | RCTRL_GRS; + iowrite32be(tmp, ®s->rctrl); - if (dtsec->fm_rev_info.major == 2) { - /* Workaround for dTSEC Errata A002 */ - usleep_range(100, 200); - } else { - /* Workaround for dTSEC Errata A004839 */ - usleep_range(10, 50); - } + if (dtsec->fm_rev_info.major == 2) { + /* Workaround for dTSEC Errata A002 */ + usleep_range(100, 200); + } else { + /* Workaround for dTSEC Errata A004839 */ + usleep_range(10, 50); } /* Graceful stop - Assert the graceful Tx stop bit */ - if (mode & COMM_MODE_TX) { - if (dtsec->fm_rev_info.major == 2) { - /* dTSEC Errata A004: Do not use TCTRL[GTS]=1 */ - pr_debug("GTS not supported due to DTSEC_A004 Errata.\n"); - } else { - tmp = ioread32be(®s->tctrl) | TCTRL_GTS; - iowrite32be(tmp, ®s->tctrl); + if (dtsec->fm_rev_info.major == 2) { + /* dTSEC Errata A004: Do not use TCTRL[GTS]=1 */ + pr_debug("GTS not supported due to DTSEC_A004 Errata.\n"); + } else { + tmp = ioread32be(®s->tctrl) | TCTRL_GTS; + iowrite32be(tmp, ®s->tctrl); - /* Workaround for dTSEC Errata A0012, A0014 */ - usleep_range(10, 50); - } + /* Workaround for dTSEC Errata A0012, A0014 */ + usleep_range(10, 50); } } -int dtsec_enable(struct fman_mac *dtsec, enum comm_mode mode) +static int dtsec_enable(struct fman_mac *dtsec) { struct dtsec_regs __iomem *regs = dtsec->regs; u32 tmp; @@ -916,58 +862,42 @@ int dtsec_enable(struct fman_mac *dtsec, enum comm_mode mode) /* Enable */ tmp = ioread32be(®s->maccfg1); - if (mode & COMM_MODE_RX) - tmp |= MACCFG1_RX_EN; - if (mode & COMM_MODE_TX) - tmp |= MACCFG1_TX_EN; - + tmp |= MACCFG1_RX_EN | MACCFG1_TX_EN; iowrite32be(tmp, ®s->maccfg1); /* Graceful start - clear the graceful Rx/Tx stop bit */ - graceful_start(dtsec, mode); + graceful_start(dtsec); return 0; } -int dtsec_disable(struct fman_mac *dtsec, enum comm_mode mode) +static void dtsec_disable(struct fman_mac *dtsec) { struct dtsec_regs __iomem *regs = dtsec->regs; u32 tmp; - if (!is_init_done(dtsec->dtsec_drv_param)) - return -EINVAL; + WARN_ON_ONCE(!is_init_done(dtsec->dtsec_drv_param)); /* Graceful stop - Assert the graceful Rx/Tx stop bit */ - graceful_stop(dtsec, mode); + graceful_stop(dtsec); tmp = ioread32be(®s->maccfg1); - if (mode & COMM_MODE_RX) - tmp &= ~MACCFG1_RX_EN; - if (mode & COMM_MODE_TX) - tmp &= ~MACCFG1_TX_EN; - + tmp &= ~(MACCFG1_RX_EN | MACCFG1_TX_EN); iowrite32be(tmp, ®s->maccfg1); - - return 0; } -int dtsec_set_tx_pause_frames(struct fman_mac *dtsec, - u8 __maybe_unused priority, - u16 pause_time, u16 __maybe_unused thresh_time) +static int dtsec_set_tx_pause_frames(struct fman_mac *dtsec, + u8 __maybe_unused priority, + u16 pause_time, + u16 __maybe_unused thresh_time) { struct dtsec_regs __iomem *regs = dtsec->regs; - enum comm_mode mode = COMM_MODE_NONE; u32 ptv = 0; if (!is_init_done(dtsec->dtsec_drv_param)) return -EINVAL; - if ((ioread32be(®s->rctrl) & RCTRL_GRS) == 0) - mode |= COMM_MODE_RX; - if ((ioread32be(®s->tctrl) & TCTRL_GTS) == 0) - mode |= COMM_MODE_TX; - - graceful_stop(dtsec, mode); + graceful_stop(dtsec); if (pause_time) { /* FM_BAD_TX_TS_IN_B_2_B_ERRATA_DTSEC_A003 Errata workaround */ @@ -989,26 +919,20 @@ int dtsec_set_tx_pause_frames(struct fman_mac *dtsec, iowrite32be(ioread32be(®s->maccfg1) & ~MACCFG1_TX_FLOW, ®s->maccfg1); - graceful_start(dtsec, mode); + graceful_start(dtsec); return 0; } -int dtsec_accept_rx_pause_frames(struct fman_mac *dtsec, bool en) +static int dtsec_accept_rx_pause_frames(struct fman_mac *dtsec, bool en) { struct dtsec_regs __iomem *regs = dtsec->regs; - enum comm_mode mode = COMM_MODE_NONE; u32 tmp; if (!is_init_done(dtsec->dtsec_drv_param)) return -EINVAL; - if ((ioread32be(®s->rctrl) & RCTRL_GRS) == 0) - mode |= COMM_MODE_RX; - if ((ioread32be(®s->tctrl) & TCTRL_GTS) == 0) - mode |= COMM_MODE_TX; - - graceful_stop(dtsec, mode); + graceful_stop(dtsec); tmp = ioread32be(®s->maccfg1); if (en) @@ -1017,25 +941,18 @@ int dtsec_accept_rx_pause_frames(struct fman_mac *dtsec, bool en) tmp &= ~MACCFG1_RX_FLOW; iowrite32be(tmp, ®s->maccfg1); - graceful_start(dtsec, mode); + graceful_start(dtsec); return 0; } -int dtsec_modify_mac_address(struct fman_mac *dtsec, const enet_addr_t *enet_addr) +static int dtsec_modify_mac_address(struct fman_mac *dtsec, + const enet_addr_t *enet_addr) { - struct dtsec_regs __iomem *regs = dtsec->regs; - enum comm_mode mode = COMM_MODE_NONE; - if (!is_init_done(dtsec->dtsec_drv_param)) return -EINVAL; - if ((ioread32be(®s->rctrl) & RCTRL_GRS) == 0) - mode |= COMM_MODE_RX; - if ((ioread32be(®s->tctrl) & TCTRL_GTS) == 0) - mode |= COMM_MODE_TX; - - graceful_stop(dtsec, mode); + graceful_stop(dtsec); /* Initialize MAC Station Address registers (1 & 2) * Station address have to be swapped (big endian to little endian @@ -1043,12 +960,13 @@ int dtsec_modify_mac_address(struct fman_mac *dtsec, const enet_addr_t *enet_add dtsec->addr = ENET_ADDR_TO_UINT64(*enet_addr); set_mac_address(dtsec->regs, (const u8 *)(*enet_addr)); - graceful_start(dtsec, mode); + graceful_start(dtsec); return 0; } -int dtsec_add_hash_mac_address(struct fman_mac *dtsec, enet_addr_t *eth_addr) +static int dtsec_add_hash_mac_address(struct fman_mac *dtsec, + enet_addr_t *eth_addr) { struct dtsec_regs __iomem *regs = dtsec->regs; struct eth_hash_entry *hash_entry; @@ -1114,7 +1032,7 @@ int dtsec_add_hash_mac_address(struct fman_mac *dtsec, enet_addr_t *eth_addr) return 0; } -int dtsec_set_allmulti(struct fman_mac *dtsec, bool enable) +static int dtsec_set_allmulti(struct fman_mac *dtsec, bool enable) { u32 tmp; struct dtsec_regs __iomem *regs = dtsec->regs; @@ -1133,7 +1051,7 @@ int dtsec_set_allmulti(struct fman_mac *dtsec, bool enable) return 0; } -int dtsec_set_tstamp(struct fman_mac *dtsec, bool enable) +static int dtsec_set_tstamp(struct fman_mac *dtsec, bool enable) { struct dtsec_regs __iomem *regs = dtsec->regs; u32 rctrl, tctrl; @@ -1158,7 +1076,8 @@ int dtsec_set_tstamp(struct fman_mac *dtsec, bool enable) return 0; } -int dtsec_del_hash_mac_address(struct fman_mac *dtsec, enet_addr_t *eth_addr) +static int dtsec_del_hash_mac_address(struct fman_mac *dtsec, + enet_addr_t *eth_addr) { struct dtsec_regs __iomem *regs = dtsec->regs; struct list_head *pos; @@ -1229,7 +1148,7 @@ int dtsec_del_hash_mac_address(struct fman_mac *dtsec, enet_addr_t *eth_addr) return 0; } -int dtsec_set_promiscuous(struct fman_mac *dtsec, bool new_val) +static int dtsec_set_promiscuous(struct fman_mac *dtsec, bool new_val) { struct dtsec_regs __iomem *regs = dtsec->regs; u32 tmp; @@ -1258,21 +1177,15 @@ int dtsec_set_promiscuous(struct fman_mac *dtsec, bool new_val) return 0; } -int dtsec_adjust_link(struct fman_mac *dtsec, u16 speed) +static int dtsec_adjust_link(struct fman_mac *dtsec, u16 speed) { struct dtsec_regs __iomem *regs = dtsec->regs; - enum comm_mode mode = COMM_MODE_NONE; u32 tmp; if (!is_init_done(dtsec->dtsec_drv_param)) return -EINVAL; - if ((ioread32be(®s->rctrl) & RCTRL_GRS) == 0) - mode |= COMM_MODE_RX; - if ((ioread32be(®s->tctrl) & TCTRL_GTS) == 0) - mode |= COMM_MODE_TX; - - graceful_stop(dtsec, mode); + graceful_stop(dtsec); tmp = ioread32be(®s->maccfg2); @@ -1293,12 +1206,12 @@ int dtsec_adjust_link(struct fman_mac *dtsec, u16 speed) tmp &= ~DTSEC_ECNTRL_R100M; iowrite32be(tmp, ®s->ecntrl); - graceful_start(dtsec, mode); + graceful_start(dtsec); return 0; } -int dtsec_restart_autoneg(struct fman_mac *dtsec) +static int dtsec_restart_autoneg(struct fman_mac *dtsec) { u16 tmp_reg16; @@ -1316,20 +1229,31 @@ int dtsec_restart_autoneg(struct fman_mac *dtsec) return 0; } -int dtsec_get_version(struct fman_mac *dtsec, u32 *mac_version) +static void adjust_link_dtsec(struct mac_device *mac_dev) { - struct dtsec_regs __iomem *regs = dtsec->regs; + struct phy_device *phy_dev = mac_dev->phy_dev; + struct fman_mac *fman_mac; + bool rx_pause, tx_pause; + int err; - if (!is_init_done(dtsec->dtsec_drv_param)) - return -EINVAL; + fman_mac = mac_dev->fman_mac; + if (!phy_dev->link) { + dtsec_restart_autoneg(fman_mac); - *mac_version = ioread32be(®s->tsec_id); + return; + } - return 0; + dtsec_adjust_link(fman_mac, phy_dev->speed); + mac_dev->update_speed(mac_dev, phy_dev->speed); + fman_get_pause_cfg(mac_dev, &rx_pause, &tx_pause); + err = fman_set_mac_active_pause(mac_dev, rx_pause, tx_pause); + if (err < 0) + dev_err(mac_dev->dev, "fman_set_mac_active_pause() = %d\n", + err); } -int dtsec_set_exception(struct fman_mac *dtsec, - enum fman_mac_exceptions exception, bool enable) +static int dtsec_set_exception(struct fman_mac *dtsec, + enum fman_mac_exceptions exception, bool enable) { struct dtsec_regs __iomem *regs = dtsec->regs; u32 bit_mask = 0; @@ -1382,7 +1306,7 @@ int dtsec_set_exception(struct fman_mac *dtsec, return 0; } -int dtsec_init(struct fman_mac *dtsec) +static int dtsec_init(struct fman_mac *dtsec) { struct dtsec_regs __iomem *regs = dtsec->regs; struct dtsec_cfg *dtsec_drv_param; @@ -1476,7 +1400,7 @@ int dtsec_init(struct fman_mac *dtsec) return 0; } -int dtsec_free(struct fman_mac *dtsec) +static int dtsec_free(struct fman_mac *dtsec) { free_init_resources(dtsec); @@ -1487,13 +1411,11 @@ int dtsec_free(struct fman_mac *dtsec) return 0; } -struct fman_mac *dtsec_config(struct fman_mac_params *params) +static struct fman_mac *dtsec_config(struct mac_device *mac_dev, + struct fman_mac_params *params) { struct fman_mac *dtsec; struct dtsec_cfg *dtsec_drv_param; - void __iomem *base_addr; - - base_addr = params->base_addr; /* allocate memory for the UCC GETH data structure. */ dtsec = kzalloc(sizeof(*dtsec), GFP_KERNEL); @@ -1510,10 +1432,10 @@ struct fman_mac *dtsec_config(struct fman_mac_params *params) set_dflts(dtsec_drv_param); - dtsec->regs = base_addr; - dtsec->addr = ENET_ADDR_TO_UINT64(params->addr); + dtsec->regs = mac_dev->vaddr; + dtsec->addr = ENET_ADDR_TO_UINT64(mac_dev->addr); dtsec->max_speed = params->max_speed; - dtsec->phy_if = params->phy_if; + dtsec->phy_if = mac_dev->phy_if; dtsec->mac_id = params->mac_id; dtsec->exceptions = (DTSEC_IMASK_BREN | DTSEC_IMASK_RXCEN | @@ -1530,34 +1452,87 @@ struct fman_mac *dtsec_config(struct fman_mac_params *params) DTSEC_IMASK_RDPEEN); dtsec->exception_cb = params->exception_cb; dtsec->event_cb = params->event_cb; - dtsec->dev_id = params->dev_id; + dtsec->dev_id = mac_dev; dtsec->ptp_tsu_enabled = dtsec->dtsec_drv_param->ptp_tsu_en; dtsec->en_tsu_err_exception = dtsec->dtsec_drv_param->ptp_exception_en; dtsec->fm = params->fm; dtsec->basex_if = params->basex_if; - if (!params->internal_phy_node) { + /* Save FMan revision */ + fman_get_revision(dtsec->fm, &dtsec->fm_rev_info); + + return dtsec; + +err_dtsec: + kfree(dtsec); + return NULL; +} + +int dtsec_initialization(struct mac_device *mac_dev, + struct device_node *mac_node, + struct fman_mac_params *params) +{ + int err; + struct fman_mac *dtsec; + struct device_node *phy_node; + + mac_dev->set_promisc = dtsec_set_promiscuous; + mac_dev->change_addr = dtsec_modify_mac_address; + mac_dev->add_hash_mac_addr = dtsec_add_hash_mac_address; + mac_dev->remove_hash_mac_addr = dtsec_del_hash_mac_address; + mac_dev->set_tx_pause = dtsec_set_tx_pause_frames; + mac_dev->set_rx_pause = dtsec_accept_rx_pause_frames; + mac_dev->set_exception = dtsec_set_exception; + mac_dev->set_allmulti = dtsec_set_allmulti; + mac_dev->set_tstamp = dtsec_set_tstamp; + mac_dev->set_multi = fman_set_multi; + mac_dev->adjust_link = adjust_link_dtsec; + mac_dev->enable = dtsec_enable; + mac_dev->disable = dtsec_disable; + + mac_dev->fman_mac = dtsec_config(mac_dev, params); + if (!mac_dev->fman_mac) { + err = -EINVAL; + goto _return; + } + + dtsec = mac_dev->fman_mac; + dtsec->dtsec_drv_param->maximum_frame = fman_get_max_frm(); + dtsec->dtsec_drv_param->tx_pad_crc = true; + + phy_node = of_parse_phandle(mac_node, "tbi-handle", 0); + if (!phy_node) { pr_err("TBI PHY node is not available\n"); - goto err_dtsec_drv_param; + err = -EINVAL; + goto _return_fm_mac_free; } - dtsec->tbiphy = of_phy_find_device(params->internal_phy_node); + dtsec->tbiphy = of_phy_find_device(phy_node); if (!dtsec->tbiphy) { pr_err("of_phy_find_device (TBI PHY) failed\n"); - goto err_dtsec_drv_param; + err = -EINVAL; + goto _return_fm_mac_free; } - put_device(&dtsec->tbiphy->mdio.dev); - /* Save FMan revision */ - fman_get_revision(dtsec->fm, &dtsec->fm_rev_info); + err = dtsec_init(dtsec); + if (err < 0) + goto _return_fm_mac_free; - return dtsec; + /* For 1G MAC, disable by default the MIB counters overflow interrupt */ + err = dtsec_set_exception(dtsec, FM_MAC_EX_1G_RX_MIB_CNT_OVFL, false); + if (err < 0) + goto _return_fm_mac_free; -err_dtsec_drv_param: - kfree(dtsec_drv_param); -err_dtsec: - kfree(dtsec); - return NULL; + dev_info(mac_dev->dev, "FMan dTSEC version: 0x%08x\n", + ioread32be(&dtsec->regs->tsec_id)); + + goto _return; + +_return_fm_mac_free: + dtsec_free(dtsec); + +_return: + return err; } diff --git a/drivers/net/ethernet/freescale/fman/fman_dtsec.h b/drivers/net/ethernet/freescale/fman/fman_dtsec.h index 68512c3bd6e5..8c72d280c51a 100644 --- a/drivers/net/ethernet/freescale/fman/fman_dtsec.h +++ b/drivers/net/ethernet/freescale/fman/fman_dtsec.h @@ -1,33 +1,6 @@ +/* SPDX-License-Identifier: BSD-3-Clause OR GPL-2.0-or-later */ /* - * Copyright 2008-2015 Freescale Semiconductor Inc. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * * Redistributions of source code must retain the above copyright - * notice, this list of conditions and the following disclaimer. - * * Redistributions in binary form must reproduce the above copyright - * notice, this list of conditions and the following disclaimer in the - * documentation and/or other materials provided with the distribution. - * * Neither the name of Freescale Semiconductor nor the - * names of its contributors may be used to endorse or promote products - * derived from this software without specific prior written permission. - * - * - * ALTERNATIVELY, this software may be distributed under the terms of the - * GNU General Public License ("GPL") as published by the Free Software - * Foundation, either version 2 of that License or (at your option) any - * later version. - * - * THIS SOFTWARE IS PROVIDED BY Freescale Semiconductor ``AS IS'' AND ANY - * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL Freescale Semiconductor BE LIABLE FOR ANY - * DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES - * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; - * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND - * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT - * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS - * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + * Copyright 2008 - 2015 Freescale Semiconductor Inc. */ #ifndef __DTSEC_H @@ -35,27 +8,10 @@ #include "fman_mac.h" -struct fman_mac *dtsec_config(struct fman_mac_params *params); -int dtsec_set_promiscuous(struct fman_mac *dtsec, bool new_val); -int dtsec_modify_mac_address(struct fman_mac *dtsec, const enet_addr_t *enet_addr); -int dtsec_adjust_link(struct fman_mac *dtsec, - u16 speed); -int dtsec_restart_autoneg(struct fman_mac *dtsec); -int dtsec_cfg_max_frame_len(struct fman_mac *dtsec, u16 new_val); -int dtsec_cfg_pad_and_crc(struct fman_mac *dtsec, bool new_val); -int dtsec_enable(struct fman_mac *dtsec, enum comm_mode mode); -int dtsec_disable(struct fman_mac *dtsec, enum comm_mode mode); -int dtsec_init(struct fman_mac *dtsec); -int dtsec_free(struct fman_mac *dtsec); -int dtsec_accept_rx_pause_frames(struct fman_mac *dtsec, bool en); -int dtsec_set_tx_pause_frames(struct fman_mac *dtsec, u8 priority, - u16 pause_time, u16 thresh_time); -int dtsec_set_exception(struct fman_mac *dtsec, - enum fman_mac_exceptions exception, bool enable); -int dtsec_add_hash_mac_address(struct fman_mac *dtsec, enet_addr_t *eth_addr); -int dtsec_del_hash_mac_address(struct fman_mac *dtsec, enet_addr_t *eth_addr); -int dtsec_get_version(struct fman_mac *dtsec, u32 *mac_version); -int dtsec_set_allmulti(struct fman_mac *dtsec, bool enable); -int dtsec_set_tstamp(struct fman_mac *dtsec, bool enable); +struct mac_device; + +int dtsec_initialization(struct mac_device *mac_dev, + struct device_node *mac_node, + struct fman_mac_params *params); #endif /* __DTSEC_H */ diff --git a/drivers/net/ethernet/freescale/fman/fman_keygen.c b/drivers/net/ethernet/freescale/fman/fman_keygen.c index e1bdfed16134..e73f6ef3c6ee 100644 --- a/drivers/net/ethernet/freescale/fman/fman_keygen.c +++ b/drivers/net/ethernet/freescale/fman/fman_keygen.c @@ -1,33 +1,6 @@ +// SPDX-License-Identifier: BSD-3-Clause OR GPL-2.0-or-later /* * Copyright 2017 NXP - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * * Redistributions of source code must retain the above copyright - * notice, this list of conditions and the following disclaimer. - * * Redistributions in binary form must reproduce the above copyright - * notice, this list of conditions and the following disclaimer in the - * documentation and/or other materials provided with the distribution. - * * Neither the name of NXP nor the - * names of its contributors may be used to endorse or promote products - * derived from this software without specific prior written permission. - * - * - * ALTERNATIVELY, this software may be distributed under the terms of the - * GNU General Public License ("GPL") as published by the Free Software - * Foundation, either version 2 of that License or (at your option) any - * later version. - * - * THIS SOFTWARE IS PROVIDED BY NXP ``AS IS'' AND ANY - * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL NXP BE LIABLE FOR ANY - * DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES - * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; - * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND - * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT - * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS - * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt diff --git a/drivers/net/ethernet/freescale/fman/fman_keygen.h b/drivers/net/ethernet/freescale/fman/fman_keygen.h index c4640de3f4cb..2cb0df453074 100644 --- a/drivers/net/ethernet/freescale/fman/fman_keygen.h +++ b/drivers/net/ethernet/freescale/fman/fman_keygen.h @@ -1,33 +1,6 @@ +/* SPDX-License-Identifier: BSD-3-Clause OR GPL-2.0-or-later */ /* * Copyright 2017 NXP - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * * Redistributions of source code must retain the above copyright - * notice, this list of conditions and the following disclaimer. - * * Redistributions in binary form must reproduce the above copyright - * notice, this list of conditions and the following disclaimer in the - * documentation and/or other materials provided with the distribution. - * * Neither the name of NXP nor the - * names of its contributors may be used to endorse or promote products - * derived from this software without specific prior written permission. - * - * - * ALTERNATIVELY, this software may be distributed under the terms of the - * GNU General Public License ("GPL") as published by the Free Software - * Foundation, either version 2 of that License or (at your option) any - * later version. - * - * THIS SOFTWARE IS PROVIDED BY NXP ``AS IS'' AND ANY - * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL NXP BE LIABLE FOR ANY - * DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES - * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; - * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND - * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT - * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS - * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ #ifndef __KEYGEN_H diff --git a/drivers/net/ethernet/freescale/fman/fman_mac.h b/drivers/net/ethernet/freescale/fman/fman_mac.h index 19f327efdaff..65887a3160d7 100644 --- a/drivers/net/ethernet/freescale/fman/fman_mac.h +++ b/drivers/net/ethernet/freescale/fman/fman_mac.h @@ -41,6 +41,7 @@ #include <linux/if_ether.h> struct fman_mac; +struct mac_device; /* Ethernet Address */ typedef u8 enet_addr_t[ETH_ALEN]; @@ -75,16 +76,6 @@ typedef u8 enet_addr_t[ETH_ALEN]; #define ETH_HASH_ENTRY_OBJ(ptr) \ hlist_entry_safe(ptr, struct eth_hash_entry, node) -/* Enumeration (bit flags) of communication modes (Transmit, - * receive or both). - */ -enum comm_mode { - COMM_MODE_NONE = 0, /* No transmit/receive communication */ - COMM_MODE_RX = 1, /* Only receive communication */ - COMM_MODE_TX = 2, /* Only transmit communication */ - COMM_MODE_RX_AND_TX = 3 /* Both transmit and receive communication */ -}; - /* FM MAC Exceptions */ enum fman_mac_exceptions { FM_MAC_EX_10G_MDIO_SCAN_EVENT = 0 @@ -168,30 +159,23 @@ struct eth_hash_entry { struct list_head node; }; -typedef void (fman_mac_exception_cb)(void *dev_id, - enum fman_mac_exceptions exceptions); +typedef void (fman_mac_exception_cb)(struct mac_device *dev_id, + enum fman_mac_exceptions exceptions); /* FMan MAC config input */ struct fman_mac_params { - /* Base of memory mapped FM MAC registers */ - void __iomem *base_addr; - /* MAC address of device; First octet is sent first */ - enet_addr_t addr; /* MAC ID; numbering of dTSEC and 1G-mEMAC: * 0 - FM_MAX_NUM_OF_1G_MACS; * numbering of 10G-MAC (TGEC) and 10G-mEMAC: * 0 - FM_MAX_NUM_OF_10G_MACS */ u8 mac_id; - /* PHY interface */ - phy_interface_t phy_if; /* Note that the speed should indicate the maximum rate that * this MAC should support rather than the actual speed; */ u16 max_speed; /* A handle to the FM object this port related to */ void *fm; - void *dev_id; /* device cookie used by the exception cbs */ fman_mac_exception_cb *event_cb; /* MDIO Events Callback Routine */ fman_mac_exception_cb *exception_cb;/* Exception Callback Routine */ /* SGMII/QSGII interface with 1000BaseX auto-negotiation between MAC @@ -200,8 +184,6 @@ struct fman_mac_params { * synchronize with far-end phy at 10Mbps, 100Mbps or 1000Mbps */ bool basex_if; - /* Pointer to TBI/PCS PHY node, used for TBI/PCS PHY access */ - struct device_node *internal_phy_node; }; struct eth_hash_t { diff --git a/drivers/net/ethernet/freescale/fman/fman_memac.c b/drivers/net/ethernet/freescale/fman/fman_memac.c index 2216b7f51d26..32d26cf17843 100644 --- a/drivers/net/ethernet/freescale/fman/fman_memac.c +++ b/drivers/net/ethernet/freescale/fman/fman_memac.c @@ -1,39 +1,13 @@ +// SPDX-License-Identifier: BSD-3-Clause OR GPL-2.0-or-later /* - * Copyright 2008-2015 Freescale Semiconductor Inc. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * * Redistributions of source code must retain the above copyright - * notice, this list of conditions and the following disclaimer. - * * Redistributions in binary form must reproduce the above copyright - * notice, this list of conditions and the following disclaimer in the - * documentation and/or other materials provided with the distribution. - * * Neither the name of Freescale Semiconductor nor the - * names of its contributors may be used to endorse or promote products - * derived from this software without specific prior written permission. - * - * - * ALTERNATIVELY, this software may be distributed under the terms of the - * GNU General Public License ("GPL") as published by the Free Software - * Foundation, either version 2 of that License or (at your option) any - * later version. - * - * THIS SOFTWARE IS PROVIDED BY Freescale Semiconductor ``AS IS'' AND ANY - * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL Freescale Semiconductor BE LIABLE FOR ANY - * DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES - * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; - * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND - * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT - * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS - * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + * Copyright 2008 - 2015 Freescale Semiconductor Inc. */ #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt #include "fman_memac.h" #include "fman.h" +#include "mac.h" #include <linux/slab.h> #include <linux/io.h> @@ -337,7 +311,7 @@ struct fman_mac { /* Ethernet physical interface */ phy_interface_t phy_if; u16 max_speed; - void *dev_id; /* device cookie used by the exception cbs */ + struct mac_device *dev_id; /* device cookie used by the exception cbs */ fman_mac_exception_cb *exception_cb; fman_mac_exception_cb *event_cb; /* Pointer to driver's global address hash table */ @@ -712,7 +686,7 @@ static bool is_init_done(struct memac_cfg *memac_drv_params) return false; } -int memac_enable(struct fman_mac *memac, enum comm_mode mode) +static int memac_enable(struct fman_mac *memac) { struct memac_regs __iomem *regs = memac->regs; u32 tmp; @@ -721,36 +695,26 @@ int memac_enable(struct fman_mac *memac, enum comm_mode mode) return -EINVAL; tmp = ioread32be(®s->command_config); - if (mode & COMM_MODE_RX) - tmp |= CMD_CFG_RX_EN; - if (mode & COMM_MODE_TX) - tmp |= CMD_CFG_TX_EN; - + tmp |= CMD_CFG_RX_EN | CMD_CFG_TX_EN; iowrite32be(tmp, ®s->command_config); return 0; } -int memac_disable(struct fman_mac *memac, enum comm_mode mode) +static void memac_disable(struct fman_mac *memac) + { struct memac_regs __iomem *regs = memac->regs; u32 tmp; - if (!is_init_done(memac->memac_drv_param)) - return -EINVAL; + WARN_ON_ONCE(!is_init_done(memac->memac_drv_param)); tmp = ioread32be(®s->command_config); - if (mode & COMM_MODE_RX) - tmp &= ~CMD_CFG_RX_EN; - if (mode & COMM_MODE_TX) - tmp &= ~CMD_CFG_TX_EN; - + tmp &= ~(CMD_CFG_RX_EN | CMD_CFG_TX_EN); iowrite32be(tmp, ®s->command_config); - - return 0; } -int memac_set_promiscuous(struct fman_mac *memac, bool new_val) +static int memac_set_promiscuous(struct fman_mac *memac, bool new_val) { struct memac_regs __iomem *regs = memac->regs; u32 tmp; @@ -769,7 +733,7 @@ int memac_set_promiscuous(struct fman_mac *memac, bool new_val) return 0; } -int memac_adjust_link(struct fman_mac *memac, u16 speed) +static int memac_adjust_link(struct fman_mac *memac, u16 speed) { struct memac_regs __iomem *regs = memac->regs; u32 tmp; @@ -809,39 +773,26 @@ int memac_adjust_link(struct fman_mac *memac, u16 speed) return 0; } -int memac_cfg_max_frame_len(struct fman_mac *memac, u16 new_val) -{ - if (is_init_done(memac->memac_drv_param)) - return -EINVAL; - - memac->memac_drv_param->max_frame_length = new_val; - - return 0; -} - -int memac_cfg_reset_on_init(struct fman_mac *memac, bool enable) -{ - if (is_init_done(memac->memac_drv_param)) - return -EINVAL; - - memac->memac_drv_param->reset_on_init = enable; - - return 0; -} - -int memac_cfg_fixed_link(struct fman_mac *memac, - struct fixed_phy_status *fixed_link) +static void adjust_link_memac(struct mac_device *mac_dev) { - if (is_init_done(memac->memac_drv_param)) - return -EINVAL; + struct phy_device *phy_dev = mac_dev->phy_dev; + struct fman_mac *fman_mac; + bool rx_pause, tx_pause; + int err; - memac->memac_drv_param->fixed_link = fixed_link; + fman_mac = mac_dev->fman_mac; + memac_adjust_link(fman_mac, phy_dev->speed); + mac_dev->update_speed(mac_dev, phy_dev->speed); - return 0; + fman_get_pause_cfg(mac_dev, &rx_pause, &tx_pause); + err = fman_set_mac_active_pause(mac_dev, rx_pause, tx_pause); + if (err < 0) + dev_err(mac_dev->dev, "fman_set_mac_active_pause() = %d\n", + err); } -int memac_set_tx_pause_frames(struct fman_mac *memac, u8 priority, - u16 pause_time, u16 thresh_time) +static int memac_set_tx_pause_frames(struct fman_mac *memac, u8 priority, + u16 pause_time, u16 thresh_time) { struct memac_regs __iomem *regs = memac->regs; u32 tmp; @@ -878,7 +829,7 @@ int memac_set_tx_pause_frames(struct fman_mac *memac, u8 priority, return 0; } -int memac_accept_rx_pause_frames(struct fman_mac *memac, bool en) +static int memac_accept_rx_pause_frames(struct fman_mac *memac, bool en) { struct memac_regs __iomem *regs = memac->regs; u32 tmp; @@ -897,7 +848,8 @@ int memac_accept_rx_pause_frames(struct fman_mac *memac, bool en) return 0; } -int memac_modify_mac_address(struct fman_mac *memac, const enet_addr_t *enet_addr) +static int memac_modify_mac_address(struct fman_mac *memac, + const enet_addr_t *enet_addr) { if (!is_init_done(memac->memac_drv_param)) return -EINVAL; @@ -907,7 +859,8 @@ int memac_modify_mac_address(struct fman_mac *memac, const enet_addr_t *enet_add return 0; } -int memac_add_hash_mac_address(struct fman_mac *memac, enet_addr_t *eth_addr) +static int memac_add_hash_mac_address(struct fman_mac *memac, + enet_addr_t *eth_addr) { struct memac_regs __iomem *regs = memac->regs; struct eth_hash_entry *hash_entry; @@ -940,7 +893,7 @@ int memac_add_hash_mac_address(struct fman_mac *memac, enet_addr_t *eth_addr) return 0; } -int memac_set_allmulti(struct fman_mac *memac, bool enable) +static int memac_set_allmulti(struct fman_mac *memac, bool enable) { u32 entry; struct memac_regs __iomem *regs = memac->regs; @@ -963,12 +916,13 @@ int memac_set_allmulti(struct fman_mac *memac, bool enable) return 0; } -int memac_set_tstamp(struct fman_mac *memac, bool enable) +static int memac_set_tstamp(struct fman_mac *memac, bool enable) { return 0; /* Always enabled. */ } -int memac_del_hash_mac_address(struct fman_mac *memac, enet_addr_t *eth_addr) +static int memac_del_hash_mac_address(struct fman_mac *memac, + enet_addr_t *eth_addr) { struct memac_regs __iomem *regs = memac->regs; struct eth_hash_entry *hash_entry = NULL; @@ -1001,8 +955,8 @@ int memac_del_hash_mac_address(struct fman_mac *memac, enet_addr_t *eth_addr) return 0; } -int memac_set_exception(struct fman_mac *memac, - enum fman_mac_exceptions exception, bool enable) +static int memac_set_exception(struct fman_mac *memac, + enum fman_mac_exceptions exception, bool enable) { u32 bit_mask = 0; @@ -1024,13 +978,13 @@ int memac_set_exception(struct fman_mac *memac, return 0; } -int memac_init(struct fman_mac *memac) +static int memac_init(struct fman_mac *memac) { struct memac_cfg *memac_drv_param; u8 i; enet_addr_t eth_addr; bool slow_10g_if = false; - struct fixed_phy_status *fixed_link; + struct fixed_phy_status *fixed_link = NULL; int err; u32 reg32 = 0; @@ -1141,7 +1095,7 @@ int memac_init(struct fman_mac *memac) return 0; } -int memac_free(struct fman_mac *memac) +static int memac_free(struct fman_mac *memac) { free_init_resources(memac); @@ -1154,13 +1108,12 @@ int memac_free(struct fman_mac *memac) return 0; } -struct fman_mac *memac_config(struct fman_mac_params *params) +static struct fman_mac *memac_config(struct mac_device *mac_dev, + struct fman_mac_params *params) { struct fman_mac *memac; struct memac_cfg *memac_drv_param; - void __iomem *base_addr; - base_addr = params->base_addr; /* allocate memory for the m_emac data structure */ memac = kzalloc(sizeof(*memac), GFP_KERNEL); if (!memac) @@ -1178,38 +1131,121 @@ struct fman_mac *memac_config(struct fman_mac_params *params) set_dflts(memac_drv_param); - memac->addr = ENET_ADDR_TO_UINT64(params->addr); + memac->addr = ENET_ADDR_TO_UINT64(mac_dev->addr); - memac->regs = base_addr; + memac->regs = mac_dev->vaddr; memac->max_speed = params->max_speed; - memac->phy_if = params->phy_if; + memac->phy_if = mac_dev->phy_if; memac->mac_id = params->mac_id; memac->exceptions = (MEMAC_IMASK_TSECC_ER | MEMAC_IMASK_TECC_ER | MEMAC_IMASK_RECC_ER | MEMAC_IMASK_MGI); memac->exception_cb = params->exception_cb; memac->event_cb = params->event_cb; - memac->dev_id = params->dev_id; + memac->dev_id = mac_dev; memac->fm = params->fm; memac->basex_if = params->basex_if; /* Save FMan revision */ fman_get_revision(memac->fm, &memac->fm_rev_info); + return memac; +} + +int memac_initialization(struct mac_device *mac_dev, + struct device_node *mac_node, + struct fman_mac_params *params) +{ + int err; + struct device_node *phy_node; + struct fixed_phy_status *fixed_link; + struct fman_mac *memac; + + mac_dev->set_promisc = memac_set_promiscuous; + mac_dev->change_addr = memac_modify_mac_address; + mac_dev->add_hash_mac_addr = memac_add_hash_mac_address; + mac_dev->remove_hash_mac_addr = memac_del_hash_mac_address; + mac_dev->set_tx_pause = memac_set_tx_pause_frames; + mac_dev->set_rx_pause = memac_accept_rx_pause_frames; + mac_dev->set_exception = memac_set_exception; + mac_dev->set_allmulti = memac_set_allmulti; + mac_dev->set_tstamp = memac_set_tstamp; + mac_dev->set_multi = fman_set_multi; + mac_dev->adjust_link = adjust_link_memac; + mac_dev->enable = memac_enable; + mac_dev->disable = memac_disable; + + if (params->max_speed == SPEED_10000) + mac_dev->phy_if = PHY_INTERFACE_MODE_XGMII; + + mac_dev->fman_mac = memac_config(mac_dev, params); + if (!mac_dev->fman_mac) { + err = -EINVAL; + goto _return; + } + + memac = mac_dev->fman_mac; + memac->memac_drv_param->max_frame_length = fman_get_max_frm(); + memac->memac_drv_param->reset_on_init = true; if (memac->phy_if == PHY_INTERFACE_MODE_SGMII || memac->phy_if == PHY_INTERFACE_MODE_QSGMII) { - if (!params->internal_phy_node) { + phy_node = of_parse_phandle(mac_node, "pcsphy-handle", 0); + if (!phy_node) { pr_err("PCS PHY node is not available\n"); - memac_free(memac); - return NULL; + err = -EINVAL; + goto _return_fm_mac_free; } - memac->pcsphy = of_phy_find_device(params->internal_phy_node); + memac->pcsphy = of_phy_find_device(phy_node); if (!memac->pcsphy) { pr_err("of_phy_find_device (PCS PHY) failed\n"); - memac_free(memac); - return NULL; + err = -EINVAL; + goto _return_fm_mac_free; } } - return memac; + if (!mac_dev->phy_node && of_phy_is_fixed_link(mac_node)) { + struct phy_device *phy; + + err = of_phy_register_fixed_link(mac_node); + if (err) + goto _return_fm_mac_free; + + fixed_link = kzalloc(sizeof(*fixed_link), GFP_KERNEL); + if (!fixed_link) { + err = -ENOMEM; + goto _return_fm_mac_free; + } + + mac_dev->phy_node = of_node_get(mac_node); + phy = of_phy_find_device(mac_dev->phy_node); + if (!phy) { + err = -EINVAL; + of_node_put(mac_dev->phy_node); + goto _return_fixed_link_free; + } + + fixed_link->link = phy->link; + fixed_link->speed = phy->speed; + fixed_link->duplex = phy->duplex; + fixed_link->pause = phy->pause; + fixed_link->asym_pause = phy->asym_pause; + + put_device(&phy->mdio.dev); + memac->memac_drv_param->fixed_link = fixed_link; + } + + err = memac_init(mac_dev->fman_mac); + if (err < 0) + goto _return_fixed_link_free; + + dev_info(mac_dev->dev, "FMan MEMAC\n"); + + goto _return; + +_return_fixed_link_free: + kfree(fixed_link); +_return_fm_mac_free: + memac_free(mac_dev->fman_mac); +_return: + return err; } diff --git a/drivers/net/ethernet/freescale/fman/fman_memac.h b/drivers/net/ethernet/freescale/fman/fman_memac.h index 3820f7a22983..5a3a14f9684f 100644 --- a/drivers/net/ethernet/freescale/fman/fman_memac.h +++ b/drivers/net/ethernet/freescale/fman/fman_memac.h @@ -1,33 +1,6 @@ +/* SPDX-License-Identifier: BSD-3-Clause OR GPL-2.0-or-later */ /* - * Copyright 2008-2015 Freescale Semiconductor Inc. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * * Redistributions of source code must retain the above copyright - * notice, this list of conditions and the following disclaimer. - * * Redistributions in binary form must reproduce the above copyright - * notice, this list of conditions and the following disclaimer in the - * documentation and/or other materials provided with the distribution. - * * Neither the name of Freescale Semiconductor nor the - * names of its contributors may be used to endorse or promote products - * derived from this software without specific prior written permission. - * - * - * ALTERNATIVELY, this software may be distributed under the terms of the - * GNU General Public License ("GPL") as published by the Free Software - * Foundation, either version 2 of that License or (at your option) any - * later version. - * - * THIS SOFTWARE IS PROVIDED BY Freescale Semiconductor ``AS IS'' AND ANY - * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL Freescale Semiconductor BE LIABLE FOR ANY - * DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES - * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; - * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND - * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT - * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS - * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + * Copyright 2008 - 2015 Freescale Semiconductor Inc. */ #ifndef __MEMAC_H @@ -38,26 +11,10 @@ #include <linux/netdevice.h> #include <linux/phy_fixed.h> -struct fman_mac *memac_config(struct fman_mac_params *params); -int memac_set_promiscuous(struct fman_mac *memac, bool new_val); -int memac_modify_mac_address(struct fman_mac *memac, const enet_addr_t *enet_addr); -int memac_adjust_link(struct fman_mac *memac, u16 speed); -int memac_cfg_max_frame_len(struct fman_mac *memac, u16 new_val); -int memac_cfg_reset_on_init(struct fman_mac *memac, bool enable); -int memac_cfg_fixed_link(struct fman_mac *memac, - struct fixed_phy_status *fixed_link); -int memac_enable(struct fman_mac *memac, enum comm_mode mode); -int memac_disable(struct fman_mac *memac, enum comm_mode mode); -int memac_init(struct fman_mac *memac); -int memac_free(struct fman_mac *memac); -int memac_accept_rx_pause_frames(struct fman_mac *memac, bool en); -int memac_set_tx_pause_frames(struct fman_mac *memac, u8 priority, - u16 pause_time, u16 thresh_time); -int memac_set_exception(struct fman_mac *memac, - enum fman_mac_exceptions exception, bool enable); -int memac_add_hash_mac_address(struct fman_mac *memac, enet_addr_t *eth_addr); -int memac_del_hash_mac_address(struct fman_mac *memac, enet_addr_t *eth_addr); -int memac_set_allmulti(struct fman_mac *memac, bool enable); -int memac_set_tstamp(struct fman_mac *memac, bool enable); +struct mac_device; + +int memac_initialization(struct mac_device *mac_dev, + struct device_node *mac_node, + struct fman_mac_params *params); #endif /* __MEMAC_H */ diff --git a/drivers/net/ethernet/freescale/fman/fman_muram.c b/drivers/net/ethernet/freescale/fman/fman_muram.c index 7ad317e622bc..f557d68e5b76 100644 --- a/drivers/net/ethernet/freescale/fman/fman_muram.c +++ b/drivers/net/ethernet/freescale/fman/fman_muram.c @@ -1,33 +1,6 @@ +// SPDX-License-Identifier: BSD-3-Clause OR GPL-2.0-or-later /* - * Copyright 2008-2015 Freescale Semiconductor Inc. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * * Redistributions of source code must retain the above copyright - * notice, this list of conditions and the following disclaimer. - * * Redistributions in binary form must reproduce the above copyright - * notice, this list of conditions and the following disclaimer in the - * documentation and/or other materials provided with the distribution. - * * Neither the name of Freescale Semiconductor nor the - * names of its contributors may be used to endorse or promote products - * derived from this software without specific prior written permission. - * - * - * ALTERNATIVELY, this software may be distributed under the terms of the - * GNU General Public License ("GPL") as published by the Free Software - * Foundation, either version 2 of that License or (at your option) any - * later version. - * - * THIS SOFTWARE IS PROVIDED BY Freescale Semiconductor ``AS IS'' AND ANY - * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL Freescale Semiconductor BE LIABLE FOR ANY - * DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES - * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; - * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND - * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT - * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS - * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + * Copyright 2008 - 2015 Freescale Semiconductor Inc. */ #include "fman_muram.h" diff --git a/drivers/net/ethernet/freescale/fman/fman_muram.h b/drivers/net/ethernet/freescale/fman/fman_muram.h index 453bf849eee1..3643af61bae2 100644 --- a/drivers/net/ethernet/freescale/fman/fman_muram.h +++ b/drivers/net/ethernet/freescale/fman/fman_muram.h @@ -1,34 +1,8 @@ +/* SPDX-License-Identifier: BSD-3-Clause OR GPL-2.0-or-later */ /* - * Copyright 2008-2015 Freescale Semiconductor Inc. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * * Redistributions of source code must retain the above copyright - * notice, this list of conditions and the following disclaimer. - * * Redistributions in binary form must reproduce the above copyright - * notice, this list of conditions and the following disclaimer in the - * documentation and/or other materials provided with the distribution. - * * Neither the name of Freescale Semiconductor nor the - * names of its contributors may be used to endorse or promote products - * derived from this software without specific prior written permission. - * - * - * ALTERNATIVELY, this software may be distributed under the terms of the - * GNU General Public License ("GPL") as published by the Free Software - * Foundation, either version 2 of that License or (at your option) any - * later version. - * - * THIS SOFTWARE IS PROVIDED BY Freescale Semiconductor ``AS IS'' AND ANY - * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL Freescale Semiconductor BE LIABLE FOR ANY - * DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES - * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; - * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND - * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT - * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS - * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + * Copyright 2008 - 2015 Freescale Semiconductor Inc. */ + #ifndef __FM_MURAM_EXT #define __FM_MURAM_EXT diff --git a/drivers/net/ethernet/freescale/fman/fman_port.c b/drivers/net/ethernet/freescale/fman/fman_port.c index 4c9d05c45c03..ab90fe2bee5e 100644 --- a/drivers/net/ethernet/freescale/fman/fman_port.c +++ b/drivers/net/ethernet/freescale/fman/fman_port.c @@ -1,33 +1,6 @@ +// SPDX-License-Identifier: BSD-3-Clause OR GPL-2.0-or-later /* * Copyright 2008 - 2015 Freescale Semiconductor Inc. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * * Redistributions of source code must retain the above copyright - * notice, this list of conditions and the following disclaimer. - * * Redistributions in binary form must reproduce the above copyright - * notice, this list of conditions and the following disclaimer in the - * documentation and/or other materials provided with the distribution. - * * Neither the name of Freescale Semiconductor nor the - * names of its contributors may be used to endorse or promote products - * derived from this software without specific prior written permission. - * - * - * ALTERNATIVELY, this software may be distributed under the terms of the - * GNU General Public License ("GPL") as published by the Free Software - * Foundation, either version 2 of that License or (at your option) any - * later version. - * - * THIS SOFTWARE IS PROVIDED BY Freescale Semiconductor ``AS IS'' AND ANY - * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL Freescale Semiconductor BE LIABLE FOR ANY - * DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES - * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; - * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND - * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT - * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS - * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt diff --git a/drivers/net/ethernet/freescale/fman/fman_port.h b/drivers/net/ethernet/freescale/fman/fman_port.h index 82f12661a46d..4917fe8f0617 100644 --- a/drivers/net/ethernet/freescale/fman/fman_port.h +++ b/drivers/net/ethernet/freescale/fman/fman_port.h @@ -1,33 +1,6 @@ +/* SPDX-License-Identifier: BSD-3-Clause OR GPL-2.0-or-later */ /* * Copyright 2008 - 2015 Freescale Semiconductor Inc. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * * Redistributions of source code must retain the above copyright - * notice, this list of conditions and the following disclaimer. - * * Redistributions in binary form must reproduce the above copyright - * notice, this list of conditions and the following disclaimer in the - * documentation and/or other materials provided with the distribution. - * * Neither the name of Freescale Semiconductor nor the - * names of its contributors may be used to endorse or promote products - * derived from this software without specific prior written permission. - * - * - * ALTERNATIVELY, this software may be distributed under the terms of the - * GNU General Public License ("GPL") as published by the Free Software - * Foundation, either version 2 of that License or (at your option) any - * later version. - * - * THIS SOFTWARE IS PROVIDED BY Freescale Semiconductor ``AS IS'' AND ANY - * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL Freescale Semiconductor BE LIABLE FOR ANY - * DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES - * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; - * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND - * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT - * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS - * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ #ifndef __FMAN_PORT_H diff --git a/drivers/net/ethernet/freescale/fman/fman_sp.c b/drivers/net/ethernet/freescale/fman/fman_sp.c index 248f5bcca468..0fac60aa5283 100644 --- a/drivers/net/ethernet/freescale/fman/fman_sp.c +++ b/drivers/net/ethernet/freescale/fman/fman_sp.c @@ -1,33 +1,6 @@ +// SPDX-License-Identifier: BSD-3-Clause OR GPL-2.0-or-later /* * Copyright 2008 - 2015 Freescale Semiconductor Inc. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * * Redistributions of source code must retain the above copyright - * notice, this list of conditions and the following disclaimer. - * * Redistributions in binary form must reproduce the above copyright - * notice, this list of conditions and the following disclaimer in the - * documentation and/or other materials provided with the distribution. - * * Neither the name of Freescale Semiconductor nor the - * names of its contributors may be used to endorse or promote products - * derived from this software without specific prior written permission. - * - * - * ALTERNATIVELY, this software may be distributed under the terms of the - * GNU General Public License ("GPL") as published by the Free Software - * Foundation, either version 2 of that License or (at your option) any - * later version. - * - * THIS SOFTWARE IS PROVIDED BY Freescale Semiconductor ``AS IS'' AND ANY - * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL Freescale Semiconductor BE LIABLE FOR ANY - * DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES - * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; - * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND - * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT - * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS - * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ #include "fman_sp.h" diff --git a/drivers/net/ethernet/freescale/fman/fman_sp.h b/drivers/net/ethernet/freescale/fman/fman_sp.h index 820b7f63088f..a62dd21c81f1 100644 --- a/drivers/net/ethernet/freescale/fman/fman_sp.h +++ b/drivers/net/ethernet/freescale/fman/fman_sp.h @@ -1,32 +1,6 @@ +/* SPDX-License-Identifier: BSD-3-Clause OR GPL-2.0-or-later */ /* * Copyright 2008 - 2015 Freescale Semiconductor Inc. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * * Redistributions of source code must retain the above copyright - * notice, this list of conditions and the following disclaimer. - * * Redistributions in binary form must reproduce the above copyright - * notice, this list of conditions and the following disclaimer in the - * documentation and/or other materials provided with the distribution. - * * Neither the name of Freescale Semiconductor nor the - * names of its contributors may be used to endorse or promote products - * derived from this software without specific prior written permission. - * - * ALTERNATIVELY, this software may be distributed under the terms of the - * GNU General Public License ("GPL") as published by the Free Software - * Foundation, either version 2 of that License or (at your option) any - * later version. - * - * THIS SOFTWARE IS PROVIDED BY Freescale Semiconductor ``AS IS'' AND ANY - * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL Freescale Semiconductor BE LIABLE FOR ANY - * DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES - * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; - * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND - * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT - * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS - * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ #ifndef __FM_SP_H diff --git a/drivers/net/ethernet/freescale/fman/fman_tgec.c b/drivers/net/ethernet/freescale/fman/fman_tgec.c index 311c1906e044..5a4be54ad459 100644 --- a/drivers/net/ethernet/freescale/fman/fman_tgec.c +++ b/drivers/net/ethernet/freescale/fman/fman_tgec.c @@ -1,39 +1,13 @@ +// SPDX-License-Identifier: BSD-3-Clause OR GPL-2.0-or-later /* - * Copyright 2008-2015 Freescale Semiconductor Inc. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * * Redistributions of source code must retain the above copyright - * notice, this list of conditions and the following disclaimer. - * * Redistributions in binary form must reproduce the above copyright - * notice, this list of conditions and the following disclaimer in the - * documentation and/or other materials provided with the distribution. - * * Neither the name of Freescale Semiconductor nor the - * names of its contributors may be used to endorse or promote products - * derived from this software without specific prior written permission. - * - * - * ALTERNATIVELY, this software may be distributed under the terms of the - * GNU General Public License ("GPL") as published by the Free Software - * Foundation, either version 2 of that License or (at your option) any - * later version. - * - * THIS SOFTWARE IS PROVIDED BY Freescale Semiconductor ``AS IS'' AND ANY - * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL Freescale Semiconductor BE LIABLE FOR ANY - * DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES - * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; - * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND - * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT - * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS - * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + * Copyright 2008 - 2015 Freescale Semiconductor Inc. */ #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt #include "fman_tgec.h" #include "fman.h" +#include "mac.h" #include <linux/slab.h> #include <linux/bitrev.h> @@ -206,7 +180,7 @@ struct fman_mac { /* MAC address of device; */ u64 addr; u16 max_speed; - void *dev_id; /* device cookie used by the exception cbs */ + struct mac_device *dev_id; /* device cookie used by the exception cbs */ fman_mac_exception_cb *exception_cb; fman_mac_exception_cb *event_cb; /* pointer to driver's global address hash table */ @@ -419,7 +393,7 @@ static bool is_init_done(struct tgec_cfg *cfg) return false; } -int tgec_enable(struct fman_mac *tgec, enum comm_mode mode) +static int tgec_enable(struct fman_mac *tgec) { struct tgec_regs __iomem *regs = tgec->regs; u32 tmp; @@ -428,34 +402,25 @@ int tgec_enable(struct fman_mac *tgec, enum comm_mode mode) return -EINVAL; tmp = ioread32be(®s->command_config); - if (mode & COMM_MODE_RX) - tmp |= CMD_CFG_RX_EN; - if (mode & COMM_MODE_TX) - tmp |= CMD_CFG_TX_EN; + tmp |= CMD_CFG_RX_EN | CMD_CFG_TX_EN; iowrite32be(tmp, ®s->command_config); return 0; } -int tgec_disable(struct fman_mac *tgec, enum comm_mode mode) +static void tgec_disable(struct fman_mac *tgec) { struct tgec_regs __iomem *regs = tgec->regs; u32 tmp; - if (!is_init_done(tgec->cfg)) - return -EINVAL; + WARN_ON_ONCE(!is_init_done(tgec->cfg)); tmp = ioread32be(®s->command_config); - if (mode & COMM_MODE_RX) - tmp &= ~CMD_CFG_RX_EN; - if (mode & COMM_MODE_TX) - tmp &= ~CMD_CFG_TX_EN; + tmp &= ~(CMD_CFG_RX_EN | CMD_CFG_TX_EN); iowrite32be(tmp, ®s->command_config); - - return 0; } -int tgec_set_promiscuous(struct fman_mac *tgec, bool new_val) +static int tgec_set_promiscuous(struct fman_mac *tgec, bool new_val) { struct tgec_regs __iomem *regs = tgec->regs; u32 tmp; @@ -473,18 +438,9 @@ int tgec_set_promiscuous(struct fman_mac *tgec, bool new_val) return 0; } -int tgec_cfg_max_frame_len(struct fman_mac *tgec, u16 new_val) -{ - if (is_init_done(tgec->cfg)) - return -EINVAL; - - tgec->cfg->max_frame_length = new_val; - - return 0; -} - -int tgec_set_tx_pause_frames(struct fman_mac *tgec, u8 __maybe_unused priority, - u16 pause_time, u16 __maybe_unused thresh_time) +static int tgec_set_tx_pause_frames(struct fman_mac *tgec, + u8 __maybe_unused priority, u16 pause_time, + u16 __maybe_unused thresh_time) { struct tgec_regs __iomem *regs = tgec->regs; @@ -496,7 +452,7 @@ int tgec_set_tx_pause_frames(struct fman_mac *tgec, u8 __maybe_unused priority, return 0; } -int tgec_accept_rx_pause_frames(struct fman_mac *tgec, bool en) +static int tgec_accept_rx_pause_frames(struct fman_mac *tgec, bool en) { struct tgec_regs __iomem *regs = tgec->regs; u32 tmp; @@ -514,7 +470,8 @@ int tgec_accept_rx_pause_frames(struct fman_mac *tgec, bool en) return 0; } -int tgec_modify_mac_address(struct fman_mac *tgec, const enet_addr_t *p_enet_addr) +static int tgec_modify_mac_address(struct fman_mac *tgec, + const enet_addr_t *p_enet_addr) { if (!is_init_done(tgec->cfg)) return -EINVAL; @@ -525,7 +482,8 @@ int tgec_modify_mac_address(struct fman_mac *tgec, const enet_addr_t *p_enet_add return 0; } -int tgec_add_hash_mac_address(struct fman_mac *tgec, enet_addr_t *eth_addr) +static int tgec_add_hash_mac_address(struct fman_mac *tgec, + enet_addr_t *eth_addr) { struct tgec_regs __iomem *regs = tgec->regs; struct eth_hash_entry *hash_entry; @@ -562,7 +520,7 @@ int tgec_add_hash_mac_address(struct fman_mac *tgec, enet_addr_t *eth_addr) return 0; } -int tgec_set_allmulti(struct fman_mac *tgec, bool enable) +static int tgec_set_allmulti(struct fman_mac *tgec, bool enable) { u32 entry; struct tgec_regs __iomem *regs = tgec->regs; @@ -585,7 +543,7 @@ int tgec_set_allmulti(struct fman_mac *tgec, bool enable) return 0; } -int tgec_set_tstamp(struct fman_mac *tgec, bool enable) +static int tgec_set_tstamp(struct fman_mac *tgec, bool enable) { struct tgec_regs __iomem *regs = tgec->regs; u32 tmp; @@ -605,7 +563,8 @@ int tgec_set_tstamp(struct fman_mac *tgec, bool enable) return 0; } -int tgec_del_hash_mac_address(struct fman_mac *tgec, enet_addr_t *eth_addr) +static int tgec_del_hash_mac_address(struct fman_mac *tgec, + enet_addr_t *eth_addr) { struct tgec_regs __iomem *regs = tgec->regs; struct eth_hash_entry *hash_entry = NULL; @@ -642,20 +601,15 @@ int tgec_del_hash_mac_address(struct fman_mac *tgec, enet_addr_t *eth_addr) return 0; } -int tgec_get_version(struct fman_mac *tgec, u32 *mac_version) +static void tgec_adjust_link(struct mac_device *mac_dev) { - struct tgec_regs __iomem *regs = tgec->regs; - - if (!is_init_done(tgec->cfg)) - return -EINVAL; + struct phy_device *phy_dev = mac_dev->phy_dev; - *mac_version = ioread32be(®s->tgec_id); - - return 0; + mac_dev->update_speed(mac_dev, phy_dev->speed); } -int tgec_set_exception(struct fman_mac *tgec, - enum fman_mac_exceptions exception, bool enable) +static int tgec_set_exception(struct fman_mac *tgec, + enum fman_mac_exceptions exception, bool enable) { struct tgec_regs __iomem *regs = tgec->regs; u32 bit_mask = 0; @@ -681,7 +635,7 @@ int tgec_set_exception(struct fman_mac *tgec, return 0; } -int tgec_init(struct fman_mac *tgec) +static int tgec_init(struct fman_mac *tgec) { struct tgec_cfg *cfg; enet_addr_t eth_addr; @@ -764,7 +718,7 @@ int tgec_init(struct fman_mac *tgec) return 0; } -int tgec_free(struct fman_mac *tgec) +static int tgec_free(struct fman_mac *tgec) { free_init_resources(tgec); @@ -774,13 +728,12 @@ int tgec_free(struct fman_mac *tgec) return 0; } -struct fman_mac *tgec_config(struct fman_mac_params *params) +static struct fman_mac *tgec_config(struct mac_device *mac_dev, + struct fman_mac_params *params) { struct fman_mac *tgec; struct tgec_cfg *cfg; - void __iomem *base_addr; - base_addr = params->base_addr; /* allocate memory for the UCC GETH data structure. */ tgec = kzalloc(sizeof(*tgec), GFP_KERNEL); if (!tgec) @@ -798,8 +751,8 @@ struct fman_mac *tgec_config(struct fman_mac_params *params) set_dflts(cfg); - tgec->regs = base_addr; - tgec->addr = ENET_ADDR_TO_UINT64(params->addr); + tgec->regs = mac_dev->vaddr; + tgec->addr = ENET_ADDR_TO_UINT64(mac_dev->addr); tgec->max_speed = params->max_speed; tgec->mac_id = params->mac_id; tgec->exceptions = (TGEC_IMASK_MDIO_SCAN_EVENT | @@ -819,7 +772,7 @@ struct fman_mac *tgec_config(struct fman_mac_params *params) TGEC_IMASK_RX_ALIGN_ER); tgec->exception_cb = params->exception_cb; tgec->event_cb = params->event_cb; - tgec->dev_id = params->dev_id; + tgec->dev_id = mac_dev; tgec->fm = params->fm; /* Save FMan revision */ @@ -827,3 +780,52 @@ struct fman_mac *tgec_config(struct fman_mac_params *params) return tgec; } + +int tgec_initialization(struct mac_device *mac_dev, + struct device_node *mac_node, + struct fman_mac_params *params) +{ + int err; + struct fman_mac *tgec; + + mac_dev->set_promisc = tgec_set_promiscuous; + mac_dev->change_addr = tgec_modify_mac_address; + mac_dev->add_hash_mac_addr = tgec_add_hash_mac_address; + mac_dev->remove_hash_mac_addr = tgec_del_hash_mac_address; + mac_dev->set_tx_pause = tgec_set_tx_pause_frames; + mac_dev->set_rx_pause = tgec_accept_rx_pause_frames; + mac_dev->set_exception = tgec_set_exception; + mac_dev->set_allmulti = tgec_set_allmulti; + mac_dev->set_tstamp = tgec_set_tstamp; + mac_dev->set_multi = fman_set_multi; + mac_dev->adjust_link = tgec_adjust_link; + mac_dev->enable = tgec_enable; + mac_dev->disable = tgec_disable; + + mac_dev->fman_mac = tgec_config(mac_dev, params); + if (!mac_dev->fman_mac) { + err = -EINVAL; + goto _return; + } + + tgec = mac_dev->fman_mac; + tgec->cfg->max_frame_length = fman_get_max_frm(); + err = tgec_init(tgec); + if (err < 0) + goto _return_fm_mac_free; + + /* For 10G MAC, disable Tx ECC exception */ + err = tgec_set_exception(tgec, FM_MAC_EX_10G_TX_ECC_ER, false); + if (err < 0) + goto _return_fm_mac_free; + + pr_info("FMan XGEC version: 0x%08x\n", + ioread32be(&tgec->regs->tgec_id)); + goto _return; + +_return_fm_mac_free: + tgec_free(mac_dev->fman_mac); + +_return: + return err; +} diff --git a/drivers/net/ethernet/freescale/fman/fman_tgec.h b/drivers/net/ethernet/freescale/fman/fman_tgec.h index b28b20b26148..768b8d165e05 100644 --- a/drivers/net/ethernet/freescale/fman/fman_tgec.h +++ b/drivers/net/ethernet/freescale/fman/fman_tgec.h @@ -1,33 +1,6 @@ +/* SPDX-License-Identifier: BSD-3-Clause OR GPL-2.0-or-later */ /* - * Copyright 2008-2015 Freescale Semiconductor Inc. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * * Redistributions of source code must retain the above copyright - * notice, this list of conditions and the following disclaimer. - * * Redistributions in binary form must reproduce the above copyright - * notice, this list of conditions and the following disclaimer in the - * documentation and/or other materials provided with the distribution. - * * Neither the name of Freescale Semiconductor nor the - * names of its contributors may be used to endorse or promote products - * derived from this software without specific prior written permission. - * - * - * ALTERNATIVELY, this software may be distributed under the terms of the - * GNU General Public License ("GPL") as published by the Free Software - * Foundation, either version 2 of that License or (at your option) any - * later version. - * - * THIS SOFTWARE IS PROVIDED BY Freescale Semiconductor ``AS IS'' AND ANY - * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL Freescale Semiconductor BE LIABLE FOR ANY - * DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES - * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; - * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND - * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT - * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS - * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + * Copyright 2008 - 2015 Freescale Semiconductor Inc. */ #ifndef __TGEC_H @@ -35,23 +8,10 @@ #include "fman_mac.h" -struct fman_mac *tgec_config(struct fman_mac_params *params); -int tgec_set_promiscuous(struct fman_mac *tgec, bool new_val); -int tgec_modify_mac_address(struct fman_mac *tgec, const enet_addr_t *enet_addr); -int tgec_cfg_max_frame_len(struct fman_mac *tgec, u16 new_val); -int tgec_enable(struct fman_mac *tgec, enum comm_mode mode); -int tgec_disable(struct fman_mac *tgec, enum comm_mode mode); -int tgec_init(struct fman_mac *tgec); -int tgec_free(struct fman_mac *tgec); -int tgec_accept_rx_pause_frames(struct fman_mac *tgec, bool en); -int tgec_set_tx_pause_frames(struct fman_mac *tgec, u8 priority, - u16 pause_time, u16 thresh_time); -int tgec_set_exception(struct fman_mac *tgec, - enum fman_mac_exceptions exception, bool enable); -int tgec_add_hash_mac_address(struct fman_mac *tgec, enet_addr_t *eth_addr); -int tgec_del_hash_mac_address(struct fman_mac *tgec, enet_addr_t *eth_addr); -int tgec_get_version(struct fman_mac *tgec, u32 *mac_version); -int tgec_set_allmulti(struct fman_mac *tgec, bool enable); -int tgec_set_tstamp(struct fman_mac *tgec, bool enable); +struct mac_device; + +int tgec_initialization(struct mac_device *mac_dev, + struct device_node *mac_node, + struct fman_mac_params *params); #endif /* __TGEC_H */ diff --git a/drivers/net/ethernet/freescale/fman/mac.c b/drivers/net/ethernet/freescale/fman/mac.c index 39ae965cd4f6..7b7526fd7da3 100644 --- a/drivers/net/ethernet/freescale/fman/mac.c +++ b/drivers/net/ethernet/freescale/fman/mac.c @@ -1,32 +1,6 @@ -/* Copyright 2008-2015 Freescale Semiconductor, Inc. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * * Redistributions of source code must retain the above copyright - * notice, this list of conditions and the following disclaimer. - * * Redistributions in binary form must reproduce the above copyright - * notice, this list of conditions and the following disclaimer in the - * documentation and/or other materials provided with the distribution. - * * Neither the name of Freescale Semiconductor nor the - * names of its contributors may be used to endorse or promote products - * derived from this software without specific prior written permission. - * - * - * ALTERNATIVELY, this software may be distributed under the terms of the - * GNU General Public License ("GPL") as published by the Free Software - * Foundation, either version 2 of that License or (at your option) any - * later version. - * - * THIS SOFTWARE IS PROVIDED BY Freescale Semiconductor ``AS IS'' AND ANY - * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL Freescale Semiconductor BE LIABLE FOR ANY - * DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES - * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; - * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND - * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT - * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS - * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +// SPDX-License-Identifier: BSD-3-Clause OR GPL-2.0-or-later +/* + * Copyright 2008 - 2015 Freescale Semiconductor Inc. */ #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt @@ -54,20 +28,12 @@ MODULE_LICENSE("Dual BSD/GPL"); MODULE_DESCRIPTION("FSL FMan MAC API based driver"); struct mac_priv_s { - struct device *dev; - void __iomem *vaddr; u8 cell_index; struct fman *fman; - struct device_node *internal_phy_node; /* List of multicast addresses */ struct list_head mc_addr_list; struct platform_device *eth_dev; - struct fixed_phy_status *fixed_link; u16 speed; - u16 max_speed; - - int (*enable)(struct fman_mac *mac_dev, enum comm_mode mode); - int (*disable)(struct fman_mac *mac_dev, enum comm_mode mode); }; struct mac_address { @@ -75,222 +41,21 @@ struct mac_address { struct list_head list; }; -static void mac_exception(void *handle, enum fman_mac_exceptions ex) +static void mac_exception(struct mac_device *mac_dev, + enum fman_mac_exceptions ex) { - struct mac_device *mac_dev; - struct mac_priv_s *priv; - - mac_dev = handle; - priv = mac_dev->priv; - if (ex == FM_MAC_EX_10G_RX_FIFO_OVFL) { /* don't flag RX FIFO after the first */ mac_dev->set_exception(mac_dev->fman_mac, FM_MAC_EX_10G_RX_FIFO_OVFL, false); - dev_err(priv->dev, "10G MAC got RX FIFO Error = %x\n", ex); + dev_err(mac_dev->dev, "10G MAC got RX FIFO Error = %x\n", ex); } - dev_dbg(priv->dev, "%s:%s() -> %d\n", KBUILD_BASENAME ".c", + dev_dbg(mac_dev->dev, "%s:%s() -> %d\n", KBUILD_BASENAME ".c", __func__, ex); } -static int set_fman_mac_params(struct mac_device *mac_dev, - struct fman_mac_params *params) -{ - struct mac_priv_s *priv = mac_dev->priv; - - params->base_addr = (typeof(params->base_addr)) - devm_ioremap(priv->dev, mac_dev->res->start, - resource_size(mac_dev->res)); - if (!params->base_addr) - return -ENOMEM; - - memcpy(¶ms->addr, mac_dev->addr, sizeof(mac_dev->addr)); - params->max_speed = priv->max_speed; - params->phy_if = mac_dev->phy_if; - params->basex_if = false; - params->mac_id = priv->cell_index; - params->fm = (void *)priv->fman; - params->exception_cb = mac_exception; - params->event_cb = mac_exception; - params->dev_id = mac_dev; - params->internal_phy_node = priv->internal_phy_node; - - return 0; -} - -static int tgec_initialization(struct mac_device *mac_dev) -{ - int err; - struct mac_priv_s *priv; - struct fman_mac_params params; - u32 version; - - priv = mac_dev->priv; - - err = set_fman_mac_params(mac_dev, ¶ms); - if (err) - goto _return; - - mac_dev->fman_mac = tgec_config(¶ms); - if (!mac_dev->fman_mac) { - err = -EINVAL; - goto _return; - } - - err = tgec_cfg_max_frame_len(mac_dev->fman_mac, fman_get_max_frm()); - if (err < 0) - goto _return_fm_mac_free; - - err = tgec_init(mac_dev->fman_mac); - if (err < 0) - goto _return_fm_mac_free; - - /* For 10G MAC, disable Tx ECC exception */ - err = mac_dev->set_exception(mac_dev->fman_mac, - FM_MAC_EX_10G_TX_ECC_ER, false); - if (err < 0) - goto _return_fm_mac_free; - - err = tgec_get_version(mac_dev->fman_mac, &version); - if (err < 0) - goto _return_fm_mac_free; - - dev_info(priv->dev, "FMan XGEC version: 0x%08x\n", version); - - goto _return; - -_return_fm_mac_free: - tgec_free(mac_dev->fman_mac); - -_return: - return err; -} - -static int dtsec_initialization(struct mac_device *mac_dev) -{ - int err; - struct mac_priv_s *priv; - struct fman_mac_params params; - u32 version; - - priv = mac_dev->priv; - - err = set_fman_mac_params(mac_dev, ¶ms); - if (err) - goto _return; - - mac_dev->fman_mac = dtsec_config(¶ms); - if (!mac_dev->fman_mac) { - err = -EINVAL; - goto _return; - } - - err = dtsec_cfg_max_frame_len(mac_dev->fman_mac, fman_get_max_frm()); - if (err < 0) - goto _return_fm_mac_free; - - err = dtsec_cfg_pad_and_crc(mac_dev->fman_mac, true); - if (err < 0) - goto _return_fm_mac_free; - - err = dtsec_init(mac_dev->fman_mac); - if (err < 0) - goto _return_fm_mac_free; - - /* For 1G MAC, disable by default the MIB counters overflow interrupt */ - err = mac_dev->set_exception(mac_dev->fman_mac, - FM_MAC_EX_1G_RX_MIB_CNT_OVFL, false); - if (err < 0) - goto _return_fm_mac_free; - - err = dtsec_get_version(mac_dev->fman_mac, &version); - if (err < 0) - goto _return_fm_mac_free; - - dev_info(priv->dev, "FMan dTSEC version: 0x%08x\n", version); - - goto _return; - -_return_fm_mac_free: - dtsec_free(mac_dev->fman_mac); - -_return: - return err; -} - -static int memac_initialization(struct mac_device *mac_dev) -{ - int err; - struct mac_priv_s *priv; - struct fman_mac_params params; - - priv = mac_dev->priv; - - err = set_fman_mac_params(mac_dev, ¶ms); - if (err) - goto _return; - - if (priv->max_speed == SPEED_10000) - params.phy_if = PHY_INTERFACE_MODE_XGMII; - - mac_dev->fman_mac = memac_config(¶ms); - if (!mac_dev->fman_mac) { - err = -EINVAL; - goto _return; - } - - err = memac_cfg_max_frame_len(mac_dev->fman_mac, fman_get_max_frm()); - if (err < 0) - goto _return_fm_mac_free; - - err = memac_cfg_reset_on_init(mac_dev->fman_mac, true); - if (err < 0) - goto _return_fm_mac_free; - - err = memac_cfg_fixed_link(mac_dev->fman_mac, priv->fixed_link); - if (err < 0) - goto _return_fm_mac_free; - - err = memac_init(mac_dev->fman_mac); - if (err < 0) - goto _return_fm_mac_free; - - dev_info(priv->dev, "FMan MEMAC\n"); - - goto _return; - -_return_fm_mac_free: - memac_free(mac_dev->fman_mac); - -_return: - return err; -} - -static int start(struct mac_device *mac_dev) -{ - int err; - struct phy_device *phy_dev = mac_dev->phy_dev; - struct mac_priv_s *priv = mac_dev->priv; - - err = priv->enable(mac_dev->fman_mac, COMM_MODE_RX_AND_TX); - if (!err && phy_dev) - phy_start(phy_dev); - - return err; -} - -static int stop(struct mac_device *mac_dev) -{ - struct mac_priv_s *priv = mac_dev->priv; - - if (mac_dev->phy_dev) - phy_stop(mac_dev->phy_dev); - - return priv->disable(mac_dev->fman_mac, COMM_MODE_RX_AND_TX); -} - -static int set_multi(struct net_device *net_dev, struct mac_device *mac_dev) +int fman_set_multi(struct net_device *net_dev, struct mac_device *mac_dev) { struct mac_priv_s *priv; struct mac_address *old_addr, *tmp; @@ -424,109 +189,6 @@ void fman_get_pause_cfg(struct mac_device *mac_dev, bool *rx_pause, } EXPORT_SYMBOL(fman_get_pause_cfg); -static void adjust_link_void(struct mac_device *mac_dev) -{ -} - -static void adjust_link_dtsec(struct mac_device *mac_dev) -{ - struct phy_device *phy_dev = mac_dev->phy_dev; - struct fman_mac *fman_mac; - bool rx_pause, tx_pause; - int err; - - fman_mac = mac_dev->fman_mac; - if (!phy_dev->link) { - dtsec_restart_autoneg(fman_mac); - - return; - } - - dtsec_adjust_link(fman_mac, phy_dev->speed); - fman_get_pause_cfg(mac_dev, &rx_pause, &tx_pause); - err = fman_set_mac_active_pause(mac_dev, rx_pause, tx_pause); - if (err < 0) - dev_err(mac_dev->priv->dev, "fman_set_mac_active_pause() = %d\n", - err); -} - -static void adjust_link_memac(struct mac_device *mac_dev) -{ - struct phy_device *phy_dev = mac_dev->phy_dev; - struct fman_mac *fman_mac; - bool rx_pause, tx_pause; - int err; - - fman_mac = mac_dev->fman_mac; - memac_adjust_link(fman_mac, phy_dev->speed); - - fman_get_pause_cfg(mac_dev, &rx_pause, &tx_pause); - err = fman_set_mac_active_pause(mac_dev, rx_pause, tx_pause); - if (err < 0) - dev_err(mac_dev->priv->dev, "fman_set_mac_active_pause() = %d\n", - err); -} - -static void setup_dtsec(struct mac_device *mac_dev) -{ - mac_dev->init = dtsec_initialization; - mac_dev->set_promisc = dtsec_set_promiscuous; - mac_dev->change_addr = dtsec_modify_mac_address; - mac_dev->add_hash_mac_addr = dtsec_add_hash_mac_address; - mac_dev->remove_hash_mac_addr = dtsec_del_hash_mac_address; - mac_dev->set_tx_pause = dtsec_set_tx_pause_frames; - mac_dev->set_rx_pause = dtsec_accept_rx_pause_frames; - mac_dev->set_exception = dtsec_set_exception; - mac_dev->set_allmulti = dtsec_set_allmulti; - mac_dev->set_tstamp = dtsec_set_tstamp; - mac_dev->set_multi = set_multi; - mac_dev->start = start; - mac_dev->stop = stop; - mac_dev->adjust_link = adjust_link_dtsec; - mac_dev->priv->enable = dtsec_enable; - mac_dev->priv->disable = dtsec_disable; -} - -static void setup_tgec(struct mac_device *mac_dev) -{ - mac_dev->init = tgec_initialization; - mac_dev->set_promisc = tgec_set_promiscuous; - mac_dev->change_addr = tgec_modify_mac_address; - mac_dev->add_hash_mac_addr = tgec_add_hash_mac_address; - mac_dev->remove_hash_mac_addr = tgec_del_hash_mac_address; - mac_dev->set_tx_pause = tgec_set_tx_pause_frames; - mac_dev->set_rx_pause = tgec_accept_rx_pause_frames; - mac_dev->set_exception = tgec_set_exception; - mac_dev->set_allmulti = tgec_set_allmulti; - mac_dev->set_tstamp = tgec_set_tstamp; - mac_dev->set_multi = set_multi; - mac_dev->start = start; - mac_dev->stop = stop; - mac_dev->adjust_link = adjust_link_void; - mac_dev->priv->enable = tgec_enable; - mac_dev->priv->disable = tgec_disable; -} - -static void setup_memac(struct mac_device *mac_dev) -{ - mac_dev->init = memac_initialization; - mac_dev->set_promisc = memac_set_promiscuous; - mac_dev->change_addr = memac_modify_mac_address; - mac_dev->add_hash_mac_addr = memac_add_hash_mac_address; - mac_dev->remove_hash_mac_addr = memac_del_hash_mac_address; - mac_dev->set_tx_pause = memac_set_tx_pause_frames; - mac_dev->set_rx_pause = memac_accept_rx_pause_frames; - mac_dev->set_exception = memac_set_exception; - mac_dev->set_allmulti = memac_set_allmulti; - mac_dev->set_tstamp = memac_set_tstamp; - mac_dev->set_multi = set_multi; - mac_dev->start = start; - mac_dev->stop = stop; - mac_dev->adjust_link = adjust_link_memac; - mac_dev->priv->enable = memac_enable; - mac_dev->priv->disable = memac_disable; -} - #define DTSEC_SUPPORTED \ (SUPPORTED_10baseT_Half \ | SUPPORTED_10baseT_Full \ @@ -577,7 +239,7 @@ static struct platform_device *dpaa_eth_add_device(int fman_id, goto no_mem; } - pdev->dev.parent = priv->dev; + pdev->dev.parent = mac_dev->dev; ret = platform_device_add_data(pdev, &data, sizeof(data)); if (ret) @@ -601,9 +263,9 @@ no_mem: } static const struct of_device_id mac_match[] = { - { .compatible = "fsl,fman-dtsec" }, - { .compatible = "fsl,fman-xgec" }, - { .compatible = "fsl,fman-memac" }, + { .compatible = "fsl,fman-dtsec", .data = dtsec_initialization }, + { .compatible = "fsl,fman-xgec", .data = tgec_initialization }, + { .compatible = "fsl,fman-memac", .data = memac_initialization }, {} }; MODULE_DEVICE_TABLE(of, mac_match); @@ -611,50 +273,33 @@ MODULE_DEVICE_TABLE(of, mac_match); static int mac_probe(struct platform_device *_of_dev) { int err, i, nph; + int (*init)(struct mac_device *mac_dev, struct device_node *mac_node, + struct fman_mac_params *params); struct device *dev; struct device_node *mac_node, *dev_node; struct mac_device *mac_dev; struct platform_device *of_dev; - struct resource res; + struct resource *res; struct mac_priv_s *priv; + struct fman_mac_params params; u32 val; u8 fman_id; phy_interface_t phy_if; dev = &_of_dev->dev; mac_node = dev->of_node; + init = of_device_get_match_data(dev); mac_dev = devm_kzalloc(dev, sizeof(*mac_dev), GFP_KERNEL); - if (!mac_dev) { - err = -ENOMEM; - goto _return; - } + if (!mac_dev) + return -ENOMEM; priv = devm_kzalloc(dev, sizeof(*priv), GFP_KERNEL); - if (!priv) { - err = -ENOMEM; - goto _return; - } + if (!priv) + return -ENOMEM; /* Save private information */ mac_dev->priv = priv; - priv->dev = dev; - - if (of_device_is_compatible(mac_node, "fsl,fman-dtsec")) { - setup_dtsec(mac_dev); - priv->internal_phy_node = of_parse_phandle(mac_node, - "tbi-handle", 0); - } else if (of_device_is_compatible(mac_node, "fsl,fman-xgec")) { - setup_tgec(mac_dev); - } else if (of_device_is_compatible(mac_node, "fsl,fman-memac")) { - setup_memac(mac_dev); - priv->internal_phy_node = of_parse_phandle(mac_node, - "pcsphy-handle", 0); - } else { - dev_err(dev, "MAC node (%pOF) contains unsupported MAC\n", - mac_node); - err = -EINVAL; - goto _return; - } + mac_dev->dev = dev; INIT_LIST_HEAD(&priv->mc_addr_list); @@ -663,8 +308,7 @@ static int mac_probe(struct platform_device *_of_dev) if (!dev_node) { dev_err(dev, "of_get_parent(%pOF) failed\n", mac_node); - err = -EINVAL; - goto _return_of_get_parent; + return -EINVAL; } of_dev = of_find_device_by_node(dev_node); @@ -694,42 +338,33 @@ static int mac_probe(struct platform_device *_of_dev) of_node_put(dev_node); /* Get the address of the memory mapped registers */ - err = of_address_to_resource(mac_node, 0, &res); - if (err < 0) { - dev_err(dev, "of_address_to_resource(%pOF) = %d\n", - mac_node, err); - goto _return_of_get_parent; + res = platform_get_mem_or_io(_of_dev, 0); + if (!res) { + dev_err(dev, "could not get registers\n"); + return -EINVAL; } - mac_dev->res = __devm_request_region(dev, - fman_get_mem_region(priv->fman), - res.start, resource_size(&res), - "mac"); - if (!mac_dev->res) { - dev_err(dev, "__devm_request_mem_region(mac) failed\n"); - err = -EBUSY; - goto _return_of_get_parent; + err = devm_request_resource(dev, fman_get_mem_region(priv->fman), res); + if (err) { + dev_err_probe(dev, err, "could not request resource\n"); + return err; } - priv->vaddr = devm_ioremap(dev, mac_dev->res->start, - resource_size(mac_dev->res)); - if (!priv->vaddr) { + mac_dev->vaddr = devm_ioremap(dev, res->start, resource_size(res)); + if (!mac_dev->vaddr) { dev_err(dev, "devm_ioremap() failed\n"); - err = -EIO; - goto _return_of_get_parent; + return -EIO; } + mac_dev->vaddr_end = mac_dev->vaddr + resource_size(res); - if (!of_device_is_available(mac_node)) { - err = -ENODEV; - goto _return_of_get_parent; - } + if (!of_device_is_available(mac_node)) + return -ENODEV; /* Get the cell-index */ err = of_property_read_u32(mac_node, "cell-index", &val); if (err) { dev_err(dev, "failed to read cell-index for %pOF\n", mac_node); - err = -EINVAL; - goto _return_of_get_parent; + return -EINVAL; } priv->cell_index = (u8)val; @@ -743,15 +378,13 @@ static int mac_probe(struct platform_device *_of_dev) if (unlikely(nph < 0)) { dev_err(dev, "of_count_phandle_with_args(%pOF, fsl,fman-ports) failed\n", mac_node); - err = nph; - goto _return_of_get_parent; + return nph; } if (nph != ARRAY_SIZE(mac_dev->port)) { dev_err(dev, "Not supported number of fman-ports handles of mac node %pOF from device tree\n", mac_node); - err = -EINVAL; - goto _return_of_get_parent; + return -EINVAL; } for (i = 0; i < ARRAY_SIZE(mac_dev->port); i++) { @@ -760,8 +393,7 @@ static int mac_probe(struct platform_device *_of_dev) if (!dev_node) { dev_err(dev, "of_parse_phandle(%pOF, fsl,fman-ports) failed\n", mac_node); - err = -EINVAL; - goto _return_of_node_put; + return -EINVAL; } of_dev = of_find_device_by_node(dev_node); @@ -793,7 +425,7 @@ static int mac_probe(struct platform_device *_of_dev) mac_dev->phy_if = phy_if; priv->speed = phy2speed[mac_dev->phy_if]; - priv->max_speed = priv->speed; + params.max_speed = priv->speed; mac_dev->if_support = DTSEC_SUPPORTED; /* We don't support half-duplex in SGMII mode */ if (mac_dev->phy_if == PHY_INTERFACE_MODE_SGMII) @@ -801,7 +433,7 @@ static int mac_probe(struct platform_device *_of_dev) SUPPORTED_100baseT_Half); /* Gigabit support (no half-duplex) */ - if (priv->max_speed == 1000) + if (params.max_speed == 1000) mac_dev->if_support |= SUPPORTED_1000baseT_Full; /* The 10G interface only supports one mode */ @@ -810,42 +442,18 @@ static int mac_probe(struct platform_device *_of_dev) /* Get the rest of the PHY information */ mac_dev->phy_node = of_parse_phandle(mac_node, "phy-handle", 0); - if (!mac_dev->phy_node && of_phy_is_fixed_link(mac_node)) { - struct phy_device *phy; - - err = of_phy_register_fixed_link(mac_node); - if (err) - goto _return_of_get_parent; - - priv->fixed_link = kzalloc(sizeof(*priv->fixed_link), - GFP_KERNEL); - if (!priv->fixed_link) { - err = -ENOMEM; - goto _return_of_get_parent; - } - - mac_dev->phy_node = of_node_get(mac_node); - phy = of_phy_find_device(mac_dev->phy_node); - if (!phy) { - err = -EINVAL; - of_node_put(mac_dev->phy_node); - goto _return_of_get_parent; - } - priv->fixed_link->link = phy->link; - priv->fixed_link->speed = phy->speed; - priv->fixed_link->duplex = phy->duplex; - priv->fixed_link->pause = phy->pause; - priv->fixed_link->asym_pause = phy->asym_pause; + params.basex_if = false; + params.mac_id = priv->cell_index; + params.fm = (void *)priv->fman; + params.exception_cb = mac_exception; + params.event_cb = mac_exception; - put_device(&phy->mdio.dev); - } - - err = mac_dev->init(mac_dev); + err = init(mac_dev, mac_node, ¶ms); if (err < 0) { dev_err(dev, "mac_dev->init() = %d\n", err); of_node_put(mac_dev->phy_node); - goto _return_of_get_parent; + return err; } /* pause frame autonegotiation enabled */ @@ -872,13 +480,10 @@ static int mac_probe(struct platform_device *_of_dev) priv->eth_dev = NULL; } - goto _return; + return err; _return_of_node_put: of_node_put(dev_node); -_return_of_get_parent: - kfree(priv->fixed_link); -_return: return err; } diff --git a/drivers/net/ethernet/freescale/fman/mac.h b/drivers/net/ethernet/freescale/fman/mac.h index daa285a9b8b2..b95d384271bd 100644 --- a/drivers/net/ethernet/freescale/fman/mac.h +++ b/drivers/net/ethernet/freescale/fman/mac.h @@ -1,32 +1,6 @@ -/* Copyright 2008-2015 Freescale Semiconductor, Inc. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * * Redistributions of source code must retain the above copyright - * notice, this list of conditions and the following disclaimer. - * * Redistributions in binary form must reproduce the above copyright - * notice, this list of conditions and the following disclaimer in the - * documentation and/or other materials provided with the distribution. - * * Neither the name of Freescale Semiconductor nor the - * names of its contributors may be used to endorse or promote products - * derived from this software without specific prior written permission. - * - * - * ALTERNATIVELY, this software may be distributed under the terms of the - * GNU General Public License ("GPL") as published by the Free Software - * Foundation, either version 2 of that License or (at your option) any - * later version. - * - * THIS SOFTWARE IS PROVIDED BY Freescale Semiconductor ``AS IS'' AND ANY - * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL Freescale Semiconductor BE LIABLE FOR ANY - * DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES - * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; - * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND - * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT - * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS - * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +/* SPDX-License-Identifier: BSD-3-Clause OR GPL-2.0-or-later */ +/* + * Copyright 2008 - 2015 Freescale Semiconductor Inc. */ #ifndef __MAC_H @@ -45,13 +19,16 @@ struct fman_mac; struct mac_priv_s; struct mac_device { - struct resource *res; + void __iomem *vaddr; + void __iomem *vaddr_end; + struct device *dev; u8 addr[ETH_ALEN]; struct fman_port *port[2]; u32 if_support; struct phy_device *phy_dev; phy_interface_t phy_if; struct device_node *phy_node; + struct net_device *net_dev; bool autoneg_pause; bool rx_pause_req; @@ -61,9 +38,8 @@ struct mac_device { bool promisc; bool allmulti; - int (*init)(struct mac_device *mac_dev); - int (*start)(struct mac_device *mac_dev); - int (*stop)(struct mac_device *mac_dev); + int (*enable)(struct fman_mac *mac_dev); + void (*disable)(struct fman_mac *mac_dev); void (*adjust_link)(struct mac_device *mac_dev); int (*set_promisc)(struct fman_mac *mac_dev, bool enable); int (*change_addr)(struct fman_mac *mac_dev, const enet_addr_t *enet_addr); @@ -81,6 +57,8 @@ struct mac_device { int (*remove_hash_mac_addr)(struct fman_mac *mac_dev, enet_addr_t *eth_addr); + void (*update_speed)(struct mac_device *mac_dev, int speed); + struct fman_mac *fman_mac; struct mac_priv_s *priv; }; @@ -97,5 +75,6 @@ int fman_set_mac_active_pause(struct mac_device *mac_dev, bool rx, bool tx); void fman_get_pause_cfg(struct mac_device *mac_dev, bool *rx_pause, bool *tx_pause); +int fman_set_multi(struct net_device *net_dev, struct mac_device *mac_dev); #endif /* __MAC_H */ diff --git a/drivers/net/ethernet/freescale/fs_enet/fs_enet-main.c b/drivers/net/ethernet/freescale/fs_enet/fs_enet-main.c index b3dae17e067e..8844a9a04fcf 100644 --- a/drivers/net/ethernet/freescale/fs_enet/fs_enet-main.c +++ b/drivers/net/ethernet/freescale/fs_enet/fs_enet-main.c @@ -791,7 +791,7 @@ static int fs_enet_close(struct net_device *dev) static void fs_get_drvinfo(struct net_device *dev, struct ethtool_drvinfo *info) { - strlcpy(info->driver, DRV_MODULE_NAME, sizeof(info->driver)); + strscpy(info->driver, DRV_MODULE_NAME, sizeof(info->driver)); } static int fs_get_regs_len(struct net_device *dev) @@ -883,9 +883,6 @@ static const struct ethtool_ops fs_ethtool_ops = { .set_tunable = fs_set_tunable, }; -extern int fs_mii_connect(struct net_device *dev); -extern void fs_mii_disconnect(struct net_device *dev); - /**************************************************************************************/ #ifdef CONFIG_FS_ENET_HAS_FEC diff --git a/drivers/net/ethernet/freescale/fs_enet/mac-fec.c b/drivers/net/ethernet/freescale/fs_enet/mac-fec.c index 99fe2c210d0f..61f4b6e50d29 100644 --- a/drivers/net/ethernet/freescale/fs_enet/mac-fec.c +++ b/drivers/net/ethernet/freescale/fs_enet/mac-fec.c @@ -98,7 +98,7 @@ static int do_pd_setup(struct fs_enet_private *fep) return -EINVAL; fep->fec.fecp = of_iomap(ofdev->dev.of_node, 0); - if (!fep->fcc.fccp) + if (!fep->fec.fecp) return -EINVAL; return 0; diff --git a/drivers/net/ethernet/freescale/gianfar.c b/drivers/net/ethernet/freescale/gianfar.c index e7bf1524b68e..b2def295523a 100644 --- a/drivers/net/ethernet/freescale/gianfar.c +++ b/drivers/net/ethernet/freescale/gianfar.c @@ -3233,7 +3233,7 @@ static int gfar_probe(struct platform_device *ofdev) /* Register for napi ...We are registering NAPI for each grp */ for (i = 0; i < priv->num_grps; i++) { netif_napi_add(dev, &priv->gfargrp[i].napi_rx, - gfar_poll_rx_sq, NAPI_POLL_WEIGHT); + gfar_poll_rx_sq); netif_napi_add_tx_weight(dev, &priv->gfargrp[i].napi_tx, gfar_poll_tx_sq, 2); } diff --git a/drivers/net/ethernet/freescale/gianfar_ethtool.c b/drivers/net/ethernet/freescale/gianfar_ethtool.c index 81fb68730138..b2b0d3c26fcc 100644 --- a/drivers/net/ethernet/freescale/gianfar_ethtool.c +++ b/drivers/net/ethernet/freescale/gianfar_ethtool.c @@ -163,7 +163,7 @@ static int gfar_sset_count(struct net_device *dev, int sset) static void gfar_gdrvinfo(struct net_device *dev, struct ethtool_drvinfo *drvinfo) { - strlcpy(drvinfo->driver, DRV_NAME, sizeof(drvinfo->driver)); + strscpy(drvinfo->driver, DRV_NAME, sizeof(drvinfo->driver)); } /* Return the length of the register structure */ diff --git a/drivers/net/ethernet/freescale/ucc_geth.c b/drivers/net/ethernet/freescale/ucc_geth.c index 823221c912ab..7a4cb4f07c32 100644 --- a/drivers/net/ethernet/freescale/ucc_geth.c +++ b/drivers/net/ethernet/freescale/ucc_geth.c @@ -3712,7 +3712,7 @@ static int ucc_geth_probe(struct platform_device* ofdev) dev->netdev_ops = &ucc_geth_netdev_ops; dev->watchdog_timeo = TX_TIMEOUT; INIT_WORK(&ugeth->timeout_work, ucc_geth_timeout_work); - netif_napi_add(dev, &ugeth->napi, ucc_geth_poll, 64); + netif_napi_add(dev, &ugeth->napi, ucc_geth_poll); dev->mtu = 1500; dev->max_mtu = 1518; diff --git a/drivers/net/ethernet/freescale/ucc_geth_ethtool.c b/drivers/net/ethernet/freescale/ucc_geth_ethtool.c index 69b2b98b1525..601beb93d3b3 100644 --- a/drivers/net/ethernet/freescale/ucc_geth_ethtool.c +++ b/drivers/net/ethernet/freescale/ucc_geth_ethtool.c @@ -337,8 +337,8 @@ static void uec_get_drvinfo(struct net_device *netdev, struct ethtool_drvinfo *drvinfo) { - strlcpy(drvinfo->driver, DRV_NAME, sizeof(drvinfo->driver)); - strlcpy(drvinfo->bus_info, "QUICC ENGINE", sizeof(drvinfo->bus_info)); + strscpy(drvinfo->driver, DRV_NAME, sizeof(drvinfo->driver)); + strscpy(drvinfo->bus_info, "QUICC ENGINE", sizeof(drvinfo->bus_info)); } #ifdef CONFIG_PM diff --git a/drivers/net/ethernet/freescale/xgmac_mdio.c b/drivers/net/ethernet/freescale/xgmac_mdio.c index ec90da1de030..d7d39a58cd80 100644 --- a/drivers/net/ethernet/freescale/xgmac_mdio.c +++ b/drivers/net/ethernet/freescale/xgmac_mdio.c @@ -355,7 +355,7 @@ static int xgmac_mdio_probe(struct platform_device *pdev) if (ret) return ret; - fwnode = pdev->dev.fwnode; + fwnode = dev_fwnode(&pdev->dev); if (is_of_node(fwnode)) ret = of_mdiobus_register(bus, to_of_node(fwnode)); else if (is_acpi_node(fwnode)) diff --git a/drivers/net/ethernet/fujitsu/fmvj18x_cs.c b/drivers/net/ethernet/fujitsu/fmvj18x_cs.c index b0d733e9a7c6..4859493471db 100644 --- a/drivers/net/ethernet/fujitsu/fmvj18x_cs.c +++ b/drivers/net/ethernet/fujitsu/fmvj18x_cs.c @@ -1046,8 +1046,8 @@ static void fjn_rx(struct net_device *dev) static void netdev_get_drvinfo(struct net_device *dev, struct ethtool_drvinfo *info) { - strlcpy(info->driver, DRV_NAME, sizeof(info->driver)); - strlcpy(info->version, DRV_VERSION, sizeof(info->version)); + strscpy(info->driver, DRV_NAME, sizeof(info->driver)); + strscpy(info->version, DRV_VERSION, sizeof(info->version)); snprintf(info->bus_info, sizeof(info->bus_info), "PCMCIA 0x%lx", dev->base_addr); } diff --git a/drivers/net/ethernet/fungible/funeth/funeth_main.c b/drivers/net/ethernet/fungible/funeth/funeth_main.c index f247b7ad3a88..095f51c4d9d9 100644 --- a/drivers/net/ethernet/fungible/funeth/funeth_main.c +++ b/drivers/net/ethernet/fungible/funeth/funeth_main.c @@ -339,8 +339,7 @@ static int fun_alloc_queue_irqs(struct net_device *dev, unsigned int ntx, return PTR_ERR(irq); fp->num_rx_irqs++; - netif_napi_add(dev, &irq->napi, fun_rxq_napi_poll, - NAPI_POLL_WEIGHT); + netif_napi_add(dev, &irq->napi, fun_rxq_napi_poll); } netif_info(fp, intr, dev, "Reserved %u/%u IRQs for Tx/Rx queues\n", @@ -1802,16 +1801,14 @@ static int fun_create_netdev(struct fun_ethdev *ed, unsigned int portid) if (rc) goto unreg_devlink; - if (fp->dl_port.devlink) - devlink_port_type_eth_set(&fp->dl_port, netdev); + devlink_port_type_eth_set(&fp->dl_port, netdev); return 0; unreg_devlink: ed->netdevs[portid] = NULL; fun_ktls_cleanup(fp); - if (fp->dl_port.devlink) - devlink_port_unregister(&fp->dl_port); + devlink_port_unregister(&fp->dl_port); free_stats: fun_free_stats_area(fp); free_rss: @@ -1830,11 +1827,9 @@ static void fun_destroy_netdev(struct net_device *netdev) struct funeth_priv *fp; fp = netdev_priv(netdev); - if (fp->dl_port.devlink) { - devlink_port_type_clear(&fp->dl_port); - devlink_port_unregister(&fp->dl_port); - } + devlink_port_type_clear(&fp->dl_port); unregister_netdev(netdev); + devlink_port_unregister(&fp->dl_port); fun_ktls_cleanup(fp); fun_free_stats_area(fp); fun_free_rss(fp); diff --git a/drivers/net/ethernet/google/gve/gve_main.c b/drivers/net/ethernet/google/gve/gve_main.c index 044db3ebb071..d3e3ac242bfc 100644 --- a/drivers/net/ethernet/google/gve/gve_main.c +++ b/drivers/net/ethernet/google/gve/gve_main.c @@ -526,8 +526,7 @@ static void gve_add_napi(struct gve_priv *priv, int ntfy_idx, { struct gve_notify_block *block = &priv->ntfy_blocks[ntfy_idx]; - netif_napi_add(priv->dev, &block->napi, gve_poll, - NAPI_POLL_WEIGHT); + netif_napi_add(priv->dev, &block->napi, gve_poll); } static void gve_remove_napi(struct gve_priv *priv, int ntfy_idx) diff --git a/drivers/net/ethernet/hisilicon/hip04_eth.c b/drivers/net/ethernet/hisilicon/hip04_eth.c index c84ef494bd60..50c3f5d6611f 100644 --- a/drivers/net/ethernet/hisilicon/hip04_eth.c +++ b/drivers/net/ethernet/hisilicon/hip04_eth.c @@ -830,8 +830,8 @@ static int hip04_set_coalesce(struct net_device *netdev, static void hip04_get_drvinfo(struct net_device *netdev, struct ethtool_drvinfo *drvinfo) { - strlcpy(drvinfo->driver, DRV_NAME, sizeof(drvinfo->driver)); - strlcpy(drvinfo->version, DRV_VERSION, sizeof(drvinfo->version)); + strscpy(drvinfo->driver, DRV_NAME, sizeof(drvinfo->driver)); + strscpy(drvinfo->version, DRV_VERSION, sizeof(drvinfo->version)); } static const struct ethtool_ops hip04_ethtool_ops = { @@ -990,7 +990,7 @@ static int hip04_mac_probe(struct platform_device *pdev) ndev->watchdog_timeo = TX_TIMEOUT; ndev->priv_flags |= IFF_UNICAST_FLT; ndev->irq = irq; - netif_napi_add(ndev, &priv->napi, hip04_rx_poll, NAPI_POLL_WEIGHT); + netif_napi_add(ndev, &priv->napi, hip04_rx_poll); hip04_reset_dreq(priv); hip04_reset_ppe(priv); diff --git a/drivers/net/ethernet/hisilicon/hix5hd2_gmac.c b/drivers/net/ethernet/hisilicon/hix5hd2_gmac.c index d7e62eca050f..ffcf797dfa90 100644 --- a/drivers/net/ethernet/hisilicon/hix5hd2_gmac.c +++ b/drivers/net/ethernet/hisilicon/hix5hd2_gmac.c @@ -1243,7 +1243,7 @@ static int hix5hd2_dev_probe(struct platform_device *pdev) if (ret) goto out_phy_node; - netif_napi_add(ndev, &priv->napi, hix5hd2_poll, NAPI_POLL_WEIGHT); + netif_napi_add(ndev, &priv->napi, hix5hd2_poll); if (HAS_CAP_TSO(priv->hw_cap)) { ret = hix5hd2_init_sg_desc_queue(priv); diff --git a/drivers/net/ethernet/hisilicon/hns/hns_enet.c b/drivers/net/ethernet/hisilicon/hns/hns_enet.c index d94cc8c6681f..7cf10d1e2b31 100644 --- a/drivers/net/ethernet/hisilicon/hns/hns_enet.c +++ b/drivers/net/ethernet/hisilicon/hns/hns_enet.c @@ -2109,8 +2109,7 @@ static int hns_nic_init_ring_data(struct hns_nic_priv *priv) rd->fini_process = is_ver1 ? hns_nic_tx_fini_pro : hns_nic_tx_fini_pro_v2; - netif_napi_add(priv->netdev, &rd->napi, - hns_nic_common_poll, NAPI_POLL_WEIGHT); + netif_napi_add(priv->netdev, &rd->napi, hns_nic_common_poll); rd->ring->irq_init_flag = RCB_IRQ_NOT_INITED; } for (i = h->q_num; i < h->q_num * 2; i++) { @@ -2122,8 +2121,7 @@ static int hns_nic_init_ring_data(struct hns_nic_priv *priv) rd->fini_process = is_ver1 ? hns_nic_rx_fini_pro : hns_nic_rx_fini_pro_v2; - netif_napi_add(priv->netdev, &rd->napi, - hns_nic_common_poll, NAPI_POLL_WEIGHT); + netif_napi_add(priv->netdev, &rd->napi, hns_nic_common_poll); rd->ring->irq_init_flag = RCB_IRQ_NOT_INITED; } diff --git a/drivers/net/ethernet/hisilicon/hns3/hclge_mbx.h b/drivers/net/ethernet/hisilicon/hns3/hclge_mbx.h index 7d4ae467f3ad..abcd7877f7d2 100644 --- a/drivers/net/ethernet/hisilicon/hns3/hclge_mbx.h +++ b/drivers/net/ethernet/hisilicon/hns3/hclge_mbx.h @@ -233,6 +233,17 @@ struct hclgevf_mbx_arq_ring { __le16 msg_q[HCLGE_MBX_MAX_ARQ_MSG_NUM][HCLGE_MBX_MAX_ARQ_MSG_SIZE]; }; +struct hclge_dev; + +#define HCLGE_MBX_OPCODE_MAX 256 +struct hclge_mbx_ops_param { + struct hclge_vport *vport; + struct hclge_mbx_vf_to_pf_cmd *req; + struct hclge_respond_to_vf_msg *resp_msg; +}; + +typedef int (*hclge_mbx_ops_fn)(struct hclge_mbx_ops_param *param); + #define hclge_mbx_ring_ptr_move_crq(crq) \ (crq->next_to_use = (crq->next_to_use + 1) % crq->desc_num) #define hclge_mbx_tail_ptr_move_arq(arq) \ diff --git a/drivers/net/ethernet/hisilicon/hns3/hnae3.h b/drivers/net/ethernet/hisilicon/hns3/hnae3.h index 94f80e1c4020..0179fc288f5f 100644 --- a/drivers/net/ethernet/hisilicon/hns3/hnae3.h +++ b/drivers/net/ethernet/hisilicon/hns3/hnae3.h @@ -97,13 +97,15 @@ enum HNAE3_DEV_CAP_BITS { HNAE3_DEV_SUPPORT_VLAN_FLTR_MDF_B, HNAE3_DEV_SUPPORT_MC_MAC_MNG_B, HNAE3_DEV_SUPPORT_CQ_B, + HNAE3_DEV_SUPPORT_FEC_STATS_B, + HNAE3_DEV_SUPPORT_LANE_NUM_B, }; -#define hnae3_dev_fd_supported(hdev) \ - test_bit(HNAE3_DEV_SUPPORT_FD_B, (hdev)->ae_dev->caps) +#define hnae3_ae_dev_fd_supported(ae_dev) \ + test_bit(HNAE3_DEV_SUPPORT_FD_B, (ae_dev)->caps) -#define hnae3_dev_gro_supported(hdev) \ - test_bit(HNAE3_DEV_SUPPORT_GRO_B, (hdev)->ae_dev->caps) +#define hnae3_ae_dev_gro_supported(ae_dev) \ + test_bit(HNAE3_DEV_SUPPORT_GRO_B, (ae_dev)->caps) #define hnae3_dev_fec_supported(hdev) \ test_bit(HNAE3_DEV_SUPPORT_FEC_B, (hdev)->ae_dev->caps) @@ -159,6 +161,12 @@ enum HNAE3_DEV_CAP_BITS { #define hnae3_ae_dev_cq_supported(ae_dev) \ test_bit(HNAE3_DEV_SUPPORT_CQ_B, (ae_dev)->caps) +#define hnae3_ae_dev_fec_stats_supported(ae_dev) \ + test_bit(HNAE3_DEV_SUPPORT_FEC_STATS_B, (ae_dev)->caps) + +#define hnae3_ae_dev_lane_num_supported(ae_dev) \ + test_bit(HNAE3_DEV_SUPPORT_LANE_NUM_B, (ae_dev)->caps) + enum HNAE3_PF_CAP_BITS { HNAE3_PF_SUPPORT_VLAN_FLTR_MDF_B = 0, }; @@ -187,6 +195,7 @@ struct hns3_mac_stats { /* hnae3 loop mode */ enum hnae3_loop { + HNAE3_LOOP_EXTERNAL, HNAE3_LOOP_APP, HNAE3_LOOP_SERIAL_SERDES, HNAE3_LOOP_PARALLEL_SERDES, @@ -223,6 +232,8 @@ enum hnae3_fec_mode { HNAE3_FEC_AUTO = 0, HNAE3_FEC_BASER, HNAE3_FEC_RS, + HNAE3_FEC_LLRS, + HNAE3_FEC_NONE, HNAE3_FEC_USER_DEF, }; @@ -270,6 +281,7 @@ enum hnae3_dbg_cmd { HNAE3_DBG_CMD_TC_SCH_INFO, HNAE3_DBG_CMD_QOS_PAUSE_CFG, HNAE3_DBG_CMD_QOS_PRI_MAP, + HNAE3_DBG_CMD_QOS_DSCP_MAP, HNAE3_DBG_CMD_QOS_BUF_CFG, HNAE3_DBG_CMD_DEV_INFO, HNAE3_DBG_CMD_TX_BD, @@ -308,6 +320,11 @@ enum hnae3_dbg_cmd { HNAE3_DBG_CMD_UNKNOWN, }; +enum hnae3_tc_map_mode { + HNAE3_TC_MAP_MODE_PRIO, + HNAE3_TC_MAP_MODE_DSCP, +}; + struct hnae3_vector_info { u8 __iomem *io_addr; int vector; @@ -560,14 +577,17 @@ struct hnae3_ae_ops { void (*client_stop)(struct hnae3_handle *handle); int (*get_status)(struct hnae3_handle *handle); void (*get_ksettings_an_result)(struct hnae3_handle *handle, - u8 *auto_neg, u32 *speed, u8 *duplex); + u8 *auto_neg, u32 *speed, u8 *duplex, + u32 *lane_num); int (*cfg_mac_speed_dup_h)(struct hnae3_handle *handle, int speed, - u8 duplex); + u8 duplex, u8 lane_num); void (*get_media_type)(struct hnae3_handle *handle, u8 *media_type, u8 *module_type); int (*check_port_speed)(struct hnae3_handle *handle, u32 speed); + void (*get_fec_stats)(struct hnae3_handle *handle, + struct ethtool_fec_stats *fec_stats); void (*get_fec)(struct hnae3_handle *handle, u8 *fec_ability, u8 *fec_mode); int (*set_fec)(struct hnae3_handle *handle, u32 fec_mode); @@ -737,6 +757,8 @@ struct hnae3_ae_ops { int (*get_link_diagnosis_info)(struct hnae3_handle *handle, u32 *status_code); void (*clean_vf_config)(struct hnae3_ae_dev *ae_dev, int num_vfs); + int (*get_dscp_prio)(struct hnae3_handle *handle, u8 dscp, + u8 *tc_map_mode, u8 *priority); }; struct hnae3_dcb_ops { @@ -745,6 +767,8 @@ struct hnae3_dcb_ops { int (*ieee_setets)(struct hnae3_handle *, struct ieee_ets *); int (*ieee_getpfc)(struct hnae3_handle *, struct ieee_pfc *); int (*ieee_setpfc)(struct hnae3_handle *, struct ieee_pfc *); + int (*ieee_setapp)(struct hnae3_handle *h, struct dcb_app *app); + int (*ieee_delapp)(struct hnae3_handle *h, struct dcb_app *app); /* DCBX configuration */ u8 (*getdcbx)(struct hnae3_handle *); @@ -774,6 +798,8 @@ struct hnae3_tc_info { bool mqprio_active; }; +#define HNAE3_MAX_DSCP 64 +#define HNAE3_PRIO_ID_INVALID 0xff struct hnae3_knic_private_info { struct net_device *netdev; /* Set by KNIC client when init instance */ u16 rss_size; /* Allocated RSS queues */ @@ -784,6 +810,9 @@ struct hnae3_knic_private_info { u32 tx_spare_buf_size; struct hnae3_tc_info tc_info; + u8 tc_map_mode; + u8 dscp_app_cnt; + u8 dscp_prio[HNAE3_MAX_DSCP]; u16 num_tqps; /* total number of TQPs in this handle */ struct hnae3_queue **tqp; /* array base of all TQPs in this instance */ @@ -815,6 +844,7 @@ struct hnae3_roce_private_info { #define HNAE3_SUPPORT_SERDES_SERIAL_LOOPBACK BIT(2) #define HNAE3_SUPPORT_VF BIT(3) #define HNAE3_SUPPORT_SERDES_PARALLEL_LOOPBACK BIT(4) +#define HNAE3_SUPPORT_EXTERNAL_LOOPBACK BIT(5) #define HNAE3_USER_UPE BIT(0) /* unicast promisc enabled by user */ #define HNAE3_USER_MPE BIT(1) /* mulitcast promisc enabled by user */ diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3_common/hclge_comm_cmd.c b/drivers/net/ethernet/hisilicon/hns3/hns3_common/hclge_comm_cmd.c index c8b151d29f53..f671a63cecde 100644 --- a/drivers/net/ethernet/hisilicon/hns3/hns3_common/hclge_comm_cmd.c +++ b/drivers/net/ethernet/hisilicon/hns3/hns3_common/hclge_comm_cmd.c @@ -52,9 +52,9 @@ void hclge_comm_cmd_reuse_desc(struct hclge_desc *desc, bool is_read) static void hclge_comm_set_default_capability(struct hnae3_ae_dev *ae_dev, bool is_pf) { - set_bit(HNAE3_DEV_SUPPORT_FD_B, ae_dev->caps); set_bit(HNAE3_DEV_SUPPORT_GRO_B, ae_dev->caps); - if (is_pf && ae_dev->dev_version == HNAE3_DEVICE_VERSION_V2) { + if (is_pf) { + set_bit(HNAE3_DEV_SUPPORT_FD_B, ae_dev->caps); set_bit(HNAE3_DEV_SUPPORT_FEC_B, ae_dev->caps); set_bit(HNAE3_DEV_SUPPORT_PAUSE_B, ae_dev->caps); } @@ -91,6 +91,7 @@ int hclge_comm_firmware_compat_config(struct hnae3_ae_dev *ae_dev, hnae3_set_bit(compat, HCLGE_COMM_PHY_IMP_EN_B, 1); hnae3_set_bit(compat, HCLGE_COMM_MAC_STATS_EXT_EN_B, 1); hnae3_set_bit(compat, HCLGE_COMM_SYNC_RX_RING_HEAD_EN_B, 1); + hnae3_set_bit(compat, HCLGE_COMM_LLRS_FEC_EN_B, 1); req->compat = cpu_to_le32(compat); } @@ -150,6 +151,10 @@ static const struct hclge_comm_caps_bit_map hclge_pf_cmd_caps[] = { HNAE3_DEV_SUPPORT_PORT_VLAN_BYPASS_B}, {HCLGE_COMM_CAP_PORT_VLAN_BYPASS_B, HNAE3_DEV_SUPPORT_VLAN_FLTR_MDF_B}, {HCLGE_COMM_CAP_CQ_B, HNAE3_DEV_SUPPORT_CQ_B}, + {HCLGE_COMM_CAP_GRO_B, HNAE3_DEV_SUPPORT_GRO_B}, + {HCLGE_COMM_CAP_FD_B, HNAE3_DEV_SUPPORT_FD_B}, + {HCLGE_COMM_CAP_FEC_STATS_B, HNAE3_DEV_SUPPORT_FEC_STATS_B}, + {HCLGE_COMM_CAP_LANE_NUM_B, HNAE3_DEV_SUPPORT_LANE_NUM_B}, }; static const struct hclge_comm_caps_bit_map hclge_vf_cmd_caps[] = { @@ -162,6 +167,7 @@ static const struct hclge_comm_caps_bit_map hclge_vf_cmd_caps[] = { {HCLGE_COMM_CAP_TX_PUSH_B, HNAE3_DEV_SUPPORT_TX_PUSH_B}, {HCLGE_COMM_CAP_RXD_ADV_LAYOUT_B, HNAE3_DEV_SUPPORT_RXD_ADV_LAYOUT_B}, {HCLGE_COMM_CAP_CQ_B, HNAE3_DEV_SUPPORT_CQ_B}, + {HCLGE_COMM_CAP_GRO_B, HNAE3_DEV_SUPPORT_GRO_B}, }; static void @@ -220,8 +226,10 @@ int hclge_comm_cmd_query_version_and_capability(struct hnae3_ae_dev *ae_dev, HNAE3_PCI_REVISION_BIT_SIZE; ae_dev->dev_version |= ae_dev->pdev->revision; - if (ae_dev->dev_version >= HNAE3_DEVICE_VERSION_V2) + if (ae_dev->dev_version == HNAE3_DEVICE_VERSION_V2) { hclge_comm_set_default_capability(ae_dev, is_pf); + return 0; + } hclge_comm_parse_capability(ae_dev, is_pf, resp); diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3_common/hclge_comm_cmd.h b/drivers/net/ethernet/hisilicon/hns3/hns3_common/hclge_comm_cmd.h index 7a7d4cf9bf35..b1f9383b418f 100644 --- a/drivers/net/ethernet/hisilicon/hns3/hns3_common/hclge_comm_cmd.h +++ b/drivers/net/ethernet/hisilicon/hns3/hns3_common/hclge_comm_cmd.h @@ -20,6 +20,7 @@ #define HCLGE_COMM_PHY_IMP_EN_B 2 #define HCLGE_COMM_MAC_STATS_EXT_EN_B 3 #define HCLGE_COMM_SYNC_RX_RING_HEAD_EN_B 4 +#define HCLGE_COMM_LLRS_FEC_EN_B 5 #define hclge_comm_dev_phy_imp_supported(ae_dev) \ test_bit(HNAE3_DEV_SUPPORT_PHY_IMP_B, (ae_dev)->caps) @@ -102,6 +103,7 @@ enum hclge_opcode_type { HCLGE_OPC_MAC_TNL_INT_EN = 0x0311, HCLGE_OPC_CLEAR_MAC_TNL_INT = 0x0312, HCLGE_OPC_COMMON_LOOPBACK = 0x0315, + HCLGE_OPC_QUERY_FEC_STATS = 0x0316, HCLGE_OPC_CONFIG_FEC_MODE = 0x031A, HCLGE_OPC_QUERY_ROH_TYPE_INFO = 0x0389, @@ -339,6 +341,10 @@ enum HCLGE_COMM_CAP_BITS { HCLGE_COMM_CAP_RXD_ADV_LAYOUT_B = 15, HCLGE_COMM_CAP_PORT_VLAN_BYPASS_B = 17, HCLGE_COMM_CAP_CQ_B = 18, + HCLGE_COMM_CAP_GRO_B = 20, + HCLGE_COMM_CAP_FD_B = 21, + HCLGE_COMM_CAP_FEC_STATS_B = 25, + HCLGE_COMM_CAP_LANE_NUM_B = 27, }; enum HCLGE_COMM_API_CAP_BITS { diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3_dcbnl.c b/drivers/net/ethernet/hisilicon/hns3/hns3_dcbnl.c index d2ec4c573bf8..3b6dbf158b98 100644 --- a/drivers/net/ethernet/hisilicon/hns3/hns3_dcbnl.c +++ b/drivers/net/ethernet/hisilicon/hns3/hns3_dcbnl.c @@ -56,6 +56,32 @@ static int hns3_dcbnl_ieee_setpfc(struct net_device *ndev, struct ieee_pfc *pfc) return -EOPNOTSUPP; } +static int hns3_dcbnl_ieee_setapp(struct net_device *ndev, struct dcb_app *app) +{ + struct hnae3_handle *h = hns3_get_handle(ndev); + + if (hns3_nic_resetting(ndev)) + return -EBUSY; + + if (h->kinfo.dcb_ops->ieee_setapp) + return h->kinfo.dcb_ops->ieee_setapp(h, app); + + return -EOPNOTSUPP; +} + +static int hns3_dcbnl_ieee_delapp(struct net_device *ndev, struct dcb_app *app) +{ + struct hnae3_handle *h = hns3_get_handle(ndev); + + if (hns3_nic_resetting(ndev)) + return -EBUSY; + + if (h->kinfo.dcb_ops->ieee_setapp) + return h->kinfo.dcb_ops->ieee_delapp(h, app); + + return -EOPNOTSUPP; +} + /* DCBX configuration */ static u8 hns3_dcbnl_getdcbx(struct net_device *ndev) { @@ -83,6 +109,8 @@ static const struct dcbnl_rtnl_ops hns3_dcbnl_ops = { .ieee_setets = hns3_dcbnl_ieee_setets, .ieee_getpfc = hns3_dcbnl_ieee_getpfc, .ieee_setpfc = hns3_dcbnl_ieee_setpfc, + .ieee_setapp = hns3_dcbnl_ieee_setapp, + .ieee_delapp = hns3_dcbnl_ieee_delapp, .getdcbx = hns3_dcbnl_getdcbx, .setdcbx = hns3_dcbnl_setdcbx, }; diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3_debugfs.c b/drivers/net/ethernet/hisilicon/hns3/hns3_debugfs.c index 93aeb615191d..66feb23f7b7b 100644 --- a/drivers/net/ethernet/hisilicon/hns3/hns3_debugfs.c +++ b/drivers/net/ethernet/hisilicon/hns3/hns3_debugfs.c @@ -106,6 +106,13 @@ static struct hns3_dbg_cmd_info hns3_dbg_cmd[] = { .init = hns3_dbg_common_file_init, }, { + .name = "qos_dscp_map", + .cmd = HNAE3_DBG_CMD_QOS_DSCP_MAP, + .dentry = HNS3_DBG_DENTRY_TM, + .buf_len = HNS3_DBG_READ_LEN, + .init = hns3_dbg_common_file_init, + }, + { .name = "qos_buf_cfg", .cmd = HNAE3_DBG_CMD_QOS_BUF_CFG, .dentry = HNS3_DBG_DENTRY_TM, @@ -395,6 +402,12 @@ static struct hns3_dbg_cap_info hns3_dbg_cap[] = { }, { .name = "support modify vlan filter state", .cap_bit = HNAE3_DEV_SUPPORT_VLAN_FLTR_MDF_B, + }, { + .name = "support FEC statistics", + .cap_bit = HNAE3_DEV_SUPPORT_FEC_STATS_B, + }, { + .name = "support lane num", + .cap_bit = HNAE3_DEV_SUPPORT_LANE_NUM_B, } }; diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3_enet.c b/drivers/net/ethernet/hisilicon/hns3/hns3_enet.c index 35d70041b9e8..4cb2421e71a7 100644 --- a/drivers/net/ethernet/hisilicon/hns3/hns3_enet.c +++ b/drivers/net/ethernet/hisilicon/hns3/hns3_enet.c @@ -2963,6 +2963,48 @@ static int hns3_nic_set_vf_mac(struct net_device *netdev, int vf_id, u8 *mac) return h->ae_algo->ops->set_vf_mac(h, vf_id, mac); } +#define HNS3_INVALID_DSCP 0xff +#define HNS3_DSCP_SHIFT 2 + +static u8 hns3_get_skb_dscp(struct sk_buff *skb) +{ + __be16 protocol = skb->protocol; + u8 dscp = HNS3_INVALID_DSCP; + + if (protocol == htons(ETH_P_8021Q)) + protocol = vlan_get_protocol(skb); + + if (protocol == htons(ETH_P_IP)) + dscp = ipv4_get_dsfield(ip_hdr(skb)) >> HNS3_DSCP_SHIFT; + else if (protocol == htons(ETH_P_IPV6)) + dscp = ipv6_get_dsfield(ipv6_hdr(skb)) >> HNS3_DSCP_SHIFT; + + return dscp; +} + +static u16 hns3_nic_select_queue(struct net_device *netdev, + struct sk_buff *skb, + struct net_device *sb_dev) +{ + struct hnae3_handle *h = hns3_get_handle(netdev); + u8 dscp; + + if (h->kinfo.tc_map_mode != HNAE3_TC_MAP_MODE_DSCP || + !h->ae_algo->ops->get_dscp_prio) + goto out; + + dscp = hns3_get_skb_dscp(skb); + if (unlikely(dscp >= HNAE3_MAX_DSCP)) + goto out; + + skb->priority = h->kinfo.dscp_prio[dscp]; + if (skb->priority == HNAE3_PRIO_ID_INVALID) + skb->priority = 0; + +out: + return netdev_pick_tx(netdev, skb, sb_dev); +} + static const struct net_device_ops hns3_nic_netdev_ops = { .ndo_open = hns3_nic_net_open, .ndo_stop = hns3_nic_net_stop, @@ -2988,6 +3030,7 @@ static const struct net_device_ops hns3_nic_netdev_ops = { .ndo_set_vf_link_state = hns3_nic_set_vf_link_state, .ndo_set_vf_rate = hns3_nic_set_vf_rate, .ndo_set_vf_mac = hns3_nic_set_vf_mac, + .ndo_select_queue = hns3_nic_select_queue, }; bool hns3_is_phys_func(struct pci_dev *pdev) @@ -3271,12 +3314,11 @@ static void hns3_set_default_feature(struct net_device *netdev) NETIF_F_GSO_GRE_CSUM | NETIF_F_GSO_UDP_TUNNEL | NETIF_F_SCTP_CRC | NETIF_F_FRAGLIST; - if (ae_dev->dev_version >= HNAE3_DEVICE_VERSION_V2) { + if (hnae3_ae_dev_gro_supported(ae_dev)) netdev->features |= NETIF_F_GRO_HW; - if (!(h->flags & HNAE3_SUPPORT_VF)) - netdev->features |= NETIF_F_NTUPLE; - } + if (hnae3_ae_dev_fd_supported(ae_dev)) + netdev->features |= NETIF_F_NTUPLE; if (test_bit(HNAE3_DEV_SUPPORT_UDP_GSO_B, ae_dev->caps)) netdev->features |= NETIF_F_GSO_UDP_L4; @@ -4650,7 +4692,7 @@ static int hns3_nic_init_vector_data(struct hns3_nic_priv *priv) goto map_ring_fail; netif_napi_add(priv->netdev, &tqp_vector->napi, - hns3_nic_common_poll, NAPI_POLL_WEIGHT); + hns3_nic_common_poll); } return 0; @@ -5782,6 +5824,57 @@ int hns3_set_channels(struct net_device *netdev, return 0; } +void hns3_external_lb_prepare(struct net_device *ndev, bool if_running) +{ + struct hns3_nic_priv *priv = netdev_priv(ndev); + struct hnae3_handle *h = priv->ae_handle; + int i; + + if (!if_running) + return; + + netif_carrier_off(ndev); + netif_tx_disable(ndev); + + for (i = 0; i < priv->vector_num; i++) + hns3_vector_disable(&priv->tqp_vector[i]); + + for (i = 0; i < h->kinfo.num_tqps; i++) + hns3_tqp_disable(h->kinfo.tqp[i]); + + /* delay ring buffer clearing to hns3_reset_notify_uninit_enet + * during reset process, because driver may not be able + * to disable the ring through firmware when downing the netdev. + */ + if (!hns3_nic_resetting(ndev)) + hns3_nic_reset_all_ring(priv->ae_handle); + + hns3_reset_tx_queue(priv->ae_handle); +} + +void hns3_external_lb_restore(struct net_device *ndev, bool if_running) +{ + struct hns3_nic_priv *priv = netdev_priv(ndev); + struct hnae3_handle *h = priv->ae_handle; + int i; + + if (!if_running) + return; + + hns3_nic_reset_all_ring(priv->ae_handle); + + for (i = 0; i < priv->vector_num; i++) + hns3_vector_enable(&priv->tqp_vector[i]); + + for (i = 0; i < h->kinfo.num_tqps; i++) + hns3_tqp_enable(h->kinfo.tqp[i]); + + netif_tx_wake_all_queues(ndev); + + if (h->ae_algo->ops->get_status(h)) + netif_carrier_on(ndev); +} + static const struct hns3_hw_error_info hns3_hw_err[] = { { .type = HNAE3_PPU_POISON_ERROR, .msg = "PPU poison" }, diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3_enet.h b/drivers/net/ethernet/hisilicon/hns3/hns3_enet.h index 4a3253692dcc..133a054af6b7 100644 --- a/drivers/net/ethernet/hisilicon/hns3/hns3_enet.h +++ b/drivers/net/ethernet/hisilicon/hns3/hns3_enet.h @@ -744,4 +744,7 @@ u16 hns3_get_max_available_channels(struct hnae3_handle *h); void hns3_cq_period_mode_init(struct hns3_nic_priv *priv, enum dim_cq_period_mode tx_mode, enum dim_cq_period_mode rx_mode); + +void hns3_external_lb_prepare(struct net_device *ndev, bool if_running); +void hns3_external_lb_restore(struct net_device *ndev, bool if_running); #endif diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3_ethtool.c b/drivers/net/ethernet/hisilicon/hns3/hns3_ethtool.c index 4c7988e308a2..cdf76fb58d45 100644 --- a/drivers/net/ethernet/hisilicon/hns3/hns3_ethtool.c +++ b/drivers/net/ethernet/hisilicon/hns3/hns3_ethtool.c @@ -69,7 +69,6 @@ static const struct hns3_stats hns3_rxq_stats[] = { #define HNS3_TQP_STATS_COUNT (HNS3_TXQ_STATS_COUNT + HNS3_RXQ_STATS_COUNT) -#define HNS3_SELF_TEST_TYPE_NUM 4 #define HNS3_NIC_LB_TEST_PKT_NUM 1 #define HNS3_NIC_LB_TEST_RING_ID 0 #define HNS3_NIC_LB_TEST_PACKET_SIZE 128 @@ -95,6 +94,7 @@ static int hns3_lp_setup(struct net_device *ndev, enum hnae3_loop loop, bool en) case HNAE3_LOOP_PARALLEL_SERDES: case HNAE3_LOOP_APP: case HNAE3_LOOP_PHY: + case HNAE3_LOOP_EXTERNAL: ret = h->ae_algo->ops->set_loopback(h, loop, en); break; default: @@ -304,6 +304,10 @@ out: static void hns3_set_selftest_param(struct hnae3_handle *h, int (*st_param)[2]) { + st_param[HNAE3_LOOP_EXTERNAL][0] = HNAE3_LOOP_EXTERNAL; + st_param[HNAE3_LOOP_EXTERNAL][1] = + h->flags & HNAE3_SUPPORT_EXTERNAL_LOOPBACK; + st_param[HNAE3_LOOP_APP][0] = HNAE3_LOOP_APP; st_param[HNAE3_LOOP_APP][1] = h->flags & HNAE3_SUPPORT_APP_LOOPBACK; @@ -322,17 +326,11 @@ static void hns3_set_selftest_param(struct hnae3_handle *h, int (*st_param)[2]) h->flags & HNAE3_SUPPORT_PHY_LOOPBACK; } -static void hns3_selftest_prepare(struct net_device *ndev, - bool if_running, int (*st_param)[2]) +static void hns3_selftest_prepare(struct net_device *ndev, bool if_running) { struct hns3_nic_priv *priv = netdev_priv(ndev); struct hnae3_handle *h = priv->ae_handle; - if (netif_msg_ifdown(h)) - netdev_info(ndev, "self test start\n"); - - hns3_set_selftest_param(h, st_param); - if (if_running) ndev->netdev_ops->ndo_stop(ndev); @@ -371,18 +369,15 @@ static void hns3_selftest_restore(struct net_device *ndev, bool if_running) if (if_running) ndev->netdev_ops->ndo_open(ndev); - - if (netif_msg_ifdown(h)) - netdev_info(ndev, "self test end\n"); } static void hns3_do_selftest(struct net_device *ndev, int (*st_param)[2], struct ethtool_test *eth_test, u64 *data) { - int test_index = 0; + int test_index = HNAE3_LOOP_APP; u32 i; - for (i = 0; i < HNS3_SELF_TEST_TYPE_NUM; i++) { + for (i = HNAE3_LOOP_APP; i < HNAE3_LOOP_NONE; i++) { enum hnae3_loop loop_type = (enum hnae3_loop)st_param[i][0]; if (!st_param[i][1]) @@ -401,6 +396,20 @@ static void hns3_do_selftest(struct net_device *ndev, int (*st_param)[2], } } +static void hns3_do_external_lb(struct net_device *ndev, + struct ethtool_test *eth_test, u64 *data) +{ + data[HNAE3_LOOP_EXTERNAL] = hns3_lp_up(ndev, HNAE3_LOOP_EXTERNAL); + if (!data[HNAE3_LOOP_EXTERNAL]) + data[HNAE3_LOOP_EXTERNAL] = hns3_lp_run_test(ndev, HNAE3_LOOP_EXTERNAL); + hns3_lp_down(ndev, HNAE3_LOOP_EXTERNAL); + + if (data[HNAE3_LOOP_EXTERNAL]) + eth_test->flags |= ETH_TEST_FL_FAILED; + + eth_test->flags |= ETH_TEST_FL_EXTERNAL_LB_DONE; +} + /** * hns3_self_test - self test * @ndev: net device @@ -410,7 +419,9 @@ static void hns3_do_selftest(struct net_device *ndev, int (*st_param)[2], static void hns3_self_test(struct net_device *ndev, struct ethtool_test *eth_test, u64 *data) { - int st_param[HNS3_SELF_TEST_TYPE_NUM][2]; + struct hns3_nic_priv *priv = netdev_priv(ndev); + struct hnae3_handle *h = priv->ae_handle; + int st_param[HNAE3_LOOP_NONE][2]; bool if_running = netif_running(ndev); if (hns3_nic_resetting(ndev)) { @@ -418,13 +429,29 @@ static void hns3_self_test(struct net_device *ndev, return; } - /* Only do offline selftest, or pass by default */ - if (eth_test->flags != ETH_TEST_FL_OFFLINE) + if (!(eth_test->flags & ETH_TEST_FL_OFFLINE)) return; - hns3_selftest_prepare(ndev, if_running, st_param); + if (netif_msg_ifdown(h)) + netdev_info(ndev, "self test start\n"); + + hns3_set_selftest_param(h, st_param); + + /* external loopback test requires that the link is up and the duplex is + * full, do external test first to reduce the whole test time + */ + if (eth_test->flags & ETH_TEST_FL_EXTERNAL_LB) { + hns3_external_lb_prepare(ndev, if_running); + hns3_do_external_lb(ndev, eth_test, data); + hns3_external_lb_restore(ndev, if_running); + } + + hns3_selftest_prepare(ndev, if_running); hns3_do_selftest(ndev, st_param, eth_test, data); hns3_selftest_restore(ndev, if_running); + + if (netif_msg_ifdown(h)) + netdev_info(ndev, "self test end\n"); } static void hns3_update_limit_promisc_mode(struct net_device *netdev, @@ -712,7 +739,8 @@ static void hns3_get_ksettings(struct hnae3_handle *h, ops->get_ksettings_an_result(h, &cmd->base.autoneg, &cmd->base.speed, - &cmd->base.duplex); + &cmd->base.duplex, + &cmd->lanes); /* 2.get link mode */ if (ops->get_link_mode) @@ -794,6 +822,7 @@ static int hns3_check_ksettings_param(const struct net_device *netdev, const struct hnae3_ae_ops *ops = handle->ae_algo->ops; u8 module_type = HNAE3_MODULE_TYPE_UNKNOWN; u8 media_type = HNAE3_MEDIA_TYPE_UNKNOWN; + u32 lane_num; u8 autoneg; u32 speed; u8 duplex; @@ -806,9 +835,9 @@ static int hns3_check_ksettings_param(const struct net_device *netdev, return 0; if (ops->get_ksettings_an_result) { - ops->get_ksettings_an_result(handle, &autoneg, &speed, &duplex); + ops->get_ksettings_an_result(handle, &autoneg, &speed, &duplex, &lane_num); if (cmd->base.autoneg == autoneg && cmd->base.speed == speed && - cmd->base.duplex == duplex) + cmd->base.duplex == duplex && cmd->lanes == lane_num) return 0; } @@ -845,10 +874,14 @@ static int hns3_set_link_ksettings(struct net_device *netdev, if (cmd->base.speed == SPEED_1000 && cmd->base.duplex == DUPLEX_HALF) return -EINVAL; + if (cmd->lanes && !hnae3_ae_dev_lane_num_supported(ae_dev)) + return -EOPNOTSUPP; + netif_dbg(handle, drv, netdev, - "set link(%s): autoneg=%u, speed=%u, duplex=%u\n", + "set link(%s): autoneg=%u, speed=%u, duplex=%u, lanes=%u\n", netdev->phydev ? "phy" : "mac", - cmd->base.autoneg, cmd->base.speed, cmd->base.duplex); + cmd->base.autoneg, cmd->base.speed, cmd->base.duplex, + cmd->lanes); /* Only support ksettings_set for netdev with phy attached for now */ if (netdev->phydev) { @@ -886,7 +919,7 @@ static int hns3_set_link_ksettings(struct net_device *netdev, if (ops->cfg_mac_speed_dup_h) ret = ops->cfg_mac_speed_dup_h(handle, cmd->base.speed, - cmd->base.duplex); + cmd->base.duplex, (u8)(cmd->lanes)); return ret; } @@ -1612,6 +1645,19 @@ static void hns3_set_msglevel(struct net_device *netdev, u32 msg_level) h->msg_enable = msg_level; } +static void hns3_get_fec_stats(struct net_device *netdev, + struct ethtool_fec_stats *fec_stats) +{ + struct hnae3_handle *handle = hns3_get_handle(netdev); + struct hnae3_ae_dev *ae_dev = pci_get_drvdata(handle->pdev); + const struct hnae3_ae_ops *ops = handle->ae_algo->ops; + + if (!hnae3_ae_dev_fec_stats_supported(ae_dev) || !ops->get_fec_stats) + return; + + ops->get_fec_stats(handle, fec_stats); +} + /* Translate local fec value into ethtool value. */ static unsigned int loc_to_eth_fec(u8 loc_fec) { @@ -1621,12 +1667,12 @@ static unsigned int loc_to_eth_fec(u8 loc_fec) eth_fec |= ETHTOOL_FEC_AUTO; if (loc_fec & BIT(HNAE3_FEC_RS)) eth_fec |= ETHTOOL_FEC_RS; + if (loc_fec & BIT(HNAE3_FEC_LLRS)) + eth_fec |= ETHTOOL_FEC_LLRS; if (loc_fec & BIT(HNAE3_FEC_BASER)) eth_fec |= ETHTOOL_FEC_BASER; - - /* if nothing is set, then FEC is off */ - if (!eth_fec) - eth_fec = ETHTOOL_FEC_OFF; + if (loc_fec & BIT(HNAE3_FEC_NONE)) + eth_fec |= ETHTOOL_FEC_OFF; return eth_fec; } @@ -1637,12 +1683,13 @@ static unsigned int eth_to_loc_fec(unsigned int eth_fec) u32 loc_fec = 0; if (eth_fec & ETHTOOL_FEC_OFF) - return loc_fec; - + loc_fec |= BIT(HNAE3_FEC_NONE); if (eth_fec & ETHTOOL_FEC_AUTO) loc_fec |= BIT(HNAE3_FEC_AUTO); if (eth_fec & ETHTOOL_FEC_RS) loc_fec |= BIT(HNAE3_FEC_RS); + if (eth_fec & ETHTOOL_FEC_LLRS) + loc_fec |= BIT(HNAE3_FEC_LLRS); if (eth_fec & ETHTOOL_FEC_BASER) loc_fec |= BIT(HNAE3_FEC_BASER); @@ -1668,6 +1715,8 @@ static int hns3_get_fecparam(struct net_device *netdev, fec->fec = loc_to_eth_fec(fec_ability); fec->active_fec = loc_to_eth_fec(fec_mode); + if (!fec->active_fec) + fec->active_fec = ETHTOOL_FEC_OFF; return 0; } @@ -2051,6 +2100,7 @@ static const struct ethtool_ops hns3vf_ethtool_ops = { static const struct ethtool_ops hns3_ethtool_ops = { .supported_coalesce_params = HNS3_ETHTOOL_COALESCE, .supported_ring_params = HNS3_ETHTOOL_RING, + .cap_link_lanes_supported = true, .self_test = hns3_self_test, .get_drvinfo = hns3_get_drvinfo, .get_link = hns3_get_link, @@ -2081,6 +2131,7 @@ static const struct ethtool_ops hns3_ethtool_ops = { .set_msglevel = hns3_set_msglevel, .get_fecparam = hns3_get_fecparam, .set_fecparam = hns3_set_fecparam, + .get_fec_stats = hns3_get_fec_stats, .get_module_info = hns3_get_module_info, .get_module_eeprom = hns3_get_module_eeprom, .get_priv_flags = hns3_get_priv_flags, diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_cmd.h b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_cmd.h index f9d89511eb32..43cada51d8cb 100644 --- a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_cmd.h +++ b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_cmd.h @@ -321,7 +321,9 @@ struct hclge_config_mac_speed_dup_cmd { #define HCLGE_CFG_MAC_SPEED_CHANGE_EN_B 0 u8 mac_change_fec_en; - u8 rsv[22]; + u8 rsv[4]; + u8 lane_num; + u8 rsv1[17]; }; #define HCLGE_TQP_ENABLE_B 0 @@ -347,7 +349,9 @@ struct hclge_sfp_info_cmd { u8 autoneg_ability; /* whether support autoneg */ __le32 speed_ability; /* speed ability for current media */ __le32 module_type; - u8 rsv[8]; + u8 fec_ability; + u8 lane_num; + u8 rsv[6]; }; #define HCLGE_MAC_CFG_FEC_AUTO_EN_B 0 @@ -359,12 +363,27 @@ struct hclge_sfp_info_cmd { #define HCLGE_MAC_FEC_OFF 0 #define HCLGE_MAC_FEC_BASER 1 #define HCLGE_MAC_FEC_RS 2 +#define HCLGE_MAC_FEC_LLRS 3 struct hclge_config_fec_cmd { u8 fec_mode; u8 default_config; u8 rsv[22]; }; +#define HCLGE_FEC_STATS_CMD_NUM 4 + +struct hclge_query_fec_stats_cmd { + /* fec rs mode total stats */ + __le32 rs_fec_corr_blocks; + __le32 rs_fec_uncorr_blocks; + __le32 rs_fec_error_blocks; + /* fec base-r mode per lanes stats */ + u8 base_r_lane_num; + u8 rsv[3]; + __le32 base_r_fec_corr_blocks; + __le32 base_r_fec_uncorr_blocks; +}; + #define HCLGE_MAC_UPLINK_PORT 0x100 struct hclge_config_max_frm_size_cmd { diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_dcb.c b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_dcb.c index 69b8673436ca..c4aded65e848 100644 --- a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_dcb.c +++ b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_dcb.c @@ -359,6 +359,93 @@ static int hclge_ieee_setpfc(struct hnae3_handle *h, struct ieee_pfc *pfc) return hclge_notify_client(hdev, HNAE3_UP_CLIENT); } +static int hclge_ieee_setapp(struct hnae3_handle *h, struct dcb_app *app) +{ + struct hclge_vport *vport = hclge_get_vport(h); + struct net_device *netdev = h->kinfo.netdev; + struct hclge_dev *hdev = vport->back; + struct dcb_app old_app; + int ret; + + if (app->selector != IEEE_8021QAZ_APP_SEL_DSCP || + app->protocol >= HNAE3_MAX_DSCP || + app->priority >= HNAE3_MAX_USER_PRIO) + return -EINVAL; + + dev_info(&hdev->pdev->dev, "setapp dscp=%u priority=%u\n", + app->protocol, app->priority); + + if (app->priority == h->kinfo.dscp_prio[app->protocol]) + return 0; + + ret = dcb_ieee_setapp(netdev, app); + if (ret) + return ret; + + old_app.selector = IEEE_8021QAZ_APP_SEL_DSCP; + old_app.protocol = app->protocol; + old_app.priority = h->kinfo.dscp_prio[app->protocol]; + + h->kinfo.dscp_prio[app->protocol] = app->priority; + ret = hclge_dscp_to_tc_map(hdev); + if (ret) { + dev_err(&hdev->pdev->dev, + "failed to set dscp to tc map, ret = %d\n", ret); + h->kinfo.dscp_prio[app->protocol] = old_app.priority; + (void)dcb_ieee_delapp(netdev, app); + return ret; + } + + vport->nic.kinfo.tc_map_mode = HNAE3_TC_MAP_MODE_DSCP; + if (old_app.priority == HNAE3_PRIO_ID_INVALID) + h->kinfo.dscp_app_cnt++; + else + ret = dcb_ieee_delapp(netdev, &old_app); + + return ret; +} + +static int hclge_ieee_delapp(struct hnae3_handle *h, struct dcb_app *app) +{ + struct hclge_vport *vport = hclge_get_vport(h); + struct net_device *netdev = h->kinfo.netdev; + struct hclge_dev *hdev = vport->back; + int ret; + + if (app->selector != IEEE_8021QAZ_APP_SEL_DSCP || + app->protocol >= HNAE3_MAX_DSCP || + app->priority >= HNAE3_MAX_USER_PRIO || + app->priority != h->kinfo.dscp_prio[app->protocol]) + return -EINVAL; + + dev_info(&hdev->pdev->dev, "delapp dscp=%u priority=%u\n", + app->protocol, app->priority); + + ret = dcb_ieee_delapp(netdev, app); + if (ret) + return ret; + + h->kinfo.dscp_prio[app->protocol] = HNAE3_PRIO_ID_INVALID; + ret = hclge_dscp_to_tc_map(hdev); + if (ret) { + dev_err(&hdev->pdev->dev, + "failed to del dscp to tc map, ret = %d\n", ret); + h->kinfo.dscp_prio[app->protocol] = app->priority; + (void)dcb_ieee_setapp(netdev, app); + return ret; + } + + if (h->kinfo.dscp_app_cnt) + h->kinfo.dscp_app_cnt--; + + if (!h->kinfo.dscp_app_cnt) { + vport->nic.kinfo.tc_map_mode = HNAE3_TC_MAP_MODE_PRIO; + ret = hclge_up_to_tc_map(hdev); + } + + return ret; +} + /* DCBX configuration */ static u8 hclge_getdcbx(struct hnae3_handle *h) { @@ -543,6 +630,8 @@ static const struct hnae3_dcb_ops hns3_dcb_ops = { .ieee_setets = hclge_ieee_setets, .ieee_getpfc = hclge_ieee_getpfc, .ieee_setpfc = hclge_ieee_setpfc, + .ieee_setapp = hclge_ieee_setapp, + .ieee_delapp = hclge_ieee_delapp, .getdcbx = hclge_getdcbx, .setdcbx = hclge_setdcbx, .setup_tc = hclge_setup_tc, diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_debugfs.c b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_debugfs.c index 9b870e79c290..142415c84c6b 100644 --- a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_debugfs.c +++ b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_debugfs.c @@ -14,6 +14,8 @@ static const char * const hclge_mac_state_str[] = { "TO_ADD", "TO_DEL", "ACTIVE" }; +static const char * const tc_map_mode_str[] = { "PRIO", "DSCP" }; + static const struct hclge_dbg_reg_type_info hclge_dbg_reg_info[] = { { .cmd = HNAE3_DBG_CMD_REG_BIOS_COMMON, .dfx_msg = &hclge_dbg_bios_common_reg[0], @@ -1115,10 +1117,11 @@ static int hclge_dbg_dump_qos_pause_cfg(struct hclge_dev *hdev, char *buf, return 0; } +#define HCLGE_DBG_TC_MASK 0x0F + static int hclge_dbg_dump_qos_pri_map(struct hclge_dev *hdev, char *buf, int len) { -#define HCLGE_DBG_TC_MASK 0x0F #define HCLGE_DBG_TC_BIT_WIDTH 4 struct hclge_qos_pri_map_cmd *pri_map; @@ -1152,6 +1155,58 @@ static int hclge_dbg_dump_qos_pri_map(struct hclge_dev *hdev, char *buf, return 0; } +static int hclge_dbg_dump_qos_dscp_map(struct hclge_dev *hdev, char *buf, + int len) +{ + struct hnae3_knic_private_info *kinfo = &hdev->vport[0].nic.kinfo; + struct hclge_desc desc[HCLGE_DSCP_MAP_TC_BD_NUM]; + u8 *req0 = (u8 *)desc[0].data; + u8 *req1 = (u8 *)desc[1].data; + u8 dscp_tc[HNAE3_MAX_DSCP]; + int pos, ret; + u8 i, j; + + pos = scnprintf(buf, len, "tc map mode: %s\n", + tc_map_mode_str[kinfo->tc_map_mode]); + + if (kinfo->tc_map_mode != HNAE3_TC_MAP_MODE_DSCP) + return 0; + + hclge_cmd_setup_basic_desc(&desc[0], HCLGE_OPC_QOS_MAP, true); + desc[0].flag |= cpu_to_le16(HCLGE_COMM_CMD_FLAG_NEXT); + hclge_cmd_setup_basic_desc(&desc[1], HCLGE_OPC_QOS_MAP, true); + ret = hclge_cmd_send(&hdev->hw, desc, HCLGE_DSCP_MAP_TC_BD_NUM); + if (ret) { + dev_err(&hdev->pdev->dev, + "failed to dump qos dscp map, ret = %d\n", ret); + return ret; + } + + pos += scnprintf(buf + pos, len - pos, "\nDSCP PRIO TC\n"); + + /* The low 32 dscp setting use bd0, high 32 dscp setting use bd1 */ + for (i = 0; i < HNAE3_MAX_DSCP / HCLGE_DSCP_MAP_TC_BD_NUM; i++) { + j = i + HNAE3_MAX_DSCP / HCLGE_DSCP_MAP_TC_BD_NUM; + /* Each dscp setting has 4 bits, so each byte saves two dscp + * setting + */ + dscp_tc[i] = req0[i >> 1] >> HCLGE_DSCP_TC_SHIFT(i); + dscp_tc[j] = req1[i >> 1] >> HCLGE_DSCP_TC_SHIFT(i); + dscp_tc[i] &= HCLGE_DBG_TC_MASK; + dscp_tc[j] &= HCLGE_DBG_TC_MASK; + } + + for (i = 0; i < HNAE3_MAX_DSCP; i++) { + if (kinfo->dscp_prio[i] == HNAE3_PRIO_ID_INVALID) + continue; + + pos += scnprintf(buf + pos, len - pos, " %2u %u %u\n", + i, kinfo->dscp_prio[i], dscp_tc[i]); + } + + return 0; +} + static int hclge_dbg_dump_tx_buf_cfg(struct hclge_dev *hdev, char *buf, int len) { struct hclge_tx_buff_alloc_cmd *tx_buf_cmd; @@ -1517,7 +1572,7 @@ static int hclge_dbg_dump_fd_tcam(struct hclge_dev *hdev, char *buf, int len) char *tcam_buf; int pos = 0; - if (!hnae3_dev_fd_supported(hdev)) { + if (!hnae3_ae_dev_fd_supported(hdev->ae_dev)) { dev_err(&hdev->pdev->dev, "Only FD-supported dev supports dump fd tcam\n"); return -EOPNOTSUPP; @@ -1585,6 +1640,9 @@ static int hclge_dbg_dump_fd_counter(struct hclge_dev *hdev, char *buf, int len) u64 cnt; u8 i; + if (!hnae3_ae_dev_fd_supported(hdev->ae_dev)) + return -EOPNOTSUPP; + pos += scnprintf(buf + pos, len - pos, "func_id\thit_times\n"); @@ -2374,6 +2432,10 @@ static const struct hclge_dbg_func hclge_dbg_cmd_func[] = { .dbg_dump = hclge_dbg_dump_qos_pri_map, }, { + .cmd = HNAE3_DBG_CMD_QOS_DSCP_MAP, + .dbg_dump = hclge_dbg_dump_qos_dscp_map, + }, + { .cmd = HNAE3_DBG_CMD_QOS_BUF_CFG, .dbg_dump = hclge_dbg_dump_qos_buf_cfg, }, diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_main.c b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_main.c index fae79764dc44..6962a9d69cf8 100644 --- a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_main.c +++ b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_main.c @@ -71,6 +71,7 @@ static void hclge_sync_mac_table(struct hclge_dev *hdev); static void hclge_restore_hw_table(struct hclge_dev *hdev); static void hclge_sync_promisc_mode(struct hclge_dev *hdev); static void hclge_sync_fd_table(struct hclge_dev *hdev); +static void hclge_update_fec_stats(struct hclge_dev *hdev); static struct hnae3_ae_algo ae_algo; @@ -148,10 +149,11 @@ static const u32 tqp_intr_reg_addr_list[] = {HCLGE_TQP_INTR_CTRL_REG, HCLGE_TQP_INTR_RL_REG}; static const char hns3_nic_test_strs[][ETH_GSTRING_LEN] = { - "App Loopback test", - "Serdes serial Loopback test", - "Serdes parallel Loopback test", - "Phy Loopback test" + "External Loopback test", + "App Loopback test", + "Serdes serial Loopback test", + "Serdes parallel Loopback test", + "Phy Loopback test" }; static const struct hclge_comm_stats_str g_mac_stats_string[] = { @@ -679,6 +681,8 @@ static void hclge_update_stats_for_all(struct hclge_dev *hdev) } } + hclge_update_fec_stats(hdev); + status = hclge_mac_update_stats(hdev); if (status) dev_err(&hdev->pdev->dev, @@ -715,7 +719,8 @@ static int hclge_get_sset_count(struct hnae3_handle *handle, int stringset) #define HCLGE_LOOPBACK_TEST_FLAGS (HNAE3_SUPPORT_APP_LOOPBACK | \ HNAE3_SUPPORT_PHY_LOOPBACK | \ HNAE3_SUPPORT_SERDES_SERIAL_LOOPBACK | \ - HNAE3_SUPPORT_SERDES_PARALLEL_LOOPBACK) + HNAE3_SUPPORT_SERDES_PARALLEL_LOOPBACK | \ + HNAE3_SUPPORT_EXTERNAL_LOOPBACK) struct hclge_vport *vport = hclge_get_vport(handle); struct hclge_dev *hdev = vport->back; @@ -737,9 +742,12 @@ static int hclge_get_sset_count(struct hnae3_handle *handle, int stringset) handle->flags |= HNAE3_SUPPORT_APP_LOOPBACK; } - count += 2; + count += 1; handle->flags |= HNAE3_SUPPORT_SERDES_SERIAL_LOOPBACK; + count += 1; handle->flags |= HNAE3_SUPPORT_SERDES_PARALLEL_LOOPBACK; + count += 1; + handle->flags |= HNAE3_SUPPORT_EXTERNAL_LOOPBACK; if ((hdev->hw.mac.phydev && hdev->hw.mac.phydev->drv && hdev->hw.mac.phydev->drv->set_loopback) || @@ -770,6 +778,11 @@ static void hclge_get_strings(struct hnae3_handle *handle, u32 stringset, size, p); p = hclge_comm_tqps_get_strings(handle, p); } else if (stringset == ETH_SS_TEST) { + if (handle->flags & HNAE3_SUPPORT_EXTERNAL_LOOPBACK) { + memcpy(p, hns3_nic_test_strs[HNAE3_LOOP_EXTERNAL], + ETH_GSTRING_LEN); + p += ETH_GSTRING_LEN; + } if (handle->flags & HNAE3_SUPPORT_APP_LOOPBACK) { memcpy(p, hns3_nic_test_strs[HNAE3_LOOP_APP], ETH_GSTRING_LEN); @@ -1003,6 +1016,27 @@ static int hclge_check_port_speed(struct hnae3_handle *handle, u32 speed) return -EINVAL; } +static void hclge_update_fec_support(struct hclge_mac *mac) +{ + linkmode_clear_bit(ETHTOOL_LINK_MODE_FEC_BASER_BIT, mac->supported); + linkmode_clear_bit(ETHTOOL_LINK_MODE_FEC_RS_BIT, mac->supported); + linkmode_clear_bit(ETHTOOL_LINK_MODE_FEC_LLRS_BIT, mac->supported); + linkmode_clear_bit(ETHTOOL_LINK_MODE_FEC_NONE_BIT, mac->supported); + + if (mac->fec_ability & BIT(HNAE3_FEC_BASER)) + linkmode_set_bit(ETHTOOL_LINK_MODE_FEC_BASER_BIT, + mac->supported); + if (mac->fec_ability & BIT(HNAE3_FEC_RS)) + linkmode_set_bit(ETHTOOL_LINK_MODE_FEC_RS_BIT, + mac->supported); + if (mac->fec_ability & BIT(HNAE3_FEC_LLRS)) + linkmode_set_bit(ETHTOOL_LINK_MODE_FEC_LLRS_BIT, + mac->supported); + if (mac->fec_ability & BIT(HNAE3_FEC_NONE)) + linkmode_set_bit(ETHTOOL_LINK_MODE_FEC_NONE_BIT, + mac->supported); +} + static void hclge_convert_setting_sr(u16 speed_ability, unsigned long *link_mode) { @@ -1101,34 +1135,36 @@ static void hclge_convert_setting_kr(u16 speed_ability, static void hclge_convert_setting_fec(struct hclge_mac *mac) { - linkmode_clear_bit(ETHTOOL_LINK_MODE_FEC_BASER_BIT, mac->supported); - linkmode_clear_bit(ETHTOOL_LINK_MODE_FEC_RS_BIT, mac->supported); + /* If firmware has reported fec_ability, don't need to convert by speed */ + if (mac->fec_ability) + goto out; switch (mac->speed) { case HCLGE_MAC_SPEED_10G: case HCLGE_MAC_SPEED_40G: - linkmode_set_bit(ETHTOOL_LINK_MODE_FEC_BASER_BIT, - mac->supported); - mac->fec_ability = - BIT(HNAE3_FEC_BASER) | BIT(HNAE3_FEC_AUTO); + mac->fec_ability = BIT(HNAE3_FEC_BASER) | BIT(HNAE3_FEC_AUTO) | + BIT(HNAE3_FEC_NONE); break; case HCLGE_MAC_SPEED_25G: case HCLGE_MAC_SPEED_50G: - linkmode_set_bit(ETHTOOL_LINK_MODE_FEC_RS_BIT, - mac->supported); - mac->fec_ability = - BIT(HNAE3_FEC_BASER) | BIT(HNAE3_FEC_RS) | - BIT(HNAE3_FEC_AUTO); + mac->fec_ability = BIT(HNAE3_FEC_BASER) | BIT(HNAE3_FEC_RS) | + BIT(HNAE3_FEC_AUTO) | BIT(HNAE3_FEC_NONE); break; case HCLGE_MAC_SPEED_100G: + mac->fec_ability = BIT(HNAE3_FEC_RS) | BIT(HNAE3_FEC_AUTO) | + BIT(HNAE3_FEC_NONE); + break; case HCLGE_MAC_SPEED_200G: - linkmode_set_bit(ETHTOOL_LINK_MODE_FEC_RS_BIT, mac->supported); - mac->fec_ability = BIT(HNAE3_FEC_RS) | BIT(HNAE3_FEC_AUTO); + mac->fec_ability = BIT(HNAE3_FEC_RS) | BIT(HNAE3_FEC_AUTO) | + BIT(HNAE3_FEC_LLRS); break; default: mac->fec_ability = 0; break; } + +out: + hclge_update_fec_support(mac); } static void hclge_parse_fiber_link_mode(struct hclge_dev *hdev, @@ -1574,7 +1610,7 @@ static int hclge_configure(struct hclge_dev *hdev) if (cfg.vlan_fliter_cap == HCLGE_VLAN_FLTR_CAN_MDF) set_bit(HNAE3_DEV_SUPPORT_VLAN_FLTR_MDF_B, ae_dev->caps); - if (hnae3_dev_fd_supported(hdev)) { + if (hnae3_ae_dev_fd_supported(hdev->ae_dev)) { hdev->fd_en = true; hdev->fd_active_type = HCLGE_FD_RULE_NONE; } @@ -1617,7 +1653,7 @@ static int hclge_config_gro(struct hclge_dev *hdev) struct hclge_desc desc; int ret; - if (!hnae3_dev_gro_supported(hdev)) + if (!hnae3_ae_dev_gro_supported(hdev->ae_dev)) return 0; hclge_cmd_setup_basic_desc(&desc, HCLGE_OPC_GRO_GENERIC_CONFIG, false); @@ -2589,7 +2625,7 @@ static int hclge_convert_to_fw_speed(u32 speed_drv, u32 *speed_fw) } static int hclge_cfg_mac_speed_dup_hw(struct hclge_dev *hdev, int speed, - u8 duplex) + u8 duplex, u8 lane_num) { struct hclge_config_mac_speed_dup_cmd *req; struct hclge_desc desc; @@ -2613,6 +2649,7 @@ static int hclge_cfg_mac_speed_dup_hw(struct hclge_dev *hdev, int speed, speed_fw); hnae3_set_bit(req->mac_change_fec_en, HCLGE_CFG_MAC_SPEED_CHANGE_EN_B, 1); + req->lane_num = lane_num; ret = hclge_cmd_send(&hdev->hw, &desc, 1); if (ret) { @@ -2624,33 +2661,35 @@ static int hclge_cfg_mac_speed_dup_hw(struct hclge_dev *hdev, int speed, return 0; } -int hclge_cfg_mac_speed_dup(struct hclge_dev *hdev, int speed, u8 duplex) +int hclge_cfg_mac_speed_dup(struct hclge_dev *hdev, int speed, u8 duplex, u8 lane_num) { struct hclge_mac *mac = &hdev->hw.mac; int ret; duplex = hclge_check_speed_dup(duplex, speed); if (!mac->support_autoneg && mac->speed == speed && - mac->duplex == duplex) + mac->duplex == duplex && (mac->lane_num == lane_num || lane_num == 0)) return 0; - ret = hclge_cfg_mac_speed_dup_hw(hdev, speed, duplex); + ret = hclge_cfg_mac_speed_dup_hw(hdev, speed, duplex, lane_num); if (ret) return ret; hdev->hw.mac.speed = speed; hdev->hw.mac.duplex = duplex; + if (!lane_num) + hdev->hw.mac.lane_num = lane_num; return 0; } static int hclge_cfg_mac_speed_dup_h(struct hnae3_handle *handle, int speed, - u8 duplex) + u8 duplex, u8 lane_num) { struct hclge_vport *vport = hclge_get_vport(handle); struct hclge_dev *hdev = vport->back; - return hclge_cfg_mac_speed_dup(hdev, speed, duplex); + return hclge_cfg_mac_speed_dup(hdev, speed, duplex, lane_num); } static int hclge_set_autoneg_en(struct hclge_dev *hdev, bool enable) @@ -2730,6 +2769,157 @@ static int hclge_halt_autoneg(struct hnae3_handle *handle, bool halt) return 0; } +static void hclge_parse_fec_stats_lanes(struct hclge_dev *hdev, + struct hclge_desc *desc, u32 desc_len) +{ + u32 lane_size = HCLGE_FEC_STATS_MAX_LANES * 2; + u32 desc_index = 0; + u32 data_index = 0; + u32 i; + + for (i = 0; i < lane_size; i++) { + if (data_index >= HCLGE_DESC_DATA_LEN) { + desc_index++; + data_index = 0; + } + + if (desc_index >= desc_len) + return; + + hdev->fec_stats.per_lanes[i] += + le32_to_cpu(desc[desc_index].data[data_index]); + data_index++; + } +} + +static void hclge_parse_fec_stats(struct hclge_dev *hdev, + struct hclge_desc *desc, u32 desc_len) +{ + struct hclge_query_fec_stats_cmd *req; + + req = (struct hclge_query_fec_stats_cmd *)desc[0].data; + + hdev->fec_stats.base_r_lane_num = req->base_r_lane_num; + hdev->fec_stats.rs_corr_blocks += + le32_to_cpu(req->rs_fec_corr_blocks); + hdev->fec_stats.rs_uncorr_blocks += + le32_to_cpu(req->rs_fec_uncorr_blocks); + hdev->fec_stats.rs_error_blocks += + le32_to_cpu(req->rs_fec_error_blocks); + hdev->fec_stats.base_r_corr_blocks += + le32_to_cpu(req->base_r_fec_corr_blocks); + hdev->fec_stats.base_r_uncorr_blocks += + le32_to_cpu(req->base_r_fec_uncorr_blocks); + + hclge_parse_fec_stats_lanes(hdev, &desc[1], desc_len - 1); +} + +static int hclge_update_fec_stats_hw(struct hclge_dev *hdev) +{ + struct hclge_desc desc[HCLGE_FEC_STATS_CMD_NUM]; + int ret; + u32 i; + + for (i = 0; i < HCLGE_FEC_STATS_CMD_NUM; i++) { + hclge_cmd_setup_basic_desc(&desc[i], HCLGE_OPC_QUERY_FEC_STATS, + true); + if (i != (HCLGE_FEC_STATS_CMD_NUM - 1)) + desc[i].flag |= cpu_to_le16(HCLGE_COMM_CMD_FLAG_NEXT); + } + + ret = hclge_cmd_send(&hdev->hw, desc, HCLGE_FEC_STATS_CMD_NUM); + if (ret) + return ret; + + hclge_parse_fec_stats(hdev, desc, HCLGE_FEC_STATS_CMD_NUM); + + return 0; +} + +static void hclge_update_fec_stats(struct hclge_dev *hdev) +{ + struct hnae3_ae_dev *ae_dev = pci_get_drvdata(hdev->pdev); + int ret; + + if (!hnae3_ae_dev_fec_stats_supported(ae_dev) || + test_and_set_bit(HCLGE_STATE_FEC_STATS_UPDATING, &hdev->state)) + return; + + ret = hclge_update_fec_stats_hw(hdev); + if (ret) + dev_err(&hdev->pdev->dev, + "failed to update fec stats, ret = %d\n", ret); + + clear_bit(HCLGE_STATE_FEC_STATS_UPDATING, &hdev->state); +} + +static void hclge_get_fec_stats_total(struct hclge_dev *hdev, + struct ethtool_fec_stats *fec_stats) +{ + fec_stats->corrected_blocks.total = hdev->fec_stats.rs_corr_blocks; + fec_stats->uncorrectable_blocks.total = + hdev->fec_stats.rs_uncorr_blocks; +} + +static void hclge_get_fec_stats_lanes(struct hclge_dev *hdev, + struct ethtool_fec_stats *fec_stats) +{ + u32 i; + + if (hdev->fec_stats.base_r_lane_num == 0 || + hdev->fec_stats.base_r_lane_num > HCLGE_FEC_STATS_MAX_LANES) { + dev_err(&hdev->pdev->dev, + "fec stats lane number(%llu) is invalid\n", + hdev->fec_stats.base_r_lane_num); + return; + } + + for (i = 0; i < hdev->fec_stats.base_r_lane_num; i++) { + fec_stats->corrected_blocks.lanes[i] = + hdev->fec_stats.base_r_corr_per_lanes[i]; + fec_stats->uncorrectable_blocks.lanes[i] = + hdev->fec_stats.base_r_uncorr_per_lanes[i]; + } +} + +static void hclge_comm_get_fec_stats(struct hclge_dev *hdev, + struct ethtool_fec_stats *fec_stats) +{ + u32 fec_mode = hdev->hw.mac.fec_mode; + + switch (fec_mode) { + case BIT(HNAE3_FEC_RS): + case BIT(HNAE3_FEC_LLRS): + hclge_get_fec_stats_total(hdev, fec_stats); + break; + case BIT(HNAE3_FEC_BASER): + hclge_get_fec_stats_lanes(hdev, fec_stats); + break; + default: + dev_err(&hdev->pdev->dev, + "fec stats is not supported by current fec mode(0x%x)\n", + fec_mode); + break; + } +} + +static void hclge_get_fec_stats(struct hnae3_handle *handle, + struct ethtool_fec_stats *fec_stats) +{ + struct hclge_vport *vport = hclge_get_vport(handle); + struct hclge_dev *hdev = vport->back; + u32 fec_mode = hdev->hw.mac.fec_mode; + + if (fec_mode == BIT(HNAE3_FEC_NONE) || + fec_mode == BIT(HNAE3_FEC_AUTO) || + fec_mode == BIT(HNAE3_FEC_USER_DEF)) + return; + + hclge_update_fec_stats(hdev); + + hclge_comm_get_fec_stats(hdev, fec_stats); +} + static int hclge_set_fec_hw(struct hclge_dev *hdev, u32 fec_mode) { struct hclge_config_fec_cmd *req; @@ -2744,6 +2934,9 @@ static int hclge_set_fec_hw(struct hclge_dev *hdev, u32 fec_mode) if (fec_mode & BIT(HNAE3_FEC_RS)) hnae3_set_field(req->fec_mode, HCLGE_MAC_CFG_FEC_MODE_M, HCLGE_MAC_CFG_FEC_MODE_S, HCLGE_MAC_FEC_RS); + if (fec_mode & BIT(HNAE3_FEC_LLRS)) + hnae3_set_field(req->fec_mode, HCLGE_MAC_CFG_FEC_MODE_M, + HCLGE_MAC_CFG_FEC_MODE_S, HCLGE_MAC_FEC_LLRS); if (fec_mode & BIT(HNAE3_FEC_BASER)) hnae3_set_field(req->fec_mode, HCLGE_MAC_CFG_FEC_MODE_M, HCLGE_MAC_CFG_FEC_MODE_S, HCLGE_MAC_FEC_BASER); @@ -2796,7 +2989,7 @@ static int hclge_mac_init(struct hclge_dev *hdev) hdev->support_sfp_query = true; hdev->hw.mac.duplex = HCLGE_MAC_FULL; ret = hclge_cfg_mac_speed_dup_hw(hdev, hdev->hw.mac.speed, - hdev->hw.mac.duplex); + hdev->hw.mac.duplex, hdev->hw.mac.lane_num); if (ret) return ret; @@ -2988,6 +3181,9 @@ static void hclge_update_fec_advertising(struct hclge_mac *mac) if (mac->fec_mode & BIT(HNAE3_FEC_RS)) linkmode_set_bit(ETHTOOL_LINK_MODE_FEC_RS_BIT, mac->advertising); + else if (mac->fec_mode & BIT(HNAE3_FEC_LLRS)) + linkmode_set_bit(ETHTOOL_LINK_MODE_FEC_LLRS_BIT, + mac->advertising); else if (mac->fec_mode & BIT(HNAE3_FEC_BASER)) linkmode_set_bit(ETHTOOL_LINK_MODE_FEC_BASER_BIT, mac->advertising); @@ -3037,7 +3233,6 @@ static void hclge_update_port_capability(struct hclge_dev *hdev, struct hclge_mac *mac) { if (hnae3_dev_fec_supported(hdev)) - /* update fec ability by speed */ hclge_convert_setting_fec(mac); /* firmware can not identify back plane type, the media type @@ -3119,10 +3314,12 @@ static int hclge_get_sfp_info(struct hclge_dev *hdev, struct hclge_mac *mac) mac->autoneg = resp->autoneg; mac->support_autoneg = resp->autoneg_ability; mac->speed_type = QUERY_ACTIVE_SPEED; + mac->lane_num = resp->lane_num; if (!resp->active_fec) mac->fec_mode = 0; else mac->fec_mode = BIT(resp->active_fec); + mac->fec_ability = resp->fec_ability; } else { mac->speed_type = QUERY_SFP_SPEED; } @@ -3302,13 +3499,13 @@ static int hclge_update_port_info(struct hclge_dev *hdev) return 0; } return hclge_cfg_mac_speed_dup(hdev, mac->speed, - HCLGE_MAC_FULL); + HCLGE_MAC_FULL, mac->lane_num); } else { if (speed == HCLGE_MAC_SPEED_UNKNOWN) return 0; /* do nothing if no SFP */ /* must config full duplex for SFP */ - return hclge_cfg_mac_speed_dup(hdev, speed, HCLGE_MAC_FULL); + return hclge_cfg_mac_speed_dup(hdev, speed, HCLGE_MAC_FULL, 0); } } @@ -5334,7 +5531,7 @@ static int hclge_init_fd_config(struct hclge_dev *hdev) struct hclge_fd_key_cfg *key_cfg; int ret; - if (!hnae3_dev_fd_supported(hdev)) + if (!hnae3_ae_dev_fd_supported(hdev->ae_dev)) return 0; ret = hclge_get_fd_mode(hdev, &hdev->fd_cfg.fd_mode); @@ -6339,7 +6536,7 @@ static int hclge_add_fd_entry(struct hnae3_handle *handle, u8 action; int ret; - if (!hnae3_dev_fd_supported(hdev)) { + if (!hnae3_ae_dev_fd_supported(hdev->ae_dev)) { dev_err(&hdev->pdev->dev, "flow table director is not supported\n"); return -EOPNOTSUPP; @@ -6395,7 +6592,7 @@ static int hclge_del_fd_entry(struct hnae3_handle *handle, struct ethtool_rx_flow_spec *fs; int ret; - if (!hnae3_dev_fd_supported(hdev)) + if (!hnae3_ae_dev_fd_supported(hdev->ae_dev)) return -EOPNOTSUPP; fs = (struct ethtool_rx_flow_spec *)&cmd->fs; @@ -6431,9 +6628,6 @@ static void hclge_clear_fd_rules_in_list(struct hclge_dev *hdev, struct hlist_node *node; u16 location; - if (!hnae3_dev_fd_supported(hdev)) - return; - spin_lock_bh(&hdev->fd_rule_lock); for_each_set_bit(location, hdev->fd_bmap, @@ -6458,6 +6652,9 @@ static void hclge_clear_fd_rules_in_list(struct hclge_dev *hdev, static void hclge_del_all_fd_entries(struct hclge_dev *hdev) { + if (!hnae3_ae_dev_fd_supported(hdev->ae_dev)) + return; + hclge_clear_fd_rules_in_list(hdev, true); hclge_fd_disable_user_def(hdev); } @@ -6473,7 +6670,7 @@ static int hclge_restore_fd_entries(struct hnae3_handle *handle) * return value. If error is returned here, the reset process will * fail. */ - if (!hnae3_dev_fd_supported(hdev)) + if (!hnae3_ae_dev_fd_supported(hdev->ae_dev)) return 0; /* if fd is disabled, should not restore it when reset */ @@ -6497,7 +6694,7 @@ static int hclge_get_fd_rule_cnt(struct hnae3_handle *handle, struct hclge_vport *vport = hclge_get_vport(handle); struct hclge_dev *hdev = vport->back; - if (!hnae3_dev_fd_supported(hdev) || hclge_is_cls_flower_active(handle)) + if (!hnae3_ae_dev_fd_supported(hdev->ae_dev) || hclge_is_cls_flower_active(handle)) return -EOPNOTSUPP; cmd->rule_cnt = hdev->hclge_fd_rule_num; @@ -6715,7 +6912,7 @@ static int hclge_get_fd_rule_info(struct hnae3_handle *handle, struct hclge_dev *hdev = vport->back; struct ethtool_rx_flow_spec *fs; - if (!hnae3_dev_fd_supported(hdev)) + if (!hnae3_ae_dev_fd_supported(hdev->ae_dev)) return -EOPNOTSUPP; fs = (struct ethtool_rx_flow_spec *)&cmd->fs; @@ -6778,7 +6975,7 @@ static int hclge_get_all_rules(struct hnae3_handle *handle, struct hlist_node *node2; int cnt = 0; - if (!hnae3_dev_fd_supported(hdev)) + if (!hnae3_ae_dev_fd_supported(hdev->ae_dev)) return -EOPNOTSUPP; cmd->data = hdev->fd_cfg.rule_num[HCLGE_FD_STAGE_1]; @@ -6878,7 +7075,7 @@ static int hclge_add_fd_entry_by_arfs(struct hnae3_handle *handle, u16 queue_id, struct hclge_fd_rule *rule; u16 bit_id; - if (!hnae3_dev_fd_supported(hdev)) + if (!hnae3_ae_dev_fd_supported(hdev->ae_dev)) return -EOPNOTSUPP; /* when there is already fd rule existed add by user, @@ -7167,6 +7364,12 @@ static int hclge_add_cls_flower(struct hnae3_handle *handle, struct hclge_fd_rule *rule; int ret; + if (!hnae3_ae_dev_fd_supported(hdev->ae_dev)) { + dev_err(&hdev->pdev->dev, + "cls flower is not supported\n"); + return -EOPNOTSUPP; + } + ret = hclge_check_cls_flower(hdev, cls_flower, tc); if (ret) { dev_err(&hdev->pdev->dev, @@ -7220,6 +7423,9 @@ static int hclge_del_cls_flower(struct hnae3_handle *handle, struct hclge_fd_rule *rule; int ret; + if (!hnae3_ae_dev_fd_supported(hdev->ae_dev)) + return -EOPNOTSUPP; + spin_lock_bh(&hdev->fd_rule_lock); rule = hclge_find_cls_flower(hdev, cls_flower->cookie); @@ -7282,6 +7488,9 @@ out: static void hclge_sync_fd_table(struct hclge_dev *hdev) { + if (!hnae3_ae_dev_fd_supported(hdev->ae_dev)) + return; + if (test_and_clear_bit(HCLGE_STATE_FD_CLEAR_ALL, &hdev->state)) { bool clear_list = hdev->fd_active_type == HCLGE_FD_ARFS_ACTIVE; @@ -7705,7 +7914,7 @@ static int hclge_set_loopback(struct hnae3_handle *handle, { struct hclge_vport *vport = hclge_get_vport(handle); struct hclge_dev *hdev = vport->back; - int ret; + int ret = 0; /* Loopback can be enabled in three places: SSU, MAC, and serdes. By * default, SSU loopback is enabled, so if the SMAC and the DMAC are @@ -7732,6 +7941,8 @@ static int hclge_set_loopback(struct hnae3_handle *handle, case HNAE3_LOOP_PHY: ret = hclge_set_phy_loopback(hdev, en); break; + case HNAE3_LOOP_EXTERNAL: + break; default: ret = -ENOTSUPP; dev_err(&hdev->pdev->dev, @@ -10793,7 +11004,7 @@ static int hclge_set_pauseparam(struct hnae3_handle *handle, u32 auto_neg, } static void hclge_get_ksettings_an_result(struct hnae3_handle *handle, - u8 *auto_neg, u32 *speed, u8 *duplex) + u8 *auto_neg, u32 *speed, u8 *duplex, u32 *lane_num) { struct hclge_vport *vport = hclge_get_vport(handle); struct hclge_dev *hdev = vport->back; @@ -10804,6 +11015,8 @@ static void hclge_get_ksettings_an_result(struct hnae3_handle *handle, *duplex = hdev->hw.mac.duplex; if (auto_neg) *auto_neg = hdev->hw.mac.autoneg; + if (lane_num) + *lane_num = hdev->hw.mac.lane_num; } static void hclge_get_media_type(struct hnae3_handle *handle, u8 *media_type, @@ -11443,6 +11656,10 @@ static int hclge_init_ae_dev(struct hnae3_ae_dev *ae_dev) if (ret) goto err_mdiobus_unreg; + ret = hclge_update_port_info(hdev); + if (ret) + goto err_mdiobus_unreg; + INIT_KFIFO(hdev->mac_tnl_log); hclge_dcb_ops_set(hdev); @@ -11510,6 +11727,7 @@ out: static void hclge_stats_clear(struct hclge_dev *hdev) { memset(&hdev->mac_stats, 0, sizeof(hdev->mac_stats)); + memset(&hdev->fec_stats, 0, sizeof(hdev->fec_stats)); } static int hclge_set_mac_spoofchk(struct hclge_dev *hdev, int vf, bool enable) @@ -12763,6 +12981,21 @@ static void hclge_clean_vport_config(struct hnae3_ae_dev *ae_dev, int num_vfs) } } +static int hclge_get_dscp_prio(struct hnae3_handle *h, u8 dscp, u8 *tc_mode, + u8 *priority) +{ + if (dscp >= HNAE3_MAX_DSCP) + return -EINVAL; + + if (tc_mode) + *tc_mode = h->kinfo.tc_map_mode; + if (priority) + *priority = h->kinfo.dscp_prio[dscp] == HNAE3_PRIO_ID_INVALID ? 0 : + h->kinfo.dscp_prio[dscp]; + + return 0; +} + static const struct hnae3_ae_ops hclge_ops = { .init_ae_dev = hclge_init_ae_dev, .uninit_ae_dev = hclge_uninit_ae_dev, @@ -12786,6 +13019,7 @@ static const struct hnae3_ae_ops hclge_ops = { .cfg_mac_speed_dup_h = hclge_cfg_mac_speed_dup_h, .get_media_type = hclge_get_media_type, .check_port_speed = hclge_check_port_speed, + .get_fec_stats = hclge_get_fec_stats, .get_fec = hclge_get_fec, .set_fec = hclge_set_fec, .get_rss_key_size = hclge_comm_get_rss_key_size, @@ -12865,6 +13099,7 @@ static const struct hnae3_ae_ops hclge_ops = { .get_ts_info = hclge_ptp_get_ts_info, .get_link_diagnosis_info = hclge_get_link_diagnosis_info, .clean_vf_config = hclge_clean_vport_config, + .get_dscp_prio = hclge_get_dscp_prio, }; static struct hnae3_ae_algo ae_algo = { @@ -12872,7 +13107,7 @@ static struct hnae3_ae_algo ae_algo = { .pdev_id_table = ae_algo_pci_tbl, }; -static int hclge_init(void) +static int __init hclge_init(void) { pr_info("%s is initializing\n", HCLGE_NAME); @@ -12887,7 +13122,7 @@ static int hclge_init(void) return 0; } -static void hclge_exit(void) +static void __exit hclge_exit(void) { hnae3_unregister_ae_algo_prepare(&ae_algo); hnae3_unregister_ae_algo(&ae_algo); diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_main.h b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_main.h index 18caddd541f8..495b639b0dc2 100644 --- a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_main.h +++ b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_main.h @@ -216,6 +216,7 @@ enum HCLGE_DEV_STATE { HCLGE_STATE_FD_USER_DEF_CHANGED, HCLGE_STATE_PTP_EN, HCLGE_STATE_PTP_TX_HANDLING, + HCLGE_STATE_FEC_STATS_UPDATING, HCLGE_STATE_MAX }; @@ -258,6 +259,7 @@ struct hclge_mac { u8 duplex; u8 support_autoneg; u8 speed_type; /* 0: sfp speed, 1: active speed */ + u8 lane_num; u32 speed; u32 max_speed; u32 speed_ability; /* speed ability supported by current media */ @@ -488,6 +490,26 @@ struct hclge_mac_stats { #define HCLGE_STATS_TIMER_INTERVAL 300UL +/* fec stats ,opcode id: 0x0316 */ +#define HCLGE_FEC_STATS_MAX_LANES 8 +struct hclge_fec_stats { + /* fec rs mode total stats */ + u64 rs_corr_blocks; + u64 rs_uncorr_blocks; + u64 rs_error_blocks; + /* fec base-r mode per lanes stats */ + u64 base_r_lane_num; + u64 base_r_corr_blocks; + u64 base_r_uncorr_blocks; + union { + struct { + u64 base_r_corr_per_lanes[HCLGE_FEC_STATS_MAX_LANES]; + u64 base_r_uncorr_per_lanes[HCLGE_FEC_STATS_MAX_LANES]; + }; + u64 per_lanes[HCLGE_FEC_STATS_MAX_LANES * 2]; + }; +}; + struct hclge_vlan_type_cfg { u16 rx_ot_fst_vlan_type; u16 rx_ot_sec_vlan_type; @@ -826,6 +848,7 @@ struct hclge_dev { struct hclge_hw hw; struct hclge_misc_vector misc_vector; struct hclge_mac_stats mac_stats; + struct hclge_fec_stats fec_stats; unsigned long state; unsigned long flr_state; unsigned long last_reset_time; @@ -1070,7 +1093,7 @@ static inline int hclge_get_queue_id(struct hnae3_queue *queue) } int hclge_inform_reset_assert_to_vf(struct hclge_vport *vport); -int hclge_cfg_mac_speed_dup(struct hclge_dev *hdev, int speed, u8 duplex); +int hclge_cfg_mac_speed_dup(struct hclge_dev *hdev, int speed, u8 duplex, u8 lane_num); int hclge_set_vlan_filter(struct hnae3_handle *handle, __be16 proto, u16 vlan_id, bool is_kill); int hclge_en_hw_strip_rxvtag(struct hnae3_handle *handle, bool enable); diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_mbx.c b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_mbx.c index e1012f7f9b73..a7b06c63143c 100644 --- a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_mbx.c +++ b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_mbx.c @@ -779,17 +779,284 @@ static void hclge_handle_vf_tbl(struct hclge_vport *vport, } } +static int +hclge_mbx_map_ring_to_vector_handler(struct hclge_mbx_ops_param *param) +{ + return hclge_map_unmap_ring_to_vf_vector(param->vport, true, + param->req); +} + +static int +hclge_mbx_unmap_ring_to_vector_handler(struct hclge_mbx_ops_param *param) +{ + return hclge_map_unmap_ring_to_vf_vector(param->vport, false, + param->req); +} + +static int +hclge_mbx_get_ring_vector_map_handler(struct hclge_mbx_ops_param *param) +{ + int ret; + + ret = hclge_get_vf_ring_vector_map(param->vport, param->req, + param->resp_msg); + if (ret) + dev_err(¶m->vport->back->pdev->dev, + "PF fail(%d) to get VF ring vector map\n", + ret); + return ret; +} + +static int hclge_mbx_set_promisc_mode_handler(struct hclge_mbx_ops_param *param) +{ + hclge_set_vf_promisc_mode(param->vport, param->req); + return 0; +} + +static int hclge_mbx_set_unicast_handler(struct hclge_mbx_ops_param *param) +{ + int ret; + + ret = hclge_set_vf_uc_mac_addr(param->vport, param->req); + if (ret) + dev_err(¶m->vport->back->pdev->dev, + "PF fail(%d) to set VF UC MAC Addr\n", + ret); + return ret; +} + +static int hclge_mbx_set_multicast_handler(struct hclge_mbx_ops_param *param) +{ + int ret; + + ret = hclge_set_vf_mc_mac_addr(param->vport, param->req); + if (ret) + dev_err(¶m->vport->back->pdev->dev, + "PF fail(%d) to set VF MC MAC Addr\n", + ret); + return ret; +} + +static int hclge_mbx_set_vlan_handler(struct hclge_mbx_ops_param *param) +{ + int ret; + + ret = hclge_set_vf_vlan_cfg(param->vport, param->req, param->resp_msg); + if (ret) + dev_err(¶m->vport->back->pdev->dev, + "PF failed(%d) to config VF's VLAN\n", + ret); + return ret; +} + +static int hclge_mbx_set_alive_handler(struct hclge_mbx_ops_param *param) +{ + int ret; + + ret = hclge_set_vf_alive(param->vport, param->req); + if (ret) + dev_err(¶m->vport->back->pdev->dev, + "PF failed(%d) to set VF's ALIVE\n", + ret); + return ret; +} + +static int hclge_mbx_get_qinfo_handler(struct hclge_mbx_ops_param *param) +{ + hclge_get_vf_queue_info(param->vport, param->resp_msg); + return 0; +} + +static int hclge_mbx_get_qdepth_handler(struct hclge_mbx_ops_param *param) +{ + hclge_get_vf_queue_depth(param->vport, param->resp_msg); + return 0; +} + +static int hclge_mbx_get_basic_info_handler(struct hclge_mbx_ops_param *param) +{ + hclge_get_basic_info(param->vport, param->resp_msg); + return 0; +} + +static int hclge_mbx_get_link_status_handler(struct hclge_mbx_ops_param *param) +{ + int ret; + + ret = hclge_push_vf_link_status(param->vport); + if (ret) + dev_err(¶m->vport->back->pdev->dev, + "failed to inform link stat to VF, ret = %d\n", + ret); + return ret; +} + +static int hclge_mbx_queue_reset_handler(struct hclge_mbx_ops_param *param) +{ + return hclge_mbx_reset_vf_queue(param->vport, param->req, + param->resp_msg); +} + +static int hclge_mbx_reset_handler(struct hclge_mbx_ops_param *param) +{ + return hclge_reset_vf(param->vport); +} + +static int hclge_mbx_keep_alive_handler(struct hclge_mbx_ops_param *param) +{ + hclge_vf_keep_alive(param->vport); + return 0; +} + +static int hclge_mbx_set_mtu_handler(struct hclge_mbx_ops_param *param) +{ + int ret; + + ret = hclge_set_vf_mtu(param->vport, param->req); + if (ret) + dev_err(¶m->vport->back->pdev->dev, + "VF fail(%d) to set mtu\n", ret); + return ret; +} + +static int hclge_mbx_get_qid_in_pf_handler(struct hclge_mbx_ops_param *param) +{ + return hclge_get_queue_id_in_pf(param->vport, param->req, + param->resp_msg); +} + +static int hclge_mbx_get_rss_key_handler(struct hclge_mbx_ops_param *param) +{ + return hclge_get_rss_key(param->vport, param->req, param->resp_msg); +} + +static int hclge_mbx_get_link_mode_handler(struct hclge_mbx_ops_param *param) +{ + hclge_get_link_mode(param->vport, param->req); + return 0; +} + +static int +hclge_mbx_get_vf_flr_status_handler(struct hclge_mbx_ops_param *param) +{ + hclge_rm_vport_all_mac_table(param->vport, false, + HCLGE_MAC_ADDR_UC); + hclge_rm_vport_all_mac_table(param->vport, false, + HCLGE_MAC_ADDR_MC); + hclge_rm_vport_all_vlan_table(param->vport, false); + return 0; +} + +static int hclge_mbx_vf_uninit_handler(struct hclge_mbx_ops_param *param) +{ + hclge_rm_vport_all_mac_table(param->vport, true, + HCLGE_MAC_ADDR_UC); + hclge_rm_vport_all_mac_table(param->vport, true, + HCLGE_MAC_ADDR_MC); + hclge_rm_vport_all_vlan_table(param->vport, true); + return 0; +} + +static int hclge_mbx_get_media_type_handler(struct hclge_mbx_ops_param *param) +{ + hclge_get_vf_media_type(param->vport, param->resp_msg); + return 0; +} + +static int hclge_mbx_push_link_status_handler(struct hclge_mbx_ops_param *param) +{ + hclge_handle_link_change_event(param->vport->back, param->req); + return 0; +} + +static int hclge_mbx_get_mac_addr_handler(struct hclge_mbx_ops_param *param) +{ + hclge_get_vf_mac_addr(param->vport, param->resp_msg); + return 0; +} + +static int hclge_mbx_ncsi_error_handler(struct hclge_mbx_ops_param *param) +{ + hclge_handle_ncsi_error(param->vport->back); + return 0; +} + +static int hclge_mbx_handle_vf_tbl_handler(struct hclge_mbx_ops_param *param) +{ + hclge_handle_vf_tbl(param->vport, param->req); + return 0; +} + +static const hclge_mbx_ops_fn hclge_mbx_ops_list[HCLGE_MBX_OPCODE_MAX] = { + [HCLGE_MBX_RESET] = hclge_mbx_reset_handler, + [HCLGE_MBX_SET_UNICAST] = hclge_mbx_set_unicast_handler, + [HCLGE_MBX_SET_MULTICAST] = hclge_mbx_set_multicast_handler, + [HCLGE_MBX_SET_VLAN] = hclge_mbx_set_vlan_handler, + [HCLGE_MBX_MAP_RING_TO_VECTOR] = hclge_mbx_map_ring_to_vector_handler, + [HCLGE_MBX_UNMAP_RING_TO_VECTOR] = hclge_mbx_unmap_ring_to_vector_handler, + [HCLGE_MBX_SET_PROMISC_MODE] = hclge_mbx_set_promisc_mode_handler, + [HCLGE_MBX_GET_QINFO] = hclge_mbx_get_qinfo_handler, + [HCLGE_MBX_GET_QDEPTH] = hclge_mbx_get_qdepth_handler, + [HCLGE_MBX_GET_BASIC_INFO] = hclge_mbx_get_basic_info_handler, + [HCLGE_MBX_GET_RSS_KEY] = hclge_mbx_get_rss_key_handler, + [HCLGE_MBX_GET_MAC_ADDR] = hclge_mbx_get_mac_addr_handler, + [HCLGE_MBX_GET_LINK_STATUS] = hclge_mbx_get_link_status_handler, + [HCLGE_MBX_QUEUE_RESET] = hclge_mbx_queue_reset_handler, + [HCLGE_MBX_KEEP_ALIVE] = hclge_mbx_keep_alive_handler, + [HCLGE_MBX_SET_ALIVE] = hclge_mbx_set_alive_handler, + [HCLGE_MBX_SET_MTU] = hclge_mbx_set_mtu_handler, + [HCLGE_MBX_GET_QID_IN_PF] = hclge_mbx_get_qid_in_pf_handler, + [HCLGE_MBX_GET_LINK_MODE] = hclge_mbx_get_link_mode_handler, + [HCLGE_MBX_GET_MEDIA_TYPE] = hclge_mbx_get_media_type_handler, + [HCLGE_MBX_VF_UNINIT] = hclge_mbx_vf_uninit_handler, + [HCLGE_MBX_HANDLE_VF_TBL] = hclge_mbx_handle_vf_tbl_handler, + [HCLGE_MBX_GET_RING_VECTOR_MAP] = hclge_mbx_get_ring_vector_map_handler, + [HCLGE_MBX_GET_VF_FLR_STATUS] = hclge_mbx_get_vf_flr_status_handler, + [HCLGE_MBX_PUSH_LINK_STATUS] = hclge_mbx_push_link_status_handler, + [HCLGE_MBX_NCSI_ERROR] = hclge_mbx_ncsi_error_handler, +}; + +static void hclge_mbx_request_handling(struct hclge_mbx_ops_param *param) +{ + hclge_mbx_ops_fn cmd_func = NULL; + struct hclge_dev *hdev; + int ret = 0; + + hdev = param->vport->back; + cmd_func = hclge_mbx_ops_list[param->req->msg.code]; + if (cmd_func) + ret = cmd_func(param); + else + dev_err(&hdev->pdev->dev, + "un-supported mailbox message, code = %u\n", + param->req->msg.code); + + /* PF driver should not reply IMP */ + if (hnae3_get_bit(param->req->mbx_need_resp, HCLGE_MBX_NEED_RESP_B) && + param->req->msg.code < HCLGE_MBX_GET_VF_FLR_STATUS) { + param->resp_msg->status = ret; + if (time_is_before_jiffies(hdev->last_mbx_scheduled + + HCLGE_MBX_SCHED_TIMEOUT)) + dev_warn(&hdev->pdev->dev, + "resp vport%u mbx(%u,%u) late\n", + param->req->mbx_src_vfid, + param->req->msg.code, + param->req->msg.subcode); + + hclge_gen_resp_to_vf(param->vport, param->req, param->resp_msg); + } +} + void hclge_mbx_handler(struct hclge_dev *hdev) { struct hclge_comm_cmq_ring *crq = &hdev->hw.hw.cmq.crq; struct hclge_respond_to_vf_msg resp_msg; struct hclge_mbx_vf_to_pf_cmd *req; - struct hclge_vport *vport; + struct hclge_mbx_ops_param param; struct hclge_desc *desc; - bool is_del = false; unsigned int flag; - int ret = 0; + param.resp_msg = &resp_msg; /* handle all the mailbox requests in the queue */ while (!hclge_cmd_crq_empty(&hdev->hw)) { if (test_bit(HCLGE_COMM_STATE_CMD_DISABLE, @@ -814,152 +1081,16 @@ void hclge_mbx_handler(struct hclge_dev *hdev) continue; } - vport = &hdev->vport[req->mbx_src_vfid]; - trace_hclge_pf_mbx_get(hdev, req); /* clear the resp_msg before processing every mailbox message */ memset(&resp_msg, 0, sizeof(resp_msg)); - - switch (req->msg.code) { - case HCLGE_MBX_MAP_RING_TO_VECTOR: - ret = hclge_map_unmap_ring_to_vf_vector(vport, true, - req); - break; - case HCLGE_MBX_UNMAP_RING_TO_VECTOR: - ret = hclge_map_unmap_ring_to_vf_vector(vport, false, - req); - break; - case HCLGE_MBX_GET_RING_VECTOR_MAP: - ret = hclge_get_vf_ring_vector_map(vport, req, - &resp_msg); - if (ret) - dev_err(&hdev->pdev->dev, - "PF fail(%d) to get VF ring vector map\n", - ret); - break; - case HCLGE_MBX_SET_PROMISC_MODE: - hclge_set_vf_promisc_mode(vport, req); - break; - case HCLGE_MBX_SET_UNICAST: - ret = hclge_set_vf_uc_mac_addr(vport, req); - if (ret) - dev_err(&hdev->pdev->dev, - "PF fail(%d) to set VF UC MAC Addr\n", - ret); - break; - case HCLGE_MBX_SET_MULTICAST: - ret = hclge_set_vf_mc_mac_addr(vport, req); - if (ret) - dev_err(&hdev->pdev->dev, - "PF fail(%d) to set VF MC MAC Addr\n", - ret); - break; - case HCLGE_MBX_SET_VLAN: - ret = hclge_set_vf_vlan_cfg(vport, req, &resp_msg); - if (ret) - dev_err(&hdev->pdev->dev, - "PF failed(%d) to config VF's VLAN\n", - ret); - break; - case HCLGE_MBX_SET_ALIVE: - ret = hclge_set_vf_alive(vport, req); - if (ret) - dev_err(&hdev->pdev->dev, - "PF failed(%d) to set VF's ALIVE\n", - ret); - break; - case HCLGE_MBX_GET_QINFO: - hclge_get_vf_queue_info(vport, &resp_msg); - break; - case HCLGE_MBX_GET_QDEPTH: - hclge_get_vf_queue_depth(vport, &resp_msg); - break; - case HCLGE_MBX_GET_BASIC_INFO: - hclge_get_basic_info(vport, &resp_msg); - break; - case HCLGE_MBX_GET_LINK_STATUS: - ret = hclge_push_vf_link_status(vport); - if (ret) - dev_err(&hdev->pdev->dev, - "failed to inform link stat to VF, ret = %d\n", - ret); - break; - case HCLGE_MBX_QUEUE_RESET: - ret = hclge_mbx_reset_vf_queue(vport, req, &resp_msg); - break; - case HCLGE_MBX_RESET: - ret = hclge_reset_vf(vport); - break; - case HCLGE_MBX_KEEP_ALIVE: - hclge_vf_keep_alive(vport); - break; - case HCLGE_MBX_SET_MTU: - ret = hclge_set_vf_mtu(vport, req); - if (ret) - dev_err(&hdev->pdev->dev, - "VF fail(%d) to set mtu\n", ret); - break; - case HCLGE_MBX_GET_QID_IN_PF: - ret = hclge_get_queue_id_in_pf(vport, req, &resp_msg); - break; - case HCLGE_MBX_GET_RSS_KEY: - ret = hclge_get_rss_key(vport, req, &resp_msg); - break; - case HCLGE_MBX_GET_LINK_MODE: - hclge_get_link_mode(vport, req); - break; - case HCLGE_MBX_GET_VF_FLR_STATUS: - case HCLGE_MBX_VF_UNINIT: - is_del = req->msg.code == HCLGE_MBX_VF_UNINIT; - hclge_rm_vport_all_mac_table(vport, is_del, - HCLGE_MAC_ADDR_UC); - hclge_rm_vport_all_mac_table(vport, is_del, - HCLGE_MAC_ADDR_MC); - hclge_rm_vport_all_vlan_table(vport, is_del); - break; - case HCLGE_MBX_GET_MEDIA_TYPE: - hclge_get_vf_media_type(vport, &resp_msg); - break; - case HCLGE_MBX_PUSH_LINK_STATUS: - hclge_handle_link_change_event(hdev, req); - break; - case HCLGE_MBX_GET_MAC_ADDR: - hclge_get_vf_mac_addr(vport, &resp_msg); - break; - case HCLGE_MBX_NCSI_ERROR: - hclge_handle_ncsi_error(hdev); - break; - case HCLGE_MBX_HANDLE_VF_TBL: - hclge_handle_vf_tbl(vport, req); - break; - default: - dev_err(&hdev->pdev->dev, - "un-supported mailbox message, code = %u\n", - req->msg.code); - break; - } - - /* PF driver should not reply IMP */ - if (hnae3_get_bit(req->mbx_need_resp, HCLGE_MBX_NEED_RESP_B) && - req->msg.code < HCLGE_MBX_GET_VF_FLR_STATUS) { - resp_msg.status = ret; - if (time_is_before_jiffies(hdev->last_mbx_scheduled + - HCLGE_MBX_SCHED_TIMEOUT)) - dev_warn(&hdev->pdev->dev, - "resp vport%u mbx(%u,%u) late\n", - req->mbx_src_vfid, - req->msg.code, - req->msg.subcode); - - hclge_gen_resp_to_vf(vport, req, &resp_msg); - } + param.vport = &hdev->vport[req->mbx_src_vfid]; + param.req = req; + hclge_mbx_request_handling(¶m); crq->desc[crq->next_to_use].flag = 0; hclge_mbx_ring_ptr_move_crq(crq); - - /* reinitialize ret after complete the mbx message processing */ - ret = 0; } /* Write back CMDQ_RQ header pointer, M7 need this pointer */ diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_mdio.c b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_mdio.c index 03d63b6a9b2b..85fb11de43a1 100644 --- a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_mdio.c +++ b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_mdio.c @@ -187,7 +187,7 @@ static void hclge_mac_adjust_link(struct net_device *netdev) speed = netdev->phydev->speed; duplex = netdev->phydev->duplex; - ret = hclge_cfg_mac_speed_dup(hdev, speed, duplex); + ret = hclge_cfg_mac_speed_dup(hdev, speed, duplex, 0); if (ret) netdev_err(netdev, "failed to adjust link.\n"); diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_tm.c b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_tm.c index 2f33b036a47a..4a33f65190e2 100644 --- a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_tm.c +++ b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_tm.c @@ -248,7 +248,7 @@ static int hclge_fill_pri_array(struct hclge_dev *hdev, u8 *pri, u8 pri_id) return 0; } -static int hclge_up_to_tc_map(struct hclge_dev *hdev) +int hclge_up_to_tc_map(struct hclge_dev *hdev) { struct hclge_desc desc; u8 *pri = (u8 *)desc.data; @@ -266,6 +266,47 @@ static int hclge_up_to_tc_map(struct hclge_dev *hdev) return hclge_cmd_send(&hdev->hw, &desc, 1); } +static void hclge_dscp_to_prio_map_init(struct hclge_dev *hdev) +{ + u8 i; + + hdev->vport[0].nic.kinfo.tc_map_mode = HNAE3_TC_MAP_MODE_PRIO; + hdev->vport[0].nic.kinfo.dscp_app_cnt = 0; + for (i = 0; i < HNAE3_MAX_DSCP; i++) + hdev->vport[0].nic.kinfo.dscp_prio[i] = HNAE3_PRIO_ID_INVALID; +} + +int hclge_dscp_to_tc_map(struct hclge_dev *hdev) +{ + struct hclge_desc desc[HCLGE_DSCP_MAP_TC_BD_NUM]; + u8 *req0 = (u8 *)desc[0].data; + u8 *req1 = (u8 *)desc[1].data; + u8 pri_id, tc_id, i, j; + + hclge_cmd_setup_basic_desc(&desc[0], HCLGE_OPC_QOS_MAP, false); + desc[0].flag |= cpu_to_le16(HCLGE_COMM_CMD_FLAG_NEXT); + hclge_cmd_setup_basic_desc(&desc[1], HCLGE_OPC_QOS_MAP, false); + + /* The low 32 dscp setting use bd0, high 32 dscp setting use bd1 */ + for (i = 0; i < HNAE3_MAX_DSCP / HCLGE_DSCP_MAP_TC_BD_NUM; i++) { + pri_id = hdev->vport[0].nic.kinfo.dscp_prio[i]; + pri_id = pri_id == HNAE3_PRIO_ID_INVALID ? 0 : pri_id; + tc_id = hdev->tm_info.prio_tc[pri_id]; + /* Each dscp setting has 4 bits, so each byte saves two dscp + * setting + */ + req0[i >> 1] |= tc_id << HCLGE_DSCP_TC_SHIFT(i); + + j = i + HNAE3_MAX_DSCP / HCLGE_DSCP_MAP_TC_BD_NUM; + pri_id = hdev->vport[0].nic.kinfo.dscp_prio[j]; + pri_id = pri_id == HNAE3_PRIO_ID_INVALID ? 0 : pri_id; + tc_id = hdev->tm_info.prio_tc[pri_id]; + req1[i >> 1] |= tc_id << HCLGE_DSCP_TC_SHIFT(i); + } + + return hclge_cmd_send(&hdev->hw, desc, HCLGE_DSCP_MAP_TC_BD_NUM); +} + static int hclge_tm_pg_to_pri_map_cfg(struct hclge_dev *hdev, u8 pg_id, u8 pri_bit_map) { @@ -1275,6 +1316,12 @@ static int hclge_tm_map_cfg(struct hclge_dev *hdev) if (ret) return ret; + if (hdev->vport[0].nic.kinfo.tc_map_mode == HNAE3_TC_MAP_MODE_DSCP) { + ret = hclge_dscp_to_tc_map(hdev); + if (ret) + return ret; + } + ret = hclge_tm_pg_to_pri_map(hdev); if (ret) return ret; @@ -1646,6 +1693,7 @@ int hclge_tm_schd_init(struct hclge_dev *hdev) return -EINVAL; hclge_tm_schd_info_init(hdev); + hclge_dscp_to_prio_map_init(hdev); return hclge_tm_init_hw(hdev, true); } diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_tm.h b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_tm.h index d943943912f7..68f28a98e380 100644 --- a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_tm.h +++ b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_tm.h @@ -30,6 +30,9 @@ enum hclge_opcode_type; #define HCLGE_TM_PF_MAX_PRI_NUM 8 #define HCLGE_TM_PF_MAX_QSET_NUM 8 +#define HCLGE_DSCP_MAP_TC_BD_NUM 2 +#define HCLGE_DSCP_TC_SHIFT(n) (((n) & 1) * 4) + struct hclge_pg_to_pri_link_cmd { u8 pg_id; u8 rsvd1[3]; @@ -262,4 +265,6 @@ int hclge_tm_get_pg_shaper(struct hclge_dev *hdev, u8 pg_id, struct hclge_tm_shaper_para *para); int hclge_tm_get_port_shaper(struct hclge_dev *hdev, struct hclge_tm_shaper_para *para); +int hclge_up_to_tc_map(struct hclge_dev *hdev); +int hclge_dscp_to_tc_map(struct hclge_dev *hdev); #endif diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_main.c b/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_main.c index 26f87330173e..db6f7cdba958 100644 --- a/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_main.c +++ b/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_main.c @@ -2125,7 +2125,7 @@ static int hclgevf_config_gro(struct hclgevf_dev *hdev) struct hclge_desc desc; int ret; - if (!hnae3_dev_gro_supported(hdev)) + if (!hnae3_ae_dev_gro_supported(hdev->ae_dev)) return 0; hclgevf_cmd_setup_basic_desc(&desc, HCLGE_OPC_GRO_GENERIC_CONFIG, @@ -3177,7 +3177,7 @@ static int hclgevf_get_status(struct hnae3_handle *handle) static void hclgevf_get_ksettings_an_result(struct hnae3_handle *handle, u8 *auto_neg, u32 *speed, - u8 *duplex) + u8 *duplex, u32 *lane_num) { struct hclgevf_dev *hdev = hclgevf_ae_get_hdev(handle); @@ -3429,7 +3429,7 @@ static struct hnae3_ae_algo ae_algovf = { .pdev_id_table = ae_algovf_pci_tbl, }; -static int hclgevf_init(void) +static int __init hclgevf_init(void) { pr_info("%s is initializing\n", HCLGEVF_NAME); @@ -3444,7 +3444,7 @@ static int hclgevf_init(void) return 0; } -static void hclgevf_exit(void) +static void __exit hclgevf_exit(void) { hnae3_unregister_ae_algo(&ae_algovf); destroy_workqueue(hclgevf_wq); diff --git a/drivers/net/ethernet/huawei/hinic/hinic_debugfs.h b/drivers/net/ethernet/huawei/hinic/hinic_debugfs.h index e9e00cfa1329..e10f739d8339 100644 --- a/drivers/net/ethernet/huawei/hinic/hinic_debugfs.h +++ b/drivers/net/ethernet/huawei/hinic/hinic_debugfs.h @@ -12,7 +12,6 @@ #define TBL_ID_FUNC_CFG_SM_INST 1 #define HINIC_FUNCTION_CONFIGURE_TABLE_SIZE 64 -#define HINIC_FUNCTION_CONFIGURE_TABLE 1 struct hinic_cmd_lt_rd { u8 status; diff --git a/drivers/net/ethernet/huawei/hinic/hinic_ethtool.c b/drivers/net/ethernet/huawei/hinic/hinic_ethtool.c index 93192f58ac88..f4b680286911 100644 --- a/drivers/net/ethernet/huawei/hinic/hinic_ethtool.c +++ b/drivers/net/ethernet/huawei/hinic/hinic_ethtool.c @@ -55,7 +55,6 @@ #define COALESCE_ALL_QUEUE 0xFFFF #define COALESCE_MAX_PENDING_LIMIT (255 * COALESCE_PENDING_LIMIT_UNIT) #define COALESCE_MAX_TIMER_CFG (255 * COALESCE_TIMER_CFG_UNIT) -#define OBJ_STR_MAX_LEN 32 struct hw2ethtool_link_mode { enum ethtool_link_mode_bit_indices link_mode_bit; diff --git a/drivers/net/ethernet/huawei/hinic/hinic_hw_cmdq.c b/drivers/net/ethernet/huawei/hinic/hinic_hw_cmdq.c index a627237f694b..78190e88cd75 100644 --- a/drivers/net/ethernet/huawei/hinic/hinic_hw_cmdq.c +++ b/drivers/net/ethernet/huawei/hinic/hinic_hw_cmdq.c @@ -82,11 +82,6 @@ struct hinic_func_to_io, \ cmdqs) -enum cmdq_wqe_type { - WQE_LCMD_TYPE = 0, - WQE_SCMD_TYPE = 1, -}; - enum completion_format { COMPLETE_DIRECT = 0, COMPLETE_SGE = 1, @@ -509,8 +504,8 @@ int hinic_cmdq_direct_resp(struct hinic_cmdqs *cmdqs, * * Return 0 - Success, negative - Failure **/ -int hinic_set_arm_bit(struct hinic_cmdqs *cmdqs, - enum hinic_set_arm_qtype q_type, u32 q_id) +static int hinic_set_arm_bit(struct hinic_cmdqs *cmdqs, + enum hinic_set_arm_qtype q_type, u32 q_id) { struct hinic_cmdq *cmdq = &cmdqs->cmdq[HINIC_CMDQ_SYNC]; struct hinic_hwif *hwif = cmdqs->hwif; diff --git a/drivers/net/ethernet/huawei/hinic/hinic_hw_cmdq.h b/drivers/net/ethernet/huawei/hinic/hinic_hw_cmdq.h index 9c413e963a04..ff09cf0ed52b 100644 --- a/drivers/net/ethernet/huawei/hinic/hinic_hw_cmdq.h +++ b/drivers/net/ethernet/huawei/hinic/hinic_hw_cmdq.h @@ -177,9 +177,6 @@ int hinic_cmdq_direct_resp(struct hinic_cmdqs *cmdqs, enum hinic_mod_type mod, u8 cmd, struct hinic_cmdq_buf *buf_in, u64 *out_param); -int hinic_set_arm_bit(struct hinic_cmdqs *cmdqs, - enum hinic_set_arm_qtype q_type, u32 q_id); - int hinic_init_cmdqs(struct hinic_cmdqs *cmdqs, struct hinic_hwif *hwif, void __iomem **db_area); diff --git a/drivers/net/ethernet/huawei/hinic/hinic_hw_csr.h b/drivers/net/ethernet/huawei/hinic/hinic_hw_csr.h index 7e84e4e33fff..d56e7413ace0 100644 --- a/drivers/net/ethernet/huawei/hinic/hinic_hw_csr.h +++ b/drivers/net/ethernet/huawei/hinic/hinic_hw_csr.h @@ -22,7 +22,6 @@ (HINIC_DMA_ATTR_BASE + (idx) * HINIC_DMA_ATTR_STRIDE) #define HINIC_PPF_ELECTION_STRIDE 0x4 -#define HINIC_CSR_MAX_PORTS 4 #define HINIC_CSR_PPF_ELECTION_ADDR(idx) \ (HINIC_ELECTION_BASE + (idx) * HINIC_PPF_ELECTION_STRIDE) diff --git a/drivers/net/ethernet/huawei/hinic/hinic_hw_dev.c b/drivers/net/ethernet/huawei/hinic/hinic_hw_dev.c index 2127a48749a8..94f470556295 100644 --- a/drivers/net/ethernet/huawei/hinic/hinic_hw_dev.c +++ b/drivers/net/ethernet/huawei/hinic/hinic_hw_dev.c @@ -29,7 +29,6 @@ #include "hinic_hw_io.h" #include "hinic_hw_dev.h" -#define IO_STATUS_TIMEOUT 100 #define OUTBOUND_STATE_TIMEOUT 100 #define DB_STATE_TIMEOUT 100 @@ -42,11 +41,6 @@ enum intr_type { INTR_MSIX_TYPE, }; -enum io_status { - IO_STOPPED = 0, - IO_RUNNING = 1, -}; - /** * parse_capability - convert device capabilities to NIC capabilities * @hwdev: the HW device to set and convert device capabilities for @@ -837,8 +831,8 @@ static int hinic_l2nic_reset(struct hinic_hwdev *hwdev) return 0; } -int hinic_get_interrupt_cfg(struct hinic_hwdev *hwdev, - struct hinic_msix_config *interrupt_info) +static int hinic_get_interrupt_cfg(struct hinic_hwdev *hwdev, + struct hinic_msix_config *interrupt_info) { u16 out_size = sizeof(*interrupt_info); struct hinic_pfhwdev *pfhwdev; @@ -1041,13 +1035,6 @@ void hinic_free_hwdev(struct hinic_hwdev *hwdev) hinic_free_hwif(hwdev->hwif); } -int hinic_hwdev_max_num_qps(struct hinic_hwdev *hwdev) -{ - struct hinic_cap *nic_cap = &hwdev->nic_cap; - - return nic_cap->max_qps; -} - /** * hinic_hwdev_num_qps - return the number QPs available for use * @hwdev: the NIC HW device diff --git a/drivers/net/ethernet/huawei/hinic/hinic_hw_dev.h b/drivers/net/ethernet/huawei/hinic/hinic_hw_dev.h index 416492e48274..d2d89b0a5ef0 100644 --- a/drivers/net/ethernet/huawei/hinic/hinic_hw_dev.h +++ b/drivers/net/ethernet/huawei/hinic/hinic_hw_dev.h @@ -566,8 +566,6 @@ struct hinic_hwdev *hinic_init_hwdev(struct pci_dev *pdev, struct devlink *devli void hinic_free_hwdev(struct hinic_hwdev *hwdev); -int hinic_hwdev_max_num_qps(struct hinic_hwdev *hwdev); - int hinic_hwdev_num_qps(struct hinic_hwdev *hwdev); struct hinic_sq *hinic_hwdev_get_sq(struct hinic_hwdev *hwdev, int i); @@ -587,9 +585,6 @@ int hinic_hwdev_hw_ci_addr_set(struct hinic_hwdev *hwdev, struct hinic_sq *sq, void hinic_hwdev_set_msix_state(struct hinic_hwdev *hwdev, u16 msix_index, enum hinic_msix_state flag); -int hinic_get_interrupt_cfg(struct hinic_hwdev *hwdev, - struct hinic_msix_config *interrupt_info); - int hinic_set_interrupt_cfg(struct hinic_hwdev *hwdev, struct hinic_msix_config *interrupt_info); diff --git a/drivers/net/ethernet/huawei/hinic/hinic_hw_if.c b/drivers/net/ethernet/huawei/hinic/hinic_hw_if.c index 0428faa68e80..88567305d06e 100644 --- a/drivers/net/ethernet/huawei/hinic/hinic_hw_if.c +++ b/drivers/net/ethernet/huawei/hinic/hinic_hw_if.c @@ -58,39 +58,6 @@ int hinic_msix_attr_set(struct hinic_hwif *hwif, u16 msix_index, } /** - * hinic_msix_attr_get - get message attribute of msix entry - * @hwif: the HW interface of a pci function device - * @msix_index: msix_index - * @pending_limit: the maximum pending interrupt events (unit 8) - * @coalesc_timer: coalesc period for interrupt (unit 8 us) - * @lli_timer: replenishing period for low latency credit (unit 8 us) - * @lli_credit_limit: maximum credits for low latency msix messages (unit 8) - * @resend_timer: maximum wait for resending msix (unit coalesc period) - * - * Return 0 - Success, negative - Failure - **/ -int hinic_msix_attr_get(struct hinic_hwif *hwif, u16 msix_index, - u8 *pending_limit, u8 *coalesc_timer, - u8 *lli_timer, u8 *lli_credit_limit, - u8 *resend_timer) -{ - u32 addr, val; - - if (!VALID_MSIX_IDX(&hwif->attr, msix_index)) - return -EINVAL; - - addr = HINIC_CSR_MSIX_CTRL_ADDR(msix_index); - val = hinic_hwif_read_reg(hwif, addr); - - *pending_limit = HINIC_MSIX_ATTR_GET(val, PENDING_LIMIT); - *coalesc_timer = HINIC_MSIX_ATTR_GET(val, COALESC_TIMER); - *lli_timer = HINIC_MSIX_ATTR_GET(val, LLI_TIMER); - *lli_credit_limit = HINIC_MSIX_ATTR_GET(val, LLI_CREDIT); - *resend_timer = HINIC_MSIX_ATTR_GET(val, RESEND_TIMER); - return 0; -} - -/** * hinic_msix_attr_cnt_clear - clear message attribute counters for msix entry * @hwif: the HW interface of a pci function device * @msix_index: msix_index @@ -115,8 +82,6 @@ int hinic_msix_attr_cnt_clear(struct hinic_hwif *hwif, u16 msix_index) * hinic_set_pf_action - set action on pf channel * @hwif: the HW interface of a pci function device * @action: action on pf channel - * - * Return 0 - Success, negative - Failure **/ void hinic_set_pf_action(struct hinic_hwif *hwif, enum hinic_pf_action action) { diff --git a/drivers/net/ethernet/huawei/hinic/hinic_hw_if.h b/drivers/net/ethernet/huawei/hinic/hinic_hw_if.h index c06f2253151e..3d588896a367 100644 --- a/drivers/net/ethernet/huawei/hinic/hinic_hw_if.h +++ b/drivers/net/ethernet/huawei/hinic/hinic_hw_if.h @@ -131,10 +131,6 @@ (((u32)(val) & HINIC_MSIX_##member##_MASK) << \ HINIC_MSIX_##member##_SHIFT) -#define HINIC_MSIX_ATTR_GET(val, member) \ - (((val) >> HINIC_MSIX_##member##_SHIFT) & \ - HINIC_MSIX_##member##_MASK) - #define HINIC_MSIX_CNT_RESEND_TIMER_SHIFT 29 #define HINIC_MSIX_CNT_RESEND_TIMER_MASK 0x1 @@ -269,11 +265,6 @@ int hinic_msix_attr_set(struct hinic_hwif *hwif, u16 msix_index, u8 lli_timer_cfg, u8 lli_credit_limit, u8 resend_timer); -int hinic_msix_attr_get(struct hinic_hwif *hwif, u16 msix_index, - u8 *pending_limit, u8 *coalesc_timer_cfg, - u8 *lli_timer, u8 *lli_credit_limit, - u8 *resend_timer); - void hinic_set_msix_state(struct hinic_hwif *hwif, u16 msix_idx, enum hinic_msix_state flag); diff --git a/drivers/net/ethernet/huawei/hinic/hinic_hw_mbox.c b/drivers/net/ethernet/huawei/hinic/hinic_hw_mbox.c index 5078c0c73863..3f9c31d29215 100644 --- a/drivers/net/ethernet/huawei/hinic/hinic_hw_mbox.c +++ b/drivers/net/ethernet/huawei/hinic/hinic_hw_mbox.c @@ -117,7 +117,6 @@ enum hinic_mbox_tx_status { #define MBOX_WB_STATUS_MASK 0xFF #define MBOX_WB_ERROR_CODE_MASK 0xFF00 #define MBOX_WB_STATUS_FINISHED_SUCCESS 0xFF -#define MBOX_WB_STATUS_FINISHED_WITH_ERR 0xFE #define MBOX_WB_STATUS_NOT_FINISHED 0x00 #define MBOX_STATUS_FINISHED(wb) \ @@ -130,11 +129,8 @@ enum hinic_mbox_tx_status { #define SEQ_ID_START_VAL 0 #define SEQ_ID_MAX_VAL 42 -#define DST_AEQ_IDX_DEFAULT_VAL 0 -#define SRC_AEQ_IDX_DEFAULT_VAL 0 #define NO_DMA_ATTRIBUTE_VAL 0 -#define HINIC_MGMT_RSP_AEQN 0 #define HINIC_MBOX_RSP_AEQN 2 #define HINIC_MBOX_RECV_AEQN 0 @@ -146,7 +142,6 @@ enum hinic_mbox_tx_status { #define IS_PF_OR_PPF_SRC(src_func_idx) ((src_func_idx) < HINIC_MAX_PF_FUNCS) -#define MBOX_RESPONSE_ERROR 0x1 #define MBOX_MSG_ID_MASK 0xFF #define MBOX_MSG_ID(func_to_func) ((func_to_func)->send_msg_id) #define MBOX_MSG_ID_INC(func_to_func_mbox) (MBOX_MSG_ID(func_to_func_mbox) = \ @@ -621,7 +616,7 @@ static bool check_vf_mbox_random_id(struct hinic_mbox_func_to_func *func_to_func return false; } -void hinic_mbox_func_aeqe_handler(void *handle, void *header, u8 size) +static void hinic_mbox_func_aeqe_handler(void *handle, void *header, u8 size) { struct hinic_mbox_func_to_func *func_to_func; u64 mbox_header = *((u64 *)header); @@ -649,7 +644,7 @@ void hinic_mbox_func_aeqe_handler(void *handle, void *header, u8 size) recv_mbox_handler(func_to_func, (u64 *)header, recv_mbox); } -void hinic_mbox_self_aeqe_handler(void *handle, void *header, u8 size) +static void hinic_mbox_self_aeqe_handler(void *handle, void *header, u8 size) { struct hinic_mbox_func_to_func *func_to_func; struct hinic_send_mbox *send_mbox; diff --git a/drivers/net/ethernet/huawei/hinic/hinic_hw_mbox.h b/drivers/net/ethernet/huawei/hinic/hinic_hw_mbox.h index 46953190d29e..33ac7814d3b3 100644 --- a/drivers/net/ethernet/huawei/hinic/hinic_hw_mbox.h +++ b/drivers/net/ethernet/huawei/hinic/hinic_hw_mbox.h @@ -150,10 +150,6 @@ void hinic_unregister_pf_mbox_cb(struct hinic_hwdev *hwdev, void hinic_unregister_vf_mbox_cb(struct hinic_hwdev *hwdev, enum hinic_mod_type mod); -void hinic_mbox_func_aeqe_handler(void *handle, void *header, u8 size); - -void hinic_mbox_self_aeqe_handler(void *handle, void *header, u8 size); - int hinic_func_to_func_init(struct hinic_hwdev *hwdev); void hinic_func_to_func_free(struct hinic_hwdev *hwdev); diff --git a/drivers/net/ethernet/huawei/hinic/hinic_hw_qp.c b/drivers/net/ethernet/huawei/hinic/hinic_hw_qp.c index 336248aa2e48..537a8098bc4e 100644 --- a/drivers/net/ethernet/huawei/hinic/hinic_hw_qp.c +++ b/drivers/net/ethernet/huawei/hinic/hinic_hw_qp.c @@ -472,8 +472,7 @@ int hinic_get_rq_free_wqebbs(struct hinic_rq *rq) return atomic_read(&wq->delta) - 1; } -static void sq_prepare_ctrl(struct hinic_sq_ctrl *ctrl, u16 prod_idx, - int nr_descs) +static void sq_prepare_ctrl(struct hinic_sq_ctrl *ctrl, int nr_descs) { u32 ctrl_size, task_size, bufdesc_size; @@ -588,18 +587,16 @@ void hinic_set_tso_inner_l4(struct hinic_sq_task *task, u32 *queue_info, /** * hinic_sq_prepare_wqe - prepare wqe before insert to the queue * @sq: send queue - * @prod_idx: pi value * @sq_wqe: wqe to prepare * @sges: sges for use by the wqe for send for buf addresses * @nr_sges: number of sges **/ -void hinic_sq_prepare_wqe(struct hinic_sq *sq, u16 prod_idx, - struct hinic_sq_wqe *sq_wqe, struct hinic_sge *sges, - int nr_sges) +void hinic_sq_prepare_wqe(struct hinic_sq *sq, struct hinic_sq_wqe *sq_wqe, + struct hinic_sge *sges, int nr_sges) { int i; - sq_prepare_ctrl(&sq_wqe->ctrl, prod_idx, nr_sges); + sq_prepare_ctrl(&sq_wqe->ctrl, nr_sges); sq_prepare_task(&sq_wqe->task); diff --git a/drivers/net/ethernet/huawei/hinic/hinic_hw_qp.h b/drivers/net/ethernet/huawei/hinic/hinic_hw_qp.h index 0dfa51ad5855..178dcc874370 100644 --- a/drivers/net/ethernet/huawei/hinic/hinic_hw_qp.h +++ b/drivers/net/ethernet/huawei/hinic/hinic_hw_qp.h @@ -175,9 +175,8 @@ void hinic_set_tso_inner_l4(struct hinic_sq_task *task, u32 l4_len, u32 offset, u32 ip_ident, u32 mss); -void hinic_sq_prepare_wqe(struct hinic_sq *sq, u16 prod_idx, - struct hinic_sq_wqe *wqe, struct hinic_sge *sges, - int nr_sges); +void hinic_sq_prepare_wqe(struct hinic_sq *sq, struct hinic_sq_wqe *wqe, + struct hinic_sge *sges, int nr_sges); void hinic_sq_write_db(struct hinic_sq *sq, u16 prod_idx, unsigned int wqe_size, unsigned int cos); diff --git a/drivers/net/ethernet/huawei/hinic/hinic_hw_wq.c b/drivers/net/ethernet/huawei/hinic/hinic_hw_wq.c index 4daf6bf291ec..e1a1735c00c1 100644 --- a/drivers/net/ethernet/huawei/hinic/hinic_hw_wq.c +++ b/drivers/net/ethernet/huawei/hinic/hinic_hw_wq.c @@ -175,8 +175,6 @@ static int cmdq_allocate_page(struct hinic_cmdq_pages *cmdq_pages) /** * cmdq_free_page - free page from cmdq * @cmdq_pages: the pages of the cmdq queue struct that hold the page - * - * Return 0 - Success, negative - Failure **/ static void cmdq_free_page(struct hinic_cmdq_pages *cmdq_pages) { diff --git a/drivers/net/ethernet/huawei/hinic/hinic_hw_wqe.h b/drivers/net/ethernet/huawei/hinic/hinic_hw_wqe.h index f4b6d2c1061f..c6bdeed5606e 100644 --- a/drivers/net/ethernet/huawei/hinic/hinic_hw_wqe.h +++ b/drivers/net/ethernet/huawei/hinic/hinic_hw_wqe.h @@ -261,23 +261,6 @@ #define HINIC_RSS_TYPE_GET(val, member) \ (((u32)(val) >> HINIC_RSS_TYPE_##member##_SHIFT) & 0x1) -enum hinic_l4offload_type { - HINIC_L4_OFF_DISABLE = 0, - HINIC_TCP_OFFLOAD_ENABLE = 1, - HINIC_SCTP_OFFLOAD_ENABLE = 2, - HINIC_UDP_OFFLOAD_ENABLE = 3, -}; - -enum hinic_vlan_offload { - HINIC_VLAN_OFF_DISABLE = 0, - HINIC_VLAN_OFF_ENABLE = 1, -}; - -enum hinic_pkt_parsed { - HINIC_PKT_NOT_PARSED = 0, - HINIC_PKT_PARSED = 1, -}; - enum hinic_l3_offload_type { L3TYPE_UNKNOWN = 0, IPV6_PKT = 1, @@ -305,18 +288,10 @@ enum hinic_outer_l3type { HINIC_OUTER_L3TYPE_IPV4_CHKSUM = 3, }; -enum hinic_media_type { - HINIC_MEDIA_UNKNOWN = 0, -}; - enum hinic_l2type { HINIC_L2TYPE_ETH = 0, }; -enum hinc_tunnel_l4type { - HINIC_TUNNEL_L4TYPE_UNKNOWN = 0, -}; - struct hinic_cmdq_header { u32 header_info; u32 saved_data; diff --git a/drivers/net/ethernet/huawei/hinic/hinic_main.c b/drivers/net/ethernet/huawei/hinic/hinic_main.c index c23ee2ddbce3..e1f54a2f28b2 100644 --- a/drivers/net/ethernet/huawei/hinic/hinic_main.c +++ b/drivers/net/ethernet/huawei/hinic/hinic_main.c @@ -960,8 +960,6 @@ static void hinic_refresh_nic_cfg(struct hinic_dev *nic_dev) * @in_size: input size * @buf_out: output buffer * @out_size: returned output size - * - * Return 0 - Success, negative - Failure **/ static void link_status_event_handler(void *handle, void *buf_in, u16 in_size, void *buf_out, u16 *out_size) @@ -1382,8 +1380,6 @@ err_pci_regions: return err; } -#define HINIC_WAIT_SRIOV_CFG_TIMEOUT 15000 - static void wait_sriov_cfg_complete(struct hinic_dev *nic_dev) { struct hinic_sriov_info *sriov_info = &nic_dev->sriov_info; diff --git a/drivers/net/ethernet/huawei/hinic/hinic_rx.c b/drivers/net/ethernet/huawei/hinic/hinic_rx.c index e5828a658caf..d649c6e323c8 100644 --- a/drivers/net/ethernet/huawei/hinic/hinic_rx.c +++ b/drivers/net/ethernet/huawei/hinic/hinic_rx.c @@ -50,7 +50,7 @@ * hinic_rxq_clean_stats - Clean the statistics of specific queue * @rxq: Logical Rx Queue **/ -void hinic_rxq_clean_stats(struct hinic_rxq *rxq) +static void hinic_rxq_clean_stats(struct hinic_rxq *rxq) { struct hinic_rxq_stats *rxq_stats = &rxq->rxq_stats; diff --git a/drivers/net/ethernet/huawei/hinic/hinic_rx.h b/drivers/net/ethernet/huawei/hinic/hinic_rx.h index 507dcbae9085..8f7bd6a049bd 100644 --- a/drivers/net/ethernet/huawei/hinic/hinic_rx.h +++ b/drivers/net/ethernet/huawei/hinic/hinic_rx.h @@ -41,8 +41,6 @@ struct hinic_rxq { struct napi_struct napi; }; -void hinic_rxq_clean_stats(struct hinic_rxq *rxq); - void hinic_rxq_get_stats(struct hinic_rxq *rxq, struct hinic_rxq_stats *stats); int hinic_init_rxq(struct hinic_rxq *rxq, struct hinic_rq *rq, diff --git a/drivers/net/ethernet/huawei/hinic/hinic_sriov.c b/drivers/net/ethernet/huawei/hinic/hinic_sriov.c index df555847afb5..a5f08b969e3f 100644 --- a/drivers/net/ethernet/huawei/hinic/hinic_sriov.c +++ b/drivers/net/ethernet/huawei/hinic/hinic_sriov.c @@ -24,6 +24,7 @@ MODULE_PARM_DESC(set_vf_link_state, "Set vf link state, 0 represents link auto, #define HINIC_VLAN_PRIORITY_SHIFT 13 #define HINIC_ADD_VLAN_IN_MAC 0x8000 #define HINIC_TX_RATE_TABLE_FULL 12 +#define HINIC_MAX_QOS 7 static int hinic_set_mac(struct hinic_hwdev *hwdev, const u8 *mac_addr, u16 vlan_id, u16 func_id) @@ -774,7 +775,7 @@ int hinic_ndo_set_vf_vlan(struct net_device *netdev, int vf, u16 vlan, u8 qos, u16 vlanprio, cur_vlanprio; sriov_info = &nic_dev->sriov_info; - if (vf >= sriov_info->num_vfs || vlan > 4095 || qos > 7) + if (vf >= sriov_info->num_vfs || vlan >= VLAN_N_VID || qos > HINIC_MAX_QOS) return -EINVAL; if (vlan_proto != htons(ETH_P_8021Q)) return -EPROTONOSUPPORT; @@ -820,7 +821,7 @@ int hinic_ndo_set_vf_trust(struct net_device *netdev, int vf, bool setting) cur_trust = nic_io->vf_infos[vf].trust; /* same request, so just return success */ - if ((setting && cur_trust) || (!setting && !cur_trust)) + if (setting == cur_trust) return 0; err = hinic_set_vf_trust(adapter->hwdev, vf, setting); @@ -940,7 +941,7 @@ int hinic_ndo_set_vf_spoofchk(struct net_device *netdev, int vf, bool setting) cur_spoofchk = nic_dev->hwdev->func_to_io.vf_infos[vf].spoofchk; /* same request, so just return success */ - if ((setting && cur_spoofchk) || (!setting && !cur_spoofchk)) + if (setting == cur_spoofchk) return 0; err = hinic_set_vf_spoofchk(sriov_info->hwdev, @@ -1131,8 +1132,8 @@ static void hinic_clear_vf_infos(struct hinic_dev *nic_dev, u16 vf_id) hinic_init_vf_infos(&nic_dev->hwdev->func_to_io, HW_VF_ID_TO_OS(vf_id)); } -static int hinic_deinit_vf_hw(struct hinic_sriov_info *sriov_info, - u16 start_vf_id, u16 end_vf_id) +static void hinic_deinit_vf_hw(struct hinic_sriov_info *sriov_info, + u16 start_vf_id, u16 end_vf_id) { struct hinic_dev *nic_dev; u16 func_idx, idx; @@ -1145,8 +1146,6 @@ static int hinic_deinit_vf_hw(struct hinic_sriov_info *sriov_info, HINIC_HW_WQ_PAGE_SIZE); hinic_clear_vf_infos(nic_dev, idx); } - - return 0; } int hinic_vf_func_init(struct hinic_hwdev *hwdev) @@ -1293,7 +1292,7 @@ int hinic_pci_sriov_disable(struct pci_dev *pdev) return 0; } -int hinic_pci_sriov_enable(struct pci_dev *pdev, int num_vfs) +static int hinic_pci_sriov_enable(struct pci_dev *pdev, int num_vfs) { struct hinic_sriov_info *sriov_info; int err; diff --git a/drivers/net/ethernet/huawei/hinic/hinic_sriov.h b/drivers/net/ethernet/huawei/hinic/hinic_sriov.h index ba627a362f9a..d4d4e63d31ea 100644 --- a/drivers/net/ethernet/huawei/hinic/hinic_sriov.h +++ b/drivers/net/ethernet/huawei/hinic/hinic_sriov.h @@ -98,8 +98,6 @@ void hinic_notify_all_vfs_link_changed(struct hinic_hwdev *hwdev, int hinic_pci_sriov_disable(struct pci_dev *dev); -int hinic_pci_sriov_enable(struct pci_dev *dev, int num_vfs); - int hinic_vf_func_init(struct hinic_hwdev *hwdev); void hinic_vf_func_free(struct hinic_hwdev *hwdev); diff --git a/drivers/net/ethernet/huawei/hinic/hinic_tx.c b/drivers/net/ethernet/huawei/hinic/hinic_tx.c index 3b6c7b585737..e91476c8ff8b 100644 --- a/drivers/net/ethernet/huawei/hinic/hinic_tx.c +++ b/drivers/net/ethernet/huawei/hinic/hinic_tx.c @@ -74,7 +74,7 @@ enum hinic_offload_type { * hinic_txq_clean_stats - Clean the statistics of specific queue * @txq: Logical Tx Queue **/ -void hinic_txq_clean_stats(struct hinic_txq *txq) +static void hinic_txq_clean_stats(struct hinic_txq *txq) { struct hinic_txq_stats *txq_stats = &txq->txq_stats; @@ -530,7 +530,7 @@ netdev_tx_t hinic_lb_xmit_frame(struct sk_buff *skb, struct net_device *netdev) } process_sq_wqe: - hinic_sq_prepare_wqe(txq->sq, prod_idx, sq_wqe, txq->sges, nr_sges); + hinic_sq_prepare_wqe(txq->sq, sq_wqe, txq->sges, nr_sges); hinic_sq_write_wqe(txq->sq, prod_idx, sq_wqe, skb, wqe_size); flush_skbs: @@ -614,7 +614,7 @@ netdev_tx_t hinic_xmit_frame(struct sk_buff *skb, struct net_device *netdev) } process_sq_wqe: - hinic_sq_prepare_wqe(txq->sq, prod_idx, sq_wqe, txq->sges, nr_sges); + hinic_sq_prepare_wqe(txq->sq, sq_wqe, txq->sges, nr_sges); err = hinic_tx_offload(skb, &sq_wqe->task, &sq_wqe->ctrl.queue_info); if (err) diff --git a/drivers/net/ethernet/huawei/hinic/hinic_tx.h b/drivers/net/ethernet/huawei/hinic/hinic_tx.h index b3c8657774a7..91dc778362f3 100644 --- a/drivers/net/ethernet/huawei/hinic/hinic_tx.h +++ b/drivers/net/ethernet/huawei/hinic/hinic_tx.h @@ -40,8 +40,6 @@ struct hinic_txq { struct napi_struct napi; }; -void hinic_txq_clean_stats(struct hinic_txq *txq); - void hinic_txq_get_stats(struct hinic_txq *txq, struct hinic_txq_stats *stats); netdev_tx_t hinic_lb_xmit_frame(struct sk_buff *skb, struct net_device *netdev); diff --git a/drivers/net/ethernet/ibm/ehea/ehea_ethtool.c b/drivers/net/ethernet/ibm/ehea/ehea_ethtool.c index 6cb86032ce46..1db5b6790a41 100644 --- a/drivers/net/ethernet/ibm/ehea/ehea_ethtool.c +++ b/drivers/net/ethernet/ibm/ehea/ehea_ethtool.c @@ -159,8 +159,8 @@ static int ehea_nway_reset(struct net_device *dev) static void ehea_get_drvinfo(struct net_device *dev, struct ethtool_drvinfo *info) { - strlcpy(info->driver, DRV_NAME, sizeof(info->driver)); - strlcpy(info->version, DRV_VERSION, sizeof(info->version)); + strscpy(info->driver, DRV_NAME, sizeof(info->driver)); + strscpy(info->version, DRV_VERSION, sizeof(info->version)); } static u32 ehea_get_msglevel(struct net_device *dev) diff --git a/drivers/net/ethernet/ibm/ehea/ehea_main.c b/drivers/net/ethernet/ibm/ehea/ehea_main.c index 5dc302880f5f..294bdbbeacc3 100644 --- a/drivers/net/ethernet/ibm/ehea/ehea_main.c +++ b/drivers/net/ethernet/ibm/ehea/ehea_main.c @@ -1546,7 +1546,7 @@ static int ehea_init_port_res(struct ehea_port *port, struct ehea_port_res *pr, kfree(init_attr); - netif_napi_add(pr->port->netdev, &pr->napi, ehea_poll, 64); + netif_napi_add(pr->port->netdev, &pr->napi, ehea_poll); ret = 0; goto out; diff --git a/drivers/net/ethernet/ibm/emac/core.c b/drivers/net/ethernet/ibm/emac/core.c index fbea9f7efe8c..9b08e41ccc29 100644 --- a/drivers/net/ethernet/ibm/emac/core.c +++ b/drivers/net/ethernet/ibm/emac/core.c @@ -2284,8 +2284,8 @@ static void emac_ethtool_get_drvinfo(struct net_device *ndev, { struct emac_instance *dev = netdev_priv(ndev); - strlcpy(info->driver, "ibm_emac", sizeof(info->driver)); - strlcpy(info->version, DRV_VERSION, sizeof(info->version)); + strscpy(info->driver, "ibm_emac", sizeof(info->driver)); + strscpy(info->version, DRV_VERSION, sizeof(info->version)); snprintf(info->bus_info, sizeof(info->bus_info), "PPC 4xx EMAC-%d %pOF", dev->cell_index, dev->ofdev->dev.of_node); } @@ -2979,11 +2979,9 @@ static int emac_init_config(struct emac_instance *dev) /* Read MAC-address */ err = of_get_ethdev_address(np, dev->ndev); - if (err) { - if (err != -EPROBE_DEFER) - dev_err(&dev->ofdev->dev, "Can't get valid [local-]mac-address from OF !\n"); - return err; - } + if (err) + return dev_err_probe(&dev->ofdev->dev, err, + "Can't get valid [local-]mac-address from OF !\n"); /* IAHT and GAHT filter parameterization */ if (emac_has_feature(dev, EMAC_FTR_EMAC4SYNC)) { diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c index 5c6a04d29f5b..3b14dc93f59d 100644 --- a/drivers/net/ethernet/ibm/ibmveth.c +++ b/drivers/net/ethernet/ibm/ibmveth.c @@ -141,6 +141,13 @@ static inline int ibmveth_rxq_csum_good(struct ibmveth_adapter *adapter) return ibmveth_rxq_flags(adapter) & IBMVETH_RXQ_CSUM_GOOD; } +static unsigned int ibmveth_real_max_tx_queues(void) +{ + unsigned int n_cpu = num_online_cpus(); + + return min(n_cpu, IBMVETH_MAX_QUEUES); +} + /* setup the initial settings for a buffer pool */ static void ibmveth_init_buffer_pool(struct ibmveth_buff_pool *pool, u32 pool_index, u32 pool_size, @@ -456,6 +463,38 @@ static void ibmveth_rxq_harvest_buffer(struct ibmveth_adapter *adapter) } } +static void ibmveth_free_tx_ltb(struct ibmveth_adapter *adapter, int idx) +{ + dma_unmap_single(&adapter->vdev->dev, adapter->tx_ltb_dma[idx], + adapter->tx_ltb_size, DMA_TO_DEVICE); + kfree(adapter->tx_ltb_ptr[idx]); + adapter->tx_ltb_ptr[idx] = NULL; +} + +static int ibmveth_allocate_tx_ltb(struct ibmveth_adapter *adapter, int idx) +{ + adapter->tx_ltb_ptr[idx] = kzalloc(adapter->tx_ltb_size, + GFP_KERNEL); + if (!adapter->tx_ltb_ptr[idx]) { + netdev_err(adapter->netdev, + "unable to allocate tx long term buffer\n"); + return -ENOMEM; + } + adapter->tx_ltb_dma[idx] = dma_map_single(&adapter->vdev->dev, + adapter->tx_ltb_ptr[idx], + adapter->tx_ltb_size, + DMA_TO_DEVICE); + if (dma_mapping_error(&adapter->vdev->dev, adapter->tx_ltb_dma[idx])) { + netdev_err(adapter->netdev, + "unable to DMA map tx long term buffer\n"); + kfree(adapter->tx_ltb_ptr[idx]); + adapter->tx_ltb_ptr[idx] = NULL; + return -ENOMEM; + } + + return 0; +} + static int ibmveth_register_logical_lan(struct ibmveth_adapter *adapter, union ibmveth_buf_desc rxq_desc, u64 mac_address) { @@ -538,6 +577,11 @@ static int ibmveth_open(struct net_device *netdev) goto out_unmap_buffer_list; } + for (i = 0; i < netdev->real_num_tx_queues; i++) { + if (ibmveth_allocate_tx_ltb(adapter, i)) + goto out_free_tx_ltb; + } + adapter->rx_queue.index = 0; adapter->rx_queue.num_slots = rxq_entries; adapter->rx_queue.toggle = 1; @@ -595,25 +639,15 @@ static int ibmveth_open(struct net_device *netdev) rc = -ENOMEM; - adapter->bounce_buffer = dma_alloc_coherent(&adapter->vdev->dev, - netdev->mtu + IBMVETH_BUFF_OH, - &adapter->bounce_buffer_dma, GFP_KERNEL); - if (!adapter->bounce_buffer) { - netdev_err(netdev, "unable to alloc bounce buffer\n"); - goto out_free_irq; - } - netdev_dbg(netdev, "initial replenish cycle\n"); ibmveth_interrupt(netdev->irq, netdev); - netif_start_queue(netdev); + netif_tx_start_all_queues(netdev); netdev_dbg(netdev, "open complete\n"); return 0; -out_free_irq: - free_irq(netdev->irq, netdev); out_free_buffer_pools: while (--i >= 0) { if (adapter->rx_buff_pool[i].active) @@ -623,6 +657,12 @@ out_free_buffer_pools: out_unmap_filter_list: dma_unmap_single(dev, adapter->filter_list_dma, 4096, DMA_BIDIRECTIONAL); + +out_free_tx_ltb: + while (--i >= 0) { + ibmveth_free_tx_ltb(adapter, i); + } + out_unmap_buffer_list: dma_unmap_single(dev, adapter->buffer_list_dma, 4096, DMA_BIDIRECTIONAL); @@ -651,7 +691,7 @@ static int ibmveth_close(struct net_device *netdev) napi_disable(&adapter->napi); if (!adapter->pool_config) - netif_stop_queue(netdev); + netif_tx_stop_all_queues(netdev); h_vio_signal(adapter->vdev->unit_address, VIO_IRQ_DISABLE); @@ -685,9 +725,8 @@ static int ibmveth_close(struct net_device *netdev) ibmveth_free_buffer_pool(adapter, &adapter->rx_buff_pool[i]); - dma_free_coherent(&adapter->vdev->dev, - adapter->netdev->mtu + IBMVETH_BUFF_OH, - adapter->bounce_buffer, adapter->bounce_buffer_dma); + for (i = 0; i < netdev->real_num_tx_queues; i++) + ibmveth_free_tx_ltb(adapter, i); netdev_dbg(netdev, "close complete\n"); @@ -727,8 +766,8 @@ static void ibmveth_init_link_settings(struct net_device *dev) static void netdev_get_drvinfo(struct net_device *dev, struct ethtool_drvinfo *info) { - strlcpy(info->driver, ibmveth_driver_name, sizeof(info->driver)); - strlcpy(info->version, ibmveth_driver_version, sizeof(info->version)); + strscpy(info->driver, ibmveth_driver_name, sizeof(info->driver)); + strscpy(info->version, ibmveth_driver_version, sizeof(info->version)); } static netdev_features_t ibmveth_fix_features(struct net_device *dev, @@ -953,6 +992,69 @@ static void ibmveth_get_ethtool_stats(struct net_device *dev, data[i] = IBMVETH_GET_STAT(adapter, ibmveth_stats[i].offset); } +static void ibmveth_get_channels(struct net_device *netdev, + struct ethtool_channels *channels) +{ + channels->max_tx = ibmveth_real_max_tx_queues(); + channels->tx_count = netdev->real_num_tx_queues; + + channels->max_rx = netdev->real_num_rx_queues; + channels->rx_count = netdev->real_num_rx_queues; +} + +static int ibmveth_set_channels(struct net_device *netdev, + struct ethtool_channels *channels) +{ + struct ibmveth_adapter *adapter = netdev_priv(netdev); + unsigned int old = netdev->real_num_tx_queues, + goal = channels->tx_count; + int rc, i; + + /* If ndo_open has not been called yet then don't allocate, just set + * desired netdev_queue's and return + */ + if (!(netdev->flags & IFF_UP)) + return netif_set_real_num_tx_queues(netdev, goal); + + /* We have IBMVETH_MAX_QUEUES netdev_queue's allocated + * but we may need to alloc/free the ltb's. + */ + netif_tx_stop_all_queues(netdev); + + /* Allocate any queue that we need */ + for (i = old; i < goal; i++) { + if (adapter->tx_ltb_ptr[i]) + continue; + + rc = ibmveth_allocate_tx_ltb(adapter, i); + if (!rc) + continue; + + /* if something goes wrong, free everything we just allocated */ + netdev_err(netdev, "Failed to allocate more tx queues, returning to %d queues\n", + old); + goal = old; + old = i; + break; + } + rc = netif_set_real_num_tx_queues(netdev, goal); + if (rc) { + netdev_err(netdev, "Failed to set real tx queues, returning to %d queues\n", + old); + goal = old; + old = i; + } + /* Free any that are no longer needed */ + for (i = old; i > goal; i--) { + if (adapter->tx_ltb_ptr[i - 1]) + ibmveth_free_tx_ltb(adapter, i - 1); + } + + netif_tx_wake_all_queues(netdev); + + return rc; +} + static const struct ethtool_ops netdev_ethtool_ops = { .get_drvinfo = netdev_get_drvinfo, .get_link = ethtool_op_get_link, @@ -961,6 +1063,8 @@ static const struct ethtool_ops netdev_ethtool_ops = { .get_ethtool_stats = ibmveth_get_ethtool_stats, .get_link_ksettings = ibmveth_get_link_ksettings, .set_link_ksettings = ibmveth_set_link_ksettings, + .get_channels = ibmveth_get_channels, + .set_channels = ibmveth_set_channels }; static int ibmveth_ioctl(struct net_device *dev, struct ifreq *ifr, int cmd) @@ -969,7 +1073,7 @@ static int ibmveth_ioctl(struct net_device *dev, struct ifreq *ifr, int cmd) } static int ibmveth_send(struct ibmveth_adapter *adapter, - union ibmveth_buf_desc *descs, unsigned long mss) + unsigned long desc, unsigned long mss) { unsigned long correlator; unsigned int retry_count; @@ -982,12 +1086,9 @@ static int ibmveth_send(struct ibmveth_adapter *adapter, retry_count = 1024; correlator = 0; do { - ret = h_send_logical_lan(adapter->vdev->unit_address, - descs[0].desc, descs[1].desc, - descs[2].desc, descs[3].desc, - descs[4].desc, descs[5].desc, - correlator, &correlator, mss, - adapter->fw_large_send_support); + ret = h_send_logical_lan(adapter->vdev->unit_address, desc, + correlator, &correlator, mss, + adapter->fw_large_send_support); } while ((ret == H_BUSY) && (retry_count--)); if (ret != H_SUCCESS && ret != H_DROPPED) { @@ -1020,34 +1121,13 @@ static netdev_tx_t ibmveth_start_xmit(struct sk_buff *skb, struct net_device *netdev) { struct ibmveth_adapter *adapter = netdev_priv(netdev); - unsigned int desc_flags; - union ibmveth_buf_desc descs[6]; - int last, i; - int force_bounce = 0; - dma_addr_t dma_addr; + unsigned int desc_flags, total_bytes; + union ibmveth_buf_desc desc; + int i, queue_num = skb_get_queue_mapping(skb); unsigned long mss = 0; if (ibmveth_is_packet_unsupported(skb, netdev)) goto out; - - /* veth doesn't handle frag_list, so linearize the skb. - * When GRO is enabled SKB's can have frag_list. - */ - if (adapter->is_active_trunk && - skb_has_frag_list(skb) && __skb_linearize(skb)) { - netdev->stats.tx_dropped++; - goto out; - } - - /* - * veth handles a maximum of 6 segments including the header, so - * we have to linearize the skb if there are more than this. - */ - if (skb_shinfo(skb)->nr_frags > 5 && __skb_linearize(skb)) { - netdev->stats.tx_dropped++; - goto out; - } - /* veth can't checksum offload UDP */ if (skb->ip_summed == CHECKSUM_PARTIAL && ((skb->protocol == htons(ETH_P_IP) && @@ -1077,56 +1157,6 @@ static netdev_tx_t ibmveth_start_xmit(struct sk_buff *skb, desc_flags |= IBMVETH_BUF_LRG_SND; } -retry_bounce: - memset(descs, 0, sizeof(descs)); - - /* - * If a linear packet is below the rx threshold then - * copy it into the static bounce buffer. This avoids the - * cost of a TCE insert and remove. - */ - if (force_bounce || (!skb_is_nonlinear(skb) && - (skb->len < tx_copybreak))) { - skb_copy_from_linear_data(skb, adapter->bounce_buffer, - skb->len); - - descs[0].fields.flags_len = desc_flags | skb->len; - descs[0].fields.address = adapter->bounce_buffer_dma; - - if (ibmveth_send(adapter, descs, 0)) { - adapter->tx_send_failed++; - netdev->stats.tx_dropped++; - } else { - netdev->stats.tx_packets++; - netdev->stats.tx_bytes += skb->len; - } - - goto out; - } - - /* Map the header */ - dma_addr = dma_map_single(&adapter->vdev->dev, skb->data, - skb_headlen(skb), DMA_TO_DEVICE); - if (dma_mapping_error(&adapter->vdev->dev, dma_addr)) - goto map_failed; - - descs[0].fields.flags_len = desc_flags | skb_headlen(skb); - descs[0].fields.address = dma_addr; - - /* Map the frags */ - for (i = 0; i < skb_shinfo(skb)->nr_frags; i++) { - const skb_frag_t *frag = &skb_shinfo(skb)->frags[i]; - - dma_addr = skb_frag_dma_map(&adapter->vdev->dev, frag, 0, - skb_frag_size(frag), DMA_TO_DEVICE); - - if (dma_mapping_error(&adapter->vdev->dev, dma_addr)) - goto map_failed_frags; - - descs[i+1].fields.flags_len = desc_flags | skb_frag_size(frag); - descs[i+1].fields.address = dma_addr; - } - if (skb->ip_summed == CHECKSUM_PARTIAL && skb_is_gso(skb)) { if (adapter->fw_large_send_support) { mss = (unsigned long)skb_shinfo(skb)->gso_size; @@ -1143,7 +1173,36 @@ retry_bounce: } } - if (ibmveth_send(adapter, descs, mss)) { + /* Copy header into mapped buffer */ + if (unlikely(skb->len > adapter->tx_ltb_size)) { + netdev_err(adapter->netdev, "tx: packet size (%u) exceeds ltb (%u)\n", + skb->len, adapter->tx_ltb_size); + netdev->stats.tx_dropped++; + goto out; + } + memcpy(adapter->tx_ltb_ptr[queue_num], skb->data, skb_headlen(skb)); + total_bytes = skb_headlen(skb); + /* Copy frags into mapped buffers */ + for (i = 0; i < skb_shinfo(skb)->nr_frags; i++) { + const skb_frag_t *frag = &skb_shinfo(skb)->frags[i]; + + memcpy(adapter->tx_ltb_ptr[queue_num] + total_bytes, + skb_frag_address_safe(frag), skb_frag_size(frag)); + total_bytes += skb_frag_size(frag); + } + + if (unlikely(total_bytes != skb->len)) { + netdev_err(adapter->netdev, "tx: incorrect packet len copied into ltb (%u != %u)\n", + skb->len, total_bytes); + netdev->stats.tx_dropped++; + goto out; + } + desc.fields.flags_len = desc_flags | skb->len; + desc.fields.address = adapter->tx_ltb_dma[queue_num]; + /* finish writing to long_term_buff before VIOS accessing it */ + dma_wmb(); + + if (ibmveth_send(adapter, desc.desc, mss)) { adapter->tx_send_failed++; netdev->stats.tx_dropped++; } else { @@ -1151,41 +1210,11 @@ retry_bounce: netdev->stats.tx_bytes += skb->len; } - dma_unmap_single(&adapter->vdev->dev, - descs[0].fields.address, - descs[0].fields.flags_len & IBMVETH_BUF_LEN_MASK, - DMA_TO_DEVICE); - - for (i = 1; i < skb_shinfo(skb)->nr_frags + 1; i++) - dma_unmap_page(&adapter->vdev->dev, descs[i].fields.address, - descs[i].fields.flags_len & IBMVETH_BUF_LEN_MASK, - DMA_TO_DEVICE); - out: dev_consume_skb_any(skb); return NETDEV_TX_OK; -map_failed_frags: - last = i+1; - for (i = 1; i < last; i++) - dma_unmap_page(&adapter->vdev->dev, descs[i].fields.address, - descs[i].fields.flags_len & IBMVETH_BUF_LEN_MASK, - DMA_TO_DEVICE); - dma_unmap_single(&adapter->vdev->dev, - descs[0].fields.address, - descs[0].fields.flags_len & IBMVETH_BUF_LEN_MASK, - DMA_TO_DEVICE); -map_failed: - if (!firmware_has_feature(FW_FEATURE_CMO)) - netdev_err(netdev, "tx: unable to map xmit buffer\n"); - adapter->tx_map_failed++; - if (skb_linearize(skb)) { - netdev->stats.tx_dropped++; - goto out; - } - force_bounce = 1; - goto retry_bounce; } static void ibmveth_rx_mss_helper(struct sk_buff *skb, u16 mss, int lrg_pkt) @@ -1568,6 +1597,8 @@ static unsigned long ibmveth_get_desired_dma(struct vio_dev *vdev) ret = IBMVETH_BUFF_LIST_SIZE + IBMVETH_FILT_LIST_SIZE; ret += IOMMU_PAGE_ALIGN(netdev->mtu, tbl); + /* add size of mapped tx buffers */ + ret += IOMMU_PAGE_ALIGN(IBMVETH_MAX_TX_BUF_SIZE, tbl); for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) { /* add the size of the active receive buffers */ @@ -1660,8 +1691,7 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id) return -EINVAL; } - netdev = alloc_etherdev(sizeof(struct ibmveth_adapter)); - + netdev = alloc_etherdev_mqs(sizeof(struct ibmveth_adapter), IBMVETH_MAX_QUEUES, 1); if (!netdev) return -ENOMEM; @@ -1727,6 +1757,17 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id) kobject_uevent(kobj, KOBJ_ADD); } + rc = netif_set_real_num_tx_queues(netdev, ibmveth_real_max_tx_queues()); + if (rc) { + netdev_dbg(netdev, "failed to set number of tx queues rc=%d\n", + rc); + free_netdev(netdev); + return rc; + } + adapter->tx_ltb_size = PAGE_ALIGN(IBMVETH_MAX_TX_BUF_SIZE); + for (i = 0; i < IBMVETH_MAX_QUEUES; i++) + adapter->tx_ltb_ptr[i] = NULL; + netdev_dbg(netdev, "adapter @ 0x%p\n", adapter); netdev_dbg(netdev, "registering netdev...\n"); diff --git a/drivers/net/ethernet/ibm/ibmveth.h b/drivers/net/ethernet/ibm/ibmveth.h index 27dfff200166..daf6f615c03f 100644 --- a/drivers/net/ethernet/ibm/ibmveth.h +++ b/drivers/net/ethernet/ibm/ibmveth.h @@ -46,23 +46,23 @@ #define h_add_logical_lan_buffer(ua, buf) \ plpar_hcall_norets(H_ADD_LOGICAL_LAN_BUFFER, ua, buf) +/* FW allows us to send 6 descriptors but we only use one so mark + * the other 5 as unused (0) + */ static inline long h_send_logical_lan(unsigned long unit_address, - unsigned long desc1, unsigned long desc2, unsigned long desc3, - unsigned long desc4, unsigned long desc5, unsigned long desc6, - unsigned long corellator_in, unsigned long *corellator_out, - unsigned long mss, unsigned long large_send_support) + unsigned long desc, unsigned long corellator_in, + unsigned long *corellator_out, unsigned long mss, + unsigned long large_send_support) { long rc; unsigned long retbuf[PLPAR_HCALL9_BUFSIZE]; if (large_send_support) rc = plpar_hcall9(H_SEND_LOGICAL_LAN, retbuf, unit_address, - desc1, desc2, desc3, desc4, desc5, desc6, - corellator_in, mss); + desc, 0, 0, 0, 0, 0, corellator_in, mss); else rc = plpar_hcall9(H_SEND_LOGICAL_LAN, retbuf, unit_address, - desc1, desc2, desc3, desc4, desc5, desc6, - corellator_in); + desc, 0, 0, 0, 0, 0, corellator_in); *corellator_out = retbuf[0]; @@ -98,6 +98,8 @@ static inline long h_illan_attributes(unsigned long unit_address, #define IBMVETH_BUFF_LIST_SIZE 4096 #define IBMVETH_FILT_LIST_SIZE 4096 #define IBMVETH_MAX_BUF_SIZE (1024 * 128) +#define IBMVETH_MAX_TX_BUF_SIZE (1024 * 64) +#define IBMVETH_MAX_QUEUES 16U static int pool_size[] = { 512, 1024 * 2, 1024 * 16, 1024 * 32, 1024 * 64 }; static int pool_count[] = { 256, 512, 256, 256, 256 }; @@ -137,6 +139,9 @@ struct ibmveth_adapter { unsigned int mcastFilterSize; void * buffer_list_addr; void * filter_list_addr; + void *tx_ltb_ptr[IBMVETH_MAX_QUEUES]; + unsigned int tx_ltb_size; + dma_addr_t tx_ltb_dma[IBMVETH_MAX_QUEUES]; dma_addr_t buffer_list_dma; dma_addr_t filter_list_dma; struct ibmveth_buff_pool rx_buff_pool[IBMVETH_NUM_BUFF_POOLS]; @@ -145,8 +150,6 @@ struct ibmveth_adapter { int rx_csum; int large_send; bool is_active_trunk; - void *bounce_buffer; - dma_addr_t bounce_buffer_dma; u64 fw_ipv6_csum_support; u64 fw_ipv4_csum_support; diff --git a/drivers/net/ethernet/ibm/ibmvnic.c b/drivers/net/ethernet/ibm/ibmvnic.c index 5ab7c0f81e9a..65dbfbec487a 100644 --- a/drivers/net/ethernet/ibm/ibmvnic.c +++ b/drivers/net/ethernet/ibm/ibmvnic.c @@ -1262,7 +1262,7 @@ static int init_napi(struct ibmvnic_adapter *adapter) for (i = 0; i < adapter->req_rx_queues; i++) { netdev_dbg(adapter->netdev, "Adding napi[%d]\n", i); netif_napi_add(adapter->netdev, &adapter->napi[i], - ibmvnic_poll, NAPI_POLL_WEIGHT); + ibmvnic_poll); } adapter->num_active_rx_napi = adapter->req_rx_queues; diff --git a/drivers/net/ethernet/intel/e100.c b/drivers/net/ethernet/intel/e100.c index 11a884aa5082..560d1d442232 100644 --- a/drivers/net/ethernet/intel/e100.c +++ b/drivers/net/ethernet/intel/e100.c @@ -2431,8 +2431,8 @@ static void e100_get_drvinfo(struct net_device *netdev, struct ethtool_drvinfo *info) { struct nic *nic = netdev_priv(netdev); - strlcpy(info->driver, DRV_NAME, sizeof(info->driver)); - strlcpy(info->bus_info, pci_name(nic->pdev), + strscpy(info->driver, DRV_NAME, sizeof(info->driver)); + strscpy(info->bus_info, pci_name(nic->pdev), sizeof(info->bus_info)); } diff --git a/drivers/net/ethernet/intel/e1000/e1000_ethtool.c b/drivers/net/ethernet/intel/e1000/e1000_ethtool.c index 32803b0cf1e8..d06d29c6c037 100644 --- a/drivers/net/ethernet/intel/e1000/e1000_ethtool.c +++ b/drivers/net/ethernet/intel/e1000/e1000_ethtool.c @@ -531,10 +531,10 @@ static void e1000_get_drvinfo(struct net_device *netdev, { struct e1000_adapter *adapter = netdev_priv(netdev); - strlcpy(drvinfo->driver, e1000_driver_name, + strscpy(drvinfo->driver, e1000_driver_name, sizeof(drvinfo->driver)); - strlcpy(drvinfo->bus_info, pci_name(adapter->pdev), + strscpy(drvinfo->bus_info, pci_name(adapter->pdev), sizeof(drvinfo->bus_info)); } diff --git a/drivers/net/ethernet/intel/e1000/e1000_main.c b/drivers/net/ethernet/intel/e1000/e1000_main.c index 23299fc56199..61e60e4de600 100644 --- a/drivers/net/ethernet/intel/e1000/e1000_main.c +++ b/drivers/net/ethernet/intel/e1000/e1000_main.c @@ -1012,7 +1012,7 @@ static int e1000_probe(struct pci_dev *pdev, const struct pci_device_id *ent) netdev->netdev_ops = &e1000_netdev_ops; e1000_set_ethtool_ops(netdev); netdev->watchdog_timeo = 5 * HZ; - netif_napi_add(netdev, &adapter->napi, e1000_clean, 64); + netif_napi_add(netdev, &adapter->napi, e1000_clean); strncpy(netdev->name, pci_name(pdev), sizeof(netdev->name) - 1); diff --git a/drivers/net/ethernet/intel/e1000e/ethtool.c b/drivers/net/ethernet/intel/e1000e/ethtool.c index b80ae9a82224..51a5afe9df2f 100644 --- a/drivers/net/ethernet/intel/e1000e/ethtool.c +++ b/drivers/net/ethernet/intel/e1000e/ethtool.c @@ -639,7 +639,7 @@ static void e1000_get_drvinfo(struct net_device *netdev, { struct e1000_adapter *adapter = netdev_priv(netdev); - strlcpy(drvinfo->driver, e1000e_driver_name, sizeof(drvinfo->driver)); + strscpy(drvinfo->driver, e1000e_driver_name, sizeof(drvinfo->driver)); /* EEPROM image version # is reported as firmware version # for * PCI-E controllers @@ -650,7 +650,7 @@ static void e1000_get_drvinfo(struct net_device *netdev, (adapter->eeprom_vers & 0x0FF0) >> 4, (adapter->eeprom_vers & 0x000F)); - strlcpy(drvinfo->bus_info, pci_name(adapter->pdev), + strscpy(drvinfo->bus_info, pci_name(adapter->pdev), sizeof(drvinfo->bus_info)); } diff --git a/drivers/net/ethernet/intel/e1000e/netdev.c b/drivers/net/ethernet/intel/e1000e/netdev.c index 321f2a95ae3a..49e926959ad3 100644 --- a/drivers/net/ethernet/intel/e1000e/netdev.c +++ b/drivers/net/ethernet/intel/e1000e/netdev.c @@ -7267,7 +7267,7 @@ static void e1000_print_device_info(struct e1000_adapter *adapter) ret_val = e1000_read_pba_string_generic(hw, pba_str, E1000_PBANUM_LENGTH); if (ret_val) - strlcpy((char *)pba_str, "Unknown", sizeof(pba_str)); + strscpy((char *)pba_str, "Unknown", sizeof(pba_str)); e_info("MAC: %d, PHY: %d, PBA No: %s\n", hw->mac.type, hw->phy.type, pba_str); } @@ -7479,8 +7479,8 @@ static int e1000_probe(struct pci_dev *pdev, const struct pci_device_id *ent) netdev->netdev_ops = &e1000e_netdev_ops; e1000e_set_ethtool_ops(netdev); netdev->watchdog_timeo = 5 * HZ; - netif_napi_add(netdev, &adapter->napi, e1000e_poll, 64); - strlcpy(netdev->name, pci_name(pdev), sizeof(netdev->name)); + netif_napi_add(netdev, &adapter->napi, e1000e_poll); + strscpy(netdev->name, pci_name(pdev), sizeof(netdev->name)); netdev->mem_start = mmio_start; netdev->mem_end = mmio_start + mmio_len; @@ -7676,7 +7676,7 @@ static int e1000_probe(struct pci_dev *pdev, const struct pci_device_id *ent) if (hw->mac.type >= e1000_pch_cnp) adapter->flags2 |= FLAG2_ENABLE_S0IX_FLOWS; - strlcpy(netdev->name, "eth%d", sizeof(netdev->name)); + strscpy(netdev->name, "eth%d", sizeof(netdev->name)); err = register_netdev(netdev); if (err) goto err_register; diff --git a/drivers/net/ethernet/intel/e1000e/phy.c b/drivers/net/ethernet/intel/e1000e/phy.c index fd07c3679bb1..060b263348ce 100644 --- a/drivers/net/ethernet/intel/e1000e/phy.c +++ b/drivers/net/ethernet/intel/e1000e/phy.c @@ -2697,9 +2697,14 @@ static s32 e1000_access_phy_wakeup_reg_bm(struct e1000_hw *hw, u32 offset, void e1000_power_up_phy_copper(struct e1000_hw *hw) { u16 mii_reg = 0; + int ret; /* The PHY will retain its settings across a power down/up cycle */ - e1e_rphy(hw, MII_BMCR, &mii_reg); + ret = e1e_rphy(hw, MII_BMCR, &mii_reg); + if (ret) { + e_dbg("Error reading PHY register\n"); + return; + } mii_reg &= ~BMCR_PDOWN; e1e_wphy(hw, MII_BMCR, mii_reg); } @@ -2715,9 +2720,14 @@ void e1000_power_up_phy_copper(struct e1000_hw *hw) void e1000_power_down_phy_copper(struct e1000_hw *hw) { u16 mii_reg = 0; + int ret; /* The PHY will retain its settings across a power down/up cycle */ - e1e_rphy(hw, MII_BMCR, &mii_reg); + ret = e1e_rphy(hw, MII_BMCR, &mii_reg); + if (ret) { + e_dbg("Error reading PHY register\n"); + return; + } mii_reg |= BMCR_PDOWN; e1e_wphy(hw, MII_BMCR, mii_reg); usleep_range(1000, 2000); @@ -3037,7 +3047,11 @@ s32 e1000_link_stall_workaround_hv(struct e1000_hw *hw) return 0; /* Do not apply workaround if in PHY loopback bit 14 set */ - e1e_rphy(hw, MII_BMCR, &data); + ret_val = e1e_rphy(hw, MII_BMCR, &data); + if (ret_val) { + e_dbg("Error reading PHY register\n"); + return ret_val; + } if (data & BMCR_LOOPBACK) return 0; diff --git a/drivers/net/ethernet/intel/fm10k/fm10k_main.c b/drivers/net/ethernet/intel/fm10k/fm10k_main.c index 3362f26d7f99..4a6630586ec9 100644 --- a/drivers/net/ethernet/intel/fm10k/fm10k_main.c +++ b/drivers/net/ethernet/intel/fm10k/fm10k_main.c @@ -1595,8 +1595,7 @@ static int fm10k_alloc_q_vector(struct fm10k_intfc *interface, return -ENOMEM; /* initialize NAPI */ - netif_napi_add(interface->netdev, &q_vector->napi, - fm10k_poll, NAPI_POLL_WEIGHT); + netif_napi_add(interface->netdev, &q_vector->napi, fm10k_poll); /* tie q_vector and interface together */ interface->q_vector[v_idx] = q_vector; diff --git a/drivers/net/ethernet/intel/i40e/i40e.h b/drivers/net/ethernet/intel/i40e/i40e.h index d86b6d349ea9..9a60d6b207f7 100644 --- a/drivers/net/ethernet/intel/i40e/i40e.h +++ b/drivers/net/ethernet/intel/i40e/i40e.h @@ -399,6 +399,20 @@ struct i40e_ddp_old_profile_list { I40E_FLEX_54_MASK | I40E_FLEX_55_MASK | \ I40E_FLEX_56_MASK | I40E_FLEX_57_MASK) +#define I40E_QINT_TQCTL_VAL(qp, vector, nextq_type) \ + (I40E_QINT_TQCTL_CAUSE_ENA_MASK | \ + (I40E_TX_ITR << I40E_QINT_TQCTL_ITR_INDX_SHIFT) | \ + ((vector) << I40E_QINT_TQCTL_MSIX_INDX_SHIFT) | \ + ((qp) << I40E_QINT_TQCTL_NEXTQ_INDX_SHIFT) | \ + (I40E_QUEUE_TYPE_##nextq_type << I40E_QINT_TQCTL_NEXTQ_TYPE_SHIFT)) + +#define I40E_QINT_RQCTL_VAL(qp, vector, nextq_type) \ + (I40E_QINT_RQCTL_CAUSE_ENA_MASK | \ + (I40E_RX_ITR << I40E_QINT_RQCTL_ITR_INDX_SHIFT) | \ + ((vector) << I40E_QINT_RQCTL_MSIX_INDX_SHIFT) | \ + ((qp) << I40E_QINT_RQCTL_NEXTQ_INDX_SHIFT) | \ + (I40E_QUEUE_TYPE_##nextq_type << I40E_QINT_RQCTL_NEXTQ_TYPE_SHIFT)) + struct i40e_flex_pit { struct list_head list; u16 src_offset; diff --git a/drivers/net/ethernet/intel/i40e/i40e_common.c b/drivers/net/ethernet/intel/i40e/i40e_common.c index 2819e261a126..4f01e2a6b6bb 100644 --- a/drivers/net/ethernet/intel/i40e/i40e_common.c +++ b/drivers/net/ethernet/intel/i40e/i40e_common.c @@ -27,6 +27,7 @@ i40e_status i40e_set_mac_type(struct i40e_hw *hw) case I40E_DEV_ID_QSFP_A: case I40E_DEV_ID_QSFP_B: case I40E_DEV_ID_QSFP_C: + case I40E_DEV_ID_1G_BASE_T_BC: case I40E_DEV_ID_5G_BASE_T_BC: case I40E_DEV_ID_10G_BASE_T: case I40E_DEV_ID_10G_BASE_T4: @@ -4974,6 +4975,7 @@ i40e_status i40e_write_phy_register(struct i40e_hw *hw, status = i40e_write_phy_register_clause22(hw, reg, phy_addr, value); break; + case I40E_DEV_ID_1G_BASE_T_BC: case I40E_DEV_ID_5G_BASE_T_BC: case I40E_DEV_ID_10G_BASE_T: case I40E_DEV_ID_10G_BASE_T4: @@ -5012,6 +5014,7 @@ i40e_status i40e_read_phy_register(struct i40e_hw *hw, status = i40e_read_phy_register_clause22(hw, reg, phy_addr, value); break; + case I40E_DEV_ID_1G_BASE_T_BC: case I40E_DEV_ID_5G_BASE_T_BC: case I40E_DEV_ID_10G_BASE_T: case I40E_DEV_ID_10G_BASE_T4: diff --git a/drivers/net/ethernet/intel/i40e/i40e_devids.h b/drivers/net/ethernet/intel/i40e/i40e_devids.h index 2610338002fe..d9c51a238dcc 100644 --- a/drivers/net/ethernet/intel/i40e/i40e_devids.h +++ b/drivers/net/ethernet/intel/i40e/i40e_devids.h @@ -24,8 +24,10 @@ #define I40E_DEV_ID_10G_B 0x104F #define I40E_DEV_ID_10G_SFP 0x104E #define I40E_DEV_ID_5G_BASE_T_BC 0x101F +#define I40E_DEV_ID_1G_BASE_T_BC 0x0DD2 #define I40E_IS_X710TL_DEVICE(d) \ - (((d) == I40E_DEV_ID_5G_BASE_T_BC) || \ + (((d) == I40E_DEV_ID_1G_BASE_T_BC) || \ + ((d) == I40E_DEV_ID_5G_BASE_T_BC) || \ ((d) == I40E_DEV_ID_10G_BASE_T_BC)) #define I40E_DEV_ID_KX_X722 0x37CE #define I40E_DEV_ID_QSFP_X722 0x37CF diff --git a/drivers/net/ethernet/intel/i40e/i40e_ethtool.c b/drivers/net/ethernet/intel/i40e/i40e_ethtool.c index e9cd0fa6a0d2..7e75706f76db 100644 --- a/drivers/net/ethernet/intel/i40e/i40e_ethtool.c +++ b/drivers/net/ethernet/intel/i40e/i40e_ethtool.c @@ -2001,10 +2001,10 @@ static void i40e_get_drvinfo(struct net_device *netdev, struct i40e_vsi *vsi = np->vsi; struct i40e_pf *pf = vsi->back; - strlcpy(drvinfo->driver, i40e_driver_name, sizeof(drvinfo->driver)); - strlcpy(drvinfo->fw_version, i40e_nvm_version_str(&pf->hw), + strscpy(drvinfo->driver, i40e_driver_name, sizeof(drvinfo->driver)); + strscpy(drvinfo->fw_version, i40e_nvm_version_str(&pf->hw), sizeof(drvinfo->fw_version)); - strlcpy(drvinfo->bus_info, pci_name(pf->pdev), + strscpy(drvinfo->bus_info, pci_name(pf->pdev), sizeof(drvinfo->bus_info)); drvinfo->n_priv_flags = I40E_PRIV_FLAGS_STR_LEN; if (pf->hw.pf_id == 0) diff --git a/drivers/net/ethernet/intel/i40e/i40e_main.c b/drivers/net/ethernet/intel/i40e/i40e_main.c index e3d9804aeb25..2c07fa8ecfc8 100644 --- a/drivers/net/ethernet/intel/i40e/i40e_main.c +++ b/drivers/net/ethernet/intel/i40e/i40e_main.c @@ -66,6 +66,7 @@ static const struct pci_device_id i40e_pci_tbl[] = { {PCI_VDEVICE(INTEL, I40E_DEV_ID_QSFP_A), 0}, {PCI_VDEVICE(INTEL, I40E_DEV_ID_QSFP_B), 0}, {PCI_VDEVICE(INTEL, I40E_DEV_ID_QSFP_C), 0}, + {PCI_VDEVICE(INTEL, I40E_DEV_ID_1G_BASE_T_BC), 0}, {PCI_VDEVICE(INTEL, I40E_DEV_ID_10G_BASE_T), 0}, {PCI_VDEVICE(INTEL, I40E_DEV_ID_10G_BASE_T4), 0}, {PCI_VDEVICE(INTEL, I40E_DEV_ID_10G_BASE_T_BC), 0}, @@ -3878,7 +3879,7 @@ static void i40e_vsi_configure_msix(struct i40e_vsi *vsi) wr32(hw, I40E_PFINT_RATEN(vector - 1), i40e_intrl_usec_to_reg(vsi->int_rate_limit)); - /* Linked list for the queuepairs assigned to this vector */ + /* begin of linked list for RX queue assigned to this vector */ wr32(hw, I40E_PFINT_LNKLSTN(vector - 1), qp); for (q = 0; q < q_vector->num_ringpairs; q++) { u32 nextqp = has_xdp ? qp + vsi->alloc_queue_pairs : qp; @@ -3894,6 +3895,7 @@ static void i40e_vsi_configure_msix(struct i40e_vsi *vsi) wr32(hw, I40E_QINT_RQCTL(qp), val); if (has_xdp) { + /* TX queue with next queue set to TX */ val = I40E_QINT_TQCTL_CAUSE_ENA_MASK | (I40E_TX_ITR << I40E_QINT_TQCTL_ITR_INDX_SHIFT) | (vector << I40E_QINT_TQCTL_MSIX_INDX_SHIFT) | @@ -3903,7 +3905,7 @@ static void i40e_vsi_configure_msix(struct i40e_vsi *vsi) wr32(hw, I40E_QINT_TQCTL(nextqp), val); } - + /* TX queue with next RX or end of linked list */ val = I40E_QINT_TQCTL_CAUSE_ENA_MASK | (I40E_TX_ITR << I40E_QINT_TQCTL_ITR_INDX_SHIFT) | (vector << I40E_QINT_TQCTL_MSIX_INDX_SHIFT) | @@ -3972,7 +3974,6 @@ static void i40e_configure_msi_and_legacy(struct i40e_vsi *vsi) struct i40e_q_vector *q_vector = vsi->q_vectors[0]; struct i40e_pf *pf = vsi->back; struct i40e_hw *hw = &pf->hw; - u32 val; /* set the ITR configuration */ q_vector->rx.next_update = jiffies + 1; @@ -3989,28 +3990,20 @@ static void i40e_configure_msi_and_legacy(struct i40e_vsi *vsi) /* FIRSTQ_INDX = 0, FIRSTQ_TYPE = 0 (rx) */ wr32(hw, I40E_PFINT_LNKLST0, 0); - /* Associate the queue pair to the vector and enable the queue int */ - val = I40E_QINT_RQCTL_CAUSE_ENA_MASK | - (I40E_RX_ITR << I40E_QINT_RQCTL_ITR_INDX_SHIFT) | - (nextqp << I40E_QINT_RQCTL_NEXTQ_INDX_SHIFT)| - (I40E_QUEUE_TYPE_TX << I40E_QINT_TQCTL_NEXTQ_TYPE_SHIFT); - - wr32(hw, I40E_QINT_RQCTL(0), val); + /* Associate the queue pair to the vector and enable the queue + * interrupt RX queue in linked list with next queue set to TX + */ + wr32(hw, I40E_QINT_RQCTL(0), I40E_QINT_RQCTL_VAL(nextqp, 0, TX)); if (i40e_enabled_xdp_vsi(vsi)) { - val = I40E_QINT_TQCTL_CAUSE_ENA_MASK | - (I40E_TX_ITR << I40E_QINT_TQCTL_ITR_INDX_SHIFT)| - (I40E_QUEUE_TYPE_TX - << I40E_QINT_TQCTL_NEXTQ_TYPE_SHIFT); - - wr32(hw, I40E_QINT_TQCTL(nextqp), val); + /* TX queue in linked list with next queue set to TX */ + wr32(hw, I40E_QINT_TQCTL(nextqp), + I40E_QINT_TQCTL_VAL(nextqp, 0, TX)); } - val = I40E_QINT_TQCTL_CAUSE_ENA_MASK | - (I40E_TX_ITR << I40E_QINT_TQCTL_ITR_INDX_SHIFT) | - (I40E_QUEUE_END_OF_LIST << I40E_QINT_TQCTL_NEXTQ_INDX_SHIFT); - - wr32(hw, I40E_QINT_TQCTL(0), val); + /* last TX queue so the next RX queue doesn't matter */ + wr32(hw, I40E_QINT_TQCTL(0), + I40E_QINT_TQCTL_VAL(I40E_QUEUE_END_OF_LIST, 0, RX)); i40e_flush(hw); } @@ -10724,7 +10717,7 @@ static void i40e_send_version(struct i40e_pf *pf) dv.minor_version = 0xff; dv.build_version = 0xff; dv.subbuild_version = 0; - strlcpy(dv.driver_string, UTS_RELEASE, sizeof(dv.driver_string)); + strscpy(dv.driver_string, UTS_RELEASE, sizeof(dv.driver_string)); i40e_aq_send_driver_version(&pf->hw, &dv, NULL); } @@ -11948,8 +11941,7 @@ static int i40e_vsi_alloc_q_vector(struct i40e_vsi *vsi, int v_idx) cpumask_copy(&q_vector->affinity_mask, cpu_possible_mask); if (vsi->netdev) - netif_napi_add(vsi->netdev, &q_vector->napi, - i40e_napi_poll, NAPI_POLL_WEIGHT); + netif_napi_add(vsi->netdev, &q_vector->napi, i40e_napi_poll); /* tie q_vector and vsi together */ vsi->q_vectors[v_idx] = q_vector; @@ -16072,23 +16064,23 @@ static int i40e_probe(struct pci_dev *pdev, const struct pci_device_id *ent) switch (hw->bus.speed) { case i40e_bus_speed_8000: - strlcpy(speed, "8.0", PCI_SPEED_SIZE); break; + strscpy(speed, "8.0", PCI_SPEED_SIZE); break; case i40e_bus_speed_5000: - strlcpy(speed, "5.0", PCI_SPEED_SIZE); break; + strscpy(speed, "5.0", PCI_SPEED_SIZE); break; case i40e_bus_speed_2500: - strlcpy(speed, "2.5", PCI_SPEED_SIZE); break; + strscpy(speed, "2.5", PCI_SPEED_SIZE); break; default: break; } switch (hw->bus.width) { case i40e_bus_width_pcie_x8: - strlcpy(width, "8", PCI_WIDTH_SIZE); break; + strscpy(width, "8", PCI_WIDTH_SIZE); break; case i40e_bus_width_pcie_x4: - strlcpy(width, "4", PCI_WIDTH_SIZE); break; + strscpy(width, "4", PCI_WIDTH_SIZE); break; case i40e_bus_width_pcie_x2: - strlcpy(width, "2", PCI_WIDTH_SIZE); break; + strscpy(width, "2", PCI_WIDTH_SIZE); break; case i40e_bus_width_pcie_x1: - strlcpy(width, "1", PCI_WIDTH_SIZE); break; + strscpy(width, "1", PCI_WIDTH_SIZE); break; default: break; } diff --git a/drivers/net/ethernet/intel/i40e/i40e_ptp.c b/drivers/net/ethernet/intel/i40e/i40e_ptp.c index 2d3533f38d7b..ffea0c9c82f1 100644 --- a/drivers/net/ethernet/intel/i40e/i40e_ptp.c +++ b/drivers/net/ethernet/intel/i40e/i40e_ptp.c @@ -1390,7 +1390,7 @@ static long i40e_ptp_create_clock(struct i40e_pf *pf) if (!IS_ERR_OR_NULL(pf->ptp_clock)) return 0; - strlcpy(pf->ptp_caps.name, i40e_driver_name, + strscpy(pf->ptp_caps.name, i40e_driver_name, sizeof(pf->ptp_caps.name) - 1); pf->ptp_caps.owner = THIS_MODULE; pf->ptp_caps.max_adj = 999999999; diff --git a/drivers/net/ethernet/intel/iavf/iavf_ethtool.c b/drivers/net/ethernet/intel/iavf/iavf_ethtool.c index e535d4c3da49..a056e1545615 100644 --- a/drivers/net/ethernet/intel/iavf/iavf_ethtool.c +++ b/drivers/net/ethernet/intel/iavf/iavf_ethtool.c @@ -581,9 +581,9 @@ static void iavf_get_drvinfo(struct net_device *netdev, { struct iavf_adapter *adapter = netdev_priv(netdev); - strlcpy(drvinfo->driver, iavf_driver_name, 32); - strlcpy(drvinfo->fw_version, "N/A", 4); - strlcpy(drvinfo->bus_info, pci_name(adapter->pdev), 32); + strscpy(drvinfo->driver, iavf_driver_name, 32); + strscpy(drvinfo->fw_version, "N/A", 4); + strscpy(drvinfo->bus_info, pci_name(adapter->pdev), 32); drvinfo->n_priv_flags = IAVF_PRIV_FLAGS_STR_LEN; } diff --git a/drivers/net/ethernet/intel/iavf/iavf_main.c b/drivers/net/ethernet/intel/iavf/iavf_main.c index 0c89f16bf1e2..3fc572341781 100644 --- a/drivers/net/ethernet/intel/iavf/iavf_main.c +++ b/drivers/net/ethernet/intel/iavf/iavf_main.c @@ -1267,66 +1267,138 @@ static void iavf_up_complete(struct iavf_adapter *adapter) } /** - * iavf_down - Shutdown the connection processing + * iavf_clear_mac_vlan_filters - Remove mac and vlan filters not sent to PF + * yet and mark other to be removed. * @adapter: board private structure - * - * Expects to be called while holding the __IAVF_IN_CRITICAL_TASK bit lock. **/ -void iavf_down(struct iavf_adapter *adapter) +static void iavf_clear_mac_vlan_filters(struct iavf_adapter *adapter) { - struct net_device *netdev = adapter->netdev; - struct iavf_vlan_filter *vlf; - struct iavf_cloud_filter *cf; - struct iavf_fdir_fltr *fdir; - struct iavf_mac_filter *f; - struct iavf_adv_rss *rss; - - if (adapter->state <= __IAVF_DOWN_PENDING) - return; - - netif_carrier_off(netdev); - netif_tx_disable(netdev); - adapter->link_up = false; - iavf_napi_disable_all(adapter); - iavf_irq_disable(adapter); + struct iavf_vlan_filter *vlf, *vlftmp; + struct iavf_mac_filter *f, *ftmp; spin_lock_bh(&adapter->mac_vlan_list_lock); - /* clear the sync flag on all filters */ __dev_uc_unsync(adapter->netdev, NULL); __dev_mc_unsync(adapter->netdev, NULL); /* remove all MAC filters */ - list_for_each_entry(f, &adapter->mac_filter_list, list) { - f->remove = true; + list_for_each_entry_safe(f, ftmp, &adapter->mac_filter_list, + list) { + if (f->add) { + list_del(&f->list); + kfree(f); + } else { + f->remove = true; + } } /* remove all VLAN filters */ - list_for_each_entry(vlf, &adapter->vlan_filter_list, list) { - vlf->remove = true; + list_for_each_entry_safe(vlf, vlftmp, &adapter->vlan_filter_list, + list) { + if (vlf->add) { + list_del(&vlf->list); + kfree(vlf); + } else { + vlf->remove = true; + } } - spin_unlock_bh(&adapter->mac_vlan_list_lock); +} + +/** + * iavf_clear_cloud_filters - Remove cloud filters not sent to PF yet and + * mark other to be removed. + * @adapter: board private structure + **/ +static void iavf_clear_cloud_filters(struct iavf_adapter *adapter) +{ + struct iavf_cloud_filter *cf, *cftmp; /* remove all cloud filters */ spin_lock_bh(&adapter->cloud_filter_list_lock); - list_for_each_entry(cf, &adapter->cloud_filter_list, list) { - cf->del = true; + list_for_each_entry_safe(cf, cftmp, &adapter->cloud_filter_list, + list) { + if (cf->add) { + list_del(&cf->list); + kfree(cf); + adapter->num_cloud_filters--; + } else { + cf->del = true; + } } spin_unlock_bh(&adapter->cloud_filter_list_lock); +} + +/** + * iavf_clear_fdir_filters - Remove fdir filters not sent to PF yet and mark + * other to be removed. + * @adapter: board private structure + **/ +static void iavf_clear_fdir_filters(struct iavf_adapter *adapter) +{ + struct iavf_fdir_fltr *fdir, *fdirtmp; /* remove all Flow Director filters */ spin_lock_bh(&adapter->fdir_fltr_lock); - list_for_each_entry(fdir, &adapter->fdir_list_head, list) { - fdir->state = IAVF_FDIR_FLTR_DEL_REQUEST; + list_for_each_entry_safe(fdir, fdirtmp, &adapter->fdir_list_head, + list) { + if (fdir->state == IAVF_FDIR_FLTR_ADD_REQUEST) { + list_del(&fdir->list); + kfree(fdir); + adapter->fdir_active_fltr--; + } else { + fdir->state = IAVF_FDIR_FLTR_DEL_REQUEST; + } } spin_unlock_bh(&adapter->fdir_fltr_lock); +} + +/** + * iavf_clear_adv_rss_conf - Remove adv rss conf not sent to PF yet and mark + * other to be removed. + * @adapter: board private structure + **/ +static void iavf_clear_adv_rss_conf(struct iavf_adapter *adapter) +{ + struct iavf_adv_rss *rss, *rsstmp; /* remove all advance RSS configuration */ spin_lock_bh(&adapter->adv_rss_lock); - list_for_each_entry(rss, &adapter->adv_rss_list_head, list) - rss->state = IAVF_ADV_RSS_DEL_REQUEST; + list_for_each_entry_safe(rss, rsstmp, &adapter->adv_rss_list_head, + list) { + if (rss->state == IAVF_ADV_RSS_ADD_REQUEST) { + list_del(&rss->list); + kfree(rss); + } else { + rss->state = IAVF_ADV_RSS_DEL_REQUEST; + } + } spin_unlock_bh(&adapter->adv_rss_lock); +} + +/** + * iavf_down - Shutdown the connection processing + * @adapter: board private structure + * + * Expects to be called while holding the __IAVF_IN_CRITICAL_TASK bit lock. + **/ +void iavf_down(struct iavf_adapter *adapter) +{ + struct net_device *netdev = adapter->netdev; + + if (adapter->state <= __IAVF_DOWN_PENDING) + return; + + netif_carrier_off(netdev); + netif_tx_disable(netdev); + adapter->link_up = false; + iavf_napi_disable_all(adapter); + iavf_irq_disable(adapter); + + iavf_clear_mac_vlan_filters(adapter); + iavf_clear_cloud_filters(adapter); + iavf_clear_fdir_filters(adapter); + iavf_clear_adv_rss_conf(adapter); if (!(adapter->flags & IAVF_FLAG_PF_COMMS_FAILED)) { /* cancel any current operation */ @@ -1335,11 +1407,16 @@ void iavf_down(struct iavf_adapter *adapter) * here for this to complete. The watchdog is still running * and it will take care of this. */ - adapter->aq_required = IAVF_FLAG_AQ_DEL_MAC_FILTER; - adapter->aq_required |= IAVF_FLAG_AQ_DEL_VLAN_FILTER; - adapter->aq_required |= IAVF_FLAG_AQ_DEL_CLOUD_FILTER; - adapter->aq_required |= IAVF_FLAG_AQ_DEL_FDIR_FILTER; - adapter->aq_required |= IAVF_FLAG_AQ_DEL_ADV_RSS_CFG; + if (!list_empty(&adapter->mac_filter_list)) + adapter->aq_required |= IAVF_FLAG_AQ_DEL_MAC_FILTER; + if (!list_empty(&adapter->vlan_filter_list)) + adapter->aq_required |= IAVF_FLAG_AQ_DEL_VLAN_FILTER; + if (!list_empty(&adapter->cloud_filter_list)) + adapter->aq_required |= IAVF_FLAG_AQ_DEL_CLOUD_FILTER; + if (!list_empty(&adapter->fdir_list_head)) + adapter->aq_required |= IAVF_FLAG_AQ_DEL_FDIR_FILTER; + if (!list_empty(&adapter->adv_rss_list_head)) + adapter->aq_required |= IAVF_FLAG_AQ_DEL_ADV_RSS_CFG; adapter->aq_required |= IAVF_FLAG_AQ_DISABLE_QUEUES; } @@ -1754,7 +1831,7 @@ static int iavf_alloc_q_vectors(struct iavf_adapter *adapter) q_vector->reg_idx = q_idx; cpumask_copy(&q_vector->affinity_mask, cpu_possible_mask); netif_napi_add(adapter->netdev, &q_vector->napi, - iavf_napi_poll, NAPI_POLL_WEIGHT); + iavf_napi_poll); } return 0; @@ -4178,6 +4255,7 @@ err_unlock: static int iavf_close(struct net_device *netdev) { struct iavf_adapter *adapter = netdev_priv(netdev); + u64 aq_to_restore; int status; mutex_lock(&adapter->crit_lock); @@ -4190,6 +4268,29 @@ static int iavf_close(struct net_device *netdev) set_bit(__IAVF_VSI_DOWN, adapter->vsi.state); if (CLIENT_ENABLED(adapter)) adapter->flags |= IAVF_FLAG_CLIENT_NEEDS_CLOSE; + /* We cannot send IAVF_FLAG_AQ_GET_OFFLOAD_VLAN_V2_CAPS before + * IAVF_FLAG_AQ_DISABLE_QUEUES because in such case there is rtnl + * deadlock with adminq_task() until iavf_close timeouts. We must send + * IAVF_FLAG_AQ_GET_CONFIG before IAVF_FLAG_AQ_DISABLE_QUEUES to make + * disable queues possible for vf. Give only necessary flags to + * iavf_down and save other to set them right before iavf_close() + * returns, when IAVF_FLAG_AQ_DISABLE_QUEUES will be already sent and + * iavf will be in DOWN state. + */ + aq_to_restore = adapter->aq_required; + adapter->aq_required &= IAVF_FLAG_AQ_GET_CONFIG; + + /* Remove flags which we do not want to send after close or we want to + * send before disable queues. + */ + aq_to_restore &= ~(IAVF_FLAG_AQ_GET_CONFIG | + IAVF_FLAG_AQ_ENABLE_QUEUES | + IAVF_FLAG_AQ_CONFIGURE_QUEUES | + IAVF_FLAG_AQ_ADD_VLAN_FILTER | + IAVF_FLAG_AQ_ADD_MAC_FILTER | + IAVF_FLAG_AQ_ADD_CLOUD_FILTER | + IAVF_FLAG_AQ_ADD_FDIR_FILTER | + IAVF_FLAG_AQ_ADD_ADV_RSS_CFG); iavf_down(adapter); iavf_change_state(adapter, __IAVF_DOWN_PENDING); @@ -4213,6 +4314,10 @@ static int iavf_close(struct net_device *netdev) msecs_to_jiffies(500)); if (!status) netdev_warn(netdev, "Device resources not yet released\n"); + + mutex_lock(&adapter->crit_lock); + adapter->aq_required |= aq_to_restore; + mutex_unlock(&adapter->crit_lock); return 0; } diff --git a/drivers/net/ethernet/intel/ice/ice.h b/drivers/net/ethernet/intel/ice/ice.h index 841fa149c407..001500afc4a6 100644 --- a/drivers/net/ethernet/intel/ice/ice.h +++ b/drivers/net/ethernet/intel/ice/ice.h @@ -864,6 +864,7 @@ ice_fetch_u64_stats_per_ring(struct u64_stats_sync *syncp, struct ice_q_stats stats, u64 *pkts, u64 *bytes); int ice_up(struct ice_vsi *vsi); int ice_down(struct ice_vsi *vsi); +int ice_down_up(struct ice_vsi *vsi); int ice_vsi_cfg(struct ice_vsi *vsi); struct ice_vsi *ice_lb_vsi_setup(struct ice_pf *pf, struct ice_port_info *pi); int ice_vsi_determine_xdp_res(struct ice_vsi *vsi); diff --git a/drivers/net/ethernet/intel/ice/ice_adminq_cmd.h b/drivers/net/ethernet/intel/ice/ice_adminq_cmd.h index 9939238573a4..1bdc70aa979d 100644 --- a/drivers/net/ethernet/intel/ice/ice_adminq_cmd.h +++ b/drivers/net/ethernet/intel/ice/ice_adminq_cmd.h @@ -1423,6 +1423,56 @@ struct ice_aqc_set_port_id_led { u8 rsvd[13]; }; +/* Get Port Options (indirect, 0x06EA) */ +struct ice_aqc_get_port_options { + u8 lport_num; + u8 lport_num_valid; + u8 port_options_count; +#define ICE_AQC_PORT_OPT_COUNT_M GENMASK(3, 0) +#define ICE_AQC_PORT_OPT_MAX 16 + + u8 innermost_phy_index; + u8 port_options; +#define ICE_AQC_PORT_OPT_ACTIVE_M GENMASK(3, 0) +#define ICE_AQC_PORT_OPT_VALID BIT(7) + + u8 pending_port_option_status; +#define ICE_AQC_PENDING_PORT_OPT_IDX_M GENMASK(3, 0) +#define ICE_AQC_PENDING_PORT_OPT_VALID BIT(7) + + u8 rsvd[2]; + __le32 addr_high; + __le32 addr_low; +}; + +struct ice_aqc_get_port_options_elem { + u8 pmd; +#define ICE_AQC_PORT_OPT_PMD_COUNT_M GENMASK(3, 0) + + u8 max_lane_speed; +#define ICE_AQC_PORT_OPT_MAX_LANE_M GENMASK(3, 0) +#define ICE_AQC_PORT_OPT_MAX_LANE_100M 0 +#define ICE_AQC_PORT_OPT_MAX_LANE_1G 1 +#define ICE_AQC_PORT_OPT_MAX_LANE_2500M 2 +#define ICE_AQC_PORT_OPT_MAX_LANE_5G 3 +#define ICE_AQC_PORT_OPT_MAX_LANE_10G 4 +#define ICE_AQC_PORT_OPT_MAX_LANE_25G 5 +#define ICE_AQC_PORT_OPT_MAX_LANE_50G 6 +#define ICE_AQC_PORT_OPT_MAX_LANE_100G 7 + + u8 global_scid[2]; + u8 phy_scid[2]; + u8 pf2port_cid[2]; +}; + +/* Set Port Option (direct, 0x06EB) */ +struct ice_aqc_set_port_option { + u8 lport_num; + u8 lport_num_valid; + u8 selected_port_option; + u8 rsvd[13]; +}; + /* Set/Get GPIO (direct, 0x06EC/0x06ED) */ struct ice_aqc_gpio { __le16 gpio_ctrl_handle; @@ -1489,6 +1539,12 @@ struct ice_aqc_nvm { #define ICE_AQC_NVM_PERST_FLAG 1 #define ICE_AQC_NVM_EMPR_FLAG 2 #define ICE_AQC_NVM_EMPR_ENA BIT(0) /* Write Activate reply only */ + /* For Write Activate, several flags are sent as part of a separate + * flags2 field using a separate byte. For simplicity of the software + * interface, we pass the flags as a 16 bit value so these flags are + * all offset by 8 bits + */ +#define ICE_AQC_NVM_ACTIV_REQ_EMPR BIT(8) /* NVM Write Activate only */ __le16 module_typeid; __le16 length; #define ICE_AQC_NVM_ERASE_LEN 0xFFFF @@ -2082,6 +2138,8 @@ struct ice_aq_desc { struct ice_aqc_gpio read_write_gpio; struct ice_aqc_sff_eeprom read_write_sff_param; struct ice_aqc_set_port_id_led set_port_id_led; + struct ice_aqc_get_port_options get_port_options; + struct ice_aqc_set_port_option set_port_option; struct ice_aqc_get_sw_cfg get_sw_conf; struct ice_aqc_set_port_params set_port_params; struct ice_aqc_sw_rules sw_rules; @@ -2243,6 +2301,8 @@ enum ice_adminq_opc { ice_aqc_opc_read_i2c = 0x06E2, ice_aqc_opc_write_i2c = 0x06E3, ice_aqc_opc_set_port_id_led = 0x06E9, + ice_aqc_opc_get_port_options = 0x06EA, + ice_aqc_opc_set_port_option = 0x06EB, ice_aqc_opc_set_gpio = 0x06EC, ice_aqc_opc_get_gpio = 0x06ED, ice_aqc_opc_sff_eeprom = 0x06EE, diff --git a/drivers/net/ethernet/intel/ice/ice_base.c b/drivers/net/ethernet/intel/ice/ice_base.c index 1e3243808178..9e36f01dfa4f 100644 --- a/drivers/net/ethernet/intel/ice/ice_base.c +++ b/drivers/net/ethernet/intel/ice/ice_base.c @@ -130,8 +130,7 @@ static int ice_vsi_alloc_q_vector(struct ice_vsi *vsi, u16 v_idx) * handler here (i.e. resume, reset/rebuild, etc.) */ if (vsi->netdev) - netif_napi_add(vsi->netdev, &q_vector->napi, ice_napi_poll, - NAPI_POLL_WEIGHT); + netif_napi_add(vsi->netdev, &q_vector->napi, ice_napi_poll); out: /* tie q_vector and VSI together */ @@ -405,7 +404,7 @@ static int ice_setup_rx_ctx(struct ice_rx_ring *ring) /* Strip the Ethernet CRC bytes before the packet is posted to host * memory. */ - rlan_ctx.crcstrip = 1; + rlan_ctx.crcstrip = !(ring->flags & ICE_RX_FLAGS_CRC_STRIP_DIS); /* L2TSEL flag defines the reported L2 Tags in the receive descriptor * and it needs to remain 1 for non-DVM capable configurations to not diff --git a/drivers/net/ethernet/intel/ice/ice_common.c b/drivers/net/ethernet/intel/ice/ice_common.c index 27d0cbbd29da..039342a0ed15 100644 --- a/drivers/net/ethernet/intel/ice/ice_common.c +++ b/drivers/net/ethernet/intel/ice/ice_common.c @@ -8,6 +8,108 @@ #define ICE_PF_RESET_WAIT_COUNT 300 +static const char * const ice_link_mode_str_low[] = { + [0] = "100BASE_TX", + [1] = "100M_SGMII", + [2] = "1000BASE_T", + [3] = "1000BASE_SX", + [4] = "1000BASE_LX", + [5] = "1000BASE_KX", + [6] = "1G_SGMII", + [7] = "2500BASE_T", + [8] = "2500BASE_X", + [9] = "2500BASE_KX", + [10] = "5GBASE_T", + [11] = "5GBASE_KR", + [12] = "10GBASE_T", + [13] = "10G_SFI_DA", + [14] = "10GBASE_SR", + [15] = "10GBASE_LR", + [16] = "10GBASE_KR_CR1", + [17] = "10G_SFI_AOC_ACC", + [18] = "10G_SFI_C2C", + [19] = "25GBASE_T", + [20] = "25GBASE_CR", + [21] = "25GBASE_CR_S", + [22] = "25GBASE_CR1", + [23] = "25GBASE_SR", + [24] = "25GBASE_LR", + [25] = "25GBASE_KR", + [26] = "25GBASE_KR_S", + [27] = "25GBASE_KR1", + [28] = "25G_AUI_AOC_ACC", + [29] = "25G_AUI_C2C", + [30] = "40GBASE_CR4", + [31] = "40GBASE_SR4", + [32] = "40GBASE_LR4", + [33] = "40GBASE_KR4", + [34] = "40G_XLAUI_AOC_ACC", + [35] = "40G_XLAUI", + [36] = "50GBASE_CR2", + [37] = "50GBASE_SR2", + [38] = "50GBASE_LR2", + [39] = "50GBASE_KR2", + [40] = "50G_LAUI2_AOC_ACC", + [41] = "50G_LAUI2", + [42] = "50G_AUI2_AOC_ACC", + [43] = "50G_AUI2", + [44] = "50GBASE_CP", + [45] = "50GBASE_SR", + [46] = "50GBASE_FR", + [47] = "50GBASE_LR", + [48] = "50GBASE_KR_PAM4", + [49] = "50G_AUI1_AOC_ACC", + [50] = "50G_AUI1", + [51] = "100GBASE_CR4", + [52] = "100GBASE_SR4", + [53] = "100GBASE_LR4", + [54] = "100GBASE_KR4", + [55] = "100G_CAUI4_AOC_ACC", + [56] = "100G_CAUI4", + [57] = "100G_AUI4_AOC_ACC", + [58] = "100G_AUI4", + [59] = "100GBASE_CR_PAM4", + [60] = "100GBASE_KR_PAM4", + [61] = "100GBASE_CP2", + [62] = "100GBASE_SR2", + [63] = "100GBASE_DR", +}; + +static const char * const ice_link_mode_str_high[] = { + [0] = "100GBASE_KR2_PAM4", + [1] = "100G_CAUI2_AOC_ACC", + [2] = "100G_CAUI2", + [3] = "100G_AUI2_AOC_ACC", + [4] = "100G_AUI2", +}; + +/** + * ice_dump_phy_type - helper function to dump phy_type + * @hw: pointer to the HW structure + * @low: 64 bit value for phy_type_low + * @high: 64 bit value for phy_type_high + * @prefix: prefix string to differentiate multiple dumps + */ +static void +ice_dump_phy_type(struct ice_hw *hw, u64 low, u64 high, const char *prefix) +{ + ice_debug(hw, ICE_DBG_PHY, "%s: phy_type_low: 0x%016llx\n", prefix, low); + + for (u32 i = 0; i < BITS_PER_TYPE(typeof(low)); i++) { + if (low & BIT_ULL(i)) + ice_debug(hw, ICE_DBG_PHY, "%s: bit(%d): %s\n", + prefix, i, ice_link_mode_str_low[i]); + } + + ice_debug(hw, ICE_DBG_PHY, "%s: phy_type_high: 0x%016llx\n", prefix, high); + + for (u32 i = 0; i < BITS_PER_TYPE(typeof(high)); i++) { + if (high & BIT_ULL(i)) + ice_debug(hw, ICE_DBG_PHY, "%s: bit(%d): %s\n", + prefix, i, ice_link_mode_str_high[i]); + } +} + /** * ice_set_mac_type - Sets MAC type * @hw: pointer to the HW structure @@ -80,9 +182,23 @@ bool ice_is_e810t(struct ice_hw *hw) { switch (hw->device_id) { case ICE_DEV_ID_E810C_SFP: - if (hw->subsystem_device_id == ICE_SUBDEV_ID_E810T || - hw->subsystem_device_id == ICE_SUBDEV_ID_E810T2) + switch (hw->subsystem_device_id) { + case ICE_SUBDEV_ID_E810T: + case ICE_SUBDEV_ID_E810T2: + case ICE_SUBDEV_ID_E810T3: + case ICE_SUBDEV_ID_E810T4: + case ICE_SUBDEV_ID_E810T6: + case ICE_SUBDEV_ID_E810T7: + return true; + } + break; + case ICE_DEV_ID_E810C_QSFP: + switch (hw->subsystem_device_id) { + case ICE_SUBDEV_ID_E810T2: + case ICE_SUBDEV_ID_E810T3: + case ICE_SUBDEV_ID_E810T5: return true; + } break; default: break; @@ -183,6 +299,7 @@ ice_aq_get_phy_caps(struct ice_port_info *pi, bool qual_mods, u8 report_mode, struct ice_aqc_get_phy_caps *cmd; u16 pcaps_size = sizeof(*pcaps); struct ice_aq_desc desc; + const char *prefix; struct ice_hw *hw; int status; @@ -204,29 +321,48 @@ ice_aq_get_phy_caps(struct ice_port_info *pi, bool qual_mods, u8 report_mode, cmd->param0 |= cpu_to_le16(report_mode); status = ice_aq_send_cmd(hw, &desc, pcaps, pcaps_size, cd); - ice_debug(hw, ICE_DBG_LINK, "get phy caps - report_mode = 0x%x\n", - report_mode); - ice_debug(hw, ICE_DBG_LINK, " phy_type_low = 0x%llx\n", - (unsigned long long)le64_to_cpu(pcaps->phy_type_low)); - ice_debug(hw, ICE_DBG_LINK, " phy_type_high = 0x%llx\n", - (unsigned long long)le64_to_cpu(pcaps->phy_type_high)); - ice_debug(hw, ICE_DBG_LINK, " caps = 0x%x\n", pcaps->caps); - ice_debug(hw, ICE_DBG_LINK, " low_power_ctrl_an = 0x%x\n", + ice_debug(hw, ICE_DBG_LINK, "get phy caps dump\n"); + + switch (report_mode) { + case ICE_AQC_REPORT_TOPO_CAP_MEDIA: + prefix = "phy_caps_media"; + break; + case ICE_AQC_REPORT_TOPO_CAP_NO_MEDIA: + prefix = "phy_caps_no_media"; + break; + case ICE_AQC_REPORT_ACTIVE_CFG: + prefix = "phy_caps_active"; + break; + case ICE_AQC_REPORT_DFLT_CFG: + prefix = "phy_caps_default"; + break; + default: + prefix = "phy_caps_invalid"; + } + + ice_dump_phy_type(hw, le64_to_cpu(pcaps->phy_type_low), + le64_to_cpu(pcaps->phy_type_high), prefix); + + ice_debug(hw, ICE_DBG_LINK, "%s: report_mode = 0x%x\n", + prefix, report_mode); + ice_debug(hw, ICE_DBG_LINK, "%s: caps = 0x%x\n", prefix, pcaps->caps); + ice_debug(hw, ICE_DBG_LINK, "%s: low_power_ctrl_an = 0x%x\n", prefix, pcaps->low_power_ctrl_an); - ice_debug(hw, ICE_DBG_LINK, " eee_cap = 0x%x\n", pcaps->eee_cap); - ice_debug(hw, ICE_DBG_LINK, " eeer_value = 0x%x\n", + ice_debug(hw, ICE_DBG_LINK, "%s: eee_cap = 0x%x\n", prefix, + pcaps->eee_cap); + ice_debug(hw, ICE_DBG_LINK, "%s: eeer_value = 0x%x\n", prefix, pcaps->eeer_value); - ice_debug(hw, ICE_DBG_LINK, " link_fec_options = 0x%x\n", + ice_debug(hw, ICE_DBG_LINK, "%s: link_fec_options = 0x%x\n", prefix, pcaps->link_fec_options); - ice_debug(hw, ICE_DBG_LINK, " module_compliance_enforcement = 0x%x\n", - pcaps->module_compliance_enforcement); - ice_debug(hw, ICE_DBG_LINK, " extended_compliance_code = 0x%x\n", - pcaps->extended_compliance_code); - ice_debug(hw, ICE_DBG_LINK, " module_type[0] = 0x%x\n", + ice_debug(hw, ICE_DBG_LINK, "%s: module_compliance_enforcement = 0x%x\n", + prefix, pcaps->module_compliance_enforcement); + ice_debug(hw, ICE_DBG_LINK, "%s: extended_compliance_code = 0x%x\n", + prefix, pcaps->extended_compliance_code); + ice_debug(hw, ICE_DBG_LINK, "%s: module_type[0] = 0x%x\n", prefix, pcaps->module_type[0]); - ice_debug(hw, ICE_DBG_LINK, " module_type[1] = 0x%x\n", + ice_debug(hw, ICE_DBG_LINK, "%s: module_type[1] = 0x%x\n", prefix, pcaps->module_type[1]); - ice_debug(hw, ICE_DBG_LINK, " module_type[2] = 0x%x\n", + ice_debug(hw, ICE_DBG_LINK, "%s: module_type[2] = 0x%x\n", prefix, pcaps->module_type[2]); if (!status && report_mode == ICE_AQC_REPORT_TOPO_CAP_MEDIA) { @@ -2397,6 +2533,8 @@ ice_parse_1588_dev_caps(struct ice_hw *hw, struct ice_hw_dev_caps *dev_p, info->tmr1_owned = ((number & ICE_TS_TMR1_OWND_M) != 0); info->tmr1_ena = ((number & ICE_TS_TMR1_ENA_M) != 0); + info->ts_ll_read = ((number & ICE_TS_LL_TX_TS_READ_M) != 0); + info->ena_ports = logical_id; info->tmr_own_map = phys_id; @@ -2414,6 +2552,8 @@ ice_parse_1588_dev_caps(struct ice_hw *hw, struct ice_hw_dev_caps *dev_p, info->tmr1_owned); ice_debug(hw, ICE_DBG_INIT, "dev caps: tmr1_ena = %u\n", info->tmr1_ena); + ice_debug(hw, ICE_DBG_INIT, "dev caps: ts_ll_read = %u\n", + info->ts_ll_read); ice_debug(hw, ICE_DBG_INIT, "dev caps: ieee_1588 ena_ports = %u\n", info->ena_ports); ice_debug(hw, ICE_DBG_INIT, "dev caps: tmr_own_map = %u\n", @@ -2776,6 +2916,26 @@ ice_aq_set_port_params(struct ice_port_info *pi, bool double_vlan, } /** + * ice_is_100m_speed_supported + * @hw: pointer to the HW struct + * + * returns true if 100M speeds are supported by the device, + * false otherwise. + */ +bool ice_is_100m_speed_supported(struct ice_hw *hw) +{ + switch (hw->device_id) { + case ICE_DEV_ID_E822C_SGMII: + case ICE_DEV_ID_E822L_SGMII: + case ICE_DEV_ID_E823L_1GBE: + case ICE_DEV_ID_E823C_SGMII: + return true; + default: + return false; + } +} + +/** * ice_get_link_speed_based_on_phy_type - returns link speed * @phy_type_low: lower part of phy_type * @phy_type_high: higher part of phy_type @@ -3535,6 +3695,121 @@ ice_aq_set_port_id_led(struct ice_port_info *pi, bool is_orig_mode, } /** + * ice_aq_get_port_options + * @hw: pointer to the HW struct + * @options: buffer for the resultant port options + * @option_count: input - size of the buffer in port options structures, + * output - number of returned port options + * @lport: logical port to call the command with (optional) + * @lport_valid: when false, FW uses port owned by the PF instead of lport, + * when PF owns more than 1 port it must be true + * @active_option_idx: index of active port option in returned buffer + * @active_option_valid: active option in returned buffer is valid + * @pending_option_idx: index of pending port option in returned buffer + * @pending_option_valid: pending option in returned buffer is valid + * + * Calls Get Port Options AQC (0x06ea) and verifies result. + */ +int +ice_aq_get_port_options(struct ice_hw *hw, + struct ice_aqc_get_port_options_elem *options, + u8 *option_count, u8 lport, bool lport_valid, + u8 *active_option_idx, bool *active_option_valid, + u8 *pending_option_idx, bool *pending_option_valid) +{ + struct ice_aqc_get_port_options *cmd; + struct ice_aq_desc desc; + int status; + u8 i; + + /* options buffer shall be able to hold max returned options */ + if (*option_count < ICE_AQC_PORT_OPT_COUNT_M) + return -EINVAL; + + cmd = &desc.params.get_port_options; + ice_fill_dflt_direct_cmd_desc(&desc, ice_aqc_opc_get_port_options); + + if (lport_valid) + cmd->lport_num = lport; + cmd->lport_num_valid = lport_valid; + + status = ice_aq_send_cmd(hw, &desc, options, + *option_count * sizeof(*options), NULL); + if (status) + return status; + + /* verify direct FW response & set output parameters */ + *option_count = FIELD_GET(ICE_AQC_PORT_OPT_COUNT_M, + cmd->port_options_count); + ice_debug(hw, ICE_DBG_PHY, "options: %x\n", *option_count); + *active_option_valid = FIELD_GET(ICE_AQC_PORT_OPT_VALID, + cmd->port_options); + if (*active_option_valid) { + *active_option_idx = FIELD_GET(ICE_AQC_PORT_OPT_ACTIVE_M, + cmd->port_options); + if (*active_option_idx > (*option_count - 1)) + return -EIO; + ice_debug(hw, ICE_DBG_PHY, "active idx: %x\n", + *active_option_idx); + } + + *pending_option_valid = FIELD_GET(ICE_AQC_PENDING_PORT_OPT_VALID, + cmd->pending_port_option_status); + if (*pending_option_valid) { + *pending_option_idx = FIELD_GET(ICE_AQC_PENDING_PORT_OPT_IDX_M, + cmd->pending_port_option_status); + if (*pending_option_idx > (*option_count - 1)) + return -EIO; + ice_debug(hw, ICE_DBG_PHY, "pending idx: %x\n", + *pending_option_idx); + } + + /* mask output options fields */ + for (i = 0; i < *option_count; i++) { + options[i].pmd = FIELD_GET(ICE_AQC_PORT_OPT_PMD_COUNT_M, + options[i].pmd); + options[i].max_lane_speed = FIELD_GET(ICE_AQC_PORT_OPT_MAX_LANE_M, + options[i].max_lane_speed); + ice_debug(hw, ICE_DBG_PHY, "pmds: %x max speed: %x\n", + options[i].pmd, options[i].max_lane_speed); + } + + return 0; +} + +/** + * ice_aq_set_port_option + * @hw: pointer to the HW struct + * @lport: logical port to call the command with + * @lport_valid: when false, FW uses port owned by the PF instead of lport, + * when PF owns more than 1 port it must be true + * @new_option: new port option to be written + * + * Calls Set Port Options AQC (0x06eb). + */ +int +ice_aq_set_port_option(struct ice_hw *hw, u8 lport, u8 lport_valid, + u8 new_option) +{ + struct ice_aqc_set_port_option *cmd; + struct ice_aq_desc desc; + + if (new_option > ICE_AQC_PORT_OPT_COUNT_M) + return -EINVAL; + + cmd = &desc.params.set_port_option; + ice_fill_dflt_direct_cmd_desc(&desc, ice_aqc_opc_set_port_option); + + if (lport_valid) + cmd->lport_num = lport; + + cmd->lport_num_valid = lport_valid; + cmd->selected_port_option = new_option; + + return ice_aq_send_cmd(hw, &desc, NULL, 0, NULL); +} + +/** * ice_aq_sff_eeprom * @hw: pointer to the HW struct * @lport: bits [7:0] = logical port, bit [8] = logical port valid @@ -5029,20 +5304,22 @@ ice_aq_get_gpio(struct ice_hw *hw, u16 gpio_ctrl_handle, u8 pin_idx, } /** - * ice_fw_supports_link_override + * ice_is_fw_api_min_ver * @hw: pointer to the hardware structure + * @maj: major version + * @min: minor version + * @patch: patch version * - * Checks if the firmware supports link override + * Checks if the firmware API is minimum version */ -bool ice_fw_supports_link_override(struct ice_hw *hw) +static bool ice_is_fw_api_min_ver(struct ice_hw *hw, u8 maj, u8 min, u8 patch) { - if (hw->api_maj_ver == ICE_FW_API_LINK_OVERRIDE_MAJ) { - if (hw->api_min_ver > ICE_FW_API_LINK_OVERRIDE_MIN) + if (hw->api_maj_ver == maj) { + if (hw->api_min_ver > min) return true; - if (hw->api_min_ver == ICE_FW_API_LINK_OVERRIDE_MIN && - hw->api_patch >= ICE_FW_API_LINK_OVERRIDE_PATCH) + if (hw->api_min_ver == min && hw->api_patch >= patch) return true; - } else if (hw->api_maj_ver > ICE_FW_API_LINK_OVERRIDE_MAJ) { + } else if (hw->api_maj_ver > maj) { return true; } @@ -5050,6 +5327,19 @@ bool ice_fw_supports_link_override(struct ice_hw *hw) } /** + * ice_fw_supports_link_override + * @hw: pointer to the hardware structure + * + * Checks if the firmware supports link override + */ +bool ice_fw_supports_link_override(struct ice_hw *hw) +{ + return ice_is_fw_api_min_ver(hw, ICE_FW_API_LINK_OVERRIDE_MAJ, + ICE_FW_API_LINK_OVERRIDE_MIN, + ICE_FW_API_LINK_OVERRIDE_PATCH); +} + +/** * ice_get_link_default_override * @ldo: pointer to the link default override struct * @pi: pointer to the port info struct @@ -5179,16 +5469,9 @@ bool ice_fw_supports_lldp_fltr_ctrl(struct ice_hw *hw) if (hw->mac_type != ICE_MAC_E810) return false; - if (hw->api_maj_ver == ICE_FW_API_LLDP_FLTR_MAJ) { - if (hw->api_min_ver > ICE_FW_API_LLDP_FLTR_MIN) - return true; - if (hw->api_min_ver == ICE_FW_API_LLDP_FLTR_MIN && - hw->api_patch >= ICE_FW_API_LLDP_FLTR_PATCH) - return true; - } else if (hw->api_maj_ver > ICE_FW_API_LLDP_FLTR_MAJ) { - return true; - } - return false; + return ice_is_fw_api_min_ver(hw, ICE_FW_API_LLDP_FLTR_MAJ, + ICE_FW_API_LLDP_FLTR_MIN, + ICE_FW_API_LLDP_FLTR_PATCH); } /** @@ -5225,14 +5508,7 @@ ice_lldp_fltr_add_remove(struct ice_hw *hw, u16 vsi_num, bool add) */ bool ice_fw_supports_report_dflt_cfg(struct ice_hw *hw) { - if (hw->api_maj_ver == ICE_FW_API_REPORT_DFLT_CFG_MAJ) { - if (hw->api_min_ver > ICE_FW_API_REPORT_DFLT_CFG_MIN) - return true; - if (hw->api_min_ver == ICE_FW_API_REPORT_DFLT_CFG_MIN && - hw->api_patch >= ICE_FW_API_REPORT_DFLT_CFG_PATCH) - return true; - } else if (hw->api_maj_ver > ICE_FW_API_REPORT_DFLT_CFG_MAJ) { - return true; - } - return false; + return ice_is_fw_api_min_ver(hw, ICE_FW_API_REPORT_DFLT_CFG_MAJ, + ICE_FW_API_REPORT_DFLT_CFG_MIN, + ICE_FW_API_REPORT_DFLT_CFG_PATCH); } diff --git a/drivers/net/ethernet/intel/ice/ice_common.h b/drivers/net/ethernet/intel/ice/ice_common.h index 61b7c60db689..8b6712b92e84 100644 --- a/drivers/net/ethernet/intel/ice/ice_common.h +++ b/drivers/net/ethernet/intel/ice/ice_common.h @@ -151,6 +151,15 @@ int ice_aq_set_port_id_led(struct ice_port_info *pi, bool is_orig_mode, struct ice_sq_cd *cd); int +ice_aq_get_port_options(struct ice_hw *hw, + struct ice_aqc_get_port_options_elem *options, + u8 *option_count, u8 lport, bool lport_valid, + u8 *active_option_idx, bool *active_option_valid, + u8 *pending_option_idx, bool *pending_option_valid); +int +ice_aq_set_port_option(struct ice_hw *hw, u8 lport, u8 lport_valid, + u8 new_option); +int ice_aq_sff_eeprom(struct ice_hw *hw, u16 lport, u8 bus_addr, u16 mem_addr, u8 page, u8 set_page, u8 *data, u8 length, bool write, struct ice_sq_cd *cd); @@ -204,6 +213,7 @@ ice_aq_set_gpio(struct ice_hw *hw, u16 gpio_ctrl_handle, u8 pin_idx, bool value, int ice_aq_get_gpio(struct ice_hw *hw, u16 gpio_ctrl_handle, u8 pin_idx, bool *value, struct ice_sq_cd *cd); +bool ice_is_100m_speed_supported(struct ice_hw *hw); int ice_aq_set_lldp_mib(struct ice_hw *hw, u8 mib_type, void *buf, u16 buf_size, struct ice_sq_cd *cd); diff --git a/drivers/net/ethernet/intel/ice/ice_devids.h b/drivers/net/ethernet/intel/ice/ice_devids.h index b41bc3dc1745..6d560d1c74a4 100644 --- a/drivers/net/ethernet/intel/ice/ice_devids.h +++ b/drivers/net/ethernet/intel/ice/ice_devids.h @@ -24,6 +24,11 @@ #define ICE_DEV_ID_E810C_SFP 0x1593 #define ICE_SUBDEV_ID_E810T 0x000E #define ICE_SUBDEV_ID_E810T2 0x000F +#define ICE_SUBDEV_ID_E810T3 0x0010 +#define ICE_SUBDEV_ID_E810T4 0x0011 +#define ICE_SUBDEV_ID_E810T5 0x0012 +#define ICE_SUBDEV_ID_E810T6 0x02E9 +#define ICE_SUBDEV_ID_E810T7 0x02EA /* Intel(R) Ethernet Controller E810-XXV for backplane */ #define ICE_DEV_ID_E810_XXV_BACKPLANE 0x1599 /* Intel(R) Ethernet Controller E810-XXV for QSFP */ diff --git a/drivers/net/ethernet/intel/ice/ice_devlink.c b/drivers/net/ethernet/intel/ice/ice_devlink.c index 3337314a7b35..e6ec20079ced 100644 --- a/drivers/net/ethernet/intel/ice/ice_devlink.c +++ b/drivers/net/ethernet/intel/ice/ice_devlink.c @@ -9,6 +9,8 @@ #include "ice_eswitch.h" #include "ice_fw_update.h" +static int ice_active_port_option = -1; + /* context for devlink info version reporting */ struct ice_info_ctx { char buf[128]; @@ -466,12 +468,259 @@ ice_devlink_reload_empr_finish(struct devlink *devlink, return 0; } +/** + * ice_devlink_port_opt_speed_str - convert speed to a string + * @speed: speed value + */ +static const char *ice_devlink_port_opt_speed_str(u8 speed) +{ + switch (speed & ICE_AQC_PORT_OPT_MAX_LANE_M) { + case ICE_AQC_PORT_OPT_MAX_LANE_100M: + return "0.1"; + case ICE_AQC_PORT_OPT_MAX_LANE_1G: + return "1"; + case ICE_AQC_PORT_OPT_MAX_LANE_2500M: + return "2.5"; + case ICE_AQC_PORT_OPT_MAX_LANE_5G: + return "5"; + case ICE_AQC_PORT_OPT_MAX_LANE_10G: + return "10"; + case ICE_AQC_PORT_OPT_MAX_LANE_25G: + return "25"; + case ICE_AQC_PORT_OPT_MAX_LANE_50G: + return "50"; + case ICE_AQC_PORT_OPT_MAX_LANE_100G: + return "100"; + } + + return "-"; +} + +#define ICE_PORT_OPT_DESC_LEN 50 +/** + * ice_devlink_port_options_print - Print available port split options + * @pf: the PF to print split port options + * + * Prints a table with available port split options and max port speeds + */ +static void ice_devlink_port_options_print(struct ice_pf *pf) +{ + u8 i, j, options_count, cnt, speed, pending_idx, active_idx; + struct ice_aqc_get_port_options_elem *options, *opt; + struct device *dev = ice_pf_to_dev(pf); + bool active_valid, pending_valid; + char desc[ICE_PORT_OPT_DESC_LEN]; + const char *str; + int status; + + options = kcalloc(ICE_AQC_PORT_OPT_MAX * ICE_MAX_PORT_PER_PCI_DEV, + sizeof(*options), GFP_KERNEL); + if (!options) + return; + + for (i = 0; i < ICE_MAX_PORT_PER_PCI_DEV; i++) { + opt = options + i * ICE_AQC_PORT_OPT_MAX; + options_count = ICE_AQC_PORT_OPT_MAX; + active_valid = 0; + + status = ice_aq_get_port_options(&pf->hw, opt, &options_count, + i, true, &active_idx, + &active_valid, &pending_idx, + &pending_valid); + if (status) { + dev_dbg(dev, "Couldn't read port option for port %d, err %d\n", + i, status); + goto err; + } + } + + dev_dbg(dev, "Available port split options and max port speeds (Gbps):\n"); + dev_dbg(dev, "Status Split Quad 0 Quad 1\n"); + dev_dbg(dev, " count L0 L1 L2 L3 L4 L5 L6 L7\n"); + + for (i = 0; i < options_count; i++) { + cnt = 0; + + if (i == ice_active_port_option) + str = "Active"; + else if ((i == pending_idx) && pending_valid) + str = "Pending"; + else + str = ""; + + cnt += snprintf(&desc[cnt], ICE_PORT_OPT_DESC_LEN - cnt, + "%-8s", str); + + cnt += snprintf(&desc[cnt], ICE_PORT_OPT_DESC_LEN - cnt, + "%-6u", options[i].pmd); + + for (j = 0; j < ICE_MAX_PORT_PER_PCI_DEV; ++j) { + speed = options[i + j * ICE_AQC_PORT_OPT_MAX].max_lane_speed; + str = ice_devlink_port_opt_speed_str(speed); + cnt += snprintf(&desc[cnt], ICE_PORT_OPT_DESC_LEN - cnt, + "%3s ", str); + } + + dev_dbg(dev, "%s\n", desc); + } + +err: + kfree(options); +} + +/** + * ice_devlink_aq_set_port_option - Send set port option admin queue command + * @pf: the PF to print split port options + * @option_idx: selected port option + * @extack: extended netdev ack structure + * + * Sends set port option admin queue command with selected port option and + * calls NVM write activate. + */ +static int +ice_devlink_aq_set_port_option(struct ice_pf *pf, u8 option_idx, + struct netlink_ext_ack *extack) +{ + struct device *dev = ice_pf_to_dev(pf); + int status; + + status = ice_aq_set_port_option(&pf->hw, 0, true, option_idx); + if (status) { + dev_dbg(dev, "ice_aq_set_port_option, err %d aq_err %d\n", + status, pf->hw.adminq.sq_last_status); + NL_SET_ERR_MSG_MOD(extack, "Port split request failed"); + return -EIO; + } + + status = ice_acquire_nvm(&pf->hw, ICE_RES_WRITE); + if (status) { + dev_dbg(dev, "ice_acquire_nvm failed, err %d aq_err %d\n", + status, pf->hw.adminq.sq_last_status); + NL_SET_ERR_MSG_MOD(extack, "Failed to acquire NVM semaphore"); + return -EIO; + } + + status = ice_nvm_write_activate(&pf->hw, ICE_AQC_NVM_ACTIV_REQ_EMPR, NULL); + if (status) { + dev_dbg(dev, "ice_nvm_write_activate failed, err %d aq_err %d\n", + status, pf->hw.adminq.sq_last_status); + NL_SET_ERR_MSG_MOD(extack, "Port split request failed to save data"); + ice_release_nvm(&pf->hw); + return -EIO; + } + + ice_release_nvm(&pf->hw); + + NL_SET_ERR_MSG_MOD(extack, "Reboot required to finish port split"); + return 0; +} + +/** + * ice_devlink_port_split - .port_split devlink handler + * @devlink: devlink instance structure + * @port: devlink port structure + * @count: number of ports to split to + * @extack: extended netdev ack structure + * + * Callback for the devlink .port_split operation. + * + * Unfortunately, the devlink expression of available options is limited + * to just a number, so search for an FW port option which supports + * the specified number. As there could be multiple FW port options with + * the same port split count, allow switching between them. When the same + * port split count request is issued again, switch to the next FW port + * option with the same port split count. + * + * Return: zero on success or an error code on failure. + */ +static int +ice_devlink_port_split(struct devlink *devlink, struct devlink_port *port, + unsigned int count, struct netlink_ext_ack *extack) +{ + struct ice_aqc_get_port_options_elem options[ICE_AQC_PORT_OPT_MAX]; + u8 i, j, active_idx, pending_idx, new_option; + struct ice_pf *pf = devlink_priv(devlink); + u8 option_count = ICE_AQC_PORT_OPT_MAX; + struct device *dev = ice_pf_to_dev(pf); + bool active_valid, pending_valid; + int status; + + status = ice_aq_get_port_options(&pf->hw, options, &option_count, + 0, true, &active_idx, &active_valid, + &pending_idx, &pending_valid); + if (status) { + dev_dbg(dev, "Couldn't read port split options, err = %d\n", + status); + NL_SET_ERR_MSG_MOD(extack, "Failed to get available port split options"); + return -EIO; + } + + new_option = ICE_AQC_PORT_OPT_MAX; + active_idx = pending_valid ? pending_idx : active_idx; + for (i = 1; i <= option_count; i++) { + /* In order to allow switching between FW port options with + * the same port split count, search for a new option starting + * from the active/pending option (with array wrap around). + */ + j = (active_idx + i) % option_count; + + if (count == options[j].pmd) { + new_option = j; + break; + } + } + + if (new_option == active_idx) { + dev_dbg(dev, "request to split: count: %u is already set and there are no other options\n", + count); + NL_SET_ERR_MSG_MOD(extack, "Requested split count is already set"); + ice_devlink_port_options_print(pf); + return -EINVAL; + } + + if (new_option == ICE_AQC_PORT_OPT_MAX) { + dev_dbg(dev, "request to split: count: %u not found\n", count); + NL_SET_ERR_MSG_MOD(extack, "Port split requested unsupported port config"); + ice_devlink_port_options_print(pf); + return -EINVAL; + } + + status = ice_devlink_aq_set_port_option(pf, new_option, extack); + if (status) + return status; + + ice_devlink_port_options_print(pf); + + return 0; +} + +/** + * ice_devlink_port_unsplit - .port_unsplit devlink handler + * @devlink: devlink instance structure + * @port: devlink port structure + * @extack: extended netdev ack structure + * + * Callback for the devlink .port_unsplit operation. + * Calls ice_devlink_port_split with split count set to 1. + * There could be no FW option available with split count 1. + * + * Return: zero on success or an error code on failure. + */ +static int +ice_devlink_port_unsplit(struct devlink *devlink, struct devlink_port *port, + struct netlink_ext_ack *extack) +{ + return ice_devlink_port_split(devlink, port, 1, extack); +} + static const struct devlink_ops ice_devlink_ops = { .supported_flash_update_params = DEVLINK_SUPPORT_FLASH_UPDATE_OVERWRITE_MASK, .reload_actions = BIT(DEVLINK_RELOAD_ACTION_FW_ACTIVATE), /* The ice driver currently does not support driver reinit */ .reload_down = ice_devlink_reload_empr_start, .reload_up = ice_devlink_reload_empr_finish, + .port_split = ice_devlink_port_split, + .port_unsplit = ice_devlink_port_unsplit, .eswitch_mode_get = ice_eswitch_mode_get, .eswitch_mode_set = ice_eswitch_mode_set, .info_get = ice_devlink_info_get, @@ -695,6 +944,39 @@ void ice_devlink_unregister_params(struct ice_pf *pf) } /** + * ice_devlink_set_port_split_options - Set port split options + * @pf: the PF to set port split options + * @attrs: devlink attributes + * + * Sets devlink port split options based on available FW port options + */ +static void +ice_devlink_set_port_split_options(struct ice_pf *pf, + struct devlink_port_attrs *attrs) +{ + struct ice_aqc_get_port_options_elem options[ICE_AQC_PORT_OPT_MAX]; + u8 i, active_idx, pending_idx, option_count = ICE_AQC_PORT_OPT_MAX; + bool active_valid, pending_valid; + int status; + + status = ice_aq_get_port_options(&pf->hw, options, &option_count, + 0, true, &active_idx, &active_valid, + &pending_idx, &pending_valid); + if (status) { + dev_dbg(ice_pf_to_dev(pf), "Couldn't read port split options, err = %d\n", + status); + return; + } + + /* find the biggest available port split count */ + for (i = 0; i < option_count; i++) + attrs->lanes = max_t(int, attrs->lanes, options[i].pmd); + + attrs->splittable = attrs->lanes ? 1 : 0; + ice_active_port_option = active_idx; +} + +/** * ice_devlink_create_pf_port - Create a devlink port for this PF * @pf: the PF to create a devlink port for * @@ -722,6 +1004,12 @@ int ice_devlink_create_pf_port(struct ice_pf *pf) attrs.flavour = DEVLINK_PORT_FLAVOUR_PHYSICAL; attrs.phys.port_number = pf->hw.bus.func; + /* As FW supports only port split options for whole device, + * set port split options only for first PF. + */ + if (pf->hw.pf_id == 0) + ice_devlink_set_port_split_options(pf, &attrs); + ice_devlink_set_switch_id(pf, &attrs.switch_id); devlink_port_attrs_set(devlink_port, &attrs); diff --git a/drivers/net/ethernet/intel/ice/ice_eswitch.c b/drivers/net/ethernet/intel/ice/ice_eswitch.c index e35371e61e07..f9f15acae90a 100644 --- a/drivers/net/ethernet/intel/ice/ice_eswitch.c +++ b/drivers/net/ethernet/intel/ice/ice_eswitch.c @@ -292,8 +292,8 @@ static int ice_eswitch_setup_reprs(struct ice_pf *pf) if (max_vsi_num < vsi->vsi_num) max_vsi_num = vsi->vsi_num; - netif_napi_add(vf->repr->netdev, &vf->repr->q_vector->napi, ice_napi_poll, - NAPI_POLL_WEIGHT); + netif_napi_add(vf->repr->netdev, &vf->repr->q_vector->napi, + ice_napi_poll); netif_keep_dst(vf->repr->netdev); } diff --git a/drivers/net/ethernet/intel/ice/ice_ethtool.c b/drivers/net/ethernet/intel/ice/ice_ethtool.c index a6fff8ebaf9d..b7be84bbe72d 100644 --- a/drivers/net/ethernet/intel/ice/ice_ethtool.c +++ b/drivers/net/ethernet/intel/ice/ice_ethtool.c @@ -136,6 +136,11 @@ static const struct ice_stats ice_gstrings_pf_stats[] = { ICE_PF_STAT("mac_remote_faults.nic", stats.mac_remote_faults), ICE_PF_STAT("fdir_sb_match.nic", stats.fd_sb_match), ICE_PF_STAT("fdir_sb_status.nic", stats.fd_sb_status), + ICE_PF_STAT("tx_hwtstamp_skipped", ptp.tx_hwtstamp_skipped), + ICE_PF_STAT("tx_hwtstamp_timeouts", ptp.tx_hwtstamp_timeouts), + ICE_PF_STAT("tx_hwtstamp_flushed", ptp.tx_hwtstamp_flushed), + ICE_PF_STAT("tx_hwtstamp_discarded", ptp.tx_hwtstamp_discarded), + ICE_PF_STAT("late_cached_phc_updates", ptp.late_cached_phc_updates), }; static const u32 ice_regs_dump_list[] = { @@ -1284,10 +1289,7 @@ static int ice_set_priv_flags(struct net_device *netdev, u32 flags) } if (test_bit(ICE_FLAG_LEGACY_RX, change_flags)) { /* down and up VSI so that changes of Rx cfg are reflected. */ - if (!test_and_set_bit(ICE_VSI_DOWN, vsi->state)) { - ice_down(vsi); - ice_up(vsi); - } + ice_down_up(vsi); } /* don't allow modification of this flag when a single VF is in * promiscuous mode because it's not supported @@ -1468,20 +1470,22 @@ ice_get_ethtool_stats(struct net_device *netdev, /** * ice_mask_min_supported_speeds + * @hw: pointer to the HW structure * @phy_types_high: PHY type high * @phy_types_low: PHY type low to apply minimum supported speeds mask * * Apply minimum supported speeds mask to PHY type low. These are the speeds * for ethtool supported link mode. */ -static -void ice_mask_min_supported_speeds(u64 phy_types_high, u64 *phy_types_low) +static void +ice_mask_min_supported_speeds(struct ice_hw *hw, + u64 phy_types_high, u64 *phy_types_low) { /* if QSFP connection with 100G speed, minimum supported speed is 25G */ if (*phy_types_low & ICE_PHY_TYPE_LOW_MASK_100G || phy_types_high & ICE_PHY_TYPE_HIGH_MASK_100G) *phy_types_low &= ~ICE_PHY_TYPE_LOW_MASK_MIN_25G; - else + else if (!ice_is_100m_speed_supported(hw)) *phy_types_low &= ~ICE_PHY_TYPE_LOW_MASK_MIN_1G; } @@ -1531,7 +1535,8 @@ ice_phy_type_to_ethtool(struct net_device *netdev, phy_types_low = le64_to_cpu(pf->nvm_phy_type_lo); phy_types_high = le64_to_cpu(pf->nvm_phy_type_hi); - ice_mask_min_supported_speeds(phy_types_high, &phy_types_low); + ice_mask_min_supported_speeds(&pf->hw, phy_types_high, + &phy_types_low); /* determine advertised modes based on link override only * if it's supported and if the FW doesn't abstract the * driver from having to account for link overrides @@ -2826,6 +2831,7 @@ ice_set_ringparam(struct net_device *netdev, struct ethtool_ringparam *ring, tx_rings[i].count = new_tx_cnt; tx_rings[i].desc = NULL; tx_rings[i].tx_buf = NULL; + tx_rings[i].tx_tstamps = &pf->ptp.port.tx; err = ice_setup_tx_ring(&tx_rings[i]); if (err) { while (i--) @@ -2884,6 +2890,7 @@ process_rx: /* clone ring and setup updated count */ rx_rings[i] = *vsi->rx_rings[i]; rx_rings[i].count = new_rx_cnt; + rx_rings[i].cached_phctime = pf->ptp.cached_phc_time; rx_rings[i].desc = NULL; rx_rings[i].rx_buf = NULL; /* this is to allow wr32 to have something to write to diff --git a/drivers/net/ethernet/intel/ice/ice_lag.c b/drivers/net/ethernet/intel/ice/ice_lag.c index c9f7393b783d..ee5b36941ba3 100644 --- a/drivers/net/ethernet/intel/ice/ice_lag.c +++ b/drivers/net/ethernet/intel/ice/ice_lag.c @@ -61,13 +61,13 @@ static void ice_lag_set_backup(struct ice_lag *lag) */ static void ice_display_lag_info(struct ice_lag *lag) { - const char *name, *peer, *upper, *role, *bonded, *master; + const char *name, *peer, *upper, *role, *bonded, *primary; struct device *dev = &lag->pf->pdev->dev; name = lag->netdev ? netdev_name(lag->netdev) : "unset"; peer = lag->peer_netdev ? netdev_name(lag->peer_netdev) : "unset"; upper = lag->upper_netdev ? netdev_name(lag->upper_netdev) : "unset"; - master = lag->master ? "TRUE" : "FALSE"; + primary = lag->primary ? "TRUE" : "FALSE"; bonded = lag->bonded ? "BONDED" : "UNBONDED"; switch (lag->role) { @@ -87,8 +87,8 @@ static void ice_display_lag_info(struct ice_lag *lag) role = "ERROR"; } - dev_dbg(dev, "%s %s, peer:%s, upper:%s, role:%s, master:%s\n", name, - bonded, peer, upper, role, master); + dev_dbg(dev, "%s %s, peer:%s, upper:%s, role:%s, primary:%s\n", name, + bonded, peer, upper, role, primary); } /** @@ -119,7 +119,7 @@ static void ice_lag_info_event(struct ice_lag *lag, void *ptr) } if (strcmp(bonding_info->slave.slave_name, lag_netdev_name)) { - netdev_dbg(lag->netdev, "Bonding event recv, but slave info not for us\n"); + netdev_dbg(lag->netdev, "Bonding event recv, but secondary info not for us\n"); goto lag_out; } @@ -164,8 +164,8 @@ ice_lag_link(struct ice_lag *lag, struct netdev_notifier_changeupper_info *info) lag->bonded = true; lag->role = ICE_LAG_UNSET; - /* if this is the first element in an LAG mark as master */ - lag->master = !!(peers == 1); + /* if this is the first element in an LAG mark as primary */ + lag->primary = !!(peers == 1); } /** @@ -264,7 +264,7 @@ static void ice_lag_changeupper_event(struct ice_lag *lag, void *ptr) netdev_dbg(netdev, "bonding %s\n", info->linking ? "LINK" : "UNLINK"); if (!netif_is_lag_master(info->upper_dev)) { - netdev_dbg(netdev, "changeupper rcvd, but not master. bail\n"); + netdev_dbg(netdev, "changeupper rcvd, but not primary. bail\n"); return; } diff --git a/drivers/net/ethernet/intel/ice/ice_lag.h b/drivers/net/ethernet/intel/ice/ice_lag.h index c2e3688dd8fd..51b5cf467ce2 100644 --- a/drivers/net/ethernet/intel/ice/ice_lag.h +++ b/drivers/net/ethernet/intel/ice/ice_lag.h @@ -24,7 +24,7 @@ struct ice_lag { struct net_device *upper_netdev; /* upper bonding netdev */ struct notifier_block notif_block; u8 bonded:1; /* currently bonded */ - u8 master:1; /* this is a master */ + u8 primary:1; /* this is primary */ u8 handler:1; /* did we register a rx_netdev_handler */ /* each thing blocking bonding will increment this value by one. * If this value is zero, then bonding is allowed. diff --git a/drivers/net/ethernet/intel/ice/ice_lib.c b/drivers/net/ethernet/intel/ice/ice_lib.c index 58d483e2f539..938ba8c215cb 100644 --- a/drivers/net/ethernet/intel/ice/ice_lib.c +++ b/drivers/net/ethernet/intel/ice/ice_lib.c @@ -1524,6 +1524,7 @@ static int ice_vsi_alloc_rings(struct ice_vsi *vsi) ring->netdev = vsi->netdev; ring->dev = dev; ring->count = vsi->num_rx_desc; + ring->cached_phctime = pf->ptp.cached_phc_time; WRITE_ONCE(vsi->rx_rings[i], ring); } @@ -1564,6 +1565,22 @@ void ice_vsi_manage_rss_lut(struct ice_vsi *vsi, bool ena) } /** + * ice_vsi_cfg_crc_strip - Configure CRC stripping for a VSI + * @vsi: VSI to be configured + * @disable: set to true to have FCS / CRC in the frame data + */ +void ice_vsi_cfg_crc_strip(struct ice_vsi *vsi, bool disable) +{ + int i; + + ice_for_each_rxq(vsi, i) + if (disable) + vsi->rx_rings[i]->flags |= ICE_RX_FLAGS_CRC_STRIP_DIS; + else + vsi->rx_rings[i]->flags &= ~ICE_RX_FLAGS_CRC_STRIP_DIS; +} + +/** * ice_vsi_cfg_rss_lut_key - Configure RSS params for a VSI * @vsi: VSI to be configured */ @@ -2971,9 +2988,6 @@ int ice_vsi_release(struct ice_vsi *vsi) clear_bit(ICE_VSI_NETDEV_REGISTERED, vsi->state); } - if (vsi->type == ICE_VSI_PF) - ice_devlink_destroy_pf_port(pf); - if (test_bit(ICE_FLAG_RSS_ENA, pf->flags)) ice_rss_clean(vsi); @@ -3031,6 +3045,9 @@ int ice_vsi_release(struct ice_vsi *vsi) } } + if (vsi->type == ICE_VSI_PF) + ice_devlink_destroy_pf_port(pf); + if (vsi->type == ICE_VSI_VF && vsi->agg_node && vsi->agg_node->valid) vsi->agg_node->num_vsis--; @@ -3278,6 +3295,12 @@ int ice_vsi_rebuild(struct ice_vsi *vsi, bool init_vsi) */ if (test_bit(ICE_FLAG_RSS_ENA, pf->flags)) ice_vsi_cfg_rss_lut_key(vsi); + + /* disable or enable CRC stripping */ + if (vsi->netdev) + ice_vsi_cfg_crc_strip(vsi, !!(vsi->netdev->features & + NETIF_F_RXFCS)); + break; case ICE_VSI_VF: ret = ice_vsi_alloc_q_vectors(vsi); diff --git a/drivers/net/ethernet/intel/ice/ice_lib.h b/drivers/net/ethernet/intel/ice/ice_lib.h index 8712b1d2ceec..ec4bf0c89857 100644 --- a/drivers/net/ethernet/intel/ice/ice_lib.h +++ b/drivers/net/ethernet/intel/ice/ice_lib.h @@ -89,6 +89,8 @@ void ice_vsi_free_tx_rings(struct ice_vsi *vsi); void ice_vsi_manage_rss_lut(struct ice_vsi *vsi, bool ena); +void ice_vsi_cfg_crc_strip(struct ice_vsi *vsi, bool disable); + void ice_update_tx_ring_stats(struct ice_tx_ring *ring, u64 pkts, u64 bytes); void ice_update_rx_ring_stats(struct ice_rx_ring *ring, u64 pkts, u64 bytes); diff --git a/drivers/net/ethernet/intel/ice/ice_main.c b/drivers/net/ethernet/intel/ice/ice_main.c index e109cb93886b..0f6718719453 100644 --- a/drivers/net/ethernet/intel/ice/ice_main.c +++ b/drivers/net/ethernet/intel/ice/ice_main.c @@ -3093,7 +3093,8 @@ static irqreturn_t ice_misc_intr(int __always_unused irq, void *data) if (oicr & PFINT_OICR_TSYN_TX_M) { ena_mask &= ~PFINT_OICR_TSYN_TX_M; - ice_ptp_process_ts(pf); + if (!hw->reset_ongoing) + ret = IRQ_WAKE_THREAD; } if (oicr & PFINT_OICR_TSYN_EVNT_M) { @@ -3128,7 +3129,8 @@ static irqreturn_t ice_misc_intr(int __always_unused irq, void *data) ice_service_task_schedule(pf); } } - ret = IRQ_HANDLED; + if (!ret) + ret = IRQ_HANDLED; ice_service_task_schedule(pf); ice_irq_dynamic_ena(hw, NULL, NULL); @@ -3137,6 +3139,24 @@ static irqreturn_t ice_misc_intr(int __always_unused irq, void *data) } /** + * ice_misc_intr_thread_fn - misc interrupt thread function + * @irq: interrupt number + * @data: pointer to a q_vector + */ +static irqreturn_t ice_misc_intr_thread_fn(int __always_unused irq, void *data) +{ + irqreturn_t ret = IRQ_HANDLED; + struct ice_pf *pf = data; + bool irq_handled; + + irq_handled = ice_ptp_process_ts(pf); + if (!irq_handled) + ret = IRQ_WAKE_THREAD; + + return ret; +} + +/** * ice_dis_ctrlq_interrupts - disable control queue interrupts * @hw: pointer to HW structure */ @@ -3248,10 +3268,12 @@ static int ice_req_irq_msix_misc(struct ice_pf *pf) pf->num_avail_sw_msix -= 1; pf->oicr_idx = (u16)oicr_idx; - err = devm_request_irq(dev, pf->msix_entries[pf->oicr_idx].vector, - ice_misc_intr, 0, pf->int_name, pf); + err = devm_request_threaded_irq(dev, + pf->msix_entries[pf->oicr_idx].vector, + ice_misc_intr, ice_misc_intr_thread_fn, + 0, pf->int_name, pf); if (err) { - dev_err(dev, "devm_request_irq for %s failed: %d\n", + dev_err(dev, "devm_request_threaded_irq for %s failed: %d\n", pf->int_name, err); ice_free_res(pf->irq_tracker, 1, ICE_RES_MISC_VEC_ID); pf->num_avail_sw_msix += 1; @@ -3288,7 +3310,7 @@ static void ice_napi_add(struct ice_vsi *vsi) ice_for_each_q_vector(vsi, v_idx) netif_napi_add(vsi->netdev, &vsi->q_vectors[v_idx]->napi, - ice_napi_poll, NAPI_POLL_WEIGHT); + ice_napi_poll); } /** @@ -3391,6 +3413,11 @@ static void ice_set_netdev_features(struct net_device *netdev) if (is_dvm_ena) netdev->hw_features |= NETIF_F_HW_VLAN_STAG_RX | NETIF_F_HW_VLAN_STAG_TX; + + /* Leave CRC / FCS stripping enabled by default, but allow the value to + * be changed at runtime + */ + netdev->hw_features |= NETIF_F_RXFCS; } /** @@ -3923,87 +3950,134 @@ static int ice_init_pf(struct ice_pf *pf) } /** + * ice_reduce_msix_usage - Reduce usage of MSI-X vectors + * @pf: board private structure + * @v_remain: number of remaining MSI-X vectors to be distributed + * + * Reduce the usage of MSI-X vectors when entire request cannot be fulfilled. + * pf->num_lan_msix and pf->num_rdma_msix values are set based on number of + * remaining vectors. + */ +static void ice_reduce_msix_usage(struct ice_pf *pf, int v_remain) +{ + int v_rdma; + + if (!ice_is_rdma_ena(pf)) { + pf->num_lan_msix = v_remain; + return; + } + + /* RDMA needs at least 1 interrupt in addition to AEQ MSIX */ + v_rdma = ICE_RDMA_NUM_AEQ_MSIX + 1; + + if (v_remain < ICE_MIN_LAN_TXRX_MSIX + ICE_MIN_RDMA_MSIX) { + dev_warn(ice_pf_to_dev(pf), "Not enough MSI-X vectors to support RDMA.\n"); + clear_bit(ICE_FLAG_RDMA_ENA, pf->flags); + + pf->num_rdma_msix = 0; + pf->num_lan_msix = ICE_MIN_LAN_TXRX_MSIX; + } else if ((v_remain < ICE_MIN_LAN_TXRX_MSIX + v_rdma) || + (v_remain - v_rdma < v_rdma)) { + /* Support minimum RDMA and give remaining vectors to LAN MSIX */ + pf->num_rdma_msix = ICE_MIN_RDMA_MSIX; + pf->num_lan_msix = v_remain - ICE_MIN_RDMA_MSIX; + } else { + /* Split remaining MSIX with RDMA after accounting for AEQ MSIX + */ + pf->num_rdma_msix = (v_remain - ICE_RDMA_NUM_AEQ_MSIX) / 2 + + ICE_RDMA_NUM_AEQ_MSIX; + pf->num_lan_msix = v_remain - pf->num_rdma_msix; + } +} + +/** * ice_ena_msix_range - Request a range of MSIX vectors from the OS * @pf: board private structure * - * compute the number of MSIX vectors required (v_budget) and request from - * the OS. Return the number of vectors reserved or negative on failure + * Compute the number of MSIX vectors wanted and request from the OS. Adjust + * device usage if there are not enough vectors. Return the number of vectors + * reserved or negative on failure. */ static int ice_ena_msix_range(struct ice_pf *pf) { - int num_cpus, v_left, v_actual, v_other, v_budget = 0; + int num_cpus, hw_num_msix, v_other, v_wanted, v_actual; struct device *dev = ice_pf_to_dev(pf); - int needed, err, i; + int err, i; - v_left = pf->hw.func_caps.common_cap.num_msix_vectors; + hw_num_msix = pf->hw.func_caps.common_cap.num_msix_vectors; num_cpus = num_online_cpus(); - /* reserve for LAN miscellaneous handler */ - needed = ICE_MIN_LAN_OICR_MSIX; - if (v_left < needed) - goto no_hw_vecs_left_err; - v_budget += needed; - v_left -= needed; + /* LAN miscellaneous handler */ + v_other = ICE_MIN_LAN_OICR_MSIX; - /* reserve for flow director */ - if (test_bit(ICE_FLAG_FD_ENA, pf->flags)) { - needed = ICE_FDIR_MSIX; - if (v_left < needed) - goto no_hw_vecs_left_err; - v_budget += needed; - v_left -= needed; - } - - /* reserve for switchdev */ - needed = ICE_ESWITCH_MSIX; - if (v_left < needed) - goto no_hw_vecs_left_err; - v_budget += needed; - v_left -= needed; - - /* total used for non-traffic vectors */ - v_other = v_budget; - - /* reserve vectors for LAN traffic */ - needed = num_cpus; - if (v_left < needed) - goto no_hw_vecs_left_err; - pf->num_lan_msix = needed; - v_budget += needed; - v_left -= needed; - - /* reserve vectors for RDMA auxiliary driver */ + /* Flow Director */ + if (test_bit(ICE_FLAG_FD_ENA, pf->flags)) + v_other += ICE_FDIR_MSIX; + + /* switchdev */ + v_other += ICE_ESWITCH_MSIX; + + v_wanted = v_other; + + /* LAN traffic */ + pf->num_lan_msix = num_cpus; + v_wanted += pf->num_lan_msix; + + /* RDMA auxiliary driver */ if (ice_is_rdma_ena(pf)) { - needed = num_cpus + ICE_RDMA_NUM_AEQ_MSIX; - if (v_left < needed) - goto no_hw_vecs_left_err; - pf->num_rdma_msix = needed; - v_budget += needed; - v_left -= needed; + pf->num_rdma_msix = num_cpus + ICE_RDMA_NUM_AEQ_MSIX; + v_wanted += pf->num_rdma_msix; } - pf->msix_entries = devm_kcalloc(dev, v_budget, + if (v_wanted > hw_num_msix) { + int v_remain; + + dev_warn(dev, "not enough device MSI-X vectors. wanted = %d, available = %d\n", + v_wanted, hw_num_msix); + + if (hw_num_msix < ICE_MIN_MSIX) { + err = -ERANGE; + goto exit_err; + } + + v_remain = hw_num_msix - v_other; + if (v_remain < ICE_MIN_LAN_TXRX_MSIX) { + v_other = ICE_MIN_MSIX - ICE_MIN_LAN_TXRX_MSIX; + v_remain = ICE_MIN_LAN_TXRX_MSIX; + } + + ice_reduce_msix_usage(pf, v_remain); + v_wanted = pf->num_lan_msix + pf->num_rdma_msix + v_other; + + dev_notice(dev, "Reducing request to %d MSI-X vectors for LAN traffic.\n", + pf->num_lan_msix); + if (ice_is_rdma_ena(pf)) + dev_notice(dev, "Reducing request to %d MSI-X vectors for RDMA.\n", + pf->num_rdma_msix); + } + + pf->msix_entries = devm_kcalloc(dev, v_wanted, sizeof(*pf->msix_entries), GFP_KERNEL); if (!pf->msix_entries) { err = -ENOMEM; goto exit_err; } - for (i = 0; i < v_budget; i++) + for (i = 0; i < v_wanted; i++) pf->msix_entries[i].entry = i; /* actually reserve the vectors */ v_actual = pci_enable_msix_range(pf->pdev, pf->msix_entries, - ICE_MIN_MSIX, v_budget); + ICE_MIN_MSIX, v_wanted); if (v_actual < 0) { dev_err(dev, "unable to reserve MSI-X vectors\n"); err = v_actual; goto msix_err; } - if (v_actual < v_budget) { + if (v_actual < v_wanted) { dev_warn(dev, "not enough OS MSI-X vectors. requested = %d, obtained = %d\n", - v_budget, v_actual); + v_wanted, v_actual); if (v_actual < ICE_MIN_MSIX) { /* error if we can't get minimum vectors */ @@ -4012,38 +4086,11 @@ static int ice_ena_msix_range(struct ice_pf *pf) goto msix_err; } else { int v_remain = v_actual - v_other; - int v_rdma = 0, v_min_rdma = 0; - if (ice_is_rdma_ena(pf)) { - /* Need at least 1 interrupt in addition to - * AEQ MSIX - */ - v_rdma = ICE_RDMA_NUM_AEQ_MSIX + 1; - v_min_rdma = ICE_MIN_RDMA_MSIX; - } + if (v_remain < ICE_MIN_LAN_TXRX_MSIX) + v_remain = ICE_MIN_LAN_TXRX_MSIX; - if (v_actual == ICE_MIN_MSIX || - v_remain < ICE_MIN_LAN_TXRX_MSIX + v_min_rdma) { - dev_warn(dev, "Not enough MSI-X vectors to support RDMA.\n"); - clear_bit(ICE_FLAG_RDMA_ENA, pf->flags); - - pf->num_rdma_msix = 0; - pf->num_lan_msix = ICE_MIN_LAN_TXRX_MSIX; - } else if ((v_remain < ICE_MIN_LAN_TXRX_MSIX + v_rdma) || - (v_remain - v_rdma < v_rdma)) { - /* Support minimum RDMA and give remaining - * vectors to LAN MSIX - */ - pf->num_rdma_msix = v_min_rdma; - pf->num_lan_msix = v_remain - v_min_rdma; - } else { - /* Split remaining MSIX with RDMA after - * accounting for AEQ MSIX - */ - pf->num_rdma_msix = (v_remain - ICE_RDMA_NUM_AEQ_MSIX) / 2 + - ICE_RDMA_NUM_AEQ_MSIX; - pf->num_lan_msix = v_remain - pf->num_rdma_msix; - } + ice_reduce_msix_usage(pf, v_remain); dev_notice(dev, "Enabled %d MSI-X vectors for LAN traffic.\n", pf->num_lan_msix); @@ -4058,12 +4105,7 @@ static int ice_ena_msix_range(struct ice_pf *pf) msix_err: devm_kfree(dev, pf->msix_entries); - goto exit_err; -no_hw_vecs_left_err: - dev_err(dev, "not enough device MSI-X vectors. requested = %d, available = %d\n", - needed, v_left); - err = -ERANGE; exit_err: pf->num_rdma_msix = 0; pf->num_lan_msix = 0; @@ -4557,6 +4599,10 @@ static int ice_register_netdev(struct ice_pf *pf) if (!vsi || !vsi->netdev) return -EIO; + err = ice_devlink_create_pf_port(pf); + if (err) + goto err_devlink_create; + err = register_netdev(vsi->netdev); if (err) goto err_register_netdev; @@ -4564,17 +4610,13 @@ static int ice_register_netdev(struct ice_pf *pf) set_bit(ICE_VSI_NETDEV_REGISTERED, vsi->state); netif_carrier_off(vsi->netdev); netif_tx_stop_all_queues(vsi->netdev); - err = ice_devlink_create_pf_port(pf); - if (err) - goto err_devlink_create; devlink_port_type_eth_set(&pf->devlink_port, vsi->netdev); return 0; -err_devlink_create: - unregister_netdev(vsi->netdev); - clear_bit(ICE_VSI_NETDEV_REGISTERED, vsi->state); err_register_netdev: + ice_devlink_destroy_pf_port(pf); +err_devlink_create: free_netdev(vsi->netdev); vsi->netdev = NULL; clear_bit(ICE_VSI_NETDEV_ALLOCD, vsi->state); @@ -4682,8 +4724,6 @@ ice_probe(struct pci_dev *pdev, const struct pci_device_id __always_unused *ent) ice_set_safe_mode_caps(hw); } - hw->ucast_shared = true; - err = ice_init_pf(pf); if (err) { dev_err(dev, "ice_init_pf failed: %d\n", err); @@ -5742,6 +5782,9 @@ ice_fdb_del(struct ndmsg *ndm, __always_unused struct nlattr *tb[], NETIF_F_HW_VLAN_STAG_RX | \ NETIF_F_HW_VLAN_STAG_TX) +#define NETIF_VLAN_STRIPPING_FEATURES (NETIF_F_HW_VLAN_CTAG_RX | \ + NETIF_F_HW_VLAN_STAG_RX) + #define NETIF_VLAN_FILTERING_FEATURES (NETIF_F_HW_VLAN_CTAG_FILTER | \ NETIF_F_HW_VLAN_STAG_FILTER) @@ -5828,6 +5871,14 @@ ice_fix_features(struct net_device *netdev, netdev_features_t features) NETIF_F_HW_VLAN_STAG_TX); } + if (!(netdev->features & NETIF_F_RXFCS) && + (features & NETIF_F_RXFCS) && + (features & NETIF_VLAN_STRIPPING_FEATURES) && + !ice_vsi_has_non_zero_vlans(np->vsi)) { + netdev_warn(netdev, "Disabling VLAN stripping as FCS/CRC stripping is also disabled and there is no VLAN configured\n"); + features &= ~NETIF_VLAN_STRIPPING_FEATURES; + } + return features; } @@ -5921,6 +5972,13 @@ ice_set_vlan_features(struct net_device *netdev, netdev_features_t features) current_vlan_features = netdev->features & NETIF_VLAN_OFFLOAD_FEATURES; requested_vlan_features = features & NETIF_VLAN_OFFLOAD_FEATURES; if (current_vlan_features ^ requested_vlan_features) { + if ((features & NETIF_F_RXFCS) && + (features & NETIF_VLAN_STRIPPING_FEATURES)) { + dev_err(ice_pf_to_dev(vsi->back), + "To enable VLAN stripping, you must first enable FCS/CRC stripping\n"); + return -EIO; + } + err = ice_set_vlan_offload_features(vsi, features); if (err) return err; @@ -6002,6 +6060,23 @@ ice_set_features(struct net_device *netdev, netdev_features_t features) if (ret) return ret; + /* Turn on receive of FCS aka CRC, and after setting this + * flag the packet data will have the 4 byte CRC appended + */ + if (changed & NETIF_F_RXFCS) { + if ((features & NETIF_F_RXFCS) && + (features & NETIF_VLAN_STRIPPING_FEATURES)) { + dev_err(ice_pf_to_dev(vsi->back), + "To disable FCS/CRC stripping, you must first disable VLAN stripping\n"); + return -EIO; + } + + ice_vsi_cfg_crc_strip(vsi, !!(features & NETIF_F_RXFCS)); + ret = ice_down_up(vsi); + if (ret) + return ret; + } + if (changed & NETIF_F_NTUPLE) { bool ena = !!(features & NETIF_F_NTUPLE); @@ -6699,6 +6774,31 @@ int ice_down(struct ice_vsi *vsi) } /** + * ice_down_up - shutdown the VSI connection and bring it up + * @vsi: the VSI to be reconnected + */ +int ice_down_up(struct ice_vsi *vsi) +{ + int ret; + + /* if DOWN already set, nothing to do */ + if (test_and_set_bit(ICE_VSI_DOWN, vsi->state)) + return 0; + + ret = ice_down(vsi); + if (ret) + return ret; + + ret = ice_up(vsi); + if (ret) { + netdev_err(vsi->netdev, "reallocating resources failed during netdev features change, may need to reload driver\n"); + return ret; + } + + return 0; +} + +/** * ice_vsi_setup_tx_rings - Allocate VSI Tx queue resources * @vsi: VSI having resources allocated * diff --git a/drivers/net/ethernet/intel/ice/ice_nvm.c b/drivers/net/ethernet/intel/ice/ice_nvm.c index 13cdb5ea594d..c262dc886e6a 100644 --- a/drivers/net/ethernet/intel/ice/ice_nvm.c +++ b/drivers/net/ethernet/intel/ice/ice_nvm.c @@ -1114,14 +1114,18 @@ int ice_nvm_validate_checksum(struct ice_hw *hw) * Update the control word with the required banks' validity bits * and dumps the Shadow RAM to flash (0x0707) * - * cmd_flags controls which banks to activate, and the preservation level to - * use when activating the NVM bank. + * cmd_flags controls which banks to activate, the preservation level to use + * when activating the NVM bank, and whether an EMP reset is required for + * activation. + * + * Note that the 16bit cmd_flags value is split between two separate 1 byte + * flag values in the descriptor. * * On successful return of the firmware command, the response_flags variable * is updated with the flags reported by firmware indicating certain status, * such as whether EMP reset is enabled. */ -int ice_nvm_write_activate(struct ice_hw *hw, u8 cmd_flags, u8 *response_flags) +int ice_nvm_write_activate(struct ice_hw *hw, u16 cmd_flags, u8 *response_flags) { struct ice_aqc_nvm *cmd; struct ice_aq_desc desc; @@ -1130,7 +1134,8 @@ int ice_nvm_write_activate(struct ice_hw *hw, u8 cmd_flags, u8 *response_flags) cmd = &desc.params.nvm; ice_fill_dflt_direct_cmd_desc(&desc, ice_aqc_opc_nvm_write_activate); - cmd->cmd_flags = cmd_flags; + cmd->cmd_flags = (u8)(cmd_flags & 0xFF); + cmd->offset_high = (u8)((cmd_flags >> 8) & 0xFF); err = ice_aq_send_cmd(hw, &desc, NULL, 0, NULL); if (!err && response_flags) diff --git a/drivers/net/ethernet/intel/ice/ice_nvm.h b/drivers/net/ethernet/intel/ice/ice_nvm.h index 856d1ad4398b..774c2317967d 100644 --- a/drivers/net/ethernet/intel/ice/ice_nvm.h +++ b/drivers/net/ethernet/intel/ice/ice_nvm.h @@ -34,7 +34,7 @@ ice_aq_update_nvm(struct ice_hw *hw, u16 module_typeid, u32 offset, int ice_aq_erase_nvm(struct ice_hw *hw, u16 module_typeid, struct ice_sq_cd *cd); int ice_nvm_validate_checksum(struct ice_hw *hw); -int ice_nvm_write_activate(struct ice_hw *hw, u8 cmd_flags, u8 *response_flags); +int ice_nvm_write_activate(struct ice_hw *hw, u16 cmd_flags, u8 *response_flags); int ice_aq_nvm_update_empr(struct ice_hw *hw); int ice_nvm_set_pkg_data(struct ice_hw *hw, bool del_pkg_data_flag, u8 *data, diff --git a/drivers/net/ethernet/intel/ice/ice_protocol_type.h b/drivers/net/ethernet/intel/ice/ice_protocol_type.h index 560efc7654c7..02a4e1cf624e 100644 --- a/drivers/net/ethernet/intel/ice/ice_protocol_type.h +++ b/drivers/net/ethernet/intel/ice/ice_protocol_type.h @@ -44,6 +44,7 @@ enum ice_protocol_type { ICE_GTP, ICE_GTP_NO_PAY, ICE_PPPOE, + ICE_L2TPV3, ICE_VLAN_EX, ICE_VLAN_IN, ICE_VXLAN_GPE, @@ -111,6 +112,7 @@ enum ice_prot_id { #define ICE_UDP_ILOS_HW 53 #define ICE_GRE_OF_HW 64 #define ICE_PPPOE_HW 103 +#define ICE_L2TPV3_HW 104 #define ICE_UDP_OF_HW 52 /* UDP Tunnels */ #define ICE_META_DATA_ID_HW 255 /* this is used for tunnel and VLAN type */ @@ -217,6 +219,11 @@ struct ice_pppoe_hdr { __be16 ppp_prot_id; /* control and data only */ }; +struct ice_l2tpv3_sess_hdr { + __be32 session_id; + __be64 cookie; +}; + struct ice_nvgre_hdr { __be16 flags; __be16 protocol; @@ -235,6 +242,7 @@ union ice_prot_hdr { struct ice_nvgre_hdr nvgre_hdr; struct ice_udp_gtp_hdr gtp_hdr; struct ice_pppoe_hdr pppoe_hdr; + struct ice_l2tpv3_sess_hdr l2tpv3_sess_hdr; }; /* This is mapping table entry that maps every word within a given protocol diff --git a/drivers/net/ethernet/intel/ice/ice_ptp.c b/drivers/net/ethernet/intel/ice/ice_ptp.c index 72b663108a4a..011b727ab190 100644 --- a/drivers/net/ethernet/intel/ice/ice_ptp.c +++ b/drivers/net/ethernet/intel/ice/ice_ptp.c @@ -491,56 +491,6 @@ ice_ptp_read_src_clk_reg(struct ice_pf *pf, struct ptp_system_timestamp *sts) } /** - * ice_ptp_update_cached_phctime - Update the cached PHC time values - * @pf: Board specific private structure - * - * This function updates the system time values which are cached in the PF - * structure and the Rx rings. - * - * This function must be called periodically to ensure that the cached value - * is never more than 2 seconds old. It must also be called whenever the PHC - * time has been changed. - * - * Return: - * * 0 - OK, successfully updated - * * -EAGAIN - PF was busy, need to reschedule the update - */ -static int ice_ptp_update_cached_phctime(struct ice_pf *pf) -{ - u64 systime; - int i; - - if (test_and_set_bit(ICE_CFG_BUSY, pf->state)) - return -EAGAIN; - - /* Read the current PHC time */ - systime = ice_ptp_read_src_clk_reg(pf, NULL); - - /* Update the cached PHC time stored in the PF structure */ - WRITE_ONCE(pf->ptp.cached_phc_time, systime); - - ice_for_each_vsi(pf, i) { - struct ice_vsi *vsi = pf->vsi[i]; - int j; - - if (!vsi) - continue; - - if (vsi->type != ICE_VSI_PF) - continue; - - ice_for_each_rxq(vsi, j) { - if (!vsi->rx_rings[j]) - continue; - WRITE_ONCE(vsi->rx_rings[j]->cached_phctime, systime); - } - } - clear_bit(ICE_CFG_BUSY, pf->state); - - return 0; -} - -/** * ice_ptp_extend_32b_ts - Convert a 32b nanoseconds timestamp to 64b * @cached_phc_time: recently cached copy of PHC time * @in_tstamp: Ingress/egress 32b nanoseconds timestamp value @@ -636,12 +586,400 @@ static u64 ice_ptp_extend_32b_ts(u64 cached_phc_time, u32 in_tstamp) static u64 ice_ptp_extend_40b_ts(struct ice_pf *pf, u64 in_tstamp) { const u64 mask = GENMASK_ULL(31, 0); + unsigned long discard_time; + + /* Discard the hardware timestamp if the cached PHC time is too old */ + discard_time = pf->ptp.cached_phc_jiffies + msecs_to_jiffies(2000); + if (time_is_before_jiffies(discard_time)) { + pf->ptp.tx_hwtstamp_discarded++; + return 0; + } return ice_ptp_extend_32b_ts(pf->ptp.cached_phc_time, (in_tstamp >> 8) & mask); } /** + * ice_ptp_tx_tstamp - Process Tx timestamps for a port + * @tx: the PTP Tx timestamp tracker + * + * Process timestamps captured by the PHY associated with this port. To do + * this, loop over each index with a waiting skb. + * + * If a given index has a valid timestamp, perform the following steps: + * + * 1) copy the timestamp out of the PHY register + * 4) clear the timestamp valid bit in the PHY register + * 5) unlock the index by clearing the associated in_use bit. + * 2) extend the 40b timestamp value to get a 64bit timestamp + * 3) send that timestamp to the stack + * + * After looping, if we still have waiting SKBs, return true. This may cause us + * effectively poll even when not strictly necessary. We do this because it's + * possible a new timestamp was requested around the same time as the interrupt. + * In some cases hardware might not interrupt us again when the timestamp is + * captured. + * + * Note that we only take the tracking lock when clearing the bit and when + * checking if we need to re-queue this task. The only place where bits can be + * set is the hard xmit routine where an SKB has a request flag set. The only + * places where we clear bits are this work function, or the periodic cleanup + * thread. If the cleanup thread clears a bit we're processing we catch it + * when we lock to clear the bit and then grab the SKB pointer. If a Tx thread + * starts a new timestamp, we might not begin processing it right away but we + * will notice it at the end when we re-queue the task. If a Tx thread starts + * a new timestamp just after this function exits without re-queuing, + * the interrupt when the timestamp finishes should trigger. Avoiding holding + * the lock for the entire function is important in order to ensure that Tx + * threads do not get blocked while waiting for the lock. + */ +static bool ice_ptp_tx_tstamp(struct ice_ptp_tx *tx) +{ + struct ice_ptp_port *ptp_port; + bool ts_handled = true; + struct ice_pf *pf; + u8 idx; + + if (!tx->init) + return false; + + ptp_port = container_of(tx, struct ice_ptp_port, tx); + pf = ptp_port_to_pf(ptp_port); + + for_each_set_bit(idx, tx->in_use, tx->len) { + struct skb_shared_hwtstamps shhwtstamps = {}; + u8 phy_idx = idx + tx->quad_offset; + u64 raw_tstamp, tstamp; + struct sk_buff *skb; + int err; + + ice_trace(tx_tstamp_fw_req, tx->tstamps[idx].skb, idx); + + err = ice_read_phy_tstamp(&pf->hw, tx->quad, phy_idx, + &raw_tstamp); + if (err) + continue; + + ice_trace(tx_tstamp_fw_done, tx->tstamps[idx].skb, idx); + + /* Check if the timestamp is invalid or stale */ + if (!(raw_tstamp & ICE_PTP_TS_VALID) || + raw_tstamp == tx->tstamps[idx].cached_tstamp) + continue; + + /* The timestamp is valid, so we'll go ahead and clear this + * index and then send the timestamp up to the stack. + */ + spin_lock(&tx->lock); + tx->tstamps[idx].cached_tstamp = raw_tstamp; + clear_bit(idx, tx->in_use); + skb = tx->tstamps[idx].skb; + tx->tstamps[idx].skb = NULL; + spin_unlock(&tx->lock); + + /* it's (unlikely but) possible we raced with the cleanup + * thread for discarding old timestamp requests. + */ + if (!skb) + continue; + + /* Extend the timestamp using cached PHC time */ + tstamp = ice_ptp_extend_40b_ts(pf, raw_tstamp); + if (tstamp) { + shhwtstamps.hwtstamp = ns_to_ktime(tstamp); + ice_trace(tx_tstamp_complete, skb, idx); + } + + skb_tstamp_tx(skb, &shhwtstamps); + dev_kfree_skb_any(skb); + } + + /* Check if we still have work to do. If so, re-queue this task to + * poll for remaining timestamps. + */ + spin_lock(&tx->lock); + if (!bitmap_empty(tx->in_use, tx->len)) + ts_handled = false; + spin_unlock(&tx->lock); + + return ts_handled; +} + +/** + * ice_ptp_alloc_tx_tracker - Initialize tracking for Tx timestamps + * @tx: Tx tracking structure to initialize + * + * Assumes that the length has already been initialized. Do not call directly, + * use the ice_ptp_init_tx_e822 or ice_ptp_init_tx_e810 instead. + */ +static int +ice_ptp_alloc_tx_tracker(struct ice_ptp_tx *tx) +{ + tx->tstamps = kcalloc(tx->len, sizeof(*tx->tstamps), GFP_KERNEL); + if (!tx->tstamps) + return -ENOMEM; + + tx->in_use = bitmap_zalloc(tx->len, GFP_KERNEL); + if (!tx->in_use) { + kfree(tx->tstamps); + tx->tstamps = NULL; + return -ENOMEM; + } + + spin_lock_init(&tx->lock); + + tx->init = 1; + + return 0; +} + +/** + * ice_ptp_flush_tx_tracker - Flush any remaining timestamps from the tracker + * @pf: Board private structure + * @tx: the tracker to flush + */ +static void +ice_ptp_flush_tx_tracker(struct ice_pf *pf, struct ice_ptp_tx *tx) +{ + u8 idx; + + for (idx = 0; idx < tx->len; idx++) { + u8 phy_idx = idx + tx->quad_offset; + + spin_lock(&tx->lock); + if (tx->tstamps[idx].skb) { + dev_kfree_skb_any(tx->tstamps[idx].skb); + tx->tstamps[idx].skb = NULL; + pf->ptp.tx_hwtstamp_flushed++; + } + clear_bit(idx, tx->in_use); + spin_unlock(&tx->lock); + + /* Clear any potential residual timestamp in the PHY block */ + if (!pf->hw.reset_ongoing) + ice_clear_phy_tstamp(&pf->hw, tx->quad, phy_idx); + } +} + +/** + * ice_ptp_release_tx_tracker - Release allocated memory for Tx tracker + * @pf: Board private structure + * @tx: Tx tracking structure to release + * + * Free memory associated with the Tx timestamp tracker. + */ +static void +ice_ptp_release_tx_tracker(struct ice_pf *pf, struct ice_ptp_tx *tx) +{ + tx->init = 0; + + ice_ptp_flush_tx_tracker(pf, tx); + + kfree(tx->tstamps); + tx->tstamps = NULL; + + bitmap_free(tx->in_use); + tx->in_use = NULL; + + tx->len = 0; +} + +/** + * ice_ptp_init_tx_e822 - Initialize tracking for Tx timestamps + * @pf: Board private structure + * @tx: the Tx tracking structure to initialize + * @port: the port this structure tracks + * + * Initialize the Tx timestamp tracker for this port. For generic MAC devices, + * the timestamp block is shared for all ports in the same quad. To avoid + * ports using the same timestamp index, logically break the block of + * registers into chunks based on the port number. + */ +static int +ice_ptp_init_tx_e822(struct ice_pf *pf, struct ice_ptp_tx *tx, u8 port) +{ + tx->quad = port / ICE_PORTS_PER_QUAD; + tx->quad_offset = (port % ICE_PORTS_PER_QUAD) * INDEX_PER_PORT; + tx->len = INDEX_PER_PORT; + + return ice_ptp_alloc_tx_tracker(tx); +} + +/** + * ice_ptp_init_tx_e810 - Initialize tracking for Tx timestamps + * @pf: Board private structure + * @tx: the Tx tracking structure to initialize + * + * Initialize the Tx timestamp tracker for this PF. For E810 devices, each + * port has its own block of timestamps, independent of the other ports. + */ +static int +ice_ptp_init_tx_e810(struct ice_pf *pf, struct ice_ptp_tx *tx) +{ + tx->quad = pf->hw.port_info->lport; + tx->quad_offset = 0; + tx->len = INDEX_PER_QUAD; + + return ice_ptp_alloc_tx_tracker(tx); +} + +/** + * ice_ptp_tx_tstamp_cleanup - Cleanup old timestamp requests that got dropped + * @pf: pointer to the PF struct + * @tx: PTP Tx tracker to clean up + * + * Loop through the Tx timestamp requests and see if any of them have been + * waiting for a long time. Discard any SKBs that have been waiting for more + * than 2 seconds. This is long enough to be reasonably sure that the + * timestamp will never be captured. This might happen if the packet gets + * discarded before it reaches the PHY timestamping block. + */ +static void ice_ptp_tx_tstamp_cleanup(struct ice_pf *pf, struct ice_ptp_tx *tx) +{ + struct ice_hw *hw = &pf->hw; + u8 idx; + + if (!tx->init) + return; + + for_each_set_bit(idx, tx->in_use, tx->len) { + struct sk_buff *skb; + u64 raw_tstamp; + + /* Check if this SKB has been waiting for too long */ + if (time_is_after_jiffies(tx->tstamps[idx].start + 2 * HZ)) + continue; + + /* Read tstamp to be able to use this register again */ + ice_read_phy_tstamp(hw, tx->quad, idx + tx->quad_offset, + &raw_tstamp); + + spin_lock(&tx->lock); + skb = tx->tstamps[idx].skb; + tx->tstamps[idx].skb = NULL; + clear_bit(idx, tx->in_use); + spin_unlock(&tx->lock); + + /* Count the number of Tx timestamps which have timed out */ + pf->ptp.tx_hwtstamp_timeouts++; + + /* Free the SKB after we've cleared the bit */ + dev_kfree_skb_any(skb); + } +} + +/** + * ice_ptp_update_cached_phctime - Update the cached PHC time values + * @pf: Board specific private structure + * + * This function updates the system time values which are cached in the PF + * structure and the Rx rings. + * + * This function must be called periodically to ensure that the cached value + * is never more than 2 seconds old. + * + * Note that the cached copy in the PF PTP structure is always updated, even + * if we can't update the copy in the Rx rings. + * + * Return: + * * 0 - OK, successfully updated + * * -EAGAIN - PF was busy, need to reschedule the update + */ +static int ice_ptp_update_cached_phctime(struct ice_pf *pf) +{ + struct device *dev = ice_pf_to_dev(pf); + unsigned long update_before; + u64 systime; + int i; + + update_before = pf->ptp.cached_phc_jiffies + msecs_to_jiffies(2000); + if (pf->ptp.cached_phc_time && + time_is_before_jiffies(update_before)) { + unsigned long time_taken = jiffies - pf->ptp.cached_phc_jiffies; + + dev_warn(dev, "%u msecs passed between update to cached PHC time\n", + jiffies_to_msecs(time_taken)); + pf->ptp.late_cached_phc_updates++; + } + + /* Read the current PHC time */ + systime = ice_ptp_read_src_clk_reg(pf, NULL); + + /* Update the cached PHC time stored in the PF structure */ + WRITE_ONCE(pf->ptp.cached_phc_time, systime); + WRITE_ONCE(pf->ptp.cached_phc_jiffies, jiffies); + + if (test_and_set_bit(ICE_CFG_BUSY, pf->state)) + return -EAGAIN; + + ice_for_each_vsi(pf, i) { + struct ice_vsi *vsi = pf->vsi[i]; + int j; + + if (!vsi) + continue; + + if (vsi->type != ICE_VSI_PF) + continue; + + ice_for_each_rxq(vsi, j) { + if (!vsi->rx_rings[j]) + continue; + WRITE_ONCE(vsi->rx_rings[j]->cached_phctime, systime); + } + } + clear_bit(ICE_CFG_BUSY, pf->state); + + return 0; +} + +/** + * ice_ptp_reset_cached_phctime - Reset cached PHC time after an update + * @pf: Board specific private structure + * + * This function must be called when the cached PHC time is no longer valid, + * such as after a time adjustment. It discards any outstanding Tx timestamps, + * and updates the cached PHC time for both the PF and Rx rings. If updating + * the PHC time cannot be done immediately, a warning message is logged and + * the work item is scheduled. + * + * These steps are required in order to ensure that we do not accidentally + * report a timestamp extended by the wrong PHC cached copy. Note that we + * do not directly update the cached timestamp here because it is possible + * this might produce an error when ICE_CFG_BUSY is set. If this occurred, we + * would have to try again. During that time window, timestamps might be + * requested and returned with an invalid extension. Thus, on failure to + * immediately update the cached PHC time we would need to zero the value + * anyways. For this reason, we just zero the value immediately and queue the + * update work item. + */ +static void ice_ptp_reset_cached_phctime(struct ice_pf *pf) +{ + struct device *dev = ice_pf_to_dev(pf); + int err; + + /* Update the cached PHC time immediately if possible, otherwise + * schedule the work item to execute soon. + */ + err = ice_ptp_update_cached_phctime(pf); + if (err) { + /* If another thread is updating the Rx rings, we won't + * properly reset them here. This could lead to reporting of + * invalid timestamps, but there isn't much we can do. + */ + dev_warn(dev, "%s: ICE_CFG_BUSY, unable to immediately update cached PHC time\n", + __func__); + + /* Queue the work item to update the Rx rings when possible */ + kthread_queue_delayed_work(pf->ptp.kworker, &pf->ptp.work, + msecs_to_jiffies(10)); + } + + /* Flush any outstanding Tx timestamps */ + ice_ptp_flush_tx_tracker(pf, &pf->ptp.port.tx); +} + +/** * ice_ptp_read_time - Read the time from the device * @pf: Board private structure * @ts: timespec structure to hold the current time value @@ -900,6 +1238,9 @@ static void ice_ptp_wait_for_offset_valid(struct kthread_work *work) hw = &pf->hw; dev = ice_pf_to_dev(pf); + if (ice_is_reset_in_progress(pf->state)) + return; + if (ice_ptp_check_offset_valid(port)) { /* Offsets not ready yet, try again later */ kthread_queue_delayed_work(pf->ptp.kworker, @@ -1509,7 +1850,7 @@ ice_ptp_settime64(struct ptp_clock_info *info, const struct timespec64 *ts) ice_ptp_unlock(hw); if (!err) - ice_ptp_update_cached_phctime(pf); + ice_ptp_reset_cached_phctime(pf); /* Reenable periodic outputs */ ice_ptp_enable_all_clkout(pf); @@ -1588,7 +1929,7 @@ static int ice_ptp_adjtime(struct ptp_clock_info *info, s64 delta) return err; } - ice_ptp_update_cached_phctime(pf); + ice_ptp_reset_cached_phctime(pf); return 0; } @@ -1796,26 +2137,31 @@ void ice_ptp_rx_hwtstamp(struct ice_rx_ring *rx_ring, union ice_32b_rx_flex_desc *rx_desc, struct sk_buff *skb) { + struct skb_shared_hwtstamps *hwtstamps; + u64 ts_ns, cached_time; u32 ts_high; - u64 ts_ns; - /* Populate timesync data into skb */ - if (rx_desc->wb.time_stamp_low & ICE_PTP_TS_VALID) { - struct skb_shared_hwtstamps *hwtstamps; + if (!(rx_desc->wb.time_stamp_low & ICE_PTP_TS_VALID)) + return; - /* Use ice_ptp_extend_32b_ts directly, using the ring-specific - * cached PHC value, rather than accessing the PF. This also - * allows us to simply pass the upper 32bits of nanoseconds - * directly. Calling ice_ptp_extend_40b_ts is unnecessary as - * it would just discard these bits itself. - */ - ts_high = le32_to_cpu(rx_desc->wb.flex_ts.ts_high); - ts_ns = ice_ptp_extend_32b_ts(rx_ring->cached_phctime, ts_high); + cached_time = READ_ONCE(rx_ring->cached_phctime); - hwtstamps = skb_hwtstamps(skb); - memset(hwtstamps, 0, sizeof(*hwtstamps)); - hwtstamps->hwtstamp = ns_to_ktime(ts_ns); - } + /* Do not report a timestamp if we don't have a cached PHC time */ + if (!cached_time) + return; + + /* Use ice_ptp_extend_32b_ts directly, using the ring-specific cached + * PHC value, rather than accessing the PF. This also allows us to + * simply pass the upper 32bits of nanoseconds directly. Calling + * ice_ptp_extend_40b_ts is unnecessary as it would just discard these + * bits itself. + */ + ts_high = le32_to_cpu(rx_desc->wb.flex_ts.ts_high); + ts_ns = ice_ptp_extend_32b_ts(cached_time, ts_high); + + hwtstamps = skb_hwtstamps(skb); + memset(hwtstamps, 0, sizeof(*hwtstamps)); + hwtstamps->hwtstamp = ns_to_ktime(ts_ns); } /** @@ -1871,49 +2217,26 @@ ice_ptp_setup_sma_pins_e810t(struct ice_pf *pf, struct ptp_clock_info *info) } /** - * ice_ptp_setup_pins_e810t - Setup PTP pins in sysfs + * ice_ptp_setup_pins_e810 - Setup PTP pins in sysfs * @pf: pointer to the PF instance * @info: PTP clock capabilities */ static void -ice_ptp_setup_pins_e810t(struct ice_pf *pf, struct ptp_clock_info *info) +ice_ptp_setup_pins_e810(struct ice_pf *pf, struct ptp_clock_info *info) { - /* Check if SMA controller is in the netlist */ - if (ice_is_feature_supported(pf, ICE_F_SMA_CTRL) && - !ice_is_pca9575_present(&pf->hw)) - ice_clear_feature_support(pf, ICE_F_SMA_CTRL); - - if (!ice_is_feature_supported(pf, ICE_F_SMA_CTRL)) { - info->n_ext_ts = N_EXT_TS_E810_NO_SMA; - info->n_per_out = N_PER_OUT_E810T_NO_SMA; - return; - } + info->n_per_out = N_PER_OUT_E810; - info->n_per_out = N_PER_OUT_E810T; + if (ice_is_feature_supported(pf, ICE_F_PTP_EXTTS)) + info->n_ext_ts = N_EXT_TS_E810; - if (ice_is_feature_supported(pf, ICE_F_PTP_EXTTS)) { + if (ice_is_feature_supported(pf, ICE_F_SMA_CTRL)) { info->n_ext_ts = N_EXT_TS_E810; info->n_pins = NUM_PTP_PINS_E810T; info->verify = ice_verify_pin_e810t; - } - /* Complete setup of the SMA pins */ - ice_ptp_setup_sma_pins_e810t(pf, info); -} - -/** - * ice_ptp_setup_pins_e810 - Setup PTP pins in sysfs - * @pf: pointer to the PF instance - * @info: PTP clock capabilities - */ -static void ice_ptp_setup_pins_e810(struct ice_pf *pf, struct ptp_clock_info *info) -{ - info->n_per_out = N_PER_OUT_E810; - - if (!ice_is_feature_supported(pf, ICE_F_PTP_EXTTS)) - return; - - info->n_ext_ts = N_EXT_TS_E810; + /* Complete setup of the SMA pins */ + ice_ptp_setup_sma_pins_e810t(pf, info); + } } /** @@ -1950,11 +2273,7 @@ static void ice_ptp_set_funcs_e810(struct ice_pf *pf, struct ptp_clock_info *info) { info->enable = ice_ptp_gpio_enable_e810; - - if (ice_is_e810t(&pf->hw)) - ice_ptp_setup_pins_e810t(pf, info); - else - ice_ptp_setup_pins_e810(pf, info); + ice_ptp_setup_pins_e810(pf, info); } /** @@ -2016,112 +2335,6 @@ static long ice_ptp_create_clock(struct ice_pf *pf) } /** - * ice_ptp_tx_tstamp_work - Process Tx timestamps for a port - * @work: pointer to the kthread_work struct - * - * Process timestamps captured by the PHY associated with this port. To do - * this, loop over each index with a waiting skb. - * - * If a given index has a valid timestamp, perform the following steps: - * - * 1) copy the timestamp out of the PHY register - * 4) clear the timestamp valid bit in the PHY register - * 5) unlock the index by clearing the associated in_use bit. - * 2) extend the 40b timestamp value to get a 64bit timestamp - * 3) send that timestamp to the stack - * - * After looping, if we still have waiting SKBs, then re-queue the work. This - * may cause us effectively poll even when not strictly necessary. We do this - * because it's possible a new timestamp was requested around the same time as - * the interrupt. In some cases hardware might not interrupt us again when the - * timestamp is captured. - * - * Note that we only take the tracking lock when clearing the bit and when - * checking if we need to re-queue this task. The only place where bits can be - * set is the hard xmit routine where an SKB has a request flag set. The only - * places where we clear bits are this work function, or the periodic cleanup - * thread. If the cleanup thread clears a bit we're processing we catch it - * when we lock to clear the bit and then grab the SKB pointer. If a Tx thread - * starts a new timestamp, we might not begin processing it right away but we - * will notice it at the end when we re-queue the work item. If a Tx thread - * starts a new timestamp just after this function exits without re-queuing, - * the interrupt when the timestamp finishes should trigger. Avoiding holding - * the lock for the entire function is important in order to ensure that Tx - * threads do not get blocked while waiting for the lock. - */ -static void ice_ptp_tx_tstamp_work(struct kthread_work *work) -{ - struct ice_ptp_port *ptp_port; - struct ice_ptp_tx *tx; - struct ice_pf *pf; - struct ice_hw *hw; - u8 idx; - - tx = container_of(work, struct ice_ptp_tx, work); - if (!tx->init) - return; - - ptp_port = container_of(tx, struct ice_ptp_port, tx); - pf = ptp_port_to_pf(ptp_port); - hw = &pf->hw; - - for_each_set_bit(idx, tx->in_use, tx->len) { - struct skb_shared_hwtstamps shhwtstamps = {}; - u8 phy_idx = idx + tx->quad_offset; - u64 raw_tstamp, tstamp; - struct sk_buff *skb; - int err; - - ice_trace(tx_tstamp_fw_req, tx->tstamps[idx].skb, idx); - - err = ice_read_phy_tstamp(hw, tx->quad, phy_idx, - &raw_tstamp); - if (err) - continue; - - ice_trace(tx_tstamp_fw_done, tx->tstamps[idx].skb, idx); - - /* Check if the timestamp is invalid or stale */ - if (!(raw_tstamp & ICE_PTP_TS_VALID) || - raw_tstamp == tx->tstamps[idx].cached_tstamp) - continue; - - /* The timestamp is valid, so we'll go ahead and clear this - * index and then send the timestamp up to the stack. - */ - spin_lock(&tx->lock); - tx->tstamps[idx].cached_tstamp = raw_tstamp; - clear_bit(idx, tx->in_use); - skb = tx->tstamps[idx].skb; - tx->tstamps[idx].skb = NULL; - spin_unlock(&tx->lock); - - /* it's (unlikely but) possible we raced with the cleanup - * thread for discarding old timestamp requests. - */ - if (!skb) - continue; - - /* Extend the timestamp using cached PHC time */ - tstamp = ice_ptp_extend_40b_ts(pf, raw_tstamp); - shhwtstamps.hwtstamp = ns_to_ktime(tstamp); - - ice_trace(tx_tstamp_complete, skb, idx); - - skb_tstamp_tx(skb, &shhwtstamps); - dev_kfree_skb_any(skb); - } - - /* Check if we still have work to do. If so, re-queue this task to - * poll for remaining timestamps. - */ - spin_lock(&tx->lock); - if (!bitmap_empty(tx->in_use, tx->len)) - kthread_queue_work(pf->ptp.kworker, &tx->work); - spin_unlock(&tx->lock); -} - -/** * ice_ptp_request_ts - Request an available Tx timestamp index * @tx: the PTP Tx timestamp tracker to request from * @skb: the SKB to associate with this timestamp request @@ -2161,177 +2374,17 @@ s8 ice_ptp_request_ts(struct ice_ptp_tx *tx, struct sk_buff *skb) } /** - * ice_ptp_process_ts - Spawn kthread work to handle timestamps + * ice_ptp_process_ts - Process the PTP Tx timestamps * @pf: Board private structure * - * Queue work required to process the PTP Tx timestamps outside of interrupt - * context. + * Returns true if timestamps are processed. */ -void ice_ptp_process_ts(struct ice_pf *pf) +bool ice_ptp_process_ts(struct ice_pf *pf) { if (pf->ptp.port.tx.init) - kthread_queue_work(pf->ptp.kworker, &pf->ptp.port.tx.work); -} - -/** - * ice_ptp_alloc_tx_tracker - Initialize tracking for Tx timestamps - * @tx: Tx tracking structure to initialize - * - * Assumes that the length has already been initialized. Do not call directly, - * use the ice_ptp_init_tx_e822 or ice_ptp_init_tx_e810 instead. - */ -static int -ice_ptp_alloc_tx_tracker(struct ice_ptp_tx *tx) -{ - tx->tstamps = kcalloc(tx->len, sizeof(*tx->tstamps), GFP_KERNEL); - if (!tx->tstamps) - return -ENOMEM; - - tx->in_use = bitmap_zalloc(tx->len, GFP_KERNEL); - if (!tx->in_use) { - kfree(tx->tstamps); - tx->tstamps = NULL; - return -ENOMEM; - } - - spin_lock_init(&tx->lock); - kthread_init_work(&tx->work, ice_ptp_tx_tstamp_work); - - tx->init = 1; - - return 0; -} - -/** - * ice_ptp_flush_tx_tracker - Flush any remaining timestamps from the tracker - * @pf: Board private structure - * @tx: the tracker to flush - */ -static void -ice_ptp_flush_tx_tracker(struct ice_pf *pf, struct ice_ptp_tx *tx) -{ - u8 idx; - - for (idx = 0; idx < tx->len; idx++) { - u8 phy_idx = idx + tx->quad_offset; - - spin_lock(&tx->lock); - if (tx->tstamps[idx].skb) { - dev_kfree_skb_any(tx->tstamps[idx].skb); - tx->tstamps[idx].skb = NULL; - } - clear_bit(idx, tx->in_use); - spin_unlock(&tx->lock); - - /* Clear any potential residual timestamp in the PHY block */ - if (!pf->hw.reset_ongoing) - ice_clear_phy_tstamp(&pf->hw, tx->quad, phy_idx); - } -} - -/** - * ice_ptp_release_tx_tracker - Release allocated memory for Tx tracker - * @pf: Board private structure - * @tx: Tx tracking structure to release - * - * Free memory associated with the Tx timestamp tracker. - */ -static void -ice_ptp_release_tx_tracker(struct ice_pf *pf, struct ice_ptp_tx *tx) -{ - tx->init = 0; - - kthread_cancel_work_sync(&tx->work); - - ice_ptp_flush_tx_tracker(pf, tx); - - kfree(tx->tstamps); - tx->tstamps = NULL; - - bitmap_free(tx->in_use); - tx->in_use = NULL; + return ice_ptp_tx_tstamp(&pf->ptp.port.tx); - tx->len = 0; -} - -/** - * ice_ptp_init_tx_e822 - Initialize tracking for Tx timestamps - * @pf: Board private structure - * @tx: the Tx tracking structure to initialize - * @port: the port this structure tracks - * - * Initialize the Tx timestamp tracker for this port. For generic MAC devices, - * the timestamp block is shared for all ports in the same quad. To avoid - * ports using the same timestamp index, logically break the block of - * registers into chunks based on the port number. - */ -static int -ice_ptp_init_tx_e822(struct ice_pf *pf, struct ice_ptp_tx *tx, u8 port) -{ - tx->quad = port / ICE_PORTS_PER_QUAD; - tx->quad_offset = (port % ICE_PORTS_PER_QUAD) * INDEX_PER_PORT; - tx->len = INDEX_PER_PORT; - - return ice_ptp_alloc_tx_tracker(tx); -} - -/** - * ice_ptp_init_tx_e810 - Initialize tracking for Tx timestamps - * @pf: Board private structure - * @tx: the Tx tracking structure to initialize - * - * Initialize the Tx timestamp tracker for this PF. For E810 devices, each - * port has its own block of timestamps, independent of the other ports. - */ -static int -ice_ptp_init_tx_e810(struct ice_pf *pf, struct ice_ptp_tx *tx) -{ - tx->quad = pf->hw.port_info->lport; - tx->quad_offset = 0; - tx->len = INDEX_PER_QUAD; - - return ice_ptp_alloc_tx_tracker(tx); -} - -/** - * ice_ptp_tx_tstamp_cleanup - Cleanup old timestamp requests that got dropped - * @hw: pointer to the hw struct - * @tx: PTP Tx tracker to clean up - * - * Loop through the Tx timestamp requests and see if any of them have been - * waiting for a long time. Discard any SKBs that have been waiting for more - * than 2 seconds. This is long enough to be reasonably sure that the - * timestamp will never be captured. This might happen if the packet gets - * discarded before it reaches the PHY timestamping block. - */ -static void ice_ptp_tx_tstamp_cleanup(struct ice_hw *hw, struct ice_ptp_tx *tx) -{ - u8 idx; - - if (!tx->init) - return; - - for_each_set_bit(idx, tx->in_use, tx->len) { - struct sk_buff *skb; - u64 raw_tstamp; - - /* Check if this SKB has been waiting for too long */ - if (time_is_after_jiffies(tx->tstamps[idx].start + 2 * HZ)) - continue; - - /* Read tstamp to be able to use this register again */ - ice_read_phy_tstamp(hw, tx->quad, idx + tx->quad_offset, - &raw_tstamp); - - spin_lock(&tx->lock); - skb = tx->tstamps[idx].skb; - tx->tstamps[idx].skb = NULL; - clear_bit(idx, tx->in_use); - spin_unlock(&tx->lock); - - /* Free the SKB after we've cleared the bit */ - dev_kfree_skb_any(skb); - } + return false; } static void ice_ptp_periodic_work(struct kthread_work *work) @@ -2345,7 +2398,7 @@ static void ice_ptp_periodic_work(struct kthread_work *work) err = ice_ptp_update_cached_phctime(pf); - ice_ptp_tx_tstamp_cleanup(&pf->hw, &pf->ptp.port.tx); + ice_ptp_tx_tstamp_cleanup(pf, &pf->ptp.port.tx); /* Run twice a second or reschedule if phc update failed */ kthread_queue_delayed_work(ptp->kworker, &ptp->work, diff --git a/drivers/net/ethernet/intel/ice/ice_ptp.h b/drivers/net/ethernet/intel/ice/ice_ptp.h index 10e396abf130..028349295b71 100644 --- a/drivers/net/ethernet/intel/ice/ice_ptp.h +++ b/drivers/net/ethernet/intel/ice/ice_ptp.h @@ -105,7 +105,6 @@ struct ice_tx_tstamp { /** * struct ice_ptp_tx - Tracking structure for all Tx timestamp requests on a port - * @work: work function to handle processing of Tx timestamps * @lock: lock to prevent concurrent write to in_use bitmap * @tstamps: array of len to store outstanding requests * @in_use: bitmap of len to indicate which slots are in use @@ -117,7 +116,6 @@ struct ice_tx_tstamp { * window, timestamps are temporarily disabled. */ struct ice_ptp_tx { - struct kthread_work work; spinlock_t lock; /* lock protecting in_use bitmap */ struct ice_tx_tstamp *tstamps; unsigned long *in_use; @@ -163,6 +161,7 @@ struct ice_ptp_port { * @work: delayed work function for periodic tasks * @extts_work: work function for handling external Tx timestamps * @cached_phc_time: a cached copy of the PHC time for timestamp extension + * @cached_phc_jiffies: jiffies when cached_phc_time was last updated * @ext_ts_chan: the external timestamp channel in use * @ext_ts_irq: the external timestamp IRQ in use * @kworker: kwork thread for handling periodic work @@ -171,12 +170,19 @@ struct ice_ptp_port { * @clock: pointer to registered PTP clock device * @tstamp_config: hardware timestamping configuration * @reset_time: kernel time after clock stop on reset + * @tx_hwtstamp_skipped: number of Tx time stamp requests skipped + * @tx_hwtstamp_timeouts: number of Tx skbs discarded with no time stamp + * @tx_hwtstamp_flushed: number of Tx skbs flushed due to interface closed + * @tx_hwtstamp_discarded: number of Tx skbs discarded due to cached PHC time + * being too old to correctly extend timestamp + * @late_cached_phc_updates: number of times cached PHC update is late */ struct ice_ptp { struct ice_ptp_port port; struct kthread_delayed_work work; struct kthread_work extts_work; u64 cached_phc_time; + unsigned long cached_phc_jiffies; u8 ext_ts_chan; u8 ext_ts_irq; struct kthread_worker *kworker; @@ -185,6 +191,11 @@ struct ice_ptp { struct ptp_clock *clock; struct hwtstamp_config tstamp_config; u64 reset_time; + u32 tx_hwtstamp_skipped; + u32 tx_hwtstamp_timeouts; + u32 tx_hwtstamp_flushed; + u32 tx_hwtstamp_discarded; + u32 late_cached_phc_updates; }; #define __ptp_port_to_ptp(p) \ @@ -224,8 +235,8 @@ struct ice_ptp { #define N_EXT_TS_E810 3 #define N_PER_OUT_E810 4 #define N_PER_OUT_E810T 3 -#define N_PER_OUT_E810T_NO_SMA 2 -#define N_EXT_TS_E810_NO_SMA 2 +#define N_PER_OUT_NO_SMA_E810T 2 +#define N_EXT_TS_NO_SMA_E810T 2 #define ETH_GLTSYN_ENA(_i) (0x03000348 + ((_i) * 4)) #if IS_ENABLED(CONFIG_PTP_1588_CLOCK) @@ -236,7 +247,7 @@ void ice_ptp_cfg_timestamp(struct ice_pf *pf, bool ena); int ice_get_ptp_clock_index(struct ice_pf *pf); s8 ice_ptp_request_ts(struct ice_ptp_tx *tx, struct sk_buff *skb); -void ice_ptp_process_ts(struct ice_pf *pf); +bool ice_ptp_process_ts(struct ice_pf *pf); void ice_ptp_rx_hwtstamp(struct ice_rx_ring *rx_ring, @@ -269,7 +280,10 @@ ice_ptp_request_ts(struct ice_ptp_tx *tx, struct sk_buff *skb) return -1; } -static inline void ice_ptp_process_ts(struct ice_pf *pf) { } +static inline bool ice_ptp_process_ts(struct ice_pf *pf) +{ + return true; +} static inline void ice_ptp_rx_hwtstamp(struct ice_rx_ring *rx_ring, union ice_32b_rx_flex_desc *rx_desc, struct sk_buff *skb) { } diff --git a/drivers/net/ethernet/intel/ice/ice_ptp_hw.c b/drivers/net/ethernet/intel/ice/ice_ptp_hw.c index 6dff97d53d81..772b1f566d6e 100644 --- a/drivers/net/ethernet/intel/ice/ice_ptp_hw.c +++ b/drivers/net/ethernet/intel/ice/ice_ptp_hw.c @@ -1,6 +1,7 @@ // SPDX-License-Identifier: GPL-2.0 /* Copyright (C) 2021, Intel Corporation. */ +#include <linux/delay.h> #include "ice_common.h" #include "ice_ptp_hw.h" #include "ice_ptp_consts.h" @@ -2587,38 +2588,113 @@ static int ice_write_phy_reg_e810(struct ice_hw *hw, u32 addr, u32 val) } /** - * ice_read_phy_tstamp_e810 - Read a PHY timestamp out of the external PHY + * ice_read_phy_tstamp_ll_e810 - Read a PHY timestamp registers through the FW + * @hw: pointer to the HW struct + * @idx: the timestamp index to read + * @hi: 8 bit timestamp high value + * @lo: 32 bit timestamp low value + * + * Read a 8bit timestamp high value and 32 bit timestamp low value out of the + * timestamp block of the external PHY on the E810 device using the low latency + * timestamp read. + */ +static int +ice_read_phy_tstamp_ll_e810(struct ice_hw *hw, u8 idx, u8 *hi, u32 *lo) +{ + u32 val; + u8 i; + + /* Write TS index to read to the PF register so the FW can read it */ + val = FIELD_PREP(TS_LL_READ_TS_IDX, idx) | TS_LL_READ_TS; + wr32(hw, PF_SB_ATQBAL, val); + + /* Read the register repeatedly until the FW provides us the TS */ + for (i = TS_LL_READ_RETRIES; i > 0; i--) { + val = rd32(hw, PF_SB_ATQBAL); + + /* When the bit is cleared, the TS is ready in the register */ + if (!(FIELD_GET(TS_LL_READ_TS, val))) { + /* High 8 bit value of the TS is on the bits 16:23 */ + *hi = FIELD_GET(TS_LL_READ_TS_HIGH, val); + + /* Read the low 32 bit value and set the TS valid bit */ + *lo = rd32(hw, PF_SB_ATQBAH) | TS_VALID; + return 0; + } + + udelay(10); + } + + /* FW failed to provide the TS in time */ + ice_debug(hw, ICE_DBG_PTP, "Failed to read PTP timestamp using low latency read\n"); + return -EINVAL; +} + +/** + * ice_read_phy_tstamp_sbq_e810 - Read a PHY timestamp registers through the sbq * @hw: pointer to the HW struct * @lport: the lport to read from * @idx: the timestamp index to read - * @tstamp: on return, the 40bit timestamp value + * @hi: 8 bit timestamp high value + * @lo: 32 bit timestamp low value * - * Read a 40bit timestamp value out of the timestamp block of the external PHY - * on the E810 device. + * Read a 8bit timestamp high value and 32 bit timestamp low value out of the + * timestamp block of the external PHY on the E810 device using sideband queue. */ static int -ice_read_phy_tstamp_e810(struct ice_hw *hw, u8 lport, u8 idx, u64 *tstamp) +ice_read_phy_tstamp_sbq_e810(struct ice_hw *hw, u8 lport, u8 idx, u8 *hi, + u32 *lo) { - u32 lo_addr, hi_addr, lo, hi; + u32 hi_addr = TS_EXT(HIGH_TX_MEMORY_BANK_START, lport, idx); + u32 lo_addr = TS_EXT(LOW_TX_MEMORY_BANK_START, lport, idx); + u32 lo_val, hi_val; int err; - lo_addr = TS_EXT(LOW_TX_MEMORY_BANK_START, lport, idx); - hi_addr = TS_EXT(HIGH_TX_MEMORY_BANK_START, lport, idx); - - err = ice_read_phy_reg_e810(hw, lo_addr, &lo); + err = ice_read_phy_reg_e810(hw, lo_addr, &lo_val); if (err) { ice_debug(hw, ICE_DBG_PTP, "Failed to read low PTP timestamp register, err %d\n", err); return err; } - err = ice_read_phy_reg_e810(hw, hi_addr, &hi); + err = ice_read_phy_reg_e810(hw, hi_addr, &hi_val); if (err) { ice_debug(hw, ICE_DBG_PTP, "Failed to read high PTP timestamp register, err %d\n", err); return err; } + *lo = lo_val; + *hi = (u8)hi_val; + + return 0; +} + +/** + * ice_read_phy_tstamp_e810 - Read a PHY timestamp out of the external PHY + * @hw: pointer to the HW struct + * @lport: the lport to read from + * @idx: the timestamp index to read + * @tstamp: on return, the 40bit timestamp value + * + * Read a 40bit timestamp value out of the timestamp block of the external PHY + * on the E810 device. + */ +static int +ice_read_phy_tstamp_e810(struct ice_hw *hw, u8 lport, u8 idx, u64 *tstamp) +{ + u32 lo = 0; + u8 hi = 0; + int err; + + if (hw->dev_caps.ts_dev_info.ts_ll_read) + err = ice_read_phy_tstamp_ll_e810(hw, idx, &hi, &lo); + else + err = ice_read_phy_tstamp_sbq_e810(hw, lport, idx, &hi, &lo); + + if (err) + return err; + /* For E810 devices, the timestamp is reported with the lower 32 bits * in the low register, and the upper 8 bits in the high register. */ diff --git a/drivers/net/ethernet/intel/ice/ice_ptp_hw.h b/drivers/net/ethernet/intel/ice/ice_ptp_hw.h index 1246e4ee4b5d..2bda64c76abc 100644 --- a/drivers/net/ethernet/intel/ice/ice_ptp_hw.h +++ b/drivers/net/ethernet/intel/ice/ice_ptp_hw.h @@ -402,6 +402,7 @@ bool ice_is_pca9575_present(struct ice_hw *hw); #define INCVAL_HIGH_M 0xFF /* Timestamp block macros */ +#define TS_VALID BIT(0) #define TS_LOW_M 0xFFFFFFFF #define TS_HIGH_M 0xFF #define TS_HIGH_S 32 @@ -413,6 +414,12 @@ bool ice_is_pca9575_present(struct ice_hw *hw); #define BYTES_PER_IDX_ADDR_L_U 8 #define BYTES_PER_IDX_ADDR_L 4 +/* Tx timestamp low latency read definitions */ +#define TS_LL_READ_RETRIES 200 +#define TS_LL_READ_TS_HIGH GENMASK(23, 16) +#define TS_LL_READ_TS_IDX GENMASK(29, 24) +#define TS_LL_READ_TS BIT(31) + /* Internal PHY timestamp address */ #define TS_L(a, idx) ((a) + ((idx) * BYTES_PER_IDX_ADDR_L_U)) #define TS_H(a, idx) ((a) + ((idx) * BYTES_PER_IDX_ADDR_L_U + \ diff --git a/drivers/net/ethernet/intel/ice/ice_repr.c b/drivers/net/ethernet/intel/ice/ice_repr.c index 0dac67cd9c77..bd31748aae1b 100644 --- a/drivers/net/ethernet/intel/ice/ice_repr.c +++ b/drivers/net/ethernet/intel/ice/ice_repr.c @@ -377,10 +377,10 @@ static void ice_repr_rem(struct ice_vf *vf) if (!vf->repr) return; - ice_devlink_destroy_vf_port(vf); kfree(vf->repr->q_vector); vf->repr->q_vector = NULL; unregister_netdev(vf->repr->netdev); + ice_devlink_destroy_vf_port(vf); free_netdev(vf->repr->netdev); vf->repr->netdev = NULL; #ifdef CONFIG_ICE_SWITCHDEV diff --git a/drivers/net/ethernet/intel/ice/ice_sched.c b/drivers/net/ethernet/intel/ice/ice_sched.c index 7947223536e3..118595763bba 100644 --- a/drivers/net/ethernet/intel/ice/ice_sched.c +++ b/drivers/net/ethernet/intel/ice/ice_sched.c @@ -1212,7 +1212,7 @@ int ice_sched_init_port(struct ice_port_info *pi) hw = pi->hw; /* Query the Default Topology from FW */ - buf = devm_kzalloc(ice_hw_to_dev(hw), ICE_AQ_MAX_BUF_LEN, GFP_KERNEL); + buf = kzalloc(ICE_AQ_MAX_BUF_LEN, GFP_KERNEL); if (!buf) return -ENOMEM; @@ -1290,7 +1290,7 @@ err_init_port: pi->root = NULL; } - devm_kfree(ice_hw_to_dev(hw), buf); + kfree(buf); return status; } diff --git a/drivers/net/ethernet/intel/ice/ice_switch.c b/drivers/net/ethernet/intel/ice/ice_switch.c index 3808034f7e7e..9b762f7972ce 100644 --- a/drivers/net/ethernet/intel/ice/ice_switch.c +++ b/drivers/net/ethernet/intel/ice/ice_switch.c @@ -42,6 +42,7 @@ enum { ICE_PKT_GTP_NOPAY = BIT(8), ICE_PKT_KMALLOC = BIT(9), ICE_PKT_PPPOE = BIT(10), + ICE_PKT_L2TPV3 = BIT(11), }; struct ice_dummy_pkt_offsets { @@ -1258,6 +1259,65 @@ ICE_DECLARE_PKT_TEMPLATE(pppoe_ipv6_udp) = { 0x00, 0x00, /* 2 bytes for 4 bytes alignment */ }; +ICE_DECLARE_PKT_OFFSETS(ipv4_l2tpv3) = { + { ICE_MAC_OFOS, 0 }, + { ICE_ETYPE_OL, 12 }, + { ICE_IPV4_OFOS, 14 }, + { ICE_L2TPV3, 34 }, + { ICE_PROTOCOL_LAST, 0 }, +}; + +ICE_DECLARE_PKT_TEMPLATE(ipv4_l2tpv3) = { + 0x00, 0x00, 0x00, 0x00, /* ICE_MAC_OFOS 0 */ + 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, + + 0x08, 0x00, /* ICE_ETYPE_OL 12 */ + + 0x45, 0x00, 0x00, 0x20, /* ICE_IPV4_IL 14 */ + 0x00, 0x00, 0x40, 0x00, + 0x40, 0x73, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, + + 0x00, 0x00, 0x00, 0x00, /* ICE_L2TPV3 34 */ + 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, /* 2 bytes for 4 bytes alignment */ +}; + +ICE_DECLARE_PKT_OFFSETS(ipv6_l2tpv3) = { + { ICE_MAC_OFOS, 0 }, + { ICE_ETYPE_OL, 12 }, + { ICE_IPV6_OFOS, 14 }, + { ICE_L2TPV3, 54 }, + { ICE_PROTOCOL_LAST, 0 }, +}; + +ICE_DECLARE_PKT_TEMPLATE(ipv6_l2tpv3) = { + 0x00, 0x00, 0x00, 0x00, /* ICE_MAC_OFOS 0 */ + 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, + + 0x86, 0xDD, /* ICE_ETYPE_OL 12 */ + + 0x60, 0x00, 0x00, 0x00, /* ICE_IPV6_IL 14 */ + 0x00, 0x0c, 0x73, 0x40, + 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, + + 0x00, 0x00, 0x00, 0x00, /* ICE_L2TPV3 54 */ + 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, /* 2 bytes for 4 bytes alignment */ +}; + static const struct ice_dummy_pkt_profile ice_dummy_pkt_profiles[] = { ICE_PKT_PROFILE(ipv6_gtp, ICE_PKT_TUN_GTPU | ICE_PKT_OUTER_IPV6 | ICE_PKT_GTP_NOPAY), @@ -1297,6 +1357,8 @@ static const struct ice_dummy_pkt_profile ice_dummy_pkt_profiles[] = { ICE_PKT_PROFILE(udp_tun_ipv6_tcp, ICE_PKT_TUN_UDP | ICE_PKT_INNER_IPV6 | ICE_PKT_INNER_TCP), + ICE_PKT_PROFILE(ipv6_l2tpv3, ICE_PKT_L2TPV3 | ICE_PKT_OUTER_IPV6), + ICE_PKT_PROFILE(ipv4_l2tpv3, ICE_PKT_L2TPV3), ICE_PKT_PROFILE(udp_tun_tcp, ICE_PKT_TUN_UDP | ICE_PKT_INNER_TCP), ICE_PKT_PROFILE(udp_tun_ipv6_udp, ICE_PKT_TUN_UDP | ICE_PKT_INNER_IPV6), @@ -2274,9 +2336,7 @@ int ice_get_initial_sw_cfg(struct ice_hw *hw) int status; u16 i; - rbuf = devm_kzalloc(ice_hw_to_dev(hw), ICE_SW_CFG_MAX_BUF_LEN, - GFP_KERNEL); - + rbuf = kzalloc(ICE_SW_CFG_MAX_BUF_LEN, GFP_KERNEL); if (!rbuf) return -ENOMEM; @@ -2324,7 +2384,7 @@ int ice_get_initial_sw_cfg(struct ice_hw *hw) } } while (req_desc && !status); - devm_kfree(ice_hw_to_dev(hw), rbuf); + kfree(rbuf); return status; } @@ -3449,31 +3509,15 @@ bool ice_vlan_fltr_exist(struct ice_hw *hw, u16 vlan_id, u16 vsi_handle) * ice_add_mac - Add a MAC address based filter rule * @hw: pointer to the hardware structure * @m_list: list of MAC addresses and forwarding information - * - * IMPORTANT: When the ucast_shared flag is set to false and m_list has - * multiple unicast addresses, the function assumes that all the - * addresses are unique in a given add_mac call. It doesn't - * check for duplicates in this case, removing duplicates from a given - * list should be taken care of in the caller of this function. */ int ice_add_mac(struct ice_hw *hw, struct list_head *m_list) { - struct ice_sw_rule_lkup_rx_tx *s_rule, *r_iter; struct ice_fltr_list_entry *m_list_itr; - struct list_head *rule_head; - u16 total_elem_left, s_rule_size; - struct ice_switch_info *sw; - struct mutex *rule_lock; /* Lock to protect filter rule list */ - u16 num_unicast = 0; int status = 0; - u8 elem_sent; if (!m_list || !hw) return -EINVAL; - s_rule = NULL; - sw = hw->switch_info; - rule_lock = &sw->recp_list[ICE_SW_LKUP_MAC].filt_rule_lock; list_for_each_entry(m_list_itr, m_list, list_entry) { u8 *add = &m_list_itr->fltr_info.l_data.mac.mac_addr[0]; u16 vsi_handle; @@ -3492,106 +3536,13 @@ int ice_add_mac(struct ice_hw *hw, struct list_head *m_list) if (m_list_itr->fltr_info.lkup_type != ICE_SW_LKUP_MAC || is_zero_ether_addr(add)) return -EINVAL; - if (is_unicast_ether_addr(add) && !hw->ucast_shared) { - /* Don't overwrite the unicast address */ - mutex_lock(rule_lock); - if (ice_find_rule_entry(hw, ICE_SW_LKUP_MAC, - &m_list_itr->fltr_info)) { - mutex_unlock(rule_lock); - return -EEXIST; - } - mutex_unlock(rule_lock); - num_unicast++; - } else if (is_multicast_ether_addr(add) || - (is_unicast_ether_addr(add) && hw->ucast_shared)) { - m_list_itr->status = - ice_add_rule_internal(hw, ICE_SW_LKUP_MAC, - m_list_itr); - if (m_list_itr->status) - return m_list_itr->status; - } - } - mutex_lock(rule_lock); - /* Exit if no suitable entries were found for adding bulk switch rule */ - if (!num_unicast) { - status = 0; - goto ice_add_mac_exit; - } - - rule_head = &sw->recp_list[ICE_SW_LKUP_MAC].filt_rules; - - /* Allocate switch rule buffer for the bulk update for unicast */ - s_rule_size = ICE_SW_RULE_RX_TX_ETH_HDR_SIZE(s_rule); - s_rule = devm_kcalloc(ice_hw_to_dev(hw), num_unicast, s_rule_size, - GFP_KERNEL); - if (!s_rule) { - status = -ENOMEM; - goto ice_add_mac_exit; - } - - r_iter = s_rule; - list_for_each_entry(m_list_itr, m_list, list_entry) { - struct ice_fltr_info *f_info = &m_list_itr->fltr_info; - u8 *mac_addr = &f_info->l_data.mac.mac_addr[0]; - - if (is_unicast_ether_addr(mac_addr)) { - ice_fill_sw_rule(hw, &m_list_itr->fltr_info, r_iter, - ice_aqc_opc_add_sw_rules); - r_iter = (typeof(s_rule))((u8 *)r_iter + s_rule_size); - } - } - - /* Call AQ bulk switch rule update for all unicast addresses */ - r_iter = s_rule; - /* Call AQ switch rule in AQ_MAX chunk */ - for (total_elem_left = num_unicast; total_elem_left > 0; - total_elem_left -= elem_sent) { - struct ice_sw_rule_lkup_rx_tx *entry = r_iter; - - elem_sent = min_t(u8, total_elem_left, - (ICE_AQ_MAX_BUF_LEN / s_rule_size)); - status = ice_aq_sw_rules(hw, entry, elem_sent * s_rule_size, - elem_sent, ice_aqc_opc_add_sw_rules, - NULL); - if (status) - goto ice_add_mac_exit; - r_iter = (typeof(s_rule)) - ((u8 *)r_iter + (elem_sent * s_rule_size)); + m_list_itr->status = ice_add_rule_internal(hw, ICE_SW_LKUP_MAC, + m_list_itr); + if (m_list_itr->status) + return m_list_itr->status; } - /* Fill up rule ID based on the value returned from FW */ - r_iter = s_rule; - list_for_each_entry(m_list_itr, m_list, list_entry) { - struct ice_fltr_info *f_info = &m_list_itr->fltr_info; - u8 *mac_addr = &f_info->l_data.mac.mac_addr[0]; - struct ice_fltr_mgmt_list_entry *fm_entry; - - if (is_unicast_ether_addr(mac_addr)) { - f_info->fltr_rule_id = le16_to_cpu(r_iter->index); - f_info->fltr_act = ICE_FWD_TO_VSI; - /* Create an entry to track this MAC address */ - fm_entry = devm_kzalloc(ice_hw_to_dev(hw), - sizeof(*fm_entry), GFP_KERNEL); - if (!fm_entry) { - status = -ENOMEM; - goto ice_add_mac_exit; - } - fm_entry->fltr_info = *f_info; - fm_entry->vsi_count = 1; - /* The book keeping entries will get removed when - * base driver calls remove filter AQ command - */ - - list_add(&fm_entry->list_entry, rule_head); - r_iter = (typeof(s_rule))((u8 *)r_iter + s_rule_size); - } - } - -ice_add_mac_exit: - mutex_unlock(rule_lock); - if (s_rule) - devm_kfree(ice_hw_to_dev(hw), s_rule); return status; } @@ -3979,38 +3930,6 @@ ice_check_if_dflt_vsi(struct ice_port_info *pi, u16 vsi_handle, } /** - * ice_find_ucast_rule_entry - Search for a unicast MAC filter rule entry - * @hw: pointer to the hardware structure - * @recp_id: lookup type for which the specified rule needs to be searched - * @f_info: rule information - * - * Helper function to search for a unicast rule entry - this is to be used - * to remove unicast MAC filter that is not shared with other VSIs on the - * PF switch. - * - * Returns pointer to entry storing the rule if found - */ -static struct ice_fltr_mgmt_list_entry * -ice_find_ucast_rule_entry(struct ice_hw *hw, u8 recp_id, - struct ice_fltr_info *f_info) -{ - struct ice_switch_info *sw = hw->switch_info; - struct ice_fltr_mgmt_list_entry *list_itr; - struct list_head *list_head; - - list_head = &sw->recp_list[recp_id].filt_rules; - list_for_each_entry(list_itr, list_head, list_entry) { - if (!memcmp(&f_info->l_data, &list_itr->fltr_info.l_data, - sizeof(f_info->l_data)) && - f_info->fwd_id.hw_vsi_id == - list_itr->fltr_info.fwd_id.hw_vsi_id && - f_info->flag == list_itr->fltr_info.flag) - return list_itr; - } - return NULL; -} - -/** * ice_remove_mac - remove a MAC address based filter rule * @hw: pointer to the hardware structure * @m_list: list of MAC addresses and forwarding information @@ -4026,15 +3945,12 @@ ice_find_ucast_rule_entry(struct ice_hw *hw, u8 recp_id, int ice_remove_mac(struct ice_hw *hw, struct list_head *m_list) { struct ice_fltr_list_entry *list_itr, *tmp; - struct mutex *rule_lock; /* Lock to protect filter rule list */ if (!m_list) return -EINVAL; - rule_lock = &hw->switch_info->recp_list[ICE_SW_LKUP_MAC].filt_rule_lock; list_for_each_entry_safe(list_itr, tmp, m_list, list_entry) { enum ice_sw_lkup_type l_type = list_itr->fltr_info.lkup_type; - u8 *add = &list_itr->fltr_info.l_data.mac.mac_addr[0]; u16 vsi_handle; if (l_type != ICE_SW_LKUP_MAC) @@ -4046,19 +3962,7 @@ int ice_remove_mac(struct ice_hw *hw, struct list_head *m_list) list_itr->fltr_info.fwd_id.hw_vsi_id = ice_get_hw_vsi_num(hw, vsi_handle); - if (is_unicast_ether_addr(add) && !hw->ucast_shared) { - /* Don't remove the unicast address that belongs to - * another VSI on the switch, since it is not being - * shared... - */ - mutex_lock(rule_lock); - if (!ice_find_ucast_rule_entry(hw, ICE_SW_LKUP_MAC, - &list_itr->fltr_info)) { - mutex_unlock(rule_lock); - return -ENOENT; - } - mutex_unlock(rule_lock); - } + list_itr->status = ice_remove_rule_internal(hw, ICE_SW_LKUP_MAC, list_itr); @@ -4648,6 +4552,7 @@ static const struct ice_prot_ext_tbl_entry ice_prot_ext[ICE_PROTOCOL_LAST] = { { ICE_GTP, { 8, 10, 12, 14, 16, 18, 20, 22 } }, { ICE_GTP_NO_PAY, { 8, 10, 12, 14 } }, { ICE_PPPOE, { 0, 2, 4, 6 } }, + { ICE_L2TPV3, { 0, 2, 4, 6, 8, 10 } }, { ICE_VLAN_EX, { 2, 0 } }, { ICE_VLAN_IN, { 2, 0 } }, }; @@ -4671,6 +4576,7 @@ static struct ice_protocol_entry ice_prot_id_tbl[ICE_PROTOCOL_LAST] = { { ICE_GTP, ICE_UDP_OF_HW }, { ICE_GTP_NO_PAY, ICE_UDP_ILOS_HW }, { ICE_PPPOE, ICE_PPPOE_HW }, + { ICE_L2TPV3, ICE_L2TPV3_HW }, { ICE_VLAN_EX, ICE_VLAN_OF_HW }, { ICE_VLAN_IN, ICE_VLAN_OL_HW }, }; @@ -5754,7 +5660,8 @@ ice_find_dummy_packet(struct ice_adv_lkup_elem *lkups, u16 lkups_cnt, if (lkups[i].h_u.pppoe_hdr.ppp_prot_id == htons(PPP_IPV6)) match |= ICE_PKT_OUTER_IPV6; - } + } else if (lkups[i].type == ICE_L2TPV3) + match |= ICE_PKT_L2TPV3; } while (ret->match && (match & ret->match) != ret->match) @@ -5855,6 +5762,9 @@ ice_fill_adv_dummy_packet(struct ice_adv_lkup_elem *lkups, u16 lkups_cnt, case ICE_PPPOE: len = sizeof(struct ice_pppoe_hdr); break; + case ICE_L2TPV3: + len = sizeof(struct ice_l2tpv3_sess_hdr); + break; default: return -EINVAL; } diff --git a/drivers/net/ethernet/intel/ice/ice_tc_lib.c b/drivers/net/ethernet/intel/ice/ice_tc_lib.c index a298862857a8..f68c555be4e9 100644 --- a/drivers/net/ethernet/intel/ice/ice_tc_lib.c +++ b/drivers/net/ethernet/intel/ice/ice_tc_lib.c @@ -36,6 +36,10 @@ ice_tc_count_lkups(u32 flags, struct ice_tc_flower_lyr_2_4_hdrs *headers, ICE_TC_FLWR_FIELD_ENC_DEST_IPV6)) lkups_cnt++; + if (flags & (ICE_TC_FLWR_FIELD_ENC_IP_TOS | + ICE_TC_FLWR_FIELD_ENC_IP_TTL)) + lkups_cnt++; + if (flags & ICE_TC_FLWR_FIELD_ENC_DEST_L4_PORT) lkups_cnt++; @@ -47,11 +51,11 @@ ice_tc_count_lkups(u32 flags, struct ice_tc_flower_lyr_2_4_hdrs *headers, lkups_cnt++; /* is VLAN specified? */ - if (flags & ICE_TC_FLWR_FIELD_VLAN) + if (flags & (ICE_TC_FLWR_FIELD_VLAN | ICE_TC_FLWR_FIELD_VLAN_PRIO)) lkups_cnt++; /* is CVLAN specified? */ - if (flags & ICE_TC_FLWR_FIELD_CVLAN) + if (flags & (ICE_TC_FLWR_FIELD_CVLAN | ICE_TC_FLWR_FIELD_CVLAN_PRIO)) lkups_cnt++; /* are PPPoE options specified? */ @@ -64,6 +68,13 @@ ice_tc_count_lkups(u32 flags, struct ice_tc_flower_lyr_2_4_hdrs *headers, ICE_TC_FLWR_FIELD_DEST_IPV6 | ICE_TC_FLWR_FIELD_SRC_IPV6)) lkups_cnt++; + if (flags & (ICE_TC_FLWR_FIELD_IP_TOS | ICE_TC_FLWR_FIELD_IP_TTL)) + lkups_cnt++; + + /* are L2TPv3 options specified? */ + if (flags & ICE_TC_FLWR_FIELD_L2TPV3_SESSID) + lkups_cnt++; + /* is L4 (TCP/UDP/any other L4 protocol fields) specified? */ if (flags & (ICE_TC_FLWR_FIELD_DEST_L4_PORT | ICE_TC_FLWR_FIELD_SRC_L4_PORT)) @@ -257,6 +268,50 @@ ice_tc_fill_tunnel_outer(u32 flags, struct ice_tc_flower_fltr *fltr, i++; } + if (fltr->inner_headers.l2_key.n_proto == htons(ETH_P_IP) && + (flags & (ICE_TC_FLWR_FIELD_ENC_IP_TOS | + ICE_TC_FLWR_FIELD_ENC_IP_TTL))) { + list[i].type = ice_proto_type_from_ipv4(false); + + if (flags & ICE_TC_FLWR_FIELD_ENC_IP_TOS) { + list[i].h_u.ipv4_hdr.tos = hdr->l3_key.tos; + list[i].m_u.ipv4_hdr.tos = hdr->l3_mask.tos; + } + + if (flags & ICE_TC_FLWR_FIELD_ENC_IP_TTL) { + list[i].h_u.ipv4_hdr.time_to_live = hdr->l3_key.ttl; + list[i].m_u.ipv4_hdr.time_to_live = hdr->l3_mask.ttl; + } + + i++; + } + + if (fltr->inner_headers.l2_key.n_proto == htons(ETH_P_IPV6) && + (flags & (ICE_TC_FLWR_FIELD_ENC_IP_TOS | + ICE_TC_FLWR_FIELD_ENC_IP_TTL))) { + struct ice_ipv6_hdr *hdr_h, *hdr_m; + + hdr_h = &list[i].h_u.ipv6_hdr; + hdr_m = &list[i].m_u.ipv6_hdr; + list[i].type = ice_proto_type_from_ipv6(false); + + if (flags & ICE_TC_FLWR_FIELD_ENC_IP_TOS) { + be32p_replace_bits(&hdr_h->be_ver_tc_flow, + hdr->l3_key.tos, + ICE_IPV6_HDR_TC_MASK); + be32p_replace_bits(&hdr_m->be_ver_tc_flow, + hdr->l3_mask.tos, + ICE_IPV6_HDR_TC_MASK); + } + + if (flags & ICE_TC_FLWR_FIELD_ENC_IP_TTL) { + hdr_h->hop_limit = hdr->l3_key.ttl; + hdr_m->hop_limit = hdr->l3_mask.ttl; + } + + i++; + } + if ((flags & ICE_TC_FLWR_FIELD_ENC_DEST_L4_PORT) && hdr->l3_key.ip_proto == IPPROTO_UDP) { list[i].type = ICE_UDP_OF; @@ -334,7 +389,7 @@ ice_tc_fill_rules(struct ice_hw *hw, u32 flags, } /* copy VLAN info */ - if (flags & ICE_TC_FLWR_FIELD_VLAN) { + if (flags & (ICE_TC_FLWR_FIELD_VLAN | ICE_TC_FLWR_FIELD_VLAN_PRIO)) { vlan_tpid = be16_to_cpu(headers->vlan_hdr.vlan_tpid); rule_info->vlan_type = ice_check_supported_vlan_tpid(vlan_tpid); @@ -343,15 +398,45 @@ ice_tc_fill_rules(struct ice_hw *hw, u32 flags, list[i].type = ICE_VLAN_EX; else list[i].type = ICE_VLAN_OFOS; - list[i].h_u.vlan_hdr.vlan = headers->vlan_hdr.vlan_id; - list[i].m_u.vlan_hdr.vlan = cpu_to_be16(0xFFFF); + + if (flags & ICE_TC_FLWR_FIELD_VLAN) { + list[i].h_u.vlan_hdr.vlan = headers->vlan_hdr.vlan_id; + list[i].m_u.vlan_hdr.vlan = cpu_to_be16(0x0FFF); + } + + if (flags & ICE_TC_FLWR_FIELD_VLAN_PRIO) { + if (flags & ICE_TC_FLWR_FIELD_VLAN) { + list[i].m_u.vlan_hdr.vlan = cpu_to_be16(0xEFFF); + } else { + list[i].m_u.vlan_hdr.vlan = cpu_to_be16(0xE000); + list[i].h_u.vlan_hdr.vlan = 0; + } + list[i].h_u.vlan_hdr.vlan |= + headers->vlan_hdr.vlan_prio; + } + i++; } - if (flags & ICE_TC_FLWR_FIELD_CVLAN) { + if (flags & (ICE_TC_FLWR_FIELD_CVLAN | ICE_TC_FLWR_FIELD_CVLAN_PRIO)) { list[i].type = ICE_VLAN_IN; - list[i].h_u.vlan_hdr.vlan = headers->cvlan_hdr.vlan_id; - list[i].m_u.vlan_hdr.vlan = cpu_to_be16(0xFFFF); + + if (flags & ICE_TC_FLWR_FIELD_CVLAN) { + list[i].h_u.vlan_hdr.vlan = headers->cvlan_hdr.vlan_id; + list[i].m_u.vlan_hdr.vlan = cpu_to_be16(0x0FFF); + } + + if (flags & ICE_TC_FLWR_FIELD_CVLAN_PRIO) { + if (flags & ICE_TC_FLWR_FIELD_CVLAN) { + list[i].m_u.vlan_hdr.vlan = cpu_to_be16(0xEFFF); + } else { + list[i].m_u.vlan_hdr.vlan = cpu_to_be16(0xE000); + list[i].h_u.vlan_hdr.vlan = 0; + } + list[i].h_u.vlan_hdr.vlan |= + headers->cvlan_hdr.vlan_prio; + } + i++; } @@ -420,6 +505,61 @@ ice_tc_fill_rules(struct ice_hw *hw, u32 flags, i++; } + if (headers->l2_key.n_proto == htons(ETH_P_IP) && + (flags & (ICE_TC_FLWR_FIELD_IP_TOS | ICE_TC_FLWR_FIELD_IP_TTL))) { + list[i].type = ice_proto_type_from_ipv4(inner); + + if (flags & ICE_TC_FLWR_FIELD_IP_TOS) { + list[i].h_u.ipv4_hdr.tos = headers->l3_key.tos; + list[i].m_u.ipv4_hdr.tos = headers->l3_mask.tos; + } + + if (flags & ICE_TC_FLWR_FIELD_IP_TTL) { + list[i].h_u.ipv4_hdr.time_to_live = + headers->l3_key.ttl; + list[i].m_u.ipv4_hdr.time_to_live = + headers->l3_mask.ttl; + } + + i++; + } + + if (headers->l2_key.n_proto == htons(ETH_P_IPV6) && + (flags & (ICE_TC_FLWR_FIELD_IP_TOS | ICE_TC_FLWR_FIELD_IP_TTL))) { + struct ice_ipv6_hdr *hdr_h, *hdr_m; + + hdr_h = &list[i].h_u.ipv6_hdr; + hdr_m = &list[i].m_u.ipv6_hdr; + list[i].type = ice_proto_type_from_ipv6(inner); + + if (flags & ICE_TC_FLWR_FIELD_IP_TOS) { + be32p_replace_bits(&hdr_h->be_ver_tc_flow, + headers->l3_key.tos, + ICE_IPV6_HDR_TC_MASK); + be32p_replace_bits(&hdr_m->be_ver_tc_flow, + headers->l3_mask.tos, + ICE_IPV6_HDR_TC_MASK); + } + + if (flags & ICE_TC_FLWR_FIELD_IP_TTL) { + hdr_h->hop_limit = headers->l3_key.ttl; + hdr_m->hop_limit = headers->l3_mask.ttl; + } + + i++; + } + + if (flags & ICE_TC_FLWR_FIELD_L2TPV3_SESSID) { + list[i].type = ICE_L2TPV3; + + list[i].h_u.l2tpv3_sess_hdr.session_id = + headers->l2tpv3_hdr.session_id; + list[i].m_u.l2tpv3_sess_hdr.session_id = + cpu_to_be32(0xFFFFFFFF); + + i++; + } + /* copy L4 (src, dest) port */ if (flags & (ICE_TC_FLWR_FIELD_DEST_L4_PORT | ICE_TC_FLWR_FIELD_SRC_L4_PORT)) { @@ -839,6 +979,40 @@ ice_tc_set_ipv6(struct flow_match_ipv6_addrs *match, } /** + * ice_tc_set_tos_ttl - Parse IP ToS/TTL from TC flower filter + * @match: Pointer to flow match structure + * @fltr: Pointer to filter structure + * @headers: inner or outer header fields + * @is_encap: set true for tunnel + */ +static void +ice_tc_set_tos_ttl(struct flow_match_ip *match, + struct ice_tc_flower_fltr *fltr, + struct ice_tc_flower_lyr_2_4_hdrs *headers, + bool is_encap) +{ + if (match->mask->tos) { + if (is_encap) + fltr->flags |= ICE_TC_FLWR_FIELD_ENC_IP_TOS; + else + fltr->flags |= ICE_TC_FLWR_FIELD_IP_TOS; + + headers->l3_key.tos = match->key->tos; + headers->l3_mask.tos = match->mask->tos; + } + + if (match->mask->ttl) { + if (is_encap) + fltr->flags |= ICE_TC_FLWR_FIELD_ENC_IP_TTL; + else + fltr->flags |= ICE_TC_FLWR_FIELD_IP_TTL; + + headers->l3_key.ttl = match->key->ttl; + headers->l3_mask.ttl = match->mask->ttl; + } +} + +/** * ice_tc_set_port - Parse ports from TC flower filter * @match: Flow match structure * @fltr: Pointer to filter structure @@ -967,10 +1141,7 @@ ice_parse_tunnel_attr(struct net_device *dev, struct flow_rule *rule, struct flow_match_ip match; flow_rule_match_enc_ip(rule, &match); - headers->l3_key.tos = match.key->tos; - headers->l3_key.ttl = match.key->ttl; - headers->l3_mask.tos = match.mask->tos; - headers->l3_mask.ttl = match.mask->ttl; + ice_tc_set_tos_ttl(&match, fltr, headers, true); } if (flow_rule_match_key(rule, FLOW_DISSECTOR_KEY_ENC_PORTS) && @@ -1039,9 +1210,11 @@ ice_parse_cls_flower(struct net_device *filter_dev, struct ice_vsi *vsi, BIT(FLOW_DISSECTOR_KEY_ENC_IPV6_ADDRS) | BIT(FLOW_DISSECTOR_KEY_ENC_PORTS) | BIT(FLOW_DISSECTOR_KEY_ENC_OPTS) | + BIT(FLOW_DISSECTOR_KEY_IP) | BIT(FLOW_DISSECTOR_KEY_ENC_IP) | BIT(FLOW_DISSECTOR_KEY_PORTS) | - BIT(FLOW_DISSECTOR_KEY_PPPOE))) { + BIT(FLOW_DISSECTOR_KEY_PPPOE) | + BIT(FLOW_DISSECTOR_KEY_L2TPV3))) { NL_SET_ERR_MSG_MOD(fltr->extack, "Unsupported key used"); return -EOPNOTSUPP; } @@ -1137,16 +1310,22 @@ ice_parse_cls_flower(struct net_device *filter_dev, struct ice_vsi *vsi, if (match.mask->vlan_id) { if (match.mask->vlan_id == VLAN_VID_MASK) { fltr->flags |= ICE_TC_FLWR_FIELD_VLAN; + headers->vlan_hdr.vlan_id = + cpu_to_be16(match.key->vlan_id & + VLAN_VID_MASK); } else { NL_SET_ERR_MSG_MOD(fltr->extack, "Bad VLAN mask"); return -EINVAL; } } - headers->vlan_hdr.vlan_id = - cpu_to_be16(match.key->vlan_id & VLAN_VID_MASK); - if (match.mask->vlan_priority) - headers->vlan_hdr.vlan_prio = match.key->vlan_priority; + if (match.mask->vlan_priority) { + fltr->flags |= ICE_TC_FLWR_FIELD_VLAN_PRIO; + headers->vlan_hdr.vlan_prio = + cpu_to_be16((match.key->vlan_priority << + VLAN_PRIO_SHIFT) & VLAN_PRIO_MASK); + } + if (match.mask->vlan_tpid) headers->vlan_hdr.vlan_tpid = match.key->vlan_tpid; } @@ -1164,6 +1343,9 @@ ice_parse_cls_flower(struct net_device *filter_dev, struct ice_vsi *vsi, if (match.mask->vlan_id) { if (match.mask->vlan_id == VLAN_VID_MASK) { fltr->flags |= ICE_TC_FLWR_FIELD_CVLAN; + headers->cvlan_hdr.vlan_id = + cpu_to_be16(match.key->vlan_id & + VLAN_VID_MASK); } else { NL_SET_ERR_MSG_MOD(fltr->extack, "Bad CVLAN mask"); @@ -1171,10 +1353,12 @@ ice_parse_cls_flower(struct net_device *filter_dev, struct ice_vsi *vsi, } } - headers->cvlan_hdr.vlan_id = - cpu_to_be16(match.key->vlan_id & VLAN_VID_MASK); - if (match.mask->vlan_priority) - headers->cvlan_hdr.vlan_prio = match.key->vlan_priority; + if (match.mask->vlan_priority) { + fltr->flags |= ICE_TC_FLWR_FIELD_CVLAN_PRIO; + headers->cvlan_hdr.vlan_prio = + cpu_to_be16((match.key->vlan_priority << + VLAN_PRIO_SHIFT) & VLAN_PRIO_MASK); + } } if (flow_rule_match_key(rule, FLOW_DISSECTOR_KEY_PPPOE)) { @@ -1217,6 +1401,22 @@ ice_parse_cls_flower(struct net_device *filter_dev, struct ice_vsi *vsi, return -EINVAL; } + if (flow_rule_match_key(rule, FLOW_DISSECTOR_KEY_IP)) { + struct flow_match_ip match; + + flow_rule_match_ip(rule, &match); + ice_tc_set_tos_ttl(&match, fltr, headers, false); + } + + if (flow_rule_match_key(rule, FLOW_DISSECTOR_KEY_L2TPV3)) { + struct flow_match_l2tpv3 match; + + flow_rule_match_l2tpv3(rule, &match); + + fltr->flags |= ICE_TC_FLWR_FIELD_L2TPV3_SESSID; + headers->l2tpv3_hdr.session_id = match.key->session_id; + } + if (flow_rule_match_key(rule, FLOW_DISSECTOR_KEY_PORTS)) { struct flow_match_ports match; diff --git a/drivers/net/ethernet/intel/ice/ice_tc_lib.h b/drivers/net/ethernet/intel/ice/ice_tc_lib.h index 91cd3d3778c7..92642faad595 100644 --- a/drivers/net/ethernet/intel/ice/ice_tc_lib.h +++ b/drivers/net/ethernet/intel/ice/ice_tc_lib.h @@ -26,9 +26,18 @@ #define ICE_TC_FLWR_FIELD_CVLAN BIT(19) #define ICE_TC_FLWR_FIELD_PPPOE_SESSID BIT(20) #define ICE_TC_FLWR_FIELD_PPP_PROTO BIT(21) +#define ICE_TC_FLWR_FIELD_IP_TOS BIT(22) +#define ICE_TC_FLWR_FIELD_IP_TTL BIT(23) +#define ICE_TC_FLWR_FIELD_ENC_IP_TOS BIT(24) +#define ICE_TC_FLWR_FIELD_ENC_IP_TTL BIT(25) +#define ICE_TC_FLWR_FIELD_L2TPV3_SESSID BIT(26) +#define ICE_TC_FLWR_FIELD_VLAN_PRIO BIT(27) +#define ICE_TC_FLWR_FIELD_CVLAN_PRIO BIT(28) #define ICE_TC_FLOWER_MASK_32 0xFFFFFFFF +#define ICE_IPV6_HDR_TC_MASK 0xFF00000 + struct ice_indr_block_priv { struct net_device *netdev; struct ice_netdev_priv *np; @@ -42,7 +51,7 @@ struct ice_tc_flower_action { struct ice_tc_vlan_hdr { __be16 vlan_id; /* Only last 12 bits valid */ - u16 vlan_prio; /* Only last 3 bits valid (valid values: 0..7) */ + __be16 vlan_prio; /* Only last 3 bits valid (valid values: 0..7) */ __be16 vlan_tpid; }; @@ -80,6 +89,10 @@ struct ice_tc_l3_hdr { u8 ttl; }; +struct ice_tc_l2tpv3_hdr { + __be32 session_id; +}; + struct ice_tc_l4_hdr { __be16 dst_port; __be16 src_port; @@ -92,6 +105,7 @@ struct ice_tc_flower_lyr_2_4_hdrs { struct ice_tc_vlan_hdr vlan_hdr; struct ice_tc_vlan_hdr cvlan_hdr; struct ice_tc_pppoe_hdr pppoe_hdr; + struct ice_tc_l2tpv3_hdr l2tpv3_hdr; /* L3 (IPv4[6]) layer fields with their mask */ struct ice_tc_l3_hdr l3_key; struct ice_tc_l3_hdr l3_mask; diff --git a/drivers/net/ethernet/intel/ice/ice_txrx.c b/drivers/net/ethernet/intel/ice/ice_txrx.c index dd2285d4bef4..dbe80e5053a8 100644 --- a/drivers/net/ethernet/intel/ice/ice_txrx.c +++ b/drivers/net/ethernet/intel/ice/ice_txrx.c @@ -2258,8 +2258,10 @@ ice_tstamp(struct ice_tx_ring *tx_ring, struct sk_buff *skb, /* Grab an open timestamp slot */ idx = ice_ptp_request_ts(tx_ring->tx_tstamps, skb); - if (idx < 0) + if (idx < 0) { + tx_ring->vsi->back->ptp.tx_hwtstamp_skipped++; return; + } off->cd_qw1 |= (u64)(ICE_TX_DESC_DTYPE_CTX | (ICE_TX_CTX_DESC_TSYN << ICE_TXD_CTX_QW1_CMD_S) | diff --git a/drivers/net/ethernet/intel/ice/ice_txrx.h b/drivers/net/ethernet/intel/ice/ice_txrx.h index ca902af54bb4..932b5661ec4d 100644 --- a/drivers/net/ethernet/intel/ice/ice_txrx.h +++ b/drivers/net/ethernet/intel/ice/ice_txrx.h @@ -295,10 +295,11 @@ struct ice_rx_ring { struct xsk_buff_pool *xsk_pool; struct sk_buff *skb; dma_addr_t dma; /* physical address of ring */ -#define ICE_RX_FLAGS_RING_BUILD_SKB BIT(1) u64 cached_phctime; u8 dcb_tc; /* Traffic class of ring */ u8 ptp_rx; +#define ICE_RX_FLAGS_RING_BUILD_SKB BIT(1) +#define ICE_RX_FLAGS_CRC_STRIP_DIS BIT(2) u8 flags; } ____cacheline_internodealigned_in_smp; diff --git a/drivers/net/ethernet/intel/ice/ice_type.h b/drivers/net/ethernet/intel/ice/ice_type.h index 861b64322959..e1abfcee96dc 100644 --- a/drivers/net/ethernet/intel/ice/ice_type.h +++ b/drivers/net/ethernet/intel/ice/ice_type.h @@ -347,6 +347,7 @@ struct ice_ts_func_info { #define ICE_TS_DEV_ENA_M BIT(24) #define ICE_TS_TMR0_ENA_M BIT(25) #define ICE_TS_TMR1_ENA_M BIT(26) +#define ICE_TS_LL_TX_TS_READ_M BIT(28) struct ice_ts_dev_info { /* Device specific info */ @@ -359,6 +360,7 @@ struct ice_ts_dev_info { u8 ena; u8 tmr0_ena; u8 tmr1_ena; + u8 ts_ll_read; }; /* Function specific capabilities */ @@ -564,6 +566,8 @@ enum ice_rl_type { #define ICE_SCHED_INVAL_PROF_ID 0xFFFF #define ICE_SCHED_DFLT_BURST_SIZE (15 * 1024) /* in bytes (15k) */ +#define ICE_MAX_PORT_PER_PCI_DEV 8 + /* Data structure for saving BW information */ enum ice_bw_type { ICE_BW_TYPE_PRIO, @@ -885,8 +889,6 @@ struct ice_hw { /* INTRL granularity in 1 us */ u8 intrl_gran; - u8 ucast_shared; /* true if VSIs can share unicast addr */ - #define ICE_PHY_PER_NAC 1 #define ICE_MAX_QUAD 2 #define ICE_NUM_QUAD_TYPE 2 diff --git a/drivers/net/ethernet/intel/igb/igb_ethtool.c b/drivers/net/ethernet/intel/igb/igb_ethtool.c index c14fc871dd41..e5f3e7680dc6 100644 --- a/drivers/net/ethernet/intel/igb/igb_ethtool.c +++ b/drivers/net/ethernet/intel/igb/igb_ethtool.c @@ -850,14 +850,14 @@ static void igb_get_drvinfo(struct net_device *netdev, { struct igb_adapter *adapter = netdev_priv(netdev); - strlcpy(drvinfo->driver, igb_driver_name, sizeof(drvinfo->driver)); + strscpy(drvinfo->driver, igb_driver_name, sizeof(drvinfo->driver)); /* EEPROM image version # is reported as firmware version # for * 82575 controllers */ - strlcpy(drvinfo->fw_version, adapter->fw_version, + strscpy(drvinfo->fw_version, adapter->fw_version, sizeof(drvinfo->fw_version)); - strlcpy(drvinfo->bus_info, pci_name(adapter->pdev), + strscpy(drvinfo->bus_info, pci_name(adapter->pdev), sizeof(drvinfo->bus_info)); drvinfo->n_priv_flags = IGB_PRIV_FLAGS_STR_LEN; diff --git a/drivers/net/ethernet/intel/igb/igb_main.c b/drivers/net/ethernet/intel/igb/igb_main.c index 2796e81d2726..f8e32833226c 100644 --- a/drivers/net/ethernet/intel/igb/igb_main.c +++ b/drivers/net/ethernet/intel/igb/igb_main.c @@ -1211,8 +1211,7 @@ static int igb_alloc_q_vector(struct igb_adapter *adapter, return -ENOMEM; /* initialize NAPI */ - netif_napi_add(adapter->netdev, &q_vector->napi, - igb_poll, 64); + netif_napi_add(adapter->netdev, &q_vector->napi, igb_poll); /* tie q_vector and adapter together */ adapter->q_vector[v_idx] = q_vector; @@ -3138,7 +3137,7 @@ static s32 igb_init_i2c(struct igb_adapter *adapter) adapter->i2c_algo.data = adapter; adapter->i2c_adap.algo_data = &adapter->i2c_algo; adapter->i2c_adap.dev.parent = &adapter->pdev->dev; - strlcpy(adapter->i2c_adap.name, "igb BB", + strscpy(adapter->i2c_adap.name, "igb BB", sizeof(adapter->i2c_adap.name)); status = i2c_bit_add_bus(&adapter->i2c_adap); return status; diff --git a/drivers/net/ethernet/intel/igbvf/ethtool.c b/drivers/net/ethernet/intel/igbvf/ethtool.c index 9d4322b74163..83b97989a6bd 100644 --- a/drivers/net/ethernet/intel/igbvf/ethtool.c +++ b/drivers/net/ethernet/intel/igbvf/ethtool.c @@ -169,8 +169,8 @@ static void igbvf_get_drvinfo(struct net_device *netdev, { struct igbvf_adapter *adapter = netdev_priv(netdev); - strlcpy(drvinfo->driver, igbvf_driver_name, sizeof(drvinfo->driver)); - strlcpy(drvinfo->bus_info, pci_name(adapter->pdev), + strscpy(drvinfo->driver, igbvf_driver_name, sizeof(drvinfo->driver)); + strscpy(drvinfo->bus_info, pci_name(adapter->pdev), sizeof(drvinfo->bus_info)); } diff --git a/drivers/net/ethernet/intel/igbvf/netdev.c b/drivers/net/ethernet/intel/igbvf/netdev.c index f4e91db89fe5..3a32809510fc 100644 --- a/drivers/net/ethernet/intel/igbvf/netdev.c +++ b/drivers/net/ethernet/intel/igbvf/netdev.c @@ -1109,7 +1109,7 @@ static int igbvf_alloc_queues(struct igbvf_adapter *adapter) return -ENOMEM; } - netif_napi_add(netdev, &adapter->rx_ring->napi, igbvf_poll, 64); + netif_napi_add(netdev, &adapter->rx_ring->napi, igbvf_poll); return 0; } diff --git a/drivers/net/ethernet/intel/igc/igc_defines.h b/drivers/net/ethernet/intel/igc/igc_defines.h index 5c66b97c0cfa..4f9d7f013a95 100644 --- a/drivers/net/ethernet/intel/igc/igc_defines.h +++ b/drivers/net/ethernet/intel/igc/igc_defines.h @@ -610,7 +610,6 @@ #define IGC_MDIC_OP_WRITE 0x04000000 #define IGC_MDIC_OP_READ 0x08000000 #define IGC_MDIC_READY 0x10000000 -#define IGC_MDIC_INT_EN 0x20000000 #define IGC_MDIC_ERROR 0x40000000 #define IGC_N0_QUEUE -1 diff --git a/drivers/net/ethernet/intel/igc/igc_main.c b/drivers/net/ethernet/intel/igc/igc_main.c index ebff0e04045d..34889be63e78 100644 --- a/drivers/net/ethernet/intel/igc/igc_main.c +++ b/drivers/net/ethernet/intel/igc/igc_main.c @@ -2129,65 +2129,102 @@ static bool igc_alloc_rx_buffers_zc(struct igc_ring *ring, u16 count) return ok; } -static int igc_xdp_init_tx_buffer(struct igc_tx_buffer *buffer, - struct xdp_frame *xdpf, - struct igc_ring *ring) -{ - dma_addr_t dma; - - dma = dma_map_single(ring->dev, xdpf->data, xdpf->len, DMA_TO_DEVICE); - if (dma_mapping_error(ring->dev, dma)) { - netdev_err_once(ring->netdev, "Failed to map DMA for TX\n"); - return -ENOMEM; - } - - buffer->type = IGC_TX_BUFFER_TYPE_XDP; - buffer->xdpf = xdpf; - buffer->protocol = 0; - buffer->bytecount = xdpf->len; - buffer->gso_segs = 1; - buffer->time_stamp = jiffies; - dma_unmap_len_set(buffer, len, xdpf->len); - dma_unmap_addr_set(buffer, dma, dma); - return 0; -} - /* This function requires __netif_tx_lock is held by the caller. */ static int igc_xdp_init_tx_descriptor(struct igc_ring *ring, struct xdp_frame *xdpf) { - struct igc_tx_buffer *buffer; - union igc_adv_tx_desc *desc; - u32 cmd_type, olinfo_status; - int err; + struct skb_shared_info *sinfo = xdp_get_shared_info_from_frame(xdpf); + u8 nr_frags = unlikely(xdp_frame_has_frags(xdpf)) ? sinfo->nr_frags : 0; + u16 count, index = ring->next_to_use; + struct igc_tx_buffer *head = &ring->tx_buffer_info[index]; + struct igc_tx_buffer *buffer = head; + union igc_adv_tx_desc *desc = IGC_TX_DESC(ring, index); + u32 olinfo_status, len = xdpf->len, cmd_type; + void *data = xdpf->data; + u16 i; - if (!igc_desc_unused(ring)) - return -EBUSY; + count = TXD_USE_COUNT(len); + for (i = 0; i < nr_frags; i++) + count += TXD_USE_COUNT(skb_frag_size(&sinfo->frags[i])); - buffer = &ring->tx_buffer_info[ring->next_to_use]; - err = igc_xdp_init_tx_buffer(buffer, xdpf, ring); - if (err) - return err; + if (igc_maybe_stop_tx(ring, count + 3)) { + /* this is a hard error */ + return -EBUSY; + } - cmd_type = IGC_ADVTXD_DTYP_DATA | IGC_ADVTXD_DCMD_DEXT | - IGC_ADVTXD_DCMD_IFCS | IGC_TXD_DCMD | - buffer->bytecount; - olinfo_status = buffer->bytecount << IGC_ADVTXD_PAYLEN_SHIFT; + i = 0; + head->bytecount = xdp_get_frame_len(xdpf); + head->type = IGC_TX_BUFFER_TYPE_XDP; + head->gso_segs = 1; + head->xdpf = xdpf; - desc = IGC_TX_DESC(ring, ring->next_to_use); - desc->read.cmd_type_len = cpu_to_le32(cmd_type); + olinfo_status = head->bytecount << IGC_ADVTXD_PAYLEN_SHIFT; desc->read.olinfo_status = cpu_to_le32(olinfo_status); - desc->read.buffer_addr = cpu_to_le64(dma_unmap_addr(buffer, dma)); - netdev_tx_sent_queue(txring_txq(ring), buffer->bytecount); + for (;;) { + dma_addr_t dma; - buffer->next_to_watch = desc; + dma = dma_map_single(ring->dev, data, len, DMA_TO_DEVICE); + if (dma_mapping_error(ring->dev, dma)) { + netdev_err_once(ring->netdev, + "Failed to map DMA for TX\n"); + goto unmap; + } - ring->next_to_use++; - if (ring->next_to_use == ring->count) - ring->next_to_use = 0; + dma_unmap_len_set(buffer, len, len); + dma_unmap_addr_set(buffer, dma, dma); + + cmd_type = IGC_ADVTXD_DTYP_DATA | IGC_ADVTXD_DCMD_DEXT | + IGC_ADVTXD_DCMD_IFCS | len; + + desc->read.cmd_type_len = cpu_to_le32(cmd_type); + desc->read.buffer_addr = cpu_to_le64(dma); + + buffer->protocol = 0; + + if (++index == ring->count) + index = 0; + + if (i == nr_frags) + break; + + buffer = &ring->tx_buffer_info[index]; + desc = IGC_TX_DESC(ring, index); + desc->read.olinfo_status = 0; + + data = skb_frag_address(&sinfo->frags[i]); + len = skb_frag_size(&sinfo->frags[i]); + i++; + } + desc->read.cmd_type_len |= cpu_to_le32(IGC_TXD_DCMD); + + netdev_tx_sent_queue(txring_txq(ring), head->bytecount); + /* set the timestamp */ + head->time_stamp = jiffies; + /* set next_to_watch value indicating a packet is present */ + head->next_to_watch = desc; + ring->next_to_use = index; return 0; + +unmap: + for (;;) { + buffer = &ring->tx_buffer_info[index]; + if (dma_unmap_len(buffer, len)) + dma_unmap_page(ring->dev, + dma_unmap_addr(buffer, dma), + dma_unmap_len(buffer, len), + DMA_TO_DEVICE); + dma_unmap_len_set(buffer, len, 0); + if (buffer == head) + break; + + if (!index) + index += ring->count; + index--; + } + + return -ENOMEM; } static struct igc_ring *igc_xdp_get_tx_ring(struct igc_adapter *adapter, @@ -2369,6 +2406,7 @@ static int igc_clean_rx_irq(struct igc_q_vector *q_vector, const int budget) xdp_prepare_buff(&xdp, pktbuf - igc_rx_offset(rx_ring), igc_rx_offset(rx_ring) + pkt_offset, size, true); + xdp_buff_clear_frags_flag(&xdp); skb = igc_xdp_run_prog(adapter, &xdp); } @@ -4356,8 +4394,7 @@ static int igc_alloc_q_vector(struct igc_adapter *adapter, return -ENOMEM; /* initialize NAPI */ - netif_napi_add(adapter->netdev, &q_vector->napi, - igc_poll, 64); + netif_napi_add(adapter->netdev, &q_vector->napi, igc_poll); /* tie q_vector and adapter together */ adapter->q_vector[v_idx] = q_vector; diff --git a/drivers/net/ethernet/intel/ixgb/ixgb_ethtool.c b/drivers/net/ethernet/intel/ixgb/ixgb_ethtool.c index 46efcfab7234..efa980514944 100644 --- a/drivers/net/ethernet/intel/ixgb/ixgb_ethtool.c +++ b/drivers/net/ethernet/intel/ixgb/ixgb_ethtool.c @@ -456,9 +456,9 @@ ixgb_get_drvinfo(struct net_device *netdev, { struct ixgb_adapter *adapter = netdev_priv(netdev); - strlcpy(drvinfo->driver, ixgb_driver_name, + strscpy(drvinfo->driver, ixgb_driver_name, sizeof(drvinfo->driver)); - strlcpy(drvinfo->bus_info, pci_name(adapter->pdev), + strscpy(drvinfo->bus_info, pci_name(adapter->pdev), sizeof(drvinfo->bus_info)); } diff --git a/drivers/net/ethernet/intel/ixgb/ixgb_main.c b/drivers/net/ethernet/intel/ixgb/ixgb_main.c index 45be9a1ab6af..b4d47e7a76c8 100644 --- a/drivers/net/ethernet/intel/ixgb/ixgb_main.c +++ b/drivers/net/ethernet/intel/ixgb/ixgb_main.c @@ -414,7 +414,7 @@ ixgb_probe(struct pci_dev *pdev, const struct pci_device_id *ent) netdev->netdev_ops = &ixgb_netdev_ops; ixgb_set_ethtool_ops(netdev); netdev->watchdog_timeo = 5 * HZ; - netif_napi_add(netdev, &adapter->napi, ixgb_clean, 64); + netif_napi_add(netdev, &adapter->napi, ixgb_clean); strncpy(netdev->name, pci_name(pdev), sizeof(netdev->name) - 1); diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_ethtool.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_ethtool.c index 04f453eabef6..e88e3dfac8c2 100644 --- a/drivers/net/ethernet/intel/ixgbe/ixgbe_ethtool.c +++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_ethtool.c @@ -1106,12 +1106,12 @@ static void ixgbe_get_drvinfo(struct net_device *netdev, { struct ixgbe_adapter *adapter = netdev_priv(netdev); - strlcpy(drvinfo->driver, ixgbe_driver_name, sizeof(drvinfo->driver)); + strscpy(drvinfo->driver, ixgbe_driver_name, sizeof(drvinfo->driver)); - strlcpy(drvinfo->fw_version, adapter->eeprom_id, + strscpy(drvinfo->fw_version, adapter->eeprom_id, sizeof(drvinfo->fw_version)); - strlcpy(drvinfo->bus_info, pci_name(adapter->pdev), + strscpy(drvinfo->bus_info, pci_name(adapter->pdev), sizeof(drvinfo->bus_info)); drvinfo->n_priv_flags = IXGBE_PRIV_FLAGS_STR_LEN; @@ -1964,15 +1964,13 @@ static bool ixgbe_check_lbtest_frame(struct ixgbe_rx_buffer *rx_buffer, frame_size >>= 1; - data = kmap(rx_buffer->page) + rx_buffer->page_offset; + data = page_address(rx_buffer->page) + rx_buffer->page_offset; if (data[3] != 0xFF || data[frame_size + 10] != 0xBE || data[frame_size + 12] != 0xAF) match = false; - kunmap(rx_buffer->page); - return match; } diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_fcoe.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_fcoe.c index 0fcd82036d4e..7311bd545acf 100644 --- a/drivers/net/ethernet/intel/ixgbe/ixgbe_fcoe.c +++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_fcoe.c @@ -1004,7 +1004,7 @@ int ixgbe_fcoe_get_hbainfo(struct net_device *netdev, ixgbe_driver_name, UTS_RELEASE); /* Firmware Version */ - strlcpy(info->firmware_version, adapter->eeprom_id, + strscpy(info->firmware_version, adapter->eeprom_id, sizeof(info->firmware_version)); /* Model */ diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_lib.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_lib.c index 86b11164655e..f8156fe4b1dc 100644 --- a/drivers/net/ethernet/intel/ixgbe/ixgbe_lib.c +++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_lib.c @@ -874,8 +874,7 @@ static int ixgbe_alloc_q_vector(struct ixgbe_adapter *adapter, #endif /* initialize NAPI */ - netif_napi_add(adapter->netdev, &q_vector->napi, - ixgbe_poll, 64); + netif_napi_add(adapter->netdev, &q_vector->napi, ixgbe_poll); /* tie q_vector and adapter together */ adapter->q_vector[v_idx] = q_vector; diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c index d1e430b8c8aa..298cfbfcb7b6 100644 --- a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c @@ -10849,7 +10849,7 @@ static int ixgbe_probe(struct pci_dev *pdev, const struct pci_device_id *ent) netdev->netdev_ops = &ixgbe_netdev_ops; ixgbe_set_ethtool_ops(netdev); netdev->watchdog_timeo = 5 * HZ; - strlcpy(netdev->name, pci_name(pdev), sizeof(netdev->name)); + strscpy(netdev->name, pci_name(pdev), sizeof(netdev->name)); /* Setup hw api */ hw->mac.ops = *ii->mac_ops; @@ -11140,7 +11140,7 @@ skip_sriov: err = ixgbe_read_pba_string_generic(hw, part_str, sizeof(part_str)); if (err) - strlcpy(part_str, "Unknown", sizeof(part_str)); + strscpy(part_str, "Unknown", sizeof(part_str)); if (ixgbe_is_sfp(hw) && hw->phy.sfp_type != ixgbe_sfp_type_not_present) e_dev_info("MAC: %d, PHY: %d, SFP+: %d, PBA No: %s\n", hw->mac.type, hw->phy.type, hw->phy.sfp_type, diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_type.h b/drivers/net/ethernet/intel/ixgbe/ixgbe_type.h index 7f7ea468ffa9..2b00db92b08f 100644 --- a/drivers/net/ethernet/intel/ixgbe/ixgbe_type.h +++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_type.h @@ -3712,7 +3712,9 @@ struct ixgbe_info { #define IXGBE_KRM_LINK_S1(P) ((P) ? 0x8200 : 0x4200) #define IXGBE_KRM_LINK_CTRL_1(P) ((P) ? 0x820C : 0x420C) #define IXGBE_KRM_AN_CNTL_1(P) ((P) ? 0x822C : 0x422C) +#define IXGBE_KRM_AN_CNTL_4(P) ((P) ? 0x8238 : 0x4238) #define IXGBE_KRM_AN_CNTL_8(P) ((P) ? 0x8248 : 0x4248) +#define IXGBE_KRM_PCS_KX_AN(P) ((P) ? 0x9918 : 0x5918) #define IXGBE_KRM_SGMII_CTRL(P) ((P) ? 0x82A0 : 0x42A0) #define IXGBE_KRM_LP_BASE_PAGE_HIGH(P) ((P) ? 0x836C : 0x436C) #define IXGBE_KRM_DSP_TXFFE_STATE_4(P) ((P) ? 0x8634 : 0x4634) @@ -3722,6 +3724,7 @@ struct ixgbe_info { #define IXGBE_KRM_PMD_FLX_MASK_ST20(P) ((P) ? 0x9054 : 0x5054) #define IXGBE_KRM_TX_COEFF_CTRL_1(P) ((P) ? 0x9520 : 0x5520) #define IXGBE_KRM_RX_ANA_CTL(P) ((P) ? 0x9A00 : 0x5A00) +#define IXGBE_KRM_FLX_TMRS_CTRL_ST31(P) ((P) ? 0x9180 : 0x5180) #define IXGBE_KRM_PMD_FLX_MASK_ST20_SFI_10G_DA ~(0x3 << 20) #define IXGBE_KRM_PMD_FLX_MASK_ST20_SFI_10G_SR BIT(20) diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_x550.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_x550.c index 35c2b9b8bd19..aa4bf6c9a2f7 100644 --- a/drivers/net/ethernet/intel/ixgbe/ixgbe_x550.c +++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_x550.c @@ -1721,9 +1721,59 @@ static s32 ixgbe_setup_sfi_x550a(struct ixgbe_hw *hw, ixgbe_link_speed *speed) return IXGBE_ERR_LINK_SETUP; } - status = mac->ops.write_iosf_sb_reg(hw, - IXGBE_KRM_PMD_FLX_MASK_ST20(hw->bus.lan_id), - IXGBE_SB_IOSF_TARGET_KR_PHY, reg_val); + (void)mac->ops.write_iosf_sb_reg(hw, + IXGBE_KRM_PMD_FLX_MASK_ST20(hw->bus.lan_id), + IXGBE_SB_IOSF_TARGET_KR_PHY, reg_val); + + /* change mode enforcement rules to hybrid */ + (void)mac->ops.read_iosf_sb_reg(hw, + IXGBE_KRM_FLX_TMRS_CTRL_ST31(hw->bus.lan_id), + IXGBE_SB_IOSF_TARGET_KR_PHY, ®_val); + reg_val |= 0x0400; + + (void)mac->ops.write_iosf_sb_reg(hw, + IXGBE_KRM_FLX_TMRS_CTRL_ST31(hw->bus.lan_id), + IXGBE_SB_IOSF_TARGET_KR_PHY, reg_val); + + /* manually control the config */ + (void)mac->ops.read_iosf_sb_reg(hw, + IXGBE_KRM_LINK_CTRL_1(hw->bus.lan_id), + IXGBE_SB_IOSF_TARGET_KR_PHY, ®_val); + reg_val |= 0x20002240; + + (void)mac->ops.write_iosf_sb_reg(hw, + IXGBE_KRM_LINK_CTRL_1(hw->bus.lan_id), + IXGBE_SB_IOSF_TARGET_KR_PHY, reg_val); + + /* move the AN base page values */ + (void)mac->ops.read_iosf_sb_reg(hw, + IXGBE_KRM_PCS_KX_AN(hw->bus.lan_id), + IXGBE_SB_IOSF_TARGET_KR_PHY, ®_val); + reg_val |= 0x1; + + (void)mac->ops.write_iosf_sb_reg(hw, + IXGBE_KRM_PCS_KX_AN(hw->bus.lan_id), + IXGBE_SB_IOSF_TARGET_KR_PHY, reg_val); + + /* set the AN37 over CB mode */ + (void)mac->ops.read_iosf_sb_reg(hw, + IXGBE_KRM_AN_CNTL_4(hw->bus.lan_id), + IXGBE_SB_IOSF_TARGET_KR_PHY, ®_val); + reg_val |= 0x20000000; + + (void)mac->ops.write_iosf_sb_reg(hw, + IXGBE_KRM_AN_CNTL_4(hw->bus.lan_id), + IXGBE_SB_IOSF_TARGET_KR_PHY, reg_val); + + /* restart AN manually */ + (void)mac->ops.read_iosf_sb_reg(hw, + IXGBE_KRM_LINK_CTRL_1(hw->bus.lan_id), + IXGBE_SB_IOSF_TARGET_KR_PHY, ®_val); + reg_val |= IXGBE_KRM_LINK_CTRL_1_TETH_AN_RESTART; + + (void)mac->ops.write_iosf_sb_reg(hw, + IXGBE_KRM_LINK_CTRL_1(hw->bus.lan_id), + IXGBE_SB_IOSF_TARGET_KR_PHY, reg_val); /* Toggle port SW reset by AN reset. */ status = ixgbe_restart_an_internal_phy_x550em(hw); diff --git a/drivers/net/ethernet/intel/ixgbevf/ethtool.c b/drivers/net/ethernet/intel/ixgbevf/ethtool.c index fed46872af2b..ccfa6b91aac6 100644 --- a/drivers/net/ethernet/intel/ixgbevf/ethtool.c +++ b/drivers/net/ethernet/intel/ixgbevf/ethtool.c @@ -213,8 +213,8 @@ static void ixgbevf_get_drvinfo(struct net_device *netdev, { struct ixgbevf_adapter *adapter = netdev_priv(netdev); - strlcpy(drvinfo->driver, ixgbevf_driver_name, sizeof(drvinfo->driver)); - strlcpy(drvinfo->bus_info, pci_name(adapter->pdev), + strscpy(drvinfo->driver, ixgbevf_driver_name, sizeof(drvinfo->driver)); + strscpy(drvinfo->bus_info, pci_name(adapter->pdev), sizeof(drvinfo->bus_info)); drvinfo->n_priv_flags = IXGBEVF_PRIV_FLAGS_STR_LEN; diff --git a/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c b/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c index 2f12fbe229c1..99933e89717a 100644 --- a/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c +++ b/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c @@ -2733,7 +2733,7 @@ static int ixgbevf_alloc_q_vector(struct ixgbevf_adapter *adapter, int v_idx, return -ENOMEM; /* initialize NAPI */ - netif_napi_add(adapter->netdev, &q_vector->napi, ixgbevf_poll, 64); + netif_napi_add(adapter->netdev, &q_vector->napi, ixgbevf_poll); /* tie q_vector and adapter together */ adapter->q_vector[v_idx] = q_vector; diff --git a/drivers/net/ethernet/jme.c b/drivers/net/ethernet/jme.c index f43d6616bc0d..1732ec3c3dbd 100644 --- a/drivers/net/ethernet/jme.c +++ b/drivers/net/ethernet/jme.c @@ -2332,9 +2332,9 @@ jme_get_drvinfo(struct net_device *netdev, { struct jme_adapter *jme = netdev_priv(netdev); - strlcpy(info->driver, DRV_NAME, sizeof(info->driver)); - strlcpy(info->version, DRV_VERSION, sizeof(info->version)); - strlcpy(info->bus_info, pci_name(jme->pdev), sizeof(info->bus_info)); + strscpy(info->driver, DRV_NAME, sizeof(info->driver)); + strscpy(info->version, DRV_VERSION, sizeof(info->version)); + strscpy(info->bus_info, pci_name(jme->pdev), sizeof(info->bus_info)); } static int @@ -3009,7 +3009,7 @@ jme_init_one(struct pci_dev *pdev, jwrite32(jme, JME_APMC, apmc); } - netif_napi_add(netdev, &jme->napi, jme_poll, NAPI_POLL_WEIGHT); + netif_napi_add(netdev, &jme->napi, jme_poll); spin_lock_init(&jme->phy_lock); spin_lock_init(&jme->macaddr_lock); diff --git a/drivers/net/ethernet/korina.c b/drivers/net/ethernet/korina.c index df9a8eefa007..2b9335cb4bb3 100644 --- a/drivers/net/ethernet/korina.c +++ b/drivers/net/ethernet/korina.c @@ -416,7 +416,8 @@ static void korina_abort_rx(struct net_device *dev) } /* transmit packet */ -static int korina_send_packet(struct sk_buff *skb, struct net_device *dev) +static netdev_tx_t korina_send_packet(struct sk_buff *skb, + struct net_device *dev) { struct korina_private *lp = netdev_priv(dev); u32 chain_prev, chain_next; @@ -938,9 +939,9 @@ static void netdev_get_drvinfo(struct net_device *dev, { struct korina_private *lp = netdev_priv(dev); - strlcpy(info->driver, DRV_NAME, sizeof(info->driver)); - strlcpy(info->version, DRV_VERSION, sizeof(info->version)); - strlcpy(info->bus_info, lp->dev->name, sizeof(info->bus_info)); + strscpy(info->driver, DRV_NAME, sizeof(info->driver)); + strscpy(info->version, DRV_VERSION, sizeof(info->version)); + strscpy(info->bus_info, lp->dev->name, sizeof(info->bus_info)); } static int netdev_get_link_ksettings(struct net_device *dev, @@ -1354,7 +1355,7 @@ static int korina_probe(struct platform_device *pdev) dev->netdev_ops = &korina_netdev_ops; dev->ethtool_ops = &netdev_ethtool_ops; dev->watchdog_timeo = TX_TIMEOUT; - netif_napi_add(dev, &lp->napi, korina_poll, NAPI_POLL_WEIGHT); + netif_napi_add(dev, &lp->napi, korina_poll); lp->mii_if.dev = dev; lp->mii_if.mdio_read = korina_mdio_read; diff --git a/drivers/net/ethernet/lantiq_etop.c b/drivers/net/ethernet/lantiq_etop.c index 7cedbe1fdfd7..59aab4086dcc 100644 --- a/drivers/net/ethernet/lantiq_etop.c +++ b/drivers/net/ethernet/lantiq_etop.c @@ -470,7 +470,7 @@ ltq_etop_stop(struct net_device *dev) return 0; } -static int +static netdev_tx_t ltq_etop_tx(struct sk_buff *skb, struct net_device *dev) { int queue = skb_get_queue_mapping(skb); diff --git a/drivers/net/ethernet/lantiq_xrx200.c b/drivers/net/ethernet/lantiq_xrx200.c index 57f27cc7724e..8d646c7f8c82 100644 --- a/drivers/net/ethernet/lantiq_xrx200.c +++ b/drivers/net/ethernet/lantiq_xrx200.c @@ -620,8 +620,7 @@ static int xrx200_probe(struct platform_device *pdev) PMAC_HD_CTL); /* setup NAPI */ - netif_napi_add(net_dev, &priv->chan_rx.napi, xrx200_poll_rx, - NAPI_POLL_WEIGHT); + netif_napi_add(net_dev, &priv->chan_rx.napi, xrx200_poll_rx); netif_napi_add_tx(net_dev, &priv->chan_tx.napi, xrx200_tx_housekeeping); diff --git a/drivers/net/ethernet/litex/litex_liteeth.c b/drivers/net/ethernet/litex/litex_liteeth.c index fdd99f0de424..35f24e0f0934 100644 --- a/drivers/net/ethernet/litex/litex_liteeth.c +++ b/drivers/net/ethernet/litex/litex_liteeth.c @@ -152,7 +152,8 @@ static int liteeth_stop(struct net_device *netdev) return 0; } -static int liteeth_start_xmit(struct sk_buff *skb, struct net_device *netdev) +static netdev_tx_t liteeth_start_xmit(struct sk_buff *skb, + struct net_device *netdev) { struct liteeth *priv = netdev_priv(netdev); void __iomem *txbuffer; diff --git a/drivers/net/ethernet/marvell/mv643xx_eth.c b/drivers/net/ethernet/marvell/mv643xx_eth.c index b6be0552a6c1..707993b445d1 100644 --- a/drivers/net/ethernet/marvell/mv643xx_eth.c +++ b/drivers/net/ethernet/marvell/mv643xx_eth.c @@ -1603,12 +1603,12 @@ mv643xx_eth_set_link_ksettings(struct net_device *dev, static void mv643xx_eth_get_drvinfo(struct net_device *dev, struct ethtool_drvinfo *drvinfo) { - strlcpy(drvinfo->driver, mv643xx_eth_driver_name, + strscpy(drvinfo->driver, mv643xx_eth_driver_name, sizeof(drvinfo->driver)); - strlcpy(drvinfo->version, mv643xx_eth_driver_version, + strscpy(drvinfo->version, mv643xx_eth_driver_version, sizeof(drvinfo->version)); - strlcpy(drvinfo->fw_version, "N/A", sizeof(drvinfo->fw_version)); - strlcpy(drvinfo->bus_info, "platform", sizeof(drvinfo->bus_info)); + strscpy(drvinfo->fw_version, "N/A", sizeof(drvinfo->fw_version)); + strscpy(drvinfo->bus_info, "platform", sizeof(drvinfo->bus_info)); } static int mv643xx_eth_get_coalesce(struct net_device *dev, @@ -3183,7 +3183,7 @@ static int mv643xx_eth_probe(struct platform_device *pdev) INIT_WORK(&mp->tx_timeout_task, tx_timeout_task); - netif_napi_add(dev, &mp->napi, mv643xx_eth_poll, NAPI_POLL_WEIGHT); + netif_napi_add(dev, &mp->napi, mv643xx_eth_poll); timer_setup(&mp->rx_oom, oom_timer_wrapper, 0); diff --git a/drivers/net/ethernet/marvell/mvneta.c b/drivers/net/ethernet/marvell/mvneta.c index 0caa2df87c04..ff3e361e06e7 100644 --- a/drivers/net/ethernet/marvell/mvneta.c +++ b/drivers/net/ethernet/marvell/mvneta.c @@ -4656,11 +4656,11 @@ mvneta_ethtool_get_coalesce(struct net_device *dev, static void mvneta_ethtool_get_drvinfo(struct net_device *dev, struct ethtool_drvinfo *drvinfo) { - strlcpy(drvinfo->driver, MVNETA_DRIVER_NAME, + strscpy(drvinfo->driver, MVNETA_DRIVER_NAME, sizeof(drvinfo->driver)); - strlcpy(drvinfo->version, MVNETA_DRIVER_VERSION, + strscpy(drvinfo->version, MVNETA_DRIVER_VERSION, sizeof(drvinfo->version)); - strlcpy(drvinfo->bus_info, dev_name(&dev->dev), + strscpy(drvinfo->bus_info, dev_name(&dev->dev), sizeof(drvinfo->bus_info)); } @@ -5600,14 +5600,13 @@ static int mvneta_probe(struct platform_device *pdev) * operation, so only single NAPI should be initialized. */ if (pp->neta_armada3700) { - netif_napi_add(dev, &pp->napi, mvneta_poll, NAPI_POLL_WEIGHT); + netif_napi_add(dev, &pp->napi, mvneta_poll); } else { for_each_present_cpu(cpu) { struct mvneta_pcpu_port *port = per_cpu_ptr(pp->ports, cpu); - netif_napi_add(dev, &port->napi, mvneta_poll, - NAPI_POLL_WEIGHT); + netif_napi_add(dev, &port->napi, mvneta_poll); port->pp = pp; } } diff --git a/drivers/net/ethernet/marvell/mvpp2/mvpp2.h b/drivers/net/ethernet/marvell/mvpp2/mvpp2.h index ad73a488fc5f..11e603686a27 100644 --- a/drivers/net/ethernet/marvell/mvpp2/mvpp2.h +++ b/drivers/net/ethernet/marvell/mvpp2/mvpp2.h @@ -1530,6 +1530,7 @@ u32 mvpp2_read(struct mvpp2 *priv, u32 offset); void mvpp2_dbgfs_init(struct mvpp2 *priv, const char *name); void mvpp2_dbgfs_cleanup(struct mvpp2 *priv); +void mvpp2_dbgfs_exit(void); void mvpp23_rx_fifo_fc_en(struct mvpp2 *priv, int port, bool en); diff --git a/drivers/net/ethernet/marvell/mvpp2/mvpp2_debugfs.c b/drivers/net/ethernet/marvell/mvpp2/mvpp2_debugfs.c index 4a3baa7e0142..75e83ea2a926 100644 --- a/drivers/net/ethernet/marvell/mvpp2/mvpp2_debugfs.c +++ b/drivers/net/ethernet/marvell/mvpp2/mvpp2_debugfs.c @@ -691,6 +691,13 @@ static int mvpp2_dbgfs_port_init(struct dentry *parent, return 0; } +static struct dentry *mvpp2_root; + +void mvpp2_dbgfs_exit(void) +{ + debugfs_remove(mvpp2_root); +} + void mvpp2_dbgfs_cleanup(struct mvpp2 *priv) { debugfs_remove_recursive(priv->dbgfs_dir); @@ -700,10 +707,9 @@ void mvpp2_dbgfs_cleanup(struct mvpp2 *priv) void mvpp2_dbgfs_init(struct mvpp2 *priv, const char *name) { - struct dentry *mvpp2_dir, *mvpp2_root; + struct dentry *mvpp2_dir; int ret, i; - mvpp2_root = debugfs_lookup(MVPP2_DRIVER_NAME, NULL); if (!mvpp2_root) mvpp2_root = debugfs_create_dir(MVPP2_DRIVER_NAME, NULL); diff --git a/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c b/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c index b84128b549b4..eb0fb8128096 100644 --- a/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c +++ b/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c @@ -5425,11 +5425,11 @@ mvpp2_ethtool_get_coalesce(struct net_device *dev, static void mvpp2_ethtool_get_drvinfo(struct net_device *dev, struct ethtool_drvinfo *drvinfo) { - strlcpy(drvinfo->driver, MVPP2_DRIVER_NAME, + strscpy(drvinfo->driver, MVPP2_DRIVER_NAME, sizeof(drvinfo->driver)); - strlcpy(drvinfo->version, MVPP2_DRIVER_VERSION, + strscpy(drvinfo->version, MVPP2_DRIVER_VERSION, sizeof(drvinfo->version)); - strlcpy(drvinfo->bus_info, dev_name(&dev->dev), + strscpy(drvinfo->bus_info, dev_name(&dev->dev), sizeof(drvinfo->bus_info)); } @@ -5770,8 +5770,7 @@ static int mvpp2_simple_queue_vectors_init(struct mvpp2_port *port, v->irq = irq_of_parse_and_map(port_node, 0); if (v->irq <= 0) return -EINVAL; - netif_napi_add(port->dev, &v->napi, mvpp2_poll, - NAPI_POLL_WEIGHT); + netif_napi_add(port->dev, &v->napi, mvpp2_poll); port->nqvecs = 1; @@ -5831,8 +5830,7 @@ static int mvpp2_multi_queue_vectors_init(struct mvpp2_port *port, goto err; } - netif_napi_add(port->dev, &v->napi, mvpp2_poll, - NAPI_POLL_WEIGHT); + netif_napi_add(port->dev, &v->napi, mvpp2_poll); } return 0; @@ -7706,7 +7704,18 @@ static struct platform_driver mvpp2_driver = { }, }; -module_platform_driver(mvpp2_driver); +static int __init mvpp2_driver_init(void) +{ + return platform_driver_register(&mvpp2_driver); +} +module_init(mvpp2_driver_init); + +static void __exit mvpp2_driver_exit(void) +{ + platform_driver_unregister(&mvpp2_driver); + mvpp2_dbgfs_exit(); +} +module_exit(mvpp2_driver_exit); MODULE_DESCRIPTION("Marvell PPv2 Ethernet Driver - www.marvell.com"); MODULE_AUTHOR("Marcin Wojtas <mw@semihalf.com>"); diff --git a/drivers/net/ethernet/marvell/octeon_ep/octep_main.c b/drivers/net/ethernet/marvell/octeon_ep/octep_main.c index 97f080c66dd4..9089adcb75f9 100644 --- a/drivers/net/ethernet/marvell/octeon_ep/octep_main.c +++ b/drivers/net/ethernet/marvell/octeon_ep/octep_main.c @@ -410,7 +410,7 @@ static void octep_napi_add(struct octep_device *oct) for (i = 0; i < oct->num_oqs; i++) { netdev_dbg(oct->netdev, "Adding NAPI on Q-%d\n", i); netif_napi_add(oct->netdev, &oct->ioq_vector[i]->napi, - octep_napi_poll, 64); + octep_napi_poll); oct->oq[i]->napi = &oct->ioq_vector[i]->napi; } } diff --git a/drivers/net/ethernet/marvell/octeon_ep/octep_rx.c b/drivers/net/ethernet/marvell/octeon_ep/octep_rx.c index d9ae0937d17a..392d9b0da0d7 100644 --- a/drivers/net/ethernet/marvell/octeon_ep/octep_rx.c +++ b/drivers/net/ethernet/marvell/octeon_ep/octep_rx.c @@ -158,8 +158,7 @@ static int octep_setup_oq(struct octep_device *oct, int q_no) goto desc_dma_alloc_err; } - oq->buff_info = (struct octep_rx_buffer *) - vzalloc(oq->max_count * OCTEP_OQ_RECVBUF_SIZE); + oq->buff_info = vzalloc(oq->max_count * OCTEP_OQ_RECVBUF_SIZE); if (unlikely(!oq->buff_info)) { dev_err(&oct->pdev->dev, "Failed to allocate buffer info for OQ-%d\n", q_no); diff --git a/drivers/net/ethernet/marvell/octeontx2/af/Makefile b/drivers/net/ethernet/marvell/octeontx2/af/Makefile index 40203560b291..3cf4c8285c90 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/Makefile +++ b/drivers/net/ethernet/marvell/octeontx2/af/Makefile @@ -11,4 +11,4 @@ rvu_mbox-y := mbox.o rvu_trace.o rvu_af-y := cgx.o rvu.o rvu_cgx.o rvu_npa.o rvu_nix.o \ rvu_reg.o rvu_npc.o rvu_debugfs.o ptp.o rvu_npc_fs.o \ rvu_cpt.o rvu_devlink.o rpm.o rvu_cn10k.o rvu_switch.o \ - rvu_sdp.o rvu_npc_hash.o + rvu_sdp.o rvu_npc_hash.o mcs.o mcs_rvu_if.o mcs_cnf10kb.o diff --git a/drivers/net/ethernet/marvell/octeontx2/af/mbox.h b/drivers/net/ethernet/marvell/octeontx2/af/mbox.h index d7762577e285..8d5d5a0f68c4 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/mbox.h +++ b/drivers/net/ethernet/marvell/octeontx2/af/mbox.h @@ -293,20 +293,74 @@ M(NIX_BANDPROF_ALLOC, 0x801d, nix_bandprof_alloc, nix_bandprof_alloc_req, \ M(NIX_BANDPROF_FREE, 0x801e, nix_bandprof_free, nix_bandprof_free_req, \ msg_rsp) \ M(NIX_BANDPROF_GET_HWINFO, 0x801f, nix_bandprof_get_hwinfo, msg_req, \ - nix_bandprof_get_hwinfo_rsp) - -/* Messages initiated by AF (range 0xC00 - 0xDFF) */ + nix_bandprof_get_hwinfo_rsp) \ +/* MCS mbox IDs (range 0xA000 - 0xBFFF) */ \ +M(MCS_ALLOC_RESOURCES, 0xa000, mcs_alloc_resources, mcs_alloc_rsrc_req, \ + mcs_alloc_rsrc_rsp) \ +M(MCS_FREE_RESOURCES, 0xa001, mcs_free_resources, mcs_free_rsrc_req, msg_rsp) \ +M(MCS_FLOWID_ENTRY_WRITE, 0xa002, mcs_flowid_entry_write, mcs_flowid_entry_write_req, \ + msg_rsp) \ +M(MCS_SECY_PLCY_WRITE, 0xa003, mcs_secy_plcy_write, mcs_secy_plcy_write_req, \ + msg_rsp) \ +M(MCS_RX_SC_CAM_WRITE, 0xa004, mcs_rx_sc_cam_write, mcs_rx_sc_cam_write_req, \ + msg_rsp) \ +M(MCS_SA_PLCY_WRITE, 0xa005, mcs_sa_plcy_write, mcs_sa_plcy_write_req, \ + msg_rsp) \ +M(MCS_TX_SC_SA_MAP_WRITE, 0xa006, mcs_tx_sc_sa_map_write, mcs_tx_sc_sa_map, \ + msg_rsp) \ +M(MCS_RX_SC_SA_MAP_WRITE, 0xa007, mcs_rx_sc_sa_map_write, mcs_rx_sc_sa_map, \ + msg_rsp) \ +M(MCS_FLOWID_ENA_ENTRY, 0xa008, mcs_flowid_ena_entry, mcs_flowid_ena_dis_entry, \ + msg_rsp) \ +M(MCS_PN_TABLE_WRITE, 0xa009, mcs_pn_table_write, mcs_pn_table_write_req, \ + msg_rsp) \ +M(MCS_SET_ACTIVE_LMAC, 0xa00a, mcs_set_active_lmac, mcs_set_active_lmac, \ + msg_rsp) \ +M(MCS_GET_HW_INFO, 0xa00b, mcs_get_hw_info, msg_req, mcs_hw_info) \ +M(MCS_GET_FLOWID_STATS, 0xa00c, mcs_get_flowid_stats, mcs_stats_req, \ + mcs_flowid_stats) \ +M(MCS_GET_SECY_STATS, 0xa00d, mcs_get_secy_stats, mcs_stats_req, \ + mcs_secy_stats) \ +M(MCS_GET_SC_STATS, 0xa00e, mcs_get_sc_stats, mcs_stats_req, mcs_sc_stats) \ +M(MCS_GET_SA_STATS, 0xa00f, mcs_get_sa_stats, mcs_stats_req, mcs_sa_stats) \ +M(MCS_GET_PORT_STATS, 0xa010, mcs_get_port_stats, mcs_stats_req, \ + mcs_port_stats) \ +M(MCS_CLEAR_STATS, 0xa011, mcs_clear_stats, mcs_clear_stats, msg_rsp) \ +M(MCS_INTR_CFG, 0xa012, mcs_intr_cfg, mcs_intr_cfg, msg_rsp) \ +M(MCS_SET_LMAC_MODE, 0xa013, mcs_set_lmac_mode, mcs_set_lmac_mode, msg_rsp) \ +M(MCS_SET_PN_THRESHOLD, 0xa014, mcs_set_pn_threshold, mcs_set_pn_threshold, \ + msg_rsp) \ +M(MCS_ALLOC_CTRL_PKT_RULE, 0xa015, mcs_alloc_ctrl_pkt_rule, \ + mcs_alloc_ctrl_pkt_rule_req, \ + mcs_alloc_ctrl_pkt_rule_rsp) \ +M(MCS_FREE_CTRL_PKT_RULE, 0xa016, mcs_free_ctrl_pkt_rule, \ + mcs_free_ctrl_pkt_rule_req, msg_rsp) \ +M(MCS_CTRL_PKT_RULE_WRITE, 0xa017, mcs_ctrl_pkt_rule_write, \ + mcs_ctrl_pkt_rule_write_req, msg_rsp) \ +M(MCS_PORT_RESET, 0xa018, mcs_port_reset, mcs_port_reset_req, msg_rsp) \ +M(MCS_PORT_CFG_SET, 0xa019, mcs_port_cfg_set, mcs_port_cfg_set_req, msg_rsp)\ +M(MCS_PORT_CFG_GET, 0xa020, mcs_port_cfg_get, mcs_port_cfg_get_req, \ + mcs_port_cfg_get_rsp) \ +M(MCS_CUSTOM_TAG_CFG_GET, 0xa021, mcs_custom_tag_cfg_get, \ + mcs_custom_tag_cfg_get_req, \ + mcs_custom_tag_cfg_get_rsp) + +/* Messages initiated by AF (range 0xC00 - 0xEFF) */ #define MBOX_UP_CGX_MESSAGES \ M(CGX_LINK_EVENT, 0xC00, cgx_link_event, cgx_link_info_msg, msg_rsp) #define MBOX_UP_CPT_MESSAGES \ M(CPT_INST_LMTST, 0xD00, cpt_inst_lmtst, cpt_inst_lmtst_req, msg_rsp) +#define MBOX_UP_MCS_MESSAGES \ +M(MCS_INTR_NOTIFY, 0xE00, mcs_intr_notify, mcs_intr_info, msg_rsp) + enum { #define M(_name, _id, _1, _2, _3) MBOX_MSG_ ## _name = _id, MBOX_MESSAGES MBOX_UP_CGX_MESSAGES MBOX_UP_CPT_MESSAGES +MBOX_UP_MCS_MESSAGES #undef M }; @@ -1471,6 +1525,7 @@ enum ptp_op { PTP_OP_GET_CLOCK = 1, PTP_OP_GET_TSTMP = 2, PTP_OP_SET_THRESH = 3, + PTP_OP_EXTTS_ON = 4, }; struct ptp_req { @@ -1478,6 +1533,7 @@ struct ptp_req { u8 op; s64 scaled_ppm; u64 thresh; + int extts_on; }; struct ptp_rsp { @@ -1655,4 +1711,415 @@ enum cgx_af_status { LMAC_AF_ERR_EXACT_MATCH_TBL_LOOK_UP_FAILED = -1110, }; +enum mcs_direction { + MCS_RX, + MCS_TX, +}; + +enum mcs_rsrc_type { + MCS_RSRC_TYPE_FLOWID, + MCS_RSRC_TYPE_SECY, + MCS_RSRC_TYPE_SC, + MCS_RSRC_TYPE_SA, +}; + +struct mcs_alloc_rsrc_req { + struct mbox_msghdr hdr; + u8 rsrc_type; + u8 rsrc_cnt; /* Resources count */ + u8 mcs_id; /* MCS block ID */ + u8 dir; /* Macsec ingress or egress side */ + u8 all; /* Allocate all resource type one each */ + u64 rsvd; +}; + +struct mcs_alloc_rsrc_rsp { + struct mbox_msghdr hdr; + u8 flow_ids[128]; /* Index of reserved entries */ + u8 secy_ids[128]; + u8 sc_ids[128]; + u8 sa_ids[256]; + u8 rsrc_type; + u8 rsrc_cnt; /* No of entries reserved */ + u8 mcs_id; + u8 dir; + u8 all; + u8 rsvd[256]; /* reserved fields for future expansion */ +}; + +struct mcs_free_rsrc_req { + struct mbox_msghdr hdr; + u8 rsrc_id; /* Index of the entry to be freed */ + u8 rsrc_type; + u8 mcs_id; + u8 dir; + u8 all; /* Free all the cam resources */ + u64 rsvd; +}; + +struct mcs_flowid_entry_write_req { + struct mbox_msghdr hdr; + u64 data[4]; + u64 mask[4]; + u64 sci; /* CNF10K-B for tx_secy_mem_map */ + u8 flow_id; + u8 secy_id; /* secyid for which flowid is mapped */ + u8 sc_id; /* Valid if dir = MCS_TX, SC_CAM id mapped to flowid */ + u8 ena; /* Enable tcam entry */ + u8 ctrl_pkt; + u8 mcs_id; + u8 dir; + u64 rsvd; +}; + +struct mcs_secy_plcy_write_req { + struct mbox_msghdr hdr; + u64 plcy; + u8 secy_id; + u8 mcs_id; + u8 dir; + u64 rsvd; +}; + +/* RX SC_CAM mapping */ +struct mcs_rx_sc_cam_write_req { + struct mbox_msghdr hdr; + u64 sci; /* SCI */ + u64 secy_id; /* secy index mapped to SC */ + u8 sc_id; /* SC CAM entry index */ + u8 mcs_id; + u64 rsvd; +}; + +struct mcs_sa_plcy_write_req { + struct mbox_msghdr hdr; + u64 plcy[2][9]; /* Support 2 SA policy */ + u8 sa_index[2]; + u8 sa_cnt; + u8 mcs_id; + u8 dir; + u64 rsvd; +}; + +struct mcs_tx_sc_sa_map { + struct mbox_msghdr hdr; + u8 sa_index0; + u8 sa_index1; + u8 rekey_ena; + u8 sa_index0_vld; + u8 sa_index1_vld; + u8 tx_sa_active; + u64 sectag_sci; + u8 sc_id; /* used as index for SA_MEM_MAP */ + u8 mcs_id; + u64 rsvd; +}; + +struct mcs_rx_sc_sa_map { + struct mbox_msghdr hdr; + u8 sa_index; + u8 sa_in_use; + u8 sc_id; + u8 an; /* value range 0-3, sc_id + an used as index SA_MEM_MAP */ + u8 mcs_id; + u64 rsvd; +}; + +struct mcs_flowid_ena_dis_entry { + struct mbox_msghdr hdr; + u8 flow_id; + u8 ena; + u8 mcs_id; + u8 dir; + u64 rsvd; +}; + +struct mcs_pn_table_write_req { + struct mbox_msghdr hdr; + u64 next_pn; + u8 pn_id; + u8 mcs_id; + u8 dir; + u64 rsvd; +}; + +struct mcs_hw_info { + struct mbox_msghdr hdr; + u8 num_mcs_blks; /* Number of MCS blocks */ + u8 tcam_entries; /* RX/TX Tcam entries per mcs block */ + u8 secy_entries; /* RX/TX SECY entries per mcs block */ + u8 sc_entries; /* RX/TX SC CAM entries per mcs block */ + u8 sa_entries; /* PN table entries = SA entries */ + u64 rsvd[16]; +}; + +struct mcs_set_active_lmac { + struct mbox_msghdr hdr; + u32 lmac_bmap; /* bitmap of active lmac per mcs block */ + u8 mcs_id; + u16 chan_base; /* MCS channel base */ + u64 rsvd; +}; + +struct mcs_set_lmac_mode { + struct mbox_msghdr hdr; + u8 mode; /* 1:Bypass 0:Operational */ + u8 lmac_id; + u8 mcs_id; + u64 rsvd; +}; + +struct mcs_port_reset_req { + struct mbox_msghdr hdr; + u8 reset; + u8 mcs_id; + u8 port_id; + u64 rsvd; +}; + +struct mcs_port_cfg_set_req { + struct mbox_msghdr hdr; + u8 cstm_tag_rel_mode_sel; + u8 custom_hdr_enb; + u8 fifo_skid; + u8 port_mode; + u8 port_id; + u8 mcs_id; + u64 rsvd; +}; + +struct mcs_port_cfg_get_req { + struct mbox_msghdr hdr; + u8 port_id; + u8 mcs_id; + u64 rsvd; +}; + +struct mcs_port_cfg_get_rsp { + struct mbox_msghdr hdr; + u8 cstm_tag_rel_mode_sel; + u8 custom_hdr_enb; + u8 fifo_skid; + u8 port_mode; + u8 port_id; + u8 mcs_id; + u64 rsvd; +}; + +struct mcs_custom_tag_cfg_get_req { + struct mbox_msghdr hdr; + u8 mcs_id; + u8 dir; + u64 rsvd; +}; + +struct mcs_custom_tag_cfg_get_rsp { + struct mbox_msghdr hdr; + u16 cstm_etype[8]; + u8 cstm_indx[8]; + u8 cstm_etype_en; + u8 mcs_id; + u8 dir; + u64 rsvd; +}; + +/* MCS mailbox error codes + * Range 1201 - 1300. + */ +enum mcs_af_status { + MCS_AF_ERR_INVALID_MCSID = -1201, + MCS_AF_ERR_NOT_MAPPED = -1202, +}; + +struct mcs_set_pn_threshold { + struct mbox_msghdr hdr; + u64 threshold; + u8 xpn; /* '1' for setting xpn threshold */ + u8 mcs_id; + u8 dir; + u64 rsvd; +}; + +enum mcs_ctrl_pkt_rulew_type { + MCS_CTRL_PKT_RULE_TYPE_ETH, + MCS_CTRL_PKT_RULE_TYPE_DA, + MCS_CTRL_PKT_RULE_TYPE_RANGE, + MCS_CTRL_PKT_RULE_TYPE_COMBO, + MCS_CTRL_PKT_RULE_TYPE_MAC, +}; + +struct mcs_alloc_ctrl_pkt_rule_req { + struct mbox_msghdr hdr; + u8 rule_type; + u8 mcs_id; /* MCS block ID */ + u8 dir; /* Macsec ingress or egress side */ + u64 rsvd; +}; + +struct mcs_alloc_ctrl_pkt_rule_rsp { + struct mbox_msghdr hdr; + u8 rule_idx; + u8 rule_type; + u8 mcs_id; + u8 dir; + u64 rsvd; +}; + +struct mcs_free_ctrl_pkt_rule_req { + struct mbox_msghdr hdr; + u8 rule_idx; + u8 rule_type; + u8 mcs_id; + u8 dir; + u8 all; + u64 rsvd; +}; + +struct mcs_ctrl_pkt_rule_write_req { + struct mbox_msghdr hdr; + u64 data0; + u64 data1; + u64 data2; + u8 rule_idx; + u8 rule_type; + u8 mcs_id; + u8 dir; + u64 rsvd; +}; + +struct mcs_stats_req { + struct mbox_msghdr hdr; + u8 id; + u8 mcs_id; + u8 dir; + u64 rsvd; +}; + +struct mcs_flowid_stats { + struct mbox_msghdr hdr; + u64 tcam_hit_cnt; + u64 rsvd; +}; + +struct mcs_secy_stats { + struct mbox_msghdr hdr; + u64 ctl_pkt_bcast_cnt; + u64 ctl_pkt_mcast_cnt; + u64 ctl_pkt_ucast_cnt; + u64 ctl_octet_cnt; + u64 unctl_pkt_bcast_cnt; + u64 unctl_pkt_mcast_cnt; + u64 unctl_pkt_ucast_cnt; + u64 unctl_octet_cnt; + /* Valid only for RX */ + u64 octet_decrypted_cnt; + u64 octet_validated_cnt; + u64 pkt_port_disabled_cnt; + u64 pkt_badtag_cnt; + u64 pkt_nosa_cnt; + u64 pkt_nosaerror_cnt; + u64 pkt_tagged_ctl_cnt; + u64 pkt_untaged_cnt; + u64 pkt_ctl_cnt; /* CN10K-B */ + u64 pkt_notag_cnt; /* CNF10K-B */ + /* Valid only for TX */ + u64 octet_encrypted_cnt; + u64 octet_protected_cnt; + u64 pkt_noactivesa_cnt; + u64 pkt_toolong_cnt; + u64 pkt_untagged_cnt; + u64 rsvd[4]; +}; + +struct mcs_port_stats { + struct mbox_msghdr hdr; + u64 tcam_miss_cnt; + u64 parser_err_cnt; + u64 preempt_err_cnt; /* CNF10K-B */ + u64 sectag_insert_err_cnt; + u64 rsvd[4]; +}; + +/* Only for CN10K-B */ +struct mcs_sa_stats { + struct mbox_msghdr hdr; + /* RX */ + u64 pkt_invalid_cnt; + u64 pkt_nosaerror_cnt; + u64 pkt_notvalid_cnt; + u64 pkt_ok_cnt; + u64 pkt_nosa_cnt; + /* TX */ + u64 pkt_encrypt_cnt; + u64 pkt_protected_cnt; + u64 rsvd[4]; +}; + +struct mcs_sc_stats { + struct mbox_msghdr hdr; + /* RX */ + u64 hit_cnt; + u64 pkt_invalid_cnt; + u64 pkt_late_cnt; + u64 pkt_notvalid_cnt; + u64 pkt_unchecked_cnt; + u64 pkt_delay_cnt; /* CNF10K-B */ + u64 pkt_ok_cnt; /* CNF10K-B */ + u64 octet_decrypt_cnt; /* CN10K-B */ + u64 octet_validate_cnt; /* CN10K-B */ + /* TX */ + u64 pkt_encrypt_cnt; + u64 pkt_protected_cnt; + u64 octet_encrypt_cnt; /* CN10K-B */ + u64 octet_protected_cnt; /* CN10K-B */ + u64 rsvd[4]; +}; + +struct mcs_clear_stats { + struct mbox_msghdr hdr; +#define MCS_FLOWID_STATS 0 +#define MCS_SECY_STATS 1 +#define MCS_SC_STATS 2 +#define MCS_SA_STATS 3 +#define MCS_PORT_STATS 4 + u8 type; /* FLOWID, SECY, SC, SA, PORT */ + u8 id; /* type = PORT, If id = FF(invalid) port no is derived from pcifunc */ + u8 mcs_id; + u8 dir; + u8 all; /* All resources stats mapped to PF are cleared */ +}; + +struct mcs_intr_cfg { + struct mbox_msghdr hdr; +#define MCS_CPM_RX_SECTAG_V_EQ1_INT BIT_ULL(0) +#define MCS_CPM_RX_SECTAG_E_EQ0_C_EQ1_INT BIT_ULL(1) +#define MCS_CPM_RX_SECTAG_SL_GTE48_INT BIT_ULL(2) +#define MCS_CPM_RX_SECTAG_ES_EQ1_SC_EQ1_INT BIT_ULL(3) +#define MCS_CPM_RX_SECTAG_SC_EQ1_SCB_EQ1_INT BIT_ULL(4) +#define MCS_CPM_RX_PACKET_XPN_EQ0_INT BIT_ULL(5) +#define MCS_CPM_RX_PN_THRESH_REACHED_INT BIT_ULL(6) +#define MCS_CPM_TX_PACKET_XPN_EQ0_INT BIT_ULL(7) +#define MCS_CPM_TX_PN_THRESH_REACHED_INT BIT_ULL(8) +#define MCS_CPM_TX_SA_NOT_VALID_INT BIT_ULL(9) +#define MCS_BBE_RX_DFIFO_OVERFLOW_INT BIT_ULL(10) +#define MCS_BBE_RX_PLFIFO_OVERFLOW_INT BIT_ULL(11) +#define MCS_BBE_TX_DFIFO_OVERFLOW_INT BIT_ULL(12) +#define MCS_BBE_TX_PLFIFO_OVERFLOW_INT BIT_ULL(13) +#define MCS_PAB_RX_CHAN_OVERFLOW_INT BIT_ULL(14) +#define MCS_PAB_TX_CHAN_OVERFLOW_INT BIT_ULL(15) + u64 intr_mask; /* Interrupt enable mask */ + u8 mcs_id; + u8 lmac_id; + u64 rsvd; +}; + +struct mcs_intr_info { + struct mbox_msghdr hdr; + u64 intr_mask; + int sa_id; + u8 mcs_id; + u8 lmac_id; + u64 rsvd; +}; + #endif /* MBOX_H */ diff --git a/drivers/net/ethernet/marvell/octeontx2/af/mcs.c b/drivers/net/ethernet/marvell/octeontx2/af/mcs.c new file mode 100644 index 000000000000..5ba618aed6ad --- /dev/null +++ b/drivers/net/ethernet/marvell/octeontx2/af/mcs.c @@ -0,0 +1,1601 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Marvell MCS driver + * + * Copyright (C) 2022 Marvell. + */ + +#include <linux/bitfield.h> +#include <linux/delay.h> +#include <linux/device.h> +#include <linux/module.h> +#include <linux/pci.h> + +#include "mcs.h" +#include "mcs_reg.h" + +#define DRV_NAME "Marvell MCS Driver" + +#define PCI_CFG_REG_BAR_NUM 0 + +static const struct pci_device_id mcs_id_table[] = { + { PCI_DEVICE(PCI_VENDOR_ID_CAVIUM, PCI_DEVID_CN10K_MCS) }, + { 0, } /* end of table */ +}; + +static LIST_HEAD(mcs_list); + +void mcs_get_tx_secy_stats(struct mcs *mcs, struct mcs_secy_stats *stats, int id) +{ + u64 reg; + + reg = MCSX_CSE_TX_MEM_SLAVE_IFOUTCTLBCPKTSX(id); + stats->ctl_pkt_bcast_cnt = mcs_reg_read(mcs, reg); + + reg = MCSX_CSE_TX_MEM_SLAVE_IFOUTCTLMCPKTSX(id); + stats->ctl_pkt_mcast_cnt = mcs_reg_read(mcs, reg); + + reg = MCSX_CSE_TX_MEM_SLAVE_IFOUTCTLOCTETSX(id); + stats->ctl_octet_cnt = mcs_reg_read(mcs, reg); + + reg = MCSX_CSE_TX_MEM_SLAVE_IFOUTCTLUCPKTSX(id); + stats->ctl_pkt_ucast_cnt = mcs_reg_read(mcs, reg); + + reg = MCSX_CSE_TX_MEM_SLAVE_IFOUTUNCTLBCPKTSX(id); + stats->unctl_pkt_bcast_cnt = mcs_reg_read(mcs, reg); + + reg = MCSX_CSE_TX_MEM_SLAVE_IFOUTUNCTLMCPKTSX(id); + stats->unctl_pkt_mcast_cnt = mcs_reg_read(mcs, reg); + + reg = MCSX_CSE_TX_MEM_SLAVE_IFOUTUNCTLOCTETSX(id); + stats->unctl_octet_cnt = mcs_reg_read(mcs, reg); + + reg = MCSX_CSE_TX_MEM_SLAVE_IFOUTUNCTLUCPKTSX(id); + stats->unctl_pkt_ucast_cnt = mcs_reg_read(mcs, reg); + + reg = MCSX_CSE_TX_MEM_SLAVE_OUTOCTETSSECYENCRYPTEDX(id); + stats->octet_encrypted_cnt = mcs_reg_read(mcs, reg); + + reg = MCSX_CSE_TX_MEM_SLAVE_OUTOCTETSSECYPROTECTEDX(id); + stats->octet_protected_cnt = mcs_reg_read(mcs, reg); + + reg = MCSX_CSE_TX_MEM_SLAVE_OUTPKTSSECYNOACTIVESAX(id); + stats->pkt_noactivesa_cnt = mcs_reg_read(mcs, reg); + + reg = MCSX_CSE_TX_MEM_SLAVE_OUTPKTSSECYTOOLONGX(id); + stats->pkt_toolong_cnt = mcs_reg_read(mcs, reg); + + reg = MCSX_CSE_TX_MEM_SLAVE_OUTPKTSSECYUNTAGGEDX(id); + stats->pkt_untagged_cnt = mcs_reg_read(mcs, reg); +} + +void mcs_get_rx_secy_stats(struct mcs *mcs, struct mcs_secy_stats *stats, int id) +{ + u64 reg; + + reg = MCSX_CSE_RX_MEM_SLAVE_IFINCTLBCPKTSX(id); + stats->ctl_pkt_bcast_cnt = mcs_reg_read(mcs, reg); + + reg = MCSX_CSE_RX_MEM_SLAVE_IFINCTLMCPKTSX(id); + stats->ctl_pkt_mcast_cnt = mcs_reg_read(mcs, reg); + + reg = MCSX_CSE_RX_MEM_SLAVE_IFINCTLOCTETSX(id); + stats->ctl_octet_cnt = mcs_reg_read(mcs, reg); + + reg = MCSX_CSE_RX_MEM_SLAVE_IFINCTLUCPKTSX(id); + stats->ctl_pkt_ucast_cnt = mcs_reg_read(mcs, reg); + + reg = MCSX_CSE_RX_MEM_SLAVE_IFINUNCTLBCPKTSX(id); + stats->unctl_pkt_bcast_cnt = mcs_reg_read(mcs, reg); + + reg = MCSX_CSE_RX_MEM_SLAVE_IFINUNCTLMCPKTSX(id); + stats->unctl_pkt_mcast_cnt = mcs_reg_read(mcs, reg); + + reg = MCSX_CSE_RX_MEM_SLAVE_IFINUNCTLOCTETSX(id); + stats->unctl_octet_cnt = mcs_reg_read(mcs, reg); + + reg = MCSX_CSE_RX_MEM_SLAVE_IFINUNCTLUCPKTSX(id); + stats->unctl_pkt_ucast_cnt = mcs_reg_read(mcs, reg); + + reg = MCSX_CSE_RX_MEM_SLAVE_INOCTETSSECYDECRYPTEDX(id); + stats->octet_decrypted_cnt = mcs_reg_read(mcs, reg); + + reg = MCSX_CSE_RX_MEM_SLAVE_INOCTETSSECYVALIDATEX(id); + stats->octet_validated_cnt = mcs_reg_read(mcs, reg); + + reg = MCSX_CSE_RX_MEM_SLAVE_INPKTSCTRLPORTDISABLEDX(id); + stats->pkt_port_disabled_cnt = mcs_reg_read(mcs, reg); + + reg = MCSX_CSE_RX_MEM_SLAVE_INPKTSSECYBADTAGX(id); + stats->pkt_badtag_cnt = mcs_reg_read(mcs, reg); + + reg = MCSX_CSE_RX_MEM_SLAVE_INPKTSSECYNOSAX(id); + stats->pkt_nosa_cnt = mcs_reg_read(mcs, reg); + + reg = MCSX_CSE_RX_MEM_SLAVE_INPKTSSECYNOSAERRORX(id); + stats->pkt_nosaerror_cnt = mcs_reg_read(mcs, reg); + + reg = MCSX_CSE_RX_MEM_SLAVE_INPKTSSECYTAGGEDCTLX(id); + stats->pkt_tagged_ctl_cnt = mcs_reg_read(mcs, reg); + + reg = MCSX_CSE_RX_MEM_SLAVE_INPKTSSECYUNTAGGEDORNOTAGX(id); + stats->pkt_untaged_cnt = mcs_reg_read(mcs, reg); + + reg = MCSX_CSE_RX_MEM_SLAVE_INPKTSSECYCTLX(id); + stats->pkt_ctl_cnt = mcs_reg_read(mcs, reg); + + if (mcs->hw->mcs_blks > 1) { + reg = MCSX_CSE_RX_MEM_SLAVE_INPKTSSECYNOTAGX(id); + stats->pkt_notag_cnt = mcs_reg_read(mcs, reg); + } +} + +void mcs_get_flowid_stats(struct mcs *mcs, struct mcs_flowid_stats *stats, + int id, int dir) +{ + u64 reg; + + if (dir == MCS_RX) + reg = MCSX_CSE_RX_MEM_SLAVE_INPKTSFLOWIDTCAMHITX(id); + else + reg = MCSX_CSE_TX_MEM_SLAVE_OUTPKTSFLOWIDTCAMHITX(id); + + stats->tcam_hit_cnt = mcs_reg_read(mcs, reg); +} + +void mcs_get_port_stats(struct mcs *mcs, struct mcs_port_stats *stats, + int id, int dir) +{ + u64 reg; + + if (dir == MCS_RX) { + reg = MCSX_CSE_RX_MEM_SLAVE_INPKTSFLOWIDTCAMMISSX(id); + stats->tcam_miss_cnt = mcs_reg_read(mcs, reg); + + reg = MCSX_CSE_RX_MEM_SLAVE_INPKTSPARSEERRX(id); + stats->parser_err_cnt = mcs_reg_read(mcs, reg); + if (mcs->hw->mcs_blks > 1) { + reg = MCSX_CSE_RX_MEM_SLAVE_INPKTSEARLYPREEMPTERRX(id); + stats->preempt_err_cnt = mcs_reg_read(mcs, reg); + } + } else { + reg = MCSX_CSE_TX_MEM_SLAVE_OUTPKTSFLOWIDTCAMMISSX(id); + stats->tcam_miss_cnt = mcs_reg_read(mcs, reg); + + reg = MCSX_CSE_TX_MEM_SLAVE_OUTPKTSPARSEERRX(id); + stats->parser_err_cnt = mcs_reg_read(mcs, reg); + + reg = MCSX_CSE_TX_MEM_SLAVE_OUTPKTSSECTAGINSERTIONERRX(id); + stats->sectag_insert_err_cnt = mcs_reg_read(mcs, reg); + } +} + +void mcs_get_sa_stats(struct mcs *mcs, struct mcs_sa_stats *stats, int id, int dir) +{ + u64 reg; + + if (dir == MCS_RX) { + reg = MCSX_CSE_RX_MEM_SLAVE_INPKTSSAINVALIDX(id); + stats->pkt_invalid_cnt = mcs_reg_read(mcs, reg); + + reg = MCSX_CSE_RX_MEM_SLAVE_INPKTSSANOTUSINGSAERRORX(id); + stats->pkt_nosaerror_cnt = mcs_reg_read(mcs, reg); + + reg = MCSX_CSE_RX_MEM_SLAVE_INPKTSSANOTVALIDX(id); + stats->pkt_notvalid_cnt = mcs_reg_read(mcs, reg); + + reg = MCSX_CSE_RX_MEM_SLAVE_INPKTSSAOKX(id); + stats->pkt_ok_cnt = mcs_reg_read(mcs, reg); + + reg = MCSX_CSE_RX_MEM_SLAVE_INPKTSSAUNUSEDSAX(id); + stats->pkt_nosa_cnt = mcs_reg_read(mcs, reg); + } else { + reg = MCSX_CSE_TX_MEM_SLAVE_OUTPKTSSAENCRYPTEDX(id); + stats->pkt_encrypt_cnt = mcs_reg_read(mcs, reg); + + reg = MCSX_CSE_TX_MEM_SLAVE_OUTPKTSSAPROTECTEDX(id); + stats->pkt_protected_cnt = mcs_reg_read(mcs, reg); + } +} + +void mcs_get_sc_stats(struct mcs *mcs, struct mcs_sc_stats *stats, + int id, int dir) +{ + u64 reg; + + if (dir == MCS_RX) { + reg = MCSX_CSE_RX_MEM_SLAVE_INPKTSSCCAMHITX(id); + stats->hit_cnt = mcs_reg_read(mcs, reg); + + reg = MCSX_CSE_RX_MEM_SLAVE_INPKTSSCINVALIDX(id); + stats->pkt_invalid_cnt = mcs_reg_read(mcs, reg); + + reg = MCSX_CSE_RX_MEM_SLAVE_INPKTSSCLATEORDELAYEDX(id); + stats->pkt_late_cnt = mcs_reg_read(mcs, reg); + + reg = MCSX_CSE_RX_MEM_SLAVE_INPKTSSCNOTVALIDX(id); + stats->pkt_notvalid_cnt = mcs_reg_read(mcs, reg); + + reg = MCSX_CSE_RX_MEM_SLAVE_INPKTSSCUNCHECKEDOROKX(id); + stats->pkt_unchecked_cnt = mcs_reg_read(mcs, reg); + + if (mcs->hw->mcs_blks > 1) { + reg = MCSX_CSE_RX_MEM_SLAVE_INPKTSSCDELAYEDX(id); + stats->pkt_delay_cnt = mcs_reg_read(mcs, reg); + + reg = MCSX_CSE_RX_MEM_SLAVE_INPKTSSCOKX(id); + stats->pkt_ok_cnt = mcs_reg_read(mcs, reg); + } + if (mcs->hw->mcs_blks == 1) { + reg = MCSX_CSE_RX_MEM_SLAVE_INOCTETSSCDECRYPTEDX(id); + stats->octet_decrypt_cnt = mcs_reg_read(mcs, reg); + + reg = MCSX_CSE_RX_MEM_SLAVE_INOCTETSSCVALIDATEX(id); + stats->octet_validate_cnt = mcs_reg_read(mcs, reg); + } + } else { + reg = MCSX_CSE_TX_MEM_SLAVE_OUTPKTSSCENCRYPTEDX(id); + stats->pkt_encrypt_cnt = mcs_reg_read(mcs, reg); + + reg = MCSX_CSE_TX_MEM_SLAVE_OUTPKTSSCPROTECTEDX(id); + stats->pkt_protected_cnt = mcs_reg_read(mcs, reg); + + if (mcs->hw->mcs_blks == 1) { + reg = MCSX_CSE_TX_MEM_SLAVE_OUTOCTETSSCENCRYPTEDX(id); + stats->octet_encrypt_cnt = mcs_reg_read(mcs, reg); + + reg = MCSX_CSE_TX_MEM_SLAVE_OUTOCTETSSCPROTECTEDX(id); + stats->octet_protected_cnt = mcs_reg_read(mcs, reg); + } + } +} + +void mcs_clear_stats(struct mcs *mcs, u8 type, u8 id, int dir) +{ + struct mcs_flowid_stats flowid_st; + struct mcs_port_stats port_st; + struct mcs_secy_stats secy_st; + struct mcs_sc_stats sc_st; + struct mcs_sa_stats sa_st; + u64 reg; + + if (dir == MCS_RX) + reg = MCSX_CSE_RX_SLAVE_CTRL; + else + reg = MCSX_CSE_TX_SLAVE_CTRL; + + mcs_reg_write(mcs, reg, BIT_ULL(0)); + + switch (type) { + case MCS_FLOWID_STATS: + mcs_get_flowid_stats(mcs, &flowid_st, id, dir); + break; + case MCS_SECY_STATS: + if (dir == MCS_RX) + mcs_get_rx_secy_stats(mcs, &secy_st, id); + else + mcs_get_tx_secy_stats(mcs, &secy_st, id); + break; + case MCS_SC_STATS: + mcs_get_sc_stats(mcs, &sc_st, id, dir); + break; + case MCS_SA_STATS: + mcs_get_sa_stats(mcs, &sa_st, id, dir); + break; + case MCS_PORT_STATS: + mcs_get_port_stats(mcs, &port_st, id, dir); + break; + } + + mcs_reg_write(mcs, reg, 0x0); +} + +int mcs_clear_all_stats(struct mcs *mcs, u16 pcifunc, int dir) +{ + struct mcs_rsrc_map *map; + int id; + + if (dir == MCS_RX) + map = &mcs->rx; + else + map = &mcs->tx; + + /* Clear FLOWID stats */ + for (id = 0; id < map->flow_ids.max; id++) { + if (map->flowid2pf_map[id] != pcifunc) + continue; + mcs_clear_stats(mcs, MCS_FLOWID_STATS, id, dir); + } + + /* Clear SECY stats */ + for (id = 0; id < map->secy.max; id++) { + if (map->secy2pf_map[id] != pcifunc) + continue; + mcs_clear_stats(mcs, MCS_SECY_STATS, id, dir); + } + + /* Clear SC stats */ + for (id = 0; id < map->secy.max; id++) { + if (map->sc2pf_map[id] != pcifunc) + continue; + mcs_clear_stats(mcs, MCS_SC_STATS, id, dir); + } + + /* Clear SA stats */ + for (id = 0; id < map->sa.max; id++) { + if (map->sa2pf_map[id] != pcifunc) + continue; + mcs_clear_stats(mcs, MCS_SA_STATS, id, dir); + } + return 0; +} + +void mcs_pn_table_write(struct mcs *mcs, u8 pn_id, u64 next_pn, u8 dir) +{ + u64 reg; + + if (dir == MCS_RX) + reg = MCSX_CPM_RX_SLAVE_SA_PN_TABLE_MEMX(pn_id); + else + reg = MCSX_CPM_TX_SLAVE_SA_PN_TABLE_MEMX(pn_id); + mcs_reg_write(mcs, reg, next_pn); +} + +void cn10kb_mcs_tx_sa_mem_map_write(struct mcs *mcs, struct mcs_tx_sc_sa_map *map) +{ + u64 reg, val; + + val = (map->sa_index0 & 0xFF) | + (map->sa_index1 & 0xFF) << 9 | + (map->rekey_ena & 0x1) << 18 | + (map->sa_index0_vld & 0x1) << 19 | + (map->sa_index1_vld & 0x1) << 20 | + (map->tx_sa_active & 0x1) << 21 | + map->sectag_sci << 22; + reg = MCSX_CPM_TX_SLAVE_SA_MAP_MEM_0X(map->sc_id); + mcs_reg_write(mcs, reg, val); + + val = map->sectag_sci >> 42; + reg = MCSX_CPM_TX_SLAVE_SA_MAP_MEM_1X(map->sc_id); + mcs_reg_write(mcs, reg, val); +} + +void cn10kb_mcs_rx_sa_mem_map_write(struct mcs *mcs, struct mcs_rx_sc_sa_map *map) +{ + u64 val, reg; + + val = (map->sa_index & 0xFF) | map->sa_in_use << 9; + + reg = MCSX_CPM_RX_SLAVE_SA_MAP_MEMX((4 * map->sc_id) + map->an); + mcs_reg_write(mcs, reg, val); +} + +void mcs_sa_plcy_write(struct mcs *mcs, u64 *plcy, int sa_id, int dir) +{ + int reg_id; + u64 reg; + + if (dir == MCS_RX) { + for (reg_id = 0; reg_id < 8; reg_id++) { + reg = MCSX_CPM_RX_SLAVE_SA_PLCY_MEMX(reg_id, sa_id); + mcs_reg_write(mcs, reg, plcy[reg_id]); + } + } else { + for (reg_id = 0; reg_id < 9; reg_id++) { + reg = MCSX_CPM_TX_SLAVE_SA_PLCY_MEMX(reg_id, sa_id); + mcs_reg_write(mcs, reg, plcy[reg_id]); + } + } +} + +void mcs_ena_dis_sc_cam_entry(struct mcs *mcs, int sc_id, int ena) +{ + u64 reg, val; + + reg = MCSX_CPM_RX_SLAVE_SC_CAM_ENA(0); + if (sc_id > 63) + reg = MCSX_CPM_RX_SLAVE_SC_CAM_ENA(1); + + if (ena) + val = mcs_reg_read(mcs, reg) | BIT_ULL(sc_id); + else + val = mcs_reg_read(mcs, reg) & ~BIT_ULL(sc_id); + + mcs_reg_write(mcs, reg, val); +} + +void mcs_rx_sc_cam_write(struct mcs *mcs, u64 sci, u64 secy, int sc_id) +{ + mcs_reg_write(mcs, MCSX_CPM_RX_SLAVE_SC_CAMX(0, sc_id), sci); + mcs_reg_write(mcs, MCSX_CPM_RX_SLAVE_SC_CAMX(1, sc_id), secy); + /* Enable SC CAM */ + mcs_ena_dis_sc_cam_entry(mcs, sc_id, true); +} + +void mcs_secy_plcy_write(struct mcs *mcs, u64 plcy, int secy_id, int dir) +{ + u64 reg; + + if (dir == MCS_RX) + reg = MCSX_CPM_RX_SLAVE_SECY_PLCY_MEM_0X(secy_id); + else + reg = MCSX_CPM_TX_SLAVE_SECY_PLCY_MEMX(secy_id); + + mcs_reg_write(mcs, reg, plcy); + + if (mcs->hw->mcs_blks == 1 && dir == MCS_RX) + mcs_reg_write(mcs, MCSX_CPM_RX_SLAVE_SECY_PLCY_MEM_1X(secy_id), 0x0ull); +} + +void cn10kb_mcs_flowid_secy_map(struct mcs *mcs, struct secy_mem_map *map, int dir) +{ + u64 reg, val; + + val = (map->secy & 0x7F) | (map->ctrl_pkt & 0x1) << 8; + if (dir == MCS_RX) { + reg = MCSX_CPM_RX_SLAVE_SECY_MAP_MEMX(map->flow_id); + } else { + val |= (map->sc & 0x7F) << 9; + reg = MCSX_CPM_TX_SLAVE_SECY_MAP_MEM_0X(map->flow_id); + } + + mcs_reg_write(mcs, reg, val); +} + +void mcs_ena_dis_flowid_entry(struct mcs *mcs, int flow_id, int dir, int ena) +{ + u64 reg, val; + + if (dir == MCS_RX) { + reg = MCSX_CPM_RX_SLAVE_FLOWID_TCAM_ENA_0; + if (flow_id > 63) + reg = MCSX_CPM_RX_SLAVE_FLOWID_TCAM_ENA_1; + } else { + reg = MCSX_CPM_TX_SLAVE_FLOWID_TCAM_ENA_0; + if (flow_id > 63) + reg = MCSX_CPM_TX_SLAVE_FLOWID_TCAM_ENA_1; + } + + /* Enable/Disable the tcam entry */ + if (ena) + val = mcs_reg_read(mcs, reg) | BIT_ULL(flow_id); + else + val = mcs_reg_read(mcs, reg) & ~BIT_ULL(flow_id); + + mcs_reg_write(mcs, reg, val); +} + +void mcs_flowid_entry_write(struct mcs *mcs, u64 *data, u64 *mask, int flow_id, int dir) +{ + int reg_id; + u64 reg; + + if (dir == MCS_RX) { + for (reg_id = 0; reg_id < 4; reg_id++) { + reg = MCSX_CPM_RX_SLAVE_FLOWID_TCAM_DATAX(reg_id, flow_id); + mcs_reg_write(mcs, reg, data[reg_id]); + reg = MCSX_CPM_RX_SLAVE_FLOWID_TCAM_MASKX(reg_id, flow_id); + mcs_reg_write(mcs, reg, mask[reg_id]); + } + } else { + for (reg_id = 0; reg_id < 4; reg_id++) { + reg = MCSX_CPM_TX_SLAVE_FLOWID_TCAM_DATAX(reg_id, flow_id); + mcs_reg_write(mcs, reg, data[reg_id]); + reg = MCSX_CPM_TX_SLAVE_FLOWID_TCAM_MASKX(reg_id, flow_id); + mcs_reg_write(mcs, reg, mask[reg_id]); + } + } +} + +int mcs_install_flowid_bypass_entry(struct mcs *mcs) +{ + int flow_id, secy_id, reg_id; + struct secy_mem_map map; + u64 reg, plcy = 0; + + /* Flow entry */ + flow_id = mcs->hw->tcam_entries - MCS_RSRC_RSVD_CNT; + for (reg_id = 0; reg_id < 4; reg_id++) { + reg = MCSX_CPM_RX_SLAVE_FLOWID_TCAM_MASKX(reg_id, flow_id); + mcs_reg_write(mcs, reg, GENMASK_ULL(63, 0)); + } + for (reg_id = 0; reg_id < 4; reg_id++) { + reg = MCSX_CPM_TX_SLAVE_FLOWID_TCAM_MASKX(reg_id, flow_id); + mcs_reg_write(mcs, reg, GENMASK_ULL(63, 0)); + } + /* secy */ + secy_id = mcs->hw->secy_entries - MCS_RSRC_RSVD_CNT; + + /* Set validate frames to NULL and enable control port */ + plcy = 0x7ull; + if (mcs->hw->mcs_blks > 1) + plcy = BIT_ULL(0) | 0x3ull << 4; + mcs_secy_plcy_write(mcs, plcy, secy_id, MCS_RX); + + /* Enable control port and set mtu to max */ + plcy = BIT_ULL(0) | GENMASK_ULL(43, 28); + if (mcs->hw->mcs_blks > 1) + plcy = BIT_ULL(0) | GENMASK_ULL(63, 48); + mcs_secy_plcy_write(mcs, plcy, secy_id, MCS_TX); + + /* Map flowid to secy */ + map.secy = secy_id; + map.ctrl_pkt = 0; + map.flow_id = flow_id; + mcs->mcs_ops->mcs_flowid_secy_map(mcs, &map, MCS_RX); + map.sc = secy_id; + mcs->mcs_ops->mcs_flowid_secy_map(mcs, &map, MCS_TX); + + /* Enable Flowid entry */ + mcs_ena_dis_flowid_entry(mcs, flow_id, MCS_RX, true); + mcs_ena_dis_flowid_entry(mcs, flow_id, MCS_TX, true); + return 0; +} + +void mcs_clear_secy_plcy(struct mcs *mcs, int secy_id, int dir) +{ + struct mcs_rsrc_map *map; + int flow_id; + + if (dir == MCS_RX) + map = &mcs->rx; + else + map = &mcs->tx; + + /* Clear secy memory to zero */ + mcs_secy_plcy_write(mcs, 0, secy_id, dir); + + /* Disable the tcam entry using this secy */ + for (flow_id = 0; flow_id < map->flow_ids.max; flow_id++) { + if (map->flowid2secy_map[flow_id] != secy_id) + continue; + mcs_ena_dis_flowid_entry(mcs, flow_id, dir, false); + } +} + +int mcs_alloc_ctrlpktrule(struct rsrc_bmap *rsrc, u16 *pf_map, u16 offset, u16 pcifunc) +{ + int rsrc_id; + + if (!rsrc->bmap) + return -EINVAL; + + rsrc_id = bitmap_find_next_zero_area(rsrc->bmap, rsrc->max, offset, 1, 0); + if (rsrc_id >= rsrc->max) + return -ENOSPC; + + bitmap_set(rsrc->bmap, rsrc_id, 1); + pf_map[rsrc_id] = pcifunc; + + return rsrc_id; +} + +int mcs_free_ctrlpktrule(struct mcs *mcs, struct mcs_free_ctrl_pkt_rule_req *req) +{ + u16 pcifunc = req->hdr.pcifunc; + struct mcs_rsrc_map *map; + u64 dis, reg; + int id, rc; + + reg = (req->dir == MCS_RX) ? MCSX_PEX_RX_SLAVE_RULE_ENABLE : MCSX_PEX_TX_SLAVE_RULE_ENABLE; + map = (req->dir == MCS_RX) ? &mcs->rx : &mcs->tx; + + if (req->all) { + for (id = 0; id < map->ctrlpktrule.max; id++) { + if (map->ctrlpktrule2pf_map[id] != pcifunc) + continue; + mcs_free_rsrc(&map->ctrlpktrule, map->ctrlpktrule2pf_map, id, pcifunc); + dis = mcs_reg_read(mcs, reg); + dis &= ~BIT_ULL(id); + mcs_reg_write(mcs, reg, dis); + } + return 0; + } + + rc = mcs_free_rsrc(&map->ctrlpktrule, map->ctrlpktrule2pf_map, req->rule_idx, pcifunc); + dis = mcs_reg_read(mcs, reg); + dis &= ~BIT_ULL(req->rule_idx); + mcs_reg_write(mcs, reg, dis); + + return rc; +} + +int mcs_ctrlpktrule_write(struct mcs *mcs, struct mcs_ctrl_pkt_rule_write_req *req) +{ + u64 reg, enb; + u64 idx; + + switch (req->rule_type) { + case MCS_CTRL_PKT_RULE_TYPE_ETH: + req->data0 &= GENMASK(15, 0); + if (req->data0 != ETH_P_PAE) + return -EINVAL; + + idx = req->rule_idx - MCS_CTRLPKT_ETYPE_RULE_OFFSET; + reg = (req->dir == MCS_RX) ? MCSX_PEX_RX_SLAVE_RULE_ETYPE_CFGX(idx) : + MCSX_PEX_TX_SLAVE_RULE_ETYPE_CFGX(idx); + + mcs_reg_write(mcs, reg, req->data0); + break; + case MCS_CTRL_PKT_RULE_TYPE_DA: + if (!(req->data0 & BIT_ULL(40))) + return -EINVAL; + + idx = req->rule_idx - MCS_CTRLPKT_DA_RULE_OFFSET; + reg = (req->dir == MCS_RX) ? MCSX_PEX_RX_SLAVE_RULE_DAX(idx) : + MCSX_PEX_TX_SLAVE_RULE_DAX(idx); + + mcs_reg_write(mcs, reg, req->data0 & GENMASK_ULL(47, 0)); + break; + case MCS_CTRL_PKT_RULE_TYPE_RANGE: + if (!(req->data0 & BIT_ULL(40)) || !(req->data1 & BIT_ULL(40))) + return -EINVAL; + + idx = req->rule_idx - MCS_CTRLPKT_DA_RANGE_RULE_OFFSET; + if (req->dir == MCS_RX) { + reg = MCSX_PEX_RX_SLAVE_RULE_DA_RANGE_MINX(idx); + mcs_reg_write(mcs, reg, req->data0 & GENMASK_ULL(47, 0)); + reg = MCSX_PEX_RX_SLAVE_RULE_DA_RANGE_MAXX(idx); + mcs_reg_write(mcs, reg, req->data1 & GENMASK_ULL(47, 0)); + } else { + reg = MCSX_PEX_TX_SLAVE_RULE_DA_RANGE_MINX(idx); + mcs_reg_write(mcs, reg, req->data0 & GENMASK_ULL(47, 0)); + reg = MCSX_PEX_TX_SLAVE_RULE_DA_RANGE_MAXX(idx); + mcs_reg_write(mcs, reg, req->data1 & GENMASK_ULL(47, 0)); + } + break; + case MCS_CTRL_PKT_RULE_TYPE_COMBO: + req->data2 &= GENMASK(15, 0); + if (req->data2 != ETH_P_PAE || !(req->data0 & BIT_ULL(40)) || + !(req->data1 & BIT_ULL(40))) + return -EINVAL; + + idx = req->rule_idx - MCS_CTRLPKT_COMBO_RULE_OFFSET; + if (req->dir == MCS_RX) { + reg = MCSX_PEX_RX_SLAVE_RULE_COMBO_MINX(idx); + mcs_reg_write(mcs, reg, req->data0 & GENMASK_ULL(47, 0)); + reg = MCSX_PEX_RX_SLAVE_RULE_COMBO_MAXX(idx); + mcs_reg_write(mcs, reg, req->data1 & GENMASK_ULL(47, 0)); + reg = MCSX_PEX_RX_SLAVE_RULE_COMBO_ETX(idx); + mcs_reg_write(mcs, reg, req->data2); + } else { + reg = MCSX_PEX_TX_SLAVE_RULE_COMBO_MINX(idx); + mcs_reg_write(mcs, reg, req->data0 & GENMASK_ULL(47, 0)); + reg = MCSX_PEX_TX_SLAVE_RULE_COMBO_MAXX(idx); + mcs_reg_write(mcs, reg, req->data1 & GENMASK_ULL(47, 0)); + reg = MCSX_PEX_TX_SLAVE_RULE_COMBO_ETX(idx); + mcs_reg_write(mcs, reg, req->data2); + } + break; + case MCS_CTRL_PKT_RULE_TYPE_MAC: + if (!(req->data0 & BIT_ULL(40))) + return -EINVAL; + + idx = req->rule_idx - MCS_CTRLPKT_MAC_EN_RULE_OFFSET; + reg = (req->dir == MCS_RX) ? MCSX_PEX_RX_SLAVE_RULE_MAC : + MCSX_PEX_TX_SLAVE_RULE_MAC; + + mcs_reg_write(mcs, reg, req->data0 & GENMASK_ULL(47, 0)); + break; + } + + reg = (req->dir == MCS_RX) ? MCSX_PEX_RX_SLAVE_RULE_ENABLE : MCSX_PEX_TX_SLAVE_RULE_ENABLE; + + enb = mcs_reg_read(mcs, reg); + enb |= BIT_ULL(req->rule_idx); + mcs_reg_write(mcs, reg, enb); + + return 0; +} + +int mcs_free_rsrc(struct rsrc_bmap *rsrc, u16 *pf_map, int rsrc_id, u16 pcifunc) +{ + /* Check if the rsrc_id is mapped to PF/VF */ + if (pf_map[rsrc_id] != pcifunc) + return -EINVAL; + + rvu_free_rsrc(rsrc, rsrc_id); + pf_map[rsrc_id] = 0; + return 0; +} + +/* Free all the cam resources mapped to pf */ +int mcs_free_all_rsrc(struct mcs *mcs, int dir, u16 pcifunc) +{ + struct mcs_rsrc_map *map; + int id; + + if (dir == MCS_RX) + map = &mcs->rx; + else + map = &mcs->tx; + + /* free tcam entries */ + for (id = 0; id < map->flow_ids.max; id++) { + if (map->flowid2pf_map[id] != pcifunc) + continue; + mcs_free_rsrc(&map->flow_ids, map->flowid2pf_map, + id, pcifunc); + mcs_ena_dis_flowid_entry(mcs, id, dir, false); + } + + /* free secy entries */ + for (id = 0; id < map->secy.max; id++) { + if (map->secy2pf_map[id] != pcifunc) + continue; + mcs_free_rsrc(&map->secy, map->secy2pf_map, + id, pcifunc); + mcs_clear_secy_plcy(mcs, id, dir); + } + + /* free sc entries */ + for (id = 0; id < map->secy.max; id++) { + if (map->sc2pf_map[id] != pcifunc) + continue; + mcs_free_rsrc(&map->sc, map->sc2pf_map, id, pcifunc); + + /* Disable SC CAM only on RX side */ + if (dir == MCS_RX) + mcs_ena_dis_sc_cam_entry(mcs, id, false); + } + + /* free sa entries */ + for (id = 0; id < map->sa.max; id++) { + if (map->sa2pf_map[id] != pcifunc) + continue; + mcs_free_rsrc(&map->sa, map->sa2pf_map, id, pcifunc); + } + return 0; +} + +int mcs_alloc_rsrc(struct rsrc_bmap *rsrc, u16 *pf_map, u16 pcifunc) +{ + int rsrc_id; + + rsrc_id = rvu_alloc_rsrc(rsrc); + if (rsrc_id < 0) + return -ENOMEM; + pf_map[rsrc_id] = pcifunc; + return rsrc_id; +} + +int mcs_alloc_all_rsrc(struct mcs *mcs, u8 *flow_id, u8 *secy_id, + u8 *sc_id, u8 *sa1_id, u8 *sa2_id, u16 pcifunc, int dir) +{ + struct mcs_rsrc_map *map; + int id; + + if (dir == MCS_RX) + map = &mcs->rx; + else + map = &mcs->tx; + + id = mcs_alloc_rsrc(&map->flow_ids, map->flowid2pf_map, pcifunc); + if (id < 0) + return -ENOMEM; + *flow_id = id; + + id = mcs_alloc_rsrc(&map->secy, map->secy2pf_map, pcifunc); + if (id < 0) + return -ENOMEM; + *secy_id = id; + + id = mcs_alloc_rsrc(&map->sc, map->sc2pf_map, pcifunc); + if (id < 0) + return -ENOMEM; + *sc_id = id; + + id = mcs_alloc_rsrc(&map->sa, map->sa2pf_map, pcifunc); + if (id < 0) + return -ENOMEM; + *sa1_id = id; + + id = mcs_alloc_rsrc(&map->sa, map->sa2pf_map, pcifunc); + if (id < 0) + return -ENOMEM; + *sa2_id = id; + + return 0; +} + +static void cn10kb_mcs_tx_pn_wrapped_handler(struct mcs *mcs) +{ + struct mcs_intr_event event = { 0 }; + struct rsrc_bmap *sc_bmap; + u64 val; + int sc; + + sc_bmap = &mcs->tx.sc; + + event.mcs_id = mcs->mcs_id; + event.intr_mask = MCS_CPM_TX_PACKET_XPN_EQ0_INT; + + for_each_set_bit(sc, sc_bmap->bmap, mcs->hw->sc_entries) { + val = mcs_reg_read(mcs, MCSX_CPM_TX_SLAVE_SA_MAP_MEM_0X(sc)); + + if (mcs->tx_sa_active[sc]) + /* SA_index1 was used and got expired */ + event.sa_id = (val >> 9) & 0xFF; + else + /* SA_index0 was used and got expired */ + event.sa_id = val & 0xFF; + + event.pcifunc = mcs->tx.sa2pf_map[event.sa_id]; + mcs_add_intr_wq_entry(mcs, &event); + } +} + +static void cn10kb_mcs_tx_pn_thresh_reached_handler(struct mcs *mcs) +{ + struct mcs_intr_event event = { 0 }; + struct rsrc_bmap *sc_bmap; + u64 val, status; + int sc; + + sc_bmap = &mcs->tx.sc; + + event.mcs_id = mcs->mcs_id; + event.intr_mask = MCS_CPM_TX_PN_THRESH_REACHED_INT; + + /* TX SA interrupt is raised only if autorekey is enabled. + * MCS_CPM_TX_SLAVE_SA_MAP_MEM_0X[sc].tx_sa_active bit gets toggled if + * one of two SAs mapped to SC gets expired. If tx_sa_active=0 implies + * SA in SA_index1 got expired else SA in SA_index0 got expired. + */ + for_each_set_bit(sc, sc_bmap->bmap, mcs->hw->sc_entries) { + val = mcs_reg_read(mcs, MCSX_CPM_TX_SLAVE_SA_MAP_MEM_0X(sc)); + /* Auto rekey is enable */ + if (!((val >> 18) & 0x1)) + continue; + + status = (val >> 21) & 0x1; + + /* Check if tx_sa_active status had changed */ + if (status == mcs->tx_sa_active[sc]) + continue; + /* SA_index0 is expired */ + if (status) + event.sa_id = val & 0xFF; + else + event.sa_id = (val >> 9) & 0xFF; + + event.pcifunc = mcs->tx.sa2pf_map[event.sa_id]; + mcs_add_intr_wq_entry(mcs, &event); + } +} + +static void mcs_rx_pn_thresh_reached_handler(struct mcs *mcs) +{ + struct mcs_intr_event event = { 0 }; + int sa, reg; + u64 intr; + + /* Check expired SAs */ + for (reg = 0; reg < (mcs->hw->sa_entries / 64); reg++) { + /* Bit high in *PN_THRESH_REACHEDX implies + * corresponding SAs are expired. + */ + intr = mcs_reg_read(mcs, MCSX_CPM_RX_SLAVE_PN_THRESH_REACHEDX(reg)); + for (sa = 0; sa < 64; sa++) { + if (!(intr & BIT_ULL(sa))) + continue; + + event.mcs_id = mcs->mcs_id; + event.intr_mask = MCS_CPM_RX_PN_THRESH_REACHED_INT; + event.sa_id = sa + (reg * 64); + event.pcifunc = mcs->rx.sa2pf_map[event.sa_id]; + mcs_add_intr_wq_entry(mcs, &event); + } + } +} + +static void mcs_rx_misc_intr_handler(struct mcs *mcs, u64 intr) +{ + struct mcs_intr_event event = { 0 }; + + event.mcs_id = mcs->mcs_id; + event.pcifunc = mcs->pf_map[0]; + + if (intr & MCS_CPM_RX_INT_SECTAG_V_EQ1) + event.intr_mask = MCS_CPM_RX_SECTAG_V_EQ1_INT; + if (intr & MCS_CPM_RX_INT_SECTAG_E_EQ0_C_EQ1) + event.intr_mask |= MCS_CPM_RX_SECTAG_E_EQ0_C_EQ1_INT; + if (intr & MCS_CPM_RX_INT_SL_GTE48) + event.intr_mask |= MCS_CPM_RX_SECTAG_SL_GTE48_INT; + if (intr & MCS_CPM_RX_INT_ES_EQ1_SC_EQ1) + event.intr_mask |= MCS_CPM_RX_SECTAG_ES_EQ1_SC_EQ1_INT; + if (intr & MCS_CPM_RX_INT_SC_EQ1_SCB_EQ1) + event.intr_mask |= MCS_CPM_RX_SECTAG_SC_EQ1_SCB_EQ1_INT; + if (intr & MCS_CPM_RX_INT_PACKET_XPN_EQ0) + event.intr_mask |= MCS_CPM_RX_PACKET_XPN_EQ0_INT; + + mcs_add_intr_wq_entry(mcs, &event); +} + +static void mcs_tx_misc_intr_handler(struct mcs *mcs, u64 intr) +{ + struct mcs_intr_event event = { 0 }; + + if (!(intr & MCS_CPM_TX_INT_SA_NOT_VALID)) + return; + + event.mcs_id = mcs->mcs_id; + event.pcifunc = mcs->pf_map[0]; + + event.intr_mask = MCS_CPM_TX_SA_NOT_VALID_INT; + + mcs_add_intr_wq_entry(mcs, &event); +} + +static void mcs_bbe_intr_handler(struct mcs *mcs, u64 intr, enum mcs_direction dir) +{ + struct mcs_intr_event event = { 0 }; + int i; + + if (!(intr & MCS_BBE_INT_MASK)) + return; + + event.mcs_id = mcs->mcs_id; + event.pcifunc = mcs->pf_map[0]; + + for (i = 0; i < MCS_MAX_BBE_INT; i++) { + if (!(intr & BIT_ULL(i))) + continue; + + /* Lower nibble denotes data fifo overflow interrupts and + * upper nibble indicates policy fifo overflow interrupts. + */ + if (intr & 0xFULL) + event.intr_mask = (dir == MCS_RX) ? + MCS_BBE_RX_DFIFO_OVERFLOW_INT : + MCS_BBE_TX_DFIFO_OVERFLOW_INT; + else + event.intr_mask = (dir == MCS_RX) ? + MCS_BBE_RX_PLFIFO_OVERFLOW_INT : + MCS_BBE_RX_PLFIFO_OVERFLOW_INT; + + /* Notify the lmac_id info which ran into BBE fatal error */ + event.lmac_id = i & 0x3ULL; + mcs_add_intr_wq_entry(mcs, &event); + } +} + +static void mcs_pab_intr_handler(struct mcs *mcs, u64 intr, enum mcs_direction dir) +{ + struct mcs_intr_event event = { 0 }; + int i; + + if (!(intr & MCS_PAB_INT_MASK)) + return; + + event.mcs_id = mcs->mcs_id; + event.pcifunc = mcs->pf_map[0]; + + for (i = 0; i < MCS_MAX_PAB_INT; i++) { + if (!(intr & BIT_ULL(i))) + continue; + + event.intr_mask = (dir == MCS_RX) ? MCS_PAB_RX_CHAN_OVERFLOW_INT : + MCS_PAB_TX_CHAN_OVERFLOW_INT; + + /* Notify the lmac_id info which ran into PAB fatal error */ + event.lmac_id = i; + mcs_add_intr_wq_entry(mcs, &event); + } +} + +static irqreturn_t mcs_ip_intr_handler(int irq, void *mcs_irq) +{ + struct mcs *mcs = (struct mcs *)mcs_irq; + u64 intr, cpm_intr, bbe_intr, pab_intr; + + /* Disable and clear the interrupt */ + mcs_reg_write(mcs, MCSX_IP_INT_ENA_W1C, BIT_ULL(0)); + mcs_reg_write(mcs, MCSX_IP_INT, BIT_ULL(0)); + + /* Check which block has interrupt*/ + intr = mcs_reg_read(mcs, MCSX_TOP_SLAVE_INT_SUM); + + /* CPM RX */ + if (intr & MCS_CPM_RX_INT_ENA) { + /* Check for PN thresh interrupt bit */ + cpm_intr = mcs_reg_read(mcs, MCSX_CPM_RX_SLAVE_RX_INT); + + if (cpm_intr & MCS_CPM_RX_INT_PN_THRESH_REACHED) + mcs_rx_pn_thresh_reached_handler(mcs); + + if (cpm_intr & MCS_CPM_RX_INT_ALL) + mcs_rx_misc_intr_handler(mcs, cpm_intr); + + /* Clear the interrupt */ + mcs_reg_write(mcs, MCSX_CPM_RX_SLAVE_RX_INT, cpm_intr); + } + + /* CPM TX */ + if (intr & MCS_CPM_TX_INT_ENA) { + cpm_intr = mcs_reg_read(mcs, MCSX_CPM_TX_SLAVE_TX_INT); + + if (cpm_intr & MCS_CPM_TX_INT_PN_THRESH_REACHED) { + if (mcs->hw->mcs_blks > 1) + cnf10kb_mcs_tx_pn_thresh_reached_handler(mcs); + else + cn10kb_mcs_tx_pn_thresh_reached_handler(mcs); + } + + if (cpm_intr & MCS_CPM_TX_INT_SA_NOT_VALID) + mcs_tx_misc_intr_handler(mcs, cpm_intr); + + if (cpm_intr & MCS_CPM_TX_INT_PACKET_XPN_EQ0) { + if (mcs->hw->mcs_blks > 1) + cnf10kb_mcs_tx_pn_wrapped_handler(mcs); + else + cn10kb_mcs_tx_pn_wrapped_handler(mcs); + } + /* Clear the interrupt */ + mcs_reg_write(mcs, MCSX_CPM_TX_SLAVE_TX_INT, cpm_intr); + } + + /* BBE RX */ + if (intr & MCS_BBE_RX_INT_ENA) { + bbe_intr = mcs_reg_read(mcs, MCSX_BBE_RX_SLAVE_BBE_INT); + mcs_bbe_intr_handler(mcs, bbe_intr, MCS_RX); + + /* Clear the interrupt */ + mcs_reg_write(mcs, MCSX_BBE_RX_SLAVE_BBE_INT_INTR_RW, 0); + mcs_reg_write(mcs, MCSX_BBE_RX_SLAVE_BBE_INT, bbe_intr); + } + + /* BBE TX */ + if (intr & MCS_BBE_TX_INT_ENA) { + bbe_intr = mcs_reg_read(mcs, MCSX_BBE_TX_SLAVE_BBE_INT); + mcs_bbe_intr_handler(mcs, bbe_intr, MCS_TX); + + /* Clear the interrupt */ + mcs_reg_write(mcs, MCSX_BBE_TX_SLAVE_BBE_INT_INTR_RW, 0); + mcs_reg_write(mcs, MCSX_BBE_TX_SLAVE_BBE_INT, bbe_intr); + } + + /* PAB RX */ + if (intr & MCS_PAB_RX_INT_ENA) { + pab_intr = mcs_reg_read(mcs, MCSX_PAB_RX_SLAVE_PAB_INT); + mcs_pab_intr_handler(mcs, pab_intr, MCS_RX); + + /* Clear the interrupt */ + mcs_reg_write(mcs, MCSX_PAB_RX_SLAVE_PAB_INT_INTR_RW, 0); + mcs_reg_write(mcs, MCSX_PAB_RX_SLAVE_PAB_INT, pab_intr); + } + + /* PAB TX */ + if (intr & MCS_PAB_TX_INT_ENA) { + pab_intr = mcs_reg_read(mcs, MCSX_PAB_TX_SLAVE_PAB_INT); + mcs_pab_intr_handler(mcs, pab_intr, MCS_TX); + + /* Clear the interrupt */ + mcs_reg_write(mcs, MCSX_PAB_TX_SLAVE_PAB_INT_INTR_RW, 0); + mcs_reg_write(mcs, MCSX_PAB_TX_SLAVE_PAB_INT, pab_intr); + } + + /* Enable the interrupt */ + mcs_reg_write(mcs, MCSX_IP_INT_ENA_W1S, BIT_ULL(0)); + + return IRQ_HANDLED; +} + +static void *alloc_mem(struct mcs *mcs, int n) +{ + return devm_kcalloc(mcs->dev, n, sizeof(u16), GFP_KERNEL); +} + +static int mcs_alloc_struct_mem(struct mcs *mcs, struct mcs_rsrc_map *res) +{ + struct hwinfo *hw = mcs->hw; + int err; + + res->flowid2pf_map = alloc_mem(mcs, hw->tcam_entries); + if (!res->flowid2pf_map) + return -ENOMEM; + + res->secy2pf_map = alloc_mem(mcs, hw->secy_entries); + if (!res->secy2pf_map) + return -ENOMEM; + + res->sc2pf_map = alloc_mem(mcs, hw->sc_entries); + if (!res->sc2pf_map) + return -ENOMEM; + + res->sa2pf_map = alloc_mem(mcs, hw->sa_entries); + if (!res->sa2pf_map) + return -ENOMEM; + + res->flowid2secy_map = alloc_mem(mcs, hw->tcam_entries); + if (!res->flowid2secy_map) + return -ENOMEM; + + res->ctrlpktrule2pf_map = alloc_mem(mcs, MCS_MAX_CTRLPKT_RULES); + if (!res->ctrlpktrule2pf_map) + return -ENOMEM; + + res->flow_ids.max = hw->tcam_entries - MCS_RSRC_RSVD_CNT; + err = rvu_alloc_bitmap(&res->flow_ids); + if (err) + return err; + + res->secy.max = hw->secy_entries - MCS_RSRC_RSVD_CNT; + err = rvu_alloc_bitmap(&res->secy); + if (err) + return err; + + res->sc.max = hw->sc_entries; + err = rvu_alloc_bitmap(&res->sc); + if (err) + return err; + + res->sa.max = hw->sa_entries; + err = rvu_alloc_bitmap(&res->sa); + if (err) + return err; + + res->ctrlpktrule.max = MCS_MAX_CTRLPKT_RULES; + err = rvu_alloc_bitmap(&res->ctrlpktrule); + if (err) + return err; + + return 0; +} + +static int mcs_register_interrupts(struct mcs *mcs) +{ + int ret = 0; + + mcs->num_vec = pci_msix_vec_count(mcs->pdev); + + ret = pci_alloc_irq_vectors(mcs->pdev, mcs->num_vec, + mcs->num_vec, PCI_IRQ_MSIX); + if (ret < 0) { + dev_err(mcs->dev, "MCS Request for %d msix vector failed err:%d\n", + mcs->num_vec, ret); + return ret; + } + + ret = request_irq(pci_irq_vector(mcs->pdev, MCS_INT_VEC_IP), + mcs_ip_intr_handler, 0, "MCS_IP", mcs); + if (ret) { + dev_err(mcs->dev, "MCS IP irq registration failed\n"); + goto exit; + } + + /* MCS enable IP interrupts */ + mcs_reg_write(mcs, MCSX_IP_INT_ENA_W1S, BIT_ULL(0)); + + /* Enable CPM Rx/Tx interrupts */ + mcs_reg_write(mcs, MCSX_TOP_SLAVE_INT_SUM_ENB, + MCS_CPM_RX_INT_ENA | MCS_CPM_TX_INT_ENA | + MCS_BBE_RX_INT_ENA | MCS_BBE_TX_INT_ENA | + MCS_PAB_RX_INT_ENA | MCS_PAB_TX_INT_ENA); + + mcs_reg_write(mcs, MCSX_CPM_TX_SLAVE_TX_INT_ENB, 0x7ULL); + mcs_reg_write(mcs, MCSX_CPM_RX_SLAVE_RX_INT_ENB, 0x7FULL); + + mcs_reg_write(mcs, MCSX_BBE_RX_SLAVE_BBE_INT_ENB, 0xff); + mcs_reg_write(mcs, MCSX_BBE_TX_SLAVE_BBE_INT_ENB, 0xff); + + mcs_reg_write(mcs, MCSX_PAB_RX_SLAVE_PAB_INT_ENB, 0xff); + mcs_reg_write(mcs, MCSX_PAB_TX_SLAVE_PAB_INT_ENB, 0xff); + + mcs->tx_sa_active = alloc_mem(mcs, mcs->hw->sc_entries); + if (!mcs->tx_sa_active) + goto exit; + + return ret; +exit: + pci_free_irq_vectors(mcs->pdev); + mcs->num_vec = 0; + return ret; +} + +int mcs_get_blkcnt(void) +{ + struct mcs *mcs; + int idmax = -ENODEV; + + /* Check MCS block is present in hardware */ + if (!pci_dev_present(mcs_id_table)) + return 0; + + list_for_each_entry(mcs, &mcs_list, mcs_list) + if (mcs->mcs_id > idmax) + idmax = mcs->mcs_id; + + if (idmax < 0) + return 0; + + return idmax + 1; +} + +struct mcs *mcs_get_pdata(int mcs_id) +{ + struct mcs *mcs_dev; + + list_for_each_entry(mcs_dev, &mcs_list, mcs_list) { + if (mcs_dev->mcs_id == mcs_id) + return mcs_dev; + } + return NULL; +} + +void mcs_set_port_cfg(struct mcs *mcs, struct mcs_port_cfg_set_req *req) +{ + u64 val = 0; + + mcs_reg_write(mcs, MCSX_PAB_RX_SLAVE_PORT_CFGX(req->port_id), + req->port_mode & MCS_PORT_MODE_MASK); + + req->cstm_tag_rel_mode_sel &= 0x3; + + if (mcs->hw->mcs_blks > 1) { + req->fifo_skid &= MCS_PORT_FIFO_SKID_MASK; + val = (u32)req->fifo_skid << 0x10; + val |= req->fifo_skid; + mcs_reg_write(mcs, MCSX_PAB_RX_SLAVE_FIFO_SKID_CFGX(req->port_id), val); + mcs_reg_write(mcs, MCSX_PEX_TX_SLAVE_CUSTOM_TAG_REL_MODE_SEL(req->port_id), + req->cstm_tag_rel_mode_sel); + val = mcs_reg_read(mcs, MCSX_PEX_RX_SLAVE_PEX_CONFIGURATION); + + if (req->custom_hdr_enb) + val |= BIT_ULL(req->port_id); + else + val &= ~BIT_ULL(req->port_id); + + mcs_reg_write(mcs, MCSX_PEX_RX_SLAVE_PEX_CONFIGURATION, val); + } else { + val = mcs_reg_read(mcs, MCSX_PEX_TX_SLAVE_PORT_CONFIG(req->port_id)); + val |= (req->cstm_tag_rel_mode_sel << 2); + mcs_reg_write(mcs, MCSX_PEX_TX_SLAVE_PORT_CONFIG(req->port_id), val); + } +} + +void mcs_get_port_cfg(struct mcs *mcs, struct mcs_port_cfg_get_req *req, + struct mcs_port_cfg_get_rsp *rsp) +{ + u64 reg = 0; + + rsp->port_mode = mcs_reg_read(mcs, MCSX_PAB_RX_SLAVE_PORT_CFGX(req->port_id)) & + MCS_PORT_MODE_MASK; + + if (mcs->hw->mcs_blks > 1) { + reg = MCSX_PAB_RX_SLAVE_FIFO_SKID_CFGX(req->port_id); + rsp->fifo_skid = mcs_reg_read(mcs, reg) & MCS_PORT_FIFO_SKID_MASK; + reg = MCSX_PEX_TX_SLAVE_CUSTOM_TAG_REL_MODE_SEL(req->port_id); + rsp->cstm_tag_rel_mode_sel = mcs_reg_read(mcs, reg) & 0x3; + if (mcs_reg_read(mcs, MCSX_PEX_RX_SLAVE_PEX_CONFIGURATION) & BIT_ULL(req->port_id)) + rsp->custom_hdr_enb = 1; + } else { + reg = MCSX_PEX_TX_SLAVE_PORT_CONFIG(req->port_id); + rsp->cstm_tag_rel_mode_sel = mcs_reg_read(mcs, reg) >> 2; + } + + rsp->port_id = req->port_id; + rsp->mcs_id = req->mcs_id; +} + +void mcs_get_custom_tag_cfg(struct mcs *mcs, struct mcs_custom_tag_cfg_get_req *req, + struct mcs_custom_tag_cfg_get_rsp *rsp) +{ + u64 reg = 0, val = 0; + u8 idx; + + for (idx = 0; idx < MCS_MAX_CUSTOM_TAGS; idx++) { + if (mcs->hw->mcs_blks > 1) + reg = (req->dir == MCS_RX) ? MCSX_PEX_RX_SLAVE_CUSTOM_TAGX(idx) : + MCSX_PEX_TX_SLAVE_CUSTOM_TAGX(idx); + else + reg = (req->dir == MCS_RX) ? MCSX_PEX_RX_SLAVE_VLAN_CFGX(idx) : + MCSX_PEX_TX_SLAVE_VLAN_CFGX(idx); + + val = mcs_reg_read(mcs, reg); + if (mcs->hw->mcs_blks > 1) { + rsp->cstm_etype[idx] = val & GENMASK(15, 0); + rsp->cstm_indx[idx] = (val >> 0x16) & 0x3; + reg = (req->dir == MCS_RX) ? MCSX_PEX_RX_SLAVE_ETYPE_ENABLE : + MCSX_PEX_TX_SLAVE_ETYPE_ENABLE; + rsp->cstm_etype_en = mcs_reg_read(mcs, reg) & 0xFF; + } else { + rsp->cstm_etype[idx] = (val >> 0x1) & GENMASK(15, 0); + rsp->cstm_indx[idx] = (val >> 0x11) & 0x3; + rsp->cstm_etype_en |= (val & 0x1) << idx; + } + } + + rsp->mcs_id = req->mcs_id; + rsp->dir = req->dir; +} + +void mcs_reset_port(struct mcs *mcs, u8 port_id, u8 reset) +{ + u64 reg = MCSX_MCS_TOP_SLAVE_PORT_RESET(port_id); + + mcs_reg_write(mcs, reg, reset & 0x1); +} + +/* Set lmac to bypass/operational mode */ +void mcs_set_lmac_mode(struct mcs *mcs, int lmac_id, u8 mode) +{ + u64 reg; + + reg = MCSX_MCS_TOP_SLAVE_CHANNEL_CFG(lmac_id * 2); + mcs_reg_write(mcs, reg, (u64)mode); +} + +void mcs_pn_threshold_set(struct mcs *mcs, struct mcs_set_pn_threshold *pn) +{ + u64 reg; + + if (pn->dir == MCS_RX) + reg = pn->xpn ? MCSX_CPM_RX_SLAVE_XPN_THRESHOLD : MCSX_CPM_RX_SLAVE_PN_THRESHOLD; + else + reg = pn->xpn ? MCSX_CPM_TX_SLAVE_XPN_THRESHOLD : MCSX_CPM_TX_SLAVE_PN_THRESHOLD; + + mcs_reg_write(mcs, reg, pn->threshold); +} + +void cn10kb_mcs_parser_cfg(struct mcs *mcs) +{ + u64 reg, val; + + /* VLAN CTag */ + val = BIT_ULL(0) | (0x8100ull & 0xFFFF) << 1 | BIT_ULL(17); + /* RX */ + reg = MCSX_PEX_RX_SLAVE_VLAN_CFGX(0); + mcs_reg_write(mcs, reg, val); + + /* TX */ + reg = MCSX_PEX_TX_SLAVE_VLAN_CFGX(0); + mcs_reg_write(mcs, reg, val); + + /* VLAN STag */ + val = BIT_ULL(0) | (0x88a8ull & 0xFFFF) << 1 | BIT_ULL(18); + /* RX */ + reg = MCSX_PEX_RX_SLAVE_VLAN_CFGX(1); + mcs_reg_write(mcs, reg, val); + + /* TX */ + reg = MCSX_PEX_TX_SLAVE_VLAN_CFGX(1); + mcs_reg_write(mcs, reg, val); +} + +static void mcs_lmac_init(struct mcs *mcs, int lmac_id) +{ + u64 reg; + + /* Port mode 25GB */ + reg = MCSX_PAB_RX_SLAVE_PORT_CFGX(lmac_id); + mcs_reg_write(mcs, reg, 0); + + if (mcs->hw->mcs_blks > 1) { + reg = MCSX_PAB_RX_SLAVE_FIFO_SKID_CFGX(lmac_id); + mcs_reg_write(mcs, reg, 0xe000e); + return; + } + + reg = MCSX_PAB_TX_SLAVE_PORT_CFGX(lmac_id); + mcs_reg_write(mcs, reg, 0); +} + +int mcs_set_lmac_channels(int mcs_id, u16 base) +{ + struct mcs *mcs; + int lmac; + u64 cfg; + + mcs = mcs_get_pdata(mcs_id); + if (!mcs) + return -ENODEV; + for (lmac = 0; lmac < mcs->hw->lmac_cnt; lmac++) { + cfg = mcs_reg_read(mcs, MCSX_LINK_LMACX_CFG(lmac)); + cfg &= ~(MCSX_LINK_LMAC_BASE_MASK | MCSX_LINK_LMAC_RANGE_MASK); + cfg |= FIELD_PREP(MCSX_LINK_LMAC_RANGE_MASK, ilog2(16)); + cfg |= FIELD_PREP(MCSX_LINK_LMAC_BASE_MASK, base); + mcs_reg_write(mcs, MCSX_LINK_LMACX_CFG(lmac), cfg); + base += 16; + } + return 0; +} + +static int mcs_x2p_calibration(struct mcs *mcs) +{ + unsigned long timeout = jiffies + usecs_to_jiffies(20000); + int i, err = 0; + u64 val; + + /* set X2P calibration */ + val = mcs_reg_read(mcs, MCSX_MIL_GLOBAL); + val |= BIT_ULL(5); + mcs_reg_write(mcs, MCSX_MIL_GLOBAL, val); + + /* Wait for calibration to complete */ + while (!(mcs_reg_read(mcs, MCSX_MIL_RX_GBL_STATUS) & BIT_ULL(0))) { + if (time_before(jiffies, timeout)) { + usleep_range(80, 100); + continue; + } else { + err = -EBUSY; + dev_err(mcs->dev, "MCS X2P calibration failed..ignoring\n"); + return err; + } + } + + val = mcs_reg_read(mcs, MCSX_MIL_RX_GBL_STATUS); + for (i = 0; i < mcs->hw->mcs_x2p_intf; i++) { + if (val & BIT_ULL(1 + i)) + continue; + err = -EBUSY; + dev_err(mcs->dev, "MCS:%d didn't respond to X2P calibration\n", i); + } + /* Clear X2P calibrate */ + mcs_reg_write(mcs, MCSX_MIL_GLOBAL, mcs_reg_read(mcs, MCSX_MIL_GLOBAL) & ~BIT_ULL(5)); + + return err; +} + +static void mcs_set_external_bypass(struct mcs *mcs, u8 bypass) +{ + u64 val; + + /* Set MCS to external bypass */ + val = mcs_reg_read(mcs, MCSX_MIL_GLOBAL); + if (bypass) + val |= BIT_ULL(6); + else + val &= ~BIT_ULL(6); + mcs_reg_write(mcs, MCSX_MIL_GLOBAL, val); +} + +static void mcs_global_cfg(struct mcs *mcs) +{ + /* Disable external bypass */ + mcs_set_external_bypass(mcs, false); + + /* Reset TX/RX stats memory */ + mcs_reg_write(mcs, MCSX_CSE_RX_SLAVE_STATS_CLEAR, 0x1F); + mcs_reg_write(mcs, MCSX_CSE_TX_SLAVE_STATS_CLEAR, 0x1F); + + /* Set MCS to perform standard IEEE802.1AE macsec processing */ + if (mcs->hw->mcs_blks == 1) { + mcs_reg_write(mcs, MCSX_IP_MODE, BIT_ULL(3)); + return; + } + + mcs_reg_write(mcs, MCSX_BBE_RX_SLAVE_CAL_ENTRY, 0xe4); + mcs_reg_write(mcs, MCSX_BBE_RX_SLAVE_CAL_LEN, 4); +} + +void cn10kb_mcs_set_hw_capabilities(struct mcs *mcs) +{ + struct hwinfo *hw = mcs->hw; + + hw->tcam_entries = 128; /* TCAM entries */ + hw->secy_entries = 128; /* SecY entries */ + hw->sc_entries = 128; /* SC CAM entries */ + hw->sa_entries = 256; /* SA entries */ + hw->lmac_cnt = 20; /* lmacs/ports per mcs block */ + hw->mcs_x2p_intf = 5; /* x2p clabration intf */ + hw->mcs_blks = 1; /* MCS blocks */ +} + +static struct mcs_ops cn10kb_mcs_ops = { + .mcs_set_hw_capabilities = cn10kb_mcs_set_hw_capabilities, + .mcs_parser_cfg = cn10kb_mcs_parser_cfg, + .mcs_tx_sa_mem_map_write = cn10kb_mcs_tx_sa_mem_map_write, + .mcs_rx_sa_mem_map_write = cn10kb_mcs_rx_sa_mem_map_write, + .mcs_flowid_secy_map = cn10kb_mcs_flowid_secy_map, +}; + +static int mcs_probe(struct pci_dev *pdev, const struct pci_device_id *id) +{ + struct device *dev = &pdev->dev; + int lmac, err = 0; + struct mcs *mcs; + + mcs = devm_kzalloc(dev, sizeof(*mcs), GFP_KERNEL); + if (!mcs) + return -ENOMEM; + + mcs->hw = devm_kzalloc(dev, sizeof(struct hwinfo), GFP_KERNEL); + if (!mcs->hw) + return -ENOMEM; + + err = pci_enable_device(pdev); + if (err) { + dev_err(dev, "Failed to enable PCI device\n"); + pci_set_drvdata(pdev, NULL); + return err; + } + + err = pci_request_regions(pdev, DRV_NAME); + if (err) { + dev_err(dev, "PCI request regions failed 0x%x\n", err); + goto exit; + } + + mcs->reg_base = pcim_iomap(pdev, PCI_CFG_REG_BAR_NUM, 0); + if (!mcs->reg_base) { + dev_err(dev, "mcs: Cannot map CSR memory space, aborting\n"); + err = -ENOMEM; + goto exit; + } + + pci_set_drvdata(pdev, mcs); + mcs->pdev = pdev; + mcs->dev = &pdev->dev; + + if (pdev->subsystem_device == PCI_SUBSYS_DEVID_CN10K_B) + mcs->mcs_ops = &cn10kb_mcs_ops; + else + mcs->mcs_ops = cnf10kb_get_mac_ops(); + + /* Set hardware capabilities */ + mcs->mcs_ops->mcs_set_hw_capabilities(mcs); + + mcs_global_cfg(mcs); + + /* Perform X2P clibration */ + err = mcs_x2p_calibration(mcs); + if (err) + goto err_x2p; + + mcs->mcs_id = (pci_resource_start(pdev, PCI_CFG_REG_BAR_NUM) >> 24) + & MCS_ID_MASK; + + /* Set mcs tx side resources */ + err = mcs_alloc_struct_mem(mcs, &mcs->tx); + if (err) + goto err_x2p; + + /* Set mcs rx side resources */ + err = mcs_alloc_struct_mem(mcs, &mcs->rx); + if (err) + goto err_x2p; + + /* per port config */ + for (lmac = 0; lmac < mcs->hw->lmac_cnt; lmac++) + mcs_lmac_init(mcs, lmac); + + /* Parser configuration */ + mcs->mcs_ops->mcs_parser_cfg(mcs); + + err = mcs_register_interrupts(mcs); + if (err) + goto exit; + + list_add(&mcs->mcs_list, &mcs_list); + mutex_init(&mcs->stats_lock); + + return 0; + +err_x2p: + /* Enable external bypass */ + mcs_set_external_bypass(mcs, true); +exit: + pci_release_regions(pdev); + pci_disable_device(pdev); + pci_set_drvdata(pdev, NULL); + return err; +} + +static void mcs_remove(struct pci_dev *pdev) +{ + struct mcs *mcs = pci_get_drvdata(pdev); + + /* Set MCS to external bypass */ + mcs_set_external_bypass(mcs, true); + pci_free_irq_vectors(pdev); + pci_release_regions(pdev); + pci_disable_device(pdev); + pci_set_drvdata(pdev, NULL); +} + +struct pci_driver mcs_driver = { + .name = DRV_NAME, + .id_table = mcs_id_table, + .probe = mcs_probe, + .remove = mcs_remove, +}; diff --git a/drivers/net/ethernet/marvell/octeontx2/af/mcs.h b/drivers/net/ethernet/marvell/octeontx2/af/mcs.h new file mode 100644 index 000000000000..64dc2b80e15d --- /dev/null +++ b/drivers/net/ethernet/marvell/octeontx2/af/mcs.h @@ -0,0 +1,246 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Marvell CN10K MCS driver + * + * Copyright (C) 2022 Marvell. + */ + +#ifndef MCS_H +#define MCS_H + +#include <linux/bits.h> +#include "rvu.h" + +#define PCI_DEVID_CN10K_MCS 0xA096 + +#define MCSX_LINK_LMAC_RANGE_MASK GENMASK_ULL(19, 16) +#define MCSX_LINK_LMAC_BASE_MASK GENMASK_ULL(11, 0) + +#define MCS_ID_MASK 0x7 +#define MCS_MAX_PFS 128 + +#define MCS_PORT_MODE_MASK 0x3 +#define MCS_PORT_FIFO_SKID_MASK 0x3F +#define MCS_MAX_CUSTOM_TAGS 0x8 + +#define MCS_CTRLPKT_ETYPE_RULE_MAX 8 +#define MCS_CTRLPKT_DA_RULE_MAX 8 +#define MCS_CTRLPKT_DA_RANGE_RULE_MAX 4 +#define MCS_CTRLPKT_COMBO_RULE_MAX 4 +#define MCS_CTRLPKT_MAC_RULE_MAX 1 + +#define MCS_MAX_CTRLPKT_RULES (MCS_CTRLPKT_ETYPE_RULE_MAX + \ + MCS_CTRLPKT_DA_RULE_MAX + \ + MCS_CTRLPKT_DA_RANGE_RULE_MAX + \ + MCS_CTRLPKT_COMBO_RULE_MAX + \ + MCS_CTRLPKT_MAC_RULE_MAX) + +#define MCS_CTRLPKT_ETYPE_RULE_OFFSET 0 +#define MCS_CTRLPKT_DA_RULE_OFFSET 8 +#define MCS_CTRLPKT_DA_RANGE_RULE_OFFSET 16 +#define MCS_CTRLPKT_COMBO_RULE_OFFSET 20 +#define MCS_CTRLPKT_MAC_EN_RULE_OFFSET 24 + +/* Reserved resources for default bypass entry */ +#define MCS_RSRC_RSVD_CNT 1 + +/* MCS Interrupt Vector Enumeration */ +enum mcs_int_vec_e { + MCS_INT_VEC_MIL_RX_GBL = 0x0, + MCS_INT_VEC_MIL_RX_LMACX = 0x1, + MCS_INT_VEC_MIL_TX_LMACX = 0x5, + MCS_INT_VEC_HIL_RX_GBL = 0x9, + MCS_INT_VEC_HIL_RX_LMACX = 0xa, + MCS_INT_VEC_HIL_TX_GBL = 0xe, + MCS_INT_VEC_HIL_TX_LMACX = 0xf, + MCS_INT_VEC_IP = 0x13, + MCS_INT_VEC_CNT = 0x14, +}; + +#define MCS_MAX_BBE_INT 8ULL +#define MCS_BBE_INT_MASK 0xFFULL + +#define MCS_MAX_PAB_INT 4ULL +#define MCS_PAB_INT_MASK 0xFULL + +#define MCS_BBE_RX_INT_ENA BIT_ULL(0) +#define MCS_BBE_TX_INT_ENA BIT_ULL(1) +#define MCS_CPM_RX_INT_ENA BIT_ULL(2) +#define MCS_CPM_TX_INT_ENA BIT_ULL(3) +#define MCS_PAB_RX_INT_ENA BIT_ULL(4) +#define MCS_PAB_TX_INT_ENA BIT_ULL(5) + +#define MCS_CPM_TX_INT_PACKET_XPN_EQ0 BIT_ULL(0) +#define MCS_CPM_TX_INT_PN_THRESH_REACHED BIT_ULL(1) +#define MCS_CPM_TX_INT_SA_NOT_VALID BIT_ULL(2) + +#define MCS_CPM_RX_INT_SECTAG_V_EQ1 BIT_ULL(0) +#define MCS_CPM_RX_INT_SECTAG_E_EQ0_C_EQ1 BIT_ULL(1) +#define MCS_CPM_RX_INT_SL_GTE48 BIT_ULL(2) +#define MCS_CPM_RX_INT_ES_EQ1_SC_EQ1 BIT_ULL(3) +#define MCS_CPM_RX_INT_SC_EQ1_SCB_EQ1 BIT_ULL(4) +#define MCS_CPM_RX_INT_PACKET_XPN_EQ0 BIT_ULL(5) +#define MCS_CPM_RX_INT_PN_THRESH_REACHED BIT_ULL(6) + +#define MCS_CPM_RX_INT_ALL (MCS_CPM_RX_INT_SECTAG_V_EQ1 | \ + MCS_CPM_RX_INT_SECTAG_E_EQ0_C_EQ1 | \ + MCS_CPM_RX_INT_SL_GTE48 | \ + MCS_CPM_RX_INT_ES_EQ1_SC_EQ1 | \ + MCS_CPM_RX_INT_SC_EQ1_SCB_EQ1 | \ + MCS_CPM_RX_INT_PACKET_XPN_EQ0 | \ + MCS_CPM_RX_INT_PN_THRESH_REACHED) + +struct mcs_pfvf { + u64 intr_mask; /* Enabled Interrupt mask */ +}; + +struct mcs_intr_event { + u16 pcifunc; + u64 intr_mask; + u64 sa_id; + u8 mcs_id; + u8 lmac_id; +}; + +struct mcs_intrq_entry { + struct list_head node; + struct mcs_intr_event intr_event; +}; + +struct secy_mem_map { + u8 flow_id; + u8 secy; + u8 ctrl_pkt; + u8 sc; + u64 sci; +}; + +struct mcs_rsrc_map { + u16 *flowid2pf_map; + u16 *secy2pf_map; + u16 *sc2pf_map; + u16 *sa2pf_map; + u16 *flowid2secy_map; /* bitmap flowid mapped to secy*/ + u16 *ctrlpktrule2pf_map; + struct rsrc_bmap flow_ids; + struct rsrc_bmap secy; + struct rsrc_bmap sc; + struct rsrc_bmap sa; + struct rsrc_bmap ctrlpktrule; +}; + +struct hwinfo { + u8 tcam_entries; + u8 secy_entries; + u8 sc_entries; + u16 sa_entries; + u8 mcs_x2p_intf; + u8 lmac_cnt; + u8 mcs_blks; + unsigned long lmac_bmap; /* bitmap of enabled mcs lmac */ +}; + +struct mcs { + void __iomem *reg_base; + struct pci_dev *pdev; + struct device *dev; + struct hwinfo *hw; + struct mcs_rsrc_map tx; + struct mcs_rsrc_map rx; + u16 pf_map[MCS_MAX_PFS]; /* List of PCIFUNC mapped to MCS */ + u8 mcs_id; + struct mcs_ops *mcs_ops; + struct list_head mcs_list; + /* Lock for mcs stats */ + struct mutex stats_lock; + struct mcs_pfvf *pf; + struct mcs_pfvf *vf; + u16 num_vec; + void *rvu; + u16 *tx_sa_active; +}; + +struct mcs_ops { + void (*mcs_set_hw_capabilities)(struct mcs *mcs); + void (*mcs_parser_cfg)(struct mcs *mcs); + void (*mcs_tx_sa_mem_map_write)(struct mcs *mcs, struct mcs_tx_sc_sa_map *map); + void (*mcs_rx_sa_mem_map_write)(struct mcs *mcs, struct mcs_rx_sc_sa_map *map); + void (*mcs_flowid_secy_map)(struct mcs *mcs, struct secy_mem_map *map, int dir); +}; + +extern struct pci_driver mcs_driver; + +static inline void mcs_reg_write(struct mcs *mcs, u64 offset, u64 val) +{ + writeq(val, mcs->reg_base + offset); +} + +static inline u64 mcs_reg_read(struct mcs *mcs, u64 offset) +{ + return readq(mcs->reg_base + offset); +} + +/* MCS APIs */ +struct mcs *mcs_get_pdata(int mcs_id); +int mcs_get_blkcnt(void); +int mcs_set_lmac_channels(int mcs_id, u16 base); +int mcs_alloc_rsrc(struct rsrc_bmap *rsrc, u16 *pf_map, u16 pcifunc); +int mcs_free_rsrc(struct rsrc_bmap *rsrc, u16 *pf_map, int rsrc_id, u16 pcifunc); +int mcs_alloc_all_rsrc(struct mcs *mcs, u8 *flowid, u8 *secy_id, + u8 *sc_id, u8 *sa1_id, u8 *sa2_id, u16 pcifunc, int dir); +int mcs_free_all_rsrc(struct mcs *mcs, int dir, u16 pcifunc); +void mcs_clear_secy_plcy(struct mcs *mcs, int secy_id, int dir); +void mcs_ena_dis_flowid_entry(struct mcs *mcs, int id, int dir, int ena); +void mcs_ena_dis_sc_cam_entry(struct mcs *mcs, int id, int ena); +void mcs_flowid_entry_write(struct mcs *mcs, u64 *data, u64 *mask, int id, int dir); +void mcs_secy_plcy_write(struct mcs *mcs, u64 plcy, int id, int dir); +void mcs_rx_sc_cam_write(struct mcs *mcs, u64 sci, u64 secy, int sc_id); +void mcs_sa_plcy_write(struct mcs *mcs, u64 *plcy, int sa, int dir); +void mcs_map_sc_to_sa(struct mcs *mcs, u64 *sa_map, int sc, int dir); +void mcs_pn_table_write(struct mcs *mcs, u8 pn_id, u64 next_pn, u8 dir); +void mcs_tx_sa_mem_map_write(struct mcs *mcs, struct mcs_tx_sc_sa_map *map); +void mcs_flowid_secy_map(struct mcs *mcs, struct secy_mem_map *map, int dir); +void mcs_rx_sa_mem_map_write(struct mcs *mcs, struct mcs_rx_sc_sa_map *map); +void mcs_pn_threshold_set(struct mcs *mcs, struct mcs_set_pn_threshold *pn); +int mcs_install_flowid_bypass_entry(struct mcs *mcs); +void mcs_set_lmac_mode(struct mcs *mcs, int lmac_id, u8 mode); +void mcs_reset_port(struct mcs *mcs, u8 port_id, u8 reset); +void mcs_set_port_cfg(struct mcs *mcs, struct mcs_port_cfg_set_req *req); +void mcs_get_port_cfg(struct mcs *mcs, struct mcs_port_cfg_get_req *req, + struct mcs_port_cfg_get_rsp *rsp); +void mcs_get_custom_tag_cfg(struct mcs *mcs, struct mcs_custom_tag_cfg_get_req *req, + struct mcs_custom_tag_cfg_get_rsp *rsp); +int mcs_alloc_ctrlpktrule(struct rsrc_bmap *rsrc, u16 *pf_map, u16 offset, u16 pcifunc); +int mcs_free_ctrlpktrule(struct mcs *mcs, struct mcs_free_ctrl_pkt_rule_req *req); +int mcs_ctrlpktrule_write(struct mcs *mcs, struct mcs_ctrl_pkt_rule_write_req *req); + +/* CN10K-B APIs */ +void cn10kb_mcs_set_hw_capabilities(struct mcs *mcs); +void cn10kb_mcs_tx_sa_mem_map_write(struct mcs *mcs, struct mcs_tx_sc_sa_map *map); +void cn10kb_mcs_flowid_secy_map(struct mcs *mcs, struct secy_mem_map *map, int dir); +void cn10kb_mcs_rx_sa_mem_map_write(struct mcs *mcs, struct mcs_rx_sc_sa_map *map); +void cn10kb_mcs_parser_cfg(struct mcs *mcs); + +/* CNF10K-B APIs */ +struct mcs_ops *cnf10kb_get_mac_ops(void); +void cnf10kb_mcs_set_hw_capabilities(struct mcs *mcs); +void cnf10kb_mcs_tx_sa_mem_map_write(struct mcs *mcs, struct mcs_tx_sc_sa_map *map); +void cnf10kb_mcs_flowid_secy_map(struct mcs *mcs, struct secy_mem_map *map, int dir); +void cnf10kb_mcs_rx_sa_mem_map_write(struct mcs *mcs, struct mcs_rx_sc_sa_map *map); +void cnf10kb_mcs_parser_cfg(struct mcs *mcs); +void cnf10kb_mcs_tx_pn_thresh_reached_handler(struct mcs *mcs); +void cnf10kb_mcs_tx_pn_wrapped_handler(struct mcs *mcs); + +/* Stats APIs */ +void mcs_get_sc_stats(struct mcs *mcs, struct mcs_sc_stats *stats, int id, int dir); +void mcs_get_sa_stats(struct mcs *mcs, struct mcs_sa_stats *stats, int id, int dir); +void mcs_get_port_stats(struct mcs *mcs, struct mcs_port_stats *stats, int id, int dir); +void mcs_get_flowid_stats(struct mcs *mcs, struct mcs_flowid_stats *stats, int id, int dir); +void mcs_get_rx_secy_stats(struct mcs *mcs, struct mcs_secy_stats *stats, int id); +void mcs_get_tx_secy_stats(struct mcs *mcs, struct mcs_secy_stats *stats, int id); +void mcs_clear_stats(struct mcs *mcs, u8 type, u8 id, int dir); +int mcs_clear_all_stats(struct mcs *mcs, u16 pcifunc, int dir); +int mcs_set_force_clk_en(struct mcs *mcs, bool set); + +int mcs_add_intr_wq_entry(struct mcs *mcs, struct mcs_intr_event *event); + +#endif /* MCS_H */ diff --git a/drivers/net/ethernet/marvell/octeontx2/af/mcs_cnf10kb.c b/drivers/net/ethernet/marvell/octeontx2/af/mcs_cnf10kb.c new file mode 100644 index 000000000000..7b6205414428 --- /dev/null +++ b/drivers/net/ethernet/marvell/octeontx2/af/mcs_cnf10kb.c @@ -0,0 +1,214 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Marvell MCS driver + * + * Copyright (C) 2022 Marvell. + */ + +#include "mcs.h" +#include "mcs_reg.h" + +static struct mcs_ops cnf10kb_mcs_ops = { + .mcs_set_hw_capabilities = cnf10kb_mcs_set_hw_capabilities, + .mcs_parser_cfg = cnf10kb_mcs_parser_cfg, + .mcs_tx_sa_mem_map_write = cnf10kb_mcs_tx_sa_mem_map_write, + .mcs_rx_sa_mem_map_write = cnf10kb_mcs_rx_sa_mem_map_write, + .mcs_flowid_secy_map = cnf10kb_mcs_flowid_secy_map, +}; + +struct mcs_ops *cnf10kb_get_mac_ops(void) +{ + return &cnf10kb_mcs_ops; +} + +void cnf10kb_mcs_set_hw_capabilities(struct mcs *mcs) +{ + struct hwinfo *hw = mcs->hw; + + hw->tcam_entries = 64; /* TCAM entries */ + hw->secy_entries = 64; /* SecY entries */ + hw->sc_entries = 64; /* SC CAM entries */ + hw->sa_entries = 128; /* SA entries */ + hw->lmac_cnt = 4; /* lmacs/ports per mcs block */ + hw->mcs_x2p_intf = 1; /* x2p clabration intf */ + hw->mcs_blks = 7; /* MCS blocks */ +} + +void cnf10kb_mcs_parser_cfg(struct mcs *mcs) +{ + u64 reg, val; + + /* VLAN Ctag */ + val = (0x8100ull & 0xFFFF) | BIT_ULL(20) | BIT_ULL(22); + + reg = MCSX_PEX_RX_SLAVE_CUSTOM_TAGX(0); + mcs_reg_write(mcs, reg, val); + + reg = MCSX_PEX_TX_SLAVE_CUSTOM_TAGX(0); + mcs_reg_write(mcs, reg, val); + + /* VLAN STag */ + val = (0x88a8ull & 0xFFFF) | BIT_ULL(20) | BIT_ULL(23); + + /* RX */ + reg = MCSX_PEX_RX_SLAVE_CUSTOM_TAGX(1); + mcs_reg_write(mcs, reg, val); + + /* TX */ + reg = MCSX_PEX_TX_SLAVE_CUSTOM_TAGX(1); + mcs_reg_write(mcs, reg, val); + + /* Enable custom tage 0 and 1 and sectag */ + val = BIT_ULL(0) | BIT_ULL(1) | BIT_ULL(12); + + reg = MCSX_PEX_RX_SLAVE_ETYPE_ENABLE; + mcs_reg_write(mcs, reg, val); + + reg = MCSX_PEX_TX_SLAVE_ETYPE_ENABLE; + mcs_reg_write(mcs, reg, val); +} + +void cnf10kb_mcs_flowid_secy_map(struct mcs *mcs, struct secy_mem_map *map, int dir) +{ + u64 reg, val; + + val = (map->secy & 0x3F) | (map->ctrl_pkt & 0x1) << 6; + if (dir == MCS_RX) { + reg = MCSX_CPM_RX_SLAVE_SECY_MAP_MEMX(map->flow_id); + } else { + reg = MCSX_CPM_TX_SLAVE_SECY_MAP_MEM_0X(map->flow_id); + mcs_reg_write(mcs, reg, map->sci); + val |= (map->sc & 0x3F) << 7; + reg = MCSX_CPM_TX_SLAVE_SECY_MAP_MEM_1X(map->flow_id); + } + + mcs_reg_write(mcs, reg, val); +} + +void cnf10kb_mcs_tx_sa_mem_map_write(struct mcs *mcs, struct mcs_tx_sc_sa_map *map) +{ + u64 reg, val; + + val = (map->sa_index0 & 0x7F) | (map->sa_index1 & 0x7F) << 7; + + reg = MCSX_CPM_TX_SLAVE_SA_MAP_MEM_0X(map->sc_id); + mcs_reg_write(mcs, reg, val); + + reg = MCSX_CPM_TX_SLAVE_AUTO_REKEY_ENABLE_0; + val = mcs_reg_read(mcs, reg); + + if (map->rekey_ena) + val |= BIT_ULL(map->sc_id); + else + val &= ~BIT_ULL(map->sc_id); + + mcs_reg_write(mcs, reg, val); + + mcs_reg_write(mcs, MCSX_CPM_TX_SLAVE_SA_INDEX0_VLDX(map->sc_id), map->sa_index0_vld); + mcs_reg_write(mcs, MCSX_CPM_TX_SLAVE_SA_INDEX1_VLDX(map->sc_id), map->sa_index1_vld); + + mcs_reg_write(mcs, MCSX_CPM_TX_SLAVE_TX_SA_ACTIVEX(map->sc_id), map->tx_sa_active); +} + +void cnf10kb_mcs_rx_sa_mem_map_write(struct mcs *mcs, struct mcs_rx_sc_sa_map *map) +{ + u64 val, reg; + + val = (map->sa_index & 0x7F) | (map->sa_in_use << 7); + + reg = MCSX_CPM_RX_SLAVE_SA_MAP_MEMX((4 * map->sc_id) + map->an); + mcs_reg_write(mcs, reg, val); +} + +int mcs_set_force_clk_en(struct mcs *mcs, bool set) +{ + unsigned long timeout = jiffies + usecs_to_jiffies(2000); + u64 val; + + val = mcs_reg_read(mcs, MCSX_MIL_GLOBAL); + + if (set) { + val |= BIT_ULL(4); + mcs_reg_write(mcs, MCSX_MIL_GLOBAL, val); + + /* Poll till mcsx_mil_ip_gbl_status.mcs_ip_stats_ready value is 1 */ + while (!(mcs_reg_read(mcs, MCSX_MIL_IP_GBL_STATUS) & BIT_ULL(0))) { + if (time_after(jiffies, timeout)) { + dev_err(mcs->dev, "MCS set force clk enable failed\n"); + break; + } + } + } else { + val &= ~BIT_ULL(4); + mcs_reg_write(mcs, MCSX_MIL_GLOBAL, val); + } + + return 0; +} + +/* TX SA interrupt is raised only if autorekey is enabled. + * MCS_CPM_TX_SLAVE_SA_MAP_MEM_0X[sc].tx_sa_active bit gets toggled if + * one of two SAs mapped to SC gets expired. If tx_sa_active=0 implies + * SA in SA_index1 got expired else SA in SA_index0 got expired. + */ +void cnf10kb_mcs_tx_pn_thresh_reached_handler(struct mcs *mcs) +{ + struct mcs_intr_event event; + struct rsrc_bmap *sc_bmap; + unsigned long rekey_ena; + u64 val, sa_status; + int sc; + + sc_bmap = &mcs->tx.sc; + + event.mcs_id = mcs->mcs_id; + event.intr_mask = MCS_CPM_TX_PN_THRESH_REACHED_INT; + + rekey_ena = mcs_reg_read(mcs, MCSX_CPM_TX_SLAVE_AUTO_REKEY_ENABLE_0); + + for_each_set_bit(sc, sc_bmap->bmap, mcs->hw->sc_entries) { + /* Auto rekey is enable */ + if (!test_bit(sc, &rekey_ena)) + continue; + sa_status = mcs_reg_read(mcs, MCSX_CPM_TX_SLAVE_TX_SA_ACTIVEX(sc)); + /* Check if tx_sa_active status had changed */ + if (sa_status == mcs->tx_sa_active[sc]) + continue; + + /* SA_index0 is expired */ + val = mcs_reg_read(mcs, MCSX_CPM_TX_SLAVE_SA_MAP_MEM_0X(sc)); + if (sa_status) + event.sa_id = val & 0x7F; + else + event.sa_id = (val >> 7) & 0x7F; + + event.pcifunc = mcs->tx.sa2pf_map[event.sa_id]; + mcs_add_intr_wq_entry(mcs, &event); + } +} + +void cnf10kb_mcs_tx_pn_wrapped_handler(struct mcs *mcs) +{ + struct mcs_intr_event event = { 0 }; + struct rsrc_bmap *sc_bmap; + u64 val; + int sc; + + sc_bmap = &mcs->tx.sc; + + event.mcs_id = mcs->mcs_id; + event.intr_mask = MCS_CPM_TX_PACKET_XPN_EQ0_INT; + + for_each_set_bit(sc, sc_bmap->bmap, mcs->hw->sc_entries) { + val = mcs_reg_read(mcs, MCSX_CPM_TX_SLAVE_SA_MAP_MEM_0X(sc)); + + if (mcs->tx_sa_active[sc]) + /* SA_index1 was used and got expired */ + event.sa_id = (val >> 7) & 0x7F; + else + /* SA_index0 was used and got expired */ + event.sa_id = val & 0x7F; + + event.pcifunc = mcs->tx.sa2pf_map[event.sa_id]; + mcs_add_intr_wq_entry(mcs, &event); + } +} diff --git a/drivers/net/ethernet/marvell/octeontx2/af/mcs_reg.h b/drivers/net/ethernet/marvell/octeontx2/af/mcs_reg.h new file mode 100644 index 000000000000..c95a8b8f5eaf --- /dev/null +++ b/drivers/net/ethernet/marvell/octeontx2/af/mcs_reg.h @@ -0,0 +1,1102 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Marvell MCS driver + * + * Copyright (C) 2022 Marvell. + */ + +#ifndef MCS_REG_H +#define MCS_REG_H + +#include <linux/bits.h> + +/* Registers */ +#define MCSX_IP_MODE 0x900c8ull +#define MCSX_MCS_TOP_SLAVE_PORT_RESET(a) ({ \ + u64 offset; \ + \ + offset = 0x408ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0xa28ull; \ + offset += (a) * 0x8ull; \ + offset; }) + + +#define MCSX_MCS_TOP_SLAVE_CHANNEL_CFG(a) ({ \ + u64 offset; \ + \ + offset = 0x808ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0xa68ull; \ + offset += (a) * 0x8ull; \ + offset; }) + +#define MCSX_MIL_GLOBAL ({ \ + u64 offset; \ + \ + offset = 0x80000ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0x60000ull; \ + offset; }) + +#define MCSX_MIL_RX_LMACX_CFG(a) ({ \ + u64 offset; \ + \ + offset = 0x900a8ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0x700a8ull; \ + offset += (a) * 0x800ull; \ + offset; }) + +#define MCSX_HIL_GLOBAL ({ \ + u64 offset; \ + \ + offset = 0xc0000ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0xa0000ull; \ + offset; }) + +#define MCSX_LINK_LMACX_CFG(a) ({ \ + u64 offset; \ + \ + offset = 0x90000ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0x70000ull; \ + offset += (a) * 0x800ull; \ + offset; }) + +#define MCSX_MIL_RX_GBL_STATUS ({ \ + u64 offset; \ + \ + offset = 0x800c8ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0x600c8ull; \ + offset; }) + +#define MCSX_MIL_IP_GBL_STATUS ({ \ + u64 offset; \ + \ + offset = 0x800d0ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0x600d0ull; \ + offset; }) + +/* PAB */ +#define MCSX_PAB_RX_SLAVE_PORT_CFGX(a) ({ \ + u64 offset; \ + \ + offset = 0x1718ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0x280ull; \ + offset += (a) * 0x40ull; \ + offset; }) + +#define MCSX_PAB_TX_SLAVE_PORT_CFGX(a) (0x2930ull + (a) * 0x40ull) + +/* PEX registers */ +#define MCSX_PEX_RX_SLAVE_VLAN_CFGX(a) (0x3b58ull + (a) * 0x8ull) +#define MCSX_PEX_TX_SLAVE_VLAN_CFGX(a) (0x46f8ull + (a) * 0x8ull) +#define MCSX_PEX_TX_SLAVE_CUSTOM_TAG_REL_MODE_SEL(a) (0x788ull + (a) * 0x8ull) +#define MCSX_PEX_TX_SLAVE_PORT_CONFIG(a) (0x4738ull + (a) * 0x8ull) +#define MCSX_PEX_RX_SLAVE_RULE_ETYPE_CFGX(a) ({ \ + u64 offset; \ + \ + offset = 0x3fc0ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0x558ull; \ + offset += (a) * 0x8ull; \ + offset; }) + +#define MCSX_PEX_RX_SLAVE_RULE_DAX(a) ({ \ + u64 offset; \ + \ + offset = 0x4000ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0x598ull; \ + offset += (a) * 0x8ull; \ + offset; }) + +#define MCSX_PEX_RX_SLAVE_RULE_DA_RANGE_MINX(a) ({ \ + u64 offset; \ + \ + offset = 0x4040ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0x5d8ull; \ + offset += (a) * 0x8ull; \ + offset; }) + +#define MCSX_PEX_RX_SLAVE_RULE_DA_RANGE_MAXX(a) ({ \ + u64 offset; \ + \ + offset = 0x4048ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0x5e0ull; \ + offset += (a) * 0x8ull; \ + offset; }) + +#define MCSX_PEX_RX_SLAVE_RULE_COMBO_MINX(a) ({ \ + u64 offset; \ + \ + offset = 0x4080ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0x648ull; \ + offset += (a) * 0x8ull; \ + offset; }) + +#define MCSX_PEX_RX_SLAVE_RULE_COMBO_MAXX(a) ({ \ + u64 offset; \ + \ + offset = 0x4088ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0x650ull; \ + offset += (a) * 0x8ull; \ + offset; }) + +#define MCSX_PEX_RX_SLAVE_RULE_COMBO_ETX(a) ({ \ + u64 offset; \ + \ + offset = 0x4090ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0x658ull; \ + offset += (a) * 0x8ull; \ + offset; }) + +#define MCSX_PEX_RX_SLAVE_RULE_MAC ({ \ + u64 offset; \ + \ + offset = 0x40e0ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0x6d8ull; \ + offset; }) + +#define MCSX_PEX_RX_SLAVE_RULE_ENABLE ({ \ + u64 offset; \ + \ + offset = 0x40e8ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0x6e0ull; \ + offset; }) + +#define MCSX_PEX_TX_SLAVE_RULE_ETYPE_CFGX(a) ({ \ + u64 offset; \ + \ + offset = 0x4b60ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0x7d8ull; \ + offset += (a) * 0x8ull; \ + offset; }) + +#define MCSX_PEX_TX_SLAVE_RULE_DAX(a) ({ \ + u64 offset; \ + \ + offset = 0x4ba0ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0x818ull; \ + offset += (a) * 0x8ull; \ + offset; }) + +#define MCSX_PEX_TX_SLAVE_RULE_DA_RANGE_MINX(a) ({ \ + u64 offset; \ + \ + offset = 0x4be0ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0x858ull; \ + offset += (a) * 0x8ull; \ + offset; }) + +#define MCSX_PEX_TX_SLAVE_RULE_DA_RANGE_MAXX(a) ({ \ + u64 offset; \ + \ + offset = 0x4be8ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0x860ull; \ + offset += (a) * 0x8ull; \ + offset; }) + +#define MCSX_PEX_TX_SLAVE_RULE_COMBO_MINX(a) ({ \ + u64 offset; \ + \ + offset = 0x4c20ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0x8c8ull; \ + offset += (a) * 0x8ull; \ + offset; }) + +#define MCSX_PEX_TX_SLAVE_RULE_COMBO_MAXX(a) ({ \ + u64 offset; \ + \ + offset = 0x4c28ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0x8d0ull; \ + offset += (a) * 0x8ull; \ + offset; }) + +#define MCSX_PEX_TX_SLAVE_RULE_COMBO_ETX(a) ({ \ + u64 offset; \ + \ + offset = 0x4c30ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0x8d8ull; \ + offset += (a) * 0x8ull; \ + offset; }) + +#define MCSX_PEX_TX_SLAVE_RULE_MAC ({ \ + u64 offset; \ + \ + offset = 0x4c80ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0x958ull; \ + offset; }) + +#define MCSX_PEX_TX_SLAVE_RULE_ENABLE ({ \ + u64 offset; \ + \ + offset = 0x4c88ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0x960ull; \ + offset; }) + +#define MCSX_PEX_RX_SLAVE_PEX_CONFIGURATION ({ \ + u64 offset; \ + \ + offset = 0x3b50ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0x4c0ull; \ + offset; }) + +/* CNF10K-B */ +#define MCSX_PEX_RX_SLAVE_CUSTOM_TAGX(a) (0x4c8ull + (a) * 0x8ull) +#define MCSX_PEX_TX_SLAVE_CUSTOM_TAGX(a) (0x748ull + (a) * 0x8ull) +#define MCSX_PEX_RX_SLAVE_ETYPE_ENABLE 0x6e8ull +#define MCSX_PEX_TX_SLAVE_ETYPE_ENABLE 0x968ull + +/* BEE */ +#define MCSX_BBE_RX_SLAVE_PADDING_CTL 0xe08ull +#define MCSX_BBE_TX_SLAVE_PADDING_CTL 0x12f8ull +#define MCSX_BBE_RX_SLAVE_CAL_ENTRY 0x180ull +#define MCSX_BBE_RX_SLAVE_CAL_LEN 0x188ull +#define MCSX_PAB_RX_SLAVE_FIFO_SKID_CFGX(a) (0x290ull + (a) * 0x40ull) + +#define MCSX_BBE_RX_SLAVE_BBE_INT ({ \ + u64 offset; \ + \ + offset = 0xe00ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0x160ull; \ + offset; }) + +#define MCSX_BBE_RX_SLAVE_BBE_INT_ENB ({ \ + u64 offset; \ + \ + offset = 0xe08ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0x168ull; \ + offset; }) + +#define MCSX_BBE_RX_SLAVE_BBE_INT_INTR_RW ({ \ + u64 offset; \ + \ + offset = 0xe08ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0x178ull; \ + offset; }) + +#define MCSX_BBE_TX_SLAVE_BBE_INT ({ \ + u64 offset; \ + \ + offset = 0x1278ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0x1e0ull; \ + offset; }) + +#define MCSX_BBE_TX_SLAVE_BBE_INT_INTR_RW ({ \ + u64 offset; \ + \ + offset = 0x1278ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0x1f8ull; \ + offset; }) + +#define MCSX_BBE_TX_SLAVE_BBE_INT_ENB ({ \ + u64 offset; \ + \ + offset = 0x1280ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0x1e8ull; \ + offset; }) + +#define MCSX_PAB_RX_SLAVE_PAB_INT ({ \ + u64 offset; \ + \ + offset = 0x16f0ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0x260ull; \ + offset; }) + +#define MCSX_PAB_RX_SLAVE_PAB_INT_ENB ({ \ + u64 offset; \ + \ + offset = 0x16f8ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0x268ull; \ + offset; }) + +#define MCSX_PAB_RX_SLAVE_PAB_INT_INTR_RW ({ \ + u64 offset; \ + \ + offset = 0x16f8ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0x278ull; \ + offset; }) + +#define MCSX_PAB_TX_SLAVE_PAB_INT ({ \ + u64 offset; \ + \ + offset = 0x2908ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0x380ull; \ + offset; }) + +#define MCSX_PAB_TX_SLAVE_PAB_INT_ENB ({ \ + u64 offset; \ + \ + offset = 0x2910ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0x388ull; \ + offset; }) + +#define MCSX_PAB_TX_SLAVE_PAB_INT_INTR_RW ({ \ + u64 offset; \ + \ + offset = 0x16f8ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0x398ull; \ + offset; }) + +/* CPM registers */ +#define MCSX_CPM_RX_SLAVE_FLOWID_TCAM_DATAX(a, b) ({ \ + u64 offset; \ + \ + offset = 0x30740ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0x3bf8ull; \ + offset += (a) * 0x8ull + (b) * 0x20ull; \ + offset; }) + +#define MCSX_CPM_RX_SLAVE_FLOWID_TCAM_MASKX(a, b) ({ \ + u64 offset; \ + \ + offset = 0x34740ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0x43f8ull; \ + offset += (a) * 0x8ull + (b) * 0x20ull; \ + offset; }) + +#define MCSX_CPM_RX_SLAVE_FLOWID_TCAM_ENA_0 ({ \ + u64 offset; \ + \ + offset = 0x30700ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0x3bd8ull; \ + offset; }) + +#define MCSX_CPM_RX_SLAVE_SC_CAMX(a, b) ({ \ + u64 offset; \ + \ + offset = 0x38780ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0x4c08ull; \ + offset += (a) * 0x8ull + (b) * 0x10ull; \ + offset; }) + +#define MCSX_CPM_RX_SLAVE_SC_CAM_ENA(a) ({ \ + u64 offset; \ + \ + offset = 0x38740ull + (a) * 0x8ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0x4bf8ull; \ + offset; }) + +#define MCSX_CPM_RX_SLAVE_SECY_MAP_MEMX(a) ({ \ + u64 offset; \ + \ + offset = 0x23ee0ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0xbd0ull; \ + offset += (a) * 0x8ull; \ + offset; }) + +#define MCSX_CPM_RX_SLAVE_SECY_PLCY_MEM_0X(a) ({ \ + u64 offset; \ + \ + offset = (0x246e0ull + (a) * 0x10ull); \ + if (mcs->hw->mcs_blks > 1) \ + offset = (0xdd0ull + (a) * 0x8ull); \ + offset; }) + +#define MCSX_CPM_RX_SLAVE_SA_KEY_LOCKOUTX(a) ({ \ + u64 offset; \ + \ + offset = 0x23E90ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0xbb0ull; \ + offset += (a) * 0x8ull; \ + offset; }) + +#define MCSX_CPM_RX_SLAVE_SA_MAP_MEMX(a) ({ \ + u64 offset; \ + \ + offset = 0x256e0ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0xfd0ull; \ + offset += (a) * 0x8ull; \ + offset; }) + +#define MCSX_CPM_RX_SLAVE_SA_PLCY_MEMX(a, b) ({ \ + u64 offset; \ + \ + offset = 0x27700ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0x17d8ull; \ + offset += (a) * 0x8ull + (b) * 0x40ull; \ + offset; }) + +#define MCSX_CPM_RX_SLAVE_SA_PN_TABLE_MEMX(a) ({ \ + u64 offset; \ + \ + offset = 0x2f700ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0x37d8; \ + offset += (a) * 0x8ull; \ + offset; }) + +#define MCSX_CPM_RX_SLAVE_XPN_THRESHOLD ({ \ + u64 offset; \ + \ + offset = 0x23e40ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0xb90ull; \ + offset; }) + +#define MCSX_CPM_RX_SLAVE_PN_THRESHOLD ({ \ + u64 offset; \ + \ + offset = 0x23e48ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0xb98ull; \ + offset; }) + +#define MCSX_CPM_RX_SLAVE_PN_THRESH_REACHEDX(a) ({ \ + u64 offset; \ + \ + offset = 0x23e50ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0xba0ull; \ + offset += (a) * 0x8ull; \ + offset; }) + +#define MCSX_CPM_RX_SLAVE_FLOWID_TCAM_ENA_1 0x30708ull +#define MCSX_CPM_RX_SLAVE_SECY_PLCY_MEM_1X(a) (0x246e8ull + (a) * 0x10ull) + +/* TX registers */ +#define MCSX_CPM_TX_SLAVE_FLOWID_TCAM_DATAX(a, b) ({ \ + u64 offset; \ + \ + offset = 0x51d50ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0xa7c0ull; \ + offset += (a) * 0x8ull + (b) * 0x20ull; \ + offset; }) + +#define MCSX_CPM_TX_SLAVE_FLOWID_TCAM_MASKX(a, b) ({ \ + u64 offset; \ + \ + offset = 0x55d50ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0xafc0ull; \ + offset += (a) * 0x8ull + (b) * 0x20ull; \ + offset; }) + +#define MCSX_CPM_TX_SLAVE_FLOWID_TCAM_ENA_0 ({ \ + u64 offset; \ + \ + offset = 0x51d10ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0xa7a0ull; \ + offset; }) + +#define MCSX_CPM_TX_SLAVE_SECY_MAP_MEM_0X(a) ({ \ + u64 offset; \ + \ + offset = 0x3e508ull + (a) * 0x8ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0x5550ull + (a) * 0x10ull; \ + offset; }) + +#define MCSX_CPM_TX_SLAVE_SECY_PLCY_MEMX(a) ({ \ + u64 offset; \ + \ + offset = 0x3ed08ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0x5950ull; \ + offset += (a) * 0x8ull; \ + offset; }) + +#define MCSX_CPM_TX_SLAVE_SA_KEY_LOCKOUTX(a) ({ \ + u64 offset; \ + \ + offset = 0x3e4c0ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0x5538ull; \ + offset += (a) * 0x8ull; \ + offset; }) + +#define MCSX_CPM_TX_SLAVE_SA_MAP_MEM_0X(a) ({ \ + u64 offset; \ + \ + offset = 0x3fd10ull + (a) * 0x10ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0x6150ull + (a) * 0x8ull; \ + offset; }) + +#define MCSX_CPM_TX_SLAVE_SA_PLCY_MEMX(a, b) ({ \ + u64 offset; \ + \ + offset = 0x40d10ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0x63a0ull; \ + offset += (a) * 0x8ull + (b) * 0x80ull; \ + offset; }) + +#define MCSX_CPM_TX_SLAVE_SA_PN_TABLE_MEMX(a) ({ \ + u64 offset; \ + \ + offset = 0x50d10ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0xa3a0ull; \ + offset += (a) * 0x8ull; \ + offset; }) + +#define MCSX_CPM_TX_SLAVE_XPN_THRESHOLD ({ \ + u64 offset; \ + \ + offset = 0x3e4b0ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0x5528ull; \ + offset; }) + +#define MCSX_CPM_TX_SLAVE_PN_THRESHOLD ({ \ + u64 offset; \ + \ + offset = 0x3e4b8ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0x5530ull; \ + offset; }) + +#define MCSX_CPM_TX_SLAVE_SA_MAP_MEM_1X(a) (0x3fd18ull + (a) * 0x10ull) +#define MCSX_CPM_TX_SLAVE_SECY_MAP_MEM_1X(a) (0x5558ull + (a) * 0x10ull) +#define MCSX_CPM_TX_SLAVE_FLOWID_TCAM_ENA_1 0x51d18ull +#define MCSX_CPM_TX_SLAVE_TX_SA_ACTIVEX(a) (0x5b50 + (a) * 0x8ull) +#define MCSX_CPM_TX_SLAVE_SA_INDEX0_VLDX(a) (0x5d50 + (a) * 0x8ull) +#define MCSX_CPM_TX_SLAVE_SA_INDEX1_VLDX(a) (0x5f50 + (a) * 0x8ull) +#define MCSX_CPM_TX_SLAVE_AUTO_REKEY_ENABLE_0 0x5500ull + +/* CSE */ +#define MCSX_CSE_RX_MEM_SLAVE_IFINCTLBCPKTSX(a) ({ \ + u64 offset; \ + \ + offset = 0x9e80ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0xc218ull; \ + offset += (a) * 0x8ull; \ + offset; }) + +#define MCSX_CSE_RX_MEM_SLAVE_IFINCTLMCPKTSX(a) ({ \ + u64 offset; \ + \ + offset = 0x9680ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0xc018ull; \ + offset += (a) * 0x8ull; \ + offset; }) + +#define MCSX_CSE_RX_MEM_SLAVE_IFINCTLOCTETSX(a) ({ \ + u64 offset; \ + \ + offset = 0x6e80ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0xbc18ull; \ + offset += (a) * 0x8ull; \ + offset; }) + +#define MCSX_CSE_RX_MEM_SLAVE_IFINCTLUCPKTSX(a) ({ \ + u64 offset; \ + \ + offset = 0x8e80ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0xbe18ull; \ + offset += (a) * 0x8ull; \ + offset; }) + +#define MCSX_CSE_RX_MEM_SLAVE_IFINUNCTLBCPKTSX(a) ({ \ + u64 offset; \ + \ + offset = 0x8680ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0xca18ull; \ + offset += (a) * 0x8ull; \ + offset; }) + +#define MCSX_CSE_RX_MEM_SLAVE_IFINUNCTLMCPKTSX(a) ({ \ + u64 offset; \ + \ + offset = 0x7e80ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0xc818ull; \ + offset += (a) * 0x8ull; \ + offset; }) + +#define MCSX_CSE_RX_MEM_SLAVE_IFINUNCTLOCTETSX(a) ({ \ + u64 offset; \ + \ + offset = 0x6680ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0xc418ull; \ + offset += (a) * 0x8ull; \ + offset; }) + +#define MCSX_CSE_RX_MEM_SLAVE_IFINUNCTLUCPKTSX(a) ({ \ + u64 offset; \ + \ + offset = 0x7680ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0xc618ull; \ + offset += (a) * 0x8ull; \ + offset; }) + +#define MCSX_CSE_RX_MEM_SLAVE_INOCTETSSECYDECRYPTEDX(a) ({ \ + u64 offset; \ + \ + offset = 0x5e80ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0xdc18ull; \ + offset += (a) * 0x8ull; \ + offset; }) + +#define MCSX_CSE_RX_MEM_SLAVE_INOCTETSSECYVALIDATEX(a)({ \ + u64 offset; \ + \ + offset = 0x5680ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0xda18ull; \ + offset += (a) * 0x8ull; \ + offset; }) + +#define MCSX_CSE_RX_MEM_SLAVE_INPKTSCTRLPORTDISABLEDX(a) ({ \ + u64 offset; \ + \ + offset = 0xd680ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0xce18ull; \ + offset += (a) * 0x8ull; \ + offset; }) + +#define MCSX_CSE_RX_MEM_SLAVE_INPKTSFLOWIDTCAMHITX(a) ({ \ + u64 offset; \ + \ + offset = 0x16a80ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0xec78ull; \ + offset += (a) * 0x8ull; \ + offset; }) + +#define MCSX_CSE_RX_MEM_SLAVE_INPKTSFLOWIDTCAMMISSX(a) ({ \ + u64 offset; \ + \ + offset = 0x16680ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0xec38ull; \ + offset += (a) * 0x8ull; \ + offset; }) + +#define MCSX_CSE_RX_MEM_SLAVE_INPKTSPARSEERRX(a) ({ \ + u64 offset; \ + \ + offset = 0x16880ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0xec18ull; \ + offset += (a) * 0x8ull; \ + offset; }) + +#define MCSX_CSE_RX_MEM_SLAVE_INPKTSSCCAMHITX(a) ({ \ + u64 offset; \ + \ + offset = 0xfe80ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0xde18ull; \ + offset += (a) * 0x8ull; \ + offset; }) + +#define MCSX_CSE_RX_MEM_SLAVE_INPKTSSCINVALIDX(a) ({ \ + u64 offset; \ + \ + offset = 0x10680ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0xe418ull; \ + offset += (a) * 0x8ull; \ + offset; }) + +#define MCSX_CSE_RX_MEM_SLAVE_INPKTSSCNOTVALIDX(a) ({ \ + u64 offset; \ + \ + offset = 0x10e80ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0xe218ull; \ + offset += (a) * 0x8ull; \ + offset; }) + +#define MCSX_CSE_RX_MEM_SLAVE_INPKTSSECYBADTAGX(a) ({ \ + u64 offset; \ + \ + offset = 0xae80ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0xd418ull; \ + offset += (a) * 0x8ull; \ + offset; }) + +#define MCSX_CSE_RX_MEM_SLAVE_INPKTSSECYNOSAX(a) ({ \ + u64 offset; \ + \ + offset = 0xc680ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0xd618ull; \ + offset += (a) * 0x8ull; \ + offset; }) + +#define MCSX_CSE_RX_MEM_SLAVE_INPKTSSECYNOSAERRORX(a) ({ \ + u64 offset; \ + \ + offset = 0xce80ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0xd818ull; \ + offset += (a) * 0x8ull; \ + offset; }) + +#define MCSX_CSE_RX_MEM_SLAVE_INPKTSSECYTAGGEDCTLX(a) ({ \ + u64 offset; \ + \ + offset = 0xbe80ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0xcc18ull; \ + offset += (a) * 0x8ull; \ + offset; }) + +#define MCSX_CSE_RX_SLAVE_CTRL ({ \ + u64 offset; \ + \ + offset = 0x52a0ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0x9c0ull; \ + offset; }) + +#define MCSX_CSE_RX_SLAVE_STATS_CLEAR ({ \ + u64 offset; \ + \ + offset = 0x52b8ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0x9d8ull; \ + offset; }) + +#define MCSX_CSE_RX_MEM_SLAVE_INOCTETSSCDECRYPTEDX(a) (0xe680ull + (a) * 0x8ull) +#define MCSX_CSE_RX_MEM_SLAVE_INOCTETSSCVALIDATEX(a) (0xde80ull + (a) * 0x8ull) +#define MCSX_CSE_RX_MEM_SLAVE_INPKTSSECYUNTAGGEDORNOTAGX(a) (0xa680ull + (a) * 0x8ull) +#define MCSX_CSE_RX_MEM_SLAVE_INPKTSSECYNOTAGX(a) (0xd218 + (a) * 0x8ull) +#define MCSX_CSE_RX_MEM_SLAVE_INPKTSSECYUNTAGGEDX(a) (0xd018ull + (a) * 0x8ull) +#define MCSX_CSE_RX_MEM_SLAVE_INPKTSSCUNCHECKEDOROKX(a) (0xee80ull + (a) * 0x8ull) +#define MCSX_CSE_RX_MEM_SLAVE_INPKTSSECYCTLX(a) (0xb680ull + (a) * 0x8ull) +#define MCSX_CSE_RX_MEM_SLAVE_INPKTSSCLATEORDELAYEDX(a) (0xf680ull + (a) * 0x8ull) +#define MCSX_CSE_RX_MEM_SLAVE_INPKTSSAINVALIDX(a) (0x12680ull + (a) * 0x8ull) +#define MCSX_CSE_RX_MEM_SLAVE_INPKTSSANOTUSINGSAERRORX(a) (0x15680ull + (a) * 0x8ull) +#define MCSX_CSE_RX_MEM_SLAVE_INPKTSSANOTVALIDX(a) (0x13680ull + (a) * 0x8ull) +#define MCSX_CSE_RX_MEM_SLAVE_INPKTSSAOKX(a) (0x11680ull + (a) * 0x8ull) +#define MCSX_CSE_RX_MEM_SLAVE_INPKTSSAUNUSEDSAX(a) (0x14680ull + (a) * 0x8ull) +#define MCSX_CSE_RX_MEM_SLAVE_INPKTSEARLYPREEMPTERRX(a) (0xec58ull + (a) * 0x8ull) +#define MCSX_CSE_RX_MEM_SLAVE_INPKTSSCOKX(a) (0xea18ull + (a) * 0x8ull) +#define MCSX_CSE_RX_MEM_SLAVE_INPKTSSCDELAYEDX(a) (0xe618ull + (a) * 0x8ull) + +/* CSE TX */ +#define MCSX_CSE_TX_MEM_SLAVE_IFOUTCOMMONOCTETSX(a) (0x18440ull + (a) * 0x8ull) +#define MCSX_CSE_TX_MEM_SLAVE_IFOUTCTLBCPKTSX(a) ({ \ + u64 offset; \ + \ + offset = 0x1c440ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0xf478ull; \ + offset += (a) * 0x8ull; \ + offset; }) + +#define MCSX_CSE_TX_MEM_SLAVE_IFOUTCTLMCPKTSX(a) ({ \ + u64 offset; \ + \ + offset = 0x1bc40ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0xf278ull; \ + offset += (a) * 0x8ull; \ + offset; }) + +#define MCSX_CSE_TX_MEM_SLAVE_IFOUTCTLOCTETSX(a) ({ \ + u64 offset; \ + \ + offset = 0x19440ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0xee78ull; \ + offset += (a) * 0x8ull; \ + offset; }) + +#define MCSX_CSE_TX_MEM_SLAVE_IFOUTCTLUCPKTSX(a) ({ \ + u64 offset; \ + \ + offset = 0x1b440ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0xf078ull; \ + offset += (a) * 0x8ull; \ + offset; }) + +#define MCSX_CSE_TX_MEM_SLAVE_IFOUTUNCTLBCPKTSX(a) ({ \ + u64 offset; \ + \ + offset = 0x1ac40ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0xfc78ull; \ + offset += (a) * 0x8ull; \ + offset; }) + +#define MCSX_CSE_TX_MEM_SLAVE_IFOUTUNCTLMCPKTSX(a) ({ \ + u64 offset; \ + \ + offset = 0x1a440ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0xfa78ull; \ + offset += (a) * 0x8ull; \ + offset; }) + +#define MCSX_CSE_TX_MEM_SLAVE_IFOUTUNCTLOCTETSX(a) ({ \ + u64 offset; \ + \ + offset = 0x18c40ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0xf678ull; \ + offset += (a) * 0x8ull; \ + offset; }) + +#define MCSX_CSE_TX_MEM_SLAVE_IFOUTUNCTLUCPKTSX(a) ({ \ + u64 offset; \ + \ + offset = 0x19c40ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0xf878ull; \ + offset += (a) * 0x8ull; \ + offset; }) + +#define MCSX_CSE_TX_MEM_SLAVE_OUTOCTETSSECYENCRYPTEDX(a) ({ \ + u64 offset; \ + \ + offset = 0x17c40ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0x10878ull; \ + offset += (a) * 0x8ull; \ + offset; }) + +#define MCSX_CSE_TX_MEM_SLAVE_OUTOCTETSSECYPROTECTEDX(a) ({ \ + u64 offset; \ + \ + offset = 0x17440ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0x10678ull; \ + offset += (a) * 0x8ull; \ + offset; }) + +#define MCSX_CSE_TX_MEM_SLAVE_OUTPKTSCTRLPORTDISABLEDX(a) ({ \ + u64 offset; \ + \ + offset = 0x1e440ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0xfe78ull; \ + offset += (a) * 0x8ull; \ + offset; }) + +#define MCSX_CSE_TX_MEM_SLAVE_OUTPKTSFLOWIDTCAMHITX(a) ({ \ + u64 offset; \ + \ + offset = 0x23240ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0x10ed8ull; \ + offset += (a) * 0x8ull; \ + offset; }) + +#define MCSX_CSE_TX_MEM_SLAVE_OUTPKTSFLOWIDTCAMMISSX(a) ({ \ + u64 offset; \ + \ + offset = 0x22c40ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0x10e98ull; \ + offset += (a) * 0x8ull; \ + offset; }) + +#define MCSX_CSE_TX_MEM_SLAVE_OUTPKTSPARSEERRX(a) ({ \ + u64 offset; \ + \ + offset = 0x22e40ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0x10e78ull; \ + offset += (a) * 0x8ull; \ + offset; }) + +#define MCSX_CSE_TX_MEM_SLAVE_OUTPKTSSCENCRYPTEDX(a) ({ \ + u64 offset; \ + \ + offset = 0x20440ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0x10c78ull; \ + offset += (a) * 0x8ull; \ + offset; }) + +#define MCSX_CSE_TX_MEM_SLAVE_OUTPKTSSCPROTECTEDX(a) ({ \ + u64 offset; \ + \ + offset = 0x1fc40ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0x10a78ull; \ + offset += (a) * 0x8ull; \ + offset; }) + +#define MCSX_CSE_TX_MEM_SLAVE_OUTPKTSSECTAGINSERTIONERRX(a) ({ \ + u64 offset; \ + \ + offset = 0x23040ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0x110d8ull; \ + offset += (a) * 0x8ull; \ + offset; }) + +#define MCSX_CSE_TX_MEM_SLAVE_OUTPKTSSECYNOACTIVESAX(a) ({ \ + u64 offset; \ + \ + offset = 0x1dc40ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0x10278ull; \ + offset += (a) * 0x8ull; \ + offset; }) + +#define MCSX_CSE_TX_MEM_SLAVE_OUTPKTSSECYTOOLONGX(a) ({ \ + u64 offset; \ + \ + offset = 0x1d440ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0x10478ull; \ + offset += (a) * 0x8ull; \ + offset; }) + +#define MCSX_CSE_TX_MEM_SLAVE_OUTPKTSSECYUNTAGGEDX(a) ({ \ + u64 offset; \ + \ + offset = 0x1cc40ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0x10078ull; \ + offset += (a) * 0x8ull; \ + offset; }) + +#define MCSX_CSE_TX_SLAVE_CTRL ({ \ + u64 offset; \ + \ + offset = 0x54a0ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0xa00ull; \ + offset; }) + +#define MCSX_CSE_TX_SLAVE_STATS_CLEAR ({ \ + u64 offset; \ + \ + offset = 0x54b8ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0xa18ull; \ + offset; }) + +#define MCSX_CSE_TX_MEM_SLAVE_OUTOCTETSSCENCRYPTEDX(a) (0x1f440ull + (a) * 0x8ull) +#define MCSX_CSE_TX_MEM_SLAVE_OUTOCTETSSCPROTECTEDX(a) (0x1ec40ull + (a) * 0x8ull) +#define MCSX_CSE_TX_MEM_SLAVE_OUTPKTSEARLYPREEMPTERRX(a) (0x10eb8ull + (a) * 0x8ull) +#define MCSX_CSE_TX_MEM_SLAVE_OUTPKTSSAENCRYPTEDX(a) (0x21c40ull + (a) * 0x8ull) +#define MCSX_CSE_TX_MEM_SLAVE_OUTPKTSSAPROTECTEDX(a) (0x20c40ull + (a) * 0x8ull) + +#define MCSX_IP_INT ({ \ + u64 offset; \ + \ + offset = 0x80028ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0x60028ull; \ + offset; }) + +#define MCSX_IP_INT_ENA_W1S ({ \ + u64 offset; \ + \ + offset = 0x80040ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0x60040ull; \ + offset; }) + +#define MCSX_IP_INT_ENA_W1C ({ \ + u64 offset; \ + \ + offset = 0x80038ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0x60038ull; \ + offset; }) + +#define MCSX_TOP_SLAVE_INT_SUM ({ \ + u64 offset; \ + \ + offset = 0xc20ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0xab8ull; \ + offset; }) + +#define MCSX_TOP_SLAVE_INT_SUM_ENB ({ \ + u64 offset; \ + \ + offset = 0xc28ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0xac0ull; \ + offset; }) + +#define MCSX_CPM_RX_SLAVE_RX_INT ({ \ + u64 offset; \ + \ + offset = 0x23c00ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0x0ad8ull; \ + offset; }) + +#define MCSX_CPM_RX_SLAVE_RX_INT_ENB ({ \ + u64 offset; \ + \ + offset = 0x23c08ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0xae0ull; \ + offset; }) + +#define MCSX_CPM_TX_SLAVE_TX_INT ({ \ + u64 offset; \ + \ + offset = 0x3d490ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0x54a0ull; \ + offset; }) + +#define MCSX_CPM_TX_SLAVE_TX_INT_ENB ({ \ + u64 offset; \ + \ + offset = 0x3d498ull; \ + if (mcs->hw->mcs_blks > 1) \ + offset = 0x54a8ull; \ + offset; }) + +#endif diff --git a/drivers/net/ethernet/marvell/octeontx2/af/mcs_rvu_if.c b/drivers/net/ethernet/marvell/octeontx2/af/mcs_rvu_if.c new file mode 100644 index 000000000000..fa8029a94068 --- /dev/null +++ b/drivers/net/ethernet/marvell/octeontx2/af/mcs_rvu_if.c @@ -0,0 +1,889 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Marvell CN10K MCS driver + * + * Copyright (C) 2022 Marvell. + */ + +#include <linux/types.h> +#include <linux/device.h> +#include <linux/module.h> +#include <linux/pci.h> + +#include "mcs.h" +#include "rvu.h" +#include "lmac_common.h" + +#define M(_name, _id, _fn_name, _req_type, _rsp_type) \ +static struct _req_type __maybe_unused \ +*otx2_mbox_alloc_msg_ ## _fn_name(struct rvu *rvu, int devid) \ +{ \ + struct _req_type *req; \ + \ + req = (struct _req_type *)otx2_mbox_alloc_msg_rsp( \ + &rvu->afpf_wq_info.mbox_up, devid, sizeof(struct _req_type), \ + sizeof(struct _rsp_type)); \ + if (!req) \ + return NULL; \ + req->hdr.sig = OTX2_MBOX_REQ_SIG; \ + req->hdr.id = _id; \ + return req; \ +} + +MBOX_UP_MCS_MESSAGES +#undef M + +int rvu_mbox_handler_mcs_set_lmac_mode(struct rvu *rvu, + struct mcs_set_lmac_mode *req, + struct msg_rsp *rsp) +{ + struct mcs *mcs; + + if (req->mcs_id >= rvu->mcs_blk_cnt) + return MCS_AF_ERR_INVALID_MCSID; + + mcs = mcs_get_pdata(req->mcs_id); + + if (BIT_ULL(req->lmac_id) & mcs->hw->lmac_bmap) + mcs_set_lmac_mode(mcs, req->lmac_id, req->mode); + + return 0; +} + +int mcs_add_intr_wq_entry(struct mcs *mcs, struct mcs_intr_event *event) +{ + struct mcs_intrq_entry *qentry; + u16 pcifunc = event->pcifunc; + struct rvu *rvu = mcs->rvu; + struct mcs_pfvf *pfvf; + + /* Check if it is PF or VF */ + if (pcifunc & RVU_PFVF_FUNC_MASK) + pfvf = &mcs->vf[rvu_get_hwvf(rvu, pcifunc)]; + else + pfvf = &mcs->pf[rvu_get_pf(pcifunc)]; + + event->intr_mask &= pfvf->intr_mask; + + /* Check PF/VF interrupt notification is enabled */ + if (!(pfvf->intr_mask && event->intr_mask)) + return 0; + + qentry = kmalloc(sizeof(*qentry), GFP_ATOMIC); + if (!qentry) + return -ENOMEM; + + qentry->intr_event = *event; + spin_lock(&rvu->mcs_intrq_lock); + list_add_tail(&qentry->node, &rvu->mcs_intrq_head); + spin_unlock(&rvu->mcs_intrq_lock); + queue_work(rvu->mcs_intr_wq, &rvu->mcs_intr_work); + + return 0; +} + +static int mcs_notify_pfvf(struct mcs_intr_event *event, struct rvu *rvu) +{ + struct mcs_intr_info *req; + int err, pf; + + pf = rvu_get_pf(event->pcifunc); + + req = otx2_mbox_alloc_msg_mcs_intr_notify(rvu, pf); + if (!req) + return -ENOMEM; + + req->mcs_id = event->mcs_id; + req->intr_mask = event->intr_mask; + req->sa_id = event->sa_id; + req->hdr.pcifunc = event->pcifunc; + req->lmac_id = event->lmac_id; + + otx2_mbox_msg_send(&rvu->afpf_wq_info.mbox_up, pf); + err = otx2_mbox_wait_for_rsp(&rvu->afpf_wq_info.mbox_up, pf); + if (err) + dev_warn(rvu->dev, "MCS notification to pf %d failed\n", pf); + + return 0; +} + +static void mcs_intr_handler_task(struct work_struct *work) +{ + struct rvu *rvu = container_of(work, struct rvu, mcs_intr_work); + struct mcs_intrq_entry *qentry; + struct mcs_intr_event *event; + unsigned long flags; + + do { + spin_lock_irqsave(&rvu->mcs_intrq_lock, flags); + qentry = list_first_entry_or_null(&rvu->mcs_intrq_head, + struct mcs_intrq_entry, + node); + if (qentry) + list_del(&qentry->node); + + spin_unlock_irqrestore(&rvu->mcs_intrq_lock, flags); + if (!qentry) + break; /* nothing more to process */ + + event = &qentry->intr_event; + + mcs_notify_pfvf(event, rvu); + kfree(qentry); + } while (1); +} + +int rvu_mbox_handler_mcs_intr_cfg(struct rvu *rvu, + struct mcs_intr_cfg *req, + struct msg_rsp *rsp) +{ + u16 pcifunc = req->hdr.pcifunc; + struct mcs_pfvf *pfvf; + struct mcs *mcs; + + if (req->mcs_id >= rvu->mcs_blk_cnt) + return MCS_AF_ERR_INVALID_MCSID; + + mcs = mcs_get_pdata(req->mcs_id); + + /* Check if it is PF or VF */ + if (pcifunc & RVU_PFVF_FUNC_MASK) + pfvf = &mcs->vf[rvu_get_hwvf(rvu, pcifunc)]; + else + pfvf = &mcs->pf[rvu_get_pf(pcifunc)]; + + mcs->pf_map[0] = pcifunc; + pfvf->intr_mask = req->intr_mask; + + return 0; +} + +int rvu_mbox_handler_mcs_get_hw_info(struct rvu *rvu, + struct msg_req *req, + struct mcs_hw_info *rsp) +{ + struct mcs *mcs; + + if (!rvu->mcs_blk_cnt) + return MCS_AF_ERR_NOT_MAPPED; + + /* MCS resources are same across all blocks */ + mcs = mcs_get_pdata(0); + rsp->num_mcs_blks = rvu->mcs_blk_cnt; + rsp->tcam_entries = mcs->hw->tcam_entries; + rsp->secy_entries = mcs->hw->secy_entries; + rsp->sc_entries = mcs->hw->sc_entries; + rsp->sa_entries = mcs->hw->sa_entries; + return 0; +} + +int rvu_mbox_handler_mcs_port_reset(struct rvu *rvu, struct mcs_port_reset_req *req, + struct msg_rsp *rsp) +{ + struct mcs *mcs; + + if (req->mcs_id >= rvu->mcs_blk_cnt) + return MCS_AF_ERR_INVALID_MCSID; + + mcs = mcs_get_pdata(req->mcs_id); + + mcs_reset_port(mcs, req->port_id, req->reset); + + return 0; +} + +int rvu_mbox_handler_mcs_clear_stats(struct rvu *rvu, + struct mcs_clear_stats *req, + struct msg_rsp *rsp) +{ + u16 pcifunc = req->hdr.pcifunc; + struct mcs *mcs; + + if (req->mcs_id >= rvu->mcs_blk_cnt) + return MCS_AF_ERR_INVALID_MCSID; + + mcs = mcs_get_pdata(req->mcs_id); + + mutex_lock(&mcs->stats_lock); + if (req->all) + mcs_clear_all_stats(mcs, pcifunc, req->dir); + else + mcs_clear_stats(mcs, req->type, req->id, req->dir); + + mutex_unlock(&mcs->stats_lock); + return 0; +} + +int rvu_mbox_handler_mcs_get_flowid_stats(struct rvu *rvu, + struct mcs_stats_req *req, + struct mcs_flowid_stats *rsp) +{ + struct mcs *mcs; + + if (req->mcs_id >= rvu->mcs_blk_cnt) + return MCS_AF_ERR_INVALID_MCSID; + + mcs = mcs_get_pdata(req->mcs_id); + + /* In CNF10K-B, before reading the statistics, + * MCSX_MIL_GLOBAL.FORCE_CLK_EN_IP needs to be set + * to get accurate statistics + */ + if (mcs->hw->mcs_blks > 1) + mcs_set_force_clk_en(mcs, true); + + mutex_lock(&mcs->stats_lock); + mcs_get_flowid_stats(mcs, rsp, req->id, req->dir); + mutex_unlock(&mcs->stats_lock); + + /* Clear MCSX_MIL_GLOBAL.FORCE_CLK_EN_IP after reading + * the statistics + */ + if (mcs->hw->mcs_blks > 1) + mcs_set_force_clk_en(mcs, false); + + return 0; +} + +int rvu_mbox_handler_mcs_get_secy_stats(struct rvu *rvu, + struct mcs_stats_req *req, + struct mcs_secy_stats *rsp) +{ struct mcs *mcs; + + if (req->mcs_id >= rvu->mcs_blk_cnt) + return MCS_AF_ERR_INVALID_MCSID; + + mcs = mcs_get_pdata(req->mcs_id); + + if (mcs->hw->mcs_blks > 1) + mcs_set_force_clk_en(mcs, true); + + mutex_lock(&mcs->stats_lock); + + if (req->dir == MCS_RX) + mcs_get_rx_secy_stats(mcs, rsp, req->id); + else + mcs_get_tx_secy_stats(mcs, rsp, req->id); + + mutex_unlock(&mcs->stats_lock); + + if (mcs->hw->mcs_blks > 1) + mcs_set_force_clk_en(mcs, false); + + return 0; +} + +int rvu_mbox_handler_mcs_get_sc_stats(struct rvu *rvu, + struct mcs_stats_req *req, + struct mcs_sc_stats *rsp) +{ + struct mcs *mcs; + + if (req->mcs_id >= rvu->mcs_blk_cnt) + return MCS_AF_ERR_INVALID_MCSID; + + mcs = mcs_get_pdata(req->mcs_id); + + if (mcs->hw->mcs_blks > 1) + mcs_set_force_clk_en(mcs, true); + + mutex_lock(&mcs->stats_lock); + mcs_get_sc_stats(mcs, rsp, req->id, req->dir); + mutex_unlock(&mcs->stats_lock); + + if (mcs->hw->mcs_blks > 1) + mcs_set_force_clk_en(mcs, false); + + return 0; +} + +int rvu_mbox_handler_mcs_get_sa_stats(struct rvu *rvu, + struct mcs_stats_req *req, + struct mcs_sa_stats *rsp) +{ + struct mcs *mcs; + + if (req->mcs_id >= rvu->mcs_blk_cnt) + return MCS_AF_ERR_INVALID_MCSID; + + mcs = mcs_get_pdata(req->mcs_id); + + if (mcs->hw->mcs_blks > 1) + mcs_set_force_clk_en(mcs, true); + + mutex_lock(&mcs->stats_lock); + mcs_get_sa_stats(mcs, rsp, req->id, req->dir); + mutex_unlock(&mcs->stats_lock); + + if (mcs->hw->mcs_blks > 1) + mcs_set_force_clk_en(mcs, false); + + return 0; +} + +int rvu_mbox_handler_mcs_get_port_stats(struct rvu *rvu, + struct mcs_stats_req *req, + struct mcs_port_stats *rsp) +{ + struct mcs *mcs; + + if (req->mcs_id >= rvu->mcs_blk_cnt) + return MCS_AF_ERR_INVALID_MCSID; + + mcs = mcs_get_pdata(req->mcs_id); + + if (mcs->hw->mcs_blks > 1) + mcs_set_force_clk_en(mcs, true); + + mutex_lock(&mcs->stats_lock); + mcs_get_port_stats(mcs, rsp, req->id, req->dir); + mutex_unlock(&mcs->stats_lock); + + if (mcs->hw->mcs_blks > 1) + mcs_set_force_clk_en(mcs, false); + + return 0; +} + +int rvu_mbox_handler_mcs_set_active_lmac(struct rvu *rvu, + struct mcs_set_active_lmac *req, + struct msg_rsp *rsp) +{ + struct mcs *mcs; + + if (req->mcs_id >= rvu->mcs_blk_cnt) + return MCS_AF_ERR_INVALID_MCSID; + + mcs = mcs_get_pdata(req->mcs_id); + if (!mcs) + return MCS_AF_ERR_NOT_MAPPED; + + mcs->hw->lmac_bmap = req->lmac_bmap; + mcs_set_lmac_channels(req->mcs_id, req->chan_base); + return 0; +} + +int rvu_mbox_handler_mcs_port_cfg_set(struct rvu *rvu, struct mcs_port_cfg_set_req *req, + struct msg_rsp *rsp) +{ + struct mcs *mcs; + + if (req->mcs_id >= rvu->mcs_blk_cnt) + return MCS_AF_ERR_INVALID_MCSID; + + mcs = mcs_get_pdata(req->mcs_id); + + if (mcs->hw->lmac_cnt <= req->port_id || !(mcs->hw->lmac_bmap & BIT_ULL(req->port_id))) + return -EINVAL; + + mcs_set_port_cfg(mcs, req); + + return 0; +} + +int rvu_mbox_handler_mcs_port_cfg_get(struct rvu *rvu, struct mcs_port_cfg_get_req *req, + struct mcs_port_cfg_get_rsp *rsp) +{ + struct mcs *mcs; + + if (req->mcs_id >= rvu->mcs_blk_cnt) + return MCS_AF_ERR_INVALID_MCSID; + + mcs = mcs_get_pdata(req->mcs_id); + + if (mcs->hw->lmac_cnt <= req->port_id || !(mcs->hw->lmac_bmap & BIT_ULL(req->port_id))) + return -EINVAL; + + mcs_get_port_cfg(mcs, req, rsp); + + return 0; +} + +int rvu_mbox_handler_mcs_custom_tag_cfg_get(struct rvu *rvu, struct mcs_custom_tag_cfg_get_req *req, + struct mcs_custom_tag_cfg_get_rsp *rsp) +{ + struct mcs *mcs; + + if (req->mcs_id >= rvu->mcs_blk_cnt) + return MCS_AF_ERR_INVALID_MCSID; + + mcs = mcs_get_pdata(req->mcs_id); + + mcs_get_custom_tag_cfg(mcs, req, rsp); + + return 0; +} + +int rvu_mcs_flr_handler(struct rvu *rvu, u16 pcifunc) +{ + struct mcs *mcs; + int mcs_id; + + /* CNF10K-B mcs0-6 are mapped to RPM2-8*/ + if (rvu->mcs_blk_cnt > 1) { + for (mcs_id = 0; mcs_id < rvu->mcs_blk_cnt; mcs_id++) { + mcs = mcs_get_pdata(mcs_id); + mcs_free_all_rsrc(mcs, MCS_RX, pcifunc); + mcs_free_all_rsrc(mcs, MCS_TX, pcifunc); + } + } else { + /* CN10K-B has only one mcs block */ + mcs = mcs_get_pdata(0); + mcs_free_all_rsrc(mcs, MCS_RX, pcifunc); + mcs_free_all_rsrc(mcs, MCS_TX, pcifunc); + } + return 0; +} + +int rvu_mbox_handler_mcs_flowid_ena_entry(struct rvu *rvu, + struct mcs_flowid_ena_dis_entry *req, + struct msg_rsp *rsp) +{ + struct mcs *mcs; + + if (req->mcs_id >= rvu->mcs_blk_cnt) + return MCS_AF_ERR_INVALID_MCSID; + + mcs = mcs_get_pdata(req->mcs_id); + mcs_ena_dis_flowid_entry(mcs, req->flow_id, req->dir, req->ena); + return 0; +} + +int rvu_mbox_handler_mcs_pn_table_write(struct rvu *rvu, + struct mcs_pn_table_write_req *req, + struct msg_rsp *rsp) +{ + struct mcs *mcs; + + if (req->mcs_id >= rvu->mcs_blk_cnt) + return MCS_AF_ERR_INVALID_MCSID; + + mcs = mcs_get_pdata(req->mcs_id); + mcs_pn_table_write(mcs, req->pn_id, req->next_pn, req->dir); + return 0; +} + +int rvu_mbox_handler_mcs_set_pn_threshold(struct rvu *rvu, + struct mcs_set_pn_threshold *req, + struct msg_rsp *rsp) +{ + struct mcs *mcs; + + if (req->mcs_id >= rvu->mcs_blk_cnt) + return MCS_AF_ERR_INVALID_MCSID; + + mcs = mcs_get_pdata(req->mcs_id); + + mcs_pn_threshold_set(mcs, req); + + return 0; +} + +int rvu_mbox_handler_mcs_rx_sc_sa_map_write(struct rvu *rvu, + struct mcs_rx_sc_sa_map *req, + struct msg_rsp *rsp) +{ + struct mcs *mcs; + + if (req->mcs_id >= rvu->mcs_blk_cnt) + return MCS_AF_ERR_INVALID_MCSID; + + mcs = mcs_get_pdata(req->mcs_id); + mcs->mcs_ops->mcs_rx_sa_mem_map_write(mcs, req); + return 0; +} + +int rvu_mbox_handler_mcs_tx_sc_sa_map_write(struct rvu *rvu, + struct mcs_tx_sc_sa_map *req, + struct msg_rsp *rsp) +{ + struct mcs *mcs; + + if (req->mcs_id >= rvu->mcs_blk_cnt) + return MCS_AF_ERR_INVALID_MCSID; + + mcs = mcs_get_pdata(req->mcs_id); + mcs->mcs_ops->mcs_tx_sa_mem_map_write(mcs, req); + mcs->tx_sa_active[req->sc_id] = req->tx_sa_active; + + return 0; +} + +int rvu_mbox_handler_mcs_sa_plcy_write(struct rvu *rvu, + struct mcs_sa_plcy_write_req *req, + struct msg_rsp *rsp) +{ + struct mcs *mcs; + int i; + + if (req->mcs_id >= rvu->mcs_blk_cnt) + return MCS_AF_ERR_INVALID_MCSID; + + mcs = mcs_get_pdata(req->mcs_id); + + for (i = 0; i < req->sa_cnt; i++) + mcs_sa_plcy_write(mcs, &req->plcy[i][0], + req->sa_index[i], req->dir); + return 0; +} + +int rvu_mbox_handler_mcs_rx_sc_cam_write(struct rvu *rvu, + struct mcs_rx_sc_cam_write_req *req, + struct msg_rsp *rsp) +{ + struct mcs *mcs; + + if (req->mcs_id >= rvu->mcs_blk_cnt) + return MCS_AF_ERR_INVALID_MCSID; + + mcs = mcs_get_pdata(req->mcs_id); + mcs_rx_sc_cam_write(mcs, req->sci, req->secy_id, req->sc_id); + return 0; +} + +int rvu_mbox_handler_mcs_secy_plcy_write(struct rvu *rvu, + struct mcs_secy_plcy_write_req *req, + struct msg_rsp *rsp) +{ struct mcs *mcs; + + if (req->mcs_id >= rvu->mcs_blk_cnt) + return MCS_AF_ERR_INVALID_MCSID; + + mcs = mcs_get_pdata(req->mcs_id); + + mcs_secy_plcy_write(mcs, req->plcy, + req->secy_id, req->dir); + return 0; +} + +int rvu_mbox_handler_mcs_flowid_entry_write(struct rvu *rvu, + struct mcs_flowid_entry_write_req *req, + struct msg_rsp *rsp) +{ + struct secy_mem_map map; + struct mcs *mcs; + + if (req->mcs_id >= rvu->mcs_blk_cnt) + return MCS_AF_ERR_INVALID_MCSID; + + mcs = mcs_get_pdata(req->mcs_id); + + /* TODO validate the flowid */ + mcs_flowid_entry_write(mcs, req->data, req->mask, + req->flow_id, req->dir); + map.secy = req->secy_id; + map.sc = req->sc_id; + map.ctrl_pkt = req->ctrl_pkt; + map.flow_id = req->flow_id; + map.sci = req->sci; + mcs->mcs_ops->mcs_flowid_secy_map(mcs, &map, req->dir); + if (req->ena) + mcs_ena_dis_flowid_entry(mcs, req->flow_id, + req->dir, true); + return 0; +} + +int rvu_mbox_handler_mcs_free_resources(struct rvu *rvu, + struct mcs_free_rsrc_req *req, + struct msg_rsp *rsp) +{ + u16 pcifunc = req->hdr.pcifunc; + struct mcs_rsrc_map *map; + struct mcs *mcs; + int rc; + + if (req->mcs_id >= rvu->mcs_blk_cnt) + return MCS_AF_ERR_INVALID_MCSID; + + mcs = mcs_get_pdata(req->mcs_id); + + if (req->dir == MCS_RX) + map = &mcs->rx; + else + map = &mcs->tx; + + mutex_lock(&rvu->rsrc_lock); + /* Free all the cam resources mapped to PF/VF */ + if (req->all) { + rc = mcs_free_all_rsrc(mcs, req->dir, pcifunc); + goto exit; + } + + switch (req->rsrc_type) { + case MCS_RSRC_TYPE_FLOWID: + rc = mcs_free_rsrc(&map->flow_ids, map->flowid2pf_map, req->rsrc_id, pcifunc); + mcs_ena_dis_flowid_entry(mcs, req->rsrc_id, req->dir, false); + break; + case MCS_RSRC_TYPE_SECY: + rc = mcs_free_rsrc(&map->secy, map->secy2pf_map, req->rsrc_id, pcifunc); + mcs_clear_secy_plcy(mcs, req->rsrc_id, req->dir); + break; + case MCS_RSRC_TYPE_SC: + rc = mcs_free_rsrc(&map->sc, map->sc2pf_map, req->rsrc_id, pcifunc); + /* Disable SC CAM only on RX side */ + if (req->dir == MCS_RX) + mcs_ena_dis_sc_cam_entry(mcs, req->rsrc_id, false); + break; + case MCS_RSRC_TYPE_SA: + rc = mcs_free_rsrc(&map->sa, map->sa2pf_map, req->rsrc_id, pcifunc); + break; + } +exit: + mutex_unlock(&rvu->rsrc_lock); + return rc; +} + +int rvu_mbox_handler_mcs_alloc_resources(struct rvu *rvu, + struct mcs_alloc_rsrc_req *req, + struct mcs_alloc_rsrc_rsp *rsp) +{ + u16 pcifunc = req->hdr.pcifunc; + struct mcs_rsrc_map *map; + struct mcs *mcs; + int rsrc_id, i; + + if (req->mcs_id >= rvu->mcs_blk_cnt) + return MCS_AF_ERR_INVALID_MCSID; + + mcs = mcs_get_pdata(req->mcs_id); + + if (req->dir == MCS_RX) + map = &mcs->rx; + else + map = &mcs->tx; + + mutex_lock(&rvu->rsrc_lock); + + if (req->all) { + rsrc_id = mcs_alloc_all_rsrc(mcs, &rsp->flow_ids[0], + &rsp->secy_ids[0], + &rsp->sc_ids[0], + &rsp->sa_ids[0], + &rsp->sa_ids[1], + pcifunc, req->dir); + goto exit; + } + + switch (req->rsrc_type) { + case MCS_RSRC_TYPE_FLOWID: + for (i = 0; i < req->rsrc_cnt; i++) { + rsrc_id = mcs_alloc_rsrc(&map->flow_ids, map->flowid2pf_map, pcifunc); + if (rsrc_id < 0) + goto exit; + rsp->flow_ids[i] = rsrc_id; + rsp->rsrc_cnt++; + } + break; + case MCS_RSRC_TYPE_SECY: + for (i = 0; i < req->rsrc_cnt; i++) { + rsrc_id = mcs_alloc_rsrc(&map->secy, map->secy2pf_map, pcifunc); + if (rsrc_id < 0) + goto exit; + rsp->secy_ids[i] = rsrc_id; + rsp->rsrc_cnt++; + } + break; + case MCS_RSRC_TYPE_SC: + for (i = 0; i < req->rsrc_cnt; i++) { + rsrc_id = mcs_alloc_rsrc(&map->sc, map->sc2pf_map, pcifunc); + if (rsrc_id < 0) + goto exit; + rsp->sc_ids[i] = rsrc_id; + rsp->rsrc_cnt++; + } + break; + case MCS_RSRC_TYPE_SA: + for (i = 0; i < req->rsrc_cnt; i++) { + rsrc_id = mcs_alloc_rsrc(&map->sa, map->sa2pf_map, pcifunc); + if (rsrc_id < 0) + goto exit; + rsp->sa_ids[i] = rsrc_id; + rsp->rsrc_cnt++; + } + break; + } + + rsp->rsrc_type = req->rsrc_type; + rsp->dir = req->dir; + rsp->mcs_id = req->mcs_id; + rsp->all = req->all; + +exit: + if (rsrc_id < 0) + dev_err(rvu->dev, "Failed to allocate the mcs resources for PCIFUNC:%d\n", pcifunc); + mutex_unlock(&rvu->rsrc_lock); + return 0; +} + +int rvu_mbox_handler_mcs_alloc_ctrl_pkt_rule(struct rvu *rvu, + struct mcs_alloc_ctrl_pkt_rule_req *req, + struct mcs_alloc_ctrl_pkt_rule_rsp *rsp) +{ + u16 pcifunc = req->hdr.pcifunc; + struct mcs_rsrc_map *map; + struct mcs *mcs; + int rsrc_id; + u16 offset; + + if (req->mcs_id >= rvu->mcs_blk_cnt) + return MCS_AF_ERR_INVALID_MCSID; + + mcs = mcs_get_pdata(req->mcs_id); + + map = (req->dir == MCS_RX) ? &mcs->rx : &mcs->tx; + + mutex_lock(&rvu->rsrc_lock); + + switch (req->rule_type) { + case MCS_CTRL_PKT_RULE_TYPE_ETH: + offset = MCS_CTRLPKT_ETYPE_RULE_OFFSET; + break; + case MCS_CTRL_PKT_RULE_TYPE_DA: + offset = MCS_CTRLPKT_DA_RULE_OFFSET; + break; + case MCS_CTRL_PKT_RULE_TYPE_RANGE: + offset = MCS_CTRLPKT_DA_RANGE_RULE_OFFSET; + break; + case MCS_CTRL_PKT_RULE_TYPE_COMBO: + offset = MCS_CTRLPKT_COMBO_RULE_OFFSET; + break; + case MCS_CTRL_PKT_RULE_TYPE_MAC: + offset = MCS_CTRLPKT_MAC_EN_RULE_OFFSET; + break; + } + + rsrc_id = mcs_alloc_ctrlpktrule(&map->ctrlpktrule, map->ctrlpktrule2pf_map, offset, + pcifunc); + if (rsrc_id < 0) + goto exit; + + rsp->rule_idx = rsrc_id; + rsp->rule_type = req->rule_type; + rsp->dir = req->dir; + rsp->mcs_id = req->mcs_id; + + mutex_unlock(&rvu->rsrc_lock); + return 0; +exit: + if (rsrc_id < 0) + dev_err(rvu->dev, "Failed to allocate the mcs ctrl pkt rule for PCIFUNC:%d\n", + pcifunc); + mutex_unlock(&rvu->rsrc_lock); + return rsrc_id; +} + +int rvu_mbox_handler_mcs_free_ctrl_pkt_rule(struct rvu *rvu, + struct mcs_free_ctrl_pkt_rule_req *req, + struct msg_rsp *rsp) +{ + struct mcs *mcs; + int rc; + + if (req->mcs_id >= rvu->mcs_blk_cnt) + return MCS_AF_ERR_INVALID_MCSID; + + mcs = mcs_get_pdata(req->mcs_id); + + mutex_lock(&rvu->rsrc_lock); + + rc = mcs_free_ctrlpktrule(mcs, req); + + mutex_unlock(&rvu->rsrc_lock); + + return rc; +} + +int rvu_mbox_handler_mcs_ctrl_pkt_rule_write(struct rvu *rvu, + struct mcs_ctrl_pkt_rule_write_req *req, + struct msg_rsp *rsp) +{ + struct mcs *mcs; + int rc; + + if (req->mcs_id >= rvu->mcs_blk_cnt) + return MCS_AF_ERR_INVALID_MCSID; + + mcs = mcs_get_pdata(req->mcs_id); + + rc = mcs_ctrlpktrule_write(mcs, req); + + return rc; +} + +static void rvu_mcs_set_lmac_bmap(struct rvu *rvu) +{ + struct mcs *mcs = mcs_get_pdata(0); + unsigned long lmac_bmap; + int cgx, lmac, port; + + for (port = 0; port < mcs->hw->lmac_cnt; port++) { + cgx = port / rvu->hw->lmac_per_cgx; + lmac = port % rvu->hw->lmac_per_cgx; + if (!is_lmac_valid(rvu_cgx_pdata(cgx, rvu), lmac)) + continue; + set_bit(port, &lmac_bmap); + } + mcs->hw->lmac_bmap = lmac_bmap; +} + +int rvu_mcs_init(struct rvu *rvu) +{ + struct rvu_hwinfo *hw = rvu->hw; + int lmac, err = 0, mcs_id; + struct mcs *mcs; + + rvu->mcs_blk_cnt = mcs_get_blkcnt(); + + if (!rvu->mcs_blk_cnt) + return 0; + + /* Needed only for CN10K-B */ + if (rvu->mcs_blk_cnt == 1) { + err = mcs_set_lmac_channels(0, hw->cgx_chan_base); + if (err) + return err; + /* Set active lmacs */ + rvu_mcs_set_lmac_bmap(rvu); + } + + /* Install default tcam bypass entry and set port to operational mode */ + for (mcs_id = 0; mcs_id < rvu->mcs_blk_cnt; mcs_id++) { + mcs = mcs_get_pdata(mcs_id); + mcs_install_flowid_bypass_entry(mcs); + for (lmac = 0; lmac < mcs->hw->lmac_cnt; lmac++) + mcs_set_lmac_mode(mcs, lmac, 0); + + mcs->rvu = rvu; + + /* Allocated memory for PFVF data */ + mcs->pf = devm_kcalloc(mcs->dev, hw->total_pfs, + sizeof(struct mcs_pfvf), GFP_KERNEL); + if (!mcs->pf) + return -ENOMEM; + + mcs->vf = devm_kcalloc(mcs->dev, hw->total_vfs, + sizeof(struct mcs_pfvf), GFP_KERNEL); + if (!mcs->vf) + return -ENOMEM; + } + + /* Initialize the wq for handling mcs interrupts */ + INIT_LIST_HEAD(&rvu->mcs_intrq_head); + INIT_WORK(&rvu->mcs_intr_work, mcs_intr_handler_task); + rvu->mcs_intr_wq = alloc_workqueue("mcs_intr_wq", 0, 0); + if (!rvu->mcs_intr_wq) { + dev_err(rvu->dev, "mcs alloc workqueue failed\n"); + return -ENOMEM; + } + + return err; +} + +void rvu_mcs_exit(struct rvu *rvu) +{ + if (!rvu->mcs_intr_wq) + return; + + flush_workqueue(rvu->mcs_intr_wq); + destroy_workqueue(rvu->mcs_intr_wq); + rvu->mcs_intr_wq = NULL; +} diff --git a/drivers/net/ethernet/marvell/octeontx2/af/ptp.c b/drivers/net/ethernet/marvell/octeontx2/af/ptp.c index 67a6821d2dff..3411e2e47d46 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/ptp.c +++ b/drivers/net/ethernet/marvell/octeontx2/af/ptp.c @@ -9,6 +9,8 @@ #include <linux/device.h> #include <linux/module.h> #include <linux/pci.h> +#include <linux/hrtimer.h> +#include <linux/ktime.h> #include "ptp.h" #include "mbox.h" @@ -50,12 +52,23 @@ #define PTP_CLOCK_COMP 0xF18ULL #define PTP_TIMESTAMP 0xF20ULL #define PTP_CLOCK_SEC 0xFD0ULL +#define PTP_SEC_ROLLOVER 0xFD8ULL #define CYCLE_MULT 1000 static struct ptp *first_ptp_block; static const struct pci_device_id ptp_id_table[]; +static bool is_ptp_dev_cnf10kb(struct ptp *ptp) +{ + return (ptp->pdev->subsystem_device == PCI_SUBSYS_DEVID_CNF10K_B_PTP) ? true : false; +} + +static bool is_ptp_dev_cn10k(struct ptp *ptp) +{ + return (ptp->pdev->device == PCI_DEVID_CN10K_PTP) ? true : false; +} + static bool cn10k_ptp_errata(struct ptp *ptp) { if (ptp->pdev->subsystem_device == PCI_SUBSYS_DEVID_CN10K_A_PTP || @@ -72,6 +85,43 @@ static bool is_ptp_tsfmt_sec_nsec(struct ptp *ptp) return false; } +static enum hrtimer_restart ptp_reset_thresh(struct hrtimer *hrtimer) +{ + struct ptp *ptp = container_of(hrtimer, struct ptp, hrtimer); + ktime_t curr_ts = ktime_get(); + ktime_t delta_ns, period_ns; + u64 ptp_clock_hi; + + /* calculate the elapsed time since last restart */ + delta_ns = ktime_to_ns(ktime_sub(curr_ts, ptp->last_ts)); + + /* if the ptp clock value has crossed 0.5 seconds, + * its too late to update pps threshold value, so + * update threshold after 1 second. + */ + ptp_clock_hi = readq(ptp->reg_base + PTP_CLOCK_HI); + if (ptp_clock_hi > 500000000) { + period_ns = ktime_set(0, (NSEC_PER_SEC + 100 - ptp_clock_hi)); + } else { + writeq(500000000, ptp->reg_base + PTP_PPS_THRESH_HI); + period_ns = ktime_set(0, (NSEC_PER_SEC + 100 - delta_ns)); + } + + hrtimer_forward_now(hrtimer, period_ns); + ptp->last_ts = curr_ts; + + return HRTIMER_RESTART; +} + +static void ptp_hrtimer_start(struct ptp *ptp, ktime_t start_ns) +{ + ktime_t period_ns; + + period_ns = ktime_set(0, (NSEC_PER_SEC + 100 - start_ns)); + hrtimer_start(&ptp->hrtimer, period_ns, HRTIMER_MODE_REL); + ptp->last_ts = ktime_get(); +} + static u64 read_ptp_tstmp_sec_nsec(struct ptp *ptp) { u64 sec, sec1, nsec; @@ -246,6 +296,10 @@ void ptp_start(struct ptp *ptp, u64 sclk, u32 ext_clk_freq, u32 extts) /* sclk is in MHz */ ptp->clock_rate = sclk * 1000000; + /* Program the seconds rollover value to 1 second */ + if (is_ptp_dev_cnf10kb(ptp)) + writeq(0x3b9aca00, ptp->reg_base + PTP_SEC_ROLLOVER); + /* Enable PTP clock */ clock_cfg = readq(ptp->reg_base + PTP_CLOCK_CFG); @@ -270,6 +324,18 @@ void ptp_start(struct ptp *ptp, u64 sclk, u32 ext_clk_freq, u32 extts) /* Set 50% duty cycle for 1Hz output */ writeq(0x1dcd650000000000, ptp->reg_base + PTP_PPS_HI_INCR); writeq(0x1dcd650000000000, ptp->reg_base + PTP_PPS_LO_INCR); + if (cn10k_ptp_errata(ptp)) { + /* The ptp_clock_hi rollsover to zero once clock cycle before it + * reaches one second boundary. so, program the pps_lo_incr in + * such a way that the pps threshold value comparison at one + * second boundary will succeed and pps edge changes. After each + * one second boundary, the hrtimer handler will be invoked and + * reprograms the pps threshold value. + */ + ptp->clock_period = NSEC_PER_SEC / ptp->clock_rate; + writeq((0x1dcd6500ULL - ptp->clock_period) << 32, + ptp->reg_base + PTP_PPS_LO_INCR); + } if (cn10k_ptp_errata(ptp)) clock_comp = ptp_calc_adjusted_comp(ptp->clock_rate); @@ -282,14 +348,39 @@ void ptp_start(struct ptp *ptp, u64 sclk, u32 ext_clk_freq, u32 extts) static int ptp_get_tstmp(struct ptp *ptp, u64 *clk) { - *clk = readq(ptp->reg_base + PTP_TIMESTAMP); + u64 timestamp; + + if (is_ptp_dev_cn10k(ptp)) { + timestamp = readq(ptp->reg_base + PTP_TIMESTAMP); + *clk = (timestamp >> 32) * NSEC_PER_SEC + (timestamp & 0xFFFFFFFF); + } else { + *clk = readq(ptp->reg_base + PTP_TIMESTAMP); + } return 0; } static int ptp_set_thresh(struct ptp *ptp, u64 thresh) { - writeq(thresh, ptp->reg_base + PTP_PPS_THRESH_HI); + if (!cn10k_ptp_errata(ptp)) + writeq(thresh, ptp->reg_base + PTP_PPS_THRESH_HI); + + return 0; +} + +static int ptp_extts_on(struct ptp *ptp, int on) +{ + u64 ptp_clock_hi; + + if (cn10k_ptp_errata(ptp)) { + if (on) { + ptp_clock_hi = readq(ptp->reg_base + PTP_CLOCK_HI); + ptp_hrtimer_start(ptp, (ktime_t)ptp_clock_hi); + } else { + if (hrtimer_active(&ptp->hrtimer)) + hrtimer_cancel(&ptp->hrtimer); + } + } return 0; } @@ -329,6 +420,11 @@ static int ptp_probe(struct pci_dev *pdev, else ptp->read_ptp_tstmp = &read_ptp_tstmp_nsec; + if (cn10k_ptp_errata(ptp)) { + hrtimer_init(&ptp->hrtimer, CLOCK_MONOTONIC, HRTIMER_MODE_REL); + ptp->hrtimer.function = ptp_reset_thresh; + } + return 0; error_free: @@ -353,6 +449,9 @@ static void ptp_remove(struct pci_dev *pdev) struct ptp *ptp = pci_get_drvdata(pdev); u64 clock_cfg; + if (cn10k_ptp_errata(ptp) && hrtimer_active(&ptp->hrtimer)) + hrtimer_cancel(&ptp->hrtimer); + if (IS_ERR_OR_NULL(ptp)) return; @@ -420,6 +519,9 @@ int rvu_mbox_handler_ptp_op(struct rvu *rvu, struct ptp_req *req, case PTP_OP_SET_THRESH: err = ptp_set_thresh(rvu->ptp, req->thresh); break; + case PTP_OP_EXTTS_ON: + err = ptp_extts_on(rvu->ptp, req->extts_on); + break; default: err = -EINVAL; break; diff --git a/drivers/net/ethernet/marvell/octeontx2/af/ptp.h b/drivers/net/ethernet/marvell/octeontx2/af/ptp.h index 95a955159f40..b9d92abc3844 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/ptp.h +++ b/drivers/net/ethernet/marvell/octeontx2/af/ptp.h @@ -17,7 +17,10 @@ struct ptp { void __iomem *reg_base; u64 (*read_ptp_tstmp)(struct ptp *ptp); spinlock_t ptp_lock; /* lock */ + struct hrtimer hrtimer; + ktime_t last_ts; u32 clock_rate; + u32 clock_period; }; struct ptp *ptp_get(void); diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rpm.c b/drivers/net/ethernet/marvell/octeontx2/af/rpm.c index ef59de43b11e..a70e1153fa04 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/rpm.c +++ b/drivers/net/ethernet/marvell/octeontx2/af/rpm.c @@ -415,11 +415,26 @@ void rpm_lmac_ptp_config(void *rpmd, int lmac_id, bool enable) return; cfg = rpm_read(rpm, lmac_id, RPMX_CMRX_CFG); - if (enable) + if (enable) { cfg |= RPMX_RX_TS_PREPEND; - else + cfg |= RPMX_TX_PTP_1S_SUPPORT; + } else { cfg &= ~RPMX_RX_TS_PREPEND; + cfg &= ~RPMX_TX_PTP_1S_SUPPORT; + } + rpm_write(rpm, lmac_id, RPMX_CMRX_CFG, cfg); + + cfg = rpm_read(rpm, lmac_id, RPMX_MTI_MAC100X_XIF_MODE); + + if (enable) { + cfg |= RPMX_ONESTEP_ENABLE; + cfg &= ~RPMX_TS_BINARY_MODE; + } else { + cfg &= ~RPMX_ONESTEP_ENABLE; + } + + rpm_write(rpm, lmac_id, RPMX_MTI_MAC100X_XIF_MODE, cfg); } int rpm_lmac_pfc_config(void *rpmd, int lmac_id, u8 tx_pause, u8 rx_pause, u16 pfc_en) diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rpm.h b/drivers/net/ethernet/marvell/octeontx2/af/rpm.h index c2bd6e54ea51..77f2ef9e1425 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/rpm.h +++ b/drivers/net/ethernet/marvell/octeontx2/af/rpm.h @@ -16,6 +16,7 @@ /* Registers */ #define RPMX_CMRX_CFG 0x00 #define RPMX_RX_TS_PREPEND BIT_ULL(22) +#define RPMX_TX_PTP_1S_SUPPORT BIT_ULL(17) #define RPMX_CMRX_SW_INT 0x180 #define RPMX_CMRX_SW_INT_W1S 0x188 #define RPMX_CMRX_SW_INT_ENA_W1S 0x198 @@ -72,6 +73,10 @@ #define RPMX_MTI_MAC100X_CL89_PAUSE_QUANTA 0x8108 #define RPM_DEFAULT_PAUSE_TIME 0x7FF +#define RPMX_MTI_MAC100X_XIF_MODE 0x8100 +#define RPMX_ONESTEP_ENABLE BIT_ULL(5) +#define RPMX_TS_BINARY_MODE BIT_ULL(11) + /* Function Declarations */ int rpm_get_nr_lmacs(void *rpmd); u8 rpm_get_lmac_type(void *rpmd, int lmac_id); diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu.c b/drivers/net/ethernet/marvell/octeontx2/af/rvu.c index 7282a826d81e..3f5e09b77d4b 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/rvu.c +++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu.c @@ -16,6 +16,7 @@ #include "rvu.h" #include "rvu_reg.h" #include "ptp.h" +#include "mcs.h" #include "rvu_trace.h" #include "rvu_npc_hash.h" @@ -23,8 +24,6 @@ #define DRV_NAME "rvu_af" #define DRV_STRING "Marvell OcteonTX2 RVU Admin Function Driver" -static int rvu_get_hwvf(struct rvu *rvu, int pcifunc); - static void rvu_set_msix_offset(struct rvu *rvu, struct rvu_pfvf *pfvf, struct rvu_block *block, int lf); static void rvu_clear_msix_offset(struct rvu *rvu, struct rvu_pfvf *pfvf, @@ -418,7 +417,7 @@ void rvu_get_pf_numvfs(struct rvu *rvu, int pf, int *numvfs, int *hwvf) *hwvf = cfg & 0xFFF; } -static int rvu_get_hwvf(struct rvu *rvu, int pcifunc) +int rvu_get_hwvf(struct rvu *rvu, int pcifunc) { int pf, func; u64 cfg; @@ -1159,6 +1158,12 @@ cpt: rvu_program_channels(rvu); + err = rvu_mcs_init(rvu); + if (err) { + dev_err(rvu->dev, "%s: Failed to initialize mcs\n", __func__); + goto nix_err; + } + return 0; nix_err: @@ -3293,6 +3298,7 @@ err_mbox: err_hwsetup: rvu_cgx_exit(rvu); rvu_fwdata_exit(rvu); + rvu_mcs_exit(rvu); rvu_reset_all_blocks(rvu); rvu_free_hw_resources(rvu); rvu_clear_rvum_blk_revid(rvu); @@ -3319,6 +3325,7 @@ static void rvu_remove(struct pci_dev *pdev) rvu_flr_wq_destroy(rvu); rvu_cgx_exit(rvu); rvu_fwdata_exit(rvu); + rvu_mcs_exit(rvu); rvu_mbox_destroy(&rvu->afpf_wq_info); rvu_disable_sriov(rvu); rvu_reset_all_blocks(rvu); @@ -3354,12 +3361,18 @@ static int __init rvu_init_module(void) if (err < 0) goto ptp_err; + err = pci_register_driver(&mcs_driver); + if (err < 0) + goto mcs_err; + err = pci_register_driver(&rvu_driver); if (err < 0) goto rvu_err; return 0; rvu_err: + pci_unregister_driver(&mcs_driver); +mcs_err: pci_unregister_driver(&ptp_driver); ptp_err: pci_unregister_driver(&cgx_driver); @@ -3370,6 +3383,7 @@ ptp_err: static void __exit rvu_cleanup_module(void) { pci_unregister_driver(&rvu_driver); + pci_unregister_driver(&mcs_driver); pci_unregister_driver(&ptp_driver); pci_unregister_driver(&cgx_driver); } diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu.h b/drivers/net/ethernet/marvell/octeontx2/af/rvu.h index d15bc443335d..76474385a602 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/rvu.h +++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu.h @@ -25,6 +25,8 @@ /* Subsystem Device ID */ #define PCI_SUBSYS_DEVID_96XX 0xB200 #define PCI_SUBSYS_DEVID_CN10K_A 0xB900 +#define PCI_SUBSYS_DEVID_CNF10K_B 0xBC00 +#define PCI_SUBSYS_DEVID_CN10K_B 0xBD00 /* PCI BAR nos */ #define PCI_AF_REG_BAR_NUM 0 @@ -62,6 +64,10 @@ struct rvu_debugfs { struct dentry *nix; struct dentry *npc; struct dentry *cpt; + struct dentry *mcs_root; + struct dentry *mcs; + struct dentry *mcs_rx; + struct dentry *mcs_tx; struct dump_ctx npa_aura_ctx; struct dump_ctx npa_pool_ctx; struct dump_ctx nix_cq_ctx; @@ -497,6 +503,8 @@ struct rvu { struct ptp *ptp; + int mcs_blk_cnt; + #ifdef CONFIG_DEBUG_FS struct rvu_debugfs rvu_dbg; #endif @@ -504,6 +512,12 @@ struct rvu { /* RVU switch implementation over NPC with DMAC rules */ struct rvu_switch rswitch; + + struct work_struct mcs_intr_work; + struct workqueue_struct *mcs_intr_wq; + struct list_head mcs_intrq_head; + /* mcs interrupt queue lock */ + spinlock_t mcs_intrq_lock; }; static inline void rvu_write64(struct rvu *rvu, u64 block, u64 offset, u64 val) @@ -868,4 +882,11 @@ void rvu_switch_update_rules(struct rvu *rvu, u16 pcifunc); int rvu_npc_set_parse_mode(struct rvu *rvu, u16 pcifunc, u64 mode, u8 dir, u64 pkind, u8 var_len_off, u8 var_len_off_mask, u8 shift_dir); +int rvu_get_hwvf(struct rvu *rvu, int pcifunc); + +/* CN10K MCS */ +int rvu_mcs_init(struct rvu *rvu); +int rvu_mcs_flr_handler(struct rvu *rvu, u16 pcifunc); +void rvu_mcs_exit(struct rvu *rvu); + #endif /* RVU_H */ diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu_debugfs.c b/drivers/net/ethernet/marvell/octeontx2/af/rvu_debugfs.c index f42a09f04b25..a1970ebedf95 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/rvu_debugfs.c +++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu_debugfs.c @@ -19,6 +19,7 @@ #include "lmac_common.h" #include "npc.h" #include "rvu_npc_hash.h" +#include "mcs.h" #define DEBUGFS_DIR_NAME "octeontx2" @@ -227,6 +228,350 @@ static const struct file_operations rvu_dbg_##name##_fops = { \ static void print_nix_qsize(struct seq_file *filp, struct rvu_pfvf *pfvf); +static int rvu_dbg_mcs_port_stats_display(struct seq_file *filp, void *unused, int dir) +{ + struct mcs *mcs = filp->private; + struct mcs_port_stats stats; + int lmac; + + seq_puts(filp, "\n port stats\n"); + mutex_lock(&mcs->stats_lock); + for_each_set_bit(lmac, &mcs->hw->lmac_bmap, mcs->hw->lmac_cnt) { + mcs_get_port_stats(mcs, &stats, lmac, dir); + seq_printf(filp, "port%d: Tcam Miss: %lld\n", lmac, stats.tcam_miss_cnt); + seq_printf(filp, "port%d: Parser errors: %lld\n", lmac, stats.parser_err_cnt); + + if (dir == MCS_RX && mcs->hw->mcs_blks > 1) + seq_printf(filp, "port%d: Preempt error: %lld\n", lmac, + stats.preempt_err_cnt); + if (dir == MCS_TX) + seq_printf(filp, "port%d: Sectag insert error: %lld\n", lmac, + stats.sectag_insert_err_cnt); + } + mutex_unlock(&mcs->stats_lock); + return 0; +} + +static int rvu_dbg_mcs_rx_port_stats_display(struct seq_file *filp, void *unused) +{ + return rvu_dbg_mcs_port_stats_display(filp, unused, MCS_RX); +} + +RVU_DEBUG_SEQ_FOPS(mcs_rx_port_stats, mcs_rx_port_stats_display, NULL); + +static int rvu_dbg_mcs_tx_port_stats_display(struct seq_file *filp, void *unused) +{ + return rvu_dbg_mcs_port_stats_display(filp, unused, MCS_TX); +} + +RVU_DEBUG_SEQ_FOPS(mcs_tx_port_stats, mcs_tx_port_stats_display, NULL); + +static int rvu_dbg_mcs_sa_stats_display(struct seq_file *filp, void *unused, int dir) +{ + struct mcs *mcs = filp->private; + struct mcs_sa_stats stats; + struct rsrc_bmap *map; + int sa_id; + + if (dir == MCS_TX) { + map = &mcs->tx.sa; + mutex_lock(&mcs->stats_lock); + for_each_set_bit(sa_id, map->bmap, mcs->hw->sa_entries) { + seq_puts(filp, "\n TX SA stats\n"); + mcs_get_sa_stats(mcs, &stats, sa_id, MCS_TX); + seq_printf(filp, "sa%d: Pkts encrypted: %lld\n", sa_id, + stats.pkt_encrypt_cnt); + + seq_printf(filp, "sa%d: Pkts protected: %lld\n", sa_id, + stats.pkt_protected_cnt); + } + mutex_unlock(&mcs->stats_lock); + return 0; + } + + /* RX stats */ + map = &mcs->rx.sa; + mutex_lock(&mcs->stats_lock); + for_each_set_bit(sa_id, map->bmap, mcs->hw->sa_entries) { + seq_puts(filp, "\n RX SA stats\n"); + mcs_get_sa_stats(mcs, &stats, sa_id, MCS_RX); + seq_printf(filp, "sa%d: Invalid pkts: %lld\n", sa_id, stats.pkt_invalid_cnt); + seq_printf(filp, "sa%d: Pkts no sa error: %lld\n", sa_id, stats.pkt_nosaerror_cnt); + seq_printf(filp, "sa%d: Pkts not valid: %lld\n", sa_id, stats.pkt_notvalid_cnt); + seq_printf(filp, "sa%d: Pkts ok: %lld\n", sa_id, stats.pkt_ok_cnt); + seq_printf(filp, "sa%d: Pkts no sa: %lld\n", sa_id, stats.pkt_nosa_cnt); + } + mutex_unlock(&mcs->stats_lock); + return 0; +} + +static int rvu_dbg_mcs_rx_sa_stats_display(struct seq_file *filp, void *unused) +{ + return rvu_dbg_mcs_sa_stats_display(filp, unused, MCS_RX); +} + +RVU_DEBUG_SEQ_FOPS(mcs_rx_sa_stats, mcs_rx_sa_stats_display, NULL); + +static int rvu_dbg_mcs_tx_sa_stats_display(struct seq_file *filp, void *unused) +{ + return rvu_dbg_mcs_sa_stats_display(filp, unused, MCS_TX); +} + +RVU_DEBUG_SEQ_FOPS(mcs_tx_sa_stats, mcs_tx_sa_stats_display, NULL); + +static int rvu_dbg_mcs_tx_sc_stats_display(struct seq_file *filp, void *unused) +{ + struct mcs *mcs = filp->private; + struct mcs_sc_stats stats; + struct rsrc_bmap *map; + int sc_id; + + map = &mcs->tx.sc; + seq_puts(filp, "\n SC stats\n"); + + mutex_lock(&mcs->stats_lock); + for_each_set_bit(sc_id, map->bmap, mcs->hw->sc_entries) { + mcs_get_sc_stats(mcs, &stats, sc_id, MCS_TX); + seq_printf(filp, "\n=======sc%d======\n\n", sc_id); + seq_printf(filp, "sc%d: Pkts encrypted: %lld\n", sc_id, stats.pkt_encrypt_cnt); + seq_printf(filp, "sc%d: Pkts protected: %lld\n", sc_id, stats.pkt_protected_cnt); + + if (mcs->hw->mcs_blks == 1) { + seq_printf(filp, "sc%d: Octets encrypted: %lld\n", sc_id, + stats.octet_encrypt_cnt); + seq_printf(filp, "sc%d: Octets protected: %lld\n", sc_id, + stats.octet_protected_cnt); + } + } + mutex_unlock(&mcs->stats_lock); + return 0; +} + +RVU_DEBUG_SEQ_FOPS(mcs_tx_sc_stats, mcs_tx_sc_stats_display, NULL); + +static int rvu_dbg_mcs_rx_sc_stats_display(struct seq_file *filp, void *unused) +{ + struct mcs *mcs = filp->private; + struct mcs_sc_stats stats; + struct rsrc_bmap *map; + int sc_id; + + map = &mcs->rx.sc; + seq_puts(filp, "\n SC stats\n"); + + mutex_lock(&mcs->stats_lock); + for_each_set_bit(sc_id, map->bmap, mcs->hw->sc_entries) { + mcs_get_sc_stats(mcs, &stats, sc_id, MCS_RX); + seq_printf(filp, "\n=======sc%d======\n\n", sc_id); + seq_printf(filp, "sc%d: Cam hits: %lld\n", sc_id, stats.hit_cnt); + seq_printf(filp, "sc%d: Invalid pkts: %lld\n", sc_id, stats.pkt_invalid_cnt); + seq_printf(filp, "sc%d: Late pkts: %lld\n", sc_id, stats.pkt_late_cnt); + seq_printf(filp, "sc%d: Notvalid pkts: %lld\n", sc_id, stats.pkt_notvalid_cnt); + seq_printf(filp, "sc%d: Unchecked pkts: %lld\n", sc_id, stats.pkt_unchecked_cnt); + + if (mcs->hw->mcs_blks > 1) { + seq_printf(filp, "sc%d: Delay pkts: %lld\n", sc_id, stats.pkt_delay_cnt); + seq_printf(filp, "sc%d: Pkts ok: %lld\n", sc_id, stats.pkt_ok_cnt); + } + if (mcs->hw->mcs_blks == 1) { + seq_printf(filp, "sc%d: Octets decrypted: %lld\n", sc_id, + stats.octet_decrypt_cnt); + seq_printf(filp, "sc%d: Octets validated: %lld\n", sc_id, + stats.octet_validate_cnt); + } + } + mutex_unlock(&mcs->stats_lock); + return 0; +} + +RVU_DEBUG_SEQ_FOPS(mcs_rx_sc_stats, mcs_rx_sc_stats_display, NULL); + +static int rvu_dbg_mcs_flowid_stats_display(struct seq_file *filp, void *unused, int dir) +{ + struct mcs *mcs = filp->private; + struct mcs_flowid_stats stats; + struct rsrc_bmap *map; + int flow_id; + + seq_puts(filp, "\n Flowid stats\n"); + + if (dir == MCS_RX) + map = &mcs->rx.flow_ids; + else + map = &mcs->tx.flow_ids; + + mutex_lock(&mcs->stats_lock); + for_each_set_bit(flow_id, map->bmap, mcs->hw->tcam_entries) { + mcs_get_flowid_stats(mcs, &stats, flow_id, dir); + seq_printf(filp, "Flowid%d: Hit:%lld\n", flow_id, stats.tcam_hit_cnt); + } + mutex_unlock(&mcs->stats_lock); + return 0; +} + +static int rvu_dbg_mcs_tx_flowid_stats_display(struct seq_file *filp, void *unused) +{ + return rvu_dbg_mcs_flowid_stats_display(filp, unused, MCS_TX); +} + +RVU_DEBUG_SEQ_FOPS(mcs_tx_flowid_stats, mcs_tx_flowid_stats_display, NULL); + +static int rvu_dbg_mcs_rx_flowid_stats_display(struct seq_file *filp, void *unused) +{ + return rvu_dbg_mcs_flowid_stats_display(filp, unused, MCS_RX); +} + +RVU_DEBUG_SEQ_FOPS(mcs_rx_flowid_stats, mcs_rx_flowid_stats_display, NULL); + +static int rvu_dbg_mcs_tx_secy_stats_display(struct seq_file *filp, void *unused) +{ + struct mcs *mcs = filp->private; + struct mcs_secy_stats stats; + struct rsrc_bmap *map; + int secy_id; + + map = &mcs->tx.secy; + seq_puts(filp, "\n MCS TX secy stats\n"); + + mutex_lock(&mcs->stats_lock); + for_each_set_bit(secy_id, map->bmap, mcs->hw->secy_entries) { + mcs_get_tx_secy_stats(mcs, &stats, secy_id); + seq_printf(filp, "\n=======Secy%d======\n\n", secy_id); + seq_printf(filp, "secy%d: Ctrl bcast pkts: %lld\n", secy_id, + stats.ctl_pkt_bcast_cnt); + seq_printf(filp, "secy%d: Ctrl Mcast pkts: %lld\n", secy_id, + stats.ctl_pkt_mcast_cnt); + seq_printf(filp, "secy%d: Ctrl ucast pkts: %lld\n", secy_id, + stats.ctl_pkt_ucast_cnt); + seq_printf(filp, "secy%d: Ctrl octets: %lld\n", secy_id, stats.ctl_octet_cnt); + seq_printf(filp, "secy%d: Unctrl bcast cnt: %lld\n", secy_id, + stats.unctl_pkt_bcast_cnt); + seq_printf(filp, "secy%d: Unctrl mcast pkts: %lld\n", secy_id, + stats.unctl_pkt_mcast_cnt); + seq_printf(filp, "secy%d: Unctrl ucast pkts: %lld\n", secy_id, + stats.unctl_pkt_ucast_cnt); + seq_printf(filp, "secy%d: Unctrl octets: %lld\n", secy_id, stats.unctl_octet_cnt); + seq_printf(filp, "secy%d: Octet encrypted: %lld\n", secy_id, + stats.octet_encrypted_cnt); + seq_printf(filp, "secy%d: octet protected: %lld\n", secy_id, + stats.octet_protected_cnt); + seq_printf(filp, "secy%d: Pkts on active sa: %lld\n", secy_id, + stats.pkt_noactivesa_cnt); + seq_printf(filp, "secy%d: Pkts too long: %lld\n", secy_id, stats.pkt_toolong_cnt); + seq_printf(filp, "secy%d: Pkts untagged: %lld\n", secy_id, stats.pkt_untagged_cnt); + } + mutex_unlock(&mcs->stats_lock); + return 0; +} + +RVU_DEBUG_SEQ_FOPS(mcs_tx_secy_stats, mcs_tx_secy_stats_display, NULL); + +static int rvu_dbg_mcs_rx_secy_stats_display(struct seq_file *filp, void *unused) +{ + struct mcs *mcs = filp->private; + struct mcs_secy_stats stats; + struct rsrc_bmap *map; + int secy_id; + + map = &mcs->rx.secy; + seq_puts(filp, "\n MCS secy stats\n"); + + mutex_lock(&mcs->stats_lock); + for_each_set_bit(secy_id, map->bmap, mcs->hw->secy_entries) { + mcs_get_rx_secy_stats(mcs, &stats, secy_id); + seq_printf(filp, "\n=======Secy%d======\n\n", secy_id); + seq_printf(filp, "secy%d: Ctrl bcast pkts: %lld\n", secy_id, + stats.ctl_pkt_bcast_cnt); + seq_printf(filp, "secy%d: Ctrl Mcast pkts: %lld\n", secy_id, + stats.ctl_pkt_mcast_cnt); + seq_printf(filp, "secy%d: Ctrl ucast pkts: %lld\n", secy_id, + stats.ctl_pkt_ucast_cnt); + seq_printf(filp, "secy%d: Ctrl octets: %lld\n", secy_id, stats.ctl_octet_cnt); + seq_printf(filp, "secy%d: Unctrl bcast cnt: %lld\n", secy_id, + stats.unctl_pkt_bcast_cnt); + seq_printf(filp, "secy%d: Unctrl mcast pkts: %lld\n", secy_id, + stats.unctl_pkt_mcast_cnt); + seq_printf(filp, "secy%d: Unctrl ucast pkts: %lld\n", secy_id, + stats.unctl_pkt_ucast_cnt); + seq_printf(filp, "secy%d: Unctrl octets: %lld\n", secy_id, stats.unctl_octet_cnt); + seq_printf(filp, "secy%d: Octet decrypted: %lld\n", secy_id, + stats.octet_decrypted_cnt); + seq_printf(filp, "secy%d: octet validated: %lld\n", secy_id, + stats.octet_validated_cnt); + seq_printf(filp, "secy%d: Pkts on disable port: %lld\n", secy_id, + stats.pkt_port_disabled_cnt); + seq_printf(filp, "secy%d: Octets validated: %lld\n", secy_id, stats.pkt_badtag_cnt); + seq_printf(filp, "secy%d: Octets validated: %lld\n", secy_id, stats.pkt_nosa_cnt); + seq_printf(filp, "secy%d: Pkts with nosaerror: %lld\n", secy_id, + stats.pkt_nosaerror_cnt); + seq_printf(filp, "secy%d: Tagged ctrl pkts: %lld\n", secy_id, + stats.pkt_tagged_ctl_cnt); + seq_printf(filp, "secy%d: Untaged pkts: %lld\n", secy_id, stats.pkt_untaged_cnt); + seq_printf(filp, "secy%d: Ctrl pkts: %lld\n", secy_id, stats.pkt_ctl_cnt); + if (mcs->hw->mcs_blks > 1) + seq_printf(filp, "secy%d: pkts notag: %lld\n", secy_id, + stats.pkt_notag_cnt); + } + mutex_unlock(&mcs->stats_lock); + return 0; +} + +RVU_DEBUG_SEQ_FOPS(mcs_rx_secy_stats, mcs_rx_secy_stats_display, NULL); + +static void rvu_dbg_mcs_init(struct rvu *rvu) +{ + struct mcs *mcs; + char dname[10]; + int i; + + if (!rvu->mcs_blk_cnt) + return; + + rvu->rvu_dbg.mcs_root = debugfs_create_dir("mcs", rvu->rvu_dbg.root); + + for (i = 0; i < rvu->mcs_blk_cnt; i++) { + mcs = mcs_get_pdata(i); + + sprintf(dname, "mcs%d", i); + rvu->rvu_dbg.mcs = debugfs_create_dir(dname, + rvu->rvu_dbg.mcs_root); + + rvu->rvu_dbg.mcs_rx = debugfs_create_dir("rx_stats", rvu->rvu_dbg.mcs); + + debugfs_create_file("flowid", 0600, rvu->rvu_dbg.mcs_rx, mcs, + &rvu_dbg_mcs_rx_flowid_stats_fops); + + debugfs_create_file("secy", 0600, rvu->rvu_dbg.mcs_rx, mcs, + &rvu_dbg_mcs_rx_secy_stats_fops); + + debugfs_create_file("sc", 0600, rvu->rvu_dbg.mcs_rx, mcs, + &rvu_dbg_mcs_rx_sc_stats_fops); + + debugfs_create_file("sa", 0600, rvu->rvu_dbg.mcs_rx, mcs, + &rvu_dbg_mcs_rx_sa_stats_fops); + + debugfs_create_file("port", 0600, rvu->rvu_dbg.mcs_rx, mcs, + &rvu_dbg_mcs_rx_port_stats_fops); + + rvu->rvu_dbg.mcs_tx = debugfs_create_dir("tx_stats", rvu->rvu_dbg.mcs); + + debugfs_create_file("flowid", 0600, rvu->rvu_dbg.mcs_tx, mcs, + &rvu_dbg_mcs_tx_flowid_stats_fops); + + debugfs_create_file("secy", 0600, rvu->rvu_dbg.mcs_tx, mcs, + &rvu_dbg_mcs_tx_secy_stats_fops); + + debugfs_create_file("sc", 0600, rvu->rvu_dbg.mcs_tx, mcs, + &rvu_dbg_mcs_tx_sc_stats_fops); + + debugfs_create_file("sa", 0600, rvu->rvu_dbg.mcs_tx, mcs, + &rvu_dbg_mcs_tx_sa_stats_fops); + + debugfs_create_file("port", 0600, rvu->rvu_dbg.mcs_tx, mcs, + &rvu_dbg_mcs_tx_port_stats_fops); + } +} + #define LMT_MAPTBL_ENTRY_SIZE 16 /* Dump LMTST map table */ static ssize_t rvu_dbg_lmtst_map_table_display(struct file *filp, @@ -3053,6 +3398,7 @@ create: rvu_dbg_npc_init(rvu); rvu_dbg_cpt_init(rvu, BLKADDR_CPT0); rvu_dbg_cpt_init(rvu, BLKADDR_CPT1); + rvu_dbg_mcs_init(rvu); } void rvu_dbg_exit(struct rvu *rvu) diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu_nix.c b/drivers/net/ethernet/marvell/octeontx2/af/rvu_nix.c index 0879a48411f3..7646bb2ec89b 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/rvu_nix.c +++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu_nix.c @@ -4296,8 +4296,14 @@ static int rvu_nix_block_init(struct rvu *rvu, struct nix_hw *nix_hw) /* Restore CINT timer delay to HW reset values */ rvu_write64(rvu, blkaddr, NIX_AF_CINT_DELAY, 0x0ULL); + cfg = rvu_read64(rvu, blkaddr, NIX_AF_SEB_CFG); + /* For better performance use NDC TX instead of NDC RX for SQ's SQEs" */ - rvu_write64(rvu, blkaddr, NIX_AF_SEB_CFG, 0x1ULL); + cfg |= 1ULL; + if (!is_rvu_otx2(rvu)) + cfg |= NIX_PTP_1STEP_EN; + + rvu_write64(rvu, blkaddr, NIX_AF_SEB_CFG, cfg); if (is_block_implemented(hw, blkaddr)) { err = nix_setup_txschq(rvu, nix_hw, blkaddr); diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu_reg.h b/drivers/net/ethernet/marvell/octeontx2/af/rvu_reg.h index 77a9ade91f3e..0e0d536645ac 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/rvu_reg.h +++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu_reg.h @@ -266,6 +266,7 @@ #define NIX_AF_TX_NPC_CAPTURE_CONFIG (0x0660) #define NIX_AF_TX_NPC_CAPTURE_INFO (0x0670) #define NIX_AF_SEB_CFG (0x05F0) +#define NIX_PTP_1STEP_EN BIT_ULL(2) #define NIX_AF_DEBUG_NPC_RESP_DATAX(a) (0x680 | (a) << 3) #define NIX_AF_SMQX_CFG(a) (0x700 | (a) << 16) diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/Makefile b/drivers/net/ethernet/marvell/octeontx2/nic/Makefile index d463dc72d80a..73fdb8798614 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/Makefile +++ b/drivers/net/ethernet/marvell/octeontx2/nic/Makefile @@ -13,5 +13,6 @@ rvu_nicvf-y := otx2_vf.o otx2_devlink.o rvu_nicpf-$(CONFIG_DCB) += otx2_dcbnl.o rvu_nicvf-$(CONFIG_DCB) += otx2_dcbnl.o +rvu_nicpf-$(CONFIG_MACSEC) += cn10k_macsec.o ccflags-y += -I$(srctree)/drivers/net/ethernet/marvell/octeontx2/af diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/cn10k.c b/drivers/net/ethernet/marvell/octeontx2/nic/cn10k.c index fd4f083c699e..826f691de259 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/cn10k.c +++ b/drivers/net/ethernet/marvell/octeontx2/nic/cn10k.c @@ -86,8 +86,7 @@ int cn10k_sq_aq_init(void *dev, u16 qidx, u16 sqb_aura) aq->sq.max_sqe_size = NIX_MAXSQESZ_W16; /* 128 byte */ aq->sq.cq_ena = 1; aq->sq.ena = 1; - /* Only one SMQ is allocated, map all SQ's to that SMQ */ - aq->sq.smq = pfvf->hw.txschq_list[NIX_TXSCH_LVL_SMQ][0]; + aq->sq.smq = otx2_get_smq_idx(pfvf, qidx); aq->sq.smq_rr_weight = mtu_to_dwrr_weight(pfvf, pfvf->tx_max_pktlen); aq->sq.default_chan = pfvf->hw.tx_chan_base; aq->sq.sqe_stype = NIX_STYPE_STF; /* Cache SQB */ diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/cn10k_macsec.c b/drivers/net/ethernet/marvell/octeontx2/nic/cn10k_macsec.c new file mode 100644 index 000000000000..64f3acd7f67b --- /dev/null +++ b/drivers/net/ethernet/marvell/octeontx2/nic/cn10k_macsec.c @@ -0,0 +1,1668 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Marvell MACSEC hardware offload driver + * + * Copyright (C) 2022 Marvell. + */ + +#include <linux/rtnetlink.h> +#include <linux/bitfield.h> +#include <net/macsec.h> +#include "otx2_common.h" + +#define MCS_TCAM0_MAC_SA_MASK GENMASK_ULL(63, 48) +#define MCS_TCAM1_MAC_SA_MASK GENMASK_ULL(31, 0) +#define MCS_TCAM1_ETYPE_MASK GENMASK_ULL(47, 32) + +#define MCS_SA_MAP_MEM_SA_USE BIT_ULL(9) + +#define MCS_RX_SECY_PLCY_RW_MASK GENMASK_ULL(49, 18) +#define MCS_RX_SECY_PLCY_RP BIT_ULL(17) +#define MCS_RX_SECY_PLCY_AUTH_ENA BIT_ULL(16) +#define MCS_RX_SECY_PLCY_CIP GENMASK_ULL(8, 5) +#define MCS_RX_SECY_PLCY_VAL GENMASK_ULL(2, 1) +#define MCS_RX_SECY_PLCY_ENA BIT_ULL(0) + +#define MCS_TX_SECY_PLCY_MTU GENMASK_ULL(43, 28) +#define MCS_TX_SECY_PLCY_ST_TCI GENMASK_ULL(27, 22) +#define MCS_TX_SECY_PLCY_ST_OFFSET GENMASK_ULL(21, 15) +#define MCS_TX_SECY_PLCY_INS_MODE BIT_ULL(14) +#define MCS_TX_SECY_PLCY_AUTH_ENA BIT_ULL(13) +#define MCS_TX_SECY_PLCY_CIP GENMASK_ULL(5, 2) +#define MCS_TX_SECY_PLCY_PROTECT BIT_ULL(1) +#define MCS_TX_SECY_PLCY_ENA BIT_ULL(0) + +#define MCS_GCM_AES_128 0 +#define MCS_GCM_AES_256 1 +#define MCS_GCM_AES_XPN_128 2 +#define MCS_GCM_AES_XPN_256 3 + +#define MCS_TCI_ES 0x40 /* end station */ +#define MCS_TCI_SC 0x20 /* SCI present */ +#define MCS_TCI_SCB 0x10 /* epon */ +#define MCS_TCI_E 0x08 /* encryption */ +#define MCS_TCI_C 0x04 /* changed text */ + +static struct cn10k_mcs_txsc *cn10k_mcs_get_txsc(struct cn10k_mcs_cfg *cfg, + struct macsec_secy *secy) +{ + struct cn10k_mcs_txsc *txsc; + + list_for_each_entry(txsc, &cfg->txsc_list, entry) { + if (txsc->sw_secy == secy) + return txsc; + } + + return NULL; +} + +static struct cn10k_mcs_rxsc *cn10k_mcs_get_rxsc(struct cn10k_mcs_cfg *cfg, + struct macsec_secy *secy, + struct macsec_rx_sc *rx_sc) +{ + struct cn10k_mcs_rxsc *rxsc; + + list_for_each_entry(rxsc, &cfg->rxsc_list, entry) { + if (rxsc->sw_rxsc == rx_sc && rxsc->sw_secy == secy) + return rxsc; + } + + return NULL; +} + +static const char *rsrc_name(enum mcs_rsrc_type rsrc_type) +{ + switch (rsrc_type) { + case MCS_RSRC_TYPE_FLOWID: + return "FLOW"; + case MCS_RSRC_TYPE_SC: + return "SC"; + case MCS_RSRC_TYPE_SECY: + return "SECY"; + case MCS_RSRC_TYPE_SA: + return "SA"; + default: + return "Unknown"; + }; + + return "Unknown"; +} + +static int cn10k_mcs_alloc_rsrc(struct otx2_nic *pfvf, enum mcs_direction dir, + enum mcs_rsrc_type type, u16 *rsrc_id) +{ + struct mbox *mbox = &pfvf->mbox; + struct mcs_alloc_rsrc_req *req; + struct mcs_alloc_rsrc_rsp *rsp; + int ret = -ENOMEM; + + mutex_lock(&mbox->lock); + + req = otx2_mbox_alloc_msg_mcs_alloc_resources(mbox); + if (!req) + goto fail; + + req->rsrc_type = type; + req->rsrc_cnt = 1; + req->dir = dir; + + ret = otx2_sync_mbox_msg(mbox); + if (ret) + goto fail; + + rsp = (struct mcs_alloc_rsrc_rsp *)otx2_mbox_get_rsp(&pfvf->mbox.mbox, + 0, &req->hdr); + if (IS_ERR(rsp) || req->rsrc_cnt != rsp->rsrc_cnt || + req->rsrc_type != rsp->rsrc_type || req->dir != rsp->dir) { + ret = -EINVAL; + goto fail; + } + + switch (rsp->rsrc_type) { + case MCS_RSRC_TYPE_FLOWID: + *rsrc_id = rsp->flow_ids[0]; + break; + case MCS_RSRC_TYPE_SC: + *rsrc_id = rsp->sc_ids[0]; + break; + case MCS_RSRC_TYPE_SECY: + *rsrc_id = rsp->secy_ids[0]; + break; + case MCS_RSRC_TYPE_SA: + *rsrc_id = rsp->sa_ids[0]; + break; + default: + ret = -EINVAL; + goto fail; + }; + + mutex_unlock(&mbox->lock); + + return 0; +fail: + dev_err(pfvf->dev, "Failed to allocate %s %s resource\n", + dir == MCS_TX ? "TX" : "RX", rsrc_name(type)); + mutex_unlock(&mbox->lock); + return ret; +} + +static void cn10k_mcs_free_rsrc(struct otx2_nic *pfvf, enum mcs_direction dir, + enum mcs_rsrc_type type, u16 hw_rsrc_id, + bool all) +{ + struct mbox *mbox = &pfvf->mbox; + struct mcs_free_rsrc_req *req; + + mutex_lock(&mbox->lock); + + req = otx2_mbox_alloc_msg_mcs_free_resources(mbox); + if (!req) + goto fail; + + req->rsrc_id = hw_rsrc_id; + req->rsrc_type = type; + req->dir = dir; + if (all) + req->all = 1; + + if (otx2_sync_mbox_msg(&pfvf->mbox)) + goto fail; + + mutex_unlock(&mbox->lock); + + return; +fail: + dev_err(pfvf->dev, "Failed to free %s %s resource\n", + dir == MCS_TX ? "TX" : "RX", rsrc_name(type)); + mutex_unlock(&mbox->lock); +} + +static int cn10k_mcs_alloc_txsa(struct otx2_nic *pfvf, u16 *hw_sa_id) +{ + return cn10k_mcs_alloc_rsrc(pfvf, MCS_TX, MCS_RSRC_TYPE_SA, hw_sa_id); +} + +static int cn10k_mcs_alloc_rxsa(struct otx2_nic *pfvf, u16 *hw_sa_id) +{ + return cn10k_mcs_alloc_rsrc(pfvf, MCS_RX, MCS_RSRC_TYPE_SA, hw_sa_id); +} + +static void cn10k_mcs_free_txsa(struct otx2_nic *pfvf, u16 hw_sa_id) +{ + cn10k_mcs_free_rsrc(pfvf, MCS_TX, MCS_RSRC_TYPE_SA, hw_sa_id, false); +} + +static void cn10k_mcs_free_rxsa(struct otx2_nic *pfvf, u16 hw_sa_id) +{ + cn10k_mcs_free_rsrc(pfvf, MCS_RX, MCS_RSRC_TYPE_SA, hw_sa_id, false); +} + +static int cn10k_mcs_write_rx_secy(struct otx2_nic *pfvf, + struct macsec_secy *secy, u8 hw_secy_id) +{ + struct mcs_secy_plcy_write_req *req; + struct mbox *mbox = &pfvf->mbox; + u64 policy; + int ret; + + mutex_lock(&mbox->lock); + + req = otx2_mbox_alloc_msg_mcs_secy_plcy_write(mbox); + if (!req) { + ret = -ENOMEM; + goto fail; + } + + policy = FIELD_PREP(MCS_RX_SECY_PLCY_RW_MASK, secy->replay_window); + if (secy->replay_protect) + policy |= MCS_RX_SECY_PLCY_RP; + + policy |= MCS_RX_SECY_PLCY_AUTH_ENA; + policy |= FIELD_PREP(MCS_RX_SECY_PLCY_CIP, MCS_GCM_AES_128); + policy |= FIELD_PREP(MCS_RX_SECY_PLCY_VAL, secy->validate_frames); + + policy |= MCS_RX_SECY_PLCY_ENA; + + req->plcy = policy; + req->secy_id = hw_secy_id; + req->dir = MCS_RX; + + ret = otx2_sync_mbox_msg(mbox); + +fail: + mutex_unlock(&mbox->lock); + return ret; +} + +static int cn10k_mcs_write_rx_flowid(struct otx2_nic *pfvf, + struct cn10k_mcs_rxsc *rxsc, u8 hw_secy_id) +{ + struct macsec_rx_sc *sw_rx_sc = rxsc->sw_rxsc; + struct mcs_flowid_entry_write_req *req; + struct mbox *mbox = &pfvf->mbox; + int ret; + + mutex_lock(&mbox->lock); + + req = otx2_mbox_alloc_msg_mcs_flowid_entry_write(mbox); + if (!req) { + ret = -ENOMEM; + goto fail; + } + + req->data[1] = FIELD_PREP(MCS_TCAM1_ETYPE_MASK, ETH_P_MACSEC); + req->mask[1] = ~0ULL; + req->mask[1] &= ~MCS_TCAM1_ETYPE_MASK; + + req->mask[0] = ~0ULL; + req->mask[2] = ~0ULL; + req->mask[3] = ~0ULL; + + req->flow_id = rxsc->hw_flow_id; + req->secy_id = hw_secy_id; + req->sc_id = rxsc->hw_sc_id; + req->dir = MCS_RX; + + if (sw_rx_sc->active) + req->ena = 1; + + ret = otx2_sync_mbox_msg(mbox); + +fail: + mutex_unlock(&mbox->lock); + return ret; +} + +static int cn10k_mcs_write_sc_cam(struct otx2_nic *pfvf, + struct cn10k_mcs_rxsc *rxsc, u8 hw_secy_id) +{ + struct macsec_rx_sc *sw_rx_sc = rxsc->sw_rxsc; + struct mcs_rx_sc_cam_write_req *sc_req; + struct mbox *mbox = &pfvf->mbox; + int ret; + + mutex_lock(&mbox->lock); + + sc_req = otx2_mbox_alloc_msg_mcs_rx_sc_cam_write(mbox); + if (!sc_req) { + return -ENOMEM; + goto fail; + } + + sc_req->sci = (__force u64)cpu_to_be64((__force u64)sw_rx_sc->sci); + sc_req->sc_id = rxsc->hw_sc_id; + sc_req->secy_id = hw_secy_id; + + ret = otx2_sync_mbox_msg(mbox); + +fail: + mutex_unlock(&mbox->lock); + return ret; +} + +static int cn10k_mcs_write_rx_sa_plcy(struct otx2_nic *pfvf, + struct macsec_secy *secy, + struct cn10k_mcs_rxsc *rxsc, + u8 assoc_num, bool sa_in_use) +{ + unsigned char *src = rxsc->sa_key[assoc_num]; + struct mcs_sa_plcy_write_req *plcy_req; + struct mcs_rx_sc_sa_map *map_req; + struct mbox *mbox = &pfvf->mbox; + u8 reg, key_len; + int ret; + + mutex_lock(&mbox->lock); + + plcy_req = otx2_mbox_alloc_msg_mcs_sa_plcy_write(mbox); + if (!plcy_req) { + ret = -ENOMEM; + goto fail; + } + + map_req = otx2_mbox_alloc_msg_mcs_rx_sc_sa_map_write(mbox); + if (!map_req) { + otx2_mbox_reset(&mbox->mbox, 0); + ret = -ENOMEM; + goto fail; + } + + for (reg = 0, key_len = 0; key_len < secy->key_len; key_len += 8) { + memcpy((u8 *)&plcy_req->plcy[0][reg], + (src + reg * 8), 8); + reg++; + } + + plcy_req->sa_index[0] = rxsc->hw_sa_id[assoc_num]; + plcy_req->sa_cnt = 1; + plcy_req->dir = MCS_RX; + + map_req->sa_index = rxsc->hw_sa_id[assoc_num]; + map_req->sa_in_use = sa_in_use; + map_req->sc_id = rxsc->hw_sc_id; + map_req->an = assoc_num; + + /* Send two messages together */ + ret = otx2_sync_mbox_msg(mbox); + +fail: + mutex_unlock(&mbox->lock); + return ret; +} + +static int cn10k_mcs_write_rx_sa_pn(struct otx2_nic *pfvf, + struct cn10k_mcs_rxsc *rxsc, + u8 assoc_num, u64 next_pn) +{ + struct mcs_pn_table_write_req *req; + struct mbox *mbox = &pfvf->mbox; + int ret; + + mutex_lock(&mbox->lock); + + req = otx2_mbox_alloc_msg_mcs_pn_table_write(mbox); + if (!req) { + ret = -ENOMEM; + goto fail; + } + + req->pn_id = rxsc->hw_sa_id[assoc_num]; + req->next_pn = next_pn; + req->dir = MCS_RX; + + ret = otx2_sync_mbox_msg(mbox); + +fail: + mutex_unlock(&mbox->lock); + return ret; +} + +static int cn10k_mcs_write_tx_secy(struct otx2_nic *pfvf, + struct macsec_secy *secy, + struct cn10k_mcs_txsc *txsc) +{ + struct mcs_secy_plcy_write_req *req; + struct mbox *mbox = &pfvf->mbox; + struct macsec_tx_sc *sw_tx_sc; + /* Insert SecTag after 12 bytes (DA+SA)*/ + u8 tag_offset = 12; + u8 sectag_tci = 0; + u64 policy; + int ret; + + sw_tx_sc = &secy->tx_sc; + + mutex_lock(&mbox->lock); + + req = otx2_mbox_alloc_msg_mcs_secy_plcy_write(mbox); + if (!req) { + ret = -ENOMEM; + goto fail; + } + + if (sw_tx_sc->send_sci) { + sectag_tci |= MCS_TCI_SC; + } else { + if (sw_tx_sc->end_station) + sectag_tci |= MCS_TCI_ES; + if (sw_tx_sc->scb) + sectag_tci |= MCS_TCI_SCB; + } + + if (sw_tx_sc->encrypt) + sectag_tci |= (MCS_TCI_E | MCS_TCI_C); + + policy = FIELD_PREP(MCS_TX_SECY_PLCY_MTU, secy->netdev->mtu); + /* Write SecTag excluding AN bits(1..0) */ + policy |= FIELD_PREP(MCS_TX_SECY_PLCY_ST_TCI, sectag_tci >> 2); + policy |= FIELD_PREP(MCS_TX_SECY_PLCY_ST_OFFSET, tag_offset); + policy |= MCS_TX_SECY_PLCY_INS_MODE; + policy |= MCS_TX_SECY_PLCY_AUTH_ENA; + policy |= FIELD_PREP(MCS_TX_SECY_PLCY_CIP, MCS_GCM_AES_128); + + if (secy->protect_frames) + policy |= MCS_TX_SECY_PLCY_PROTECT; + + /* If the encodingsa does not exist/active and protect is + * not set then frames can be sent out as it is. Hence enable + * the policy irrespective of secy operational when !protect. + */ + if (!secy->protect_frames || secy->operational) + policy |= MCS_TX_SECY_PLCY_ENA; + + req->plcy = policy; + req->secy_id = txsc->hw_secy_id_tx; + req->dir = MCS_TX; + + ret = otx2_sync_mbox_msg(mbox); + +fail: + mutex_unlock(&mbox->lock); + return ret; +} + +static int cn10k_mcs_write_tx_flowid(struct otx2_nic *pfvf, + struct macsec_secy *secy, + struct cn10k_mcs_txsc *txsc) +{ + struct mcs_flowid_entry_write_req *req; + struct mbox *mbox = &pfvf->mbox; + u64 mac_sa; + int ret; + + mutex_lock(&mbox->lock); + + req = otx2_mbox_alloc_msg_mcs_flowid_entry_write(mbox); + if (!req) { + ret = -ENOMEM; + goto fail; + } + + mac_sa = ether_addr_to_u64(secy->netdev->dev_addr); + + req->data[0] = FIELD_PREP(MCS_TCAM0_MAC_SA_MASK, mac_sa); + req->data[1] = FIELD_PREP(MCS_TCAM1_MAC_SA_MASK, mac_sa >> 16); + + req->mask[0] = ~0ULL; + req->mask[0] &= ~MCS_TCAM0_MAC_SA_MASK; + + req->mask[1] = ~0ULL; + req->mask[1] &= ~MCS_TCAM1_MAC_SA_MASK; + + req->mask[2] = ~0ULL; + req->mask[3] = ~0ULL; + + req->flow_id = txsc->hw_flow_id; + req->secy_id = txsc->hw_secy_id_tx; + req->sc_id = txsc->hw_sc_id; + req->sci = (__force u64)cpu_to_be64((__force u64)secy->sci); + req->dir = MCS_TX; + /* This can be enabled since stack xmits packets only when interface is up */ + req->ena = 1; + + ret = otx2_sync_mbox_msg(mbox); + +fail: + mutex_unlock(&mbox->lock); + return ret; +} + +static int cn10k_mcs_link_tx_sa2sc(struct otx2_nic *pfvf, + struct macsec_secy *secy, + struct cn10k_mcs_txsc *txsc, + u8 sa_num, bool sa_active) +{ + struct mcs_tx_sc_sa_map *map_req; + struct mbox *mbox = &pfvf->mbox; + int ret; + + /* Link the encoding_sa only to SC out of all SAs */ + if (txsc->encoding_sa != sa_num) + return 0; + + mutex_lock(&mbox->lock); + + map_req = otx2_mbox_alloc_msg_mcs_tx_sc_sa_map_write(mbox); + if (!map_req) { + otx2_mbox_reset(&mbox->mbox, 0); + ret = -ENOMEM; + goto fail; + } + + map_req->sa_index0 = txsc->hw_sa_id[sa_num]; + map_req->sa_index0_vld = sa_active; + map_req->sectag_sci = (__force u64)cpu_to_be64((__force u64)secy->sci); + map_req->sc_id = txsc->hw_sc_id; + + ret = otx2_sync_mbox_msg(mbox); + +fail: + mutex_unlock(&mbox->lock); + return ret; +} + +static int cn10k_mcs_write_tx_sa_plcy(struct otx2_nic *pfvf, + struct macsec_secy *secy, + struct cn10k_mcs_txsc *txsc, + u8 assoc_num) +{ + unsigned char *src = txsc->sa_key[assoc_num]; + struct mcs_sa_plcy_write_req *plcy_req; + struct mbox *mbox = &pfvf->mbox; + u8 reg, key_len; + int ret; + + mutex_lock(&mbox->lock); + + plcy_req = otx2_mbox_alloc_msg_mcs_sa_plcy_write(mbox); + if (!plcy_req) { + ret = -ENOMEM; + goto fail; + } + + for (reg = 0, key_len = 0; key_len < secy->key_len; key_len += 8) { + memcpy((u8 *)&plcy_req->plcy[0][reg], (src + reg * 8), 8); + reg++; + } + + plcy_req->plcy[0][8] = assoc_num; + plcy_req->sa_index[0] = txsc->hw_sa_id[assoc_num]; + plcy_req->sa_cnt = 1; + plcy_req->dir = MCS_TX; + + ret = otx2_sync_mbox_msg(mbox); + +fail: + mutex_unlock(&mbox->lock); + return ret; +} + +static int cn10k_write_tx_sa_pn(struct otx2_nic *pfvf, + struct cn10k_mcs_txsc *txsc, + u8 assoc_num, u64 next_pn) +{ + struct mcs_pn_table_write_req *req; + struct mbox *mbox = &pfvf->mbox; + int ret; + + mutex_lock(&mbox->lock); + + req = otx2_mbox_alloc_msg_mcs_pn_table_write(mbox); + if (!req) { + ret = -ENOMEM; + goto fail; + } + + req->pn_id = txsc->hw_sa_id[assoc_num]; + req->next_pn = next_pn; + req->dir = MCS_TX; + + ret = otx2_sync_mbox_msg(mbox); + +fail: + mutex_unlock(&mbox->lock); + return ret; +} + +static int cn10k_mcs_ena_dis_flowid(struct otx2_nic *pfvf, u16 hw_flow_id, + bool enable, enum mcs_direction dir) +{ + struct mcs_flowid_ena_dis_entry *req; + struct mbox *mbox = &pfvf->mbox; + int ret; + + mutex_lock(&mbox->lock); + + req = otx2_mbox_alloc_msg_mcs_flowid_ena_entry(mbox); + if (!req) { + return -ENOMEM; + goto fail; + } + + req->flow_id = hw_flow_id; + req->ena = enable; + req->dir = dir; + + ret = otx2_sync_mbox_msg(mbox); + +fail: + mutex_unlock(&mbox->lock); + return ret; +} + +static int cn10k_mcs_sa_stats(struct otx2_nic *pfvf, u8 hw_sa_id, + struct mcs_sa_stats *rsp_p, + enum mcs_direction dir, bool clear) +{ + struct mcs_clear_stats *clear_req; + struct mbox *mbox = &pfvf->mbox; + struct mcs_stats_req *req; + struct mcs_sa_stats *rsp; + int ret; + + mutex_lock(&mbox->lock); + + req = otx2_mbox_alloc_msg_mcs_get_sa_stats(mbox); + if (!req) { + ret = -ENOMEM; + goto fail; + } + + req->id = hw_sa_id; + req->dir = dir; + + if (!clear) + goto send_msg; + + clear_req = otx2_mbox_alloc_msg_mcs_clear_stats(mbox); + if (!clear_req) { + ret = -ENOMEM; + goto fail; + } + clear_req->id = hw_sa_id; + clear_req->dir = dir; + clear_req->type = MCS_RSRC_TYPE_SA; + +send_msg: + ret = otx2_sync_mbox_msg(mbox); + if (ret) + goto fail; + + rsp = (struct mcs_sa_stats *)otx2_mbox_get_rsp(&pfvf->mbox.mbox, + 0, &req->hdr); + if (IS_ERR(rsp)) { + ret = PTR_ERR(rsp); + goto fail; + } + + memcpy(rsp_p, rsp, sizeof(*rsp_p)); + + mutex_unlock(&mbox->lock); + + return 0; +fail: + mutex_unlock(&mbox->lock); + return ret; +} + +static int cn10k_mcs_sc_stats(struct otx2_nic *pfvf, u8 hw_sc_id, + struct mcs_sc_stats *rsp_p, + enum mcs_direction dir, bool clear) +{ + struct mcs_clear_stats *clear_req; + struct mbox *mbox = &pfvf->mbox; + struct mcs_stats_req *req; + struct mcs_sc_stats *rsp; + int ret; + + mutex_lock(&mbox->lock); + + req = otx2_mbox_alloc_msg_mcs_get_sc_stats(mbox); + if (!req) { + ret = -ENOMEM; + goto fail; + } + + req->id = hw_sc_id; + req->dir = dir; + + if (!clear) + goto send_msg; + + clear_req = otx2_mbox_alloc_msg_mcs_clear_stats(mbox); + if (!clear_req) { + ret = -ENOMEM; + goto fail; + } + clear_req->id = hw_sc_id; + clear_req->dir = dir; + clear_req->type = MCS_RSRC_TYPE_SC; + +send_msg: + ret = otx2_sync_mbox_msg(mbox); + if (ret) + goto fail; + + rsp = (struct mcs_sc_stats *)otx2_mbox_get_rsp(&pfvf->mbox.mbox, + 0, &req->hdr); + if (IS_ERR(rsp)) { + ret = PTR_ERR(rsp); + goto fail; + } + + memcpy(rsp_p, rsp, sizeof(*rsp_p)); + + mutex_unlock(&mbox->lock); + + return 0; +fail: + mutex_unlock(&mbox->lock); + return ret; +} + +static int cn10k_mcs_secy_stats(struct otx2_nic *pfvf, u8 hw_secy_id, + struct mcs_secy_stats *rsp_p, + enum mcs_direction dir, bool clear) +{ + struct mcs_clear_stats *clear_req; + struct mbox *mbox = &pfvf->mbox; + struct mcs_secy_stats *rsp; + struct mcs_stats_req *req; + int ret; + + mutex_lock(&mbox->lock); + + req = otx2_mbox_alloc_msg_mcs_get_secy_stats(mbox); + if (!req) { + ret = -ENOMEM; + goto fail; + } + + req->id = hw_secy_id; + req->dir = dir; + + if (!clear) + goto send_msg; + + clear_req = otx2_mbox_alloc_msg_mcs_clear_stats(mbox); + if (!clear_req) { + ret = -ENOMEM; + goto fail; + } + clear_req->id = hw_secy_id; + clear_req->dir = dir; + clear_req->type = MCS_RSRC_TYPE_SECY; + +send_msg: + ret = otx2_sync_mbox_msg(mbox); + if (ret) + goto fail; + + rsp = (struct mcs_secy_stats *)otx2_mbox_get_rsp(&pfvf->mbox.mbox, + 0, &req->hdr); + if (IS_ERR(rsp)) { + ret = PTR_ERR(rsp); + goto fail; + } + + memcpy(rsp_p, rsp, sizeof(*rsp_p)); + + mutex_unlock(&mbox->lock); + + return 0; +fail: + mutex_unlock(&mbox->lock); + return ret; +} + +static struct cn10k_mcs_txsc *cn10k_mcs_create_txsc(struct otx2_nic *pfvf) +{ + struct cn10k_mcs_txsc *txsc; + int ret; + + txsc = kzalloc(sizeof(*txsc), GFP_KERNEL); + if (!txsc) + return ERR_PTR(-ENOMEM); + + ret = cn10k_mcs_alloc_rsrc(pfvf, MCS_TX, MCS_RSRC_TYPE_FLOWID, + &txsc->hw_flow_id); + if (ret) + goto fail; + + /* For a SecY, one TX secy and one RX secy HW resources are needed */ + ret = cn10k_mcs_alloc_rsrc(pfvf, MCS_TX, MCS_RSRC_TYPE_SECY, + &txsc->hw_secy_id_tx); + if (ret) + goto free_flowid; + + ret = cn10k_mcs_alloc_rsrc(pfvf, MCS_RX, MCS_RSRC_TYPE_SECY, + &txsc->hw_secy_id_rx); + if (ret) + goto free_tx_secy; + + ret = cn10k_mcs_alloc_rsrc(pfvf, MCS_TX, MCS_RSRC_TYPE_SC, + &txsc->hw_sc_id); + if (ret) + goto free_rx_secy; + + return txsc; +free_rx_secy: + cn10k_mcs_free_rsrc(pfvf, MCS_RX, MCS_RSRC_TYPE_SECY, + txsc->hw_secy_id_rx, false); +free_tx_secy: + cn10k_mcs_free_rsrc(pfvf, MCS_TX, MCS_RSRC_TYPE_SECY, + txsc->hw_secy_id_tx, false); +free_flowid: + cn10k_mcs_free_rsrc(pfvf, MCS_TX, MCS_RSRC_TYPE_FLOWID, + txsc->hw_flow_id, false); +fail: + return ERR_PTR(ret); +} + +/* Free Tx SC and its SAs(if any) resources to AF + */ +static void cn10k_mcs_delete_txsc(struct otx2_nic *pfvf, + struct cn10k_mcs_txsc *txsc) +{ + u8 sa_bmap = txsc->sa_bmap; + u8 sa_num = 0; + + while (sa_bmap) { + if (sa_bmap & 1) { + cn10k_mcs_write_tx_sa_plcy(pfvf, txsc->sw_secy, + txsc, sa_num); + cn10k_mcs_free_txsa(pfvf, txsc->hw_sa_id[sa_num]); + } + sa_num++; + sa_bmap >>= 1; + } + + cn10k_mcs_free_rsrc(pfvf, MCS_TX, MCS_RSRC_TYPE_SC, + txsc->hw_sc_id, false); + cn10k_mcs_free_rsrc(pfvf, MCS_RX, MCS_RSRC_TYPE_SECY, + txsc->hw_secy_id_rx, false); + cn10k_mcs_free_rsrc(pfvf, MCS_TX, MCS_RSRC_TYPE_SECY, + txsc->hw_secy_id_tx, false); + cn10k_mcs_free_rsrc(pfvf, MCS_TX, MCS_RSRC_TYPE_FLOWID, + txsc->hw_flow_id, false); +} + +static struct cn10k_mcs_rxsc *cn10k_mcs_create_rxsc(struct otx2_nic *pfvf) +{ + struct cn10k_mcs_rxsc *rxsc; + int ret; + + rxsc = kzalloc(sizeof(*rxsc), GFP_KERNEL); + if (!rxsc) + return ERR_PTR(-ENOMEM); + + ret = cn10k_mcs_alloc_rsrc(pfvf, MCS_RX, MCS_RSRC_TYPE_FLOWID, + &rxsc->hw_flow_id); + if (ret) + goto fail; + + ret = cn10k_mcs_alloc_rsrc(pfvf, MCS_RX, MCS_RSRC_TYPE_SC, + &rxsc->hw_sc_id); + if (ret) + goto free_flowid; + + return rxsc; +free_flowid: + cn10k_mcs_free_rsrc(pfvf, MCS_RX, MCS_RSRC_TYPE_FLOWID, + rxsc->hw_flow_id, false); +fail: + return ERR_PTR(ret); +} + +/* Free Rx SC and its SAs(if any) resources to AF + */ +static void cn10k_mcs_delete_rxsc(struct otx2_nic *pfvf, + struct cn10k_mcs_rxsc *rxsc) +{ + u8 sa_bmap = rxsc->sa_bmap; + u8 sa_num = 0; + + while (sa_bmap) { + if (sa_bmap & 1) { + cn10k_mcs_write_rx_sa_plcy(pfvf, rxsc->sw_secy, rxsc, + sa_num, false); + cn10k_mcs_free_rxsa(pfvf, rxsc->hw_sa_id[sa_num]); + } + sa_num++; + sa_bmap >>= 1; + } + + cn10k_mcs_free_rsrc(pfvf, MCS_RX, MCS_RSRC_TYPE_SC, + rxsc->hw_sc_id, false); + cn10k_mcs_free_rsrc(pfvf, MCS_RX, MCS_RSRC_TYPE_FLOWID, + rxsc->hw_flow_id, false); +} + +static int cn10k_mcs_secy_tx_cfg(struct otx2_nic *pfvf, struct macsec_secy *secy, + struct cn10k_mcs_txsc *txsc, + struct macsec_tx_sa *sw_tx_sa, u8 sa_num) +{ + if (sw_tx_sa) { + cn10k_mcs_write_tx_sa_plcy(pfvf, secy, txsc, sa_num); + cn10k_write_tx_sa_pn(pfvf, txsc, sa_num, + sw_tx_sa->next_pn_halves.lower); + cn10k_mcs_link_tx_sa2sc(pfvf, secy, txsc, sa_num, + sw_tx_sa->active); + } + + cn10k_mcs_write_tx_secy(pfvf, secy, txsc); + cn10k_mcs_write_tx_flowid(pfvf, secy, txsc); + /* When updating secy, change RX secy also */ + cn10k_mcs_write_rx_secy(pfvf, secy, txsc->hw_secy_id_rx); + + return 0; +} + +static int cn10k_mcs_secy_rx_cfg(struct otx2_nic *pfvf, + struct macsec_secy *secy, u8 hw_secy_id) +{ + struct cn10k_mcs_cfg *cfg = pfvf->macsec_cfg; + struct cn10k_mcs_rxsc *mcs_rx_sc; + struct macsec_rx_sc *sw_rx_sc; + struct macsec_rx_sa *sw_rx_sa; + u8 sa_num; + + for (sw_rx_sc = rcu_dereference_bh(secy->rx_sc); sw_rx_sc && sw_rx_sc->active; + sw_rx_sc = rcu_dereference_bh(sw_rx_sc->next)) { + mcs_rx_sc = cn10k_mcs_get_rxsc(cfg, secy, sw_rx_sc); + if (unlikely(!mcs_rx_sc)) + continue; + + for (sa_num = 0; sa_num < CN10K_MCS_SA_PER_SC; sa_num++) { + sw_rx_sa = rcu_dereference_bh(sw_rx_sc->sa[sa_num]); + if (!sw_rx_sa) + continue; + + cn10k_mcs_write_rx_sa_plcy(pfvf, secy, mcs_rx_sc, + sa_num, sw_rx_sa->active); + cn10k_mcs_write_rx_sa_pn(pfvf, mcs_rx_sc, sa_num, + sw_rx_sa->next_pn_halves.lower); + } + + cn10k_mcs_write_rx_flowid(pfvf, mcs_rx_sc, hw_secy_id); + cn10k_mcs_write_sc_cam(pfvf, mcs_rx_sc, hw_secy_id); + } + + return 0; +} + +static int cn10k_mcs_disable_rxscs(struct otx2_nic *pfvf, + struct macsec_secy *secy, + bool delete) +{ + struct cn10k_mcs_cfg *cfg = pfvf->macsec_cfg; + struct cn10k_mcs_rxsc *mcs_rx_sc; + struct macsec_rx_sc *sw_rx_sc; + int ret; + + for (sw_rx_sc = rcu_dereference_bh(secy->rx_sc); sw_rx_sc && sw_rx_sc->active; + sw_rx_sc = rcu_dereference_bh(sw_rx_sc->next)) { + mcs_rx_sc = cn10k_mcs_get_rxsc(cfg, secy, sw_rx_sc); + if (unlikely(!mcs_rx_sc)) + continue; + + ret = cn10k_mcs_ena_dis_flowid(pfvf, mcs_rx_sc->hw_flow_id, + false, MCS_RX); + if (ret) + dev_err(pfvf->dev, "Failed to disable TCAM for SC %d\n", + mcs_rx_sc->hw_sc_id); + if (delete) { + cn10k_mcs_delete_rxsc(pfvf, mcs_rx_sc); + list_del(&mcs_rx_sc->entry); + kfree(mcs_rx_sc); + } + } + + return 0; +} + +static void cn10k_mcs_sync_stats(struct otx2_nic *pfvf, struct macsec_secy *secy, + struct cn10k_mcs_txsc *txsc) +{ + struct cn10k_mcs_cfg *cfg = pfvf->macsec_cfg; + struct mcs_secy_stats rx_rsp = { 0 }; + struct mcs_sc_stats sc_rsp = { 0 }; + struct cn10k_mcs_rxsc *rxsc; + + /* Because of shared counters for some stats in the hardware, when + * updating secy policy take a snapshot of current stats and reset them. + * Below are the effected stats because of shared counters. + */ + + /* Check if sync is really needed */ + if (secy->validate_frames == txsc->last_validate_frames && + secy->protect_frames == txsc->last_protect_frames) + return; + + cn10k_mcs_secy_stats(pfvf, txsc->hw_secy_id_rx, &rx_rsp, MCS_RX, true); + + txsc->stats.InPktsBadTag += rx_rsp.pkt_badtag_cnt; + txsc->stats.InPktsUnknownSCI += rx_rsp.pkt_nosa_cnt; + txsc->stats.InPktsNoSCI += rx_rsp.pkt_nosaerror_cnt; + if (txsc->last_validate_frames == MACSEC_VALIDATE_STRICT) + txsc->stats.InPktsNoTag += rx_rsp.pkt_untaged_cnt; + else + txsc->stats.InPktsUntagged += rx_rsp.pkt_untaged_cnt; + + list_for_each_entry(rxsc, &cfg->rxsc_list, entry) { + cn10k_mcs_sc_stats(pfvf, rxsc->hw_sc_id, &sc_rsp, MCS_RX, true); + + rxsc->stats.InOctetsValidated += sc_rsp.octet_validate_cnt; + rxsc->stats.InOctetsDecrypted += sc_rsp.octet_decrypt_cnt; + + rxsc->stats.InPktsInvalid += sc_rsp.pkt_invalid_cnt; + rxsc->stats.InPktsNotValid += sc_rsp.pkt_notvalid_cnt; + + if (txsc->last_protect_frames) + rxsc->stats.InPktsLate += sc_rsp.pkt_late_cnt; + else + rxsc->stats.InPktsDelayed += sc_rsp.pkt_late_cnt; + + if (txsc->last_validate_frames == MACSEC_VALIDATE_CHECK) + rxsc->stats.InPktsUnchecked += sc_rsp.pkt_unchecked_cnt; + else + rxsc->stats.InPktsOK += sc_rsp.pkt_unchecked_cnt; + } + + txsc->last_validate_frames = secy->validate_frames; + txsc->last_protect_frames = secy->protect_frames; +} + +static int cn10k_mdo_open(struct macsec_context *ctx) +{ + struct otx2_nic *pfvf = netdev_priv(ctx->netdev); + struct cn10k_mcs_cfg *cfg = pfvf->macsec_cfg; + struct macsec_secy *secy = ctx->secy; + struct macsec_tx_sa *sw_tx_sa; + struct cn10k_mcs_txsc *txsc; + u8 sa_num; + int err; + + txsc = cn10k_mcs_get_txsc(cfg, ctx->secy); + if (!txsc) + return -ENOENT; + + sa_num = txsc->encoding_sa; + sw_tx_sa = rcu_dereference_bh(secy->tx_sc.sa[sa_num]); + + err = cn10k_mcs_secy_tx_cfg(pfvf, secy, txsc, sw_tx_sa, sa_num); + if (err) + return err; + + return cn10k_mcs_secy_rx_cfg(pfvf, secy, txsc->hw_secy_id_rx); +} + +static int cn10k_mdo_stop(struct macsec_context *ctx) +{ + struct otx2_nic *pfvf = netdev_priv(ctx->netdev); + struct cn10k_mcs_cfg *cfg = pfvf->macsec_cfg; + struct cn10k_mcs_txsc *txsc; + int err; + + txsc = cn10k_mcs_get_txsc(cfg, ctx->secy); + if (!txsc) + return -ENOENT; + + err = cn10k_mcs_ena_dis_flowid(pfvf, txsc->hw_flow_id, false, MCS_TX); + if (err) + return err; + + return cn10k_mcs_disable_rxscs(pfvf, ctx->secy, false); +} + +static int cn10k_mdo_add_secy(struct macsec_context *ctx) +{ + struct otx2_nic *pfvf = netdev_priv(ctx->netdev); + struct cn10k_mcs_cfg *cfg = pfvf->macsec_cfg; + struct macsec_secy *secy = ctx->secy; + struct cn10k_mcs_txsc *txsc; + + if (secy->icv_len != MACSEC_DEFAULT_ICV_LEN) + return -EOPNOTSUPP; + + /* Stick to 16 bytes key len until XPN support is added */ + if (secy->key_len != 16) + return -EOPNOTSUPP; + + if (secy->xpn) + return -EOPNOTSUPP; + + txsc = cn10k_mcs_create_txsc(pfvf); + if (IS_ERR(txsc)) + return -ENOSPC; + + txsc->sw_secy = secy; + txsc->encoding_sa = secy->tx_sc.encoding_sa; + txsc->last_validate_frames = secy->validate_frames; + txsc->last_protect_frames = secy->protect_frames; + + list_add(&txsc->entry, &cfg->txsc_list); + + if (netif_running(secy->netdev)) + return cn10k_mcs_secy_tx_cfg(pfvf, secy, txsc, NULL, 0); + + return 0; +} + +static int cn10k_mdo_upd_secy(struct macsec_context *ctx) +{ + struct otx2_nic *pfvf = netdev_priv(ctx->netdev); + struct cn10k_mcs_cfg *cfg = pfvf->macsec_cfg; + struct macsec_secy *secy = ctx->secy; + struct macsec_tx_sa *sw_tx_sa; + struct cn10k_mcs_txsc *txsc; + u8 sa_num; + int err; + + txsc = cn10k_mcs_get_txsc(cfg, secy); + if (!txsc) + return -ENOENT; + + txsc->encoding_sa = secy->tx_sc.encoding_sa; + + sa_num = txsc->encoding_sa; + sw_tx_sa = rcu_dereference_bh(secy->tx_sc.sa[sa_num]); + + if (netif_running(secy->netdev)) { + cn10k_mcs_sync_stats(pfvf, secy, txsc); + + err = cn10k_mcs_secy_tx_cfg(pfvf, secy, txsc, sw_tx_sa, sa_num); + if (err) + return err; + } + + return 0; +} + +static int cn10k_mdo_del_secy(struct macsec_context *ctx) +{ + struct otx2_nic *pfvf = netdev_priv(ctx->netdev); + struct cn10k_mcs_cfg *cfg = pfvf->macsec_cfg; + struct cn10k_mcs_txsc *txsc; + + txsc = cn10k_mcs_get_txsc(cfg, ctx->secy); + if (!txsc) + return -ENOENT; + + cn10k_mcs_ena_dis_flowid(pfvf, txsc->hw_flow_id, false, MCS_TX); + cn10k_mcs_disable_rxscs(pfvf, ctx->secy, true); + cn10k_mcs_delete_txsc(pfvf, txsc); + list_del(&txsc->entry); + kfree(txsc); + + return 0; +} + +static int cn10k_mdo_add_txsa(struct macsec_context *ctx) +{ + struct otx2_nic *pfvf = netdev_priv(ctx->netdev); + struct macsec_tx_sa *sw_tx_sa = ctx->sa.tx_sa; + struct cn10k_mcs_cfg *cfg = pfvf->macsec_cfg; + struct macsec_secy *secy = ctx->secy; + u8 sa_num = ctx->sa.assoc_num; + struct cn10k_mcs_txsc *txsc; + int err; + + txsc = cn10k_mcs_get_txsc(cfg, secy); + if (!txsc) + return -ENOENT; + + if (sa_num >= CN10K_MCS_SA_PER_SC) + return -EOPNOTSUPP; + + if (cn10k_mcs_alloc_txsa(pfvf, &txsc->hw_sa_id[sa_num])) + return -ENOSPC; + + memcpy(&txsc->sa_key[sa_num], ctx->sa.key, secy->key_len); + txsc->sa_bmap |= 1 << sa_num; + + if (netif_running(secy->netdev)) { + err = cn10k_mcs_write_tx_sa_plcy(pfvf, secy, txsc, sa_num); + if (err) + return err; + + err = cn10k_write_tx_sa_pn(pfvf, txsc, sa_num, + sw_tx_sa->next_pn_halves.lower); + if (err) + return err; + + err = cn10k_mcs_link_tx_sa2sc(pfvf, secy, txsc, + sa_num, sw_tx_sa->active); + if (err) + return err; + } + + return 0; +} + +static int cn10k_mdo_upd_txsa(struct macsec_context *ctx) +{ + struct otx2_nic *pfvf = netdev_priv(ctx->netdev); + struct macsec_tx_sa *sw_tx_sa = ctx->sa.tx_sa; + struct cn10k_mcs_cfg *cfg = pfvf->macsec_cfg; + struct macsec_secy *secy = ctx->secy; + u8 sa_num = ctx->sa.assoc_num; + struct cn10k_mcs_txsc *txsc; + int err; + + txsc = cn10k_mcs_get_txsc(cfg, secy); + if (!txsc) + return -ENOENT; + + if (sa_num >= CN10K_MCS_SA_PER_SC) + return -EOPNOTSUPP; + + if (netif_running(secy->netdev)) { + /* Keys cannot be changed after creation */ + err = cn10k_write_tx_sa_pn(pfvf, txsc, sa_num, + sw_tx_sa->next_pn_halves.lower); + if (err) + return err; + + err = cn10k_mcs_link_tx_sa2sc(pfvf, secy, txsc, + sa_num, sw_tx_sa->active); + if (err) + return err; + } + + return 0; +} + +static int cn10k_mdo_del_txsa(struct macsec_context *ctx) +{ + struct otx2_nic *pfvf = netdev_priv(ctx->netdev); + struct cn10k_mcs_cfg *cfg = pfvf->macsec_cfg; + u8 sa_num = ctx->sa.assoc_num; + struct cn10k_mcs_txsc *txsc; + + txsc = cn10k_mcs_get_txsc(cfg, ctx->secy); + if (!txsc) + return -ENOENT; + + if (sa_num >= CN10K_MCS_SA_PER_SC) + return -EOPNOTSUPP; + + cn10k_mcs_free_txsa(pfvf, txsc->hw_sa_id[sa_num]); + txsc->sa_bmap &= ~(1 << sa_num); + + return 0; +} + +static int cn10k_mdo_add_rxsc(struct macsec_context *ctx) +{ + struct otx2_nic *pfvf = netdev_priv(ctx->netdev); + struct cn10k_mcs_cfg *cfg = pfvf->macsec_cfg; + struct macsec_secy *secy = ctx->secy; + struct cn10k_mcs_rxsc *rxsc; + struct cn10k_mcs_txsc *txsc; + int err; + + txsc = cn10k_mcs_get_txsc(cfg, secy); + if (!txsc) + return -ENOENT; + + rxsc = cn10k_mcs_create_rxsc(pfvf); + if (IS_ERR(rxsc)) + return -ENOSPC; + + rxsc->sw_secy = ctx->secy; + rxsc->sw_rxsc = ctx->rx_sc; + list_add(&rxsc->entry, &cfg->rxsc_list); + + if (netif_running(secy->netdev)) { + err = cn10k_mcs_write_rx_flowid(pfvf, rxsc, txsc->hw_secy_id_rx); + if (err) + return err; + + err = cn10k_mcs_write_sc_cam(pfvf, rxsc, txsc->hw_secy_id_rx); + if (err) + return err; + } + + return 0; +} + +static int cn10k_mdo_upd_rxsc(struct macsec_context *ctx) +{ + struct otx2_nic *pfvf = netdev_priv(ctx->netdev); + struct cn10k_mcs_cfg *cfg = pfvf->macsec_cfg; + struct macsec_secy *secy = ctx->secy; + bool enable = ctx->rx_sc->active; + struct cn10k_mcs_rxsc *rxsc; + + rxsc = cn10k_mcs_get_rxsc(cfg, secy, ctx->rx_sc); + if (!rxsc) + return -ENOENT; + + if (netif_running(secy->netdev)) + return cn10k_mcs_ena_dis_flowid(pfvf, rxsc->hw_flow_id, + enable, MCS_RX); + + return 0; +} + +static int cn10k_mdo_del_rxsc(struct macsec_context *ctx) +{ + struct otx2_nic *pfvf = netdev_priv(ctx->netdev); + struct cn10k_mcs_cfg *cfg = pfvf->macsec_cfg; + struct cn10k_mcs_rxsc *rxsc; + + rxsc = cn10k_mcs_get_rxsc(cfg, ctx->secy, ctx->rx_sc); + if (!rxsc) + return -ENOENT; + + cn10k_mcs_ena_dis_flowid(pfvf, rxsc->hw_flow_id, false, MCS_RX); + cn10k_mcs_delete_rxsc(pfvf, rxsc); + list_del(&rxsc->entry); + kfree(rxsc); + + return 0; +} + +static int cn10k_mdo_add_rxsa(struct macsec_context *ctx) +{ + struct macsec_rx_sc *sw_rx_sc = ctx->sa.rx_sa->sc; + struct otx2_nic *pfvf = netdev_priv(ctx->netdev); + struct cn10k_mcs_cfg *cfg = pfvf->macsec_cfg; + struct macsec_rx_sa *rx_sa = ctx->sa.rx_sa; + u64 next_pn = rx_sa->next_pn_halves.lower; + struct macsec_secy *secy = ctx->secy; + bool sa_in_use = rx_sa->active; + u8 sa_num = ctx->sa.assoc_num; + struct cn10k_mcs_rxsc *rxsc; + int err; + + rxsc = cn10k_mcs_get_rxsc(cfg, secy, sw_rx_sc); + if (!rxsc) + return -ENOENT; + + if (sa_num >= CN10K_MCS_SA_PER_SC) + return -EOPNOTSUPP; + + if (cn10k_mcs_alloc_rxsa(pfvf, &rxsc->hw_sa_id[sa_num])) + return -ENOSPC; + + memcpy(&rxsc->sa_key[sa_num], ctx->sa.key, ctx->secy->key_len); + rxsc->sa_bmap |= 1 << sa_num; + + if (netif_running(secy->netdev)) { + err = cn10k_mcs_write_rx_sa_plcy(pfvf, secy, rxsc, + sa_num, sa_in_use); + if (err) + return err; + + err = cn10k_mcs_write_rx_sa_pn(pfvf, rxsc, sa_num, next_pn); + if (err) + return err; + } + + return 0; +} + +static int cn10k_mdo_upd_rxsa(struct macsec_context *ctx) +{ + struct macsec_rx_sc *sw_rx_sc = ctx->sa.rx_sa->sc; + struct otx2_nic *pfvf = netdev_priv(ctx->netdev); + struct cn10k_mcs_cfg *cfg = pfvf->macsec_cfg; + struct macsec_rx_sa *rx_sa = ctx->sa.rx_sa; + u64 next_pn = rx_sa->next_pn_halves.lower; + struct macsec_secy *secy = ctx->secy; + bool sa_in_use = rx_sa->active; + u8 sa_num = ctx->sa.assoc_num; + struct cn10k_mcs_rxsc *rxsc; + int err; + + rxsc = cn10k_mcs_get_rxsc(cfg, secy, sw_rx_sc); + if (!rxsc) + return -ENOENT; + + if (sa_num >= CN10K_MCS_SA_PER_SC) + return -EOPNOTSUPP; + + if (netif_running(secy->netdev)) { + err = cn10k_mcs_write_rx_sa_plcy(pfvf, secy, rxsc, sa_num, sa_in_use); + if (err) + return err; + + err = cn10k_mcs_write_rx_sa_pn(pfvf, rxsc, sa_num, next_pn); + if (err) + return err; + } + + return 0; +} + +static int cn10k_mdo_del_rxsa(struct macsec_context *ctx) +{ + struct macsec_rx_sc *sw_rx_sc = ctx->sa.rx_sa->sc; + struct otx2_nic *pfvf = netdev_priv(ctx->netdev); + struct cn10k_mcs_cfg *cfg = pfvf->macsec_cfg; + u8 sa_num = ctx->sa.assoc_num; + struct cn10k_mcs_rxsc *rxsc; + + rxsc = cn10k_mcs_get_rxsc(cfg, ctx->secy, sw_rx_sc); + if (!rxsc) + return -ENOENT; + + if (sa_num >= CN10K_MCS_SA_PER_SC) + return -EOPNOTSUPP; + + cn10k_mcs_write_rx_sa_plcy(pfvf, ctx->secy, rxsc, sa_num, false); + cn10k_mcs_free_rxsa(pfvf, rxsc->hw_sa_id[sa_num]); + + rxsc->sa_bmap &= ~(1 << sa_num); + + return 0; +} + +static int cn10k_mdo_get_dev_stats(struct macsec_context *ctx) +{ + struct mcs_secy_stats tx_rsp = { 0 }, rx_rsp = { 0 }; + struct otx2_nic *pfvf = netdev_priv(ctx->netdev); + struct cn10k_mcs_cfg *cfg = pfvf->macsec_cfg; + struct macsec_secy *secy = ctx->secy; + struct cn10k_mcs_txsc *txsc; + + txsc = cn10k_mcs_get_txsc(cfg, ctx->secy); + if (!txsc) + return -ENOENT; + + cn10k_mcs_secy_stats(pfvf, txsc->hw_secy_id_tx, &tx_rsp, MCS_TX, false); + ctx->stats.dev_stats->OutPktsUntagged = tx_rsp.pkt_untagged_cnt; + ctx->stats.dev_stats->OutPktsTooLong = tx_rsp.pkt_toolong_cnt; + + cn10k_mcs_secy_stats(pfvf, txsc->hw_secy_id_rx, &rx_rsp, MCS_RX, true); + txsc->stats.InPktsBadTag += rx_rsp.pkt_badtag_cnt; + txsc->stats.InPktsUnknownSCI += rx_rsp.pkt_nosa_cnt; + txsc->stats.InPktsNoSCI += rx_rsp.pkt_nosaerror_cnt; + if (secy->validate_frames == MACSEC_VALIDATE_STRICT) + txsc->stats.InPktsNoTag += rx_rsp.pkt_untaged_cnt; + else + txsc->stats.InPktsUntagged += rx_rsp.pkt_untaged_cnt; + txsc->stats.InPktsOverrun = 0; + + ctx->stats.dev_stats->InPktsNoTag = txsc->stats.InPktsNoTag; + ctx->stats.dev_stats->InPktsUntagged = txsc->stats.InPktsUntagged; + ctx->stats.dev_stats->InPktsBadTag = txsc->stats.InPktsBadTag; + ctx->stats.dev_stats->InPktsUnknownSCI = txsc->stats.InPktsUnknownSCI; + ctx->stats.dev_stats->InPktsNoSCI = txsc->stats.InPktsNoSCI; + ctx->stats.dev_stats->InPktsOverrun = txsc->stats.InPktsOverrun; + + return 0; +} + +static int cn10k_mdo_get_tx_sc_stats(struct macsec_context *ctx) +{ + struct otx2_nic *pfvf = netdev_priv(ctx->netdev); + struct cn10k_mcs_cfg *cfg = pfvf->macsec_cfg; + struct mcs_sc_stats rsp = { 0 }; + struct cn10k_mcs_txsc *txsc; + + txsc = cn10k_mcs_get_txsc(cfg, ctx->secy); + if (!txsc) + return -ENOENT; + + cn10k_mcs_sc_stats(pfvf, txsc->hw_sc_id, &rsp, MCS_TX, false); + + ctx->stats.tx_sc_stats->OutPktsProtected = rsp.pkt_protected_cnt; + ctx->stats.tx_sc_stats->OutPktsEncrypted = rsp.pkt_encrypt_cnt; + ctx->stats.tx_sc_stats->OutOctetsProtected = rsp.octet_protected_cnt; + ctx->stats.tx_sc_stats->OutOctetsEncrypted = rsp.octet_encrypt_cnt; + + return 0; +} + +static int cn10k_mdo_get_tx_sa_stats(struct macsec_context *ctx) +{ + struct otx2_nic *pfvf = netdev_priv(ctx->netdev); + struct cn10k_mcs_cfg *cfg = pfvf->macsec_cfg; + struct mcs_sa_stats rsp = { 0 }; + u8 sa_num = ctx->sa.assoc_num; + struct cn10k_mcs_txsc *txsc; + + txsc = cn10k_mcs_get_txsc(cfg, ctx->secy); + if (!txsc) + return -ENOENT; + + if (sa_num >= CN10K_MCS_SA_PER_SC) + return -EOPNOTSUPP; + + cn10k_mcs_sa_stats(pfvf, txsc->hw_sa_id[sa_num], &rsp, MCS_TX, false); + + ctx->stats.tx_sa_stats->OutPktsProtected = rsp.pkt_protected_cnt; + ctx->stats.tx_sa_stats->OutPktsEncrypted = rsp.pkt_encrypt_cnt; + + return 0; +} + +static int cn10k_mdo_get_rx_sc_stats(struct macsec_context *ctx) +{ + struct otx2_nic *pfvf = netdev_priv(ctx->netdev); + struct cn10k_mcs_cfg *cfg = pfvf->macsec_cfg; + struct macsec_secy *secy = ctx->secy; + struct mcs_sc_stats rsp = { 0 }; + struct cn10k_mcs_rxsc *rxsc; + + rxsc = cn10k_mcs_get_rxsc(cfg, secy, ctx->rx_sc); + if (!rxsc) + return -ENOENT; + + cn10k_mcs_sc_stats(pfvf, rxsc->hw_sc_id, &rsp, MCS_RX, true); + + rxsc->stats.InOctetsValidated += rsp.octet_validate_cnt; + rxsc->stats.InOctetsDecrypted += rsp.octet_decrypt_cnt; + + rxsc->stats.InPktsInvalid += rsp.pkt_invalid_cnt; + rxsc->stats.InPktsNotValid += rsp.pkt_notvalid_cnt; + + if (secy->protect_frames) + rxsc->stats.InPktsLate += rsp.pkt_late_cnt; + else + rxsc->stats.InPktsDelayed += rsp.pkt_late_cnt; + + if (secy->validate_frames == MACSEC_VALIDATE_CHECK) + rxsc->stats.InPktsUnchecked += rsp.pkt_unchecked_cnt; + else + rxsc->stats.InPktsOK += rsp.pkt_unchecked_cnt; + + ctx->stats.rx_sc_stats->InOctetsValidated = rxsc->stats.InOctetsValidated; + ctx->stats.rx_sc_stats->InOctetsDecrypted = rxsc->stats.InOctetsDecrypted; + ctx->stats.rx_sc_stats->InPktsInvalid = rxsc->stats.InPktsInvalid; + ctx->stats.rx_sc_stats->InPktsNotValid = rxsc->stats.InPktsNotValid; + ctx->stats.rx_sc_stats->InPktsLate = rxsc->stats.InPktsLate; + ctx->stats.rx_sc_stats->InPktsDelayed = rxsc->stats.InPktsDelayed; + ctx->stats.rx_sc_stats->InPktsUnchecked = rxsc->stats.InPktsUnchecked; + ctx->stats.rx_sc_stats->InPktsOK = rxsc->stats.InPktsOK; + + return 0; +} + +static int cn10k_mdo_get_rx_sa_stats(struct macsec_context *ctx) +{ + struct macsec_rx_sc *sw_rx_sc = ctx->sa.rx_sa->sc; + struct otx2_nic *pfvf = netdev_priv(ctx->netdev); + struct cn10k_mcs_cfg *cfg = pfvf->macsec_cfg; + struct mcs_sa_stats rsp = { 0 }; + u8 sa_num = ctx->sa.assoc_num; + struct cn10k_mcs_rxsc *rxsc; + + rxsc = cn10k_mcs_get_rxsc(cfg, ctx->secy, sw_rx_sc); + if (!rxsc) + return -ENOENT; + + if (sa_num >= CN10K_MCS_SA_PER_SC) + return -EOPNOTSUPP; + + cn10k_mcs_sa_stats(pfvf, rxsc->hw_sa_id[sa_num], &rsp, MCS_RX, false); + + ctx->stats.rx_sa_stats->InPktsOK = rsp.pkt_ok_cnt; + ctx->stats.rx_sa_stats->InPktsInvalid = rsp.pkt_invalid_cnt; + ctx->stats.rx_sa_stats->InPktsNotValid = rsp.pkt_notvalid_cnt; + ctx->stats.rx_sa_stats->InPktsNotUsingSA = rsp.pkt_nosaerror_cnt; + ctx->stats.rx_sa_stats->InPktsUnusedSA = rsp.pkt_nosa_cnt; + + return 0; +} + +static const struct macsec_ops cn10k_mcs_ops = { + .mdo_dev_open = cn10k_mdo_open, + .mdo_dev_stop = cn10k_mdo_stop, + .mdo_add_secy = cn10k_mdo_add_secy, + .mdo_upd_secy = cn10k_mdo_upd_secy, + .mdo_del_secy = cn10k_mdo_del_secy, + .mdo_add_rxsc = cn10k_mdo_add_rxsc, + .mdo_upd_rxsc = cn10k_mdo_upd_rxsc, + .mdo_del_rxsc = cn10k_mdo_del_rxsc, + .mdo_add_rxsa = cn10k_mdo_add_rxsa, + .mdo_upd_rxsa = cn10k_mdo_upd_rxsa, + .mdo_del_rxsa = cn10k_mdo_del_rxsa, + .mdo_add_txsa = cn10k_mdo_add_txsa, + .mdo_upd_txsa = cn10k_mdo_upd_txsa, + .mdo_del_txsa = cn10k_mdo_del_txsa, + .mdo_get_dev_stats = cn10k_mdo_get_dev_stats, + .mdo_get_tx_sc_stats = cn10k_mdo_get_tx_sc_stats, + .mdo_get_tx_sa_stats = cn10k_mdo_get_tx_sa_stats, + .mdo_get_rx_sc_stats = cn10k_mdo_get_rx_sc_stats, + .mdo_get_rx_sa_stats = cn10k_mdo_get_rx_sa_stats, +}; + +void cn10k_handle_mcs_event(struct otx2_nic *pfvf, struct mcs_intr_info *event) +{ + struct cn10k_mcs_cfg *cfg = pfvf->macsec_cfg; + struct macsec_tx_sa *sw_tx_sa = NULL; + struct macsec_secy *secy = NULL; + struct cn10k_mcs_txsc *txsc; + u8 an; + + if (!test_bit(CN10K_HW_MACSEC, &pfvf->hw.cap_flag)) + return; + + if (!(event->intr_mask & MCS_CPM_TX_PACKET_XPN_EQ0_INT)) + return; + + /* Find the SecY to which the expired hardware SA is mapped */ + list_for_each_entry(txsc, &cfg->txsc_list, entry) { + for (an = 0; an < CN10K_MCS_SA_PER_SC; an++) + if (txsc->hw_sa_id[an] == event->sa_id) { + secy = txsc->sw_secy; + sw_tx_sa = rcu_dereference_bh(secy->tx_sc.sa[an]); + } + } + + if (secy && sw_tx_sa) + macsec_pn_wrapped(secy, sw_tx_sa); +} + +int cn10k_mcs_init(struct otx2_nic *pfvf) +{ + struct mbox *mbox = &pfvf->mbox; + struct cn10k_mcs_cfg *cfg; + struct mcs_intr_cfg *req; + + if (!test_bit(CN10K_HW_MACSEC, &pfvf->hw.cap_flag)) + return 0; + + cfg = kzalloc(sizeof(*cfg), GFP_KERNEL); + if (!cfg) + return -ENOMEM; + + INIT_LIST_HEAD(&cfg->txsc_list); + INIT_LIST_HEAD(&cfg->rxsc_list); + pfvf->macsec_cfg = cfg; + + pfvf->netdev->features |= NETIF_F_HW_MACSEC; + pfvf->netdev->macsec_ops = &cn10k_mcs_ops; + + mutex_lock(&mbox->lock); + + req = otx2_mbox_alloc_msg_mcs_intr_cfg(mbox); + if (!req) + goto fail; + + req->intr_mask = MCS_CPM_TX_PACKET_XPN_EQ0_INT; + + if (otx2_sync_mbox_msg(mbox)) + goto fail; + + mutex_unlock(&mbox->lock); + + return 0; +fail: + dev_err(pfvf->dev, "Cannot notify PN wrapped event\n"); + return 0; +} + +void cn10k_mcs_free(struct otx2_nic *pfvf) +{ + if (!test_bit(CN10K_HW_MACSEC, &pfvf->hw.cap_flag)) + return; + + cn10k_mcs_free_rsrc(pfvf, MCS_TX, MCS_RSRC_TYPE_SECY, 0, true); + cn10k_mcs_free_rsrc(pfvf, MCS_RX, MCS_RSRC_TYPE_SECY, 0, true); + kfree(pfvf->macsec_cfg); + pfvf->macsec_cfg = NULL; +} diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.c b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.c index d686c7b6252f..9ac9e6615ae7 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.c +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.c @@ -586,8 +586,9 @@ void otx2_get_mac_from_af(struct net_device *netdev) } EXPORT_SYMBOL(otx2_get_mac_from_af); -int otx2_txschq_config(struct otx2_nic *pfvf, int lvl) +int otx2_txschq_config(struct otx2_nic *pfvf, int lvl, int prio, bool txschq_for_pfc) { + u16 (*schq_list)[MAX_TXSCHQ_PER_FUNC]; struct otx2_hw *hw = &pfvf->hw; struct nix_txschq_config *req; u64 schq, parent; @@ -602,7 +603,13 @@ int otx2_txschq_config(struct otx2_nic *pfvf, int lvl) req->lvl = lvl; req->num_regs = 1; - schq = hw->txschq_list[lvl][0]; + schq_list = hw->txschq_list; +#ifdef CONFIG_DCB + if (txschq_for_pfc) + schq_list = pfvf->pfc_schq_list; +#endif + + schq = schq_list[lvl][prio]; /* Set topology e.t.c configuration */ if (lvl == NIX_TXSCH_LVL_SMQ) { req->reg[0] = NIX_AF_SMQX_CFG(schq); @@ -611,7 +618,7 @@ int otx2_txschq_config(struct otx2_nic *pfvf, int lvl) (0x2ULL << 36); req->num_regs++; /* MDQ config */ - parent = hw->txschq_list[NIX_TXSCH_LVL_TL4][0]; + parent = schq_list[NIX_TXSCH_LVL_TL4][prio]; req->reg[1] = NIX_AF_MDQX_PARENT(schq); req->regval[1] = parent << 16; req->num_regs++; @@ -619,14 +626,14 @@ int otx2_txschq_config(struct otx2_nic *pfvf, int lvl) req->reg[2] = NIX_AF_MDQX_SCHEDULE(schq); req->regval[2] = dwrr_val; } else if (lvl == NIX_TXSCH_LVL_TL4) { - parent = hw->txschq_list[NIX_TXSCH_LVL_TL3][0]; + parent = schq_list[NIX_TXSCH_LVL_TL3][prio]; req->reg[0] = NIX_AF_TL4X_PARENT(schq); req->regval[0] = parent << 16; req->num_regs++; req->reg[1] = NIX_AF_TL4X_SCHEDULE(schq); req->regval[1] = dwrr_val; } else if (lvl == NIX_TXSCH_LVL_TL3) { - parent = hw->txschq_list[NIX_TXSCH_LVL_TL2][0]; + parent = schq_list[NIX_TXSCH_LVL_TL2][prio]; req->reg[0] = NIX_AF_TL3X_PARENT(schq); req->regval[0] = parent << 16; req->num_regs++; @@ -635,11 +642,13 @@ int otx2_txschq_config(struct otx2_nic *pfvf, int lvl) if (lvl == hw->txschq_link_cfg_lvl) { req->num_regs++; req->reg[2] = NIX_AF_TL3_TL2X_LINKX_CFG(schq, hw->tx_link); - /* Enable this queue and backpressure */ - req->regval[2] = BIT_ULL(13) | BIT_ULL(12); + /* Enable this queue and backpressure + * and set relative channel + */ + req->regval[2] = BIT_ULL(13) | BIT_ULL(12) | prio; } } else if (lvl == NIX_TXSCH_LVL_TL2) { - parent = hw->txschq_list[NIX_TXSCH_LVL_TL1][0]; + parent = schq_list[NIX_TXSCH_LVL_TL1][prio]; req->reg[0] = NIX_AF_TL2X_PARENT(schq); req->regval[0] = parent << 16; @@ -650,8 +659,10 @@ int otx2_txschq_config(struct otx2_nic *pfvf, int lvl) if (lvl == hw->txschq_link_cfg_lvl) { req->num_regs++; req->reg[2] = NIX_AF_TL3_TL2X_LINKX_CFG(schq, hw->tx_link); - /* Enable this queue and backpressure */ - req->regval[2] = BIT_ULL(13) | BIT_ULL(12); + /* Enable this queue and backpressure + * and set relative channel + */ + req->regval[2] = BIT_ULL(13) | BIT_ULL(12) | prio; } } else if (lvl == NIX_TXSCH_LVL_TL1) { /* Default config for TL1. @@ -676,6 +687,31 @@ int otx2_txschq_config(struct otx2_nic *pfvf, int lvl) return otx2_sync_mbox_msg(&pfvf->mbox); } +EXPORT_SYMBOL(otx2_txschq_config); + +int otx2_smq_flush(struct otx2_nic *pfvf, int smq) +{ + struct nix_txschq_config *req; + int rc; + + mutex_lock(&pfvf->mbox.lock); + + req = otx2_mbox_alloc_msg_nix_txschq_cfg(&pfvf->mbox); + if (!req) { + mutex_unlock(&pfvf->mbox.lock); + return -ENOMEM; + } + + req->lvl = NIX_TXSCH_LVL_SMQ; + req->reg[0] = NIX_AF_SMQX_CFG(smq); + req->regval[0] |= BIT_ULL(49); + req->num_regs++; + + rc = otx2_sync_mbox_msg(&pfvf->mbox); + mutex_unlock(&pfvf->mbox.lock); + return rc; +} +EXPORT_SYMBOL(otx2_smq_flush); int otx2_txsch_alloc(struct otx2_nic *pfvf) { @@ -806,8 +842,7 @@ int otx2_sq_aq_init(void *dev, u16 qidx, u16 sqb_aura) aq->sq.max_sqe_size = NIX_MAXSQESZ_W16; /* 128 byte */ aq->sq.cq_ena = 1; aq->sq.ena = 1; - /* Only one SMQ is allocated, map all SQ's to that SMQ */ - aq->sq.smq = pfvf->hw.txschq_list[NIX_TXSCH_LVL_SMQ][0]; + aq->sq.smq = otx2_get_smq_idx(pfvf, qidx); aq->sq.smq_rr_quantum = mtu_to_dwrr_weight(pfvf, pfvf->tx_max_pktlen); aq->sq.default_chan = pfvf->hw.tx_chan_base; aq->sq.sqe_stype = NIX_STYPE_STF; /* Cache SQB */ @@ -1792,4 +1827,5 @@ otx2_mbox_up_handler_ ## _fn_name(struct otx2_nic *pfvf, \ } \ EXPORT_SYMBOL(otx2_mbox_up_handler_ ## _fn_name); MBOX_UP_CGX_MESSAGES +MBOX_UP_MCS_MESSAGES #undef M diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h index b28029cc4316..282db6fe3b08 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h @@ -19,6 +19,7 @@ #include <net/devlink.h> #include <linux/time64.h> #include <linux/dim.h> +#include <uapi/linux/if_macsec.h> #include <mbox.h> #include <npc.h> @@ -33,6 +34,7 @@ #define PCI_DEVID_OCTEONTX2_RVU_AFVF 0xA0F8 #define PCI_SUBSYS_DEVID_96XX_RVU_PFVF 0xB200 +#define PCI_SUBSYS_DEVID_CN10K_B_RVU_PFVF 0xBD00 /* PCI BAR nos */ #define PCI_CFG_REG_BAR_NUM 2 @@ -40,6 +42,11 @@ #define NAME_SIZE 32 +#ifdef CONFIG_DCB +/* Max priority supported for PFC */ +#define NIX_PF_PFC_PRIO_MAX 8 +#endif + enum arua_mapped_qtypes { AURA_NIX_RQ, AURA_NIX_SQ, @@ -196,7 +203,7 @@ struct otx2_hw { /* NIX */ u8 txschq_link_cfg_lvl; - u16 txschq_list[NIX_TXSCH_LVL_CNT][MAX_TXSCHQ_PER_FUNC]; + u16 txschq_list[NIX_TXSCH_LVL_CNT][MAX_TXSCHQ_PER_FUNC]; u16 matchall_ipolicer; u32 dwrr_mtu; @@ -238,6 +245,8 @@ struct otx2_hw { #define CN10K_MBOX 1 #define CN10K_LMTST 2 #define CN10K_RPM 3 +#define CN10K_PTP_ONESTEP 4 +#define CN10K_HW_MACSEC 5 unsigned long cap_flag; #define LMT_LINE_SIZE 128 @@ -271,6 +280,13 @@ struct refill_work { struct otx2_nic *pf; }; +/* PTPv2 originTimestamp structure */ +struct ptpv2_tstamp { + __be16 seconds_msb; /* 16 bits + */ + __be32 seconds_lsb; /* 32 bits = 48 bits*/ + __be32 nanoseconds; +} __packed; + struct otx2_ptp { struct ptp_clock_info ptp_info; struct ptp_clock *ptp_clock; @@ -286,6 +302,9 @@ struct otx2_ptp { struct ptp_pin_desc extts_config; u64 (*convert_rx_ptp_tstmp)(u64 timestamp); u64 (*convert_tx_ptp_tstmp)(u64 timestamp); + struct delayed_work synctstamp_work; + u64 tstamp; + u32 base_ns; }; #define OTX2_HW_TIMESTAMP_LEN 8 @@ -335,6 +354,66 @@ struct dev_hw_ops { void (*aura_freeptr)(void *dev, int aura, u64 buf); }; +#define CN10K_MCS_SA_PER_SC 4 + +/* Stats which need to be accumulated in software because + * of shared counters in hardware. + */ +struct cn10k_txsc_stats { + u64 InPktsUntagged; + u64 InPktsNoTag; + u64 InPktsBadTag; + u64 InPktsUnknownSCI; + u64 InPktsNoSCI; + u64 InPktsOverrun; +}; + +struct cn10k_rxsc_stats { + u64 InOctetsValidated; + u64 InOctetsDecrypted; + u64 InPktsUnchecked; + u64 InPktsDelayed; + u64 InPktsOK; + u64 InPktsInvalid; + u64 InPktsLate; + u64 InPktsNotValid; + u64 InPktsNotUsingSA; + u64 InPktsUnusedSA; +}; + +struct cn10k_mcs_txsc { + struct macsec_secy *sw_secy; + struct cn10k_txsc_stats stats; + struct list_head entry; + enum macsec_validation_type last_validate_frames; + bool last_protect_frames; + u16 hw_secy_id_tx; + u16 hw_secy_id_rx; + u16 hw_flow_id; + u16 hw_sc_id; + u16 hw_sa_id[CN10K_MCS_SA_PER_SC]; + u8 sa_bmap; + u8 sa_key[CN10K_MCS_SA_PER_SC][MACSEC_MAX_KEY_LEN]; + u8 encoding_sa; +}; + +struct cn10k_mcs_rxsc { + struct macsec_secy *sw_secy; + struct macsec_rx_sc *sw_rxsc; + struct cn10k_rxsc_stats stats; + struct list_head entry; + u16 hw_flow_id; + u16 hw_sc_id; + u16 hw_sa_id[CN10K_MCS_SA_PER_SC]; + u8 sa_bmap; + u8 sa_key[CN10K_MCS_SA_PER_SC][MACSEC_MAX_KEY_LEN]; +}; + +struct cn10k_mcs_cfg { + struct list_head txsc_list; + struct list_head rxsc_list; +}; + struct otx2_nic { void __iomem *reg_base; struct net_device *netdev; @@ -358,6 +437,7 @@ struct otx2_nic { #define OTX2_FLAG_TC_MATCHALL_EGRESS_ENABLED BIT_ULL(12) #define OTX2_FLAG_TC_MATCHALL_INGRESS_ENABLED BIT_ULL(13) #define OTX2_FLAG_DMACFLTR_SUPPORT BIT_ULL(14) +#define OTX2_FLAG_PTP_ONESTEP_SYNC BIT_ULL(15) #define OTX2_FLAG_ADPTV_INT_COAL_ENABLED BIT_ULL(16) u64 flags; u64 *cq_op_addr; @@ -415,10 +495,16 @@ struct otx2_nic { /* PFC */ u8 pfc_en; u8 *queue_to_pfc_map; + u16 pfc_schq_list[NIX_TXSCH_LVL_CNT][MAX_TXSCHQ_PER_FUNC]; + bool pfc_alloc_status[NIX_PF_PFC_PRIO_MAX]; #endif /* napi event count. It is needed for adaptive irq coalescing. */ u32 napi_events; + +#if IS_ENABLED(CONFIG_MACSEC) + struct cn10k_mcs_cfg *macsec_cfg; +#endif }; static inline bool is_otx2_lbkvf(struct pci_dev *pdev) @@ -458,6 +544,11 @@ static inline bool is_dev_otx2(struct pci_dev *pdev) midr == PCI_REVISION_ID_95XXMM || midr == PCI_REVISION_ID_95XXO); } +static inline bool is_dev_cn10kb(struct pci_dev *pdev) +{ + return pdev->subsystem_device == PCI_SUBSYS_DEVID_CN10K_B_RVU_PFVF; +} + static inline void otx2_setup_dev_hw_settings(struct otx2_nic *pfvf) { struct otx2_hw *hw = &pfvf->hw; @@ -487,7 +578,11 @@ static inline void otx2_setup_dev_hw_settings(struct otx2_nic *pfvf) __set_bit(CN10K_MBOX, &hw->cap_flag); __set_bit(CN10K_LMTST, &hw->cap_flag); __set_bit(CN10K_RPM, &hw->cap_flag); + __set_bit(CN10K_PTP_ONESTEP, &hw->cap_flag); } + + if (is_dev_cn10kb(pfvf->pdev)) + __set_bit(CN10K_HW_MACSEC, &hw->cap_flag); } /* Register read/write APIs */ @@ -743,6 +838,7 @@ otx2_mbox_up_handler_ ## _fn_name(struct otx2_nic *pfvf, \ struct _rsp_type *rsp); \ MBOX_UP_CGX_MESSAGES +MBOX_UP_MCS_MESSAGES #undef M /* Time to wait before watchdog kicks off */ @@ -785,6 +881,16 @@ static inline void otx2_dma_unmap_page(struct otx2_nic *pfvf, dir, DMA_ATTR_SKIP_CPU_SYNC); } +static inline u16 otx2_get_smq_idx(struct otx2_nic *pfvf, u16 qidx) +{ +#ifdef CONFIG_DCB + if (pfvf->pfc_alloc_status[qidx]) + return pfvf->pfc_schq_list[NIX_TXSCH_LVL_SMQ][qidx]; +#endif + + return pfvf->hw.txschq_list[NIX_TXSCH_LVL_SMQ][0]; +} + /* MSI-X APIs */ void otx2_free_cints(struct otx2_nic *pfvf, int n); void otx2_set_cints_affinity(struct otx2_nic *pfvf); @@ -807,7 +913,7 @@ void otx2_free_aura_ptr(struct otx2_nic *pfvf, int type); void otx2_sq_free_sqbs(struct otx2_nic *pfvf); int otx2_config_nix(struct otx2_nic *pfvf); int otx2_config_nix_queues(struct otx2_nic *pfvf); -int otx2_txschq_config(struct otx2_nic *pfvf, int lvl); +int otx2_txschq_config(struct otx2_nic *pfvf, int lvl, int prio, bool pfc_en); int otx2_txsch_alloc(struct otx2_nic *pfvf); int otx2_txschq_stop(struct otx2_nic *pfvf); void otx2_sqb_flush(struct otx2_nic *pfvf); @@ -888,6 +994,8 @@ bool otx2_xdp_sq_append_pkt(struct otx2_nic *pfvf, u64 iova, int len, u16 qidx); u16 otx2_get_max_mtu(struct otx2_nic *pfvf); int otx2_handle_ntuple_tc_features(struct net_device *netdev, netdev_features_t features); +int otx2_smq_flush(struct otx2_nic *pfvf, int smq); + /* tc support */ int otx2_init_tc(struct otx2_nic *nic); void otx2_shutdown_tc(struct otx2_nic *nic); @@ -907,5 +1015,24 @@ void otx2_dmacflt_update_pfmac_flow(struct otx2_nic *pfvf); void otx2_update_bpid_in_rqctx(struct otx2_nic *pfvf, int vlan_prio, int qidx, bool pfc_enable); int otx2_config_priority_flow_ctrl(struct otx2_nic *pfvf); int otx2_dcbnl_set_ops(struct net_device *dev); +/* PFC support */ +int otx2_pfc_txschq_config(struct otx2_nic *pfvf); +int otx2_pfc_txschq_alloc(struct otx2_nic *pfvf); +int otx2_pfc_txschq_update(struct otx2_nic *pfvf); +int otx2_pfc_txschq_stop(struct otx2_nic *pfvf); #endif + +#if IS_ENABLED(CONFIG_MACSEC) +/* MACSEC offload support */ +int cn10k_mcs_init(struct otx2_nic *pfvf); +void cn10k_mcs_free(struct otx2_nic *pfvf); +void cn10k_handle_mcs_event(struct otx2_nic *pfvf, struct mcs_intr_info *event); +#else +static inline int cn10k_mcs_init(struct otx2_nic *pfvf) { return 0; } +static inline void cn10k_mcs_free(struct otx2_nic *pfvf) {} +static inline void cn10k_handle_mcs_event(struct otx2_nic *pfvf, + struct mcs_intr_info *event) +{} +#endif /* CONFIG_MACSEC */ + #endif /* OTX2_COMMON_H */ diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_dcbnl.c b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_dcbnl.c index 723d2506d309..ccaf97bb1ce0 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_dcbnl.c +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_dcbnl.c @@ -7,6 +7,289 @@ #include "otx2_common.h" +static int otx2_check_pfc_config(struct otx2_nic *pfvf) +{ + u8 tx_queues = pfvf->hw.tx_queues, prio; + u8 pfc_en = pfvf->pfc_en; + + for (prio = 0; prio < NIX_PF_PFC_PRIO_MAX; prio++) { + if ((pfc_en & (1 << prio)) && + prio > tx_queues - 1) { + dev_warn(pfvf->dev, + "Increase number of tx queues from %d to %d to support PFC.\n", + tx_queues, prio + 1); + return -EINVAL; + } + } + + return 0; +} + +int otx2_pfc_txschq_config(struct otx2_nic *pfvf) +{ + u8 pfc_en, pfc_bit_set; + int prio, lvl, err; + + pfc_en = pfvf->pfc_en; + for (prio = 0; prio < NIX_PF_PFC_PRIO_MAX; prio++) { + pfc_bit_set = pfc_en & (1 << prio); + + /* Either PFC bit is not set + * or tx scheduler is not allocated for the priority + */ + if (!pfc_bit_set || !pfvf->pfc_alloc_status[prio]) + continue; + + /* configure the scheduler for the tls*/ + for (lvl = 0; lvl < NIX_TXSCH_LVL_CNT; lvl++) { + err = otx2_txschq_config(pfvf, lvl, prio, true); + if (err) { + dev_err(pfvf->dev, + "%s configure PFC tx schq for lvl:%d, prio:%d failed!\n", + __func__, lvl, prio); + return err; + } + } + } + + return 0; +} + +static int otx2_pfc_txschq_alloc_one(struct otx2_nic *pfvf, u8 prio) +{ + struct nix_txsch_alloc_req *req; + struct nix_txsch_alloc_rsp *rsp; + int lvl, rc; + + /* Get memory to put this msg */ + req = otx2_mbox_alloc_msg_nix_txsch_alloc(&pfvf->mbox); + if (!req) + return -ENOMEM; + + /* Request one schq per level upto max level as configured + * link config level. These rest of the scheduler can be + * same as hw.txschq_list. + */ + for (lvl = 0; lvl < pfvf->hw.txschq_link_cfg_lvl; lvl++) + req->schq[lvl] = 1; + + rc = otx2_sync_mbox_msg(&pfvf->mbox); + if (rc) + return rc; + + rsp = (struct nix_txsch_alloc_rsp *) + otx2_mbox_get_rsp(&pfvf->mbox.mbox, 0, &req->hdr); + if (IS_ERR(rsp)) + return PTR_ERR(rsp); + + /* Setup transmit scheduler list */ + for (lvl = 0; lvl < pfvf->hw.txschq_link_cfg_lvl; lvl++) { + if (!rsp->schq[lvl]) + return -ENOSPC; + + pfvf->pfc_schq_list[lvl][prio] = rsp->schq_list[lvl][0]; + } + + /* Set the Tx schedulers for rest of the levels same as + * hw.txschq_list as those will be common for all. + */ + for (; lvl < NIX_TXSCH_LVL_CNT; lvl++) + pfvf->pfc_schq_list[lvl][prio] = pfvf->hw.txschq_list[lvl][0]; + + pfvf->pfc_alloc_status[prio] = true; + return 0; +} + +int otx2_pfc_txschq_alloc(struct otx2_nic *pfvf) +{ + u8 pfc_en = pfvf->pfc_en; + u8 pfc_bit_set; + int err, prio; + + for (prio = 0; prio < NIX_PF_PFC_PRIO_MAX; prio++) { + pfc_bit_set = pfc_en & (1 << prio); + + if (!pfc_bit_set || pfvf->pfc_alloc_status[prio]) + continue; + + /* Add new scheduler to the priority */ + err = otx2_pfc_txschq_alloc_one(pfvf, prio); + if (err) { + dev_err(pfvf->dev, "%s failed to allocate PFC TX schedulers\n", __func__); + return err; + } + } + + return 0; +} + +static int otx2_pfc_txschq_stop_one(struct otx2_nic *pfvf, u8 prio) +{ + struct nix_txsch_free_req *free_req; + + mutex_lock(&pfvf->mbox.lock); + /* free PFC TLx nodes */ + free_req = otx2_mbox_alloc_msg_nix_txsch_free(&pfvf->mbox); + if (!free_req) { + mutex_unlock(&pfvf->mbox.lock); + return -ENOMEM; + } + + free_req->flags = TXSCHQ_FREE_ALL; + otx2_sync_mbox_msg(&pfvf->mbox); + mutex_unlock(&pfvf->mbox.lock); + + pfvf->pfc_alloc_status[prio] = false; + return 0; +} + +static int otx2_pfc_update_sq_smq_mapping(struct otx2_nic *pfvf, int prio) +{ + struct nix_cn10k_aq_enq_req *cn10k_sq_aq; + struct net_device *dev = pfvf->netdev; + bool if_up = netif_running(dev); + struct nix_aq_enq_req *sq_aq; + + if (if_up) { + if (pfvf->pfc_alloc_status[prio]) + netif_tx_stop_all_queues(pfvf->netdev); + else + netif_tx_stop_queue(netdev_get_tx_queue(dev, prio)); + } + + if (test_bit(CN10K_LMTST, &pfvf->hw.cap_flag)) { + cn10k_sq_aq = otx2_mbox_alloc_msg_nix_cn10k_aq_enq(&pfvf->mbox); + if (!cn10k_sq_aq) + return -ENOMEM; + + /* Fill AQ info */ + cn10k_sq_aq->qidx = prio; + cn10k_sq_aq->ctype = NIX_AQ_CTYPE_SQ; + cn10k_sq_aq->op = NIX_AQ_INSTOP_WRITE; + + /* Fill fields to update */ + cn10k_sq_aq->sq.ena = 1; + cn10k_sq_aq->sq_mask.ena = 1; + cn10k_sq_aq->sq_mask.smq = GENMASK(9, 0); + cn10k_sq_aq->sq.smq = otx2_get_smq_idx(pfvf, prio); + } else { + sq_aq = otx2_mbox_alloc_msg_nix_aq_enq(&pfvf->mbox); + if (!sq_aq) + return -ENOMEM; + + /* Fill AQ info */ + sq_aq->qidx = prio; + sq_aq->ctype = NIX_AQ_CTYPE_SQ; + sq_aq->op = NIX_AQ_INSTOP_WRITE; + + /* Fill fields to update */ + sq_aq->sq.ena = 1; + sq_aq->sq_mask.ena = 1; + sq_aq->sq_mask.smq = GENMASK(8, 0); + sq_aq->sq.smq = otx2_get_smq_idx(pfvf, prio); + } + + otx2_sync_mbox_msg(&pfvf->mbox); + + if (if_up) { + if (pfvf->pfc_alloc_status[prio]) + netif_tx_start_all_queues(pfvf->netdev); + else + netif_tx_start_queue(netdev_get_tx_queue(dev, prio)); + } + + return 0; +} + +int otx2_pfc_txschq_update(struct otx2_nic *pfvf) +{ + bool if_up = netif_running(pfvf->netdev); + u8 pfc_en = pfvf->pfc_en, pfc_bit_set; + struct mbox *mbox = &pfvf->mbox; + int err, prio; + + mutex_lock(&mbox->lock); + for (prio = 0; prio < NIX_PF_PFC_PRIO_MAX; prio++) { + pfc_bit_set = pfc_en & (1 << prio); + + /* tx scheduler was created but user wants to disable now */ + if (!pfc_bit_set && pfvf->pfc_alloc_status[prio]) { + mutex_unlock(&mbox->lock); + if (if_up) + netif_tx_stop_all_queues(pfvf->netdev); + + otx2_smq_flush(pfvf, pfvf->pfc_schq_list[NIX_TXSCH_LVL_SMQ][prio]); + if (if_up) + netif_tx_start_all_queues(pfvf->netdev); + + /* delete the schq */ + err = otx2_pfc_txschq_stop_one(pfvf, prio); + if (err) { + dev_err(pfvf->dev, + "%s failed to stop PFC tx schedulers for priority: %d\n", + __func__, prio); + return err; + } + + mutex_lock(&mbox->lock); + goto update_sq_smq_map; + } + + /* Either PFC bit is not set + * or Tx scheduler is already mapped for the priority + */ + if (!pfc_bit_set || pfvf->pfc_alloc_status[prio]) + continue; + + /* Add new scheduler to the priority */ + err = otx2_pfc_txschq_alloc_one(pfvf, prio); + if (err) { + mutex_unlock(&mbox->lock); + dev_err(pfvf->dev, + "%s failed to allocate PFC tx schedulers for priority: %d\n", + __func__, prio); + return err; + } + +update_sq_smq_map: + err = otx2_pfc_update_sq_smq_mapping(pfvf, prio); + if (err) { + mutex_unlock(&mbox->lock); + dev_err(pfvf->dev, "%s failed PFC Tx schq sq:%d mapping", __func__, prio); + return err; + } + } + + err = otx2_pfc_txschq_config(pfvf); + mutex_unlock(&mbox->lock); + if (err) + return err; + + return 0; +} + +int otx2_pfc_txschq_stop(struct otx2_nic *pfvf) +{ + u8 pfc_en, pfc_bit_set; + int prio, err; + + pfc_en = pfvf->pfc_en; + for (prio = 0; prio < NIX_PF_PFC_PRIO_MAX; prio++) { + pfc_bit_set = pfc_en & (1 << prio); + if (!pfc_bit_set || !pfvf->pfc_alloc_status[prio]) + continue; + + /* Delete the existing scheduler */ + err = otx2_pfc_txschq_stop_one(pfvf, prio); + if (err) { + dev_err(pfvf->dev, "%s failed to stop PFC TX schedulers\n", __func__); + return err; + } + } + + return 0; +} + int otx2_config_priority_flow_ctrl(struct otx2_nic *pfvf) { struct cgx_pfc_cfg *req; @@ -128,6 +411,17 @@ static int otx2_dcbnl_ieee_setpfc(struct net_device *dev, struct ieee_pfc *pfc) /* Save PFC configuration to interface */ pfvf->pfc_en = pfc->pfc_en; + if (pfvf->hw.tx_queues >= NIX_PF_PFC_PRIO_MAX) + goto process_pfc; + + /* Check if the PFC configuration can be + * supported by the tx queue configuration + */ + err = otx2_check_pfc_config(pfvf); + if (err) + return err; + +process_pfc: err = otx2_config_priority_flow_ctrl(pfvf); if (err) return err; @@ -136,6 +430,12 @@ static int otx2_dcbnl_ieee_setpfc(struct net_device *dev, struct ieee_pfc *pfc) if (pfc->pfc_en) otx2_nix_config_bp(pfvf, true); + err = otx2_pfc_txschq_update(pfvf); + if (err) { + dev_err(pfvf->dev, "%s failed to update TX schedulers\n", __func__); + return err; + } + return 0; } diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_ethtool.c b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_ethtool.c index 3f60a80e34c8..0eb74e8c553d 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_ethtool.c +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_ethtool.c @@ -76,8 +76,8 @@ static void otx2_get_drvinfo(struct net_device *netdev, { struct otx2_nic *pfvf = netdev_priv(netdev); - strlcpy(info->driver, DRV_NAME, sizeof(info->driver)); - strlcpy(info->bus_info, pci_name(pfvf->pdev), sizeof(info->bus_info)); + strscpy(info->driver, DRV_NAME, sizeof(info->driver)); + strscpy(info->bus_info, pci_name(pfvf->pdev), sizeof(info->bus_info)); } static void otx2_get_qset_strings(struct otx2_nic *pfvf, u8 **data, int qset) @@ -963,10 +963,12 @@ static int otx2_get_ts_info(struct net_device *netdev, info->phc_index = otx2_ptp_clock_index(pfvf); - info->tx_types = (1 << HWTSTAMP_TX_OFF) | (1 << HWTSTAMP_TX_ON); + info->tx_types = BIT(HWTSTAMP_TX_OFF) | BIT(HWTSTAMP_TX_ON); + if (test_bit(CN10K_PTP_ONESTEP, &pfvf->hw.cap_flag)) + info->tx_types |= BIT(HWTSTAMP_TX_ONESTEP_SYNC); - info->rx_filters = (1 << HWTSTAMP_FILTER_NONE) | - (1 << HWTSTAMP_FILTER_ALL); + info->rx_filters = BIT(HWTSTAMP_FILTER_NONE) | + BIT(HWTSTAMP_FILTER_ALL); return 0; } @@ -1313,8 +1315,8 @@ static void otx2vf_get_drvinfo(struct net_device *netdev, { struct otx2_nic *vf = netdev_priv(netdev); - strlcpy(info->driver, DRV_VF_NAME, sizeof(info->driver)); - strlcpy(info->bus_info, pci_name(vf->pdev), sizeof(info->bus_info)); + strscpy(info->driver, DRV_VF_NAME, sizeof(info->driver)); + strscpy(info->bus_info, pci_name(vf->pdev), sizeof(info->bus_info)); } static void otx2vf_get_strings(struct net_device *netdev, u32 sset, u8 *data) diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c index 9376d0e62914..5803d7f9137c 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c @@ -858,6 +858,15 @@ static void otx2_handle_link_event(struct otx2_nic *pf) } } +int otx2_mbox_up_handler_mcs_intr_notify(struct otx2_nic *pf, + struct mcs_intr_info *event, + struct msg_rsp *rsp) +{ + cn10k_handle_mcs_event(pf, event); + + return 0; +} + int otx2_mbox_up_handler_cgx_link_event(struct otx2_nic *pf, struct cgx_link_info_msg *msg, struct msg_rsp *rsp) @@ -917,6 +926,7 @@ static int otx2_process_mbox_msg_up(struct otx2_nic *pf, return err; \ } MBOX_UP_CGX_MESSAGES +MBOX_UP_MCS_MESSAGES #undef M break; default: @@ -1389,18 +1399,40 @@ static int otx2_init_hw_resources(struct otx2_nic *pf) goto err_free_sq_ptrs; } +#ifdef CONFIG_DCB + if (pf->pfc_en) { + err = otx2_pfc_txschq_alloc(pf); + if (err) { + mutex_unlock(&mbox->lock); + goto err_free_sq_ptrs; + } + } +#endif + err = otx2_config_nix_queues(pf); if (err) { mutex_unlock(&mbox->lock); goto err_free_txsch; } + for (lvl = 0; lvl < NIX_TXSCH_LVL_CNT; lvl++) { - err = otx2_txschq_config(pf, lvl); + err = otx2_txschq_config(pf, lvl, 0, false); + if (err) { + mutex_unlock(&mbox->lock); + goto err_free_nix_queues; + } + } + +#ifdef CONFIG_DCB + if (pf->pfc_en) { + err = otx2_pfc_txschq_config(pf); if (err) { mutex_unlock(&mbox->lock); goto err_free_nix_queues; } } +#endif + mutex_unlock(&mbox->lock); return err; @@ -1455,6 +1487,11 @@ static void otx2_free_hw_resources(struct otx2_nic *pf) if (err) dev_err(pf->dev, "RVUPF: Failed to stop/free TX schedulers\n"); +#ifdef CONFIG_DCB + if (pf->pfc_en) + otx2_pfc_txschq_stop(pf); +#endif + mutex_lock(&mbox->lock); /* Disable backpressure */ if (!(pf->pcifunc & RVU_PFVF_FUNC_MASK)) @@ -1634,8 +1671,7 @@ int otx2_open(struct net_device *netdev) cq_poll->dev = (void *)pf; cq_poll->dim.mode = DIM_CQ_PERIOD_MODE_START_FROM_CQE; INIT_WORK(&cq_poll->dim.work, otx2_dim_work); - netif_napi_add(netdev, &cq_poll->napi, - otx2_napi_handler, NAPI_POLL_WEIGHT); + netif_napi_add(netdev, &cq_poll->napi, otx2_napi_handler); napi_enable(&cq_poll->napi); } @@ -1853,6 +1889,30 @@ static netdev_tx_t otx2_xmit(struct sk_buff *skb, struct net_device *netdev) return NETDEV_TX_OK; } +static u16 otx2_select_queue(struct net_device *netdev, struct sk_buff *skb, + struct net_device *sb_dev) +{ +#ifdef CONFIG_DCB + struct otx2_nic *pf = netdev_priv(netdev); + u8 vlan_prio; +#endif + +#ifdef CONFIG_DCB + if (!skb->vlan_present) + goto pick_tx; + + vlan_prio = skb->vlan_tci >> 13; + if ((vlan_prio > pf->hw.tx_queues - 1) || + !pf->pfc_alloc_status[vlan_prio]) + goto pick_tx; + + return vlan_prio; + +pick_tx: +#endif + return netdev_pick_tx(netdev, skb, NULL); +} + static netdev_features_t otx2_fix_features(struct net_device *dev, netdev_features_t features) { @@ -1987,8 +2047,19 @@ int otx2_config_hwtstamp(struct net_device *netdev, struct ifreq *ifr) switch (config.tx_type) { case HWTSTAMP_TX_OFF: + if (pfvf->flags & OTX2_FLAG_PTP_ONESTEP_SYNC) + pfvf->flags &= ~OTX2_FLAG_PTP_ONESTEP_SYNC; + + cancel_delayed_work(&pfvf->ptp->synctstamp_work); otx2_config_hw_tx_tstamp(pfvf, false); break; + case HWTSTAMP_TX_ONESTEP_SYNC: + if (!test_bit(CN10K_PTP_ONESTEP, &pfvf->hw.cap_flag)) + return -ERANGE; + pfvf->flags |= OTX2_FLAG_PTP_ONESTEP_SYNC; + schedule_delayed_work(&pfvf->ptp->synctstamp_work, + msecs_to_jiffies(500)); + fallthrough; case HWTSTAMP_TX_ON: otx2_config_hw_tx_tstamp(pfvf, true); break; @@ -2447,6 +2518,7 @@ static const struct net_device_ops otx2_netdev_ops = { .ndo_open = otx2_open, .ndo_stop = otx2_stop, .ndo_start_xmit = otx2_xmit, + .ndo_select_queue = otx2_select_queue, .ndo_fix_features = otx2_fix_features, .ndo_set_mac_address = otx2_set_mac_address, .ndo_change_mtu = otx2_change_mtu, @@ -2702,6 +2774,10 @@ static int otx2_probe(struct pci_dev *pdev, const struct pci_device_id *id) if (err) goto err_ptp_destroy; + err = cn10k_mcs_init(pf); + if (err) + goto err_del_mcam_entries; + if (pf->flags & OTX2_FLAG_NTUPLE_SUPPORT) netdev->hw_features |= NETIF_F_NTUPLE; @@ -2916,6 +2992,8 @@ static void otx2_remove(struct pci_dev *pdev) otx2_config_pause_frm(pf); } + cn10k_mcs_free(pf); + #ifdef CONFIG_DCB /* Disable PFC config */ if (pf->pfc_en) { diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_ptp.c b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_ptp.c index fdc2c9315b91..896b2f9bac34 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_ptp.c +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_ptp.c @@ -10,6 +10,33 @@ #include "otx2_common.h" #include "otx2_ptp.h" +static u64 otx2_ptp_get_clock(struct otx2_ptp *ptp) +{ + struct ptp_req *req; + struct ptp_rsp *rsp; + int err; + + if (!ptp->nic) + return 0; + + req = otx2_mbox_alloc_msg_ptp_op(&ptp->nic->mbox); + if (!req) + return 0; + + req->op = PTP_OP_GET_CLOCK; + + err = otx2_sync_mbox_msg(&ptp->nic->mbox); + if (err) + return 0; + + rsp = (struct ptp_rsp *)otx2_mbox_get_rsp(&ptp->nic->mbox.mbox, 0, + &req->hdr); + if (IS_ERR(rsp)) + return 0; + + return rsp->clk; +} + static int otx2_ptp_adjfine(struct ptp_clock_info *ptp_info, long scaled_ppm) { struct otx2_ptp *ptp = container_of(ptp_info, struct otx2_ptp, @@ -46,32 +73,28 @@ static int ptp_set_thresh(struct otx2_ptp *ptp, u64 thresh) return otx2_sync_mbox_msg(&ptp->nic->mbox); } -static u64 ptp_cc_read(const struct cyclecounter *cc) +static int ptp_extts_on(struct otx2_ptp *ptp, int on) { - struct otx2_ptp *ptp = container_of(cc, struct otx2_ptp, cycle_counter); struct ptp_req *req; - struct ptp_rsp *rsp; - int err; if (!ptp->nic) - return 0; + return -ENODEV; req = otx2_mbox_alloc_msg_ptp_op(&ptp->nic->mbox); if (!req) - return 0; + return -ENOMEM; - req->op = PTP_OP_GET_CLOCK; + req->op = PTP_OP_EXTTS_ON; + req->extts_on = on; - err = otx2_sync_mbox_msg(&ptp->nic->mbox); - if (err) - return 0; + return otx2_sync_mbox_msg(&ptp->nic->mbox); +} - rsp = (struct ptp_rsp *)otx2_mbox_get_rsp(&ptp->nic->mbox.mbox, 0, - &req->hdr); - if (IS_ERR(rsp)) - return 0; +static u64 ptp_cc_read(const struct cyclecounter *cc) +{ + struct otx2_ptp *ptp = container_of(cc, struct otx2_ptp, cycle_counter); - return rsp->clk; + return otx2_ptp_get_clock(ptp); } static u64 ptp_tstmp_read(struct otx2_ptp *ptp) @@ -101,6 +124,15 @@ static u64 ptp_tstmp_read(struct otx2_ptp *ptp) return rsp->clk; } +static void otx2_get_ptpclock(struct otx2_ptp *ptp, u64 *tstamp) +{ + struct otx2_nic *pfvf = ptp->nic; + + mutex_lock(&pfvf->mbox.lock); + *tstamp = timecounter_read(&ptp->time_counter); + mutex_unlock(&pfvf->mbox.lock); +} + static int otx2_ptp_adjtime(struct ptp_clock_info *ptp_info, s64 delta) { struct otx2_ptp *ptp = container_of(ptp_info, struct otx2_ptp, @@ -119,14 +151,10 @@ static int otx2_ptp_gettime(struct ptp_clock_info *ptp_info, { struct otx2_ptp *ptp = container_of(ptp_info, struct otx2_ptp, ptp_info); - struct otx2_nic *pfvf = ptp->nic; - u64 nsec; + u64 tstamp; - mutex_lock(&pfvf->mbox.lock); - nsec = timecounter_read(&ptp->time_counter); - mutex_unlock(&pfvf->mbox.lock); - - *ts = ns_to_timespec64(nsec); + otx2_get_ptpclock(ptp, &tstamp); + *ts = ns_to_timespec64(tstamp); return 0; } @@ -178,8 +206,6 @@ static void otx2_ptp_extts_check(struct work_struct *work) event.index = 0; event.timestamp = timecounter_cyc2time(&ptp->time_counter, tstmp); ptp_clock_event(ptp->ptp_clock, &event); - ptp->last_extts = tstmp; - new_thresh = tstmp % 500000000; if (ptp->thresh != new_thresh) { mutex_lock(&ptp->nic->mbox.lock); @@ -187,10 +213,28 @@ static void otx2_ptp_extts_check(struct work_struct *work) mutex_unlock(&ptp->nic->mbox.lock); ptp->thresh = new_thresh; } + ptp->last_extts = tstmp; } schedule_delayed_work(&ptp->extts_work, msecs_to_jiffies(200)); } +static void otx2_sync_tstamp(struct work_struct *work) +{ + struct otx2_ptp *ptp = container_of(work, struct otx2_ptp, + synctstamp_work.work); + struct otx2_nic *pfvf = ptp->nic; + u64 tstamp; + + mutex_lock(&pfvf->mbox.lock); + tstamp = otx2_ptp_get_clock(ptp); + mutex_unlock(&pfvf->mbox.lock); + + ptp->tstamp = timecounter_cyc2time(&pfvf->ptp->time_counter, tstamp); + ptp->base_ns = tstamp % NSEC_PER_SEC; + + schedule_delayed_work(&ptp->synctstamp_work, msecs_to_jiffies(250)); +} + static int otx2_ptp_enable(struct ptp_clock_info *ptp_info, struct ptp_clock_request *rq, int on) { @@ -207,10 +251,13 @@ static int otx2_ptp_enable(struct ptp_clock_info *ptp_info, rq->extts.index); if (pin < 0) return -EBUSY; - if (on) + if (on) { + ptp_extts_on(ptp, on); schedule_delayed_work(&ptp->extts_work, msecs_to_jiffies(200)); - else + } else { + ptp_extts_on(ptp, on); cancel_delayed_work_sync(&ptp->extts_work); + } return 0; default: break; @@ -302,6 +349,8 @@ int otx2_ptp_init(struct otx2_nic *pfvf) ptp_ptr->convert_tx_ptp_tstmp = &cn10k_ptp_convert_timestamp; } + INIT_DELAYED_WORK(&ptp_ptr->synctstamp_work, otx2_sync_tstamp); + pfvf->ptp = ptp_ptr; error: @@ -316,6 +365,8 @@ void otx2_ptp_destroy(struct otx2_nic *pfvf) if (!ptp) return; + cancel_delayed_work(&pfvf->ptp->synctstamp_work); + ptp_clock_unregister(ptp->ptp_clock); kfree(ptp); pfvf->ptp = NULL; diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_struct.h b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_struct.h index 4bbd12ff26e6..aa205a0d158f 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_struct.h +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_struct.h @@ -236,8 +236,15 @@ struct nix_sqe_sg_s { /* NIX send memory subdescriptor structure */ struct nix_sqe_mem_s { - u64 offset : 16; /* W0 */ - u64 rsvd_51_16 : 36; + u64 start_offset : 8; + u64 rsvd_11_8 : 4; + u64 rsvd_12 : 1; + u64 udp_csum_crt : 1; + u64 update64 : 1; + u64 rsvd_15_16 : 1; + u64 base_ns : 32; + u64 step_type : 1; + u64 rsvd_51_49 : 3; u64 per_lso_seg : 1; u64 wmem : 1; u64 dsz : 2; diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_txrx.c b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_txrx.c index a18e8efd0f1e..5ec11d71bf60 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_txrx.c +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_txrx.c @@ -19,6 +19,12 @@ #include "cn10k.h" #define CQE_ADDR(CQ, idx) ((CQ)->cqe_base + ((CQ)->cqe_size * (idx))) +#define PTP_PORT 0x13F +/* PTPv2 header Original Timestamp starts at byte offset 34 and + * contains 6 byte seconds field and 4 byte nano seconds field. + */ +#define PTP_SYNC_SEC_OFFSET 34 + static bool otx2_xdp_rcv_pkt_handler(struct otx2_nic *pfvf, struct bpf_prog *prog, struct nix_cqe_rx_s *cqe, @@ -686,7 +692,8 @@ static void otx2_sqe_add_ext(struct otx2_nic *pfvf, struct otx2_snd_queue *sq, } static void otx2_sqe_add_mem(struct otx2_snd_queue *sq, int *offset, - int alg, u64 iova) + int alg, u64 iova, int ptp_offset, + u64 base_ns, int udp_csum) { struct nix_sqe_mem_s *mem; @@ -696,6 +703,13 @@ static void otx2_sqe_add_mem(struct otx2_snd_queue *sq, int *offset, mem->wmem = 1; /* wait for the memory operation */ mem->addr = iova; + if (ptp_offset) { + mem->start_offset = ptp_offset; + mem->udp_csum_crt = udp_csum; + mem->base_ns = base_ns; + mem->step_type = 1; + } + *offset += sizeof(*mem); } @@ -952,16 +966,102 @@ static int otx2_get_sqe_count(struct otx2_nic *pfvf, struct sk_buff *skb) return skb_shinfo(skb)->gso_segs; } +static bool otx2_validate_network_transport(struct sk_buff *skb) +{ + if ((ip_hdr(skb)->protocol == IPPROTO_UDP) || + (ipv6_hdr(skb)->nexthdr == IPPROTO_UDP)) { + struct udphdr *udph = udp_hdr(skb); + + if (udph->source == htons(PTP_PORT) && + udph->dest == htons(PTP_PORT)) + return true; + } + + return false; +} + +static bool otx2_ptp_is_sync(struct sk_buff *skb, int *offset, int *udp_csum) +{ + struct ethhdr *eth = (struct ethhdr *)(skb->data); + u16 nix_offload_hlen = 0, inner_vhlen = 0; + u8 *data = skb->data, *msgtype; + __be16 proto = eth->h_proto; + int network_depth = 0; + + /* NIX is programmed to offload outer VLAN header + * in case of single vlan protocol field holds Network header ETH_IP/V6 + * in case of stacked vlan protocol field holds Inner vlan (8100) + */ + if (skb->dev->features & NETIF_F_HW_VLAN_CTAG_TX && + skb->dev->features & NETIF_F_HW_VLAN_STAG_TX) { + if (skb->vlan_proto == htons(ETH_P_8021AD)) { + /* Get vlan protocol */ + proto = __vlan_get_protocol(skb, eth->h_proto, NULL); + /* SKB APIs like skb_transport_offset does not include + * offloaded vlan header length. Need to explicitly add + * the length + */ + nix_offload_hlen = VLAN_HLEN; + inner_vhlen = VLAN_HLEN; + } else if (skb->vlan_proto == htons(ETH_P_8021Q)) { + nix_offload_hlen = VLAN_HLEN; + } + } else if (eth_type_vlan(eth->h_proto)) { + proto = __vlan_get_protocol(skb, eth->h_proto, &network_depth); + } + + switch (ntohs(proto)) { + case ETH_P_1588: + if (network_depth) + *offset = network_depth; + else + *offset = ETH_HLEN + nix_offload_hlen + + inner_vhlen; + break; + case ETH_P_IP: + case ETH_P_IPV6: + if (!otx2_validate_network_transport(skb)) + return false; + + *udp_csum = 1; + *offset = nix_offload_hlen + skb_transport_offset(skb) + + sizeof(struct udphdr); + } + + msgtype = data + *offset; + + /* Check PTP messageId is SYNC or not */ + return (*msgtype & 0xf) == 0; +} + static void otx2_set_txtstamp(struct otx2_nic *pfvf, struct sk_buff *skb, struct otx2_snd_queue *sq, int *offset) { + struct ptpv2_tstamp *origin_tstamp; + int ptp_offset = 0, udp_csum = 0; + struct timespec64 ts; u64 iova; - if (!skb_shinfo(skb)->gso_size && - skb_shinfo(skb)->tx_flags & SKBTX_HW_TSTAMP) { - skb_shinfo(skb)->tx_flags |= SKBTX_IN_PROGRESS; + if (unlikely(!skb_shinfo(skb)->gso_size && + (skb_shinfo(skb)->tx_flags & SKBTX_HW_TSTAMP))) { + if (unlikely(pfvf->flags & OTX2_FLAG_PTP_ONESTEP_SYNC)) { + if (otx2_ptp_is_sync(skb, &ptp_offset, &udp_csum)) { + origin_tstamp = (struct ptpv2_tstamp *) + ((u8 *)skb->data + ptp_offset + + PTP_SYNC_SEC_OFFSET); + ts = ns_to_timespec64(pfvf->ptp->tstamp); + origin_tstamp->seconds_msb = htons((ts.tv_sec >> 32) & 0xffff); + origin_tstamp->seconds_lsb = htonl(ts.tv_sec & 0xffffffff); + origin_tstamp->nanoseconds = htonl(ts.tv_nsec); + /* Point to correction field in PTP packet */ + ptp_offset += 8; + } + } else { + skb_shinfo(skb)->tx_flags |= SKBTX_IN_PROGRESS; + } iova = sq->timestamps->iova + (sq->head * sizeof(u64)); - otx2_sqe_add_mem(sq, offset, NIX_SENDMEMALG_E_SETTSTMP, iova); + otx2_sqe_add_mem(sq, offset, NIX_SENDMEMALG_E_SETTSTMP, iova, + ptp_offset, pfvf->ptp->base_ns, udp_csum); } else { skb_tx_timestamp(skb); } diff --git a/drivers/net/ethernet/marvell/prestera/Makefile b/drivers/net/ethernet/marvell/prestera/Makefile index d395f4131648..df14cee80153 100644 --- a/drivers/net/ethernet/marvell/prestera/Makefile +++ b/drivers/net/ethernet/marvell/prestera/Makefile @@ -4,6 +4,6 @@ prestera-objs := prestera_main.o prestera_hw.o prestera_dsa.o \ prestera_rxtx.o prestera_devlink.o prestera_ethtool.o \ prestera_switchdev.o prestera_acl.o prestera_flow.o \ prestera_flower.o prestera_span.o prestera_counter.o \ - prestera_router.o prestera_router_hw.o + prestera_router.o prestera_router_hw.o prestera_matchall.o obj-$(CONFIG_PRESTERA_PCI) += prestera_pci.o diff --git a/drivers/net/ethernet/marvell/prestera/prestera.h b/drivers/net/ethernet/marvell/prestera/prestera.h index 2f84d0fb4094..35554ee805cd 100644 --- a/drivers/net/ethernet/marvell/prestera/prestera.h +++ b/drivers/net/ethernet/marvell/prestera/prestera.h @@ -306,17 +306,27 @@ struct prestera_switch { struct prestera_counter *counter; u8 lag_member_max; u8 lag_max; + u32 size_tbl_router_nexthop; }; struct prestera_router { struct prestera_switch *sw; struct list_head vr_list; struct list_head rif_entry_list; + struct rhashtable nh_neigh_ht; + struct rhashtable nexthop_group_ht; struct rhashtable fib_ht; + struct rhashtable kern_neigh_cache_ht; struct rhashtable kern_fib_cache_ht; struct notifier_block inetaddr_nb; struct notifier_block inetaddr_valid_nb; struct notifier_block fib_nb; + struct notifier_block netevent_nb; + u8 *nhgrp_hw_state_cache; /* Bitmap cached hw state of nhs */ + unsigned long nhgrp_hw_cache_kick; /* jiffies */ + struct { + struct delayed_work dw; + } neighs_update; }; struct prestera_rxtx_params { @@ -362,11 +372,15 @@ int prestera_port_cfg_mac_write(struct prestera_port *port, struct prestera_port *prestera_port_dev_lower_find(struct net_device *dev); void prestera_queue_work(struct work_struct *work); +void prestera_queue_delayed_work(struct delayed_work *work, unsigned long delay); +void prestera_queue_drain(void); int prestera_port_learning_set(struct prestera_port *port, bool learn_enable); int prestera_port_uc_flood_set(struct prestera_port *port, bool flood); int prestera_port_mc_flood_set(struct prestera_port *port, bool flood); +int prestera_port_br_locked_set(struct prestera_port *port, bool br_locked); + int prestera_port_pvid_set(struct prestera_port *port, u16 vid); bool prestera_netdev_check(const struct net_device *dev); diff --git a/drivers/net/ethernet/marvell/prestera/prestera_acl.c b/drivers/net/ethernet/marvell/prestera/prestera_acl.c index 3d4b85f2d541..cba89fda504b 100644 --- a/drivers/net/ethernet/marvell/prestera/prestera_acl.c +++ b/drivers/net/ethernet/marvell/prestera/prestera_acl.c @@ -54,6 +54,10 @@ struct prestera_acl_ruleset { struct prestera_acl_ruleset_ht_key ht_key; struct rhashtable rule_ht; struct prestera_acl *acl; + struct { + u32 min; + u32 max; + } prio; unsigned long rule_count; refcount_t refcount; void *keymask; @@ -162,6 +166,9 @@ prestera_acl_ruleset_create(struct prestera_acl *acl, ruleset->pcl_id = PRESTERA_ACL_PCL_ID_MAKE((u8)uid, chain_index); ruleset->index = uid; + ruleset->prio.min = UINT_MAX; + ruleset->prio.max = 0; + err = rhashtable_insert_fast(&acl->ruleset_ht, &ruleset->ht_node, prestera_acl_ruleset_ht_params); if (err) @@ -178,10 +185,14 @@ err_rhashtable_init: return ERR_PTR(err); } -void prestera_acl_ruleset_keymask_set(struct prestera_acl_ruleset *ruleset, - void *keymask) +int prestera_acl_ruleset_keymask_set(struct prestera_acl_ruleset *ruleset, + void *keymask) { ruleset->keymask = kmemdup(keymask, ACL_KEYMASK_SIZE, GFP_KERNEL); + if (!ruleset->keymask) + return -ENOMEM; + + return 0; } int prestera_acl_ruleset_offload(struct prestera_acl_ruleset *ruleset) @@ -365,6 +376,26 @@ prestera_acl_ruleset_block_unbind(struct prestera_acl_ruleset *ruleset, block->ruleset_zero = NULL; } +static void +prestera_acl_ruleset_prio_refresh(struct prestera_acl *acl, + struct prestera_acl_ruleset *ruleset) +{ + struct prestera_acl_rule *rule; + + ruleset->prio.min = UINT_MAX; + ruleset->prio.max = 0; + + list_for_each_entry(rule, &acl->rules, list) { + if (ruleset->ingress != rule->ruleset->ingress) + continue; + if (ruleset->ht_key.chain_index != rule->chain_index) + continue; + + ruleset->prio.min = min(ruleset->prio.min, rule->priority); + ruleset->prio.max = max(ruleset->prio.max, rule->priority); + } +} + void prestera_acl_rule_keymask_pcl_id_set(struct prestera_acl_rule *rule, u16 pcl_id) { @@ -389,6 +420,13 @@ u32 prestera_acl_ruleset_index_get(const struct prestera_acl_ruleset *ruleset) return ruleset->index; } +void prestera_acl_ruleset_prio_get(struct prestera_acl_ruleset *ruleset, + u32 *prio_min, u32 *prio_max) +{ + *prio_min = ruleset->prio.min; + *prio_max = ruleset->prio.max; +} + bool prestera_acl_ruleset_is_offload(struct prestera_acl_ruleset *ruleset) { return ruleset->offload; @@ -429,6 +467,13 @@ void prestera_acl_rule_destroy(struct prestera_acl_rule *rule) kfree(rule); } +static void prestera_acl_ruleset_prio_update(struct prestera_acl_ruleset *ruleset, + u32 prio) +{ + ruleset->prio.min = min(ruleset->prio.min, prio); + ruleset->prio.max = max(ruleset->prio.max, prio); +} + int prestera_acl_rule_add(struct prestera_switch *sw, struct prestera_acl_rule *rule) { @@ -468,6 +513,7 @@ int prestera_acl_rule_add(struct prestera_switch *sw, list_add_tail(&rule->list, &sw->acl->rules); ruleset->rule_count++; + prestera_acl_ruleset_prio_update(ruleset, rule->priority); return 0; err_acl_block_bind: @@ -492,6 +538,7 @@ void prestera_acl_rule_del(struct prestera_switch *sw, list_del(&rule->list); prestera_acl_rule_entry_destroy(sw->acl, rule->re); + prestera_acl_ruleset_prio_refresh(sw->acl, ruleset); /* unbind block (all ports) */ if (!ruleset->ht_key.chain_index && !ruleset->rule_count) diff --git a/drivers/net/ethernet/marvell/prestera/prestera_acl.h b/drivers/net/ethernet/marvell/prestera/prestera_acl.h index 03fc5b9dc925..a35cc0609a1d 100644 --- a/drivers/net/ethernet/marvell/prestera/prestera_acl.h +++ b/drivers/net/ethernet/marvell/prestera/prestera_acl.h @@ -185,8 +185,8 @@ struct prestera_acl_ruleset * prestera_acl_ruleset_lookup(struct prestera_acl *acl, struct prestera_flow_block *block, u32 chain_index); -void prestera_acl_ruleset_keymask_set(struct prestera_acl_ruleset *ruleset, - void *keymask); +int prestera_acl_ruleset_keymask_set(struct prestera_acl_ruleset *ruleset, + void *keymask); bool prestera_acl_ruleset_is_offload(struct prestera_acl_ruleset *ruleset); int prestera_acl_ruleset_offload(struct prestera_acl_ruleset *ruleset); void prestera_acl_ruleset_put(struct prestera_acl_ruleset *ruleset); @@ -195,6 +195,8 @@ int prestera_acl_ruleset_bind(struct prestera_acl_ruleset *ruleset, int prestera_acl_ruleset_unbind(struct prestera_acl_ruleset *ruleset, struct prestera_port *port); u32 prestera_acl_ruleset_index_get(const struct prestera_acl_ruleset *ruleset); +void prestera_acl_ruleset_prio_get(struct prestera_acl_ruleset *ruleset, + u32 *prio_min, u32 *prio_max); void prestera_acl_rule_keymask_pcl_id_set(struct prestera_acl_rule *rule, u16 pcl_id); diff --git a/drivers/net/ethernet/marvell/prestera/prestera_ethtool.c b/drivers/net/ethernet/marvell/prestera/prestera_ethtool.c index 1da7ff889417..2f52daba58e6 100644 --- a/drivers/net/ethernet/marvell/prestera/prestera_ethtool.c +++ b/drivers/net/ethernet/marvell/prestera/prestera_ethtool.c @@ -300,8 +300,8 @@ static void prestera_ethtool_get_drvinfo(struct net_device *dev, struct prestera_port *port = netdev_priv(dev); struct prestera_switch *sw = port->sw; - strlcpy(drvinfo->driver, driver_kind, sizeof(drvinfo->driver)); - strlcpy(drvinfo->bus_info, dev_name(prestera_dev(sw)), + strscpy(drvinfo->driver, driver_kind, sizeof(drvinfo->driver)); + strscpy(drvinfo->bus_info, dev_name(prestera_dev(sw)), sizeof(drvinfo->bus_info)); snprintf(drvinfo->fw_version, sizeof(drvinfo->fw_version), "%d.%d.%d", diff --git a/drivers/net/ethernet/marvell/prestera/prestera_flow.c b/drivers/net/ethernet/marvell/prestera/prestera_flow.c index 2262693bd5cf..9f4267f326b0 100644 --- a/drivers/net/ethernet/marvell/prestera/prestera_flow.c +++ b/drivers/net/ethernet/marvell/prestera/prestera_flow.c @@ -7,8 +7,9 @@ #include "prestera.h" #include "prestera_acl.h" #include "prestera_flow.h" -#include "prestera_span.h" #include "prestera_flower.h" +#include "prestera_matchall.h" +#include "prestera_span.h" static LIST_HEAD(prestera_block_cb_list); @@ -17,9 +18,9 @@ static int prestera_flow_block_mall_cb(struct prestera_flow_block *block, { switch (f->command) { case TC_CLSMATCHALL_REPLACE: - return prestera_span_replace(block, f); + return prestera_mall_replace(block, f); case TC_CLSMATCHALL_DESTROY: - prestera_span_destroy(block); + prestera_mall_destroy(block); return 0; default: return -EOPNOTSUPP; @@ -89,6 +90,9 @@ prestera_flow_block_create(struct prestera_switch *sw, INIT_LIST_HEAD(&block->template_list); block->net = net; block->sw = sw; + block->mall.prio_min = UINT_MAX; + block->mall.prio_max = 0; + block->mall.bound = false; block->ingress = ingress; return block; @@ -263,7 +267,7 @@ static void prestera_setup_flow_block_unbind(struct prestera_port *port, block = flow_block_cb_priv(block_cb); - prestera_span_destroy(block); + prestera_mall_destroy(block); err = prestera_flow_block_unbind(block, port); if (err) diff --git a/drivers/net/ethernet/marvell/prestera/prestera_flow.h b/drivers/net/ethernet/marvell/prestera/prestera_flow.h index 0c9e13263261..a85a3eb40279 100644 --- a/drivers/net/ethernet/marvell/prestera/prestera_flow.h +++ b/drivers/net/ethernet/marvell/prestera/prestera_flow.h @@ -22,6 +22,11 @@ struct prestera_flow_block { struct prestera_acl_ruleset *ruleset_zero; struct flow_block_cb *block_cb; struct list_head template_list; + struct { + u32 prio_min; + u32 prio_max; + bool bound; + } mall; unsigned int rule_count; bool ingress; }; diff --git a/drivers/net/ethernet/marvell/prestera/prestera_flower.c b/drivers/net/ethernet/marvell/prestera/prestera_flower.c index 19d3b55c578e..91a478b75cbf 100644 --- a/drivers/net/ethernet/marvell/prestera/prestera_flower.c +++ b/drivers/net/ethernet/marvell/prestera/prestera_flower.c @@ -5,6 +5,7 @@ #include "prestera_acl.h" #include "prestera_flow.h" #include "prestera_flower.h" +#include "prestera_matchall.h" struct prestera_flower_template { struct prestera_acl_ruleset *ruleset; @@ -360,6 +361,49 @@ static int prestera_flower_parse(struct prestera_flow_block *block, f->common.extack); } +static int prestera_flower_prio_check(struct prestera_flow_block *block, + struct flow_cls_offload *f) +{ + u32 mall_prio_min; + u32 mall_prio_max; + int err; + + err = prestera_mall_prio_get(block, &mall_prio_min, &mall_prio_max); + if (err == -ENOENT) + /* No matchall filters installed on this chain. */ + return 0; + + if (err) { + NL_SET_ERR_MSG(f->common.extack, "Failed to get matchall priorities"); + return err; + } + + if (f->common.prio <= mall_prio_max && block->ingress) { + NL_SET_ERR_MSG(f->common.extack, + "Failed to add in front of existing matchall rules"); + return -EOPNOTSUPP; + } + if (f->common.prio >= mall_prio_min && !block->ingress) { + NL_SET_ERR_MSG(f->common.extack, "Failed to add behind of existing matchall rules"); + return -EOPNOTSUPP; + } + + return 0; +} + +int prestera_flower_prio_get(struct prestera_flow_block *block, u32 chain_index, + u32 *prio_min, u32 *prio_max) +{ + struct prestera_acl_ruleset *ruleset; + + ruleset = prestera_acl_ruleset_lookup(block->sw->acl, block, chain_index); + if (IS_ERR(ruleset)) + return PTR_ERR(ruleset); + + prestera_acl_ruleset_prio_get(ruleset, prio_min, prio_max); + return 0; +} + int prestera_flower_replace(struct prestera_flow_block *block, struct flow_cls_offload *f) { @@ -368,6 +412,10 @@ int prestera_flower_replace(struct prestera_flow_block *block, struct prestera_acl_rule *rule; int err; + err = prestera_flower_prio_check(block, f); + if (err) + return err; + ruleset = prestera_acl_ruleset_get(acl, block, f->common.chain_index); if (IS_ERR(ruleset)) return PTR_ERR(ruleset); @@ -452,7 +500,9 @@ int prestera_flower_tmplt_create(struct prestera_flow_block *block, } /* preserve keymask/template to this ruleset */ - prestera_acl_ruleset_keymask_set(ruleset, rule.re_key.match.mask); + err = prestera_acl_ruleset_keymask_set(ruleset, rule.re_key.match.mask); + if (err) + goto err_ruleset_keymask_set; /* skip error, as it is not possible to reject template operation, * so, keep the reference to the ruleset for rules to be added @@ -468,6 +518,8 @@ int prestera_flower_tmplt_create(struct prestera_flow_block *block, list_add_rcu(&template->list, &block->template_list); return 0; +err_ruleset_keymask_set: + prestera_acl_ruleset_put(ruleset); err_ruleset_get: kfree(template); err_malloc: diff --git a/drivers/net/ethernet/marvell/prestera/prestera_flower.h b/drivers/net/ethernet/marvell/prestera/prestera_flower.h index 495f151e6fa9..1181115fe6fa 100644 --- a/drivers/net/ethernet/marvell/prestera/prestera_flower.h +++ b/drivers/net/ethernet/marvell/prestera/prestera_flower.h @@ -19,5 +19,7 @@ int prestera_flower_tmplt_create(struct prestera_flow_block *block, void prestera_flower_tmplt_destroy(struct prestera_flow_block *block, struct flow_cls_offload *f); void prestera_flower_template_cleanup(struct prestera_flow_block *block); +int prestera_flower_prio_get(struct prestera_flow_block *block, u32 chain_index, + u32 *prio_min, u32 *prio_max); #endif /* _PRESTERA_FLOWER_H_ */ diff --git a/drivers/net/ethernet/marvell/prestera/prestera_hw.c b/drivers/net/ethernet/marvell/prestera/prestera_hw.c index 962d7e0c0cb5..fc6f7d2746e8 100644 --- a/drivers/net/ethernet/marvell/prestera/prestera_hw.c +++ b/drivers/net/ethernet/marvell/prestera/prestera_hw.c @@ -10,11 +10,14 @@ #include "prestera_hw.h" #include "prestera_acl.h" #include "prestera_counter.h" +#include "prestera_router_hw.h" #define PRESTERA_SWITCH_INIT_TIMEOUT_MS (30 * 1000) #define PRESTERA_MIN_MTU 64 +#define PRESTERA_MSG_CHUNK_SIZE 1024 + enum prestera_cmd_type_t { PRESTERA_CMD_TYPE_SWITCH_INIT = 0x1, PRESTERA_CMD_TYPE_SWITCH_ATTR_SET = 0x2, @@ -57,6 +60,10 @@ enum prestera_cmd_type_t { PRESTERA_CMD_TYPE_ROUTER_RIF_DELETE = 0x601, PRESTERA_CMD_TYPE_ROUTER_LPM_ADD = 0x610, PRESTERA_CMD_TYPE_ROUTER_LPM_DELETE = 0x611, + PRESTERA_CMD_TYPE_ROUTER_NH_GRP_SET = 0x622, + PRESTERA_CMD_TYPE_ROUTER_NH_GRP_BLK_GET = 0x645, + PRESTERA_CMD_TYPE_ROUTER_NH_GRP_ADD = 0x623, + PRESTERA_CMD_TYPE_ROUTER_NH_GRP_DELETE = 0x624, PRESTERA_CMD_TYPE_ROUTER_VR_CREATE = 0x630, PRESTERA_CMD_TYPE_ROUTER_VR_DELETE = 0x631, @@ -78,9 +85,11 @@ enum prestera_cmd_type_t { PRESTERA_CMD_TYPE_STP_PORT_SET = 0x1000, PRESTERA_CMD_TYPE_SPAN_GET = 0x1100, - PRESTERA_CMD_TYPE_SPAN_BIND = 0x1101, - PRESTERA_CMD_TYPE_SPAN_UNBIND = 0x1102, + PRESTERA_CMD_TYPE_SPAN_INGRESS_BIND = 0x1101, + PRESTERA_CMD_TYPE_SPAN_INGRESS_UNBIND = 0x1102, PRESTERA_CMD_TYPE_SPAN_RELEASE = 0x1103, + PRESTERA_CMD_TYPE_SPAN_EGRESS_BIND = 0x1104, + PRESTERA_CMD_TYPE_SPAN_EGRESS_UNBIND = 0x1105, PRESTERA_CMD_TYPE_POLICER_CREATE = 0x1500, PRESTERA_CMD_TYPE_POLICER_RELEASE = 0x1501, @@ -101,6 +110,7 @@ enum { PRESTERA_CMD_PORT_ATTR_LEARNING = 7, PRESTERA_CMD_PORT_ATTR_FLOOD = 8, PRESTERA_CMD_PORT_ATTR_CAPABILITY = 9, + PRESTERA_CMD_PORT_ATTR_LOCKED = 10, PRESTERA_CMD_PORT_ATTR_PHY_MODE = 12, PRESTERA_CMD_PORT_ATTR_TYPE = 13, PRESTERA_CMD_PORT_ATTR_STATS = 17, @@ -285,6 +295,7 @@ union prestera_msg_port_param { u8 duplex; u8 fec; u8 fc; + u8 br_locked; union { struct { u8 admin; @@ -538,6 +549,14 @@ struct prestera_msg_ip_addr { u8 __pad[3]; }; +struct prestera_msg_nh { + struct prestera_msg_iface oif; + __le32 hw_id; + u8 mac[ETH_ALEN]; + u8 is_active; + u8 pad; +}; + struct prestera_msg_rif_req { struct prestera_msg_cmd cmd; struct prestera_msg_iface iif; @@ -563,6 +582,34 @@ struct prestera_msg_lpm_req { u8 __pad[2]; }; +struct prestera_msg_nh_req { + struct prestera_msg_cmd cmd; + struct prestera_msg_nh nh[PRESTERA_NHGR_SIZE_MAX]; + __le32 size; + __le32 grp_id; +}; + +struct prestera_msg_nh_chunk_req { + struct prestera_msg_cmd cmd; + __le32 offset; +}; + +struct prestera_msg_nh_chunk_resp { + struct prestera_msg_ret ret; + u8 hw_state[PRESTERA_MSG_CHUNK_SIZE]; +}; + +struct prestera_msg_nh_grp_req { + struct prestera_msg_cmd cmd; + __le32 grp_id; + __le32 size; +}; + +struct prestera_msg_nh_grp_resp { + struct prestera_msg_ret ret; + __le32 grp_id; +}; + struct prestera_msg_vr_req { struct prestera_msg_cmd cmd; __le16 vr_id; @@ -725,11 +772,15 @@ static void prestera_hw_build_tests(void) BUILD_BUG_ON(sizeof(struct prestera_msg_flood_domain_ports_reset_req) != 8); BUILD_BUG_ON(sizeof(struct prestera_msg_mdb_create_req) != 16); BUILD_BUG_ON(sizeof(struct prestera_msg_mdb_destroy_req) != 16); + BUILD_BUG_ON(sizeof(struct prestera_msg_nh_req) != 124); + BUILD_BUG_ON(sizeof(struct prestera_msg_nh_chunk_req) != 8); + BUILD_BUG_ON(sizeof(struct prestera_msg_nh_grp_req) != 12); /* structure that are part of req/resp fw messages */ BUILD_BUG_ON(sizeof(struct prestera_msg_iface) != 16); BUILD_BUG_ON(sizeof(struct prestera_msg_ip_addr) != 20); BUILD_BUG_ON(sizeof(struct prestera_msg_flood_domain_port) != 12); + BUILD_BUG_ON(sizeof(struct prestera_msg_nh) != 28); /* check responses */ BUILD_BUG_ON(sizeof(struct prestera_msg_common_resp) != 8); @@ -745,6 +796,9 @@ static void prestera_hw_build_tests(void) BUILD_BUG_ON(sizeof(struct prestera_msg_rif_resp) != 12); BUILD_BUG_ON(sizeof(struct prestera_msg_vr_resp) != 12); BUILD_BUG_ON(sizeof(struct prestera_msg_policer_resp) != 12); + BUILD_BUG_ON(sizeof(struct prestera_msg_flood_domain_create_resp) != 12); + BUILD_BUG_ON(sizeof(struct prestera_msg_nh_chunk_resp) != 1032); + BUILD_BUG_ON(sizeof(struct prestera_msg_nh_grp_resp) != 12); /* check events */ BUILD_BUG_ON(sizeof(struct prestera_msg_event_port) != 20); @@ -1022,6 +1076,8 @@ int prestera_hw_switch_init(struct prestera_switch *sw) sw->id = resp.switch_id; sw->lag_member_max = resp.lag_member_max; sw->lag_max = resp.lag_max; + sw->size_tbl_router_nexthop = + __le32_to_cpu(resp.size_tbl_router_nexthop); return 0; } @@ -1431,27 +1487,39 @@ int prestera_hw_span_get(const struct prestera_port *port, u8 *span_id) return 0; } -int prestera_hw_span_bind(const struct prestera_port *port, u8 span_id) +int prestera_hw_span_bind(const struct prestera_port *port, u8 span_id, + bool ingress) { struct prestera_msg_span_req req = { .port = __cpu_to_le32(port->hw_id), .dev = __cpu_to_le32(port->dev_id), .id = span_id, }; + enum prestera_cmd_type_t cmd_type; + + if (ingress) + cmd_type = PRESTERA_CMD_TYPE_SPAN_INGRESS_BIND; + else + cmd_type = PRESTERA_CMD_TYPE_SPAN_EGRESS_BIND; + + return prestera_cmd(port->sw, cmd_type, &req.cmd, sizeof(req)); - return prestera_cmd(port->sw, PRESTERA_CMD_TYPE_SPAN_BIND, - &req.cmd, sizeof(req)); } -int prestera_hw_span_unbind(const struct prestera_port *port) +int prestera_hw_span_unbind(const struct prestera_port *port, bool ingress) { struct prestera_msg_span_req req = { .port = __cpu_to_le32(port->hw_id), .dev = __cpu_to_le32(port->dev_id), }; + enum prestera_cmd_type_t cmd_type; - return prestera_cmd(port->sw, PRESTERA_CMD_TYPE_SPAN_UNBIND, - &req.cmd, sizeof(req)); + if (ingress) + cmd_type = PRESTERA_CMD_TYPE_SPAN_INGRESS_UNBIND; + else + cmd_type = PRESTERA_CMD_TYPE_SPAN_EGRESS_UNBIND; + + return prestera_cmd(port->sw, cmd_type, &req.cmd, sizeof(req)); } int prestera_hw_span_release(struct prestera_switch *sw, u8 span_id) @@ -1639,6 +1707,22 @@ int prestera_hw_port_mc_flood_set(const struct prestera_port *port, bool flood) &req.cmd, sizeof(req)); } +int prestera_hw_port_br_locked_set(const struct prestera_port *port, + bool br_locked) +{ + struct prestera_msg_port_attr_req req = { + .attr = __cpu_to_le32(PRESTERA_CMD_PORT_ATTR_LOCKED), + .port = __cpu_to_le32(port->hw_id), + .dev = __cpu_to_le32(port->dev_id), + .param = { + .br_locked = br_locked, + } + }; + + return prestera_cmd(port->sw, PRESTERA_CMD_TYPE_PORT_ATTR_SET, + &req.cmd, sizeof(req)); +} + int prestera_hw_vlan_create(struct prestera_switch *sw, u16 vid) { struct prestera_msg_vlan_req req = { @@ -2004,6 +2088,85 @@ int prestera_hw_lpm_del(struct prestera_switch *sw, u16 vr_id, sizeof(req)); } +int prestera_hw_nh_entries_set(struct prestera_switch *sw, int count, + struct prestera_neigh_info *nhs, u32 grp_id) +{ + struct prestera_msg_nh_req req = { .size = __cpu_to_le32((u32)count), + .grp_id = __cpu_to_le32(grp_id) }; + int i, err; + + for (i = 0; i < count; i++) { + req.nh[i].is_active = nhs[i].connected; + memcpy(&req.nh[i].mac, nhs[i].ha, ETH_ALEN); + err = prestera_iface_to_msg(&nhs[i].iface, &req.nh[i].oif); + if (err) + return err; + } + + return prestera_cmd(sw, PRESTERA_CMD_TYPE_ROUTER_NH_GRP_SET, &req.cmd, + sizeof(req)); +} + +int prestera_hw_nhgrp_blk_get(struct prestera_switch *sw, + u8 *hw_state, u32 buf_size /* Buffer in bytes */) +{ + static struct prestera_msg_nh_chunk_resp resp; + struct prestera_msg_nh_chunk_req req; + u32 buf_offset; + int err; + + memset(&hw_state[0], 0, buf_size); + buf_offset = 0; + while (1) { + if (buf_offset >= buf_size) + break; + + memset(&req, 0, sizeof(req)); + req.offset = __cpu_to_le32(buf_offset * 8); /* 8 bits in u8 */ + err = prestera_cmd_ret(sw, + PRESTERA_CMD_TYPE_ROUTER_NH_GRP_BLK_GET, + &req.cmd, sizeof(req), &resp.ret, + sizeof(resp)); + if (err) + return err; + + memcpy(&hw_state[buf_offset], &resp.hw_state[0], + buf_offset + PRESTERA_MSG_CHUNK_SIZE > buf_size ? + buf_size - buf_offset : PRESTERA_MSG_CHUNK_SIZE); + buf_offset += PRESTERA_MSG_CHUNK_SIZE; + } + + return 0; +} + +int prestera_hw_nh_group_create(struct prestera_switch *sw, u16 nh_count, + u32 *grp_id) +{ + struct prestera_msg_nh_grp_req req = { .size = __cpu_to_le32((u32)nh_count) }; + struct prestera_msg_nh_grp_resp resp; + int err; + + err = prestera_cmd_ret(sw, PRESTERA_CMD_TYPE_ROUTER_NH_GRP_ADD, + &req.cmd, sizeof(req), &resp.ret, sizeof(resp)); + if (err) + return err; + + *grp_id = __le32_to_cpu(resp.grp_id); + return err; +} + +int prestera_hw_nh_group_delete(struct prestera_switch *sw, u16 nh_count, + u32 grp_id) +{ + struct prestera_msg_nh_grp_req req = { + .grp_id = __cpu_to_le32(grp_id), + .size = __cpu_to_le32(nh_count) + }; + + return prestera_cmd(sw, PRESTERA_CMD_TYPE_ROUTER_NH_GRP_DELETE, + &req.cmd, sizeof(req)); +} + int prestera_hw_rxtx_init(struct prestera_switch *sw, struct prestera_rxtx_params *params) { diff --git a/drivers/net/ethernet/marvell/prestera/prestera_hw.h b/drivers/net/ethernet/marvell/prestera/prestera_hw.h index 56e043146dd2..0a929279e1ce 100644 --- a/drivers/net/ethernet/marvell/prestera/prestera_hw.h +++ b/drivers/net/ethernet/marvell/prestera/prestera_hw.h @@ -146,6 +146,7 @@ struct prestera_counter_stats; struct prestera_iface; struct prestera_flood_domain; struct prestera_mdb_entry; +struct prestera_neigh_info; /* Switch API */ int prestera_hw_switch_init(struct prestera_switch *sw); @@ -183,6 +184,8 @@ int prestera_hw_port_speed_get(const struct prestera_port *port, u32 *speed); int prestera_hw_port_learning_set(struct prestera_port *port, bool enable); int prestera_hw_port_uc_flood_set(const struct prestera_port *port, bool flood); int prestera_hw_port_mc_flood_set(const struct prestera_port *port, bool flood); +int prestera_hw_port_br_locked_set(const struct prestera_port *port, + bool br_locked); int prestera_hw_port_accept_frm_type(struct prestera_port *port, enum prestera_accept_frm_type type); /* Vlan API */ @@ -243,8 +246,9 @@ int prestera_hw_counter_clear(struct prestera_switch *sw, u32 block_id, /* SPAN API */ int prestera_hw_span_get(const struct prestera_port *port, u8 *span_id); -int prestera_hw_span_bind(const struct prestera_port *port, u8 span_id); -int prestera_hw_span_unbind(const struct prestera_port *port); +int prestera_hw_span_bind(const struct prestera_port *port, u8 span_id, + bool ingress); +int prestera_hw_span_unbind(const struct prestera_port *port, bool ingress); int prestera_hw_span_release(struct prestera_switch *sw, u8 span_id); /* Router API */ @@ -263,6 +267,16 @@ int prestera_hw_lpm_add(struct prestera_switch *sw, u16 vr_id, int prestera_hw_lpm_del(struct prestera_switch *sw, u16 vr_id, __be32 dst, u32 dst_len); +/* NH API */ +int prestera_hw_nh_entries_set(struct prestera_switch *sw, int count, + struct prestera_neigh_info *nhs, u32 grp_id); +int prestera_hw_nhgrp_blk_get(struct prestera_switch *sw, + u8 *hw_state, u32 buf_size /* Buffer in bytes */); +int prestera_hw_nh_group_create(struct prestera_switch *sw, u16 nh_count, + u32 *grp_id); +int prestera_hw_nh_group_delete(struct prestera_switch *sw, u16 nh_count, + u32 grp_id); + /* Event handlers */ int prestera_hw_event_handler_register(struct prestera_switch *sw, enum prestera_event_type type, diff --git a/drivers/net/ethernet/marvell/prestera/prestera_main.c b/drivers/net/ethernet/marvell/prestera/prestera_main.c index a895862b4821..24f9d6024745 100644 --- a/drivers/net/ethernet/marvell/prestera/prestera_main.c +++ b/drivers/net/ethernet/marvell/prestera/prestera_main.c @@ -36,6 +36,17 @@ void prestera_queue_work(struct work_struct *work) queue_work(prestera_owq, work); } +void prestera_queue_delayed_work(struct delayed_work *work, unsigned long delay) +{ + queue_delayed_work(prestera_wq, work, delay); +} + +void prestera_queue_drain(void) +{ + drain_workqueue(prestera_wq); + drain_workqueue(prestera_owq); +} + int prestera_port_learning_set(struct prestera_port *port, bool learn) { return prestera_hw_port_learning_set(port, learn); @@ -51,6 +62,11 @@ int prestera_port_mc_flood_set(struct prestera_port *port, bool flood) return prestera_hw_port_mc_flood_set(port, flood); } +int prestera_port_br_locked_set(struct prestera_port *port, bool br_locked) +{ + return prestera_hw_port_br_locked_set(port, br_locked); +} + int prestera_port_pvid_set(struct prestera_port *port, u16 vid) { enum prestera_accept_frm_type frm_type; @@ -799,32 +815,30 @@ static void prestera_port_handle_event(struct prestera_switch *sw, caching_dw = &port->cached_hw_stats.caching_dw; - if (port->phy_link) { - memset(&smac, 0, sizeof(smac)); - smac.valid = true; - smac.oper = pevt->data.mac.oper; - if (smac.oper) { - smac.mode = pevt->data.mac.mode; - smac.speed = pevt->data.mac.speed; - smac.duplex = pevt->data.mac.duplex; - smac.fc = pevt->data.mac.fc; - smac.fec = pevt->data.mac.fec; - phylink_mac_change(port->phy_link, true); - } else { - phylink_mac_change(port->phy_link, false); - } - prestera_port_mac_state_cache_write(port, &smac); + memset(&smac, 0, sizeof(smac)); + smac.valid = true; + smac.oper = pevt->data.mac.oper; + if (smac.oper) { + smac.mode = pevt->data.mac.mode; + smac.speed = pevt->data.mac.speed; + smac.duplex = pevt->data.mac.duplex; + smac.fc = pevt->data.mac.fc; + smac.fec = pevt->data.mac.fec; } + prestera_port_mac_state_cache_write(port, &smac); if (port->state_mac.oper) { - if (!port->phy_link) + if (port->phy_link) + phylink_mac_change(port->phy_link, true); + else netif_carrier_on(port->dev); if (!delayed_work_pending(caching_dw)) queue_delayed_work(prestera_wq, caching_dw, 0); - } else if (netif_running(port->dev) && - netif_carrier_ok(port->dev)) { - if (!port->phy_link) + } else { + if (port->phy_link) + phylink_mac_change(port->phy_link, false); + else if (netif_running(port->dev) && netif_carrier_ok(port->dev)) netif_carrier_off(port->dev); if (delayed_work_pending(caching_dw)) diff --git a/drivers/net/ethernet/marvell/prestera/prestera_matchall.c b/drivers/net/ethernet/marvell/prestera/prestera_matchall.c new file mode 100644 index 000000000000..6f2b95a5263e --- /dev/null +++ b/drivers/net/ethernet/marvell/prestera/prestera_matchall.c @@ -0,0 +1,125 @@ +// SPDX-License-Identifier: BSD-3-Clause OR GPL-2.0 +/* Copyright (c) 2019-2022 Marvell International Ltd. All rights reserved */ + +#include <linux/kernel.h> +#include <linux/list.h> + +#include "prestera.h" +#include "prestera_hw.h" +#include "prestera_flow.h" +#include "prestera_flower.h" +#include "prestera_matchall.h" +#include "prestera_span.h" + +static int prestera_mall_prio_check(struct prestera_flow_block *block, + struct tc_cls_matchall_offload *f) +{ + u32 flower_prio_min; + u32 flower_prio_max; + int err; + + err = prestera_flower_prio_get(block, f->common.chain_index, + &flower_prio_min, &flower_prio_max); + if (err == -ENOENT) + /* No flower filters installed on this chain. */ + return 0; + + if (err) { + NL_SET_ERR_MSG(f->common.extack, "Failed to get flower priorities"); + return err; + } + + if (f->common.prio <= flower_prio_max && !block->ingress) { + NL_SET_ERR_MSG(f->common.extack, "Failed to add in front of existing flower rules"); + return -EOPNOTSUPP; + } + if (f->common.prio >= flower_prio_min && block->ingress) { + NL_SET_ERR_MSG(f->common.extack, "Failed to add behind of existing flower rules"); + return -EOPNOTSUPP; + } + + return 0; +} + +int prestera_mall_prio_get(struct prestera_flow_block *block, + u32 *prio_min, u32 *prio_max) +{ + if (!block->mall.bound) + return -ENOENT; + + *prio_min = block->mall.prio_min; + *prio_max = block->mall.prio_max; + return 0; +} + +static void prestera_mall_prio_update(struct prestera_flow_block *block, + struct tc_cls_matchall_offload *f) +{ + block->mall.prio_min = min(block->mall.prio_min, f->common.prio); + block->mall.prio_max = max(block->mall.prio_max, f->common.prio); +} + +int prestera_mall_replace(struct prestera_flow_block *block, + struct tc_cls_matchall_offload *f) +{ + struct prestera_flow_block_binding *binding; + __be16 protocol = f->common.protocol; + struct flow_action_entry *act; + struct prestera_port *port; + int err; + + if (!flow_offload_has_one_action(&f->rule->action)) { + NL_SET_ERR_MSG(f->common.extack, + "Only singular actions are supported"); + return -EOPNOTSUPP; + } + + act = &f->rule->action.entries[0]; + + if (!prestera_netdev_check(act->dev)) { + NL_SET_ERR_MSG(f->common.extack, + "Only Marvell Prestera port is supported"); + return -EINVAL; + } + if (!tc_cls_can_offload_and_chain0(act->dev, &f->common)) + return -EOPNOTSUPP; + if (act->id != FLOW_ACTION_MIRRED) + return -EOPNOTSUPP; + if (protocol != htons(ETH_P_ALL)) + return -EOPNOTSUPP; + + err = prestera_mall_prio_check(block, f); + if (err) + return err; + + port = netdev_priv(act->dev); + + list_for_each_entry(binding, &block->binding_list, list) { + err = prestera_span_rule_add(binding, port, block->ingress); + if (err) + goto rollback; + } + + prestera_mall_prio_update(block, f); + + block->mall.bound = true; + return 0; + +rollback: + list_for_each_entry_continue_reverse(binding, + &block->binding_list, list) + prestera_span_rule_del(binding, block->ingress); + return err; +} + +void prestera_mall_destroy(struct prestera_flow_block *block) +{ + struct prestera_flow_block_binding *binding; + + list_for_each_entry(binding, &block->binding_list, list) + prestera_span_rule_del(binding, block->ingress); + + block->mall.prio_min = UINT_MAX; + block->mall.prio_max = 0; + block->mall.bound = false; +} diff --git a/drivers/net/ethernet/marvell/prestera/prestera_matchall.h b/drivers/net/ethernet/marvell/prestera/prestera_matchall.h new file mode 100644 index 000000000000..fed08be80257 --- /dev/null +++ b/drivers/net/ethernet/marvell/prestera/prestera_matchall.h @@ -0,0 +1,17 @@ +/* SPDX-License-Identifier: BSD-3-Clause OR GPL-2.0 */ +/* Copyright (c) 2022 Marvell International Ltd. All rights reserved. */ + +#ifndef _PRESTERA_MATCHALL_H_ +#define _PRESTERA_MATCHALL_H_ + +#include <net/pkt_cls.h> + +struct prestera_flow_block; + +int prestera_mall_replace(struct prestera_flow_block *block, + struct tc_cls_matchall_offload *f); +void prestera_mall_destroy(struct prestera_flow_block *block); +int prestera_mall_prio_get(struct prestera_flow_block *block, + u32 *prio_min, u32 *prio_max); + +#endif /* _PRESTERA_MATCHALL_H_ */ diff --git a/drivers/net/ethernet/marvell/prestera/prestera_router.c b/drivers/net/ethernet/marvell/prestera/prestera_router.c index 58f4e44d5ad7..4046be0e86ff 100644 --- a/drivers/net/ethernet/marvell/prestera/prestera_router.c +++ b/drivers/net/ethernet/marvell/prestera/prestera_router.c @@ -7,10 +7,35 @@ #include <net/inet_dscp.h> #include <net/switchdev.h> #include <linux/rhashtable.h> +#include <net/nexthop.h> +#include <net/arp.h> +#include <linux/if_vlan.h> +#include <linux/if_macvlan.h> +#include <net/netevent.h> #include "prestera.h" #include "prestera_router_hw.h" +#define PRESTERA_IMPLICITY_RESOLVE_DEAD_NEIGH +#define PRESTERA_NH_PROBE_INTERVAL 5000 /* ms */ + +struct prestera_kern_neigh_cache_key { + struct prestera_ip_addr addr; + struct net_device *dev; +}; + +struct prestera_kern_neigh_cache { + struct prestera_kern_neigh_cache_key key; + struct rhash_head ht_node; + struct list_head kern_fib_cache_list; + /* Hold prepared nh_neigh info if is in_kernel */ + struct prestera_neigh_info nh_neigh_info; + /* Indicate if neighbour is reachable by direct route */ + bool reachable; + /* Lock cache if neigh is present in kernel */ + bool in_kernel; +}; + struct prestera_kern_fib_cache_key { struct prestera_ip_addr addr; u32 prefix_len; @@ -23,15 +48,29 @@ struct prestera_kern_fib_cache { struct { struct prestera_fib_key fib_key; enum prestera_fib_type fib_type; + struct prestera_nexthop_group_key nh_grp_key; } lpm_info; /* hold prepared lpm info */ /* Indicate if route is not overlapped by another table */ struct rhash_head ht_node; /* node of prestera_router */ - struct fib_info *fi; - dscp_t kern_dscp; - u8 kern_type; + struct prestera_kern_neigh_cache_head { + struct prestera_kern_fib_cache *this; + struct list_head head; + struct prestera_kern_neigh_cache *n_cache; + } kern_neigh_cache_head[PRESTERA_NHGR_SIZE_MAX]; + union { + struct fib_notifier_info info; /* point to any of 4/6 */ + struct fib_entry_notifier_info fen4_info; + }; bool reachable; }; +static const struct rhashtable_params __prestera_kern_neigh_cache_ht_params = { + .key_offset = offsetof(struct prestera_kern_neigh_cache, key), + .head_offset = offsetof(struct prestera_kern_neigh_cache, ht_node), + .key_len = sizeof(struct prestera_kern_neigh_cache_key), + .automatic_shrinking = true, +}; + static const struct rhashtable_params __prestera_kern_fib_cache_ht_params = { .key_offset = offsetof(struct prestera_kern_fib_cache, key), .head_offset = offsetof(struct prestera_kern_fib_cache, ht_node), @@ -51,15 +90,450 @@ static u32 prestera_fix_tb_id(u32 tb_id) } static void -prestera_util_fen_info2fib_cache_key(struct fib_entry_notifier_info *fen_info, +prestera_util_fen_info2fib_cache_key(struct fib_notifier_info *info, struct prestera_kern_fib_cache_key *key) { + struct fib_entry_notifier_info *fen_info = + container_of(info, struct fib_entry_notifier_info, info); + memset(key, 0, sizeof(*key)); + key->addr.v = PRESTERA_IPV4; key->addr.u.ipv4 = cpu_to_be32(fen_info->dst); key->prefix_len = fen_info->dst_len; key->kern_tb_id = fen_info->tb_id; } +static int prestera_util_nhc2nc_key(struct prestera_switch *sw, + struct fib_nh_common *nhc, + struct prestera_kern_neigh_cache_key *nk) +{ + memset(nk, 0, sizeof(*nk)); + if (nhc->nhc_gw_family == AF_INET) { + nk->addr.v = PRESTERA_IPV4; + nk->addr.u.ipv4 = nhc->nhc_gw.ipv4; + } else { + nk->addr.v = PRESTERA_IPV6; + nk->addr.u.ipv6 = nhc->nhc_gw.ipv6; + } + + nk->dev = nhc->nhc_dev; + return 0; +} + +static void +prestera_util_nc_key2nh_key(struct prestera_kern_neigh_cache_key *ck, + struct prestera_nh_neigh_key *nk) +{ + memset(nk, 0, sizeof(*nk)); + nk->addr = ck->addr; + nk->rif = (void *)ck->dev; +} + +static bool +prestera_util_nhc_eq_n_cache_key(struct prestera_switch *sw, + struct fib_nh_common *nhc, + struct prestera_kern_neigh_cache_key *nk) +{ + struct prestera_kern_neigh_cache_key tk; + int err; + + err = prestera_util_nhc2nc_key(sw, nhc, &tk); + if (err) + return false; + + if (memcmp(&tk, nk, sizeof(tk))) + return false; + + return true; +} + +static int +prestera_util_neigh2nc_key(struct prestera_switch *sw, struct neighbour *n, + struct prestera_kern_neigh_cache_key *key) +{ + memset(key, 0, sizeof(*key)); + if (n->tbl->family == AF_INET) { + key->addr.v = PRESTERA_IPV4; + key->addr.u.ipv4 = *(__be32 *)n->primary_key; + } else { + return -ENOENT; + } + + key->dev = n->dev; + + return 0; +} + +static bool __prestera_fi_is_direct(struct fib_info *fi) +{ + struct fib_nh *fib_nh; + + if (fib_info_num_path(fi) == 1) { + fib_nh = fib_info_nh(fi, 0); + if (fib_nh->fib_nh_gw_family == AF_UNSPEC) + return true; + } + + return false; +} + +static bool prestera_fi_is_direct(struct fib_info *fi) +{ + if (fi->fib_type != RTN_UNICAST) + return false; + + return __prestera_fi_is_direct(fi); +} + +static bool prestera_fi_is_nh(struct fib_info *fi) +{ + if (fi->fib_type != RTN_UNICAST) + return false; + + return !__prestera_fi_is_direct(fi); +} + +static bool __prestera_fi6_is_direct(struct fib6_info *fi) +{ + if (!fi->fib6_nh->nh_common.nhc_gw_family) + return true; + + return false; +} + +static bool prestera_fi6_is_direct(struct fib6_info *fi) +{ + if (fi->fib6_type != RTN_UNICAST) + return false; + + return __prestera_fi6_is_direct(fi); +} + +static bool prestera_fi6_is_nh(struct fib6_info *fi) +{ + if (fi->fib6_type != RTN_UNICAST) + return false; + + return !__prestera_fi6_is_direct(fi); +} + +static bool prestera_fib_info_is_direct(struct fib_notifier_info *info) +{ + struct fib6_entry_notifier_info *fen6_info = + container_of(info, struct fib6_entry_notifier_info, info); + struct fib_entry_notifier_info *fen_info = + container_of(info, struct fib_entry_notifier_info, info); + + if (info->family == AF_INET) + return prestera_fi_is_direct(fen_info->fi); + else + return prestera_fi6_is_direct(fen6_info->rt); +} + +static bool prestera_fib_info_is_nh(struct fib_notifier_info *info) +{ + struct fib6_entry_notifier_info *fen6_info = + container_of(info, struct fib6_entry_notifier_info, info); + struct fib_entry_notifier_info *fen_info = + container_of(info, struct fib_entry_notifier_info, info); + + if (info->family == AF_INET) + return prestera_fi_is_nh(fen_info->fi); + else + return prestera_fi6_is_nh(fen6_info->rt); +} + +/* must be called with rcu_read_lock() */ +static int prestera_util_kern_get_route(struct fib_result *res, u32 tb_id, + __be32 *addr) +{ + struct flowi4 fl4; + + /* TODO: walkthrough appropriate tables in kernel + * to know if the same prefix exists in several tables + */ + memset(&fl4, 0, sizeof(fl4)); + fl4.daddr = *addr; + return fib_lookup(&init_net, &fl4, res, 0 /* FIB_LOOKUP_NOREF */); +} + +static bool +__prestera_util_kern_n_is_reachable_v4(u32 tb_id, __be32 *addr, + struct net_device *dev) +{ + struct fib_nh *fib_nh; + struct fib_result res; + bool reachable; + + reachable = false; + + if (!prestera_util_kern_get_route(&res, tb_id, addr)) + if (prestera_fi_is_direct(res.fi)) { + fib_nh = fib_info_nh(res.fi, 0); + if (dev == fib_nh->fib_nh_dev) + reachable = true; + } + + return reachable; +} + +/* Check if neigh route is reachable */ +static bool +prestera_util_kern_n_is_reachable(u32 tb_id, + struct prestera_ip_addr *addr, + struct net_device *dev) +{ + if (addr->v == PRESTERA_IPV4) + return __prestera_util_kern_n_is_reachable_v4(tb_id, + &addr->u.ipv4, + dev); + else + return false; +} + +static void prestera_util_kern_set_neigh_offload(struct neighbour *n, + bool offloaded) +{ + if (offloaded) + n->flags |= NTF_OFFLOADED; + else + n->flags &= ~NTF_OFFLOADED; +} + +static void +prestera_util_kern_set_nh_offload(struct fib_nh_common *nhc, bool offloaded, bool trap) +{ + if (offloaded) + nhc->nhc_flags |= RTNH_F_OFFLOAD; + else + nhc->nhc_flags &= ~RTNH_F_OFFLOAD; + + if (trap) + nhc->nhc_flags |= RTNH_F_TRAP; + else + nhc->nhc_flags &= ~RTNH_F_TRAP; +} + +static struct fib_nh_common * +prestera_kern_fib_info_nhc(struct fib_notifier_info *info, int n) +{ + struct fib6_entry_notifier_info *fen6_info; + struct fib_entry_notifier_info *fen4_info; + struct fib6_info *iter; + + if (info->family == AF_INET) { + fen4_info = container_of(info, struct fib_entry_notifier_info, + info); + return &fib_info_nh(fen4_info->fi, n)->nh_common; + } else if (info->family == AF_INET6) { + fen6_info = container_of(info, struct fib6_entry_notifier_info, + info); + if (!n) + return &fen6_info->rt->fib6_nh->nh_common; + + list_for_each_entry(iter, &fen6_info->rt->fib6_siblings, + fib6_siblings) { + if (!--n) + return &iter->fib6_nh->nh_common; + } + } + + /* if family is incorrect - than upper functions has BUG */ + /* if doesn't find requested index - there is alsi bug, because + * valid index must be produced by nhs, which checks list length + */ + WARN(1, "Invalid parameters passed to %s n=%d i=%p", + __func__, n, info); + return NULL; +} + +static int prestera_kern_fib_info_nhs(struct fib_notifier_info *info) +{ + struct fib6_entry_notifier_info *fen6_info; + struct fib_entry_notifier_info *fen4_info; + + if (info->family == AF_INET) { + fen4_info = container_of(info, struct fib_entry_notifier_info, + info); + return fib_info_num_path(fen4_info->fi); + } else if (info->family == AF_INET6) { + fen6_info = container_of(info, struct fib6_entry_notifier_info, + info); + return fen6_info->rt->fib6_nsiblings + 1; + } + + return 0; +} + +static unsigned char +prestera_kern_fib_info_type(struct fib_notifier_info *info) +{ + struct fib6_entry_notifier_info *fen6_info; + struct fib_entry_notifier_info *fen4_info; + + if (info->family == AF_INET) { + fen4_info = container_of(info, struct fib_entry_notifier_info, + info); + return fen4_info->fi->fib_type; + } else if (info->family == AF_INET6) { + fen6_info = container_of(info, struct fib6_entry_notifier_info, + info); + /* TODO: ECMP in ipv6 is several routes. + * Every route has single nh. + */ + return fen6_info->rt->fib6_type; + } + + return RTN_UNSPEC; +} + +/* Decided, that uc_nh route with key==nh is obviously neighbour route */ +static bool +prestera_fib_node_util_is_neighbour(struct prestera_fib_node *fib_node) +{ + if (fib_node->info.type != PRESTERA_FIB_TYPE_UC_NH) + return false; + + if (fib_node->info.nh_grp->nh_neigh_head[1].neigh) + return false; + + if (!fib_node->info.nh_grp->nh_neigh_head[0].neigh) + return false; + + if (memcmp(&fib_node->info.nh_grp->nh_neigh_head[0].neigh->key.addr, + &fib_node->key.addr, sizeof(struct prestera_ip_addr))) + return false; + + return true; +} + +static int prestera_dev_if_type(const struct net_device *dev) +{ + struct macvlan_dev *vlan; + + if (is_vlan_dev(dev) && + netif_is_bridge_master(vlan_dev_real_dev(dev))) { + return PRESTERA_IF_VID_E; + } else if (netif_is_bridge_master(dev)) { + return PRESTERA_IF_VID_E; + } else if (netif_is_lag_master(dev)) { + return PRESTERA_IF_LAG_E; + } else if (netif_is_macvlan(dev)) { + vlan = netdev_priv(dev); + return prestera_dev_if_type(vlan->lowerdev); + } else { + return PRESTERA_IF_PORT_E; + } +} + +static int +prestera_neigh_iface_init(struct prestera_switch *sw, + struct prestera_iface *iface, + struct neighbour *n) +{ + struct prestera_port *port; + + iface->vlan_id = 0; /* TODO: vlan egress */ + iface->type = prestera_dev_if_type(n->dev); + if (iface->type != PRESTERA_IF_PORT_E) + return -EINVAL; + + if (!prestera_netdev_check(n->dev)) + return -EINVAL; + + port = netdev_priv(n->dev); + iface->dev_port.hw_dev_num = port->dev_id; + iface->dev_port.port_num = port->hw_id; + + return 0; +} + +static struct prestera_kern_neigh_cache * +prestera_kern_neigh_cache_find(struct prestera_switch *sw, + struct prestera_kern_neigh_cache_key *key) +{ + struct prestera_kern_neigh_cache *n_cache; + + n_cache = + rhashtable_lookup_fast(&sw->router->kern_neigh_cache_ht, key, + __prestera_kern_neigh_cache_ht_params); + return IS_ERR(n_cache) ? NULL : n_cache; +} + +static void +__prestera_kern_neigh_cache_destruct(struct prestera_switch *sw, + struct prestera_kern_neigh_cache *n_cache) +{ + dev_put(n_cache->key.dev); +} + +static void +__prestera_kern_neigh_cache_destroy(struct prestera_switch *sw, + struct prestera_kern_neigh_cache *n_cache) +{ + rhashtable_remove_fast(&sw->router->kern_neigh_cache_ht, + &n_cache->ht_node, + __prestera_kern_neigh_cache_ht_params); + __prestera_kern_neigh_cache_destruct(sw, n_cache); + kfree(n_cache); +} + +static struct prestera_kern_neigh_cache * +__prestera_kern_neigh_cache_create(struct prestera_switch *sw, + struct prestera_kern_neigh_cache_key *key) +{ + struct prestera_kern_neigh_cache *n_cache; + int err; + + n_cache = kzalloc(sizeof(*n_cache), GFP_KERNEL); + if (!n_cache) + goto err_kzalloc; + + memcpy(&n_cache->key, key, sizeof(*key)); + dev_hold(n_cache->key.dev); + + INIT_LIST_HEAD(&n_cache->kern_fib_cache_list); + err = rhashtable_insert_fast(&sw->router->kern_neigh_cache_ht, + &n_cache->ht_node, + __prestera_kern_neigh_cache_ht_params); + if (err) + goto err_ht_insert; + + return n_cache; + +err_ht_insert: + dev_put(n_cache->key.dev); + kfree(n_cache); +err_kzalloc: + return NULL; +} + +static struct prestera_kern_neigh_cache * +prestera_kern_neigh_cache_get(struct prestera_switch *sw, + struct prestera_kern_neigh_cache_key *key) +{ + struct prestera_kern_neigh_cache *n_cache; + + n_cache = prestera_kern_neigh_cache_find(sw, key); + if (!n_cache) + n_cache = __prestera_kern_neigh_cache_create(sw, key); + + return n_cache; +} + +static struct prestera_kern_neigh_cache * +prestera_kern_neigh_cache_put(struct prestera_switch *sw, + struct prestera_kern_neigh_cache *n_cache) +{ + if (!n_cache->in_kernel && + list_empty(&n_cache->kern_fib_cache_list)) { + __prestera_kern_neigh_cache_destroy(sw, n_cache); + return NULL; + } + + return n_cache; +} + static struct prestera_kern_fib_cache * prestera_kern_fib_cache_find(struct prestera_switch *sw, struct prestera_kern_fib_cache_key *key) @@ -73,24 +547,79 @@ prestera_kern_fib_cache_find(struct prestera_switch *sw, } static void +__prestera_kern_fib_cache_destruct(struct prestera_switch *sw, + struct prestera_kern_fib_cache *fib_cache) +{ + struct prestera_kern_neigh_cache *n_cache; + int i; + + for (i = 0; i < PRESTERA_NHGR_SIZE_MAX; i++) { + n_cache = fib_cache->kern_neigh_cache_head[i].n_cache; + if (n_cache) { + list_del(&fib_cache->kern_neigh_cache_head[i].head); + prestera_kern_neigh_cache_put(sw, n_cache); + } + } + + fib_info_put(fib_cache->fen4_info.fi); +} + +static void prestera_kern_fib_cache_destroy(struct prestera_switch *sw, struct prestera_kern_fib_cache *fib_cache) { - fib_info_put(fib_cache->fi); rhashtable_remove_fast(&sw->router->kern_fib_cache_ht, &fib_cache->ht_node, __prestera_kern_fib_cache_ht_params); + __prestera_kern_fib_cache_destruct(sw, fib_cache); kfree(fib_cache); } +static int +__prestera_kern_fib_cache_create_nhs(struct prestera_switch *sw, + struct prestera_kern_fib_cache *fc) +{ + struct prestera_kern_neigh_cache_key nc_key; + struct prestera_kern_neigh_cache *n_cache; + struct fib_nh_common *nhc; + int i, nhs, err; + + if (!prestera_fib_info_is_nh(&fc->info)) + return 0; + + nhs = prestera_kern_fib_info_nhs(&fc->info); + if (nhs > PRESTERA_NHGR_SIZE_MAX) + return 0; + + for (i = 0; i < nhs; i++) { + nhc = prestera_kern_fib_info_nhc(&fc->fen4_info.info, i); + err = prestera_util_nhc2nc_key(sw, nhc, &nc_key); + if (err) + return 0; + + n_cache = prestera_kern_neigh_cache_get(sw, &nc_key); + if (!n_cache) + return 0; + + fc->kern_neigh_cache_head[i].this = fc; + fc->kern_neigh_cache_head[i].n_cache = n_cache; + list_add(&fc->kern_neigh_cache_head[i].head, + &n_cache->kern_fib_cache_list); + } + + return 0; +} + /* Operations on fi (offload, etc) must be wrapped in utils. * This function just create storage. */ static struct prestera_kern_fib_cache * prestera_kern_fib_cache_create(struct prestera_switch *sw, struct prestera_kern_fib_cache_key *key, - struct fib_info *fi, dscp_t dscp, u8 type) + struct fib_notifier_info *info) { + struct fib_entry_notifier_info *fen_info = + container_of(info, struct fib_entry_notifier_info, info); struct prestera_kern_fib_cache *fib_cache; int err; @@ -99,10 +628,8 @@ prestera_kern_fib_cache_create(struct prestera_switch *sw, goto err_kzalloc; memcpy(&fib_cache->key, key, sizeof(*key)); - fib_info_hold(fi); - fib_cache->fi = fi; - fib_cache->kern_dscp = dscp; - fib_cache->kern_type = type; + fib_info_hold(fen_info->fi); + memcpy(&fib_cache->fen4_info, fen_info, sizeof(*fen_info)); err = rhashtable_insert_fast(&sw->router->kern_fib_cache_ht, &fib_cache->ht_node, @@ -110,48 +637,270 @@ prestera_kern_fib_cache_create(struct prestera_switch *sw, if (err) goto err_ht_insert; + /* Handle nexthops */ + err = __prestera_kern_fib_cache_create_nhs(sw, fib_cache); + if (err) + goto out; /* Not critical */ + +out: return fib_cache; err_ht_insert: - fib_info_put(fi); + fib_info_put(fen_info->fi); kfree(fib_cache); err_kzalloc: return NULL; } static void +__prestera_k_arb_fib_nh_offload_set(struct prestera_switch *sw, + struct prestera_kern_fib_cache *fibc, + struct prestera_kern_neigh_cache *nc, + bool offloaded, bool trap) +{ + struct fib_nh_common *nhc; + int i, nhs; + + nhs = prestera_kern_fib_info_nhs(&fibc->info); + for (i = 0; i < nhs; i++) { + nhc = prestera_kern_fib_info_nhc(&fibc->info, i); + if (!nc) { + prestera_util_kern_set_nh_offload(nhc, offloaded, trap); + continue; + } + + if (prestera_util_nhc_eq_n_cache_key(sw, nhc, &nc->key)) { + prestera_util_kern_set_nh_offload(nhc, offloaded, trap); + break; + } + } +} + +static void +__prestera_k_arb_n_offload_set(struct prestera_switch *sw, + struct prestera_kern_neigh_cache *nc, + bool offloaded) +{ + struct neighbour *n; + + n = neigh_lookup(&arp_tbl, &nc->key.addr.u.ipv4, + nc->key.dev); + if (!n) + return; + + prestera_util_kern_set_neigh_offload(n, offloaded); + neigh_release(n); +} + +static void __prestera_k_arb_fib_lpm_offload_set(struct prestera_switch *sw, struct prestera_kern_fib_cache *fc, bool fail, bool offload, bool trap) { struct fib_rt_info fri; - if (fc->key.addr.v != PRESTERA_IPV4) + switch (fc->key.addr.v) { + case PRESTERA_IPV4: + fri.fi = fc->fen4_info.fi; + fri.tb_id = fc->key.kern_tb_id; + fri.dst = fc->key.addr.u.ipv4; + fri.dst_len = fc->key.prefix_len; + fri.dscp = fc->fen4_info.dscp; + fri.type = fc->fen4_info.type; + /* flags begin */ + fri.offload = offload; + fri.trap = trap; + fri.offload_failed = fail; + /* flags end */ + fib_alias_hw_flags_set(&init_net, &fri); + return; + case PRESTERA_IPV6: + /* TODO */ return; + } +} + +static void +__prestera_k_arb_n_lpm_set(struct prestera_switch *sw, + struct prestera_kern_neigh_cache *n_cache, + bool enabled) +{ + struct prestera_nexthop_group_key nh_grp_key; + struct prestera_kern_fib_cache_key fc_key; + struct prestera_kern_fib_cache *fib_cache; + struct prestera_fib_node *fib_node; + struct prestera_fib_key fib_key; + + /* Exception for fc with prefix 32: LPM entry is already used by fib */ + memset(&fc_key, 0, sizeof(fc_key)); + fc_key.addr = n_cache->key.addr; + fc_key.prefix_len = PRESTERA_IP_ADDR_PLEN(n_cache->key.addr.v); + /* But better to use tb_id of route, which pointed to this neighbour. */ + /* We take it from rif, because rif inconsistent. + * Must be separated in_rif and out_rif. + * Also note: for each fib pointed to this neigh should be separated + * neigh lpm entry (for each ingress vr) + */ + fc_key.kern_tb_id = l3mdev_fib_table(n_cache->key.dev); + fib_cache = prestera_kern_fib_cache_find(sw, &fc_key); + memset(&fib_key, 0, sizeof(fib_key)); + fib_key.addr = n_cache->key.addr; + fib_key.prefix_len = PRESTERA_IP_ADDR_PLEN(n_cache->key.addr.v); + fib_key.tb_id = prestera_fix_tb_id(fc_key.kern_tb_id); + fib_node = prestera_fib_node_find(sw, &fib_key); + if (!fib_cache || !fib_cache->reachable) { + if (!enabled && fib_node) { + if (prestera_fib_node_util_is_neighbour(fib_node)) + prestera_fib_node_destroy(sw, fib_node); + return; + } + } + + if (enabled && !fib_node) { + memset(&nh_grp_key, 0, sizeof(nh_grp_key)); + prestera_util_nc_key2nh_key(&n_cache->key, + &nh_grp_key.neigh[0]); + fib_node = prestera_fib_node_create(sw, &fib_key, + PRESTERA_FIB_TYPE_UC_NH, + &nh_grp_key); + if (!fib_node) + pr_err("%s failed ip=%pI4n", "prestera_fib_node_create", + &fib_key.addr.u.ipv4); + return; + } +} + +static void +__prestera_k_arb_nc_kern_fib_fetch(struct prestera_switch *sw, + struct prestera_kern_neigh_cache *nc) +{ + if (prestera_util_kern_n_is_reachable(l3mdev_fib_table(nc->key.dev), + &nc->key.addr, nc->key.dev)) + nc->reachable = true; + else + nc->reachable = false; +} + +/* Kernel neighbour -> neigh_cache info */ +static void +__prestera_k_arb_nc_kern_n_fetch(struct prestera_switch *sw, + struct prestera_kern_neigh_cache *nc) +{ + struct neighbour *n; + int err; + + memset(&nc->nh_neigh_info, 0, sizeof(nc->nh_neigh_info)); + n = neigh_lookup(&arp_tbl, &nc->key.addr.u.ipv4, nc->key.dev); + if (!n) + goto out; + + read_lock_bh(&n->lock); + if (n->nud_state & NUD_VALID && !n->dead) { + err = prestera_neigh_iface_init(sw, &nc->nh_neigh_info.iface, + n); + if (err) + goto n_read_out; - fri.fi = fc->fi; - fri.tb_id = fc->key.kern_tb_id; - fri.dst = fc->key.addr.u.ipv4; - fri.dst_len = fc->key.prefix_len; - fri.dscp = fc->kern_dscp; - fri.type = fc->kern_type; - /* flags begin */ - fri.offload = offload; - fri.trap = trap; - fri.offload_failed = fail; - /* flags end */ - fib_alias_hw_flags_set(&init_net, &fri); + memcpy(&nc->nh_neigh_info.ha[0], &n->ha[0], ETH_ALEN); + nc->nh_neigh_info.connected = true; + } +n_read_out: + read_unlock_bh(&n->lock); +out: + nc->in_kernel = nc->nh_neigh_info.connected; + if (n) + neigh_release(n); +} + +/* neigh_cache info -> lpm update */ +static void +__prestera_k_arb_nc_apply(struct prestera_switch *sw, + struct prestera_kern_neigh_cache *nc) +{ + struct prestera_kern_neigh_cache_head *nhead; + struct prestera_nh_neigh_key nh_key; + struct prestera_nh_neigh *nh_neigh; + int err; + + __prestera_k_arb_n_lpm_set(sw, nc, nc->reachable && nc->in_kernel); + __prestera_k_arb_n_offload_set(sw, nc, nc->reachable && nc->in_kernel); + + prestera_util_nc_key2nh_key(&nc->key, &nh_key); + nh_neigh = prestera_nh_neigh_find(sw, &nh_key); + if (!nh_neigh) + goto out; + + /* Do hw update only if something changed to prevent nh flap */ + if (memcmp(&nc->nh_neigh_info, &nh_neigh->info, + sizeof(nh_neigh->info))) { + memcpy(&nh_neigh->info, &nc->nh_neigh_info, + sizeof(nh_neigh->info)); + err = prestera_nh_neigh_set(sw, nh_neigh); + if (err) { + pr_err("%s failed with err=%d ip=%pI4n mac=%pM", + "prestera_nh_neigh_set", err, + &nh_neigh->key.addr.u.ipv4, + &nh_neigh->info.ha[0]); + goto out; + } + } + +out: + list_for_each_entry(nhead, &nc->kern_fib_cache_list, head) { + __prestera_k_arb_fib_nh_offload_set(sw, nhead->this, nc, + nc->in_kernel, + !nc->in_kernel); + } } static int __prestera_pr_k_arb_fc_lpm_info_calc(struct prestera_switch *sw, struct prestera_kern_fib_cache *fc) { + struct fib_nh_common *nhc; + int nh_cnt; + memset(&fc->lpm_info, 0, sizeof(fc->lpm_info)); - switch (fc->fi->fib_type) { + switch (prestera_kern_fib_info_type(&fc->info)) { case RTN_UNICAST: - fc->lpm_info.fib_type = PRESTERA_FIB_TYPE_TRAP; + if (prestera_fib_info_is_direct(&fc->info) && + fc->key.prefix_len == + PRESTERA_IP_ADDR_PLEN(fc->key.addr.v)) { + /* This is special case. + * When prefix is 32. Than we will have conflict in lpm + * for direct route - once TRAP added, there is no + * place for neighbour entry. So represent direct route + * with prefix 32, as NH. So neighbour will be resolved + * as nexthop of this route. + */ + nhc = prestera_kern_fib_info_nhc(&fc->info, 0); + fc->lpm_info.fib_type = PRESTERA_FIB_TYPE_UC_NH; + fc->lpm_info.nh_grp_key.neigh[0].addr = + fc->key.addr; + fc->lpm_info.nh_grp_key.neigh[0].rif = + nhc->nhc_dev; + + break; + } + + /* We can also get nh_grp_key from fi. This will be correct to + * because cache not always represent, what actually written to + * lpm. But we use nh cache, as well for now (for this case). + */ + for (nh_cnt = 0; nh_cnt < PRESTERA_NHGR_SIZE_MAX; nh_cnt++) { + if (!fc->kern_neigh_cache_head[nh_cnt].n_cache) + break; + + fc->lpm_info.nh_grp_key.neigh[nh_cnt].addr = + fc->kern_neigh_cache_head[nh_cnt].n_cache->key.addr; + fc->lpm_info.nh_grp_key.neigh[nh_cnt].rif = + fc->kern_neigh_cache_head[nh_cnt].n_cache->key.dev; + } + + fc->lpm_info.fib_type = nh_cnt ? + PRESTERA_FIB_TYPE_UC_NH : + PRESTERA_FIB_TYPE_TRAP; break; /* Unsupported. Leave it for kernel: */ case RTN_BROADCAST: @@ -191,7 +940,8 @@ static int __prestera_k_arb_f_lpm_set(struct prestera_switch *sw, return 0; fib_node = prestera_fib_node_create(sw, &fc->lpm_info.fib_key, - fc->lpm_info.fib_type); + fc->lpm_info.fib_type, + &fc->lpm_info.nh_grp_key); if (!fib_node) { dev_err(sw->dev->dev, "fib_node=NULL %pI4n/%d kern_tb_id = %d", @@ -220,6 +970,10 @@ static int __prestera_k_arb_fc_apply(struct prestera_switch *sw, } switch (fc->lpm_info.fib_type) { + case PRESTERA_FIB_TYPE_UC_NH: + __prestera_k_arb_fib_lpm_offload_set(sw, fc, false, + fc->reachable, false); + break; case PRESTERA_FIB_TYPE_TRAP: __prestera_k_arb_fib_lpm_offload_set(sw, fc, false, false, fc->reachable); @@ -271,17 +1025,140 @@ __prestera_k_arb_util_fib_overlapped(struct prestera_switch *sw, return rfc; } +static void __prestera_k_arb_hw_state_upd(struct prestera_switch *sw, + struct prestera_kern_neigh_cache *nc) +{ + struct prestera_nh_neigh_key nh_key; + struct prestera_nh_neigh *nh_neigh; + struct neighbour *n; + bool hw_active; + + prestera_util_nc_key2nh_key(&nc->key, &nh_key); + nh_neigh = prestera_nh_neigh_find(sw, &nh_key); + if (!nh_neigh) { + pr_err("Cannot find nh_neigh for cached %pI4n", + &nc->key.addr.u.ipv4); + return; + } + + hw_active = prestera_nh_neigh_util_hw_state(sw, nh_neigh); + +#ifdef PRESTERA_IMPLICITY_RESOLVE_DEAD_NEIGH + if (!hw_active && nc->in_kernel) + goto out; +#else /* PRESTERA_IMPLICITY_RESOLVE_DEAD_NEIGH */ + if (!hw_active) + goto out; +#endif /* PRESTERA_IMPLICITY_RESOLVE_DEAD_NEIGH */ + + if (nc->key.addr.v == PRESTERA_IPV4) { + n = neigh_lookup(&arp_tbl, &nc->key.addr.u.ipv4, + nc->key.dev); + if (!n) + n = neigh_create(&arp_tbl, &nc->key.addr.u.ipv4, + nc->key.dev); + } else { + n = NULL; + } + + if (!IS_ERR(n) && n) { + neigh_event_send(n, NULL); + neigh_release(n); + } else { + pr_err("Cannot create neighbour %pI4n", &nc->key.addr.u.ipv4); + } + +out: + return; +} + +/* Propagate hw state to kernel */ +static void prestera_k_arb_hw_evt(struct prestera_switch *sw) +{ + struct prestera_kern_neigh_cache *n_cache; + struct rhashtable_iter iter; + + rhashtable_walk_enter(&sw->router->kern_neigh_cache_ht, &iter); + rhashtable_walk_start(&iter); + while (1) { + n_cache = rhashtable_walk_next(&iter); + + if (!n_cache) + break; + + if (IS_ERR(n_cache)) + continue; + + rhashtable_walk_stop(&iter); + __prestera_k_arb_hw_state_upd(sw, n_cache); + rhashtable_walk_start(&iter); + } + rhashtable_walk_stop(&iter); + rhashtable_walk_exit(&iter); +} + +/* Propagate kernel event to hw */ +static void prestera_k_arb_n_evt(struct prestera_switch *sw, + struct neighbour *n) +{ + struct prestera_kern_neigh_cache_key n_key; + struct prestera_kern_neigh_cache *n_cache; + int err; + + err = prestera_util_neigh2nc_key(sw, n, &n_key); + if (err) + return; + + n_cache = prestera_kern_neigh_cache_find(sw, &n_key); + if (!n_cache) { + n_cache = prestera_kern_neigh_cache_get(sw, &n_key); + if (!n_cache) + return; + __prestera_k_arb_nc_kern_fib_fetch(sw, n_cache); + } + + __prestera_k_arb_nc_kern_n_fetch(sw, n_cache); + __prestera_k_arb_nc_apply(sw, n_cache); + + prestera_kern_neigh_cache_put(sw, n_cache); +} + +static void __prestera_k_arb_fib_evt2nc(struct prestera_switch *sw) +{ + struct prestera_kern_neigh_cache *n_cache; + struct rhashtable_iter iter; + + rhashtable_walk_enter(&sw->router->kern_neigh_cache_ht, &iter); + rhashtable_walk_start(&iter); + while (1) { + n_cache = rhashtable_walk_next(&iter); + + if (!n_cache) + break; + + if (IS_ERR(n_cache)) + continue; + + rhashtable_walk_stop(&iter); + __prestera_k_arb_nc_kern_fib_fetch(sw, n_cache); + __prestera_k_arb_nc_apply(sw, n_cache); + rhashtable_walk_start(&iter); + } + rhashtable_walk_stop(&iter); + rhashtable_walk_exit(&iter); +} + static int prestera_k_arb_fib_evt(struct prestera_switch *sw, bool replace, /* replace or del */ - struct fib_entry_notifier_info *fen_info) + struct fib_notifier_info *info) { struct prestera_kern_fib_cache *tfib_cache, *bfib_cache; /* top/btm */ struct prestera_kern_fib_cache_key fc_key; struct prestera_kern_fib_cache *fib_cache; int err; - prestera_util_fen_info2fib_cache_key(fen_info, &fc_key); + prestera_util_fen_info2fib_cache_key(info, &fc_key); fib_cache = prestera_kern_fib_cache_find(sw, &fc_key); if (fib_cache) { fib_cache->reachable = false; @@ -304,10 +1181,7 @@ prestera_k_arb_fib_evt(struct prestera_switch *sw, } if (replace) { - fib_cache = prestera_kern_fib_cache_create(sw, &fc_key, - fen_info->fi, - fen_info->dscp, - fen_info->type); + fib_cache = prestera_kern_fib_cache_create(sw, &fc_key, info); if (!fib_cache) { dev_err(sw->dev->dev, "fib_cache == NULL"); return -ENOENT; @@ -331,9 +1205,65 @@ prestera_k_arb_fib_evt(struct prestera_switch *sw, dev_err(sw->dev->dev, "Applying fib_cache failed"); } + /* Update all neighs to resolve overlapped and apply related */ + __prestera_k_arb_fib_evt2nc(sw); + return 0; } +static void __prestera_k_arb_abort_neigh_ht_cb(void *ptr, void *arg) +{ + struct prestera_kern_neigh_cache *n_cache = ptr; + struct prestera_switch *sw = arg; + + if (!list_empty(&n_cache->kern_fib_cache_list)) { + WARN_ON(1); /* BUG */ + return; + } + __prestera_k_arb_n_offload_set(sw, n_cache, false); + n_cache->in_kernel = false; + /* No need to destroy lpm. + * It will be aborted by destroy_ht + */ + __prestera_kern_neigh_cache_destruct(sw, n_cache); + kfree(n_cache); +} + +static void __prestera_k_arb_abort_fib_ht_cb(void *ptr, void *arg) +{ + struct prestera_kern_fib_cache *fib_cache = ptr; + struct prestera_switch *sw = arg; + + __prestera_k_arb_fib_lpm_offload_set(sw, fib_cache, + false, false, + false); + __prestera_k_arb_fib_nh_offload_set(sw, fib_cache, NULL, + false, false); + /* No need to destroy lpm. + * It will be aborted by destroy_ht + */ + __prestera_kern_fib_cache_destruct(sw, fib_cache); + kfree(fib_cache); +} + +static void prestera_k_arb_abort(struct prestera_switch *sw) +{ + /* Function to remove all arbiter entries and related hw objects. */ + /* Sequence: + * 1) Clear arbiter tables, but don't touch hw + * 2) Clear hw + * We use such approach, because arbiter object is not directly mapped + * to hw. So deletion of one arbiter object may even lead to creation of + * hw object (e.g. in case of overlapped routes). + */ + rhashtable_free_and_destroy(&sw->router->kern_fib_cache_ht, + __prestera_k_arb_abort_fib_ht_cb, + sw); + rhashtable_free_and_destroy(&sw->router->kern_neigh_cache_ht, + __prestera_k_arb_abort_neigh_ht_cb, + sw); +} + static int __prestera_inetaddr_port_event(struct net_device *port_dev, unsigned long event, struct netlink_ext_ack *extack) @@ -469,13 +1399,15 @@ static void __prestera_router_fib_event_work(struct work_struct *work) switch (fib_work->event) { case FIB_EVENT_ENTRY_REPLACE: - err = prestera_k_arb_fib_evt(sw, true, &fib_work->fen_info); + err = prestera_k_arb_fib_evt(sw, true, + &fib_work->fen_info.info); if (err) goto err_out; break; case FIB_EVENT_ENTRY_DEL: - err = prestera_k_arb_fib_evt(sw, false, &fib_work->fen_info); + err = prestera_k_arb_fib_evt(sw, false, + &fib_work->fen_info.info); if (err) goto err_out; @@ -534,10 +1466,89 @@ static int __prestera_router_fib_event(struct notifier_block *nb, return NOTIFY_DONE; } +struct prestera_netevent_work { + struct work_struct work; + struct prestera_switch *sw; + struct neighbour *n; +}; + +static void prestera_router_neigh_event_work(struct work_struct *work) +{ + struct prestera_netevent_work *net_work = + container_of(work, struct prestera_netevent_work, work); + struct prestera_switch *sw = net_work->sw; + struct neighbour *n = net_work->n; + + /* neigh - its not hw related object. It stored only in kernel. So... */ + rtnl_lock(); + + prestera_k_arb_n_evt(sw, n); + + neigh_release(n); + rtnl_unlock(); + kfree(net_work); +} + +static int prestera_router_netevent_event(struct notifier_block *nb, + unsigned long event, void *ptr) +{ + struct prestera_netevent_work *net_work; + struct prestera_router *router; + struct neighbour *n = ptr; + + router = container_of(nb, struct prestera_router, netevent_nb); + + switch (event) { + case NETEVENT_NEIGH_UPDATE: + if (n->tbl->family != AF_INET) + return NOTIFY_DONE; + + net_work = kzalloc(sizeof(*net_work), GFP_ATOMIC); + if (WARN_ON(!net_work)) + return NOTIFY_BAD; + + neigh_clone(n); + net_work->n = n; + net_work->sw = router->sw; + INIT_WORK(&net_work->work, prestera_router_neigh_event_work); + prestera_queue_work(&net_work->work); + } + + return NOTIFY_DONE; +} + +static void prestera_router_update_neighs_work(struct work_struct *work) +{ + struct prestera_router *router; + + router = container_of(work, struct prestera_router, + neighs_update.dw.work); + rtnl_lock(); + + prestera_k_arb_hw_evt(router->sw); + + rtnl_unlock(); + prestera_queue_delayed_work(&router->neighs_update.dw, + msecs_to_jiffies(PRESTERA_NH_PROBE_INTERVAL)); +} + +static int prestera_neigh_work_init(struct prestera_switch *sw) +{ + INIT_DELAYED_WORK(&sw->router->neighs_update.dw, + prestera_router_update_neighs_work); + prestera_queue_delayed_work(&sw->router->neighs_update.dw, 0); + return 0; +} + +static void prestera_neigh_work_fini(struct prestera_switch *sw) +{ + cancel_delayed_work_sync(&sw->router->neighs_update.dw); +} + int prestera_router_init(struct prestera_switch *sw) { struct prestera_router *router; - int err; + int err, nhgrp_cache_bytes; router = kzalloc(sizeof(*sw->router), GFP_KERNEL); if (!router) @@ -555,6 +1566,22 @@ int prestera_router_init(struct prestera_switch *sw) if (err) goto err_kern_fib_cache_ht_init; + err = rhashtable_init(&router->kern_neigh_cache_ht, + &__prestera_kern_neigh_cache_ht_params); + if (err) + goto err_kern_neigh_cache_ht_init; + + nhgrp_cache_bytes = sw->size_tbl_router_nexthop / 8 + 1; + router->nhgrp_hw_state_cache = kzalloc(nhgrp_cache_bytes, GFP_KERNEL); + if (!router->nhgrp_hw_state_cache) { + err = -ENOMEM; + goto err_nh_state_cache_alloc; + } + + err = prestera_neigh_work_init(sw); + if (err) + goto err_neigh_work_init; + router->inetaddr_valid_nb.notifier_call = __prestera_inetaddr_valid_cb; err = register_inetaddr_validator_notifier(&router->inetaddr_valid_nb); if (err) @@ -565,6 +1592,11 @@ int prestera_router_init(struct prestera_switch *sw) if (err) goto err_register_inetaddr_notifier; + router->netevent_nb.notifier_call = prestera_router_netevent_event; + err = register_netevent_notifier(&router->netevent_nb); + if (err) + goto err_register_netevent_notifier; + router->fib_nb.notifier_call = __prestera_router_fib_event; err = register_fib_notifier(&init_net, &router->fib_nb, /* TODO: flush fib entries */ NULL, NULL); @@ -574,10 +1606,18 @@ int prestera_router_init(struct prestera_switch *sw) return 0; err_register_fib_notifier: + unregister_netevent_notifier(&router->netevent_nb); +err_register_netevent_notifier: unregister_inetaddr_notifier(&router->inetaddr_nb); err_register_inetaddr_notifier: unregister_inetaddr_validator_notifier(&router->inetaddr_valid_nb); err_register_inetaddr_validator_notifier: + prestera_neigh_work_fini(sw); +err_neigh_work_init: + kfree(router->nhgrp_hw_state_cache); +err_nh_state_cache_alloc: + rhashtable_destroy(&router->kern_neigh_cache_ht); +err_kern_neigh_cache_ht_init: rhashtable_destroy(&router->kern_fib_cache_ht); err_kern_fib_cache_ht_init: prestera_router_hw_fini(sw); @@ -589,8 +1629,15 @@ err_router_lib_init: void prestera_router_fini(struct prestera_switch *sw) { unregister_fib_notifier(&init_net, &sw->router->fib_nb); + unregister_netevent_notifier(&sw->router->netevent_nb); unregister_inetaddr_notifier(&sw->router->inetaddr_nb); unregister_inetaddr_validator_notifier(&sw->router->inetaddr_valid_nb); + prestera_neigh_work_fini(sw); + prestera_queue_drain(); + + prestera_k_arb_abort(sw); + + kfree(sw->router->nhgrp_hw_state_cache); rhashtable_destroy(&sw->router->kern_fib_cache_ht); prestera_router_hw_fini(sw); kfree(sw->router); diff --git a/drivers/net/ethernet/marvell/prestera/prestera_router_hw.c b/drivers/net/ethernet/marvell/prestera/prestera_router_hw.c index 5b0cf3be9a9e..4f65df0ae5e8 100644 --- a/drivers/net/ethernet/marvell/prestera/prestera_router_hw.c +++ b/drivers/net/ethernet/marvell/prestera/prestera_router_hw.c @@ -8,10 +8,16 @@ #include "prestera_router_hw.h" #include "prestera_acl.h" -/* +--+ - * +------->|vr|<-+ - * | +--+ | - * | | +/* Nexthop is pointed + * to port (not rif) + * +-------+ + * +>|nexthop| + * | +-------+ + * | + * +--+ +-----++ + * +------->|vr|<-+ +>|nh_grp| + * | +--+ | | +------+ + * | | | * +-+-------+ +--+---+-+ * |rif_entry| |fib_node| * +---------+ +--------+ @@ -23,6 +29,8 @@ #define PRESTERA_NHGR_UNUSED (0) #define PRESTERA_NHGR_DROP (0xFFFFFFFF) +/* Need to merge it with router_manager */ +#define PRESTERA_NH_ACTIVE_JIFFER_FILTER 3000 /* ms */ static const struct rhashtable_params __prestera_fib_ht_params = { .key_offset = offsetof(struct prestera_fib_node, key), @@ -31,10 +39,45 @@ static const struct rhashtable_params __prestera_fib_ht_params = { .automatic_shrinking = true, }; +static const struct rhashtable_params __prestera_nh_neigh_ht_params = { + .key_offset = offsetof(struct prestera_nh_neigh, key), + .key_len = sizeof(struct prestera_nh_neigh_key), + .head_offset = offsetof(struct prestera_nh_neigh, ht_node), +}; + +static const struct rhashtable_params __prestera_nexthop_group_ht_params = { + .key_offset = offsetof(struct prestera_nexthop_group, key), + .key_len = sizeof(struct prestera_nexthop_group_key), + .head_offset = offsetof(struct prestera_nexthop_group, ht_node), +}; + +static int prestera_nexthop_group_set(struct prestera_switch *sw, + struct prestera_nexthop_group *nh_grp); +static bool +prestera_nexthop_group_util_hw_state(struct prestera_switch *sw, + struct prestera_nexthop_group *nh_grp); +static void prestera_fib_node_destroy_ht_cb(void *ptr, void *arg); + +/* TODO: move to router.h as macros */ +static bool prestera_nh_neigh_key_is_valid(struct prestera_nh_neigh_key *key) +{ + return memchr_inv(key, 0, sizeof(*key)) ? true : false; +} + int prestera_router_hw_init(struct prestera_switch *sw) { int err; + err = rhashtable_init(&sw->router->nh_neigh_ht, + &__prestera_nh_neigh_ht_params); + if (err) + goto err_nh_neigh_ht_init; + + err = rhashtable_init(&sw->router->nexthop_group_ht, + &__prestera_nexthop_group_ht_params); + if (err) + goto err_nexthop_grp_ht_init; + err = rhashtable_init(&sw->router->fib_ht, &__prestera_fib_ht_params); if (err) @@ -43,15 +86,25 @@ int prestera_router_hw_init(struct prestera_switch *sw) INIT_LIST_HEAD(&sw->router->vr_list); INIT_LIST_HEAD(&sw->router->rif_entry_list); + return 0; + err_fib_ht_init: + rhashtable_destroy(&sw->router->nexthop_group_ht); +err_nexthop_grp_ht_init: + rhashtable_destroy(&sw->router->nh_neigh_ht); +err_nh_neigh_ht_init: return 0; } void prestera_router_hw_fini(struct prestera_switch *sw) { + rhashtable_free_and_destroy(&sw->router->fib_ht, + prestera_fib_node_destroy_ht_cb, sw); WARN_ON(!list_empty(&sw->router->vr_list)); WARN_ON(!list_empty(&sw->router->rif_entry_list)); rhashtable_destroy(&sw->router->fib_ht); + rhashtable_destroy(&sw->router->nexthop_group_ht); + rhashtable_destroy(&sw->router->nh_neigh_ht); } static struct prestera_vr *__prestera_vr_find(struct prestera_switch *sw, @@ -232,6 +285,286 @@ err_kzalloc: return NULL; } +static void __prestera_nh_neigh_destroy(struct prestera_switch *sw, + struct prestera_nh_neigh *neigh) +{ + rhashtable_remove_fast(&sw->router->nh_neigh_ht, + &neigh->ht_node, + __prestera_nh_neigh_ht_params); + kfree(neigh); +} + +static struct prestera_nh_neigh * +__prestera_nh_neigh_create(struct prestera_switch *sw, + struct prestera_nh_neigh_key *key) +{ + struct prestera_nh_neigh *neigh; + int err; + + neigh = kzalloc(sizeof(*neigh), GFP_KERNEL); + if (!neigh) + goto err_kzalloc; + + memcpy(&neigh->key, key, sizeof(*key)); + neigh->info.connected = false; + INIT_LIST_HEAD(&neigh->nexthop_group_list); + err = rhashtable_insert_fast(&sw->router->nh_neigh_ht, + &neigh->ht_node, + __prestera_nh_neigh_ht_params); + if (err) + goto err_rhashtable_insert; + + return neigh; + +err_rhashtable_insert: + kfree(neigh); +err_kzalloc: + return NULL; +} + +struct prestera_nh_neigh * +prestera_nh_neigh_find(struct prestera_switch *sw, + struct prestera_nh_neigh_key *key) +{ + struct prestera_nh_neigh *nh_neigh; + + nh_neigh = rhashtable_lookup_fast(&sw->router->nh_neigh_ht, + key, __prestera_nh_neigh_ht_params); + return IS_ERR(nh_neigh) ? NULL : nh_neigh; +} + +struct prestera_nh_neigh * +prestera_nh_neigh_get(struct prestera_switch *sw, + struct prestera_nh_neigh_key *key) +{ + struct prestera_nh_neigh *neigh; + + neigh = prestera_nh_neigh_find(sw, key); + if (!neigh) + return __prestera_nh_neigh_create(sw, key); + + return neigh; +} + +void prestera_nh_neigh_put(struct prestera_switch *sw, + struct prestera_nh_neigh *neigh) +{ + if (list_empty(&neigh->nexthop_group_list)) + __prestera_nh_neigh_destroy(sw, neigh); +} + +/* Updates new prestera_neigh_info */ +int prestera_nh_neigh_set(struct prestera_switch *sw, + struct prestera_nh_neigh *neigh) +{ + struct prestera_nh_neigh_head *nh_head; + struct prestera_nexthop_group *nh_grp; + int err; + + list_for_each_entry(nh_head, &neigh->nexthop_group_list, head) { + nh_grp = nh_head->this; + err = prestera_nexthop_group_set(sw, nh_grp); + if (err) + return err; + } + + return 0; +} + +bool prestera_nh_neigh_util_hw_state(struct prestera_switch *sw, + struct prestera_nh_neigh *nh_neigh) +{ + bool state; + struct prestera_nh_neigh_head *nh_head, *tmp; + + state = false; + list_for_each_entry_safe(nh_head, tmp, + &nh_neigh->nexthop_group_list, head) { + state = prestera_nexthop_group_util_hw_state(sw, nh_head->this); + if (state) + goto out; + } + +out: + return state; +} + +static struct prestera_nexthop_group * +__prestera_nexthop_group_create(struct prestera_switch *sw, + struct prestera_nexthop_group_key *key) +{ + struct prestera_nexthop_group *nh_grp; + struct prestera_nh_neigh *nh_neigh; + int nh_cnt, err, gid; + + nh_grp = kzalloc(sizeof(*nh_grp), GFP_KERNEL); + if (!nh_grp) + goto err_kzalloc; + + memcpy(&nh_grp->key, key, sizeof(*key)); + for (nh_cnt = 0; nh_cnt < PRESTERA_NHGR_SIZE_MAX; nh_cnt++) { + if (!prestera_nh_neigh_key_is_valid(&nh_grp->key.neigh[nh_cnt])) + break; + + nh_neigh = prestera_nh_neigh_get(sw, + &nh_grp->key.neigh[nh_cnt]); + if (!nh_neigh) + goto err_nh_neigh_get; + + nh_grp->nh_neigh_head[nh_cnt].neigh = nh_neigh; + nh_grp->nh_neigh_head[nh_cnt].this = nh_grp; + list_add(&nh_grp->nh_neigh_head[nh_cnt].head, + &nh_neigh->nexthop_group_list); + } + + err = prestera_hw_nh_group_create(sw, nh_cnt, &nh_grp->grp_id); + if (err) + goto err_nh_group_create; + + err = prestera_nexthop_group_set(sw, nh_grp); + if (err) + goto err_nexthop_group_set; + + err = rhashtable_insert_fast(&sw->router->nexthop_group_ht, + &nh_grp->ht_node, + __prestera_nexthop_group_ht_params); + if (err) + goto err_ht_insert; + + /* reset cache for created group */ + gid = nh_grp->grp_id; + sw->router->nhgrp_hw_state_cache[gid / 8] &= ~BIT(gid % 8); + + return nh_grp; + +err_ht_insert: +err_nexthop_group_set: + prestera_hw_nh_group_delete(sw, nh_cnt, nh_grp->grp_id); +err_nh_group_create: +err_nh_neigh_get: + for (nh_cnt--; nh_cnt >= 0; nh_cnt--) { + list_del(&nh_grp->nh_neigh_head[nh_cnt].head); + prestera_nh_neigh_put(sw, nh_grp->nh_neigh_head[nh_cnt].neigh); + } + + kfree(nh_grp); +err_kzalloc: + return NULL; +} + +static void +__prestera_nexthop_group_destroy(struct prestera_switch *sw, + struct prestera_nexthop_group *nh_grp) +{ + struct prestera_nh_neigh *nh_neigh; + int nh_cnt; + + rhashtable_remove_fast(&sw->router->nexthop_group_ht, + &nh_grp->ht_node, + __prestera_nexthop_group_ht_params); + + for (nh_cnt = 0; nh_cnt < PRESTERA_NHGR_SIZE_MAX; nh_cnt++) { + nh_neigh = nh_grp->nh_neigh_head[nh_cnt].neigh; + if (!nh_neigh) + break; + + list_del(&nh_grp->nh_neigh_head[nh_cnt].head); + prestera_nh_neigh_put(sw, nh_neigh); + } + + prestera_hw_nh_group_delete(sw, nh_cnt, nh_grp->grp_id); + kfree(nh_grp); +} + +static struct prestera_nexthop_group * +__prestera_nexthop_group_find(struct prestera_switch *sw, + struct prestera_nexthop_group_key *key) +{ + struct prestera_nexthop_group *nh_grp; + + nh_grp = rhashtable_lookup_fast(&sw->router->nexthop_group_ht, + key, __prestera_nexthop_group_ht_params); + return IS_ERR(nh_grp) ? NULL : nh_grp; +} + +static struct prestera_nexthop_group * +prestera_nexthop_group_get(struct prestera_switch *sw, + struct prestera_nexthop_group_key *key) +{ + struct prestera_nexthop_group *nh_grp; + + nh_grp = __prestera_nexthop_group_find(sw, key); + if (nh_grp) { + refcount_inc(&nh_grp->refcount); + } else { + nh_grp = __prestera_nexthop_group_create(sw, key); + if (IS_ERR(nh_grp)) + return ERR_CAST(nh_grp); + + refcount_set(&nh_grp->refcount, 1); + } + + return nh_grp; +} + +static void prestera_nexthop_group_put(struct prestera_switch *sw, + struct prestera_nexthop_group *nh_grp) +{ + if (refcount_dec_and_test(&nh_grp->refcount)) + __prestera_nexthop_group_destroy(sw, nh_grp); +} + +/* Updates with new nh_neigh's info */ +static int prestera_nexthop_group_set(struct prestera_switch *sw, + struct prestera_nexthop_group *nh_grp) +{ + struct prestera_neigh_info info[PRESTERA_NHGR_SIZE_MAX]; + struct prestera_nh_neigh *neigh; + int nh_cnt; + + memset(&info[0], 0, sizeof(info)); + for (nh_cnt = 0; nh_cnt < PRESTERA_NHGR_SIZE_MAX; nh_cnt++) { + neigh = nh_grp->nh_neigh_head[nh_cnt].neigh; + if (!neigh) + break; + + memcpy(&info[nh_cnt], &neigh->info, sizeof(neigh->info)); + } + + return prestera_hw_nh_entries_set(sw, nh_cnt, &info[0], nh_grp->grp_id); +} + +static bool +prestera_nexthop_group_util_hw_state(struct prestera_switch *sw, + struct prestera_nexthop_group *nh_grp) +{ + int err; + u32 buf_size = sw->size_tbl_router_nexthop / 8 + 1; + u32 gid = nh_grp->grp_id; + u8 *cache = sw->router->nhgrp_hw_state_cache; + + /* Antijitter + * Prevent situation, when we read state of nh_grp twice in short time, + * and state bit is still cleared on second call. So just stuck active + * state for PRESTERA_NH_ACTIVE_JIFFER_FILTER, after last occurred. + */ + if (!time_before(jiffies, sw->router->nhgrp_hw_cache_kick + + msecs_to_jiffies(PRESTERA_NH_ACTIVE_JIFFER_FILTER))) { + err = prestera_hw_nhgrp_blk_get(sw, cache, buf_size); + if (err) { + pr_err("Failed to get hw state nh_grp's"); + return false; + } + + sw->router->nhgrp_hw_cache_kick = jiffies; + } + + if (cache[gid / 8] & BIT(gid % 8)) + return true; + + return false; +} + struct prestera_fib_node * prestera_fib_node_find(struct prestera_switch *sw, struct prestera_fib_key *key) { @@ -251,6 +584,9 @@ static void __prestera_fib_node_destruct(struct prestera_switch *sw, prestera_hw_lpm_del(sw, vr->hw_vr_id, fib_node->key.addr.u.ipv4, fib_node->key.prefix_len); switch (fib_node->info.type) { + case PRESTERA_FIB_TYPE_UC_NH: + prestera_nexthop_group_put(sw, fib_node->info.nh_grp); + break; case PRESTERA_FIB_TYPE_TRAP: break; case PRESTERA_FIB_TYPE_DROP: @@ -272,10 +608,20 @@ void prestera_fib_node_destroy(struct prestera_switch *sw, kfree(fib_node); } +static void prestera_fib_node_destroy_ht_cb(void *ptr, void *arg) +{ + struct prestera_fib_node *node = ptr; + struct prestera_switch *sw = arg; + + __prestera_fib_node_destruct(sw, node); + kfree(node); +} + struct prestera_fib_node * prestera_fib_node_create(struct prestera_switch *sw, struct prestera_fib_key *key, - enum prestera_fib_type fib_type) + enum prestera_fib_type fib_type, + struct prestera_nexthop_group_key *nh_grp_key) { struct prestera_fib_node *fib_node; u32 grp_id; @@ -302,6 +648,14 @@ prestera_fib_node_create(struct prestera_switch *sw, case PRESTERA_FIB_TYPE_DROP: grp_id = PRESTERA_NHGR_DROP; break; + case PRESTERA_FIB_TYPE_UC_NH: + fib_node->info.nh_grp = prestera_nexthop_group_get(sw, + nh_grp_key); + if (!fib_node->info.nh_grp) + goto err_nh_grp_get; + + grp_id = fib_node->info.nh_grp->grp_id; + break; default: pr_err("Unsupported fib_type %d", fib_type); goto err_nh_grp_get; @@ -323,6 +677,8 @@ err_ht_insert: prestera_hw_lpm_del(sw, vr->hw_vr_id, key->addr.u.ipv4, key->prefix_len); err_lpm_add: + if (fib_type == PRESTERA_FIB_TYPE_UC_NH) + prestera_nexthop_group_put(sw, fib_node->info.nh_grp); err_nh_grp_get: prestera_vr_put(sw, vr); err_vr_get: diff --git a/drivers/net/ethernet/marvell/prestera/prestera_router_hw.h b/drivers/net/ethernet/marvell/prestera/prestera_router_hw.h index 67dbb49c8bd4..9ca97919c863 100644 --- a/drivers/net/ethernet/marvell/prestera/prestera_router_hw.h +++ b/drivers/net/ethernet/marvell/prestera/prestera_router_hw.h @@ -31,6 +31,63 @@ struct prestera_ip_addr { PRESTERA_IPV4 = 0, PRESTERA_IPV6 } v; +#define PRESTERA_IP_ADDR_PLEN(V) ((V) == PRESTERA_IPV4 ? 32 : \ + /* (V) == PRESTERA_IPV6 ? */ 128 /* : 0 */) +}; + +struct prestera_nh_neigh_key { + struct prestera_ip_addr addr; + /* Seems like rif is obsolete, because there is iface in info ? + * Key can contain functional fields, or fields, which is used to + * filter duplicate objects on logical level (before you pass it to + * HW)... also key can be used to cover hardware restrictions. + * In our case rif - is logical interface (even can be VLAN), which + * is used in combination with IP address (which is also not related to + * hardware nexthop) to provide logical compression of created nexthops. + * You even can imagine, that rif+IPaddr is just cookie. + */ + /* struct prestera_rif *rif; */ + /* Use just as cookie, to divide ARP domains (in order with addr) */ + void *rif; +}; + +/* Used for hw call */ +struct prestera_neigh_info { + struct prestera_iface iface; + unsigned char ha[ETH_ALEN]; + u8 connected; /* bool. indicate, if mac/oif valid */ + u8 __pad[1]; +}; + +/* Used to notify nh about neigh change */ +struct prestera_nh_neigh { + struct prestera_nh_neigh_key key; + struct prestera_neigh_info info; + struct rhash_head ht_node; /* node of prestera_vr */ + struct list_head nexthop_group_list; +}; + +#define PRESTERA_NHGR_SIZE_MAX 4 + +struct prestera_nexthop_group { + struct prestera_nexthop_group_key { + struct prestera_nh_neigh_key neigh[PRESTERA_NHGR_SIZE_MAX]; + } key; + /* Store intermediate object here. + * This prevent overhead kzalloc call. + */ + /* nh_neigh is used only to notify nexthop_group */ + struct prestera_nh_neigh_head { + struct prestera_nexthop_group *this; + struct list_head head; + /* ptr to neigh is not necessary. + * It used to prevent lookup of nh_neigh by key (n) on destroy + */ + struct prestera_nh_neigh *neigh; + } nh_neigh_head[PRESTERA_NHGR_SIZE_MAX]; + struct rhash_head ht_node; /* node of prestera_vr */ + refcount_t refcount; + u32 grp_id; /* hw */ }; struct prestera_fib_key { @@ -44,12 +101,16 @@ struct prestera_fib_info { struct list_head vr_node; enum prestera_fib_type { PRESTERA_FIB_TYPE_INVALID = 0, + /* must be pointer to nh_grp id */ + PRESTERA_FIB_TYPE_UC_NH, /* It can be connected route * and will be overlapped with neighbours */ PRESTERA_FIB_TYPE_TRAP, PRESTERA_FIB_TYPE_DROP } type; + /* Valid only if type = UC_NH*/ + struct prestera_nexthop_group *nh_grp; }; struct prestera_fib_node { @@ -67,6 +128,18 @@ struct prestera_rif_entry * prestera_rif_entry_create(struct prestera_switch *sw, struct prestera_rif_entry_key *k, u32 tb_id, const unsigned char *addr); +struct prestera_nh_neigh * +prestera_nh_neigh_find(struct prestera_switch *sw, + struct prestera_nh_neigh_key *key); +struct prestera_nh_neigh * +prestera_nh_neigh_get(struct prestera_switch *sw, + struct prestera_nh_neigh_key *key); +void prestera_nh_neigh_put(struct prestera_switch *sw, + struct prestera_nh_neigh *neigh); +int prestera_nh_neigh_set(struct prestera_switch *sw, + struct prestera_nh_neigh *neigh); +bool prestera_nh_neigh_util_hw_state(struct prestera_switch *sw, + struct prestera_nh_neigh *nh_neigh); struct prestera_fib_node *prestera_fib_node_find(struct prestera_switch *sw, struct prestera_fib_key *key); void prestera_fib_node_destroy(struct prestera_switch *sw, @@ -74,7 +147,8 @@ void prestera_fib_node_destroy(struct prestera_switch *sw, struct prestera_fib_node * prestera_fib_node_create(struct prestera_switch *sw, struct prestera_fib_key *key, - enum prestera_fib_type fib_type); + enum prestera_fib_type fib_type, + struct prestera_nexthop_group_key *nh_grp_key); int prestera_router_hw_init(struct prestera_switch *sw); void prestera_router_hw_fini(struct prestera_switch *sw); diff --git a/drivers/net/ethernet/marvell/prestera/prestera_rxtx.c b/drivers/net/ethernet/marvell/prestera/prestera_rxtx.c index dc3e3ddc60bf..42ee963e9f75 100644 --- a/drivers/net/ethernet/marvell/prestera/prestera_rxtx.c +++ b/drivers/net/ethernet/marvell/prestera/prestera_rxtx.c @@ -659,7 +659,7 @@ static int prestera_sdma_switch_init(struct prestera_switch *sw) init_dummy_netdev(&sdma->napi_dev); - netif_napi_add(&sdma->napi_dev, &sdma->rx_napi, prestera_sdma_rx_poll, 64); + netif_napi_add(&sdma->napi_dev, &sdma->rx_napi, prestera_sdma_rx_poll); napi_enable(&sdma->rx_napi); return 0; diff --git a/drivers/net/ethernet/marvell/prestera/prestera_span.c b/drivers/net/ethernet/marvell/prestera/prestera_span.c index 845e9d8c8cc7..f0e9d6ea88c5 100644 --- a/drivers/net/ethernet/marvell/prestera/prestera_span.c +++ b/drivers/net/ethernet/marvell/prestera/prestera_span.c @@ -120,8 +120,9 @@ static int prestera_span_put(struct prestera_switch *sw, u8 span_id) return 0; } -static int prestera_span_rule_add(struct prestera_flow_block_binding *binding, - struct prestera_port *to_port) +int prestera_span_rule_add(struct prestera_flow_block_binding *binding, + struct prestera_port *to_port, + bool ingress) { struct prestera_switch *sw = binding->port->sw; u8 span_id; @@ -135,7 +136,7 @@ static int prestera_span_rule_add(struct prestera_flow_block_binding *binding, if (err) return err; - err = prestera_hw_span_bind(binding->port, span_id); + err = prestera_hw_span_bind(binding->port, span_id, ingress); if (err) { prestera_span_put(sw, span_id); return err; @@ -145,11 +146,12 @@ static int prestera_span_rule_add(struct prestera_flow_block_binding *binding, return 0; } -static int prestera_span_rule_del(struct prestera_flow_block_binding *binding) +int prestera_span_rule_del(struct prestera_flow_block_binding *binding, + bool ingress) { int err; - err = prestera_hw_span_unbind(binding->port); + err = prestera_hw_span_unbind(binding->port, ingress); if (err) return err; @@ -161,60 +163,6 @@ static int prestera_span_rule_del(struct prestera_flow_block_binding *binding) return 0; } -int prestera_span_replace(struct prestera_flow_block *block, - struct tc_cls_matchall_offload *f) -{ - struct prestera_flow_block_binding *binding; - __be16 protocol = f->common.protocol; - struct flow_action_entry *act; - struct prestera_port *port; - int err; - - if (!flow_offload_has_one_action(&f->rule->action)) { - NL_SET_ERR_MSG(f->common.extack, - "Only singular actions are supported"); - return -EOPNOTSUPP; - } - - act = &f->rule->action.entries[0]; - - if (!prestera_netdev_check(act->dev)) { - NL_SET_ERR_MSG(f->common.extack, - "Only Marvell Prestera port is supported"); - return -EINVAL; - } - if (!tc_cls_can_offload_and_chain0(act->dev, &f->common)) - return -EOPNOTSUPP; - if (act->id != FLOW_ACTION_MIRRED) - return -EOPNOTSUPP; - if (protocol != htons(ETH_P_ALL)) - return -EOPNOTSUPP; - - port = netdev_priv(act->dev); - - list_for_each_entry(binding, &block->binding_list, list) { - err = prestera_span_rule_add(binding, port); - if (err) - goto rollback; - } - - return 0; - -rollback: - list_for_each_entry_continue_reverse(binding, - &block->binding_list, list) - prestera_span_rule_del(binding); - return err; -} - -void prestera_span_destroy(struct prestera_flow_block *block) -{ - struct prestera_flow_block_binding *binding; - - list_for_each_entry(binding, &block->binding_list, list) - prestera_span_rule_del(binding); -} - int prestera_span_init(struct prestera_switch *sw) { struct prestera_span *span; diff --git a/drivers/net/ethernet/marvell/prestera/prestera_span.h b/drivers/net/ethernet/marvell/prestera/prestera_span.h index f0644521f78a..493b68524bcb 100644 --- a/drivers/net/ethernet/marvell/prestera/prestera_span.h +++ b/drivers/net/ethernet/marvell/prestera/prestera_span.h @@ -8,13 +8,17 @@ #define PRESTERA_SPAN_INVALID_ID -1 +struct prestera_port; struct prestera_switch; -struct prestera_flow_block; +struct prestera_flow_block_binding; int prestera_span_init(struct prestera_switch *sw); void prestera_span_fini(struct prestera_switch *sw); -int prestera_span_replace(struct prestera_flow_block *block, - struct tc_cls_matchall_offload *f); -void prestera_span_destroy(struct prestera_flow_block *block); + +int prestera_span_rule_add(struct prestera_flow_block_binding *binding, + struct prestera_port *to_port, + bool ingress); +int prestera_span_rule_del(struct prestera_flow_block_binding *binding, + bool ingress); #endif /* _PRESTERA_SPAN_H_ */ diff --git a/drivers/net/ethernet/marvell/prestera/prestera_switchdev.c b/drivers/net/ethernet/marvell/prestera/prestera_switchdev.c index 71cde97d85c8..e548cd32582e 100644 --- a/drivers/net/ethernet/marvell/prestera/prestera_switchdev.c +++ b/drivers/net/ethernet/marvell/prestera/prestera_switchdev.c @@ -143,6 +143,7 @@ prestera_br_port_flags_reset(struct prestera_bridge_port *br_port, prestera_port_uc_flood_set(port, false); prestera_port_mc_flood_set(port, false); prestera_port_learning_set(port, false); + prestera_port_br_locked_set(port, false); } static int prestera_br_port_flags_set(struct prestera_bridge_port *br_port, @@ -162,6 +163,11 @@ static int prestera_br_port_flags_set(struct prestera_bridge_port *br_port, if (err) goto err_out; + err = prestera_port_br_locked_set(port, + br_port->flags & BR_PORT_LOCKED); + if (err) + goto err_out; + return 0; err_out: @@ -1163,7 +1169,7 @@ static int prestera_port_obj_attr_set(struct net_device *dev, const void *ctx, break; case SWITCHDEV_ATTR_ID_PORT_PRE_BRIDGE_FLAGS: if (attr->u.brport_flags.mask & - ~(BR_LEARNING | BR_FLOOD | BR_MCAST_FLOOD)) + ~(BR_LEARNING | BR_FLOOD | BR_MCAST_FLOOD | BR_PORT_LOCKED)) err = -EINVAL; break; case SWITCHDEV_ATTR_ID_PORT_BRIDGE_FLAGS: diff --git a/drivers/net/ethernet/marvell/pxa168_eth.c b/drivers/net/ethernet/marvell/pxa168_eth.c index 349b8a94e939..cf456d62677f 100644 --- a/drivers/net/ethernet/marvell/pxa168_eth.c +++ b/drivers/net/ethernet/marvell/pxa168_eth.c @@ -1354,10 +1354,10 @@ static void pxa168_eth_netpoll(struct net_device *dev) static void pxa168_get_drvinfo(struct net_device *dev, struct ethtool_drvinfo *info) { - strlcpy(info->driver, DRIVER_NAME, sizeof(info->driver)); - strlcpy(info->version, DRIVER_VERSION, sizeof(info->version)); - strlcpy(info->fw_version, "N/A", sizeof(info->fw_version)); - strlcpy(info->bus_info, "N/A", sizeof(info->bus_info)); + strscpy(info->driver, DRIVER_NAME, sizeof(info->driver)); + strscpy(info->version, DRIVER_VERSION, sizeof(info->version)); + strscpy(info->fw_version, "N/A", sizeof(info->fw_version)); + strscpy(info->bus_info, "N/A", sizeof(info->bus_info)); } static const struct ethtool_ops pxa168_ethtool_ops = { diff --git a/drivers/net/ethernet/marvell/skge.c b/drivers/net/ethernet/marvell/skge.c index c1e985416c0e..1b43704baceb 100644 --- a/drivers/net/ethernet/marvell/skge.c +++ b/drivers/net/ethernet/marvell/skge.c @@ -394,9 +394,9 @@ static void skge_get_drvinfo(struct net_device *dev, { struct skge_port *skge = netdev_priv(dev); - strlcpy(info->driver, DRV_NAME, sizeof(info->driver)); - strlcpy(info->version, DRV_VERSION, sizeof(info->version)); - strlcpy(info->bus_info, pci_name(skge->hw->pdev), + strscpy(info->driver, DRV_NAME, sizeof(info->driver)); + strscpy(info->version, DRV_VERSION, sizeof(info->version)); + strscpy(info->bus_info, pci_name(skge->hw->pdev), sizeof(info->bus_info)); } @@ -3832,7 +3832,7 @@ static struct net_device *skge_devinit(struct skge_hw *hw, int port, dev->features |= NETIF_F_HIGHDMA; skge = netdev_priv(dev); - netif_napi_add(dev, &skge->napi, skge_poll, NAPI_POLL_WEIGHT); + netif_napi_add(dev, &skge->napi, skge_poll); skge->netdev = dev; skge->hw = hw; skge->msg_enable = netif_msg_init(debug, default_msg); diff --git a/drivers/net/ethernet/marvell/sky2.c b/drivers/net/ethernet/marvell/sky2.c index bbea5458000b..ab33ba1c3023 100644 --- a/drivers/net/ethernet/marvell/sky2.c +++ b/drivers/net/ethernet/marvell/sky2.c @@ -3687,9 +3687,9 @@ static void sky2_get_drvinfo(struct net_device *dev, { struct sky2_port *sky2 = netdev_priv(dev); - strlcpy(info->driver, DRV_NAME, sizeof(info->driver)); - strlcpy(info->version, DRV_VERSION, sizeof(info->version)); - strlcpy(info->bus_info, pci_name(sky2->hw->pdev), + strscpy(info->driver, DRV_NAME, sizeof(info->driver)); + strscpy(info->version, DRV_VERSION, sizeof(info->version)); + strscpy(info->bus_info, pci_name(sky2->hw->pdev), sizeof(info->bus_info)); } @@ -4937,7 +4937,7 @@ static int sky2_probe(struct pci_dev *pdev, const struct pci_device_id *ent) } } - netif_napi_add(dev, &hw->napi, sky2_poll, NAPI_POLL_WEIGHT); + netif_napi_add(dev, &hw->napi, sky2_poll); err = register_netdev(dev); if (err) { diff --git a/drivers/net/ethernet/mediatek/mtk_eth_soc.c b/drivers/net/ethernet/mediatek/mtk_eth_soc.c index b344632beadd..4fba7cb0144b 100644 --- a/drivers/net/ethernet/mediatek/mtk_eth_soc.c +++ b/drivers/net/ethernet/mediatek/mtk_eth_soc.c @@ -73,6 +73,12 @@ static const struct mtk_reg_map mtk_reg_map = { .fq_blen = 0x1b2c, }, .gdm1_cnt = 0x2400, + .gdma_to_ppe = 0x4444, + .ppe_base = 0x0c00, + .wdma_base = { + [0] = 0x2800, + [1] = 0x2c00, + }, }; static const struct mtk_reg_map mt7628_reg_map = { @@ -126,6 +132,12 @@ static const struct mtk_reg_map mt7986_reg_map = { .fq_blen = 0x472c, }, .gdm1_cnt = 0x1c00, + .gdma_to_ppe = 0x3333, + .ppe_base = 0x2000, + .wdma_base = { + [0] = 0x4800, + [1] = 0x4c00, + }, }; /* strings used by ethtool */ @@ -1573,8 +1585,8 @@ static int mtk_xdp_submit_frame(struct mtk_eth *eth, struct xdp_frame *xdpf, .last = !xdp_frame_has_frags(xdpf), }; int err, index = 0, n_desc = 1, nr_frags; - struct mtk_tx_dma *htxd, *txd, *txd_pdma; struct mtk_tx_buf *htx_buf, *tx_buf; + struct mtk_tx_dma *htxd, *txd; void *data = xdpf->data; if (unlikely(test_bit(MTK_RESETTING, ð->state))) @@ -1608,7 +1620,6 @@ static int mtk_xdp_submit_frame(struct mtk_eth *eth, struct xdp_frame *xdpf, if (MTK_HAS_CAPS(soc->caps, MTK_QDMA) || (index & 0x1)) { txd = mtk_qdma_phys_to_virt(ring, txd->txd2); - txd_pdma = qdma_to_pdma(ring, txd); if (txd == ring->last_free) goto unmap; @@ -1629,7 +1640,8 @@ static int mtk_xdp_submit_frame(struct mtk_eth *eth, struct xdp_frame *xdpf, htx_buf->data = xdpf; if (!MTK_HAS_CAPS(soc->caps, MTK_QDMA)) { - txd_pdma = qdma_to_pdma(ring, txd); + struct mtk_tx_dma *txd_pdma = qdma_to_pdma(ring, txd); + if (index & 1) txd_pdma->txd2 |= TX_DMA_LS0; else @@ -1660,13 +1672,15 @@ static int mtk_xdp_submit_frame(struct mtk_eth *eth, struct xdp_frame *xdpf, unmap: while (htxd != txd) { - txd_pdma = qdma_to_pdma(ring, htxd); tx_buf = mtk_desc_to_tx_buf(ring, htxd, soc->txrx.txd_size); mtk_tx_unmap(eth, tx_buf, NULL, false); htxd->txd3 = TX_DMA_LS0 | TX_DMA_OWNER_CPU; - if (!MTK_HAS_CAPS(soc->caps, MTK_QDMA)) + if (!MTK_HAS_CAPS(soc->caps, MTK_QDMA)) { + struct mtk_tx_dma *txd_pdma = qdma_to_pdma(ring, htxd); + txd_pdma->txd2 = TX_DMA_DESP2_DEF; + } htxd = mtk_qdma_phys_to_virt(ring, htxd->txd2); } @@ -1892,12 +1906,14 @@ static int mtk_poll_rx(struct napi_struct *napi, int budget, bytes += skb->len; if (MTK_HAS_CAPS(eth->soc->caps, MTK_NETSYS_V2)) { + reason = FIELD_GET(MTK_RXD5_PPE_CPU_REASON, trxd.rxd5); hash = trxd.rxd5 & MTK_RXD5_FOE_ENTRY; if (hash != MTK_RXD5_FOE_ENTRY) skb_set_hash(skb, jhash_1word(hash, 0), PKT_HASH_TYPE_L4); rxdcsum = &trxd.rxd3; } else { + reason = FIELD_GET(MTK_RXD4_PPE_CPU_REASON, trxd.rxd4); hash = trxd.rxd4 & MTK_RXD4_FOE_ENTRY; if (hash != MTK_RXD4_FOE_ENTRY) skb_set_hash(skb, jhash_1word(hash, 0), @@ -1911,9 +1927,8 @@ static int mtk_poll_rx(struct napi_struct *napi, int budget, skb_checksum_none_assert(skb); skb->protocol = eth_type_trans(skb, netdev); - reason = FIELD_GET(MTK_RXD4_PPE_CPU_REASON, trxd.rxd4); if (reason == MTK_PPE_CPU_REASON_HIT_UNBIND_RATE_REACHED) - mtk_ppe_check_skb(eth->ppe, skb, hash); + mtk_ppe_check_skb(eth->ppe[0], skb, hash); if (netdev->features & NETIF_F_HW_VLAN_CTAG_RX) { if (MTK_HAS_CAPS(eth->soc->caps, MTK_NETSYS_V2)) { @@ -2976,21 +2991,25 @@ static int mtk_open(struct net_device *dev) /* we run 2 netdevs on the same dma ring so we only bring it up once */ if (!refcount_read(ð->dma_refcnt)) { - u32 gdm_config = MTK_GDMA_TO_PDMA; + const struct mtk_soc_data *soc = eth->soc; + u32 gdm_config; + int i; err = mtk_start_dma(eth); if (err) return err; - if (eth->soc->offload_version && mtk_ppe_start(eth->ppe) == 0) - gdm_config = MTK_GDMA_TO_PPE; + for (i = 0; i < ARRAY_SIZE(eth->ppe); i++) + mtk_ppe_start(eth->ppe[i]); + gdm_config = soc->offload_version ? soc->reg_map->gdma_to_ppe + : MTK_GDMA_TO_PDMA; mtk_gdm_config(eth, gdm_config); napi_enable(ð->tx_napi); napi_enable(ð->rx_napi); mtk_tx_irq_enable(eth, MTK_TX_DONE_INT); - mtk_rx_irq_enable(eth, eth->soc->txrx.rx_irq_done_mask); + mtk_rx_irq_enable(eth, soc->txrx.rx_irq_done_mask); refcount_set(ð->dma_refcnt, 1); } else @@ -3028,6 +3047,7 @@ static int mtk_stop(struct net_device *dev) { struct mtk_mac *mac = netdev_priv(dev); struct mtk_eth *eth = mac->hw; + int i; phylink_stop(mac->phylink); @@ -3055,8 +3075,8 @@ static int mtk_stop(struct net_device *dev) mtk_dma_free(eth); - if (eth->soc->offload_version) - mtk_ppe_stop(eth->ppe); + for (i = 0; i < ARRAY_SIZE(eth->ppe); i++) + mtk_ppe_stop(eth->ppe[i]); return 0; } @@ -3556,8 +3576,8 @@ static void mtk_get_drvinfo(struct net_device *dev, { struct mtk_mac *mac = netdev_priv(dev); - strlcpy(info->driver, mac->hw->dev->driver->name, sizeof(info->driver)); - strlcpy(info->bus_info, dev_name(mac->hw->dev), sizeof(info->bus_info)); + strscpy(info->driver, mac->hw->dev->driver->name, sizeof(info->driver)); + strscpy(info->bus_info, dev_name(mac->hw->dev), sizeof(info->bus_info)); info->n_stats = ARRAY_SIZE(mtk_ethtool_stats); } @@ -3925,6 +3945,7 @@ void mtk_eth_set_dma_device(struct mtk_eth *eth, struct device *dma_dev) static int mtk_probe(struct platform_device *pdev) { + struct resource *res = NULL; struct device_node *mac_np; struct mtk_eth *eth; int err, i; @@ -4005,20 +4026,31 @@ static int mtk_probe(struct platform_device *pdev) } } - for (i = 0;; i++) { - struct device_node *np = of_parse_phandle(pdev->dev.of_node, - "mediatek,wed", i); - static const u32 wdma_regs[] = { - MTK_WDMA0_BASE, - MTK_WDMA1_BASE - }; - void __iomem *wdma; - - if (!np || i >= ARRAY_SIZE(wdma_regs)) - break; + if (MTK_HAS_CAPS(eth->soc->caps, MTK_NETSYS_V2)) { + res = platform_get_resource(pdev, IORESOURCE_MEM, 0); + if (!res) + return -EINVAL; + } - wdma = eth->base + wdma_regs[i]; - mtk_wed_add_hw(np, eth, wdma, i); + if (eth->soc->offload_version) { + for (i = 0;; i++) { + struct device_node *np; + phys_addr_t wdma_phy; + u32 wdma_base; + + if (i >= ARRAY_SIZE(eth->soc->reg_map->wdma_base)) + break; + + np = of_parse_phandle(pdev->dev.of_node, + "mediatek,wed", i); + if (!np) + break; + + wdma_base = eth->soc->reg_map->wdma_base[i]; + wdma_phy = res ? res->start + wdma_base : 0; + mtk_wed_add_hw(np, eth, eth->base + wdma_base, + wdma_phy, i); + } } for (i = 0; i < 3; i++) { @@ -4096,10 +4128,19 @@ static int mtk_probe(struct platform_device *pdev) } if (eth->soc->offload_version) { - eth->ppe = mtk_ppe_init(eth, eth->base + MTK_ETH_PPE_BASE, 2); - if (!eth->ppe) { - err = -ENOMEM; - goto err_free_dev; + u32 num_ppe; + + num_ppe = MTK_HAS_CAPS(eth->soc->caps, MTK_NETSYS_V2) ? 2 : 1; + num_ppe = min_t(u32, ARRAY_SIZE(eth->ppe), num_ppe); + for (i = 0; i < num_ppe; i++) { + u32 ppe_addr = eth->soc->reg_map->ppe_base + i * 0x400; + + eth->ppe[i] = mtk_ppe_init(eth, eth->base + ppe_addr, + eth->soc->offload_version, i); + if (!eth->ppe[i]) { + err = -ENOMEM; + goto err_free_dev; + } } err = mtk_eth_offload_init(eth); @@ -4125,10 +4166,8 @@ static int mtk_probe(struct platform_device *pdev) * for NAPI to work */ init_dummy_netdev(ð->dummy_dev); - netif_napi_add(ð->dummy_dev, ð->tx_napi, mtk_napi_tx, - NAPI_POLL_WEIGHT); - netif_napi_add(ð->dummy_dev, ð->rx_napi, mtk_napi_rx, - NAPI_POLL_WEIGHT); + netif_napi_add(ð->dummy_dev, ð->tx_napi, mtk_napi_tx); + netif_napi_add(ð->dummy_dev, ð->rx_napi, mtk_napi_rx); platform_set_drvdata(pdev, eth); @@ -4192,6 +4231,8 @@ static const struct mtk_soc_data mt7621_data = { .required_clks = MT7621_CLKS_BITMAP, .required_pctl = false, .offload_version = 2, + .hash_offset = 2, + .foe_entry_size = sizeof(struct mtk_foe_entry) - 16, .txrx = { .txd_size = sizeof(struct mtk_tx_dma), .rxd_size = sizeof(struct mtk_rx_dma), @@ -4210,6 +4251,8 @@ static const struct mtk_soc_data mt7622_data = { .required_clks = MT7622_CLKS_BITMAP, .required_pctl = false, .offload_version = 2, + .hash_offset = 2, + .foe_entry_size = sizeof(struct mtk_foe_entry) - 16, .txrx = { .txd_size = sizeof(struct mtk_tx_dma), .rxd_size = sizeof(struct mtk_rx_dma), @@ -4227,6 +4270,8 @@ static const struct mtk_soc_data mt7623_data = { .required_clks = MT7623_CLKS_BITMAP, .required_pctl = true, .offload_version = 2, + .hash_offset = 2, + .foe_entry_size = sizeof(struct mtk_foe_entry) - 16, .txrx = { .txd_size = sizeof(struct mtk_tx_dma), .rxd_size = sizeof(struct mtk_rx_dma), @@ -4258,8 +4303,11 @@ static const struct mtk_soc_data mt7986_data = { .reg_map = &mt7986_reg_map, .ana_rgc3 = 0x128, .caps = MT7986_CAPS, + .hw_features = MTK_HW_FEATURES, .required_clks = MT7986_CLKS_BITMAP, .required_pctl = false, + .hash_offset = 4, + .foe_entry_size = sizeof(struct mtk_foe_entry), .txrx = { .txd_size = sizeof(struct mtk_tx_dma_v2), .rxd_size = sizeof(struct mtk_rx_dma_v2), diff --git a/drivers/net/ethernet/mediatek/mtk_eth_soc.h b/drivers/net/ethernet/mediatek/mtk_eth_soc.h index 0f9668a4079d..b52f3b0177ef 100644 --- a/drivers/net/ethernet/mediatek/mtk_eth_soc.h +++ b/drivers/net/ethernet/mediatek/mtk_eth_soc.h @@ -105,7 +105,6 @@ #define MTK_GDMA_TCS_EN BIT(21) #define MTK_GDMA_UCS_EN BIT(20) #define MTK_GDMA_TO_PDMA 0x0 -#define MTK_GDMA_TO_PPE 0x4444 #define MTK_GDMA_DROP_ALL 0x7777 /* Unicast Filter MAC Address Register - Low */ @@ -269,9 +268,6 @@ #define TX_DMA_FPORT_MASK_V2 0xf #define TX_DMA_SWC_V2 BIT(30) -#define MTK_WDMA0_BASE 0x2800 -#define MTK_WDMA1_BASE 0x2c00 - /* QDMA descriptor txd4 */ #define TX_DMA_CHKSUM (0x7 << 29) #define TX_DMA_TSO BIT(28) @@ -955,6 +951,9 @@ struct mtk_reg_map { u32 fq_blen; /* fq free page buffer length */ } qdma; u32 gdm1_cnt; + u32 gdma_to_ppe; + u32 ppe_base; + u32 wdma_base[2]; }; /* struct mtk_eth_data - This is the structure holding all differences @@ -968,6 +967,8 @@ struct mtk_reg_map { * the target SoC * @required_pctl A bool value to show whether the SoC requires * the extra setup for those pins used by GMAC. + * @hash_offset Flow table hash offset. + * @foe_entry_size Foe table entry size. * @txd_size Tx DMA descriptor size. * @rxd_size Rx DMA descriptor size. * @rx_irq_done_mask Rx irq done register mask. @@ -982,6 +983,8 @@ struct mtk_soc_data { u32 required_clks; bool required_pctl; u8 offload_version; + u8 hash_offset; + u16 foe_entry_size; netdev_features_t hw_features; struct { u32 txd_size; @@ -1111,7 +1114,7 @@ struct mtk_eth { int ip_align; - struct mtk_ppe *ppe; + struct mtk_ppe *ppe[2]; struct rhashtable flow_table; struct bpf_prog __rcu *prog; @@ -1142,6 +1145,86 @@ struct mtk_mac { /* the struct describing the SoC. these are declared in the soc_xyz.c files */ extern const struct of_device_id of_mtk_match[]; +static inline struct mtk_foe_entry * +mtk_foe_get_entry(struct mtk_ppe *ppe, u16 hash) +{ + const struct mtk_soc_data *soc = ppe->eth->soc; + + return ppe->foe_table + hash * soc->foe_entry_size; +} + +static inline u32 mtk_get_ib1_ts_mask(struct mtk_eth *eth) +{ + if (MTK_HAS_CAPS(eth->soc->caps, MTK_NETSYS_V2)) + return MTK_FOE_IB1_BIND_TIMESTAMP_V2; + + return MTK_FOE_IB1_BIND_TIMESTAMP; +} + +static inline u32 mtk_get_ib1_ppoe_mask(struct mtk_eth *eth) +{ + if (MTK_HAS_CAPS(eth->soc->caps, MTK_NETSYS_V2)) + return MTK_FOE_IB1_BIND_PPPOE_V2; + + return MTK_FOE_IB1_BIND_PPPOE; +} + +static inline u32 mtk_get_ib1_vlan_tag_mask(struct mtk_eth *eth) +{ + if (MTK_HAS_CAPS(eth->soc->caps, MTK_NETSYS_V2)) + return MTK_FOE_IB1_BIND_VLAN_TAG_V2; + + return MTK_FOE_IB1_BIND_VLAN_TAG; +} + +static inline u32 mtk_get_ib1_vlan_layer_mask(struct mtk_eth *eth) +{ + if (MTK_HAS_CAPS(eth->soc->caps, MTK_NETSYS_V2)) + return MTK_FOE_IB1_BIND_VLAN_LAYER_V2; + + return MTK_FOE_IB1_BIND_VLAN_LAYER; +} + +static inline u32 mtk_prep_ib1_vlan_layer(struct mtk_eth *eth, u32 val) +{ + if (MTK_HAS_CAPS(eth->soc->caps, MTK_NETSYS_V2)) + return FIELD_PREP(MTK_FOE_IB1_BIND_VLAN_LAYER_V2, val); + + return FIELD_PREP(MTK_FOE_IB1_BIND_VLAN_LAYER, val); +} + +static inline u32 mtk_get_ib1_vlan_layer(struct mtk_eth *eth, u32 val) +{ + if (MTK_HAS_CAPS(eth->soc->caps, MTK_NETSYS_V2)) + return FIELD_GET(MTK_FOE_IB1_BIND_VLAN_LAYER_V2, val); + + return FIELD_GET(MTK_FOE_IB1_BIND_VLAN_LAYER, val); +} + +static inline u32 mtk_get_ib1_pkt_type_mask(struct mtk_eth *eth) +{ + if (MTK_HAS_CAPS(eth->soc->caps, MTK_NETSYS_V2)) + return MTK_FOE_IB1_PACKET_TYPE_V2; + + return MTK_FOE_IB1_PACKET_TYPE; +} + +static inline u32 mtk_get_ib1_pkt_type(struct mtk_eth *eth, u32 val) +{ + if (MTK_HAS_CAPS(eth->soc->caps, MTK_NETSYS_V2)) + return FIELD_GET(MTK_FOE_IB1_PACKET_TYPE_V2, val); + + return FIELD_GET(MTK_FOE_IB1_PACKET_TYPE, val); +} + +static inline u32 mtk_get_ib2_multicast_mask(struct mtk_eth *eth) +{ + if (MTK_HAS_CAPS(eth->soc->caps, MTK_NETSYS_V2)) + return MTK_FOE_IB2_MULTICAST_V2; + + return MTK_FOE_IB2_MULTICAST; +} + /* read the hardware status register */ void mtk_stats_update_mac(struct mtk_mac *mac); diff --git a/drivers/net/ethernet/mediatek/mtk_ppe.c b/drivers/net/ethernet/mediatek/mtk_ppe.c index cfe804bc8d20..ae00e572390d 100644 --- a/drivers/net/ethernet/mediatek/mtk_ppe.c +++ b/drivers/net/ethernet/mediatek/mtk_ppe.c @@ -56,7 +56,7 @@ static u32 ppe_clear(struct mtk_ppe *ppe, u32 reg, u32 val) static u32 mtk_eth_timestamp(struct mtk_eth *eth) { - return mtk_r32(eth, 0x0010) & MTK_FOE_IB1_BIND_TIMESTAMP; + return mtk_r32(eth, 0x0010) & mtk_get_ib1_ts_mask(eth); } static int mtk_ppe_wait_busy(struct mtk_ppe *ppe) @@ -88,12 +88,12 @@ static void mtk_ppe_cache_enable(struct mtk_ppe *ppe, bool enable) enable * MTK_PPE_CACHE_CTL_EN); } -static u32 mtk_ppe_hash_entry(struct mtk_foe_entry *e) +static u32 mtk_ppe_hash_entry(struct mtk_eth *eth, struct mtk_foe_entry *e) { u32 hv1, hv2, hv3; u32 hash; - switch (FIELD_GET(MTK_FOE_IB1_PACKET_TYPE, e->ib1)) { + switch (mtk_get_ib1_pkt_type(eth, e->ib1)) { case MTK_PPE_PKT_TYPE_IPV4_ROUTE: case MTK_PPE_PKT_TYPE_IPV4_HNAPT: hv1 = e->ipv4.orig.ports; @@ -122,16 +122,16 @@ static u32 mtk_ppe_hash_entry(struct mtk_foe_entry *e) hash = (hash >> 24) | ((hash & 0xffffff) << 8); hash ^= hv1 ^ hv2 ^ hv3; hash ^= hash >> 16; - hash <<= 1; + hash <<= (ffs(eth->soc->hash_offset) - 1); hash &= MTK_PPE_ENTRIES - 1; return hash; } static inline struct mtk_foe_mac_info * -mtk_foe_entry_l2(struct mtk_foe_entry *entry) +mtk_foe_entry_l2(struct mtk_eth *eth, struct mtk_foe_entry *entry) { - int type = FIELD_GET(MTK_FOE_IB1_PACKET_TYPE, entry->ib1); + int type = mtk_get_ib1_pkt_type(eth, entry->ib1); if (type == MTK_PPE_PKT_TYPE_BRIDGE) return &entry->bridge.l2; @@ -143,9 +143,9 @@ mtk_foe_entry_l2(struct mtk_foe_entry *entry) } static inline u32 * -mtk_foe_entry_ib2(struct mtk_foe_entry *entry) +mtk_foe_entry_ib2(struct mtk_eth *eth, struct mtk_foe_entry *entry) { - int type = FIELD_GET(MTK_FOE_IB1_PACKET_TYPE, entry->ib1); + int type = mtk_get_ib1_pkt_type(eth, entry->ib1); if (type == MTK_PPE_PKT_TYPE_BRIDGE) return &entry->bridge.ib2; @@ -156,27 +156,38 @@ mtk_foe_entry_ib2(struct mtk_foe_entry *entry) return &entry->ipv4.ib2; } -int mtk_foe_entry_prepare(struct mtk_foe_entry *entry, int type, int l4proto, - u8 pse_port, u8 *src_mac, u8 *dest_mac) +int mtk_foe_entry_prepare(struct mtk_eth *eth, struct mtk_foe_entry *entry, + int type, int l4proto, u8 pse_port, u8 *src_mac, + u8 *dest_mac) { struct mtk_foe_mac_info *l2; u32 ports_pad, val; memset(entry, 0, sizeof(*entry)); - val = FIELD_PREP(MTK_FOE_IB1_STATE, MTK_FOE_STATE_BIND) | - FIELD_PREP(MTK_FOE_IB1_PACKET_TYPE, type) | - FIELD_PREP(MTK_FOE_IB1_UDP, l4proto == IPPROTO_UDP) | - MTK_FOE_IB1_BIND_TTL | - MTK_FOE_IB1_BIND_CACHE; - entry->ib1 = val; + if (MTK_HAS_CAPS(eth->soc->caps, MTK_NETSYS_V2)) { + val = FIELD_PREP(MTK_FOE_IB1_STATE, MTK_FOE_STATE_BIND) | + FIELD_PREP(MTK_FOE_IB1_PACKET_TYPE_V2, type) | + FIELD_PREP(MTK_FOE_IB1_UDP, l4proto == IPPROTO_UDP) | + MTK_FOE_IB1_BIND_CACHE_V2 | MTK_FOE_IB1_BIND_TTL_V2; + entry->ib1 = val; - val = FIELD_PREP(MTK_FOE_IB2_PORT_MG, 0x3f) | - FIELD_PREP(MTK_FOE_IB2_PORT_AG, 0x1f) | - FIELD_PREP(MTK_FOE_IB2_DEST_PORT, pse_port); + val = FIELD_PREP(MTK_FOE_IB2_DEST_PORT_V2, pse_port) | + FIELD_PREP(MTK_FOE_IB2_PORT_AG_V2, 0xf); + } else { + val = FIELD_PREP(MTK_FOE_IB1_STATE, MTK_FOE_STATE_BIND) | + FIELD_PREP(MTK_FOE_IB1_PACKET_TYPE, type) | + FIELD_PREP(MTK_FOE_IB1_UDP, l4proto == IPPROTO_UDP) | + MTK_FOE_IB1_BIND_CACHE | MTK_FOE_IB1_BIND_TTL; + entry->ib1 = val; + + val = FIELD_PREP(MTK_FOE_IB2_DEST_PORT, pse_port) | + FIELD_PREP(MTK_FOE_IB2_PORT_MG, 0x3f) | + FIELD_PREP(MTK_FOE_IB2_PORT_AG, 0x1f); + } if (is_multicast_ether_addr(dest_mac)) - val |= MTK_FOE_IB2_MULTICAST; + val |= mtk_get_ib2_multicast_mask(eth); ports_pad = 0xa5a5a500 | (l4proto & 0xff); if (type == MTK_PPE_PKT_TYPE_IPV4_ROUTE) @@ -210,24 +221,30 @@ int mtk_foe_entry_prepare(struct mtk_foe_entry *entry, int type, int l4proto, return 0; } -int mtk_foe_entry_set_pse_port(struct mtk_foe_entry *entry, u8 port) +int mtk_foe_entry_set_pse_port(struct mtk_eth *eth, + struct mtk_foe_entry *entry, u8 port) { - u32 *ib2 = mtk_foe_entry_ib2(entry); - u32 val; + u32 *ib2 = mtk_foe_entry_ib2(eth, entry); + u32 val = *ib2; - val = *ib2; - val &= ~MTK_FOE_IB2_DEST_PORT; - val |= FIELD_PREP(MTK_FOE_IB2_DEST_PORT, port); + if (MTK_HAS_CAPS(eth->soc->caps, MTK_NETSYS_V2)) { + val &= ~MTK_FOE_IB2_DEST_PORT_V2; + val |= FIELD_PREP(MTK_FOE_IB2_DEST_PORT_V2, port); + } else { + val &= ~MTK_FOE_IB2_DEST_PORT; + val |= FIELD_PREP(MTK_FOE_IB2_DEST_PORT, port); + } *ib2 = val; return 0; } -int mtk_foe_entry_set_ipv4_tuple(struct mtk_foe_entry *entry, bool egress, +int mtk_foe_entry_set_ipv4_tuple(struct mtk_eth *eth, + struct mtk_foe_entry *entry, bool egress, __be32 src_addr, __be16 src_port, __be32 dest_addr, __be16 dest_port) { - int type = FIELD_GET(MTK_FOE_IB1_PACKET_TYPE, entry->ib1); + int type = mtk_get_ib1_pkt_type(eth, entry->ib1); struct mtk_ipv4_tuple *t; switch (type) { @@ -262,11 +279,12 @@ int mtk_foe_entry_set_ipv4_tuple(struct mtk_foe_entry *entry, bool egress, return 0; } -int mtk_foe_entry_set_ipv6_tuple(struct mtk_foe_entry *entry, +int mtk_foe_entry_set_ipv6_tuple(struct mtk_eth *eth, + struct mtk_foe_entry *entry, __be32 *src_addr, __be16 src_port, __be32 *dest_addr, __be16 dest_port) { - int type = FIELD_GET(MTK_FOE_IB1_PACKET_TYPE, entry->ib1); + int type = mtk_get_ib1_pkt_type(eth, entry->ib1); u32 *src, *dest; int i; @@ -297,39 +315,41 @@ int mtk_foe_entry_set_ipv6_tuple(struct mtk_foe_entry *entry, return 0; } -int mtk_foe_entry_set_dsa(struct mtk_foe_entry *entry, int port) +int mtk_foe_entry_set_dsa(struct mtk_eth *eth, struct mtk_foe_entry *entry, + int port) { - struct mtk_foe_mac_info *l2 = mtk_foe_entry_l2(entry); + struct mtk_foe_mac_info *l2 = mtk_foe_entry_l2(eth, entry); l2->etype = BIT(port); - if (!(entry->ib1 & MTK_FOE_IB1_BIND_VLAN_LAYER)) - entry->ib1 |= FIELD_PREP(MTK_FOE_IB1_BIND_VLAN_LAYER, 1); + if (!(entry->ib1 & mtk_get_ib1_vlan_layer_mask(eth))) + entry->ib1 |= mtk_prep_ib1_vlan_layer(eth, 1); else l2->etype |= BIT(8); - entry->ib1 &= ~MTK_FOE_IB1_BIND_VLAN_TAG; + entry->ib1 &= ~mtk_get_ib1_vlan_tag_mask(eth); return 0; } -int mtk_foe_entry_set_vlan(struct mtk_foe_entry *entry, int vid) +int mtk_foe_entry_set_vlan(struct mtk_eth *eth, struct mtk_foe_entry *entry, + int vid) { - struct mtk_foe_mac_info *l2 = mtk_foe_entry_l2(entry); + struct mtk_foe_mac_info *l2 = mtk_foe_entry_l2(eth, entry); - switch (FIELD_GET(MTK_FOE_IB1_BIND_VLAN_LAYER, entry->ib1)) { + switch (mtk_get_ib1_vlan_layer(eth, entry->ib1)) { case 0: - entry->ib1 |= MTK_FOE_IB1_BIND_VLAN_TAG | - FIELD_PREP(MTK_FOE_IB1_BIND_VLAN_LAYER, 1); + entry->ib1 |= mtk_get_ib1_vlan_tag_mask(eth) | + mtk_prep_ib1_vlan_layer(eth, 1); l2->vlan1 = vid; return 0; case 1: - if (!(entry->ib1 & MTK_FOE_IB1_BIND_VLAN_TAG)) { + if (!(entry->ib1 & mtk_get_ib1_vlan_tag_mask(eth))) { l2->vlan1 = vid; l2->etype |= BIT(8); } else { l2->vlan2 = vid; - entry->ib1 += FIELD_PREP(MTK_FOE_IB1_BIND_VLAN_LAYER, 1); + entry->ib1 += mtk_prep_ib1_vlan_layer(eth, 1); } return 0; default: @@ -337,34 +357,42 @@ int mtk_foe_entry_set_vlan(struct mtk_foe_entry *entry, int vid) } } -int mtk_foe_entry_set_pppoe(struct mtk_foe_entry *entry, int sid) +int mtk_foe_entry_set_pppoe(struct mtk_eth *eth, struct mtk_foe_entry *entry, + int sid) { - struct mtk_foe_mac_info *l2 = mtk_foe_entry_l2(entry); + struct mtk_foe_mac_info *l2 = mtk_foe_entry_l2(eth, entry); - if (!(entry->ib1 & MTK_FOE_IB1_BIND_VLAN_LAYER) || - (entry->ib1 & MTK_FOE_IB1_BIND_VLAN_TAG)) + if (!(entry->ib1 & mtk_get_ib1_vlan_layer_mask(eth)) || + (entry->ib1 & mtk_get_ib1_vlan_tag_mask(eth))) l2->etype = ETH_P_PPP_SES; - entry->ib1 |= MTK_FOE_IB1_BIND_PPPOE; + entry->ib1 |= mtk_get_ib1_ppoe_mask(eth); l2->pppoe_id = sid; return 0; } -int mtk_foe_entry_set_wdma(struct mtk_foe_entry *entry, int wdma_idx, int txq, - int bss, int wcid) +int mtk_foe_entry_set_wdma(struct mtk_eth *eth, struct mtk_foe_entry *entry, + int wdma_idx, int txq, int bss, int wcid) { - struct mtk_foe_mac_info *l2 = mtk_foe_entry_l2(entry); - u32 *ib2 = mtk_foe_entry_ib2(entry); - - *ib2 &= ~MTK_FOE_IB2_PORT_MG; - *ib2 |= MTK_FOE_IB2_WDMA_WINFO; - if (wdma_idx) - *ib2 |= MTK_FOE_IB2_WDMA_DEVIDX; + struct mtk_foe_mac_info *l2 = mtk_foe_entry_l2(eth, entry); + u32 *ib2 = mtk_foe_entry_ib2(eth, entry); - l2->vlan2 = FIELD_PREP(MTK_FOE_VLAN2_WINFO_BSS, bss) | - FIELD_PREP(MTK_FOE_VLAN2_WINFO_WCID, wcid) | - FIELD_PREP(MTK_FOE_VLAN2_WINFO_RING, txq); + if (MTK_HAS_CAPS(eth->soc->caps, MTK_NETSYS_V2)) { + *ib2 &= ~MTK_FOE_IB2_PORT_MG_V2; + *ib2 |= FIELD_PREP(MTK_FOE_IB2_RX_IDX, txq) | + MTK_FOE_IB2_WDMA_WINFO_V2; + l2->winfo = FIELD_PREP(MTK_FOE_WINFO_WCID, wcid) | + FIELD_PREP(MTK_FOE_WINFO_BSS, bss); + } else { + *ib2 &= ~MTK_FOE_IB2_PORT_MG; + *ib2 |= MTK_FOE_IB2_WDMA_WINFO; + if (wdma_idx) + *ib2 |= MTK_FOE_IB2_WDMA_DEVIDX; + l2->vlan2 = FIELD_PREP(MTK_FOE_VLAN2_WINFO_BSS, bss) | + FIELD_PREP(MTK_FOE_VLAN2_WINFO_WCID, wcid) | + FIELD_PREP(MTK_FOE_VLAN2_WINFO_RING, txq); + } return 0; } @@ -376,14 +404,15 @@ static inline bool mtk_foe_entry_usable(struct mtk_foe_entry *entry) } static bool -mtk_flow_entry_match(struct mtk_flow_entry *entry, struct mtk_foe_entry *data) +mtk_flow_entry_match(struct mtk_eth *eth, struct mtk_flow_entry *entry, + struct mtk_foe_entry *data) { int type, len; if ((data->ib1 ^ entry->data.ib1) & MTK_FOE_IB1_UDP) return false; - type = FIELD_GET(MTK_FOE_IB1_PACKET_TYPE, entry->data.ib1); + type = mtk_get_ib1_pkt_type(eth, entry->data.ib1); if (type > MTK_PPE_PKT_TYPE_IPV4_DSLITE) len = offsetof(struct mtk_foe_entry, ipv6._rsv); else @@ -410,9 +439,10 @@ __mtk_foe_entry_clear(struct mtk_ppe *ppe, struct mtk_flow_entry *entry) hlist_del_init(&entry->list); if (entry->hash != 0xffff) { - ppe->foe_table[entry->hash].ib1 &= ~MTK_FOE_IB1_STATE; - ppe->foe_table[entry->hash].ib1 |= FIELD_PREP(MTK_FOE_IB1_STATE, - MTK_FOE_STATE_UNBIND); + struct mtk_foe_entry *hwe = mtk_foe_get_entry(ppe, entry->hash); + + hwe->ib1 &= ~MTK_FOE_IB1_STATE; + hwe->ib1 |= FIELD_PREP(MTK_FOE_IB1_STATE, MTK_FOE_STATE_INVALID); dma_wmb(); } entry->hash = 0xffff; @@ -426,14 +456,12 @@ __mtk_foe_entry_clear(struct mtk_ppe *ppe, struct mtk_flow_entry *entry) static int __mtk_foe_entry_idle_time(struct mtk_ppe *ppe, u32 ib1) { - u16 timestamp; - u16 now; - - now = mtk_eth_timestamp(ppe->eth) & MTK_FOE_IB1_BIND_TIMESTAMP; - timestamp = ib1 & MTK_FOE_IB1_BIND_TIMESTAMP; + u32 ib1_ts_mask = mtk_get_ib1_ts_mask(ppe->eth); + u16 now = mtk_eth_timestamp(ppe->eth); + u16 timestamp = ib1 & ib1_ts_mask; if (timestamp > now) - return MTK_FOE_IB1_BIND_TIMESTAMP + 1 - timestamp + now; + return ib1_ts_mask + 1 - timestamp + now; else return now - timestamp; } @@ -441,6 +469,7 @@ static int __mtk_foe_entry_idle_time(struct mtk_ppe *ppe, u32 ib1) static void mtk_flow_entry_update_l2(struct mtk_ppe *ppe, struct mtk_flow_entry *entry) { + u32 ib1_ts_mask = mtk_get_ib1_ts_mask(ppe->eth); struct mtk_flow_entry *cur; struct mtk_foe_entry *hwe; struct hlist_node *tmp; @@ -451,7 +480,7 @@ mtk_flow_entry_update_l2(struct mtk_ppe *ppe, struct mtk_flow_entry *entry) int cur_idle; u32 ib1; - hwe = &ppe->foe_table[cur->hash]; + hwe = mtk_foe_get_entry(ppe, cur->hash); ib1 = READ_ONCE(hwe->ib1); if (FIELD_GET(MTK_FOE_IB1_STATE, ib1) != MTK_FOE_STATE_BIND) { @@ -465,16 +494,16 @@ mtk_flow_entry_update_l2(struct mtk_ppe *ppe, struct mtk_flow_entry *entry) continue; idle = cur_idle; - entry->data.ib1 &= ~MTK_FOE_IB1_BIND_TIMESTAMP; - entry->data.ib1 |= hwe->ib1 & MTK_FOE_IB1_BIND_TIMESTAMP; + entry->data.ib1 &= ~ib1_ts_mask; + entry->data.ib1 |= hwe->ib1 & ib1_ts_mask; } } static void mtk_flow_entry_update(struct mtk_ppe *ppe, struct mtk_flow_entry *entry) { + struct mtk_foe_entry foe = {}; struct mtk_foe_entry *hwe; - struct mtk_foe_entry foe; spin_lock_bh(&ppe_lock); @@ -486,9 +515,9 @@ mtk_flow_entry_update(struct mtk_ppe *ppe, struct mtk_flow_entry *entry) if (entry->hash == 0xffff) goto out; - hwe = &ppe->foe_table[entry->hash]; - memcpy(&foe, hwe, sizeof(foe)); - if (!mtk_flow_entry_match(entry, &foe)) { + hwe = mtk_foe_get_entry(ppe, entry->hash); + memcpy(&foe, hwe, ppe->eth->soc->foe_entry_size); + if (!mtk_flow_entry_match(ppe->eth, entry, &foe)) { entry->hash = 0xffff; goto out; } @@ -503,16 +532,22 @@ static void __mtk_foe_entry_commit(struct mtk_ppe *ppe, struct mtk_foe_entry *entry, u16 hash) { + struct mtk_eth *eth = ppe->eth; + u16 timestamp = mtk_eth_timestamp(eth); struct mtk_foe_entry *hwe; - u16 timestamp; - timestamp = mtk_eth_timestamp(ppe->eth); - timestamp &= MTK_FOE_IB1_BIND_TIMESTAMP; - entry->ib1 &= ~MTK_FOE_IB1_BIND_TIMESTAMP; - entry->ib1 |= FIELD_PREP(MTK_FOE_IB1_BIND_TIMESTAMP, timestamp); + if (MTK_HAS_CAPS(eth->soc->caps, MTK_NETSYS_V2)) { + entry->ib1 &= ~MTK_FOE_IB1_BIND_TIMESTAMP_V2; + entry->ib1 |= FIELD_PREP(MTK_FOE_IB1_BIND_TIMESTAMP_V2, + timestamp); + } else { + entry->ib1 &= ~MTK_FOE_IB1_BIND_TIMESTAMP; + entry->ib1 |= FIELD_PREP(MTK_FOE_IB1_BIND_TIMESTAMP, + timestamp); + } - hwe = &ppe->foe_table[hash]; - memcpy(&hwe->data, &entry->data, sizeof(hwe->data)); + hwe = mtk_foe_get_entry(ppe, hash); + memcpy(&hwe->data, &entry->data, eth->soc->foe_entry_size - sizeof(hwe->ib1)); wmb(); hwe->ib1 = entry->ib1; @@ -539,16 +574,17 @@ mtk_foe_entry_commit_l2(struct mtk_ppe *ppe, struct mtk_flow_entry *entry) int mtk_foe_entry_commit(struct mtk_ppe *ppe, struct mtk_flow_entry *entry) { - int type = FIELD_GET(MTK_FOE_IB1_PACKET_TYPE, entry->data.ib1); + const struct mtk_soc_data *soc = ppe->eth->soc; + int type = mtk_get_ib1_pkt_type(ppe->eth, entry->data.ib1); u32 hash; if (type == MTK_PPE_PKT_TYPE_BRIDGE) return mtk_foe_entry_commit_l2(ppe, entry); - hash = mtk_ppe_hash_entry(&entry->data); + hash = mtk_ppe_hash_entry(ppe->eth, &entry->data); entry->hash = 0xffff; spin_lock_bh(&ppe_lock); - hlist_add_head(&entry->list, &ppe->foe_flow[hash / 2]); + hlist_add_head(&entry->list, &ppe->foe_flow[hash / soc->hash_offset]); spin_unlock_bh(&ppe_lock); return 0; @@ -558,10 +594,11 @@ static void mtk_foe_entry_commit_subflow(struct mtk_ppe *ppe, struct mtk_flow_entry *entry, u16 hash) { + const struct mtk_soc_data *soc = ppe->eth->soc; struct mtk_flow_entry *flow_info; - struct mtk_foe_entry foe, *hwe; + struct mtk_foe_entry foe = {}, *hwe; struct mtk_foe_mac_info *l2; - u32 ib1_mask = MTK_FOE_IB1_PACKET_TYPE | MTK_FOE_IB1_UDP; + u32 ib1_mask = mtk_get_ib1_pkt_type_mask(ppe->eth) | MTK_FOE_IB1_UDP; int type; flow_info = kzalloc(offsetof(struct mtk_flow_entry, l2_data.end), @@ -572,32 +609,34 @@ mtk_foe_entry_commit_subflow(struct mtk_ppe *ppe, struct mtk_flow_entry *entry, flow_info->l2_data.base_flow = entry; flow_info->type = MTK_FLOW_TYPE_L2_SUBFLOW; flow_info->hash = hash; - hlist_add_head(&flow_info->list, &ppe->foe_flow[hash / 2]); + hlist_add_head(&flow_info->list, + &ppe->foe_flow[hash / soc->hash_offset]); hlist_add_head(&flow_info->l2_data.list, &entry->l2_flows); - hwe = &ppe->foe_table[hash]; - memcpy(&foe, hwe, sizeof(foe)); + hwe = mtk_foe_get_entry(ppe, hash); + memcpy(&foe, hwe, soc->foe_entry_size); foe.ib1 &= ib1_mask; foe.ib1 |= entry->data.ib1 & ~ib1_mask; - l2 = mtk_foe_entry_l2(&foe); + l2 = mtk_foe_entry_l2(ppe->eth, &foe); memcpy(l2, &entry->data.bridge.l2, sizeof(*l2)); - type = FIELD_GET(MTK_FOE_IB1_PACKET_TYPE, foe.ib1); + type = mtk_get_ib1_pkt_type(ppe->eth, foe.ib1); if (type == MTK_PPE_PKT_TYPE_IPV4_HNAPT) memcpy(&foe.ipv4.new, &foe.ipv4.orig, sizeof(foe.ipv4.new)); else if (type >= MTK_PPE_PKT_TYPE_IPV6_ROUTE_3T && l2->etype == ETH_P_IP) l2->etype = ETH_P_IPV6; - *mtk_foe_entry_ib2(&foe) = entry->data.bridge.ib2; + *mtk_foe_entry_ib2(ppe->eth, &foe) = entry->data.bridge.ib2; __mtk_foe_entry_commit(ppe, &foe, hash); } void __mtk_ppe_check_skb(struct mtk_ppe *ppe, struct sk_buff *skb, u16 hash) { - struct hlist_head *head = &ppe->foe_flow[hash / 2]; - struct mtk_foe_entry *hwe = &ppe->foe_table[hash]; + const struct mtk_soc_data *soc = ppe->eth->soc; + struct hlist_head *head = &ppe->foe_flow[hash / soc->hash_offset]; + struct mtk_foe_entry *hwe = mtk_foe_get_entry(ppe, hash); struct mtk_flow_entry *entry; struct mtk_foe_bridge key = {}; struct hlist_node *n; @@ -621,7 +660,7 @@ void __mtk_ppe_check_skb(struct mtk_ppe *ppe, struct sk_buff *skb, u16 hash) continue; } - if (found || !mtk_flow_entry_match(entry, hwe)) { + if (found || !mtk_flow_entry_match(ppe->eth, entry, hwe)) { if (entry->hash != 0xffff) entry->hash = 0xffff; continue; @@ -678,11 +717,13 @@ int mtk_foe_entry_idle_time(struct mtk_ppe *ppe, struct mtk_flow_entry *entry) } struct mtk_ppe *mtk_ppe_init(struct mtk_eth *eth, void __iomem *base, - int version) + int version, int index) { + const struct mtk_soc_data *soc = eth->soc; struct device *dev = eth->dev; - struct mtk_foe_entry *foe; struct mtk_ppe *ppe; + u32 foe_flow_size; + void *foe; ppe = devm_kzalloc(dev, sizeof(*ppe), GFP_KERNEL); if (!ppe) @@ -698,14 +739,21 @@ struct mtk_ppe *mtk_ppe_init(struct mtk_eth *eth, void __iomem *base, ppe->dev = dev; ppe->version = version; - foe = dmam_alloc_coherent(ppe->dev, MTK_PPE_ENTRIES * sizeof(*foe), + foe = dmam_alloc_coherent(ppe->dev, + MTK_PPE_ENTRIES * soc->foe_entry_size, &ppe->foe_phys, GFP_KERNEL); if (!foe) return NULL; ppe->foe_table = foe; - mtk_ppe_debugfs_init(ppe); + foe_flow_size = (MTK_PPE_ENTRIES / soc->hash_offset) * + sizeof(*ppe->foe_flow); + ppe->foe_flow = devm_kzalloc(dev, foe_flow_size, GFP_KERNEL); + if (!ppe->foe_flow) + return NULL; + + mtk_ppe_debugfs_init(ppe, index); return ppe; } @@ -715,21 +763,30 @@ static void mtk_ppe_init_foe_table(struct mtk_ppe *ppe) static const u8 skip[] = { 12, 25, 38, 51, 76, 89, 102 }; int i, k; - memset(ppe->foe_table, 0, MTK_PPE_ENTRIES * sizeof(*ppe->foe_table)); + memset(ppe->foe_table, 0, + MTK_PPE_ENTRIES * ppe->eth->soc->foe_entry_size); if (!IS_ENABLED(CONFIG_SOC_MT7621)) return; /* skip all entries that cross the 1024 byte boundary */ - for (i = 0; i < MTK_PPE_ENTRIES; i += 128) - for (k = 0; k < ARRAY_SIZE(skip); k++) - ppe->foe_table[i + skip[k]].ib1 |= MTK_FOE_IB1_STATIC; + for (i = 0; i < MTK_PPE_ENTRIES; i += 128) { + for (k = 0; k < ARRAY_SIZE(skip); k++) { + struct mtk_foe_entry *hwe; + + hwe = mtk_foe_get_entry(ppe, i + skip[k]); + hwe->ib1 |= MTK_FOE_IB1_STATIC; + } + } } -int mtk_ppe_start(struct mtk_ppe *ppe) +void mtk_ppe_start(struct mtk_ppe *ppe) { u32 val; + if (!ppe) + return; + mtk_ppe_init_foe_table(ppe); ppe_w32(ppe, MTK_PPE_TB_BASE, ppe->foe_phys); @@ -748,6 +805,8 @@ int mtk_ppe_start(struct mtk_ppe *ppe) MTK_PPE_SCAN_MODE_KEEPALIVE_AGE) | FIELD_PREP(MTK_PPE_TB_CFG_ENTRY_NUM, MTK_PPE_ENTRIES_SHIFT); + if (MTK_HAS_CAPS(ppe->eth->soc->caps, MTK_NETSYS_V2)) + val |= MTK_PPE_TB_CFG_INFO_SEL; ppe_w32(ppe, MTK_PPE_TB_CFG, val); ppe_w32(ppe, MTK_PPE_IP_PROTO_CHK, @@ -755,15 +814,21 @@ int mtk_ppe_start(struct mtk_ppe *ppe) mtk_ppe_cache_enable(ppe, true); - val = MTK_PPE_FLOW_CFG_IP4_TCP_FRAG | - MTK_PPE_FLOW_CFG_IP4_UDP_FRAG | - MTK_PPE_FLOW_CFG_IP6_3T_ROUTE | + val = MTK_PPE_FLOW_CFG_IP6_3T_ROUTE | MTK_PPE_FLOW_CFG_IP6_5T_ROUTE | MTK_PPE_FLOW_CFG_IP6_6RD | MTK_PPE_FLOW_CFG_IP4_NAT | MTK_PPE_FLOW_CFG_IP4_NAPT | MTK_PPE_FLOW_CFG_IP4_DSLITE | MTK_PPE_FLOW_CFG_IP4_NAT_FRAG; + if (MTK_HAS_CAPS(ppe->eth->soc->caps, MTK_NETSYS_V2)) + val |= MTK_PPE_MD_TOAP_BYP_CRSN0 | + MTK_PPE_MD_TOAP_BYP_CRSN1 | + MTK_PPE_MD_TOAP_BYP_CRSN2 | + MTK_PPE_FLOW_CFG_IP4_HASH_GRE_KEY; + else + val |= MTK_PPE_FLOW_CFG_IP4_TCP_FRAG | + MTK_PPE_FLOW_CFG_IP4_UDP_FRAG; ppe_w32(ppe, MTK_PPE_FLOW_CFG, val); val = FIELD_PREP(MTK_PPE_UNBIND_AGE_MIN_PACKETS, 1000) | @@ -798,7 +863,10 @@ int mtk_ppe_start(struct mtk_ppe *ppe) ppe_w32(ppe, MTK_PPE_DEFAULT_CPU_PORT, 0); - return 0; + if (MTK_HAS_CAPS(ppe->eth->soc->caps, MTK_NETSYS_V2)) { + ppe_w32(ppe, MTK_PPE_DEFAULT_CPU_PORT1, 0xcb777); + ppe_w32(ppe, MTK_PPE_SBW_CTRL, 0x7f); + } } int mtk_ppe_stop(struct mtk_ppe *ppe) @@ -806,9 +874,15 @@ int mtk_ppe_stop(struct mtk_ppe *ppe) u32 val; int i; - for (i = 0; i < MTK_PPE_ENTRIES; i++) - ppe->foe_table[i].ib1 = FIELD_PREP(MTK_FOE_IB1_STATE, - MTK_FOE_STATE_INVALID); + if (!ppe) + return 0; + + for (i = 0; i < MTK_PPE_ENTRIES; i++) { + struct mtk_foe_entry *hwe = mtk_foe_get_entry(ppe, i); + + hwe->ib1 = FIELD_PREP(MTK_FOE_IB1_STATE, + MTK_FOE_STATE_INVALID); + } mtk_ppe_cache_enable(ppe, false); diff --git a/drivers/net/ethernet/mediatek/mtk_ppe.h b/drivers/net/ethernet/mediatek/mtk_ppe.h index 69ffce04d630..0b7a67a958e4 100644 --- a/drivers/net/ethernet/mediatek/mtk_ppe.h +++ b/drivers/net/ethernet/mediatek/mtk_ppe.h @@ -8,8 +8,6 @@ #include <linux/bitfield.h> #include <linux/rhashtable.h> -#define MTK_ETH_PPE_BASE 0xc00 - #define MTK_PPE_ENTRIES_SHIFT 3 #define MTK_PPE_ENTRIES (1024 << MTK_PPE_ENTRIES_SHIFT) #define MTK_PPE_HASH_MASK (MTK_PPE_ENTRIES - 1) @@ -34,6 +32,15 @@ #define MTK_FOE_IB1_UDP BIT(30) #define MTK_FOE_IB1_STATIC BIT(31) +/* CONFIG_MEDIATEK_NETSYS_V2 */ +#define MTK_FOE_IB1_BIND_TIMESTAMP_V2 GENMASK(7, 0) +#define MTK_FOE_IB1_BIND_VLAN_LAYER_V2 GENMASK(16, 14) +#define MTK_FOE_IB1_BIND_PPPOE_V2 BIT(17) +#define MTK_FOE_IB1_BIND_VLAN_TAG_V2 BIT(18) +#define MTK_FOE_IB1_BIND_CACHE_V2 BIT(20) +#define MTK_FOE_IB1_BIND_TTL_V2 BIT(22) +#define MTK_FOE_IB1_PACKET_TYPE_V2 GENMASK(27, 23) + enum { MTK_PPE_PKT_TYPE_IPV4_HNAPT = 0, MTK_PPE_PKT_TYPE_IPV4_ROUTE = 1, @@ -55,14 +62,25 @@ enum { #define MTK_FOE_IB2_PORT_MG GENMASK(17, 12) +#define MTK_FOE_IB2_RX_IDX GENMASK(18, 17) #define MTK_FOE_IB2_PORT_AG GENMASK(23, 18) #define MTK_FOE_IB2_DSCP GENMASK(31, 24) +/* CONFIG_MEDIATEK_NETSYS_V2 */ +#define MTK_FOE_IB2_PORT_MG_V2 BIT(7) +#define MTK_FOE_IB2_DEST_PORT_V2 GENMASK(12, 9) +#define MTK_FOE_IB2_MULTICAST_V2 BIT(13) +#define MTK_FOE_IB2_WDMA_WINFO_V2 BIT(19) +#define MTK_FOE_IB2_PORT_AG_V2 GENMASK(23, 20) + #define MTK_FOE_VLAN2_WINFO_BSS GENMASK(5, 0) #define MTK_FOE_VLAN2_WINFO_WCID GENMASK(13, 6) #define MTK_FOE_VLAN2_WINFO_RING GENMASK(15, 14) +#define MTK_FOE_WINFO_BSS GENMASK(5, 0) +#define MTK_FOE_WINFO_WCID GENMASK(15, 6) + enum { MTK_FOE_STATE_INVALID, MTK_FOE_STATE_UNBIND, @@ -83,6 +101,9 @@ struct mtk_foe_mac_info { u16 pppoe_id; u16 src_mac_lo; + + u16 minfo; + u16 winfo; }; /* software-only entry type */ @@ -200,7 +221,7 @@ struct mtk_foe_entry { struct mtk_foe_ipv4_dslite dslite; struct mtk_foe_ipv6 ipv6; struct mtk_foe_ipv6_6rd ipv6_6rd; - u32 data[19]; + u32 data[23]; }; }; @@ -249,6 +270,7 @@ struct mtk_flow_entry { }; u8 type; s8 wed_index; + u8 ppe_index; u16 hash; union { struct mtk_foe_entry data; @@ -267,20 +289,22 @@ struct mtk_ppe { struct device *dev; void __iomem *base; int version; + char dirname[5]; - struct mtk_foe_entry *foe_table; + void *foe_table; dma_addr_t foe_phys; u16 foe_check_time[MTK_PPE_ENTRIES]; - struct hlist_head foe_flow[MTK_PPE_ENTRIES / 2]; + struct hlist_head *foe_flow; struct rhashtable l2_flows; void *acct_table; }; -struct mtk_ppe *mtk_ppe_init(struct mtk_eth *eth, void __iomem *base, int version); -int mtk_ppe_start(struct mtk_ppe *ppe); +struct mtk_ppe *mtk_ppe_init(struct mtk_eth *eth, void __iomem *base, + int version, int index); +void mtk_ppe_start(struct mtk_ppe *ppe); int mtk_ppe_stop(struct mtk_ppe *ppe); void __mtk_ppe_check_skb(struct mtk_ppe *ppe, struct sk_buff *skb, u16 hash); @@ -305,34 +329,30 @@ mtk_ppe_check_skb(struct mtk_ppe *ppe, struct sk_buff *skb, u16 hash) __mtk_ppe_check_skb(ppe, skb, hash); } -static inline int -mtk_foe_entry_timestamp(struct mtk_ppe *ppe, u16 hash) -{ - u32 ib1 = READ_ONCE(ppe->foe_table[hash].ib1); - - if (FIELD_GET(MTK_FOE_IB1_STATE, ib1) != MTK_FOE_STATE_BIND) - return -1; - - return FIELD_GET(MTK_FOE_IB1_BIND_TIMESTAMP, ib1); -} - -int mtk_foe_entry_prepare(struct mtk_foe_entry *entry, int type, int l4proto, - u8 pse_port, u8 *src_mac, u8 *dest_mac); -int mtk_foe_entry_set_pse_port(struct mtk_foe_entry *entry, u8 port); -int mtk_foe_entry_set_ipv4_tuple(struct mtk_foe_entry *entry, bool orig, +int mtk_foe_entry_prepare(struct mtk_eth *eth, struct mtk_foe_entry *entry, + int type, int l4proto, u8 pse_port, u8 *src_mac, + u8 *dest_mac); +int mtk_foe_entry_set_pse_port(struct mtk_eth *eth, + struct mtk_foe_entry *entry, u8 port); +int mtk_foe_entry_set_ipv4_tuple(struct mtk_eth *eth, + struct mtk_foe_entry *entry, bool orig, __be32 src_addr, __be16 src_port, __be32 dest_addr, __be16 dest_port); -int mtk_foe_entry_set_ipv6_tuple(struct mtk_foe_entry *entry, +int mtk_foe_entry_set_ipv6_tuple(struct mtk_eth *eth, + struct mtk_foe_entry *entry, __be32 *src_addr, __be16 src_port, __be32 *dest_addr, __be16 dest_port); -int mtk_foe_entry_set_dsa(struct mtk_foe_entry *entry, int port); -int mtk_foe_entry_set_vlan(struct mtk_foe_entry *entry, int vid); -int mtk_foe_entry_set_pppoe(struct mtk_foe_entry *entry, int sid); -int mtk_foe_entry_set_wdma(struct mtk_foe_entry *entry, int wdma_idx, int txq, - int bss, int wcid); +int mtk_foe_entry_set_dsa(struct mtk_eth *eth, struct mtk_foe_entry *entry, + int port); +int mtk_foe_entry_set_vlan(struct mtk_eth *eth, struct mtk_foe_entry *entry, + int vid); +int mtk_foe_entry_set_pppoe(struct mtk_eth *eth, struct mtk_foe_entry *entry, + int sid); +int mtk_foe_entry_set_wdma(struct mtk_eth *eth, struct mtk_foe_entry *entry, + int wdma_idx, int txq, int bss, int wcid); int mtk_foe_entry_commit(struct mtk_ppe *ppe, struct mtk_flow_entry *entry); void mtk_foe_entry_clear(struct mtk_ppe *ppe, struct mtk_flow_entry *entry); int mtk_foe_entry_idle_time(struct mtk_ppe *ppe, struct mtk_flow_entry *entry); -int mtk_ppe_debugfs_init(struct mtk_ppe *ppe); +int mtk_ppe_debugfs_init(struct mtk_ppe *ppe, int index); #endif diff --git a/drivers/net/ethernet/mediatek/mtk_ppe_debugfs.c b/drivers/net/ethernet/mediatek/mtk_ppe_debugfs.c index eb0b598f14e4..391b071bcff3 100644 --- a/drivers/net/ethernet/mediatek/mtk_ppe_debugfs.c +++ b/drivers/net/ethernet/mediatek/mtk_ppe_debugfs.c @@ -79,7 +79,7 @@ mtk_ppe_debugfs_foe_show(struct seq_file *m, void *private, bool bind) int i; for (i = 0; i < MTK_PPE_ENTRIES; i++) { - struct mtk_foe_entry *entry = &ppe->foe_table[i]; + struct mtk_foe_entry *entry = mtk_foe_get_entry(ppe, i); struct mtk_foe_mac_info *l2; struct mtk_flow_addr_info ai = {}; unsigned char h_source[ETH_ALEN]; @@ -162,52 +162,28 @@ mtk_ppe_debugfs_foe_show(struct seq_file *m, void *private, bool bind) } static int -mtk_ppe_debugfs_foe_show_all(struct seq_file *m, void *private) +mtk_ppe_debugfs_foe_all_show(struct seq_file *m, void *private) { return mtk_ppe_debugfs_foe_show(m, private, false); } +DEFINE_SHOW_ATTRIBUTE(mtk_ppe_debugfs_foe_all); static int -mtk_ppe_debugfs_foe_show_bind(struct seq_file *m, void *private) +mtk_ppe_debugfs_foe_bind_show(struct seq_file *m, void *private) { return mtk_ppe_debugfs_foe_show(m, private, true); } +DEFINE_SHOW_ATTRIBUTE(mtk_ppe_debugfs_foe_bind); -static int -mtk_ppe_debugfs_foe_open_all(struct inode *inode, struct file *file) +int mtk_ppe_debugfs_init(struct mtk_ppe *ppe, int index) { - return single_open(file, mtk_ppe_debugfs_foe_show_all, - inode->i_private); -} - -static int -mtk_ppe_debugfs_foe_open_bind(struct inode *inode, struct file *file) -{ - return single_open(file, mtk_ppe_debugfs_foe_show_bind, - inode->i_private); -} - -int mtk_ppe_debugfs_init(struct mtk_ppe *ppe) -{ - static const struct file_operations fops_all = { - .open = mtk_ppe_debugfs_foe_open_all, - .read = seq_read, - .llseek = seq_lseek, - .release = single_release, - }; - - static const struct file_operations fops_bind = { - .open = mtk_ppe_debugfs_foe_open_bind, - .read = seq_read, - .llseek = seq_lseek, - .release = single_release, - }; - struct dentry *root; - root = debugfs_create_dir("mtk_ppe", NULL); - debugfs_create_file("entries", S_IRUGO, root, ppe, &fops_all); - debugfs_create_file("bind", S_IRUGO, root, ppe, &fops_bind); + snprintf(ppe->dirname, sizeof(ppe->dirname), "ppe%d", index); + + root = debugfs_create_dir(ppe->dirname, NULL); + debugfs_create_file("entries", S_IRUGO, root, ppe, &mtk_ppe_debugfs_foe_all_fops); + debugfs_create_file("bind", S_IRUGO, root, ppe, &mtk_ppe_debugfs_foe_bind_fops); return 0; } diff --git a/drivers/net/ethernet/mediatek/mtk_ppe_offload.c b/drivers/net/ethernet/mediatek/mtk_ppe_offload.c index 25dc3c3aa31d..28bbd1df3e30 100644 --- a/drivers/net/ethernet/mediatek/mtk_ppe_offload.c +++ b/drivers/net/ethernet/mediatek/mtk_ppe_offload.c @@ -52,18 +52,19 @@ static const struct rhashtable_params mtk_flow_ht_params = { }; static int -mtk_flow_set_ipv4_addr(struct mtk_foe_entry *foe, struct mtk_flow_data *data, - bool egress) +mtk_flow_set_ipv4_addr(struct mtk_eth *eth, struct mtk_foe_entry *foe, + struct mtk_flow_data *data, bool egress) { - return mtk_foe_entry_set_ipv4_tuple(foe, egress, + return mtk_foe_entry_set_ipv4_tuple(eth, foe, egress, data->v4.src_addr, data->src_port, data->v4.dst_addr, data->dst_port); } static int -mtk_flow_set_ipv6_addr(struct mtk_foe_entry *foe, struct mtk_flow_data *data) +mtk_flow_set_ipv6_addr(struct mtk_eth *eth, struct mtk_foe_entry *foe, + struct mtk_flow_data *data) { - return mtk_foe_entry_set_ipv6_tuple(foe, + return mtk_foe_entry_set_ipv6_tuple(eth, foe, data->v6.src_addr.s6_addr32, data->src_port, data->v6.dst_addr.s6_addr32, data->dst_port); } @@ -173,7 +174,7 @@ mtk_flow_get_dsa_port(struct net_device **dev) if (dp->cpu_dp->tag_ops->proto != DSA_TAG_PROTO_MTK) return -ENODEV; - *dev = dp->cpu_dp->master; + *dev = dsa_port_to_master(dp); return dp->index; #else @@ -190,16 +191,29 @@ mtk_flow_set_output_device(struct mtk_eth *eth, struct mtk_foe_entry *foe, int pse_port, dsa_port; if (mtk_flow_get_wdma_info(dev, dest_mac, &info) == 0) { - mtk_foe_entry_set_wdma(foe, info.wdma_idx, info.queue, info.bss, - info.wcid); - pse_port = 3; + mtk_foe_entry_set_wdma(eth, foe, info.wdma_idx, info.queue, + info.bss, info.wcid); + if (MTK_HAS_CAPS(eth->soc->caps, MTK_NETSYS_V2)) { + switch (info.wdma_idx) { + case 0: + pse_port = 8; + break; + case 1: + pse_port = 9; + break; + default: + return -EINVAL; + } + } else { + pse_port = 3; + } *wed_index = info.wdma_idx; goto out; } dsa_port = mtk_flow_get_dsa_port(&dev); if (dsa_port >= 0) - mtk_foe_entry_set_dsa(foe, dsa_port); + mtk_foe_entry_set_dsa(eth, foe, dsa_port); if (dev == eth->netdev[0]) pse_port = 1; @@ -209,7 +223,7 @@ mtk_flow_set_output_device(struct mtk_eth *eth, struct mtk_foe_entry *foe, return -EOPNOTSUPP; out: - mtk_foe_entry_set_pse_port(foe, pse_port); + mtk_foe_entry_set_pse_port(eth, foe, pse_port); return 0; } @@ -333,9 +347,8 @@ mtk_flow_offload_replace(struct mtk_eth *eth, struct flow_cls_offload *f) !is_valid_ether_addr(data.eth.h_dest)) return -EINVAL; - err = mtk_foe_entry_prepare(&foe, offload_type, l4proto, 0, - data.eth.h_source, - data.eth.h_dest); + err = mtk_foe_entry_prepare(eth, &foe, offload_type, l4proto, 0, + data.eth.h_source, data.eth.h_dest); if (err) return err; @@ -360,7 +373,7 @@ mtk_flow_offload_replace(struct mtk_eth *eth, struct flow_cls_offload *f) data.v4.src_addr = addrs.key->src; data.v4.dst_addr = addrs.key->dst; - mtk_flow_set_ipv4_addr(&foe, &data, false); + mtk_flow_set_ipv4_addr(eth, &foe, &data, false); } if (addr_type == FLOW_DISSECTOR_KEY_IPV6_ADDRS) { @@ -371,7 +384,7 @@ mtk_flow_offload_replace(struct mtk_eth *eth, struct flow_cls_offload *f) data.v6.src_addr = addrs.key->src; data.v6.dst_addr = addrs.key->dst; - mtk_flow_set_ipv6_addr(&foe, &data); + mtk_flow_set_ipv6_addr(eth, &foe, &data); } flow_action_for_each(i, act, &rule->action) { @@ -401,7 +414,7 @@ mtk_flow_offload_replace(struct mtk_eth *eth, struct flow_cls_offload *f) } if (addr_type == FLOW_DISSECTOR_KEY_IPV4_ADDRS) { - err = mtk_flow_set_ipv4_addr(&foe, &data, true); + err = mtk_flow_set_ipv4_addr(eth, &foe, &data, true); if (err) return err; } @@ -413,10 +426,10 @@ mtk_flow_offload_replace(struct mtk_eth *eth, struct flow_cls_offload *f) if (data.vlan.proto != htons(ETH_P_8021Q)) return -EOPNOTSUPP; - mtk_foe_entry_set_vlan(&foe, data.vlan.id); + mtk_foe_entry_set_vlan(eth, &foe, data.vlan.id); } if (data.pppoe.num == 1) - mtk_foe_entry_set_pppoe(&foe, data.pppoe.sid); + mtk_foe_entry_set_pppoe(eth, &foe, data.pppoe.sid); err = mtk_flow_set_output_device(eth, &foe, odev, data.eth.h_dest, &wed_index); @@ -434,7 +447,7 @@ mtk_flow_offload_replace(struct mtk_eth *eth, struct flow_cls_offload *f) memcpy(&entry->data, &foe, sizeof(entry->data)); entry->wed_index = wed_index; - err = mtk_foe_entry_commit(eth->ppe, entry); + err = mtk_foe_entry_commit(eth->ppe[entry->ppe_index], entry); if (err < 0) goto free; @@ -446,7 +459,7 @@ mtk_flow_offload_replace(struct mtk_eth *eth, struct flow_cls_offload *f) return 0; clear: - mtk_foe_entry_clear(eth->ppe, entry); + mtk_foe_entry_clear(eth->ppe[entry->ppe_index], entry); free: kfree(entry); if (wed_index >= 0) @@ -464,7 +477,7 @@ mtk_flow_offload_destroy(struct mtk_eth *eth, struct flow_cls_offload *f) if (!entry) return -ENOENT; - mtk_foe_entry_clear(eth->ppe, entry); + mtk_foe_entry_clear(eth->ppe[entry->ppe_index], entry); rhashtable_remove_fast(ð->flow_table, &entry->node, mtk_flow_ht_params); if (entry->wed_index >= 0) @@ -485,7 +498,7 @@ mtk_flow_offload_stats(struct mtk_eth *eth, struct flow_cls_offload *f) if (!entry) return -ENOENT; - idle = mtk_foe_entry_idle_time(eth->ppe, entry); + idle = mtk_foe_entry_idle_time(eth->ppe[entry->ppe_index], entry); f->stats.lastused = jiffies - idle * HZ; return 0; @@ -537,7 +550,7 @@ mtk_eth_setup_tc_block(struct net_device *dev, struct flow_block_offload *f) struct flow_block_cb *block_cb; flow_setup_cb_t *cb; - if (!eth->ppe || !eth->ppe->foe_table) + if (!eth->soc->offload_version) return -EOPNOTSUPP; if (f->binder_type != FLOW_BLOCK_BINDER_TYPE_CLSACT_INGRESS) @@ -589,8 +602,5 @@ int mtk_eth_setup_tc(struct net_device *dev, enum tc_setup_type type, int mtk_eth_offload_init(struct mtk_eth *eth) { - if (!eth->ppe || !eth->ppe->foe_table) - return 0; - return rhashtable_init(ð->flow_table, &mtk_flow_ht_params); } diff --git a/drivers/net/ethernet/mediatek/mtk_ppe_regs.h b/drivers/net/ethernet/mediatek/mtk_ppe_regs.h index 0c45ea0900f1..59596d823d8b 100644 --- a/drivers/net/ethernet/mediatek/mtk_ppe_regs.h +++ b/drivers/net/ethernet/mediatek/mtk_ppe_regs.h @@ -21,6 +21,9 @@ #define MTK_PPE_GLO_CFG_BUSY BIT(31) #define MTK_PPE_FLOW_CFG 0x204 +#define MTK_PPE_MD_TOAP_BYP_CRSN0 BIT(1) +#define MTK_PPE_MD_TOAP_BYP_CRSN1 BIT(2) +#define MTK_PPE_MD_TOAP_BYP_CRSN2 BIT(3) #define MTK_PPE_FLOW_CFG_IP4_TCP_FRAG BIT(6) #define MTK_PPE_FLOW_CFG_IP4_UDP_FRAG BIT(7) #define MTK_PPE_FLOW_CFG_IP6_3T_ROUTE BIT(8) @@ -54,6 +57,7 @@ #define MTK_PPE_TB_CFG_HASH_MODE GENMASK(15, 14) #define MTK_PPE_TB_CFG_SCAN_MODE GENMASK(17, 16) #define MTK_PPE_TB_CFG_HASH_DEBUG GENMASK(19, 18) +#define MTK_PPE_TB_CFG_INFO_SEL BIT(20) enum { MTK_PPE_SCAN_MODE_DISABLED, @@ -112,6 +116,8 @@ enum { #define MTK_PPE_DEFAULT_CPU_PORT 0x248 #define MTK_PPE_DEFAULT_CPU_PORT_MASK(_n) (GENMASK(2, 0) << ((_n) * 4)) +#define MTK_PPE_DEFAULT_CPU_PORT1 0x24c + #define MTK_PPE_MTU_DROP 0x308 #define MTK_PPE_VLAN_MTU0 0x30c @@ -141,4 +147,6 @@ enum { #define MTK_PPE_MIB_CACHE_CTL_EN BIT(0) #define MTK_PPE_MIB_CACHE_CTL_FLUSH BIT(2) +#define MTK_PPE_SBW_CTRL 0x374 + #endif diff --git a/drivers/net/ethernet/mediatek/mtk_star_emac.c b/drivers/net/ethernet/mediatek/mtk_star_emac.c index 3f0e5e64de50..7e890f81148e 100644 --- a/drivers/net/ethernet/mediatek/mtk_star_emac.c +++ b/drivers/net/ethernet/mediatek/mtk_star_emac.c @@ -1255,7 +1255,7 @@ static const struct net_device_ops mtk_star_netdev_ops = { static void mtk_star_get_drvinfo(struct net_device *dev, struct ethtool_drvinfo *info) { - strlcpy(info->driver, MTK_STAR_DRVNAME, sizeof(info->driver)); + strscpy(info->driver, MTK_STAR_DRVNAME, sizeof(info->driver)); } /* TODO Add ethtool stats. */ @@ -1651,8 +1651,7 @@ static int mtk_star_probe(struct platform_device *pdev) ndev->netdev_ops = &mtk_star_netdev_ops; ndev->ethtool_ops = &mtk_star_ethtool_ops; - netif_napi_add(ndev, &priv->rx_napi, mtk_star_rx_poll, - NAPI_POLL_WEIGHT); + netif_napi_add(ndev, &priv->rx_napi, mtk_star_rx_poll); netif_napi_add_tx(ndev, &priv->tx_napi, mtk_star_tx_poll); return devm_register_netdev(dev, ndev); diff --git a/drivers/net/ethernet/mediatek/mtk_wed.c b/drivers/net/ethernet/mediatek/mtk_wed.c index 29be2fcafea3..099b6e0df619 100644 --- a/drivers/net/ethernet/mediatek/mtk_wed.c +++ b/drivers/net/ethernet/mediatek/mtk_wed.c @@ -25,6 +25,11 @@ #define MTK_WED_TX_RING_SIZE 2048 #define MTK_WED_WDMA_RING_SIZE 1024 +#define MTK_WED_MAX_GROUP_SIZE 0x100 +#define MTK_WED_VLD_GROUP_SIZE 0x40 +#define MTK_WED_PER_GROUP_PKT 128 + +#define MTK_WED_FBUF_SIZE 128 static struct mtk_wed_hw *hw_list[2]; static DEFINE_MUTEX(hw_lock); @@ -80,11 +85,31 @@ static struct mtk_wed_hw * mtk_wed_assign(struct mtk_wed_device *dev) { struct mtk_wed_hw *hw; + int i; + + if (dev->wlan.bus_type == MTK_WED_BUS_PCIE) { + hw = hw_list[pci_domain_nr(dev->wlan.pci_dev->bus)]; + if (!hw) + return NULL; + + if (!hw->wed_dev) + goto out; + + if (hw->version == 1) + return NULL; + + /* MT7986 WED devices do not have any pcie slot restrictions */ + } + /* MT7986 PCIE or AXI */ + for (i = 0; i < ARRAY_SIZE(hw_list); i++) { + hw = hw_list[i]; + if (hw && !hw->wed_dev) + goto out; + } - hw = hw_list[pci_domain_nr(dev->wlan.pci_dev->bus)]; - if (!hw || hw->wed_dev) - return NULL; + return NULL; +out: hw->wed_dev = dev; return hw; } @@ -150,10 +175,17 @@ mtk_wed_buffer_alloc(struct mtk_wed_device *dev) desc->buf0 = cpu_to_le32(buf_phys); desc->buf1 = cpu_to_le32(buf_phys + txd_size); - ctrl = FIELD_PREP(MTK_WDMA_DESC_CTRL_LEN0, txd_size) | - FIELD_PREP(MTK_WDMA_DESC_CTRL_LEN1, - MTK_WED_BUF_SIZE - txd_size) | - MTK_WDMA_DESC_CTRL_LAST_SEG1; + + if (dev->hw->version == 1) + ctrl = FIELD_PREP(MTK_WDMA_DESC_CTRL_LEN0, txd_size) | + FIELD_PREP(MTK_WDMA_DESC_CTRL_LEN1, + MTK_WED_BUF_SIZE - txd_size) | + MTK_WDMA_DESC_CTRL_LAST_SEG1; + else + ctrl = FIELD_PREP(MTK_WDMA_DESC_CTRL_LEN0, txd_size) | + FIELD_PREP(MTK_WDMA_DESC_CTRL_LEN1_V2, + MTK_WED_BUF_SIZE - txd_size) | + MTK_WDMA_DESC_CTRL_LAST_SEG0; desc->ctrl = cpu_to_le32(ctrl); desc->info = 0; desc++; @@ -209,7 +241,7 @@ mtk_wed_free_ring(struct mtk_wed_device *dev, struct mtk_wed_ring *ring) if (!ring->desc) return; - dma_free_coherent(dev->hw->dev, ring->size * sizeof(*ring->desc), + dma_free_coherent(dev->hw->dev, ring->size * ring->desc_size, ring->desc, ring->desc_phys); } @@ -229,6 +261,14 @@ mtk_wed_set_ext_int(struct mtk_wed_device *dev, bool en) { u32 mask = MTK_WED_EXT_INT_STATUS_ERROR_MASK; + if (dev->hw->version == 1) + mask |= MTK_WED_EXT_INT_STATUS_TX_DRV_R_RESP_ERR; + else + mask |= MTK_WED_EXT_INT_STATUS_RX_FBUF_LO_TH | + MTK_WED_EXT_INT_STATUS_RX_FBUF_HI_TH | + MTK_WED_EXT_INT_STATUS_RX_DRV_COHERENT | + MTK_WED_EXT_INT_STATUS_TX_DMA_W_RESP_ERR; + if (!dev->hw->num_flows) mask &= ~MTK_WED_EXT_INT_STATUS_TKID_WO_PYLD; @@ -237,9 +277,54 @@ mtk_wed_set_ext_int(struct mtk_wed_device *dev, bool en) } static void +mtk_wed_set_512_support(struct mtk_wed_device *dev, bool enable) +{ + if (enable) { + wed_w32(dev, MTK_WED_TXDP_CTRL, MTK_WED_TXDP_DW9_OVERWR); + wed_w32(dev, MTK_WED_TXP_DW1, + FIELD_PREP(MTK_WED_WPDMA_WRITE_TXP, 0x0103)); + } else { + wed_w32(dev, MTK_WED_TXP_DW1, + FIELD_PREP(MTK_WED_WPDMA_WRITE_TXP, 0x0100)); + wed_clr(dev, MTK_WED_TXDP_CTRL, MTK_WED_TXDP_DW9_OVERWR); + } +} + +static void +mtk_wed_dma_disable(struct mtk_wed_device *dev) +{ + wed_clr(dev, MTK_WED_WPDMA_GLO_CFG, + MTK_WED_WPDMA_GLO_CFG_TX_DRV_EN | + MTK_WED_WPDMA_GLO_CFG_RX_DRV_EN); + + wed_clr(dev, MTK_WED_WDMA_GLO_CFG, MTK_WED_WDMA_GLO_CFG_RX_DRV_EN); + + wed_clr(dev, MTK_WED_GLO_CFG, + MTK_WED_GLO_CFG_TX_DMA_EN | + MTK_WED_GLO_CFG_RX_DMA_EN); + + wdma_m32(dev, MTK_WDMA_GLO_CFG, + MTK_WDMA_GLO_CFG_TX_DMA_EN | + MTK_WDMA_GLO_CFG_RX_INFO1_PRERES | + MTK_WDMA_GLO_CFG_RX_INFO2_PRERES, 0); + + if (dev->hw->version == 1) { + regmap_write(dev->hw->mirror, dev->hw->index * 4, 0); + wdma_m32(dev, MTK_WDMA_GLO_CFG, + MTK_WDMA_GLO_CFG_RX_INFO3_PRERES, 0); + } else { + wed_clr(dev, MTK_WED_WPDMA_GLO_CFG, + MTK_WED_WPDMA_GLO_CFG_RX_DRV_R0_PKT_PROC | + MTK_WED_WPDMA_GLO_CFG_RX_DRV_R0_CRX_SYNC); + + mtk_wed_set_512_support(dev, false); + } +} + +static void mtk_wed_stop(struct mtk_wed_device *dev) { - regmap_write(dev->hw->mirror, dev->hw->index * 4, 0); + mtk_wed_dma_disable(dev); mtk_wed_set_ext_int(dev, false); wed_clr(dev, MTK_WED_CTRL, @@ -252,21 +337,11 @@ mtk_wed_stop(struct mtk_wed_device *dev) wdma_w32(dev, MTK_WDMA_INT_MASK, 0); wdma_w32(dev, MTK_WDMA_INT_GRP2, 0); wed_w32(dev, MTK_WED_WPDMA_INT_MASK, 0); - - wed_clr(dev, MTK_WED_GLO_CFG, - MTK_WED_GLO_CFG_TX_DMA_EN | - MTK_WED_GLO_CFG_RX_DMA_EN); - wed_clr(dev, MTK_WED_WPDMA_GLO_CFG, - MTK_WED_WPDMA_GLO_CFG_TX_DRV_EN | - MTK_WED_WPDMA_GLO_CFG_RX_DRV_EN); - wed_clr(dev, MTK_WED_WDMA_GLO_CFG, - MTK_WED_WDMA_GLO_CFG_RX_DRV_EN); } static void mtk_wed_detach(struct mtk_wed_device *dev) { - struct device_node *wlan_node = dev->wlan.pci_dev->dev.of_node; struct mtk_wed_hw *hw = dev->hw; mutex_lock(&hw_lock); @@ -281,9 +356,14 @@ mtk_wed_detach(struct mtk_wed_device *dev) mtk_wed_free_buffer(dev); mtk_wed_free_tx_rings(dev); - if (of_dma_is_coherent(wlan_node)) - regmap_update_bits(hw->hifsys, HIFSYS_DMA_AG_MAP, - BIT(hw->index), BIT(hw->index)); + if (dev->wlan.bus_type == MTK_WED_BUS_PCIE) { + struct device_node *wlan_node; + + wlan_node = dev->wlan.pci_dev->dev.of_node; + if (of_dma_is_coherent(wlan_node) && hw->hifsys) + regmap_update_bits(hw->hifsys, HIFSYS_DMA_AG_MAP, + BIT(hw->index), BIT(hw->index)); + } if (!hw_list[!hw->index]->wed_dev && hw->eth->dma_dev != hw->eth->dev) @@ -296,14 +376,76 @@ mtk_wed_detach(struct mtk_wed_device *dev) mutex_unlock(&hw_lock); } +#define PCIE_BASE_ADDR0 0x11280000 +static void +mtk_wed_bus_init(struct mtk_wed_device *dev) +{ + switch (dev->wlan.bus_type) { + case MTK_WED_BUS_PCIE: { + struct device_node *np = dev->hw->eth->dev->of_node; + struct regmap *regs; + + regs = syscon_regmap_lookup_by_phandle(np, + "mediatek,wed-pcie"); + if (IS_ERR(regs)) + break; + + regmap_update_bits(regs, 0, BIT(0), BIT(0)); + + wed_w32(dev, MTK_WED_PCIE_INT_CTRL, + FIELD_PREP(MTK_WED_PCIE_INT_CTRL_POLL_EN, 2)); + + /* pcie interrupt control: pola/source selection */ + wed_set(dev, MTK_WED_PCIE_INT_CTRL, + MTK_WED_PCIE_INT_CTRL_MSK_EN_POLA | + FIELD_PREP(MTK_WED_PCIE_INT_CTRL_SRC_SEL, 1)); + wed_r32(dev, MTK_WED_PCIE_INT_CTRL); + + wed_w32(dev, MTK_WED_PCIE_CFG_INTM, PCIE_BASE_ADDR0 | 0x180); + wed_w32(dev, MTK_WED_PCIE_CFG_BASE, PCIE_BASE_ADDR0 | 0x184); + + /* pcie interrupt status trigger register */ + wed_w32(dev, MTK_WED_PCIE_INT_TRIGGER, BIT(24)); + wed_r32(dev, MTK_WED_PCIE_INT_TRIGGER); + + /* pola setting */ + wed_set(dev, MTK_WED_PCIE_INT_CTRL, + MTK_WED_PCIE_INT_CTRL_MSK_EN_POLA); + break; + } + case MTK_WED_BUS_AXI: + wed_set(dev, MTK_WED_WPDMA_INT_CTRL, + MTK_WED_WPDMA_INT_CTRL_SIG_SRC | + FIELD_PREP(MTK_WED_WPDMA_INT_CTRL_SRC_SEL, 0)); + break; + default: + break; + } +} + +static void +mtk_wed_set_wpdma(struct mtk_wed_device *dev) +{ + if (dev->hw->version == 1) { + wed_w32(dev, MTK_WED_WPDMA_CFG_BASE, dev->wlan.wpdma_phys); + } else { + mtk_wed_bus_init(dev); + + wed_w32(dev, MTK_WED_WPDMA_CFG_BASE, dev->wlan.wpdma_int); + wed_w32(dev, MTK_WED_WPDMA_CFG_INT_MASK, dev->wlan.wpdma_mask); + wed_w32(dev, MTK_WED_WPDMA_CFG_TX, dev->wlan.wpdma_tx); + wed_w32(dev, MTK_WED_WPDMA_CFG_TX_FREE, dev->wlan.wpdma_txfree); + } +} + static void mtk_wed_hw_init_early(struct mtk_wed_device *dev) { u32 mask, set; - u32 offset; mtk_wed_stop(dev); mtk_wed_reset(dev, MTK_WED_RESET_WED); + mtk_wed_set_wpdma(dev); mask = MTK_WED_WDMA_GLO_CFG_BT_SIZE | MTK_WED_WDMA_GLO_CFG_DYNAMIC_DMAD_RECYCLE | @@ -313,14 +455,33 @@ mtk_wed_hw_init_early(struct mtk_wed_device *dev) MTK_WED_WDMA_GLO_CFG_IDLE_DMAD_SUPPLY; wed_m32(dev, MTK_WED_WDMA_GLO_CFG, mask, set); - wdma_set(dev, MTK_WDMA_GLO_CFG, MTK_WDMA_GLO_CFG_RX_INFO_PRERES); + if (dev->hw->version == 1) { + u32 offset = dev->hw->index ? 0x04000400 : 0; - offset = dev->hw->index ? 0x04000400 : 0; - wed_w32(dev, MTK_WED_WDMA_OFFSET0, 0x2a042a20 + offset); - wed_w32(dev, MTK_WED_WDMA_OFFSET1, 0x29002800 + offset); + wdma_set(dev, MTK_WDMA_GLO_CFG, + MTK_WDMA_GLO_CFG_RX_INFO1_PRERES | + MTK_WDMA_GLO_CFG_RX_INFO2_PRERES | + MTK_WDMA_GLO_CFG_RX_INFO3_PRERES); - wed_w32(dev, MTK_WED_PCIE_CFG_BASE, MTK_PCIE_BASE(dev->hw->index)); - wed_w32(dev, MTK_WED_WPDMA_CFG_BASE, dev->wlan.wpdma_phys); + wed_w32(dev, MTK_WED_WDMA_OFFSET0, 0x2a042a20 + offset); + wed_w32(dev, MTK_WED_WDMA_OFFSET1, 0x29002800 + offset); + wed_w32(dev, MTK_WED_PCIE_CFG_BASE, + MTK_PCIE_BASE(dev->hw->index)); + } else { + wed_w32(dev, MTK_WED_WDMA_CFG_BASE, dev->hw->wdma_phy); + wed_set(dev, MTK_WED_CTRL, MTK_WED_CTRL_ETH_DMAD_FMT); + wed_w32(dev, MTK_WED_WDMA_OFFSET0, + FIELD_PREP(MTK_WED_WDMA_OFST0_GLO_INTS, + MTK_WDMA_INT_STATUS) | + FIELD_PREP(MTK_WED_WDMA_OFST0_GLO_CFG, + MTK_WDMA_GLO_CFG)); + + wed_w32(dev, MTK_WED_WDMA_OFFSET1, + FIELD_PREP(MTK_WED_WDMA_OFST1_TX_CTRL, + MTK_WDMA_RING_TX(0)) | + FIELD_PREP(MTK_WED_WDMA_OFST1_RX_CTRL, + MTK_WDMA_RING_RX(0))); + } } static void @@ -340,37 +501,65 @@ mtk_wed_hw_init(struct mtk_wed_device *dev) wed_w32(dev, MTK_WED_TX_BM_BASE, dev->buf_ring.desc_phys); - wed_w32(dev, MTK_WED_TX_BM_TKID, - FIELD_PREP(MTK_WED_TX_BM_TKID_START, - dev->wlan.token_start) | - FIELD_PREP(MTK_WED_TX_BM_TKID_END, - dev->wlan.token_start + dev->wlan.nbuf - 1)); - wed_w32(dev, MTK_WED_TX_BM_BUF_LEN, MTK_WED_PKT_SIZE); - wed_w32(dev, MTK_WED_TX_BM_DYN_THR, - FIELD_PREP(MTK_WED_TX_BM_DYN_THR_LO, 1) | - MTK_WED_TX_BM_DYN_THR_HI); + if (dev->hw->version == 1) { + wed_w32(dev, MTK_WED_TX_BM_TKID, + FIELD_PREP(MTK_WED_TX_BM_TKID_START, + dev->wlan.token_start) | + FIELD_PREP(MTK_WED_TX_BM_TKID_END, + dev->wlan.token_start + + dev->wlan.nbuf - 1)); + wed_w32(dev, MTK_WED_TX_BM_DYN_THR, + FIELD_PREP(MTK_WED_TX_BM_DYN_THR_LO, 1) | + MTK_WED_TX_BM_DYN_THR_HI); + } else { + wed_w32(dev, MTK_WED_TX_BM_TKID_V2, + FIELD_PREP(MTK_WED_TX_BM_TKID_START, + dev->wlan.token_start) | + FIELD_PREP(MTK_WED_TX_BM_TKID_END, + dev->wlan.token_start + + dev->wlan.nbuf - 1)); + wed_w32(dev, MTK_WED_TX_BM_DYN_THR, + FIELD_PREP(MTK_WED_TX_BM_DYN_THR_LO_V2, 0) | + MTK_WED_TX_BM_DYN_THR_HI_V2); + wed_w32(dev, MTK_WED_TX_TKID_CTRL, + MTK_WED_TX_TKID_CTRL_PAUSE | + FIELD_PREP(MTK_WED_TX_TKID_CTRL_VLD_GRP_NUM, + dev->buf_ring.size / 128) | + FIELD_PREP(MTK_WED_TX_TKID_CTRL_RSV_GRP_NUM, + dev->buf_ring.size / 128)); + wed_w32(dev, MTK_WED_TX_TKID_DYN_THR, + FIELD_PREP(MTK_WED_TX_TKID_DYN_THR_LO, 0) | + MTK_WED_TX_TKID_DYN_THR_HI); + } mtk_wed_reset(dev, MTK_WED_RESET_TX_BM); - wed_set(dev, MTK_WED_CTRL, - MTK_WED_CTRL_WED_TX_BM_EN | - MTK_WED_CTRL_WED_TX_FREE_AGENT_EN); + if (dev->hw->version == 1) + wed_set(dev, MTK_WED_CTRL, + MTK_WED_CTRL_WED_TX_BM_EN | + MTK_WED_CTRL_WED_TX_FREE_AGENT_EN); + else + wed_clr(dev, MTK_WED_TX_TKID_CTRL, MTK_WED_TX_TKID_CTRL_PAUSE); wed_clr(dev, MTK_WED_TX_BM_CTRL, MTK_WED_TX_BM_CTRL_PAUSE); } static void -mtk_wed_ring_reset(struct mtk_wdma_desc *desc, int size) +mtk_wed_ring_reset(struct mtk_wed_ring *ring, int size) { + void *head = (void *)ring->desc; int i; for (i = 0; i < size; i++) { - desc[i].buf0 = 0; - desc[i].ctrl = cpu_to_le32(MTK_WDMA_DESC_CTRL_DMA_DONE); - desc[i].buf1 = 0; - desc[i].info = 0; + struct mtk_wdma_desc *desc; + + desc = (struct mtk_wdma_desc *)(head + i * ring->desc_size); + desc->buf0 = 0; + desc->ctrl = cpu_to_le32(MTK_WDMA_DESC_CTRL_DMA_DONE); + desc->buf1 = 0; + desc->info = 0; } } @@ -421,12 +610,10 @@ mtk_wed_reset_dma(struct mtk_wed_device *dev) int i; for (i = 0; i < ARRAY_SIZE(dev->tx_ring); i++) { - struct mtk_wdma_desc *desc = dev->tx_ring[i].desc; - - if (!desc) + if (!dev->tx_ring[i].desc) continue; - mtk_wed_ring_reset(desc, MTK_WED_TX_RING_SIZE); + mtk_wed_ring_reset(&dev->tx_ring[i], MTK_WED_TX_RING_SIZE); } if (mtk_wed_poll_busy(dev)) @@ -483,16 +670,16 @@ mtk_wed_reset_dma(struct mtk_wed_device *dev) static int mtk_wed_ring_alloc(struct mtk_wed_device *dev, struct mtk_wed_ring *ring, - int size) + int size, u32 desc_size) { - ring->desc = dma_alloc_coherent(dev->hw->dev, - size * sizeof(*ring->desc), + ring->desc = dma_alloc_coherent(dev->hw->dev, size * desc_size, &ring->desc_phys, GFP_KERNEL); if (!ring->desc) return -ENOMEM; + ring->desc_size = desc_size; ring->size = size; - mtk_wed_ring_reset(ring->desc, size); + mtk_wed_ring_reset(ring, size); return 0; } @@ -500,9 +687,10 @@ mtk_wed_ring_alloc(struct mtk_wed_device *dev, struct mtk_wed_ring *ring, static int mtk_wed_wdma_ring_setup(struct mtk_wed_device *dev, int idx, int size) { + u32 desc_size = sizeof(struct mtk_wdma_desc) * dev->hw->version; struct mtk_wed_ring *wdma = &dev->tx_wdma[idx]; - if (mtk_wed_ring_alloc(dev, wdma, MTK_WED_WDMA_RING_SIZE)) + if (mtk_wed_ring_alloc(dev, wdma, MTK_WED_WDMA_RING_SIZE, desc_size)) return -ENOMEM; wdma_w32(dev, MTK_WDMA_RING_RX(idx) + MTK_WED_RING_OFS_BASE, @@ -520,43 +708,63 @@ mtk_wed_wdma_ring_setup(struct mtk_wed_device *dev, int idx, int size) } static void -mtk_wed_start(struct mtk_wed_device *dev, u32 irq_mask) +mtk_wed_configure_irq(struct mtk_wed_device *dev, u32 irq_mask) { - u32 wdma_mask; - u32 val; - int i; - - for (i = 0; i < ARRAY_SIZE(dev->tx_wdma); i++) - if (!dev->tx_wdma[i].desc) - mtk_wed_wdma_ring_setup(dev, i, 16); - - wdma_mask = FIELD_PREP(MTK_WDMA_INT_MASK_RX_DONE, GENMASK(1, 0)); - - mtk_wed_hw_init(dev); + u32 wdma_mask = FIELD_PREP(MTK_WDMA_INT_MASK_RX_DONE, GENMASK(1, 0)); + /* wed control cr set */ wed_set(dev, MTK_WED_CTRL, MTK_WED_CTRL_WDMA_INT_AGENT_EN | MTK_WED_CTRL_WPDMA_INT_AGENT_EN | MTK_WED_CTRL_WED_TX_BM_EN | MTK_WED_CTRL_WED_TX_FREE_AGENT_EN); - wed_w32(dev, MTK_WED_PCIE_INT_TRIGGER, MTK_WED_PCIE_INT_TRIGGER_STATUS); + if (dev->hw->version == 1) { + wed_w32(dev, MTK_WED_PCIE_INT_TRIGGER, + MTK_WED_PCIE_INT_TRIGGER_STATUS); - wed_w32(dev, MTK_WED_WPDMA_INT_TRIGGER, - MTK_WED_WPDMA_INT_TRIGGER_RX_DONE | - MTK_WED_WPDMA_INT_TRIGGER_TX_DONE); + wed_w32(dev, MTK_WED_WPDMA_INT_TRIGGER, + MTK_WED_WPDMA_INT_TRIGGER_RX_DONE | + MTK_WED_WPDMA_INT_TRIGGER_TX_DONE); - wed_set(dev, MTK_WED_WPDMA_INT_CTRL, - MTK_WED_WPDMA_INT_CTRL_SUBRT_ADV); + wed_clr(dev, MTK_WED_WDMA_INT_CTRL, wdma_mask); + } else { + /* initail tx interrupt trigger */ + wed_w32(dev, MTK_WED_WPDMA_INT_CTRL_TX, + MTK_WED_WPDMA_INT_CTRL_TX0_DONE_EN | + MTK_WED_WPDMA_INT_CTRL_TX0_DONE_CLR | + MTK_WED_WPDMA_INT_CTRL_TX1_DONE_EN | + MTK_WED_WPDMA_INT_CTRL_TX1_DONE_CLR | + FIELD_PREP(MTK_WED_WPDMA_INT_CTRL_TX0_DONE_TRIG, + dev->wlan.tx_tbit[0]) | + FIELD_PREP(MTK_WED_WPDMA_INT_CTRL_TX1_DONE_TRIG, + dev->wlan.tx_tbit[1])); + + /* initail txfree interrupt trigger */ + wed_w32(dev, MTK_WED_WPDMA_INT_CTRL_TX_FREE, + MTK_WED_WPDMA_INT_CTRL_TX_FREE_DONE_EN | + MTK_WED_WPDMA_INT_CTRL_TX_FREE_DONE_CLR | + FIELD_PREP(MTK_WED_WPDMA_INT_CTRL_TX_FREE_DONE_TRIG, + dev->wlan.txfree_tbit)); + + wed_w32(dev, MTK_WED_WDMA_INT_CLR, wdma_mask); + wed_set(dev, MTK_WED_WDMA_INT_CTRL, + FIELD_PREP(MTK_WED_WDMA_INT_CTRL_POLL_SRC_SEL, + dev->wdma_idx)); + } wed_w32(dev, MTK_WED_WDMA_INT_TRIGGER, wdma_mask); - wed_clr(dev, MTK_WED_WDMA_INT_CTRL, wdma_mask); wdma_w32(dev, MTK_WDMA_INT_MASK, wdma_mask); wdma_w32(dev, MTK_WDMA_INT_GRP2, wdma_mask); - wed_w32(dev, MTK_WED_WPDMA_INT_MASK, irq_mask); wed_w32(dev, MTK_WED_INT_MASK, irq_mask); +} + +static void +mtk_wed_dma_enable(struct mtk_wed_device *dev) +{ + wed_set(dev, MTK_WED_WPDMA_INT_CTRL, MTK_WED_WPDMA_INT_CTRL_SUBRT_ADV); wed_set(dev, MTK_WED_GLO_CFG, MTK_WED_GLO_CFG_TX_DMA_EN | @@ -567,16 +775,54 @@ mtk_wed_start(struct mtk_wed_device *dev, u32 irq_mask) wed_set(dev, MTK_WED_WDMA_GLO_CFG, MTK_WED_WDMA_GLO_CFG_RX_DRV_EN); + wdma_set(dev, MTK_WDMA_GLO_CFG, + MTK_WDMA_GLO_CFG_TX_DMA_EN | + MTK_WDMA_GLO_CFG_RX_INFO1_PRERES | + MTK_WDMA_GLO_CFG_RX_INFO2_PRERES); + + if (dev->hw->version == 1) { + wdma_set(dev, MTK_WDMA_GLO_CFG, + MTK_WDMA_GLO_CFG_RX_INFO3_PRERES); + } else { + wed_set(dev, MTK_WED_WPDMA_CTRL, + MTK_WED_WPDMA_CTRL_SDL1_FIXED); + + wed_set(dev, MTK_WED_WPDMA_GLO_CFG, + MTK_WED_WPDMA_GLO_CFG_RX_DRV_R0_PKT_PROC | + MTK_WED_WPDMA_GLO_CFG_RX_DRV_R0_CRX_SYNC); + + wed_clr(dev, MTK_WED_WPDMA_GLO_CFG, + MTK_WED_WPDMA_GLO_CFG_TX_TKID_KEEP | + MTK_WED_WPDMA_GLO_CFG_TX_DMAD_DW3_PREV); + } +} + +static void +mtk_wed_start(struct mtk_wed_device *dev, u32 irq_mask) +{ + int i; + + for (i = 0; i < ARRAY_SIZE(dev->tx_wdma); i++) + if (!dev->tx_wdma[i].desc) + mtk_wed_wdma_ring_setup(dev, i, 16); + + mtk_wed_hw_init(dev); + mtk_wed_configure_irq(dev, irq_mask); + mtk_wed_set_ext_int(dev, true); - val = dev->wlan.wpdma_phys | - MTK_PCIE_MIRROR_MAP_EN | - FIELD_PREP(MTK_PCIE_MIRROR_MAP_WED_ID, dev->hw->index); - if (dev->hw->index) - val |= BIT(1); - val |= BIT(0); - regmap_write(dev->hw->mirror, dev->hw->index * 4, val); + if (dev->hw->version == 1) { + u32 val = dev->wlan.wpdma_phys | MTK_PCIE_MIRROR_MAP_EN | + FIELD_PREP(MTK_PCIE_MIRROR_MAP_WED_ID, + dev->hw->index); + + val |= BIT(0) | (BIT(1) * !!dev->hw->index); + regmap_write(dev->hw->mirror, dev->hw->index * 4, val); + } else { + mtk_wed_set_512_support(dev, true); + } + mtk_wed_dma_enable(dev); dev->running = true; } @@ -585,12 +831,14 @@ mtk_wed_attach(struct mtk_wed_device *dev) __releases(RCU) { struct mtk_wed_hw *hw; + struct device *device; int ret = 0; RCU_LOCKDEP_WARN(!rcu_read_lock_held(), "mtk_wed_attach without holding the RCU read lock"); - if (pci_domain_nr(dev->wlan.pci_dev->bus) > 1 || + if ((dev->wlan.bus_type == MTK_WED_BUS_PCIE && + pci_domain_nr(dev->wlan.pci_dev->bus) > 1) || !try_module_get(THIS_MODULE)) ret = -ENODEV; @@ -608,7 +856,11 @@ mtk_wed_attach(struct mtk_wed_device *dev) goto out; } - dev_info(&dev->wlan.pci_dev->dev, "attaching wed device %d\n", hw->index); + device = dev->wlan.bus_type == MTK_WED_BUS_PCIE + ? &dev->wlan.pci_dev->dev + : &dev->wlan.platform_dev->dev; + dev_info(device, "attaching wed device %d version %d\n", + hw->index, hw->version); dev->hw = hw; dev->dev = hw->dev; @@ -626,7 +878,9 @@ mtk_wed_attach(struct mtk_wed_device *dev) } mtk_wed_hw_init_early(dev); - regmap_update_bits(hw->hifsys, HIFSYS_DMA_AG_MAP, BIT(hw->index), 0); + if (hw->hifsys) + regmap_update_bits(hw->hifsys, HIFSYS_DMA_AG_MAP, + BIT(hw->index), 0); out: mutex_unlock(&hw_lock); @@ -653,7 +907,8 @@ mtk_wed_tx_ring_setup(struct mtk_wed_device *dev, int idx, void __iomem *regs) BUG_ON(idx >= ARRAY_SIZE(dev->tx_ring)); - if (mtk_wed_ring_alloc(dev, ring, MTK_WED_TX_RING_SIZE)) + if (mtk_wed_ring_alloc(dev, ring, MTK_WED_TX_RING_SIZE, + sizeof(*ring->desc))) return -ENOMEM; if (mtk_wed_wdma_ring_setup(dev, idx, MTK_WED_WDMA_RING_SIZE)) @@ -680,21 +935,21 @@ static int mtk_wed_txfree_ring_setup(struct mtk_wed_device *dev, void __iomem *regs) { struct mtk_wed_ring *ring = &dev->txfree_ring; - int i; + int i, index = dev->hw->version == 1; /* * For txfree event handling, the same DMA ring is shared between WED * and WLAN. The WLAN driver accesses the ring index registers through * WED */ - ring->reg_base = MTK_WED_RING_RX(1); + ring->reg_base = MTK_WED_RING_RX(index); ring->wpdma = regs; for (i = 0; i < 12; i += 4) { u32 val = readl(regs + i); - wed_w32(dev, MTK_WED_RING_RX(1) + i, val); - wed_w32(dev, MTK_WED_WPDMA_RING_RX(1) + i, val); + wed_w32(dev, MTK_WED_RING_RX(index) + i, val); + wed_w32(dev, MTK_WED_WPDMA_RING_RX(index) + i, val); } return 0; @@ -703,11 +958,19 @@ mtk_wed_txfree_ring_setup(struct mtk_wed_device *dev, void __iomem *regs) static u32 mtk_wed_irq_get(struct mtk_wed_device *dev, u32 mask) { - u32 val; + u32 val, ext_mask = MTK_WED_EXT_INT_STATUS_ERROR_MASK; + + if (dev->hw->version == 1) + ext_mask |= MTK_WED_EXT_INT_STATUS_TX_DRV_R_RESP_ERR; + else + ext_mask |= MTK_WED_EXT_INT_STATUS_RX_FBUF_LO_TH | + MTK_WED_EXT_INT_STATUS_RX_FBUF_HI_TH | + MTK_WED_EXT_INT_STATUS_RX_DRV_COHERENT | + MTK_WED_EXT_INT_STATUS_TX_DMA_W_RESP_ERR; val = wed_r32(dev, MTK_WED_EXT_INT_STATUS); wed_w32(dev, MTK_WED_EXT_INT_STATUS, val); - val &= MTK_WED_EXT_INT_STATUS_ERROR_MASK; + val &= ext_mask; if (!dev->hw->num_flows) val &= ~MTK_WED_EXT_INT_STATUS_TKID_WO_PYLD; if (val && net_ratelimit()) @@ -782,7 +1045,8 @@ out: } void mtk_wed_add_hw(struct device_node *np, struct mtk_eth *eth, - void __iomem *wdma, int index) + void __iomem *wdma, phys_addr_t wdma_phy, + int index) { static const struct mtk_wed_ops wed_ops = { .attach = mtk_wed_attach, @@ -829,26 +1093,33 @@ void mtk_wed_add_hw(struct device_node *np, struct mtk_eth *eth, hw = kzalloc(sizeof(*hw), GFP_KERNEL); if (!hw) goto unlock; + hw->node = np; hw->regs = regs; hw->eth = eth; hw->dev = &pdev->dev; + hw->wdma_phy = wdma_phy; hw->wdma = wdma; hw->index = index; hw->irq = irq; - hw->mirror = syscon_regmap_lookup_by_phandle(eth_np, - "mediatek,pcie-mirror"); - hw->hifsys = syscon_regmap_lookup_by_phandle(eth_np, - "mediatek,hifsys"); - if (IS_ERR(hw->mirror) || IS_ERR(hw->hifsys)) { - kfree(hw); - goto unlock; - } + hw->version = MTK_HAS_CAPS(eth->soc->caps, MTK_NETSYS_V2) ? 2 : 1; + + if (hw->version == 1) { + hw->mirror = syscon_regmap_lookup_by_phandle(eth_np, + "mediatek,pcie-mirror"); + hw->hifsys = syscon_regmap_lookup_by_phandle(eth_np, + "mediatek,hifsys"); + if (IS_ERR(hw->mirror) || IS_ERR(hw->hifsys)) { + kfree(hw); + goto unlock; + } - if (!index) { - regmap_write(hw->mirror, 0, 0); - regmap_write(hw->mirror, 4, 0); + if (!index) { + regmap_write(hw->mirror, 0, 0); + regmap_write(hw->mirror, 4, 0); + } } + mtk_wed_hw_add_debugfs(hw); hw_list[index] = hw; diff --git a/drivers/net/ethernet/mediatek/mtk_wed.h b/drivers/net/ethernet/mediatek/mtk_wed.h index 981ec613f4b0..ae420ca01a48 100644 --- a/drivers/net/ethernet/mediatek/mtk_wed.h +++ b/drivers/net/ethernet/mediatek/mtk_wed.h @@ -18,11 +18,13 @@ struct mtk_wed_hw { struct regmap *hifsys; struct device *dev; void __iomem *wdma; + phys_addr_t wdma_phy; struct regmap *mirror; struct dentry *debugfs_dir; struct mtk_wed_device *wed_dev; u32 debugfs_reg; u32 num_flows; + u8 version; char dirname[5]; int irq; int index; @@ -101,14 +103,16 @@ wpdma_txfree_w32(struct mtk_wed_device *dev, u32 reg, u32 val) } void mtk_wed_add_hw(struct device_node *np, struct mtk_eth *eth, - void __iomem *wdma, int index); + void __iomem *wdma, phys_addr_t wdma_phy, + int index); void mtk_wed_exit(void); int mtk_wed_flow_add(int index); void mtk_wed_flow_remove(int index); #else static inline void mtk_wed_add_hw(struct device_node *np, struct mtk_eth *eth, - void __iomem *wdma, int index) + void __iomem *wdma, phys_addr_t wdma_phy, + int index) { } static inline void diff --git a/drivers/net/ethernet/mediatek/mtk_wed_debugfs.c b/drivers/net/ethernet/mediatek/mtk_wed_debugfs.c index a81d3fd1a439..f420f187e837 100644 --- a/drivers/net/ethernet/mediatek/mtk_wed_debugfs.c +++ b/drivers/net/ethernet/mediatek/mtk_wed_debugfs.c @@ -116,6 +116,9 @@ wed_txinfo_show(struct seq_file *s, void *data) DUMP_WDMA(WDMA_GLO_CFG), DUMP_WDMA_RING(WDMA_RING_RX(0)), DUMP_WDMA_RING(WDMA_RING_RX(1)), + + DUMP_STR("TX FREE"), + DUMP_WED(WED_RX_MIB(0)), }; struct mtk_wed_hw *hw = s->private; struct mtk_wed_device *dev = hw->wed_dev; diff --git a/drivers/net/ethernet/mediatek/mtk_wed_regs.h b/drivers/net/ethernet/mediatek/mtk_wed_regs.h index 0a0465ea58b4..e270fb336143 100644 --- a/drivers/net/ethernet/mediatek/mtk_wed_regs.h +++ b/drivers/net/ethernet/mediatek/mtk_wed_regs.h @@ -5,6 +5,7 @@ #define __MTK_WED_REGS_H #define MTK_WDMA_DESC_CTRL_LEN1 GENMASK(14, 0) +#define MTK_WDMA_DESC_CTRL_LEN1_V2 GENMASK(13, 0) #define MTK_WDMA_DESC_CTRL_LAST_SEG1 BIT(15) #define MTK_WDMA_DESC_CTRL_BURST BIT(16) #define MTK_WDMA_DESC_CTRL_LEN0 GENMASK(29, 16) @@ -41,6 +42,7 @@ struct mtk_wdma_desc { #define MTK_WED_CTRL_RESERVE_EN BIT(12) #define MTK_WED_CTRL_RESERVE_BUSY BIT(13) #define MTK_WED_CTRL_FINAL_DIDX_READ BIT(24) +#define MTK_WED_CTRL_ETH_DMAD_FMT BIT(25) #define MTK_WED_CTRL_MIB_READ_CLEAR BIT(28) #define MTK_WED_EXT_INT_STATUS 0x020 @@ -57,7 +59,8 @@ struct mtk_wdma_desc { #define MTK_WED_EXT_INT_STATUS_RX_DRV_INIT_WDMA_EN BIT(19) #define MTK_WED_EXT_INT_STATUS_RX_DRV_BM_DMAD_COHERENT BIT(20) #define MTK_WED_EXT_INT_STATUS_TX_DRV_R_RESP_ERR BIT(21) -#define MTK_WED_EXT_INT_STATUS_TX_DRV_W_RESP_ERR BIT(22) +#define MTK_WED_EXT_INT_STATUS_TX_DMA_R_RESP_ERR BIT(22) +#define MTK_WED_EXT_INT_STATUS_TX_DMA_W_RESP_ERR BIT(23) #define MTK_WED_EXT_INT_STATUS_RX_DRV_DMA_RECYCLE BIT(24) #define MTK_WED_EXT_INT_STATUS_ERROR_MASK (MTK_WED_EXT_INT_STATUS_TF_LEN_ERR | \ MTK_WED_EXT_INT_STATUS_TKID_WO_PYLD | \ @@ -65,8 +68,7 @@ struct mtk_wdma_desc { MTK_WED_EXT_INT_STATUS_RX_DRV_R_RESP_ERR | \ MTK_WED_EXT_INT_STATUS_RX_DRV_W_RESP_ERR | \ MTK_WED_EXT_INT_STATUS_RX_DRV_INIT_WDMA_EN | \ - MTK_WED_EXT_INT_STATUS_TX_DRV_R_RESP_ERR | \ - MTK_WED_EXT_INT_STATUS_TX_DRV_W_RESP_ERR) + MTK_WED_EXT_INT_STATUS_TX_DMA_R_RESP_ERR) #define MTK_WED_EXT_INT_MASK 0x028 @@ -81,6 +83,7 @@ struct mtk_wdma_desc { #define MTK_WED_TX_BM_BASE 0x084 #define MTK_WED_TX_BM_TKID 0x088 +#define MTK_WED_TX_BM_TKID_V2 0x0c8 #define MTK_WED_TX_BM_TKID_START GENMASK(15, 0) #define MTK_WED_TX_BM_TKID_END GENMASK(31, 16) @@ -94,7 +97,25 @@ struct mtk_wdma_desc { #define MTK_WED_TX_BM_DYN_THR 0x0a0 #define MTK_WED_TX_BM_DYN_THR_LO GENMASK(6, 0) +#define MTK_WED_TX_BM_DYN_THR_LO_V2 GENMASK(8, 0) #define MTK_WED_TX_BM_DYN_THR_HI GENMASK(22, 16) +#define MTK_WED_TX_BM_DYN_THR_HI_V2 GENMASK(24, 16) + +#define MTK_WED_TX_TKID_CTRL 0x0c0 +#define MTK_WED_TX_TKID_CTRL_VLD_GRP_NUM GENMASK(6, 0) +#define MTK_WED_TX_TKID_CTRL_RSV_GRP_NUM GENMASK(22, 16) +#define MTK_WED_TX_TKID_CTRL_PAUSE BIT(28) + +#define MTK_WED_TX_TKID_DYN_THR 0x0e0 +#define MTK_WED_TX_TKID_DYN_THR_LO GENMASK(6, 0) +#define MTK_WED_TX_TKID_DYN_THR_HI GENMASK(22, 16) + +#define MTK_WED_TXP_DW0 0x120 +#define MTK_WED_TXP_DW1 0x124 +#define MTK_WED_WPDMA_WRITE_TXP GENMASK(31, 16) +#define MTK_WED_TXDP_CTRL 0x130 +#define MTK_WED_TXDP_DW9_OVERWR BIT(9) +#define MTK_WED_RX_BM_TKID_MIB 0x1cc #define MTK_WED_INT_STATUS 0x200 #define MTK_WED_INT_MASK 0x204 @@ -125,6 +146,7 @@ struct mtk_wdma_desc { #define MTK_WED_RESET_IDX_RX GENMASK(17, 16) #define MTK_WED_TX_MIB(_n) (0x2a0 + (_n) * 4) +#define MTK_WED_RX_MIB(_n) (0x2e0 + (_n) * 4) #define MTK_WED_RING_TX(_n) (0x300 + (_n) * 0x10) @@ -155,21 +177,64 @@ struct mtk_wdma_desc { #define MTK_WED_WPDMA_GLO_CFG_BYTE_SWAP BIT(29) #define MTK_WED_WPDMA_GLO_CFG_RX_2B_OFFSET BIT(31) +/* CONFIG_MEDIATEK_NETSYS_V2 */ +#define MTK_WED_WPDMA_GLO_CFG_RX_DRV_R0_PKT_PROC BIT(4) +#define MTK_WED_WPDMA_GLO_CFG_RX_DRV_R1_PKT_PROC BIT(5) +#define MTK_WED_WPDMA_GLO_CFG_RX_DRV_R0_CRX_SYNC BIT(6) +#define MTK_WED_WPDMA_GLO_CFG_RX_DRV_R1_CRX_SYNC BIT(7) +#define MTK_WED_WPDMA_GLO_CFG_RX_DRV_EVENT_PKT_FMT_VER GENMASK(18, 16) +#define MTK_WED_WPDMA_GLO_CFG_RX_DRV_UNSUPPORT_FMT BIT(19) +#define MTK_WED_WPDMA_GLO_CFG_RX_DRV_UEVENT_PKT_FMT_CHK BIT(20) +#define MTK_WED_WPDMA_GLO_CFG_RX_DDONE2_WR BIT(21) +#define MTK_WED_WPDMA_GLO_CFG_TX_TKID_KEEP BIT(24) +#define MTK_WED_WPDMA_GLO_CFG_TX_DMAD_DW3_PREV BIT(28) + #define MTK_WED_WPDMA_RESET_IDX 0x50c #define MTK_WED_WPDMA_RESET_IDX_TX GENMASK(3, 0) #define MTK_WED_WPDMA_RESET_IDX_RX GENMASK(17, 16) +#define MTK_WED_WPDMA_CTRL 0x518 +#define MTK_WED_WPDMA_CTRL_SDL1_FIXED BIT(31) + #define MTK_WED_WPDMA_INT_CTRL 0x520 #define MTK_WED_WPDMA_INT_CTRL_SUBRT_ADV BIT(21) +#define MTK_WED_WPDMA_INT_CTRL_SIG_SRC BIT(22) +#define MTK_WED_WPDMA_INT_CTRL_SRC_SEL GENMASK(17, 16) #define MTK_WED_WPDMA_INT_MASK 0x524 +#define MTK_WED_WPDMA_INT_CTRL_TX 0x530 +#define MTK_WED_WPDMA_INT_CTRL_TX0_DONE_EN BIT(0) +#define MTK_WED_WPDMA_INT_CTRL_TX0_DONE_CLR BIT(1) +#define MTK_WED_WPDMA_INT_CTRL_TX0_DONE_TRIG GENMASK(6, 2) +#define MTK_WED_WPDMA_INT_CTRL_TX1_DONE_EN BIT(8) +#define MTK_WED_WPDMA_INT_CTRL_TX1_DONE_CLR BIT(9) +#define MTK_WED_WPDMA_INT_CTRL_TX1_DONE_TRIG GENMASK(14, 10) + +#define MTK_WED_WPDMA_INT_CTRL_RX 0x534 + +#define MTK_WED_WPDMA_INT_CTRL_TX_FREE 0x538 +#define MTK_WED_WPDMA_INT_CTRL_TX_FREE_DONE_EN BIT(0) +#define MTK_WED_WPDMA_INT_CTRL_TX_FREE_DONE_CLR BIT(1) +#define MTK_WED_WPDMA_INT_CTRL_TX_FREE_DONE_TRIG GENMASK(6, 2) + #define MTK_WED_PCIE_CFG_BASE 0x560 +#define MTK_WED_PCIE_CFG_BASE 0x560 +#define MTK_WED_PCIE_CFG_INTM 0x564 +#define MTK_WED_PCIE_CFG_MSIS 0x568 #define MTK_WED_PCIE_INT_TRIGGER 0x570 #define MTK_WED_PCIE_INT_TRIGGER_STATUS BIT(16) +#define MTK_WED_PCIE_INT_CTRL 0x57c +#define MTK_WED_PCIE_INT_CTRL_MSK_EN_POLA BIT(20) +#define MTK_WED_PCIE_INT_CTRL_SRC_SEL GENMASK(17, 16) +#define MTK_WED_PCIE_INT_CTRL_POLL_EN GENMASK(13, 12) + #define MTK_WED_WPDMA_CFG_BASE 0x580 +#define MTK_WED_WPDMA_CFG_INT_MASK 0x584 +#define MTK_WED_WPDMA_CFG_TX 0x588 +#define MTK_WED_WPDMA_CFG_TX_FREE 0x58c #define MTK_WED_WPDMA_TX_MIB(_n) (0x5a0 + (_n) * 4) #define MTK_WED_WPDMA_TX_COHERENT_MIB(_n) (0x5d0 + (_n) * 4) @@ -203,15 +268,24 @@ struct mtk_wdma_desc { #define MTK_WED_WDMA_RESET_IDX_RX GENMASK(17, 16) #define MTK_WED_WDMA_RESET_IDX_DRV GENMASK(25, 24) +#define MTK_WED_WDMA_INT_CLR 0xa24 +#define MTK_WED_WDMA_INT_CLR_RX_DONE GENMASK(17, 16) + #define MTK_WED_WDMA_INT_TRIGGER 0xa28 #define MTK_WED_WDMA_INT_TRIGGER_RX_DONE GENMASK(17, 16) #define MTK_WED_WDMA_INT_CTRL 0xa2c #define MTK_WED_WDMA_INT_CTRL_POLL_SRC_SEL GENMASK(17, 16) +#define MTK_WED_WDMA_CFG_BASE 0xaa0 #define MTK_WED_WDMA_OFFSET0 0xaa4 #define MTK_WED_WDMA_OFFSET1 0xaa8 +#define MTK_WED_WDMA_OFST0_GLO_INTS GENMASK(15, 0) +#define MTK_WED_WDMA_OFST0_GLO_CFG GENMASK(31, 16) +#define MTK_WED_WDMA_OFST1_TX_CTRL GENMASK(15, 0) +#define MTK_WED_WDMA_OFST1_RX_CTRL GENMASK(31, 16) + #define MTK_WED_WDMA_RX_MIB(_n) (0xae0 + (_n) * 4) #define MTK_WED_WDMA_RX_RECYCLE_MIB(_n) (0xae8 + (_n) * 4) #define MTK_WED_WDMA_RX_PROCESSED_MIB(_n) (0xaf0 + (_n) * 4) @@ -221,15 +295,22 @@ struct mtk_wdma_desc { #define MTK_WED_RING_OFS_CPU_IDX 0x08 #define MTK_WED_RING_OFS_DMA_IDX 0x0c +#define MTK_WDMA_RING_TX(_n) (0x000 + (_n) * 0x10) #define MTK_WDMA_RING_RX(_n) (0x100 + (_n) * 0x10) #define MTK_WDMA_GLO_CFG 0x204 -#define MTK_WDMA_GLO_CFG_RX_INFO_PRERES GENMASK(28, 26) +#define MTK_WDMA_GLO_CFG_TX_DMA_EN BIT(0) +#define MTK_WDMA_GLO_CFG_RX_DMA_EN BIT(2) +#define MTK_WDMA_GLO_CFG_RX_INFO3_PRERES BIT(26) +#define MTK_WDMA_GLO_CFG_RX_INFO2_PRERES BIT(27) +#define MTK_WDMA_GLO_CFG_RX_INFO1_PRERES BIT(28) #define MTK_WDMA_RESET_IDX 0x208 #define MTK_WDMA_RESET_IDX_TX GENMASK(3, 0) #define MTK_WDMA_RESET_IDX_RX GENMASK(17, 16) +#define MTK_WDMA_INT_STATUS 0x220 + #define MTK_WDMA_INT_MASK 0x228 #define MTK_WDMA_INT_MASK_TX_DONE GENMASK(3, 0) #define MTK_WDMA_INT_MASK_RX_DONE GENMASK(17, 16) diff --git a/drivers/net/ethernet/mellanox/mlx4/en_cq.c b/drivers/net/ethernet/mellanox/mlx4/en_cq.c index 6affbd241264..1184ac5751e1 100644 --- a/drivers/net/ethernet/mellanox/mlx4/en_cq.c +++ b/drivers/net/ethernet/mellanox/mlx4/en_cq.c @@ -152,7 +152,7 @@ int mlx4_en_activate_cq(struct mlx4_en_priv *priv, struct mlx4_en_cq *cq, break; case RX: cq->mcq.comp = mlx4_en_rx_irq; - netif_napi_add(cq->dev, &cq->napi, mlx4_en_poll_rx_cq, 64); + netif_napi_add(cq->dev, &cq->napi, mlx4_en_poll_rx_cq); napi_enable(&cq->napi); break; case TX_XDP: diff --git a/drivers/net/ethernet/mellanox/mlx4/en_ethtool.c b/drivers/net/ethernet/mellanox/mlx4/en_ethtool.c index 6400a827173c..7d45f1d55f79 100644 --- a/drivers/net/ethernet/mellanox/mlx4/en_ethtool.c +++ b/drivers/net/ethernet/mellanox/mlx4/en_ethtool.c @@ -89,15 +89,15 @@ mlx4_en_get_drvinfo(struct net_device *dev, struct ethtool_drvinfo *drvinfo) struct mlx4_en_priv *priv = netdev_priv(dev); struct mlx4_en_dev *mdev = priv->mdev; - strlcpy(drvinfo->driver, DRV_NAME, sizeof(drvinfo->driver)); - strlcpy(drvinfo->version, DRV_VERSION, + strscpy(drvinfo->driver, DRV_NAME, sizeof(drvinfo->driver)); + strscpy(drvinfo->version, DRV_VERSION, sizeof(drvinfo->version)); snprintf(drvinfo->fw_version, sizeof(drvinfo->fw_version), "%d.%d.%d", (u16) (mdev->dev->caps.fw_ver >> 32), (u16) ((mdev->dev->caps.fw_ver >> 16) & 0xffff), (u16) (mdev->dev->caps.fw_ver & 0xffff)); - strlcpy(drvinfo->bus_info, pci_name(mdev->dev->persist->pdev), + strscpy(drvinfo->bus_info, pci_name(mdev->dev->persist->pdev), sizeof(drvinfo->bus_info)); } diff --git a/drivers/net/ethernet/mellanox/mlx4/fw.c b/drivers/net/ethernet/mellanox/mlx4/fw.c index dcb9eb1899ce..fe48d20d6118 100644 --- a/drivers/net/ethernet/mellanox/mlx4/fw.c +++ b/drivers/net/ethernet/mellanox/mlx4/fw.c @@ -1779,7 +1779,7 @@ static void get_board_id(void *vsd, char *board_id) if (be16_to_cpup(vsd + VSD_OFFSET_SIG1) == VSD_SIGNATURE_TOPSPIN && be16_to_cpup(vsd + VSD_OFFSET_SIG2) == VSD_SIGNATURE_TOPSPIN) { - strlcpy(board_id, vsd + VSD_OFFSET_TS_BOARD_ID, MLX4_BOARD_ID_LEN); + strscpy(board_id, vsd + VSD_OFFSET_TS_BOARD_ID, MLX4_BOARD_ID_LEN); } else { /* * The board ID is a string but the firmware byte diff --git a/drivers/net/ethernet/mellanox/mlx4/icm.c b/drivers/net/ethernet/mellanox/mlx4/icm.c index d89a3da89e5a..59b8b3c73582 100644 --- a/drivers/net/ethernet/mellanox/mlx4/icm.c +++ b/drivers/net/ethernet/mellanox/mlx4/icm.c @@ -208,7 +208,7 @@ struct mlx4_icm *mlx4_alloc_icm(struct mlx4_dev *dev, int npages, chunk->sg, chunk->npages, DMA_BIDIRECTIONAL); - if (chunk->nsg <= 0) + if (!chunk->nsg) goto fail; } @@ -222,7 +222,7 @@ struct mlx4_icm *mlx4_alloc_icm(struct mlx4_dev *dev, int npages, chunk->nsg = dma_map_sg(&dev->persist->pdev->dev, chunk->sg, chunk->npages, DMA_BIDIRECTIONAL); - if (chunk->nsg <= 0) + if (!chunk->nsg) goto fail; } diff --git a/drivers/net/ethernet/mellanox/mlx4/main.c b/drivers/net/ethernet/mellanox/mlx4/main.c index 78c5f40382c9..d3fc86cd3c1d 100644 --- a/drivers/net/ethernet/mellanox/mlx4/main.c +++ b/drivers/net/ethernet/mellanox/mlx4/main.c @@ -3071,6 +3071,7 @@ static int mlx4_init_port_info(struct mlx4_dev *dev, int port) err = device_create_file(&dev->persist->pdev->dev, &info->port_attr); if (err) { mlx4_err(dev, "Failed to create file for port %d\n", port); + devlink_port_type_clear(&info->devlink_port); devl_port_unregister(&info->devlink_port); info->port = -1; return err; @@ -3093,6 +3094,7 @@ static int mlx4_init_port_info(struct mlx4_dev *dev, int port) mlx4_err(dev, "Failed to create mtu file for port %d\n", port); device_remove_file(&info->dev->persist->pdev->dev, &info->port_attr); + devlink_port_type_clear(&info->devlink_port); devl_port_unregister(&info->devlink_port); info->port = -1; return err; @@ -3109,6 +3111,7 @@ static void mlx4_cleanup_port_info(struct mlx4_port_info *info) device_remove_file(&info->dev->persist->pdev->dev, &info->port_attr); device_remove_file(&info->dev->persist->pdev->dev, &info->port_mtu_attr); + devlink_port_type_clear(&info->devlink_port); devl_port_unregister(&info->devlink_port); #ifdef CONFIG_RFS_ACCEL diff --git a/drivers/net/ethernet/mellanox/mlx5/core/Kconfig b/drivers/net/ethernet/mellanox/mlx5/core/Kconfig index bfc0cd5ec423..26685fd0fdaa 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/Kconfig +++ b/drivers/net/ethernet/mellanox/mlx5/core/Kconfig @@ -139,6 +139,14 @@ config MLX5_CORE_IPOIB help MLX5 IPoIB offloads & acceleration support. +config MLX5_EN_MACSEC + bool "Connect-X support for MACSec offload" + depends on MLX5_CORE_EN + depends on MACSEC + default n + help + Build support for MACsec cryptography-offload acceleration in the NIC. + config MLX5_EN_IPSEC bool "Mellanox Technologies IPsec Connect-X support" depends on MLX5_CORE_EN diff --git a/drivers/net/ethernet/mellanox/mlx5/core/Makefile b/drivers/net/ethernet/mellanox/mlx5/core/Makefile index a3773a8177ed..a22c32aabf11 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/Makefile +++ b/drivers/net/ethernet/mellanox/mlx5/core/Makefile @@ -92,6 +92,9 @@ mlx5_core-$(CONFIG_MLX5_CORE_IPOIB) += ipoib/ipoib.o ipoib/ethtool.o ipoib/ipoib # mlx5_core-$(CONFIG_MLX5_FPGA) += fpga/cmd.o fpga/core.o fpga/conn.o fpga/sdk.o +mlx5_core-$(CONFIG_MLX5_EN_MACSEC) += en_accel/macsec.o en_accel/macsec_fs.o \ + en_accel/macsec_stats.o + mlx5_core-$(CONFIG_MLX5_EN_IPSEC) += en_accel/ipsec.o en_accel/ipsec_rxtx.o \ en_accel/ipsec_stats.o en_accel/ipsec_fs.o \ en_accel/ipsec_offload.o diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en.h b/drivers/net/ethernet/mellanox/mlx5/core/en.h index a560df446bac..26a23047f1f3 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en.h @@ -93,29 +93,26 @@ struct page_pool; #define MLX5_MPWRQ_DEF_LOG_STRIDE_SZ(mdev) \ MLX5_MPWRQ_LOG_STRIDE_SZ(mdev, order_base_2(MLX5E_RX_MAX_HEAD)) -#define MLX5_MPWRQ_LOG_WQE_SZ 18 -#define MLX5_MPWRQ_WQE_PAGE_ORDER (MLX5_MPWRQ_LOG_WQE_SZ - PAGE_SHIFT > 0 ? \ - MLX5_MPWRQ_LOG_WQE_SZ - PAGE_SHIFT : 0) -#define MLX5_MPWRQ_PAGES_PER_WQE BIT(MLX5_MPWRQ_WQE_PAGE_ORDER) - -#define MLX5_ALIGN_MTTS(mtts) (ALIGN(mtts, 8)) -#define MLX5_ALIGNED_MTTS_OCTW(mtts) ((mtts) / 2) -#define MLX5_MTT_OCTW(mtts) (MLX5_ALIGNED_MTTS_OCTW(MLX5_ALIGN_MTTS(mtts))) -/* Add another page to MLX5E_REQUIRED_WQE_MTTS as a buffer between - * WQEs, This page will absorb write overflow by the hardware, when - * receiving packets larger than MTU. These oversize packets are - * dropped by the driver at a later stage. +#define MLX5_MPWRQ_MAX_LOG_WQE_SZ 18 + +/* Keep in sync with mlx5e_mpwrq_log_wqe_sz. + * These are theoretical maximums, which can be further restricted by + * capabilities. These values are used for static resource allocations and + * sanity checks. + * MLX5_SEND_WQE_MAX_SIZE is a bit bigger than the maximum cacheline-aligned WQE + * size actually used at runtime, but it's not a problem when calculating static + * array sizes. */ -#define MLX5E_REQUIRED_WQE_MTTS (MLX5_ALIGN_MTTS(MLX5_MPWRQ_PAGES_PER_WQE + 1)) -#define MLX5E_REQUIRED_MTTS(wqes) (wqes * MLX5E_REQUIRED_WQE_MTTS) +#define MLX5_UMR_MAX_MTT_SPACE \ + (ALIGN_DOWN(MLX5_SEND_WQE_MAX_SIZE - sizeof(struct mlx5e_umr_wqe), \ + MLX5_UMR_MTT_ALIGNMENT)) +#define MLX5_MPWRQ_MAX_PAGES_PER_WQE \ + rounddown_pow_of_two(MLX5_UMR_MAX_MTT_SPACE / sizeof(struct mlx5_mtt)) + #define MLX5E_MAX_RQ_NUM_MTTS \ - (ALIGN_DOWN(U16_MAX, 4) * 2) /* So that MLX5_MTT_OCTW(num_mtts) fits into u16 */ + (ALIGN_DOWN(U16_MAX, 4) * 2) /* Fits into u16 and aligned by WQEBB. */ +#define MLX5E_MAX_RQ_NUM_KSMS (U16_MAX - 1) /* So that num_ksms fits into u16. */ #define MLX5E_ORDER2_MAX_PACKET_MTU (order_base_2(10 * 1024)) -#define MLX5E_PARAMS_MAXIMUM_LOG_RQ_SIZE_MPW \ - (ilog2(MLX5E_MAX_RQ_NUM_MTTS / MLX5E_REQUIRED_WQE_MTTS)) -#define MLX5E_LOG_MAX_RQ_NUM_PACKETS_MPW \ - (MLX5E_PARAMS_MAXIMUM_LOG_RQ_SIZE_MPW + \ - (MLX5_MPWRQ_LOG_WQE_SZ - MLX5E_ORDER2_MAX_PACKET_MTU)) #define MLX5E_MIN_SKB_FRAG_SZ (MLX5_SKB_FRAG_SZ(MLX5_RX_HEADROOM)) #define MLX5E_LOG_MAX_RX_WQE_BULK \ @@ -127,8 +124,7 @@ struct page_pool; #define MLX5E_PARAMS_MINIMUM_LOG_RQ_SIZE (1 + MLX5E_LOG_MAX_RX_WQE_BULK) #define MLX5E_PARAMS_DEFAULT_LOG_RQ_SIZE 0xa -#define MLX5E_PARAMS_MAXIMUM_LOG_RQ_SIZE min_t(u8, 0xd, \ - MLX5E_LOG_MAX_RQ_NUM_PACKETS_MPW) +#define MLX5E_PARAMS_MAXIMUM_LOG_RQ_SIZE 0xd #define MLX5E_PARAMS_MINIMUM_LOG_RQ_SIZE_MPW 0x2 @@ -150,13 +146,6 @@ struct page_pool; #define MLX5E_TX_XSK_POLL_BUDGET 64 #define MLX5E_SQ_RECOVER_MIN_INTERVAL 500 /* msecs */ -#define MLX5E_UMR_WQE_INLINE_SZ \ - (sizeof(struct mlx5e_umr_wqe) + \ - ALIGN(MLX5_MPWRQ_PAGES_PER_WQE * sizeof(struct mlx5_mtt), \ - MLX5_UMR_MTT_ALIGNMENT)) -#define MLX5E_UMR_WQEBBS \ - (DIV_ROUND_UP(MLX5E_UMR_WQE_INLINE_SZ, MLX5_SEND_WQE_BB)) - #define MLX5E_KLM_UMR_WQE_SZ(sgl_len)\ (sizeof(struct mlx5e_umr_wqe) +\ (sizeof(struct mlx5_klm) * (sgl_len))) @@ -174,8 +163,7 @@ struct page_pool; ALIGN_DOWN(MLX5E_KLM_MAX_ENTRIES_PER_WQE(wqe_size), MLX5_UMR_KLM_ALIGNMENT) #define MLX5E_MAX_KLM_PER_WQE(mdev) \ - MLX5E_KLM_ENTRIES_PER_WQE(MLX5_SEND_WQE_BB * \ - mlx5e_get_sw_max_sq_mpw_wqebbs(mlx5e_get_max_sq_wqebbs(mdev))) + MLX5E_KLM_ENTRIES_PER_WQE(MLX5_SEND_WQE_BB * mlx5e_get_max_sq_aligned_wqebbs(mdev)) #define MLX5E_MSG_LEVEL NETIF_MSG_LINK @@ -189,12 +177,6 @@ do { \ #define mlx5e_state_dereference(priv, p) \ rcu_dereference_protected((p), lockdep_is_held(&(priv)->state_lock)) -enum mlx5e_rq_group { - MLX5E_RQ_GROUP_REGULAR, - MLX5E_RQ_GROUP_XSK, -#define MLX5E_NUM_RQ_GROUPS(g) (1 + MLX5E_RQ_GROUP_##g) -}; - static inline u8 mlx5e_get_num_lag_ports(struct mlx5_core_dev *mdev) { if (mlx5_lag_is_lacp_owner(mdev)) @@ -227,13 +209,15 @@ static inline int mlx5e_get_max_num_channels(struct mlx5_core_dev *mdev) * bytes units. Driver hardens the limitation to 1KB (16 * WQEBBs), unless firmware capability is stricter. */ -static inline u16 mlx5e_get_max_sq_wqebbs(struct mlx5_core_dev *mdev) +static inline u8 mlx5e_get_max_sq_wqebbs(struct mlx5_core_dev *mdev) { - return min_t(u16, MLX5_SEND_WQE_MAX_WQEBBS, - MLX5_CAP_GEN(mdev, max_wqe_sz_sq) / MLX5_SEND_WQE_BB); + BUILD_BUG_ON(MLX5_SEND_WQE_MAX_WQEBBS > U8_MAX); + + return (u8)min_t(u16, MLX5_SEND_WQE_MAX_WQEBBS, + MLX5_CAP_GEN(mdev, max_wqe_sz_sq) / MLX5_SEND_WQE_BB); } -static inline u8 mlx5e_get_sw_max_sq_mpw_wqebbs(u8 max_sq_wqebbs) +static inline u8 mlx5e_get_max_sq_aligned_wqebbs(struct mlx5_core_dev *mdev) { /* The return value will be multiplied by MLX5_SEND_WQEBB_NUM_DS. * Since max_sq_wqebbs may be up to MLX5_SEND_WQE_MAX_WQEBBS == 16, @@ -242,8 +226,9 @@ static inline u8 mlx5e_get_sw_max_sq_mpw_wqebbs(u8 max_sq_wqebbs) * than MLX5_SEND_WQE_MAX_WQEBBS to let a full-session WQE be * cache-aligned. */ - u8 wqebbs = min_t(u8, max_sq_wqebbs, MLX5_SEND_WQE_MAX_WQEBBS - 1); + u8 wqebbs = mlx5e_get_max_sq_wqebbs(mdev); + wqebbs = min_t(u8, wqebbs, MLX5_SEND_WQE_MAX_WQEBBS - 1); #if L1_CACHE_BYTES >= 128 wqebbs = ALIGN_DOWN(wqebbs, 2); #endif @@ -272,6 +257,7 @@ struct mlx5e_umr_wqe { union { DECLARE_FLEX_ARRAY(struct mlx5_mtt, inline_mtts); DECLARE_FLEX_ARRAY(struct mlx5_klm, inline_klms); + DECLARE_FLEX_ARRAY(struct mlx5_ksm, inline_ksms); }; }; @@ -476,15 +462,11 @@ struct mlx5e_txqsq { struct work_struct recover_work; struct mlx5e_ptpsq *ptpsq; cqe_ts_to_ns ptp_cyc2time; - u16 max_sq_wqebbs; } ____cacheline_aligned_in_smp; -struct mlx5e_dma_info { - dma_addr_t addr; - union { - struct page *page; - struct xdp_buff *xsk; - }; +union mlx5e_alloc_unit { + struct page *page; + struct xdp_buff *xsk; }; /* XDP packets can be transmitted in different ways. On completion, we need to @@ -580,7 +562,6 @@ struct mlx5e_xdpsq { /* control path */ struct mlx5_wq_ctrl wq_ctrl; struct mlx5e_channel *channel; - u16 max_sq_wqebbs; } ____cacheline_aligned_in_smp; struct mlx5e_ktls_resync_resp; @@ -609,25 +590,20 @@ struct mlx5e_icosq { /* control path */ struct mlx5_wq_ctrl wq_ctrl; struct mlx5e_channel *channel; - u16 max_sq_wqebbs; struct work_struct recover_work; } ____cacheline_aligned_in_smp; struct mlx5e_wqe_frag_info { - struct mlx5e_dma_info *di; + union mlx5e_alloc_unit *au; u32 offset; bool last_in_page; }; -struct mlx5e_umr_dma_info { - struct mlx5e_dma_info dma_info[MLX5_MPWRQ_PAGES_PER_WQE]; -}; - struct mlx5e_mpw_info { - struct mlx5e_umr_dma_info umr; u16 consumed_strides; - DECLARE_BITMAP(xdp_xmit_bitmap, MLX5_MPWRQ_PAGES_PER_WQE); + DECLARE_BITMAP(xdp_xmit_bitmap, MLX5_MPWRQ_MAX_PAGES_PER_WQE); + union mlx5e_alloc_unit alloc_units[]; }; #define MLX5E_MAX_RX_FRAGS 4 @@ -635,13 +611,13 @@ struct mlx5e_mpw_info { /* a single cache unit is capable to serve one napi call (for non-striding rq) * or a MPWQE (for striding rq). */ -#define MLX5E_CACHE_UNIT (MLX5_MPWRQ_PAGES_PER_WQE > NAPI_POLL_WEIGHT ? \ - MLX5_MPWRQ_PAGES_PER_WQE : NAPI_POLL_WEIGHT) +#define MLX5E_CACHE_UNIT (MLX5_MPWRQ_MAX_PAGES_PER_WQE > NAPI_POLL_WEIGHT ? \ + MLX5_MPWRQ_MAX_PAGES_PER_WQE : NAPI_POLL_WEIGHT) #define MLX5E_CACHE_SIZE (4 * roundup_pow_of_two(MLX5E_CACHE_UNIT)) struct mlx5e_page_cache { u32 head; u32 tail; - struct mlx5e_dma_info page_cache[MLX5E_CACHE_SIZE]; + struct page *page_cache[MLX5E_CACHE_SIZE]; }; struct mlx5e_rq; @@ -674,6 +650,12 @@ struct mlx5e_rq_frags_info { u8 num_frags; u8 log_num_frags; u8 wqe_bulk; + u8 wqe_index_mask; +}; + +struct mlx5e_dma_info { + dma_addr_t addr; + struct page *page; }; struct mlx5e_shampo_hd { @@ -695,13 +677,20 @@ struct mlx5e_hw_gro_data { int second_ip_id; }; +enum mlx5e_mpwrq_umr_mode { + MLX5E_MPWRQ_UMR_MODE_ALIGNED, + MLX5E_MPWRQ_UMR_MODE_UNALIGNED, + MLX5E_MPWRQ_UMR_MODE_OVERSIZED, + MLX5E_MPWRQ_UMR_MODE_TRIPLE, +}; + struct mlx5e_rq { /* data path */ union { struct { struct mlx5_wq_cyc wq; struct mlx5e_wqe_frag_info *frags; - struct mlx5e_dma_info *di; + union mlx5e_alloc_unit *alloc_units; struct mlx5e_rq_frags_info info; mlx5e_fp_skb_from_cqe skb_from_cqe; } wqe; @@ -710,6 +699,7 @@ struct mlx5e_rq { struct mlx5e_umr_wqe umr_wqe; struct mlx5e_mpw_info *info; mlx5e_fp_skb_from_cqe_mpwrq skb_from_cqe_mpwrq; + __be32 umr_mkey_be; u16 num_strides; u16 actual_wq_head; u8 log_stride_sz; @@ -717,6 +707,11 @@ struct mlx5e_rq { u8 umr_last_bulk; u8 umr_completed; u8 min_wqe_bulk; + u8 page_shift; + u8 pages_per_wqe; + u8 umr_wqebbs; + u8 mtts_per_wqe; + u8 umr_mode; struct mlx5e_shampo_hd *shampo; } mpwqe; }; @@ -767,7 +762,6 @@ struct mlx5e_rq { u32 rqn; struct mlx5_core_dev *mdev; struct mlx5e_channel *channel; - u32 umr_mkey; struct mlx5e_dma_info wqe_overflow; /* XDP read-mostly */ @@ -856,11 +850,6 @@ enum { MLX5E_STATE_XDP_ACTIVE, }; -enum { - MLX5E_TC_PRIO = 0, - MLX5E_NIC_PRIO -}; - struct mlx5e_modify_sq_param { int curr_state; int next_state; @@ -959,6 +948,9 @@ struct mlx5e_priv { const struct mlx5e_profile *profile; void *ppriv; +#ifdef CONFIG_MLX5_EN_MACSEC + struct mlx5e_macsec *macsec; +#endif #ifdef CONFIG_MLX5_EN_IPSEC struct mlx5e_ipsec *ipsec; #endif @@ -1010,7 +1002,6 @@ struct mlx5e_profile { mlx5e_stats_grp_t *stats_grps; const struct mlx5e_rx_handlers *rx_handlers; int max_tc; - u8 rq_groups; u32 features; }; @@ -1019,7 +1010,8 @@ struct mlx5e_profile { void mlx5e_build_ptys2ethtool_map(void); -bool mlx5e_check_fragmented_striding_rq_cap(struct mlx5_core_dev *mdev); +bool mlx5e_check_fragmented_striding_rq_cap(struct mlx5_core_dev *mdev, u8 page_shift, + enum mlx5e_mpwrq_umr_mode umr_mode); void mlx5e_shampo_dealloc_hd(struct mlx5e_rq *rq, u16 len, u16 start, bool close); void mlx5e_get_stats(struct net_device *dev, struct rtnl_link_stats64 *stats); @@ -1047,6 +1039,7 @@ struct mlx5e_rq_param; int mlx5e_open_rq(struct mlx5e_params *params, struct mlx5e_rq_param *param, struct mlx5e_xsk_param *xsk, int node, struct mlx5e_rq *rq); +#define MLX5E_RQ_WQES_TIMEOUT 20000 /* msecs */ int mlx5e_wait_for_min_rx_wqes(struct mlx5e_rq *rq, int wait_time); void mlx5e_close_rq(struct mlx5e_rq *rq); int mlx5e_create_rq(struct mlx5e_rq *rq, struct mlx5e_rq_param *param); @@ -1101,7 +1094,7 @@ void mlx5e_activate_priv_channels(struct mlx5e_priv *priv); void mlx5e_deactivate_priv_channels(struct mlx5e_priv *priv); int mlx5e_ptp_rx_manage_fs_ctx(struct mlx5e_priv *priv, void *ctx); -int mlx5e_modify_rq_state(struct mlx5e_rq *rq, int curr_state, int next_state); +int mlx5e_flush_rq(struct mlx5e_rq *rq, int curr_state); void mlx5e_activate_rq(struct mlx5e_rq *rq); void mlx5e_deactivate_rq(struct mlx5e_rq *rq); void mlx5e_activate_icosq(struct mlx5e_icosq *icosq); @@ -1136,6 +1129,7 @@ static inline bool mlx5_tx_swp_supported(struct mlx5_core_dev *mdev) extern const struct ethtool_ops mlx5e_ethtool_ops; +int mlx5e_create_mkey(struct mlx5_core_dev *mdev, u32 pdn, u32 *mkey); int mlx5e_create_mdev_resources(struct mlx5_core_dev *mdev); void mlx5e_destroy_mdev_resources(struct mlx5_core_dev *mdev); int mlx5e_refresh_tirs(struct mlx5e_priv *priv, bool enable_uc_lb, @@ -1148,8 +1142,6 @@ void mlx5e_destroy_q_counters(struct mlx5e_priv *priv); int mlx5e_open_drop_rq(struct mlx5e_priv *priv, struct mlx5e_rq *drop_rq); void mlx5e_close_drop_rq(struct mlx5e_rq *drop_rq); -int mlx5e_init_di_list(struct mlx5e_rq *rq, int wq_sz, int node); -void mlx5e_free_di_list(struct mlx5e_rq *rq); int mlx5e_create_tis(struct mlx5_core_dev *mdev, void *in, u32 *tisn); void mlx5e_destroy_tis(struct mlx5_core_dev *mdev, u32 tisn); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/channels.c b/drivers/net/ethernet/mellanox/mlx5/core/en/channels.c index e7c14c0de0a7..48581ea3adcb 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/channels.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/channels.c @@ -10,28 +10,33 @@ unsigned int mlx5e_channels_get_num(struct mlx5e_channels *chs) return chs->num; } -void mlx5e_channels_get_regular_rqn(struct mlx5e_channels *chs, unsigned int ix, u32 *rqn) +static struct mlx5e_channel *mlx5e_channels_get(struct mlx5e_channels *chs, unsigned int ix) { - struct mlx5e_channel *c; + WARN_ON_ONCE(ix >= mlx5e_channels_get_num(chs)); + return chs->c[ix]; +} - WARN_ON(ix >= mlx5e_channels_get_num(chs)); - c = chs->c[ix]; +bool mlx5e_channels_is_xsk(struct mlx5e_channels *chs, unsigned int ix) +{ + struct mlx5e_channel *c = mlx5e_channels_get(chs, ix); - *rqn = c->rq.rqn; + return test_bit(MLX5E_CHANNEL_STATE_XSK, c->state); } -bool mlx5e_channels_get_xsk_rqn(struct mlx5e_channels *chs, unsigned int ix, u32 *rqn) +void mlx5e_channels_get_regular_rqn(struct mlx5e_channels *chs, unsigned int ix, u32 *rqn) { - struct mlx5e_channel *c; + struct mlx5e_channel *c = mlx5e_channels_get(chs, ix); - WARN_ON(ix >= mlx5e_channels_get_num(chs)); - c = chs->c[ix]; + *rqn = c->rq.rqn; +} - if (!test_bit(MLX5E_CHANNEL_STATE_XSK, c->state)) - return false; +void mlx5e_channels_get_xsk_rqn(struct mlx5e_channels *chs, unsigned int ix, u32 *rqn) +{ + struct mlx5e_channel *c = mlx5e_channels_get(chs, ix); + + WARN_ON_ONCE(!test_bit(MLX5E_CHANNEL_STATE_XSK, c->state)); *rqn = c->xskrq.rqn; - return true; } bool mlx5e_channels_get_ptp_rqn(struct mlx5e_channels *chs, u32 *rqn) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/channels.h b/drivers/net/ethernet/mellanox/mlx5/core/en/channels.h index ca00cbc827cb..637ca90daaa8 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/channels.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/channels.h @@ -9,8 +9,9 @@ struct mlx5e_channels; unsigned int mlx5e_channels_get_num(struct mlx5e_channels *chs); +bool mlx5e_channels_is_xsk(struct mlx5e_channels *chs, unsigned int ix); void mlx5e_channels_get_regular_rqn(struct mlx5e_channels *chs, unsigned int ix, u32 *rqn); -bool mlx5e_channels_get_xsk_rqn(struct mlx5e_channels *chs, unsigned int ix, u32 *rqn); +void mlx5e_channels_get_xsk_rqn(struct mlx5e_channels *chs, unsigned int ix, u32 *rqn); bool mlx5e_channels_get_ptp_rqn(struct mlx5e_channels *chs, u32 *rqn); #endif /* __MLX5_EN_CHANNELS_H__ */ diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/fs.h b/drivers/net/ethernet/mellanox/mlx5/core/en/fs.h index 9b8cdf2e68ad..bf2741eb7f9b 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/fs.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/fs.h @@ -8,6 +8,7 @@ #include "lib/fs_ttc.h" struct mlx5e_post_act; +struct mlx5e_tc_table; enum { MLX5E_TC_FT_LEVEL = 0, @@ -15,6 +16,11 @@ enum { MLX5E_TC_MISS_LEVEL, }; +enum { + MLX5E_TC_PRIO = 0, + MLX5E_NIC_PRIO +}; + struct mlx5e_flow_table { int num_groups; struct mlx5_flow_table *t; @@ -83,54 +89,28 @@ enum { #endif }; -struct mlx5e_priv; - -#ifdef CONFIG_MLX5_EN_RXNFC - -struct mlx5e_ethtool_table { - struct mlx5_flow_table *ft; - int num_rules; -}; - -#define ETHTOOL_NUM_L3_L4_FTS 7 -#define ETHTOOL_NUM_L2_FTS 4 - -struct mlx5e_ethtool_steering { - struct mlx5e_ethtool_table l3_l4_ft[ETHTOOL_NUM_L3_L4_FTS]; - struct mlx5e_ethtool_table l2_ft[ETHTOOL_NUM_L2_FTS]; - struct list_head rules; - int tot_num_rules; -}; - -void mlx5e_ethtool_init_steering(struct mlx5e_priv *priv); -void mlx5e_ethtool_cleanup_steering(struct mlx5e_priv *priv); -int mlx5e_ethtool_set_rxnfc(struct mlx5e_priv *priv, struct ethtool_rxnfc *cmd); -int mlx5e_ethtool_get_rxnfc(struct mlx5e_priv *priv, - struct ethtool_rxnfc *info, u32 *rule_locs); -#else -static inline void mlx5e_ethtool_init_steering(struct mlx5e_priv *priv) { } -static inline void mlx5e_ethtool_cleanup_steering(struct mlx5e_priv *priv) { } -static inline int mlx5e_ethtool_set_rxnfc(struct mlx5e_priv *priv, struct ethtool_rxnfc *cmd) -{ return -EOPNOTSUPP; } -static inline int mlx5e_ethtool_get_rxnfc(struct mlx5e_priv *priv, - struct ethtool_rxnfc *info, u32 *rule_locs) -{ return -EOPNOTSUPP; } -#endif /* CONFIG_MLX5_EN_RXNFC */ +struct mlx5e_flow_steering; +struct mlx5e_rx_res; #ifdef CONFIG_MLX5_EN_ARFS struct mlx5e_arfs_tables; -int mlx5e_arfs_create_tables(struct mlx5e_priv *priv); -void mlx5e_arfs_destroy_tables(struct mlx5e_priv *priv); -int mlx5e_arfs_enable(struct mlx5e_priv *priv); -int mlx5e_arfs_disable(struct mlx5e_priv *priv); +int mlx5e_arfs_create_tables(struct mlx5e_flow_steering *fs, + struct mlx5e_rx_res *rx_res, bool ntuple); +void mlx5e_arfs_destroy_tables(struct mlx5e_flow_steering *fs, bool ntuple); +int mlx5e_arfs_enable(struct mlx5e_flow_steering *fs); +int mlx5e_arfs_disable(struct mlx5e_flow_steering *fs); int mlx5e_rx_flow_steer(struct net_device *dev, const struct sk_buff *skb, u16 rxq_index, u32 flow_id); #else -static inline int mlx5e_arfs_create_tables(struct mlx5e_priv *priv) { return 0; } -static inline void mlx5e_arfs_destroy_tables(struct mlx5e_priv *priv) {} -static inline int mlx5e_arfs_enable(struct mlx5e_priv *priv) { return -EOPNOTSUPP; } -static inline int mlx5e_arfs_disable(struct mlx5e_priv *priv) { return -EOPNOTSUPP; } +static inline int mlx5e_arfs_create_tables(struct mlx5e_flow_steering *fs, + struct mlx5e_rx_res *rx_res, bool ntuple) +{ return 0; } +static inline void mlx5e_arfs_destroy_tables(struct mlx5e_flow_steering *fs, bool ntuple) {} +static inline int mlx5e_arfs_enable(struct mlx5e_flow_steering *fs) +{ return -EOPNOTSUPP; } +static inline int mlx5e_arfs_disable(struct mlx5e_flow_steering *fs) +{ return -EOPNOTSUPP; } #endif #ifdef CONFIG_MLX5_EN_TLS @@ -142,54 +122,63 @@ struct mlx5e_fs_udp; struct mlx5e_fs_any; struct mlx5e_ptp_fs; -struct mlx5e_flow_steering { - bool state_destroy; - bool vlan_strip_disable; - struct mlx5_core_dev *mdev; - struct mlx5_flow_namespace *ns; -#ifdef CONFIG_MLX5_EN_RXNFC - struct mlx5e_ethtool_steering ethtool; -#endif - struct mlx5e_tc_table *tc; - struct mlx5e_promisc_table promisc; - struct mlx5e_vlan_table *vlan; - struct mlx5e_l2_table l2; - struct mlx5_ttc_table *ttc; - struct mlx5_ttc_table *inner_ttc; -#ifdef CONFIG_MLX5_EN_ARFS - struct mlx5e_arfs_tables *arfs; -#endif -#ifdef CONFIG_MLX5_EN_TLS - struct mlx5e_accel_fs_tcp *accel_tcp; -#endif - struct mlx5e_fs_udp *udp; - struct mlx5e_fs_any *any; - struct mlx5e_ptp_fs *ptp_fs; -}; - -void mlx5e_set_ttc_params(struct mlx5e_priv *priv, +void mlx5e_set_ttc_params(struct mlx5e_flow_steering *fs, + struct mlx5e_rx_res *rx_res, struct ttc_params *ttc_params, bool tunnel); -void mlx5e_destroy_ttc_table(struct mlx5e_priv *priv); -int mlx5e_create_ttc_table(struct mlx5e_priv *priv); +void mlx5e_destroy_ttc_table(struct mlx5e_flow_steering *fs); +int mlx5e_create_ttc_table(struct mlx5e_flow_steering *fs, + struct mlx5e_rx_res *rx_res); void mlx5e_destroy_flow_table(struct mlx5e_flow_table *ft); -void mlx5e_enable_cvlan_filter(struct mlx5e_priv *priv); -void mlx5e_disable_cvlan_filter(struct mlx5e_priv *priv); +void mlx5e_enable_cvlan_filter(struct mlx5e_flow_steering *fs, bool promisc); +void mlx5e_disable_cvlan_filter(struct mlx5e_flow_steering *fs, bool promisc); -int mlx5e_create_flow_steering(struct mlx5e_priv *priv); -void mlx5e_destroy_flow_steering(struct mlx5e_priv *priv); +int mlx5e_create_flow_steering(struct mlx5e_flow_steering *fs, + struct mlx5e_rx_res *rx_res, + const struct mlx5e_profile *profile, + struct net_device *netdev); +void mlx5e_destroy_flow_steering(struct mlx5e_flow_steering *fs, bool ntuple, + const struct mlx5e_profile *profile); struct mlx5e_flow_steering *mlx5e_fs_init(const struct mlx5e_profile *profile, struct mlx5_core_dev *mdev, bool state_destroy); void mlx5e_fs_cleanup(struct mlx5e_flow_steering *fs); - -int mlx5e_add_vlan_trap(struct mlx5e_priv *priv, int trap_id, int tir_num); -void mlx5e_remove_vlan_trap(struct mlx5e_priv *priv); -int mlx5e_add_mac_trap(struct mlx5e_priv *priv, int trap_id, int tir_num); -void mlx5e_remove_mac_trap(struct mlx5e_priv *priv); +struct mlx5e_vlan_table *mlx5e_fs_get_vlan(struct mlx5e_flow_steering *fs); +void mlx5e_fs_set_tc(struct mlx5e_flow_steering *fs, struct mlx5e_tc_table *tc); +struct mlx5e_tc_table *mlx5e_fs_get_tc(struct mlx5e_flow_steering *fs); +struct mlx5e_l2_table *mlx5e_fs_get_l2(struct mlx5e_flow_steering *fs); +struct mlx5_flow_namespace *mlx5e_fs_get_ns(struct mlx5e_flow_steering *fs, bool egress); +void mlx5e_fs_set_ns(struct mlx5e_flow_steering *fs, struct mlx5_flow_namespace *ns, bool egress); +#ifdef CONFIG_MLX5_EN_RXNFC +struct mlx5e_ethtool_steering *mlx5e_fs_get_ethtool(struct mlx5e_flow_steering *fs); +#endif +struct mlx5_ttc_table *mlx5e_fs_get_ttc(struct mlx5e_flow_steering *fs, bool inner); +void mlx5e_fs_set_ttc(struct mlx5e_flow_steering *fs, struct mlx5_ttc_table *ttc, bool inner); +#ifdef CONFIG_MLX5_EN_ARFS +struct mlx5e_arfs_tables *mlx5e_fs_get_arfs(struct mlx5e_flow_steering *fs); +void mlx5e_fs_set_arfs(struct mlx5e_flow_steering *fs, struct mlx5e_arfs_tables *arfs); +#endif +struct mlx5e_ptp_fs *mlx5e_fs_get_ptp(struct mlx5e_flow_steering *fs); +void mlx5e_fs_set_ptp(struct mlx5e_flow_steering *fs, struct mlx5e_ptp_fs *ptp_fs); +struct mlx5e_fs_any *mlx5e_fs_get_any(struct mlx5e_flow_steering *fs); +void mlx5e_fs_set_any(struct mlx5e_flow_steering *fs, struct mlx5e_fs_any *any); +struct mlx5e_fs_udp *mlx5e_fs_get_udp(struct mlx5e_flow_steering *fs); +void mlx5e_fs_set_udp(struct mlx5e_flow_steering *fs, struct mlx5e_fs_udp *udp); +#ifdef CONFIG_MLX5_EN_TLS +struct mlx5e_accel_fs_tcp *mlx5e_fs_get_accel_tcp(struct mlx5e_flow_steering *fs); +void mlx5e_fs_set_accel_tcp(struct mlx5e_flow_steering *fs, struct mlx5e_accel_fs_tcp *accel_tcp); +#endif +void mlx5e_fs_set_state_destroy(struct mlx5e_flow_steering *fs, bool state_destroy); +void mlx5e_fs_set_vlan_strip_disable(struct mlx5e_flow_steering *fs, bool vlan_strip_disable); + +struct mlx5_core_dev *mlx5e_fs_get_mdev(struct mlx5e_flow_steering *fs); +int mlx5e_add_vlan_trap(struct mlx5e_flow_steering *fs, int trap_id, int tir_num); +void mlx5e_remove_vlan_trap(struct mlx5e_flow_steering *fs); +int mlx5e_add_mac_trap(struct mlx5e_flow_steering *fs, int trap_id, int tir_num); +void mlx5e_remove_mac_trap(struct mlx5e_flow_steering *fs); void mlx5e_fs_set_rx_mode_work(struct mlx5e_flow_steering *fs, struct net_device *netdev); int mlx5e_fs_vlan_rx_add_vid(struct mlx5e_flow_steering *fs, struct net_device *netdev, @@ -198,5 +187,18 @@ int mlx5e_fs_vlan_rx_kill_vid(struct mlx5e_flow_steering *fs, struct net_device *netdev, __be16 proto, u16 vid); void mlx5e_fs_init_l2_addr(struct mlx5e_flow_steering *fs, struct net_device *netdev); + +#define fs_err(fs, fmt, ...) \ + mlx5_core_err(mlx5e_fs_get_mdev(fs), fmt, ##__VA_ARGS__) + +#define fs_dbg(fs, fmt, ...) \ + mlx5_core_dbg(mlx5e_fs_get_mdev(fs), fmt, ##__VA_ARGS__) + +#define fs_warn(fs, fmt, ...) \ + mlx5_core_warn(mlx5e_fs_get_mdev(fs), fmt, ##__VA_ARGS__) + +#define fs_warn_once(fs, fmt, ...) \ + mlx5_core_warn_once(mlx5e_fs_get_mdev(fs), fmt, ##__VA_ARGS__) + #endif /* __MLX5E_FLOW_STEER_H__ */ diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/fs_ethtool.h b/drivers/net/ethernet/mellanox/mlx5/core/en/fs_ethtool.h new file mode 100644 index 000000000000..9e276fd3c0cf --- /dev/null +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/fs_ethtool.h @@ -0,0 +1,29 @@ +/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB */ +/* Copyright (c) 2022, NVIDIA CORPORATION & AFFILIATES. */ + +#ifndef __MLX5E_FS_ETHTOOL_H__ +#define __MLX5E_FS_ETHTOOL_H__ + +struct mlx5e_priv; +struct mlx5e_ethtool_steering; +#ifdef CONFIG_MLX5_EN_RXNFC +int mlx5e_ethtool_alloc(struct mlx5e_ethtool_steering **ethtool); +void mlx5e_ethtool_free(struct mlx5e_ethtool_steering *ethtool); +void mlx5e_ethtool_init_steering(struct mlx5e_flow_steering *fs); +void mlx5e_ethtool_cleanup_steering(struct mlx5e_flow_steering *fs); +int mlx5e_ethtool_set_rxnfc(struct mlx5e_priv *priv, struct ethtool_rxnfc *cmd); +int mlx5e_ethtool_get_rxnfc(struct mlx5e_priv *priv, + struct ethtool_rxnfc *info, u32 *rule_locs); +#else +static inline int mlx5e_ethtool_alloc(struct mlx5e_ethtool_steering **ethtool) +{ return 0; } +static inline void mlx5e_ethtool_free(struct mlx5e_ethtool_steering *ethtool) { } +static inline void mlx5e_ethtool_init_steering(struct mlx5e_flow_steering *fs) { } +static inline void mlx5e_ethtool_cleanup_steering(struct mlx5e_flow_steering *fs) { } +static inline int mlx5e_ethtool_set_rxnfc(struct mlx5e_priv *priv, struct ethtool_rxnfc *cmd) +{ return -EOPNOTSUPP; } +static inline int mlx5e_ethtool_get_rxnfc(struct mlx5e_priv *priv, + struct ethtool_rxnfc *info, u32 *rule_locs) +{ return -EOPNOTSUPP; } +#endif +#endif diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/fs_tt_redirect.c b/drivers/net/ethernet/mellanox/mlx5/core/en/fs_tt_redirect.c index e153d6119e02..03cb79adf912 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/fs_tt_redirect.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/fs_tt_redirect.c @@ -1,9 +1,9 @@ // SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB /* Copyright (c) 2021, Mellanox Technologies inc. All rights reserved. */ -#include <linux/netdevice.h> #include "en/fs_tt_redirect.h" #include "fs_core.h" +#include "mlx5_core.h" enum fs_udp_type { FS_IPV4_UDP, @@ -74,17 +74,17 @@ static void fs_udp_set_dport_flow(struct mlx5_flow_spec *spec, enum fs_udp_type } struct mlx5_flow_handle * -mlx5e_fs_tt_redirect_udp_add_rule(struct mlx5e_priv *priv, +mlx5e_fs_tt_redirect_udp_add_rule(struct mlx5e_flow_steering *fs, enum mlx5_traffic_types ttc_type, u32 tir_num, u16 d_port) { + struct mlx5e_fs_udp *fs_udp = mlx5e_fs_get_udp(fs); enum fs_udp_type type = tt2fs_udp(ttc_type); struct mlx5_flow_destination dest = {}; struct mlx5_flow_table *ft = NULL; MLX5_DECLARE_FLOW_ACT(flow_act); struct mlx5_flow_handle *rule; struct mlx5_flow_spec *spec; - struct mlx5e_fs_udp *fs_udp; int err; if (type == FS_UDP_NUM_TYPES) @@ -94,7 +94,6 @@ mlx5e_fs_tt_redirect_udp_add_rule(struct mlx5e_priv *priv, if (!spec) return ERR_PTR(-ENOMEM); - fs_udp = priv->fs->udp; ft = fs_udp->tables[type].t; fs_udp_set_dport_flow(spec, type, d_port); @@ -106,31 +105,30 @@ mlx5e_fs_tt_redirect_udp_add_rule(struct mlx5e_priv *priv, if (IS_ERR(rule)) { err = PTR_ERR(rule); - netdev_err(priv->netdev, "%s: add %s rule failed, err %d\n", - __func__, fs_udp_type2str(type), err); + fs_err(fs, "%s: add %s rule failed, err %d\n", + __func__, fs_udp_type2str(type), err); } return rule; } -static int fs_udp_add_default_rule(struct mlx5e_priv *priv, enum fs_udp_type type) +static int fs_udp_add_default_rule(struct mlx5e_flow_steering *fs, enum fs_udp_type type) { + struct mlx5_ttc_table *ttc = mlx5e_fs_get_ttc(fs, false); + struct mlx5e_fs_udp *fs_udp = mlx5e_fs_get_udp(fs); struct mlx5e_flow_table *fs_udp_t; struct mlx5_flow_destination dest; MLX5_DECLARE_FLOW_ACT(flow_act); struct mlx5_flow_handle *rule; - struct mlx5e_fs_udp *fs_udp; int err; - fs_udp = priv->fs->udp; fs_udp_t = &fs_udp->tables[type]; - dest = mlx5_ttc_get_default_dest(priv->fs->ttc, fs_udp2tt(type)); + dest = mlx5_ttc_get_default_dest(ttc, fs_udp2tt(type)); rule = mlx5_add_flow_rules(fs_udp_t->t, NULL, &flow_act, &dest, 1); if (IS_ERR(rule)) { err = PTR_ERR(rule); - netdev_err(priv->netdev, - "%s: add default rule failed, fs type=%d, err %d\n", - __func__, type, err); + fs_err(fs, "%s: add default rule failed, fs type=%d, err %d\n", + __func__, type, err); return err; } @@ -206,33 +204,36 @@ out: return err; } -static int fs_udp_create_table(struct mlx5e_priv *priv, enum fs_udp_type type) +static int fs_udp_create_table(struct mlx5e_flow_steering *fs, enum fs_udp_type type) { - struct mlx5e_flow_table *ft = &priv->fs->udp->tables[type]; + struct mlx5_flow_namespace *ns = mlx5e_fs_get_ns(fs, false); + struct mlx5e_fs_udp *fs_udp = mlx5e_fs_get_udp(fs); struct mlx5_flow_table_attr ft_attr = {}; + struct mlx5e_flow_table *ft; int err; + ft = &fs_udp->tables[type]; ft->num_groups = 0; ft_attr.max_fte = MLX5E_FS_UDP_TABLE_SIZE; ft_attr.level = MLX5E_FS_TT_UDP_FT_LEVEL; ft_attr.prio = MLX5E_NIC_PRIO; - ft->t = mlx5_create_flow_table(priv->fs->ns, &ft_attr); + ft->t = mlx5_create_flow_table(ns, &ft_attr); if (IS_ERR(ft->t)) { err = PTR_ERR(ft->t); ft->t = NULL; return err; } - netdev_dbg(priv->netdev, "Created fs %s table id %u level %u\n", - fs_udp_type2str(type), ft->t->id, ft->t->level); + mlx5_core_dbg(mlx5e_fs_get_mdev(fs), "Created fs %s table id %u level %u\n", + fs_udp_type2str(type), ft->t->id, ft->t->level); err = fs_udp_create_groups(ft, type); if (err) goto err; - err = fs_udp_add_default_rule(priv, type); + err = fs_udp_add_default_rule(fs, type); if (err) goto err; @@ -253,17 +254,17 @@ static void fs_udp_destroy_table(struct mlx5e_fs_udp *fs_udp, int i) fs_udp->tables[i].t = NULL; } -static int fs_udp_disable(struct mlx5e_priv *priv) +static int fs_udp_disable(struct mlx5e_flow_steering *fs) { + struct mlx5_ttc_table *ttc = mlx5e_fs_get_ttc(fs, false); int err, i; for (i = 0; i < FS_UDP_NUM_TYPES; i++) { /* Modify ttc rules destination to point back to the indir TIRs */ - err = mlx5_ttc_fwd_default_dest(priv->fs->ttc, fs_udp2tt(i)); + err = mlx5_ttc_fwd_default_dest(ttc, fs_udp2tt(i)); if (err) { - netdev_err(priv->netdev, - "%s: modify ttc[%d] default destination failed, err(%d)\n", - __func__, fs_udp2tt(i), err); + fs_err(fs, "%s: modify ttc[%d] default destination failed, err(%d)\n", + __func__, fs_udp2tt(i), err); return err; } } @@ -271,30 +272,31 @@ static int fs_udp_disable(struct mlx5e_priv *priv) return 0; } -static int fs_udp_enable(struct mlx5e_priv *priv) +static int fs_udp_enable(struct mlx5e_flow_steering *fs) { + struct mlx5_ttc_table *ttc = mlx5e_fs_get_ttc(fs, false); + struct mlx5e_fs_udp *udp = mlx5e_fs_get_udp(fs); struct mlx5_flow_destination dest = {}; int err, i; dest.type = MLX5_FLOW_DESTINATION_TYPE_FLOW_TABLE; for (i = 0; i < FS_UDP_NUM_TYPES; i++) { - dest.ft = priv->fs->udp->tables[i].t; + dest.ft = udp->tables[i].t; /* Modify ttc rules destination to point on the accel_fs FTs */ - err = mlx5_ttc_fwd_dest(priv->fs->ttc, fs_udp2tt(i), &dest); + err = mlx5_ttc_fwd_dest(ttc, fs_udp2tt(i), &dest); if (err) { - netdev_err(priv->netdev, - "%s: modify ttc[%d] destination to accel failed, err(%d)\n", - __func__, fs_udp2tt(i), err); + fs_err(fs, "%s: modify ttc[%d] destination to accel failed, err(%d)\n", + __func__, fs_udp2tt(i), err); return err; } } return 0; } -void mlx5e_fs_tt_redirect_udp_destroy(struct mlx5e_priv *priv) +void mlx5e_fs_tt_redirect_udp_destroy(struct mlx5e_flow_steering *fs) { - struct mlx5e_fs_udp *fs_udp = priv->fs->udp; + struct mlx5e_fs_udp *fs_udp = mlx5e_fs_get_udp(fs); int i; if (!fs_udp) @@ -303,48 +305,50 @@ void mlx5e_fs_tt_redirect_udp_destroy(struct mlx5e_priv *priv) if (--fs_udp->ref_cnt) return; - fs_udp_disable(priv); + fs_udp_disable(fs); for (i = 0; i < FS_UDP_NUM_TYPES; i++) fs_udp_destroy_table(fs_udp, i); kfree(fs_udp); - priv->fs->udp = NULL; + mlx5e_fs_set_udp(fs, NULL); } -int mlx5e_fs_tt_redirect_udp_create(struct mlx5e_priv *priv) +int mlx5e_fs_tt_redirect_udp_create(struct mlx5e_flow_steering *fs) { + struct mlx5e_fs_udp *udp = mlx5e_fs_get_udp(fs); int i, err; - if (priv->fs->udp) { - priv->fs->udp->ref_cnt++; + if (udp) { + udp->ref_cnt++; return 0; } - priv->fs->udp = kzalloc(sizeof(*priv->fs->udp), GFP_KERNEL); - if (!priv->fs->udp) + udp = kzalloc(sizeof(*udp), GFP_KERNEL); + if (!udp) return -ENOMEM; + mlx5e_fs_set_udp(fs, udp); for (i = 0; i < FS_UDP_NUM_TYPES; i++) { - err = fs_udp_create_table(priv, i); + err = fs_udp_create_table(fs, i); if (err) goto err_destroy_tables; } - err = fs_udp_enable(priv); + err = fs_udp_enable(fs); if (err) goto err_destroy_tables; - priv->fs->udp->ref_cnt = 1; + udp->ref_cnt = 1; return 0; err_destroy_tables: while (--i >= 0) - fs_udp_destroy_table(priv->fs->udp, i); + fs_udp_destroy_table(udp, i); - kfree(priv->fs->udp); - priv->fs->udp = NULL; + kfree(udp); + mlx5e_fs_set_udp(fs, NULL); return err; } @@ -356,22 +360,21 @@ static void fs_any_set_ethertype_flow(struct mlx5_flow_spec *spec, u16 ether_typ } struct mlx5_flow_handle * -mlx5e_fs_tt_redirect_any_add_rule(struct mlx5e_priv *priv, +mlx5e_fs_tt_redirect_any_add_rule(struct mlx5e_flow_steering *fs, u32 tir_num, u16 ether_type) { + struct mlx5e_fs_any *fs_any = mlx5e_fs_get_any(fs); struct mlx5_flow_destination dest = {}; struct mlx5_flow_table *ft = NULL; MLX5_DECLARE_FLOW_ACT(flow_act); struct mlx5_flow_handle *rule; struct mlx5_flow_spec *spec; - struct mlx5e_fs_any *fs_any; int err; spec = kvzalloc(sizeof(*spec), GFP_KERNEL); if (!spec) return ERR_PTR(-ENOMEM); - fs_any = priv->fs->any; ft = fs_any->table.t; fs_any_set_ethertype_flow(spec, ether_type); @@ -383,31 +386,29 @@ mlx5e_fs_tt_redirect_any_add_rule(struct mlx5e_priv *priv, if (IS_ERR(rule)) { err = PTR_ERR(rule); - netdev_err(priv->netdev, "%s: add ANY rule failed, err %d\n", - __func__, err); + fs_err(fs, "%s: add ANY rule failed, err %d\n", + __func__, err); } return rule; } -static int fs_any_add_default_rule(struct mlx5e_priv *priv) +static int fs_any_add_default_rule(struct mlx5e_flow_steering *fs) { + struct mlx5_ttc_table *ttc = mlx5e_fs_get_ttc(fs, false); + struct mlx5e_fs_any *fs_any = mlx5e_fs_get_any(fs); struct mlx5e_flow_table *fs_any_t; struct mlx5_flow_destination dest; MLX5_DECLARE_FLOW_ACT(flow_act); struct mlx5_flow_handle *rule; - struct mlx5e_fs_any *fs_any; int err; - fs_any = priv->fs->any; fs_any_t = &fs_any->table; - - dest = mlx5_ttc_get_default_dest(priv->fs->ttc, MLX5_TT_ANY); + dest = mlx5_ttc_get_default_dest(ttc, MLX5_TT_ANY); rule = mlx5_add_flow_rules(fs_any_t->t, NULL, &flow_act, &dest, 1); if (IS_ERR(rule)) { err = PTR_ERR(rule); - netdev_err(priv->netdev, - "%s: add default rule failed, fs type=ANY, err %d\n", - __func__, err); + fs_err(fs, "%s: add default rule failed, fs type=ANY, err %d\n", + __func__, err); return err; } @@ -472,9 +473,11 @@ err: return err; } -static int fs_any_create_table(struct mlx5e_priv *priv) +static int fs_any_create_table(struct mlx5e_flow_steering *fs) { - struct mlx5e_flow_table *ft = &priv->fs->any->table; + struct mlx5_flow_namespace *ns = mlx5e_fs_get_ns(fs, false); + struct mlx5e_fs_any *fs_any = mlx5e_fs_get_any(fs); + struct mlx5e_flow_table *ft = &fs_any->table; struct mlx5_flow_table_attr ft_attr = {}; int err; @@ -484,21 +487,21 @@ static int fs_any_create_table(struct mlx5e_priv *priv) ft_attr.level = MLX5E_FS_TT_ANY_FT_LEVEL; ft_attr.prio = MLX5E_NIC_PRIO; - ft->t = mlx5_create_flow_table(priv->fs->ns, &ft_attr); + ft->t = mlx5_create_flow_table(ns, &ft_attr); if (IS_ERR(ft->t)) { err = PTR_ERR(ft->t); ft->t = NULL; return err; } - netdev_dbg(priv->netdev, "Created fs ANY table id %u level %u\n", - ft->t->id, ft->t->level); + mlx5_core_dbg(mlx5e_fs_get_mdev(fs), "Created fs ANY table id %u level %u\n", + ft->t->id, ft->t->level); err = fs_any_create_groups(ft); if (err) goto err; - err = fs_any_add_default_rule(priv); + err = fs_any_add_default_rule(fs); if (err) goto err; @@ -509,35 +512,38 @@ err: return err; } -static int fs_any_disable(struct mlx5e_priv *priv) +static int fs_any_disable(struct mlx5e_flow_steering *fs) { + struct mlx5_ttc_table *ttc = mlx5e_fs_get_ttc(fs, false); int err; /* Modify ttc rules destination to point back to the indir TIRs */ - err = mlx5_ttc_fwd_default_dest(priv->fs->ttc, MLX5_TT_ANY); + err = mlx5_ttc_fwd_default_dest(ttc, MLX5_TT_ANY); if (err) { - netdev_err(priv->netdev, - "%s: modify ttc[%d] default destination failed, err(%d)\n", - __func__, MLX5_TT_ANY, err); + fs_err(fs, + "%s: modify ttc[%d] default destination failed, err(%d)\n", + __func__, MLX5_TT_ANY, err); return err; } return 0; } -static int fs_any_enable(struct mlx5e_priv *priv) +static int fs_any_enable(struct mlx5e_flow_steering *fs) { + struct mlx5_ttc_table *ttc = mlx5e_fs_get_ttc(fs, false); + struct mlx5e_fs_any *any = mlx5e_fs_get_any(fs); struct mlx5_flow_destination dest = {}; int err; dest.type = MLX5_FLOW_DESTINATION_TYPE_FLOW_TABLE; - dest.ft = priv->fs->any->table.t; + dest.ft = any->table.t; /* Modify ttc rules destination to point on the accel_fs FTs */ - err = mlx5_ttc_fwd_dest(priv->fs->ttc, MLX5_TT_ANY, &dest); + err = mlx5_ttc_fwd_dest(ttc, MLX5_TT_ANY, &dest); if (err) { - netdev_err(priv->netdev, - "%s: modify ttc[%d] destination to accel failed, err(%d)\n", - __func__, MLX5_TT_ANY, err); + fs_err(fs, + "%s: modify ttc[%d] destination to accel failed, err(%d)\n", + __func__, MLX5_TT_ANY, err); return err; } return 0; @@ -553,9 +559,9 @@ static void fs_any_destroy_table(struct mlx5e_fs_any *fs_any) fs_any->table.t = NULL; } -void mlx5e_fs_tt_redirect_any_destroy(struct mlx5e_priv *priv) +void mlx5e_fs_tt_redirect_any_destroy(struct mlx5e_flow_steering *fs) { - struct mlx5e_fs_any *fs_any = priv->fs->any; + struct mlx5e_fs_any *fs_any = mlx5e_fs_get_any(fs); if (!fs_any) return; @@ -563,43 +569,45 @@ void mlx5e_fs_tt_redirect_any_destroy(struct mlx5e_priv *priv) if (--fs_any->ref_cnt) return; - fs_any_disable(priv); + fs_any_disable(fs); fs_any_destroy_table(fs_any); kfree(fs_any); - priv->fs->any = NULL; + mlx5e_fs_set_any(fs, NULL); } -int mlx5e_fs_tt_redirect_any_create(struct mlx5e_priv *priv) +int mlx5e_fs_tt_redirect_any_create(struct mlx5e_flow_steering *fs) { + struct mlx5e_fs_any *fs_any = mlx5e_fs_get_any(fs); int err; - if (priv->fs->any) { - priv->fs->any->ref_cnt++; + if (fs_any) { + fs_any->ref_cnt++; return 0; } - priv->fs->any = kzalloc(sizeof(*priv->fs->any), GFP_KERNEL); - if (!priv->fs->any) + fs_any = kzalloc(sizeof(*fs_any), GFP_KERNEL); + if (!fs_any) return -ENOMEM; + mlx5e_fs_set_any(fs, fs_any); - err = fs_any_create_table(priv); + err = fs_any_create_table(fs); if (err) return err; - err = fs_any_enable(priv); + err = fs_any_enable(fs); if (err) goto err_destroy_table; - priv->fs->any->ref_cnt = 1; + fs_any->ref_cnt = 1; return 0; err_destroy_table: - fs_any_destroy_table(priv->fs->any); + fs_any_destroy_table(fs_any); - kfree(priv->fs->any); - priv->fs->any = NULL; + kfree(fs_any); + mlx5e_fs_set_any(fs, NULL); return err; } diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/fs_tt_redirect.h b/drivers/net/ethernet/mellanox/mlx5/core/en/fs_tt_redirect.h index 7a70c4f38fda..5780fd7ad507 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/fs_tt_redirect.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/fs_tt_redirect.h @@ -4,23 +4,22 @@ #ifndef __MLX5E_FS_TT_REDIRECT_H__ #define __MLX5E_FS_TT_REDIRECT_H__ -#include "en.h" #include "en/fs.h" void mlx5e_fs_tt_redirect_del_rule(struct mlx5_flow_handle *rule); /* UDP traffic type redirect */ struct mlx5_flow_handle * -mlx5e_fs_tt_redirect_udp_add_rule(struct mlx5e_priv *priv, +mlx5e_fs_tt_redirect_udp_add_rule(struct mlx5e_flow_steering *fs, enum mlx5_traffic_types ttc_type, u32 tir_num, u16 d_port); -void mlx5e_fs_tt_redirect_udp_destroy(struct mlx5e_priv *priv); -int mlx5e_fs_tt_redirect_udp_create(struct mlx5e_priv *priv); +void mlx5e_fs_tt_redirect_udp_destroy(struct mlx5e_flow_steering *fs); +int mlx5e_fs_tt_redirect_udp_create(struct mlx5e_flow_steering *fs); /* ANY traffic type redirect*/ struct mlx5_flow_handle * -mlx5e_fs_tt_redirect_any_add_rule(struct mlx5e_priv *priv, +mlx5e_fs_tt_redirect_any_add_rule(struct mlx5e_flow_steering *fs, u32 tir_num, u16 ether_type); -void mlx5e_fs_tt_redirect_any_destroy(struct mlx5e_priv *priv); -int mlx5e_fs_tt_redirect_any_create(struct mlx5e_priv *priv); +void mlx5e_fs_tt_redirect_any_destroy(struct mlx5e_flow_steering *fs); +int mlx5e_fs_tt_redirect_any_create(struct mlx5e_flow_steering *fs); #endif diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/params.c b/drivers/net/ethernet/mellanox/mlx5/core/en/params.c index e025040350ba..29dd3a04c154 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/params.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/params.c @@ -6,11 +6,212 @@ #include "en/port.h" #include "en_accel/en_accel.h" #include "en_accel/ipsec.h" +#include <net/xdp_sock_drv.h> -static bool mlx5e_rx_is_xdp(struct mlx5e_params *params, - struct mlx5e_xsk_param *xsk) +static u8 mlx5e_mpwrq_min_page_shift(struct mlx5_core_dev *mdev) +{ + u8 min_page_shift = MLX5_CAP_GEN_2(mdev, log_min_mkey_entity_size); + + return min_page_shift ? : 12; +} + +u8 mlx5e_mpwrq_page_shift(struct mlx5_core_dev *mdev, struct mlx5e_xsk_param *xsk) +{ + u8 req_page_shift = xsk ? order_base_2(xsk->chunk_size) : PAGE_SHIFT; + u8 min_page_shift = mlx5e_mpwrq_min_page_shift(mdev); + + /* Regular RQ uses order-0 pages, the NIC must be able to map them. */ + if (WARN_ON_ONCE(!xsk && req_page_shift < min_page_shift)) + min_page_shift = req_page_shift; + + return max(req_page_shift, min_page_shift); +} + +enum mlx5e_mpwrq_umr_mode +mlx5e_mpwrq_umr_mode(struct mlx5_core_dev *mdev, struct mlx5e_xsk_param *xsk) +{ + /* Different memory management schemes use different mechanisms to map + * user-mode memory. The stricter guarantees we have, the faster + * mechanisms we use: + * 1. MTT - direct mapping in page granularity. + * 2. KSM - indirect mapping to another MKey to arbitrary addresses, but + * all mappings have the same size. + * 3. KLM - indirect mapping to another MKey to arbitrary addresses, and + * mappings can have different sizes. + */ + u8 page_shift = mlx5e_mpwrq_page_shift(mdev, xsk); + bool unaligned = xsk ? xsk->unaligned : false; + bool oversized = false; + + if (xsk) { + oversized = xsk->chunk_size < (1 << page_shift); + WARN_ON_ONCE(xsk->chunk_size > (1 << page_shift)); + } + + /* XSK frame size doesn't match the UMR page size, either because the + * frame size is not a power of two, or it's smaller than the minimal + * page size supported by the firmware. + * It's possible to receive packets bigger than MTU in certain setups. + * To avoid writing over the XSK frame boundary, the top region of each + * stride is mapped to a garbage page, resulting in two mappings of + * different sizes per frame. + */ + if (oversized) { + /* An optimization for frame sizes equal to 3 * power_of_two. + * 3 KSMs point to the frame, and one KSM points to the garbage + * page, which works faster than KLM. + */ + if (xsk->chunk_size % 3 == 0 && is_power_of_2(xsk->chunk_size / 3)) + return MLX5E_MPWRQ_UMR_MODE_TRIPLE; + + return MLX5E_MPWRQ_UMR_MODE_OVERSIZED; + } + + /* XSK frames can start at arbitrary unaligned locations, but they all + * have the same size which is a power of two. It allows to optimize to + * one KSM per frame. + */ + if (unaligned) + return MLX5E_MPWRQ_UMR_MODE_UNALIGNED; + + /* XSK: frames are naturally aligned, MTT can be used. + * Non-XSK: Allocations happen in units of CPU pages, therefore, the + * mappings are naturally aligned. + */ + return MLX5E_MPWRQ_UMR_MODE_ALIGNED; +} + +u8 mlx5e_mpwrq_umr_entry_size(enum mlx5e_mpwrq_umr_mode mode) +{ + switch (mode) { + case MLX5E_MPWRQ_UMR_MODE_ALIGNED: + return sizeof(struct mlx5_mtt); + case MLX5E_MPWRQ_UMR_MODE_UNALIGNED: + return sizeof(struct mlx5_ksm); + case MLX5E_MPWRQ_UMR_MODE_OVERSIZED: + return sizeof(struct mlx5_klm) * 2; + case MLX5E_MPWRQ_UMR_MODE_TRIPLE: + return sizeof(struct mlx5_ksm) * 4; + } + WARN_ONCE(1, "MPWRQ UMR mode %d is not known\n", mode); + return 0; +} + +u8 mlx5e_mpwrq_log_wqe_sz(struct mlx5_core_dev *mdev, u8 page_shift, + enum mlx5e_mpwrq_umr_mode umr_mode) +{ + u8 umr_entry_size = mlx5e_mpwrq_umr_entry_size(umr_mode); + u8 max_pages_per_wqe, max_log_mpwqe_size; + u16 max_wqe_size; + + /* Keep in sync with MLX5_MPWRQ_MAX_PAGES_PER_WQE. */ + max_wqe_size = mlx5e_get_max_sq_aligned_wqebbs(mdev) * MLX5_SEND_WQE_BB; + max_pages_per_wqe = ALIGN_DOWN(max_wqe_size - sizeof(struct mlx5e_umr_wqe), + MLX5_UMR_MTT_ALIGNMENT) / umr_entry_size; + max_log_mpwqe_size = ilog2(max_pages_per_wqe) + page_shift; + + WARN_ON_ONCE(max_log_mpwqe_size < MLX5E_ORDER2_MAX_PACKET_MTU); + + return min_t(u8, max_log_mpwqe_size, MLX5_MPWRQ_MAX_LOG_WQE_SZ); +} + +u8 mlx5e_mpwrq_pages_per_wqe(struct mlx5_core_dev *mdev, u8 page_shift, + enum mlx5e_mpwrq_umr_mode umr_mode) { - return params->xdp_prog || xsk; + u8 log_wqe_sz = mlx5e_mpwrq_log_wqe_sz(mdev, page_shift, umr_mode); + u8 pages_per_wqe; + + pages_per_wqe = log_wqe_sz > page_shift ? (1 << (log_wqe_sz - page_shift)) : 1; + + /* Two MTTs are needed to form an octword. The number of MTTs is encoded + * in octwords in a UMR WQE, so we need at least two to avoid mapping + * garbage addresses. + */ + if (WARN_ON_ONCE(pages_per_wqe < 2 && umr_mode == MLX5E_MPWRQ_UMR_MODE_ALIGNED)) + pages_per_wqe = 2; + + /* Sanity check for further calculations to succeed. */ + BUILD_BUG_ON(MLX5_MPWRQ_MAX_PAGES_PER_WQE > 64); + if (WARN_ON_ONCE(pages_per_wqe > MLX5_MPWRQ_MAX_PAGES_PER_WQE)) + return MLX5_MPWRQ_MAX_PAGES_PER_WQE; + + return pages_per_wqe; +} + +u16 mlx5e_mpwrq_umr_wqe_sz(struct mlx5_core_dev *mdev, u8 page_shift, + enum mlx5e_mpwrq_umr_mode umr_mode) +{ + u8 pages_per_wqe = mlx5e_mpwrq_pages_per_wqe(mdev, page_shift, umr_mode); + u8 umr_entry_size = mlx5e_mpwrq_umr_entry_size(umr_mode); + u16 umr_wqe_sz; + + umr_wqe_sz = sizeof(struct mlx5e_umr_wqe) + + ALIGN(pages_per_wqe * umr_entry_size, MLX5_UMR_MTT_ALIGNMENT); + + WARN_ON_ONCE(DIV_ROUND_UP(umr_wqe_sz, MLX5_SEND_WQE_DS) > MLX5_WQE_CTRL_DS_MASK); + + return umr_wqe_sz; +} + +u8 mlx5e_mpwrq_umr_wqebbs(struct mlx5_core_dev *mdev, u8 page_shift, + enum mlx5e_mpwrq_umr_mode umr_mode) +{ + return DIV_ROUND_UP(mlx5e_mpwrq_umr_wqe_sz(mdev, page_shift, umr_mode), + MLX5_SEND_WQE_BB); +} + +u8 mlx5e_mpwrq_mtts_per_wqe(struct mlx5_core_dev *mdev, u8 page_shift, + enum mlx5e_mpwrq_umr_mode umr_mode) +{ + u8 pages_per_wqe = mlx5e_mpwrq_pages_per_wqe(mdev, page_shift, umr_mode); + + /* Add another page as a buffer between WQEs. This page will absorb + * write overflow by the hardware, when receiving packets larger than + * MTU. These oversize packets are dropped by the driver at a later + * stage. + */ + return ALIGN(pages_per_wqe + 1, + MLX5_SEND_WQE_BB / mlx5e_mpwrq_umr_entry_size(umr_mode)); +} + +u32 mlx5e_mpwrq_max_num_entries(struct mlx5_core_dev *mdev, + enum mlx5e_mpwrq_umr_mode umr_mode) +{ + /* Same limits apply to KSMs and KLMs. */ + u32 klm_limit = min(MLX5E_MAX_RQ_NUM_KSMS, + 1 << MLX5_CAP_GEN(mdev, log_max_klm_list_size)); + + switch (umr_mode) { + case MLX5E_MPWRQ_UMR_MODE_ALIGNED: + return MLX5E_MAX_RQ_NUM_MTTS; + case MLX5E_MPWRQ_UMR_MODE_UNALIGNED: + return klm_limit; + case MLX5E_MPWRQ_UMR_MODE_OVERSIZED: + /* Each entry is two KLMs. */ + return klm_limit / 2; + case MLX5E_MPWRQ_UMR_MODE_TRIPLE: + /* Each entry is four KSMs. */ + return klm_limit / 4; + } + WARN_ONCE(1, "MPWRQ UMR mode %d is not known\n", umr_mode); + return 0; +} + +static u8 mlx5e_mpwrq_max_log_rq_size(struct mlx5_core_dev *mdev, u8 page_shift, + enum mlx5e_mpwrq_umr_mode umr_mode) +{ + u8 mtts_per_wqe = mlx5e_mpwrq_mtts_per_wqe(mdev, page_shift, umr_mode); + u32 max_entries = mlx5e_mpwrq_max_num_entries(mdev, umr_mode); + + return ilog2(max_entries / mtts_per_wqe); +} + +u8 mlx5e_mpwrq_max_log_rq_pkts(struct mlx5_core_dev *mdev, u8 page_shift, + enum mlx5e_mpwrq_umr_mode umr_mode) +{ + return mlx5e_mpwrq_max_log_rq_size(mdev, page_shift, umr_mode) + + mlx5e_mpwrq_log_wqe_sz(mdev, page_shift, umr_mode) - + MLX5E_ORDER2_MAX_PACKET_MTU; } u16 mlx5e_get_linear_rq_headroom(struct mlx5e_params *params, @@ -22,7 +223,7 @@ u16 mlx5e_get_linear_rq_headroom(struct mlx5e_params *params, return xsk->headroom; headroom = NET_IP_ALIGN; - if (mlx5e_rx_is_xdp(params, xsk)) + if (params->xdp_prog) headroom += XDP_PACKET_HEADROOM; else headroom += MLX5_RX_HEADROOM; @@ -30,70 +231,80 @@ u16 mlx5e_get_linear_rq_headroom(struct mlx5e_params *params, return headroom; } -u32 mlx5e_rx_get_min_frag_sz(struct mlx5e_params *params, - struct mlx5e_xsk_param *xsk) +static u32 mlx5e_rx_get_linear_sz_xsk(struct mlx5e_params *params, + struct mlx5e_xsk_param *xsk) { u32 hw_mtu = MLX5E_SW2HW_MTU(params, params->sw_mtu); - u16 linear_rq_headroom = mlx5e_get_linear_rq_headroom(params, xsk); - return linear_rq_headroom + hw_mtu; + return xsk->headroom + hw_mtu; } -static u32 mlx5e_rx_get_linear_frag_sz(struct mlx5e_params *params, - struct mlx5e_xsk_param *xsk) +static u32 mlx5e_rx_get_linear_sz_skb(struct mlx5e_params *params, bool xsk) { - u32 frag_sz = mlx5e_rx_get_min_frag_sz(params, xsk); - - /* AF_XDP doesn't build SKBs in place. */ - if (!xsk) - frag_sz = MLX5_SKB_FRAG_SZ(frag_sz); + /* SKBs built on XDP_PASS on XSK RQs don't have headroom. */ + u16 headroom = xsk ? 0 : mlx5e_get_linear_rq_headroom(params, NULL); + u32 hw_mtu = MLX5E_SW2HW_MTU(params, params->sw_mtu); - /* XDP in mlx5e doesn't support multiple packets per page. AF_XDP is a - * special case. It can run with frames smaller than a page, as it - * doesn't allocate pages dynamically. However, here we pretend that - * fragments are page-sized: it allows to treat XSK frames like pages - * by redirecting alloc and free operations to XSK rings and by using - * the fact there are no multiple packets per "page" (which is a frame). - * The latter is important, because frames may come in a random order, - * and we will have trouble assemblying a real page of multiple frames. - */ - if (mlx5e_rx_is_xdp(params, xsk)) - frag_sz = max_t(u32, frag_sz, PAGE_SIZE); + return MLX5_SKB_FRAG_SZ(headroom + hw_mtu); +} - /* Even if we can go with a smaller fragment size, we must not put - * multiple packets into a single frame. +static u32 mlx5e_rx_get_linear_stride_sz(struct mlx5_core_dev *mdev, + struct mlx5e_params *params, + struct mlx5e_xsk_param *xsk, + bool mpwqe) +{ + /* XSK frames are mapped as individual pages, because frames may come in + * an arbitrary order from random locations in the UMEM. */ if (xsk) - frag_sz = max_t(u32, frag_sz, xsk->chunk_size); + return mpwqe ? 1 << mlx5e_mpwrq_page_shift(mdev, xsk) : PAGE_SIZE; + + /* XDP in mlx5e doesn't support multiple packets per page. */ + if (params->xdp_prog) + return PAGE_SIZE; - return frag_sz; + return roundup_pow_of_two(mlx5e_rx_get_linear_sz_skb(params, false)); } -u8 mlx5e_mpwqe_log_pkts_per_wqe(struct mlx5e_params *params, - struct mlx5e_xsk_param *xsk) +static u8 mlx5e_mpwqe_log_pkts_per_wqe(struct mlx5_core_dev *mdev, + struct mlx5e_params *params, + struct mlx5e_xsk_param *xsk) { - u32 linear_frag_sz = mlx5e_rx_get_linear_frag_sz(params, xsk); + u32 linear_stride_sz = mlx5e_rx_get_linear_stride_sz(mdev, params, xsk, true); + enum mlx5e_mpwrq_umr_mode umr_mode = mlx5e_mpwrq_umr_mode(mdev, xsk); + u8 page_shift = mlx5e_mpwrq_page_shift(mdev, xsk); - return MLX5_MPWRQ_LOG_WQE_SZ - order_base_2(linear_frag_sz); + return mlx5e_mpwrq_log_wqe_sz(mdev, page_shift, umr_mode) - + order_base_2(linear_stride_sz); } -bool mlx5e_rx_is_linear_skb(struct mlx5e_params *params, +bool mlx5e_rx_is_linear_skb(struct mlx5_core_dev *mdev, + struct mlx5e_params *params, struct mlx5e_xsk_param *xsk) { - /* AF_XDP allocates SKBs on XDP_PASS - ensure they don't occupy more - * than one page. For this, check both with and without xsk. + if (params->packet_merge.type != MLX5E_PACKET_MERGE_NONE) + return false; + + /* Both XSK and non-XSK cases allocate an SKB on XDP_PASS. Packet data + * must fit into a CPU page. */ - u32 linear_frag_sz = max(mlx5e_rx_get_linear_frag_sz(params, xsk), - mlx5e_rx_get_linear_frag_sz(params, NULL)); + if (mlx5e_rx_get_linear_sz_skb(params, xsk) > PAGE_SIZE) + return false; + + /* XSK frames must be big enough to hold the packet data. */ + if (xsk && mlx5e_rx_get_linear_sz_xsk(params, xsk) > xsk->chunk_size) + return false; - return params->packet_merge.type == MLX5E_PACKET_MERGE_NONE && - linear_frag_sz <= PAGE_SIZE; + return true; } -bool mlx5e_verify_rx_mpwqe_strides(struct mlx5_core_dev *mdev, - u8 log_stride_sz, u8 log_num_strides) +static bool mlx5e_verify_rx_mpwqe_strides(struct mlx5_core_dev *mdev, + u8 log_stride_sz, u8 log_num_strides, + u8 page_shift, + enum mlx5e_mpwrq_umr_mode umr_mode) { - if (log_stride_sz + log_num_strides != MLX5_MPWRQ_LOG_WQE_SZ) + if (log_stride_sz + log_num_strides != + mlx5e_mpwrq_log_wqe_sz(mdev, page_shift, umr_mode)) return false; if (log_stride_sz < MLX5_MPWQE_LOG_STRIDE_SZ_BASE || @@ -113,28 +324,53 @@ bool mlx5e_rx_mpwqe_is_linear_skb(struct mlx5_core_dev *mdev, struct mlx5e_params *params, struct mlx5e_xsk_param *xsk) { - s8 log_num_strides; + enum mlx5e_mpwrq_umr_mode umr_mode = mlx5e_mpwrq_umr_mode(mdev, xsk); + u8 page_shift = mlx5e_mpwrq_page_shift(mdev, xsk); + u8 log_num_strides; u8 log_stride_sz; + u8 log_wqe_sz; - if (!mlx5e_rx_is_linear_skb(params, xsk)) + if (!mlx5e_rx_is_linear_skb(mdev, params, xsk)) return false; - log_stride_sz = order_base_2(mlx5e_rx_get_linear_frag_sz(params, xsk)); - log_num_strides = MLX5_MPWRQ_LOG_WQE_SZ - log_stride_sz; + log_stride_sz = order_base_2(mlx5e_rx_get_linear_stride_sz(mdev, params, xsk, true)); + log_wqe_sz = mlx5e_mpwrq_log_wqe_sz(mdev, page_shift, umr_mode); - return mlx5e_verify_rx_mpwqe_strides(mdev, log_stride_sz, log_num_strides); + if (log_wqe_sz < log_stride_sz) + return false; + + log_num_strides = log_wqe_sz - log_stride_sz; + + return mlx5e_verify_rx_mpwqe_strides(mdev, log_stride_sz, + log_num_strides, page_shift, + umr_mode); } -u8 mlx5e_mpwqe_get_log_rq_size(struct mlx5e_params *params, +u8 mlx5e_mpwqe_get_log_rq_size(struct mlx5_core_dev *mdev, + struct mlx5e_params *params, struct mlx5e_xsk_param *xsk) { - u8 log_pkts_per_wqe = mlx5e_mpwqe_log_pkts_per_wqe(params, xsk); + enum mlx5e_mpwrq_umr_mode umr_mode = mlx5e_mpwrq_umr_mode(mdev, xsk); + u8 log_pkts_per_wqe, page_shift, max_log_rq_size; + + log_pkts_per_wqe = mlx5e_mpwqe_log_pkts_per_wqe(mdev, params, xsk); + page_shift = mlx5e_mpwrq_page_shift(mdev, xsk); + max_log_rq_size = mlx5e_mpwrq_max_log_rq_size(mdev, page_shift, umr_mode); /* Numbers are unsigned, don't subtract to avoid underflow. */ if (params->log_rq_mtu_frames < log_pkts_per_wqe + MLX5E_PARAMS_MINIMUM_LOG_RQ_SIZE_MPW) return MLX5E_PARAMS_MINIMUM_LOG_RQ_SIZE_MPW; + /* Ethtool's rx_max_pending is calculated for regular RQ, that uses + * pages of PAGE_SIZE. Max length of an XSK RQ might differ if it uses a + * frame size not equal to PAGE_SIZE. + * A stricter condition is checked in mlx5e_mpwrq_validate_xsk, WARN on + * unexpected failure. + */ + if (WARN_ON_ONCE(params->log_rq_mtu_frames > log_pkts_per_wqe + max_log_rq_size)) + return max_log_rq_size; + return params->log_rq_mtu_frames - log_pkts_per_wqe; } @@ -164,7 +400,7 @@ u8 mlx5e_mpwqe_get_log_stride_size(struct mlx5_core_dev *mdev, struct mlx5e_xsk_param *xsk) { if (mlx5e_rx_mpwqe_is_linear_skb(mdev, params, xsk)) - return order_base_2(mlx5e_rx_get_linear_frag_sz(params, xsk)); + return order_base_2(mlx5e_rx_get_linear_stride_sz(mdev, params, xsk, true)); return MLX5_MPWRQ_DEF_LOG_STRIDE_SZ(mdev); } @@ -173,7 +409,10 @@ u8 mlx5e_mpwqe_get_log_num_strides(struct mlx5_core_dev *mdev, struct mlx5e_params *params, struct mlx5e_xsk_param *xsk) { - return MLX5_MPWRQ_LOG_WQE_SZ - + enum mlx5e_mpwrq_umr_mode umr_mode = mlx5e_mpwrq_umr_mode(mdev, xsk); + u8 page_shift = mlx5e_mpwrq_page_shift(mdev, xsk); + + return mlx5e_mpwrq_log_wqe_sz(mdev, page_shift, umr_mode) - mlx5e_mpwqe_get_log_stride_size(mdev, params, xsk); } @@ -209,11 +448,11 @@ u16 mlx5e_calc_sq_stop_room(struct mlx5_core_dev *mdev, struct mlx5e_params *par stop_room = mlx5e_ktls_get_stop_room(mdev, params); stop_room += mlx5e_stop_room_for_max_wqe(mdev); if (is_mpwqe) - /* A MPWQE can take up to the maximum-sized WQE + all the normal - * stop room can be taken if a new packet breaks the active - * MPWQE session and allocates its WQEs right away. + /* A MPWQE can take up to the maximum cacheline-aligned WQE + + * all the normal stop room can be taken if a new packet breaks + * the active MPWQE session and allocates its WQEs right away. */ - stop_room += mlx5e_stop_room_for_max_wqe(mdev); + stop_room += mlx5e_stop_room_for_mpwqe(mdev); return stop_room; } @@ -320,22 +559,46 @@ bool slow_pci_heuristic(struct mlx5_core_dev *mdev) link_speed > MLX5E_SLOW_PCI_RATIO * pci_bw; } -bool mlx5e_striding_rq_possible(struct mlx5_core_dev *mdev, - struct mlx5e_params *params) +int mlx5e_mpwrq_validate_regular(struct mlx5_core_dev *mdev, struct mlx5e_params *params) { - if (!mlx5e_check_fragmented_striding_rq_cap(mdev)) - return false; + enum mlx5e_mpwrq_umr_mode umr_mode = mlx5e_mpwrq_umr_mode(mdev, NULL); + u8 page_shift = mlx5e_mpwrq_page_shift(mdev, NULL); - if (params->xdp_prog) { - /* XSK params are not considered here. If striding RQ is in use, - * and an XSK is being opened, mlx5e_rx_mpwqe_is_linear_skb will - * be called with the known XSK params. - */ - if (!mlx5e_rx_mpwqe_is_linear_skb(mdev, params, NULL)) - return false; + if (!mlx5e_check_fragmented_striding_rq_cap(mdev, page_shift, umr_mode)) + return -EOPNOTSUPP; + + if (params->xdp_prog && !mlx5e_rx_mpwqe_is_linear_skb(mdev, params, NULL)) + return -EINVAL; + + return 0; +} + +int mlx5e_mpwrq_validate_xsk(struct mlx5_core_dev *mdev, struct mlx5e_params *params, + struct mlx5e_xsk_param *xsk) +{ + enum mlx5e_mpwrq_umr_mode umr_mode = mlx5e_mpwrq_umr_mode(mdev, xsk); + u8 page_shift = mlx5e_mpwrq_page_shift(mdev, xsk); + bool unaligned = xsk ? xsk->unaligned : false; + u16 max_mtu_pkts; + + if (!mlx5e_check_fragmented_striding_rq_cap(mdev, page_shift, umr_mode)) + return -EOPNOTSUPP; + + if (!mlx5e_rx_mpwqe_is_linear_skb(mdev, params, xsk)) + return -EINVAL; + + /* Current RQ length is too big for the given frame size, the + * needed number of WQEs exceeds the maximum. + */ + max_mtu_pkts = min_t(u8, MLX5E_PARAMS_MAXIMUM_LOG_RQ_SIZE, + mlx5e_mpwrq_max_log_rq_pkts(mdev, page_shift, unaligned)); + if (params->log_rq_mtu_frames > max_mtu_pkts) { + mlx5_core_err(mdev, "Current RQ length %d is too big for XSK with given frame size %u\n", + 1 << params->log_rq_mtu_frames, xsk->chunk_size); + return -EINVAL; } - return true; + return 0; } void mlx5e_init_rq_type_params(struct mlx5_core_dev *mdev, @@ -348,7 +611,7 @@ void mlx5e_init_rq_type_params(struct mlx5_core_dev *mdev, mlx5_core_info(mdev, "MLX5E: StrdRq(%d) RqSz(%ld) StrdSz(%ld) RxCqeCmprss(%d)\n", params->rq_wq_type == MLX5_WQ_TYPE_LINKED_LIST_STRIDING_RQ, params->rq_wq_type == MLX5_WQ_TYPE_LINKED_LIST_STRIDING_RQ ? - BIT(mlx5e_mpwqe_get_log_rq_size(params, NULL)) : + BIT(mlx5e_mpwqe_get_log_rq_size(mdev, params, NULL)) : BIT(params->log_rq_mtu_frames), BIT(mlx5e_mpwqe_get_log_stride_size(mdev, params, NULL)), MLX5E_GET_PFLAG(params, MLX5E_PFLAG_RX_CQE_COMPRESS)); @@ -356,8 +619,7 @@ void mlx5e_init_rq_type_params(struct mlx5_core_dev *mdev, void mlx5e_set_rq_type(struct mlx5_core_dev *mdev, struct mlx5e_params *params) { - params->rq_wq_type = mlx5e_striding_rq_possible(mdev, params) && - MLX5E_GET_PFLAG(params, MLX5E_PFLAG_RX_STRIDING_RQ) ? + params->rq_wq_type = MLX5E_GET_PFLAG(params, MLX5E_PFLAG_RX_STRIDING_RQ) ? MLX5_WQ_TYPE_LINKED_LIST_STRIDING_RQ : MLX5_WQ_TYPE_CYCLIC; } @@ -374,9 +636,9 @@ void mlx5e_build_rq_params(struct mlx5_core_dev *mdev, */ if ((!MLX5E_GET_PFLAG(params, MLX5E_PFLAG_RX_CQE_COMPRESS) || MLX5_CAP_GEN(mdev, mini_cqe_resp_stride_index)) && - mlx5e_striding_rq_possible(mdev, params) && + !mlx5e_mpwrq_validate_regular(mdev, params) && (mlx5e_rx_mpwqe_is_linear_skb(mdev, params, NULL) || - !mlx5e_rx_is_linear_skb(params, NULL))) + !mlx5e_rx_is_linear_skb(mdev, params, NULL))) MLX5E_SET_PFLAG(params, MLX5E_PFLAG_RX_STRIDING_RQ, true); mlx5e_set_rq_type(mdev, params); mlx5e_init_rq_type_params(mdev, params); @@ -419,16 +681,22 @@ static int mlx5e_build_rq_frags_info(struct mlx5_core_dev *mdev, int max_mtu; int i; - if (mlx5e_rx_is_linear_skb(params, xsk)) { + if (mlx5e_rx_is_linear_skb(mdev, params, xsk)) { int frag_stride; - frag_stride = mlx5e_rx_get_linear_frag_sz(params, xsk); - frag_stride = roundup_pow_of_two(frag_stride); + frag_stride = mlx5e_rx_get_linear_stride_sz(mdev, params, xsk, false); info->arr[0].frag_size = byte_count; info->arr[0].frag_stride = frag_stride; info->num_frags = 1; - info->wqe_bulk = PAGE_SIZE / frag_stride; + + /* N WQEs share the same page, N = PAGE_SIZE / frag_stride. The + * first WQE in the page is responsible for allocation of this + * page, this WQE's index is k*N. If WQEs [k*N+1; k*N+N-1] are + * still not completed, the allocation must stop before k*N. + */ + info->wqe_index_mask = (PAGE_SIZE / frag_stride) - 1; + goto out; } @@ -477,11 +745,40 @@ static int mlx5e_build_rq_frags_info(struct mlx5_core_dev *mdev, i++; } info->num_frags = i; - /* number of different wqes sharing a page */ - info->wqe_bulk = 1 + (info->num_frags % 2); + + /* The last fragment of WQE with index 2*N may share the page with the + * first fragment of WQE with index 2*N+1 in certain cases. If WQE 2*N+1 + * is not completed yet, WQE 2*N must not be allocated, as it's + * responsible for allocating a new page. + */ + if (frag_size_max == PAGE_SIZE) { + /* No WQE can start in the middle of a page. */ + info->wqe_index_mask = 0; + } else { + /* PAGE_SIZEs starting from 8192 don't use 2K-sized fragments, + * because there would be more than MLX5E_MAX_RX_FRAGS of them. + */ + WARN_ON(PAGE_SIZE != 2 * DEFAULT_FRAG_SIZE); + + /* Odd number of fragments allows to pack the last fragment of + * the previous WQE and the first fragment of the next WQE into + * the same page. + * As long as DEFAULT_FRAG_SIZE is 2048, and MLX5E_MAX_RX_FRAGS + * is 4, the last fragment can be bigger than the rest only if + * it's the fourth one, so WQEs consisting of 3 fragments will + * always share a page. + * When a page is shared, WQE bulk size is 2, otherwise just 1. + */ + info->wqe_index_mask = info->num_frags % 2; + } out: - info->wqe_bulk = max_t(u8, info->wqe_bulk, 8); + /* Bulking optimization to skip allocation until at least 8 WQEs can be + * allocated in a row. At the same time, never start allocation when + * the page is still used by older WQEs. + */ + info->wqe_bulk = max_t(u8, info->wqe_index_mask + 1, 8); + info->log_num_frags = order_base_2(info->num_frags); return 0; @@ -520,7 +817,7 @@ static u32 mlx5e_shampo_get_log_cq_size(struct mlx5_core_dev *mdev, u16 num_strides = BIT(mlx5e_mpwqe_get_log_num_strides(mdev, params, xsk)); int pkt_per_rsrv = BIT(mlx5e_shampo_get_log_pkt_per_rsrv(mdev, params)); u8 log_stride_sz = mlx5e_mpwqe_get_log_stride_size(mdev, params, xsk); - int wq_size = BIT(mlx5e_mpwqe_get_log_rq_size(params, xsk)); + int wq_size = BIT(mlx5e_mpwqe_get_log_rq_size(mdev, params, xsk)); int wqe_size = BIT(log_stride_sz) * num_strides; /* +1 is for the case that the pkt_per_rsrv dont consume the reservation @@ -544,7 +841,7 @@ static void mlx5e_build_rx_cq_param(struct mlx5_core_dev *mdev, if (params->packet_merge.type == MLX5E_PACKET_MERGE_SHAMPO) log_cq_size = mlx5e_shampo_get_log_cq_size(mdev, params, xsk); else - log_cq_size = mlx5e_mpwqe_get_log_rq_size(params, xsk) + + log_cq_size = mlx5e_mpwqe_get_log_rq_size(mdev, params, xsk) + mlx5e_mpwqe_get_log_num_strides(mdev, params, xsk); break; default: /* MLX5_WQ_TYPE_CYCLIC */ @@ -587,12 +884,16 @@ int mlx5e_build_rq_param(struct mlx5_core_dev *mdev, case MLX5_WQ_TYPE_LINKED_LIST_STRIDING_RQ: { u8 log_wqe_num_of_strides = mlx5e_mpwqe_get_log_num_strides(mdev, params, xsk); u8 log_wqe_stride_size = mlx5e_mpwqe_get_log_stride_size(mdev, params, xsk); + enum mlx5e_mpwrq_umr_mode umr_mode = mlx5e_mpwrq_umr_mode(mdev, xsk); + u8 page_shift = mlx5e_mpwrq_page_shift(mdev, xsk); if (!mlx5e_verify_rx_mpwqe_strides(mdev, log_wqe_stride_size, - log_wqe_num_of_strides)) { + log_wqe_num_of_strides, + page_shift, umr_mode)) { mlx5_core_err(mdev, - "Bad RX MPWQE params: log_stride_size %u, log_num_strides %u\n", - log_wqe_stride_size, log_wqe_num_of_strides); + "Bad RX MPWQE params: log_stride_size %u, log_num_strides %u, umr_mode %d\n", + log_wqe_stride_size, log_wqe_num_of_strides, + umr_mode); return -EINVAL; } @@ -600,7 +901,7 @@ int mlx5e_build_rq_param(struct mlx5_core_dev *mdev, log_wqe_num_of_strides - MLX5_MPWQE_LOG_NUM_STRIDES_BASE); MLX5_SET(wq, wq, log_wqe_stride_size, log_wqe_stride_size - MLX5_MPWQE_LOG_STRIDE_SZ_BASE); - MLX5_SET(wq, wq, log_wq_sz, mlx5e_mpwqe_get_log_rq_size(params, xsk)); + MLX5_SET(wq, wq, log_wq_sz, mlx5e_mpwqe_get_log_rq_size(mdev, params, xsk)); if (params->packet_merge.type == MLX5E_PACKET_MERGE_SHAMPO) { MLX5_SET(wq, wq, shampo_enable, true); MLX5_SET(wq, wq, log_reservation_size, @@ -712,13 +1013,6 @@ static void mlx5e_build_ico_cq_param(struct mlx5_core_dev *mdev, param->cq_period_mode = DIM_CQ_PERIOD_MODE_START_FROM_EQE; } -static u8 mlx5e_get_rq_log_wq_sz(void *rqc) -{ - void *wq = MLX5_ADDR_OF(rqc, rqc, wq); - - return MLX5_GET(wq, wq, log_wq_sz); -} - /* This function calculates the maximum number of headers entries that are needed * per WQE, the formula is based on the size of the reservations and the * restriction we have about max packets for reservation that is equal to max @@ -779,31 +1073,92 @@ static u32 mlx5e_shampo_icosq_sz(struct mlx5_core_dev *mdev, return wqebbs; } +static u32 mlx5e_mpwrq_total_umr_wqebbs(struct mlx5_core_dev *mdev, + struct mlx5e_params *params, + struct mlx5e_xsk_param *xsk) +{ + enum mlx5e_mpwrq_umr_mode umr_mode = mlx5e_mpwrq_umr_mode(mdev, xsk); + u8 page_shift = mlx5e_mpwrq_page_shift(mdev, xsk); + u8 umr_wqebbs; + + umr_wqebbs = mlx5e_mpwrq_umr_wqebbs(mdev, page_shift, umr_mode); + + return umr_wqebbs * (1 << mlx5e_mpwqe_get_log_rq_size(mdev, params, xsk)); +} + static u8 mlx5e_build_icosq_log_wq_sz(struct mlx5_core_dev *mdev, struct mlx5e_params *params, struct mlx5e_rq_param *rqp) { - u32 wqebbs; + u32 wqebbs, total_pages, useful_space; /* MLX5_WQ_TYPE_CYCLIC */ if (params->rq_wq_type != MLX5_WQ_TYPE_LINKED_LIST_STRIDING_RQ) return MLX5E_PARAMS_MINIMUM_LOG_SQ_SIZE; - wqebbs = MLX5E_UMR_WQEBBS * BIT(mlx5e_get_rq_log_wq_sz(rqp->rqc)); + /* UMR WQEs for the regular RQ. */ + wqebbs = mlx5e_mpwrq_total_umr_wqebbs(mdev, params, NULL); /* If XDP program is attached, XSK may be turned on at any time without * restarting the channel. ICOSQ must be big enough to fit UMR WQEs of * both regular RQ and XSK RQ. - * Although mlx5e_mpwqe_get_log_rq_size accepts mlx5e_xsk_param, it - * doesn't affect its return value, as long as params->xdp_prog != NULL, - * so we can just multiply by 2. + * + * XSK uses different values of page_shift, and the total number of UMR + * WQEBBs depends on it. This dependency is complex and not monotonic, + * especially taking into consideration that some of the parameters come + * from capabilities. Hence, we have to try all valid values of XSK + * frame size (and page_shift) to find the maximum. */ - if (params->xdp_prog) - wqebbs *= 2; + if (params->xdp_prog) { + u32 max_xsk_wqebbs = 0; + u8 frame_shift; + + for (frame_shift = XDP_UMEM_MIN_CHUNK_SHIFT; + frame_shift <= PAGE_SHIFT; frame_shift++) { + /* The headroom doesn't affect the calculation. */ + struct mlx5e_xsk_param xsk = { + .chunk_size = 1 << frame_shift, + .unaligned = false, + }; + + /* XSK aligned mode. */ + max_xsk_wqebbs = max(max_xsk_wqebbs, + mlx5e_mpwrq_total_umr_wqebbs(mdev, params, &xsk)); + + /* XSK unaligned mode, frame size is a power of two. */ + xsk.unaligned = true; + max_xsk_wqebbs = max(max_xsk_wqebbs, + mlx5e_mpwrq_total_umr_wqebbs(mdev, params, &xsk)); + + /* XSK unaligned mode, frame size is not equal to stride size. */ + xsk.chunk_size -= 1; + max_xsk_wqebbs = max(max_xsk_wqebbs, + mlx5e_mpwrq_total_umr_wqebbs(mdev, params, &xsk)); + + /* XSK unaligned mode, frame size is a triple power of two. */ + xsk.chunk_size = (1 << frame_shift) / 4 * 3; + max_xsk_wqebbs = max(max_xsk_wqebbs, + mlx5e_mpwrq_total_umr_wqebbs(mdev, params, &xsk)); + } + + wqebbs += max_xsk_wqebbs; + } if (params->packet_merge.type == MLX5E_PACKET_MERGE_SHAMPO) wqebbs += mlx5e_shampo_icosq_sz(mdev, params, rqp); + /* UMR WQEs don't cross the page boundary, they are padded with NOPs. + * This padding is always smaller than the max WQE size. That gives us + * at least (PAGE_SIZE - (max WQE size - MLX5_SEND_WQE_BB)) useful bytes + * per page. The number of pages is estimated as the total size of WQEs + * divided by the useful space in page, rounding up. If some WQEs don't + * fully fit into the useful space, they can occupy part of the padding, + * which proves this estimation to be correct (reserve enough space). + */ + useful_space = PAGE_SIZE - mlx5e_get_max_sq_wqebbs(mdev) + MLX5_SEND_WQE_BB; + total_pages = DIV_ROUND_UP(wqebbs * MLX5_SEND_WQE_BB, useful_space); + wqebbs = total_pages * (PAGE_SIZE / MLX5_SEND_WQE_BB); + return max_t(u8, MLX5E_PARAMS_MINIMUM_LOG_SQ_SIZE, order_base_2(wqebbs)); } @@ -857,7 +1212,7 @@ void mlx5e_build_xdpsq_param(struct mlx5_core_dev *mdev, mlx5e_build_sq_param_common(mdev, param); MLX5_SET(wq, wq, log_wq_sz, params->log_sq_size); param->is_mpw = MLX5E_GET_PFLAG(params, MLX5E_PFLAG_XDP_TX_MPWQE); - param->is_xdp_mb = !mlx5e_rx_is_linear_skb(params, xsk); + param->is_xdp_mb = !mlx5e_rx_is_linear_skb(mdev, params, xsk); mlx5e_build_tx_cq_param(mdev, params, ¶m->cqp); } diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/params.h b/drivers/net/ethernet/mellanox/mlx5/core/en/params.h index f5c46e78eebc..034debd140bc 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/params.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/params.h @@ -9,6 +9,7 @@ struct mlx5e_xsk_param { u16 headroom; u16 chunk_size; + bool unaligned; }; struct mlx5e_cq_param { @@ -52,37 +53,26 @@ struct mlx5e_create_sq_param { u8 min_inline_mode; }; -static inline bool mlx5e_qid_get_ch_if_in_group(struct mlx5e_params *params, - u16 qid, - enum mlx5e_rq_group group, - u16 *ix) -{ - int nch = params->num_channels; - int ch = qid - nch * group; - - if (ch < 0 || ch >= nch) - return false; - - *ix = ch; - return true; -} - -static inline void mlx5e_qid_get_ch_and_group(struct mlx5e_params *params, - u16 qid, - u16 *ix, - enum mlx5e_rq_group *group) -{ - u16 nch = params->num_channels; - - *ix = qid % nch; - *group = qid / nch; -} - -static inline bool mlx5e_qid_validate(const struct mlx5e_profile *profile, - struct mlx5e_params *params, u64 qid) -{ - return qid < params->num_channels * profile->rq_groups; -} +/* Striding RQ dynamic parameters */ + +u8 mlx5e_mpwrq_page_shift(struct mlx5_core_dev *mdev, struct mlx5e_xsk_param *xsk); +enum mlx5e_mpwrq_umr_mode +mlx5e_mpwrq_umr_mode(struct mlx5_core_dev *mdev, struct mlx5e_xsk_param *xsk); +u8 mlx5e_mpwrq_umr_entry_size(enum mlx5e_mpwrq_umr_mode mode); +u8 mlx5e_mpwrq_log_wqe_sz(struct mlx5_core_dev *mdev, u8 page_shift, + enum mlx5e_mpwrq_umr_mode umr_mode); +u8 mlx5e_mpwrq_pages_per_wqe(struct mlx5_core_dev *mdev, u8 page_shift, + enum mlx5e_mpwrq_umr_mode umr_mode); +u16 mlx5e_mpwrq_umr_wqe_sz(struct mlx5_core_dev *mdev, u8 page_shift, + enum mlx5e_mpwrq_umr_mode umr_mode); +u8 mlx5e_mpwrq_umr_wqebbs(struct mlx5_core_dev *mdev, u8 page_shift, + enum mlx5e_mpwrq_umr_mode umr_mode); +u8 mlx5e_mpwrq_mtts_per_wqe(struct mlx5_core_dev *mdev, u8 page_shift, + enum mlx5e_mpwrq_umr_mode umr_mode); +u32 mlx5e_mpwrq_max_num_entries(struct mlx5_core_dev *mdev, + enum mlx5e_mpwrq_umr_mode umr_mode); +u8 mlx5e_mpwrq_max_log_rq_pkts(struct mlx5_core_dev *mdev, u8 page_shift, + enum mlx5e_mpwrq_umr_mode umr_mode); /* Parameter calculations */ @@ -92,25 +82,23 @@ void mlx5e_set_tx_cq_mode_params(struct mlx5e_params *params, u8 cq_period_mode) void mlx5e_set_rx_cq_mode_params(struct mlx5e_params *params, u8 cq_period_mode); bool slow_pci_heuristic(struct mlx5_core_dev *mdev); -bool mlx5e_striding_rq_possible(struct mlx5_core_dev *mdev, struct mlx5e_params *params); +int mlx5e_mpwrq_validate_regular(struct mlx5_core_dev *mdev, struct mlx5e_params *params); +int mlx5e_mpwrq_validate_xsk(struct mlx5_core_dev *mdev, struct mlx5e_params *params, + struct mlx5e_xsk_param *xsk); void mlx5e_build_rq_params(struct mlx5_core_dev *mdev, struct mlx5e_params *params); void mlx5e_set_rq_type(struct mlx5_core_dev *mdev, struct mlx5e_params *params); void mlx5e_init_rq_type_params(struct mlx5_core_dev *mdev, struct mlx5e_params *params); -bool mlx5e_verify_rx_mpwqe_strides(struct mlx5_core_dev *mdev, - u8 log_stride_sz, u8 log_num_strides); u16 mlx5e_get_linear_rq_headroom(struct mlx5e_params *params, struct mlx5e_xsk_param *xsk); -u32 mlx5e_rx_get_min_frag_sz(struct mlx5e_params *params, - struct mlx5e_xsk_param *xsk); -u8 mlx5e_mpwqe_log_pkts_per_wqe(struct mlx5e_params *params, - struct mlx5e_xsk_param *xsk); -bool mlx5e_rx_is_linear_skb(struct mlx5e_params *params, +bool mlx5e_rx_is_linear_skb(struct mlx5_core_dev *mdev, + struct mlx5e_params *params, struct mlx5e_xsk_param *xsk); bool mlx5e_rx_mpwqe_is_linear_skb(struct mlx5_core_dev *mdev, struct mlx5e_params *params, struct mlx5e_xsk_param *xsk); -u8 mlx5e_mpwqe_get_log_rq_size(struct mlx5e_params *params, +u8 mlx5e_mpwqe_get_log_rq_size(struct mlx5_core_dev *mdev, + struct mlx5e_params *params, struct mlx5e_xsk_param *xsk); u8 mlx5e_shampo_get_log_hd_entry_size(struct mlx5_core_dev *mdev, struct mlx5e_params *params); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/ptp.c b/drivers/net/ethernet/mellanox/mlx5/core/en/ptp.c index 903de88bab53..8469e9c38670 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/ptp.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/ptp.c @@ -622,37 +622,39 @@ static int mlx5e_ptp_set_state(struct mlx5e_ptp *c, struct mlx5e_params *params) return bitmap_empty(c->state, MLX5E_PTP_STATE_NUM_STATES) ? -EINVAL : 0; } -static void mlx5e_ptp_rx_unset_fs(struct mlx5e_priv *priv) +static void mlx5e_ptp_rx_unset_fs(struct mlx5e_flow_steering *fs) { - struct mlx5e_ptp_fs *ptp_fs = priv->fs->ptp_fs; + struct mlx5e_ptp_fs *ptp_fs = mlx5e_fs_get_ptp(fs); if (!ptp_fs->valid) return; mlx5e_fs_tt_redirect_del_rule(ptp_fs->l2_rule); - mlx5e_fs_tt_redirect_any_destroy(priv); + mlx5e_fs_tt_redirect_any_destroy(fs); mlx5e_fs_tt_redirect_del_rule(ptp_fs->udp_v6_rule); mlx5e_fs_tt_redirect_del_rule(ptp_fs->udp_v4_rule); - mlx5e_fs_tt_redirect_udp_destroy(priv); + mlx5e_fs_tt_redirect_udp_destroy(fs); ptp_fs->valid = false; } static int mlx5e_ptp_rx_set_fs(struct mlx5e_priv *priv) { u32 tirn = mlx5e_rx_res_get_tirn_ptp(priv->rx_res); - struct mlx5e_ptp_fs *ptp_fs = priv->fs->ptp_fs; + struct mlx5e_flow_steering *fs = priv->fs; struct mlx5_flow_handle *rule; + struct mlx5e_ptp_fs *ptp_fs; int err; + ptp_fs = mlx5e_fs_get_ptp(fs); if (ptp_fs->valid) return 0; - err = mlx5e_fs_tt_redirect_udp_create(priv); + err = mlx5e_fs_tt_redirect_udp_create(fs); if (err) goto out_free; - rule = mlx5e_fs_tt_redirect_udp_add_rule(priv, MLX5_TT_IPV4_UDP, + rule = mlx5e_fs_tt_redirect_udp_add_rule(fs, MLX5_TT_IPV4_UDP, tirn, PTP_EV_PORT); if (IS_ERR(rule)) { err = PTR_ERR(rule); @@ -660,7 +662,7 @@ static int mlx5e_ptp_rx_set_fs(struct mlx5e_priv *priv) } ptp_fs->udp_v4_rule = rule; - rule = mlx5e_fs_tt_redirect_udp_add_rule(priv, MLX5_TT_IPV6_UDP, + rule = mlx5e_fs_tt_redirect_udp_add_rule(fs, MLX5_TT_IPV6_UDP, tirn, PTP_EV_PORT); if (IS_ERR(rule)) { err = PTR_ERR(rule); @@ -668,11 +670,11 @@ static int mlx5e_ptp_rx_set_fs(struct mlx5e_priv *priv) } ptp_fs->udp_v6_rule = rule; - err = mlx5e_fs_tt_redirect_any_create(priv); + err = mlx5e_fs_tt_redirect_any_create(fs); if (err) goto out_destroy_udp_v6_rule; - rule = mlx5e_fs_tt_redirect_any_add_rule(priv, tirn, ETH_P_1588); + rule = mlx5e_fs_tt_redirect_any_add_rule(fs, tirn, ETH_P_1588); if (IS_ERR(rule)) { err = PTR_ERR(rule); goto out_destroy_fs_any; @@ -683,13 +685,13 @@ static int mlx5e_ptp_rx_set_fs(struct mlx5e_priv *priv) return 0; out_destroy_fs_any: - mlx5e_fs_tt_redirect_any_destroy(priv); + mlx5e_fs_tt_redirect_any_destroy(fs); out_destroy_udp_v6_rule: mlx5e_fs_tt_redirect_del_rule(ptp_fs->udp_v6_rule); out_destroy_udp_v4_rule: mlx5e_fs_tt_redirect_del_rule(ptp_fs->udp_v4_rule); out_destroy_fs_udp: - mlx5e_fs_tt_redirect_udp_destroy(priv); + mlx5e_fs_tt_redirect_udp_destroy(fs); out_free: return err; } @@ -723,7 +725,7 @@ int mlx5e_ptp_open(struct mlx5e_priv *priv, struct mlx5e_params *params, if (err) goto err_free; - netif_napi_add(netdev, &c->napi, mlx5e_ptp_napi_poll, 64); + netif_napi_add(netdev, &c->napi, mlx5e_ptp_napi_poll); mlx5e_ptp_build_params(c, cparams, params); @@ -797,29 +799,31 @@ int mlx5e_ptp_get_rqn(struct mlx5e_ptp *c, u32 *rqn) return 0; } -int mlx5e_ptp_alloc_rx_fs(struct mlx5e_priv *priv) +int mlx5e_ptp_alloc_rx_fs(struct mlx5e_flow_steering *fs, + const struct mlx5e_profile *profile) { struct mlx5e_ptp_fs *ptp_fs; - if (!mlx5e_profile_feature_cap(priv->profile, PTP_RX)) + if (!mlx5e_profile_feature_cap(profile, PTP_RX)) return 0; ptp_fs = kzalloc(sizeof(*ptp_fs), GFP_KERNEL); if (!ptp_fs) return -ENOMEM; + mlx5e_fs_set_ptp(fs, ptp_fs); - priv->fs->ptp_fs = ptp_fs; return 0; } -void mlx5e_ptp_free_rx_fs(struct mlx5e_priv *priv) +void mlx5e_ptp_free_rx_fs(struct mlx5e_flow_steering *fs, + const struct mlx5e_profile *profile) { - struct mlx5e_ptp_fs *ptp_fs = priv->fs->ptp_fs; + struct mlx5e_ptp_fs *ptp_fs = mlx5e_fs_get_ptp(fs); - if (!mlx5e_profile_feature_cap(priv->profile, PTP_RX)) + if (!mlx5e_profile_feature_cap(profile, PTP_RX)) return; - mlx5e_ptp_rx_unset_fs(priv); + mlx5e_ptp_rx_unset_fs(fs); kfree(ptp_fs); } @@ -845,6 +849,6 @@ int mlx5e_ptp_rx_manage_fs(struct mlx5e_priv *priv, bool set) netdev_WARN_ONCE(priv->netdev, "Don't try to remove PTP RX-FS rules"); return -EINVAL; } - mlx5e_ptp_rx_unset_fs(priv); + mlx5e_ptp_rx_unset_fs(priv->fs); return 0; } diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/ptp.h b/drivers/net/ethernet/mellanox/mlx5/core/en/ptp.h index 92dbbec472ec..5bce554e131a 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/ptp.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/ptp.h @@ -74,8 +74,10 @@ void mlx5e_ptp_close(struct mlx5e_ptp *c); void mlx5e_ptp_activate_channel(struct mlx5e_ptp *c); void mlx5e_ptp_deactivate_channel(struct mlx5e_ptp *c); int mlx5e_ptp_get_rqn(struct mlx5e_ptp *c, u32 *rqn); -int mlx5e_ptp_alloc_rx_fs(struct mlx5e_priv *priv); -void mlx5e_ptp_free_rx_fs(struct mlx5e_priv *priv); +int mlx5e_ptp_alloc_rx_fs(struct mlx5e_flow_steering *fs, + const struct mlx5e_profile *profile); +void mlx5e_ptp_free_rx_fs(struct mlx5e_flow_steering *fs, + const struct mlx5e_profile *profile); int mlx5e_ptp_rx_manage_fs(struct mlx5e_priv *priv, bool set); enum { diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/reporter_rx.c b/drivers/net/ethernet/mellanox/mlx5/core/en/reporter_rx.c index fc366e66d0b0..5f6f95ad6888 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/reporter_rx.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/reporter_rx.c @@ -134,38 +134,17 @@ out: return err; } -static int mlx5e_rq_to_ready(struct mlx5e_rq *rq, int curr_state) -{ - struct net_device *dev = rq->netdev; - int err; - - err = mlx5e_modify_rq_state(rq, curr_state, MLX5_RQC_STATE_RST); - if (err) { - netdev_err(dev, "Failed to move rq 0x%x to reset\n", rq->rqn); - return err; - } - err = mlx5e_modify_rq_state(rq, MLX5_RQC_STATE_RST, MLX5_RQC_STATE_RDY); - if (err) { - netdev_err(dev, "Failed to move rq 0x%x to ready\n", rq->rqn); - return err; - } - - return 0; -} - static int mlx5e_rx_reporter_err_rq_cqe_recover(void *ctx) { struct mlx5e_rq *rq = ctx; int err; mlx5e_deactivate_rq(rq); - mlx5e_free_rx_descs(rq); - - err = mlx5e_rq_to_ready(rq, MLX5_RQC_STATE_ERR); + err = mlx5e_flush_rq(rq, MLX5_RQC_STATE_ERR); + clear_bit(MLX5E_RQ_STATE_RECOVERING, &rq->state); if (err) - goto out; + return err; - clear_bit(MLX5E_RQ_STATE_RECOVERING, &rq->state); mlx5e_activate_rq(rq); rq->stats->recover++; if (rq->channel) @@ -173,9 +152,6 @@ static int mlx5e_rx_reporter_err_rq_cqe_recover(void *ctx) else mlx5e_trigger_napi_sched(rq->cq.napi); return 0; -out: - clear_bit(MLX5E_RQ_STATE_RECOVERING, &rq->state); - return err; } static int mlx5e_rx_reporter_timeout_recover(void *ctx) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/rx_res.c b/drivers/net/ethernet/mellanox/mlx5/core/en/rx_res.c index 24c32f73040a..e1095bc36543 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/rx_res.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/rx_res.c @@ -24,8 +24,6 @@ struct mlx5e_rx_res { struct { struct mlx5e_rqt direct_rqt; struct mlx5e_tir direct_tir; - struct mlx5e_rqt xsk_rqt; - struct mlx5e_tir xsk_tir; } *channels; struct { @@ -320,48 +318,8 @@ static int mlx5e_rx_res_channels_init(struct mlx5e_rx_res *res) mlx5e_tir_builder_clear(builder); } - if (!(res->features & MLX5E_RX_RES_FEATURE_XSK)) - goto out; - - for (ix = 0; ix < res->max_nch; ix++) { - err = mlx5e_rqt_init_direct(&res->channels[ix].xsk_rqt, - res->mdev, false, res->drop_rqn); - if (err) { - mlx5_core_warn(res->mdev, "Failed to create an XSK RQT: err = %d, ix = %u\n", - err, ix); - goto err_destroy_xsk_rqts; - } - } - - for (ix = 0; ix < res->max_nch; ix++) { - mlx5e_tir_builder_build_rqt(builder, res->mdev->mlx5e_res.hw_objs.td.tdn, - mlx5e_rqt_get_rqtn(&res->channels[ix].xsk_rqt), - inner_ft_support); - mlx5e_tir_builder_build_packet_merge(builder, &res->pkt_merge_param); - mlx5e_tir_builder_build_direct(builder); - - err = mlx5e_tir_init(&res->channels[ix].xsk_tir, builder, res->mdev, true); - if (err) { - mlx5_core_warn(res->mdev, "Failed to create an XSK TIR: err = %d, ix = %u\n", - err, ix); - goto err_destroy_xsk_tirs; - } - - mlx5e_tir_builder_clear(builder); - } - goto out; -err_destroy_xsk_tirs: - while (--ix >= 0) - mlx5e_tir_destroy(&res->channels[ix].xsk_tir); - - ix = res->max_nch; -err_destroy_xsk_rqts: - while (--ix >= 0) - mlx5e_rqt_destroy(&res->channels[ix].xsk_rqt); - - ix = res->max_nch; err_destroy_direct_tirs: while (--ix >= 0) mlx5e_tir_destroy(&res->channels[ix].direct_tir); @@ -420,12 +378,6 @@ static void mlx5e_rx_res_channels_destroy(struct mlx5e_rx_res *res) for (ix = 0; ix < res->max_nch; ix++) { mlx5e_tir_destroy(&res->channels[ix].direct_tir); mlx5e_rqt_destroy(&res->channels[ix].direct_rqt); - - if (!(res->features & MLX5E_RX_RES_FEATURE_XSK)) - continue; - - mlx5e_tir_destroy(&res->channels[ix].xsk_tir); - mlx5e_rqt_destroy(&res->channels[ix].xsk_rqt); } kvfree(res->channels); @@ -491,13 +443,6 @@ u32 mlx5e_rx_res_get_tirn_direct(struct mlx5e_rx_res *res, unsigned int ix) return mlx5e_tir_get_tirn(&res->channels[ix].direct_tir); } -u32 mlx5e_rx_res_get_tirn_xsk(struct mlx5e_rx_res *res, unsigned int ix) -{ - WARN_ON(!(res->features & MLX5E_RX_RES_FEATURE_XSK)); - - return mlx5e_tir_get_tirn(&res->channels[ix].xsk_tir); -} - u32 mlx5e_rx_res_get_tirn_rss(struct mlx5e_rx_res *res, enum mlx5_traffic_types tt) { struct mlx5e_rss *rss = res->rss[0]; @@ -523,56 +468,53 @@ static u32 mlx5e_rx_res_get_rqtn_direct(struct mlx5e_rx_res *res, unsigned int i return mlx5e_rqt_get_rqtn(&res->channels[ix].direct_rqt); } -void mlx5e_rx_res_channels_activate(struct mlx5e_rx_res *res, struct mlx5e_channels *chs) +static void mlx5e_rx_res_channel_activate_direct(struct mlx5e_rx_res *res, + struct mlx5e_channels *chs, + unsigned int ix) { - unsigned int nch, ix; + u32 rqn = res->rss_rqns[ix]; int err; - nch = mlx5e_channels_get_num(chs); - - for (ix = 0; ix < chs->num; ix++) - mlx5e_channels_get_regular_rqn(chs, ix, &res->rss_rqns[ix]); - res->rss_nch = chs->num; + err = mlx5e_rqt_redirect_direct(&res->channels[ix].direct_rqt, rqn); + if (err) + mlx5_core_warn(res->mdev, "Failed to redirect direct RQT %#x to RQ %#x (channel %u): err = %d\n", + mlx5e_rqt_get_rqtn(&res->channels[ix].direct_rqt), + rqn, ix, err); +} - mlx5e_rx_res_rss_enable(res); +static void mlx5e_rx_res_channel_deactivate_direct(struct mlx5e_rx_res *res, + unsigned int ix) +{ + int err; - for (ix = 0; ix < nch; ix++) { - u32 rqn; + err = mlx5e_rqt_redirect_direct(&res->channels[ix].direct_rqt, res->drop_rqn); + if (err) + mlx5_core_warn(res->mdev, "Failed to redirect direct RQT %#x to drop RQ %#x (channel %u): err = %d\n", + mlx5e_rqt_get_rqtn(&res->channels[ix].direct_rqt), + res->drop_rqn, ix, err); +} - mlx5e_channels_get_regular_rqn(chs, ix, &rqn); - err = mlx5e_rqt_redirect_direct(&res->channels[ix].direct_rqt, rqn); - if (err) - mlx5_core_warn(res->mdev, "Failed to redirect direct RQT %#x to RQ %#x (channel %u): err = %d\n", - mlx5e_rqt_get_rqtn(&res->channels[ix].direct_rqt), - rqn, ix, err); +void mlx5e_rx_res_channels_activate(struct mlx5e_rx_res *res, struct mlx5e_channels *chs) +{ + unsigned int nch, ix; + int err; - if (!(res->features & MLX5E_RX_RES_FEATURE_XSK)) - continue; + nch = mlx5e_channels_get_num(chs); - if (!mlx5e_channels_get_xsk_rqn(chs, ix, &rqn)) - rqn = res->drop_rqn; - err = mlx5e_rqt_redirect_direct(&res->channels[ix].xsk_rqt, rqn); - if (err) - mlx5_core_warn(res->mdev, "Failed to redirect XSK RQT %#x to RQ %#x (channel %u): err = %d\n", - mlx5e_rqt_get_rqtn(&res->channels[ix].xsk_rqt), - rqn, ix, err); + for (ix = 0; ix < chs->num; ix++) { + if (mlx5e_channels_is_xsk(chs, ix)) + mlx5e_channels_get_xsk_rqn(chs, ix, &res->rss_rqns[ix]); + else + mlx5e_channels_get_regular_rqn(chs, ix, &res->rss_rqns[ix]); } - for (ix = nch; ix < res->max_nch; ix++) { - err = mlx5e_rqt_redirect_direct(&res->channels[ix].direct_rqt, res->drop_rqn); - if (err) - mlx5_core_warn(res->mdev, "Failed to redirect direct RQT %#x to drop RQ %#x (channel %u): err = %d\n", - mlx5e_rqt_get_rqtn(&res->channels[ix].direct_rqt), - res->drop_rqn, ix, err); + res->rss_nch = chs->num; - if (!(res->features & MLX5E_RX_RES_FEATURE_XSK)) - continue; + mlx5e_rx_res_rss_enable(res); - err = mlx5e_rqt_redirect_direct(&res->channels[ix].xsk_rqt, res->drop_rqn); - if (err) - mlx5_core_warn(res->mdev, "Failed to redirect XSK RQT %#x to drop RQ %#x (channel %u): err = %d\n", - mlx5e_rqt_get_rqtn(&res->channels[ix].xsk_rqt), - res->drop_rqn, ix, err); - } + for (ix = 0; ix < nch; ix++) + mlx5e_rx_res_channel_activate_direct(res, chs, ix); + for (ix = nch; ix < res->max_nch; ix++) + mlx5e_rx_res_channel_deactivate_direct(res, ix); if (res->features & MLX5E_RX_RES_FEATURE_PTP) { u32 rqn; @@ -595,22 +537,8 @@ void mlx5e_rx_res_channels_deactivate(struct mlx5e_rx_res *res) mlx5e_rx_res_rss_disable(res); - for (ix = 0; ix < res->max_nch; ix++) { - err = mlx5e_rqt_redirect_direct(&res->channels[ix].direct_rqt, res->drop_rqn); - if (err) - mlx5_core_warn(res->mdev, "Failed to redirect direct RQT %#x to drop RQ %#x (channel %u): err = %d\n", - mlx5e_rqt_get_rqtn(&res->channels[ix].direct_rqt), - res->drop_rqn, ix, err); - - if (!(res->features & MLX5E_RX_RES_FEATURE_XSK)) - continue; - - err = mlx5e_rqt_redirect_direct(&res->channels[ix].xsk_rqt, res->drop_rqn); - if (err) - mlx5_core_warn(res->mdev, "Failed to redirect XSK RQT %#x to drop RQ %#x (channel %u): err = %d\n", - mlx5e_rqt_get_rqtn(&res->channels[ix].xsk_rqt), - res->drop_rqn, ix, err); - } + for (ix = 0; ix < res->max_nch; ix++) + mlx5e_rx_res_channel_deactivate_direct(res, ix); if (res->features & MLX5E_RX_RES_FEATURE_PTP) { err = mlx5e_rqt_redirect_direct(&res->ptp.rqt, res->drop_rqn); @@ -621,33 +549,17 @@ void mlx5e_rx_res_channels_deactivate(struct mlx5e_rx_res *res) } } -int mlx5e_rx_res_xsk_activate(struct mlx5e_rx_res *res, struct mlx5e_channels *chs, - unsigned int ix) +void mlx5e_rx_res_xsk_update(struct mlx5e_rx_res *res, struct mlx5e_channels *chs, + unsigned int ix, bool xsk) { - u32 rqn; - int err; - - if (!mlx5e_channels_get_xsk_rqn(chs, ix, &rqn)) - return -EINVAL; - - err = mlx5e_rqt_redirect_direct(&res->channels[ix].xsk_rqt, rqn); - if (err) - mlx5_core_warn(res->mdev, "Failed to redirect XSK RQT %#x to XSK RQ %#x (channel %u): err = %d\n", - mlx5e_rqt_get_rqtn(&res->channels[ix].xsk_rqt), - rqn, ix, err); - return err; -} + if (xsk) + mlx5e_channels_get_xsk_rqn(chs, ix, &res->rss_rqns[ix]); + else + mlx5e_channels_get_regular_rqn(chs, ix, &res->rss_rqns[ix]); -int mlx5e_rx_res_xsk_deactivate(struct mlx5e_rx_res *res, unsigned int ix) -{ - int err; + mlx5e_rx_res_rss_enable(res); - err = mlx5e_rqt_redirect_direct(&res->channels[ix].xsk_rqt, res->drop_rqn); - if (err) - mlx5_core_warn(res->mdev, "Failed to redirect XSK RQT %#x to drop RQ %#x (channel %u): err = %d\n", - mlx5e_rqt_get_rqtn(&res->channels[ix].xsk_rqt), - res->drop_rqn, ix, err); - return err; + mlx5e_rx_res_channel_activate_direct(res, chs, ix); } int mlx5e_rx_res_packet_merge_set_param(struct mlx5e_rx_res *res, diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/rx_res.h b/drivers/net/ethernet/mellanox/mlx5/core/en/rx_res.h index b39b20a720e0..5d5f64fab60f 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/rx_res.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/rx_res.h @@ -17,8 +17,7 @@ struct mlx5e_rss_params_hash; enum mlx5e_rx_res_features { MLX5E_RX_RES_FEATURE_INNER_FT = BIT(0), - MLX5E_RX_RES_FEATURE_XSK = BIT(1), - MLX5E_RX_RES_FEATURE_PTP = BIT(2), + MLX5E_RX_RES_FEATURE_PTP = BIT(1), }; /* Setup */ @@ -32,7 +31,6 @@ void mlx5e_rx_res_free(struct mlx5e_rx_res *res); /* TIRN getters for flow steering */ u32 mlx5e_rx_res_get_tirn_direct(struct mlx5e_rx_res *res, unsigned int ix); -u32 mlx5e_rx_res_get_tirn_xsk(struct mlx5e_rx_res *res, unsigned int ix); u32 mlx5e_rx_res_get_tirn_rss(struct mlx5e_rx_res *res, enum mlx5_traffic_types tt); u32 mlx5e_rx_res_get_tirn_rss_inner(struct mlx5e_rx_res *res, enum mlx5_traffic_types tt); u32 mlx5e_rx_res_get_tirn_ptp(struct mlx5e_rx_res *res); @@ -40,9 +38,8 @@ u32 mlx5e_rx_res_get_tirn_ptp(struct mlx5e_rx_res *res); /* Activate/deactivate API */ void mlx5e_rx_res_channels_activate(struct mlx5e_rx_res *res, struct mlx5e_channels *chs); void mlx5e_rx_res_channels_deactivate(struct mlx5e_rx_res *res); -int mlx5e_rx_res_xsk_activate(struct mlx5e_rx_res *res, struct mlx5e_channels *chs, - unsigned int ix); -int mlx5e_rx_res_xsk_deactivate(struct mlx5e_rx_res *res, unsigned int ix); +void mlx5e_rx_res_xsk_update(struct mlx5e_rx_res *res, struct mlx5e_channels *chs, + unsigned int ix, bool xsk); /* Configuration API */ void mlx5e_rx_res_rss_set_indir_uniform(struct mlx5e_rx_res *res, unsigned int nch); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/tc/act/goto.c b/drivers/net/ethernet/mellanox/mlx5/core/en/tc/act/goto.c index 69949ab830b6..25174f68613e 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/tc/act/goto.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/tc/act/goto.c @@ -12,6 +12,7 @@ validate_goto_chain(struct mlx5e_priv *priv, const struct flow_action_entry *act, struct netlink_ext_ack *extack) { + struct mlx5e_tc_table *tc = mlx5e_fs_get_tc(priv->fs); bool is_esw = mlx5e_is_eswitch_flow(flow); bool ft_flow = mlx5e_is_ft_flow(flow); u32 dest_chain = act->chain_index; @@ -21,7 +22,7 @@ validate_goto_chain(struct mlx5e_priv *priv, u32 max_chain; esw = priv->mdev->priv.eswitch; - chains = is_esw ? esw_chains(esw) : mlx5e_nic_chains(priv->fs->tc); + chains = is_esw ? esw_chains(esw) : mlx5e_nic_chains(tc); max_chain = mlx5_chains_get_chain_range(chains); reformat_and_fwd = is_esw ? MLX5_CAP_ESW_FLOWTABLE_FDB(priv->mdev, reformat_and_fwd_to_table) : diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/trap.c b/drivers/net/ethernet/mellanox/mlx5/core/en/trap.c index 11f2a7fb72a9..201ac7dd338f 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/trap.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/trap.c @@ -147,7 +147,7 @@ static struct mlx5e_trap *mlx5e_open_trap(struct mlx5e_priv *priv) t->mkey_be = cpu_to_be32(priv->mdev->mlx5e_res.hw_objs.mkey); t->stats = &priv->trap_stats.ch; - netif_napi_add(netdev, &t->napi, mlx5e_trap_napi_poll, 64); + netif_napi_add(netdev, &t->napi, mlx5e_trap_napi_poll); err = mlx5e_open_trap_rq(priv, t); if (unlikely(err)) @@ -230,12 +230,12 @@ static int mlx5e_handle_action_trap(struct mlx5e_priv *priv, int trap_id) switch (trap_id) { case DEVLINK_TRAP_GENERIC_ID_INGRESS_VLAN_FILTER: - err = mlx5e_add_vlan_trap(priv, trap_id, mlx5e_trap_get_tirn(priv->en_trap)); + err = mlx5e_add_vlan_trap(priv->fs, trap_id, mlx5e_trap_get_tirn(priv->en_trap)); if (err) goto err_out; break; case DEVLINK_TRAP_GENERIC_ID_DMAC_FILTER: - err = mlx5e_add_mac_trap(priv, trap_id, mlx5e_trap_get_tirn(priv->en_trap)); + err = mlx5e_add_mac_trap(priv->fs, trap_id, mlx5e_trap_get_tirn(priv->en_trap)); if (err) goto err_out; break; @@ -256,10 +256,10 @@ static int mlx5e_handle_action_drop(struct mlx5e_priv *priv, int trap_id) { switch (trap_id) { case DEVLINK_TRAP_GENERIC_ID_INGRESS_VLAN_FILTER: - mlx5e_remove_vlan_trap(priv); + mlx5e_remove_vlan_trap(priv->fs); break; case DEVLINK_TRAP_GENERIC_ID_DMAC_FILTER: - mlx5e_remove_mac_trap(priv); + mlx5e_remove_mac_trap(priv->fs); break; default: netdev_warn(priv->netdev, "%s: Unknown trap id %d\n", __func__, trap_id); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/txrx.h b/drivers/net/ethernet/mellanox/mlx5/core/en/txrx.h index c208ea307bff..4456ad5cedf1 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/txrx.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/txrx.h @@ -439,16 +439,24 @@ static inline u16 mlx5e_stop_room_for_max_wqe(struct mlx5_core_dev *mdev) return MLX5E_STOP_ROOM(mlx5e_get_max_sq_wqebbs(mdev)); } -static inline bool mlx5e_icosq_can_post_wqe(struct mlx5e_icosq *sq, u16 wqe_size) +static inline u16 mlx5e_stop_room_for_mpwqe(struct mlx5_core_dev *mdev) { - u16 room = sq->reserved_room; + u8 mpwqe_wqebbs = mlx5e_get_max_sq_aligned_wqebbs(mdev); - WARN_ONCE(wqe_size > sq->max_sq_wqebbs, - "wqe_size %u is greater than max SQ WQEBBs %u", - wqe_size, sq->max_sq_wqebbs); + return mlx5e_stop_room_for_wqe(mdev, mpwqe_wqebbs); +} - room += MLX5E_STOP_ROOM(wqe_size); +static inline bool mlx5e_icosq_can_post_wqe(struct mlx5e_icosq *sq, u16 wqe_size) +{ + u16 room = sq->reserved_room + MLX5E_STOP_ROOM(wqe_size); return mlx5e_wqc_has_room_for(&sq->wq, sq->cc, sq->pc, room); } + +static inline struct mlx5e_mpw_info *mlx5e_get_mpw_info(struct mlx5e_rq *rq, int i) +{ + size_t isz = struct_size(rq->mpwqe.info, alloc_units, rq->mpwqe.pages_per_wqe); + + return (struct mlx5e_mpw_info *)((char *)rq->mpwqe.info + array_size(i, isz)); +} #endif diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.c b/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.c index 8f321a6c0809..4685c652c97e 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.c @@ -333,7 +333,7 @@ mlx5e_xmit_xdp_frame_mpwqe(struct mlx5e_xdpsq *sq, struct mlx5e_xmit_data *xdptx mlx5e_xdp_mpwqe_add_dseg(sq, xdptxd, stats); - if (unlikely(mlx5e_xdp_mpqwe_is_full(session, sq->max_sq_mpw_wqebbs))) + if (unlikely(mlx5e_xdp_mpwqe_is_full(session, sq->max_sq_mpw_wqebbs))) mlx5e_xdp_mpwqe_complete(sq); stats->xmit++; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.h b/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.h index 287e17911251..bc2d9034af5b 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.h @@ -122,7 +122,7 @@ static inline bool mlx5e_xdp_get_inline_state(struct mlx5e_xdpsq *sq, bool cur) return cur; } -static inline bool mlx5e_xdp_mpqwe_is_full(struct mlx5e_tx_mpwqe *session, u8 max_sq_mpw_wqebbs) +static inline bool mlx5e_xdp_mpwqe_is_full(struct mlx5e_tx_mpwqe *session, u8 max_sq_mpw_wqebbs) { if (session->inline_on) return session->ds_count + MLX5E_XDP_INLINE_WQE_MAX_DS_CNT > diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/pool.c b/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/pool.c index 2c520394aa1d..ebada0c5af3c 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/pool.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/pool.c @@ -72,6 +72,7 @@ void mlx5e_build_xsk_param(struct xsk_buff_pool *pool, struct mlx5e_xsk_param *x { xsk->headroom = xsk_pool_get_headroom(pool); xsk->chunk_size = xsk_pool_get_chunk_size(pool); + xsk->unaligned = pool->unaligned; } static int mlx5e_xsk_enable_locked(struct mlx5e_priv *priv, @@ -98,6 +99,15 @@ static int mlx5e_xsk_enable_locked(struct mlx5e_priv *priv, mlx5e_build_xsk_param(pool, &xsk); + if (priv->channels.params.rq_wq_type == MLX5_WQ_TYPE_LINKED_LIST_STRIDING_RQ && + mlx5e_mpwrq_umr_mode(priv->mdev, &xsk) == MLX5E_MPWRQ_UMR_MODE_OVERSIZED) { + const char *recommendation = is_power_of_2(xsk.chunk_size) ? + "Upgrade firmware" : "Disable striding RQ"; + + mlx5_core_warn(priv->mdev, "Expected slowdown with XSK frame size %u. %s for better performance.\n", + xsk.chunk_size, recommendation); + } + if (!test_bit(MLX5E_STATE_OPENED, &priv->state)) { /* XSK objects will be created on open. */ goto validate_closed; @@ -123,15 +133,12 @@ static int mlx5e_xsk_enable_locked(struct mlx5e_priv *priv, * any Fill Ring entries at the setup stage. */ - err = mlx5e_rx_res_xsk_activate(priv->rx_res, &priv->channels, ix); - if (unlikely(err)) - goto err_deactivate; + mlx5e_rx_res_xsk_update(priv->rx_res, &priv->channels, ix, true); - return 0; + mlx5e_deactivate_rq(&c->rq); + mlx5e_flush_rq(&c->rq, MLX5_RQC_STATE_RDY); -err_deactivate: - mlx5e_deactivate_xsk(c); - mlx5e_close_xsk(c); + return 0; err_remove_pool: mlx5e_xsk_remove_pool(&priv->xsk, ix); @@ -170,7 +177,13 @@ static int mlx5e_xsk_disable_locked(struct mlx5e_priv *priv, u16 ix) goto remove_pool; c = priv->channels.c[ix]; - mlx5e_rx_res_xsk_deactivate(priv->rx_res, ix); + + mlx5e_activate_rq(&c->rq); + mlx5e_trigger_napi_icosq(c); + mlx5e_wait_for_min_rx_wqes(&c->rq, MLX5E_RQ_WQES_TIMEOUT); + + mlx5e_rx_res_xsk_update(priv->rx_res, &priv->channels, ix, false); + mlx5e_deactivate_xsk(c); mlx5e_close_xsk(c); @@ -208,11 +221,10 @@ int mlx5e_xsk_setup_pool(struct net_device *dev, struct xsk_buff_pool *pool, u16 { struct mlx5e_priv *priv = netdev_priv(dev); struct mlx5e_params *params = &priv->channels.params; - u16 ix; - if (unlikely(!mlx5e_qid_get_ch_if_in_group(params, qid, MLX5E_RQ_GROUP_XSK, &ix))) + if (unlikely(qid >= params->num_channels)) return -EINVAL; - return pool ? mlx5e_xsk_enable_pool(priv, pool, ix) : - mlx5e_xsk_disable_pool(priv, ix); + return pool ? mlx5e_xsk_enable_pool(priv, pool, qid) : + mlx5e_xsk_disable_pool(priv, qid); } diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/rx.c b/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/rx.c index 9a1553598a7c..c91b54d9ff27 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/rx.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/rx.c @@ -8,18 +8,221 @@ /* RX data path */ -static struct sk_buff *mlx5e_xsk_construct_skb(struct mlx5e_rq *rq, void *data, - u32 cqe_bcnt) +int mlx5e_xsk_alloc_rx_mpwqe(struct mlx5e_rq *rq, u16 ix) { + struct mlx5e_mpw_info *wi = mlx5e_get_mpw_info(rq, ix); + struct mlx5e_icosq *icosq = rq->icosq; + struct mlx5_wq_cyc *wq = &icosq->wq; + struct mlx5e_umr_wqe *umr_wqe; + int batch, i; + u32 offset; /* 17-bit value with MTT. */ + u16 pi; + + if (unlikely(!xsk_buff_can_alloc(rq->xsk_pool, rq->mpwqe.pages_per_wqe))) + goto err; + + BUILD_BUG_ON(sizeof(wi->alloc_units[0]) != sizeof(wi->alloc_units[0].xsk)); + batch = xsk_buff_alloc_batch(rq->xsk_pool, (struct xdp_buff **)wi->alloc_units, + rq->mpwqe.pages_per_wqe); + + /* If batch < pages_per_wqe, either: + * 1. Some (or all) descriptors were invalid. + * 2. dma_need_sync is true, and it fell back to allocating one frame. + * In either case, try to continue allocating frames one by one, until + * the first error, which will mean there are no more valid descriptors. + */ + for (; batch < rq->mpwqe.pages_per_wqe; batch++) { + wi->alloc_units[batch].xsk = xsk_buff_alloc(rq->xsk_pool); + if (unlikely(!wi->alloc_units[batch].xsk)) + goto err_reuse_batch; + } + + pi = mlx5e_icosq_get_next_pi(icosq, rq->mpwqe.umr_wqebbs); + umr_wqe = mlx5_wq_cyc_get_wqe(wq, pi); + memcpy(umr_wqe, &rq->mpwqe.umr_wqe, sizeof(struct mlx5e_umr_wqe)); + + if (likely(rq->mpwqe.umr_mode == MLX5E_MPWRQ_UMR_MODE_ALIGNED)) { + for (i = 0; i < batch; i++) { + dma_addr_t addr = xsk_buff_xdp_get_frame_dma(wi->alloc_units[i].xsk); + + umr_wqe->inline_mtts[i] = (struct mlx5_mtt) { + .ptag = cpu_to_be64(addr | MLX5_EN_WR), + }; + } + } else if (unlikely(rq->mpwqe.umr_mode == MLX5E_MPWRQ_UMR_MODE_UNALIGNED)) { + for (i = 0; i < batch; i++) { + dma_addr_t addr = xsk_buff_xdp_get_frame_dma(wi->alloc_units[i].xsk); + + umr_wqe->inline_ksms[i] = (struct mlx5_ksm) { + .key = rq->mkey_be, + .va = cpu_to_be64(addr), + }; + } + } else if (likely(rq->mpwqe.umr_mode == MLX5E_MPWRQ_UMR_MODE_TRIPLE)) { + u32 mapping_size = 1 << (rq->mpwqe.page_shift - 2); + + for (i = 0; i < batch; i++) { + dma_addr_t addr = xsk_buff_xdp_get_frame_dma(wi->alloc_units[i].xsk); + + umr_wqe->inline_ksms[i << 2] = (struct mlx5_ksm) { + .key = rq->mkey_be, + .va = cpu_to_be64(addr), + }; + umr_wqe->inline_ksms[(i << 2) + 1] = (struct mlx5_ksm) { + .key = rq->mkey_be, + .va = cpu_to_be64(addr + mapping_size), + }; + umr_wqe->inline_ksms[(i << 2) + 2] = (struct mlx5_ksm) { + .key = rq->mkey_be, + .va = cpu_to_be64(addr + mapping_size * 2), + }; + umr_wqe->inline_ksms[(i << 2) + 3] = (struct mlx5_ksm) { + .key = rq->mkey_be, + .va = cpu_to_be64(rq->wqe_overflow.addr), + }; + } + } else { + __be32 pad_size = cpu_to_be32((1 << rq->mpwqe.page_shift) - + rq->xsk_pool->chunk_size); + __be32 frame_size = cpu_to_be32(rq->xsk_pool->chunk_size); + + for (i = 0; i < batch; i++) { + dma_addr_t addr = xsk_buff_xdp_get_frame_dma(wi->alloc_units[i].xsk); + + umr_wqe->inline_klms[i << 1] = (struct mlx5_klm) { + .key = rq->mkey_be, + .va = cpu_to_be64(addr), + .bcount = frame_size, + }; + umr_wqe->inline_klms[(i << 1) + 1] = (struct mlx5_klm) { + .key = rq->mkey_be, + .va = cpu_to_be64(rq->wqe_overflow.addr), + .bcount = pad_size, + }; + } + } + + bitmap_zero(wi->xdp_xmit_bitmap, rq->mpwqe.pages_per_wqe); + wi->consumed_strides = 0; + + umr_wqe->ctrl.opmod_idx_opcode = + cpu_to_be32((icosq->pc << MLX5_WQE_CTRL_WQE_INDEX_SHIFT) | MLX5_OPCODE_UMR); + + /* Optimized for speed: keep in sync with mlx5e_mpwrq_umr_entry_size. */ + offset = ix * rq->mpwqe.mtts_per_wqe; + if (likely(rq->mpwqe.umr_mode == MLX5E_MPWRQ_UMR_MODE_ALIGNED)) + offset = offset * sizeof(struct mlx5_mtt) / MLX5_OCTWORD; + else if (unlikely(rq->mpwqe.umr_mode == MLX5E_MPWRQ_UMR_MODE_OVERSIZED)) + offset = offset * sizeof(struct mlx5_klm) * 2 / MLX5_OCTWORD; + else if (unlikely(rq->mpwqe.umr_mode == MLX5E_MPWRQ_UMR_MODE_TRIPLE)) + offset = offset * sizeof(struct mlx5_ksm) * 4 / MLX5_OCTWORD; + umr_wqe->uctrl.xlt_offset = cpu_to_be16(offset); + + icosq->db.wqe_info[pi] = (struct mlx5e_icosq_wqe_info) { + .wqe_type = MLX5E_ICOSQ_WQE_UMR_RX, + .num_wqebbs = rq->mpwqe.umr_wqebbs, + .umr.rq = rq, + }; + + icosq->pc += rq->mpwqe.umr_wqebbs; + + icosq->doorbell_cseg = &umr_wqe->ctrl; + + return 0; + +err_reuse_batch: + while (--batch >= 0) + xsk_buff_free(wi->alloc_units[batch].xsk); + +err: + rq->stats->buff_alloc_err++; + return -ENOMEM; +} + +int mlx5e_xsk_alloc_rx_wqes_batched(struct mlx5e_rq *rq, u16 ix, int wqe_bulk) +{ + struct mlx5_wq_cyc *wq = &rq->wqe.wq; + struct xdp_buff **buffs; + u32 contig, alloc; + int i; + + /* mlx5e_init_frags_partition creates a 1:1 mapping between + * rq->wqe.frags and rq->wqe.alloc_units, which allows us to + * allocate XDP buffers straight into alloc_units. + */ + BUILD_BUG_ON(sizeof(rq->wqe.alloc_units[0]) != + sizeof(rq->wqe.alloc_units[0].xsk)); + buffs = (struct xdp_buff **)rq->wqe.alloc_units; + contig = mlx5_wq_cyc_get_size(wq) - ix; + if (wqe_bulk <= contig) { + alloc = xsk_buff_alloc_batch(rq->xsk_pool, buffs + ix, wqe_bulk); + } else { + alloc = xsk_buff_alloc_batch(rq->xsk_pool, buffs + ix, contig); + if (likely(alloc == contig)) + alloc += xsk_buff_alloc_batch(rq->xsk_pool, buffs, wqe_bulk - contig); + } + + for (i = 0; i < alloc; i++) { + int j = mlx5_wq_cyc_ctr2ix(wq, ix + i); + struct mlx5e_wqe_frag_info *frag; + struct mlx5e_rx_wqe_cyc *wqe; + dma_addr_t addr; + + wqe = mlx5_wq_cyc_get_wqe(wq, j); + /* Assumes log_num_frags == 0. */ + frag = &rq->wqe.frags[j]; + + addr = xsk_buff_xdp_get_frame_dma(frag->au->xsk); + wqe->data[0].addr = cpu_to_be64(addr + rq->buff.headroom); + } + + return alloc; +} + +int mlx5e_xsk_alloc_rx_wqes(struct mlx5e_rq *rq, u16 ix, int wqe_bulk) +{ + struct mlx5_wq_cyc *wq = &rq->wqe.wq; + int i; + + for (i = 0; i < wqe_bulk; i++) { + int j = mlx5_wq_cyc_ctr2ix(wq, ix + i); + struct mlx5e_wqe_frag_info *frag; + struct mlx5e_rx_wqe_cyc *wqe; + dma_addr_t addr; + + wqe = mlx5_wq_cyc_get_wqe(wq, j); + /* Assumes log_num_frags == 0. */ + frag = &rq->wqe.frags[j]; + + frag->au->xsk = xsk_buff_alloc(rq->xsk_pool); + if (unlikely(!frag->au->xsk)) + return i; + + addr = xsk_buff_xdp_get_frame_dma(frag->au->xsk); + wqe->data[0].addr = cpu_to_be64(addr + rq->buff.headroom); + } + + return wqe_bulk; +} + +static struct sk_buff *mlx5e_xsk_construct_skb(struct mlx5e_rq *rq, struct xdp_buff *xdp) +{ + u32 totallen = xdp->data_end - xdp->data_meta; + u32 metalen = xdp->data - xdp->data_meta; struct sk_buff *skb; - skb = napi_alloc_skb(rq->cq.napi, cqe_bcnt); + skb = napi_alloc_skb(rq->cq.napi, totallen); if (unlikely(!skb)) { rq->stats->buff_alloc_err++; return NULL; } - skb_put_data(skb, data, cqe_bcnt); + skb_put_data(skb, xdp->data_meta, totallen); + + if (metalen) { + skb_metadata_set(skb, metalen); + __skb_pull(skb, metalen); + } return skb; } @@ -30,7 +233,7 @@ struct sk_buff *mlx5e_xsk_skb_from_cqe_mpwrq_linear(struct mlx5e_rq *rq, u32 head_offset, u32 page_idx) { - struct xdp_buff *xdp = wi->umr.dma_info[page_idx].xsk; + struct xdp_buff *xdp = wi->alloc_units[page_idx].xsk; struct bpf_prog *prog; /* Check packet size. Note LRO doesn't use linear SKB */ @@ -46,8 +249,7 @@ struct sk_buff *mlx5e_xsk_skb_from_cqe_mpwrq_linear(struct mlx5e_rq *rq, */ WARN_ON_ONCE(head_offset); - xdp->data_end = xdp->data + cqe_bcnt; - xdp_set_data_meta_invalid(xdp); + xsk_buff_set_size(xdp, cqe_bcnt); xsk_buff_dma_sync_for_cpu(xdp, rq->xsk_pool); net_prefetch(xdp->data); @@ -76,14 +278,14 @@ struct sk_buff *mlx5e_xsk_skb_from_cqe_mpwrq_linear(struct mlx5e_rq *rq, /* XDP_PASS: copy the data from the UMEM to a new SKB and reuse the * frame. On SKB allocation failure, NULL is returned. */ - return mlx5e_xsk_construct_skb(rq, xdp->data, xdp->data_end - xdp->data); + return mlx5e_xsk_construct_skb(rq, xdp); } struct sk_buff *mlx5e_xsk_skb_from_cqe_linear(struct mlx5e_rq *rq, struct mlx5e_wqe_frag_info *wi, u32 cqe_bcnt) { - struct xdp_buff *xdp = wi->di->xsk; + struct xdp_buff *xdp = wi->au->xsk; struct bpf_prog *prog; /* wi->offset is not used in this function, because xdp->data and the @@ -93,8 +295,7 @@ struct sk_buff *mlx5e_xsk_skb_from_cqe_linear(struct mlx5e_rq *rq, */ WARN_ON_ONCE(wi->offset); - xdp->data_end = xdp->data + cqe_bcnt; - xdp_set_data_meta_invalid(xdp); + xsk_buff_set_size(xdp, cqe_bcnt); xsk_buff_dma_sync_for_cpu(xdp, rq->xsk_pool); net_prefetch(xdp->data); @@ -103,8 +304,8 @@ struct sk_buff *mlx5e_xsk_skb_from_cqe_linear(struct mlx5e_rq *rq, return NULL; /* page/packet was consumed by XDP */ /* XDP_PASS: copy the data from the UMEM to a new SKB. The frame reuse - * will be handled by mlx5e_put_rx_frag. + * will be handled by mlx5e_free_rx_wqe. * On SKB allocation failure, NULL is returned. */ - return mlx5e_xsk_construct_skb(rq, xdp->data, xdp->data_end - xdp->data); + return mlx5e_xsk_construct_skb(rq, xdp); } diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/rx.h b/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/rx.h index cc18d97d8ee0..087c943bd8e9 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/rx.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/rx.h @@ -5,12 +5,12 @@ #define __MLX5_EN_XSK_RX_H__ #include "en.h" -#include <net/xdp_sock_drv.h> - -#define MLX5E_MTT_PTAG_MASK 0xfffffffffffffff8ULL /* RX data path */ +int mlx5e_xsk_alloc_rx_mpwqe(struct mlx5e_rq *rq, u16 ix); +int mlx5e_xsk_alloc_rx_wqes_batched(struct mlx5e_rq *rq, u16 ix, int wqe_bulk); +int mlx5e_xsk_alloc_rx_wqes(struct mlx5e_rq *rq, u16 ix, int wqe_bulk); struct sk_buff *mlx5e_xsk_skb_from_cqe_mpwrq_linear(struct mlx5e_rq *rq, struct mlx5e_mpw_info *wi, u16 cqe_bcnt, @@ -20,46 +20,4 @@ struct sk_buff *mlx5e_xsk_skb_from_cqe_linear(struct mlx5e_rq *rq, struct mlx5e_wqe_frag_info *wi, u32 cqe_bcnt); -static inline int mlx5e_xsk_page_alloc_pool(struct mlx5e_rq *rq, - struct mlx5e_dma_info *dma_info) -{ -retry: - dma_info->xsk = xsk_buff_alloc(rq->xsk_pool); - if (!dma_info->xsk) - return -ENOMEM; - - /* Store the DMA address without headroom. In striding RQ case, we just - * provide pages for UMR, and headroom is counted at the setup stage - * when creating a WQE. In non-striding RQ case, headroom is accounted - * in mlx5e_alloc_rx_wqe. - */ - dma_info->addr = xsk_buff_xdp_get_frame_dma(dma_info->xsk); - - /* MTT page mapping has alignment requirements. If they are not - * satisfied, leak the descriptor so that it won't come again, and try - * to allocate a new one. - */ - if (rq->wq_type == MLX5_WQ_TYPE_LINKED_LIST_STRIDING_RQ) { - if (unlikely(dma_info->addr & ~MLX5E_MTT_PTAG_MASK)) { - xsk_buff_discard(dma_info->xsk); - goto retry; - } - } - - return 0; -} - -static inline bool mlx5e_xsk_update_rx_wakeup(struct mlx5e_rq *rq, bool alloc_err) -{ - if (!xsk_uses_need_wakeup(rq->xsk_pool)) - return alloc_err; - - if (unlikely(alloc_err)) - xsk_set_rx_need_wakeup(rq->xsk_pool); - else - xsk_clear_rx_need_wakeup(rq->xsk_pool); - - return false; -} - #endif /* __MLX5_EN_XSK_RX_H__ */ diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/setup.c b/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/setup.c index 98ed9ef3a6bd..ff03c43833bb 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/setup.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/setup.c @@ -5,24 +5,19 @@ #include "en/params.h" #include "en/txrx.h" #include "en/health.h" +#include <net/xdp_sock_drv.h> -/* It matches XDP_UMEM_MIN_CHUNK_SIZE, but as this constant is private and may - * change unexpectedly, and mlx5e has a minimum valid stride size for striding - * RQ, keep this check in the driver. +/* The limitation of 2048 can be altered, but shouldn't go beyond the minimal + * stride size of striding RQ. */ -#define MLX5E_MIN_XSK_CHUNK_SIZE 2048 +#define MLX5E_MIN_XSK_CHUNK_SIZE max(2048, XDP_UMEM_MIN_CHUNK_SIZE) bool mlx5e_validate_xsk_param(struct mlx5e_params *params, struct mlx5e_xsk_param *xsk, struct mlx5_core_dev *mdev) { /* AF_XDP doesn't support frames larger than PAGE_SIZE. */ - if (xsk->chunk_size > PAGE_SIZE || - xsk->chunk_size < MLX5E_MIN_XSK_CHUNK_SIZE) - return false; - - /* Current MTU and XSK headroom don't allow packets to fit the frames. */ - if (mlx5e_rx_get_min_frag_sz(params, xsk) > xsk->chunk_size) + if (xsk->chunk_size > PAGE_SIZE || xsk->chunk_size < MLX5E_MIN_XSK_CHUNK_SIZE) return false; /* frag_sz is different for regular and XSK RQs, so ensure that linear @@ -30,9 +25,9 @@ bool mlx5e_validate_xsk_param(struct mlx5e_params *params, */ switch (params->rq_wq_type) { case MLX5_WQ_TYPE_LINKED_LIST_STRIDING_RQ: - return mlx5e_rx_mpwqe_is_linear_skb(mdev, params, xsk); + return !mlx5e_mpwrq_validate_xsk(mdev, params, xsk); default: /* MLX5_WQ_TYPE_CYCLIC */ - return mlx5e_rx_is_linear_skb(params, xsk); + return mlx5e_rx_is_linear_skb(mdev, params, xsk); } } @@ -71,7 +66,7 @@ static int mlx5e_init_xsk_rq(struct mlx5e_channel *c, rq->xsk_pool = pool; rq->stats = &c->priv->channel_stats[c->ix]->xskrq; rq->ptp_cyc2time = mlx5_rq_ts_translator(mdev); - rq_xdp_ix = c->ix + params->num_channels * MLX5E_RQ_GROUP_XSK; + rq_xdp_ix = c->ix; err = mlx5e_rq_set_handlers(rq, params, xsk); if (err) return err; @@ -159,7 +154,7 @@ err_free_cparam: void mlx5e_close_xsk(struct mlx5e_channel *c) { clear_bit(MLX5E_CHANNEL_STATE_XSK, c->state); - synchronize_net(); /* Sync with the XSK wakeup and with NAPI. */ + synchronize_net(); /* Sync with NAPI. */ mlx5e_close_rq(&c->xskrq); mlx5e_close_cq(&c->xskrq.cq); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/tx.c b/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/tx.c index 4902ef74fedf..367a9505ca4f 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/tx.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/tx.c @@ -12,18 +12,14 @@ int mlx5e_xsk_wakeup(struct net_device *dev, u32 qid, u32 flags) struct mlx5e_priv *priv = netdev_priv(dev); struct mlx5e_params *params = &priv->channels.params; struct mlx5e_channel *c; - u16 ix; if (unlikely(!mlx5e_xdp_is_active(priv))) return -ENETDOWN; - if (unlikely(!mlx5e_qid_get_ch_if_in_group(params, qid, MLX5E_RQ_GROUP_XSK, &ix))) + if (unlikely(qid >= params->num_channels)) return -EINVAL; - c = priv->channels.c[ix]; - - if (unlikely(!test_bit(MLX5E_CHANNEL_STATE_XSK, c->state))) - return -EINVAL; + c = priv->channels.c[qid]; if (!napi_if_scheduled_mark_missed(&c->napi)) { /* To avoid WQE overrun, don't post a NOP if async_icosq is not @@ -36,9 +32,7 @@ int mlx5e_xsk_wakeup(struct net_device *dev, u32 qid, u32 flags) if (test_and_set_bit(MLX5E_SQ_STATE_PENDING_XSK_TX, &c->async_icosq.state)) return 0; - spin_lock_bh(&c->async_icosq_lock); - mlx5e_trigger_irq(&c->async_icosq); - spin_unlock_bh(&c->async_icosq_lock); + mlx5e_trigger_napi_icosq(c); } return 0; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/tx.h b/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/tx.h index a05085035f23..9c505158b975 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/tx.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/tx.h @@ -5,7 +5,6 @@ #define __MLX5_EN_XSK_TX_H__ #include "en.h" -#include <net/xdp_sock_drv.h> /* TX data path */ @@ -13,15 +12,4 @@ int mlx5e_xsk_wakeup(struct net_device *dev, u32 qid, u32 flags); bool mlx5e_xsk_tx(struct mlx5e_xdpsq *sq, unsigned int budget); -static inline void mlx5e_xsk_update_tx_wakeup(struct mlx5e_xdpsq *sq) -{ - if (!xsk_uses_need_wakeup(sq->xsk_pool)) - return; - - if (sq->pc != sq->cc) - xsk_clear_tx_need_wakeup(sq->xsk_pool); - else - xsk_set_tx_need_wakeup(sq->xsk_pool); -} - #endif /* __MLX5_EN_XSK_TX_H__ */ diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/en_accel.h b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/en_accel.h index 1839f1ab1ddd..07187028f0d3 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/en_accel.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/en_accel.h @@ -39,6 +39,7 @@ #include "en_accel/ipsec_rxtx.h" #include "en_accel/ktls.h" #include "en_accel/ktls_txrx.h" +#include <en_accel/macsec.h> #include "en.h" #include "en/txrx.h" @@ -137,6 +138,15 @@ static inline bool mlx5e_accel_tx_begin(struct net_device *dev, } #endif +#ifdef CONFIG_MLX5_EN_MACSEC + if (unlikely(mlx5e_macsec_skb_is_offload(skb))) { + struct mlx5e_priv *priv = netdev_priv(dev); + + if (unlikely(!mlx5e_macsec_handle_tx_skb(priv->macsec, skb))) + return false; + } +#endif + return true; } @@ -163,6 +173,11 @@ static inline void mlx5e_accel_tx_eseg(struct mlx5e_priv *priv, mlx5e_ipsec_tx_build_eseg(priv, skb, eseg); #endif +#ifdef CONFIG_MLX5_EN_MACSEC + if (unlikely(mlx5e_macsec_skb_is_offload(skb))) + mlx5e_macsec_tx_build_eseg(priv->macsec, skb, eseg); +#endif + #if IS_ENABLED(CONFIG_GENEVE) if (skb->encapsulation && skb->ip_summed == CHECKSUM_PARTIAL) mlx5e_tx_tunnel_accel(skb, eseg, ihs); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/fs_tcp.c b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/fs_tcp.c index 20a4f1e585af..285d32d2fd08 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/fs_tcp.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/fs_tcp.c @@ -1,7 +1,7 @@ // SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB /* Copyright (c) 2020, Mellanox Technologies inc. All rights reserved. */ -#include <linux/netdevice.h> +#include <mlx5_core.h> #include "en_accel/fs_tcp.h" #include "fs_core.h" @@ -71,13 +71,13 @@ void mlx5e_accel_fs_del_sk(struct mlx5_flow_handle *rule) mlx5_del_flow_rules(rule); } -struct mlx5_flow_handle *mlx5e_accel_fs_add_sk(struct mlx5e_priv *priv, +struct mlx5_flow_handle *mlx5e_accel_fs_add_sk(struct mlx5e_flow_steering *fs, struct sock *sk, u32 tirn, uint32_t flow_tag) { + struct mlx5e_accel_fs_tcp *fs_tcp = mlx5e_fs_get_accel_tcp(fs); struct mlx5_flow_destination dest = {}; struct mlx5e_flow_table *ft = NULL; - struct mlx5e_accel_fs_tcp *fs_tcp; MLX5_DECLARE_FLOW_ACT(flow_act); struct mlx5_flow_handle *flow; struct mlx5_flow_spec *spec; @@ -86,19 +86,17 @@ struct mlx5_flow_handle *mlx5e_accel_fs_add_sk(struct mlx5e_priv *priv, if (!spec) return ERR_PTR(-ENOMEM); - fs_tcp = priv->fs->accel_tcp; - spec->match_criteria_enable = MLX5_MATCH_OUTER_HEADERS; switch (sk->sk_family) { case AF_INET: accel_fs_tcp_set_ipv4_flow(spec, sk); ft = &fs_tcp->tables[ACCEL_FS_IPV4_TCP]; - mlx5e_dbg(HW, priv, "%s flow is %pI4:%d -> %pI4:%d\n", __func__, - &inet_sk(sk)->inet_rcv_saddr, - inet_sk(sk)->inet_sport, - &inet_sk(sk)->inet_daddr, - inet_sk(sk)->inet_dport); + fs_dbg(fs, "%s flow is %pI4:%d -> %pI4:%d\n", __func__, + &inet_sk(sk)->inet_rcv_saddr, + inet_sk(sk)->inet_sport, + &inet_sk(sk)->inet_daddr, + inet_sk(sk)->inet_dport); break; #if IS_ENABLED(CONFIG_IPV6) case AF_INET6: @@ -140,34 +138,32 @@ struct mlx5_flow_handle *mlx5e_accel_fs_add_sk(struct mlx5e_priv *priv, flow = mlx5_add_flow_rules(ft->t, spec, &flow_act, &dest, 1); if (IS_ERR(flow)) - netdev_err(priv->netdev, "mlx5_add_flow_rules() failed, flow is %ld\n", - PTR_ERR(flow)); + fs_err(fs, "mlx5_add_flow_rules() failed, flow is %ld\n", PTR_ERR(flow)); out: kvfree(spec); return flow; } -static int accel_fs_tcp_add_default_rule(struct mlx5e_priv *priv, +static int accel_fs_tcp_add_default_rule(struct mlx5e_flow_steering *fs, enum accel_fs_tcp_type type) { + struct mlx5e_accel_fs_tcp *fs_tcp = mlx5e_fs_get_accel_tcp(fs); + struct mlx5_ttc_table *ttc = mlx5e_fs_get_ttc(fs, false); struct mlx5e_flow_table *accel_fs_t; struct mlx5_flow_destination dest; - struct mlx5e_accel_fs_tcp *fs_tcp; MLX5_DECLARE_FLOW_ACT(flow_act); struct mlx5_flow_handle *rule; int err = 0; - fs_tcp = priv->fs->accel_tcp; accel_fs_t = &fs_tcp->tables[type]; - dest = mlx5_ttc_get_default_dest(priv->fs->ttc, fs_accel2tt(type)); + dest = mlx5_ttc_get_default_dest(ttc, fs_accel2tt(type)); rule = mlx5_add_flow_rules(accel_fs_t->t, NULL, &flow_act, &dest, 1); if (IS_ERR(rule)) { err = PTR_ERR(rule); - netdev_err(priv->netdev, - "%s: add default rule failed, accel_fs type=%d, err %d\n", - __func__, type, err); + fs_err(fs, "%s: add default rule failed, accel_fs type=%d, err %d\n", + __func__, type, err); return err; } @@ -265,9 +261,11 @@ out: return err; } -static int accel_fs_tcp_create_table(struct mlx5e_priv *priv, enum accel_fs_tcp_type type) +static int accel_fs_tcp_create_table(struct mlx5e_flow_steering *fs, enum accel_fs_tcp_type type) { - struct mlx5e_flow_table *ft = &priv->fs->accel_tcp->tables[type]; + struct mlx5e_accel_fs_tcp *accel_tcp = mlx5e_fs_get_accel_tcp(fs); + struct mlx5_flow_namespace *ns = mlx5e_fs_get_ns(fs, false); + struct mlx5e_flow_table *ft = &accel_tcp->tables[type]; struct mlx5_flow_table_attr ft_attr = {}; int err; @@ -277,21 +275,21 @@ static int accel_fs_tcp_create_table(struct mlx5e_priv *priv, enum accel_fs_tcp_ ft_attr.level = MLX5E_ACCEL_FS_TCP_FT_LEVEL; ft_attr.prio = MLX5E_NIC_PRIO; - ft->t = mlx5_create_flow_table(priv->fs->ns, &ft_attr); + ft->t = mlx5_create_flow_table(ns, &ft_attr); if (IS_ERR(ft->t)) { err = PTR_ERR(ft->t); ft->t = NULL; return err; } - netdev_dbg(priv->netdev, "Created fs accel table id %u level %u\n", - ft->t->id, ft->t->level); + fs_dbg(fs, "Created fs accel table id %u level %u\n", + ft->t->id, ft->t->level); err = accel_fs_tcp_create_groups(ft, type); if (err) goto err; - err = accel_fs_tcp_add_default_rule(priv, type); + err = accel_fs_tcp_add_default_rule(fs, type); if (err) goto err; @@ -301,17 +299,18 @@ err: return err; } -static int accel_fs_tcp_disable(struct mlx5e_priv *priv) +static int accel_fs_tcp_disable(struct mlx5e_flow_steering *fs) { + struct mlx5_ttc_table *ttc = mlx5e_fs_get_ttc(fs, false); int err, i; for (i = 0; i < ACCEL_FS_TCP_NUM_TYPES; i++) { /* Modify ttc rules destination to point back to the indir TIRs */ - err = mlx5_ttc_fwd_default_dest(priv->fs->ttc, fs_accel2tt(i)); + err = mlx5_ttc_fwd_default_dest(ttc, fs_accel2tt(i)); if (err) { - netdev_err(priv->netdev, - "%s: modify ttc[%d] default destination failed, err(%d)\n", - __func__, fs_accel2tt(i), err); + fs_err(fs, + "%s: modify ttc[%d] default destination failed, err(%d)\n", + __func__, fs_accel2tt(i), err); return err; } } @@ -319,32 +318,32 @@ static int accel_fs_tcp_disable(struct mlx5e_priv *priv) return 0; } -static int accel_fs_tcp_enable(struct mlx5e_priv *priv) +static int accel_fs_tcp_enable(struct mlx5e_flow_steering *fs) { + struct mlx5e_accel_fs_tcp *accel_tcp = mlx5e_fs_get_accel_tcp(fs); + struct mlx5_ttc_table *ttc = mlx5e_fs_get_ttc(fs, false); struct mlx5_flow_destination dest = {}; int err, i; dest.type = MLX5_FLOW_DESTINATION_TYPE_FLOW_TABLE; for (i = 0; i < ACCEL_FS_TCP_NUM_TYPES; i++) { - dest.ft = priv->fs->accel_tcp->tables[i].t; + dest.ft = accel_tcp->tables[i].t; /* Modify ttc rules destination to point on the accel_fs FTs */ - err = mlx5_ttc_fwd_dest(priv->fs->ttc, fs_accel2tt(i), &dest); + err = mlx5_ttc_fwd_dest(ttc, fs_accel2tt(i), &dest); if (err) { - netdev_err(priv->netdev, - "%s: modify ttc[%d] destination to accel failed, err(%d)\n", - __func__, fs_accel2tt(i), err); + fs_err(fs, "%s: modify ttc[%d] destination to accel failed, err(%d)\n", + __func__, fs_accel2tt(i), err); return err; } } return 0; } -static void accel_fs_tcp_destroy_table(struct mlx5e_priv *priv, int i) +static void accel_fs_tcp_destroy_table(struct mlx5e_flow_steering *fs, int i) { - struct mlx5e_accel_fs_tcp *fs_tcp; + struct mlx5e_accel_fs_tcp *fs_tcp = mlx5e_fs_get_accel_tcp(fs); - fs_tcp = priv->fs->accel_tcp; if (IS_ERR_OR_NULL(fs_tcp->tables[i].t)) return; @@ -353,40 +352,43 @@ static void accel_fs_tcp_destroy_table(struct mlx5e_priv *priv, int i) fs_tcp->tables[i].t = NULL; } -void mlx5e_accel_fs_tcp_destroy(struct mlx5e_priv *priv) +void mlx5e_accel_fs_tcp_destroy(struct mlx5e_flow_steering *fs) { + struct mlx5e_accel_fs_tcp *accel_tcp = mlx5e_fs_get_accel_tcp(fs); int i; - if (!priv->fs->accel_tcp) + if (!accel_tcp) return; - accel_fs_tcp_disable(priv); + accel_fs_tcp_disable(fs); for (i = 0; i < ACCEL_FS_TCP_NUM_TYPES; i++) - accel_fs_tcp_destroy_table(priv, i); + accel_fs_tcp_destroy_table(fs, i); - kfree(priv->fs->accel_tcp); - priv->fs->accel_tcp = NULL; + kfree(accel_tcp); + mlx5e_fs_set_accel_tcp(fs, NULL); } -int mlx5e_accel_fs_tcp_create(struct mlx5e_priv *priv) +int mlx5e_accel_fs_tcp_create(struct mlx5e_flow_steering *fs) { + struct mlx5e_accel_fs_tcp *accel_tcp; int i, err; - if (!MLX5_CAP_FLOWTABLE_NIC_RX(priv->mdev, ft_field_support.outer_ip_version)) + if (!MLX5_CAP_FLOWTABLE_NIC_RX(mlx5e_fs_get_mdev(fs), ft_field_support.outer_ip_version)) return -EOPNOTSUPP; - priv->fs->accel_tcp = kzalloc(sizeof(*priv->fs->accel_tcp), GFP_KERNEL); - if (!priv->fs->accel_tcp) + accel_tcp = kvzalloc(sizeof(*accel_tcp), GFP_KERNEL); + if (!accel_tcp) return -ENOMEM; + mlx5e_fs_set_accel_tcp(fs, accel_tcp); for (i = 0; i < ACCEL_FS_TCP_NUM_TYPES; i++) { - err = accel_fs_tcp_create_table(priv, i); + err = accel_fs_tcp_create_table(fs, i); if (err) goto err_destroy_tables; } - err = accel_fs_tcp_enable(priv); + err = accel_fs_tcp_enable(fs); if (err) goto err_destroy_tables; @@ -394,9 +396,8 @@ int mlx5e_accel_fs_tcp_create(struct mlx5e_priv *priv) err_destroy_tables: while (--i >= 0) - accel_fs_tcp_destroy_table(priv, i); - - kfree(priv->fs->accel_tcp); - priv->fs->accel_tcp = NULL; + accel_fs_tcp_destroy_table(fs, i); + kfree(accel_tcp); + mlx5e_fs_set_accel_tcp(fs, NULL); return err; } diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/fs_tcp.h b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/fs_tcp.h index 589235824543..a032bff482a6 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/fs_tcp.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/fs_tcp.h @@ -4,19 +4,19 @@ #ifndef __MLX5E_ACCEL_FS_TCP_H__ #define __MLX5E_ACCEL_FS_TCP_H__ -#include "en.h" +#include "en/fs.h" #ifdef CONFIG_MLX5_EN_TLS -int mlx5e_accel_fs_tcp_create(struct mlx5e_priv *priv); -void mlx5e_accel_fs_tcp_destroy(struct mlx5e_priv *priv); -struct mlx5_flow_handle *mlx5e_accel_fs_add_sk(struct mlx5e_priv *priv, +int mlx5e_accel_fs_tcp_create(struct mlx5e_flow_steering *fs); +void mlx5e_accel_fs_tcp_destroy(struct mlx5e_flow_steering *fs); +struct mlx5_flow_handle *mlx5e_accel_fs_add_sk(struct mlx5e_flow_steering *fs, struct sock *sk, u32 tirn, uint32_t flow_tag); void mlx5e_accel_fs_del_sk(struct mlx5_flow_handle *rule); #else -static inline int mlx5e_accel_fs_tcp_create(struct mlx5e_priv *priv) { return 0; } -static inline void mlx5e_accel_fs_tcp_destroy(struct mlx5e_priv *priv) {} -static inline struct mlx5_flow_handle *mlx5e_accel_fs_add_sk(struct mlx5e_priv *priv, +static inline int mlx5e_accel_fs_tcp_create(struct mlx5e_flow_steering *fs) { return 0; } +static inline void mlx5e_accel_fs_tcp_destroy(struct mlx5e_flow_steering *fs) {} +static inline struct mlx5_flow_handle *mlx5e_accel_fs_add_sk(struct mlx5e_flow_steering *fs, struct sock *sk, u32 tirn, uint32_t flow_tag) { return ERR_PTR(-EOPNOTSUPP); } diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec_fs.c b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec_fs.c index f8113fd23265..b859e4a4c744 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec_fs.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec_fs.c @@ -174,6 +174,8 @@ static void rx_destroy(struct mlx5e_priv *priv, enum accel_fs_esp_type type) static int rx_create(struct mlx5e_priv *priv, enum accel_fs_esp_type type) { + struct mlx5_flow_namespace *ns = mlx5e_fs_get_ns(priv->fs, false); + struct mlx5_ttc_table *ttc = mlx5e_fs_get_ttc(priv->fs, false); struct mlx5_flow_table_attr ft_attr = {}; struct mlx5e_accel_fs_esp_prot *fs_prot; struct mlx5e_accel_fs_esp *accel_esp; @@ -182,15 +184,14 @@ static int rx_create(struct mlx5e_priv *priv, enum accel_fs_esp_type type) accel_esp = priv->ipsec->rx_fs; fs_prot = &accel_esp->fs_prot[type]; - fs_prot->default_dest = - mlx5_ttc_get_default_dest(priv->fs->ttc, fs_esp2tt(type)); + mlx5_ttc_get_default_dest(ttc, fs_esp2tt(type)); ft_attr.max_fte = 1; ft_attr.autogroup.max_num_groups = 1; ft_attr.level = MLX5E_ACCEL_FS_ESP_FT_ERR_LEVEL; ft_attr.prio = MLX5E_NIC_PRIO; - ft = mlx5_create_auto_grouped_flow_table(priv->fs->ns, &ft_attr); + ft = mlx5_create_auto_grouped_flow_table(ns, &ft_attr); if (IS_ERR(ft)) return PTR_ERR(ft); @@ -205,7 +206,7 @@ static int rx_create(struct mlx5e_priv *priv, enum accel_fs_esp_type type) ft_attr.prio = MLX5E_NIC_PRIO; ft_attr.autogroup.num_reserved_entries = 1; ft_attr.autogroup.max_num_groups = 1; - ft = mlx5_create_auto_grouped_flow_table(priv->fs->ns, &ft_attr); + ft = mlx5_create_auto_grouped_flow_table(ns, &ft_attr); if (IS_ERR(ft)) { err = PTR_ERR(ft); goto err_fs_ft; @@ -230,6 +231,7 @@ err_add: static int rx_ft_get(struct mlx5e_priv *priv, enum accel_fs_esp_type type) { + struct mlx5_ttc_table *ttc = mlx5e_fs_get_ttc(priv->fs, false); struct mlx5e_accel_fs_esp_prot *fs_prot; struct mlx5_flow_destination dest = {}; struct mlx5e_accel_fs_esp *accel_esp; @@ -249,7 +251,7 @@ static int rx_ft_get(struct mlx5e_priv *priv, enum accel_fs_esp_type type) /* connect */ dest.type = MLX5_FLOW_DESTINATION_TYPE_FLOW_TABLE; dest.ft = fs_prot->ft; - mlx5_ttc_fwd_dest(priv->fs->ttc, fs_esp2tt(type), &dest); + mlx5_ttc_fwd_dest(ttc, fs_esp2tt(type), &dest); skip: fs_prot->refcnt++; @@ -260,6 +262,7 @@ out: static void rx_ft_put(struct mlx5e_priv *priv, enum accel_fs_esp_type type) { + struct mlx5_ttc_table *ttc = mlx5e_fs_get_ttc(priv->fs, false); struct mlx5e_accel_fs_esp_prot *fs_prot; struct mlx5e_accel_fs_esp *accel_esp; @@ -271,7 +274,7 @@ static void rx_ft_put(struct mlx5e_priv *priv, enum accel_fs_esp_type type) goto out; /* disconnect */ - mlx5_ttc_fwd_default_dest(priv->fs->ttc, fs_esp2tt(type)); + mlx5_ttc_fwd_default_dest(ttc, fs_esp2tt(type)); /* remove FT */ rx_destroy(priv, type); @@ -385,7 +388,8 @@ static void setup_fte_common(struct mlx5_accel_esp_xfrm_attrs *attrs, 0xff, 16); } - flow_act->ipsec_obj_id = ipsec_obj_id; + flow_act->crypto.type = MLX5_FLOW_CONTEXT_ENCRYPT_DECRYPT_TYPE_IPSEC; + flow_act->crypto.obj_id = ipsec_obj_id; flow_act->flags |= FLOW_ACT_NO_APPEND; } @@ -441,7 +445,7 @@ static int rx_add_rule(struct mlx5e_priv *priv, } flow_act.action = MLX5_FLOW_CONTEXT_ACTION_FWD_DEST | - MLX5_FLOW_CONTEXT_ACTION_IPSEC_DECRYPT | + MLX5_FLOW_CONTEXT_ACTION_CRYPTO_DECRYPT | MLX5_FLOW_CONTEXT_ACTION_MOD_HDR; dest.type = MLX5_FLOW_DESTINATION_TYPE_FLOW_TABLE; flow_act.modify_hdr = modify_hdr; @@ -497,7 +501,7 @@ static int tx_add_rule(struct mlx5e_priv *priv, MLX5_ETH_WQE_FT_META_IPSEC); flow_act.action = MLX5_FLOW_CONTEXT_ACTION_ALLOW | - MLX5_FLOW_CONTEXT_ACTION_IPSEC_ENCRYPT; + MLX5_FLOW_CONTEXT_ACTION_CRYPTO_ENCRYPT; rule = mlx5_add_flow_rules(priv->ipsec->tx_fs->ft, spec, &flow_act, NULL, 0); if (IS_ERR(rule)) { err = PTR_ERR(rule); @@ -573,7 +577,7 @@ int mlx5e_accel_ipsec_fs_init(struct mlx5e_ipsec *ipsec) int err = -ENOMEM; ns = mlx5_get_flow_namespace(ipsec->mdev, - MLX5_FLOW_NAMESPACE_EGRESS_KERNEL); + MLX5_FLOW_NAMESPACE_EGRESS_IPSEC); if (!ns) return -EOPNOTSUPP; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec_rxtx.h b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec_rxtx.h index 0ae4e12ce528..1878a70b9031 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec_rxtx.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec_rxtx.h @@ -39,9 +39,9 @@ #include "en.h" #include "en/txrx.h" -/* Bit31: IPsec marker, Bit30-24: IPsec syndrome, Bit23-0: IPsec obj id */ +/* Bit31: IPsec marker, Bit30: reserved, Bit29-24: IPsec syndrome, Bit23-0: IPsec obj id */ #define MLX5_IPSEC_METADATA_MARKER(metadata) (((metadata) >> 31) & 0x1) -#define MLX5_IPSEC_METADATA_SYNDROM(metadata) (((metadata) >> 24) & GENMASK(6, 0)) +#define MLX5_IPSEC_METADATA_SYNDROM(metadata) (((metadata) >> 24) & GENMASK(5, 0)) #define MLX5_IPSEC_METADATA_HANDLE(metadata) ((metadata) & GENMASK(23, 0)) struct mlx5e_accel_tx_ipsec_state { @@ -77,11 +77,6 @@ static inline bool mlx5_ipsec_is_rx_flow(struct mlx5_cqe64 *cqe) return MLX5_IPSEC_METADATA_MARKER(be32_to_cpu(cqe->ft_metadata)); } -static inline bool mlx5e_ipsec_is_tx_flow(struct mlx5e_accel_tx_ipsec_state *ipsec_st) -{ - return ipsec_st->x; -} - static inline bool mlx5e_ipsec_eseg_meta(struct mlx5_wqe_eth_seg *eseg) { return eseg->flow_table_metadata & cpu_to_be32(MLX5_ETH_WQE_FT_META_IPSEC); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls.c b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls.c index 30a70d139046..da2184c94203 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls.c @@ -92,6 +92,24 @@ static const struct tlsdev_ops mlx5e_ktls_ops = { .tls_dev_resync = mlx5e_ktls_resync, }; +bool mlx5e_is_ktls_rx(struct mlx5_core_dev *mdev) +{ + u8 max_sq_wqebbs = mlx5e_get_max_sq_wqebbs(mdev); + + if (is_kdump_kernel() || !MLX5_CAP_GEN(mdev, tls_rx)) + return false; + + /* Check the possibility to post the required ICOSQ WQEs. */ + if (WARN_ON_ONCE(max_sq_wqebbs < MLX5E_TLS_SET_STATIC_PARAMS_WQEBBS)) + return false; + if (WARN_ON_ONCE(max_sq_wqebbs < MLX5E_TLS_SET_PROGRESS_PARAMS_WQEBBS)) + return false; + if (WARN_ON_ONCE(max_sq_wqebbs < MLX5E_KTLS_GET_PROGRESS_WQEBBS)) + return false; + + return true; +} + void mlx5e_ktls_build_netdev(struct mlx5e_priv *priv) { struct net_device *netdev = priv->netdev; @@ -118,9 +136,9 @@ int mlx5e_ktls_set_feature_rx(struct net_device *netdev, bool enable) mutex_lock(&priv->state_lock); if (enable) - err = mlx5e_accel_fs_tcp_create(priv); + err = mlx5e_accel_fs_tcp_create(priv->fs); else - mlx5e_accel_fs_tcp_destroy(priv); + mlx5e_accel_fs_tcp_destroy(priv->fs); mutex_unlock(&priv->state_lock); return err; @@ -138,7 +156,7 @@ int mlx5e_ktls_init_rx(struct mlx5e_priv *priv) return -ENOMEM; if (priv->netdev->features & NETIF_F_HW_TLS_RX) { - err = mlx5e_accel_fs_tcp_create(priv); + err = mlx5e_accel_fs_tcp_create(priv->fs); if (err) { destroy_workqueue(priv->tls->rx_wq); return err; @@ -154,7 +172,7 @@ void mlx5e_ktls_cleanup_rx(struct mlx5e_priv *priv) return; if (priv->netdev->features & NETIF_F_HW_TLS_RX) - mlx5e_accel_fs_tcp_destroy(priv); + mlx5e_accel_fs_tcp_destroy(priv->fs); destroy_workqueue(priv->tls->rx_wq); } diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls.h b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls.h index 948400dee525..1c35045e41fb 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls.h @@ -25,7 +25,8 @@ static inline bool mlx5e_is_ktls_device(struct mlx5_core_dev *mdev) if (!MLX5_CAP_GEN(mdev, log_max_dek)) return false; - return MLX5_CAP_TLS(mdev, tls_1_2_aes_gcm_128); + return (MLX5_CAP_TLS(mdev, tls_1_2_aes_gcm_128) || + MLX5_CAP_TLS(mdev, tls_1_2_aes_gcm_256)); } static inline bool mlx5e_ktls_type_check(struct mlx5_core_dev *mdev, @@ -36,6 +37,10 @@ static inline bool mlx5e_ktls_type_check(struct mlx5_core_dev *mdev, if (crypto_info->version == TLS_1_2_VERSION) return MLX5_CAP_TLS(mdev, tls_1_2_aes_gcm_128); break; + case TLS_CIPHER_AES_GCM_256: + if (crypto_info->version == TLS_1_2_VERSION) + return MLX5_CAP_TLS(mdev, tls_1_2_aes_gcm_256); + break; } return false; @@ -56,10 +61,7 @@ static inline bool mlx5e_is_ktls_tx(struct mlx5_core_dev *mdev) return !is_kdump_kernel() && MLX5_CAP_GEN(mdev, tls_tx); } -static inline bool mlx5e_is_ktls_rx(struct mlx5_core_dev *mdev) -{ - return !is_kdump_kernel() && MLX5_CAP_GEN(mdev, tls_rx); -} +bool mlx5e_is_ktls_rx(struct mlx5_core_dev *mdev); struct mlx5e_tls_sw_stats { atomic64_t tx_tls_ctx; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls_rx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls_rx.c index 27483aa7be8a..3e54834747ce 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls_rx.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls_rx.c @@ -43,7 +43,7 @@ struct mlx5e_ktls_rx_resync_ctx { }; struct mlx5e_ktls_offload_context_rx { - struct tls12_crypto_info_aes_gcm_128 crypto_info; + union mlx5e_crypto_info crypto_info; struct accel_rule rule; struct sock *sk; struct mlx5e_rq_stats *rq_stats; @@ -111,7 +111,7 @@ static void accel_rule_handle_work(struct work_struct *work) if (unlikely(test_bit(MLX5E_PRIV_RX_FLAG_DELETING, priv_rx->flags))) goto out; - rule = mlx5e_accel_fs_add_sk(accel_rule->priv, priv_rx->sk, + rule = mlx5e_accel_fs_add_sk(accel_rule->priv->fs, priv_rx->sk, mlx5e_tir_get_tirn(&priv_rx->tir), MLX5_FS_DEFAULT_FLOW_TAG); if (!IS_ERR_OR_NULL(rule)) @@ -362,7 +362,6 @@ static void resync_init(struct mlx5e_ktls_rx_resync_ctx *resync, static void resync_handle_seq_match(struct mlx5e_ktls_offload_context_rx *priv_rx, struct mlx5e_channel *c) { - struct tls12_crypto_info_aes_gcm_128 *info = &priv_rx->crypto_info; struct mlx5e_ktls_resync_resp *ktls_resync; struct mlx5e_icosq *sq; bool trigger_poll; @@ -373,7 +372,31 @@ static void resync_handle_seq_match(struct mlx5e_ktls_offload_context_rx *priv_r spin_lock_bh(&ktls_resync->lock); spin_lock_bh(&priv_rx->lock); - memcpy(info->rec_seq, &priv_rx->resync.sw_rcd_sn_be, sizeof(info->rec_seq)); + switch (priv_rx->crypto_info.crypto_info.cipher_type) { + case TLS_CIPHER_AES_GCM_128: { + struct tls12_crypto_info_aes_gcm_128 *info = + &priv_rx->crypto_info.crypto_info_128; + + memcpy(info->rec_seq, &priv_rx->resync.sw_rcd_sn_be, + sizeof(info->rec_seq)); + break; + } + case TLS_CIPHER_AES_GCM_256: { + struct tls12_crypto_info_aes_gcm_256 *info = + &priv_rx->crypto_info.crypto_info_256; + + memcpy(info->rec_seq, &priv_rx->resync.sw_rcd_sn_be, + sizeof(info->rec_seq)); + break; + } + default: + WARN_ONCE(1, "Unsupported cipher type %u\n", + priv_rx->crypto_info.crypto_info.cipher_type); + spin_unlock_bh(&priv_rx->lock); + spin_unlock_bh(&ktls_resync->lock); + return; + } + if (list_empty(&priv_rx->list)) { list_add_tail(&priv_rx->list, &ktls_resync->list); trigger_poll = !test_and_set_bit(MLX5E_SQ_STATE_PENDING_TLS_RX_RESYNC, &sq->state); @@ -461,6 +484,7 @@ static void resync_update_sn(struct mlx5e_rq *rq, struct sk_buff *skb) { struct ethhdr *eth = (struct ethhdr *)(skb->data); struct net_device *netdev = rq->netdev; + struct net *net = dev_net(netdev); struct sock *sk = NULL; unsigned int datalen; struct iphdr *iph; @@ -475,7 +499,7 @@ static void resync_update_sn(struct mlx5e_rq *rq, struct sk_buff *skb) depth += sizeof(struct iphdr); th = (void *)iph + sizeof(struct iphdr); - sk = inet_lookup_established(dev_net(netdev), &tcp_hashinfo, + sk = inet_lookup_established(net, net->ipv4.tcp_death_row.hashinfo, iph->saddr, th->source, iph->daddr, th->dest, netdev->ifindex); #if IS_ENABLED(CONFIG_IPV6) @@ -485,7 +509,7 @@ static void resync_update_sn(struct mlx5e_rq *rq, struct sk_buff *skb) depth += sizeof(struct ipv6hdr); th = (void *)ipv6h + sizeof(struct ipv6hdr); - sk = __inet6_lookup_established(dev_net(netdev), &tcp_hashinfo, + sk = __inet6_lookup_established(net, net->ipv4.tcp_death_row.hashinfo, &ipv6h->saddr, th->source, &ipv6h->daddr, ntohs(th->dest), netdev->ifindex, 0); @@ -603,8 +627,20 @@ int mlx5e_ktls_add_rx(struct net_device *netdev, struct sock *sk, INIT_LIST_HEAD(&priv_rx->list); spin_lock_init(&priv_rx->lock); - priv_rx->crypto_info = - *(struct tls12_crypto_info_aes_gcm_128 *)crypto_info; + switch (crypto_info->cipher_type) { + case TLS_CIPHER_AES_GCM_128: + priv_rx->crypto_info.crypto_info_128 = + *(struct tls12_crypto_info_aes_gcm_128 *)crypto_info; + break; + case TLS_CIPHER_AES_GCM_256: + priv_rx->crypto_info.crypto_info_256 = + *(struct tls12_crypto_info_aes_gcm_256 *)crypto_info; + break; + default: + WARN_ONCE(1, "Unsupported cipher type %u\n", + crypto_info->cipher_type); + return -EOPNOTSUPP; + } rxq = mlx5e_ktls_sk_get_rxq(sk); priv_rx->rxq = rxq; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls_tx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls_tx.c index 3a1f76eac542..2e0335246967 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls_tx.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls_tx.c @@ -93,7 +93,7 @@ struct mlx5e_ktls_offload_context_tx { bool ctx_post_pending; /* control / resync */ struct list_head list_node; /* member of the pool */ - struct tls12_crypto_info_aes_gcm_128 crypto_info; + union mlx5e_crypto_info crypto_info; struct tls_offload_context_tx *tx_ctx; struct mlx5_core_dev *mdev; struct mlx5e_tls_sw_stats *sw_stats; @@ -485,8 +485,20 @@ int mlx5e_ktls_add_tx(struct net_device *netdev, struct sock *sk, goto err_create_key; priv_tx->expected_seq = start_offload_tcp_sn; - priv_tx->crypto_info = - *(struct tls12_crypto_info_aes_gcm_128 *)crypto_info; + switch (crypto_info->cipher_type) { + case TLS_CIPHER_AES_GCM_128: + priv_tx->crypto_info.crypto_info_128 = + *(struct tls12_crypto_info_aes_gcm_128 *)crypto_info; + break; + case TLS_CIPHER_AES_GCM_256: + priv_tx->crypto_info.crypto_info_256 = + *(struct tls12_crypto_info_aes_gcm_256 *)crypto_info; + break; + default: + WARN_ONCE(1, "Unsupported cipher type %u\n", + crypto_info->cipher_type); + return -EOPNOTSUPP; + } priv_tx->tx_ctx = tls_offload_ctx_tx(tls_ctx); mlx5e_set_ktls_tx_priv_ctx(tls_ctx, priv_tx); @@ -671,14 +683,31 @@ tx_post_resync_params(struct mlx5e_txqsq *sq, struct mlx5e_ktls_offload_context_tx *priv_tx, u64 rcd_sn) { - struct tls12_crypto_info_aes_gcm_128 *info = &priv_tx->crypto_info; __be64 rn_be = cpu_to_be64(rcd_sn); bool skip_static_post; u16 rec_seq_sz; char *rec_seq; - rec_seq = info->rec_seq; - rec_seq_sz = sizeof(info->rec_seq); + switch (priv_tx->crypto_info.crypto_info.cipher_type) { + case TLS_CIPHER_AES_GCM_128: { + struct tls12_crypto_info_aes_gcm_128 *info = &priv_tx->crypto_info.crypto_info_128; + + rec_seq = info->rec_seq; + rec_seq_sz = sizeof(info->rec_seq); + break; + } + case TLS_CIPHER_AES_GCM_256: { + struct tls12_crypto_info_aes_gcm_256 *info = &priv_tx->crypto_info.crypto_info_256; + + rec_seq = info->rec_seq; + rec_seq_sz = sizeof(info->rec_seq); + break; + } + default: + WARN_ONCE(1, "Unsupported cipher type %u\n", + priv_tx->crypto_info.crypto_info.cipher_type); + return; + } skip_static_post = !memcmp(rec_seq, &rn_be, rec_seq_sz); if (!skip_static_post) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls_txrx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls_txrx.c index ac29aeb8af49..570a912dd6fa 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls_txrx.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls_txrx.c @@ -21,7 +21,7 @@ enum { static void fill_static_params(struct mlx5_wqe_tls_static_params_seg *params, - struct tls12_crypto_info_aes_gcm_128 *info, + union mlx5e_crypto_info *crypto_info, u32 key_id, u32 resync_tcp_sn) { char *initial_rn, *gcm_iv; @@ -32,7 +32,26 @@ fill_static_params(struct mlx5_wqe_tls_static_params_seg *params, ctx = params->ctx; - EXTRACT_INFO_FIELDS; + switch (crypto_info->crypto_info.cipher_type) { + case TLS_CIPHER_AES_GCM_128: { + struct tls12_crypto_info_aes_gcm_128 *info = + &crypto_info->crypto_info_128; + + EXTRACT_INFO_FIELDS; + break; + } + case TLS_CIPHER_AES_GCM_256: { + struct tls12_crypto_info_aes_gcm_256 *info = + &crypto_info->crypto_info_256; + + EXTRACT_INFO_FIELDS; + break; + } + default: + WARN_ONCE(1, "Unsupported cipher type %u\n", + crypto_info->crypto_info.cipher_type); + return; + } gcm_iv = MLX5_ADDR_OF(tls_static_params, ctx, gcm_iv); initial_rn = MLX5_ADDR_OF(tls_static_params, ctx, initial_record_number); @@ -54,7 +73,7 @@ fill_static_params(struct mlx5_wqe_tls_static_params_seg *params, void mlx5e_ktls_build_static_params(struct mlx5e_set_tls_static_params_wqe *wqe, u16 pc, u32 sqn, - struct tls12_crypto_info_aes_gcm_128 *info, + union mlx5e_crypto_info *crypto_info, u32 tis_tir_num, u32 key_id, u32 resync_tcp_sn, bool fence, enum tls_offload_ctx_dir direction) { @@ -75,7 +94,7 @@ mlx5e_ktls_build_static_params(struct mlx5e_set_tls_static_params_wqe *wqe, ucseg->flags = MLX5_UMR_INLINE; ucseg->bsf_octowords = cpu_to_be16(MLX5_ST_SZ_BYTES(tls_static_params) / 16); - fill_static_params(&wqe->params, info, key_id, resync_tcp_sn); + fill_static_params(&wqe->params, crypto_info, key_id, resync_tcp_sn); } static void diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls_utils.h b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls_utils.h index 0dc715c4c10d..3d79cd379890 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls_utils.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls_utils.h @@ -27,6 +27,12 @@ int mlx5e_ktls_add_rx(struct net_device *netdev, struct sock *sk, void mlx5e_ktls_del_rx(struct net_device *netdev, struct tls_context *tls_ctx); void mlx5e_ktls_rx_resync(struct net_device *netdev, struct sock *sk, u32 seq, u8 *rcd_sn); +union mlx5e_crypto_info { + struct tls_crypto_info crypto_info; + struct tls12_crypto_info_aes_gcm_128 crypto_info_128; + struct tls12_crypto_info_aes_gcm_256 crypto_info_256; +}; + struct mlx5e_set_tls_static_params_wqe { struct mlx5_wqe_ctrl_seg ctrl; struct mlx5_wqe_umr_ctrl_seg uctrl; @@ -72,7 +78,7 @@ struct mlx5e_get_tls_progress_params_wqe { void mlx5e_ktls_build_static_params(struct mlx5e_set_tls_static_params_wqe *wqe, u16 pc, u32 sqn, - struct tls12_crypto_info_aes_gcm_128 *info, + union mlx5e_crypto_info *crypto_info, u32 tis_tir_num, u32 key_id, u32 resync_tcp_sn, bool fence, enum tls_offload_ctx_dir direction); void diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/macsec.c b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/macsec.c new file mode 100644 index 000000000000..5da746da898d --- /dev/null +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/macsec.c @@ -0,0 +1,1870 @@ +// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB +/* Copyright (c) 2022, NVIDIA CORPORATION & AFFILIATES. All rights reserved. */ + +#include <linux/mlx5/device.h> +#include <linux/mlx5/mlx5_ifc.h> +#include <linux/xarray.h> + +#include "en.h" +#include "lib/aso.h" +#include "lib/mlx5.h" +#include "en_accel/macsec.h" +#include "en_accel/macsec_fs.h" + +#define MLX5_MACSEC_EPN_SCOPE_MID 0x80000000L +#define MLX5E_MACSEC_ASO_CTX_SZ MLX5_ST_SZ_BYTES(macsec_aso) + +enum mlx5_macsec_aso_event_arm { + MLX5E_ASO_EPN_ARM = BIT(0), +}; + +enum { + MLX5_MACSEC_ASO_REMOVE_FLOW_PKT_CNT_OFFSET, +}; + +struct mlx5e_macsec_handle { + struct mlx5e_macsec *macsec; + u32 obj_id; + u8 idx; +}; + +enum { + MLX5_MACSEC_EPN, +}; + +struct mlx5e_macsec_aso_out { + u8 event_arm; + u32 mode_param; +}; + +struct mlx5e_macsec_aso_in { + u8 mode; + u32 obj_id; +}; + +struct mlx5e_macsec_epn_state { + u32 epn_msb; + u8 epn_enabled; + u8 overlap; +}; + +struct mlx5e_macsec_async_work { + struct mlx5e_macsec *macsec; + struct mlx5_core_dev *mdev; + struct work_struct work; + u32 obj_id; +}; + +struct mlx5e_macsec_sa { + bool active; + u8 assoc_num; + u32 macsec_obj_id; + u32 enc_key_id; + u32 next_pn; + sci_t sci; + salt_t salt; + + struct rhash_head hash; + u32 fs_id; + union mlx5e_macsec_rule *macsec_rule; + struct rcu_head rcu_head; + struct mlx5e_macsec_epn_state epn_state; +}; + +struct mlx5e_macsec_rx_sc; +struct mlx5e_macsec_rx_sc_xarray_element { + u32 fs_id; + struct mlx5e_macsec_rx_sc *rx_sc; +}; + +struct mlx5e_macsec_rx_sc { + bool active; + sci_t sci; + struct mlx5e_macsec_sa *rx_sa[MACSEC_NUM_AN]; + struct list_head rx_sc_list_element; + struct mlx5e_macsec_rx_sc_xarray_element *sc_xarray_element; + struct metadata_dst *md_dst; + struct rcu_head rcu_head; +}; + +struct mlx5e_macsec_umr { + dma_addr_t dma_addr; + u8 ctx[MLX5_ST_SZ_BYTES(macsec_aso)]; + u32 mkey; +}; + +struct mlx5e_macsec_aso { + /* ASO */ + struct mlx5_aso *maso; + /* Protects macsec ASO */ + struct mutex aso_lock; + /* UMR */ + struct mlx5e_macsec_umr *umr; + + u32 pdn; +}; + +static const struct rhashtable_params rhash_sci = { + .key_len = sizeof_field(struct mlx5e_macsec_sa, sci), + .key_offset = offsetof(struct mlx5e_macsec_sa, sci), + .head_offset = offsetof(struct mlx5e_macsec_sa, hash), + .automatic_shrinking = true, + .min_size = 1, +}; + +struct mlx5e_macsec_device { + const struct net_device *netdev; + struct mlx5e_macsec_sa *tx_sa[MACSEC_NUM_AN]; + struct list_head macsec_rx_sc_list_head; + unsigned char *dev_addr; + struct list_head macsec_device_list_element; +}; + +struct mlx5e_macsec { + struct list_head macsec_device_list_head; + int num_of_devices; + struct mlx5e_macsec_fs *macsec_fs; + struct mutex lock; /* Protects mlx5e_macsec internal contexts */ + + /* Tx sci -> fs id mapping handling */ + struct rhashtable sci_hash; /* sci -> mlx5e_macsec_sa */ + + /* Rx fs_id -> rx_sc mapping */ + struct xarray sc_xarray; + + struct mlx5_core_dev *mdev; + + /* Stats manage */ + struct mlx5e_macsec_stats stats; + + /* ASO */ + struct mlx5e_macsec_aso aso; + + struct notifier_block nb; + struct workqueue_struct *wq; +}; + +struct mlx5_macsec_obj_attrs { + u32 aso_pdn; + u32 next_pn; + __be64 sci; + u32 enc_key_id; + bool encrypt; + struct mlx5e_macsec_epn_state epn_state; + salt_t salt; + __be32 ssci; + bool replay_protect; + u32 replay_window; +}; + +struct mlx5_aso_ctrl_param { + u8 data_mask_mode; + u8 condition_0_operand; + u8 condition_1_operand; + u8 condition_0_offset; + u8 condition_1_offset; + u8 data_offset; + u8 condition_operand; + u32 condition_0_data; + u32 condition_0_mask; + u32 condition_1_data; + u32 condition_1_mask; + u64 bitwise_data; + u64 data_mask; +}; + +static int mlx5e_macsec_aso_reg_mr(struct mlx5_core_dev *mdev, struct mlx5e_macsec_aso *aso) +{ + struct mlx5e_macsec_umr *umr; + struct device *dma_device; + dma_addr_t dma_addr; + int err; + + umr = kzalloc(sizeof(*umr), GFP_KERNEL); + if (!umr) { + err = -ENOMEM; + return err; + } + + dma_device = &mdev->pdev->dev; + dma_addr = dma_map_single(dma_device, umr->ctx, sizeof(umr->ctx), DMA_BIDIRECTIONAL); + err = dma_mapping_error(dma_device, dma_addr); + if (err) { + mlx5_core_err(mdev, "Can't map dma device, err=%d\n", err); + goto out_dma; + } + + err = mlx5e_create_mkey(mdev, aso->pdn, &umr->mkey); + if (err) { + mlx5_core_err(mdev, "Can't create mkey, err=%d\n", err); + goto out_mkey; + } + + umr->dma_addr = dma_addr; + + aso->umr = umr; + + return 0; + +out_mkey: + dma_unmap_single(dma_device, dma_addr, sizeof(umr->ctx), DMA_BIDIRECTIONAL); +out_dma: + kfree(umr); + return err; +} + +static void mlx5e_macsec_aso_dereg_mr(struct mlx5_core_dev *mdev, struct mlx5e_macsec_aso *aso) +{ + struct mlx5e_macsec_umr *umr = aso->umr; + + mlx5_core_destroy_mkey(mdev, umr->mkey); + dma_unmap_single(&mdev->pdev->dev, umr->dma_addr, sizeof(umr->ctx), DMA_BIDIRECTIONAL); + kfree(umr); +} + +static int macsec_set_replay_protection(struct mlx5_macsec_obj_attrs *attrs, void *aso_ctx) +{ + u8 window_sz; + + if (!attrs->replay_protect) + return 0; + + switch (attrs->replay_window) { + case 256: + window_sz = MLX5_MACSEC_ASO_REPLAY_WIN_256BIT; + break; + case 128: + window_sz = MLX5_MACSEC_ASO_REPLAY_WIN_128BIT; + break; + case 64: + window_sz = MLX5_MACSEC_ASO_REPLAY_WIN_64BIT; + break; + case 32: + window_sz = MLX5_MACSEC_ASO_REPLAY_WIN_32BIT; + break; + default: + return -EINVAL; + } + MLX5_SET(macsec_aso, aso_ctx, window_size, window_sz); + MLX5_SET(macsec_aso, aso_ctx, mode, MLX5_MACSEC_ASO_REPLAY_PROTECTION); + + return 0; +} + +static int mlx5e_macsec_create_object(struct mlx5_core_dev *mdev, + struct mlx5_macsec_obj_attrs *attrs, + bool is_tx, + u32 *macsec_obj_id) +{ + u32 in[MLX5_ST_SZ_DW(create_macsec_obj_in)] = {}; + u32 out[MLX5_ST_SZ_DW(general_obj_out_cmd_hdr)]; + void *aso_ctx; + void *obj; + int err; + + obj = MLX5_ADDR_OF(create_macsec_obj_in, in, macsec_object); + aso_ctx = MLX5_ADDR_OF(macsec_offload_obj, obj, macsec_aso); + + MLX5_SET(macsec_offload_obj, obj, confidentiality_en, attrs->encrypt); + MLX5_SET(macsec_offload_obj, obj, dekn, attrs->enc_key_id); + MLX5_SET(macsec_offload_obj, obj, aso_return_reg, MLX5_MACSEC_ASO_REG_C_4_5); + MLX5_SET(macsec_offload_obj, obj, macsec_aso_access_pd, attrs->aso_pdn); + MLX5_SET(macsec_aso, aso_ctx, mode_parameter, attrs->next_pn); + + /* Epn */ + if (attrs->epn_state.epn_enabled) { + void *salt_p; + int i; + + MLX5_SET(macsec_aso, aso_ctx, epn_event_arm, 1); + MLX5_SET(macsec_offload_obj, obj, epn_en, 1); + MLX5_SET(macsec_offload_obj, obj, epn_msb, attrs->epn_state.epn_msb); + MLX5_SET(macsec_offload_obj, obj, epn_overlap, attrs->epn_state.overlap); + MLX5_SET64(macsec_offload_obj, obj, sci, (__force u64)attrs->ssci); + salt_p = MLX5_ADDR_OF(macsec_offload_obj, obj, salt); + for (i = 0; i < 3 ; i++) + memcpy((u32 *)salt_p + i, &attrs->salt.bytes[4 * (2 - i)], 4); + } else { + MLX5_SET64(macsec_offload_obj, obj, sci, (__force u64)(attrs->sci)); + } + + MLX5_SET(macsec_aso, aso_ctx, valid, 0x1); + if (is_tx) { + MLX5_SET(macsec_aso, aso_ctx, mode, MLX5_MACSEC_ASO_INC_SN); + } else { + err = macsec_set_replay_protection(attrs, aso_ctx); + if (err) + return err; + } + + /* general object fields set */ + MLX5_SET(general_obj_in_cmd_hdr, in, opcode, MLX5_CMD_OP_CREATE_GENERAL_OBJECT); + MLX5_SET(general_obj_in_cmd_hdr, in, obj_type, MLX5_GENERAL_OBJECT_TYPES_MACSEC); + + err = mlx5_cmd_exec(mdev, in, sizeof(in), out, sizeof(out)); + if (err) { + mlx5_core_err(mdev, + "MACsec offload: Failed to create MACsec object (err = %d)\n", + err); + return err; + } + + *macsec_obj_id = MLX5_GET(general_obj_out_cmd_hdr, out, obj_id); + + return err; +} + +static void mlx5e_macsec_destroy_object(struct mlx5_core_dev *mdev, u32 macsec_obj_id) +{ + u32 in[MLX5_ST_SZ_DW(general_obj_in_cmd_hdr)] = {}; + u32 out[MLX5_ST_SZ_DW(general_obj_out_cmd_hdr)]; + + MLX5_SET(general_obj_in_cmd_hdr, in, opcode, MLX5_CMD_OP_DESTROY_GENERAL_OBJECT); + MLX5_SET(general_obj_in_cmd_hdr, in, obj_type, MLX5_GENERAL_OBJECT_TYPES_MACSEC); + MLX5_SET(general_obj_in_cmd_hdr, in, obj_id, macsec_obj_id); + + mlx5_cmd_exec(mdev, in, sizeof(in), out, sizeof(out)); +} + +static void mlx5e_macsec_cleanup_sa(struct mlx5e_macsec *macsec, + struct mlx5e_macsec_sa *sa, + bool is_tx) +{ + int action = (is_tx) ? MLX5_ACCEL_MACSEC_ACTION_ENCRYPT : + MLX5_ACCEL_MACSEC_ACTION_DECRYPT; + + if ((is_tx) && sa->fs_id) { + /* Make sure ongoing datapath readers sees a valid SA */ + rhashtable_remove_fast(&macsec->sci_hash, &sa->hash, rhash_sci); + sa->fs_id = 0; + } + + if (!sa->macsec_rule) + return; + + mlx5e_macsec_fs_del_rule(macsec->macsec_fs, sa->macsec_rule, action); + mlx5e_macsec_destroy_object(macsec->mdev, sa->macsec_obj_id); + sa->macsec_rule = NULL; +} + +static int mlx5e_macsec_init_sa(struct macsec_context *ctx, + struct mlx5e_macsec_sa *sa, + bool encrypt, + bool is_tx) +{ + struct mlx5e_priv *priv = netdev_priv(ctx->netdev); + struct mlx5e_macsec *macsec = priv->macsec; + struct mlx5_macsec_rule_attrs rule_attrs; + struct mlx5_core_dev *mdev = priv->mdev; + struct mlx5_macsec_obj_attrs obj_attrs; + union mlx5e_macsec_rule *macsec_rule; + struct macsec_key *key; + int err; + + obj_attrs.next_pn = sa->next_pn; + obj_attrs.sci = cpu_to_be64((__force u64)sa->sci); + obj_attrs.enc_key_id = sa->enc_key_id; + obj_attrs.encrypt = encrypt; + obj_attrs.aso_pdn = macsec->aso.pdn; + obj_attrs.epn_state = sa->epn_state; + + if (is_tx) { + obj_attrs.ssci = cpu_to_be32((__force u32)ctx->sa.tx_sa->ssci); + key = &ctx->sa.tx_sa->key; + } else { + obj_attrs.ssci = cpu_to_be32((__force u32)ctx->sa.rx_sa->ssci); + key = &ctx->sa.rx_sa->key; + } + + memcpy(&obj_attrs.salt, &key->salt, sizeof(key->salt)); + obj_attrs.replay_window = ctx->secy->replay_window; + obj_attrs.replay_protect = ctx->secy->replay_protect; + + err = mlx5e_macsec_create_object(mdev, &obj_attrs, is_tx, &sa->macsec_obj_id); + if (err) + return err; + + rule_attrs.macsec_obj_id = sa->macsec_obj_id; + rule_attrs.sci = sa->sci; + rule_attrs.assoc_num = sa->assoc_num; + rule_attrs.action = (is_tx) ? MLX5_ACCEL_MACSEC_ACTION_ENCRYPT : + MLX5_ACCEL_MACSEC_ACTION_DECRYPT; + + macsec_rule = mlx5e_macsec_fs_add_rule(macsec->macsec_fs, ctx, &rule_attrs, &sa->fs_id); + if (!macsec_rule) { + err = -ENOMEM; + goto destroy_macsec_object; + } + + sa->macsec_rule = macsec_rule; + + if (is_tx) { + err = rhashtable_insert_fast(&macsec->sci_hash, &sa->hash, rhash_sci); + if (err) + goto destroy_macsec_object_and_rule; + } + + return 0; + +destroy_macsec_object_and_rule: + mlx5e_macsec_cleanup_sa(macsec, sa, is_tx); +destroy_macsec_object: + mlx5e_macsec_destroy_object(mdev, sa->macsec_obj_id); + + return err; +} + +static struct mlx5e_macsec_rx_sc * +mlx5e_macsec_get_rx_sc_from_sc_list(const struct list_head *list, sci_t sci) +{ + struct mlx5e_macsec_rx_sc *iter; + + list_for_each_entry_rcu(iter, list, rx_sc_list_element) { + if (iter->sci == sci) + return iter; + } + + return NULL; +} + +static int mlx5e_macsec_update_rx_sa(struct mlx5e_macsec *macsec, + struct mlx5e_macsec_sa *rx_sa, + bool active) +{ + struct mlx5_core_dev *mdev = macsec->mdev; + struct mlx5_macsec_obj_attrs attrs; + int err = 0; + + if (rx_sa->active != active) + return 0; + + rx_sa->active = active; + if (!active) { + mlx5e_macsec_cleanup_sa(macsec, rx_sa, false); + return 0; + } + + attrs.sci = rx_sa->sci; + attrs.enc_key_id = rx_sa->enc_key_id; + err = mlx5e_macsec_create_object(mdev, &attrs, false, &rx_sa->macsec_obj_id); + if (err) + return err; + + return 0; +} + +static bool mlx5e_macsec_secy_features_validate(struct macsec_context *ctx) +{ + const struct net_device *netdev = ctx->netdev; + const struct macsec_secy *secy = ctx->secy; + + if (secy->validate_frames != MACSEC_VALIDATE_STRICT) { + netdev_err(netdev, + "MACsec offload is supported only when validate_frame is in strict mode\n"); + return false; + } + + if (secy->icv_len != MACSEC_DEFAULT_ICV_LEN) { + netdev_err(netdev, "MACsec offload is supported only when icv_len is %d\n", + MACSEC_DEFAULT_ICV_LEN); + return false; + } + + if (!secy->protect_frames) { + netdev_err(netdev, + "MACsec offload is supported only when protect_frames is set\n"); + return false; + } + + return true; +} + +static struct mlx5e_macsec_device * +mlx5e_macsec_get_macsec_device_context(const struct mlx5e_macsec *macsec, + const struct macsec_context *ctx) +{ + struct mlx5e_macsec_device *iter; + const struct list_head *list; + + list = &macsec->macsec_device_list_head; + list_for_each_entry_rcu(iter, list, macsec_device_list_element) { + if (iter->netdev == ctx->secy->netdev) + return iter; + } + + return NULL; +} + +static void update_macsec_epn(struct mlx5e_macsec_sa *sa, const struct macsec_key *key, + const pn_t *next_pn_halves) +{ + struct mlx5e_macsec_epn_state *epn_state = &sa->epn_state; + + sa->salt = key->salt; + epn_state->epn_enabled = 1; + epn_state->epn_msb = next_pn_halves->upper; + epn_state->overlap = next_pn_halves->lower < MLX5_MACSEC_EPN_SCOPE_MID ? 0 : 1; +} + +static int mlx5e_macsec_add_txsa(struct macsec_context *ctx) +{ + const struct macsec_tx_sc *tx_sc = &ctx->secy->tx_sc; + const struct macsec_tx_sa *ctx_tx_sa = ctx->sa.tx_sa; + struct mlx5e_priv *priv = netdev_priv(ctx->netdev); + const struct macsec_secy *secy = ctx->secy; + struct mlx5e_macsec_device *macsec_device; + struct mlx5_core_dev *mdev = priv->mdev; + u8 assoc_num = ctx->sa.assoc_num; + struct mlx5e_macsec_sa *tx_sa; + struct mlx5e_macsec *macsec; + int err = 0; + + mutex_lock(&priv->macsec->lock); + + macsec = priv->macsec; + macsec_device = mlx5e_macsec_get_macsec_device_context(macsec, ctx); + if (!macsec_device) { + netdev_err(ctx->netdev, "MACsec offload: Failed to find device context\n"); + err = -EEXIST; + goto out; + } + + if (macsec_device->tx_sa[assoc_num]) { + netdev_err(ctx->netdev, "MACsec offload tx_sa: %d already exist\n", assoc_num); + err = -EEXIST; + goto out; + } + + tx_sa = kzalloc(sizeof(*tx_sa), GFP_KERNEL); + if (!tx_sa) { + err = -ENOMEM; + goto out; + } + + tx_sa->active = ctx_tx_sa->active; + tx_sa->next_pn = ctx_tx_sa->next_pn_halves.lower; + tx_sa->sci = secy->sci; + tx_sa->assoc_num = assoc_num; + + if (secy->xpn) + update_macsec_epn(tx_sa, &ctx_tx_sa->key, &ctx_tx_sa->next_pn_halves); + + err = mlx5_create_encryption_key(mdev, ctx->sa.key, secy->key_len, + MLX5_ACCEL_OBJ_MACSEC_KEY, + &tx_sa->enc_key_id); + if (err) + goto destroy_sa; + + macsec_device->tx_sa[assoc_num] = tx_sa; + if (!secy->operational || + assoc_num != tx_sc->encoding_sa || + !tx_sa->active) + goto out; + + err = mlx5e_macsec_init_sa(ctx, tx_sa, tx_sc->encrypt, true); + if (err) + goto destroy_encryption_key; + + mutex_unlock(&macsec->lock); + + return 0; + +destroy_encryption_key: + macsec_device->tx_sa[assoc_num] = NULL; + mlx5_destroy_encryption_key(mdev, tx_sa->enc_key_id); +destroy_sa: + kfree(tx_sa); +out: + mutex_unlock(&macsec->lock); + + return err; +} + +static int mlx5e_macsec_upd_txsa(struct macsec_context *ctx) +{ + const struct macsec_tx_sc *tx_sc = &ctx->secy->tx_sc; + const struct macsec_tx_sa *ctx_tx_sa = ctx->sa.tx_sa; + struct mlx5e_priv *priv = netdev_priv(ctx->netdev); + struct mlx5e_macsec_device *macsec_device; + u8 assoc_num = ctx->sa.assoc_num; + struct mlx5e_macsec_sa *tx_sa; + struct mlx5e_macsec *macsec; + struct net_device *netdev; + int err = 0; + + mutex_lock(&priv->macsec->lock); + + macsec = priv->macsec; + netdev = ctx->netdev; + macsec_device = mlx5e_macsec_get_macsec_device_context(macsec, ctx); + if (!macsec_device) { + netdev_err(netdev, "MACsec offload: Failed to find device context\n"); + err = -EINVAL; + goto out; + } + + tx_sa = macsec_device->tx_sa[assoc_num]; + if (!tx_sa) { + netdev_err(netdev, "MACsec offload: TX sa 0x%x doesn't exist\n", assoc_num); + err = -EEXIST; + goto out; + } + + if (tx_sa->next_pn != ctx_tx_sa->next_pn_halves.lower) { + netdev_err(netdev, "MACsec offload: update TX sa %d PN isn't supported\n", + assoc_num); + err = -EINVAL; + goto out; + } + + if (tx_sa->active == ctx_tx_sa->active) + goto out; + + if (tx_sa->assoc_num != tx_sc->encoding_sa) + goto out; + + if (ctx_tx_sa->active) { + err = mlx5e_macsec_init_sa(ctx, tx_sa, tx_sc->encrypt, true); + if (err) + goto out; + } else { + if (!tx_sa->macsec_rule) { + err = -EINVAL; + goto out; + } + + mlx5e_macsec_cleanup_sa(macsec, tx_sa, true); + } + + tx_sa->active = ctx_tx_sa->active; +out: + mutex_unlock(&macsec->lock); + + return err; +} + +static int mlx5e_macsec_del_txsa(struct macsec_context *ctx) +{ + struct mlx5e_priv *priv = netdev_priv(ctx->netdev); + struct mlx5e_macsec_device *macsec_device; + u8 assoc_num = ctx->sa.assoc_num; + struct mlx5e_macsec_sa *tx_sa; + struct mlx5e_macsec *macsec; + int err = 0; + + mutex_lock(&priv->macsec->lock); + macsec = priv->macsec; + macsec_device = mlx5e_macsec_get_macsec_device_context(macsec, ctx); + if (!macsec_device) { + netdev_err(ctx->netdev, "MACsec offload: Failed to find device context\n"); + err = -EINVAL; + goto out; + } + + tx_sa = macsec_device->tx_sa[assoc_num]; + if (!tx_sa) { + netdev_err(ctx->netdev, "MACsec offload: TX sa 0x%x doesn't exist\n", assoc_num); + err = -EEXIST; + goto out; + } + + mlx5e_macsec_cleanup_sa(macsec, tx_sa, true); + mlx5_destroy_encryption_key(macsec->mdev, tx_sa->enc_key_id); + kfree_rcu(tx_sa); + macsec_device->tx_sa[assoc_num] = NULL; + +out: + mutex_unlock(&macsec->lock); + + return err; +} + +static u32 mlx5e_macsec_get_sa_from_hashtable(struct rhashtable *sci_hash, sci_t *sci) +{ + struct mlx5e_macsec_sa *macsec_sa; + u32 fs_id = 0; + + rcu_read_lock(); + macsec_sa = rhashtable_lookup(sci_hash, sci, rhash_sci); + if (macsec_sa) + fs_id = macsec_sa->fs_id; + rcu_read_unlock(); + + return fs_id; +} + +static int mlx5e_macsec_add_rxsc(struct macsec_context *ctx) +{ + struct mlx5e_macsec_rx_sc_xarray_element *sc_xarray_element; + struct mlx5e_priv *priv = netdev_priv(ctx->netdev); + const struct macsec_rx_sc *ctx_rx_sc = ctx->rx_sc; + struct mlx5e_macsec_device *macsec_device; + struct mlx5e_macsec_rx_sc *rx_sc; + struct list_head *rx_sc_list; + struct mlx5e_macsec *macsec; + int err = 0; + + mutex_lock(&priv->macsec->lock); + macsec = priv->macsec; + macsec_device = mlx5e_macsec_get_macsec_device_context(macsec, ctx); + if (!macsec_device) { + netdev_err(ctx->netdev, "MACsec offload: Failed to find device context\n"); + err = -EINVAL; + goto out; + } + + rx_sc_list = &macsec_device->macsec_rx_sc_list_head; + rx_sc = mlx5e_macsec_get_rx_sc_from_sc_list(rx_sc_list, ctx_rx_sc->sci); + if (rx_sc) { + netdev_err(ctx->netdev, "MACsec offload: rx_sc (sci %lld) already exists\n", + ctx_rx_sc->sci); + err = -EEXIST; + goto out; + } + + rx_sc = kzalloc(sizeof(*rx_sc), GFP_KERNEL); + if (!rx_sc) { + err = -ENOMEM; + goto out; + } + + sc_xarray_element = kzalloc(sizeof(*sc_xarray_element), GFP_KERNEL); + if (!sc_xarray_element) { + err = -ENOMEM; + goto destroy_rx_sc; + } + + sc_xarray_element->rx_sc = rx_sc; + err = xa_alloc(&macsec->sc_xarray, &sc_xarray_element->fs_id, sc_xarray_element, + XA_LIMIT(1, USHRT_MAX), GFP_KERNEL); + if (err) + goto destroy_sc_xarray_elemenet; + + rx_sc->md_dst = metadata_dst_alloc(0, METADATA_MACSEC, GFP_KERNEL); + if (!rx_sc->md_dst) { + err = -ENOMEM; + goto erase_xa_alloc; + } + + rx_sc->sci = ctx_rx_sc->sci; + rx_sc->active = ctx_rx_sc->active; + list_add_rcu(&rx_sc->rx_sc_list_element, rx_sc_list); + + rx_sc->sc_xarray_element = sc_xarray_element; + rx_sc->md_dst->u.macsec_info.sci = rx_sc->sci; + mutex_unlock(&macsec->lock); + + return 0; + +erase_xa_alloc: + xa_erase(&macsec->sc_xarray, sc_xarray_element->fs_id); +destroy_sc_xarray_elemenet: + kfree(sc_xarray_element); +destroy_rx_sc: + kfree(rx_sc); + +out: + mutex_unlock(&macsec->lock); + + return err; +} + +static int mlx5e_macsec_upd_rxsc(struct macsec_context *ctx) +{ + struct mlx5e_priv *priv = netdev_priv(ctx->netdev); + const struct macsec_rx_sc *ctx_rx_sc = ctx->rx_sc; + struct mlx5e_macsec_device *macsec_device; + struct mlx5e_macsec_rx_sc *rx_sc; + struct mlx5e_macsec_sa *rx_sa; + struct mlx5e_macsec *macsec; + struct list_head *list; + int i; + int err = 0; + + mutex_lock(&priv->macsec->lock); + + macsec = priv->macsec; + macsec_device = mlx5e_macsec_get_macsec_device_context(macsec, ctx); + if (!macsec_device) { + netdev_err(ctx->netdev, "MACsec offload: Failed to find device context\n"); + err = -EINVAL; + goto out; + } + + list = &macsec_device->macsec_rx_sc_list_head; + rx_sc = mlx5e_macsec_get_rx_sc_from_sc_list(list, ctx_rx_sc->sci); + if (!rx_sc) { + err = -EINVAL; + goto out; + } + + rx_sc->active = ctx_rx_sc->active; + if (rx_sc->active == ctx_rx_sc->active) + goto out; + + for (i = 0; i < MACSEC_NUM_AN; ++i) { + rx_sa = rx_sc->rx_sa[i]; + if (!rx_sa) + continue; + + err = mlx5e_macsec_update_rx_sa(macsec, rx_sa, rx_sa->active && ctx_rx_sc->active); + if (err) + goto out; + } + +out: + mutex_unlock(&macsec->lock); + + return err; +} + +static int mlx5e_macsec_del_rxsc(struct macsec_context *ctx) +{ + struct mlx5e_priv *priv = netdev_priv(ctx->netdev); + struct mlx5e_macsec_device *macsec_device; + struct mlx5e_macsec_rx_sc *rx_sc; + struct mlx5e_macsec_sa *rx_sa; + struct mlx5e_macsec *macsec; + struct list_head *list; + int err = 0; + int i; + + mutex_lock(&priv->macsec->lock); + + macsec = priv->macsec; + macsec_device = mlx5e_macsec_get_macsec_device_context(macsec, ctx); + if (!macsec_device) { + netdev_err(ctx->netdev, "MACsec offload: Failed to find device context\n"); + err = -EINVAL; + goto out; + } + + list = &macsec_device->macsec_rx_sc_list_head; + rx_sc = mlx5e_macsec_get_rx_sc_from_sc_list(list, ctx->rx_sc->sci); + if (!rx_sc) { + netdev_err(ctx->netdev, + "MACsec offload rx_sc sci %lld doesn't exist\n", + ctx->sa.rx_sa->sc->sci); + err = -EINVAL; + goto out; + } + + for (i = 0; i < MACSEC_NUM_AN; ++i) { + rx_sa = rx_sc->rx_sa[i]; + if (!rx_sa) + continue; + + mlx5e_macsec_cleanup_sa(macsec, rx_sa, false); + mlx5_destroy_encryption_key(macsec->mdev, rx_sa->enc_key_id); + + kfree(rx_sa); + rx_sc->rx_sa[i] = NULL; + } + +/* + * At this point the relevant MACsec offload Rx rule already removed at + * mlx5e_macsec_cleanup_sa need to wait for datapath to finish current + * Rx related data propagating using xa_erase which uses rcu to sync, + * once fs_id is erased then this rx_sc is hidden from datapath. + */ + list_del_rcu(&rx_sc->rx_sc_list_element); + xa_erase(&macsec->sc_xarray, rx_sc->sc_xarray_element->fs_id); + metadata_dst_free(rx_sc->md_dst); + kfree(rx_sc->sc_xarray_element); + + kfree_rcu(rx_sc); + +out: + mutex_unlock(&macsec->lock); + + return err; +} + +static int mlx5e_macsec_add_rxsa(struct macsec_context *ctx) +{ + const struct macsec_rx_sa *ctx_rx_sa = ctx->sa.rx_sa; + struct mlx5e_priv *priv = netdev_priv(ctx->netdev); + struct mlx5e_macsec_device *macsec_device; + struct mlx5_core_dev *mdev = priv->mdev; + u8 assoc_num = ctx->sa.assoc_num; + struct mlx5e_macsec_rx_sc *rx_sc; + sci_t sci = ctx_rx_sa->sc->sci; + struct mlx5e_macsec_sa *rx_sa; + struct mlx5e_macsec *macsec; + struct list_head *list; + int err = 0; + + mutex_lock(&priv->macsec->lock); + + macsec = priv->macsec; + macsec_device = mlx5e_macsec_get_macsec_device_context(macsec, ctx); + if (!macsec_device) { + netdev_err(ctx->netdev, "MACsec offload: Failed to find device context\n"); + err = -EINVAL; + goto out; + } + + list = &macsec_device->macsec_rx_sc_list_head; + rx_sc = mlx5e_macsec_get_rx_sc_from_sc_list(list, sci); + if (!rx_sc) { + netdev_err(ctx->netdev, + "MACsec offload rx_sc sci %lld doesn't exist\n", + ctx->sa.rx_sa->sc->sci); + err = -EINVAL; + goto out; + } + + if (rx_sc->rx_sa[assoc_num]) { + netdev_err(ctx->netdev, + "MACsec offload rx_sc sci %lld rx_sa %d already exist\n", + sci, assoc_num); + err = -EEXIST; + goto out; + } + + rx_sa = kzalloc(sizeof(*rx_sa), GFP_KERNEL); + if (!rx_sa) { + err = -ENOMEM; + goto out; + } + + rx_sa->active = ctx_rx_sa->active; + rx_sa->next_pn = ctx_rx_sa->next_pn; + rx_sa->sci = sci; + rx_sa->assoc_num = assoc_num; + rx_sa->fs_id = rx_sc->sc_xarray_element->fs_id; + + if (ctx->secy->xpn) + update_macsec_epn(rx_sa, &ctx_rx_sa->key, &ctx_rx_sa->next_pn_halves); + + err = mlx5_create_encryption_key(mdev, ctx->sa.key, ctx->secy->key_len, + MLX5_ACCEL_OBJ_MACSEC_KEY, + &rx_sa->enc_key_id); + if (err) + goto destroy_sa; + + rx_sc->rx_sa[assoc_num] = rx_sa; + if (!rx_sa->active) + goto out; + + //TODO - add support for both authentication and encryption flows + err = mlx5e_macsec_init_sa(ctx, rx_sa, true, false); + if (err) + goto destroy_encryption_key; + + goto out; + +destroy_encryption_key: + rx_sc->rx_sa[assoc_num] = NULL; + mlx5_destroy_encryption_key(mdev, rx_sa->enc_key_id); +destroy_sa: + kfree(rx_sa); +out: + mutex_unlock(&macsec->lock); + + return err; +} + +static int mlx5e_macsec_upd_rxsa(struct macsec_context *ctx) +{ + const struct macsec_rx_sa *ctx_rx_sa = ctx->sa.rx_sa; + struct mlx5e_priv *priv = netdev_priv(ctx->netdev); + struct mlx5e_macsec_device *macsec_device; + u8 assoc_num = ctx->sa.assoc_num; + struct mlx5e_macsec_rx_sc *rx_sc; + sci_t sci = ctx_rx_sa->sc->sci; + struct mlx5e_macsec_sa *rx_sa; + struct mlx5e_macsec *macsec; + struct list_head *list; + int err = 0; + + mutex_lock(&priv->macsec->lock); + + macsec = priv->macsec; + macsec_device = mlx5e_macsec_get_macsec_device_context(macsec, ctx); + if (!macsec_device) { + netdev_err(ctx->netdev, "MACsec offload: Failed to find device context\n"); + err = -EINVAL; + goto out; + } + + list = &macsec_device->macsec_rx_sc_list_head; + rx_sc = mlx5e_macsec_get_rx_sc_from_sc_list(list, sci); + if (!rx_sc) { + netdev_err(ctx->netdev, + "MACsec offload rx_sc sci %lld doesn't exist\n", + ctx->sa.rx_sa->sc->sci); + err = -EINVAL; + goto out; + } + + rx_sa = rx_sc->rx_sa[assoc_num]; + if (rx_sa) { + netdev_err(ctx->netdev, + "MACsec offload rx_sc sci %lld rx_sa %d already exist\n", + sci, assoc_num); + err = -EEXIST; + goto out; + } + + if (rx_sa->next_pn != ctx_rx_sa->next_pn_halves.lower) { + netdev_err(ctx->netdev, + "MACsec offload update RX sa %d PN isn't supported\n", + assoc_num); + err = -EINVAL; + goto out; + } + + err = mlx5e_macsec_update_rx_sa(macsec, rx_sa, ctx_rx_sa->active); +out: + mutex_unlock(&macsec->lock); + + return err; +} + +static int mlx5e_macsec_del_rxsa(struct macsec_context *ctx) +{ + struct mlx5e_priv *priv = netdev_priv(ctx->netdev); + struct mlx5e_macsec_device *macsec_device; + sci_t sci = ctx->sa.rx_sa->sc->sci; + struct mlx5e_macsec_rx_sc *rx_sc; + u8 assoc_num = ctx->sa.assoc_num; + struct mlx5e_macsec_sa *rx_sa; + struct mlx5e_macsec *macsec; + struct list_head *list; + int err = 0; + + mutex_lock(&priv->macsec->lock); + + macsec = priv->macsec; + macsec_device = mlx5e_macsec_get_macsec_device_context(macsec, ctx); + if (!macsec_device) { + netdev_err(ctx->netdev, "MACsec offload: Failed to find device context\n"); + err = -EINVAL; + goto out; + } + + list = &macsec_device->macsec_rx_sc_list_head; + rx_sc = mlx5e_macsec_get_rx_sc_from_sc_list(list, sci); + if (!rx_sc) { + netdev_err(ctx->netdev, + "MACsec offload rx_sc sci %lld doesn't exist\n", + ctx->sa.rx_sa->sc->sci); + err = -EINVAL; + goto out; + } + + rx_sa = rx_sc->rx_sa[assoc_num]; + if (rx_sa) { + netdev_err(ctx->netdev, + "MACsec offload rx_sc sci %lld rx_sa %d already exist\n", + sci, assoc_num); + err = -EEXIST; + goto out; + } + + mlx5e_macsec_cleanup_sa(macsec, rx_sa, false); + mlx5_destroy_encryption_key(macsec->mdev, rx_sa->enc_key_id); + kfree(rx_sa); + rx_sc->rx_sa[assoc_num] = NULL; + +out: + mutex_unlock(&macsec->lock); + + return err; +} + +static int mlx5e_macsec_add_secy(struct macsec_context *ctx) +{ + struct mlx5e_priv *priv = netdev_priv(ctx->netdev); + const struct net_device *dev = ctx->secy->netdev; + const struct net_device *netdev = ctx->netdev; + struct mlx5e_macsec_device *macsec_device; + struct mlx5e_macsec *macsec; + int err = 0; + + if (!mlx5e_macsec_secy_features_validate(ctx)) + return -EINVAL; + + mutex_lock(&priv->macsec->lock); + macsec = priv->macsec; + if (mlx5e_macsec_get_macsec_device_context(macsec, ctx)) { + netdev_err(netdev, "MACsec offload: MACsec net_device already exist\n"); + goto out; + } + + if (macsec->num_of_devices >= MLX5_MACSEC_NUM_OF_SUPPORTED_INTERFACES) { + netdev_err(netdev, "Currently, only %d MACsec offload devices can be set\n", + MLX5_MACSEC_NUM_OF_SUPPORTED_INTERFACES); + err = -EBUSY; + goto out; + } + + macsec_device = kzalloc(sizeof(*macsec_device), GFP_KERNEL); + if (!macsec_device) { + err = -ENOMEM; + goto out; + } + + macsec_device->dev_addr = kmemdup(dev->dev_addr, dev->addr_len, GFP_KERNEL); + if (!macsec_device->dev_addr) { + kfree(macsec_device); + err = -ENOMEM; + goto out; + } + + macsec_device->netdev = dev; + + INIT_LIST_HEAD_RCU(&macsec_device->macsec_rx_sc_list_head); + list_add_rcu(&macsec_device->macsec_device_list_element, &macsec->macsec_device_list_head); + + ++macsec->num_of_devices; +out: + mutex_unlock(&macsec->lock); + + return err; +} + +static int macsec_upd_secy_hw_address(struct macsec_context *ctx, + struct mlx5e_macsec_device *macsec_device) +{ + struct mlx5e_priv *priv = netdev_priv(ctx->netdev); + const struct net_device *dev = ctx->secy->netdev; + struct mlx5e_macsec *macsec = priv->macsec; + struct mlx5e_macsec_rx_sc *rx_sc, *tmp; + struct mlx5e_macsec_sa *rx_sa; + struct list_head *list; + int i, err = 0; + + + list = &macsec_device->macsec_rx_sc_list_head; + list_for_each_entry_safe(rx_sc, tmp, list, rx_sc_list_element) { + for (i = 0; i < MACSEC_NUM_AN; ++i) { + rx_sa = rx_sc->rx_sa[i]; + if (!rx_sa || !rx_sa->macsec_rule) + continue; + + mlx5e_macsec_cleanup_sa(macsec, rx_sa, false); + } + } + + list_for_each_entry_safe(rx_sc, tmp, list, rx_sc_list_element) { + for (i = 0; i < MACSEC_NUM_AN; ++i) { + rx_sa = rx_sc->rx_sa[i]; + if (!rx_sa) + continue; + + if (rx_sa->active) { + err = mlx5e_macsec_init_sa(ctx, rx_sa, false, false); + if (err) + goto out; + } + } + } + + memcpy(macsec_device->dev_addr, dev->dev_addr, dev->addr_len); +out: + return err; +} + +/* this function is called from 2 macsec ops functions: + * macsec_set_mac_address – MAC address was changed, therefore we need to destroy + * and create new Tx contexts(macsec object + steering). + * macsec_changelink – in this case the tx SC or SecY may be changed, therefore need to + * destroy Tx and Rx contexts(macsec object + steering) + */ +static int mlx5e_macsec_upd_secy(struct macsec_context *ctx) +{ + const struct macsec_tx_sc *tx_sc = &ctx->secy->tx_sc; + struct mlx5e_priv *priv = netdev_priv(ctx->netdev); + const struct net_device *dev = ctx->secy->netdev; + struct mlx5e_macsec_device *macsec_device; + struct mlx5e_macsec_sa *tx_sa; + struct mlx5e_macsec *macsec; + int i, err = 0; + + if (!mlx5e_macsec_secy_features_validate(ctx)) + return -EINVAL; + + mutex_lock(&priv->macsec->lock); + + macsec = priv->macsec; + macsec_device = mlx5e_macsec_get_macsec_device_context(macsec, ctx); + if (!macsec_device) { + netdev_err(ctx->netdev, "MACsec offload: Failed to find device context\n"); + err = -EINVAL; + goto out; + } + + /* if the dev_addr hasn't change, it mean the callback is from macsec_changelink */ + if (!memcmp(macsec_device->dev_addr, dev->dev_addr, dev->addr_len)) { + err = macsec_upd_secy_hw_address(ctx, macsec_device); + if (err) + goto out; + } + + for (i = 0; i < MACSEC_NUM_AN; ++i) { + tx_sa = macsec_device->tx_sa[i]; + if (!tx_sa) + continue; + + mlx5e_macsec_cleanup_sa(macsec, tx_sa, true); + } + + for (i = 0; i < MACSEC_NUM_AN; ++i) { + tx_sa = macsec_device->tx_sa[i]; + if (!tx_sa) + continue; + + if (tx_sa->assoc_num == tx_sc->encoding_sa && tx_sa->active) { + err = mlx5e_macsec_init_sa(ctx, tx_sa, tx_sc->encrypt, true); + if (err) + goto out; + } + } + +out: + mutex_unlock(&macsec->lock); + + return err; +} + +static int mlx5e_macsec_del_secy(struct macsec_context *ctx) +{ + struct mlx5e_priv *priv = netdev_priv(ctx->netdev); + struct mlx5e_macsec_device *macsec_device; + struct mlx5e_macsec_rx_sc *rx_sc, *tmp; + struct mlx5e_macsec_sa *rx_sa; + struct mlx5e_macsec_sa *tx_sa; + struct mlx5e_macsec *macsec; + struct list_head *list; + int err = 0; + int i; + + mutex_lock(&priv->macsec->lock); + macsec = priv->macsec; + macsec_device = mlx5e_macsec_get_macsec_device_context(macsec, ctx); + if (!macsec_device) { + netdev_err(ctx->netdev, "MACsec offload: Failed to find device context\n"); + err = -EINVAL; + + goto out; + } + + for (i = 0; i < MACSEC_NUM_AN; ++i) { + tx_sa = macsec_device->tx_sa[i]; + if (!tx_sa) + continue; + + mlx5e_macsec_cleanup_sa(macsec, tx_sa, true); + mlx5_destroy_encryption_key(macsec->mdev, tx_sa->enc_key_id); + kfree(tx_sa); + macsec_device->tx_sa[i] = NULL; + } + + list = &macsec_device->macsec_rx_sc_list_head; + list_for_each_entry_safe(rx_sc, tmp, list, rx_sc_list_element) { + for (i = 0; i < MACSEC_NUM_AN; ++i) { + rx_sa = rx_sc->rx_sa[i]; + if (!rx_sa) + continue; + + mlx5e_macsec_cleanup_sa(macsec, rx_sa, false); + mlx5_destroy_encryption_key(macsec->mdev, rx_sa->enc_key_id); + kfree(rx_sa); + rx_sc->rx_sa[i] = NULL; + } + + list_del_rcu(&rx_sc->rx_sc_list_element); + + kfree_rcu(rx_sc); + } + + kfree(macsec_device->dev_addr); + macsec_device->dev_addr = NULL; + + list_del_rcu(&macsec_device->macsec_device_list_element); + --macsec->num_of_devices; + +out: + mutex_unlock(&macsec->lock); + + return err; +} + +static void macsec_build_accel_attrs(struct mlx5e_macsec_sa *sa, + struct mlx5_macsec_obj_attrs *attrs) +{ + attrs->epn_state.epn_msb = sa->epn_state.epn_msb; + attrs->epn_state.overlap = sa->epn_state.overlap; +} + +static void macsec_aso_build_wqe_ctrl_seg(struct mlx5e_macsec_aso *macsec_aso, + struct mlx5_wqe_aso_ctrl_seg *aso_ctrl, + struct mlx5_aso_ctrl_param *param) +{ + memset(aso_ctrl, 0, sizeof(*aso_ctrl)); + if (macsec_aso->umr->dma_addr) { + aso_ctrl->va_l = cpu_to_be32(macsec_aso->umr->dma_addr | ASO_CTRL_READ_EN); + aso_ctrl->va_h = cpu_to_be32((u64)macsec_aso->umr->dma_addr >> 32); + aso_ctrl->l_key = cpu_to_be32(macsec_aso->umr->mkey); + } + + if (!param) + return; + + aso_ctrl->data_mask_mode = param->data_mask_mode << 6; + aso_ctrl->condition_1_0_operand = param->condition_1_operand | + param->condition_0_operand << 4; + aso_ctrl->condition_1_0_offset = param->condition_1_offset | + param->condition_0_offset << 4; + aso_ctrl->data_offset_condition_operand = param->data_offset | + param->condition_operand << 6; + aso_ctrl->condition_0_data = cpu_to_be32(param->condition_0_data); + aso_ctrl->condition_0_mask = cpu_to_be32(param->condition_0_mask); + aso_ctrl->condition_1_data = cpu_to_be32(param->condition_1_data); + aso_ctrl->condition_1_mask = cpu_to_be32(param->condition_1_mask); + aso_ctrl->bitwise_data = cpu_to_be64(param->bitwise_data); + aso_ctrl->data_mask = cpu_to_be64(param->data_mask); +} + +static int mlx5e_macsec_modify_obj(struct mlx5_core_dev *mdev, struct mlx5_macsec_obj_attrs *attrs, + u32 macsec_id) +{ + u32 in[MLX5_ST_SZ_DW(modify_macsec_obj_in)] = {}; + u32 out[MLX5_ST_SZ_DW(query_macsec_obj_out)]; + u64 modify_field_select = 0; + void *obj; + int err; + + /* General object fields set */ + MLX5_SET(general_obj_in_cmd_hdr, in, opcode, MLX5_CMD_OP_QUERY_GENERAL_OBJECT); + MLX5_SET(general_obj_in_cmd_hdr, in, obj_type, MLX5_GENERAL_OBJECT_TYPES_MACSEC); + MLX5_SET(general_obj_in_cmd_hdr, in, obj_id, macsec_id); + err = mlx5_cmd_exec(mdev, in, sizeof(in), out, sizeof(out)); + if (err) { + mlx5_core_err(mdev, "Query MACsec object failed (Object id %d), err = %d\n", + macsec_id, err); + return err; + } + + obj = MLX5_ADDR_OF(query_macsec_obj_out, out, macsec_object); + modify_field_select = MLX5_GET64(macsec_offload_obj, obj, modify_field_select); + + /* EPN */ + if (!(modify_field_select & MLX5_MODIFY_MACSEC_BITMASK_EPN_OVERLAP) || + !(modify_field_select & MLX5_MODIFY_MACSEC_BITMASK_EPN_MSB)) { + mlx5_core_dbg(mdev, "MACsec object field is not modifiable (Object id %d)\n", + macsec_id); + return -EOPNOTSUPP; + } + + obj = MLX5_ADDR_OF(modify_macsec_obj_in, in, macsec_object); + MLX5_SET64(macsec_offload_obj, obj, modify_field_select, + MLX5_MODIFY_MACSEC_BITMASK_EPN_OVERLAP | MLX5_MODIFY_MACSEC_BITMASK_EPN_MSB); + MLX5_SET(macsec_offload_obj, obj, epn_msb, attrs->epn_state.epn_msb); + MLX5_SET(macsec_offload_obj, obj, epn_overlap, attrs->epn_state.overlap); + + /* General object fields set */ + MLX5_SET(general_obj_in_cmd_hdr, in, opcode, MLX5_CMD_OP_MODIFY_GENERAL_OBJECT); + + return mlx5_cmd_exec(mdev, in, sizeof(in), out, sizeof(out)); +} + +static void macsec_aso_build_ctrl(struct mlx5e_macsec_aso *aso, + struct mlx5_wqe_aso_ctrl_seg *aso_ctrl, + struct mlx5e_macsec_aso_in *in) +{ + struct mlx5_aso_ctrl_param param = {}; + + param.data_mask_mode = MLX5_ASO_DATA_MASK_MODE_BITWISE_64BIT; + param.condition_0_operand = MLX5_ASO_ALWAYS_TRUE; + param.condition_1_operand = MLX5_ASO_ALWAYS_TRUE; + if (in->mode == MLX5_MACSEC_EPN) { + param.data_offset = MLX5_MACSEC_ASO_REMOVE_FLOW_PKT_CNT_OFFSET; + param.bitwise_data = BIT_ULL(54); + param.data_mask = param.bitwise_data; + } + macsec_aso_build_wqe_ctrl_seg(aso, aso_ctrl, ¶m); +} + +static int macsec_aso_set_arm_event(struct mlx5_core_dev *mdev, struct mlx5e_macsec *macsec, + struct mlx5e_macsec_aso_in *in) +{ + struct mlx5e_macsec_aso *aso; + struct mlx5_aso_wqe *aso_wqe; + struct mlx5_aso *maso; + int err; + + aso = &macsec->aso; + maso = aso->maso; + + mutex_lock(&aso->aso_lock); + aso_wqe = mlx5_aso_get_wqe(maso); + mlx5_aso_build_wqe(maso, MLX5_MACSEC_ASO_DS_CNT, aso_wqe, in->obj_id, + MLX5_ACCESS_ASO_OPC_MOD_MACSEC); + macsec_aso_build_ctrl(aso, &aso_wqe->aso_ctrl, in); + mlx5_aso_post_wqe(maso, false, &aso_wqe->ctrl); + err = mlx5_aso_poll_cq(maso, false, 10); + mutex_unlock(&aso->aso_lock); + + return err; +} + +static int macsec_aso_query(struct mlx5_core_dev *mdev, struct mlx5e_macsec *macsec, + struct mlx5e_macsec_aso_in *in, struct mlx5e_macsec_aso_out *out) +{ + struct mlx5e_macsec_aso *aso; + struct mlx5_aso_wqe *aso_wqe; + struct mlx5_aso *maso; + int err; + + aso = &macsec->aso; + maso = aso->maso; + + mutex_lock(&aso->aso_lock); + + aso_wqe = mlx5_aso_get_wqe(maso); + mlx5_aso_build_wqe(maso, MLX5_MACSEC_ASO_DS_CNT, aso_wqe, in->obj_id, + MLX5_ACCESS_ASO_OPC_MOD_MACSEC); + macsec_aso_build_wqe_ctrl_seg(aso, &aso_wqe->aso_ctrl, NULL); + + mlx5_aso_post_wqe(maso, false, &aso_wqe->ctrl); + err = mlx5_aso_poll_cq(maso, false, 10); + if (err) + goto err_out; + + if (MLX5_GET(macsec_aso, aso->umr->ctx, epn_event_arm)) + out->event_arm |= MLX5E_ASO_EPN_ARM; + + out->mode_param = MLX5_GET(macsec_aso, aso->umr->ctx, mode_parameter); + +err_out: + mutex_unlock(&aso->aso_lock); + return err; +} + +static struct mlx5e_macsec_sa *get_macsec_tx_sa_from_obj_id(const struct mlx5e_macsec *macsec, + const u32 obj_id) +{ + const struct list_head *device_list; + struct mlx5e_macsec_sa *macsec_sa; + struct mlx5e_macsec_device *iter; + int i; + + device_list = &macsec->macsec_device_list_head; + + list_for_each_entry(iter, device_list, macsec_device_list_element) { + for (i = 0; i < MACSEC_NUM_AN; ++i) { + macsec_sa = iter->tx_sa[i]; + if (!macsec_sa || !macsec_sa->active) + continue; + if (macsec_sa->macsec_obj_id == obj_id) + return macsec_sa; + } + } + + return NULL; +} + +static struct mlx5e_macsec_sa *get_macsec_rx_sa_from_obj_id(const struct mlx5e_macsec *macsec, + const u32 obj_id) +{ + const struct list_head *device_list, *sc_list; + struct mlx5e_macsec_rx_sc *mlx5e_rx_sc; + struct mlx5e_macsec_sa *macsec_sa; + struct mlx5e_macsec_device *iter; + int i; + + device_list = &macsec->macsec_device_list_head; + + list_for_each_entry(iter, device_list, macsec_device_list_element) { + sc_list = &iter->macsec_rx_sc_list_head; + list_for_each_entry(mlx5e_rx_sc, sc_list, rx_sc_list_element) { + for (i = 0; i < MACSEC_NUM_AN; ++i) { + macsec_sa = mlx5e_rx_sc->rx_sa[i]; + if (!macsec_sa || !macsec_sa->active) + continue; + if (macsec_sa->macsec_obj_id == obj_id) + return macsec_sa; + } + } + } + + return NULL; +} + +static void macsec_epn_update(struct mlx5e_macsec *macsec, struct mlx5_core_dev *mdev, + struct mlx5e_macsec_sa *sa, u32 obj_id, u32 mode_param) +{ + struct mlx5_macsec_obj_attrs attrs = {}; + struct mlx5e_macsec_aso_in in = {}; + + /* When the bottom of the replay protection window (mode_param) crosses 2^31 (half sequence + * number wraparound) hence mode_param > MLX5_MACSEC_EPN_SCOPE_MID the SW should update the + * esn_overlap to OLD (1). + * When the bottom of the replay protection window (mode_param) crosses 2^32 (full sequence + * number wraparound) hence mode_param < MLX5_MACSEC_EPN_SCOPE_MID since it did a + * wraparound, the SW should update the esn_overlap to NEW (0), and increment the esn_msb. + */ + + if (mode_param < MLX5_MACSEC_EPN_SCOPE_MID) { + sa->epn_state.epn_msb++; + sa->epn_state.overlap = 0; + } else { + sa->epn_state.overlap = 1; + } + + macsec_build_accel_attrs(sa, &attrs); + mlx5e_macsec_modify_obj(mdev, &attrs, obj_id); + + /* Re-set EPN arm event */ + in.obj_id = obj_id; + in.mode = MLX5_MACSEC_EPN; + macsec_aso_set_arm_event(mdev, macsec, &in); +} + +static void macsec_async_event(struct work_struct *work) +{ + struct mlx5e_macsec_async_work *async_work; + struct mlx5e_macsec_aso_out out = {}; + struct mlx5e_macsec_aso_in in = {}; + struct mlx5e_macsec_sa *macsec_sa; + struct mlx5e_macsec *macsec; + struct mlx5_core_dev *mdev; + u32 obj_id; + + async_work = container_of(work, struct mlx5e_macsec_async_work, work); + macsec = async_work->macsec; + mdev = async_work->mdev; + obj_id = async_work->obj_id; + macsec_sa = get_macsec_tx_sa_from_obj_id(macsec, obj_id); + if (!macsec_sa) { + macsec_sa = get_macsec_rx_sa_from_obj_id(macsec, obj_id); + if (!macsec_sa) { + mlx5_core_dbg(mdev, "MACsec SA is not found (SA object id %d)\n", obj_id); + goto out_async_work; + } + } + + /* Query MACsec ASO context */ + in.obj_id = obj_id; + macsec_aso_query(mdev, macsec, &in, &out); + + /* EPN case */ + if (macsec_sa->epn_state.epn_enabled && !(out.event_arm & MLX5E_ASO_EPN_ARM)) + macsec_epn_update(macsec, mdev, macsec_sa, obj_id, out.mode_param); + +out_async_work: + kfree(async_work); +} + +static int macsec_obj_change_event(struct notifier_block *nb, unsigned long event, void *data) +{ + struct mlx5e_macsec *macsec = container_of(nb, struct mlx5e_macsec, nb); + struct mlx5e_macsec_async_work *async_work; + struct mlx5_eqe_obj_change *obj_change; + struct mlx5_eqe *eqe = data; + u16 obj_type; + u32 obj_id; + + if (event != MLX5_EVENT_TYPE_OBJECT_CHANGE) + return NOTIFY_DONE; + + obj_change = &eqe->data.obj_change; + obj_type = be16_to_cpu(obj_change->obj_type); + obj_id = be32_to_cpu(obj_change->obj_id); + + if (obj_type != MLX5_GENERAL_OBJECT_TYPES_MACSEC) + return NOTIFY_DONE; + + async_work = kzalloc(sizeof(*async_work), GFP_ATOMIC); + if (!async_work) + return NOTIFY_DONE; + + async_work->macsec = macsec; + async_work->mdev = macsec->mdev; + async_work->obj_id = obj_id; + + INIT_WORK(&async_work->work, macsec_async_event); + + WARN_ON(!queue_work(macsec->wq, &async_work->work)); + + return NOTIFY_OK; +} + +static int mlx5e_macsec_aso_init(struct mlx5e_macsec_aso *aso, struct mlx5_core_dev *mdev) +{ + struct mlx5_aso *maso; + int err; + + err = mlx5_core_alloc_pd(mdev, &aso->pdn); + if (err) { + mlx5_core_err(mdev, + "MACsec offload: Failed to alloc pd for MACsec ASO, err=%d\n", + err); + return err; + } + + maso = mlx5_aso_create(mdev, aso->pdn); + if (IS_ERR(maso)) { + err = PTR_ERR(maso); + goto err_aso; + } + + err = mlx5e_macsec_aso_reg_mr(mdev, aso); + if (err) + goto err_aso_reg; + + mutex_init(&aso->aso_lock); + + aso->maso = maso; + + return 0; + +err_aso_reg: + mlx5_aso_destroy(maso); +err_aso: + mlx5_core_dealloc_pd(mdev, aso->pdn); + return err; +} + +static void mlx5e_macsec_aso_cleanup(struct mlx5e_macsec_aso *aso, struct mlx5_core_dev *mdev) +{ + if (!aso) + return; + + mlx5e_macsec_aso_dereg_mr(mdev, aso); + + mlx5_aso_destroy(aso->maso); + + mlx5_core_dealloc_pd(mdev, aso->pdn); +} + +bool mlx5e_is_macsec_device(const struct mlx5_core_dev *mdev) +{ + if (!(MLX5_CAP_GEN_64(mdev, general_obj_types) & + MLX5_GENERAL_OBJ_TYPES_CAP_MACSEC_OFFLOAD)) + return false; + + if (!MLX5_CAP_GEN(mdev, log_max_dek)) + return false; + + if (!MLX5_CAP_MACSEC(mdev, log_max_macsec_offload)) + return false; + + if (!MLX5_CAP_FLOWTABLE_NIC_RX(mdev, macsec_decrypt) || + !MLX5_CAP_FLOWTABLE_NIC_RX(mdev, reformat_remove_macsec)) + return false; + + if (!MLX5_CAP_FLOWTABLE_NIC_TX(mdev, macsec_encrypt) || + !MLX5_CAP_FLOWTABLE_NIC_TX(mdev, reformat_add_macsec)) + return false; + + if (!MLX5_CAP_MACSEC(mdev, macsec_crypto_esp_aes_gcm_128_encrypt) && + !MLX5_CAP_MACSEC(mdev, macsec_crypto_esp_aes_gcm_256_encrypt)) + return false; + + if (!MLX5_CAP_MACSEC(mdev, macsec_crypto_esp_aes_gcm_128_decrypt) && + !MLX5_CAP_MACSEC(mdev, macsec_crypto_esp_aes_gcm_256_decrypt)) + return false; + + return true; +} + +void mlx5e_macsec_get_stats_fill(struct mlx5e_macsec *macsec, void *macsec_stats) +{ + mlx5e_macsec_fs_get_stats_fill(macsec->macsec_fs, macsec_stats); +} + +struct mlx5e_macsec_stats *mlx5e_macsec_get_stats(struct mlx5e_macsec *macsec) +{ + if (!macsec) + return NULL; + + return &macsec->stats; +} + +static const struct macsec_ops macsec_offload_ops = { + .mdo_add_txsa = mlx5e_macsec_add_txsa, + .mdo_upd_txsa = mlx5e_macsec_upd_txsa, + .mdo_del_txsa = mlx5e_macsec_del_txsa, + .mdo_add_rxsc = mlx5e_macsec_add_rxsc, + .mdo_upd_rxsc = mlx5e_macsec_upd_rxsc, + .mdo_del_rxsc = mlx5e_macsec_del_rxsc, + .mdo_add_rxsa = mlx5e_macsec_add_rxsa, + .mdo_upd_rxsa = mlx5e_macsec_upd_rxsa, + .mdo_del_rxsa = mlx5e_macsec_del_rxsa, + .mdo_add_secy = mlx5e_macsec_add_secy, + .mdo_upd_secy = mlx5e_macsec_upd_secy, + .mdo_del_secy = mlx5e_macsec_del_secy, +}; + +bool mlx5e_macsec_handle_tx_skb(struct mlx5e_macsec *macsec, struct sk_buff *skb) +{ + struct metadata_dst *md_dst = skb_metadata_dst(skb); + u32 fs_id; + + fs_id = mlx5e_macsec_get_sa_from_hashtable(&macsec->sci_hash, &md_dst->u.macsec_info.sci); + if (!fs_id) + goto err_out; + + return true; + +err_out: + dev_kfree_skb_any(skb); + return false; +} + +void mlx5e_macsec_tx_build_eseg(struct mlx5e_macsec *macsec, + struct sk_buff *skb, + struct mlx5_wqe_eth_seg *eseg) +{ + struct metadata_dst *md_dst = skb_metadata_dst(skb); + u32 fs_id; + + fs_id = mlx5e_macsec_get_sa_from_hashtable(&macsec->sci_hash, &md_dst->u.macsec_info.sci); + if (!fs_id) + return; + + eseg->flow_table_metadata = cpu_to_be32(MLX5_ETH_WQE_FT_META_MACSEC | fs_id << 2); +} + +void mlx5e_macsec_offload_handle_rx_skb(struct net_device *netdev, + struct sk_buff *skb, + struct mlx5_cqe64 *cqe) +{ + struct mlx5e_macsec_rx_sc_xarray_element *sc_xarray_element; + u32 macsec_meta_data = be32_to_cpu(cqe->ft_metadata); + struct mlx5e_priv *priv = netdev_priv(netdev); + struct mlx5e_macsec_rx_sc *rx_sc; + struct mlx5e_macsec *macsec; + u32 fs_id; + + macsec = priv->macsec; + if (!macsec) + return; + + fs_id = MLX5_MACSEC_METADATA_HANDLE(macsec_meta_data); + + rcu_read_lock(); + sc_xarray_element = xa_load(&macsec->sc_xarray, fs_id); + rx_sc = sc_xarray_element->rx_sc; + if (rx_sc) { + dst_hold(&rx_sc->md_dst->dst); + skb_dst_set(skb, &rx_sc->md_dst->dst); + } + + rcu_read_unlock(); +} + +void mlx5e_macsec_build_netdev(struct mlx5e_priv *priv) +{ + struct net_device *netdev = priv->netdev; + + if (!mlx5e_is_macsec_device(priv->mdev)) + return; + + /* Enable MACsec */ + mlx5_core_dbg(priv->mdev, "mlx5e: MACsec acceleration enabled\n"); + netdev->macsec_ops = &macsec_offload_ops; + netdev->features |= NETIF_F_HW_MACSEC; + netif_keep_dst(netdev); +} + +int mlx5e_macsec_init(struct mlx5e_priv *priv) +{ + struct mlx5_core_dev *mdev = priv->mdev; + struct mlx5e_macsec *macsec = NULL; + struct mlx5e_macsec_fs *macsec_fs; + int err; + + if (!mlx5e_is_macsec_device(priv->mdev)) { + mlx5_core_dbg(mdev, "Not a MACsec offload device\n"); + return 0; + } + + macsec = kzalloc(sizeof(*macsec), GFP_KERNEL); + if (!macsec) + return -ENOMEM; + + INIT_LIST_HEAD(&macsec->macsec_device_list_head); + mutex_init(&macsec->lock); + + err = rhashtable_init(&macsec->sci_hash, &rhash_sci); + if (err) { + mlx5_core_err(mdev, "MACsec offload: Failed to init SCI hash table, err=%d\n", + err); + goto err_hash; + } + + err = mlx5e_macsec_aso_init(&macsec->aso, priv->mdev); + if (err) { + mlx5_core_err(mdev, "MACsec offload: Failed to init aso, err=%d\n", err); + goto err_aso; + } + + macsec->wq = alloc_ordered_workqueue("mlx5e_macsec_%s", 0, priv->netdev->name); + if (!macsec->wq) { + err = -ENOMEM; + goto err_wq; + } + + xa_init_flags(&macsec->sc_xarray, XA_FLAGS_ALLOC1); + + priv->macsec = macsec; + + macsec->mdev = mdev; + + macsec_fs = mlx5e_macsec_fs_init(mdev, priv->netdev); + if (!macsec_fs) { + err = -ENOMEM; + goto err_out; + } + + macsec->macsec_fs = macsec_fs; + + macsec->nb.notifier_call = macsec_obj_change_event; + mlx5_notifier_register(mdev, &macsec->nb); + + mlx5_core_dbg(mdev, "MACsec attached to netdevice\n"); + + return 0; + +err_out: + destroy_workqueue(macsec->wq); +err_wq: + mlx5e_macsec_aso_cleanup(&macsec->aso, priv->mdev); +err_aso: + rhashtable_destroy(&macsec->sci_hash); +err_hash: + kfree(macsec); + priv->macsec = NULL; + return err; +} + +void mlx5e_macsec_cleanup(struct mlx5e_priv *priv) +{ + struct mlx5e_macsec *macsec = priv->macsec; + struct mlx5_core_dev *mdev = macsec->mdev; + + if (!macsec) + return; + + mlx5_notifier_unregister(mdev, &macsec->nb); + + mlx5e_macsec_fs_cleanup(macsec->macsec_fs); + + /* Cleanup workqueue */ + destroy_workqueue(macsec->wq); + + mlx5e_macsec_aso_cleanup(&macsec->aso, mdev); + + priv->macsec = NULL; + + rhashtable_destroy(&macsec->sci_hash); + + mutex_destroy(&macsec->lock); + + kfree(macsec); +} diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/macsec.h b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/macsec.h new file mode 100644 index 000000000000..d580b4a91253 --- /dev/null +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/macsec.h @@ -0,0 +1,71 @@ +/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB */ +/* Copyright (c) 2022, NVIDIA CORPORATION & AFFILIATES. All rights reserved. */ + +#ifndef __MLX5_EN_ACCEL_MACSEC_H__ +#define __MLX5_EN_ACCEL_MACSEC_H__ + +#ifdef CONFIG_MLX5_EN_MACSEC + +#include <linux/mlx5/driver.h> +#include <net/macsec.h> +#include <net/dst_metadata.h> + +/* Bit31 - 30: MACsec marker, Bit3-0: MACsec id */ +#define MLX5_MACSEC_METADATA_MARKER(metadata) ((((metadata) >> 30) & 0x3) == 0x1) +#define MLX5_MACSEC_METADATA_HANDLE(metadata) ((metadata) & GENMASK(3, 0)) + +struct mlx5e_priv; +struct mlx5e_macsec; + +struct mlx5e_macsec_stats { + u64 macsec_rx_pkts; + u64 macsec_rx_bytes; + u64 macsec_rx_pkts_drop; + u64 macsec_rx_bytes_drop; + u64 macsec_tx_pkts; + u64 macsec_tx_bytes; + u64 macsec_tx_pkts_drop; + u64 macsec_tx_bytes_drop; +}; + +void mlx5e_macsec_build_netdev(struct mlx5e_priv *priv); +int mlx5e_macsec_init(struct mlx5e_priv *priv); +void mlx5e_macsec_cleanup(struct mlx5e_priv *priv); +bool mlx5e_macsec_handle_tx_skb(struct mlx5e_macsec *macsec, struct sk_buff *skb); +void mlx5e_macsec_tx_build_eseg(struct mlx5e_macsec *macsec, + struct sk_buff *skb, + struct mlx5_wqe_eth_seg *eseg); + +static inline bool mlx5e_macsec_skb_is_offload(struct sk_buff *skb) +{ + struct metadata_dst *md_dst = skb_metadata_dst(skb); + + return md_dst && (md_dst->type == METADATA_MACSEC); +} + +static inline bool mlx5e_macsec_is_rx_flow(struct mlx5_cqe64 *cqe) +{ + return MLX5_MACSEC_METADATA_MARKER(be32_to_cpu(cqe->ft_metadata)); +} + +void mlx5e_macsec_offload_handle_rx_skb(struct net_device *netdev, struct sk_buff *skb, + struct mlx5_cqe64 *cqe); +bool mlx5e_is_macsec_device(const struct mlx5_core_dev *mdev); +void mlx5e_macsec_get_stats_fill(struct mlx5e_macsec *macsec, void *macsec_stats); +struct mlx5e_macsec_stats *mlx5e_macsec_get_stats(struct mlx5e_macsec *macsec); + +#else + +static inline void mlx5e_macsec_build_netdev(struct mlx5e_priv *priv) {} +static inline int mlx5e_macsec_init(struct mlx5e_priv *priv) { return 0; } +static inline void mlx5e_macsec_cleanup(struct mlx5e_priv *priv) {} +static inline bool mlx5e_macsec_skb_is_offload(struct sk_buff *skb) { return false; } +static inline bool mlx5e_macsec_is_rx_flow(struct mlx5_cqe64 *cqe) { return false; } +static inline void mlx5e_macsec_offload_handle_rx_skb(struct net_device *netdev, + struct sk_buff *skb, + struct mlx5_cqe64 *cqe) +{} +static inline bool mlx5e_is_macsec_device(const struct mlx5_core_dev *mdev) { return false; } +#endif /* CONFIG_MLX5_EN_MACSEC */ + +#endif /* __MLX5_ACCEL_EN_MACSEC_H__ */ diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/macsec_fs.c b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/macsec_fs.c new file mode 100644 index 000000000000..13dc628b988a --- /dev/null +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/macsec_fs.c @@ -0,0 +1,1384 @@ +// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB +/* Copyright (c) 2022, NVIDIA CORPORATION & AFFILIATES. All rights reserved. */ + +#include <net/macsec.h> +#include <linux/netdevice.h> +#include <linux/mlx5/qp.h> +#include "fs_core.h" +#include "en/fs.h" +#include "en_accel/macsec_fs.h" +#include "mlx5_core.h" + +/* MACsec TX flow steering */ +#define CRYPTO_NUM_MAXSEC_FTE BIT(15) +#define CRYPTO_TABLE_DEFAULT_RULE_GROUP_SIZE 1 + +#define TX_CRYPTO_TABLE_LEVEL 0 +#define TX_CRYPTO_TABLE_NUM_GROUPS 3 +#define TX_CRYPTO_TABLE_MKE_GROUP_SIZE 1 +#define TX_CRYPTO_TABLE_SA_GROUP_SIZE \ + (CRYPTO_NUM_MAXSEC_FTE - (TX_CRYPTO_TABLE_MKE_GROUP_SIZE + \ + CRYPTO_TABLE_DEFAULT_RULE_GROUP_SIZE)) +#define TX_CHECK_TABLE_LEVEL 1 +#define TX_CHECK_TABLE_NUM_FTE 2 +#define RX_CRYPTO_TABLE_LEVEL 0 +#define RX_CHECK_TABLE_LEVEL 1 +#define RX_CHECK_TABLE_NUM_FTE 3 +#define RX_CRYPTO_TABLE_NUM_GROUPS 3 +#define RX_CRYPTO_TABLE_SA_RULE_WITH_SCI_GROUP_SIZE \ + ((CRYPTO_NUM_MAXSEC_FTE - CRYPTO_TABLE_DEFAULT_RULE_GROUP_SIZE) / 2) +#define RX_CRYPTO_TABLE_SA_RULE_WITHOUT_SCI_GROUP_SIZE \ + (CRYPTO_NUM_MAXSEC_FTE - RX_CRYPTO_TABLE_SA_RULE_WITH_SCI_GROUP_SIZE) +#define RX_NUM_OF_RULES_PER_SA 2 + +#define MLX5_MACSEC_TAG_LEN 8 /* SecTAG length with ethertype and without the optional SCI */ +#define MLX5_MACSEC_SECTAG_TCI_AN_FIELD_BITMASK 0x23 +#define MLX5_MACSEC_SECTAG_TCI_AN_FIELD_OFFSET 0x8 +#define MLX5_MACSEC_SECTAG_TCI_SC_FIELD_OFFSET 0x5 +#define MLX5_MACSEC_SECTAG_TCI_SC_FIELD_BIT (0x1 << MLX5_MACSEC_SECTAG_TCI_SC_FIELD_OFFSET) +#define MLX5_SECTAG_HEADER_SIZE_WITHOUT_SCI 0x8 +#define MLX5_SECTAG_HEADER_SIZE_WITH_SCI (MLX5_SECTAG_HEADER_SIZE_WITHOUT_SCI + MACSEC_SCI_LEN) + +/* MACsec RX flow steering */ +#define MLX5_ETH_WQE_FT_META_MACSEC_MASK 0x3E + +struct mlx5_sectag_header { + __be16 ethertype; + u8 tci_an; + u8 sl; + u32 pn; + u8 sci[MACSEC_SCI_LEN]; /* optional */ +} __packed; + +struct mlx5e_macsec_tx_rule { + struct mlx5_flow_handle *rule; + struct mlx5_pkt_reformat *pkt_reformat; + u32 fs_id; +}; + +struct mlx5e_macsec_tables { + struct mlx5e_flow_table ft_crypto; + struct mlx5_flow_handle *crypto_miss_rule; + + struct mlx5_flow_table *ft_check; + struct mlx5_flow_group *ft_check_group; + struct mlx5_fc *check_miss_rule_counter; + struct mlx5_flow_handle *check_miss_rule; + struct mlx5_fc *check_rule_counter; + + u32 refcnt; +}; + +struct mlx5e_macsec_tx { + struct mlx5_flow_handle *crypto_mke_rule; + struct mlx5_flow_handle *check_rule; + + struct ida tx_halloc; + + struct mlx5e_macsec_tables tables; +}; + +struct mlx5e_macsec_rx_rule { + struct mlx5_flow_handle *rule[RX_NUM_OF_RULES_PER_SA]; + struct mlx5_modify_hdr *meta_modhdr; +}; + +struct mlx5e_macsec_rx { + struct mlx5_flow_handle *check_rule[2]; + struct mlx5_pkt_reformat *check_rule_pkt_reformat[2]; + + struct mlx5e_macsec_tables tables; +}; + +union mlx5e_macsec_rule { + struct mlx5e_macsec_tx_rule tx_rule; + struct mlx5e_macsec_rx_rule rx_rule; +}; + +struct mlx5e_macsec_fs { + struct mlx5_core_dev *mdev; + struct net_device *netdev; + struct mlx5e_macsec_tx *tx_fs; + struct mlx5e_macsec_rx *rx_fs; +}; + +static void macsec_fs_tx_destroy(struct mlx5e_macsec_fs *macsec_fs) +{ + struct mlx5e_macsec_tx *tx_fs = macsec_fs->tx_fs; + struct mlx5e_macsec_tables *tx_tables; + + tx_tables = &tx_fs->tables; + + /* Tx check table */ + if (tx_fs->check_rule) { + mlx5_del_flow_rules(tx_fs->check_rule); + tx_fs->check_rule = NULL; + } + + if (tx_tables->check_miss_rule) { + mlx5_del_flow_rules(tx_tables->check_miss_rule); + tx_tables->check_miss_rule = NULL; + } + + if (tx_tables->ft_check_group) { + mlx5_destroy_flow_group(tx_tables->ft_check_group); + tx_tables->ft_check_group = NULL; + } + + if (tx_tables->ft_check) { + mlx5_destroy_flow_table(tx_tables->ft_check); + tx_tables->ft_check = NULL; + } + + /* Tx crypto table */ + if (tx_fs->crypto_mke_rule) { + mlx5_del_flow_rules(tx_fs->crypto_mke_rule); + tx_fs->crypto_mke_rule = NULL; + } + + if (tx_tables->crypto_miss_rule) { + mlx5_del_flow_rules(tx_tables->crypto_miss_rule); + tx_tables->crypto_miss_rule = NULL; + } + + mlx5e_destroy_flow_table(&tx_tables->ft_crypto); +} + +static int macsec_fs_tx_create_crypto_table_groups(struct mlx5e_flow_table *ft) +{ + int inlen = MLX5_ST_SZ_BYTES(create_flow_group_in); + int mclen = MLX5_ST_SZ_BYTES(fte_match_param); + int ix = 0; + u32 *in; + int err; + u8 *mc; + + ft->g = kcalloc(TX_CRYPTO_TABLE_NUM_GROUPS, sizeof(*ft->g), GFP_KERNEL); + if (!ft->g) + return -ENOMEM; + in = kvzalloc(inlen, GFP_KERNEL); + + if (!in) { + kfree(ft->g); + return -ENOMEM; + } + + mc = MLX5_ADDR_OF(create_flow_group_in, in, match_criteria); + + /* Flow Group for MKE match */ + MLX5_SET_CFG(in, match_criteria_enable, MLX5_MATCH_OUTER_HEADERS); + MLX5_SET_TO_ONES(fte_match_param, mc, outer_headers.ethertype); + + MLX5_SET_CFG(in, start_flow_index, ix); + ix += TX_CRYPTO_TABLE_MKE_GROUP_SIZE; + MLX5_SET_CFG(in, end_flow_index, ix - 1); + ft->g[ft->num_groups] = mlx5_create_flow_group(ft->t, in); + if (IS_ERR(ft->g[ft->num_groups])) + goto err; + ft->num_groups++; + + /* Flow Group for SA rules */ + memset(in, 0, inlen); + memset(mc, 0, mclen); + MLX5_SET_CFG(in, match_criteria_enable, MLX5_MATCH_MISC_PARAMETERS_2); + MLX5_SET(fte_match_param, mc, misc_parameters_2.metadata_reg_a, + MLX5_ETH_WQE_FT_META_MACSEC_MASK); + + MLX5_SET_CFG(in, start_flow_index, ix); + ix += TX_CRYPTO_TABLE_SA_GROUP_SIZE; + MLX5_SET_CFG(in, end_flow_index, ix - 1); + ft->g[ft->num_groups] = mlx5_create_flow_group(ft->t, in); + if (IS_ERR(ft->g[ft->num_groups])) + goto err; + ft->num_groups++; + + /* Flow Group for l2 traps */ + memset(in, 0, inlen); + memset(mc, 0, mclen); + MLX5_SET_CFG(in, start_flow_index, ix); + ix += CRYPTO_TABLE_DEFAULT_RULE_GROUP_SIZE; + MLX5_SET_CFG(in, end_flow_index, ix - 1); + ft->g[ft->num_groups] = mlx5_create_flow_group(ft->t, in); + if (IS_ERR(ft->g[ft->num_groups])) + goto err; + ft->num_groups++; + + kvfree(in); + return 0; + +err: + err = PTR_ERR(ft->g[ft->num_groups]); + ft->g[ft->num_groups] = NULL; + kvfree(in); + + return err; +} + +static struct mlx5_flow_table + *macsec_fs_auto_group_table_create(struct mlx5_flow_namespace *ns, int flags, + int level, int max_fte) +{ + struct mlx5_flow_table_attr ft_attr = {}; + struct mlx5_flow_table *fdb = NULL; + + /* reserve entry for the match all miss group and rule */ + ft_attr.autogroup.num_reserved_entries = 1; + ft_attr.autogroup.max_num_groups = 1; + ft_attr.prio = 0; + ft_attr.flags = flags; + ft_attr.level = level; + ft_attr.max_fte = max_fte; + + fdb = mlx5_create_auto_grouped_flow_table(ns, &ft_attr); + + return fdb; +} + +static int macsec_fs_tx_create(struct mlx5e_macsec_fs *macsec_fs) +{ + int inlen = MLX5_ST_SZ_BYTES(create_flow_group_in); + struct mlx5e_macsec_tx *tx_fs = macsec_fs->tx_fs; + struct net_device *netdev = macsec_fs->netdev; + struct mlx5_flow_table_attr ft_attr = {}; + struct mlx5_flow_destination dest = {}; + struct mlx5e_macsec_tables *tx_tables; + struct mlx5_flow_act flow_act = {}; + struct mlx5e_flow_table *ft_crypto; + struct mlx5_flow_table *flow_table; + struct mlx5_flow_group *flow_group; + struct mlx5_flow_namespace *ns; + struct mlx5_flow_handle *rule; + struct mlx5_flow_spec *spec; + u32 *flow_group_in; + int err = 0; + + ns = mlx5_get_flow_namespace(macsec_fs->mdev, MLX5_FLOW_NAMESPACE_EGRESS_MACSEC); + if (!ns) + return -ENOMEM; + + spec = kvzalloc(sizeof(*spec), GFP_KERNEL); + if (!spec) + return -ENOMEM; + + flow_group_in = kvzalloc(inlen, GFP_KERNEL); + if (!flow_group_in) + goto out_spec; + + tx_tables = &tx_fs->tables; + ft_crypto = &tx_tables->ft_crypto; + + /* Tx crypto table */ + ft_attr.flags = MLX5_FLOW_TABLE_TUNNEL_EN_REFORMAT; + ft_attr.level = TX_CRYPTO_TABLE_LEVEL; + ft_attr.max_fte = CRYPTO_NUM_MAXSEC_FTE; + + flow_table = mlx5_create_flow_table(ns, &ft_attr); + if (IS_ERR(flow_table)) { + err = PTR_ERR(flow_table); + netdev_err(netdev, "Failed to create MACsec Tx crypto table err(%d)\n", err); + goto out_flow_group; + } + ft_crypto->t = flow_table; + + /* Tx crypto table groups */ + err = macsec_fs_tx_create_crypto_table_groups(ft_crypto); + if (err) { + netdev_err(netdev, + "Failed to create default flow group for MACsec Tx crypto table err(%d)\n", + err); + goto err; + } + + /* Tx crypto table MKE rule - MKE packets shouldn't be offloaded */ + memset(&flow_act, 0, sizeof(flow_act)); + memset(spec, 0, sizeof(*spec)); + spec->match_criteria_enable = MLX5_MATCH_OUTER_HEADERS; + + MLX5_SET_TO_ONES(fte_match_param, spec->match_criteria, outer_headers.ethertype); + MLX5_SET(fte_match_param, spec->match_value, outer_headers.ethertype, ETH_P_PAE); + flow_act.action = MLX5_FLOW_CONTEXT_ACTION_ALLOW; + + rule = mlx5_add_flow_rules(ft_crypto->t, spec, &flow_act, NULL, 0); + if (IS_ERR(rule)) { + err = PTR_ERR(rule); + netdev_err(netdev, "Failed to add MACsec TX MKE rule, err=%d\n", err); + goto err; + } + tx_fs->crypto_mke_rule = rule; + + /* Tx crypto table Default miss rule */ + memset(&flow_act, 0, sizeof(flow_act)); + flow_act.action = MLX5_FLOW_CONTEXT_ACTION_ALLOW; + rule = mlx5_add_flow_rules(ft_crypto->t, NULL, &flow_act, NULL, 0); + if (IS_ERR(rule)) { + err = PTR_ERR(rule); + netdev_err(netdev, "Failed to add MACsec Tx table default miss rule %d\n", err); + goto err; + } + tx_tables->crypto_miss_rule = rule; + + /* Tx check table */ + flow_table = macsec_fs_auto_group_table_create(ns, 0, TX_CHECK_TABLE_LEVEL, + TX_CHECK_TABLE_NUM_FTE); + if (IS_ERR(flow_table)) { + err = PTR_ERR(flow_table); + netdev_err(netdev, "fail to create MACsec TX check table, err(%d)\n", err); + goto err; + } + tx_tables->ft_check = flow_table; + + /* Tx check table Default miss group/rule */ + memset(flow_group_in, 0, inlen); + MLX5_SET(create_flow_group_in, flow_group_in, start_flow_index, flow_table->max_fte - 1); + MLX5_SET(create_flow_group_in, flow_group_in, end_flow_index, flow_table->max_fte - 1); + flow_group = mlx5_create_flow_group(tx_tables->ft_check, flow_group_in); + if (IS_ERR(flow_group)) { + err = PTR_ERR(flow_group); + netdev_err(netdev, + "Failed to create default flow group for MACsec Tx crypto table err(%d)\n", + err); + goto err; + } + tx_tables->ft_check_group = flow_group; + + /* Tx check table default drop rule */ + memset(&dest, 0, sizeof(struct mlx5_flow_destination)); + memset(&flow_act, 0, sizeof(flow_act)); + dest.type = MLX5_FLOW_DESTINATION_TYPE_COUNTER; + dest.counter_id = mlx5_fc_id(tx_tables->check_miss_rule_counter); + flow_act.action = MLX5_FLOW_CONTEXT_ACTION_DROP | MLX5_FLOW_CONTEXT_ACTION_COUNT; + rule = mlx5_add_flow_rules(tx_tables->ft_check, NULL, &flow_act, &dest, 1); + if (IS_ERR(rule)) { + err = PTR_ERR(rule); + netdev_err(netdev, "Failed to added MACsec tx check drop rule, err(%d)\n", err); + goto err; + } + tx_tables->check_miss_rule = rule; + + /* Tx check table rule */ + memset(spec, 0, sizeof(struct mlx5_flow_spec)); + memset(&dest, 0, sizeof(struct mlx5_flow_destination)); + memset(&flow_act, 0, sizeof(flow_act)); + + MLX5_SET_TO_ONES(fte_match_param, spec->match_criteria, misc_parameters_2.metadata_reg_c_4); + MLX5_SET(fte_match_param, spec->match_value, misc_parameters_2.metadata_reg_c_4, 0); + spec->match_criteria_enable = MLX5_MATCH_MISC_PARAMETERS_2; + + flow_act.flags = FLOW_ACT_NO_APPEND; + flow_act.action = MLX5_FLOW_CONTEXT_ACTION_ALLOW | MLX5_FLOW_CONTEXT_ACTION_COUNT; + dest.type = MLX5_FLOW_DESTINATION_TYPE_COUNTER; + dest.counter_id = mlx5_fc_id(tx_tables->check_rule_counter); + rule = mlx5_add_flow_rules(tx_tables->ft_check, spec, &flow_act, &dest, 1); + if (IS_ERR(rule)) { + err = PTR_ERR(rule); + netdev_err(netdev, "Failed to add MACsec check rule, err=%d\n", err); + goto err; + } + tx_fs->check_rule = rule; + + goto out_flow_group; + +err: + macsec_fs_tx_destroy(macsec_fs); +out_flow_group: + kvfree(flow_group_in); +out_spec: + kvfree(spec); + return err; +} + +static int macsec_fs_tx_ft_get(struct mlx5e_macsec_fs *macsec_fs) +{ + struct mlx5e_macsec_tx *tx_fs = macsec_fs->tx_fs; + struct mlx5e_macsec_tables *tx_tables; + int err = 0; + + tx_tables = &tx_fs->tables; + if (tx_tables->refcnt) + goto out; + + err = macsec_fs_tx_create(macsec_fs); + if (err) + return err; + +out: + tx_tables->refcnt++; + return err; +} + +static void macsec_fs_tx_ft_put(struct mlx5e_macsec_fs *macsec_fs) +{ + struct mlx5e_macsec_tables *tx_tables = &macsec_fs->tx_fs->tables; + + if (--tx_tables->refcnt) + return; + + macsec_fs_tx_destroy(macsec_fs); +} + +static int macsec_fs_tx_setup_fte(struct mlx5e_macsec_fs *macsec_fs, + struct mlx5_flow_spec *spec, + struct mlx5_flow_act *flow_act, + u32 macsec_obj_id, + u32 *fs_id) +{ + struct mlx5e_macsec_tx *tx_fs = macsec_fs->tx_fs; + int err = 0; + u32 id; + + err = ida_alloc_range(&tx_fs->tx_halloc, 1, + MLX5_MACSEC_NUM_OF_SUPPORTED_INTERFACES, + GFP_KERNEL); + if (err < 0) + return err; + + id = err; + spec->match_criteria_enable |= MLX5_MATCH_MISC_PARAMETERS_2; + + /* Metadata match */ + MLX5_SET(fte_match_param, spec->match_criteria, misc_parameters_2.metadata_reg_a, + MLX5_ETH_WQE_FT_META_MACSEC_MASK); + MLX5_SET(fte_match_param, spec->match_value, misc_parameters_2.metadata_reg_a, + MLX5_ETH_WQE_FT_META_MACSEC | id << 2); + + *fs_id = id; + flow_act->crypto.type = MLX5_FLOW_CONTEXT_ENCRYPT_DECRYPT_TYPE_MACSEC; + flow_act->crypto.obj_id = macsec_obj_id; + + mlx5_core_dbg(macsec_fs->mdev, "Tx fte: macsec obj_id %u, fs_id %u\n", macsec_obj_id, id); + return 0; +} + +static void macsec_fs_tx_create_sectag_header(const struct macsec_context *ctx, + char *reformatbf, + size_t *reformat_size) +{ + const struct macsec_secy *secy = ctx->secy; + bool sci_present = macsec_send_sci(secy); + struct mlx5_sectag_header sectag = {}; + const struct macsec_tx_sc *tx_sc; + + tx_sc = &secy->tx_sc; + sectag.ethertype = htons(ETH_P_MACSEC); + + if (sci_present) { + sectag.tci_an |= MACSEC_TCI_SC; + memcpy(§ag.sci, &secy->sci, + sizeof(sectag.sci)); + } else { + if (tx_sc->end_station) + sectag.tci_an |= MACSEC_TCI_ES; + if (tx_sc->scb) + sectag.tci_an |= MACSEC_TCI_SCB; + } + + /* With GCM, C/E clear for !encrypt, both set for encrypt */ + if (tx_sc->encrypt) + sectag.tci_an |= MACSEC_TCI_CONFID; + else if (secy->icv_len != MACSEC_DEFAULT_ICV_LEN) + sectag.tci_an |= MACSEC_TCI_C; + + sectag.tci_an |= tx_sc->encoding_sa; + + *reformat_size = MLX5_MACSEC_TAG_LEN + (sci_present ? MACSEC_SCI_LEN : 0); + + memcpy(reformatbf, §ag, *reformat_size); +} + +static void macsec_fs_tx_del_rule(struct mlx5e_macsec_fs *macsec_fs, + struct mlx5e_macsec_tx_rule *tx_rule) +{ + if (tx_rule->rule) { + mlx5_del_flow_rules(tx_rule->rule); + tx_rule->rule = NULL; + } + + if (tx_rule->pkt_reformat) { + mlx5_packet_reformat_dealloc(macsec_fs->mdev, tx_rule->pkt_reformat); + tx_rule->pkt_reformat = NULL; + } + + if (tx_rule->fs_id) { + ida_free(&macsec_fs->tx_fs->tx_halloc, tx_rule->fs_id); + tx_rule->fs_id = 0; + } + + kfree(tx_rule); + + macsec_fs_tx_ft_put(macsec_fs); +} + +static union mlx5e_macsec_rule * +macsec_fs_tx_add_rule(struct mlx5e_macsec_fs *macsec_fs, + const struct macsec_context *macsec_ctx, + struct mlx5_macsec_rule_attrs *attrs, + u32 *sa_fs_id) +{ + char reformatbf[MLX5_MACSEC_TAG_LEN + MACSEC_SCI_LEN]; + struct mlx5_pkt_reformat_params reformat_params = {}; + struct mlx5e_macsec_tx *tx_fs = macsec_fs->tx_fs; + struct net_device *netdev = macsec_fs->netdev; + union mlx5e_macsec_rule *macsec_rule = NULL; + struct mlx5_flow_destination dest = {}; + struct mlx5e_macsec_tables *tx_tables; + struct mlx5e_macsec_tx_rule *tx_rule; + struct mlx5_flow_act flow_act = {}; + struct mlx5_flow_handle *rule; + struct mlx5_flow_spec *spec; + size_t reformat_size; + int err = 0; + u32 fs_id; + + tx_tables = &tx_fs->tables; + + spec = kvzalloc(sizeof(*spec), GFP_KERNEL); + if (!spec) + return NULL; + + err = macsec_fs_tx_ft_get(macsec_fs); + if (err) + goto out_spec; + + macsec_rule = kzalloc(sizeof(*macsec_rule), GFP_KERNEL); + if (!macsec_rule) { + macsec_fs_tx_ft_put(macsec_fs); + goto out_spec; + } + + tx_rule = &macsec_rule->tx_rule; + + /* Tx crypto table crypto rule */ + macsec_fs_tx_create_sectag_header(macsec_ctx, reformatbf, &reformat_size); + + reformat_params.type = MLX5_REFORMAT_TYPE_ADD_MACSEC; + reformat_params.size = reformat_size; + reformat_params.data = reformatbf; + flow_act.pkt_reformat = mlx5_packet_reformat_alloc(macsec_fs->mdev, + &reformat_params, + MLX5_FLOW_NAMESPACE_EGRESS_MACSEC); + if (IS_ERR(flow_act.pkt_reformat)) { + err = PTR_ERR(flow_act.pkt_reformat); + netdev_err(netdev, "Failed to allocate MACsec Tx reformat context err=%d\n", err); + goto err; + } + tx_rule->pkt_reformat = flow_act.pkt_reformat; + + err = macsec_fs_tx_setup_fte(macsec_fs, spec, &flow_act, attrs->macsec_obj_id, &fs_id); + if (err) { + netdev_err(netdev, + "Failed to add packet reformat for MACsec TX crypto rule, err=%d\n", + err); + goto err; + } + + tx_rule->fs_id = fs_id; + *sa_fs_id = fs_id; + + flow_act.action = MLX5_FLOW_CONTEXT_ACTION_FWD_DEST | + MLX5_FLOW_CONTEXT_ACTION_CRYPTO_ENCRYPT | + MLX5_FLOW_CONTEXT_ACTION_PACKET_REFORMAT; + dest.type = MLX5_FLOW_DESTINATION_TYPE_FLOW_TABLE; + dest.ft = tx_tables->ft_check; + rule = mlx5_add_flow_rules(tx_tables->ft_crypto.t, spec, &flow_act, &dest, 1); + if (IS_ERR(rule)) { + err = PTR_ERR(rule); + netdev_err(netdev, "Failed to add MACsec TX crypto rule, err=%d\n", err); + goto err; + } + tx_rule->rule = rule; + + goto out_spec; + +err: + macsec_fs_tx_del_rule(macsec_fs, tx_rule); + macsec_rule = NULL; +out_spec: + kvfree(spec); + + return macsec_rule; +} + +static void macsec_fs_tx_cleanup(struct mlx5e_macsec_fs *macsec_fs) +{ + struct mlx5e_macsec_tx *tx_fs = macsec_fs->tx_fs; + struct mlx5_core_dev *mdev = macsec_fs->mdev; + struct mlx5e_macsec_tables *tx_tables; + + if (!tx_fs) + return; + + tx_tables = &tx_fs->tables; + if (tx_tables->refcnt) { + netdev_err(macsec_fs->netdev, + "Can't destroy MACsec offload tx_fs, refcnt(%u) isn't 0\n", + tx_tables->refcnt); + return; + } + + ida_destroy(&tx_fs->tx_halloc); + + if (tx_tables->check_miss_rule_counter) { + mlx5_fc_destroy(mdev, tx_tables->check_miss_rule_counter); + tx_tables->check_miss_rule_counter = NULL; + } + + if (tx_tables->check_rule_counter) { + mlx5_fc_destroy(mdev, tx_tables->check_rule_counter); + tx_tables->check_rule_counter = NULL; + } + + kfree(tx_fs); + macsec_fs->tx_fs = NULL; +} + +static int macsec_fs_tx_init(struct mlx5e_macsec_fs *macsec_fs) +{ + struct net_device *netdev = macsec_fs->netdev; + struct mlx5_core_dev *mdev = macsec_fs->mdev; + struct mlx5e_macsec_tables *tx_tables; + struct mlx5e_macsec_tx *tx_fs; + struct mlx5_fc *flow_counter; + int err; + + tx_fs = kzalloc(sizeof(*tx_fs), GFP_KERNEL); + if (!tx_fs) + return -ENOMEM; + + tx_tables = &tx_fs->tables; + + flow_counter = mlx5_fc_create(mdev, false); + if (IS_ERR(flow_counter)) { + err = PTR_ERR(flow_counter); + netdev_err(netdev, + "Failed to create MACsec Tx encrypt flow counter, err(%d)\n", + err); + goto err_encrypt_counter; + } + tx_tables->check_rule_counter = flow_counter; + + flow_counter = mlx5_fc_create(mdev, false); + if (IS_ERR(flow_counter)) { + err = PTR_ERR(flow_counter); + netdev_err(netdev, + "Failed to create MACsec Tx drop flow counter, err(%d)\n", + err); + goto err_drop_counter; + } + tx_tables->check_miss_rule_counter = flow_counter; + + ida_init(&tx_fs->tx_halloc); + + macsec_fs->tx_fs = tx_fs; + + return 0; + +err_drop_counter: + mlx5_fc_destroy(mdev, tx_tables->check_rule_counter); + tx_tables->check_rule_counter = NULL; + +err_encrypt_counter: + kfree(tx_fs); + macsec_fs->tx_fs = NULL; + + return err; +} + +static void macsec_fs_rx_destroy(struct mlx5e_macsec_fs *macsec_fs) +{ + struct mlx5e_macsec_rx *rx_fs = macsec_fs->rx_fs; + struct mlx5e_macsec_tables *rx_tables; + int i; + + /* Rx check table */ + for (i = 1; i >= 0; --i) { + if (rx_fs->check_rule[i]) { + mlx5_del_flow_rules(rx_fs->check_rule[i]); + rx_fs->check_rule[i] = NULL; + } + + if (rx_fs->check_rule_pkt_reformat[i]) { + mlx5_packet_reformat_dealloc(macsec_fs->mdev, + rx_fs->check_rule_pkt_reformat[i]); + rx_fs->check_rule_pkt_reformat[i] = NULL; + } + } + + rx_tables = &rx_fs->tables; + + if (rx_tables->check_miss_rule) { + mlx5_del_flow_rules(rx_tables->check_miss_rule); + rx_tables->check_miss_rule = NULL; + } + + if (rx_tables->ft_check_group) { + mlx5_destroy_flow_group(rx_tables->ft_check_group); + rx_tables->ft_check_group = NULL; + } + + if (rx_tables->ft_check) { + mlx5_destroy_flow_table(rx_tables->ft_check); + rx_tables->ft_check = NULL; + } + + /* Rx crypto table */ + if (rx_tables->crypto_miss_rule) { + mlx5_del_flow_rules(rx_tables->crypto_miss_rule); + rx_tables->crypto_miss_rule = NULL; + } + + mlx5e_destroy_flow_table(&rx_tables->ft_crypto); +} + +static int macsec_fs_rx_create_crypto_table_groups(struct mlx5e_flow_table *ft) +{ + int inlen = MLX5_ST_SZ_BYTES(create_flow_group_in); + int mclen = MLX5_ST_SZ_BYTES(fte_match_param); + int ix = 0; + u32 *in; + int err; + u8 *mc; + + ft->g = kcalloc(RX_CRYPTO_TABLE_NUM_GROUPS, sizeof(*ft->g), GFP_KERNEL); + if (!ft->g) + return -ENOMEM; + + in = kvzalloc(inlen, GFP_KERNEL); + if (!in) { + kfree(ft->g); + return -ENOMEM; + } + + mc = MLX5_ADDR_OF(create_flow_group_in, in, match_criteria); + + /* Flow group for SA rule with SCI */ + MLX5_SET_CFG(in, match_criteria_enable, MLX5_MATCH_OUTER_HEADERS | + MLX5_MATCH_MISC_PARAMETERS_5); + MLX5_SET_TO_ONES(fte_match_param, mc, outer_headers.ethertype); + + MLX5_SET(fte_match_param, mc, misc_parameters_5.macsec_tag_0, + MLX5_MACSEC_SECTAG_TCI_AN_FIELD_BITMASK << + MLX5_MACSEC_SECTAG_TCI_AN_FIELD_OFFSET); + MLX5_SET_TO_ONES(fte_match_param, mc, misc_parameters_5.macsec_tag_2); + MLX5_SET_TO_ONES(fte_match_param, mc, misc_parameters_5.macsec_tag_3); + + MLX5_SET_CFG(in, start_flow_index, ix); + ix += RX_CRYPTO_TABLE_SA_RULE_WITH_SCI_GROUP_SIZE; + MLX5_SET_CFG(in, end_flow_index, ix - 1); + ft->g[ft->num_groups] = mlx5_create_flow_group(ft->t, in); + if (IS_ERR(ft->g[ft->num_groups])) + goto err; + ft->num_groups++; + + /* Flow group for SA rule without SCI */ + memset(in, 0, inlen); + memset(mc, 0, mclen); + MLX5_SET_CFG(in, match_criteria_enable, MLX5_MATCH_OUTER_HEADERS | + MLX5_MATCH_MISC_PARAMETERS_5); + MLX5_SET_TO_ONES(fte_match_param, mc, outer_headers.smac_47_16); + MLX5_SET_TO_ONES(fte_match_param, mc, outer_headers.smac_15_0); + MLX5_SET_TO_ONES(fte_match_param, mc, outer_headers.ethertype); + + MLX5_SET(fte_match_param, mc, misc_parameters_5.macsec_tag_0, + MLX5_MACSEC_SECTAG_TCI_AN_FIELD_BITMASK << MLX5_MACSEC_SECTAG_TCI_AN_FIELD_OFFSET); + + MLX5_SET_CFG(in, start_flow_index, ix); + ix += RX_CRYPTO_TABLE_SA_RULE_WITHOUT_SCI_GROUP_SIZE; + MLX5_SET_CFG(in, end_flow_index, ix - 1); + ft->g[ft->num_groups] = mlx5_create_flow_group(ft->t, in); + if (IS_ERR(ft->g[ft->num_groups])) + goto err; + ft->num_groups++; + + /* Flow Group for l2 traps */ + memset(in, 0, inlen); + memset(mc, 0, mclen); + MLX5_SET_CFG(in, start_flow_index, ix); + ix += CRYPTO_TABLE_DEFAULT_RULE_GROUP_SIZE; + MLX5_SET_CFG(in, end_flow_index, ix - 1); + ft->g[ft->num_groups] = mlx5_create_flow_group(ft->t, in); + if (IS_ERR(ft->g[ft->num_groups])) + goto err; + ft->num_groups++; + + kvfree(in); + return 0; + +err: + err = PTR_ERR(ft->g[ft->num_groups]); + ft->g[ft->num_groups] = NULL; + kvfree(in); + + return err; +} + +static int macsec_fs_rx_create_check_decap_rule(struct mlx5e_macsec_fs *macsec_fs, + struct mlx5_flow_destination *dest, + struct mlx5_flow_act *flow_act, + struct mlx5_flow_spec *spec, + int reformat_param_size) +{ + int rule_index = (reformat_param_size == MLX5_SECTAG_HEADER_SIZE_WITH_SCI) ? 0 : 1; + u8 mlx5_reformat_buf[MLX5_SECTAG_HEADER_SIZE_WITH_SCI]; + struct mlx5_pkt_reformat_params reformat_params = {}; + struct mlx5e_macsec_rx *rx_fs = macsec_fs->rx_fs; + struct net_device *netdev = macsec_fs->netdev; + struct mlx5e_macsec_tables *rx_tables; + struct mlx5_flow_handle *rule; + int err = 0; + + rx_tables = &rx_fs->tables; + + /* Rx check table decap 16B rule */ + memset(dest, 0, sizeof(*dest)); + memset(flow_act, 0, sizeof(*flow_act)); + memset(spec, 0, sizeof(*spec)); + + reformat_params.type = MLX5_REFORMAT_TYPE_DEL_MACSEC; + reformat_params.size = reformat_param_size; + reformat_params.data = mlx5_reformat_buf; + flow_act->pkt_reformat = mlx5_packet_reformat_alloc(macsec_fs->mdev, + &reformat_params, + MLX5_FLOW_NAMESPACE_KERNEL_RX_MACSEC); + if (IS_ERR(flow_act->pkt_reformat)) { + err = PTR_ERR(flow_act->pkt_reformat); + netdev_err(netdev, "Failed to allocate MACsec Rx reformat context err=%d\n", err); + return err; + } + rx_fs->check_rule_pkt_reformat[rule_index] = flow_act->pkt_reformat; + + spec->match_criteria_enable = MLX5_MATCH_MISC_PARAMETERS_2; + /* MACsec syndrome match */ + MLX5_SET_TO_ONES(fte_match_param, spec->match_criteria, misc_parameters_2.macsec_syndrome); + MLX5_SET(fte_match_param, spec->match_value, misc_parameters_2.macsec_syndrome, 0); + /* ASO return reg syndrome match */ + MLX5_SET_TO_ONES(fte_match_param, spec->match_criteria, misc_parameters_2.metadata_reg_c_4); + MLX5_SET(fte_match_param, spec->match_value, misc_parameters_2.metadata_reg_c_4, 0); + + spec->match_criteria_enable |= MLX5_MATCH_MISC_PARAMETERS_5; + /* Sectag TCI SC present bit*/ + MLX5_SET(fte_match_param, spec->match_criteria, misc_parameters_5.macsec_tag_0, + MLX5_MACSEC_SECTAG_TCI_SC_FIELD_BIT << MLX5_MACSEC_SECTAG_TCI_AN_FIELD_OFFSET); + + if (reformat_param_size == MLX5_SECTAG_HEADER_SIZE_WITH_SCI) + MLX5_SET(fte_match_param, spec->match_value, misc_parameters_5.macsec_tag_0, + MLX5_MACSEC_SECTAG_TCI_SC_FIELD_BIT << + MLX5_MACSEC_SECTAG_TCI_AN_FIELD_OFFSET); + + flow_act->flags = FLOW_ACT_NO_APPEND; + flow_act->action = MLX5_FLOW_CONTEXT_ACTION_FWD_NEXT_PRIO | + MLX5_FLOW_CONTEXT_ACTION_PACKET_REFORMAT | + MLX5_FLOW_CONTEXT_ACTION_COUNT; + dest->type = MLX5_FLOW_DESTINATION_TYPE_COUNTER; + dest->counter_id = mlx5_fc_id(rx_tables->check_rule_counter); + rule = mlx5_add_flow_rules(rx_tables->ft_check, spec, flow_act, dest, 1); + if (IS_ERR(rule)) { + err = PTR_ERR(rule); + netdev_err(netdev, "Failed to add MACsec Rx check rule, err=%d\n", err); + return err; + } + + rx_fs->check_rule[rule_index] = rule; + + return 0; +} + +static int macsec_fs_rx_create(struct mlx5e_macsec_fs *macsec_fs) +{ + int inlen = MLX5_ST_SZ_BYTES(create_flow_group_in); + struct mlx5e_macsec_rx *rx_fs = macsec_fs->rx_fs; + struct net_device *netdev = macsec_fs->netdev; + struct mlx5_flow_table_attr ft_attr = {}; + struct mlx5_flow_destination dest = {}; + struct mlx5e_macsec_tables *rx_tables; + struct mlx5e_flow_table *ft_crypto; + struct mlx5_flow_table *flow_table; + struct mlx5_flow_group *flow_group; + struct mlx5_flow_act flow_act = {}; + struct mlx5_flow_namespace *ns; + struct mlx5_flow_handle *rule; + struct mlx5_flow_spec *spec; + u32 *flow_group_in; + int err = 0; + + ns = mlx5_get_flow_namespace(macsec_fs->mdev, MLX5_FLOW_NAMESPACE_KERNEL_RX_MACSEC); + if (!ns) + return -ENOMEM; + + spec = kvzalloc(sizeof(*spec), GFP_KERNEL); + if (!spec) + return -ENOMEM; + + flow_group_in = kvzalloc(inlen, GFP_KERNEL); + if (!flow_group_in) + goto free_spec; + + rx_tables = &rx_fs->tables; + ft_crypto = &rx_tables->ft_crypto; + + /* Rx crypto table */ + ft_attr.level = RX_CRYPTO_TABLE_LEVEL; + ft_attr.max_fte = CRYPTO_NUM_MAXSEC_FTE; + + flow_table = mlx5_create_flow_table(ns, &ft_attr); + if (IS_ERR(flow_table)) { + err = PTR_ERR(flow_table); + netdev_err(netdev, "Failed to create MACsec Rx crypto table err(%d)\n", err); + goto out_flow_group; + } + ft_crypto->t = flow_table; + + /* Rx crypto table groups */ + err = macsec_fs_rx_create_crypto_table_groups(ft_crypto); + if (err) { + netdev_err(netdev, + "Failed to create default flow group for MACsec Tx crypto table err(%d)\n", + err); + goto err; + } + + flow_act.action = MLX5_FLOW_CONTEXT_ACTION_FWD_NEXT_PRIO; + rule = mlx5_add_flow_rules(ft_crypto->t, NULL, &flow_act, NULL, 0); + if (IS_ERR(rule)) { + err = PTR_ERR(rule); + netdev_err(netdev, + "Failed to add MACsec Rx crypto table default miss rule %d\n", + err); + goto err; + } + rx_tables->crypto_miss_rule = rule; + + /* Rx check table */ + flow_table = macsec_fs_auto_group_table_create(ns, + MLX5_FLOW_TABLE_TUNNEL_EN_REFORMAT, + RX_CHECK_TABLE_LEVEL, + RX_CHECK_TABLE_NUM_FTE); + if (IS_ERR(flow_table)) { + err = PTR_ERR(flow_table); + netdev_err(netdev, "fail to create MACsec RX check table, err(%d)\n", err); + goto err; + } + rx_tables->ft_check = flow_table; + + /* Rx check table Default miss group/rule */ + MLX5_SET(create_flow_group_in, flow_group_in, start_flow_index, flow_table->max_fte - 1); + MLX5_SET(create_flow_group_in, flow_group_in, end_flow_index, flow_table->max_fte - 1); + flow_group = mlx5_create_flow_group(rx_tables->ft_check, flow_group_in); + if (IS_ERR(flow_group)) { + err = PTR_ERR(flow_group); + netdev_err(netdev, + "Failed to create default flow group for MACsec Rx check table err(%d)\n", + err); + goto err; + } + rx_tables->ft_check_group = flow_group; + + /* Rx check table default drop rule */ + memset(&flow_act, 0, sizeof(flow_act)); + + dest.type = MLX5_FLOW_DESTINATION_TYPE_COUNTER; + dest.counter_id = mlx5_fc_id(rx_tables->check_miss_rule_counter); + flow_act.action = MLX5_FLOW_CONTEXT_ACTION_DROP | MLX5_FLOW_CONTEXT_ACTION_COUNT; + rule = mlx5_add_flow_rules(rx_tables->ft_check, NULL, &flow_act, &dest, 1); + if (IS_ERR(rule)) { + err = PTR_ERR(rule); + netdev_err(netdev, "Failed to added MACsec Rx check drop rule, err(%d)\n", err); + goto err; + } + rx_tables->check_miss_rule = rule; + + /* Rx check table decap rules */ + err = macsec_fs_rx_create_check_decap_rule(macsec_fs, &dest, &flow_act, spec, + MLX5_SECTAG_HEADER_SIZE_WITH_SCI); + if (err) + goto err; + + err = macsec_fs_rx_create_check_decap_rule(macsec_fs, &dest, &flow_act, spec, + MLX5_SECTAG_HEADER_SIZE_WITHOUT_SCI); + if (err) + goto err; + + goto out_flow_group; + +err: + macsec_fs_rx_destroy(macsec_fs); +out_flow_group: + kvfree(flow_group_in); +free_spec: + kvfree(spec); + return err; +} + +static int macsec_fs_rx_ft_get(struct mlx5e_macsec_fs *macsec_fs) +{ + struct mlx5e_macsec_tables *rx_tables = &macsec_fs->rx_fs->tables; + int err = 0; + + if (rx_tables->refcnt) + goto out; + + err = macsec_fs_rx_create(macsec_fs); + if (err) + return err; + +out: + rx_tables->refcnt++; + return err; +} + +static void macsec_fs_rx_ft_put(struct mlx5e_macsec_fs *macsec_fs) +{ + struct mlx5e_macsec_tables *rx_tables = &macsec_fs->rx_fs->tables; + + if (--rx_tables->refcnt) + return; + + macsec_fs_rx_destroy(macsec_fs); +} + +static void macsec_fs_rx_del_rule(struct mlx5e_macsec_fs *macsec_fs, + struct mlx5e_macsec_rx_rule *rx_rule) +{ + int i; + + for (i = 0; i < RX_NUM_OF_RULES_PER_SA; ++i) { + if (rx_rule->rule[i]) { + mlx5_del_flow_rules(rx_rule->rule[i]); + rx_rule->rule[i] = NULL; + } + } + + if (rx_rule->meta_modhdr) { + mlx5_modify_header_dealloc(macsec_fs->mdev, rx_rule->meta_modhdr); + rx_rule->meta_modhdr = NULL; + } + + kfree(rx_rule); + + macsec_fs_rx_ft_put(macsec_fs); +} + +static void macsec_fs_rx_setup_fte(struct mlx5_flow_spec *spec, + struct mlx5_flow_act *flow_act, + struct mlx5_macsec_rule_attrs *attrs, + bool sci_present) +{ + u8 tci_an = (sci_present << MLX5_MACSEC_SECTAG_TCI_SC_FIELD_OFFSET) | attrs->assoc_num; + struct mlx5_flow_act_crypto_params *crypto_params = &flow_act->crypto; + __be32 *sci_p = (__be32 *)(&attrs->sci); + + spec->match_criteria_enable = MLX5_MATCH_OUTER_HEADERS; + + /* MACsec ethertype */ + MLX5_SET_TO_ONES(fte_match_param, spec->match_criteria, outer_headers.ethertype); + MLX5_SET(fte_match_param, spec->match_value, outer_headers.ethertype, ETH_P_MACSEC); + + spec->match_criteria_enable |= MLX5_MATCH_MISC_PARAMETERS_5; + + /* Sectag AN + TCI SC present bit*/ + MLX5_SET(fte_match_param, spec->match_criteria, misc_parameters_5.macsec_tag_0, + MLX5_MACSEC_SECTAG_TCI_AN_FIELD_BITMASK << MLX5_MACSEC_SECTAG_TCI_AN_FIELD_OFFSET); + MLX5_SET(fte_match_param, spec->match_value, misc_parameters_5.macsec_tag_0, + tci_an << MLX5_MACSEC_SECTAG_TCI_AN_FIELD_OFFSET); + + if (sci_present) { + MLX5_SET_TO_ONES(fte_match_param, spec->match_criteria, + misc_parameters_5.macsec_tag_2); + MLX5_SET(fte_match_param, spec->match_value, misc_parameters_5.macsec_tag_2, + be32_to_cpu(sci_p[0])); + + MLX5_SET_TO_ONES(fte_match_param, spec->match_criteria, + misc_parameters_5.macsec_tag_3); + MLX5_SET(fte_match_param, spec->match_value, misc_parameters_5.macsec_tag_3, + be32_to_cpu(sci_p[1])); + } else { + /* When SCI isn't present in the Sectag, need to match the source */ + /* MAC address only if the SCI contains the default MACsec PORT */ + MLX5_SET_TO_ONES(fte_match_param, spec->match_criteria, outer_headers.smac_47_16); + MLX5_SET_TO_ONES(fte_match_param, spec->match_criteria, outer_headers.smac_15_0); + memcpy(MLX5_ADDR_OF(fte_match_param, spec->match_value, outer_headers.smac_47_16), + sci_p, ETH_ALEN); + } + + crypto_params->type = MLX5_FLOW_CONTEXT_ENCRYPT_DECRYPT_TYPE_MACSEC; + crypto_params->obj_id = attrs->macsec_obj_id; +} + +static union mlx5e_macsec_rule * +macsec_fs_rx_add_rule(struct mlx5e_macsec_fs *macsec_fs, + const struct macsec_context *macsec_ctx, + struct mlx5_macsec_rule_attrs *attrs, + u32 fs_id) +{ + u8 action[MLX5_UN_SZ_BYTES(set_add_copy_action_in_auto)] = {}; + struct mlx5e_macsec_rx *rx_fs = macsec_fs->rx_fs; + struct net_device *netdev = macsec_fs->netdev; + union mlx5e_macsec_rule *macsec_rule = NULL; + struct mlx5_modify_hdr *modify_hdr = NULL; + struct mlx5_flow_destination dest = {}; + struct mlx5e_macsec_tables *rx_tables; + struct mlx5e_macsec_rx_rule *rx_rule; + struct mlx5_flow_act flow_act = {}; + struct mlx5e_flow_table *ft_crypto; + struct mlx5_flow_handle *rule; + struct mlx5_flow_spec *spec; + int err = 0; + + spec = kvzalloc(sizeof(*spec), GFP_KERNEL); + if (!spec) + return NULL; + + err = macsec_fs_rx_ft_get(macsec_fs); + if (err) + goto out_spec; + + macsec_rule = kzalloc(sizeof(*macsec_rule), GFP_KERNEL); + if (!macsec_rule) { + macsec_fs_rx_ft_put(macsec_fs); + goto out_spec; + } + + rx_rule = &macsec_rule->rx_rule; + rx_tables = &rx_fs->tables; + ft_crypto = &rx_tables->ft_crypto; + + /* Set bit[31 - 30] macsec marker - 0x01 */ + /* Set bit[3-0] fs id */ + MLX5_SET(set_action_in, action, action_type, MLX5_ACTION_TYPE_SET); + MLX5_SET(set_action_in, action, field, MLX5_ACTION_IN_FIELD_METADATA_REG_B); + MLX5_SET(set_action_in, action, data, fs_id | BIT(30)); + MLX5_SET(set_action_in, action, offset, 0); + MLX5_SET(set_action_in, action, length, 32); + + modify_hdr = mlx5_modify_header_alloc(macsec_fs->mdev, MLX5_FLOW_NAMESPACE_KERNEL_RX_MACSEC, + 1, action); + if (IS_ERR(modify_hdr)) { + err = PTR_ERR(modify_hdr); + netdev_err(netdev, "fail to alloc MACsec set modify_header_id err=%d\n", err); + modify_hdr = NULL; + goto err; + } + rx_rule->meta_modhdr = modify_hdr; + + /* Rx crypto table with SCI rule */ + macsec_fs_rx_setup_fte(spec, &flow_act, attrs, true); + + flow_act.modify_hdr = modify_hdr; + flow_act.action = MLX5_FLOW_CONTEXT_ACTION_FWD_DEST | + MLX5_FLOW_CONTEXT_ACTION_CRYPTO_DECRYPT | + MLX5_FLOW_CONTEXT_ACTION_MOD_HDR; + + dest.type = MLX5_FLOW_DESTINATION_TYPE_FLOW_TABLE; + dest.ft = rx_tables->ft_check; + rule = mlx5_add_flow_rules(ft_crypto->t, spec, &flow_act, &dest, 1); + if (IS_ERR(rule)) { + err = PTR_ERR(rule); + netdev_err(netdev, + "Failed to add SA with SCI rule to Rx crypto rule, err=%d\n", + err); + goto err; + } + rx_rule->rule[0] = rule; + + /* Rx crypto table without SCI rule */ + if (cpu_to_be64((__force u64)attrs->sci) & ntohs(MACSEC_PORT_ES)) { + memset(spec, 0, sizeof(struct mlx5_flow_spec)); + memset(&dest, 0, sizeof(struct mlx5_flow_destination)); + memset(&flow_act, 0, sizeof(flow_act)); + + macsec_fs_rx_setup_fte(spec, &flow_act, attrs, false); + + flow_act.modify_hdr = modify_hdr; + flow_act.action = MLX5_FLOW_CONTEXT_ACTION_FWD_DEST | + MLX5_FLOW_CONTEXT_ACTION_CRYPTO_DECRYPT | + MLX5_FLOW_CONTEXT_ACTION_MOD_HDR; + + dest.type = MLX5_FLOW_DESTINATION_TYPE_FLOW_TABLE; + dest.ft = rx_tables->ft_check; + rule = mlx5_add_flow_rules(ft_crypto->t, spec, &flow_act, &dest, 1); + if (IS_ERR(rule)) { + err = PTR_ERR(rule); + netdev_err(netdev, + "Failed to add SA without SCI rule to Rx crypto rule, err=%d\n", + err); + goto err; + } + rx_rule->rule[1] = rule; + } + + return macsec_rule; + +err: + macsec_fs_rx_del_rule(macsec_fs, rx_rule); + macsec_rule = NULL; +out_spec: + kvfree(spec); + return macsec_rule; +} + +static int macsec_fs_rx_init(struct mlx5e_macsec_fs *macsec_fs) +{ + struct net_device *netdev = macsec_fs->netdev; + struct mlx5_core_dev *mdev = macsec_fs->mdev; + struct mlx5e_macsec_tables *rx_tables; + struct mlx5e_macsec_rx *rx_fs; + struct mlx5_fc *flow_counter; + int err; + + rx_fs = kzalloc(sizeof(*rx_fs), GFP_KERNEL); + if (!rx_fs) + return -ENOMEM; + + flow_counter = mlx5_fc_create(mdev, false); + if (IS_ERR(flow_counter)) { + err = PTR_ERR(flow_counter); + netdev_err(netdev, + "Failed to create MACsec Rx encrypt flow counter, err(%d)\n", + err); + goto err_encrypt_counter; + } + + rx_tables = &rx_fs->tables; + rx_tables->check_rule_counter = flow_counter; + + flow_counter = mlx5_fc_create(mdev, false); + if (IS_ERR(flow_counter)) { + err = PTR_ERR(flow_counter); + netdev_err(netdev, + "Failed to create MACsec Rx drop flow counter, err(%d)\n", + err); + goto err_drop_counter; + } + rx_tables->check_miss_rule_counter = flow_counter; + + macsec_fs->rx_fs = rx_fs; + + return 0; + +err_drop_counter: + mlx5_fc_destroy(mdev, rx_tables->check_rule_counter); + rx_tables->check_rule_counter = NULL; + +err_encrypt_counter: + kfree(rx_fs); + macsec_fs->rx_fs = NULL; + + return err; +} + +static void macsec_fs_rx_cleanup(struct mlx5e_macsec_fs *macsec_fs) +{ + struct mlx5e_macsec_rx *rx_fs = macsec_fs->rx_fs; + struct mlx5_core_dev *mdev = macsec_fs->mdev; + struct mlx5e_macsec_tables *rx_tables; + + if (!rx_fs) + return; + + rx_tables = &rx_fs->tables; + + if (rx_tables->refcnt) { + netdev_err(macsec_fs->netdev, + "Can't destroy MACsec offload rx_fs, refcnt(%u) isn't 0\n", + rx_tables->refcnt); + return; + } + + if (rx_tables->check_miss_rule_counter) { + mlx5_fc_destroy(mdev, rx_tables->check_miss_rule_counter); + rx_tables->check_miss_rule_counter = NULL; + } + + if (rx_tables->check_rule_counter) { + mlx5_fc_destroy(mdev, rx_tables->check_rule_counter); + rx_tables->check_rule_counter = NULL; + } + + kfree(rx_fs); + macsec_fs->rx_fs = NULL; +} + +void mlx5e_macsec_fs_get_stats_fill(struct mlx5e_macsec_fs *macsec_fs, void *macsec_stats) +{ + struct mlx5e_macsec_stats *stats = (struct mlx5e_macsec_stats *)macsec_stats; + struct mlx5e_macsec_tables *tx_tables = &macsec_fs->tx_fs->tables; + struct mlx5e_macsec_tables *rx_tables = &macsec_fs->rx_fs->tables; + struct mlx5_core_dev *mdev = macsec_fs->mdev; + + if (tx_tables->check_rule_counter) + mlx5_fc_query(mdev, tx_tables->check_rule_counter, + &stats->macsec_tx_pkts, &stats->macsec_tx_bytes); + + if (tx_tables->check_miss_rule_counter) + mlx5_fc_query(mdev, tx_tables->check_miss_rule_counter, + &stats->macsec_tx_pkts_drop, &stats->macsec_tx_bytes_drop); + + if (rx_tables->check_rule_counter) + mlx5_fc_query(mdev, rx_tables->check_rule_counter, + &stats->macsec_rx_pkts, &stats->macsec_rx_bytes); + + if (rx_tables->check_miss_rule_counter) + mlx5_fc_query(mdev, rx_tables->check_miss_rule_counter, + &stats->macsec_rx_pkts_drop, &stats->macsec_rx_bytes_drop); +} + +union mlx5e_macsec_rule * +mlx5e_macsec_fs_add_rule(struct mlx5e_macsec_fs *macsec_fs, + const struct macsec_context *macsec_ctx, + struct mlx5_macsec_rule_attrs *attrs, + u32 *sa_fs_id) +{ + return (attrs->action == MLX5_ACCEL_MACSEC_ACTION_ENCRYPT) ? + macsec_fs_tx_add_rule(macsec_fs, macsec_ctx, attrs, sa_fs_id) : + macsec_fs_rx_add_rule(macsec_fs, macsec_ctx, attrs, *sa_fs_id); +} + +void mlx5e_macsec_fs_del_rule(struct mlx5e_macsec_fs *macsec_fs, + union mlx5e_macsec_rule *macsec_rule, + int action) +{ + (action == MLX5_ACCEL_MACSEC_ACTION_ENCRYPT) ? + macsec_fs_tx_del_rule(macsec_fs, &macsec_rule->tx_rule) : + macsec_fs_rx_del_rule(macsec_fs, &macsec_rule->rx_rule); +} + +void mlx5e_macsec_fs_cleanup(struct mlx5e_macsec_fs *macsec_fs) +{ + macsec_fs_rx_cleanup(macsec_fs); + macsec_fs_tx_cleanup(macsec_fs); + kfree(macsec_fs); +} + +struct mlx5e_macsec_fs * +mlx5e_macsec_fs_init(struct mlx5_core_dev *mdev, + struct net_device *netdev) +{ + struct mlx5e_macsec_fs *macsec_fs; + int err; + + macsec_fs = kzalloc(sizeof(*macsec_fs), GFP_KERNEL); + if (!macsec_fs) + return NULL; + + macsec_fs->mdev = mdev; + macsec_fs->netdev = netdev; + + err = macsec_fs_tx_init(macsec_fs); + if (err) { + netdev_err(netdev, "MACsec offload: Failed to init tx_fs, err=%d\n", err); + goto err; + } + + err = macsec_fs_rx_init(macsec_fs); + if (err) { + netdev_err(netdev, "MACsec offload: Failed to init tx_fs, err=%d\n", err); + goto tx_cleanup; + } + + return macsec_fs; + +tx_cleanup: + macsec_fs_tx_cleanup(macsec_fs); +err: + kfree(macsec_fs); + return NULL; +} diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/macsec_fs.h b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/macsec_fs.h new file mode 100644 index 000000000000..b429648d4ee7 --- /dev/null +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/macsec_fs.h @@ -0,0 +1,47 @@ +/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB */ +/* Copyright (c) 2022, NVIDIA CORPORATION & AFFILIATES. All rights reserved. */ + +#ifndef __MLX5_MACSEC_STEERING_H__ +#define __MLX5_MACSEC_STEERING_H__ + +#ifdef CONFIG_MLX5_EN_MACSEC + +#include "en_accel/macsec.h" + +#define MLX5_MACSEC_NUM_OF_SUPPORTED_INTERFACES 16 + +struct mlx5e_macsec_fs; +union mlx5e_macsec_rule; + +struct mlx5_macsec_rule_attrs { + sci_t sci; + u32 macsec_obj_id; + u8 assoc_num; + int action; +}; + +enum mlx5_macsec_action { + MLX5_ACCEL_MACSEC_ACTION_ENCRYPT, + MLX5_ACCEL_MACSEC_ACTION_DECRYPT, +}; + +void mlx5e_macsec_fs_cleanup(struct mlx5e_macsec_fs *macsec_fs); + +struct mlx5e_macsec_fs * +mlx5e_macsec_fs_init(struct mlx5_core_dev *mdev, struct net_device *netdev); + +union mlx5e_macsec_rule * +mlx5e_macsec_fs_add_rule(struct mlx5e_macsec_fs *macsec_fs, + const struct macsec_context *ctx, + struct mlx5_macsec_rule_attrs *attrs, + u32 *sa_fs_id); + +void mlx5e_macsec_fs_del_rule(struct mlx5e_macsec_fs *macsec_fs, + union mlx5e_macsec_rule *macsec_rule, + int action); + +void mlx5e_macsec_fs_get_stats_fill(struct mlx5e_macsec_fs *macsec_fs, void *macsec_stats); + +#endif + +#endif /* __MLX5_MACSEC_STEERING_H__ */ diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/macsec_stats.c b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/macsec_stats.c new file mode 100644 index 000000000000..e50a2e3f3d18 --- /dev/null +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/macsec_stats.c @@ -0,0 +1,72 @@ +// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB +/* Copyright (c) 2022, NVIDIA CORPORATION & AFFILIATES. All rights reserved. */ + +#include <linux/ethtool.h> +#include <net/sock.h> + +#include "en.h" +#include "en_accel/macsec.h" + +static const struct counter_desc mlx5e_macsec_hw_stats_desc[] = { + { MLX5E_DECLARE_STAT(struct mlx5e_macsec_stats, macsec_rx_pkts) }, + { MLX5E_DECLARE_STAT(struct mlx5e_macsec_stats, macsec_rx_bytes) }, + { MLX5E_DECLARE_STAT(struct mlx5e_macsec_stats, macsec_rx_pkts_drop) }, + { MLX5E_DECLARE_STAT(struct mlx5e_macsec_stats, macsec_rx_bytes_drop) }, + { MLX5E_DECLARE_STAT(struct mlx5e_macsec_stats, macsec_tx_pkts) }, + { MLX5E_DECLARE_STAT(struct mlx5e_macsec_stats, macsec_tx_bytes) }, + { MLX5E_DECLARE_STAT(struct mlx5e_macsec_stats, macsec_tx_pkts_drop) }, + { MLX5E_DECLARE_STAT(struct mlx5e_macsec_stats, macsec_tx_bytes_drop) }, +}; + +#define NUM_MACSEC_HW_COUNTERS ARRAY_SIZE(mlx5e_macsec_hw_stats_desc) + +static MLX5E_DECLARE_STATS_GRP_OP_NUM_STATS(macsec_hw) +{ + if (!priv->macsec) + return 0; + + if (mlx5e_is_macsec_device(priv->mdev)) + return NUM_MACSEC_HW_COUNTERS; + + return 0; +} + +static MLX5E_DECLARE_STATS_GRP_OP_UPDATE_STATS(macsec_hw) {} + +static MLX5E_DECLARE_STATS_GRP_OP_FILL_STRS(macsec_hw) +{ + unsigned int i; + + if (!priv->macsec) + return idx; + + if (!mlx5e_is_macsec_device(priv->mdev)) + return idx; + + for (i = 0; i < NUM_MACSEC_HW_COUNTERS; i++) + strcpy(data + (idx++) * ETH_GSTRING_LEN, + mlx5e_macsec_hw_stats_desc[i].format); + + return idx; +} + +static MLX5E_DECLARE_STATS_GRP_OP_FILL_STATS(macsec_hw) +{ + int i; + + if (!priv->macsec) + return idx; + + if (!mlx5e_is_macsec_device(priv->mdev)) + return idx; + + mlx5e_macsec_get_stats_fill(priv->macsec, mlx5e_macsec_get_stats(priv->macsec)); + for (i = 0; i < NUM_MACSEC_HW_COUNTERS; i++) + data[idx++] = MLX5E_READ_CTR64_CPU(mlx5e_macsec_get_stats(priv->macsec), + mlx5e_macsec_hw_stats_desc, + i); + + return idx; +} + +MLX5E_DEFINE_STATS_GRP(macsec_hw, 0); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_arfs.c b/drivers/net/ethernet/mellanox/mlx5/core/en_arfs.c index cd7f245dcf14..0ae1865086ff 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_arfs.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_arfs.c @@ -114,47 +114,49 @@ static enum mlx5_traffic_types arfs_get_tt(enum arfs_type type) } } -static int arfs_disable(struct mlx5e_priv *priv) +static int arfs_disable(struct mlx5e_flow_steering *fs) { + struct mlx5_ttc_table *ttc = mlx5e_fs_get_ttc(fs, false); int err, i; for (i = 0; i < ARFS_NUM_TYPES; i++) { /* Modify ttc rules destination back to their default */ - err = mlx5_ttc_fwd_default_dest(priv->fs->ttc, arfs_get_tt(i)); + err = mlx5_ttc_fwd_default_dest(ttc, arfs_get_tt(i)); if (err) { - netdev_err(priv->netdev, - "%s: modify ttc[%d] default destination failed, err(%d)\n", - __func__, arfs_get_tt(i), err); + fs_err(fs, + "%s: modify ttc[%d] default destination failed, err(%d)\n", + __func__, arfs_get_tt(i), err); return err; } } return 0; } -static void arfs_del_rules(struct mlx5e_priv *priv); +static void arfs_del_rules(struct mlx5e_flow_steering *fs); -int mlx5e_arfs_disable(struct mlx5e_priv *priv) +int mlx5e_arfs_disable(struct mlx5e_flow_steering *fs) { - arfs_del_rules(priv); + arfs_del_rules(fs); - return arfs_disable(priv); + return arfs_disable(fs); } -int mlx5e_arfs_enable(struct mlx5e_priv *priv) +int mlx5e_arfs_enable(struct mlx5e_flow_steering *fs) { + struct mlx5_ttc_table *ttc = mlx5e_fs_get_ttc(fs, false); + struct mlx5e_arfs_tables *arfs = mlx5e_fs_get_arfs(fs); struct mlx5_flow_destination dest = {}; int err, i; dest.type = MLX5_FLOW_DESTINATION_TYPE_FLOW_TABLE; for (i = 0; i < ARFS_NUM_TYPES; i++) { - dest.ft = priv->fs->arfs->arfs_tables[i].ft.t; + dest.ft = arfs->arfs_tables[i].ft.t; /* Modify ttc rules destination to point on the aRFS FTs */ - err = mlx5_ttc_fwd_dest(priv->fs->ttc, arfs_get_tt(i), &dest); + err = mlx5_ttc_fwd_dest(ttc, arfs_get_tt(i), &dest); if (err) { - netdev_err(priv->netdev, - "%s: modify ttc[%d] dest to arfs, failed err(%d)\n", - __func__, arfs_get_tt(i), err); - arfs_disable(priv); + fs_err(fs, "%s: modify ttc[%d] dest to arfs, failed err(%d)\n", + __func__, arfs_get_tt(i), err); + arfs_disable(fs); return err; } } @@ -167,31 +169,37 @@ static void arfs_destroy_table(struct arfs_table *arfs_t) mlx5e_destroy_flow_table(&arfs_t->ft); } -static void _mlx5e_cleanup_tables(struct mlx5e_priv *priv) +static void _mlx5e_cleanup_tables(struct mlx5e_flow_steering *fs) { + struct mlx5e_arfs_tables *arfs = mlx5e_fs_get_arfs(fs); int i; - arfs_del_rules(priv); - destroy_workqueue(priv->fs->arfs->wq); + arfs_del_rules(fs); + destroy_workqueue(arfs->wq); for (i = 0; i < ARFS_NUM_TYPES; i++) { - if (!IS_ERR_OR_NULL(priv->fs->arfs->arfs_tables[i].ft.t)) - arfs_destroy_table(&priv->fs->arfs->arfs_tables[i]); + if (!IS_ERR_OR_NULL(arfs->arfs_tables[i].ft.t)) + arfs_destroy_table(&arfs->arfs_tables[i]); } } -void mlx5e_arfs_destroy_tables(struct mlx5e_priv *priv) +void mlx5e_arfs_destroy_tables(struct mlx5e_flow_steering *fs, bool ntuple) { - if (!(priv->netdev->hw_features & NETIF_F_NTUPLE)) + struct mlx5e_arfs_tables *arfs = mlx5e_fs_get_arfs(fs); + + if (!ntuple) return; - _mlx5e_cleanup_tables(priv); - kvfree(priv->fs->arfs); + _mlx5e_cleanup_tables(fs); + mlx5e_fs_set_arfs(fs, NULL); + kvfree(arfs); } -static int arfs_add_default_rule(struct mlx5e_priv *priv, +static int arfs_add_default_rule(struct mlx5e_flow_steering *fs, + struct mlx5e_rx_res *rx_res, enum arfs_type type) { - struct arfs_table *arfs_t = &priv->fs->arfs->arfs_tables[type]; + struct mlx5e_arfs_tables *arfs = mlx5e_fs_get_arfs(fs); + struct arfs_table *arfs_t = &arfs->arfs_tables[type]; struct mlx5_flow_destination dest = {}; MLX5_DECLARE_FLOW_ACT(flow_act); enum mlx5_traffic_types tt; @@ -200,23 +208,21 @@ static int arfs_add_default_rule(struct mlx5e_priv *priv, dest.type = MLX5_FLOW_DESTINATION_TYPE_TIR; tt = arfs_get_tt(type); if (tt == -EINVAL) { - netdev_err(priv->netdev, "%s: bad arfs_type: %d\n", - __func__, type); + fs_err(fs, "%s: bad arfs_type: %d\n", __func__, type); return -EINVAL; } /* FIXME: Must use mlx5_ttc_get_default_dest(), * but can't since TTC default is not setup yet ! */ - dest.tir_num = mlx5e_rx_res_get_tirn_rss(priv->rx_res, tt); + dest.tir_num = mlx5e_rx_res_get_tirn_rss(rx_res, tt); arfs_t->default_rule = mlx5_add_flow_rules(arfs_t->ft.t, NULL, &flow_act, &dest, 1); if (IS_ERR(arfs_t->default_rule)) { err = PTR_ERR(arfs_t->default_rule); arfs_t->default_rule = NULL; - netdev_err(priv->netdev, "%s: add rule failed, arfs type=%d\n", - __func__, type); + fs_err(fs, "%s: add rule failed, arfs type=%d\n", __func__, type); } return err; @@ -318,10 +324,12 @@ out: return err; } -static int arfs_create_table(struct mlx5e_priv *priv, +static int arfs_create_table(struct mlx5e_flow_steering *fs, + struct mlx5e_rx_res *rx_res, enum arfs_type type) { - struct mlx5e_arfs_tables *arfs = priv->fs->arfs; + struct mlx5_flow_namespace *ns = mlx5e_fs_get_ns(fs, false); + struct mlx5e_arfs_tables *arfs = mlx5e_fs_get_arfs(fs); struct mlx5e_flow_table *ft = &arfs->arfs_tables[type].ft; struct mlx5_flow_table_attr ft_attr = {}; int err; @@ -332,7 +340,7 @@ static int arfs_create_table(struct mlx5e_priv *priv, ft_attr.level = MLX5E_ARFS_FT_LEVEL; ft_attr.prio = MLX5E_NIC_PRIO; - ft->t = mlx5_create_flow_table(priv->fs->ns, &ft_attr); + ft->t = mlx5_create_flow_table(ns, &ft_attr); if (IS_ERR(ft->t)) { err = PTR_ERR(ft->t); ft->t = NULL; @@ -343,7 +351,7 @@ static int arfs_create_table(struct mlx5e_priv *priv, if (err) goto err; - err = arfs_add_default_rule(priv, type); + err = arfs_add_default_rule(fs, rx_res, type); if (err) goto err; @@ -353,35 +361,40 @@ err: return err; } -int mlx5e_arfs_create_tables(struct mlx5e_priv *priv) +int mlx5e_arfs_create_tables(struct mlx5e_flow_steering *fs, + struct mlx5e_rx_res *rx_res, bool ntuple) { + struct mlx5e_arfs_tables *arfs; int err = -ENOMEM; int i; - if (!(priv->netdev->hw_features & NETIF_F_NTUPLE)) + if (!ntuple) return 0; - priv->fs->arfs = kvzalloc(sizeof(*priv->fs->arfs), GFP_KERNEL); - if (!priv->fs->arfs) + arfs = kvzalloc(sizeof(*arfs), GFP_KERNEL); + if (!arfs) return -ENOMEM; - spin_lock_init(&priv->fs->arfs->arfs_lock); - INIT_LIST_HEAD(&priv->fs->arfs->rules); - priv->fs->arfs->wq = create_singlethread_workqueue("mlx5e_arfs"); - if (!priv->fs->arfs->wq) + spin_lock_init(&arfs->arfs_lock); + INIT_LIST_HEAD(&arfs->rules); + arfs->wq = create_singlethread_workqueue("mlx5e_arfs"); + if (!arfs->wq) goto err; + mlx5e_fs_set_arfs(fs, arfs); + for (i = 0; i < ARFS_NUM_TYPES; i++) { - err = arfs_create_table(priv, i); + err = arfs_create_table(fs, rx_res, i); if (err) goto err_des; } return 0; err_des: - _mlx5e_cleanup_tables(priv); + _mlx5e_cleanup_tables(fs); err: - kvfree(priv->fs->arfs); + mlx5e_fs_set_arfs(fs, NULL); + kvfree(arfs); return err; } @@ -389,6 +402,7 @@ err: static void arfs_may_expire_flow(struct mlx5e_priv *priv) { + struct mlx5e_arfs_tables *arfs = mlx5e_fs_get_arfs(priv->fs); struct arfs_rule *arfs_rule; struct hlist_node *htmp; HLIST_HEAD(del_list); @@ -396,8 +410,8 @@ static void arfs_may_expire_flow(struct mlx5e_priv *priv) int i; int j; - spin_lock_bh(&priv->fs->arfs->arfs_lock); - mlx5e_for_each_arfs_rule(arfs_rule, htmp, priv->fs->arfs->arfs_tables, i, j) { + spin_lock_bh(&arfs->arfs_lock); + mlx5e_for_each_arfs_rule(arfs_rule, htmp, arfs->arfs_tables, i, j) { if (!work_pending(&arfs_rule->arfs_work) && rps_may_expire_flow(priv->netdev, arfs_rule->rxq, arfs_rule->flow_id, @@ -408,7 +422,7 @@ static void arfs_may_expire_flow(struct mlx5e_priv *priv) break; } } - spin_unlock_bh(&priv->fs->arfs->arfs_lock); + spin_unlock_bh(&arfs->arfs_lock); hlist_for_each_entry_safe(arfs_rule, htmp, &del_list, hlist) { if (arfs_rule->rule) mlx5_del_flow_rules(arfs_rule->rule); @@ -417,20 +431,21 @@ static void arfs_may_expire_flow(struct mlx5e_priv *priv) } } -static void arfs_del_rules(struct mlx5e_priv *priv) +static void arfs_del_rules(struct mlx5e_flow_steering *fs) { + struct mlx5e_arfs_tables *arfs = mlx5e_fs_get_arfs(fs); struct hlist_node *htmp; struct arfs_rule *rule; HLIST_HEAD(del_list); int i; int j; - spin_lock_bh(&priv->fs->arfs->arfs_lock); - mlx5e_for_each_arfs_rule(rule, htmp, priv->fs->arfs->arfs_tables, i, j) { + spin_lock_bh(&arfs->arfs_lock); + mlx5e_for_each_arfs_rule(rule, htmp, arfs->arfs_tables, i, j) { hlist_del_init(&rule->hlist); hlist_add_head(&rule->hlist, &del_list); } - spin_unlock_bh(&priv->fs->arfs->arfs_lock); + spin_unlock_bh(&arfs->arfs_lock); hlist_for_each_entry_safe(rule, htmp, &del_list, hlist) { cancel_work_sync(&rule->arfs_work); @@ -474,7 +489,7 @@ static struct arfs_table *arfs_get_table(struct mlx5e_arfs_tables *arfs, static struct mlx5_flow_handle *arfs_add_rule(struct mlx5e_priv *priv, struct arfs_rule *arfs_rule) { - struct mlx5e_arfs_tables *arfs = priv->fs->arfs; + struct mlx5e_arfs_tables *arfs = mlx5e_fs_get_arfs(priv->fs); struct arfs_tuple *tuple = &arfs_rule->tuple; struct mlx5_flow_handle *rule = NULL; struct mlx5_flow_destination dest = {}; @@ -588,13 +603,15 @@ static void arfs_handle_work(struct work_struct *work) struct arfs_rule, arfs_work); struct mlx5e_priv *priv = arfs_rule->priv; + struct mlx5e_arfs_tables *arfs; struct mlx5_flow_handle *rule; + arfs = mlx5e_fs_get_arfs(priv->fs); mutex_lock(&priv->state_lock); if (!test_bit(MLX5E_STATE_OPENED, &priv->state)) { - spin_lock_bh(&priv->fs->arfs->arfs_lock); + spin_lock_bh(&arfs->arfs_lock); hlist_del(&arfs_rule->hlist); - spin_unlock_bh(&priv->fs->arfs->arfs_lock); + spin_unlock_bh(&arfs->arfs_lock); mutex_unlock(&priv->state_lock); kfree(arfs_rule); @@ -620,6 +637,7 @@ static struct arfs_rule *arfs_alloc_rule(struct mlx5e_priv *priv, const struct flow_keys *fk, u16 rxq, u32 flow_id) { + struct mlx5e_arfs_tables *arfs = mlx5e_fs_get_arfs(priv->fs); struct arfs_rule *rule; struct arfs_tuple *tuple; @@ -647,7 +665,7 @@ static struct arfs_rule *arfs_alloc_rule(struct mlx5e_priv *priv, tuple->dst_port = fk->ports.dst; rule->flow_id = flow_id; - rule->filter_id = priv->fs->arfs->last_filter_id++ % RPS_NO_FILTER; + rule->filter_id = arfs->last_filter_id++ % RPS_NO_FILTER; hlist_add_head(&rule->hlist, arfs_hash_bucket(arfs_t, tuple->src_port, @@ -691,11 +709,12 @@ int mlx5e_rx_flow_steer(struct net_device *dev, const struct sk_buff *skb, u16 rxq_index, u32 flow_id) { struct mlx5e_priv *priv = netdev_priv(dev); - struct mlx5e_arfs_tables *arfs = priv->fs->arfs; - struct arfs_table *arfs_t; + struct mlx5e_arfs_tables *arfs; struct arfs_rule *arfs_rule; + struct arfs_table *arfs_t; struct flow_keys fk; + arfs = mlx5e_fs_get_arfs(priv->fs); if (!skb_flow_dissect_flow_keys(skb, &fk, 0)) return -EPROTONOSUPPORT; @@ -725,7 +744,7 @@ int mlx5e_rx_flow_steer(struct net_device *dev, const struct sk_buff *skb, return -ENOMEM; } } - queue_work(priv->fs->arfs->wq, &arfs_rule->arfs_work); + queue_work(arfs->wq, &arfs_rule->arfs_work); spin_unlock_bh(&arfs->arfs_lock); return arfs_rule->filter_id; } diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_common.c b/drivers/net/ethernet/mellanox/mlx5/core/en_common.c index c0f409c195bf..68f19324db93 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_common.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_common.c @@ -46,8 +46,7 @@ void mlx5e_mkey_set_relaxed_ordering(struct mlx5_core_dev *mdev, void *mkc) MLX5_SET(mkc, mkc, relaxed_ordering_write, ro_pci_enable && ro_write); } -static int mlx5e_create_mkey(struct mlx5_core_dev *mdev, u32 pdn, - u32 *mkey) +int mlx5e_create_mkey(struct mlx5_core_dev *mdev, u32 pdn, u32 *mkey) { int inlen = MLX5_ST_SZ_BYTES(create_mkey_in); void *mkc; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c b/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c index b811207fe5ed..24aa25da482b 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c @@ -38,18 +38,19 @@ #include "en/xsk/pool.h" #include "en/ptp.h" #include "lib/clock.h" +#include "en/fs_ethtool.h" void mlx5e_ethtool_get_drvinfo(struct mlx5e_priv *priv, struct ethtool_drvinfo *drvinfo) { struct mlx5_core_dev *mdev = priv->mdev; - strlcpy(drvinfo->driver, KBUILD_MODNAME, sizeof(drvinfo->driver)); + strscpy(drvinfo->driver, KBUILD_MODNAME, sizeof(drvinfo->driver)); snprintf(drvinfo->fw_version, sizeof(drvinfo->fw_version), "%d.%d.%04d (%.16s)", fw_rev_maj(mdev), fw_rev_min(mdev), fw_rev_sub(mdev), mdev->board_id); - strlcpy(drvinfo->bus_info, dev_name(mdev->device), + strscpy(drvinfo->bus_info, dev_name(mdev->device), sizeof(drvinfo->bus_info)); } @@ -310,7 +311,15 @@ void mlx5e_ethtool_get_ringparam(struct mlx5e_priv *priv, struct ethtool_ringparam *param, struct kernel_ethtool_ringparam *kernel_param) { - param->rx_max_pending = 1 << MLX5E_PARAMS_MAXIMUM_LOG_RQ_SIZE; + /* Limitation for regular RQ. XSK RQ may clamp the queue length in + * mlx5e_mpwqe_get_log_rq_size. + */ + u8 max_log_mpwrq_pkts = mlx5e_mpwrq_max_log_rq_pkts(priv->mdev, + PAGE_SHIFT, + MLX5E_MPWRQ_UMR_MODE_ALIGNED); + + param->rx_max_pending = 1 << min_t(u8, MLX5E_PARAMS_MAXIMUM_LOG_RQ_SIZE, + max_log_mpwrq_pkts); param->tx_max_pending = 1 << MLX5E_PARAMS_MAXIMUM_LOG_SQ_SIZE; param->rx_pending = 1 << priv->channels.params.log_rq_mtu_frames; param->tx_pending = 1 << priv->channels.params.log_sq_size; @@ -494,14 +503,14 @@ int mlx5e_ethtool_set_channels(struct mlx5e_priv *priv, arfs_enabled = opened && (priv->netdev->features & NETIF_F_NTUPLE); if (arfs_enabled) - mlx5e_arfs_disable(priv); + mlx5e_arfs_disable(priv->fs); /* Switch to new channels, set new parameters and close old ones */ err = mlx5e_safe_switch_params(priv, &new_params, mlx5e_num_channels_changed_ctx, NULL, true); if (arfs_enabled) { - int err2 = mlx5e_arfs_enable(priv); + int err2 = mlx5e_arfs_enable(priv->fs); if (err2) netdev_err(priv->netdev, "%s: mlx5e_arfs_enable failed: %d\n", @@ -1996,10 +2005,14 @@ static int set_pflag_rx_striding_rq(struct net_device *netdev, bool enable) struct mlx5e_params new_params; if (enable) { - if (!mlx5e_check_fragmented_striding_rq_cap(mdev)) - return -EOPNOTSUPP; - if (!mlx5e_striding_rq_possible(mdev, &priv->channels.params)) - return -EINVAL; + /* Checking the regular RQ here; mlx5e_validate_xsk_param called + * from mlx5e_open_xsk will check for each XSK queue, and + * mlx5e_safe_switch_params will be reverted if any check fails. + */ + int err = mlx5e_mpwrq_validate_regular(mdev, &priv->channels.params); + + if (err) + return err; } else if (priv->channels.params.packet_merge.type != MLX5E_PACKET_MERGE_NONE) { netdev_warn(netdev, "Can't set legacy RQ with HW-GRO/LRO, disable them first\n"); return -EINVAL; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_fs.c b/drivers/net/ethernet/mellanox/mlx5/core/en_fs.c index e0ce5a233d0b..1892ccb889b3 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_fs.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_fs.c @@ -36,10 +36,38 @@ #include <linux/tcp.h> #include <linux/mlx5/fs.h> #include <linux/mlx5/mpfs.h> -#include "en.h" #include "en_tc.h" #include "lib/mpfs.h" #include "en/ptp.h" +#include "en/fs_ethtool.h" + +struct mlx5e_flow_steering { + struct work_struct set_rx_mode_work; + bool state_destroy; + bool vlan_strip_disable; + struct mlx5_core_dev *mdev; + struct net_device *netdev; + struct mlx5_flow_namespace *ns; + struct mlx5_flow_namespace *egress_ns; +#ifdef CONFIG_MLX5_EN_RXNFC + struct mlx5e_ethtool_steering *ethtool; +#endif + struct mlx5e_tc_table *tc; + struct mlx5e_promisc_table promisc; + struct mlx5e_vlan_table *vlan; + struct mlx5e_l2_table l2; + struct mlx5_ttc_table *ttc; + struct mlx5_ttc_table *inner_ttc; +#ifdef CONFIG_MLX5_EN_ARFS + struct mlx5e_arfs_tables *arfs; +#endif +#ifdef CONFIG_MLX5_EN_TLS + struct mlx5e_accel_fs_tcp *accel_tcp; +#endif + struct mlx5e_fs_udp *udp; + struct mlx5e_fs_any *any; + struct mlx5e_ptp_fs *ptp_fs; +}; static int mlx5e_add_l2_flow_rule(struct mlx5e_flow_steering *fs, struct mlx5e_l2_rule *ai, int type); @@ -148,9 +176,8 @@ static int mlx5e_vport_context_update_vlans(struct mlx5e_flow_steering *fs) max_list_size = 1 << MLX5_CAP_GEN(fs->mdev, log_max_vlan_list); if (list_size > max_list_size) { - mlx5_core_warn(fs->mdev, - "netdev vlans list size (%d) > (%d) max vport list size, some vlans will be dropped\n", - list_size, max_list_size); + fs_warn(fs, "netdev vlans list size (%d) > (%d) max vport list size, some vlans will be dropped\n", + list_size, max_list_size); list_size = max_list_size; } @@ -167,8 +194,8 @@ static int mlx5e_vport_context_update_vlans(struct mlx5e_flow_steering *fs) err = mlx5_modify_nic_vport_vlans(fs->mdev, vlans, list_size); if (err) - mlx5_core_err(fs->mdev, "Failed to modify vport vlans list err(%d)\n", - err); + fs_err(fs, "Failed to modify vport vlans list err(%d)\n", + err); kvfree(vlans); return err; @@ -249,7 +276,7 @@ static int __mlx5e_add_vlan_rule(struct mlx5e_flow_steering *fs, if (IS_ERR(*rule_p)) { err = PTR_ERR(*rule_p); *rule_p = NULL; - mlx5_core_err(fs->mdev, "%s: add rule failed\n", __func__); + fs_err(fs, "%s: add rule failed\n", __func__); } return err; @@ -351,78 +378,78 @@ mlx5e_add_trap_rule(struct mlx5_flow_table *ft, int trap_id, int tir_num) return rule; } -int mlx5e_add_vlan_trap(struct mlx5e_priv *priv, int trap_id, int tir_num) +int mlx5e_add_vlan_trap(struct mlx5e_flow_steering *fs, int trap_id, int tir_num) { - struct mlx5_flow_table *ft = priv->fs->vlan->ft.t; + struct mlx5_flow_table *ft = fs->vlan->ft.t; struct mlx5_flow_handle *rule; int err; rule = mlx5e_add_trap_rule(ft, trap_id, tir_num); if (IS_ERR(rule)) { err = PTR_ERR(rule); - priv->fs->vlan->trap_rule = NULL; - mlx5_core_err(priv->fs->mdev, "%s: add VLAN trap rule failed, err %d\n", - __func__, err); + fs->vlan->trap_rule = NULL; + fs_err(fs, "%s: add VLAN trap rule failed, err %d\n", + __func__, err); return err; } - priv->fs->vlan->trap_rule = rule; + fs->vlan->trap_rule = rule; return 0; } -void mlx5e_remove_vlan_trap(struct mlx5e_priv *priv) +void mlx5e_remove_vlan_trap(struct mlx5e_flow_steering *fs) { - if (priv->fs->vlan->trap_rule) { - mlx5_del_flow_rules(priv->fs->vlan->trap_rule); - priv->fs->vlan->trap_rule = NULL; + if (fs->vlan->trap_rule) { + mlx5_del_flow_rules(fs->vlan->trap_rule); + fs->vlan->trap_rule = NULL; } } -int mlx5e_add_mac_trap(struct mlx5e_priv *priv, int trap_id, int tir_num) +int mlx5e_add_mac_trap(struct mlx5e_flow_steering *fs, int trap_id, int tir_num) { - struct mlx5_flow_table *ft = priv->fs->l2.ft.t; + struct mlx5_flow_table *ft = fs->l2.ft.t; struct mlx5_flow_handle *rule; int err; rule = mlx5e_add_trap_rule(ft, trap_id, tir_num); if (IS_ERR(rule)) { err = PTR_ERR(rule); - priv->fs->l2.trap_rule = NULL; - mlx5_core_err(priv->fs->mdev, "%s: add MAC trap rule failed, err %d\n", - __func__, err); + fs->l2.trap_rule = NULL; + fs_err(fs, "%s: add MAC trap rule failed, err %d\n", + __func__, err); return err; } - priv->fs->l2.trap_rule = rule; + fs->l2.trap_rule = rule; return 0; } -void mlx5e_remove_mac_trap(struct mlx5e_priv *priv) +void mlx5e_remove_mac_trap(struct mlx5e_flow_steering *fs) { - if (priv->fs->l2.trap_rule) { - mlx5_del_flow_rules(priv->fs->l2.trap_rule); - priv->fs->l2.trap_rule = NULL; + if (fs->l2.trap_rule) { + mlx5_del_flow_rules(fs->l2.trap_rule); + fs->l2.trap_rule = NULL; } } -void mlx5e_enable_cvlan_filter(struct mlx5e_priv *priv) +void mlx5e_enable_cvlan_filter(struct mlx5e_flow_steering *fs, bool promisc) { - if (!priv->fs->vlan->cvlan_filter_disabled) + if (!fs->vlan->cvlan_filter_disabled) return; - priv->fs->vlan->cvlan_filter_disabled = false; - if (priv->netdev->flags & IFF_PROMISC) + fs->vlan->cvlan_filter_disabled = false; + if (promisc) return; - mlx5e_fs_del_vlan_rule(priv->fs, MLX5E_VLAN_RULE_TYPE_ANY_CTAG_VID, 0); + mlx5e_fs_del_vlan_rule(fs, MLX5E_VLAN_RULE_TYPE_ANY_CTAG_VID, 0); } -void mlx5e_disable_cvlan_filter(struct mlx5e_priv *priv) +void mlx5e_disable_cvlan_filter(struct mlx5e_flow_steering *fs, bool promisc) { - if (priv->fs->vlan->cvlan_filter_disabled) + if (fs->vlan->cvlan_filter_disabled) return; - priv->fs->vlan->cvlan_filter_disabled = true; - if (priv->netdev->flags & IFF_PROMISC) + fs->vlan->cvlan_filter_disabled = true; + if (promisc) return; - mlx5e_add_vlan_rule(priv->fs, MLX5E_VLAN_RULE_TYPE_ANY_CTAG_VID, 0); + mlx5e_add_vlan_rule(fs, MLX5E_VLAN_RULE_TYPE_ANY_CTAG_VID, 0); } static int mlx5e_vlan_rx_add_cvid(struct mlx5e_flow_steering *fs, u16 vid) @@ -462,7 +489,7 @@ int mlx5e_fs_vlan_rx_add_vid(struct mlx5e_flow_steering *fs, { if (!fs->vlan) { - mlx5_core_err(fs->mdev, "Vlan doesn't exist\n"); + fs_err(fs, "Vlan doesn't exist\n"); return -EINVAL; } @@ -479,7 +506,7 @@ int mlx5e_fs_vlan_rx_kill_vid(struct mlx5e_flow_steering *fs, __be16 proto, u16 vid) { if (!fs->vlan) { - mlx5_core_err(fs->mdev, "Vlan doesn't exist\n"); + fs_err(fs, "Vlan doesn't exist\n"); return -EINVAL; } @@ -512,28 +539,28 @@ static void mlx5e_fs_add_vlan_rules(struct mlx5e_flow_steering *fs) mlx5e_fs_add_any_vid_rules(fs); } -static void mlx5e_del_vlan_rules(struct mlx5e_priv *priv) +static void mlx5e_del_vlan_rules(struct mlx5e_flow_steering *fs) { int i; - mlx5e_fs_del_vlan_rule(priv->fs, MLX5E_VLAN_RULE_TYPE_UNTAGGED, 0); + mlx5e_fs_del_vlan_rule(fs, MLX5E_VLAN_RULE_TYPE_UNTAGGED, 0); - for_each_set_bit(i, priv->fs->vlan->active_cvlans, VLAN_N_VID) { - mlx5e_fs_del_vlan_rule(priv->fs, MLX5E_VLAN_RULE_TYPE_MATCH_CTAG_VID, i); + for_each_set_bit(i, fs->vlan->active_cvlans, VLAN_N_VID) { + mlx5e_fs_del_vlan_rule(fs, MLX5E_VLAN_RULE_TYPE_MATCH_CTAG_VID, i); } - for_each_set_bit(i, priv->fs->vlan->active_svlans, VLAN_N_VID) - mlx5e_fs_del_vlan_rule(priv->fs, MLX5E_VLAN_RULE_TYPE_MATCH_STAG_VID, i); + for_each_set_bit(i, fs->vlan->active_svlans, VLAN_N_VID) + mlx5e_fs_del_vlan_rule(fs, MLX5E_VLAN_RULE_TYPE_MATCH_STAG_VID, i); - WARN_ON_ONCE(priv->fs->state_destroy); + WARN_ON_ONCE(fs->state_destroy); - mlx5e_remove_vlan_trap(priv); + mlx5e_remove_vlan_trap(fs); /* must be called after DESTROY bit is set and * set_rx_mode is called and flushed */ - if (priv->fs->vlan->cvlan_filter_disabled) - mlx5e_fs_del_any_vid_rules(priv->fs); + if (fs->vlan->cvlan_filter_disabled) + mlx5e_fs_del_any_vid_rules(fs); } #define mlx5e_for_each_hash_node(hn, tmp, hash, i) \ @@ -568,8 +595,9 @@ static void mlx5e_execute_l2_action(struct mlx5e_flow_steering *fs, } if (l2_err) - mlx5_core_warn(fs->mdev, "MPFS, failed to %s mac %pM, err(%d)\n", - action == MLX5E_ACTION_ADD ? "add" : "del", mac_addr, l2_err); + fs_warn(fs, "MPFS, failed to %s mac %pM, err(%d)\n", + action == MLX5E_ACTION_ADD ? "add" : "del", + mac_addr, l2_err); } static void mlx5e_sync_netdev_addr(struct mlx5e_flow_steering *fs, @@ -640,9 +668,8 @@ static void mlx5e_vport_context_update_addr_list(struct mlx5e_flow_steering *fs, size++; if (size > max_size) { - mlx5_core_warn(fs->mdev, - "mdev %s list size (%d) > (%d) max vport list size, some addresses will be dropped\n", - is_uc ? "UC" : "MC", size, max_size); + fs_warn(fs, "mdev %s list size (%d) > (%d) max vport list size, some addresses will be dropped\n", + is_uc ? "UC" : "MC", size, max_size); size = max_size; } @@ -658,9 +685,8 @@ static void mlx5e_vport_context_update_addr_list(struct mlx5e_flow_steering *fs, err = mlx5_modify_nic_vport_mac_list(fs->mdev, list_type, addr_array, size); out: if (err) - mlx5_core_err(fs->mdev, - "Failed to modify vport %s list err(%d)\n", - is_uc ? "UC" : "MC", err); + fs_err(fs, "Failed to modify vport %s list err(%d)\n", + is_uc ? "UC" : "MC", err); kfree(addr_array); } @@ -730,7 +756,7 @@ static int mlx5e_add_promisc_rule(struct mlx5e_flow_steering *fs) if (IS_ERR(*rule_p)) { err = PTR_ERR(*rule_p); *rule_p = NULL; - mlx5_core_err(fs->mdev, "%s: add promiscuous rule failed\n", __func__); + fs_err(fs, "%s: add promiscuous rule failed\n", __func__); } kvfree(spec); return err; @@ -750,7 +776,7 @@ static int mlx5e_create_promisc_table(struct mlx5e_flow_steering *fs) ft->t = mlx5_create_auto_grouped_flow_table(fs->ns, &ft_attr); if (IS_ERR(ft->t)) { err = PTR_ERR(ft->t); - mlx5_core_err(fs->mdev, "fail to create promisc table err=%d\n", err); + fs_err(fs, "fail to create promisc table err=%d\n", err); return err; } @@ -807,8 +833,8 @@ void mlx5e_fs_set_rx_mode_work(struct mlx5e_flow_steering *fs, if (err) enable_promisc = false; if (!fs->vlan_strip_disable && !err) - mlx5_core_warn_once(fs->mdev, - "S-tagged traffic will be dropped while C-tag vlan stripping is enabled\n"); + fs_warn_once(fs, + "S-tagged traffic will be dropped while C-tag vlan stripping is enabled\n"); } if (enable_allmulti) mlx5e_add_l2_flow_rule(fs, &ea->allmulti, MLX5E_ALLMULTI); @@ -856,14 +882,15 @@ void mlx5e_destroy_flow_table(struct mlx5e_flow_table *ft) ft->t = NULL; } -static void mlx5e_set_inner_ttc_params(struct mlx5e_priv *priv, +static void mlx5e_set_inner_ttc_params(struct mlx5e_flow_steering *fs, + struct mlx5e_rx_res *rx_res, struct ttc_params *ttc_params) { struct mlx5_flow_table_attr *ft_attr = &ttc_params->ft_attr; int tt; memset(ttc_params, 0, sizeof(*ttc_params)); - ttc_params->ns = mlx5_get_flow_namespace(priv->fs->mdev, + ttc_params->ns = mlx5_get_flow_namespace(fs->mdev, MLX5_FLOW_NAMESPACE_KERNEL); ft_attr->level = MLX5E_INNER_TTC_FT_LEVEL; ft_attr->prio = MLX5E_NIC_PRIO; @@ -872,13 +899,14 @@ static void mlx5e_set_inner_ttc_params(struct mlx5e_priv *priv, ttc_params->dests[tt].type = MLX5_FLOW_DESTINATION_TYPE_TIR; ttc_params->dests[tt].tir_num = tt == MLX5_TT_ANY ? - mlx5e_rx_res_get_tirn_direct(priv->rx_res, 0) : - mlx5e_rx_res_get_tirn_rss_inner(priv->rx_res, + mlx5e_rx_res_get_tirn_direct(rx_res, 0) : + mlx5e_rx_res_get_tirn_rss_inner(rx_res, tt); } } -void mlx5e_set_ttc_params(struct mlx5e_priv *priv, +void mlx5e_set_ttc_params(struct mlx5e_flow_steering *fs, + struct mlx5e_rx_res *rx_res, struct ttc_params *ttc_params, bool tunnel) { @@ -886,7 +914,7 @@ void mlx5e_set_ttc_params(struct mlx5e_priv *priv, int tt; memset(ttc_params, 0, sizeof(*ttc_params)); - ttc_params->ns = mlx5_get_flow_namespace(priv->fs->mdev, + ttc_params->ns = mlx5_get_flow_namespace(fs->mdev, MLX5_FLOW_NAMESPACE_KERNEL); ft_attr->level = MLX5E_TTC_FT_LEVEL; ft_attr->prio = MLX5E_NIC_PRIO; @@ -895,19 +923,19 @@ void mlx5e_set_ttc_params(struct mlx5e_priv *priv, ttc_params->dests[tt].type = MLX5_FLOW_DESTINATION_TYPE_TIR; ttc_params->dests[tt].tir_num = tt == MLX5_TT_ANY ? - mlx5e_rx_res_get_tirn_direct(priv->rx_res, 0) : - mlx5e_rx_res_get_tirn_rss(priv->rx_res, tt); + mlx5e_rx_res_get_tirn_direct(rx_res, 0) : + mlx5e_rx_res_get_tirn_rss(rx_res, tt); } ttc_params->inner_ttc = tunnel; - if (!tunnel || !mlx5_tunnel_inner_ft_supported(priv->fs->mdev)) + if (!tunnel || !mlx5_tunnel_inner_ft_supported(fs->mdev)) return; for (tt = 0; tt < MLX5_NUM_TUNNEL_TT; tt++) { ttc_params->tunnel_dests[tt].type = MLX5_FLOW_DESTINATION_TYPE_FLOW_TABLE; ttc_params->tunnel_dests[tt].ft = - mlx5_get_ttc_flow_table(priv->fs->inner_ttc); + mlx5_get_ttc_flow_table(fs->inner_ttc); } } @@ -959,8 +987,7 @@ static int mlx5e_add_l2_flow_rule(struct mlx5e_flow_steering *fs, ai->rule = mlx5_add_flow_rules(ft, spec, &flow_act, &dest, 1); if (IS_ERR(ai->rule)) { - mlx5_core_err(fs->mdev, "%s: add l2 rule(mac:%pM) failed\n", - __func__, mv_dmac); + fs_err(fs, "%s: add l2 rule(mac:%pM) failed\n", __func__, mv_dmac); err = PTR_ERR(ai->rule); ai->rule = NULL; } @@ -1044,14 +1071,14 @@ err_destroy_groups: return err; } -static void mlx5e_destroy_l2_table(struct mlx5e_priv *priv) +static void mlx5e_destroy_l2_table(struct mlx5e_flow_steering *fs) { - mlx5e_destroy_flow_table(&priv->fs->l2.ft); + mlx5e_destroy_flow_table(&fs->l2.ft); } -static int mlx5e_create_l2_table(struct mlx5e_priv *priv) +static int mlx5e_create_l2_table(struct mlx5e_flow_steering *fs) { - struct mlx5e_l2_table *l2_table = &priv->fs->l2; + struct mlx5e_l2_table *l2_table = &fs->l2; struct mlx5e_flow_table *ft = &l2_table->ft; struct mlx5_flow_table_attr ft_attr = {}; int err; @@ -1062,7 +1089,7 @@ static int mlx5e_create_l2_table(struct mlx5e_priv *priv) ft_attr.level = MLX5E_L2_FT_LEVEL; ft_attr.prio = MLX5E_NIC_PRIO; - ft->t = mlx5_create_flow_table(priv->fs->ns, &ft_attr); + ft->t = mlx5_create_flow_table(fs->ns, &ft_attr); if (IS_ERR(ft->t)) { err = PTR_ERR(ft->t); ft->t = NULL; @@ -1221,126 +1248,128 @@ err_destroy_vlan_table: return err; } -static void mlx5e_destroy_vlan_table(struct mlx5e_priv *priv) +static void mlx5e_destroy_vlan_table(struct mlx5e_flow_steering *fs) { - mlx5e_del_vlan_rules(priv); - mlx5e_destroy_flow_table(&priv->fs->vlan->ft); + mlx5e_del_vlan_rules(fs); + mlx5e_destroy_flow_table(&fs->vlan->ft); } -static void mlx5e_destroy_inner_ttc_table(struct mlx5e_priv *priv) +static void mlx5e_destroy_inner_ttc_table(struct mlx5e_flow_steering *fs) { - if (!mlx5_tunnel_inner_ft_supported(priv->fs->mdev)) + if (!mlx5_tunnel_inner_ft_supported(fs->mdev)) return; - mlx5_destroy_ttc_table(priv->fs->inner_ttc); + mlx5_destroy_ttc_table(fs->inner_ttc); } -void mlx5e_destroy_ttc_table(struct mlx5e_priv *priv) +void mlx5e_destroy_ttc_table(struct mlx5e_flow_steering *fs) { - mlx5_destroy_ttc_table(priv->fs->ttc); + mlx5_destroy_ttc_table(fs->ttc); } -static int mlx5e_create_inner_ttc_table(struct mlx5e_priv *priv) +static int mlx5e_create_inner_ttc_table(struct mlx5e_flow_steering *fs, + struct mlx5e_rx_res *rx_res) { struct ttc_params ttc_params = {}; - if (!mlx5_tunnel_inner_ft_supported(priv->fs->mdev)) + if (!mlx5_tunnel_inner_ft_supported(fs->mdev)) return 0; - mlx5e_set_inner_ttc_params(priv, &ttc_params); - priv->fs->inner_ttc = mlx5_create_inner_ttc_table(priv->fs->mdev, - &ttc_params); - if (IS_ERR(priv->fs->inner_ttc)) - return PTR_ERR(priv->fs->inner_ttc); + mlx5e_set_inner_ttc_params(fs, rx_res, &ttc_params); + fs->inner_ttc = mlx5_create_inner_ttc_table(fs->mdev, + &ttc_params); + if (IS_ERR(fs->inner_ttc)) + return PTR_ERR(fs->inner_ttc); return 0; } -int mlx5e_create_ttc_table(struct mlx5e_priv *priv) +int mlx5e_create_ttc_table(struct mlx5e_flow_steering *fs, + struct mlx5e_rx_res *rx_res) { struct ttc_params ttc_params = {}; - mlx5e_set_ttc_params(priv, &ttc_params, true); - priv->fs->ttc = mlx5_create_ttc_table(priv->fs->mdev, &ttc_params); - if (IS_ERR(priv->fs->ttc)) - return PTR_ERR(priv->fs->ttc); + mlx5e_set_ttc_params(fs, rx_res, &ttc_params, true); + fs->ttc = mlx5_create_ttc_table(fs->mdev, &ttc_params); + if (IS_ERR(fs->ttc)) + return PTR_ERR(fs->ttc); return 0; } -int mlx5e_create_flow_steering(struct mlx5e_priv *priv) +int mlx5e_create_flow_steering(struct mlx5e_flow_steering *fs, + struct mlx5e_rx_res *rx_res, + const struct mlx5e_profile *profile, + struct net_device *netdev) { + struct mlx5_flow_namespace *ns = mlx5_get_flow_namespace(fs->mdev, + MLX5_FLOW_NAMESPACE_KERNEL); int err; - priv->fs->ns = mlx5_get_flow_namespace(priv->fs->mdev, - MLX5_FLOW_NAMESPACE_KERNEL); - - if (!priv->fs->ns) + if (!ns) return -EOPNOTSUPP; - err = mlx5e_arfs_create_tables(priv); + mlx5e_fs_set_ns(fs, ns, false); + err = mlx5e_arfs_create_tables(fs, rx_res, + !!(netdev->hw_features & NETIF_F_NTUPLE)); if (err) { - mlx5_core_err(priv->fs->mdev, "Failed to create arfs tables, err=%d\n", - err); - priv->netdev->hw_features &= ~NETIF_F_NTUPLE; + fs_err(fs, "Failed to create arfs tables, err=%d\n", err); + netdev->hw_features &= ~NETIF_F_NTUPLE; } - err = mlx5e_create_inner_ttc_table(priv); + err = mlx5e_create_inner_ttc_table(fs, rx_res); if (err) { - mlx5_core_err(priv->fs->mdev, - "Failed to create inner ttc table, err=%d\n", err); + fs_err(fs, "Failed to create inner ttc table, err=%d\n", err); goto err_destroy_arfs_tables; } - err = mlx5e_create_ttc_table(priv); + err = mlx5e_create_ttc_table(fs, rx_res); if (err) { - mlx5_core_err(priv->fs->mdev, "Failed to create ttc table, err=%d\n", - err); + fs_err(fs, "Failed to create ttc table, err=%d\n", err); goto err_destroy_inner_ttc_table; } - err = mlx5e_create_l2_table(priv); + err = mlx5e_create_l2_table(fs); if (err) { - mlx5_core_err(priv->fs->mdev, "Failed to create l2 table, err=%d\n", - err); + fs_err(fs, "Failed to create l2 table, err=%d\n", err); goto err_destroy_ttc_table; } - err = mlx5e_fs_create_vlan_table(priv->fs); + err = mlx5e_fs_create_vlan_table(fs); if (err) { - mlx5_core_err(priv->fs->mdev, "Failed to create vlan table, err=%d\n", - err); + fs_err(fs, "Failed to create vlan table, err=%d\n", err); goto err_destroy_l2_table; } - err = mlx5e_ptp_alloc_rx_fs(priv); + err = mlx5e_ptp_alloc_rx_fs(fs, profile); if (err) goto err_destory_vlan_table; - mlx5e_ethtool_init_steering(priv); + mlx5e_ethtool_init_steering(fs); return 0; err_destory_vlan_table: - mlx5e_destroy_vlan_table(priv); + mlx5e_destroy_vlan_table(fs); err_destroy_l2_table: - mlx5e_destroy_l2_table(priv); + mlx5e_destroy_l2_table(fs); err_destroy_ttc_table: - mlx5e_destroy_ttc_table(priv); + mlx5e_destroy_ttc_table(fs); err_destroy_inner_ttc_table: - mlx5e_destroy_inner_ttc_table(priv); + mlx5e_destroy_inner_ttc_table(fs); err_destroy_arfs_tables: - mlx5e_arfs_destroy_tables(priv); + mlx5e_arfs_destroy_tables(fs, !!(netdev->hw_features & NETIF_F_NTUPLE)); return err; } -void mlx5e_destroy_flow_steering(struct mlx5e_priv *priv) +void mlx5e_destroy_flow_steering(struct mlx5e_flow_steering *fs, bool ntuple, + const struct mlx5e_profile *profile) { - mlx5e_ptp_free_rx_fs(priv); - mlx5e_destroy_vlan_table(priv); - mlx5e_destroy_l2_table(priv); - mlx5e_destroy_ttc_table(priv); - mlx5e_destroy_inner_ttc_table(priv); - mlx5e_arfs_destroy_tables(priv); - mlx5e_ethtool_cleanup_steering(priv); + mlx5e_ptp_free_rx_fs(fs, profile); + mlx5e_destroy_vlan_table(fs); + mlx5e_destroy_l2_table(fs); + mlx5e_destroy_ttc_table(fs); + mlx5e_destroy_inner_ttc_table(fs); + mlx5e_arfs_destroy_tables(fs, ntuple); + mlx5e_ethtool_cleanup_steering(fs); } static int mlx5e_fs_vlan_alloc(struct mlx5e_flow_steering *fs) @@ -1356,6 +1385,11 @@ static void mlx5e_fs_vlan_free(struct mlx5e_flow_steering *fs) kvfree(fs->vlan); } +struct mlx5e_vlan_table *mlx5e_fs_get_vlan(struct mlx5e_flow_steering *fs) +{ + return fs->vlan; +} + static int mlx5e_fs_tc_alloc(struct mlx5e_flow_steering *fs) { fs->tc = mlx5e_tc_table_alloc(); @@ -1369,6 +1403,32 @@ static void mlx5e_fs_tc_free(struct mlx5e_flow_steering *fs) mlx5e_tc_table_free(fs->tc); } +struct mlx5e_tc_table *mlx5e_fs_get_tc(struct mlx5e_flow_steering *fs) +{ + return fs->tc; +} + +#ifdef CONFIG_MLX5_EN_RXNFC +static int mlx5e_fs_ethtool_alloc(struct mlx5e_flow_steering *fs) +{ + return mlx5e_ethtool_alloc(&fs->ethtool); +} + +static void mlx5e_fs_ethtool_free(struct mlx5e_flow_steering *fs) +{ + mlx5e_ethtool_free(fs->ethtool); +} + +struct mlx5e_ethtool_steering *mlx5e_fs_get_ethtool(struct mlx5e_flow_steering *fs) +{ + return fs->ethtool; +} +#else +static int mlx5e_fs_ethtool_alloc(struct mlx5e_flow_steering *fs) +{ return 0; } +static void mlx5e_fs_ethtool_free(struct mlx5e_flow_steering *fs) { } +#endif + struct mlx5e_flow_steering *mlx5e_fs_init(const struct mlx5e_profile *profile, struct mlx5_core_dev *mdev, bool state_destroy) @@ -1394,8 +1454,13 @@ struct mlx5e_flow_steering *mlx5e_fs_init(const struct mlx5e_profile *profile, goto err_free_vlan; } - return fs; + err = mlx5e_fs_ethtool_alloc(fs); + if (err) + goto err_free_tc; + return fs; +err_free_tc: + mlx5e_fs_tc_free(fs); err_free_vlan: mlx5e_fs_vlan_free(fs); err_free_fs: @@ -1406,7 +1471,109 @@ err: void mlx5e_fs_cleanup(struct mlx5e_flow_steering *fs) { + mlx5e_fs_ethtool_free(fs); mlx5e_fs_tc_free(fs); mlx5e_fs_vlan_free(fs); kvfree(fs); } + +struct mlx5e_l2_table *mlx5e_fs_get_l2(struct mlx5e_flow_steering *fs) +{ + return &fs->l2; +} + +struct mlx5_flow_namespace *mlx5e_fs_get_ns(struct mlx5e_flow_steering *fs, bool egress) +{ + return egress ? fs->egress_ns : fs->ns; +} + +void mlx5e_fs_set_ns(struct mlx5e_flow_steering *fs, struct mlx5_flow_namespace *ns, bool egress) +{ + if (!egress) + fs->ns = ns; + else + fs->egress_ns = ns; +} + +struct mlx5_ttc_table *mlx5e_fs_get_ttc(struct mlx5e_flow_steering *fs, bool inner) +{ + return inner ? fs->inner_ttc : fs->ttc; +} + +void mlx5e_fs_set_ttc(struct mlx5e_flow_steering *fs, struct mlx5_ttc_table *ttc, bool inner) +{ + if (!inner) + fs->ttc = ttc; + else + fs->inner_ttc = ttc; +} + +#ifdef CONFIG_MLX5_EN_ARFS +struct mlx5e_arfs_tables *mlx5e_fs_get_arfs(struct mlx5e_flow_steering *fs) +{ + return fs->arfs; +} + +void mlx5e_fs_set_arfs(struct mlx5e_flow_steering *fs, struct mlx5e_arfs_tables *arfs) +{ + fs->arfs = arfs; +} +#endif + +struct mlx5e_ptp_fs *mlx5e_fs_get_ptp(struct mlx5e_flow_steering *fs) +{ + return fs->ptp_fs; +} + +void mlx5e_fs_set_ptp(struct mlx5e_flow_steering *fs, struct mlx5e_ptp_fs *ptp_fs) +{ + fs->ptp_fs = ptp_fs; +} + +struct mlx5e_fs_any *mlx5e_fs_get_any(struct mlx5e_flow_steering *fs) +{ + return fs->any; +} + +void mlx5e_fs_set_any(struct mlx5e_flow_steering *fs, struct mlx5e_fs_any *any) +{ + fs->any = any; +} + +#ifdef CONFIG_MLX5_EN_TLS +struct mlx5e_accel_fs_tcp *mlx5e_fs_get_accel_tcp(struct mlx5e_flow_steering *fs) +{ + return fs->accel_tcp; +} + +void mlx5e_fs_set_accel_tcp(struct mlx5e_flow_steering *fs, struct mlx5e_accel_fs_tcp *accel_tcp) +{ + fs->accel_tcp = accel_tcp; +} +#endif + +void mlx5e_fs_set_state_destroy(struct mlx5e_flow_steering *fs, bool state_destroy) +{ + fs->state_destroy = state_destroy; +} + +void mlx5e_fs_set_vlan_strip_disable(struct mlx5e_flow_steering *fs, + bool vlan_strip_disable) +{ + fs->vlan_strip_disable = vlan_strip_disable; +} + +struct mlx5e_fs_udp *mlx5e_fs_get_udp(struct mlx5e_flow_steering *fs) +{ + return fs->udp; +} + +void mlx5e_fs_set_udp(struct mlx5e_flow_steering *fs, struct mlx5e_fs_udp *udp) +{ + fs->udp = udp; +} + +struct mlx5_core_dev *mlx5e_fs_get_mdev(struct mlx5e_flow_steering *fs) +{ + return fs->mdev; +} diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_fs_ethtool.c b/drivers/net/ethernet/mellanox/mlx5/core/en_fs_ethtool.c index 3e4bc7836ef4..aac32e505c14 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_fs_ethtool.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_fs_ethtool.c @@ -34,6 +34,22 @@ #include "en.h" #include "en/params.h" #include "en/xsk/pool.h" +#include "en/fs_ethtool.h" + +struct mlx5e_ethtool_table { + struct mlx5_flow_table *ft; + int num_rules; +}; + +#define ETHTOOL_NUM_L3_L4_FTS 7 +#define ETHTOOL_NUM_L2_FTS 4 + +struct mlx5e_ethtool_steering { + struct mlx5e_ethtool_table l3_l4_ft[ETHTOOL_NUM_L3_L4_FTS]; + struct mlx5e_ethtool_table l2_ft[ETHTOOL_NUM_L2_FTS]; + struct list_head rules; + int tot_num_rules; +}; static int flow_type_to_traffic_type(u32 flow_type); @@ -66,6 +82,7 @@ static struct mlx5e_ethtool_table *get_flow_table(struct mlx5e_priv *priv, struct ethtool_rx_flow_spec *fs, int num_tuples) { + struct mlx5e_ethtool_steering *ethtool = mlx5e_fs_get_ethtool(priv->fs); struct mlx5_flow_table_attr ft_attr = {}; struct mlx5e_ethtool_table *eth_ft; struct mlx5_flow_namespace *ns; @@ -81,18 +98,18 @@ static struct mlx5e_ethtool_table *get_flow_table(struct mlx5e_priv *priv, case UDP_V6_FLOW: max_tuples = ETHTOOL_NUM_L3_L4_FTS; prio = MLX5E_ETHTOOL_L3_L4_PRIO + (max_tuples - num_tuples); - eth_ft = &priv->fs->ethtool.l3_l4_ft[prio]; + eth_ft = ðtool->l3_l4_ft[prio]; break; case IP_USER_FLOW: case IPV6_USER_FLOW: max_tuples = ETHTOOL_NUM_L3_L4_FTS; prio = MLX5E_ETHTOOL_L3_L4_PRIO + (max_tuples - num_tuples); - eth_ft = &priv->fs->ethtool.l3_l4_ft[prio]; + eth_ft = ðtool->l3_l4_ft[prio]; break; case ETHER_FLOW: max_tuples = ETHTOOL_NUM_L2_FTS; prio = max_tuples - num_tuples; - eth_ft = &priv->fs->ethtool.l2_ft[prio]; + eth_ft = ðtool->l2_ft[prio]; prio += MLX5E_ETHTOOL_L2_PRIO; break; default: @@ -382,15 +399,16 @@ static int set_flow_attrs(u32 *match_c, u32 *match_v, static void add_rule_to_list(struct mlx5e_priv *priv, struct mlx5e_ethtool_rule *rule) { + struct mlx5e_ethtool_steering *ethtool = mlx5e_fs_get_ethtool(priv->fs); + struct list_head *head = ðtool->rules; struct mlx5e_ethtool_rule *iter; - struct list_head *head = &priv->fs->ethtool.rules; - list_for_each_entry(iter, &priv->fs->ethtool.rules, list) { + list_for_each_entry(iter, ðtool->rules, list) { if (iter->flow_spec.location > rule->flow_spec.location) break; head = &iter->list; } - priv->fs->ethtool.tot_num_rules++; + ethtool->tot_num_rules++; list_add(&rule->list, head); } @@ -433,15 +451,7 @@ static int flow_get_tirn(struct mlx5e_priv *priv, eth_rule->rss = rss; mlx5e_rss_refcnt_inc(eth_rule->rss); } else { - struct mlx5e_params *params = &priv->channels.params; - enum mlx5e_rq_group group; - u16 ix; - - mlx5e_qid_get_ch_and_group(params, fs->ring_cookie, &ix, &group); - - *tirn = group == MLX5E_RQ_GROUP_XSK ? - mlx5e_rx_res_get_tirn_xsk(priv->rx_res, ix) : - mlx5e_rx_res_get_tirn_direct(priv->rx_res, ix); + *tirn = mlx5e_rx_res_get_tirn_direct(priv->rx_res, fs->ring_cookie); } return 0; @@ -499,15 +509,16 @@ free: return err ? ERR_PTR(err) : rule; } -static void del_ethtool_rule(struct mlx5e_priv *priv, +static void del_ethtool_rule(struct mlx5e_flow_steering *fs, struct mlx5e_ethtool_rule *eth_rule) { + struct mlx5e_ethtool_steering *ethtool = mlx5e_fs_get_ethtool(fs); if (eth_rule->rule) mlx5_del_flow_rules(eth_rule->rule); if (eth_rule->rss) mlx5e_rss_refcnt_dec(eth_rule->rss); list_del(ð_rule->list); - priv->fs->ethtool.tot_num_rules--; + ethtool->tot_num_rules--; put_flow_table(eth_rule->eth_ft); kfree(eth_rule); } @@ -515,9 +526,10 @@ static void del_ethtool_rule(struct mlx5e_priv *priv, static struct mlx5e_ethtool_rule *find_ethtool_rule(struct mlx5e_priv *priv, int location) { + struct mlx5e_ethtool_steering *ethtool = mlx5e_fs_get_ethtool(priv->fs); struct mlx5e_ethtool_rule *iter; - list_for_each_entry(iter, &priv->fs->ethtool.rules, list) { + list_for_each_entry(iter, ðtool->rules, list) { if (iter->flow_spec.location == location) return iter; } @@ -531,7 +543,7 @@ static struct mlx5e_ethtool_rule *get_ethtool_rule(struct mlx5e_priv *priv, eth_rule = find_ethtool_rule(priv, location); if (eth_rule) - del_ethtool_rule(priv, eth_rule); + del_ethtool_rule(priv->fs, eth_rule); eth_rule = kzalloc(sizeof(*eth_rule), GFP_KERNEL); if (!eth_rule) @@ -662,8 +674,7 @@ static int validate_flow(struct mlx5e_priv *priv, return -ENOSPC; if (fs->ring_cookie != RX_CLS_FLOW_DISC) - if (!mlx5e_qid_validate(priv->profile, &priv->channels.params, - fs->ring_cookie)) + if (fs->ring_cookie >= priv->channels.params.num_channels) return -EINVAL; switch (flow_type_mask(fs->flow_type)) { @@ -754,7 +765,7 @@ mlx5e_ethtool_flow_replace(struct mlx5e_priv *priv, return 0; del_ethtool_rule: - del_ethtool_rule(priv, eth_rule); + del_ethtool_rule(priv->fs, eth_rule); return err; } @@ -774,7 +785,7 @@ mlx5e_ethtool_flow_remove(struct mlx5e_priv *priv, int location) goto out; } - del_ethtool_rule(priv, eth_rule); + del_ethtool_rule(priv->fs, eth_rule); out: return err; } @@ -783,12 +794,13 @@ static int mlx5e_ethtool_get_flow(struct mlx5e_priv *priv, struct ethtool_rxnfc *info, int location) { + struct mlx5e_ethtool_steering *ethtool = mlx5e_fs_get_ethtool(priv->fs); struct mlx5e_ethtool_rule *eth_rule; if (location < 0 || location >= MAX_NUM_OF_ETHTOOL_RULES) return -EINVAL; - list_for_each_entry(eth_rule, &priv->fs->ethtool.rules, list) { + list_for_each_entry(eth_rule, ðtool->rules, list) { int index; if (eth_rule->flow_spec.location != location) @@ -826,18 +838,34 @@ mlx5e_ethtool_get_all_flows(struct mlx5e_priv *priv, return err; } -void mlx5e_ethtool_cleanup_steering(struct mlx5e_priv *priv) +int mlx5e_ethtool_alloc(struct mlx5e_ethtool_steering **ethtool) { + *ethtool = kvzalloc(sizeof(**ethtool), GFP_KERNEL); + if (!*ethtool) + return -ENOMEM; + return 0; +} + +void mlx5e_ethtool_free(struct mlx5e_ethtool_steering *ethtool) +{ + kvfree(ethtool); +} + +void mlx5e_ethtool_cleanup_steering(struct mlx5e_flow_steering *fs) +{ + struct mlx5e_ethtool_steering *ethtool = mlx5e_fs_get_ethtool(fs); struct mlx5e_ethtool_rule *iter; struct mlx5e_ethtool_rule *temp; - list_for_each_entry_safe(iter, temp, &priv->fs->ethtool.rules, list) - del_ethtool_rule(priv, iter); + list_for_each_entry_safe(iter, temp, ðtool->rules, list) + del_ethtool_rule(fs, iter); } -void mlx5e_ethtool_init_steering(struct mlx5e_priv *priv) +void mlx5e_ethtool_init_steering(struct mlx5e_flow_steering *fs) { - INIT_LIST_HEAD(&priv->fs->ethtool.rules); + struct mlx5e_ethtool_steering *ethtool = mlx5e_fs_get_ethtool(fs); + + INIT_LIST_HEAD(ðtool->rules); } static int flow_type_to_traffic_type(u32 flow_type) @@ -959,11 +987,12 @@ int mlx5e_ethtool_set_rxnfc(struct mlx5e_priv *priv, struct ethtool_rxnfc *cmd) int mlx5e_ethtool_get_rxnfc(struct mlx5e_priv *priv, struct ethtool_rxnfc *info, u32 *rule_locs) { + struct mlx5e_ethtool_steering *ethtool = mlx5e_fs_get_ethtool(priv->fs); int err = 0; switch (info->cmd) { case ETHTOOL_GRXCLSRLCNT: - info->rule_cnt = priv->fs->ethtool.tot_num_rules; + info->rule_cnt = ethtool->tot_num_rules; break; case ETHTOOL_GRXCLSRULE: err = mlx5e_ethtool_get_flow(priv, info, info->fs.location); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c index 02eb2f0fa2ae..364f04309149 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c @@ -45,6 +45,7 @@ #include "en_tc.h" #include "en_rep.h" #include "en_accel/ipsec.h" +#include "en_accel/macsec.h" #include "en_accel/en_accel.h" #include "en_accel/ktls.h" #include "lib/vxlan.h" @@ -67,22 +68,25 @@ #include "qos.h" #include "en/trap.h" -bool mlx5e_check_fragmented_striding_rq_cap(struct mlx5_core_dev *mdev) +bool mlx5e_check_fragmented_striding_rq_cap(struct mlx5_core_dev *mdev, u8 page_shift, + enum mlx5e_mpwrq_umr_mode umr_mode) { - bool striding_rq_umr, inline_umr; - u16 max_wqe_sz_cap; + u16 umr_wqebbs, max_wqebbs; + bool striding_rq_umr; striding_rq_umr = MLX5_CAP_GEN(mdev, striding_rq) && MLX5_CAP_GEN(mdev, umr_ptr_rlky) && MLX5_CAP_ETH(mdev, reg_umr_sq); - max_wqe_sz_cap = mlx5e_get_max_sq_wqebbs(mdev) * MLX5_SEND_WQE_BB; - inline_umr = max_wqe_sz_cap >= MLX5E_UMR_WQE_INLINE_SZ; if (!striding_rq_umr) return false; - if (!inline_umr) { - mlx5_core_warn(mdev, "Cannot support Striding RQ: UMR WQE size (%d) exceeds maximum supported (%d).\n", - (int)MLX5E_UMR_WQE_INLINE_SZ, max_wqe_sz_cap); + + umr_wqebbs = mlx5e_mpwrq_umr_wqebbs(mdev, page_shift, umr_mode); + max_wqebbs = mlx5e_get_max_sq_aligned_wqebbs(mdev); + /* Sanity check; should never happen, because mlx5e_mpwrq_umr_wqebbs is + * calculated from mlx5e_get_max_sq_aligned_wqebbs. + */ + if (WARN_ON(umr_wqebbs > max_wqebbs)) return false; - } + return true; } @@ -199,21 +203,35 @@ static void mlx5e_disable_blocking_events(struct mlx5e_priv *priv) mlx5_blocking_notifier_unregister(priv->mdev, &priv->blocking_events_nb); } +static u16 mlx5e_mpwrq_umr_octowords(u32 entries, enum mlx5e_mpwrq_umr_mode umr_mode) +{ + u8 umr_entry_size = mlx5e_mpwrq_umr_entry_size(umr_mode); + + WARN_ON_ONCE(entries * umr_entry_size % MLX5_OCTWORD); + + return entries * umr_entry_size / MLX5_OCTWORD; +} + static inline void mlx5e_build_umr_wqe(struct mlx5e_rq *rq, struct mlx5e_icosq *sq, struct mlx5e_umr_wqe *wqe) { struct mlx5_wqe_ctrl_seg *cseg = &wqe->ctrl; struct mlx5_wqe_umr_ctrl_seg *ucseg = &wqe->uctrl; - u8 ds_cnt = DIV_ROUND_UP(MLX5E_UMR_WQE_INLINE_SZ, MLX5_SEND_WQE_DS); + u16 octowords; + u8 ds_cnt; + + ds_cnt = DIV_ROUND_UP(mlx5e_mpwrq_umr_wqe_sz(rq->mdev, rq->mpwqe.page_shift, + rq->mpwqe.umr_mode), + MLX5_SEND_WQE_DS); cseg->qpn_ds = cpu_to_be32((sq->sqn << MLX5_WQE_CTRL_QPN_SHIFT) | ds_cnt); - cseg->umr_mkey = rq->mkey_be; + cseg->umr_mkey = rq->mpwqe.umr_mkey_be; ucseg->flags = MLX5_UMR_TRANSLATION_OFFSET_EN | MLX5_UMR_INLINE; - ucseg->xlt_octowords = - cpu_to_be16(MLX5_MTT_OCTW(MLX5_MPWRQ_PAGES_PER_WQE)); + octowords = mlx5e_mpwrq_umr_octowords(rq->mpwqe.pages_per_wqe, rq->mpwqe.umr_mode); + ucseg->xlt_octowords = cpu_to_be16(octowords); ucseg->mkey_mask = cpu_to_be64(MLX5_MKEY_MASK_FREE); } @@ -259,10 +277,12 @@ static void mlx5e_rq_shampo_hd_info_free(struct mlx5e_rq *rq) static int mlx5e_rq_alloc_mpwqe_info(struct mlx5e_rq *rq, int node) { int wq_sz = mlx5_wq_ll_get_size(&rq->mpwqe.wq); + size_t alloc_size; + + alloc_size = array_size(wq_sz, struct_size(rq->mpwqe.info, alloc_units, + rq->mpwqe.pages_per_wqe)); - rq->mpwqe.info = kvzalloc_node(array_size(wq_sz, - sizeof(*rq->mpwqe.info)), - GFP_KERNEL, node); + rq->mpwqe.info = kvzalloc_node(alloc_size, GFP_KERNEL, node); if (!rq->mpwqe.info) return -ENOMEM; @@ -271,18 +291,52 @@ static int mlx5e_rq_alloc_mpwqe_info(struct mlx5e_rq *rq, int node) return 0; } -static int mlx5e_create_umr_mtt_mkey(struct mlx5_core_dev *mdev, - u64 npages, u8 page_shift, u32 *umr_mkey, - dma_addr_t filler_addr) + +static u8 mlx5e_mpwrq_access_mode(enum mlx5e_mpwrq_umr_mode umr_mode) +{ + switch (umr_mode) { + case MLX5E_MPWRQ_UMR_MODE_ALIGNED: + return MLX5_MKC_ACCESS_MODE_MTT; + case MLX5E_MPWRQ_UMR_MODE_UNALIGNED: + return MLX5_MKC_ACCESS_MODE_KSM; + case MLX5E_MPWRQ_UMR_MODE_OVERSIZED: + return MLX5_MKC_ACCESS_MODE_KLMS; + case MLX5E_MPWRQ_UMR_MODE_TRIPLE: + return MLX5_MKC_ACCESS_MODE_KSM; + } + WARN_ONCE(1, "MPWRQ UMR mode %d is not known\n", umr_mode); + return 0; +} + +static int mlx5e_create_umr_mkey(struct mlx5_core_dev *mdev, + u32 npages, u8 page_shift, u32 *umr_mkey, + dma_addr_t filler_addr, + enum mlx5e_mpwrq_umr_mode umr_mode, + u32 xsk_chunk_size) { struct mlx5_mtt *mtt; + struct mlx5_ksm *ksm; + struct mlx5_klm *klm; + u32 octwords; int inlen; void *mkc; u32 *in; int err; int i; - inlen = MLX5_ST_SZ_BYTES(create_mkey_in) + sizeof(*mtt) * npages; + if ((umr_mode == MLX5E_MPWRQ_UMR_MODE_UNALIGNED || + umr_mode == MLX5E_MPWRQ_UMR_MODE_TRIPLE) && + !MLX5_CAP_GEN(mdev, fixed_buffer_size)) { + mlx5_core_warn(mdev, "Unaligned AF_XDP requires fixed_buffer_size capability\n"); + return -EINVAL; + } + + octwords = mlx5e_mpwrq_umr_octowords(npages, umr_mode); + + inlen = MLX5_FLEXIBLE_INLEN(mdev, MLX5_ST_SZ_BYTES(create_mkey_in), + MLX5_OCTWORD, octwords); + if (inlen < 0) + return inlen; in = kvzalloc(inlen, GFP_KERNEL); if (!in) @@ -294,16 +348,17 @@ static int mlx5e_create_umr_mtt_mkey(struct mlx5_core_dev *mdev, MLX5_SET(mkc, mkc, umr_en, 1); MLX5_SET(mkc, mkc, lw, 1); MLX5_SET(mkc, mkc, lr, 1); - MLX5_SET(mkc, mkc, access_mode_1_0, MLX5_MKC_ACCESS_MODE_MTT); + MLX5_SET(mkc, mkc, access_mode_1_0, mlx5e_mpwrq_access_mode(umr_mode)); mlx5e_mkey_set_relaxed_ordering(mdev, mkc); MLX5_SET(mkc, mkc, qpn, 0xffffff); MLX5_SET(mkc, mkc, pd, mdev->mlx5e_res.hw_objs.pdn); MLX5_SET64(mkc, mkc, len, npages << page_shift); - MLX5_SET(mkc, mkc, translations_octword_size, - MLX5_MTT_OCTW(npages)); - MLX5_SET(mkc, mkc, log_page_size, page_shift); - MLX5_SET(create_mkey_in, in, translations_octword_actual_size, - MLX5_MTT_OCTW(npages)); + MLX5_SET(mkc, mkc, translations_octword_size, octwords); + if (umr_mode == MLX5E_MPWRQ_UMR_MODE_TRIPLE) + MLX5_SET(mkc, mkc, log_page_size, page_shift - 2); + else if (umr_mode != MLX5E_MPWRQ_UMR_MODE_OVERSIZED) + MLX5_SET(mkc, mkc, log_page_size, page_shift); + MLX5_SET(create_mkey_in, in, translations_octword_actual_size, octwords); /* Initialize the mkey with all MTTs pointing to a default * page (filler_addr). When the channels are activated, UMR @@ -311,9 +366,47 @@ static int mlx5e_create_umr_mtt_mkey(struct mlx5_core_dev *mdev, * the RQ's pool, while the gaps (wqe_overflow) remain mapped * to the default page. */ - mtt = MLX5_ADDR_OF(create_mkey_in, in, klm_pas_mtt); - for (i = 0 ; i < npages ; i++) - mtt[i].ptag = cpu_to_be64(filler_addr); + switch (umr_mode) { + case MLX5E_MPWRQ_UMR_MODE_OVERSIZED: + klm = MLX5_ADDR_OF(create_mkey_in, in, klm_pas_mtt); + for (i = 0; i < npages; i++) { + klm[i << 1] = (struct mlx5_klm) { + .va = cpu_to_be64(filler_addr), + .bcount = cpu_to_be32(xsk_chunk_size), + .key = cpu_to_be32(mdev->mlx5e_res.hw_objs.mkey), + }; + klm[(i << 1) + 1] = (struct mlx5_klm) { + .va = cpu_to_be64(filler_addr), + .bcount = cpu_to_be32((1 << page_shift) - xsk_chunk_size), + .key = cpu_to_be32(mdev->mlx5e_res.hw_objs.mkey), + }; + } + break; + case MLX5E_MPWRQ_UMR_MODE_UNALIGNED: + ksm = MLX5_ADDR_OF(create_mkey_in, in, klm_pas_mtt); + for (i = 0; i < npages; i++) + ksm[i] = (struct mlx5_ksm) { + .key = cpu_to_be32(mdev->mlx5e_res.hw_objs.mkey), + .va = cpu_to_be64(filler_addr), + }; + break; + case MLX5E_MPWRQ_UMR_MODE_ALIGNED: + mtt = MLX5_ADDR_OF(create_mkey_in, in, klm_pas_mtt); + for (i = 0; i < npages; i++) + mtt[i] = (struct mlx5_mtt) { + .ptag = cpu_to_be64(filler_addr), + }; + break; + case MLX5E_MPWRQ_UMR_MODE_TRIPLE: + ksm = MLX5_ADDR_OF(create_mkey_in, in, klm_pas_mtt); + for (i = 0; i < npages * 4; i++) { + ksm[i] = (struct mlx5_ksm) { + .key = cpu_to_be32(mdev->mlx5e_res.hw_objs.mkey), + .va = cpu_to_be64(filler_addr), + }; + } + break; + } err = mlx5_core_create_mkey(mdev, umr_mkey, in, inlen); @@ -356,10 +449,27 @@ static int mlx5e_create_umr_klm_mkey(struct mlx5_core_dev *mdev, static int mlx5e_create_rq_umr_mkey(struct mlx5_core_dev *mdev, struct mlx5e_rq *rq) { - u64 num_mtts = MLX5E_REQUIRED_MTTS(mlx5_wq_ll_get_size(&rq->mpwqe.wq)); + u32 xsk_chunk_size = rq->xsk_pool ? rq->xsk_pool->chunk_size : 0; + u32 wq_size = mlx5_wq_ll_get_size(&rq->mpwqe.wq); + u32 num_entries, max_num_entries; + u32 umr_mkey; + int err; + + max_num_entries = mlx5e_mpwrq_max_num_entries(mdev, rq->mpwqe.umr_mode); + + /* Shouldn't overflow, the result is at most MLX5E_MAX_RQ_NUM_MTTS. */ + if (WARN_ON_ONCE(check_mul_overflow(wq_size, (u32)rq->mpwqe.mtts_per_wqe, + &num_entries) || + num_entries > max_num_entries)) + mlx5_core_err(mdev, "%s: multiplication overflow: %u * %u > %u\n", + __func__, wq_size, rq->mpwqe.mtts_per_wqe, + max_num_entries); - return mlx5e_create_umr_mtt_mkey(mdev, num_mtts, PAGE_SHIFT, - &rq->umr_mkey, rq->wqe_overflow.addr); + err = mlx5e_create_umr_mkey(mdev, num_entries, rq->mpwqe.page_shift, + &umr_mkey, rq->wqe_overflow.addr, + rq->mpwqe.umr_mode, xsk_chunk_size); + rq->mpwqe.umr_mkey_be = cpu_to_be32(umr_mkey); + return err; } static int mlx5e_create_rq_hd_umr_mkey(struct mlx5_core_dev *mdev, @@ -376,18 +486,20 @@ static int mlx5e_create_rq_hd_umr_mkey(struct mlx5_core_dev *mdev, &rq->mpwqe.shampo->mkey); } -static u64 mlx5e_get_mpwqe_offset(u16 wqe_ix) -{ - return MLX5E_REQUIRED_MTTS(wqe_ix) << PAGE_SHIFT; -} - static void mlx5e_init_frags_partition(struct mlx5e_rq *rq) { struct mlx5e_wqe_frag_info next_frag = {}; struct mlx5e_wqe_frag_info *prev = NULL; int i; - next_frag.di = &rq->wqe.di[0]; + if (rq->xsk_pool) { + /* Assumptions used by XSK batched allocator. */ + WARN_ON(rq->wqe.info.num_frags != 1); + WARN_ON(rq->wqe.info.log_num_frags != 0); + WARN_ON(rq->wqe.info.arr[0].frag_stride != PAGE_SIZE); + } + + next_frag.au = &rq->wqe.alloc_units[0]; for (i = 0; i < mlx5_wq_cyc_get_size(&rq->wqe.wq); i++) { struct mlx5e_rq_frag_info *frag_info = &rq->wqe.info.arr[0]; @@ -397,7 +509,7 @@ static void mlx5e_init_frags_partition(struct mlx5e_rq *rq) for (f = 0; f < rq->wqe.info.num_frags; f++, frag++) { if (next_frag.offset + frag_info[f].frag_stride > PAGE_SIZE) { - next_frag.di++; + next_frag.au++; next_frag.offset = 0; if (prev) prev->last_in_page = true; @@ -414,12 +526,13 @@ static void mlx5e_init_frags_partition(struct mlx5e_rq *rq) prev->last_in_page = true; } -int mlx5e_init_di_list(struct mlx5e_rq *rq, int wq_sz, int node) +static int mlx5e_init_au_list(struct mlx5e_rq *rq, int wq_sz, int node) { int len = wq_sz << rq->wqe.info.log_num_frags; - rq->wqe.di = kvzalloc_node(array_size(len, sizeof(*rq->wqe.di)), GFP_KERNEL, node); - if (!rq->wqe.di) + rq->wqe.alloc_units = kvzalloc_node(array_size(len, sizeof(*rq->wqe.alloc_units)), + GFP_KERNEL, node); + if (!rq->wqe.alloc_units) return -ENOMEM; mlx5e_init_frags_partition(rq); @@ -427,9 +540,9 @@ int mlx5e_init_di_list(struct mlx5e_rq *rq, int wq_sz, int node) return 0; } -void mlx5e_free_di_list(struct mlx5e_rq *rq) +static void mlx5e_free_au_list(struct mlx5e_rq *rq) { - kvfree(rq->wqe.di); + kvfree(rq->wqe.alloc_units); } static void mlx5e_rq_err_cqe_work(struct work_struct *recover_work) @@ -485,7 +598,7 @@ static int mlx5e_init_rxq_rq(struct mlx5e_channel *c, struct mlx5e_params *param if (err) return err; - return xdp_rxq_info_reg(&rq->xdp_rxq, rq->netdev, rq->ix, 0); + return xdp_rxq_info_reg(&rq->xdp_rxq, rq->netdev, rq->ix, c->napi.napi_id); } static int mlx5_rq_shampo_alloc(struct mlx5_core_dev *mdev, @@ -572,6 +685,8 @@ static int mlx5e_alloc_rq(struct mlx5e_params *params, rq->buff.headroom = mlx5e_get_rq_headroom(mdev, params, xsk); pool_size = 1 << params->log_rq_mtu_frames; + rq->mkey_be = cpu_to_be32(mdev->mlx5e_res.hw_objs.mkey); + switch (rq->wq_type) { case MLX5_WQ_TYPE_LINKED_LIST_STRIDING_RQ: err = mlx5_wq_ll_create(mdev, &rqp->wq, rqc_wq, &rq->mpwqe.wq, @@ -587,8 +702,20 @@ static int mlx5e_alloc_rq(struct mlx5e_params *params, wq_sz = mlx5_wq_ll_get_size(&rq->mpwqe.wq); - pool_size = MLX5_MPWRQ_PAGES_PER_WQE << - mlx5e_mpwqe_get_log_rq_size(params, xsk); + rq->mpwqe.page_shift = mlx5e_mpwrq_page_shift(mdev, xsk); + rq->mpwqe.umr_mode = mlx5e_mpwrq_umr_mode(mdev, xsk); + rq->mpwqe.pages_per_wqe = + mlx5e_mpwrq_pages_per_wqe(mdev, rq->mpwqe.page_shift, + rq->mpwqe.umr_mode); + rq->mpwqe.umr_wqebbs = + mlx5e_mpwrq_umr_wqebbs(mdev, rq->mpwqe.page_shift, + rq->mpwqe.umr_mode); + rq->mpwqe.mtts_per_wqe = + mlx5e_mpwrq_mtts_per_wqe(mdev, rq->mpwqe.page_shift, + rq->mpwqe.umr_mode); + + pool_size = rq->mpwqe.pages_per_wqe << + mlx5e_mpwqe_get_log_rq_size(mdev, params, xsk); rq->mpwqe.log_stride_sz = mlx5e_mpwqe_get_log_stride_size(mdev, params, xsk); rq->mpwqe.num_strides = @@ -600,7 +727,6 @@ static int mlx5e_alloc_rq(struct mlx5e_params *params, err = mlx5e_create_rq_umr_mkey(mdev, rq); if (err) goto err_rq_drop_page; - rq->mkey_be = cpu_to_be32(rq->umr_mkey); err = mlx5e_rq_alloc_mpwqe_info(rq, node); if (err) @@ -608,7 +734,7 @@ static int mlx5e_alloc_rq(struct mlx5e_params *params, err = mlx5_rq_shampo_alloc(mdev, params, rqp, rq, &pool_size, node); if (err) - goto err_free_by_rq_type; + goto err_free_mpwqe_info; break; default: /* MLX5_WQ_TYPE_CYCLIC */ @@ -633,11 +759,9 @@ static int mlx5e_alloc_rq(struct mlx5e_params *params, goto err_rq_wq_destroy; } - err = mlx5e_init_di_list(rq, wq_sz, node); + err = mlx5e_init_au_list(rq, wq_sz, node); if (err) goto err_rq_frags; - - rq->mkey_be = cpu_to_be32(mdev->mlx5e_res.hw_objs.mkey); } if (xsk) { @@ -662,14 +786,14 @@ static int mlx5e_alloc_rq(struct mlx5e_params *params, if (IS_ERR(rq->page_pool)) { err = PTR_ERR(rq->page_pool); rq->page_pool = NULL; - goto err_free_shampo; + goto err_free_by_rq_type; } if (xdp_rxq_info_is_reg(&rq->xdp_rxq)) err = xdp_rxq_info_reg_mem_model(&rq->xdp_rxq, MEM_TYPE_PAGE_POOL, rq->page_pool); } if (err) - goto err_free_shampo; + goto err_destroy_page_pool; for (i = 0; i < wq_sz; i++) { if (rq->wq_type == MLX5_WQ_TYPE_LINKED_LIST_STRIDING_RQ) { @@ -677,13 +801,14 @@ static int mlx5e_alloc_rq(struct mlx5e_params *params, mlx5_wq_ll_get_wqe(&rq->mpwqe.wq, i); u32 byte_count = rq->mpwqe.num_strides << rq->mpwqe.log_stride_sz; - u64 dma_offset = mlx5e_get_mpwqe_offset(i); + u64 dma_offset = mul_u32_u32(i, rq->mpwqe.mtts_per_wqe) << + rq->mpwqe.page_shift; u16 headroom = test_bit(MLX5E_RQ_STATE_SHAMPO, &rq->state) ? 0 : rq->buff.headroom; wqe->data[0].addr = cpu_to_be64(dma_offset + headroom); wqe->data[0].byte_count = cpu_to_be32(byte_count); - wqe->data[0].lkey = rq->mkey_be; + wqe->data[0].lkey = rq->mpwqe.umr_mkey_be; } else { struct mlx5e_rx_wqe_cyc *wqe = mlx5_wq_cyc_get_wqe(&rq->wqe.wq, i); @@ -721,19 +846,21 @@ static int mlx5e_alloc_rq(struct mlx5e_params *params, return 0; -err_free_shampo: - mlx5e_rq_free_shampo(rq); +err_destroy_page_pool: + page_pool_destroy(rq->page_pool); err_free_by_rq_type: switch (rq->wq_type) { case MLX5_WQ_TYPE_LINKED_LIST_STRIDING_RQ: + mlx5e_rq_free_shampo(rq); +err_free_mpwqe_info: kvfree(rq->mpwqe.info); err_rq_mkey: - mlx5_core_destroy_mkey(mdev, rq->umr_mkey); + mlx5_core_destroy_mkey(mdev, be32_to_cpu(rq->mpwqe.umr_mkey_be)); err_rq_drop_page: mlx5e_free_mpwqe_rq_drop_page(rq); break; default: /* MLX5_WQ_TYPE_CYCLIC */ - mlx5e_free_di_list(rq); + mlx5e_free_au_list(rq); err_rq_frags: kvfree(rq->wqe.frags); } @@ -761,24 +888,22 @@ static void mlx5e_free_rq(struct mlx5e_rq *rq) switch (rq->wq_type) { case MLX5_WQ_TYPE_LINKED_LIST_STRIDING_RQ: kvfree(rq->mpwqe.info); - mlx5_core_destroy_mkey(rq->mdev, rq->umr_mkey); + mlx5_core_destroy_mkey(rq->mdev, be32_to_cpu(rq->mpwqe.umr_mkey_be)); mlx5e_free_mpwqe_rq_drop_page(rq); mlx5e_rq_free_shampo(rq); break; default: /* MLX5_WQ_TYPE_CYCLIC */ kvfree(rq->wqe.frags); - mlx5e_free_di_list(rq); + mlx5e_free_au_list(rq); } for (i = rq->page_cache.head; i != rq->page_cache.tail; i = (i + 1) & (MLX5E_CACHE_SIZE - 1)) { - struct mlx5e_dma_info *dma_info = &rq->page_cache.page_cache[i]; - /* With AF_XDP, page_cache is not used, so this loop is not * entered, and it's safe to call mlx5e_page_release_dynamic * directly. */ - mlx5e_page_release_dynamic(rq, dma_info->page, false); + mlx5e_page_release_dynamic(rq, rq->page_cache.page_cache[i], false); } xdp_rxq_info_unreg(&rq->xdp_rxq); @@ -833,7 +958,7 @@ int mlx5e_create_rq(struct mlx5e_rq *rq, struct mlx5e_rq_param *param) return err; } -int mlx5e_modify_rq_state(struct mlx5e_rq *rq, int curr_state, int next_state) +static int mlx5e_modify_rq_state(struct mlx5e_rq *rq, int curr_state, int next_state) { struct mlx5_core_dev *mdev = rq->mdev; @@ -862,6 +987,32 @@ int mlx5e_modify_rq_state(struct mlx5e_rq *rq, int curr_state, int next_state) return err; } +static int mlx5e_rq_to_ready(struct mlx5e_rq *rq, int curr_state) +{ + struct net_device *dev = rq->netdev; + int err; + + err = mlx5e_modify_rq_state(rq, curr_state, MLX5_RQC_STATE_RST); + if (err) { + netdev_err(dev, "Failed to move rq 0x%x to reset\n", rq->rqn); + return err; + } + err = mlx5e_modify_rq_state(rq, MLX5_RQC_STATE_RST, MLX5_RQC_STATE_RDY); + if (err) { + netdev_err(dev, "Failed to move rq 0x%x to ready\n", rq->rqn); + return err; + } + + return 0; +} + +int mlx5e_flush_rq(struct mlx5e_rq *rq, int curr_state) +{ + mlx5e_free_rx_descs(rq); + + return mlx5e_rq_to_ready(rq, curr_state); +} + static int mlx5e_modify_rq_scatter_fcs(struct mlx5e_rq *rq, bool enable) { struct mlx5_core_dev *mdev = rq->mdev; @@ -1154,9 +1305,9 @@ static int mlx5e_alloc_xdpsq(struct mlx5e_channel *c, is_redirect ? &c->priv->channel_stats[c->ix]->xdpsq : &c->priv->channel_stats[c->ix]->rq_xdpsq; - sq->max_sq_wqebbs = mlx5e_get_max_sq_wqebbs(mdev); - sq->stop_room = MLX5E_STOP_ROOM(sq->max_sq_wqebbs); - sq->max_sq_mpw_wqebbs = mlx5e_get_sw_max_sq_mpw_wqebbs(sq->max_sq_wqebbs); + sq->stop_room = param->is_mpw ? mlx5e_stop_room_for_mpwqe(mdev) : + mlx5e_stop_room_for_max_wqe(mdev); + sq->max_sq_mpw_wqebbs = mlx5e_get_max_sq_aligned_wqebbs(mdev); param->wq.db_numa_node = cpu_to_node(c->cpu); err = mlx5_wq_cyc_create(mdev, ¶m->wq, sqc_wq, wq, &sq->wq_ctrl); @@ -1231,7 +1382,6 @@ static int mlx5e_alloc_icosq(struct mlx5e_channel *c, sq->channel = c; sq->uar_map = mdev->mlx5e_res.hw_objs.bfreg.map; sq->reserved_room = param->stop_room; - sq->max_sq_wqebbs = mlx5e_get_max_sq_wqebbs(mdev); param->wq.db_numa_node = cpu_to_node(c->cpu); err = mlx5_wq_cyc_create(mdev, ¶m->wq, sqc_wq, wq, &sq->wq_ctrl); @@ -1317,8 +1467,7 @@ static int mlx5e_alloc_txqsq(struct mlx5e_channel *c, sq->uar_map = mdev->mlx5e_res.hw_objs.bfreg.map; sq->min_inline_mode = params->tx_min_inline_mode; sq->hw_mtu = MLX5E_SW2HW_MTU(params, params->sw_mtu); - sq->max_sq_wqebbs = mlx5e_get_max_sq_wqebbs(mdev); - sq->max_sq_mpw_wqebbs = mlx5e_get_sw_max_sq_mpw_wqebbs(sq->max_sq_wqebbs); + sq->max_sq_mpw_wqebbs = mlx5e_get_max_sq_aligned_wqebbs(mdev); INIT_WORK(&sq->recover_work, mlx5e_tx_err_cqe_work); if (!MLX5_CAP_ETH(mdev, wqe_vlan_insert)) set_bit(MLX5E_SQ_STATE_VLAN_NEED_L2_INLINE, &sq->state); @@ -2280,7 +2429,7 @@ static int mlx5e_open_channel(struct mlx5e_priv *priv, int ix, c->aff_mask = irq_get_effective_affinity_mask(irq); c->lag_port = mlx5e_enumerate_lag_port(priv->mdev, ix); - netif_napi_add(netdev, &c->napi, mlx5e_napi_poll, 64); + netif_napi_add(netdev, &c->napi, mlx5e_napi_poll); err = mlx5e_open_queues(c, params, cparam); if (unlikely(err)) @@ -2318,10 +2467,11 @@ static void mlx5e_activate_channel(struct mlx5e_channel *c) mlx5e_activate_txqsq(&c->sq[tc]); mlx5e_activate_icosq(&c->icosq); mlx5e_activate_icosq(&c->async_icosq); - mlx5e_activate_rq(&c->rq); if (test_bit(MLX5E_CHANNEL_STATE_XSK, c->state)) mlx5e_activate_xsk(c); + else + mlx5e_activate_rq(&c->rq); mlx5e_trigger_napi_icosq(c); } @@ -2332,8 +2482,9 @@ static void mlx5e_deactivate_channel(struct mlx5e_channel *c) if (test_bit(MLX5E_CHANNEL_STATE_XSK, c->state)) mlx5e_deactivate_xsk(c); + else + mlx5e_deactivate_rq(&c->rq); - mlx5e_deactivate_rq(&c->rq); mlx5e_deactivate_icosq(&c->async_icosq); mlx5e_deactivate_icosq(&c->icosq); for (tc = 0; tc < c->num_tc; tc++) @@ -2425,8 +2576,6 @@ static void mlx5e_activate_channels(struct mlx5e_channels *chs) mlx5e_ptp_activate_channel(chs->ptp); } -#define MLX5E_RQ_WQES_TIMEOUT 20000 /* msecs */ - static int mlx5e_wait_channels_min_rx_wqes(struct mlx5e_channels *chs) { int err = 0; @@ -2434,8 +2583,12 @@ static int mlx5e_wait_channels_min_rx_wqes(struct mlx5e_channels *chs) for (i = 0; i < chs->num; i++) { int timeout = err ? 0 : MLX5E_RQ_WQES_TIMEOUT; + struct mlx5e_channel *c = chs->c[i]; - err |= mlx5e_wait_for_min_rx_wqes(&chs->c[i]->rq, timeout); + if (test_bit(MLX5E_CHANNEL_STATE_XSK, c->state)) + continue; + + err |= mlx5e_wait_for_min_rx_wqes(&c->rq, timeout); /* Don't wait on the XSK RQ, because the newer xdpsock sample * doesn't provide any Fill Ring entries at the setup stage. @@ -2600,7 +2753,7 @@ static int mlx5e_update_netdev_queues(struct mlx5e_priv *priv) struct netdev_tc_txq old_tc_to_txq[TC_MAX_QUEUE], *tc_to_txq; struct net_device *netdev = priv->netdev; int old_num_txqs, old_ntc; - int num_rxqs, nch, ntc; + int nch, ntc; int err; int i; @@ -2611,7 +2764,6 @@ static int mlx5e_update_netdev_queues(struct mlx5e_priv *priv) nch = priv->channels.params.num_channels; ntc = priv->channels.params.mqprio.num_tc; - num_rxqs = nch * priv->profile->rq_groups; tc_to_txq = priv->channels.params.mqprio.tc_to_txq; err = mlx5e_netdev_set_tcs(netdev, nch, ntc, tc_to_txq); @@ -2620,7 +2772,7 @@ static int mlx5e_update_netdev_queues(struct mlx5e_priv *priv) err = mlx5e_update_tx_netdev_queues(priv); if (err) goto err_tcs; - err = netif_set_real_num_rx_queues(netdev, num_rxqs); + err = netif_set_real_num_rx_queues(netdev, nch); if (err) { netdev_warn(netdev, "netif_set_real_num_rx_queues failed, %d\n", err); goto err_txqs; @@ -2738,7 +2890,7 @@ void mlx5e_activate_priv_channels(struct mlx5e_priv *priv) netif_tx_start_all_queues(priv->netdev); if (mlx5e_is_vport_rep(priv)) - mlx5e_add_sqs_fwd_rules(priv); + mlx5e_rep_activate_channels(priv); mlx5e_wait_channels_min_rx_wqes(&priv->channels); @@ -2752,7 +2904,7 @@ void mlx5e_deactivate_priv_channels(struct mlx5e_priv *priv) mlx5e_rx_res_channels_deactivate(priv->rx_res); if (mlx5e_is_vport_rep(priv)) - mlx5e_remove_sqs_fwd_rules(priv); + mlx5e_rep_deactivate_channels(priv); /* The results of ndo_select_queue are unreliable, while netdev config * is being changed (real_num_tx_queues, num_tc). Stop all queues to @@ -3547,7 +3699,8 @@ mlx5e_get_stats(struct net_device *dev, struct rtnl_link_stats64 *stats) stats->rx_length_errors = PPORT_802_3_GET(pstats, a_in_range_length_errors) + PPORT_802_3_GET(pstats, a_out_of_range_length_field) + - PPORT_802_3_GET(pstats, a_frame_too_long_errors); + PPORT_802_3_GET(pstats, a_frame_too_long_errors) + + VNIC_ENV_GET(&priv->stats.vnic, eth_wqe_too_small); stats->rx_crc_errors = PPORT_802_3_GET(pstats, a_frame_check_sequence_errors); stats->rx_frame_errors = PPORT_802_3_GET(pstats, a_alignment_errors); @@ -3669,9 +3822,11 @@ static int set_feature_cvlan_filter(struct net_device *netdev, bool enable) struct mlx5e_priv *priv = netdev_priv(netdev); if (enable) - mlx5e_enable_cvlan_filter(priv); + mlx5e_enable_cvlan_filter(priv->fs, + !!(priv->netdev->flags & IFF_PROMISC)); else - mlx5e_disable_cvlan_filter(priv); + mlx5e_disable_cvlan_filter(priv->fs, + !!(priv->netdev->flags & IFF_PROMISC)); return 0; } @@ -3780,7 +3935,7 @@ static int set_feature_rx_vlan(struct net_device *netdev, bool enable) mutex_lock(&priv->state_lock); - priv->fs->vlan_strip_disable = !enable; + mlx5e_fs_set_vlan_strip_disable(priv->fs, !enable); priv->channels.params.vlan_strip_disable = !enable; if (!test_bit(MLX5E_STATE_OPENED, &priv->state)) @@ -3788,7 +3943,7 @@ static int set_feature_rx_vlan(struct net_device *netdev, bool enable) err = mlx5e_modify_channels_vsd(&priv->channels, !enable); if (err) { - priv->fs->vlan_strip_disable = enable; + mlx5e_fs_set_vlan_strip_disable(priv->fs, enable); priv->channels.params.vlan_strip_disable = enable; } unlock: @@ -3826,9 +3981,9 @@ static int set_feature_arfs(struct net_device *netdev, bool enable) int err; if (enable) - err = mlx5e_arfs_enable(priv); + err = mlx5e_arfs_enable(priv->fs); else - err = mlx5e_arfs_disable(priv); + err = mlx5e_arfs_disable(priv->fs); return err; } @@ -3912,12 +4067,14 @@ static netdev_features_t mlx5e_fix_features(struct net_device *netdev, netdev_features_t features) { struct mlx5e_priv *priv = netdev_priv(netdev); + struct mlx5e_vlan_table *vlan; struct mlx5e_params *params; + vlan = mlx5e_fs_get_vlan(priv->fs); mutex_lock(&priv->state_lock); params = &priv->channels.params; - if (!priv->fs->vlan || - !bitmap_empty(mlx5e_vlan_get_active_svlans(priv->fs->vlan), VLAN_N_VID)) { + if (!vlan || + !bitmap_empty(mlx5e_vlan_get_active_svlans(vlan), VLAN_N_VID)) { /* HW strips the outer C-tag header, this is a problem * for S-tag traffic. */ @@ -4006,7 +4163,7 @@ static bool mlx5e_xsk_validate_mtu(struct net_device *netdev, * 2. Size of SKBs allocated on XDP_PASS <= PAGE_SIZE. */ max_mtu_frame = MLX5E_HW2SW_MTU(new_params, xsk.chunk_size - hr); - max_mtu_page = mlx5e_xdp_max_mtu(new_params, &xsk); + max_mtu_page = MLX5E_HW2SW_MTU(new_params, SKB_MAX_HEAD(0)); max_mtu = min(max_mtu_frame, max_mtu_page); netdev_err(netdev, "MTU %d is too big for an XSK running on channel %u. Try MTU <= %d\n", @@ -4018,14 +4175,16 @@ static bool mlx5e_xsk_validate_mtu(struct net_device *netdev, return true; } -static bool mlx5e_params_validate_xdp(struct net_device *netdev, struct mlx5e_params *params) +static bool mlx5e_params_validate_xdp(struct net_device *netdev, + struct mlx5_core_dev *mdev, + struct mlx5e_params *params) { bool is_linear; /* No XSK params: AF_XDP can't be enabled yet at the point of setting * the XDP program. */ - is_linear = mlx5e_rx_is_linear_skb(params, NULL); + is_linear = mlx5e_rx_is_linear_skb(mdev, params, NULL); if (!is_linear && params->rq_wq_type != MLX5_WQ_TYPE_CYCLIC) { netdev_warn(netdev, "XDP is not allowed with striding RQ and MTU(%d) > %d\n", @@ -4062,7 +4221,8 @@ int mlx5e_change_mtu(struct net_device *netdev, int new_mtu, if (err) goto out; - if (new_params.xdp_prog && !mlx5e_params_validate_xdp(netdev, &new_params)) { + if (new_params.xdp_prog && !mlx5e_params_validate_xdp(netdev, priv->mdev, + &new_params)) { err = -EINVAL; goto out; } @@ -4077,19 +4237,21 @@ int mlx5e_change_mtu(struct net_device *netdev, int new_mtu, if (params->packet_merge.type == MLX5E_PACKET_MERGE_LRO) reset = false; - if (params->rq_wq_type == MLX5_WQ_TYPE_LINKED_LIST_STRIDING_RQ) { + if (params->rq_wq_type == MLX5_WQ_TYPE_LINKED_LIST_STRIDING_RQ && + params->packet_merge.type != MLX5E_PACKET_MERGE_SHAMPO) { bool is_linear_old = mlx5e_rx_mpwqe_is_linear_skb(priv->mdev, params, NULL); bool is_linear_new = mlx5e_rx_mpwqe_is_linear_skb(priv->mdev, &new_params, NULL); - u8 ppw_old = mlx5e_mpwqe_log_pkts_per_wqe(params, NULL); - u8 ppw_new = mlx5e_mpwqe_log_pkts_per_wqe(&new_params, NULL); + u8 sz_old = mlx5e_mpwqe_get_log_rq_size(priv->mdev, params, NULL); + u8 sz_new = mlx5e_mpwqe_get_log_rq_size(priv->mdev, &new_params, NULL); /* Always reset in linear mode - hw_mtu is used in data path. * Check that the mode was non-linear and didn't change. * If XSK is active, XSK RQs are linear. + * Reset if the RQ size changed, even if it's non-linear. */ if (!is_linear_old && !is_linear_new && !priv->xsk.refcnt && - ppw_old == ppw_new) + sz_old == sz_new) reset = false; } @@ -4539,7 +4701,7 @@ static int mlx5e_xdp_allowed(struct mlx5e_priv *priv, struct bpf_prog *prog) new_params = priv->channels.params; new_params.xdp_prog = prog; - if (!mlx5e_params_validate_xdp(netdev, &new_params)) + if (!mlx5e_params_validate_xdp(netdev, priv->mdev, &new_params)) return -EINVAL; return 0; @@ -4577,8 +4739,20 @@ static int mlx5e_xdp_set(struct net_device *netdev, struct bpf_prog *prog) new_params = priv->channels.params; new_params.xdp_prog = prog; - if (reset) - mlx5e_set_rq_type(priv->mdev, &new_params); + + /* XDP affects striding RQ parameters. Block XDP if striding RQ won't be + * supported with the new parameters: if PAGE_SIZE is bigger than + * MLX5_MPWQE_LOG_STRIDE_SZ_MAX, striding RQ can't be used, even though + * the MTU is small enough for the linear mode, because XDP uses strides + * of PAGE_SIZE on regular RQs. + */ + if (reset && MLX5E_GET_PFLAG(&new_params, MLX5E_PFLAG_RX_STRIDING_RQ)) { + /* Checking for regular RQs here; XSK RQs were checked on XSK bind. */ + err = mlx5e_mpwrq_validate_regular(priv->mdev, &new_params); + if (err) + goto unlock; + } + old_prog = priv->channels.params.xdp_prog; err = mlx5e_safe_switch_params(priv, &new_params, NULL, NULL, reset); @@ -4898,7 +5072,8 @@ static void mlx5e_build_nic_netdev(struct net_device *netdev) if (!!MLX5_CAP_ETH(mdev, lro_cap) && !MLX5_CAP_ETH(mdev, tunnel_lro_vxlan) && !MLX5_CAP_ETH(mdev, tunnel_lro_gre) && - mlx5e_check_fragmented_striding_rq_cap(mdev)) + mlx5e_check_fragmented_striding_rq_cap(mdev, PAGE_SHIFT, + MLX5E_MPWRQ_UMR_MODE_ALIGNED)) netdev->vlan_features |= NETIF_F_LRO; netdev->hw_features = netdev->vlan_features; @@ -4986,6 +5161,7 @@ static void mlx5e_build_nic_netdev(struct net_device *netdev) netif_set_tso_max_size(netdev, GSO_MAX_SIZE); mlx5e_set_netdev_dev_addr(netdev); + mlx5e_macsec_build_netdev(priv); mlx5e_ipsec_build_netdev(priv); mlx5e_ktls_build_netdev(priv); } @@ -5087,7 +5263,7 @@ static int mlx5e_init_nic_rx(struct mlx5e_priv *priv) goto err_destroy_q_counters; } - features = MLX5E_RX_RES_FEATURE_XSK | MLX5E_RX_RES_FEATURE_PTP; + features = MLX5E_RX_RES_FEATURE_PTP; if (priv->channels.params.tunneled_offload_en) features |= MLX5E_RX_RES_FEATURE_INNER_FT; err = mlx5e_rx_res_init(priv->rx_res, priv->mdev, features, @@ -5097,7 +5273,8 @@ static int mlx5e_init_nic_rx(struct mlx5e_priv *priv) if (err) goto err_close_drop_rq; - err = mlx5e_create_flow_steering(priv); + err = mlx5e_create_flow_steering(priv->fs, priv->rx_res, priv->profile, + priv->netdev); if (err) { mlx5_core_warn(mdev, "create flow steering failed, %d\n", err); goto err_destroy_rx_res; @@ -5120,7 +5297,8 @@ static int mlx5e_init_nic_rx(struct mlx5e_priv *priv) err_tc_nic_cleanup: mlx5e_tc_nic_cleanup(priv); err_destroy_flow_steering: - mlx5e_destroy_flow_steering(priv); + mlx5e_destroy_flow_steering(priv->fs, !!(priv->netdev->hw_features & NETIF_F_NTUPLE), + priv->profile); err_destroy_rx_res: mlx5e_rx_res_destroy(priv->rx_res); err_close_drop_rq: @@ -5136,7 +5314,8 @@ static void mlx5e_cleanup_nic_rx(struct mlx5e_priv *priv) { mlx5e_accel_cleanup_rx(priv); mlx5e_tc_nic_cleanup(priv); - mlx5e_destroy_flow_steering(priv); + mlx5e_destroy_flow_steering(priv->fs, !!(priv->netdev->hw_features & NETIF_F_NTUPLE), + priv->profile); mlx5e_rx_res_destroy(priv->rx_res); mlx5e_close_drop_rq(&priv->drop_rq); mlx5e_destroy_q_counters(priv); @@ -5188,9 +5367,14 @@ static void mlx5e_nic_enable(struct mlx5e_priv *priv) { struct net_device *netdev = priv->netdev; struct mlx5_core_dev *mdev = priv->mdev; + int err; mlx5e_fs_init_l2_addr(priv->fs, netdev); + err = mlx5e_macsec_init(priv); + if (err) + mlx5_core_err(mdev, "MACsec initialization failed, %d\n", err); + /* Marking the link as currently not needed by the Driver */ if (!netif_running(netdev)) mlx5e_modify_admin_state(mdev, MLX5_PORT_DOWN); @@ -5248,6 +5432,7 @@ static void mlx5e_nic_disable(struct mlx5e_priv *priv) mlx5e_disable_async_events(priv); mlx5_lag_remove_netdev(mdev, priv->netdev); mlx5_vxlan_reset_to_default(mdev->vxlan); + mlx5e_macsec_cleanup(priv); } int mlx5e_update_nic_rx(struct mlx5e_priv *priv) @@ -5269,7 +5454,6 @@ static const struct mlx5e_profile mlx5e_nic_profile = { .update_carrier = mlx5e_update_carrier, .rx_handlers = &mlx5e_rx_handlers_nic, .max_tc = MLX5E_MAX_NUM_TC, - .rq_groups = MLX5E_NUM_RQ_GROUPS(XSK), .stats_grps = mlx5e_nic_stats_grps, .stats_grps_num = mlx5e_nic_stats_grps_num, .features = BIT(MLX5E_PROFILE_FEATURE_PTP_RX) | @@ -5302,8 +5486,7 @@ mlx5e_calc_max_nch(struct mlx5_core_dev *mdev, struct net_device *netdev, max_nch = mlx5e_profile_max_num_channels(mdev, profile); /* netdev rx queues */ - tmp = netdev->num_rx_queues / max_t(u8, profile->rq_groups, 1); - max_nch = min_t(unsigned int, max_nch, tmp); + max_nch = min_t(unsigned int, max_nch, netdev->num_rx_queues); /* netdev tx queues */ tmp = netdev->num_tx_queues; @@ -5447,11 +5630,7 @@ static unsigned int mlx5e_get_max_num_txqs(struct mlx5_core_dev *mdev, static unsigned int mlx5e_get_max_num_rxqs(struct mlx5_core_dev *mdev, const struct mlx5e_profile *profile) { - unsigned int nch; - - nch = mlx5e_profile_max_num_channels(mdev, profile); - - return nch * profile->rq_groups; + return mlx5e_profile_max_num_channels(mdev, profile); } struct net_device * @@ -5512,7 +5691,8 @@ int mlx5e_attach_netdev(struct mlx5e_priv *priv) clear_bit(MLX5E_STATE_DESTROYING, &priv->state); if (priv->fs) - priv->fs->state_destroy = !test_bit(MLX5E_STATE_DESTROYING, &priv->state); + mlx5e_fs_set_state_destroy(priv->fs, + !test_bit(MLX5E_STATE_DESTROYING, &priv->state)); /* max number of channels may have changed */ max_nch = mlx5e_calc_max_nch(priv->mdev, priv->netdev, profile); @@ -5573,7 +5753,8 @@ out: mlx5e_reset_channels(priv->netdev); set_bit(MLX5E_STATE_DESTROYING, &priv->state); if (priv->fs) - priv->fs->state_destroy = !test_bit(MLX5E_STATE_DESTROYING, &priv->state); + mlx5e_fs_set_state_destroy(priv->fs, + !test_bit(MLX5E_STATE_DESTROYING, &priv->state)); cancel_work_sync(&priv->update_stats_work); return err; } @@ -5584,7 +5765,8 @@ void mlx5e_detach_netdev(struct mlx5e_priv *priv) set_bit(MLX5E_STATE_DESTROYING, &priv->state); if (priv->fs) - priv->fs->state_destroy = !test_bit(MLX5E_STATE_DESTROYING, &priv->state); + mlx5e_fs_set_state_destroy(priv->fs, + !test_bit(MLX5E_STATE_DESTROYING, &priv->state)); if (profile->disable) profile->disable(priv); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_rep.c b/drivers/net/ethernet/mellanox/mlx5/core/en_rep.c index 759f7d3c2cfd..794cd8dfe9c9 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_rep.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_rep.c @@ -56,6 +56,7 @@ #include "en_accel/ipsec.h" #include "en/tc/int_port.h" #include "en/ptp.h" +#include "en/fs_ethtool.h" #define MLX5E_REP_PARAMS_DEF_LOG_SQ_SIZE \ max(0x7, MLX5E_PARAMS_MINIMUM_LOG_SQ_SIZE) @@ -69,7 +70,7 @@ static void mlx5e_rep_get_drvinfo(struct net_device *dev, struct mlx5e_priv *priv = netdev_priv(dev); struct mlx5_core_dev *mdev = priv->mdev; - strlcpy(drvinfo->driver, mlx5e_rep_driver_name, + strscpy(drvinfo->driver, mlx5e_rep_driver_name, sizeof(drvinfo->driver)); snprintf(drvinfo->fw_version, sizeof(drvinfo->fw_version), "%d.%d.%04d (%.16s)", @@ -397,7 +398,8 @@ out_err: return err; } -int mlx5e_add_sqs_fwd_rules(struct mlx5e_priv *priv) +static int +mlx5e_add_sqs_fwd_rules(struct mlx5e_priv *priv) { int sqs_per_channel = mlx5e_get_dcb_num_tc(&priv->channels.params); struct mlx5_eswitch *esw = priv->mdev->priv.eswitch; @@ -451,7 +453,8 @@ out: return err; } -void mlx5e_remove_sqs_fwd_rules(struct mlx5e_priv *priv) +static void +mlx5e_remove_sqs_fwd_rules(struct mlx5e_priv *priv) { struct mlx5_eswitch *esw = priv->mdev->priv.eswitch; struct mlx5e_rep_priv *rpriv = priv->ppriv; @@ -460,6 +463,49 @@ void mlx5e_remove_sqs_fwd_rules(struct mlx5e_priv *priv) mlx5e_sqs2vport_stop(esw, rep); } +static int +mlx5e_rep_add_meta_tunnel_rule(struct mlx5e_priv *priv) +{ + struct mlx5_eswitch *esw = priv->mdev->priv.eswitch; + struct mlx5e_rep_priv *rpriv = priv->ppriv; + struct mlx5_eswitch_rep *rep = rpriv->rep; + struct mlx5_flow_handle *flow_rule; + struct mlx5_flow_group *g; + + g = esw->fdb_table.offloads.send_to_vport_meta_grp; + if (!g) + return 0; + + flow_rule = mlx5_eswitch_add_send_to_vport_meta_rule(esw, rep->vport); + if (IS_ERR(flow_rule)) + return PTR_ERR(flow_rule); + + rpriv->send_to_vport_meta_rule = flow_rule; + + return 0; +} + +static void +mlx5e_rep_del_meta_tunnel_rule(struct mlx5e_priv *priv) +{ + struct mlx5e_rep_priv *rpriv = priv->ppriv; + + if (rpriv->send_to_vport_meta_rule) + mlx5_eswitch_del_send_to_vport_meta_rule(rpriv->send_to_vport_meta_rule); +} + +void mlx5e_rep_activate_channels(struct mlx5e_priv *priv) +{ + mlx5e_add_sqs_fwd_rules(priv); + mlx5e_rep_add_meta_tunnel_rule(priv); +} + +void mlx5e_rep_deactivate_channels(struct mlx5e_priv *priv) +{ + mlx5e_rep_del_meta_tunnel_rule(priv); + mlx5e_remove_sqs_fwd_rules(priv); +} + static int mlx5e_rep_open(struct net_device *dev) { struct mlx5e_priv *priv = netdev_priv(dev); @@ -747,19 +793,20 @@ static int mlx5e_create_rep_ttc_table(struct mlx5e_priv *priv) struct ttc_params ttc_params = {}; int err; - priv->fs->ns = mlx5_get_flow_namespace(priv->mdev, - MLX5_FLOW_NAMESPACE_KERNEL); + mlx5e_fs_set_ns(priv->fs, + mlx5_get_flow_namespace(priv->mdev, + MLX5_FLOW_NAMESPACE_KERNEL), false); /* The inner_ttc in the ttc params is intentionally not set */ - mlx5e_set_ttc_params(priv, &ttc_params, false); + mlx5e_set_ttc_params(priv->fs, priv->rx_res, &ttc_params, false); if (rep->vport != MLX5_VPORT_UPLINK) /* To give uplik rep TTC a lower level for chaining from root ft */ ttc_params.ft_attr.level = MLX5E_TTC_FT_LEVEL + 1; - priv->fs->ttc = mlx5_create_ttc_table(priv->mdev, &ttc_params); - if (IS_ERR(priv->fs->ttc)) { - err = PTR_ERR(priv->fs->ttc); + mlx5e_fs_set_ttc(priv->fs, mlx5_create_ttc_table(priv->mdev, &ttc_params), false); + if (IS_ERR(mlx5e_fs_get_ttc(priv->fs, false))) { + err = PTR_ERR(mlx5e_fs_get_ttc(priv->fs, false)); netdev_err(priv->netdev, "Failed to create rep ttc table, err=%d\n", err); return err; @@ -779,7 +826,7 @@ static int mlx5e_create_rep_root_ft(struct mlx5e_priv *priv) /* non uplik reps will skip any bypass tables and go directly to * their own ttc */ - rpriv->root_ft = mlx5_get_ttc_flow_table(priv->fs->ttc); + rpriv->root_ft = mlx5_get_ttc_flow_table(mlx5e_fs_get_ttc(priv->fs, false)); return 0; } @@ -887,14 +934,14 @@ static int mlx5e_init_rep_rx(struct mlx5e_priv *priv) if (err) goto err_destroy_root_ft; - mlx5e_ethtool_init_steering(priv); + mlx5e_ethtool_init_steering(priv->fs); return 0; err_destroy_root_ft: mlx5e_destroy_rep_root_ft(priv); err_destroy_ttc_table: - mlx5_destroy_ttc_table(priv->fs->ttc); + mlx5_destroy_ttc_table(mlx5e_fs_get_ttc(priv->fs, false)); err_destroy_rx_res: mlx5e_rx_res_destroy(priv->rx_res); err_close_drop_rq: @@ -908,10 +955,10 @@ err_free_fs: static void mlx5e_cleanup_rep_rx(struct mlx5e_priv *priv) { - mlx5e_ethtool_cleanup_steering(priv); + mlx5e_ethtool_cleanup_steering(priv->fs); rep_vport_rx_rule_destroy(priv); mlx5e_destroy_rep_root_ft(priv); - mlx5_destroy_ttc_table(priv->fs->ttc); + mlx5_destroy_ttc_table(mlx5e_fs_get_ttc(priv->fs, false)); mlx5e_rx_res_destroy(priv->rx_res); mlx5e_close_drop_rq(&priv->drop_rq); mlx5e_rx_res_free(priv->rx_res); @@ -1177,7 +1224,6 @@ static const struct mlx5e_profile mlx5e_rep_profile = { .update_stats = mlx5e_stats_update_ndo_stats, .rx_handlers = &mlx5e_rx_handlers_rep, .max_tc = 1, - .rq_groups = MLX5E_NUM_RQ_GROUPS(REGULAR), .stats_grps = mlx5e_rep_stats_grps, .stats_grps_num = mlx5e_rep_stats_grps_num, .max_nch_limit = mlx5e_rep_max_nch_limit, @@ -1197,8 +1243,6 @@ static const struct mlx5e_profile mlx5e_uplink_rep_profile = { .update_carrier = mlx5e_update_carrier, .rx_handlers = &mlx5e_rx_handlers_rep, .max_tc = MLX5E_MAX_NUM_TC, - /* XSK is needed so we can replace profile with NIC netdev */ - .rq_groups = MLX5E_NUM_RQ_GROUPS(XSK), .stats_grps = mlx5e_ul_rep_stats_grps, .stats_grps_num = mlx5e_ul_rep_stats_grps_num, }; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_rep.h b/drivers/net/ethernet/mellanox/mlx5/core/en_rep.h index dec183ccd4ac..b4e691760da9 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_rep.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_rep.h @@ -111,6 +111,7 @@ struct mlx5e_rep_priv { struct list_head vport_sqs_list; struct mlx5_rep_uplink_priv uplink_priv; /* valid for uplink rep */ struct rtnl_link_stats64 prev_vf_vport_stats; + struct mlx5_flow_handle *send_to_vport_meta_rule; struct rhashtable tc_ht; }; @@ -241,8 +242,8 @@ int mlx5e_rep_get_offload_stats(int attr_id, const struct net_device *dev, void *sp); bool mlx5e_is_uplink_rep(struct mlx5e_priv *priv); -int mlx5e_add_sqs_fwd_rules(struct mlx5e_priv *priv); -void mlx5e_remove_sqs_fwd_rules(struct mlx5e_priv *priv); +void mlx5e_rep_activate_channels(struct mlx5e_priv *priv); +void mlx5e_rep_deactivate_channels(struct mlx5e_priv *priv); void mlx5e_rep_queue_neigh_stats_work(struct mlx5e_priv *priv); @@ -256,8 +257,8 @@ static inline bool mlx5e_eswitch_rep(const struct net_device *netdev) #else /* CONFIG_MLX5_ESWITCH */ static inline bool mlx5e_is_uplink_rep(struct mlx5e_priv *priv) { return false; } -static inline int mlx5e_add_sqs_fwd_rules(struct mlx5e_priv *priv) { return 0; } -static inline void mlx5e_remove_sqs_fwd_rules(struct mlx5e_priv *priv) {} +static inline void mlx5e_rep_activate_channels(struct mlx5e_priv *priv) {} +static inline void mlx5e_rep_deactivate_channels(struct mlx5e_priv *priv) {} static inline int mlx5e_rep_init(void) { return 0; }; static inline void mlx5e_rep_cleanup(void) {}; static inline bool mlx5e_rep_has_offload_stats(const struct net_device *dev, diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c index 24de37b79f5a..58084650151f 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c @@ -41,6 +41,7 @@ #include <net/gro.h> #include <net/udp.h> #include <net/tcp.h> +#include <net/xdp_sock_drv.h> #include "en.h" #include "en/txrx.h" #include "en_tc.h" @@ -49,6 +50,7 @@ #include "en/rep/tc.h" #include "ipoib/ipoib.h" #include "en_accel/ipsec.h" +#include "en_accel/macsec.h" #include "en_accel/ipsec_rxtx.h" #include "en_accel/ktls_txrx.h" #include "en/xdp.h" @@ -237,69 +239,61 @@ static inline bool mlx5e_rx_cache_put(struct mlx5e_rq *rq, struct page *page) return false; } - cache->page_cache[cache->tail].page = page; - cache->page_cache[cache->tail].addr = page_pool_get_dma_addr(page); + cache->page_cache[cache->tail] = page; cache->tail = tail_next; return true; } -static inline bool mlx5e_rx_cache_get(struct mlx5e_rq *rq, - struct mlx5e_dma_info *dma_info) +static inline bool mlx5e_rx_cache_get(struct mlx5e_rq *rq, union mlx5e_alloc_unit *au) { struct mlx5e_page_cache *cache = &rq->page_cache; struct mlx5e_rq_stats *stats = rq->stats; + dma_addr_t addr; if (unlikely(cache->head == cache->tail)) { stats->cache_empty++; return false; } - if (page_ref_count(cache->page_cache[cache->head].page) != 1) { + if (page_ref_count(cache->page_cache[cache->head]) != 1) { stats->cache_busy++; return false; } - *dma_info = cache->page_cache[cache->head]; + au->page = cache->page_cache[cache->head]; cache->head = (cache->head + 1) & (MLX5E_CACHE_SIZE - 1); stats->cache_reuse++; - dma_sync_single_for_device(rq->pdev, dma_info->addr, - PAGE_SIZE, - DMA_FROM_DEVICE); + addr = page_pool_get_dma_addr(au->page); + /* Non-XSK always uses PAGE_SIZE. */ + dma_sync_single_for_device(rq->pdev, addr, PAGE_SIZE, DMA_FROM_DEVICE); return true; } -static inline int mlx5e_page_alloc_pool(struct mlx5e_rq *rq, - struct mlx5e_dma_info *dma_info) +static inline int mlx5e_page_alloc_pool(struct mlx5e_rq *rq, union mlx5e_alloc_unit *au) { - if (mlx5e_rx_cache_get(rq, dma_info)) + dma_addr_t addr; + + if (mlx5e_rx_cache_get(rq, au)) return 0; - dma_info->page = page_pool_dev_alloc_pages(rq->page_pool); - if (unlikely(!dma_info->page)) + au->page = page_pool_dev_alloc_pages(rq->page_pool); + if (unlikely(!au->page)) return -ENOMEM; - dma_info->addr = dma_map_page_attrs(rq->pdev, dma_info->page, 0, PAGE_SIZE, - rq->buff.map_dir, DMA_ATTR_SKIP_CPU_SYNC); - if (unlikely(dma_mapping_error(rq->pdev, dma_info->addr))) { - page_pool_recycle_direct(rq->page_pool, dma_info->page); - dma_info->page = NULL; + /* Non-XSK always uses PAGE_SIZE. */ + addr = dma_map_page_attrs(rq->pdev, au->page, 0, PAGE_SIZE, + rq->buff.map_dir, DMA_ATTR_SKIP_CPU_SYNC); + if (unlikely(dma_mapping_error(rq->pdev, addr))) { + page_pool_recycle_direct(rq->page_pool, au->page); + au->page = NULL; return -ENOMEM; } - page_pool_set_dma_addr(dma_info->page, dma_info->addr); + page_pool_set_dma_addr(au->page, addr); return 0; } -static inline int mlx5e_page_alloc(struct mlx5e_rq *rq, - struct mlx5e_dma_info *dma_info) -{ - if (rq->xsk_pool) - return mlx5e_xsk_page_alloc_pool(rq, dma_info); - else - return mlx5e_page_alloc_pool(rq, dma_info); -} - void mlx5e_page_dma_unmap(struct mlx5e_rq *rq, struct page *page) { dma_addr_t dma_addr = page_pool_get_dma_addr(page); @@ -324,32 +318,18 @@ void mlx5e_page_release_dynamic(struct mlx5e_rq *rq, struct page *page, bool rec } } -static inline void mlx5e_page_release(struct mlx5e_rq *rq, - struct mlx5e_dma_info *dma_info, - bool recycle) -{ - if (rq->xsk_pool) - /* The `recycle` parameter is ignored, and the page is always - * put into the Reuse Ring, because there is no way to return - * the page to the userspace when the interface goes down. - */ - xsk_buff_free(dma_info->xsk); - else - mlx5e_page_release_dynamic(rq, dma_info->page, recycle); -} - static inline int mlx5e_get_rx_frag(struct mlx5e_rq *rq, struct mlx5e_wqe_frag_info *frag) { int err = 0; if (!frag->offset) - /* On first frag (offset == 0), replenish page (dma_info actually). - * Other frags that point to the same dma_info (with a different + /* On first frag (offset == 0), replenish page (alloc_unit actually). + * Other frags that point to the same alloc_unit (with a different * offset) should just use the new one without replenishing again * by themselves. */ - err = mlx5e_page_alloc(rq, frag->di); + err = mlx5e_page_alloc_pool(rq, frag->au); return err; } @@ -359,7 +339,7 @@ static inline void mlx5e_put_rx_frag(struct mlx5e_rq *rq, bool recycle) { if (frag->last_in_page) - mlx5e_page_release(rq, frag->di, recycle); + mlx5e_page_release_dynamic(rq, frag->au->page, recycle); } static inline struct mlx5e_wqe_frag_info *get_frag(struct mlx5e_rq *rq, u16 ix) @@ -375,6 +355,7 @@ static int mlx5e_alloc_rx_wqe(struct mlx5e_rq *rq, struct mlx5e_rx_wqe_cyc *wqe, int i; for (i = 0; i < rq->wqe.info.num_frags; i++, frag++) { + dma_addr_t addr; u16 headroom; err = mlx5e_get_rx_frag(rq, frag); @@ -382,8 +363,8 @@ static int mlx5e_alloc_rx_wqe(struct mlx5e_rq *rq, struct mlx5e_rx_wqe_cyc *wqe, goto free_frags; headroom = i == 0 ? rq->buff.headroom : 0; - wqe->data[i].addr = cpu_to_be64(frag->di->addr + - frag->offset + headroom); + addr = page_pool_get_dma_addr(frag->au->page); + wqe->data[i].addr = cpu_to_be64(addr + frag->offset + headroom); } return 0; @@ -401,6 +382,15 @@ static inline void mlx5e_free_rx_wqe(struct mlx5e_rq *rq, { int i; + if (rq->xsk_pool) { + /* The `recycle` parameter is ignored, and the page is always + * put into the Reuse Ring, because there is no way to return + * the page to the userspace when the interface goes down. + */ + xsk_buff_free(wi->au->xsk); + return; + } + for (i = 0; i < rq->wqe.info.num_frags; i++, wi++) mlx5e_put_rx_frag(rq, wi, recycle); } @@ -412,84 +402,76 @@ static void mlx5e_dealloc_rx_wqe(struct mlx5e_rq *rq, u16 ix) mlx5e_free_rx_wqe(rq, wi, false); } -static int mlx5e_alloc_rx_wqes(struct mlx5e_rq *rq, u16 ix, u8 wqe_bulk) +static int mlx5e_alloc_rx_wqes(struct mlx5e_rq *rq, u16 ix, int wqe_bulk) { struct mlx5_wq_cyc *wq = &rq->wqe.wq; - int err; int i; - if (rq->xsk_pool) { - int pages_desired = wqe_bulk << rq->wqe.info.log_num_frags; - - /* Check in advance that we have enough frames, instead of - * allocating one-by-one, failing and moving frames to the - * Reuse Ring. - */ - if (unlikely(!xsk_buff_can_alloc(rq->xsk_pool, pages_desired))) - return -ENOMEM; - } - for (i = 0; i < wqe_bulk; i++) { - struct mlx5e_rx_wqe_cyc *wqe = mlx5_wq_cyc_get_wqe(wq, ix + i); - - err = mlx5e_alloc_rx_wqe(rq, wqe, ix + i); - if (unlikely(err)) - goto free_wqes; - } + int j = mlx5_wq_cyc_ctr2ix(wq, ix + i); + struct mlx5e_rx_wqe_cyc *wqe; - return 0; + wqe = mlx5_wq_cyc_get_wqe(wq, j); -free_wqes: - while (--i >= 0) - mlx5e_dealloc_rx_wqe(rq, ix + i); + if (unlikely(mlx5e_alloc_rx_wqe(rq, wqe, j))) + break; + } - return err; + return i; } static inline void mlx5e_add_skb_frag(struct mlx5e_rq *rq, struct sk_buff *skb, - struct mlx5e_dma_info *di, u32 frag_offset, u32 len, + union mlx5e_alloc_unit *au, u32 frag_offset, u32 len, unsigned int truesize) { - dma_sync_single_for_cpu(rq->pdev, - di->addr + frag_offset, - len, DMA_FROM_DEVICE); - page_ref_inc(di->page); + dma_addr_t addr = page_pool_get_dma_addr(au->page); + + dma_sync_single_for_cpu(rq->pdev, addr + frag_offset, len, DMA_FROM_DEVICE); + page_ref_inc(au->page); skb_add_rx_frag(skb, skb_shinfo(skb)->nr_frags, - di->page, frag_offset, len, truesize); + au->page, frag_offset, len, truesize); } static inline void mlx5e_copy_skb_header(struct device *pdev, struct sk_buff *skb, - struct mlx5e_dma_info *dma_info, + struct page *page, dma_addr_t addr, int offset_from, int dma_offset, u32 headlen) { - const void *from = page_address(dma_info->page) + offset_from; + const void *from = page_address(page) + offset_from; /* Aligning len to sizeof(long) optimizes memcpy performance */ unsigned int len = ALIGN(headlen, sizeof(long)); - dma_sync_single_for_cpu(pdev, dma_info->addr + dma_offset, len, - DMA_FROM_DEVICE); + dma_sync_single_for_cpu(pdev, addr + dma_offset, len, DMA_FROM_DEVICE); skb_copy_to_linear_data(skb, from, len); } static void mlx5e_free_rx_mpwqe(struct mlx5e_rq *rq, struct mlx5e_mpw_info *wi, bool recycle) { + union mlx5e_alloc_unit *alloc_units = wi->alloc_units; bool no_xdp_xmit; - struct mlx5e_dma_info *dma_info = wi->umr.dma_info; int i; /* A common case for AF_XDP. */ - if (bitmap_full(wi->xdp_xmit_bitmap, MLX5_MPWRQ_PAGES_PER_WQE)) + if (bitmap_full(wi->xdp_xmit_bitmap, rq->mpwqe.pages_per_wqe)) return; - no_xdp_xmit = bitmap_empty(wi->xdp_xmit_bitmap, - MLX5_MPWRQ_PAGES_PER_WQE); + no_xdp_xmit = bitmap_empty(wi->xdp_xmit_bitmap, rq->mpwqe.pages_per_wqe); - for (i = 0; i < MLX5_MPWRQ_PAGES_PER_WQE; i++) - if (no_xdp_xmit || !test_bit(i, wi->xdp_xmit_bitmap)) - mlx5e_page_release(rq, &dma_info[i], recycle); + if (rq->xsk_pool) { + /* The `recycle` parameter is ignored, and the page is always + * put into the Reuse Ring, because there is no way to return + * the page to the userspace when the interface goes down. + */ + for (i = 0; i < rq->mpwqe.pages_per_wqe; i++) + if (no_xdp_xmit || !test_bit(i, wi->xdp_xmit_bitmap)) + xsk_buff_free(alloc_units[i].xsk); + } else { + for (i = 0; i < rq->mpwqe.pages_per_wqe; i++) + if (no_xdp_xmit || !test_bit(i, wi->xdp_xmit_bitmap)) + mlx5e_page_release_dynamic(rq, alloc_units[i].page, recycle); + } } static void mlx5e_post_rx_mpwqe(struct mlx5e_rq *rq, u8 n) @@ -574,11 +556,13 @@ static int mlx5e_build_shampo_hd_umr(struct mlx5e_rq *rq, header_offset = (index & (MLX5E_SHAMPO_WQ_HEADER_PER_PAGE - 1)) << MLX5E_SHAMPO_LOG_MAX_HEADER_ENTRY_SIZE; if (!(header_offset & (PAGE_SIZE - 1))) { - err = mlx5e_page_alloc(rq, dma_info); + union mlx5e_alloc_unit au; + + err = mlx5e_page_alloc_pool(rq, &au); if (unlikely(err)) goto err_unmap; - addr = dma_info->addr; - page = dma_info->page; + page = dma_info->page = au.page; + addr = dma_info->addr = page_pool_get_dma_addr(au.page); } else { dma_info->addr = addr + header_offset; dma_info->page = page; @@ -611,7 +595,7 @@ err_unmap: dma_info = &shampo->info[--index]; if (!(i & (MLX5E_SHAMPO_WQ_HEADER_PER_PAGE - 1))) { dma_info->addr = ALIGN_DOWN(dma_info->addr, PAGE_SIZE); - mlx5e_page_release(rq, dma_info, true); + mlx5e_page_release_dynamic(rq, dma_info->page, true); } } rq->stats->buff_alloc_err++; @@ -659,57 +643,55 @@ static int mlx5e_alloc_rx_hd_mpwqe(struct mlx5e_rq *rq) static int mlx5e_alloc_rx_mpwqe(struct mlx5e_rq *rq, u16 ix) { - struct mlx5e_mpw_info *wi = &rq->mpwqe.info[ix]; - struct mlx5e_dma_info *dma_info = &wi->umr.dma_info[0]; + struct mlx5e_mpw_info *wi = mlx5e_get_mpw_info(rq, ix); + union mlx5e_alloc_unit *au = &wi->alloc_units[0]; struct mlx5e_icosq *sq = rq->icosq; struct mlx5_wq_cyc *wq = &sq->wq; struct mlx5e_umr_wqe *umr_wqe; + u32 offset; /* 17-bit value with MTT. */ u16 pi; int err; int i; - /* Check in advance that we have enough frames, instead of allocating - * one-by-one, failing and moving frames to the Reuse Ring. - */ - if (rq->xsk_pool && - unlikely(!xsk_buff_can_alloc(rq->xsk_pool, MLX5_MPWRQ_PAGES_PER_WQE))) { - err = -ENOMEM; - goto err; - } - if (test_bit(MLX5E_RQ_STATE_SHAMPO, &rq->state)) { err = mlx5e_alloc_rx_hd_mpwqe(rq); if (unlikely(err)) goto err; } - pi = mlx5e_icosq_get_next_pi(sq, MLX5E_UMR_WQEBBS); + pi = mlx5e_icosq_get_next_pi(sq, rq->mpwqe.umr_wqebbs); umr_wqe = mlx5_wq_cyc_get_wqe(wq, pi); - memcpy(umr_wqe, &rq->mpwqe.umr_wqe, offsetof(struct mlx5e_umr_wqe, inline_mtts)); + memcpy(umr_wqe, &rq->mpwqe.umr_wqe, sizeof(struct mlx5e_umr_wqe)); + + for (i = 0; i < rq->mpwqe.pages_per_wqe; i++, au++) { + dma_addr_t addr; - for (i = 0; i < MLX5_MPWRQ_PAGES_PER_WQE; i++, dma_info++) { - err = mlx5e_page_alloc(rq, dma_info); + err = mlx5e_page_alloc_pool(rq, au); if (unlikely(err)) goto err_unmap; - umr_wqe->inline_mtts[i].ptag = cpu_to_be64(dma_info->addr | MLX5_EN_WR); + addr = page_pool_get_dma_addr(au->page); + umr_wqe->inline_mtts[i] = (struct mlx5_mtt) { + .ptag = cpu_to_be64(addr | MLX5_EN_WR), + }; } - bitmap_zero(wi->xdp_xmit_bitmap, MLX5_MPWRQ_PAGES_PER_WQE); + bitmap_zero(wi->xdp_xmit_bitmap, rq->mpwqe.pages_per_wqe); wi->consumed_strides = 0; umr_wqe->ctrl.opmod_idx_opcode = cpu_to_be32((sq->pc << MLX5_WQE_CTRL_WQE_INDEX_SHIFT) | MLX5_OPCODE_UMR); - umr_wqe->uctrl.xlt_offset = - cpu_to_be16(MLX5_ALIGNED_MTTS_OCTW(MLX5E_REQUIRED_MTTS(ix))); + + offset = (ix * rq->mpwqe.mtts_per_wqe) * sizeof(struct mlx5_mtt) / MLX5_OCTWORD; + umr_wqe->uctrl.xlt_offset = cpu_to_be16(offset); sq->db.wqe_info[pi] = (struct mlx5e_icosq_wqe_info) { .wqe_type = MLX5E_ICOSQ_WQE_UMR_RX, - .num_wqebbs = MLX5E_UMR_WQEBBS, + .num_wqebbs = rq->mpwqe.umr_wqebbs, .umr.rq = rq, }; - sq->pc += MLX5E_UMR_WQEBBS; + sq->pc += rq->mpwqe.umr_wqebbs; sq->doorbell_cseg = &umr_wqe->ctrl; @@ -717,8 +699,8 @@ static int mlx5e_alloc_rx_mpwqe(struct mlx5e_rq *rq, u16 ix) err_unmap: while (--i >= 0) { - dma_info--; - mlx5e_page_release(rq, dma_info, true); + au--; + mlx5e_page_release_dynamic(rq, au->page, true); } err: @@ -752,7 +734,7 @@ void mlx5e_shampo_dealloc_hd(struct mlx5e_rq *rq, u16 len, u16 start, bool close hd_info->addr = ALIGN_DOWN(hd_info->addr, PAGE_SIZE); if (hd_info->page != deleted_page) { deleted_page = hd_info->page; - mlx5e_page_release(rq, hd_info, false); + mlx5e_page_release_dynamic(rq, hd_info->page, false); } } @@ -767,7 +749,7 @@ void mlx5e_shampo_dealloc_hd(struct mlx5e_rq *rq, u16 len, u16 start, bool close static void mlx5e_dealloc_rx_mpwqe(struct mlx5e_rq *rq, u16 ix) { - struct mlx5e_mpw_info *wi = &rq->mpwqe.info[ix]; + struct mlx5e_mpw_info *wi = mlx5e_get_mpw_info(rq, ix); /* Don't recycle, this function is called on rq/netdev close */ mlx5e_free_rx_mpwqe(rq, wi, false); } @@ -775,38 +757,51 @@ static void mlx5e_dealloc_rx_mpwqe(struct mlx5e_rq *rq, u16 ix) INDIRECT_CALLABLE_SCOPE bool mlx5e_post_rx_wqes(struct mlx5e_rq *rq) { struct mlx5_wq_cyc *wq = &rq->wqe.wq; - u8 wqe_bulk; - int err; + int wqe_bulk, count; + bool busy = false; + u16 head; if (unlikely(!test_bit(MLX5E_RQ_STATE_ENABLED, &rq->state))) return false; - wqe_bulk = rq->wqe.info.wqe_bulk; - - if (mlx5_wq_cyc_missing(wq) < wqe_bulk) + if (mlx5_wq_cyc_missing(wq) < rq->wqe.info.wqe_bulk) return false; if (rq->page_pool) page_pool_nid_changed(rq->page_pool, numa_mem_id()); - do { - u16 head = mlx5_wq_cyc_get_head(wq); + wqe_bulk = mlx5_wq_cyc_missing(wq); + head = mlx5_wq_cyc_get_head(wq); - err = mlx5e_alloc_rx_wqes(rq, head, wqe_bulk); - if (unlikely(err)) { - rq->stats->buff_alloc_err++; - break; - } + /* Don't allow any newly allocated WQEs to share the same page with old + * WQEs that aren't completed yet. Stop earlier. + */ + wqe_bulk -= (head + wqe_bulk) & rq->wqe.info.wqe_index_mask; - mlx5_wq_cyc_push_n(wq, wqe_bulk); - } while (mlx5_wq_cyc_missing(wq) >= wqe_bulk); + if (!rq->xsk_pool) + count = mlx5e_alloc_rx_wqes(rq, head, wqe_bulk); + else if (likely(!rq->xsk_pool->dma_need_sync)) + count = mlx5e_xsk_alloc_rx_wqes_batched(rq, head, wqe_bulk); + else + /* If dma_need_sync is true, it's more efficient to call + * xsk_buff_alloc in a loop, rather than xsk_buff_alloc_batch, + * because the latter does the same check and returns only one + * frame. + */ + count = mlx5e_xsk_alloc_rx_wqes(rq, head, wqe_bulk); + + mlx5_wq_cyc_push_n(wq, count); + if (unlikely(count != wqe_bulk)) { + rq->stats->buff_alloc_err++; + busy = true; + } /* ensure wqes are visible to device before updating doorbell record */ dma_wmb(); mlx5_wq_cyc_update_db_record(wq); - return !!err; + return busy; } void mlx5e_free_icosq_descs(struct mlx5e_icosq *sq) @@ -974,7 +969,8 @@ INDIRECT_CALLABLE_SCOPE bool mlx5e_post_rx_mpwqes(struct mlx5e_rq *rq) head = rq->mpwqe.actual_wq_head; i = missing; do { - alloc_err = mlx5e_alloc_rx_mpwqe(rq, head); + alloc_err = rq->xsk_pool ? mlx5e_xsk_alloc_rx_mpwqe(rq, head) : + mlx5e_alloc_rx_mpwqe(rq, head); if (unlikely(alloc_err)) break; @@ -1421,6 +1417,9 @@ static inline void mlx5e_build_rx_skb(struct mlx5_cqe64 *cqe, if (unlikely(mlx5_ipsec_is_rx_flow(cqe))) mlx5e_ipsec_offload_handle_rx_skb(netdev, skb, cqe); + if (unlikely(mlx5e_macsec_is_rx_flow(cqe))) + mlx5e_macsec_offload_handle_rx_skb(netdev, skb, cqe); + if (lro_num_seg > 1) { mlx5e_lro_update_hdr(skb, cqe, cqe_bcnt); skb_shinfo(skb)->gso_size = DIV_ROUND_UP(cqe_bcnt, lro_num_seg); @@ -1524,19 +1523,21 @@ static struct sk_buff * mlx5e_skb_from_cqe_linear(struct mlx5e_rq *rq, struct mlx5e_wqe_frag_info *wi, u32 cqe_bcnt) { - struct mlx5e_dma_info *di = wi->di; + union mlx5e_alloc_unit *au = wi->au; u16 rx_headroom = rq->buff.headroom; struct bpf_prog *prog; struct sk_buff *skb; u32 metasize = 0; void *va, *data; + dma_addr_t addr; u32 frag_size; - va = page_address(di->page) + wi->offset; + va = page_address(au->page) + wi->offset; data = va + rx_headroom; frag_size = MLX5_SKB_FRAG_SZ(rx_headroom + cqe_bcnt); - dma_sync_single_range_for_cpu(rq->pdev, di->addr, wi->offset, + addr = page_pool_get_dma_addr(au->page); + dma_sync_single_range_for_cpu(rq->pdev, addr, wi->offset, frag_size, DMA_FROM_DEVICE); net_prefetch(data); @@ -1546,7 +1547,7 @@ mlx5e_skb_from_cqe_linear(struct mlx5e_rq *rq, struct mlx5e_wqe_frag_info *wi, net_prefetchw(va); /* xdp_frame data area */ mlx5e_fill_xdp_buff(rq, va, rx_headroom, cqe_bcnt, &xdp); - if (mlx5e_xdp_handle(rq, di->page, prog, &xdp)) + if (mlx5e_xdp_handle(rq, au->page, prog, &xdp)) return NULL; /* page/packet was consumed by XDP */ rx_headroom = xdp.data - xdp.data_hard_start; @@ -1559,7 +1560,7 @@ mlx5e_skb_from_cqe_linear(struct mlx5e_rq *rq, struct mlx5e_wqe_frag_info *wi, return NULL; /* queue up for recycling/reuse */ - page_ref_inc(di->page); + page_ref_inc(au->page); return skb; } @@ -1570,20 +1571,22 @@ mlx5e_skb_from_cqe_nonlinear(struct mlx5e_rq *rq, struct mlx5e_wqe_frag_info *wi { struct mlx5e_rq_frag_info *frag_info = &rq->wqe.info.arr[0]; struct mlx5e_wqe_frag_info *head_wi = wi; + union mlx5e_alloc_unit *au = wi->au; u16 rx_headroom = rq->buff.headroom; - struct mlx5e_dma_info *di = wi->di; struct skb_shared_info *sinfo; u32 frag_consumed_bytes; struct bpf_prog *prog; struct xdp_buff xdp; struct sk_buff *skb; + dma_addr_t addr; u32 truesize; void *va; - va = page_address(di->page) + wi->offset; + va = page_address(au->page) + wi->offset; frag_consumed_bytes = min_t(u32, frag_info->frag_size, cqe_bcnt); - dma_sync_single_range_for_cpu(rq->pdev, di->addr, wi->offset, + addr = page_pool_get_dma_addr(au->page); + dma_sync_single_range_for_cpu(rq->pdev, addr, wi->offset, rq->buff.frame0_sz, DMA_FROM_DEVICE); net_prefetchw(va); /* xdp_frame data area */ net_prefetch(va + rx_headroom); @@ -1599,11 +1602,12 @@ mlx5e_skb_from_cqe_nonlinear(struct mlx5e_rq *rq, struct mlx5e_wqe_frag_info *wi while (cqe_bcnt) { skb_frag_t *frag; - di = wi->di; + au = wi->au; frag_consumed_bytes = min_t(u32, frag_info->frag_size, cqe_bcnt); - dma_sync_single_for_cpu(rq->pdev, di->addr + wi->offset, + addr = page_pool_get_dma_addr(au->page); + dma_sync_single_for_cpu(rq->pdev, addr + wi->offset, frag_consumed_bytes, DMA_FROM_DEVICE); if (!xdp_buff_has_frags(&xdp)) { @@ -1616,11 +1620,11 @@ mlx5e_skb_from_cqe_nonlinear(struct mlx5e_rq *rq, struct mlx5e_wqe_frag_info *wi } frag = &sinfo->frags[sinfo->nr_frags++]; - __skb_frag_set_page(frag, di->page); + __skb_frag_set_page(frag, au->page); skb_frag_off_set(frag, wi->offset); skb_frag_size_set(frag, frag_consumed_bytes); - if (page_is_pfmemalloc(di->page)) + if (page_is_pfmemalloc(au->page)) xdp_buff_set_frag_pfmemalloc(&xdp); sinfo->xdp_frags_size += frag_consumed_bytes; @@ -1631,10 +1635,10 @@ mlx5e_skb_from_cqe_nonlinear(struct mlx5e_rq *rq, struct mlx5e_wqe_frag_info *wi wi++; } - di = head_wi->di; + au = head_wi->au; prog = rcu_dereference(rq->xdp_prog); - if (prog && mlx5e_xdp_handle(rq, di->page, prog, &xdp)) { + if (prog && mlx5e_xdp_handle(rq, au->page, prog, &xdp)) { if (test_bit(MLX5E_RQ_FLAG_XDP_XMIT, rq->flags)) { int i; @@ -1651,7 +1655,7 @@ mlx5e_skb_from_cqe_nonlinear(struct mlx5e_rq *rq, struct mlx5e_wqe_frag_info *wi if (unlikely(!skb)) return NULL; - page_ref_inc(di->page); + page_ref_inc(au->page); if (unlikely(xdp_buff_has_frags(&xdp))) { int i; @@ -1706,9 +1710,10 @@ static void mlx5e_handle_rx_cqe(struct mlx5e_rq *rq, struct mlx5_cqe64 *cqe) goto free_wqe; } - skb = INDIRECT_CALL_2(rq->wqe.skb_from_cqe, + skb = INDIRECT_CALL_3(rq->wqe.skb_from_cqe, mlx5e_skb_from_cqe_linear, mlx5e_skb_from_cqe_nonlinear, + mlx5e_xsk_skb_from_cqe_linear, rq, wi, cqe_bcnt); if (!skb) { /* probably for XDP */ @@ -1791,11 +1796,11 @@ static void mlx5e_handle_rx_cqe_mpwrq_rep(struct mlx5e_rq *rq, struct mlx5_cqe64 { u16 cstrides = mpwrq_get_cqe_consumed_strides(cqe); u16 wqe_id = be16_to_cpu(cqe->wqe_id); - struct mlx5e_mpw_info *wi = &rq->mpwqe.info[wqe_id]; + struct mlx5e_mpw_info *wi = mlx5e_get_mpw_info(rq, wqe_id); u16 stride_ix = mpwrq_get_cqe_stride_index(cqe); u32 wqe_offset = stride_ix << rq->mpwqe.log_stride_sz; - u32 head_offset = wqe_offset & (PAGE_SIZE - 1); - u32 page_idx = wqe_offset >> PAGE_SHIFT; + u32 head_offset = wqe_offset & ((1 << rq->mpwqe.page_shift) - 1); + u32 page_idx = wqe_offset >> rq->mpwqe.page_shift; struct mlx5e_rx_wqe_ll *wqe; struct mlx5_wq_ll *wq; struct sk_buff *skb; @@ -1846,12 +1851,13 @@ const struct mlx5e_rx_handlers mlx5e_rx_handlers_rep = { #endif static void -mlx5e_fill_skb_data(struct sk_buff *skb, struct mlx5e_rq *rq, struct mlx5e_dma_info *di, - u32 data_bcnt, u32 data_offset) +mlx5e_fill_skb_data(struct sk_buff *skb, struct mlx5e_rq *rq, + union mlx5e_alloc_unit *au, u32 data_bcnt, u32 data_offset) { net_prefetchw(skb->data); while (data_bcnt) { + /* Non-linear mode, hence non-XSK, which always uses PAGE_SIZE. */ u32 pg_consumed_bytes = min_t(u32, PAGE_SIZE - data_offset, data_bcnt); unsigned int truesize; @@ -1860,12 +1866,12 @@ mlx5e_fill_skb_data(struct sk_buff *skb, struct mlx5e_rq *rq, struct mlx5e_dma_i else truesize = ALIGN(pg_consumed_bytes, BIT(rq->mpwqe.log_stride_sz)); - mlx5e_add_skb_frag(rq, skb, di, data_offset, + mlx5e_add_skb_frag(rq, skb, au, data_offset, pg_consumed_bytes, truesize); data_bcnt -= pg_consumed_bytes; data_offset = 0; - di++; + au++; } } @@ -1873,12 +1879,13 @@ static struct sk_buff * mlx5e_skb_from_cqe_mpwrq_nonlinear(struct mlx5e_rq *rq, struct mlx5e_mpw_info *wi, u16 cqe_bcnt, u32 head_offset, u32 page_idx) { + union mlx5e_alloc_unit *au = &wi->alloc_units[page_idx]; u16 headlen = min_t(u16, MLX5E_RX_MAX_HEAD, cqe_bcnt); - struct mlx5e_dma_info *di = &wi->umr.dma_info[page_idx]; u32 frag_offset = head_offset + headlen; u32 byte_cnt = cqe_bcnt - headlen; - struct mlx5e_dma_info *head_di = di; + union mlx5e_alloc_unit *head_au = au; struct sk_buff *skb; + dma_addr_t addr; skb = napi_alloc_skb(rq->cq.napi, ALIGN(MLX5E_RX_MAX_HEAD, sizeof(long))); @@ -1889,14 +1896,17 @@ mlx5e_skb_from_cqe_mpwrq_nonlinear(struct mlx5e_rq *rq, struct mlx5e_mpw_info *w net_prefetchw(skb->data); + /* Non-linear mode, hence non-XSK, which always uses PAGE_SIZE. */ if (unlikely(frag_offset >= PAGE_SIZE)) { - di++; + au++; frag_offset -= PAGE_SIZE; } - mlx5e_fill_skb_data(skb, rq, di, byte_cnt, frag_offset); + mlx5e_fill_skb_data(skb, rq, au, byte_cnt, frag_offset); /* copy header */ - mlx5e_copy_skb_header(rq->pdev, skb, head_di, head_offset, head_offset, headlen); + addr = page_pool_get_dma_addr(head_au->page); + mlx5e_copy_skb_header(rq->pdev, skb, head_au->page, addr, + head_offset, head_offset, headlen); /* skb linear part was allocated with headlen and aligned to long */ skb->tail += headlen; skb->len += headlen; @@ -1908,12 +1918,13 @@ static struct sk_buff * mlx5e_skb_from_cqe_mpwrq_linear(struct mlx5e_rq *rq, struct mlx5e_mpw_info *wi, u16 cqe_bcnt, u32 head_offset, u32 page_idx) { - struct mlx5e_dma_info *di = &wi->umr.dma_info[page_idx]; + union mlx5e_alloc_unit *au = &wi->alloc_units[page_idx]; u16 rx_headroom = rq->buff.headroom; struct bpf_prog *prog; struct sk_buff *skb; u32 metasize = 0; void *va, *data; + dma_addr_t addr; u32 frag_size; /* Check packet size. Note LRO doesn't use linear SKB */ @@ -1922,11 +1933,12 @@ mlx5e_skb_from_cqe_mpwrq_linear(struct mlx5e_rq *rq, struct mlx5e_mpw_info *wi, return NULL; } - va = page_address(di->page) + head_offset; + va = page_address(au->page) + head_offset; data = va + rx_headroom; frag_size = MLX5_SKB_FRAG_SZ(rx_headroom + cqe_bcnt); - dma_sync_single_range_for_cpu(rq->pdev, di->addr, head_offset, + addr = page_pool_get_dma_addr(au->page); + dma_sync_single_range_for_cpu(rq->pdev, addr, head_offset, frag_size, DMA_FROM_DEVICE); net_prefetch(data); @@ -1936,7 +1948,7 @@ mlx5e_skb_from_cqe_mpwrq_linear(struct mlx5e_rq *rq, struct mlx5e_mpw_info *wi, net_prefetchw(va); /* xdp_frame data area */ mlx5e_fill_xdp_buff(rq, va, rx_headroom, cqe_bcnt, &xdp); - if (mlx5e_xdp_handle(rq, di->page, prog, &xdp)) { + if (mlx5e_xdp_handle(rq, au->page, prog, &xdp)) { if (__test_and_clear_bit(MLX5E_RQ_FLAG_XDP_XMIT, rq->flags)) __set_bit(page_idx, wi->xdp_xmit_bitmap); /* non-atomic */ return NULL; /* page/packet was consumed by XDP */ @@ -1952,7 +1964,7 @@ mlx5e_skb_from_cqe_mpwrq_linear(struct mlx5e_rq *rq, struct mlx5e_mpw_info *wi, return NULL; /* queue up for recycling/reuse */ - page_ref_inc(di->page); + page_ref_inc(au->page); return skb; } @@ -1997,7 +2009,7 @@ mlx5e_skb_from_cqe_shampo(struct mlx5e_rq *rq, struct mlx5e_mpw_info *wi, } prefetchw(skb->data); - mlx5e_copy_skb_header(rq->pdev, skb, head, + mlx5e_copy_skb_header(rq->pdev, skb, head->page, head->addr, head_offset + rx_headroom, rx_headroom, head_size); /* skb linear part was allocated with headlen and aligned to long */ @@ -2049,7 +2061,7 @@ mlx5e_free_rx_shampo_hd_entry(struct mlx5e_rq *rq, u16 header_index) if (((header_index + 1) & (MLX5E_SHAMPO_WQ_HEADER_PER_PAGE - 1)) == 0) { shampo->info[header_index].addr = ALIGN_DOWN(addr, PAGE_SIZE); - mlx5e_page_release(rq, &shampo->info[header_index], true); + mlx5e_page_release_dynamic(rq, shampo->info[header_index].page, true); } bitmap_clear(shampo->bitmap, header_index, 1); } @@ -2070,11 +2082,11 @@ static void mlx5e_handle_rx_cqe_mpwrq_shampo(struct mlx5e_rq *rq, struct mlx5_cq bool match = cqe->shampo.match; struct mlx5e_rq_stats *stats = rq->stats; struct mlx5e_rx_wqe_ll *wqe; - struct mlx5e_dma_info *di; + union mlx5e_alloc_unit *au; struct mlx5e_mpw_info *wi; struct mlx5_wq_ll *wq; - wi = &rq->mpwqe.info[wqe_id]; + wi = mlx5e_get_mpw_info(rq, wqe_id); wi->consumed_strides += cstrides; if (unlikely(MLX5E_RX_ERR_CQE(cqe))) { @@ -2120,8 +2132,8 @@ static void mlx5e_handle_rx_cqe_mpwrq_shampo(struct mlx5e_rq *rq, struct mlx5_cq } if (likely(head_size)) { - di = &wi->umr.dma_info[page_idx]; - mlx5e_fill_skb_data(*skb, rq, di, data_bcnt, data_offset); + au = &wi->alloc_units[page_idx]; + mlx5e_fill_skb_data(*skb, rq, au, data_bcnt, data_offset); } mlx5e_shampo_complete_rx_cqe(rq, cqe, cqe_bcnt, *skb); @@ -2143,11 +2155,11 @@ static void mlx5e_handle_rx_cqe_mpwrq(struct mlx5e_rq *rq, struct mlx5_cqe64 *cq { u16 cstrides = mpwrq_get_cqe_consumed_strides(cqe); u16 wqe_id = be16_to_cpu(cqe->wqe_id); - struct mlx5e_mpw_info *wi = &rq->mpwqe.info[wqe_id]; + struct mlx5e_mpw_info *wi = mlx5e_get_mpw_info(rq, wqe_id); u16 stride_ix = mpwrq_get_cqe_stride_index(cqe); u32 wqe_offset = stride_ix << rq->mpwqe.log_stride_sz; - u32 head_offset = wqe_offset & (PAGE_SIZE - 1); - u32 page_idx = wqe_offset >> PAGE_SHIFT; + u32 head_offset = wqe_offset & ((1 << rq->mpwqe.page_shift) - 1); + u32 page_idx = wqe_offset >> rq->mpwqe.page_shift; struct mlx5e_rx_wqe_ll *wqe; struct mlx5_wq_ll *wq; struct sk_buff *skb; @@ -2170,9 +2182,10 @@ static void mlx5e_handle_rx_cqe_mpwrq(struct mlx5e_rq *rq, struct mlx5_cqe64 *cq cqe_bcnt = mpwrq_get_cqe_byte_cnt(cqe); - skb = INDIRECT_CALL_2(rq->mpwqe.skb_from_cqe_mpwrq, + skb = INDIRECT_CALL_3(rq->mpwqe.skb_from_cqe_mpwrq, mlx5e_skb_from_cqe_mpwrq_linear, mlx5e_skb_from_cqe_mpwrq_nonlinear, + mlx5e_xsk_skb_from_cqe_mpwrq_linear, rq, wi, cqe_bcnt, head_offset, page_idx); if (!skb) goto mpwrq_cqe_out; @@ -2417,7 +2430,7 @@ int mlx5e_rq_set_handlers(struct mlx5e_rq *rq, struct mlx5e_params *params, bool default: /* MLX5_WQ_TYPE_CYCLIC */ rq->wqe.skb_from_cqe = xsk ? mlx5e_xsk_skb_from_cqe_linear : - mlx5e_rx_is_linear_skb(params, NULL) ? + mlx5e_rx_is_linear_skb(mdev, params, NULL) ? mlx5e_skb_from_cqe_linear : mlx5e_skb_from_cqe_nonlinear; rq->post_wqes = mlx5e_post_rx_wqes; @@ -2471,7 +2484,7 @@ free_wqe: void mlx5e_rq_set_trap_handlers(struct mlx5e_rq *rq, struct mlx5e_params *params) { - rq->wqe.skb_from_cqe = mlx5e_rx_is_linear_skb(params, NULL) ? + rq->wqe.skb_from_cqe = mlx5e_rx_is_linear_skb(rq->mdev, params, NULL) ? mlx5e_skb_from_cqe_linear : mlx5e_skb_from_cqe_nonlinear; rq->post_wqes = mlx5e_post_rx_wqes; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_stats.c b/drivers/net/ethernet/mellanox/mlx5/core/en_stats.c index 7409829d1201..03c1841970f1 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_stats.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_stats.c @@ -641,17 +641,26 @@ static const struct counter_desc vnic_env_stats_dev_oob_desc[] = { VNIC_ENV_OFF(vport_env.internal_rq_out_of_buffer) }, }; +static const struct counter_desc vnic_env_stats_drop_desc[] = { + { "rx_oversize_pkts_buffer", + VNIC_ENV_OFF(vport_env.eth_wqe_too_small) }, +}; + #define NUM_VNIC_ENV_STEER_COUNTERS(dev) \ (MLX5_CAP_GEN(dev, nic_receive_steering_discard) ? \ ARRAY_SIZE(vnic_env_stats_steer_desc) : 0) #define NUM_VNIC_ENV_DEV_OOB_COUNTERS(dev) \ (MLX5_CAP_GEN(dev, vnic_env_int_rq_oob) ? \ ARRAY_SIZE(vnic_env_stats_dev_oob_desc) : 0) +#define NUM_VNIC_ENV_DROP_COUNTERS(dev) \ + (MLX5_CAP_GEN(dev, eth_wqe_too_small) ? \ + ARRAY_SIZE(vnic_env_stats_drop_desc) : 0) static MLX5E_DECLARE_STATS_GRP_OP_NUM_STATS(vnic_env) { return NUM_VNIC_ENV_STEER_COUNTERS(priv->mdev) + - NUM_VNIC_ENV_DEV_OOB_COUNTERS(priv->mdev); + NUM_VNIC_ENV_DEV_OOB_COUNTERS(priv->mdev) + + NUM_VNIC_ENV_DROP_COUNTERS(priv->mdev); } static MLX5E_DECLARE_STATS_GRP_OP_FILL_STRS(vnic_env) @@ -665,6 +674,11 @@ static MLX5E_DECLARE_STATS_GRP_OP_FILL_STRS(vnic_env) for (i = 0; i < NUM_VNIC_ENV_DEV_OOB_COUNTERS(priv->mdev); i++) strcpy(data + (idx++) * ETH_GSTRING_LEN, vnic_env_stats_dev_oob_desc[i].format); + + for (i = 0; i < NUM_VNIC_ENV_DROP_COUNTERS(priv->mdev); i++) + strcpy(data + (idx++) * ETH_GSTRING_LEN, + vnic_env_stats_drop_desc[i].format); + return idx; } @@ -679,6 +693,11 @@ static MLX5E_DECLARE_STATS_GRP_OP_FILL_STATS(vnic_env) for (i = 0; i < NUM_VNIC_ENV_DEV_OOB_COUNTERS(priv->mdev); i++) data[idx++] = MLX5E_READ_CTR32_BE(priv->stats.vnic.query_vnic_env_out, vnic_env_stats_dev_oob_desc, i); + + for (i = 0; i < NUM_VNIC_ENV_DROP_COUNTERS(priv->mdev); i++) + data[idx++] = MLX5E_READ_CTR32_BE(priv->stats.vnic.query_vnic_env_out, + vnic_env_stats_drop_desc, i); + return idx; } @@ -2451,6 +2470,9 @@ mlx5e_stats_grp_t mlx5e_nic_stats_grps[] = { &MLX5E_STATS_GRP(per_port_buff_congest), &MLX5E_STATS_GRP(ptp), &MLX5E_STATS_GRP(qos), +#ifdef CONFIG_MLX5_EN_MACSEC + &MLX5E_STATS_GRP(macsec_hw), +#endif }; unsigned int mlx5e_nic_stats_grps_num(struct mlx5e_priv *priv) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_stats.h b/drivers/net/ethernet/mellanox/mlx5/core/en_stats.h index ed4fc940e4ef..9f781085be47 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_stats.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_stats.h @@ -273,6 +273,10 @@ struct mlx5e_qcounter_stats { u32 rx_if_down_packets; }; +#define VNIC_ENV_GET(vnic_env_stats, c) \ + MLX5_GET(query_vnic_env_out, (vnic_env_stats)->query_vnic_env_out, \ + vport_env.c) + struct mlx5e_vnic_env_stats { __be64 query_vnic_env_out[MLX5_ST_SZ_QW(query_vnic_env_out)]; }; @@ -486,5 +490,6 @@ extern MLX5E_DECLARE_STATS_GRP(channels); extern MLX5E_DECLARE_STATS_GRP(per_port_buff_congest); extern MLX5E_DECLARE_STATS_GRP(ipsec_sw); extern MLX5E_DECLARE_STATS_GRP(ptp); +extern MLX5E_DECLARE_STATS_GRP(macsec_hw); #endif /* __MLX5_EN_STATS_H__ */ diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_tc.c b/drivers/net/ethernet/mellanox/mlx5/core/en_tc.c index f154bda668ad..70a7a61f9708 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_tc.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_tc.c @@ -311,6 +311,7 @@ mlx5e_get_flow_meters(struct mlx5_core_dev *dev) static struct mlx5_tc_ct_priv * get_ct_priv(struct mlx5e_priv *priv) { + struct mlx5e_tc_table *tc = mlx5e_fs_get_tc(priv->fs); struct mlx5_eswitch *esw = priv->mdev->priv.eswitch; struct mlx5_rep_uplink_priv *uplink_priv; struct mlx5e_rep_priv *uplink_rpriv; @@ -322,7 +323,7 @@ get_ct_priv(struct mlx5e_priv *priv) return uplink_priv->ct_priv; } - return priv->fs->tc->ct; + return tc->ct; } static struct mlx5e_tc_psample * @@ -345,6 +346,7 @@ get_sample_priv(struct mlx5e_priv *priv) static struct mlx5e_post_act * get_post_action(struct mlx5e_priv *priv) { + struct mlx5e_tc_table *tc = mlx5e_fs_get_tc(priv->fs); struct mlx5_eswitch *esw = priv->mdev->priv.eswitch; struct mlx5_rep_uplink_priv *uplink_priv; struct mlx5e_rep_priv *uplink_rpriv; @@ -356,7 +358,7 @@ get_post_action(struct mlx5e_priv *priv) return uplink_priv->post_act; } - return priv->fs->tc->post_act; + return tc->post_act; } struct mlx5_flow_handle * @@ -607,11 +609,12 @@ int mlx5e_get_flow_namespace(struct mlx5e_tc_flow *flow) static struct mod_hdr_tbl * get_mod_hdr_table(struct mlx5e_priv *priv, struct mlx5e_tc_flow *flow) { + struct mlx5e_tc_table *tc = mlx5e_fs_get_tc(priv->fs); struct mlx5_eswitch *esw = priv->mdev->priv.eswitch; return mlx5e_get_flow_namespace(flow) == MLX5_FLOW_NAMESPACE_FDB ? &esw->offloads.mod_hdr : - &priv->fs->tc->mod_hdr; + &tc->mod_hdr; } static int mlx5e_attach_mod_hdr(struct mlx5e_priv *priv, @@ -810,6 +813,7 @@ static int mlx5e_hairpin_rss_init(struct mlx5e_hairpin *hp) { struct mlx5e_priv *priv = hp->func_priv; struct ttc_params ttc_params; + struct mlx5_ttc_table *ttc; int err; err = mlx5e_hairpin_create_indirect_rqt(hp); @@ -827,9 +831,10 @@ static int mlx5e_hairpin_rss_init(struct mlx5e_hairpin *hp) goto err_create_ttc_table; } + ttc = mlx5e_fs_get_ttc(priv->fs, false); netdev_dbg(priv->netdev, "add hairpin: using %d channels rss ttc table id %x\n", hp->num_channels, - mlx5_get_ttc_flow_table(priv->fs->ttc)->id); + mlx5_get_ttc_flow_table(ttc)->id); return 0; @@ -916,10 +921,11 @@ static inline u32 hash_hairpin_info(u16 peer_vhca_id, u8 prio) static struct mlx5e_hairpin_entry *mlx5e_hairpin_get(struct mlx5e_priv *priv, u16 peer_vhca_id, u8 prio) { + struct mlx5e_tc_table *tc = mlx5e_fs_get_tc(priv->fs); struct mlx5e_hairpin_entry *hpe; u32 hash_key = hash_hairpin_info(peer_vhca_id, prio); - hash_for_each_possible(priv->fs->tc->hairpin_tbl, hpe, + hash_for_each_possible(tc->hairpin_tbl, hpe, hairpin_hlist, hash_key) { if (hpe->peer_vhca_id == peer_vhca_id && hpe->prio == prio) { refcount_inc(&hpe->refcnt); @@ -933,11 +939,12 @@ static struct mlx5e_hairpin_entry *mlx5e_hairpin_get(struct mlx5e_priv *priv, static void mlx5e_hairpin_put(struct mlx5e_priv *priv, struct mlx5e_hairpin_entry *hpe) { + struct mlx5e_tc_table *tc = mlx5e_fs_get_tc(priv->fs); /* no more hairpin flows for us, release the hairpin pair */ - if (!refcount_dec_and_mutex_lock(&hpe->refcnt, &priv->fs->tc->hairpin_tbl_lock)) + if (!refcount_dec_and_mutex_lock(&hpe->refcnt, &tc->hairpin_tbl_lock)) return; hash_del(&hpe->hairpin_hlist); - mutex_unlock(&priv->fs->tc->hairpin_tbl_lock); + mutex_unlock(&tc->hairpin_tbl_lock); if (!IS_ERR_OR_NULL(hpe->hp)) { netdev_dbg(priv->netdev, "del hairpin: peer %s\n", @@ -993,6 +1000,7 @@ static int mlx5e_hairpin_flow_add(struct mlx5e_priv *priv, struct mlx5e_tc_flow_parse_attr *parse_attr, struct netlink_ext_ack *extack) { + struct mlx5e_tc_table *tc = mlx5e_fs_get_tc(priv->fs); int peer_ifindex = parse_attr->mirred_ifindex[0]; struct mlx5_hairpin_params params; struct mlx5_core_dev *peer_mdev; @@ -1021,10 +1029,10 @@ static int mlx5e_hairpin_flow_add(struct mlx5e_priv *priv, if (err) return err; - mutex_lock(&priv->fs->tc->hairpin_tbl_lock); + mutex_lock(&tc->hairpin_tbl_lock); hpe = mlx5e_hairpin_get(priv, peer_id, match_prio); if (hpe) { - mutex_unlock(&priv->fs->tc->hairpin_tbl_lock); + mutex_unlock(&tc->hairpin_tbl_lock); wait_for_completion(&hpe->res_ready); if (IS_ERR(hpe->hp)) { @@ -1036,7 +1044,7 @@ static int mlx5e_hairpin_flow_add(struct mlx5e_priv *priv, hpe = kzalloc(sizeof(*hpe), GFP_KERNEL); if (!hpe) { - mutex_unlock(&priv->fs->tc->hairpin_tbl_lock); + mutex_unlock(&tc->hairpin_tbl_lock); return -ENOMEM; } @@ -1048,9 +1056,9 @@ static int mlx5e_hairpin_flow_add(struct mlx5e_priv *priv, refcount_set(&hpe->refcnt, 1); init_completion(&hpe->res_ready); - hash_add(priv->fs->tc->hairpin_tbl, &hpe->hairpin_hlist, + hash_add(tc->hairpin_tbl, &hpe->hairpin_hlist, hash_hairpin_info(peer_id, match_prio)); - mutex_unlock(&priv->fs->tc->hairpin_tbl_lock); + mutex_unlock(&tc->hairpin_tbl_lock); params.log_data_size = 16; params.log_data_size = min_t(u8, params.log_data_size, @@ -1126,8 +1134,9 @@ mlx5e_add_offloaded_nic_rule(struct mlx5e_priv *priv, struct mlx5_flow_attr *attr) { struct mlx5_flow_context *flow_context = &spec->flow_context; + struct mlx5e_vlan_table *vlan = mlx5e_fs_get_vlan(priv->fs); + struct mlx5e_tc_table *tc = mlx5e_fs_get_tc(priv->fs); struct mlx5_nic_flow_attr *nic_attr = attr->nic_attr; - struct mlx5e_tc_table *tc = priv->fs->tc; struct mlx5_flow_destination dest[2] = {}; struct mlx5_fs_chains *nic_chains; struct mlx5_flow_act flow_act = { @@ -1163,7 +1172,7 @@ mlx5e_add_offloaded_nic_rule(struct mlx5e_priv *priv, if (IS_ERR(dest[dest_ix].ft)) return ERR_CAST(dest[dest_ix].ft); } else { - dest[dest_ix].ft = mlx5e_vlan_get_flowtable(priv->fs->vlan); + dest[dest_ix].ft = mlx5e_vlan_get_flowtable(vlan); } dest_ix++; } @@ -1191,7 +1200,7 @@ mlx5e_add_offloaded_nic_rule(struct mlx5e_priv *priv, mutex_unlock(&tc->t_lock); netdev_err(priv->netdev, "Failed to create tc offload table\n"); - rule = ERR_CAST(priv->fs->tc->t); + rule = ERR_CAST(tc->t); goto err_ft_get; } } @@ -1293,8 +1302,10 @@ void mlx5e_del_offloaded_nic_rule(struct mlx5e_priv *priv, struct mlx5_flow_handle *rule, struct mlx5_flow_attr *attr) { - struct mlx5_fs_chains *nic_chains = mlx5e_nic_chains(priv->fs->tc); + struct mlx5e_tc_table *tc = mlx5e_fs_get_tc(priv->fs); + struct mlx5_fs_chains *nic_chains; + nic_chains = mlx5e_nic_chains(tc); mlx5_del_flow_rules(rule); if (attr->chain || attr->prio) @@ -1309,8 +1320,8 @@ void mlx5e_del_offloaded_nic_rule(struct mlx5e_priv *priv, static void mlx5e_tc_del_nic_flow(struct mlx5e_priv *priv, struct mlx5e_tc_flow *flow) { + struct mlx5e_tc_table *tc = mlx5e_fs_get_tc(priv->fs); struct mlx5_flow_attr *attr = flow->attr; - struct mlx5e_tc_table *tc = priv->fs->tc; flow_flag_clear(flow, OFFLOADED); @@ -1322,13 +1333,13 @@ static void mlx5e_tc_del_nic_flow(struct mlx5e_priv *priv, /* Remove root table if no rules are left to avoid * extra steering hops. */ - mutex_lock(&priv->fs->tc->t_lock); + mutex_lock(&tc->t_lock); if (!mlx5e_tc_num_filters(priv, MLX5_TC_FLAG(NIC_OFFLOAD)) && !IS_ERR_OR_NULL(tc->t)) { mlx5_chains_put_table(mlx5e_nic_chains(tc), 0, 1, MLX5E_TC_FT_LEVEL); - priv->fs->tc->t = NULL; + tc->t = NULL; } - mutex_unlock(&priv->fs->tc->t_lock); + mutex_unlock(&tc->t_lock); if (attr->action & MLX5_FLOW_CONTEXT_ACTION_MOD_HDR) mlx5e_detach_mod_hdr(priv, flow); @@ -1494,8 +1505,11 @@ bool mlx5e_tc_is_vf_tunnel(struct net_device *out_dev, struct net_device *route_ route_priv = netdev_priv(route_dev); route_mdev = route_priv->mdev; - if (out_mdev->coredev_type != MLX5_COREDEV_PF || - route_mdev->coredev_type != MLX5_COREDEV_VF) + if (out_mdev->coredev_type != MLX5_COREDEV_PF) + return false; + + if (route_mdev->coredev_type != MLX5_COREDEV_VF && + route_mdev->coredev_type != MLX5_COREDEV_SF) return false; return mlx5e_same_hw_devs(out_priv, route_priv); @@ -4058,13 +4072,14 @@ static const struct rhashtable_params tc_ht_params = { static struct rhashtable *get_tc_ht(struct mlx5e_priv *priv, unsigned long flags) { + struct mlx5e_tc_table *tc = mlx5e_fs_get_tc(priv->fs); struct mlx5e_rep_priv *rpriv; if (flags & MLX5_TC_FLAG(ESW_OFFLOAD)) { rpriv = priv->ppriv; return &rpriv->tc_ht; } else /* NIC offload */ - return &priv->fs->tc->ht; + return &tc->ht; } static bool is_peer_flow_needed(struct mlx5e_tc_flow *flow) @@ -4448,7 +4463,7 @@ int mlx5e_configure_flower(struct net_device *dev, struct mlx5e_priv *priv, int err = 0; if (!mlx5_esw_hold(priv->mdev)) - return -EAGAIN; + return -EBUSY; mlx5_esw_get(priv->mdev); @@ -4772,6 +4787,7 @@ void mlx5e_tc_stats_matchall(struct mlx5e_priv *priv, static void mlx5e_tc_hairpin_update_dead_peer(struct mlx5e_priv *priv, struct mlx5e_priv *peer_priv) { + struct mlx5e_tc_table *tc = mlx5e_fs_get_tc(priv->fs); struct mlx5_core_dev *peer_mdev = peer_priv->mdev; struct mlx5e_hairpin_entry *hpe, *tmp; LIST_HEAD(init_wait_list); @@ -4783,11 +4799,11 @@ static void mlx5e_tc_hairpin_update_dead_peer(struct mlx5e_priv *priv, peer_vhca_id = MLX5_CAP_GEN(peer_mdev, vhca_id); - mutex_lock(&priv->fs->tc->hairpin_tbl_lock); - hash_for_each(priv->fs->tc->hairpin_tbl, bkt, hpe, hairpin_hlist) + mutex_lock(&tc->hairpin_tbl_lock); + hash_for_each(tc->hairpin_tbl, bkt, hpe, hairpin_hlist) if (refcount_inc_not_zero(&hpe->refcnt)) list_add(&hpe->dead_peer_wait_list, &init_wait_list); - mutex_unlock(&priv->fs->tc->hairpin_tbl_lock); + mutex_unlock(&tc->hairpin_tbl_lock); list_for_each_entry_safe(hpe, tmp, &init_wait_list, dead_peer_wait_list) { wait_for_completion(&hpe->res_ready); @@ -4841,7 +4857,8 @@ static int mlx5e_tc_nic_get_ft_size(struct mlx5_core_dev *dev) static int mlx5e_tc_nic_create_miss_table(struct mlx5e_priv *priv) { - struct mlx5_flow_table **ft = &priv->fs->tc->miss_t; + struct mlx5e_tc_table *tc = mlx5e_fs_get_tc(priv->fs); + struct mlx5_flow_table **ft = &tc->miss_t; struct mlx5_flow_table_attr ft_attr = {}; struct mlx5_flow_namespace *ns; int err = 0; @@ -4863,12 +4880,14 @@ static int mlx5e_tc_nic_create_miss_table(struct mlx5e_priv *priv) static void mlx5e_tc_nic_destroy_miss_table(struct mlx5e_priv *priv) { - mlx5_destroy_flow_table(priv->fs->tc->miss_t); + struct mlx5e_tc_table *tc = mlx5e_fs_get_tc(priv->fs); + + mlx5_destroy_flow_table(tc->miss_t); } int mlx5e_tc_nic_init(struct mlx5e_priv *priv) { - struct mlx5e_tc_table *tc = priv->fs->tc; + struct mlx5e_tc_table *tc = mlx5e_fs_get_tc(priv->fs); struct mlx5_core_dev *dev = priv->mdev; struct mapping_ctx *chains_mapping; struct mlx5_chains_attr attr = {}; @@ -4909,7 +4928,7 @@ int mlx5e_tc_nic_init(struct mlx5e_priv *priv) attr.ns = MLX5_FLOW_NAMESPACE_KERNEL; attr.max_ft_sz = mlx5e_tc_nic_get_ft_size(dev); attr.max_grp_num = MLX5E_TC_TABLE_NUM_GROUPS; - attr.default_ft = priv->fs->tc->miss_t; + attr.default_ft = tc->miss_t; attr.mapping = chains_mapping; tc->chains = mlx5_chains_create(dev, &attr); @@ -4958,7 +4977,7 @@ static void _mlx5e_tc_del_flow(void *ptr, void *arg) void mlx5e_tc_nic_cleanup(struct mlx5e_priv *priv) { - struct mlx5e_tc_table *tc = priv->fs->tc; + struct mlx5e_tc_table *tc = mlx5e_fs_get_tc(priv->fs); if (tc->netdevice_nb.notifier_call) unregister_netdevice_notifier_dev_net(priv->netdev, @@ -5163,13 +5182,13 @@ bool mlx5e_tc_update_skb(struct mlx5_cqe64 *cqe, #if IS_ENABLED(CONFIG_NET_TC_SKB_EXT) u32 chain = 0, chain_tag, reg_b, zone_restore_id; struct mlx5e_priv *priv = netdev_priv(skb->dev); - struct mlx5e_tc_table *tc = priv->fs->tc; struct mlx5_mapped_obj mapped_obj; struct tc_skb_ext *tc_skb_ext; + struct mlx5e_tc_table *tc; int err; reg_b = be32_to_cpu(cqe->ft_metadata); - + tc = mlx5e_fs_get_tc(priv->fs); chain_tag = reg_b & MLX5E_TC_TABLE_CHAIN_TAG_MASK; err = mapping_find(tc->mapping, chain_tag, &mapped_obj); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_tc.h b/drivers/net/ethernet/mellanox/mlx5/core/en_tc.h index 6ce1ab6b86b7..48241317a535 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_tc.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_tc.h @@ -54,6 +54,7 @@ ESW_FLOW_ATTR_SZ :\ NIC_FLOW_ATTR_SZ) +struct mlx5_fs_chains *mlx5e_nic_chains(struct mlx5e_tc_table *tc); int mlx5e_tc_num_filters(struct mlx5e_priv *priv, unsigned long flags); struct mlx5e_tc_update_priv { diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c index 27f791feb517..bf2232a2a836 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c @@ -39,6 +39,7 @@ #include "ipoib/ipoib.h" #include "en_accel/en_accel.h" #include "en_accel/ipsec_rxtx.h" +#include "en_accel/macsec.h" #include "en/ptp.h" #include <net/ipv6.h> @@ -485,7 +486,7 @@ err_drop: static bool mlx5e_tx_skb_supports_mpwqe(struct sk_buff *skb, struct mlx5e_tx_attr *attr) { return !skb_is_nonlinear(skb) && !skb_vlan_tag_present(skb) && !attr->ihs && - !attr->insz; + !attr->insz && !mlx5e_macsec_skb_is_offload(skb); } static bool mlx5e_tx_mpwqe_same_eseg(struct mlx5e_txqsq *sq, struct mlx5_wqe_eth_seg *eseg) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_txrx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_txrx.c index 833be29170a1..9a458a5d9853 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_txrx.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_txrx.c @@ -31,6 +31,7 @@ */ #include <linux/irq.h> +#include <net/xdp_sock_drv.h> #include "en.h" #include "en/txrx.h" #include "en/xdp.h" @@ -86,26 +87,36 @@ void mlx5e_trigger_irq(struct mlx5e_icosq *sq) static bool mlx5e_napi_xsk_post(struct mlx5e_xdpsq *xsksq, struct mlx5e_rq *xskrq) { + bool need_wakeup = xsk_uses_need_wakeup(xskrq->xsk_pool); bool busy_xsk = false, xsk_rx_alloc_err; - /* Handle the race between the application querying need_wakeup and the - * driver setting it: - * 1. Update need_wakeup both before and after the TX. If it goes to - * "yes", it can only happen with the first update. - * 2. If the application queried need_wakeup before we set it, the - * packets will be transmitted anyway, even w/o a wakeup. - * 3. Give a chance to clear need_wakeup after new packets were queued - * for TX. + /* If SQ is empty, there are no TX completions to trigger NAPI, so set + * need_wakeup. Do it before queuing packets for TX to avoid race + * condition with userspace. */ - mlx5e_xsk_update_tx_wakeup(xsksq); + if (need_wakeup && xsksq->pc == xsksq->cc) + xsk_set_tx_need_wakeup(xsksq->xsk_pool); busy_xsk |= mlx5e_xsk_tx(xsksq, MLX5E_TX_XSK_POLL_BUDGET); - mlx5e_xsk_update_tx_wakeup(xsksq); + /* If we queued some packets for TX, no need for wakeup anymore. */ + if (need_wakeup && xsksq->pc != xsksq->cc) + xsk_clear_tx_need_wakeup(xsksq->xsk_pool); + /* If WQ is empty, RX won't trigger NAPI, so set need_wakeup. Do it + * before refilling to avoid race condition with userspace. + */ + if (need_wakeup && !mlx5e_rqwq_get_cur_sz(xskrq)) + xsk_set_rx_need_wakeup(xskrq->xsk_pool); xsk_rx_alloc_err = INDIRECT_CALL_2(xskrq->post_wqes, mlx5e_post_rx_mpwqes, mlx5e_post_rx_wqes, xskrq); - busy_xsk |= mlx5e_xsk_update_rx_wakeup(xskrq, xsk_rx_alloc_err); + /* Ask for wakeup if WQ is not full after refill. */ + if (!need_wakeup) + busy_xsk |= xsk_rx_alloc_err; + else if (xsk_rx_alloc_err) + xsk_set_rx_need_wakeup(xskrq->xsk_pool); + else + xsk_clear_rx_need_wakeup(xskrq->xsk_pool); return busy_xsk; } diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eq.c b/drivers/net/ethernet/mellanox/mlx5/core/eq.c index 229728c80233..a0242dc15741 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/eq.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/eq.c @@ -575,6 +575,9 @@ static void gather_async_events_mask(struct mlx5_core_dev *dev, u64 mask[4]) if (MLX5_CAP_GEN_MAX(dev, vhca_state)) async_event_mask |= (1ull << MLX5_EVENT_TYPE_VHCA_STATE_CHANGE); + if (MLX5_CAP_MACSEC(dev, log_max_macsec_offload)) + async_event_mask |= (1ull << MLX5_EVENT_TYPE_OBJECT_CHANGE); + mask[0] = async_event_mask; if (MLX5_CAP_GEN(dev, event_cap)) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/esw/indir_table.c b/drivers/net/ethernet/mellanox/mlx5/core/esw/indir_table.c index 0abef71cb839..c9a91158e99c 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/esw/indir_table.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/esw/indir_table.c @@ -78,12 +78,16 @@ mlx5_esw_indir_table_needed(struct mlx5_eswitch *esw, struct mlx5_core_dev *dest_mdev) { struct mlx5_esw_flow_attr *esw_attr = attr->esw_attr; + bool vf_sf_vport; + + vf_sf_vport = mlx5_eswitch_is_vf_vport(esw, vport_num) || + mlx5_esw_is_sf_vport(esw, vport_num); /* Use indirect table for all IP traffic from UL to VF with vport * destination when source rewrite flag is set. */ return esw_attr->in_rep->vport == MLX5_VPORT_UPLINK && - mlx5_eswitch_is_vf_vport(esw, vport_num) && + vf_sf_vport && esw->dev == dest_mdev && attr->ip_version && attr->flags & MLX5_ATTR_FLAG_SRC_REWRITE; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/esw/qos.c b/drivers/net/ethernet/mellanox/mlx5/core/esw/qos.c index 694c54066955..4f8a24d84a86 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/esw/qos.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/esw/qos.c @@ -924,12 +924,16 @@ int mlx5_esw_qos_vport_update_group(struct mlx5_eswitch *esw, struct mlx5_esw_rate_group *group, struct netlink_ext_ack *extack) { - int err; + int err = 0; mutex_lock(&esw->state_lock); + if (!vport->qos.enabled && !group) + goto unlock; + err = esw_qos_vport_enable(esw, vport, 0, 0, extack); if (!err) err = esw_qos_vport_update_group(esw, vport, group, extack); +unlock: mutex_unlock(&esw->state_lock); return err; } diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c index 6aa58044b949..c59107fa9e6d 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c @@ -1360,7 +1360,6 @@ void mlx5_eswitch_disable_sriov(struct mlx5_eswitch *esw, bool clear_vf) if (esw->mode == MLX5_ESWITCH_OFFLOADS) { struct devlink *devlink = priv_to_devlink(esw->dev); - esw_offloads_del_send_to_vport_meta_rules(esw); devl_rate_nodes_destroy(devlink); } diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h index 87ce5a208cb5..f68dc2d0dbe6 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h @@ -244,6 +244,8 @@ struct mlx5_esw_offload { struct mlx5_flow_table *ft_offloads; struct mlx5_flow_group *vport_rx_group; + struct mlx5_flow_group *vport_rx_drop_group; + struct mlx5_flow_handle *vport_rx_drop_rule; struct xarray vport_reps; struct list_head peer_flows; struct mutex peer_mutex; @@ -344,7 +346,10 @@ void esw_offloads_disable(struct mlx5_eswitch *esw); int esw_offloads_enable(struct mlx5_eswitch *esw); void esw_offloads_cleanup_reps(struct mlx5_eswitch *esw); int esw_offloads_init_reps(struct mlx5_eswitch *esw); -void esw_offloads_del_send_to_vport_meta_rules(struct mlx5_eswitch *esw); + +struct mlx5_flow_handle * +mlx5_eswitch_add_send_to_vport_meta_rule(struct mlx5_eswitch *esw, u16 vport_num); +void mlx5_eswitch_del_send_to_vport_meta_rule(struct mlx5_flow_handle *rule); bool mlx5_esw_vport_match_metadata_supported(const struct mlx5_eswitch *esw); int mlx5_esw_offloads_vport_metadata_set(struct mlx5_eswitch *esw, bool enable); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c index a9f4c652f859..4e50df3139c6 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c @@ -70,6 +70,8 @@ #define MLX5_ESW_VPORT_TBL_SIZE 128 #define MLX5_ESW_VPORT_TBL_NUM_GROUPS 4 +#define MLX5_ESW_FT_OFFLOADS_DROP_RULE (1) + static const struct esw_vport_tbl_namespace mlx5_esw_vport_tbl_mirror_ns = { .max_fte = MLX5_ESW_VPORT_TBL_SIZE, .max_num_groups = MLX5_ESW_VPORT_TBL_NUM_GROUPS, @@ -481,25 +483,27 @@ esw_setup_dests(struct mlx5_flow_destination *dest, !(attr->flags & MLX5_ATTR_FLAG_SLOW_PATH)) { esw_setup_sampler_dest(dest, flow_act, attr->sample_attr.sampler_id, *i); (*i)++; - } else if (attr->dest_ft) { - esw_setup_ft_dest(dest, flow_act, esw, attr, spec, *i); - (*i)++; } else if (attr->flags & MLX5_ATTR_FLAG_SLOW_PATH) { esw_setup_slow_path_dest(dest, flow_act, esw, *i); (*i)++; } else if (attr->flags & MLX5_ATTR_FLAG_ACCEPT) { esw_setup_accept_dest(dest, flow_act, chains, *i); (*i)++; - } else if (attr->dest_chain) { - err = esw_setup_chain_dest(dest, flow_act, chains, attr->dest_chain, - 1, 0, *i); - (*i)++; } else if (esw_is_indir_table(esw, attr)) { err = esw_setup_indir_table(dest, flow_act, esw, attr, spec, true, i); } else if (esw_is_chain_src_port_rewrite(esw, esw_attr)) { err = esw_setup_chain_src_port_rewrite(dest, flow_act, esw, chains, attr, i); } else { *i = esw_setup_vport_dests(dest, flow_act, esw, esw_attr, *i); + + if (attr->dest_ft) { + err = esw_setup_ft_dest(dest, flow_act, esw, attr, spec, *i); + (*i)++; + } else if (attr->dest_chain) { + err = esw_setup_chain_dest(dest, flow_act, chains, attr->dest_chain, + 1, 0, *i); + (*i)++; + } } return err; @@ -1058,52 +1062,23 @@ void mlx5_eswitch_del_send_to_vport_rule(struct mlx5_flow_handle *rule) mlx5_del_flow_rules(rule); } -static void mlx5_eswitch_del_send_to_vport_meta_rules(struct mlx5_eswitch *esw) +void mlx5_eswitch_del_send_to_vport_meta_rule(struct mlx5_flow_handle *rule) { - struct mlx5_flow_handle **flows = esw->fdb_table.offloads.send_to_vport_meta_rules; - int i = 0, num_vfs = esw->esw_funcs.num_vfs; - - if (!num_vfs || !flows) - return; - - for (i = 0; i < num_vfs; i++) - mlx5_del_flow_rules(flows[i]); - - kvfree(flows); - /* If changing eswitch mode from switchdev to legacy, but num_vfs is not 0, - * meta rules could be freed again. So set it to NULL. - */ - esw->fdb_table.offloads.send_to_vport_meta_rules = NULL; + if (rule) + mlx5_del_flow_rules(rule); } -void esw_offloads_del_send_to_vport_meta_rules(struct mlx5_eswitch *esw) -{ - mlx5_eswitch_del_send_to_vport_meta_rules(esw); -} - -static int -mlx5_eswitch_add_send_to_vport_meta_rules(struct mlx5_eswitch *esw) +struct mlx5_flow_handle * +mlx5_eswitch_add_send_to_vport_meta_rule(struct mlx5_eswitch *esw, u16 vport_num) { struct mlx5_flow_destination dest = {}; struct mlx5_flow_act flow_act = {0}; - int num_vfs, rule_idx = 0, err = 0; struct mlx5_flow_handle *flow_rule; - struct mlx5_flow_handle **flows; struct mlx5_flow_spec *spec; - struct mlx5_vport *vport; - unsigned long i; - u16 vport_num; - - num_vfs = esw->esw_funcs.num_vfs; - flows = kvcalloc(num_vfs, sizeof(*flows), GFP_KERNEL); - if (!flows) - return -ENOMEM; spec = kvzalloc(sizeof(*spec), GFP_KERNEL); - if (!spec) { - err = -ENOMEM; - goto alloc_err; - } + if (!spec) + return ERR_PTR(-ENOMEM); MLX5_SET(fte_match_param, spec->match_criteria, misc_parameters_2.metadata_reg_c_0, mlx5_eswitch_get_vport_metadata_mask()); @@ -1116,34 +1091,18 @@ mlx5_eswitch_add_send_to_vport_meta_rules(struct mlx5_eswitch *esw) dest.type = MLX5_FLOW_DESTINATION_TYPE_VPORT; flow_act.action = MLX5_FLOW_CONTEXT_ACTION_FWD_DEST; - mlx5_esw_for_each_vf_vport(esw, i, vport, num_vfs) { - vport_num = vport->vport; - MLX5_SET(fte_match_param, spec->match_value, misc_parameters_2.metadata_reg_c_0, - mlx5_eswitch_get_vport_metadata_for_match(esw, vport_num)); - dest.vport.num = vport_num; - - flow_rule = mlx5_add_flow_rules(esw->fdb_table.offloads.slow_fdb, - spec, &flow_act, &dest, 1); - if (IS_ERR(flow_rule)) { - err = PTR_ERR(flow_rule); - esw_warn(esw->dev, "FDB: Failed to add send to vport meta rule idx %d, err %ld\n", - rule_idx, PTR_ERR(flow_rule)); - goto rule_err; - } - flows[rule_idx++] = flow_rule; - } + MLX5_SET(fte_match_param, spec->match_value, misc_parameters_2.metadata_reg_c_0, + mlx5_eswitch_get_vport_metadata_for_match(esw, vport_num)); + dest.vport.num = vport_num; - esw->fdb_table.offloads.send_to_vport_meta_rules = flows; - kvfree(spec); - return 0; + flow_rule = mlx5_add_flow_rules(esw->fdb_table.offloads.slow_fdb, + spec, &flow_act, &dest, 1); + if (IS_ERR(flow_rule)) + esw_warn(esw->dev, "FDB: Failed to add send to vport meta rule vport %d, err %ld\n", + vport_num, PTR_ERR(flow_rule)); -rule_err: - while (--rule_idx >= 0) - mlx5_del_flow_rules(flows[rule_idx]); kvfree(spec); -alloc_err: - kvfree(flows); - return err; + return flow_rule; } static bool mlx5_eswitch_reg_c1_loopback_supported(struct mlx5_eswitch *esw) @@ -1668,18 +1627,200 @@ esw_chains_destroy(struct mlx5_eswitch *esw, struct mlx5_fs_chains *chains) #endif +static int +esw_create_send_to_vport_group(struct mlx5_eswitch *esw, + struct mlx5_flow_table *fdb, + u32 *flow_group_in, + int *ix) +{ + int inlen = MLX5_ST_SZ_BYTES(create_flow_group_in); + struct mlx5_flow_group *g; + void *match_criteria; + int count, err = 0; + + memset(flow_group_in, 0, inlen); + + MLX5_SET(create_flow_group_in, flow_group_in, match_criteria_enable, + MLX5_MATCH_MISC_PARAMETERS); + + match_criteria = MLX5_ADDR_OF(create_flow_group_in, flow_group_in, match_criteria); + + MLX5_SET_TO_ONES(fte_match_param, match_criteria, misc_parameters.source_sqn); + MLX5_SET_TO_ONES(fte_match_param, match_criteria, misc_parameters.source_port); + if (MLX5_CAP_ESW(esw->dev, merged_eswitch)) { + MLX5_SET_TO_ONES(fte_match_param, match_criteria, + misc_parameters.source_eswitch_owner_vhca_id); + MLX5_SET(create_flow_group_in, flow_group_in, + source_eswitch_owner_vhca_id_valid, 1); + } + + /* See comment at table_size calculation */ + count = MLX5_MAX_PORTS * (esw->total_vports * MAX_SQ_NVPORTS + MAX_PF_SQ); + MLX5_SET(create_flow_group_in, flow_group_in, start_flow_index, 0); + MLX5_SET(create_flow_group_in, flow_group_in, end_flow_index, *ix + count - 1); + *ix += count; + + g = mlx5_create_flow_group(fdb, flow_group_in); + if (IS_ERR(g)) { + err = PTR_ERR(g); + esw_warn(esw->dev, "Failed to create send-to-vport flow group err(%d)\n", err); + goto out; + } + esw->fdb_table.offloads.send_to_vport_grp = g; + +out: + return err; +} + +static int +esw_create_meta_send_to_vport_group(struct mlx5_eswitch *esw, + struct mlx5_flow_table *fdb, + u32 *flow_group_in, + int *ix) +{ + int inlen = MLX5_ST_SZ_BYTES(create_flow_group_in); + struct mlx5_flow_group *g; + void *match_criteria; + int err = 0; + + if (!esw_src_port_rewrite_supported(esw)) + return 0; + + memset(flow_group_in, 0, inlen); + + MLX5_SET(create_flow_group_in, flow_group_in, match_criteria_enable, + MLX5_MATCH_MISC_PARAMETERS_2); + + match_criteria = MLX5_ADDR_OF(create_flow_group_in, flow_group_in, match_criteria); + + MLX5_SET(fte_match_param, match_criteria, + misc_parameters_2.metadata_reg_c_0, + mlx5_eswitch_get_vport_metadata_mask()); + MLX5_SET(fte_match_param, match_criteria, + misc_parameters_2.metadata_reg_c_1, ESW_TUN_MASK); + + MLX5_SET(create_flow_group_in, flow_group_in, start_flow_index, *ix); + MLX5_SET(create_flow_group_in, flow_group_in, + end_flow_index, *ix + esw->total_vports - 1); + *ix += esw->total_vports; + + g = mlx5_create_flow_group(fdb, flow_group_in); + if (IS_ERR(g)) { + err = PTR_ERR(g); + esw_warn(esw->dev, + "Failed to create send-to-vport meta flow group err(%d)\n", err); + goto send_vport_meta_err; + } + esw->fdb_table.offloads.send_to_vport_meta_grp = g; + + return 0; + +send_vport_meta_err: + return err; +} + +static int +esw_create_peer_esw_miss_group(struct mlx5_eswitch *esw, + struct mlx5_flow_table *fdb, + u32 *flow_group_in, + int *ix) +{ + int inlen = MLX5_ST_SZ_BYTES(create_flow_group_in); + struct mlx5_flow_group *g; + void *match_criteria; + int err = 0; + + if (!MLX5_CAP_ESW(esw->dev, merged_eswitch)) + return 0; + + memset(flow_group_in, 0, inlen); + + esw_set_flow_group_source_port(esw, flow_group_in); + + if (!mlx5_eswitch_vport_match_metadata_enabled(esw)) { + match_criteria = MLX5_ADDR_OF(create_flow_group_in, + flow_group_in, + match_criteria); + + MLX5_SET_TO_ONES(fte_match_param, match_criteria, + misc_parameters.source_eswitch_owner_vhca_id); + + MLX5_SET(create_flow_group_in, flow_group_in, + source_eswitch_owner_vhca_id_valid, 1); + } + + MLX5_SET(create_flow_group_in, flow_group_in, start_flow_index, *ix); + MLX5_SET(create_flow_group_in, flow_group_in, end_flow_index, + *ix + esw->total_vports - 1); + *ix += esw->total_vports; + + g = mlx5_create_flow_group(fdb, flow_group_in); + if (IS_ERR(g)) { + err = PTR_ERR(g); + esw_warn(esw->dev, "Failed to create peer miss flow group err(%d)\n", err); + goto out; + } + esw->fdb_table.offloads.peer_miss_grp = g; + +out: + return err; +} + +static int +esw_create_miss_group(struct mlx5_eswitch *esw, + struct mlx5_flow_table *fdb, + u32 *flow_group_in, + int *ix) +{ + int inlen = MLX5_ST_SZ_BYTES(create_flow_group_in); + struct mlx5_flow_group *g; + void *match_criteria; + int err = 0; + u8 *dmac; + + memset(flow_group_in, 0, inlen); + + MLX5_SET(create_flow_group_in, flow_group_in, match_criteria_enable, + MLX5_MATCH_OUTER_HEADERS); + match_criteria = MLX5_ADDR_OF(create_flow_group_in, flow_group_in, + match_criteria); + dmac = MLX5_ADDR_OF(fte_match_param, match_criteria, + outer_headers.dmac_47_16); + dmac[0] = 0x01; + + MLX5_SET(create_flow_group_in, flow_group_in, start_flow_index, *ix); + MLX5_SET(create_flow_group_in, flow_group_in, end_flow_index, + *ix + MLX5_ESW_MISS_FLOWS); + + g = mlx5_create_flow_group(fdb, flow_group_in); + if (IS_ERR(g)) { + err = PTR_ERR(g); + esw_warn(esw->dev, "Failed to create miss flow group err(%d)\n", err); + goto miss_err; + } + esw->fdb_table.offloads.miss_grp = g; + + err = esw_add_fdb_miss_rule(esw); + if (err) + goto miss_rule_err; + + return 0; + +miss_rule_err: + mlx5_destroy_flow_group(esw->fdb_table.offloads.miss_grp); +miss_err: + return err; +} + static int esw_create_offloads_fdb_tables(struct mlx5_eswitch *esw) { int inlen = MLX5_ST_SZ_BYTES(create_flow_group_in); struct mlx5_flow_table_attr ft_attr = {}; - int num_vfs, table_size, ix, err = 0; struct mlx5_core_dev *dev = esw->dev; struct mlx5_flow_namespace *root_ns; struct mlx5_flow_table *fdb = NULL; + int table_size, ix = 0, err = 0; u32 flags = 0, *flow_group_in; - struct mlx5_flow_group *g; - void *match_criteria; - u8 *dmac; esw_debug(esw->dev, "Create offloads FDB Tables\n"); @@ -1713,7 +1854,7 @@ static int esw_create_offloads_fdb_tables(struct mlx5_eswitch *esw) * total vports of the peer (currently is also uses esw->total_vports). */ table_size = MLX5_MAX_PORTS * (esw->total_vports * MAX_SQ_NVPORTS + MAX_PF_SQ) + - MLX5_ESW_MISS_FLOWS + esw->total_vports + esw->esw_funcs.num_vfs; + esw->total_vports * 2 + MLX5_ESW_MISS_FLOWS; /* create the slow path fdb with encap set, so further table instances * can be created at run time while VFs are probed if the FW allows that. @@ -1754,139 +1895,29 @@ static int esw_create_offloads_fdb_tables(struct mlx5_eswitch *esw) goto fdb_chains_err; } - /* create send-to-vport group */ - MLX5_SET(create_flow_group_in, flow_group_in, match_criteria_enable, - MLX5_MATCH_MISC_PARAMETERS); - - match_criteria = MLX5_ADDR_OF(create_flow_group_in, flow_group_in, match_criteria); - - MLX5_SET_TO_ONES(fte_match_param, match_criteria, misc_parameters.source_sqn); - MLX5_SET_TO_ONES(fte_match_param, match_criteria, misc_parameters.source_port); - if (MLX5_CAP_ESW(esw->dev, merged_eswitch)) { - MLX5_SET_TO_ONES(fte_match_param, match_criteria, - misc_parameters.source_eswitch_owner_vhca_id); - MLX5_SET(create_flow_group_in, flow_group_in, - source_eswitch_owner_vhca_id_valid, 1); - } - - /* See comment above table_size calculation */ - ix = MLX5_MAX_PORTS * (esw->total_vports * MAX_SQ_NVPORTS + MAX_PF_SQ); - MLX5_SET(create_flow_group_in, flow_group_in, start_flow_index, 0); - MLX5_SET(create_flow_group_in, flow_group_in, end_flow_index, ix - 1); - - g = mlx5_create_flow_group(fdb, flow_group_in); - if (IS_ERR(g)) { - err = PTR_ERR(g); - esw_warn(dev, "Failed to create send-to-vport flow group err(%d)\n", err); + err = esw_create_send_to_vport_group(esw, fdb, flow_group_in, &ix); + if (err) goto send_vport_err; - } - esw->fdb_table.offloads.send_to_vport_grp = g; - - if (esw_src_port_rewrite_supported(esw)) { - /* meta send to vport */ - memset(flow_group_in, 0, inlen); - MLX5_SET(create_flow_group_in, flow_group_in, match_criteria_enable, - MLX5_MATCH_MISC_PARAMETERS_2); - - match_criteria = MLX5_ADDR_OF(create_flow_group_in, flow_group_in, match_criteria); - - MLX5_SET(fte_match_param, match_criteria, - misc_parameters_2.metadata_reg_c_0, - mlx5_eswitch_get_vport_metadata_mask()); - MLX5_SET(fte_match_param, match_criteria, - misc_parameters_2.metadata_reg_c_1, ESW_TUN_MASK); - - num_vfs = esw->esw_funcs.num_vfs; - if (num_vfs) { - MLX5_SET(create_flow_group_in, flow_group_in, start_flow_index, ix); - MLX5_SET(create_flow_group_in, flow_group_in, - end_flow_index, ix + num_vfs - 1); - ix += num_vfs; - - g = mlx5_create_flow_group(fdb, flow_group_in); - if (IS_ERR(g)) { - err = PTR_ERR(g); - esw_warn(dev, "Failed to create send-to-vport meta flow group err(%d)\n", - err); - goto send_vport_meta_err; - } - esw->fdb_table.offloads.send_to_vport_meta_grp = g; - - err = mlx5_eswitch_add_send_to_vport_meta_rules(esw); - if (err) - goto meta_rule_err; - } - } - - if (MLX5_CAP_ESW(esw->dev, merged_eswitch)) { - /* create peer esw miss group */ - memset(flow_group_in, 0, inlen); - - esw_set_flow_group_source_port(esw, flow_group_in); - - if (!mlx5_eswitch_vport_match_metadata_enabled(esw)) { - match_criteria = MLX5_ADDR_OF(create_flow_group_in, - flow_group_in, - match_criteria); - - MLX5_SET_TO_ONES(fte_match_param, match_criteria, - misc_parameters.source_eswitch_owner_vhca_id); - - MLX5_SET(create_flow_group_in, flow_group_in, - source_eswitch_owner_vhca_id_valid, 1); - } - - MLX5_SET(create_flow_group_in, flow_group_in, start_flow_index, ix); - MLX5_SET(create_flow_group_in, flow_group_in, end_flow_index, - ix + esw->total_vports - 1); - ix += esw->total_vports; - - g = mlx5_create_flow_group(fdb, flow_group_in); - if (IS_ERR(g)) { - err = PTR_ERR(g); - esw_warn(dev, "Failed to create peer miss flow group err(%d)\n", err); - goto peer_miss_err; - } - esw->fdb_table.offloads.peer_miss_grp = g; - } - /* create miss group */ - memset(flow_group_in, 0, inlen); - MLX5_SET(create_flow_group_in, flow_group_in, match_criteria_enable, - MLX5_MATCH_OUTER_HEADERS); - match_criteria = MLX5_ADDR_OF(create_flow_group_in, flow_group_in, - match_criteria); - dmac = MLX5_ADDR_OF(fte_match_param, match_criteria, - outer_headers.dmac_47_16); - dmac[0] = 0x01; - - MLX5_SET(create_flow_group_in, flow_group_in, start_flow_index, ix); - MLX5_SET(create_flow_group_in, flow_group_in, end_flow_index, - ix + MLX5_ESW_MISS_FLOWS); + err = esw_create_meta_send_to_vport_group(esw, fdb, flow_group_in, &ix); + if (err) + goto send_vport_meta_err; - g = mlx5_create_flow_group(fdb, flow_group_in); - if (IS_ERR(g)) { - err = PTR_ERR(g); - esw_warn(dev, "Failed to create miss flow group err(%d)\n", err); - goto miss_err; - } - esw->fdb_table.offloads.miss_grp = g; + err = esw_create_peer_esw_miss_group(esw, fdb, flow_group_in, &ix); + if (err) + goto peer_miss_err; - err = esw_add_fdb_miss_rule(esw); + err = esw_create_miss_group(esw, fdb, flow_group_in, &ix); if (err) - goto miss_rule_err; + goto miss_err; kvfree(flow_group_in); return 0; -miss_rule_err: - mlx5_destroy_flow_group(esw->fdb_table.offloads.miss_grp); miss_err: if (MLX5_CAP_ESW(esw->dev, merged_eswitch)) mlx5_destroy_flow_group(esw->fdb_table.offloads.peer_miss_grp); peer_miss_err: - mlx5_eswitch_del_send_to_vport_meta_rules(esw); -meta_rule_err: if (esw->fdb_table.offloads.send_to_vport_meta_grp) mlx5_destroy_flow_group(esw->fdb_table.offloads.send_to_vport_meta_grp); send_vport_meta_err: @@ -1913,7 +1944,6 @@ static void esw_destroy_offloads_fdb_tables(struct mlx5_eswitch *esw) esw_debug(esw->dev, "Destroy offloads FDB Tables\n"); mlx5_del_flow_rules(esw->fdb_table.offloads.miss_rule_multi); mlx5_del_flow_rules(esw->fdb_table.offloads.miss_rule_uni); - mlx5_eswitch_del_send_to_vport_meta_rules(esw); mlx5_destroy_flow_group(esw->fdb_table.offloads.send_to_vport_grp); if (esw->fdb_table.offloads.send_to_vport_meta_grp) mlx5_destroy_flow_group(esw->fdb_table.offloads.send_to_vport_meta_grp); @@ -1931,7 +1961,7 @@ static void esw_destroy_offloads_fdb_tables(struct mlx5_eswitch *esw) atomic64_set(&esw->user_count, 0); } -static int esw_get_offloads_ft_size(struct mlx5_eswitch *esw) +static int esw_get_nr_ft_offloads_steering_src_ports(struct mlx5_eswitch *esw) { int nvports; @@ -1956,7 +1986,8 @@ static int esw_create_offloads_table(struct mlx5_eswitch *esw) return -EOPNOTSUPP; } - ft_attr.max_fte = esw_get_offloads_ft_size(esw); + ft_attr.max_fte = esw_get_nr_ft_offloads_steering_src_ports(esw) + + MLX5_ESW_FT_OFFLOADS_DROP_RULE; ft_attr.prio = 1; ft_offloads = mlx5_create_flow_table(ns, &ft_attr); @@ -1985,7 +2016,7 @@ static int esw_create_vport_rx_group(struct mlx5_eswitch *esw) int nvports; int err = 0; - nvports = esw_get_offloads_ft_size(esw); + nvports = esw_get_nr_ft_offloads_steering_src_ports(esw); flow_group_in = kvzalloc(inlen, GFP_KERNEL); if (!flow_group_in) return -ENOMEM; @@ -2015,6 +2046,52 @@ static void esw_destroy_vport_rx_group(struct mlx5_eswitch *esw) mlx5_destroy_flow_group(esw->offloads.vport_rx_group); } +static int esw_create_vport_rx_drop_rule_index(struct mlx5_eswitch *esw) +{ + /* ft_offloads table is enlarged by MLX5_ESW_FT_OFFLOADS_DROP_RULE (1) + * for the drop rule, which is placed at the end of the table. + * So return the total of vport and int_port as rule index. + */ + return esw_get_nr_ft_offloads_steering_src_ports(esw); +} + +static int esw_create_vport_rx_drop_group(struct mlx5_eswitch *esw) +{ + int inlen = MLX5_ST_SZ_BYTES(create_flow_group_in); + struct mlx5_flow_group *g; + u32 *flow_group_in; + int flow_index; + int err = 0; + + flow_index = esw_create_vport_rx_drop_rule_index(esw); + + flow_group_in = kvzalloc(inlen, GFP_KERNEL); + if (!flow_group_in) + return -ENOMEM; + + MLX5_SET(create_flow_group_in, flow_group_in, start_flow_index, flow_index); + MLX5_SET(create_flow_group_in, flow_group_in, end_flow_index, flow_index); + + g = mlx5_create_flow_group(esw->offloads.ft_offloads, flow_group_in); + + if (IS_ERR(g)) { + err = PTR_ERR(g); + mlx5_core_warn(esw->dev, "Failed to create vport rx drop group err %d\n", err); + goto out; + } + + esw->offloads.vport_rx_drop_group = g; +out: + kvfree(flow_group_in); + return err; +} + +static void esw_destroy_vport_rx_drop_group(struct mlx5_eswitch *esw) +{ + if (esw->offloads.vport_rx_drop_group) + mlx5_destroy_flow_group(esw->offloads.vport_rx_drop_group); +} + struct mlx5_flow_handle * mlx5_eswitch_create_vport_rx_rule(struct mlx5_eswitch *esw, u16 vport, struct mlx5_flow_destination *dest) @@ -2063,6 +2140,32 @@ out: return flow_rule; } +static int esw_create_vport_rx_drop_rule(struct mlx5_eswitch *esw) +{ + struct mlx5_flow_act flow_act = {}; + struct mlx5_flow_handle *flow_rule; + + flow_act.action = MLX5_FLOW_CONTEXT_ACTION_DROP; + flow_rule = mlx5_add_flow_rules(esw->offloads.ft_offloads, NULL, + &flow_act, NULL, 0); + if (IS_ERR(flow_rule)) { + esw_warn(esw->dev, + "fs offloads: Failed to add vport rx drop rule err %ld\n", + PTR_ERR(flow_rule)); + return PTR_ERR(flow_rule); + } + + esw->offloads.vport_rx_drop_rule = flow_rule; + + return 0; +} + +static void esw_destroy_vport_rx_drop_rule(struct mlx5_eswitch *esw) +{ + if (esw->offloads.vport_rx_drop_rule) + mlx5_del_flow_rules(esw->offloads.vport_rx_drop_rule); +} + static int mlx5_eswitch_inline_mode_get(struct mlx5_eswitch *esw, u8 *mode) { u8 prev_mlx5_mode, mlx5_mode = MLX5_INLINE_MODE_L2; @@ -3063,8 +3166,20 @@ static int esw_offloads_steering_init(struct mlx5_eswitch *esw) if (err) goto create_fg_err; + err = esw_create_vport_rx_drop_group(esw); + if (err) + goto create_rx_drop_fg_err; + + err = esw_create_vport_rx_drop_rule(esw); + if (err) + goto create_rx_drop_rule_err; + return 0; +create_rx_drop_rule_err: + esw_destroy_vport_rx_drop_group(esw); +create_rx_drop_fg_err: + esw_destroy_vport_rx_group(esw); create_fg_err: esw_destroy_offloads_fdb_tables(esw); create_fdb_err: @@ -3082,6 +3197,8 @@ create_indir_err: static void esw_offloads_steering_cleanup(struct mlx5_eswitch *esw) { + esw_destroy_vport_rx_drop_rule(esw); + esw_destroy_vport_rx_drop_group(esw); esw_destroy_vport_rx_group(esw); esw_destroy_offloads_fdb_tables(esw); esw_destroy_restore_table(esw); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/events.c b/drivers/net/ethernet/mellanox/mlx5/core/events.c index a1ac3a654962..9459e56ee90a 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/events.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/events.c @@ -36,6 +36,7 @@ static struct mlx5_nb events_nbs_ref[] = { /* Events to be forwarded (as is) to mlx5 core interfaces (mlx5e/mlx5_ib) */ {.nb.notifier_call = forward_event, .event_type = MLX5_EVENT_TYPE_PORT_CHANGE }, {.nb.notifier_call = forward_event, .event_type = MLX5_EVENT_TYPE_GENERAL_EVENT }, + {.nb.notifier_call = forward_event, .event_type = MLX5_EVENT_TYPE_OBJECT_CHANGE }, /* QP/WQ resource events to forward */ {.nb.notifier_call = forward_event, .event_type = MLX5_EVENT_TYPE_DCT_DRAINED }, {.nb.notifier_call = forward_event, .event_type = MLX5_EVENT_TYPE_PATH_MIG }, @@ -132,6 +133,8 @@ static const char *eqe_type_str(u8 type) return "MLX5_EVENT_TYPE_MONITOR_COUNTER"; case MLX5_EVENT_TYPE_DEVICE_TRACER: return "MLX5_EVENT_TYPE_DEVICE_TRACER"; + case MLX5_EVENT_TYPE_OBJECT_CHANGE: + return "MLX5_EVENT_TYPE_OBJECT_CHANGE"; default: return "Unrecognized event"; } diff --git a/drivers/net/ethernet/mellanox/mlx5/core/fs_cmd.c b/drivers/net/ethernet/mellanox/mlx5/core/fs_cmd.c index e735e19461ba..32d4c967469c 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/fs_cmd.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/fs_cmd.c @@ -577,7 +577,10 @@ static int mlx5_cmd_set_fte(struct mlx5_core_dev *dev, MLX5_SET(flow_context, in_flow_context, modify_header_id, fte->action.modify_hdr->id); - MLX5_SET(flow_context, in_flow_context, ipsec_obj_id, fte->action.ipsec_obj_id); + MLX5_SET(flow_context, in_flow_context, encrypt_decrypt_type, + fte->action.crypto.type); + MLX5_SET(flow_context, in_flow_context, encrypt_decrypt_obj_id, + fte->action.crypto.obj_id); vlan = MLX5_ADDR_OF(flow_context, in_flow_context, push_vlan); @@ -919,13 +922,15 @@ static int mlx5_cmd_modify_header_alloc(struct mlx5_flow_root_namespace *ns, max_actions = MLX5_CAP_ESW_FLOWTABLE_FDB(dev, max_modify_header_actions); table_type = FS_FT_FDB; break; + case MLX5_FLOW_NAMESPACE_KERNEL_RX_MACSEC: case MLX5_FLOW_NAMESPACE_KERNEL: case MLX5_FLOW_NAMESPACE_BYPASS: max_actions = MLX5_CAP_FLOWTABLE_NIC_RX(dev, max_modify_header_actions); table_type = FS_FT_NIC_RX; break; case MLX5_FLOW_NAMESPACE_EGRESS: - case MLX5_FLOW_NAMESPACE_EGRESS_KERNEL: + case MLX5_FLOW_NAMESPACE_EGRESS_IPSEC: + case MLX5_FLOW_NAMESPACE_EGRESS_MACSEC: max_actions = MLX5_CAP_FLOWTABLE_NIC_TX(dev, max_modify_header_actions); table_type = FS_FT_NIC_TX; break; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/fs_core.c b/drivers/net/ethernet/mellanox/mlx5/core/fs_core.c index e3960cdf5131..d53749248fa0 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/fs_core.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/fs_core.c @@ -104,6 +104,10 @@ #define BY_PASS_MIN_LEVEL (ETHTOOL_MIN_LEVEL + MLX5_BY_PASS_NUM_PRIOS +\ LEFTOVERS_NUM_PRIOS) +#define KERNEL_RX_MACSEC_NUM_PRIOS 1 +#define KERNEL_RX_MACSEC_NUM_LEVELS 2 +#define KERNEL_RX_MACSEC_MIN_LEVEL (BY_PASS_MIN_LEVEL + KERNEL_RX_MACSEC_NUM_PRIOS) + #define ETHTOOL_PRIO_NUM_LEVELS 1 #define ETHTOOL_NUM_PRIOS 11 #define ETHTOOL_MIN_LEVEL (KERNEL_MIN_LEVEL + ETHTOOL_NUM_PRIOS) @@ -126,11 +130,15 @@ #define LAG_PRIO_NUM_LEVELS 1 #define LAG_NUM_PRIOS 1 -#define LAG_MIN_LEVEL (OFFLOADS_MIN_LEVEL + 1) +#define LAG_MIN_LEVEL (OFFLOADS_MIN_LEVEL + KERNEL_RX_MACSEC_MIN_LEVEL + 1) #define KERNEL_TX_IPSEC_NUM_PRIOS 1 #define KERNEL_TX_IPSEC_NUM_LEVELS 1 -#define KERNEL_TX_MIN_LEVEL (KERNEL_TX_IPSEC_NUM_LEVELS) +#define KERNEL_TX_IPSEC_MIN_LEVEL (KERNEL_TX_IPSEC_NUM_LEVELS) + +#define KERNEL_TX_MACSEC_NUM_PRIOS 1 +#define KERNEL_TX_MACSEC_NUM_LEVELS 2 +#define KERNEL_TX_MACSEC_MIN_LEVEL (KERNEL_TX_IPSEC_MIN_LEVEL + KERNEL_TX_MACSEC_NUM_PRIOS) struct node_caps { size_t arr_sz; @@ -149,12 +157,16 @@ static struct init_tree_node { enum mlx5_flow_table_miss_action def_miss_action; } root_fs = { .type = FS_TYPE_NAMESPACE, - .ar_size = 7, + .ar_size = 8, .children = (struct init_tree_node[]){ ADD_PRIO(0, BY_PASS_MIN_LEVEL, 0, FS_CHAINING_CAPS, ADD_NS(MLX5_FLOW_TABLE_MISS_ACTION_DEF, ADD_MULTIPLE_PRIO(MLX5_BY_PASS_NUM_PRIOS, BY_PASS_PRIO_NUM_LEVELS))), + ADD_PRIO(0, KERNEL_RX_MACSEC_MIN_LEVEL, 0, FS_CHAINING_CAPS, + ADD_NS(MLX5_FLOW_TABLE_MISS_ACTION_DEF, + ADD_MULTIPLE_PRIO(KERNEL_RX_MACSEC_NUM_PRIOS, + KERNEL_RX_MACSEC_NUM_LEVELS))), ADD_PRIO(0, LAG_MIN_LEVEL, 0, FS_CHAINING_CAPS, ADD_NS(MLX5_FLOW_TABLE_MISS_ACTION_DEF, ADD_MULTIPLE_PRIO(LAG_NUM_PRIOS, @@ -186,18 +198,23 @@ static struct init_tree_node { static struct init_tree_node egress_root_fs = { .type = FS_TYPE_NAMESPACE, - .ar_size = 2, + .ar_size = 3, .children = (struct init_tree_node[]) { ADD_PRIO(0, MLX5_BY_PASS_NUM_PRIOS, 0, FS_CHAINING_CAPS_EGRESS, ADD_NS(MLX5_FLOW_TABLE_MISS_ACTION_DEF, ADD_MULTIPLE_PRIO(MLX5_BY_PASS_NUM_PRIOS, BY_PASS_PRIO_NUM_LEVELS))), - ADD_PRIO(0, KERNEL_TX_MIN_LEVEL, 0, + ADD_PRIO(0, KERNEL_TX_IPSEC_MIN_LEVEL, 0, FS_CHAINING_CAPS_EGRESS, ADD_NS(MLX5_FLOW_TABLE_MISS_ACTION_DEF, ADD_MULTIPLE_PRIO(KERNEL_TX_IPSEC_NUM_PRIOS, KERNEL_TX_IPSEC_NUM_LEVELS))), + ADD_PRIO(0, KERNEL_TX_MACSEC_MIN_LEVEL, 0, + FS_CHAINING_CAPS_EGRESS, + ADD_NS(MLX5_FLOW_TABLE_MISS_ACTION_DEF, + ADD_MULTIPLE_PRIO(KERNEL_TX_MACSEC_NUM_PRIOS, + KERNEL_TX_MACSEC_NUM_LEVELS))), } }; @@ -2269,6 +2286,7 @@ static bool is_nic_rx_ns(enum mlx5_flow_namespace_type type) { switch (type) { case MLX5_FLOW_NAMESPACE_BYPASS: + case MLX5_FLOW_NAMESPACE_KERNEL_RX_MACSEC: case MLX5_FLOW_NAMESPACE_LAG: case MLX5_FLOW_NAMESPACE_OFFLOADS: case MLX5_FLOW_NAMESPACE_ETHTOOL: @@ -2315,7 +2333,8 @@ struct mlx5_flow_namespace *mlx5_get_flow_namespace(struct mlx5_core_dev *dev, prio = FDB_BYPASS_PATH; break; case MLX5_FLOW_NAMESPACE_EGRESS: - case MLX5_FLOW_NAMESPACE_EGRESS_KERNEL: + case MLX5_FLOW_NAMESPACE_EGRESS_IPSEC: + case MLX5_FLOW_NAMESPACE_EGRESS_MACSEC: root_ns = steering->egress_root_ns; prio = type - MLX5_FLOW_NAMESPACE_EGRESS; break; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/fw.c b/drivers/net/ethernet/mellanox/mlx5/core/fw.c index 079fa44ada71..f34e758a2f1f 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/fw.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/fw.c @@ -273,6 +273,19 @@ int mlx5_query_hca_caps(struct mlx5_core_dev *dev) return err; } + if (MLX5_CAP_GEN_64(dev, general_obj_types) & + MLX5_GENERAL_OBJ_TYPES_CAP_MACSEC_OFFLOAD) { + err = mlx5_core_get_caps(dev, MLX5_CAP_MACSEC); + if (err) + return err; + } + + if (MLX5_CAP_GEN(dev, adv_virtualization)) { + err = mlx5_core_get_caps(dev, MLX5_CAP_ADV_VIRTUALIZATION); + if (err) + return err; + } + return 0; } diff --git a/drivers/net/ethernet/mellanox/mlx5/core/health.c b/drivers/net/ethernet/mellanox/mlx5/core/health.c index 2cf2c9948446..86ed87d704f7 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/health.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/health.c @@ -601,7 +601,7 @@ static void mlx5_fw_reporter_err_work(struct work_struct *work) fw_reporter_ctx.miss_counter = health->miss_counter; if (fw_reporter_ctx.err_synd) { devlink_health_report(health->fw_reporter, - "FW syndrom reported", &fw_reporter_ctx); + "FW syndrome reported", &fw_reporter_ctx); return; } if (fw_reporter_ctx.miss_counter) @@ -702,11 +702,25 @@ static const struct devlink_health_reporter_ops mlx5_fw_fatal_reporter_ops = { .dump = mlx5_fw_fatal_reporter_dump, }; -#define MLX5_REPORTER_FW_GRACEFUL_PERIOD 1200000 +#define MLX5_FW_REPORTER_ECPF_GRACEFUL_PERIOD 180000 +#define MLX5_FW_REPORTER_PF_GRACEFUL_PERIOD 60000 +#define MLX5_FW_REPORTER_VF_GRACEFUL_PERIOD 30000 +#define MLX5_FW_REPORTER_DEFAULT_GRACEFUL_PERIOD MLX5_FW_REPORTER_VF_GRACEFUL_PERIOD + static void mlx5_fw_reporters_create(struct mlx5_core_dev *dev) { struct mlx5_core_health *health = &dev->priv.health; struct devlink *devlink = priv_to_devlink(dev); + u64 grace_period; + + if (mlx5_core_is_ecpf(dev)) { + grace_period = MLX5_FW_REPORTER_ECPF_GRACEFUL_PERIOD; + } else if (mlx5_core_is_pf(dev)) { + grace_period = MLX5_FW_REPORTER_PF_GRACEFUL_PERIOD; + } else { + /* VF or SF */ + grace_period = MLX5_FW_REPORTER_DEFAULT_GRACEFUL_PERIOD; + } health->fw_reporter = devlink_health_reporter_create(devlink, &mlx5_fw_reporter_ops, @@ -718,7 +732,7 @@ static void mlx5_fw_reporters_create(struct mlx5_core_dev *dev) health->fw_fatal_reporter = devlink_health_reporter_create(devlink, &mlx5_fw_fatal_reporter_ops, - MLX5_REPORTER_FW_GRACEFUL_PERIOD, + grace_period, dev); if (IS_ERR(health->fw_fatal_reporter)) mlx5_core_warn(dev, "Failed to create fw fatal reporter, err = %ld\n", @@ -843,9 +857,6 @@ void mlx5_start_health_poll(struct mlx5_core_dev *dev) health->timer.expires = jiffies + msecs_to_jiffies(poll_interval_ms); add_timer(&health->timer); - - if (mlx5_core_is_pf(dev) && MLX5_CAP_MCAM_REG(dev, mrtc)) - queue_delayed_work(health->wq, &health->update_fw_log_ts_work, 0); } void mlx5_stop_health_poll(struct mlx5_core_dev *dev, bool disable_health) @@ -862,6 +873,14 @@ void mlx5_stop_health_poll(struct mlx5_core_dev *dev, bool disable_health) del_timer_sync(&health->timer); } +void mlx5_start_health_fw_log_up(struct mlx5_core_dev *dev) +{ + struct mlx5_core_health *health = &dev->priv.health; + + if (mlx5_core_is_pf(dev) && MLX5_CAP_MCAM_REG(dev, mrtc)) + queue_delayed_work(health->wq, &health->update_fw_log_ts_work, 0); +} + void mlx5_drain_health_wq(struct mlx5_core_dev *dev) { struct mlx5_core_health *health = &dev->priv.health; @@ -875,13 +894,6 @@ void mlx5_drain_health_wq(struct mlx5_core_dev *dev) cancel_work_sync(&health->fatal_report_work); } -void mlx5_health_flush(struct mlx5_core_dev *dev) -{ - struct mlx5_core_health *health = &dev->priv.health; - - flush_workqueue(health->wq); -} - void mlx5_health_cleanup(struct mlx5_core_dev *dev) { struct mlx5_core_health *health = &dev->priv.health; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/ipoib/ethtool.c b/drivers/net/ethernet/mellanox/mlx5/core/ipoib/ethtool.c index ac3757beaea2..c247cca154e9 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/ipoib/ethtool.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/ipoib/ethtool.c @@ -32,6 +32,7 @@ #include "en.h" #include "ipoib.h" +#include "en/fs_ethtool.h" static void mlx5i_get_drvinfo(struct net_device *dev, struct ethtool_drvinfo *drvinfo) @@ -39,7 +40,7 @@ static void mlx5i_get_drvinfo(struct net_device *dev, struct mlx5e_priv *priv = mlx5i_epriv(dev); mlx5e_ethtool_get_drvinfo(priv, drvinfo); - strlcpy(drvinfo->driver, KBUILD_MODNAME "[ib_ipoib]", + strscpy(drvinfo->driver, KBUILD_MODNAME "[ib_ipoib]", sizeof(drvinfo->driver)); } diff --git a/drivers/net/ethernet/mellanox/mlx5/core/ipoib/ipoib.c b/drivers/net/ethernet/mellanox/mlx5/core/ipoib/ipoib.c index c02b7b08fb4c..4e3a75496dd9 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/ipoib/ipoib.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/ipoib/ipoib.c @@ -35,6 +35,7 @@ #include "en.h" #include "en/params.h" #include "ipoib.h" +#include "en/fs_ethtool.h" #define IB_DEFAULT_Q_KEY 0xb1b #define MLX5I_PARAMS_DEFAULT_LOG_RQ_SIZE 9 @@ -320,43 +321,47 @@ static void mlx5i_cleanup_tx(struct mlx5e_priv *priv) static int mlx5i_create_flow_steering(struct mlx5e_priv *priv) { + struct mlx5_flow_namespace *ns = + mlx5_get_flow_namespace(priv->mdev, MLX5_FLOW_NAMESPACE_KERNEL); int err; - priv->fs->ns = mlx5_get_flow_namespace(priv->mdev, - MLX5_FLOW_NAMESPACE_KERNEL); - if (!priv->fs->ns) + if (!ns) return -EINVAL; - err = mlx5e_arfs_create_tables(priv); + mlx5e_fs_set_ns(priv->fs, ns, false); + err = mlx5e_arfs_create_tables(priv->fs, priv->rx_res, + !!(priv->netdev->hw_features & NETIF_F_NTUPLE)); if (err) { netdev_err(priv->netdev, "Failed to create arfs tables, err=%d\n", err); priv->netdev->hw_features &= ~NETIF_F_NTUPLE; } - err = mlx5e_create_ttc_table(priv); + err = mlx5e_create_ttc_table(priv->fs, priv->rx_res); if (err) { netdev_err(priv->netdev, "Failed to create ttc table, err=%d\n", err); goto err_destroy_arfs_tables; } - mlx5e_ethtool_init_steering(priv); + mlx5e_ethtool_init_steering(priv->fs); return 0; err_destroy_arfs_tables: - mlx5e_arfs_destroy_tables(priv); + mlx5e_arfs_destroy_tables(priv->fs, + !!(priv->netdev->hw_features & NETIF_F_NTUPLE)); return err; } static void mlx5i_destroy_flow_steering(struct mlx5e_priv *priv) { - mlx5e_destroy_ttc_table(priv); - mlx5e_arfs_destroy_tables(priv); - mlx5e_ethtool_cleanup_steering(priv); + mlx5e_destroy_ttc_table(priv->fs); + mlx5e_arfs_destroy_tables(priv->fs, + !!(priv->netdev->hw_features & NETIF_F_NTUPLE)); + mlx5e_ethtool_cleanup_steering(priv->fs); } static int mlx5i_init_rx(struct mlx5e_priv *priv) @@ -458,7 +463,6 @@ static const struct mlx5e_profile mlx5i_nic_profile = { .update_carrier = NULL, /* no HW update in IB link */ .rx_handlers = &mlx5i_rx_handlers, .max_tc = MLX5I_MAX_NUM_TC, - .rq_groups = MLX5E_NUM_RQ_GROUPS(REGULAR), .stats_grps = mlx5i_stats_grps, .stats_grps_num = mlx5i_stats_grps_num, }; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/ipoib/ipoib_vlan.c b/drivers/net/ethernet/mellanox/mlx5/core/ipoib/ipoib_vlan.c index 0b86e78dbc0e..0227a521d301 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/ipoib/ipoib_vlan.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/ipoib/ipoib_vlan.c @@ -349,7 +349,6 @@ static const struct mlx5e_profile mlx5i_pkey_nic_profile = { .update_stats = NULL, .rx_handlers = &mlx5i_rx_handlers, .max_tc = MLX5I_MAX_NUM_TC, - .rq_groups = MLX5E_NUM_RQ_GROUPS(REGULAR), }; const struct mlx5e_profile *mlx5i_pkey_get_profile(void) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c b/drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c index 065102278cb8..a9f4ede4a9bf 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c @@ -65,6 +65,21 @@ static int get_port_sel_mode(enum mlx5_lag_mode mode, unsigned long flags) return MLX5_LAG_PORT_SELECT_MODE_QUEUE_AFFINITY; } +static u8 lag_active_port_bits(struct mlx5_lag *ldev) +{ + u8 enabled_ports[MLX5_MAX_PORTS] = {}; + u8 active_port = 0; + int num_enabled; + int idx; + + mlx5_infer_tx_enabled(&ldev->tracker, ldev->ports, enabled_ports, + &num_enabled); + for (idx = 0; idx < num_enabled; idx++) + active_port |= BIT_MASK(enabled_ports[idx]); + + return active_port; +} + static int mlx5_cmd_create_lag(struct mlx5_core_dev *dev, u8 *ports, int mode, unsigned long flags) { @@ -77,9 +92,21 @@ static int mlx5_cmd_create_lag(struct mlx5_core_dev *dev, u8 *ports, int mode, lag_ctx = MLX5_ADDR_OF(create_lag_in, in, ctx); MLX5_SET(create_lag_in, in, opcode, MLX5_CMD_OP_CREATE_LAG); MLX5_SET(lagc, lag_ctx, fdb_selection_mode, fdb_sel_mode); - if (port_sel_mode == MLX5_LAG_PORT_SELECT_MODE_QUEUE_AFFINITY) { + + switch (port_sel_mode) { + case MLX5_LAG_PORT_SELECT_MODE_QUEUE_AFFINITY: MLX5_SET(lagc, lag_ctx, tx_remap_affinity_1, ports[0]); MLX5_SET(lagc, lag_ctx, tx_remap_affinity_2, ports[1]); + break; + case MLX5_LAG_PORT_SELECT_MODE_PORT_SELECT_FT: + if (!MLX5_CAP_PORT_SELECTION(dev, port_select_flow_table_bypass)) + break; + + MLX5_SET(lagc, lag_ctx, active_port, + lag_active_port_bits(mlx5_lag_dev(dev))); + break; + default: + break; } MLX5_SET(lagc, lag_ctx, port_select_mode, port_sel_mode); @@ -386,12 +413,37 @@ static void mlx5_lag_drop_rule_setup(struct mlx5_lag *ldev, } } +static int mlx5_cmd_modify_active_port(struct mlx5_core_dev *dev, u8 ports) +{ + u32 in[MLX5_ST_SZ_DW(modify_lag_in)] = {}; + void *lag_ctx; + + lag_ctx = MLX5_ADDR_OF(modify_lag_in, in, ctx); + + MLX5_SET(modify_lag_in, in, opcode, MLX5_CMD_OP_MODIFY_LAG); + MLX5_SET(modify_lag_in, in, field_select, 0x2); + + MLX5_SET(lagc, lag_ctx, active_port, ports); + + return mlx5_cmd_exec_in(dev, modify_lag, in); +} + static int _mlx5_modify_lag(struct mlx5_lag *ldev, u8 *ports) { struct mlx5_core_dev *dev0 = ldev->pf[MLX5_LAG_P1].dev; + u8 active_ports; + int ret; + + if (test_bit(MLX5_LAG_MODE_FLAG_HASH_BASED, &ldev->mode_flags)) { + ret = mlx5_lag_port_sel_modify(ldev, ports); + if (ret || + !MLX5_CAP_PORT_SELECTION(dev0, port_select_flow_table_bypass)) + return ret; - if (test_bit(MLX5_LAG_MODE_FLAG_HASH_BASED, &ldev->mode_flags)) - return mlx5_lag_port_sel_modify(ldev, ports); + active_ports = lag_active_port_bits(ldev); + + return mlx5_cmd_modify_active_port(dev0, active_ports); + } return mlx5_cmd_modify_lag(dev0, ldev->ports, ports); } @@ -432,21 +484,22 @@ void mlx5_modify_lag(struct mlx5_lag *ldev, mlx5_lag_drop_rule_setup(ldev, tracker); } -#define MLX5_LAG_ROCE_HASH_PORTS_SUPPORTED 4 static int mlx5_lag_set_port_sel_mode_roce(struct mlx5_lag *ldev, unsigned long *flags) { - struct lag_func *dev0 = &ldev->pf[MLX5_LAG_P1]; + struct mlx5_core_dev *dev0 = ldev->pf[MLX5_LAG_P1].dev; - if (ldev->ports == MLX5_LAG_ROCE_HASH_PORTS_SUPPORTED) { - /* Four ports are support only in hash mode */ - if (!MLX5_CAP_PORT_SELECTION(dev0->dev, port_select_flow_table)) - return -EINVAL; - set_bit(MLX5_LAG_MODE_FLAG_HASH_BASED, flags); + if (!MLX5_CAP_PORT_SELECTION(dev0, port_select_flow_table)) { if (ldev->ports > 2) - ldev->buckets = MLX5_LAG_MAX_HASH_BUCKETS; + return -EINVAL; + return 0; } + if (ldev->ports > 2) + ldev->buckets = MLX5_LAG_MAX_HASH_BUCKETS; + + set_bit(MLX5_LAG_MODE_FLAG_HASH_BASED, flags); + return 0; } @@ -1275,6 +1328,22 @@ bool mlx5_lag_is_active(struct mlx5_core_dev *dev) } EXPORT_SYMBOL(mlx5_lag_is_active); +bool mlx5_lag_mode_is_hash(struct mlx5_core_dev *dev) +{ + struct mlx5_lag *ldev; + unsigned long flags; + bool res = 0; + + spin_lock_irqsave(&lag_lock, flags); + ldev = mlx5_lag_dev(dev); + if (ldev) + res = test_bit(MLX5_LAG_MODE_FLAG_HASH_BASED, &ldev->mode_flags); + spin_unlock_irqrestore(&lag_lock, flags); + + return res; +} +EXPORT_SYMBOL(mlx5_lag_mode_is_hash); + bool mlx5_lag_is_master(struct mlx5_core_dev *dev) { struct mlx5_lag *ldev; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/lib/aso.h b/drivers/net/ethernet/mellanox/mlx5/core/lib/aso.h index b3bbf284fe71..d854e01d7fc5 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/lib/aso.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/lib/aso.h @@ -11,7 +11,9 @@ (DIV_ROUND_UP(sizeof(struct mlx5_aso_wqe), MLX5_SEND_WQE_BB)) #define MLX5_ASO_WQEBBS_DATA \ (DIV_ROUND_UP(sizeof(struct mlx5_aso_wqe_data), MLX5_SEND_WQE_BB)) +#define ASO_CTRL_READ_EN BIT(0) #define MLX5_WQE_CTRL_WQE_OPC_MOD_SHIFT 24 +#define MLX5_MACSEC_ASO_DS_CNT (DIV_ROUND_UP(sizeof(struct mlx5_aso_wqe), MLX5_SEND_WQE_DS)) struct mlx5_wqe_aso_ctrl_seg { __be32 va_h; @@ -70,6 +72,7 @@ enum { enum { MLX5_ACCESS_ASO_OPC_MOD_FLOW_METER = 0x2, + MLX5_ACCESS_ASO_OPC_MOD_MACSEC = 0x5, }; struct mlx5_aso; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/lib/clock.c b/drivers/net/ethernet/mellanox/mlx5/core/lib/clock.c index 91e806c1aa21..d3a9ae80fd30 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/lib/clock.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/lib/clock.c @@ -65,6 +65,8 @@ enum { MLX5_MTPPS_FS_TIME_STAMP = BIT(0x4), MLX5_MTPPS_FS_OUT_PULSE_DURATION = BIT(0x5), MLX5_MTPPS_FS_ENH_OUT_PER_ADJ = BIT(0x7), + MLX5_MTPPS_FS_NPPS_PERIOD = BIT(0x9), + MLX5_MTPPS_FS_OUT_PULSE_DURATION_NS = BIT(0xa), }; static bool mlx5_real_time_mode(struct mlx5_core_dev *mdev) @@ -72,6 +74,13 @@ static bool mlx5_real_time_mode(struct mlx5_core_dev *mdev) return (mlx5_is_real_time_rq(mdev) || mlx5_is_real_time_sq(mdev)); } +static bool mlx5_npps_real_time_supported(struct mlx5_core_dev *mdev) +{ + return (mlx5_real_time_mode(mdev) && + MLX5_CAP_MCAM_FEATURE(mdev, npps_period) && + MLX5_CAP_MCAM_FEATURE(mdev, out_pulse_duration_ns)); +} + static bool mlx5_modify_mtutc_allowed(struct mlx5_core_dev *mdev) { return MLX5_CAP_MCAM_FEATURE(mdev, ptpcyc2realtime_modify); @@ -459,9 +468,95 @@ static u64 perout_conf_internal_timer(struct mlx5_core_dev *mdev, s64 sec) return find_target_cycles(mdev, target_ns); } -static u64 perout_conf_real_time(s64 sec) +static u64 perout_conf_real_time(s64 sec, u32 nsec) +{ + return (u64)nsec | (u64)sec << 32; +} + +static int perout_conf_1pps(struct mlx5_core_dev *mdev, struct ptp_clock_request *rq, + u64 *time_stamp, bool real_time) +{ + struct timespec64 ts; + s64 ns; + + ts.tv_nsec = rq->perout.period.nsec; + ts.tv_sec = rq->perout.period.sec; + ns = timespec64_to_ns(&ts); + + if ((ns >> 1) != 500000000LL) + return -EINVAL; + + *time_stamp = real_time ? perout_conf_real_time(rq->perout.start.sec, 0) : + perout_conf_internal_timer(mdev, rq->perout.start.sec); + + return 0; +} + +#define MLX5_MAX_PULSE_DURATION (BIT(__mlx5_bit_sz(mtpps_reg, out_pulse_duration_ns)) - 1) +static int mlx5_perout_conf_out_pulse_duration(struct mlx5_core_dev *mdev, + struct ptp_clock_request *rq, + u32 *out_pulse_duration_ns) { - return (u64)sec << 32; + struct mlx5_pps *pps_info = &mdev->clock.pps_info; + u32 out_pulse_duration; + struct timespec64 ts; + + if (rq->perout.flags & PTP_PEROUT_DUTY_CYCLE) { + ts.tv_sec = rq->perout.on.sec; + ts.tv_nsec = rq->perout.on.nsec; + out_pulse_duration = (u32)timespec64_to_ns(&ts); + } else { + /* out_pulse_duration_ns should be up to 50% of the + * pulse period as default + */ + ts.tv_sec = rq->perout.period.sec; + ts.tv_nsec = rq->perout.period.nsec; + out_pulse_duration = (u32)timespec64_to_ns(&ts) >> 1; + } + + if (out_pulse_duration < pps_info->min_out_pulse_duration_ns || + out_pulse_duration > MLX5_MAX_PULSE_DURATION) { + mlx5_core_err(mdev, "NPPS pulse duration %u is not in [%llu, %lu]\n", + out_pulse_duration, pps_info->min_out_pulse_duration_ns, + MLX5_MAX_PULSE_DURATION); + return -EINVAL; + } + *out_pulse_duration_ns = out_pulse_duration; + + return 0; +} + +static int perout_conf_npps_real_time(struct mlx5_core_dev *mdev, struct ptp_clock_request *rq, + u32 *field_select, u32 *out_pulse_duration_ns, + u64 *period, u64 *time_stamp) +{ + struct mlx5_pps *pps_info = &mdev->clock.pps_info; + struct ptp_clock_time *time = &rq->perout.start; + struct timespec64 ts; + + ts.tv_sec = rq->perout.period.sec; + ts.tv_nsec = rq->perout.period.nsec; + if (timespec64_to_ns(&ts) < pps_info->min_npps_period) { + mlx5_core_err(mdev, "NPPS period is lower than minimal npps period %llu\n", + pps_info->min_npps_period); + return -EINVAL; + } + *period = perout_conf_real_time(rq->perout.period.sec, rq->perout.period.nsec); + + if (mlx5_perout_conf_out_pulse_duration(mdev, rq, out_pulse_duration_ns)) + return -EINVAL; + + *time_stamp = perout_conf_real_time(time->sec, time->nsec); + *field_select |= MLX5_MTPPS_FS_NPPS_PERIOD | + MLX5_MTPPS_FS_OUT_PULSE_DURATION_NS; + + return 0; +} + +static bool mlx5_perout_verify_flags(struct mlx5_core_dev *mdev, unsigned int flags) +{ + return ((!mlx5_npps_real_time_supported(mdev) && flags) || + (mlx5_npps_real_time_supported(mdev) && flags & ~PTP_PEROUT_DUTY_CYCLE)); } static int mlx5_perout_configure(struct ptp_clock_info *ptp, @@ -474,20 +569,20 @@ static int mlx5_perout_configure(struct ptp_clock_info *ptp, container_of(clock, struct mlx5_core_dev, clock); bool rt_mode = mlx5_real_time_mode(mdev); u32 in[MLX5_ST_SZ_DW(mtpps_reg)] = {0}; - struct timespec64 ts; + u32 out_pulse_duration_ns = 0; u32 field_select = 0; + u64 npps_period = 0; u64 time_stamp = 0; u8 pin_mode = 0; u8 pattern = 0; int pin = -1; int err = 0; - s64 ns; if (!MLX5_PPS_CAP(mdev)) return -EOPNOTSUPP; /* Reject requests with unsupported flags */ - if (rq->perout.flags) + if (mlx5_perout_verify_flags(mdev, rq->perout.flags)) return -EOPNOTSUPP; if (rq->perout.index >= clock->ptp_info.n_pins) @@ -500,29 +595,25 @@ static int mlx5_perout_configure(struct ptp_clock_info *ptp, if (on) { bool rt_mode = mlx5_real_time_mode(mdev); - s64 sec = rq->perout.start.sec; - - if (rq->perout.start.nsec) - return -EINVAL; pin_mode = MLX5_PIN_MODE_OUT; pattern = MLX5_OUT_PATTERN_PERIODIC; - ts.tv_sec = rq->perout.period.sec; - ts.tv_nsec = rq->perout.period.nsec; - ns = timespec64_to_ns(&ts); - if ((ns >> 1) != 500000000LL) + if (rt_mode && rq->perout.start.sec > U32_MAX) return -EINVAL; - if (rt_mode && sec > U32_MAX) - return -EINVAL; - - time_stamp = rt_mode ? perout_conf_real_time(sec) : - perout_conf_internal_timer(mdev, sec); - field_select |= MLX5_MTPPS_FS_PIN_MODE | MLX5_MTPPS_FS_PATTERN | MLX5_MTPPS_FS_TIME_STAMP; + + if (mlx5_npps_real_time_supported(mdev)) + err = perout_conf_npps_real_time(mdev, rq, &field_select, + &out_pulse_duration_ns, &npps_period, + &time_stamp); + else + err = perout_conf_1pps(mdev, rq, &time_stamp, rt_mode); + if (err) + return err; } MLX5_SET(mtpps_reg, in, pin, pin); @@ -531,7 +622,8 @@ static int mlx5_perout_configure(struct ptp_clock_info *ptp, MLX5_SET(mtpps_reg, in, enable, on); MLX5_SET64(mtpps_reg, in, time_stamp, time_stamp); MLX5_SET(mtpps_reg, in, field_select, field_select); - + MLX5_SET64(mtpps_reg, in, npps_period, npps_period); + MLX5_SET(mtpps_reg, in, out_pulse_duration_ns, out_pulse_duration_ns); err = mlx5_set_mtpps(mdev, in, sizeof(in)); if (err) return err; @@ -687,6 +779,13 @@ static void mlx5_get_pps_caps(struct mlx5_core_dev *mdev) clock->ptp_info.n_per_out = MLX5_GET(mtpps_reg, out, cap_max_num_of_pps_out_pins); + if (MLX5_CAP_MCAM_FEATURE(mdev, npps_period)) + clock->pps_info.min_npps_period = 1 << MLX5_GET(mtpps_reg, out, + cap_log_min_npps_period); + if (MLX5_CAP_MCAM_FEATURE(mdev, out_pulse_duration_ns)) + clock->pps_info.min_out_pulse_duration_ns = 1 << MLX5_GET(mtpps_reg, out, + cap_log_min_out_pulse_duration_ns); + clock->pps_info.pin_caps[0] = MLX5_GET(mtpps_reg, out, cap_pin_0_mode); clock->pps_info.pin_caps[1] = MLX5_GET(mtpps_reg, out, cap_pin_1_mode); clock->pps_info.pin_caps[2] = MLX5_GET(mtpps_reg, out, cap_pin_2_mode); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/lib/mlx5.h b/drivers/net/ethernet/mellanox/mlx5/core/lib/mlx5.h index 2f536c5d30b1..032adb21ad4b 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/lib/mlx5.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/lib/mlx5.h @@ -83,6 +83,7 @@ int mlx5_notifier_call_chain(struct mlx5_events *events, unsigned int event, voi enum { MLX5_ACCEL_OBJ_TLS_KEY = MLX5_GENERAL_OBJECT_TYPE_ENCRYPTION_KEY_TYPE_TLS, MLX5_ACCEL_OBJ_IPSEC_KEY = MLX5_GENERAL_OBJECT_TYPE_ENCRYPTION_KEY_TYPE_IPSEC, + MLX5_ACCEL_OBJ_MACSEC_KEY = MLX5_GENERAL_OBJECT_TYPE_ENCRYPTION_KEY_TYPE_MACSEC, }; int mlx5_create_encryption_key(struct mlx5_core_dev *mdev, diff --git a/drivers/net/ethernet/mellanox/mlx5/core/main.c b/drivers/net/ethernet/mellanox/mlx5/core/main.c index 89b2d9cea33f..0b459d841c3a 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/main.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/main.c @@ -671,6 +671,33 @@ static int handle_hca_cap_roce(struct mlx5_core_dev *dev, void *set_ctx) return err; } +static int handle_hca_cap_port_selection(struct mlx5_core_dev *dev, + void *set_ctx) +{ + void *set_hca_cap; + int err; + + if (!MLX5_CAP_GEN(dev, port_selection_cap)) + return 0; + + err = mlx5_core_get_caps(dev, MLX5_CAP_PORT_SELECTION); + if (err) + return err; + + if (MLX5_CAP_PORT_SELECTION(dev, port_select_flow_table_bypass) || + !MLX5_CAP_PORT_SELECTION_MAX(dev, port_select_flow_table_bypass)) + return 0; + + set_hca_cap = MLX5_ADDR_OF(set_hca_cap_in, set_ctx, capability); + memcpy(set_hca_cap, dev->caps.hca[MLX5_CAP_PORT_SELECTION]->cur, + MLX5_ST_SZ_BYTES(port_selection_cap)); + MLX5_SET(port_selection_cap, set_hca_cap, port_select_flow_table_bypass, 1); + + err = set_caps(dev, set_ctx, MLX5_SET_HCA_CAP_OP_MODE_PORT_SELECTION); + + return err; +} + static int set_hca_cap(struct mlx5_core_dev *dev) { int set_sz = MLX5_ST_SZ_BYTES(set_hca_cap_in); @@ -715,6 +742,13 @@ static int set_hca_cap(struct mlx5_core_dev *dev) goto out; } + memset(set_ctx, 0, set_sz); + err = handle_hca_cap_port_selection(dev, set_ctx); + if (err) { + mlx5_core_err(dev, "handle_hca_cap_port_selection failed\n"); + goto out; + } + out: kfree(set_ctx); return err; @@ -1058,7 +1092,7 @@ static void mlx5_cleanup_once(struct mlx5_core_dev *dev) mlx5_devcom_unregister_device(dev->priv.devcom); } -static int mlx5_function_setup(struct mlx5_core_dev *dev, u64 timeout) +static int mlx5_function_setup(struct mlx5_core_dev *dev, bool boot, u64 timeout) { int err; @@ -1096,10 +1130,12 @@ static int mlx5_function_setup(struct mlx5_core_dev *dev, u64 timeout) mlx5_cmd_set_state(dev, MLX5_CMDIF_STATE_UP); + mlx5_start_health_poll(dev); + err = mlx5_core_enable_hca(dev, 0); if (err) { mlx5_core_err(dev, "enable hca failed\n"); - goto err_cmd_cleanup; + goto stop_health_poll; } err = mlx5_core_set_issi(dev); @@ -1151,8 +1187,7 @@ static int mlx5_function_setup(struct mlx5_core_dev *dev, u64 timeout) mlx5_core_err(dev, "query hca failed\n"); goto reclaim_boot_pages; } - - mlx5_start_health_poll(dev); + mlx5_start_health_fw_log_up(dev); return 0; @@ -1160,6 +1195,8 @@ reclaim_boot_pages: mlx5_reclaim_startup_pages(dev); err_disable_hca: mlx5_core_disable_hca(dev, 0); +stop_health_poll: + mlx5_stop_health_poll(dev, boot); err_cmd_cleanup: mlx5_cmd_set_state(dev, MLX5_CMDIF_STATE_DOWN); mlx5_cmd_cleanup(dev); @@ -1171,7 +1208,6 @@ static int mlx5_function_teardown(struct mlx5_core_dev *dev, bool boot) { int err; - mlx5_stop_health_poll(dev, boot); err = mlx5_cmd_teardown_hca(dev); if (err) { mlx5_core_err(dev, "tear_down_hca failed, skip cleanup\n"); @@ -1179,6 +1215,7 @@ static int mlx5_function_teardown(struct mlx5_core_dev *dev, bool boot) } mlx5_reclaim_startup_pages(dev); mlx5_core_disable_hca(dev, 0); + mlx5_stop_health_poll(dev, boot); mlx5_cmd_set_state(dev, MLX5_CMDIF_STATE_DOWN); mlx5_cmd_cleanup(dev); @@ -1328,7 +1365,7 @@ int mlx5_init_one(struct mlx5_core_dev *dev) mutex_lock(&dev->intf_state_mutex); dev->state = MLX5_DEVICE_STATE_UP; - err = mlx5_function_setup(dev, mlx5_tout_ms(dev, FW_PRE_INIT_TIMEOUT)); + err = mlx5_function_setup(dev, true, mlx5_tout_ms(dev, FW_PRE_INIT_TIMEOUT)); if (err) goto err_function; @@ -1416,7 +1453,7 @@ int mlx5_load_one_devl_locked(struct mlx5_core_dev *dev, bool recovery) timeout = mlx5_tout_ms(dev, FW_PRE_INIT_ON_RECOVERY_TIMEOUT); else timeout = mlx5_tout_ms(dev, FW_PRE_INIT_TIMEOUT); - err = mlx5_function_setup(dev, timeout); + err = mlx5_function_setup(dev, false, timeout); if (err) goto err_function; @@ -1507,6 +1544,8 @@ static const int types[] = { MLX5_CAP_IPSEC, MLX5_CAP_PORT_SELECTION, MLX5_CAP_DEV_SHAMPO, + MLX5_CAP_MACSEC, + MLX5_CAP_ADV_VIRTUALIZATION, }; static void mlx5_hca_caps_free(struct mlx5_core_dev *dev) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.h b/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.h index ad61b86d5769..a806e3de7b7c 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.h @@ -143,6 +143,36 @@ enum mlx5_semaphore_space_address { #define MLX5_DEFAULT_PROF 2 +static inline int mlx5_flexible_inlen(struct mlx5_core_dev *dev, size_t fixed, + size_t item_size, size_t num_items, + const char *func, int line) +{ + int inlen; + + if (fixed > INT_MAX || item_size > INT_MAX || num_items > INT_MAX) { + mlx5_core_err(dev, "%s: %s:%d: input values too big: %zu + %zu * %zu\n", + __func__, func, line, fixed, item_size, num_items); + return -ENOMEM; + } + + if (check_mul_overflow((int)item_size, (int)num_items, &inlen)) { + mlx5_core_err(dev, "%s: %s:%d: multiplication overflow: %zu + %zu * %zu\n", + __func__, func, line, fixed, item_size, num_items); + return -ENOMEM; + } + + if (check_add_overflow((int)fixed, inlen, &inlen)) { + mlx5_core_err(dev, "%s: %s:%d: addition overflow: %zu + %zu * %zu\n", + __func__, func, line, fixed, item_size, num_items); + return -ENOMEM; + } + + return inlen; +} + +#define MLX5_FLEXIBLE_INLEN(dev, fixed, item_size, num_items) \ + mlx5_flexible_inlen(dev, fixed, item_size, num_items, __func__, __LINE__) + int mlx5_query_hca_caps(struct mlx5_core_dev *dev); int mlx5_query_board_id(struct mlx5_core_dev *dev); int mlx5_cmd_init(struct mlx5_core_dev *dev); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/port.c b/drivers/net/ethernet/mellanox/mlx5/core/port.c index e1bd54574ea5..a1548e6bfb35 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/port.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/port.c @@ -493,29 +493,6 @@ int mlx5_query_port_vl_hw_cap(struct mlx5_core_dev *dev, } EXPORT_SYMBOL_GPL(mlx5_query_port_vl_hw_cap); -int mlx5_core_query_ib_ppcnt(struct mlx5_core_dev *dev, - u8 port_num, void *out, size_t sz) -{ - u32 *in; - int err; - - in = kvzalloc(sz, GFP_KERNEL); - if (!in) { - err = -ENOMEM; - return err; - } - - MLX5_SET(ppcnt_reg, in, local_port, port_num); - - MLX5_SET(ppcnt_reg, in, grp, MLX5_INFINIBAND_PORT_COUNTERS_GROUP); - err = mlx5_core_access_reg(dev, in, sz, out, - sz, MLX5_REG_PPCNT, 0, 0); - - kvfree(in); - return err; -} -EXPORT_SYMBOL_GPL(mlx5_core_query_ib_ppcnt); - static int mlx5_query_pfcc_reg(struct mlx5_core_dev *dev, u32 *out, u32 out_size) { diff --git a/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_types.h b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_types.h index 062c7c74a1f3..1777a1e508e7 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_types.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_types.h @@ -1294,20 +1294,6 @@ struct mlx5dr_cmd_gid_attr { u32 roce_ver; }; -struct mlx5dr_cmd_qp_create_attr { - u32 page_id; - u32 pdn; - u32 cqn; - u32 pm_state; - u32 service_type; - u32 buff_umem_id; - u32 db_umem_id; - u32 sq_wqe_cnt; - u32 rq_wqe_cnt; - u32 rq_wqe_shift; - u8 isolate_vl_tc:1; -}; - int mlx5dr_cmd_query_gid(struct mlx5_core_dev *mdev, u8 vhca_port_num, u16 index, struct mlx5dr_cmd_gid_attr *attr); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/steering/fs_dr.h b/drivers/net/ethernet/mellanox/mlx5/core/steering/fs_dr.h index 1fb185d6ac7f..d168622063d5 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/steering/fs_dr.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/steering/fs_dr.h @@ -14,10 +14,6 @@ struct mlx5_fs_dr_action { struct mlx5dr_action *dr_action; }; -struct mlx5_fs_dr_ns { - struct mlx5_dr_ns *dr_ns; -}; - struct mlx5_fs_dr_rule { struct mlx5dr_rule *dr_rule; /* Only actions created by fs_dr */ diff --git a/drivers/net/ethernet/mellanox/mlx5/core/wq.h b/drivers/net/ethernet/mellanox/mlx5/core/wq.h index e5c4dcd1425e..4d629e5ddbc7 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/wq.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/wq.h @@ -123,7 +123,7 @@ static inline void mlx5_wq_cyc_push(struct mlx5_wq_cyc *wq) wq->cur_sz++; } -static inline void mlx5_wq_cyc_push_n(struct mlx5_wq_cyc *wq, u8 n) +static inline void mlx5_wq_cyc_push_n(struct mlx5_wq_cyc *wq, u16 n) { wq->wqe_ctr += n; wq->cur_sz += n; diff --git a/drivers/net/ethernet/mellanox/mlxbf_gige/mlxbf_gige_main.c b/drivers/net/ethernet/mellanox/mlxbf_gige/mlxbf_gige_main.c index b03e1c66bac0..2292d63a279c 100644 --- a/drivers/net/ethernet/mellanox/mlxbf_gige/mlxbf_gige_main.c +++ b/drivers/net/ethernet/mellanox/mlxbf_gige/mlxbf_gige_main.c @@ -156,7 +156,7 @@ static int mlxbf_gige_open(struct net_device *netdev) phy_start(phydev); - netif_napi_add(netdev, &priv->napi, mlxbf_gige_poll, NAPI_POLL_WEIGHT); + netif_napi_add(netdev, &priv->napi, mlxbf_gige_poll); napi_enable(&priv->napi); netif_start_queue(netdev); diff --git a/drivers/net/ethernet/mellanox/mlxsw/cmd.h b/drivers/net/ethernet/mellanox/mlxsw/cmd.h index 60232fb8ccd7..09bef04b11d1 100644 --- a/drivers/net/ethernet/mellanox/mlxsw/cmd.h +++ b/drivers/net/ethernet/mellanox/mlxsw/cmd.h @@ -703,6 +703,9 @@ MLXSW_ITEM32(cmd_mbox, config_profile, max_vepa_channels, 0x10, 0, 8); /* cmd_mbox_config_profile_max_lag * Maximum number of LAG IDs requested. + * Reserved when Spectrum-1/2/3, supported from Spectrum-4 and above. + * For Spectrum-4, firmware sets 128 for values between 1-128 and 256 for values + * between 129-256. */ MLXSW_ITEM32(cmd_mbox, config_profile, max_lag, 0x14, 0, 16); diff --git a/drivers/net/ethernet/mellanox/mlxsw/core.c b/drivers/net/ethernet/mellanox/mlxsw/core.c index 75553eb2c7f2..e2a985ec2c76 100644 --- a/drivers/net/ethernet/mellanox/mlxsw/core.c +++ b/drivers/net/ethernet/mellanox/mlxsw/core.c @@ -70,6 +70,8 @@ struct mlxsw_core { struct workqueue_struct *emad_wq; struct list_head rx_listener_list; struct list_head event_listener_list; + struct list_head irq_event_handler_list; + struct mutex irq_event_handler_lock; /* Locks access to handlers list */ struct { atomic64_t tid; struct list_head trans_list; @@ -184,6 +186,23 @@ unsigned int mlxsw_core_max_ports(const struct mlxsw_core *mlxsw_core) } EXPORT_SYMBOL(mlxsw_core_max_ports); +int mlxsw_core_max_lag(struct mlxsw_core *mlxsw_core, u16 *p_max_lag) +{ + struct mlxsw_driver *driver = mlxsw_core->driver; + + if (driver->profile->used_max_lag) { + *p_max_lag = driver->profile->max_lag; + return 0; + } + + if (!MLXSW_CORE_RES_VALID(mlxsw_core, MAX_LAG)) + return -EIO; + + *p_max_lag = MLXSW_CORE_RES_GET(mlxsw_core, MAX_LAG); + return 0; +} +EXPORT_SYMBOL(mlxsw_core_max_lag); + void *mlxsw_core_driver_priv(struct mlxsw_core *mlxsw_core) { return mlxsw_core->driver_priv; @@ -633,7 +652,7 @@ static void mlxsw_emad_process_string_tlv(const struct sk_buff *skb, return; string = mlxsw_emad_string_tlv_string_data(string_tlv); - strlcpy(trans->emad_err_string, string, + strscpy(trans->emad_err_string, string, MLXSW_EMAD_STRING_TLV_STRING_LEN); } @@ -1305,21 +1324,6 @@ mlxsw_devlink_sb_pool_set(struct devlink *devlink, extack); } -static int mlxsw_devlink_port_type_set(struct devlink_port *devlink_port, - enum devlink_port_type port_type) -{ - struct mlxsw_core *mlxsw_core = devlink_priv(devlink_port->devlink); - struct mlxsw_driver *mlxsw_driver = mlxsw_core->driver; - struct mlxsw_core_port *mlxsw_core_port = __dl_port(devlink_port); - - if (!mlxsw_driver->port_type_set) - return -EOPNOTSUPP; - - return mlxsw_driver->port_type_set(mlxsw_core, - mlxsw_core_port->local_port, - port_type); -} - static int mlxsw_devlink_sb_port_pool_get(struct devlink_port *devlink_port, unsigned int sb_index, u16 pool_index, u32 *p_threshold) @@ -1650,7 +1654,6 @@ static const struct devlink_ops mlxsw_devlink_ops = { BIT(DEVLINK_RELOAD_ACTION_FW_ACTIVATE), .reload_down = mlxsw_devlink_core_bus_device_reload_down, .reload_up = mlxsw_devlink_core_bus_device_reload_up, - .port_type_set = mlxsw_devlink_port_type_set, .port_split = mlxsw_devlink_port_split, .port_unsplit = mlxsw_devlink_port_unsplit, .sb_pool_get = mlxsw_devlink_sb_pool_get, @@ -2090,6 +2093,18 @@ static void mlxsw_core_health_fini(struct mlxsw_core *mlxsw_core) devlink_health_reporter_destroy(mlxsw_core->health.fw_fatal); } +static void mlxsw_core_irq_event_handler_init(struct mlxsw_core *mlxsw_core) +{ + INIT_LIST_HEAD(&mlxsw_core->irq_event_handler_list); + mutex_init(&mlxsw_core->irq_event_handler_lock); +} + +static void mlxsw_core_irq_event_handler_fini(struct mlxsw_core *mlxsw_core) +{ + mutex_destroy(&mlxsw_core->irq_event_handler_lock); + WARN_ON(!list_empty(&mlxsw_core->irq_event_handler_list)); +} + static int __mlxsw_core_bus_device_register(const struct mlxsw_bus_info *mlxsw_bus_info, const struct mlxsw_bus *mlxsw_bus, @@ -2101,6 +2116,7 @@ __mlxsw_core_bus_device_register(const struct mlxsw_bus_info *mlxsw_bus_info, struct mlxsw_core *mlxsw_core; struct mlxsw_driver *mlxsw_driver; size_t alloc_size; + u16 max_lag; int err; mlxsw_driver = mlxsw_core_driver_get(device_kind); @@ -2125,6 +2141,7 @@ __mlxsw_core_bus_device_register(const struct mlxsw_bus_info *mlxsw_bus_info, mlxsw_core->bus = mlxsw_bus; mlxsw_core->bus_priv = bus_priv; mlxsw_core->bus_info = mlxsw_bus_info; + mlxsw_core_irq_event_handler_init(mlxsw_core); err = mlxsw_bus->init(bus_priv, mlxsw_core, mlxsw_driver->profile, &mlxsw_core->res); @@ -2141,10 +2158,9 @@ __mlxsw_core_bus_device_register(const struct mlxsw_bus_info *mlxsw_bus_info, if (err) goto err_ports_init; - if (MLXSW_CORE_RES_VALID(mlxsw_core, MAX_LAG) && - MLXSW_CORE_RES_VALID(mlxsw_core, MAX_LAG_MEMBERS)) { - alloc_size = sizeof(*mlxsw_core->lag.mapping) * - MLXSW_CORE_RES_GET(mlxsw_core, MAX_LAG) * + err = mlxsw_core_max_lag(mlxsw_core, &max_lag); + if (!err && MLXSW_CORE_RES_VALID(mlxsw_core, MAX_LAG_MEMBERS)) { + alloc_size = sizeof(*mlxsw_core->lag.mapping) * max_lag * MLXSW_CORE_RES_GET(mlxsw_core, MAX_LAG_MEMBERS); mlxsw_core->lag.mapping = kzalloc(alloc_size, GFP_KERNEL); if (!mlxsw_core->lag.mapping) { @@ -2233,6 +2249,7 @@ err_ports_init: err_register_resources: mlxsw_bus->fini(bus_priv); err_bus_init: + mlxsw_core_irq_event_handler_fini(mlxsw_core); if (!reload) { devl_unlock(devlink); devlink_free(devlink); @@ -2302,6 +2319,7 @@ void mlxsw_core_bus_device_unregister(struct mlxsw_core *mlxsw_core, if (!reload) devl_resources_unregister(devlink); mlxsw_core->bus->fini(mlxsw_core->bus_priv); + mlxsw_core_irq_event_handler_fini(mlxsw_core); if (!reload) { devl_unlock(devlink); devlink_free(devlink); @@ -2772,6 +2790,57 @@ int mlxsw_reg_trans_bulk_wait(struct list_head *bulk_list) } EXPORT_SYMBOL(mlxsw_reg_trans_bulk_wait); +struct mlxsw_core_irq_event_handler_item { + struct list_head list; + void (*cb)(struct mlxsw_core *mlxsw_core); +}; + +int mlxsw_core_irq_event_handler_register(struct mlxsw_core *mlxsw_core, + mlxsw_irq_event_cb_t cb) +{ + struct mlxsw_core_irq_event_handler_item *item; + + item = kzalloc(sizeof(*item), GFP_KERNEL); + if (!item) + return -ENOMEM; + item->cb = cb; + mutex_lock(&mlxsw_core->irq_event_handler_lock); + list_add_tail(&item->list, &mlxsw_core->irq_event_handler_list); + mutex_unlock(&mlxsw_core->irq_event_handler_lock); + return 0; +} +EXPORT_SYMBOL(mlxsw_core_irq_event_handler_register); + +void mlxsw_core_irq_event_handler_unregister(struct mlxsw_core *mlxsw_core, + mlxsw_irq_event_cb_t cb) +{ + struct mlxsw_core_irq_event_handler_item *item, *tmp; + + mutex_lock(&mlxsw_core->irq_event_handler_lock); + list_for_each_entry_safe(item, tmp, + &mlxsw_core->irq_event_handler_list, list) { + if (item->cb == cb) { + list_del(&item->list); + kfree(item); + } + } + mutex_unlock(&mlxsw_core->irq_event_handler_lock); +} +EXPORT_SYMBOL(mlxsw_core_irq_event_handler_unregister); + +void mlxsw_core_irq_event_handlers_call(struct mlxsw_core *mlxsw_core) +{ + struct mlxsw_core_irq_event_handler_item *item; + + mutex_lock(&mlxsw_core->irq_event_handler_lock); + list_for_each_entry(item, &mlxsw_core->irq_event_handler_list, list) { + if (item->cb) + item->cb(mlxsw_core); + } + mutex_unlock(&mlxsw_core->irq_event_handler_lock); +} +EXPORT_SYMBOL(mlxsw_core_irq_event_handlers_call); + static int mlxsw_core_reg_access_cmd(struct mlxsw_core *mlxsw_core, const struct mlxsw_reg_info *reg, char *payload, @@ -3115,18 +3184,6 @@ void mlxsw_core_port_eth_set(struct mlxsw_core *mlxsw_core, u16 local_port, } EXPORT_SYMBOL(mlxsw_core_port_eth_set); -void mlxsw_core_port_ib_set(struct mlxsw_core *mlxsw_core, u16 local_port, - void *port_driver_priv) -{ - struct mlxsw_core_port *mlxsw_core_port = - &mlxsw_core->ports[local_port]; - struct devlink_port *devlink_port = &mlxsw_core_port->devlink_port; - - mlxsw_core_port->port_driver_priv = port_driver_priv; - devlink_port_type_ib_set(devlink_port, NULL); -} -EXPORT_SYMBOL(mlxsw_core_port_ib_set); - void mlxsw_core_port_clear(struct mlxsw_core *mlxsw_core, u16 local_port, void *port_driver_priv) { @@ -3139,18 +3196,6 @@ void mlxsw_core_port_clear(struct mlxsw_core *mlxsw_core, u16 local_port, } EXPORT_SYMBOL(mlxsw_core_port_clear); -enum devlink_port_type mlxsw_core_port_type_get(struct mlxsw_core *mlxsw_core, - u16 local_port) -{ - struct mlxsw_core_port *mlxsw_core_port = - &mlxsw_core->ports[local_port]; - struct devlink_port *devlink_port = &mlxsw_core_port->devlink_port; - - return devlink_port->type; -} -EXPORT_SYMBOL(mlxsw_core_port_type_get); - - struct devlink_port * mlxsw_core_port_devlink_port_get(struct mlxsw_core *mlxsw_core, u16 local_port) diff --git a/drivers/net/ethernet/mellanox/mlxsw/core.h b/drivers/net/ethernet/mellanox/mlxsw/core.h index 02d9cc2ef0c8..ca0c3d2bee6b 100644 --- a/drivers/net/ethernet/mellanox/mlxsw/core.h +++ b/drivers/net/ethernet/mellanox/mlxsw/core.h @@ -35,6 +35,8 @@ struct mlxsw_fw_rev; unsigned int mlxsw_core_max_ports(const struct mlxsw_core *mlxsw_core); +int mlxsw_core_max_lag(struct mlxsw_core *mlxsw_core, u16 *p_max_lag); + void *mlxsw_core_driver_priv(struct mlxsw_core *mlxsw_core); struct mlxsw_linecards *mlxsw_core_linecards(struct mlxsw_core *mlxsw_core); @@ -215,6 +217,14 @@ int mlxsw_reg_trans_write(struct mlxsw_core *mlxsw_core, mlxsw_reg_trans_cb_t *cb, unsigned long cb_priv); int mlxsw_reg_trans_bulk_wait(struct list_head *bulk_list); +typedef void mlxsw_irq_event_cb_t(struct mlxsw_core *mlxsw_core); + +int mlxsw_core_irq_event_handler_register(struct mlxsw_core *mlxsw_core, + mlxsw_irq_event_cb_t cb); +void mlxsw_core_irq_event_handler_unregister(struct mlxsw_core *mlxsw_core, + mlxsw_irq_event_cb_t cb); +void mlxsw_core_irq_event_handlers_call(struct mlxsw_core *mlxsw_core); + int mlxsw_reg_query(struct mlxsw_core *mlxsw_core, const struct mlxsw_reg_info *reg, char *payload); int mlxsw_reg_write(struct mlxsw_core *mlxsw_core, @@ -256,12 +266,8 @@ int mlxsw_core_cpu_port_init(struct mlxsw_core *mlxsw_core, void mlxsw_core_cpu_port_fini(struct mlxsw_core *mlxsw_core); void mlxsw_core_port_eth_set(struct mlxsw_core *mlxsw_core, u16 local_port, void *port_driver_priv, struct net_device *dev); -void mlxsw_core_port_ib_set(struct mlxsw_core *mlxsw_core, u16 local_port, - void *port_driver_priv); void mlxsw_core_port_clear(struct mlxsw_core *mlxsw_core, u16 local_port, void *port_driver_priv); -enum devlink_port_type mlxsw_core_port_type_get(struct mlxsw_core *mlxsw_core, - u16 local_port); struct devlink_port * mlxsw_core_port_devlink_port_get(struct mlxsw_core *mlxsw_core, u16 local_port); @@ -291,6 +297,7 @@ struct mlxsw_swid_config { struct mlxsw_config_profile { u16 used_max_vepa_channels:1, + used_max_lag:1, used_max_mid:1, used_max_pgt:1, used_max_system_port:1, @@ -306,6 +313,7 @@ struct mlxsw_config_profile { used_kvd_sizes:1, used_cqe_time_stamp_type:1; u8 max_vepa_channels; + u16 max_lag; u16 max_mid; u16 max_pgt; u16 max_system_port; @@ -341,8 +349,6 @@ struct mlxsw_driver { const struct mlxsw_bus_info *mlxsw_bus_info, struct netlink_ext_ack *extack); void (*fini)(struct mlxsw_core *mlxsw_core); - int (*port_type_set)(struct mlxsw_core *mlxsw_core, u16 local_port, - enum devlink_port_type new_type); int (*port_split)(struct mlxsw_core *mlxsw_core, u16 local_port, unsigned int count, struct netlink_ext_ack *extack); int (*port_unsplit)(struct mlxsw_core *mlxsw_core, u16 local_port, diff --git a/drivers/net/ethernet/mellanox/mlxsw/core_acl_flex_actions.c b/drivers/net/ethernet/mellanox/mlxsw/core_acl_flex_actions.c index 636db9a87457..9dfe7148199f 100644 --- a/drivers/net/ethernet/mellanox/mlxsw/core_acl_flex_actions.c +++ b/drivers/net/ethernet/mellanox/mlxsw/core_acl_flex_actions.c @@ -737,8 +737,9 @@ mlxsw_afa_cookie_create(struct mlxsw_afa *mlxsw_afa, if (!cookie) return ERR_PTR(-ENOMEM); refcount_set(&cookie->ref_count, 1); - memcpy(&cookie->fa_cookie, fa_cookie, - sizeof(*fa_cookie) + fa_cookie->cookie_len); + cookie->fa_cookie = *fa_cookie; + memcpy(cookie->fa_cookie.cookie, fa_cookie->cookie, + fa_cookie->cookie_len); err = rhashtable_insert_fast(&mlxsw_afa->cookie_ht, &cookie->ht_node, mlxsw_afa_cookie_ht_params); diff --git a/drivers/net/ethernet/mellanox/mlxsw/core_linecards.c b/drivers/net/ethernet/mellanox/mlxsw/core_linecards.c index ca59f0b946da..83d2dc91ba2c 100644 --- a/drivers/net/ethernet/mellanox/mlxsw/core_linecards.c +++ b/drivers/net/ethernet/mellanox/mlxsw/core_linecards.c @@ -785,6 +785,21 @@ static int mlxsw_linecard_status_get_and_process(struct mlxsw_core *mlxsw_core, return mlxsw_linecard_status_process(linecards, linecard, mddq_pl); } +static void mlxsw_linecards_irq_event_handler(struct mlxsw_core *mlxsw_core) +{ + struct mlxsw_linecards *linecards = mlxsw_core_linecards(mlxsw_core); + int i; + + /* Handle change of line card active state. */ + for (i = 0; i < linecards->count; i++) { + struct mlxsw_linecard *linecard = mlxsw_linecard_get(linecards, + i + 1); + + mlxsw_linecard_status_get_and_process(mlxsw_core, linecards, + linecard); + } +} + static const char * const mlxsw_linecard_status_event_type_name[] = { [MLXSW_LINECARD_STATUS_EVENT_TYPE_PROVISION] = "provision", [MLXSW_LINECARD_STATUS_EVENT_TYPE_UNPROVISION] = "unprovision", @@ -1238,7 +1253,6 @@ static int mlxsw_linecard_init(struct mlxsw_core *mlxsw_core, { struct devlink_linecard *devlink_linecard; struct mlxsw_linecard *linecard; - int err; linecard = mlxsw_linecard_get(linecards, slot_index); linecard->slot_index = slot_index; @@ -1248,17 +1262,45 @@ static int mlxsw_linecard_init(struct mlxsw_core *mlxsw_core, devlink_linecard = devlink_linecard_create(priv_to_devlink(mlxsw_core), slot_index, &mlxsw_linecard_ops, linecard); - if (IS_ERR(devlink_linecard)) { - err = PTR_ERR(devlink_linecard); - goto err_devlink_linecard_create; - } + if (IS_ERR(devlink_linecard)) + return PTR_ERR(devlink_linecard); + linecard->devlink_linecard = devlink_linecard; INIT_DELAYED_WORK(&linecard->status_event_to_dw, &mlxsw_linecard_status_event_to_work); + return 0; +} + +static void mlxsw_linecard_fini(struct mlxsw_core *mlxsw_core, + struct mlxsw_linecards *linecards, + u8 slot_index) +{ + struct mlxsw_linecard *linecard; + + linecard = mlxsw_linecard_get(linecards, slot_index); + cancel_delayed_work_sync(&linecard->status_event_to_dw); + /* Make sure all scheduled events are processed */ + mlxsw_core_flush_owq(); + if (linecard->active) + mlxsw_linecard_active_clear(linecard); + mlxsw_linecard_bdev_del(linecard); + devlink_linecard_destroy(linecard->devlink_linecard); + mutex_destroy(&linecard->lock); +} + +static int +mlxsw_linecard_event_delivery_init(struct mlxsw_core *mlxsw_core, + struct mlxsw_linecards *linecards, + u8 slot_index) +{ + struct mlxsw_linecard *linecard; + int err; + + linecard = mlxsw_linecard_get(linecards, slot_index); err = mlxsw_linecard_event_delivery_set(mlxsw_core, linecard, true); if (err) - goto err_event_delivery_set; + return err; err = mlxsw_linecard_status_get_and_process(mlxsw_core, linecards, linecard); @@ -1269,29 +1311,18 @@ static int mlxsw_linecard_init(struct mlxsw_core *mlxsw_core, err_status_get_and_process: mlxsw_linecard_event_delivery_set(mlxsw_core, linecard, false); -err_event_delivery_set: - devlink_linecard_destroy(linecard->devlink_linecard); -err_devlink_linecard_create: - mutex_destroy(&linecard->lock); return err; } -static void mlxsw_linecard_fini(struct mlxsw_core *mlxsw_core, - struct mlxsw_linecards *linecards, - u8 slot_index) +static void +mlxsw_linecard_event_delivery_fini(struct mlxsw_core *mlxsw_core, + struct mlxsw_linecards *linecards, + u8 slot_index) { struct mlxsw_linecard *linecard; linecard = mlxsw_linecard_get(linecards, slot_index); mlxsw_linecard_event_delivery_set(mlxsw_core, linecard, false); - cancel_delayed_work_sync(&linecard->status_event_to_dw); - /* Make sure all scheduled events are processed */ - mlxsw_core_flush_owq(); - if (linecard->active) - mlxsw_linecard_active_clear(linecard); - mlxsw_linecard_bdev_del(linecard); - devlink_linecard_destroy(linecard->devlink_linecard); - mutex_destroy(&linecard->lock); } /* LINECARDS INI BUNDLE FILE @@ -1505,6 +1536,11 @@ int mlxsw_linecards_init(struct mlxsw_core *mlxsw_core, if (err) goto err_traps_register; + err = mlxsw_core_irq_event_handler_register(mlxsw_core, + mlxsw_linecards_irq_event_handler); + if (err) + goto err_irq_event_handler_register; + mlxsw_core_linecards_set(mlxsw_core, linecards); for (i = 0; i < linecards->count; i++) { @@ -1513,11 +1549,25 @@ int mlxsw_linecards_init(struct mlxsw_core *mlxsw_core, goto err_linecard_init; } + for (i = 0; i < linecards->count; i++) { + err = mlxsw_linecard_event_delivery_init(mlxsw_core, linecards, + i + 1); + if (err) + goto err_linecard_event_delivery_init; + } + return 0; +err_linecard_event_delivery_init: + for (i--; i >= 0; i--) + mlxsw_linecard_event_delivery_fini(mlxsw_core, linecards, i + 1); + i = linecards->count; err_linecard_init: for (i--; i >= 0; i--) mlxsw_linecard_fini(mlxsw_core, linecards, i + 1); + mlxsw_core_irq_event_handler_unregister(mlxsw_core, + mlxsw_linecards_irq_event_handler); +err_irq_event_handler_register: mlxsw_core_traps_unregister(mlxsw_core, mlxsw_linecard_listener, ARRAY_SIZE(mlxsw_linecard_listener), mlxsw_core); @@ -1536,7 +1586,11 @@ void mlxsw_linecards_fini(struct mlxsw_core *mlxsw_core) if (!linecards) return; for (i = 0; i < linecards->count; i++) + mlxsw_linecard_event_delivery_fini(mlxsw_core, linecards, i + 1); + for (i = 0; i < linecards->count; i++) mlxsw_linecard_fini(mlxsw_core, linecards, i + 1); + mlxsw_core_irq_event_handler_unregister(mlxsw_core, + mlxsw_linecards_irq_event_handler); mlxsw_core_traps_unregister(mlxsw_core, mlxsw_linecard_listener, ARRAY_SIZE(mlxsw_linecard_listener), mlxsw_core); diff --git a/drivers/net/ethernet/mellanox/mlxsw/i2c.c b/drivers/net/ethernet/mellanox/mlxsw/i2c.c index ce843ea91464..716c73e4fd59 100644 --- a/drivers/net/ethernet/mellanox/mlxsw/i2c.c +++ b/drivers/net/ethernet/mellanox/mlxsw/i2c.c @@ -9,6 +9,7 @@ #include <linux/mutex.h> #include <linux/module.h> #include <linux/mod_devicetable.h> +#include <linux/platform_data/mlxreg.h> #include <linux/slab.h> #include "cmd.h" @@ -51,6 +52,15 @@ #define MLXSW_I2C_TIMEOUT_MSECS 5000 #define MLXSW_I2C_MAX_DATA_SIZE 256 +/* Driver can be initialized by kernel platform driver or from the user + * space. In the first case IRQ line number is passed through the platform + * data, otherwise default IRQ line is to be used. Default IRQ is relevant + * only for specific I2C slave address, allowing 3.4 MHz I2C path to the chip + * (special hardware feature for I2C acceleration). + */ +#define MLXSW_I2C_DEFAULT_IRQ 17 +#define MLXSW_FAST_I2C_SLAVE 0x37 + /** * struct mlxsw_i2c - device private data: * @cmd: command attributes; @@ -63,6 +73,9 @@ * @core: switch core pointer; * @bus_info: bus info block; * @block_size: maximum block size allowed to pass to under layer; + * @pdata: device platform data; + * @irq_work: interrupts work item; + * @irq: IRQ line number; */ struct mlxsw_i2c { struct { @@ -76,6 +89,9 @@ struct mlxsw_i2c { struct mlxsw_core *core; struct mlxsw_bus_info bus_info; u16 block_size; + struct mlxreg_core_hotplug_platform_data *pdata; + struct work_struct irq_work; + int irq; }; #define MLXSW_I2C_READ_MSG(_client, _addr_buf, _buf, _len) { \ @@ -546,6 +562,67 @@ static void mlxsw_i2c_fini(void *bus_priv) mlxsw_i2c->core = NULL; } +static void mlxsw_i2c_work_handler(struct work_struct *work) +{ + struct mlxsw_i2c *mlxsw_i2c; + + mlxsw_i2c = container_of(work, struct mlxsw_i2c, irq_work); + mlxsw_core_irq_event_handlers_call(mlxsw_i2c->core); +} + +static irqreturn_t mlxsw_i2c_irq_handler(int irq, void *dev) +{ + struct mlxsw_i2c *mlxsw_i2c = dev; + + mlxsw_core_schedule_work(&mlxsw_i2c->irq_work); + + /* Interrupt handler shares IRQ line with 'main' interrupt handler. + * Return here IRQ_NONE, while main handler will return IRQ_HANDLED. + */ + return IRQ_NONE; +} + +static int mlxsw_i2c_irq_init(struct mlxsw_i2c *mlxsw_i2c, u8 addr) +{ + int err; + + /* Initialize interrupt handler if system hotplug driver is reachable, + * otherwise interrupt line is not enabled and interrupts will not be + * raised to CPU. Also request_irq() call will be not valid. + */ + if (!IS_REACHABLE(CONFIG_MLXREG_HOTPLUG)) + return 0; + + /* Set default interrupt line. */ + if (mlxsw_i2c->pdata && mlxsw_i2c->pdata->irq) + mlxsw_i2c->irq = mlxsw_i2c->pdata->irq; + else if (addr == MLXSW_FAST_I2C_SLAVE) + mlxsw_i2c->irq = MLXSW_I2C_DEFAULT_IRQ; + + if (!mlxsw_i2c->irq) + return 0; + + INIT_WORK(&mlxsw_i2c->irq_work, mlxsw_i2c_work_handler); + err = request_irq(mlxsw_i2c->irq, mlxsw_i2c_irq_handler, + IRQF_TRIGGER_FALLING | IRQF_SHARED, "mlxsw-i2c", + mlxsw_i2c); + if (err) { + dev_err(mlxsw_i2c->bus_info.dev, "Failed to request irq: %d\n", + err); + return err; + } + + return 0; +} + +static void mlxsw_i2c_irq_fini(struct mlxsw_i2c *mlxsw_i2c) +{ + if (!IS_REACHABLE(CONFIG_MLXREG_HOTPLUG) || !mlxsw_i2c->irq) + return; + cancel_work_sync(&mlxsw_i2c->irq_work); + free_irq(mlxsw_i2c->irq, mlxsw_i2c); +} + static const struct mlxsw_bus mlxsw_i2c_bus = { .kind = "i2c", .init = mlxsw_i2c_init, @@ -638,17 +715,24 @@ static int mlxsw_i2c_probe(struct i2c_client *client, mlxsw_i2c->bus_info.dev = &client->dev; mlxsw_i2c->bus_info.low_frequency = true; mlxsw_i2c->dev = &client->dev; + mlxsw_i2c->pdata = client->dev.platform_data; + + err = mlxsw_i2c_irq_init(mlxsw_i2c, client->addr); + if (err) + goto errout; err = mlxsw_core_bus_device_register(&mlxsw_i2c->bus_info, &mlxsw_i2c_bus, mlxsw_i2c, false, NULL, NULL); if (err) { dev_err(&client->dev, "Fail to register core bus\n"); - return err; + goto err_bus_device_register; } return 0; +err_bus_device_register: + mlxsw_i2c_irq_fini(mlxsw_i2c); errout: mutex_destroy(&mlxsw_i2c->cmd.lock); i2c_set_clientdata(client, NULL); @@ -661,6 +745,7 @@ static int mlxsw_i2c_remove(struct i2c_client *client) struct mlxsw_i2c *mlxsw_i2c = i2c_get_clientdata(client); mlxsw_core_bus_device_unregister(mlxsw_i2c->core, false); + mlxsw_i2c_irq_fini(mlxsw_i2c); mutex_destroy(&mlxsw_i2c->cmd.lock); return 0; diff --git a/drivers/net/ethernet/mellanox/mlxsw/minimal.c b/drivers/net/ethernet/mellanox/mlxsw/minimal.c index bb1cd4bae82e..55b3c42bb007 100644 --- a/drivers/net/ethernet/mellanox/mlxsw/minimal.c +++ b/drivers/net/ethernet/mellanox/mlxsw/minimal.c @@ -26,20 +26,29 @@ static const struct mlxsw_fw_rev mlxsw_m_fw_rev = { struct mlxsw_m_port; +struct mlxsw_m_line_card { + bool active; + int module_to_port[]; +}; + struct mlxsw_m { struct mlxsw_m_port **ports; - int *module_to_port; struct mlxsw_core *core; const struct mlxsw_bus_info *bus_info; u8 base_mac[ETH_ALEN]; u8 max_ports; + u8 max_modules_per_slot; /* Maximum number of modules per-slot. */ + u8 num_of_slots; /* Including the main board. */ + struct mlxsw_m_line_card **line_cards; }; struct mlxsw_m_port { struct net_device *dev; struct mlxsw_m *mlxsw_m; u16 local_port; + u8 slot_index; u8 module; + u8 module_offset; }; static int mlxsw_m_base_mac_get(struct mlxsw_m *mlxsw_m) @@ -94,14 +103,14 @@ static void mlxsw_m_module_get_drvinfo(struct net_device *dev, struct mlxsw_m_port *mlxsw_m_port = netdev_priv(dev); struct mlxsw_m *mlxsw_m = mlxsw_m_port->mlxsw_m; - strlcpy(drvinfo->driver, mlxsw_m->bus_info->device_kind, + strscpy(drvinfo->driver, mlxsw_m->bus_info->device_kind, sizeof(drvinfo->driver)); snprintf(drvinfo->fw_version, sizeof(drvinfo->fw_version), "%d.%d.%d", mlxsw_m->bus_info->fw_rev.major, mlxsw_m->bus_info->fw_rev.minor, mlxsw_m->bus_info->fw_rev.subminor); - strlcpy(drvinfo->bus_info, mlxsw_m->bus_info->device_name, + strscpy(drvinfo->bus_info, mlxsw_m->bus_info->device_name, sizeof(drvinfo->bus_info)); } @@ -111,8 +120,9 @@ static int mlxsw_m_get_module_info(struct net_device *netdev, struct mlxsw_m_port *mlxsw_m_port = netdev_priv(netdev); struct mlxsw_core *core = mlxsw_m_port->mlxsw_m->core; - return mlxsw_env_get_module_info(netdev, core, 0, mlxsw_m_port->module, - modinfo); + return mlxsw_env_get_module_info(netdev, core, + mlxsw_m_port->slot_index, + mlxsw_m_port->module, modinfo); } static int @@ -122,7 +132,8 @@ mlxsw_m_get_module_eeprom(struct net_device *netdev, struct ethtool_eeprom *ee, struct mlxsw_m_port *mlxsw_m_port = netdev_priv(netdev); struct mlxsw_core *core = mlxsw_m_port->mlxsw_m->core; - return mlxsw_env_get_module_eeprom(netdev, core, 0, + return mlxsw_env_get_module_eeprom(netdev, core, + mlxsw_m_port->slot_index, mlxsw_m_port->module, ee, data); } @@ -134,7 +145,8 @@ mlxsw_m_get_module_eeprom_by_page(struct net_device *netdev, struct mlxsw_m_port *mlxsw_m_port = netdev_priv(netdev); struct mlxsw_core *core = mlxsw_m_port->mlxsw_m->core; - return mlxsw_env_get_module_eeprom_by_page(core, 0, + return mlxsw_env_get_module_eeprom_by_page(core, + mlxsw_m_port->slot_index, mlxsw_m_port->module, page, extack); } @@ -144,7 +156,8 @@ static int mlxsw_m_reset(struct net_device *netdev, u32 *flags) struct mlxsw_m_port *mlxsw_m_port = netdev_priv(netdev); struct mlxsw_core *core = mlxsw_m_port->mlxsw_m->core; - return mlxsw_env_reset_module(netdev, core, 0, mlxsw_m_port->module, + return mlxsw_env_reset_module(netdev, core, mlxsw_m_port->slot_index, + mlxsw_m_port->module, flags); } @@ -156,7 +169,8 @@ mlxsw_m_get_module_power_mode(struct net_device *netdev, struct mlxsw_m_port *mlxsw_m_port = netdev_priv(netdev); struct mlxsw_core *core = mlxsw_m_port->mlxsw_m->core; - return mlxsw_env_get_module_power_mode(core, 0, mlxsw_m_port->module, + return mlxsw_env_get_module_power_mode(core, mlxsw_m_port->slot_index, + mlxsw_m_port->module, params, extack); } @@ -168,7 +182,8 @@ mlxsw_m_set_module_power_mode(struct net_device *netdev, struct mlxsw_m_port *mlxsw_m_port = netdev_priv(netdev); struct mlxsw_core *core = mlxsw_m_port->mlxsw_m->core; - return mlxsw_env_set_module_power_mode(core, 0, mlxsw_m_port->module, + return mlxsw_env_set_module_power_mode(core, mlxsw_m_port->slot_index, + mlxsw_m_port->module, params->policy, extack); } @@ -184,7 +199,7 @@ static const struct ethtool_ops mlxsw_m_port_ethtool_ops = { static int mlxsw_m_port_module_info_get(struct mlxsw_m *mlxsw_m, u16 local_port, - u8 *p_module, u8 *p_width) + u8 *p_module, u8 *p_width, u8 *p_slot_index) { char pmlp_pl[MLXSW_REG_PMLP_LEN]; int err; @@ -195,6 +210,7 @@ mlxsw_m_port_module_info_get(struct mlxsw_m *mlxsw_m, u16 local_port, return err; *p_module = mlxsw_reg_pmlp_module_get(pmlp_pl, 0); *p_width = mlxsw_reg_pmlp_width_get(pmlp_pl); + *p_slot_index = mlxsw_reg_pmlp_slot_index_get(pmlp_pl, 0); return 0; } @@ -212,18 +228,25 @@ mlxsw_m_port_dev_addr_get(struct mlxsw_m_port *mlxsw_m_port) if (err) return err; mlxsw_reg_ppad_mac_memcpy_from(ppad_pl, addr); - eth_hw_addr_gen(mlxsw_m_port->dev, addr, mlxsw_m_port->module + 1); + eth_hw_addr_gen(mlxsw_m_port->dev, addr, mlxsw_m_port->module + 1 + + mlxsw_m_port->module_offset); return 0; } +static bool mlxsw_m_port_created(struct mlxsw_m *mlxsw_m, u16 local_port) +{ + return mlxsw_m->ports[local_port]; +} + static int -mlxsw_m_port_create(struct mlxsw_m *mlxsw_m, u16 local_port, u8 module) +mlxsw_m_port_create(struct mlxsw_m *mlxsw_m, u16 local_port, u8 slot_index, + u8 module) { struct mlxsw_m_port *mlxsw_m_port; struct net_device *dev; int err; - err = mlxsw_core_port_init(mlxsw_m->core, local_port, 0, + err = mlxsw_core_port_init(mlxsw_m->core, local_port, slot_index, module + 1, false, 0, false, 0, mlxsw_m->base_mac, sizeof(mlxsw_m->base_mac)); @@ -246,6 +269,15 @@ mlxsw_m_port_create(struct mlxsw_m *mlxsw_m, u16 local_port, u8 module) mlxsw_m_port->mlxsw_m = mlxsw_m; mlxsw_m_port->local_port = local_port; mlxsw_m_port->module = module; + mlxsw_m_port->slot_index = slot_index; + /* Add module offset for line card. Offset for main board iz zero. + * For line card in slot #n offset is calculated as (#n - 1) + * multiplied by maximum modules number, which could be found on a line + * card. + */ + mlxsw_m_port->module_offset = mlxsw_m_port->slot_index ? + (mlxsw_m_port->slot_index - 1) * + mlxsw_m->max_modules_per_slot : 0; dev->netdev_ops = &mlxsw_m_port_netdev_ops; dev->ethtool_ops = &mlxsw_m_port_ethtool_ops; @@ -291,19 +323,29 @@ static void mlxsw_m_port_remove(struct mlxsw_m *mlxsw_m, u16 local_port) mlxsw_core_port_fini(mlxsw_m->core, local_port); } +static int* +mlxsw_m_port_mapping_get(struct mlxsw_m *mlxsw_m, u8 slot_index, u8 module) +{ + return &mlxsw_m->line_cards[slot_index]->module_to_port[module]; +} + static int mlxsw_m_port_module_map(struct mlxsw_m *mlxsw_m, u16 local_port, u8 *last_module) { unsigned int max_ports = mlxsw_core_max_ports(mlxsw_m->core); - u8 module, width; + u8 module, width, slot_index; + int *module_to_port; int err; /* Fill out to local port mapping array */ err = mlxsw_m_port_module_info_get(mlxsw_m, local_port, &module, - &width); + &width, &slot_index); if (err) return err; + /* Skip if line card has been already configured */ + if (mlxsw_m->line_cards[slot_index]->active) + return 0; if (!width) return 0; /* Skip, if port belongs to the cluster */ @@ -313,91 +355,220 @@ static int mlxsw_m_port_module_map(struct mlxsw_m *mlxsw_m, u16 local_port, if (WARN_ON_ONCE(module >= max_ports)) return -EINVAL; - mlxsw_env_module_port_map(mlxsw_m->core, 0, module); - mlxsw_m->module_to_port[module] = ++mlxsw_m->max_ports; + mlxsw_env_module_port_map(mlxsw_m->core, slot_index, module); + module_to_port = mlxsw_m_port_mapping_get(mlxsw_m, slot_index, module); + *module_to_port = local_port; return 0; } -static void mlxsw_m_port_module_unmap(struct mlxsw_m *mlxsw_m, u8 module) +static void +mlxsw_m_port_module_unmap(struct mlxsw_m *mlxsw_m, u8 slot_index, u8 module) { - mlxsw_m->module_to_port[module] = -1; - mlxsw_env_module_port_unmap(mlxsw_m->core, 0, module); + int *module_to_port = mlxsw_m_port_mapping_get(mlxsw_m, slot_index, + module); + *module_to_port = -1; + mlxsw_env_module_port_unmap(mlxsw_m->core, slot_index, module); } -static int mlxsw_m_ports_create(struct mlxsw_m *mlxsw_m) +static int mlxsw_m_linecards_init(struct mlxsw_m *mlxsw_m) { unsigned int max_ports = mlxsw_core_max_ports(mlxsw_m->core); - u8 last_module = max_ports; - int i; - int err; + char mgpir_pl[MLXSW_REG_MGPIR_LEN]; + u8 num_of_modules; + int i, j, err; + + mlxsw_reg_mgpir_pack(mgpir_pl, 0); + err = mlxsw_reg_query(mlxsw_m->core, MLXSW_REG(mgpir), mgpir_pl); + if (err) + return err; + + mlxsw_reg_mgpir_unpack(mgpir_pl, NULL, NULL, NULL, &num_of_modules, + &mlxsw_m->num_of_slots); + /* If the system is modular, get the maximum number of modules per-slot. + * Otherwise, get the maximum number of modules on the main board. + */ + if (mlxsw_m->num_of_slots) + mlxsw_m->max_modules_per_slot = + mlxsw_reg_mgpir_max_modules_per_slot_get(mgpir_pl); + else + mlxsw_m->max_modules_per_slot = num_of_modules; + /* Add slot for main board. */ + mlxsw_m->num_of_slots += 1; mlxsw_m->ports = kcalloc(max_ports, sizeof(*mlxsw_m->ports), GFP_KERNEL); if (!mlxsw_m->ports) return -ENOMEM; - mlxsw_m->module_to_port = kmalloc_array(max_ports, sizeof(int), - GFP_KERNEL); - if (!mlxsw_m->module_to_port) { + mlxsw_m->line_cards = kcalloc(mlxsw_m->num_of_slots, + sizeof(*mlxsw_m->line_cards), + GFP_KERNEL); + if (!mlxsw_m->line_cards) { err = -ENOMEM; - goto err_module_to_port_alloc; + goto err_kcalloc; } - /* Invalidate the entries of module to local port mapping array */ - for (i = 0; i < max_ports; i++) - mlxsw_m->module_to_port[i] = -1; + for (i = 0; i < mlxsw_m->num_of_slots; i++) { + mlxsw_m->line_cards[i] = + kzalloc(struct_size(mlxsw_m->line_cards[i], + module_to_port, + mlxsw_m->max_modules_per_slot), + GFP_KERNEL); + if (!mlxsw_m->line_cards[i]) { + err = -ENOMEM; + goto err_kmalloc_array; + } - /* Fill out module to local port mapping array */ - for (i = 1; i < max_ports; i++) { - err = mlxsw_m_port_module_map(mlxsw_m, i, &last_module); - if (err) - goto err_module_to_port_map; + /* Invalidate the entries of module to local port mapping array. */ + for (j = 0; j < mlxsw_m->max_modules_per_slot; j++) + mlxsw_m->line_cards[i]->module_to_port[j] = -1; } - /* Create port objects for each valid entry */ - for (i = 0; i < mlxsw_m->max_ports; i++) { - if (mlxsw_m->module_to_port[i] > 0) { - err = mlxsw_m_port_create(mlxsw_m, - mlxsw_m->module_to_port[i], - i); + return 0; + +err_kmalloc_array: + for (i--; i >= 0; i--) + kfree(mlxsw_m->line_cards[i]); +err_kcalloc: + kfree(mlxsw_m->ports); + return err; +} + +static void mlxsw_m_linecards_fini(struct mlxsw_m *mlxsw_m) +{ + int i = mlxsw_m->num_of_slots; + + for (i--; i >= 0; i--) + kfree(mlxsw_m->line_cards[i]); + kfree(mlxsw_m->line_cards); + kfree(mlxsw_m->ports); +} + +static void +mlxsw_m_linecard_port_module_unmap(struct mlxsw_m *mlxsw_m, u8 slot_index) +{ + int i; + + for (i = mlxsw_m->max_modules_per_slot - 1; i >= 0; i--) { + int *module_to_port; + + module_to_port = mlxsw_m_port_mapping_get(mlxsw_m, slot_index, i); + if (*module_to_port > 0) + mlxsw_m_port_module_unmap(mlxsw_m, slot_index, i); + } +} + +static int +mlxsw_m_linecard_ports_create(struct mlxsw_m *mlxsw_m, u8 slot_index) +{ + int *module_to_port; + int i, err; + + for (i = 0; i < mlxsw_m->max_modules_per_slot; i++) { + module_to_port = mlxsw_m_port_mapping_get(mlxsw_m, slot_index, i); + if (*module_to_port > 0) { + err = mlxsw_m_port_create(mlxsw_m, *module_to_port, + slot_index, i); if (err) - goto err_module_to_port_create; + goto err_port_create; + /* Mark slot as active */ + if (!mlxsw_m->line_cards[slot_index]->active) + mlxsw_m->line_cards[slot_index]->active = true; } } - return 0; -err_module_to_port_create: +err_port_create: for (i--; i >= 0; i--) { - if (mlxsw_m->module_to_port[i] > 0) - mlxsw_m_port_remove(mlxsw_m, - mlxsw_m->module_to_port[i]); + module_to_port = mlxsw_m_port_mapping_get(mlxsw_m, slot_index, i); + if (*module_to_port > 0 && + mlxsw_m_port_created(mlxsw_m, *module_to_port)) { + mlxsw_m_port_remove(mlxsw_m, *module_to_port); + /* Mark slot as inactive */ + if (mlxsw_m->line_cards[slot_index]->active) + mlxsw_m->line_cards[slot_index]->active = false; + } } - i = max_ports; -err_module_to_port_map: - for (i--; i > 0; i--) - mlxsw_m_port_module_unmap(mlxsw_m, i); - kfree(mlxsw_m->module_to_port); -err_module_to_port_alloc: - kfree(mlxsw_m->ports); return err; } -static void mlxsw_m_ports_remove(struct mlxsw_m *mlxsw_m) +static void +mlxsw_m_linecard_ports_remove(struct mlxsw_m *mlxsw_m, u8 slot_index) { int i; - for (i = 0; i < mlxsw_m->max_ports; i++) { - if (mlxsw_m->module_to_port[i] > 0) { - mlxsw_m_port_remove(mlxsw_m, - mlxsw_m->module_to_port[i]); - mlxsw_m_port_module_unmap(mlxsw_m, i); + for (i = 0; i < mlxsw_m->max_modules_per_slot; i++) { + int *module_to_port = mlxsw_m_port_mapping_get(mlxsw_m, + slot_index, i); + + if (*module_to_port > 0 && + mlxsw_m_port_created(mlxsw_m, *module_to_port)) { + mlxsw_m_port_remove(mlxsw_m, *module_to_port); + mlxsw_m_port_module_unmap(mlxsw_m, slot_index, i); } } +} - kfree(mlxsw_m->module_to_port); - kfree(mlxsw_m->ports); +static int mlxsw_m_ports_module_map(struct mlxsw_m *mlxsw_m) +{ + unsigned int max_ports = mlxsw_core_max_ports(mlxsw_m->core); + u8 last_module = max_ports; + int i, err; + + for (i = 1; i < max_ports; i++) { + err = mlxsw_m_port_module_map(mlxsw_m, i, &last_module); + if (err) + return err; + } + + return 0; +} + +static int mlxsw_m_ports_create(struct mlxsw_m *mlxsw_m) +{ + int err; + + /* Fill out module to local port mapping array */ + err = mlxsw_m_ports_module_map(mlxsw_m); + if (err) + goto err_ports_module_map; + + /* Create port objects for each valid entry */ + err = mlxsw_m_linecard_ports_create(mlxsw_m, 0); + if (err) + goto err_linecard_ports_create; + + return 0; + +err_linecard_ports_create: +err_ports_module_map: + mlxsw_m_linecard_port_module_unmap(mlxsw_m, 0); + + return err; +} + +static void mlxsw_m_ports_remove(struct mlxsw_m *mlxsw_m) +{ + mlxsw_m_linecard_ports_remove(mlxsw_m, 0); +} + +static void +mlxsw_m_ports_remove_selected(struct mlxsw_core *mlxsw_core, + bool (*selector)(void *priv, u16 local_port), + void *priv) +{ + struct mlxsw_m *mlxsw_m = mlxsw_core_driver_priv(mlxsw_core); + struct mlxsw_linecard *linecard_priv = priv; + struct mlxsw_m_line_card *linecard; + + linecard = mlxsw_m->line_cards[linecard_priv->slot_index]; + + if (WARN_ON(!linecard->active)) + return; + + mlxsw_m_linecard_ports_remove(mlxsw_m, linecard_priv->slot_index); + linecard->active = false; } static int mlxsw_m_fw_rev_validate(struct mlxsw_m *mlxsw_m) @@ -418,6 +589,60 @@ static int mlxsw_m_fw_rev_validate(struct mlxsw_m *mlxsw_m) return -EINVAL; } +static void +mlxsw_m_got_active(struct mlxsw_core *mlxsw_core, u8 slot_index, void *priv) +{ + struct mlxsw_m_line_card *linecard; + struct mlxsw_m *mlxsw_m = priv; + int err; + + linecard = mlxsw_m->line_cards[slot_index]; + /* Skip if line card has been already configured during init */ + if (linecard->active) + return; + + /* Fill out module to local port mapping array */ + err = mlxsw_m_ports_module_map(mlxsw_m); + if (err) + goto err_ports_module_map; + + /* Create port objects for each valid entry */ + err = mlxsw_m_linecard_ports_create(mlxsw_m, slot_index); + if (err) { + dev_err(mlxsw_m->bus_info->dev, "Failed to create port for line card at slot %d\n", + slot_index); + goto err_linecard_ports_create; + } + + linecard->active = true; + + return; + +err_linecard_ports_create: +err_ports_module_map: + mlxsw_m_linecard_port_module_unmap(mlxsw_m, slot_index); +} + +static void +mlxsw_m_got_inactive(struct mlxsw_core *mlxsw_core, u8 slot_index, void *priv) +{ + struct mlxsw_m_line_card *linecard; + struct mlxsw_m *mlxsw_m = priv; + + linecard = mlxsw_m->line_cards[slot_index]; + + if (WARN_ON(!linecard->active)) + return; + + mlxsw_m_linecard_ports_remove(mlxsw_m, slot_index); + linecard->active = false; +} + +static struct mlxsw_linecards_event_ops mlxsw_m_event_ops = { + .got_active = mlxsw_m_got_active, + .got_inactive = mlxsw_m_got_inactive, +}; + static int mlxsw_m_init(struct mlxsw_core *mlxsw_core, const struct mlxsw_bus_info *mlxsw_bus_info, struct netlink_ext_ack *extack) @@ -438,13 +663,33 @@ static int mlxsw_m_init(struct mlxsw_core *mlxsw_core, return err; } + err = mlxsw_m_linecards_init(mlxsw_m); + if (err) { + dev_err(mlxsw_m->bus_info->dev, "Failed to create line cards\n"); + return err; + } + + err = mlxsw_linecards_event_ops_register(mlxsw_core, + &mlxsw_m_event_ops, mlxsw_m); + if (err) { + dev_err(mlxsw_m->bus_info->dev, "Failed to register line cards operations\n"); + goto linecards_event_ops_register; + } + err = mlxsw_m_ports_create(mlxsw_m); if (err) { dev_err(mlxsw_m->bus_info->dev, "Failed to create ports\n"); - return err; + goto err_ports_create; } return 0; + +err_ports_create: + mlxsw_linecards_event_ops_unregister(mlxsw_core, + &mlxsw_m_event_ops, mlxsw_m); +linecards_event_ops_register: + mlxsw_m_linecards_fini(mlxsw_m); + return err; } static void mlxsw_m_fini(struct mlxsw_core *mlxsw_core) @@ -452,6 +697,9 @@ static void mlxsw_m_fini(struct mlxsw_core *mlxsw_core) struct mlxsw_m *mlxsw_m = mlxsw_core_driver_priv(mlxsw_core); mlxsw_m_ports_remove(mlxsw_m); + mlxsw_linecards_event_ops_unregister(mlxsw_core, + &mlxsw_m_event_ops, mlxsw_m); + mlxsw_m_linecards_fini(mlxsw_m); } static const struct mlxsw_config_profile mlxsw_m_config_profile; @@ -461,6 +709,7 @@ static struct mlxsw_driver mlxsw_m_driver = { .priv_size = sizeof(struct mlxsw_m), .init = mlxsw_m_init, .fini = mlxsw_m_fini, + .ports_remove_selected = mlxsw_m_ports_remove_selected, .profile = &mlxsw_m_config_profile, }; diff --git a/drivers/net/ethernet/mellanox/mlxsw/pci.c b/drivers/net/ethernet/mellanox/mlxsw/pci.c index 50527adc5b5a..c968309657dd 100644 --- a/drivers/net/ethernet/mellanox/mlxsw/pci.c +++ b/drivers/net/ethernet/mellanox/mlxsw/pci.c @@ -1187,6 +1187,11 @@ static int mlxsw_pci_config_profile(struct mlxsw_pci *mlxsw_pci, char *mbox, mlxsw_cmd_mbox_config_profile_max_vepa_channels_set( mbox, profile->max_vepa_channels); } + if (profile->used_max_lag) { + mlxsw_cmd_mbox_config_profile_set_max_lag_set(mbox, 1); + mlxsw_cmd_mbox_config_profile_max_lag_set(mbox, + profile->max_lag); + } if (profile->used_max_mid) { mlxsw_cmd_mbox_config_profile_set_max_mid_set( mbox, 1); diff --git a/drivers/net/ethernet/mellanox/mlxsw/reg.h b/drivers/net/ethernet/mellanox/mlxsw/reg.h index f27bdecdf952..0777bed5bb1a 100644 --- a/drivers/net/ethernet/mellanox/mlxsw/reg.h +++ b/drivers/net/ethernet/mellanox/mlxsw/reg.h @@ -2218,76 +2218,6 @@ static inline void mlxsw_reg_smpe_pack(char *payload, u16 local_port, mlxsw_reg_smpe_evid_set(payload, evid); } -/* SFTR-V2 - Switch Flooding Table Version 2 Register - * -------------------------------------------------- - * The switch flooding table is used for flooding packet replication. The table - * defines a bit mask of ports for packet replication. - */ -#define MLXSW_REG_SFTR2_ID 0x202F -#define MLXSW_REG_SFTR2_LEN 0x120 - -MLXSW_REG_DEFINE(sftr2, MLXSW_REG_SFTR2_ID, MLXSW_REG_SFTR2_LEN); - -/* reg_sftr2_swid - * Switch partition ID with which to associate the port. - * Access: Index - */ -MLXSW_ITEM32(reg, sftr2, swid, 0x00, 24, 8); - -/* reg_sftr2_flood_table - * Flooding table index to associate with the specific type on the specific - * switch partition. - * Access: Index - */ -MLXSW_ITEM32(reg, sftr2, flood_table, 0x00, 16, 6); - -/* reg_sftr2_index - * Index. Used as an index into the Flooding Table in case the table is - * configured to use VID / FID or FID Offset. - * Access: Index - */ -MLXSW_ITEM32(reg, sftr2, index, 0x00, 0, 16); - -/* reg_sftr2_table_type - * See mlxsw_flood_table_type - * Access: RW - */ -MLXSW_ITEM32(reg, sftr2, table_type, 0x04, 16, 3); - -/* reg_sftr2_range - * Range of entries to update - * Access: Index - */ -MLXSW_ITEM32(reg, sftr2, range, 0x04, 0, 16); - -/* reg_sftr2_port - * Local port membership (1 bit per port). - * Access: RW - */ -MLXSW_ITEM_BIT_ARRAY(reg, sftr2, port, 0x20, 0x80, 1); - -/* reg_sftr2_port_mask - * Local port mask (1 bit per port). - * Access: WO - */ -MLXSW_ITEM_BIT_ARRAY(reg, sftr2, port_mask, 0xA0, 0x80, 1); - -static inline void mlxsw_reg_sftr2_pack(char *payload, - unsigned int flood_table, - unsigned int index, - enum mlxsw_flood_table_type table_type, - unsigned int range, u16 port, bool set) -{ - MLXSW_REG_ZERO(sftr2, payload); - mlxsw_reg_sftr2_swid_set(payload, 0); - mlxsw_reg_sftr2_flood_table_set(payload, flood_table); - mlxsw_reg_sftr2_index_set(payload, index); - mlxsw_reg_sftr2_table_type_set(payload, table_type); - mlxsw_reg_sftr2_range_set(payload, range); - mlxsw_reg_sftr2_port_set(payload, port, set); - mlxsw_reg_sftr2_port_mask_set(payload, port, 1); -} - /* SMID-V2 - Switch Multicast ID Version 2 Register * ------------------------------------------------ * The MID record maps from a MID (Multicast ID), which is a unique identifier @@ -4729,25 +4659,6 @@ MLXSW_ITEM32(reg, ptys, ext_eth_proto_cap, 0x08, 0, 32); */ MLXSW_ITEM32(reg, ptys, eth_proto_cap, 0x0C, 0, 32); -/* reg_ptys_ib_link_width_cap - * IB port supported widths. - * Access: RO - */ -MLXSW_ITEM32(reg, ptys, ib_link_width_cap, 0x10, 16, 16); - -#define MLXSW_REG_PTYS_IB_SPEED_SDR BIT(0) -#define MLXSW_REG_PTYS_IB_SPEED_DDR BIT(1) -#define MLXSW_REG_PTYS_IB_SPEED_QDR BIT(2) -#define MLXSW_REG_PTYS_IB_SPEED_FDR10 BIT(3) -#define MLXSW_REG_PTYS_IB_SPEED_FDR BIT(4) -#define MLXSW_REG_PTYS_IB_SPEED_EDR BIT(5) - -/* reg_ptys_ib_proto_cap - * IB port supported speeds and protocols. - * Access: RO - */ -MLXSW_ITEM32(reg, ptys, ib_proto_cap, 0x10, 0, 16); - /* reg_ptys_ext_eth_proto_admin * Extended speed and protocol to set port to. * Access: RW @@ -4760,18 +4671,6 @@ MLXSW_ITEM32(reg, ptys, ext_eth_proto_admin, 0x14, 0, 32); */ MLXSW_ITEM32(reg, ptys, eth_proto_admin, 0x18, 0, 32); -/* reg_ptys_ib_link_width_admin - * IB width to set port to. - * Access: RW - */ -MLXSW_ITEM32(reg, ptys, ib_link_width_admin, 0x1C, 16, 16); - -/* reg_ptys_ib_proto_admin - * IB speeds and protocols to set port to. - * Access: RW - */ -MLXSW_ITEM32(reg, ptys, ib_proto_admin, 0x1C, 0, 16); - /* reg_ptys_ext_eth_proto_oper * The extended current speed and protocol configured for the port. * Access: RO @@ -4784,18 +4683,6 @@ MLXSW_ITEM32(reg, ptys, ext_eth_proto_oper, 0x20, 0, 32); */ MLXSW_ITEM32(reg, ptys, eth_proto_oper, 0x24, 0, 32); -/* reg_ptys_ib_link_width_oper - * The current IB width to set port to. - * Access: RO - */ -MLXSW_ITEM32(reg, ptys, ib_link_width_oper, 0x28, 16, 16); - -/* reg_ptys_ib_proto_oper - * The current IB speed and protocol. - * Access: RO - */ -MLXSW_ITEM32(reg, ptys, ib_proto_oper, 0x28, 0, 16); - enum mlxsw_reg_ptys_connector_type { MLXSW_REG_PTYS_CONNECTOR_TYPE_UNKNOWN_OR_NO_CONNECTOR, MLXSW_REG_PTYS_CONNECTOR_TYPE_PORT_NONE, @@ -4866,33 +4753,6 @@ static inline void mlxsw_reg_ptys_ext_eth_unpack(char *payload, mlxsw_reg_ptys_ext_eth_proto_oper_get(payload); } -static inline void mlxsw_reg_ptys_ib_pack(char *payload, u16 local_port, - u16 proto_admin, u16 link_width) -{ - MLXSW_REG_ZERO(ptys, payload); - mlxsw_reg_ptys_local_port_set(payload, local_port); - mlxsw_reg_ptys_proto_mask_set(payload, MLXSW_REG_PTYS_PROTO_MASK_IB); - mlxsw_reg_ptys_ib_proto_admin_set(payload, proto_admin); - mlxsw_reg_ptys_ib_link_width_admin_set(payload, link_width); -} - -static inline void mlxsw_reg_ptys_ib_unpack(char *payload, u16 *p_ib_proto_cap, - u16 *p_ib_link_width_cap, - u16 *p_ib_proto_oper, - u16 *p_ib_link_width_oper) -{ - if (p_ib_proto_cap) - *p_ib_proto_cap = mlxsw_reg_ptys_ib_proto_cap_get(payload); - if (p_ib_link_width_cap) - *p_ib_link_width_cap = - mlxsw_reg_ptys_ib_link_width_cap_get(payload); - if (p_ib_proto_oper) - *p_ib_proto_oper = mlxsw_reg_ptys_ib_proto_oper_get(payload); - if (p_ib_link_width_oper) - *p_ib_link_width_oper = - mlxsw_reg_ptys_ib_link_width_oper_get(payload); -} - /* PPAD - Port Physical Address Register * ------------------------------------- * The PPAD register configures the per port physical MAC address. @@ -5666,27 +5526,6 @@ static inline void mlxsw_reg_ppcnt_pack(char *payload, u16 local_port, mlxsw_reg_ppcnt_prio_tc_set(payload, prio_tc); } -/* PLIB - Port Local to InfiniBand Port - * ------------------------------------ - * The PLIB register performs mapping from Local Port into InfiniBand Port. - */ -#define MLXSW_REG_PLIB_ID 0x500A -#define MLXSW_REG_PLIB_LEN 0x10 - -MLXSW_REG_DEFINE(plib, MLXSW_REG_PLIB_ID, MLXSW_REG_PLIB_LEN); - -/* reg_plib_local_port - * Local port number. - * Access: Index - */ -MLXSW_ITEM32_LP(reg, plib, 0x00, 16, 0x00, 12); - -/* reg_plib_ib_port - * InfiniBand port remapping for local_port. - * Access: RW - */ -MLXSW_ITEM32(reg, plib, ib_port, 0x00, 0, 8); - /* PPTB - Port Prio To Buffer Register * ----------------------------------- * Configures the switch priority to buffer table. @@ -12924,7 +12763,6 @@ static const struct mlxsw_reg_info *mlxsw_reg_infos[] = { MLXSW_REG(spvc), MLXSW_REG(spevet), MLXSW_REG(smpe), - MLXSW_REG(sftr2), MLXSW_REG(smid2), MLXSW_REG(cwtp), MLXSW_REG(cwtpm), @@ -12962,7 +12800,6 @@ static const struct mlxsw_reg_info *mlxsw_reg_infos[] = { MLXSW_REG(paos), MLXSW_REG(pfcc), MLXSW_REG(ppcnt), - MLXSW_REG(plib), MLXSW_REG(pptb), MLXSW_REG(pbmc), MLXSW_REG(pspa), diff --git a/drivers/net/ethernet/mellanox/mlxsw/spectrum.c b/drivers/net/ethernet/mellanox/mlxsw/spectrum.c index 30c7b0e15721..5bcf5bceff71 100644 --- a/drivers/net/ethernet/mellanox/mlxsw/spectrum.c +++ b/drivers/net/ethernet/mellanox/mlxsw/spectrum.c @@ -2691,6 +2691,7 @@ static void mlxsw_sp_traps_fini(struct mlxsw_sp *mlxsw_sp) static int mlxsw_sp_lag_init(struct mlxsw_sp *mlxsw_sp) { char slcr_pl[MLXSW_REG_SLCR_LEN]; + u16 max_lag; u32 seed; int err; @@ -2709,12 +2710,14 @@ static int mlxsw_sp_lag_init(struct mlxsw_sp *mlxsw_sp) if (err) return err; - if (!MLXSW_CORE_RES_VALID(mlxsw_sp->core, MAX_LAG) || - !MLXSW_CORE_RES_VALID(mlxsw_sp->core, MAX_LAG_MEMBERS)) + err = mlxsw_core_max_lag(mlxsw_sp->core, &max_lag); + if (err) + return err; + + if (!MLXSW_CORE_RES_VALID(mlxsw_sp->core, MAX_LAG_MEMBERS)) return -EIO; - mlxsw_sp->lags = kcalloc(MLXSW_CORE_RES_GET(mlxsw_sp->core, MAX_LAG), - sizeof(struct mlxsw_sp_upper), + mlxsw_sp->lags = kcalloc(max_lag, sizeof(struct mlxsw_sp_upper), GFP_KERNEL); if (!mlxsw_sp->lags) return -ENOMEM; @@ -3509,6 +3512,33 @@ static const struct mlxsw_config_profile mlxsw_sp2_config_profile = { .cqe_time_stamp_type = MLXSW_CMD_MBOX_CONFIG_PROFILE_CQE_TIME_STAMP_TYPE_UTC, }; +/* Reduce number of LAGs from full capacity (256) to the maximum supported LAGs + * in Spectrum-2/3, to avoid regression in number of free entries in the PGT + * table. + */ +#define MLXSW_SP4_CONFIG_PROFILE_MAX_LAG 128 + +static const struct mlxsw_config_profile mlxsw_sp4_config_profile = { + .used_max_lag = 1, + .max_lag = MLXSW_SP4_CONFIG_PROFILE_MAX_LAG, + .used_flood_mode = 1, + .flood_mode = MLXSW_CMD_MBOX_CONFIG_PROFILE_FLOOD_MODE_CONTROLLED, + .used_max_ib_mc = 1, + .max_ib_mc = 0, + .used_max_pkey = 1, + .max_pkey = 0, + .used_ubridge = 1, + .ubridge = 1, + .swid_config = { + { + .used_type = 1, + .type = MLXSW_PORT_SWID_TYPE_ETH, + } + }, + .used_cqe_time_stamp_type = 1, + .cqe_time_stamp_type = MLXSW_CMD_MBOX_CONFIG_PROFILE_CQE_TIME_STAMP_TYPE_UTC, +}; + static void mlxsw_sp_resource_size_params_prepare(struct mlxsw_core *mlxsw_core, struct devlink_resource_size_params *kvd_size_params, @@ -4039,7 +4069,7 @@ static struct mlxsw_driver mlxsw_sp4_driver = { .params_unregister = mlxsw_sp2_params_unregister, .ptp_transmitted = mlxsw_sp_ptp_transmitted, .txhdr_len = MLXSW_TXHDR_LEN, - .profile = &mlxsw_sp2_config_profile, + .profile = &mlxsw_sp4_config_profile, .sdq_supports_cqe_v2 = true, }; @@ -4263,10 +4293,13 @@ static int mlxsw_sp_lag_index_get(struct mlxsw_sp *mlxsw_sp, { struct mlxsw_sp_upper *lag; int free_lag_id = -1; - u64 max_lag; - int i; + u16 max_lag; + int err, i; + + err = mlxsw_core_max_lag(mlxsw_sp->core, &max_lag); + if (err) + return err; - max_lag = MLXSW_CORE_RES_GET(mlxsw_sp->core, MAX_LAG); for (i = 0; i < max_lag; i++) { lag = mlxsw_sp_lag_get(mlxsw_sp, i); if (lag->ref_count) { diff --git a/drivers/net/ethernet/mellanox/mlxsw/spectrum_ethtool.c b/drivers/net/ethernet/mellanox/mlxsw/spectrum_ethtool.c index 915dffb85a1c..dcd79d7e2af4 100644 --- a/drivers/net/ethernet/mellanox/mlxsw/spectrum_ethtool.c +++ b/drivers/net/ethernet/mellanox/mlxsw/spectrum_ethtool.c @@ -14,16 +14,16 @@ static void mlxsw_sp_port_get_drvinfo(struct net_device *dev, struct mlxsw_sp_port *mlxsw_sp_port = netdev_priv(dev); struct mlxsw_sp *mlxsw_sp = mlxsw_sp_port->mlxsw_sp; - strlcpy(drvinfo->driver, mlxsw_sp->bus_info->device_kind, + strscpy(drvinfo->driver, mlxsw_sp->bus_info->device_kind, sizeof(drvinfo->driver)); - strlcpy(drvinfo->version, mlxsw_sp_driver_version, + strscpy(drvinfo->version, mlxsw_sp_driver_version, sizeof(drvinfo->version)); snprintf(drvinfo->fw_version, sizeof(drvinfo->fw_version), "%d.%d.%d", mlxsw_sp->bus_info->fw_rev.major, mlxsw_sp->bus_info->fw_rev.minor, mlxsw_sp->bus_info->fw_rev.subminor); - strlcpy(drvinfo->bus_info, mlxsw_sp->bus_info->device_name, + strscpy(drvinfo->bus_info, mlxsw_sp->bus_info->device_name, sizeof(drvinfo->bus_info)); } diff --git a/drivers/net/ethernet/mellanox/mlxsw/spectrum_router.c b/drivers/net/ethernet/mellanox/mlxsw/spectrum_router.c index 2c4443c6b964..48f1fa62a4fd 100644 --- a/drivers/net/ethernet/mellanox/mlxsw/spectrum_router.c +++ b/drivers/net/ethernet/mellanox/mlxsw/spectrum_router.c @@ -1819,7 +1819,7 @@ void mlxsw_sp_ipip_entry_demote_tunnel(struct mlxsw_sp *mlxsw_sp, /* The configuration where several tunnels have the same local address in the * same underlay table needs special treatment in the HW. That is currently not * implemented in the driver. This function finds and demotes the first tunnel - * with a given source address, except the one passed in in the argument + * with a given source address, except the one passed in the argument * `except'. */ bool diff --git a/drivers/net/ethernet/micrel/ks8851.h b/drivers/net/ethernet/micrel/ks8851.h index 6f34a61739b6..fecd43754cea 100644 --- a/drivers/net/ethernet/micrel/ks8851.h +++ b/drivers/net/ethernet/micrel/ks8851.h @@ -403,7 +403,7 @@ struct ks8851_net { struct eeprom_93cx6 eeprom; struct regulator *vdd_reg; struct regulator *vdd_io; - int gpio; + struct gpio_desc *gpio; struct mii_bus *mii_bus; void (*lock)(struct ks8851_net *ks, diff --git a/drivers/net/ethernet/micrel/ks8851_common.c b/drivers/net/ethernet/micrel/ks8851_common.c index 691206f19ea7..cfbc900d4aeb 100644 --- a/drivers/net/ethernet/micrel/ks8851_common.c +++ b/drivers/net/ethernet/micrel/ks8851_common.c @@ -17,10 +17,9 @@ #include <linux/cache.h> #include <linux/crc32.h> #include <linux/mii.h> +#include <linux/gpio/consumer.h> #include <linux/regulator/consumer.h> -#include <linux/gpio.h> -#include <linux/of_gpio.h> #include <linux/of_mdio.h> #include <linux/of_net.h> @@ -703,9 +702,9 @@ static const struct net_device_ops ks8851_netdev_ops = { static void ks8851_get_drvinfo(struct net_device *dev, struct ethtool_drvinfo *di) { - strlcpy(di->driver, "KS8851", sizeof(di->driver)); - strlcpy(di->version, "1.00", sizeof(di->version)); - strlcpy(di->bus_info, dev_name(dev->dev.parent), sizeof(di->bus_info)); + strscpy(di->driver, "KS8851", sizeof(di->driver)); + strscpy(di->version, "1.00", sizeof(di->version)); + strscpy(di->bus_info, dev_name(dev->dev.parent), sizeof(di->bus_info)); } static u32 ks8851_get_msglevel(struct net_device *dev) @@ -1117,24 +1116,23 @@ int ks8851_probe_common(struct net_device *netdev, struct device *dev, { struct ks8851_net *ks = netdev_priv(netdev); unsigned cider; - int gpio; int ret; ks->netdev = netdev; ks->tx_space = 6144; - gpio = of_get_named_gpio_flags(dev->of_node, "reset-gpios", 0, NULL); - if (gpio == -EPROBE_DEFER) - return gpio; - - ks->gpio = gpio; - if (gpio_is_valid(gpio)) { - ret = devm_gpio_request_one(dev, gpio, - GPIOF_OUT_INIT_LOW, "ks8851_rst_n"); - if (ret) { - dev_err(dev, "reset gpio request failed\n"); - return ret; - } + ks->gpio = devm_gpiod_get_optional(dev, "reset", GPIOD_OUT_HIGH); + ret = PTR_ERR_OR_ZERO(ks->gpio); + if (ret) { + if (ret != -EPROBE_DEFER) + dev_err(dev, "reset gpio request failed: %d\n", ret); + return ret; + } + + ret = gpiod_set_consumer_name(ks->gpio, "ks8851_rst_n"); + if (ret) { + dev_err(dev, "failed to set reset gpio name: %d\n", ret); + return ret; } ks->vdd_io = devm_regulator_get(dev, "vdd-io"); @@ -1161,9 +1159,9 @@ int ks8851_probe_common(struct net_device *netdev, struct device *dev, goto err_reg; } - if (gpio_is_valid(gpio)) { + if (ks->gpio) { usleep_range(10000, 11000); - gpio_set_value(gpio, 1); + gpiod_set_value_cansleep(ks->gpio, 0); } spin_lock_init(&ks->statelock); @@ -1239,8 +1237,8 @@ int ks8851_probe_common(struct net_device *netdev, struct device *dev, err_id: ks8851_unregister_mdiobus(ks); err_mdio: - if (gpio_is_valid(gpio)) - gpio_set_value(gpio, 0); + if (ks->gpio) + gpiod_set_value_cansleep(ks->gpio, 1); regulator_disable(ks->vdd_reg); err_reg: regulator_disable(ks->vdd_io); @@ -1259,8 +1257,8 @@ void ks8851_remove_common(struct device *dev) dev_info(dev, "remove\n"); unregister_netdev(priv->netdev); - if (gpio_is_valid(priv->gpio)) - gpio_set_value(priv->gpio, 0); + if (priv->gpio) + gpiod_set_value_cansleep(priv->gpio, 1); regulator_disable(priv->vdd_reg); regulator_disable(priv->vdd_io); } diff --git a/drivers/net/ethernet/micrel/ks8851_spi.c b/drivers/net/ethernet/micrel/ks8851_spi.c index 82d55fc27edc..70bc7253454f 100644 --- a/drivers/net/ethernet/micrel/ks8851_spi.c +++ b/drivers/net/ethernet/micrel/ks8851_spi.c @@ -413,7 +413,8 @@ static int ks8851_probe_spi(struct spi_device *spi) spi->bits_per_word = 8; - ks = netdev_priv(netdev); + kss = netdev_priv(netdev); + ks = &kss->ks8851; ks->lock = ks8851_lock_spi; ks->unlock = ks8851_unlock_spi; @@ -433,8 +434,6 @@ static int ks8851_probe_spi(struct spi_device *spi) IRQ_RXPSI) /* RX process stop */ ks->rc_ier = STD_IRQ; - kss = to_ks8851_spi(ks); - kss->spidev = spi; mutex_init(&kss->lock); INIT_WORK(&kss->tx_work, ks8851_tx_work); diff --git a/drivers/net/ethernet/micrel/ksz884x.c b/drivers/net/ethernet/micrel/ksz884x.c index 2b3eb5ed8233..468520079c65 100644 --- a/drivers/net/ethernet/micrel/ksz884x.c +++ b/drivers/net/ethernet/micrel/ksz884x.c @@ -5998,9 +5998,9 @@ static void netdev_get_drvinfo(struct net_device *dev, struct dev_priv *priv = netdev_priv(dev); struct dev_info *hw_priv = priv->adapter; - strlcpy(info->driver, DRV_NAME, sizeof(info->driver)); - strlcpy(info->version, DRV_VERSION, sizeof(info->version)); - strlcpy(info->bus_info, pci_name(hw_priv->pdev), + strscpy(info->driver, DRV_NAME, sizeof(info->driver)); + strscpy(info->version, DRV_VERSION, sizeof(info->version)); + strscpy(info->bus_info, pci_name(hw_priv->pdev), sizeof(info->bus_info)); } diff --git a/drivers/net/ethernet/microchip/enc28j60.c b/drivers/net/ethernet/microchip/enc28j60.c index 559ad94a44d0..176efbeae127 100644 --- a/drivers/net/ethernet/microchip/enc28j60.c +++ b/drivers/net/ethernet/microchip/enc28j60.c @@ -1467,9 +1467,9 @@ static void enc28j60_restart_work_handler(struct work_struct *work) static void enc28j60_get_drvinfo(struct net_device *dev, struct ethtool_drvinfo *info) { - strlcpy(info->driver, DRV_NAME, sizeof(info->driver)); - strlcpy(info->version, DRV_VERSION, sizeof(info->version)); - strlcpy(info->bus_info, + strscpy(info->driver, DRV_NAME, sizeof(info->driver)); + strscpy(info->version, DRV_VERSION, sizeof(info->version)); + strscpy(info->bus_info, dev_name(dev->dev.parent), sizeof(info->bus_info)); } diff --git a/drivers/net/ethernet/microchip/encx24j600.c b/drivers/net/ethernet/microchip/encx24j600.c index dc1840cb5b10..d7c8aa77ec75 100644 --- a/drivers/net/ethernet/microchip/encx24j600.c +++ b/drivers/net/ethernet/microchip/encx24j600.c @@ -925,9 +925,9 @@ static void encx24j600_get_regs(struct net_device *dev, static void encx24j600_get_drvinfo(struct net_device *dev, struct ethtool_drvinfo *info) { - strlcpy(info->driver, DRV_NAME, sizeof(info->driver)); - strlcpy(info->version, DRV_VERSION, sizeof(info->version)); - strlcpy(info->bus_info, dev_name(dev->dev.parent), + strscpy(info->driver, DRV_NAME, sizeof(info->driver)); + strscpy(info->version, DRV_VERSION, sizeof(info->version)); + strscpy(info->bus_info, dev_name(dev->dev.parent), sizeof(info->bus_info)); } diff --git a/drivers/net/ethernet/microchip/lan743x_ethtool.c b/drivers/net/ethernet/microchip/lan743x_ethtool.c index b1c74e6cb012..c739d60ee17d 100644 --- a/drivers/net/ethernet/microchip/lan743x_ethtool.c +++ b/drivers/net/ethernet/microchip/lan743x_ethtool.c @@ -579,8 +579,8 @@ static void lan743x_ethtool_get_drvinfo(struct net_device *netdev, { struct lan743x_adapter *adapter = netdev_priv(netdev); - strlcpy(info->driver, DRIVER_NAME, sizeof(info->driver)); - strlcpy(info->bus_info, + strscpy(info->driver, DRIVER_NAME, sizeof(info->driver)); + strscpy(info->bus_info, pci_name(adapter->pdev), sizeof(info->bus_info)); } diff --git a/drivers/net/ethernet/microchip/lan743x_main.c b/drivers/net/ethernet/microchip/lan743x_main.c index a9a1dea6d731..50eeecba1f18 100644 --- a/drivers/net/ethernet/microchip/lan743x_main.c +++ b/drivers/net/ethernet/microchip/lan743x_main.c @@ -1585,6 +1585,9 @@ static void lan743x_rfe_set_multicast(struct lan743x_adapter *adapter) rfctl |= RFE_CTL_AM_; } + if (netdev->features & NETIF_F_RXCSUM) + rfctl |= RFE_CTL_IP_COE_ | RFE_CTL_TCP_UDP_COE_; + memset(hash_table, 0, DP_SEL_VHF_HASH_LEN * sizeof(u32)); if (netdev_mc_count(netdev)) { struct netdev_hw_addr *ha; @@ -2066,11 +2069,13 @@ static netdev_tx_t lan743x_tx_xmit_frame(struct lan743x_tx *tx, { int required_number_of_descriptors = 0; unsigned int start_frame_length = 0; + netdev_tx_t retval = NETDEV_TX_OK; unsigned int frame_length = 0; unsigned int head_length = 0; unsigned long irq_flags = 0; bool do_timestamp = false; bool ignore_sync = false; + struct netdev_queue *txq; int nr_frags = 0; bool gso = false; int j; @@ -2083,9 +2088,12 @@ static netdev_tx_t lan743x_tx_xmit_frame(struct lan743x_tx *tx, if (required_number_of_descriptors > (tx->ring_size - 1)) { dev_kfree_skb_irq(skb); } else { - /* save to overflow buffer */ - tx->overflow_skb = skb; - netif_stop_queue(tx->adapter->netdev); + /* save how many descriptors we needed to restart the queue */ + tx->rqd_descriptors = required_number_of_descriptors; + retval = NETDEV_TX_BUSY; + txq = netdev_get_tx_queue(tx->adapter->netdev, + tx->channel_number); + netif_tx_stop_queue(txq); } goto unlock; } @@ -2144,15 +2152,15 @@ finish: unlock: spin_unlock_irqrestore(&tx->ring_lock, irq_flags); - return NETDEV_TX_OK; + return retval; } static int lan743x_tx_napi_poll(struct napi_struct *napi, int weight) { struct lan743x_tx *tx = container_of(napi, struct lan743x_tx, napi); struct lan743x_adapter *adapter = tx->adapter; - bool start_transmitter = false; unsigned long irq_flags = 0; + struct netdev_queue *txq; u32 ioc_bit = 0; ioc_bit = DMAC_INT_BIT_TX_IOC_(tx->channel_number); @@ -2163,24 +2171,20 @@ static int lan743x_tx_napi_poll(struct napi_struct *napi, int weight) /* clean up tx ring */ lan743x_tx_release_completed_descriptors(tx); - if (netif_queue_stopped(adapter->netdev)) { - if (tx->overflow_skb) { - if (lan743x_tx_get_desc_cnt(tx, tx->overflow_skb) <= - lan743x_tx_get_avail_desc(tx)) - start_transmitter = true; + txq = netdev_get_tx_queue(adapter->netdev, tx->channel_number); + if (netif_tx_queue_stopped(txq)) { + if (tx->rqd_descriptors) { + if (tx->rqd_descriptors <= + lan743x_tx_get_avail_desc(tx)) { + tx->rqd_descriptors = 0; + netif_tx_wake_queue(txq); + } } else { - netif_wake_queue(adapter->netdev); + netif_tx_wake_queue(txq); } } spin_unlock_irqrestore(&tx->ring_lock, irq_flags); - if (start_transmitter) { - /* space is now available, transmit overflow skb */ - lan743x_tx_xmit_frame(tx, tx->overflow_skb); - tx->overflow_skb = NULL; - netif_wake_queue(adapter->netdev); - } - if (!napi_complete(napi)) goto done; @@ -2304,10 +2308,7 @@ static void lan743x_tx_close(struct lan743x_tx *tx) lan743x_tx_release_all_descriptors(tx); - if (tx->overflow_skb) { - dev_kfree_skb(tx->overflow_skb); - tx->overflow_skb = NULL; - } + tx->rqd_descriptors = 0; lan743x_tx_ring_cleanup(tx); } @@ -2387,7 +2388,7 @@ static int lan743x_tx_open(struct lan743x_tx *tx) (tx->channel_number)); netif_napi_add_tx_weight(adapter->netdev, &tx->napi, lan743x_tx_napi_poll, - tx->ring_size - 1); + NAPI_POLL_WEIGHT); napi_enable(&tx->napi); data = 0; @@ -2549,6 +2550,7 @@ static int lan743x_rx_process_buffer(struct lan743x_rx *rx) int result = RX_PROCESS_RESULT_NOTHING_TO_DO; struct lan743x_rx_buffer_info *buffer_info; int frame_length, buffer_length; + bool is_ice, is_tce, is_icsm; int extension_index = -1; bool is_last, is_first; struct sk_buff *skb; @@ -2595,6 +2597,9 @@ static int lan743x_rx_process_buffer(struct lan743x_rx *rx) frame_length = RX_DESC_DATA0_FRAME_LENGTH_GET_(le32_to_cpu(descriptor->data0)); buffer_length = buffer_info->buffer_length; + is_ice = le32_to_cpu(descriptor->data1) & RX_DESC_DATA1_STATUS_ICE_; + is_tce = le32_to_cpu(descriptor->data1) & RX_DESC_DATA1_STATUS_TCE_; + is_icsm = le32_to_cpu(descriptor->data1) & RX_DESC_DATA1_STATUS_ICSM_; netdev_dbg(netdev, "%s%schunk: %d/%d", is_first ? "first " : " ", @@ -2663,6 +2668,10 @@ process_extension: if (is_last && rx->skb_head) { rx->skb_head->protocol = eth_type_trans(rx->skb_head, rx->adapter->netdev); + if (rx->adapter->netdev->features & NETIF_F_RXCSUM) { + if (!is_ice && !is_tce && !is_icsm) + skb->ip_summed = CHECKSUM_UNNECESSARY; + } netdev_dbg(netdev, "sending %d byte frame to OS", rx->skb_head->len); napi_gro_receive(&rx->napi, rx->skb_head); @@ -2866,9 +2875,7 @@ static int lan743x_rx_open(struct lan743x_rx *rx) if (ret) goto return_error; - netif_napi_add(adapter->netdev, - &rx->napi, lan743x_rx_napi_poll, - NAPI_POLL_WEIGHT); + netif_napi_add(adapter->netdev, &rx->napi, lan743x_rx_napi_poll); lan743x_csr_write(adapter, DMAC_CMD, DMAC_CMD_RX_SWR_(rx->channel_number)); @@ -3347,8 +3354,10 @@ static int lan743x_pcidev_probe(struct pci_dev *pdev, PCI11X1X_USED_TX_CHANNELS, LAN743X_USED_RX_CHANNELS); } else { - netdev = devm_alloc_etherdev(&pdev->dev, - sizeof(struct lan743x_adapter)); + netdev = devm_alloc_etherdev_mqs(&pdev->dev, + sizeof(struct lan743x_adapter), + LAN743X_USED_TX_CHANNELS, + LAN743X_USED_RX_CHANNELS); } if (!netdev) @@ -3383,7 +3392,8 @@ static int lan743x_pcidev_probe(struct pci_dev *pdev, adapter->netdev->netdev_ops = &lan743x_netdev_ops; adapter->netdev->ethtool_ops = &lan743x_ethtool_ops; - adapter->netdev->features = NETIF_F_SG | NETIF_F_TSO | NETIF_F_HW_CSUM; + adapter->netdev->features = NETIF_F_SG | NETIF_F_TSO | + NETIF_F_HW_CSUM | NETIF_F_RXCSUM; adapter->netdev->hw_features = adapter->netdev->features; /* carrier off reporting is important to ethtool even BEFORE open */ diff --git a/drivers/net/ethernet/microchip/lan743x_main.h b/drivers/net/ethernet/microchip/lan743x_main.h index 72adae4f2aa0..67877d3b6dd9 100644 --- a/drivers/net/ethernet/microchip/lan743x_main.h +++ b/drivers/net/ethernet/microchip/lan743x_main.h @@ -266,6 +266,8 @@ #define RFE_ADDR_FILT_LO(x) (0x404 + (8 * (x))) #define RFE_CTL (0x508) +#define RFE_CTL_TCP_UDP_COE_ BIT(12) +#define RFE_CTL_IP_COE_ BIT(11) #define RFE_CTL_AB_ BIT(10) #define RFE_CTL_AM_ BIT(9) #define RFE_CTL_AU_ BIT(8) @@ -954,8 +956,7 @@ struct lan743x_tx { struct napi_struct napi; u32 frame_count; - - struct sk_buff *overflow_skb; + u32 rqd_descriptors; }; void lan743x_tx_set_timestamping_mode(struct lan743x_tx *tx, @@ -1110,7 +1111,7 @@ struct lan743x_tx_buffer_info { unsigned int buffer_length; }; -#define LAN743X_TX_RING_SIZE (50) +#define LAN743X_TX_RING_SIZE (128) /* OWN bit is set. ie, Descs are owned by RX DMAC */ #define RX_DESC_DATA0_OWN_ (0x00008000) @@ -1122,6 +1123,9 @@ struct lan743x_tx_buffer_info { (((data0) & RX_DESC_DATA0_FRAME_LENGTH_MASK_) >> 16) #define RX_DESC_DATA0_EXT_ (0x00004000) #define RX_DESC_DATA0_BUF_LENGTH_MASK_ (0x00003FFF) +#define RX_DESC_DATA1_STATUS_ICE_ (0x00020000) +#define RX_DESC_DATA1_STATUS_TCE_ (0x00010000) +#define RX_DESC_DATA1_STATUS_ICSM_ (0x00000001) #define RX_DESC_DATA2_TS_NS_MASK_ (0x3FFFFFFF) #if ((NET_IP_ALIGN != 0) && (NET_IP_ALIGN != 2)) diff --git a/drivers/net/ethernet/microchip/lan743x_ptp.c b/drivers/net/ethernet/microchip/lan743x_ptp.c index 6a11e2ceb013..da3ea905adbb 100644 --- a/drivers/net/ethernet/microchip/lan743x_ptp.c +++ b/drivers/net/ethernet/microchip/lan743x_ptp.c @@ -1049,6 +1049,10 @@ static int lan743x_ptpci_verify_pin_config(struct ptp_clock_info *ptp, enum ptp_pin_function func, unsigned int chan) { + struct lan743x_ptp *lan_ptp = + container_of(ptp, struct lan743x_ptp, ptp_clock_info); + struct lan743x_adapter *adapter = + container_of(lan_ptp, struct lan743x_adapter, ptp); int result = 0; /* Confirm the requested function is supported. Parameter @@ -1057,7 +1061,10 @@ static int lan743x_ptpci_verify_pin_config(struct ptp_clock_info *ptp, switch (func) { case PTP_PF_NONE: case PTP_PF_PEROUT: + break; case PTP_PF_EXTTS: + if (!adapter->is_pci11x1x) + result = -1; break; case PTP_PF_PHYSYNC: default: diff --git a/drivers/net/ethernet/microchip/lan966x/Kconfig b/drivers/net/ethernet/microchip/lan966x/Kconfig index 4241ff0e5098..49e1464a4313 100644 --- a/drivers/net/ethernet/microchip/lan966x/Kconfig +++ b/drivers/net/ethernet/microchip/lan966x/Kconfig @@ -4,6 +4,7 @@ config LAN966X_SWITCH depends on HAS_IOMEM depends on OF depends on NET_SWITCHDEV + depends on BRIDGE || BRIDGE=n select PHYLINK select PACKING help diff --git a/drivers/net/ethernet/microchip/lan966x/Makefile b/drivers/net/ethernet/microchip/lan966x/Makefile index fd2e0ebb2427..962f7c5f9e7d 100644 --- a/drivers/net/ethernet/microchip/lan966x/Makefile +++ b/drivers/net/ethernet/microchip/lan966x/Makefile @@ -8,4 +8,7 @@ obj-$(CONFIG_LAN966X_SWITCH) += lan966x-switch.o lan966x-switch-objs := lan966x_main.o lan966x_phylink.o lan966x_port.o \ lan966x_mac.o lan966x_ethtool.o lan966x_switchdev.o \ lan966x_vlan.o lan966x_fdb.o lan966x_mdb.o \ - lan966x_ptp.o lan966x_fdma.o + lan966x_ptp.o lan966x_fdma.o lan966x_lag.o \ + lan966x_tc.o lan966x_mqprio.o lan966x_taprio.o \ + lan966x_tbf.o lan966x_cbs.o lan966x_ets.o \ + lan966x_tc_matchall.o lan966x_police.o lan966x_mirror.o diff --git a/drivers/net/ethernet/microchip/lan966x/lan966x_cbs.c b/drivers/net/ethernet/microchip/lan966x/lan966x_cbs.c new file mode 100644 index 000000000000..70cbbf8d2b67 --- /dev/null +++ b/drivers/net/ethernet/microchip/lan966x/lan966x_cbs.c @@ -0,0 +1,70 @@ +// SPDX-License-Identifier: GPL-2.0+ + +#include "lan966x_main.h" + +int lan966x_cbs_add(struct lan966x_port *port, + struct tc_cbs_qopt_offload *qopt) +{ + struct lan966x *lan966x = port->lan966x; + u32 cir, cbs; + u8 se_idx; + + /* Check for invalid values */ + if (qopt->idleslope <= 0 || + qopt->sendslope >= 0 || + qopt->locredit >= qopt->hicredit) + return -EINVAL; + + se_idx = SE_IDX_QUEUE + port->chip_port * NUM_PRIO_QUEUES + qopt->queue; + cir = qopt->idleslope; + cbs = (qopt->idleslope - qopt->sendslope) * + (qopt->hicredit - qopt->locredit) / + -qopt->sendslope; + + /* Rate unit is 100 kbps */ + cir = DIV_ROUND_UP(cir, 100); + /* Avoid using zero rate */ + cir = cir ?: 1; + /* Burst unit is 4kB */ + cbs = DIV_ROUND_UP(cbs, 4096); + /* Avoid using zero burst */ + cbs = cbs ?: 1; + + /* Check that actually the result can be written */ + if (cir > GENMASK(15, 0) || + cbs > GENMASK(6, 0)) + return -EINVAL; + + lan_rmw(QSYS_SE_CFG_SE_AVB_ENA_SET(1) | + QSYS_SE_CFG_SE_FRM_MODE_SET(1), + QSYS_SE_CFG_SE_AVB_ENA | + QSYS_SE_CFG_SE_FRM_MODE, + lan966x, QSYS_SE_CFG(se_idx)); + + lan_wr(QSYS_CIR_CFG_CIR_RATE_SET(cir) | + QSYS_CIR_CFG_CIR_BURST_SET(cbs), + lan966x, QSYS_CIR_CFG(se_idx)); + + return 0; +} + +int lan966x_cbs_del(struct lan966x_port *port, + struct tc_cbs_qopt_offload *qopt) +{ + struct lan966x *lan966x = port->lan966x; + u8 se_idx; + + se_idx = SE_IDX_QUEUE + port->chip_port * NUM_PRIO_QUEUES + qopt->queue; + + lan_rmw(QSYS_SE_CFG_SE_AVB_ENA_SET(1) | + QSYS_SE_CFG_SE_FRM_MODE_SET(0), + QSYS_SE_CFG_SE_AVB_ENA | + QSYS_SE_CFG_SE_FRM_MODE, + lan966x, QSYS_SE_CFG(se_idx)); + + lan_wr(QSYS_CIR_CFG_CIR_RATE_SET(0) | + QSYS_CIR_CFG_CIR_BURST_SET(0), + lan966x, QSYS_CIR_CFG(se_idx)); + + return 0; +} diff --git a/drivers/net/ethernet/microchip/lan966x/lan966x_ets.c b/drivers/net/ethernet/microchip/lan966x/lan966x_ets.c new file mode 100644 index 000000000000..8310d3f35404 --- /dev/null +++ b/drivers/net/ethernet/microchip/lan966x/lan966x_ets.c @@ -0,0 +1,96 @@ +// SPDX-License-Identifier: GPL-2.0+ + +#include "lan966x_main.h" + +#define DWRR_COST_BIT_WIDTH BIT(5) + +static u32 lan966x_ets_hw_cost(u32 w_min, u32 weight) +{ + u32 res; + + /* Round half up: Multiply with 16 before division, + * add 8 and divide result with 16 again + */ + res = (((DWRR_COST_BIT_WIDTH << 4) * w_min / weight) + 8) >> 4; + return max_t(u32, 1, res) - 1; +} + +int lan966x_ets_add(struct lan966x_port *port, + struct tc_ets_qopt_offload *qopt) +{ + struct tc_ets_qopt_offload_replace_params *params; + struct lan966x *lan966x = port->lan966x; + u32 w_min = 100; + u8 count = 0; + u32 se_idx; + u8 i; + + /* Check the input */ + if (qopt->parent != TC_H_ROOT) + return -EINVAL; + + params = &qopt->replace_params; + if (params->bands != NUM_PRIO_QUEUES) + return -EINVAL; + + for (i = 0; i < params->bands; ++i) { + /* In the switch the DWRR is always on the lowest consecutive + * priorities. Due to this, the first priority must map to the + * first DWRR band. + */ + if (params->priomap[i] != (7 - i)) + return -EINVAL; + + if (params->quanta[i] && params->weights[i] == 0) + return -EINVAL; + } + + se_idx = SE_IDX_PORT + port->chip_port; + + /* Find minimum weight */ + for (i = 0; i < params->bands; ++i) { + if (params->quanta[i] == 0) + continue; + + w_min = min(w_min, params->weights[i]); + } + + for (i = 0; i < params->bands; ++i) { + if (params->quanta[i] == 0) + continue; + + ++count; + + lan_wr(lan966x_ets_hw_cost(w_min, params->weights[i]), + lan966x, QSYS_SE_DWRR_CFG(se_idx, 7 - i)); + } + + lan_rmw(QSYS_SE_CFG_SE_DWRR_CNT_SET(count) | + QSYS_SE_CFG_SE_RR_ENA_SET(0), + QSYS_SE_CFG_SE_DWRR_CNT | + QSYS_SE_CFG_SE_RR_ENA, + lan966x, QSYS_SE_CFG(se_idx)); + + return 0; +} + +int lan966x_ets_del(struct lan966x_port *port, + struct tc_ets_qopt_offload *qopt) +{ + struct lan966x *lan966x = port->lan966x; + u32 se_idx; + int i; + + se_idx = SE_IDX_PORT + port->chip_port; + + for (i = 0; i < NUM_PRIO_QUEUES; ++i) + lan_wr(0, lan966x, QSYS_SE_DWRR_CFG(se_idx, i)); + + lan_rmw(QSYS_SE_CFG_SE_DWRR_CNT_SET(0) | + QSYS_SE_CFG_SE_RR_ENA_SET(0), + QSYS_SE_CFG_SE_DWRR_CNT | + QSYS_SE_CFG_SE_RR_ENA, + lan966x, QSYS_SE_CFG(se_idx)); + + return 0; +} diff --git a/drivers/net/ethernet/microchip/lan966x/lan966x_fdb.c b/drivers/net/ethernet/microchip/lan966x/lan966x_fdb.c index da5ca7188679..2ea263e893ee 100644 --- a/drivers/net/ethernet/microchip/lan966x/lan966x_fdb.c +++ b/drivers/net/ethernet/microchip/lan966x/lan966x_fdb.c @@ -8,6 +8,7 @@ struct lan966x_fdb_event_work { struct work_struct work; struct switchdev_notifier_fdb_info fdb_info; struct net_device *dev; + struct net_device *orig_dev; struct lan966x *lan966x; unsigned long event; }; @@ -127,75 +128,119 @@ void lan966x_fdb_deinit(struct lan966x *lan966x) lan966x_fdb_purge_entries(lan966x); } -static void lan966x_fdb_event_work(struct work_struct *work) +void lan966x_fdb_flush_workqueue(struct lan966x *lan966x) +{ + flush_workqueue(lan966x->fdb_work); +} + +static void lan966x_fdb_port_event_work(struct lan966x_fdb_event_work *fdb_work) { - struct lan966x_fdb_event_work *fdb_work = - container_of(work, struct lan966x_fdb_event_work, work); struct switchdev_notifier_fdb_info *fdb_info; - struct net_device *dev = fdb_work->dev; struct lan966x_port *port; struct lan966x *lan966x; - int ret; - fdb_info = &fdb_work->fdb_info; lan966x = fdb_work->lan966x; + port = netdev_priv(fdb_work->orig_dev); + fdb_info = &fdb_work->fdb_info; - if (lan966x_netdevice_check(dev)) { - port = netdev_priv(dev); + switch (fdb_work->event) { + case SWITCHDEV_FDB_ADD_TO_DEVICE: + if (!fdb_info->added_by_user) + break; + lan966x_mac_add_entry(lan966x, port, fdb_info->addr, + fdb_info->vid); + break; + case SWITCHDEV_FDB_DEL_TO_DEVICE: + if (!fdb_info->added_by_user) + break; + lan966x_mac_del_entry(lan966x, fdb_info->addr, + fdb_info->vid); + break; + } +} + +static void lan966x_fdb_bridge_event_work(struct lan966x_fdb_event_work *fdb_work) +{ + struct switchdev_notifier_fdb_info *fdb_info; + struct lan966x *lan966x; + int ret; - switch (fdb_work->event) { - case SWITCHDEV_FDB_ADD_TO_DEVICE: - if (!fdb_info->added_by_user) - break; - lan966x_mac_add_entry(lan966x, port, fdb_info->addr, - fdb_info->vid); + lan966x = fdb_work->lan966x; + fdb_info = &fdb_work->fdb_info; + + /* In case the bridge is called */ + switch (fdb_work->event) { + case SWITCHDEV_FDB_ADD_TO_DEVICE: + /* If there is no front port in this vlan, there is no + * point to copy the frame to CPU because it would be + * just dropped at later point. So add it only if + * there is a port but it is required to store the fdb + * entry for later point when a port actually gets in + * the vlan. + */ + lan966x_fdb_add_entry(lan966x, fdb_info); + if (!lan966x_vlan_cpu_member_cpu_vlan_mask(lan966x, + fdb_info->vid)) break; - case SWITCHDEV_FDB_DEL_TO_DEVICE: - if (!fdb_info->added_by_user) - break; - lan966x_mac_del_entry(lan966x, fdb_info->addr, - fdb_info->vid); + + lan966x_mac_cpu_learn(lan966x, fdb_info->addr, + fdb_info->vid); + break; + case SWITCHDEV_FDB_DEL_TO_DEVICE: + ret = lan966x_fdb_del_entry(lan966x, fdb_info); + if (!lan966x_vlan_cpu_member_cpu_vlan_mask(lan966x, + fdb_info->vid)) break; - } - } else { - if (!netif_is_bridge_master(dev)) - goto out; - - /* In case the bridge is called */ - switch (fdb_work->event) { - case SWITCHDEV_FDB_ADD_TO_DEVICE: - /* If there is no front port in this vlan, there is no - * point to copy the frame to CPU because it would be - * just dropped at later point. So add it only if - * there is a port but it is required to store the fdb - * entry for later point when a port actually gets in - * the vlan. - */ - lan966x_fdb_add_entry(lan966x, fdb_info); - if (!lan966x_vlan_cpu_member_cpu_vlan_mask(lan966x, - fdb_info->vid)) - break; - - lan966x_mac_cpu_learn(lan966x, fdb_info->addr, - fdb_info->vid); + + if (ret) + lan966x_mac_cpu_forget(lan966x, fdb_info->addr, + fdb_info->vid); + break; + } +} + +static void lan966x_fdb_lag_event_work(struct lan966x_fdb_event_work *fdb_work) +{ + struct switchdev_notifier_fdb_info *fdb_info; + struct lan966x_port *port; + struct lan966x *lan966x; + + if (!lan966x_lag_first_port(fdb_work->orig_dev, fdb_work->dev)) + return; + + lan966x = fdb_work->lan966x; + port = netdev_priv(fdb_work->dev); + fdb_info = &fdb_work->fdb_info; + + switch (fdb_work->event) { + case SWITCHDEV_FDB_ADD_TO_DEVICE: + if (!fdb_info->added_by_user) break; - case SWITCHDEV_FDB_DEL_TO_DEVICE: - ret = lan966x_fdb_del_entry(lan966x, fdb_info); - if (!lan966x_vlan_cpu_member_cpu_vlan_mask(lan966x, - fdb_info->vid)) - break; - - if (ret) - lan966x_mac_cpu_forget(lan966x, fdb_info->addr, - fdb_info->vid); + lan966x_mac_add_entry(lan966x, port, fdb_info->addr, + fdb_info->vid); + break; + case SWITCHDEV_FDB_DEL_TO_DEVICE: + if (!fdb_info->added_by_user) break; - } + lan966x_mac_del_entry(lan966x, fdb_info->addr, fdb_info->vid); + break; } +} + +static void lan966x_fdb_event_work(struct work_struct *work) +{ + struct lan966x_fdb_event_work *fdb_work = + container_of(work, struct lan966x_fdb_event_work, work); + + if (lan966x_netdevice_check(fdb_work->orig_dev)) + lan966x_fdb_port_event_work(fdb_work); + else if (netif_is_bridge_master(fdb_work->orig_dev)) + lan966x_fdb_bridge_event_work(fdb_work); + else if (netif_is_lag_master(fdb_work->orig_dev)) + lan966x_fdb_lag_event_work(fdb_work); -out: kfree(fdb_work->fdb_info.addr); kfree(fdb_work); - dev_put(dev); } int lan966x_handle_fdb(struct net_device *dev, @@ -221,7 +266,8 @@ int lan966x_handle_fdb(struct net_device *dev, if (!fdb_work) return -ENOMEM; - fdb_work->dev = orig_dev; + fdb_work->dev = dev; + fdb_work->orig_dev = orig_dev; fdb_work->lan966x = lan966x; fdb_work->event = event; INIT_WORK(&fdb_work->work, lan966x_fdb_event_work); @@ -231,7 +277,6 @@ int lan966x_handle_fdb(struct net_device *dev, goto err_addr_alloc; ether_addr_copy((u8 *)fdb_work->fdb_info.addr, fdb_info->addr); - dev_hold(orig_dev); queue_work(lan966x->fdb_work, &fdb_work->work); break; diff --git a/drivers/net/ethernet/microchip/lan966x/lan966x_fdma.c b/drivers/net/ethernet/microchip/lan966x/lan966x_fdma.c index 51f8a0816377..7e4061c854f0 100644 --- a/drivers/net/ethernet/microchip/lan966x/lan966x_fdma.c +++ b/drivers/net/ethernet/microchip/lan966x/lan966x_fdma.c @@ -787,8 +787,7 @@ void lan966x_fdma_netdev_init(struct lan966x *lan966x, struct net_device *dev) return; lan966x->fdma_ndev = dev; - netif_napi_add(dev, &lan966x->napi, lan966x_fdma_napi_poll, - NAPI_POLL_WEIGHT); + netif_napi_add(dev, &lan966x->napi, lan966x_fdma_napi_poll); napi_enable(&lan966x->napi); } diff --git a/drivers/net/ethernet/microchip/lan966x/lan966x_lag.c b/drivers/net/ethernet/microchip/lan966x/lan966x_lag.c new file mode 100644 index 000000000000..41fa2523d91d --- /dev/null +++ b/drivers/net/ethernet/microchip/lan966x/lan966x_lag.c @@ -0,0 +1,363 @@ +// SPDX-License-Identifier: GPL-2.0+ + +#include <linux/if_bridge.h> + +#include "lan966x_main.h" + +static void lan966x_lag_set_aggr_pgids(struct lan966x *lan966x) +{ + u32 visited = GENMASK(lan966x->num_phys_ports - 1, 0); + int p, lag, i; + + /* Reset destination and aggregation PGIDS */ + for (p = 0; p < lan966x->num_phys_ports; ++p) + lan_wr(ANA_PGID_PGID_SET(BIT(p)), + lan966x, ANA_PGID(p)); + + for (p = PGID_AGGR; p < PGID_SRC; ++p) + lan_wr(ANA_PGID_PGID_SET(visited), + lan966x, ANA_PGID(p)); + + /* The visited ports bitmask holds the list of ports offloading any + * bonding interface. Initially we mark all these ports as unvisited, + * then every time we visit a port in this bitmask, we know that it is + * the lowest numbered port, i.e. the one whose logical ID == physical + * port ID == LAG ID. So we mark as visited all further ports in the + * bitmask that are offloading the same bonding interface. This way, + * we set up the aggregation PGIDs only once per bonding interface. + */ + for (p = 0; p < lan966x->num_phys_ports; ++p) { + struct lan966x_port *port = lan966x->ports[p]; + + if (!port || !port->bond) + continue; + + visited &= ~BIT(p); + } + + /* Now, set PGIDs for each active LAG */ + for (lag = 0; lag < lan966x->num_phys_ports; ++lag) { + struct net_device *bond = lan966x->ports[lag]->bond; + int num_active_ports = 0; + unsigned long bond_mask; + u8 aggr_idx[16]; + + if (!bond || (visited & BIT(lag))) + continue; + + bond_mask = lan966x_lag_get_mask(lan966x, bond); + + for_each_set_bit(p, &bond_mask, lan966x->num_phys_ports) { + struct lan966x_port *port = lan966x->ports[p]; + + lan_wr(ANA_PGID_PGID_SET(bond_mask), + lan966x, ANA_PGID(p)); + if (port->lag_tx_active) + aggr_idx[num_active_ports++] = p; + } + + for (i = PGID_AGGR; i < PGID_SRC; ++i) { + u32 ac; + + ac = lan_rd(lan966x, ANA_PGID(i)); + ac &= ~bond_mask; + /* Don't do division by zero if there was no active + * port. Just make all aggregation codes zero. + */ + if (num_active_ports) + ac |= BIT(aggr_idx[i % num_active_ports]); + lan_wr(ANA_PGID_PGID_SET(ac), + lan966x, ANA_PGID(i)); + } + + /* Mark all ports in the same LAG as visited to avoid applying + * the same config again. + */ + for (p = lag; p < lan966x->num_phys_ports; p++) { + struct lan966x_port *port = lan966x->ports[p]; + + if (!port) + continue; + + if (port->bond == bond) + visited |= BIT(p); + } + } +} + +static void lan966x_lag_set_port_ids(struct lan966x *lan966x) +{ + struct lan966x_port *port; + u32 bond_mask; + u32 lag_id; + int p; + + for (p = 0; p < lan966x->num_phys_ports; ++p) { + port = lan966x->ports[p]; + if (!port) + continue; + + lag_id = port->chip_port; + + bond_mask = lan966x_lag_get_mask(lan966x, port->bond); + if (bond_mask) + lag_id = __ffs(bond_mask); + + lan_rmw(ANA_PORT_CFG_PORTID_VAL_SET(lag_id), + ANA_PORT_CFG_PORTID_VAL, + lan966x, ANA_PORT_CFG(port->chip_port)); + } +} + +static void lan966x_lag_update_ids(struct lan966x *lan966x) +{ + lan966x_lag_set_port_ids(lan966x); + lan966x_update_fwd_mask(lan966x); + lan966x_lag_set_aggr_pgids(lan966x); +} + +int lan966x_lag_port_join(struct lan966x_port *port, + struct net_device *brport_dev, + struct net_device *bond, + struct netlink_ext_ack *extack) +{ + struct lan966x *lan966x = port->lan966x; + struct net_device *dev = port->dev; + u32 lag_id = -1; + u32 bond_mask; + int err; + + bond_mask = lan966x_lag_get_mask(lan966x, bond); + if (bond_mask) + lag_id = __ffs(bond_mask); + + port->bond = bond; + lan966x_lag_update_ids(lan966x); + + err = switchdev_bridge_port_offload(brport_dev, dev, port, + &lan966x_switchdev_nb, + &lan966x_switchdev_blocking_nb, + false, extack); + if (err) + goto out; + + lan966x_port_stp_state_set(port, br_port_get_stp_state(brport_dev)); + + if (lan966x_lag_first_port(port->bond, port->dev) && + lag_id != -1) + lan966x_mac_lag_replace_port_entry(lan966x, + lan966x->ports[lag_id], + port); + + return 0; + +out: + port->bond = NULL; + lan966x_lag_update_ids(lan966x); + + return err; +} + +void lan966x_lag_port_leave(struct lan966x_port *port, struct net_device *bond) +{ + struct lan966x *lan966x = port->lan966x; + u32 bond_mask; + u32 lag_id; + + if (lan966x_lag_first_port(port->bond, port->dev)) { + bond_mask = lan966x_lag_get_mask(lan966x, port->bond); + bond_mask &= ~BIT(port->chip_port); + if (bond_mask) { + lag_id = __ffs(bond_mask); + lan966x_mac_lag_replace_port_entry(lan966x, port, + lan966x->ports[lag_id]); + } else { + lan966x_mac_lag_remove_port_entry(lan966x, port); + } + } + + port->bond = NULL; + lan966x_lag_update_ids(lan966x); + lan966x_port_stp_state_set(port, BR_STATE_FORWARDING); +} + +static bool lan966x_lag_port_check_hash_types(struct lan966x *lan966x, + enum netdev_lag_hash hash_type) +{ + int p; + + for (p = 0; p < lan966x->num_phys_ports; ++p) { + struct lan966x_port *port = lan966x->ports[p]; + + if (!port || !port->bond) + continue; + + if (port->hash_type != hash_type) + return false; + } + + return true; +} + +int lan966x_lag_port_prechangeupper(struct net_device *dev, + struct netdev_notifier_changeupper_info *info) +{ + struct lan966x_port *port = netdev_priv(dev); + struct lan966x *lan966x = port->lan966x; + struct netdev_lag_upper_info *lui; + struct netlink_ext_ack *extack; + + extack = netdev_notifier_info_to_extack(&info->info); + lui = info->upper_info; + if (!lui) { + port->hash_type = NETDEV_LAG_HASH_NONE; + return NOTIFY_DONE; + } + + if (lui->tx_type != NETDEV_LAG_TX_TYPE_HASH) { + NL_SET_ERR_MSG_MOD(extack, + "LAG device using unsupported Tx type"); + return -EINVAL; + } + + if (!lan966x_lag_port_check_hash_types(lan966x, lui->hash_type)) { + NL_SET_ERR_MSG_MOD(extack, + "LAG devices can have only the same hash_type"); + return -EINVAL; + } + + switch (lui->hash_type) { + case NETDEV_LAG_HASH_L2: + lan_wr(ANA_AGGR_CFG_AC_DMAC_ENA_SET(1) | + ANA_AGGR_CFG_AC_SMAC_ENA_SET(1), + lan966x, ANA_AGGR_CFG); + break; + case NETDEV_LAG_HASH_L34: + lan_wr(ANA_AGGR_CFG_AC_IP6_TCPUDP_ENA_SET(1) | + ANA_AGGR_CFG_AC_IP4_TCPUDP_ENA_SET(1) | + ANA_AGGR_CFG_AC_IP4_SIPDIP_ENA_SET(1), + lan966x, ANA_AGGR_CFG); + break; + case NETDEV_LAG_HASH_L23: + lan_wr(ANA_AGGR_CFG_AC_DMAC_ENA_SET(1) | + ANA_AGGR_CFG_AC_SMAC_ENA_SET(1) | + ANA_AGGR_CFG_AC_IP6_TCPUDP_ENA_SET(1) | + ANA_AGGR_CFG_AC_IP4_TCPUDP_ENA_SET(1), + lan966x, ANA_AGGR_CFG); + break; + default: + NL_SET_ERR_MSG_MOD(extack, + "LAG device using unsupported hash type"); + return -EINVAL; + } + + port->hash_type = lui->hash_type; + + return NOTIFY_OK; +} + +int lan966x_lag_port_changelowerstate(struct net_device *dev, + struct netdev_notifier_changelowerstate_info *info) +{ + struct netdev_lag_lower_state_info *lag = info->lower_state_info; + struct lan966x_port *port = netdev_priv(dev); + struct lan966x *lan966x = port->lan966x; + bool is_active; + + if (!port->bond) + return NOTIFY_DONE; + + is_active = lag->link_up && lag->tx_enabled; + if (port->lag_tx_active == is_active) + return NOTIFY_DONE; + + port->lag_tx_active = is_active; + lan966x_lag_set_aggr_pgids(lan966x); + + return NOTIFY_OK; +} + +int lan966x_lag_netdev_prechangeupper(struct net_device *dev, + struct netdev_notifier_changeupper_info *info) +{ + struct lan966x_port *port; + struct net_device *lower; + struct list_head *iter; + int err; + + netdev_for_each_lower_dev(dev, lower, iter) { + if (!lan966x_netdevice_check(lower)) + continue; + + port = netdev_priv(lower); + if (port->bond != dev) + continue; + + err = lan966x_port_prechangeupper(lower, dev, info); + if (err) + return err; + } + + return NOTIFY_DONE; +} + +int lan966x_lag_netdev_changeupper(struct net_device *dev, + struct netdev_notifier_changeupper_info *info) +{ + struct lan966x_port *port; + struct net_device *lower; + struct list_head *iter; + int err; + + netdev_for_each_lower_dev(dev, lower, iter) { + if (!lan966x_netdevice_check(lower)) + continue; + + port = netdev_priv(lower); + if (port->bond != dev) + continue; + + err = lan966x_port_changeupper(lower, dev, info); + if (err) + return err; + } + + return NOTIFY_DONE; +} + +bool lan966x_lag_first_port(struct net_device *lag, struct net_device *dev) +{ + struct lan966x_port *port = netdev_priv(dev); + struct lan966x *lan966x = port->lan966x; + unsigned long bond_mask; + + if (port->bond != lag) + return false; + + bond_mask = lan966x_lag_get_mask(lan966x, lag); + if (bond_mask && port->chip_port == __ffs(bond_mask)) + return true; + + return false; +} + +u32 lan966x_lag_get_mask(struct lan966x *lan966x, struct net_device *bond) +{ + struct lan966x_port *port; + u32 mask = 0; + int p; + + if (!bond) + return mask; + + for (p = 0; p < lan966x->num_phys_ports; p++) { + port = lan966x->ports[p]; + if (!port) + continue; + + if (port->bond == bond) + mask |= BIT(p); + } + + return mask; +} diff --git a/drivers/net/ethernet/microchip/lan966x/lan966x_mac.c b/drivers/net/ethernet/microchip/lan966x/lan966x_mac.c index 5893770bfd94..baa3a30c039f 100644 --- a/drivers/net/ethernet/microchip/lan966x/lan966x_mac.c +++ b/drivers/net/ethernet/microchip/lan966x/lan966x_mac.c @@ -22,6 +22,7 @@ struct lan966x_mac_entry { u16 vid; u16 port_index; int row; + bool lag; }; struct lan966x_mac_raw_entry { @@ -69,15 +70,14 @@ static void lan966x_mac_select(struct lan966x *lan966x, lan_wr(mach, lan966x, ANA_MACHDATA); } -static int __lan966x_mac_learn(struct lan966x *lan966x, int pgid, - bool cpu_copy, - const unsigned char mac[ETH_ALEN], - unsigned int vid, - enum macaccess_entry_type type) +static int __lan966x_mac_learn_locked(struct lan966x *lan966x, int pgid, + bool cpu_copy, + const unsigned char mac[ETH_ALEN], + unsigned int vid, + enum macaccess_entry_type type) { - int ret; + lockdep_assert_held(&lan966x->mac_lock); - spin_lock(&lan966x->mac_lock); lan966x_mac_select(lan966x, mac, vid); /* Issue a write command */ @@ -89,7 +89,19 @@ static int __lan966x_mac_learn(struct lan966x *lan966x, int pgid, ANA_MACACCESS_MAC_TABLE_CMD_SET(MACACCESS_CMD_LEARN), lan966x, ANA_MACACCESS); - ret = lan966x_mac_wait_for_completion(lan966x); + return lan966x_mac_wait_for_completion(lan966x); +} + +static int __lan966x_mac_learn(struct lan966x *lan966x, int pgid, + bool cpu_copy, + const unsigned char mac[ETH_ALEN], + unsigned int vid, + enum macaccess_entry_type type) +{ + int ret; + + spin_lock(&lan966x->mac_lock); + ret = __lan966x_mac_learn_locked(lan966x, pgid, cpu_copy, mac, vid, type); spin_unlock(&lan966x->mac_lock); return ret; @@ -119,6 +131,16 @@ int lan966x_mac_learn(struct lan966x *lan966x, int port, return __lan966x_mac_learn(lan966x, port, false, mac, vid, type); } +static int lan966x_mac_learn_locked(struct lan966x *lan966x, int port, + const unsigned char mac[ETH_ALEN], + unsigned int vid, + enum macaccess_entry_type type) +{ + WARN_ON(type != ENTRYTYPE_NORMAL && type != ENTRYTYPE_LOCKED); + + return __lan966x_mac_learn_locked(lan966x, port, false, mac, vid, type); +} + static int lan966x_mac_forget_locked(struct lan966x *lan966x, const unsigned char mac[ETH_ALEN], unsigned int vid, @@ -178,8 +200,9 @@ void lan966x_mac_init(struct lan966x *lan966x) INIT_LIST_HEAD(&lan966x->mac_entries); } -static struct lan966x_mac_entry *lan966x_mac_alloc_entry(const unsigned char *mac, - u16 vid, u16 port_index) +static struct lan966x_mac_entry *lan966x_mac_alloc_entry(struct lan966x_port *port, + const unsigned char *mac, + u16 vid) { struct lan966x_mac_entry *mac_entry; @@ -189,8 +212,9 @@ static struct lan966x_mac_entry *lan966x_mac_alloc_entry(const unsigned char *ma memcpy(mac_entry->mac, mac, ETH_ALEN); mac_entry->vid = vid; - mac_entry->port_index = port_index; + mac_entry->port_index = port->chip_port; mac_entry->row = LAN966X_MAC_INVALID_ROW; + mac_entry->lag = port->bond ? true : false; return mac_entry; } @@ -269,7 +293,7 @@ int lan966x_mac_add_entry(struct lan966x *lan966x, struct lan966x_port *port, goto mac_learn; } - mac_entry = lan966x_mac_alloc_entry(addr, vid, port->chip_port); + mac_entry = lan966x_mac_alloc_entry(port, addr, vid); if (!mac_entry) { spin_unlock(&lan966x->mac_lock); return -ENOMEM; @@ -278,7 +302,8 @@ int lan966x_mac_add_entry(struct lan966x *lan966x, struct lan966x_port *port, list_add_tail(&mac_entry->list, &lan966x->mac_entries); spin_unlock(&lan966x->mac_lock); - lan966x_fdb_call_notifiers(SWITCHDEV_FDB_OFFLOADED, addr, vid, port->dev); + lan966x_fdb_call_notifiers(SWITCHDEV_FDB_OFFLOADED, addr, vid, + port->bond ?: port->dev); mac_learn: lan966x_mac_learn(lan966x, port->chip_port, addr, vid, ENTRYTYPE_LOCKED); @@ -309,6 +334,50 @@ int lan966x_mac_del_entry(struct lan966x *lan966x, const unsigned char *addr, return 0; } +void lan966x_mac_lag_replace_port_entry(struct lan966x *lan966x, + struct lan966x_port *src, + struct lan966x_port *dst) +{ + struct lan966x_mac_entry *mac_entry; + + spin_lock(&lan966x->mac_lock); + list_for_each_entry(mac_entry, &lan966x->mac_entries, list) { + if (mac_entry->port_index == src->chip_port && + mac_entry->lag) { + lan966x_mac_forget_locked(lan966x, mac_entry->mac, + mac_entry->vid, + ENTRYTYPE_LOCKED); + + lan966x_mac_learn_locked(lan966x, dst->chip_port, + mac_entry->mac, mac_entry->vid, + ENTRYTYPE_LOCKED); + mac_entry->port_index = dst->chip_port; + } + } + spin_unlock(&lan966x->mac_lock); +} + +void lan966x_mac_lag_remove_port_entry(struct lan966x *lan966x, + struct lan966x_port *src) +{ + struct lan966x_mac_entry *mac_entry, *tmp; + + spin_lock(&lan966x->mac_lock); + list_for_each_entry_safe(mac_entry, tmp, &lan966x->mac_entries, + list) { + if (mac_entry->port_index == src->chip_port && + mac_entry->lag) { + lan966x_mac_forget_locked(lan966x, mac_entry->mac, + mac_entry->vid, + ENTRYTYPE_LOCKED); + + list_del(&mac_entry->list); + kfree(mac_entry); + } + } + spin_unlock(&lan966x->mac_lock); +} + void lan966x_mac_purge_entries(struct lan966x *lan966x) { struct lan966x_mac_entry *mac_entry, *tmp; @@ -354,6 +423,7 @@ static void lan966x_mac_irq_process(struct lan966x *lan966x, u32 row, struct lan966x_mac_entry *mac_entry, *tmp; unsigned char mac[ETH_ALEN] __aligned(2); struct list_head mac_deleted_entries; + struct lan966x_port *port; u32 dest_idx; u32 column; u16 vid; @@ -406,9 +476,10 @@ static void lan966x_mac_irq_process(struct lan966x *lan966x, u32 row, /* Notify the bridge that the entry doesn't exist * anymore in the HW */ + port = lan966x->ports[mac_entry->port_index]; lan966x_mac_notifiers(SWITCHDEV_FDB_DEL_TO_BRIDGE, mac_entry->mac, mac_entry->vid, - lan966x->ports[mac_entry->port_index]->dev); + port->bond ?: port->dev); list_del(&mac_entry->list); kfree(mac_entry); } @@ -440,7 +511,8 @@ static void lan966x_mac_irq_process(struct lan966x *lan966x, u32 row, continue; } - mac_entry = lan966x_mac_alloc_entry(mac, vid, dest_idx); + port = lan966x->ports[dest_idx]; + mac_entry = lan966x_mac_alloc_entry(port, mac, vid); if (!mac_entry) { spin_unlock(&lan966x->mac_lock); return; @@ -451,7 +523,7 @@ static void lan966x_mac_irq_process(struct lan966x *lan966x, u32 row, spin_unlock(&lan966x->mac_lock); lan966x_mac_notifiers(SWITCHDEV_FDB_ADD_TO_BRIDGE, - mac, vid, lan966x->ports[dest_idx]->dev); + mac, vid, port->bond ?: port->dev); } } diff --git a/drivers/net/ethernet/microchip/lan966x/lan966x_main.c b/drivers/net/ethernet/microchip/lan966x/lan966x_main.c index d928b75f3780..be2fd030cccb 100644 --- a/drivers/net/ethernet/microchip/lan966x/lan966x_main.c +++ b/drivers/net/ethernet/microchip/lan966x/lan966x_main.c @@ -344,7 +344,8 @@ static void lan966x_ifh_set_timestamp(void *ifh, u64 timestamp) IFH_POS_TIMESTAMP, IFH_LEN * 4, PACK, 0); } -static int lan966x_port_xmit(struct sk_buff *skb, struct net_device *dev) +static netdev_tx_t lan966x_port_xmit(struct sk_buff *skb, + struct net_device *dev) { struct lan966x_port *port = netdev_priv(dev); struct lan966x *lan966x = port->lan966x; @@ -466,6 +467,7 @@ static const struct net_device_ops lan966x_port_netdev_ops = { .ndo_set_mac_address = lan966x_port_set_mac_address, .ndo_get_port_parent_id = lan966x_port_get_parent_id, .ndo_eth_ioctl = lan966x_port_ioctl, + .ndo_setup_tc = lan966x_tc_setup, }; bool lan966x_netdevice_check(const struct net_device *dev) @@ -738,7 +740,8 @@ static int lan966x_probe_port(struct lan966x *lan966x, u32 p, return -EINVAL; dev = devm_alloc_etherdev_mqs(lan966x->dev, - sizeof(struct lan966x_port), 8, 1); + sizeof(struct lan966x_port), + NUM_PRIO_QUEUES, 1); if (!dev) return -ENOMEM; @@ -754,7 +757,9 @@ static int lan966x_probe_port(struct lan966x *lan966x, u32 p, dev->netdev_ops = &lan966x_port_netdev_ops; dev->ethtool_ops = &lan966x_ethtool_ops; dev->features |= NETIF_F_HW_VLAN_CTAG_TX | - NETIF_F_HW_VLAN_STAG_TX; + NETIF_F_HW_VLAN_STAG_TX | + NETIF_F_HW_TC; + dev->hw_features |= NETIF_F_HW_TC; dev->needed_headroom = IFH_LEN * sizeof(u32); eth_hw_addr_gen(dev, lan966x->base_mac, p + 1); @@ -770,6 +775,7 @@ static int lan966x_probe_port(struct lan966x *lan966x, u32 p, port->phylink_config.mac_capabilities = MAC_ASYM_PAUSE | MAC_SYM_PAUSE | MAC_10 | MAC_100 | MAC_1000FD | MAC_2500FD; + phy_interface_set_rgmii(port->phylink_config.supported_interfaces); __set_bit(PHY_INTERFACE_MODE_MII, port->phylink_config.supported_interfaces); __set_bit(PHY_INTERFACE_MODE_GMII, @@ -778,6 +784,8 @@ static int lan966x_probe_port(struct lan966x *lan966x, u32 p, port->phylink_config.supported_interfaces); __set_bit(PHY_INTERFACE_MODE_QSGMII, port->phylink_config.supported_interfaces); + __set_bit(PHY_INTERFACE_MODE_QUSGMII, + port->phylink_config.supported_interfaces); __set_bit(PHY_INTERFACE_MODE_1000BASEX, port->phylink_config.supported_interfaces); __set_bit(PHY_INTERFACE_MODE_2500BASEX, @@ -956,6 +964,8 @@ static void lan966x_init(struct lan966x *lan966x) lan966x, ANA_ANAINTR); spin_lock_init(&lan966x->tx_lock); + + lan966x_taprio_init(lan966x); } static int lan966x_ram_init(struct lan966x *lan966x) @@ -969,7 +979,8 @@ static int lan966x_reset_switch(struct lan966x *lan966x) int val = 0; int ret; - switch_reset = devm_reset_control_get_shared(lan966x->dev, "switch"); + switch_reset = devm_reset_control_get_optional_shared(lan966x->dev, + "switch"); if (IS_ERR(switch_reset)) return dev_err_probe(lan966x->dev, PTR_ERR(switch_reset), "Could not obtain switch reset"); @@ -1164,6 +1175,7 @@ static int lan966x_remove(struct platform_device *pdev) { struct lan966x *lan966x = platform_get_drvdata(pdev); + lan966x_taprio_deinit(lan966x); lan966x_fdma_deinit(lan966x); lan966x_cleanup_ports(lan966x); diff --git a/drivers/net/ethernet/microchip/lan966x/lan966x_main.h b/drivers/net/ethernet/microchip/lan966x/lan966x_main.h index 2787055c1847..9656071b8289 100644 --- a/drivers/net/ethernet/microchip/lan966x/lan966x_main.h +++ b/drivers/net/ethernet/microchip/lan966x/lan966x_main.h @@ -9,6 +9,8 @@ #include <linux/phy.h> #include <linux/phylink.h> #include <linux/ptp_clock_kernel.h> +#include <net/pkt_cls.h> +#include <net/pkt_sched.h> #include <net/switchdev.h> #include "lan966x_regs.h" @@ -36,6 +38,7 @@ #define NUM_PHYS_PORTS 8 #define CPU_PORT 8 +#define NUM_PRIO_QUEUES 8 /* Reserved PGIDs */ #define PGID_CPU (PGID_AGGR - 6) @@ -79,6 +82,9 @@ #define FDMA_INJ_CHANNEL 0 #define FDMA_DCB_MAX 512 +#define SE_IDX_QUEUE 0 /* 0-79 : Queue scheduler elements */ +#define SE_IDX_PORT 80 /* 80-89 : Port schedular elements */ + /* MAC table entry types. * ENTRYTYPE_NORMAL is subject to aging. * ENTRYTYPE_LOCKED is not subject to aging. @@ -258,6 +264,11 @@ struct lan966x { struct lan966x_rx rx; struct lan966x_tx tx; struct napi_struct napi; + + /* Mirror */ + struct lan966x_port *mirror_monitor; + u32 mirror_mask[2]; + u32 mirror_count; }; struct lan966x_port_config { @@ -270,6 +281,15 @@ struct lan966x_port_config { bool autoneg; }; +struct lan966x_port_tc { + bool ingress_shared_block; + unsigned long police_id; + unsigned long ingress_mirror_id; + unsigned long egress_mirror_id; + struct flow_stats police_stat; + struct flow_stats mirror_stat; +}; + struct lan966x_port { struct net_device *dev; struct lan966x *lan966x; @@ -292,11 +312,19 @@ struct lan966x_port { u8 ptp_cmd; u16 ts_id; struct sk_buff_head tx_skbs; + + struct net_device *bond; + bool lag_tx_active; + enum netdev_lag_hash hash_type; + + struct lan966x_port_tc tc; }; extern const struct phylink_mac_ops lan966x_phylink_mac_ops; extern const struct phylink_pcs_ops lan966x_phylink_pcs_ops; extern const struct ethtool_ops lan966x_ethtool_ops; +extern struct notifier_block lan966x_switchdev_nb __read_mostly; +extern struct notifier_block lan966x_switchdev_blocking_nb __read_mostly; bool lan966x_netdevice_check(const struct net_device *dev); @@ -345,6 +373,11 @@ int lan966x_mac_add_entry(struct lan966x *lan966x, struct lan966x_port *port, const unsigned char *addr, u16 vid); +void lan966x_mac_lag_replace_port_entry(struct lan966x *lan966x, + struct lan966x_port *src, + struct lan966x_port *dst); +void lan966x_mac_lag_remove_port_entry(struct lan966x *lan966x, + struct lan966x_port *src); void lan966x_mac_purge_entries(struct lan966x *lan966x); irqreturn_t lan966x_mac_irq_handler(struct lan966x *lan966x); @@ -369,6 +402,7 @@ void lan966x_fdb_write_entries(struct lan966x *lan966x, u16 vid); void lan966x_fdb_erase_entries(struct lan966x *lan966x, u16 vid); int lan966x_fdb_init(struct lan966x *lan966x); void lan966x_fdb_deinit(struct lan966x *lan966x); +void lan966x_fdb_flush_workqueue(struct lan966x *lan966x); int lan966x_handle_fdb(struct net_device *dev, struct net_device *orig_dev, unsigned long event, const void *ctx, @@ -397,6 +431,8 @@ void lan966x_ptp_txtstamp_release(struct lan966x_port *port, struct sk_buff *skb); irqreturn_t lan966x_ptp_irq_handler(int irq, void *args); irqreturn_t lan966x_ptp_ext_irq_handler(int irq, void *args); +u32 lan966x_ptp_get_period_ps(void); +int lan966x_ptp_gettime64(struct ptp_clock_info *ptp, struct timespec64 *ts); int lan966x_fdma_xmit(struct sk_buff *skb, __be32 *ifh, struct net_device *dev); int lan966x_fdma_change_mtu(struct lan966x *lan966x); @@ -406,6 +442,89 @@ int lan966x_fdma_init(struct lan966x *lan966x); void lan966x_fdma_deinit(struct lan966x *lan966x); irqreturn_t lan966x_fdma_irq_handler(int irq, void *args); +int lan966x_lag_port_join(struct lan966x_port *port, + struct net_device *brport_dev, + struct net_device *bond, + struct netlink_ext_ack *extack); +void lan966x_lag_port_leave(struct lan966x_port *port, struct net_device *bond); +int lan966x_lag_port_prechangeupper(struct net_device *dev, + struct netdev_notifier_changeupper_info *info); +int lan966x_lag_port_changelowerstate(struct net_device *dev, + struct netdev_notifier_changelowerstate_info *info); +int lan966x_lag_netdev_prechangeupper(struct net_device *dev, + struct netdev_notifier_changeupper_info *info); +int lan966x_lag_netdev_changeupper(struct net_device *dev, + struct netdev_notifier_changeupper_info *info); +bool lan966x_lag_first_port(struct net_device *lag, struct net_device *dev); +u32 lan966x_lag_get_mask(struct lan966x *lan966x, struct net_device *bond); + +int lan966x_port_changeupper(struct net_device *dev, + struct net_device *brport_dev, + struct netdev_notifier_changeupper_info *info); +int lan966x_port_prechangeupper(struct net_device *dev, + struct net_device *brport_dev, + struct netdev_notifier_changeupper_info *info); +void lan966x_port_stp_state_set(struct lan966x_port *port, u8 state); +void lan966x_port_ageing_set(struct lan966x_port *port, + unsigned long ageing_clock_t); +void lan966x_update_fwd_mask(struct lan966x *lan966x); + +int lan966x_tc_setup(struct net_device *dev, enum tc_setup_type type, + void *type_data); + +int lan966x_mqprio_add(struct lan966x_port *port, u8 num_tc); +int lan966x_mqprio_del(struct lan966x_port *port); + +void lan966x_taprio_init(struct lan966x *lan966x); +void lan966x_taprio_deinit(struct lan966x *lan966x); +int lan966x_taprio_add(struct lan966x_port *port, + struct tc_taprio_qopt_offload *qopt); +int lan966x_taprio_del(struct lan966x_port *port); +int lan966x_taprio_speed_set(struct lan966x_port *port, int speed); + +int lan966x_tbf_add(struct lan966x_port *port, + struct tc_tbf_qopt_offload *qopt); +int lan966x_tbf_del(struct lan966x_port *port, + struct tc_tbf_qopt_offload *qopt); + +int lan966x_cbs_add(struct lan966x_port *port, + struct tc_cbs_qopt_offload *qopt); +int lan966x_cbs_del(struct lan966x_port *port, + struct tc_cbs_qopt_offload *qopt); + +int lan966x_ets_add(struct lan966x_port *port, + struct tc_ets_qopt_offload *qopt); +int lan966x_ets_del(struct lan966x_port *port, + struct tc_ets_qopt_offload *qopt); + +int lan966x_tc_matchall(struct lan966x_port *port, + struct tc_cls_matchall_offload *f, + bool ingress); + +int lan966x_police_port_add(struct lan966x_port *port, + struct flow_action *action, + struct flow_action_entry *act, + unsigned long police_id, + bool ingress, + struct netlink_ext_ack *extack); +int lan966x_police_port_del(struct lan966x_port *port, + unsigned long police_id, + struct netlink_ext_ack *extack); +void lan966x_police_port_stats(struct lan966x_port *port, + struct flow_stats *stats); + +int lan966x_mirror_port_add(struct lan966x_port *port, + struct flow_action_entry *action, + unsigned long mirror_id, + bool ingress, + struct netlink_ext_ack *extack); +int lan966x_mirror_port_del(struct lan966x_port *port, + bool ingress, + struct netlink_ext_ack *extack); +void lan966x_mirror_port_stats(struct lan966x_port *port, + struct flow_stats *stats, + bool ingress); + static inline void __iomem *lan_addr(void __iomem *base[], int id, int tinst, int tcnt, int gbase, int ginst, diff --git a/drivers/net/ethernet/microchip/lan966x/lan966x_mirror.c b/drivers/net/ethernet/microchip/lan966x/lan966x_mirror.c new file mode 100644 index 000000000000..7e1ba3f40c35 --- /dev/null +++ b/drivers/net/ethernet/microchip/lan966x/lan966x_mirror.c @@ -0,0 +1,138 @@ +// SPDX-License-Identifier: GPL-2.0+ + +#include "lan966x_main.h" + +int lan966x_mirror_port_add(struct lan966x_port *port, + struct flow_action_entry *action, + unsigned long mirror_id, + bool ingress, + struct netlink_ext_ack *extack) +{ + struct lan966x *lan966x = port->lan966x; + struct lan966x_port *monitor_port; + + if (!lan966x_netdevice_check(action->dev)) { + NL_SET_ERR_MSG_MOD(extack, + "Destination not an lan966x port"); + return -EOPNOTSUPP; + } + + monitor_port = netdev_priv(action->dev); + + if (lan966x->mirror_mask[ingress] & BIT(port->chip_port)) { + NL_SET_ERR_MSG_MOD(extack, + "Mirror already exists"); + return -EEXIST; + } + + if (lan966x->mirror_monitor && + lan966x->mirror_monitor != monitor_port) { + NL_SET_ERR_MSG_MOD(extack, + "Cannot change mirror port while in use"); + return -EBUSY; + } + + if (port == monitor_port) { + NL_SET_ERR_MSG_MOD(extack, + "Cannot mirror the monitor port"); + return -EINVAL; + } + + lan966x->mirror_mask[ingress] |= BIT(port->chip_port); + + lan966x->mirror_monitor = monitor_port; + lan_wr(BIT(monitor_port->chip_port), lan966x, ANA_MIRRORPORTS); + + if (ingress) { + lan_rmw(ANA_PORT_CFG_SRC_MIRROR_ENA_SET(1), + ANA_PORT_CFG_SRC_MIRROR_ENA, + lan966x, ANA_PORT_CFG(port->chip_port)); + } else { + lan_wr(lan966x->mirror_mask[0], lan966x, + ANA_EMIRRORPORTS); + } + + lan966x->mirror_count++; + + if (ingress) + port->tc.ingress_mirror_id = mirror_id; + else + port->tc.egress_mirror_id = mirror_id; + + return 0; +} + +int lan966x_mirror_port_del(struct lan966x_port *port, + bool ingress, + struct netlink_ext_ack *extack) +{ + struct lan966x *lan966x = port->lan966x; + + if (!(lan966x->mirror_mask[ingress] & BIT(port->chip_port))) { + NL_SET_ERR_MSG_MOD(extack, + "There is no mirroring for this port"); + return -ENOENT; + } + + lan966x->mirror_mask[ingress] &= ~BIT(port->chip_port); + + if (ingress) { + lan_rmw(ANA_PORT_CFG_SRC_MIRROR_ENA_SET(0), + ANA_PORT_CFG_SRC_MIRROR_ENA, + lan966x, ANA_PORT_CFG(port->chip_port)); + } else { + lan_wr(lan966x->mirror_mask[0], lan966x, + ANA_EMIRRORPORTS); + } + + lan966x->mirror_count--; + + if (lan966x->mirror_count == 0) { + lan966x->mirror_monitor = NULL; + lan_wr(0, lan966x, ANA_MIRRORPORTS); + } + + if (ingress) + port->tc.ingress_mirror_id = 0; + else + port->tc.egress_mirror_id = 0; + + return 0; +} + +void lan966x_mirror_port_stats(struct lan966x_port *port, + struct flow_stats *stats, + bool ingress) +{ + struct rtnl_link_stats64 new_stats; + struct flow_stats *old_stats; + + old_stats = &port->tc.mirror_stat; + lan966x_stats_get(port->dev, &new_stats); + + if (ingress) { + flow_stats_update(stats, + new_stats.rx_bytes - old_stats->bytes, + new_stats.rx_packets - old_stats->pkts, + new_stats.rx_dropped - old_stats->drops, + old_stats->lastused, + FLOW_ACTION_HW_STATS_IMMEDIATE); + + old_stats->bytes = new_stats.rx_bytes; + old_stats->pkts = new_stats.rx_packets; + old_stats->drops = new_stats.rx_dropped; + old_stats->lastused = jiffies; + } else { + flow_stats_update(stats, + new_stats.tx_bytes - old_stats->bytes, + new_stats.tx_packets - old_stats->pkts, + new_stats.tx_dropped - old_stats->drops, + old_stats->lastused, + FLOW_ACTION_HW_STATS_IMMEDIATE); + + old_stats->bytes = new_stats.tx_bytes; + old_stats->pkts = new_stats.tx_packets; + old_stats->drops = new_stats.tx_dropped; + old_stats->lastused = jiffies; + } +} diff --git a/drivers/net/ethernet/microchip/lan966x/lan966x_mqprio.c b/drivers/net/ethernet/microchip/lan966x/lan966x_mqprio.c new file mode 100644 index 000000000000..7fa76e74f9e2 --- /dev/null +++ b/drivers/net/ethernet/microchip/lan966x/lan966x_mqprio.c @@ -0,0 +1,28 @@ +// SPDX-License-Identifier: GPL-2.0+ + +#include "lan966x_main.h" + +int lan966x_mqprio_add(struct lan966x_port *port, u8 num_tc) +{ + u8 i; + + if (num_tc != NUM_PRIO_QUEUES) { + netdev_err(port->dev, "Only %d traffic classes supported\n", + NUM_PRIO_QUEUES); + return -EINVAL; + } + + netdev_set_num_tc(port->dev, num_tc); + + for (i = 0; i < num_tc; ++i) + netdev_set_tc_queue(port->dev, i, 1, i); + + return 0; +} + +int lan966x_mqprio_del(struct lan966x_port *port) +{ + netdev_reset_tc(port->dev); + + return 0; +} diff --git a/drivers/net/ethernet/microchip/lan966x/lan966x_phylink.c b/drivers/net/ethernet/microchip/lan966x/lan966x_phylink.c index 38a7e95d69b4..e4ac59480514 100644 --- a/drivers/net/ethernet/microchip/lan966x/lan966x_phylink.c +++ b/drivers/net/ethernet/microchip/lan966x/lan966x_phylink.c @@ -28,11 +28,12 @@ static int lan966x_phylink_mac_prepare(struct phylink_config *config, phy_interface_t iface) { struct lan966x_port *port = netdev_priv(to_net_dev(config->dev)); + phy_interface_t serdes_mode = iface; int err; if (port->serdes) { err = phy_set_mode_ext(port->serdes, PHY_MODE_ETHERNET, - iface); + serdes_mode); if (err) { netdev_err(to_net_dev(config->dev), "Could not set mode of SerDes\n"); @@ -59,6 +60,9 @@ static void lan966x_phylink_mac_link_up(struct phylink_config *config, port_config->pause |= tx_pause ? MLO_PAUSE_TX : 0; port_config->pause |= rx_pause ? MLO_PAUSE_RX : 0; + if (phy_interface_mode_is_rgmii(interface)) + phy_set_speed(port->serdes, speed); + lan966x_port_config_up(port); } diff --git a/drivers/net/ethernet/microchip/lan966x/lan966x_police.c b/drivers/net/ethernet/microchip/lan966x/lan966x_police.c new file mode 100644 index 000000000000..a9aec900d608 --- /dev/null +++ b/drivers/net/ethernet/microchip/lan966x/lan966x_police.c @@ -0,0 +1,235 @@ +// SPDX-License-Identifier: GPL-2.0+ + +#include "lan966x_main.h" + +/* 0-8 : 9 port policers */ +#define POL_IDX_PORT 0 + +/* Policer order: Serial (QoS -> Port -> VCAP) */ +#define POL_ORDER 0x1d3 + +struct lan966x_tc_policer { + /* kilobit per second */ + u32 rate; + /* bytes */ + u32 burst; +}; + +static int lan966x_police_add(struct lan966x_port *port, + struct lan966x_tc_policer *pol, + u16 pol_idx) +{ + struct lan966x *lan966x = port->lan966x; + + /* Rate unit is 33 1/3 kpps */ + pol->rate = DIV_ROUND_UP(pol->rate * 3, 100); + /* Avoid zero burst size */ + pol->burst = pol->burst ?: 1; + /* Unit is 4kB */ + pol->burst = DIV_ROUND_UP(pol->burst, 4096); + + if (pol->rate > GENMASK(15, 0) || + pol->burst > GENMASK(6, 0)) + return -EINVAL; + + lan_wr(ANA_POL_MODE_DROP_ON_YELLOW_ENA_SET(0) | + ANA_POL_MODE_MARK_ALL_FRMS_RED_ENA_SET(0) | + ANA_POL_MODE_IPG_SIZE_SET(20) | + ANA_POL_MODE_FRM_MODE_SET(1) | + ANA_POL_MODE_OVERSHOOT_ENA_SET(1), + lan966x, ANA_POL_MODE(pol_idx)); + + lan_wr(ANA_POL_PIR_STATE_PIR_LVL_SET(0), + lan966x, ANA_POL_PIR_STATE(pol_idx)); + + lan_wr(ANA_POL_PIR_CFG_PIR_RATE_SET(pol->rate) | + ANA_POL_PIR_CFG_PIR_BURST_SET(pol->burst), + lan966x, ANA_POL_PIR_CFG(pol_idx)); + + return 0; +} + +static int lan966x_police_del(struct lan966x_port *port, + u16 pol_idx) +{ + struct lan966x *lan966x = port->lan966x; + + lan_wr(ANA_POL_MODE_DROP_ON_YELLOW_ENA_SET(0) | + ANA_POL_MODE_MARK_ALL_FRMS_RED_ENA_SET(0) | + ANA_POL_MODE_IPG_SIZE_SET(20) | + ANA_POL_MODE_FRM_MODE_SET(2) | + ANA_POL_MODE_OVERSHOOT_ENA_SET(1), + lan966x, ANA_POL_MODE(pol_idx)); + + lan_wr(ANA_POL_PIR_STATE_PIR_LVL_SET(0), + lan966x, ANA_POL_PIR_STATE(pol_idx)); + + lan_wr(ANA_POL_PIR_CFG_PIR_RATE_SET(GENMASK(14, 0)) | + ANA_POL_PIR_CFG_PIR_BURST_SET(0), + lan966x, ANA_POL_PIR_CFG(pol_idx)); + + return 0; +} + +static int lan966x_police_validate(struct lan966x_port *port, + const struct flow_action *action, + const struct flow_action_entry *act, + unsigned long police_id, + bool ingress, + struct netlink_ext_ack *extack) +{ + if (act->police.exceed.act_id != FLOW_ACTION_DROP) { + NL_SET_ERR_MSG_MOD(extack, + "Offload not supported when exceed action is not drop"); + return -EOPNOTSUPP; + } + + if (act->police.notexceed.act_id != FLOW_ACTION_PIPE && + act->police.notexceed.act_id != FLOW_ACTION_ACCEPT) { + NL_SET_ERR_MSG_MOD(extack, + "Offload not supported when conform action is not pipe or ok"); + return -EOPNOTSUPP; + } + + if (act->police.notexceed.act_id == FLOW_ACTION_ACCEPT && + !flow_action_is_last_entry(action, act)) { + NL_SET_ERR_MSG_MOD(extack, + "Offload not supported when conform action is ok, but action is not last"); + return -EOPNOTSUPP; + } + + if (act->police.peakrate_bytes_ps || + act->police.avrate || act->police.overhead) { + NL_SET_ERR_MSG_MOD(extack, + "Offload not supported when peakrate/avrate/overhead is configured"); + return -EOPNOTSUPP; + } + + if (act->police.rate_pkt_ps) { + NL_SET_ERR_MSG_MOD(extack, + "QoS offload not support packets per second"); + return -EOPNOTSUPP; + } + + if (!ingress) { + NL_SET_ERR_MSG_MOD(extack, + "Policer is not supported on egress"); + return -EOPNOTSUPP; + } + + if (port->tc.ingress_shared_block) { + NL_SET_ERR_MSG_MOD(extack, + "Policer is not supported on shared ingress blocks"); + return -EOPNOTSUPP; + } + + if (port->tc.police_id && port->tc.police_id != police_id) { + NL_SET_ERR_MSG_MOD(extack, + "Only one policer per port is supported"); + return -EEXIST; + } + + return 0; +} + +int lan966x_police_port_add(struct lan966x_port *port, + struct flow_action *action, + struct flow_action_entry *act, + unsigned long police_id, + bool ingress, + struct netlink_ext_ack *extack) +{ + struct lan966x *lan966x = port->lan966x; + struct rtnl_link_stats64 new_stats; + struct lan966x_tc_policer pol; + struct flow_stats *old_stats; + int err; + + err = lan966x_police_validate(port, action, act, police_id, ingress, + extack); + if (err) + return err; + + memset(&pol, 0, sizeof(pol)); + + pol.rate = div_u64(act->police.rate_bytes_ps, 1000) * 8; + pol.burst = act->police.burst; + + err = lan966x_police_add(port, &pol, POL_IDX_PORT + port->chip_port); + if (err) { + NL_SET_ERR_MSG_MOD(extack, + "Failed to add policer to port"); + return err; + } + + lan_rmw(ANA_POL_CFG_PORT_POL_ENA_SET(1) | + ANA_POL_CFG_POL_ORDER_SET(POL_ORDER), + ANA_POL_CFG_PORT_POL_ENA | + ANA_POL_CFG_POL_ORDER, + lan966x, ANA_POL_CFG(port->chip_port)); + + port->tc.police_id = police_id; + + /* Setup initial stats */ + old_stats = &port->tc.police_stat; + lan966x_stats_get(port->dev, &new_stats); + old_stats->bytes = new_stats.rx_bytes; + old_stats->pkts = new_stats.rx_packets; + old_stats->drops = new_stats.rx_dropped; + old_stats->lastused = jiffies; + + return 0; +} + +int lan966x_police_port_del(struct lan966x_port *port, + unsigned long police_id, + struct netlink_ext_ack *extack) +{ + struct lan966x *lan966x = port->lan966x; + int err; + + if (port->tc.police_id != police_id) { + NL_SET_ERR_MSG_MOD(extack, + "Invalid policer id"); + return -EINVAL; + } + + err = lan966x_police_del(port, port->tc.police_id); + if (err) { + NL_SET_ERR_MSG_MOD(extack, + "Failed to add policer to port"); + return err; + } + + lan_rmw(ANA_POL_CFG_PORT_POL_ENA_SET(0) | + ANA_POL_CFG_POL_ORDER_SET(POL_ORDER), + ANA_POL_CFG_PORT_POL_ENA | + ANA_POL_CFG_POL_ORDER, + lan966x, ANA_POL_CFG(port->chip_port)); + + port->tc.police_id = 0; + + return 0; +} + +void lan966x_police_port_stats(struct lan966x_port *port, + struct flow_stats *stats) +{ + struct rtnl_link_stats64 new_stats; + struct flow_stats *old_stats; + + old_stats = &port->tc.police_stat; + lan966x_stats_get(port->dev, &new_stats); + + flow_stats_update(stats, + new_stats.rx_bytes - old_stats->bytes, + new_stats.rx_packets - old_stats->pkts, + new_stats.rx_dropped - old_stats->drops, + old_stats->lastused, + FLOW_ACTION_HW_STATS_IMMEDIATE); + + old_stats->bytes = new_stats.rx_bytes; + old_stats->pkts = new_stats.rx_packets; + old_stats->drops = new_stats.rx_dropped; + old_stats->lastused = jiffies; +} diff --git a/drivers/net/ethernet/microchip/lan966x/lan966x_port.c b/drivers/net/ethernet/microchip/lan966x/lan966x_port.c index f141644e4372..1a61c6cdb077 100644 --- a/drivers/net/ethernet/microchip/lan966x/lan966x_port.c +++ b/drivers/net/ethernet/microchip/lan966x/lan966x_port.c @@ -165,10 +165,12 @@ static void lan966x_port_link_up(struct lan966x_port *port) break; } + lan966x_taprio_speed_set(port, config->speed); + /* Also the GIGA_MODE_ENA(1) needs to be set regardless of the * port speed for QSGMII ports. */ - if (config->portmode == PHY_INTERFACE_MODE_QSGMII) + if (phy_interface_num_ports(config->portmode) == 4) mode = DEV_MAC_MODE_CFG_GIGA_MODE_ENA_SET(1); lan_wr(config->duplex | mode, @@ -331,10 +333,14 @@ int lan966x_port_pcs_set(struct lan966x_port *port, struct lan966x *lan966x = port->lan966x; bool inband_aneg = false; bool outband; + bool full_preamble = false; + + if (config->portmode == PHY_INTERFACE_MODE_QUSGMII) + full_preamble = true; if (config->inband) { if (config->portmode == PHY_INTERFACE_MODE_SGMII || - config->portmode == PHY_INTERFACE_MODE_QSGMII) + phy_interface_num_ports(config->portmode) == 4) inband_aneg = true; /* Cisco-SGMII in-band-aneg */ else if (config->portmode == PHY_INTERFACE_MODE_1000BASEX && config->autoneg) @@ -345,9 +351,15 @@ int lan966x_port_pcs_set(struct lan966x_port *port, outband = true; } - /* Disable or enable inband */ - lan_rmw(DEV_PCS1G_MODE_CFG_SGMII_MODE_ENA_SET(outband), - DEV_PCS1G_MODE_CFG_SGMII_MODE_ENA, + /* Disable or enable inband. + * For QUSGMII, we rely on the preamble to transmit data such as + * timestamps, therefore force full preamble transmission, and prevent + * premable shortening + */ + lan_rmw(DEV_PCS1G_MODE_CFG_SGMII_MODE_ENA_SET(outband) | + DEV_PCS1G_MODE_CFG_SAVE_PREAMBLE_ENA_SET(full_preamble), + DEV_PCS1G_MODE_CFG_SGMII_MODE_ENA | + DEV_PCS1G_MODE_CFG_SAVE_PREAMBLE_ENA, lan966x, DEV_PCS1G_MODE_CFG(port->chip_port)); /* Enable PCS */ @@ -396,7 +408,7 @@ void lan966x_port_init(struct lan966x_port *port) if (lan966x->fdma) lan966x_fdma_netdev_init(lan966x, port->dev); - if (config->portmode != PHY_INTERFACE_MODE_QSGMII) + if (phy_interface_num_ports(config->portmode) != 4) return; lan_rmw(DEV_CLOCK_CFG_PCS_RX_RST_SET(0) | diff --git a/drivers/net/ethernet/microchip/lan966x/lan966x_ptp.c b/drivers/net/ethernet/microchip/lan966x/lan966x_ptp.c index 3a621c5165bc..e5a2bbe064f8 100644 --- a/drivers/net/ethernet/microchip/lan966x/lan966x_ptp.c +++ b/drivers/net/ethernet/microchip/lan966x/lan966x_ptp.c @@ -464,8 +464,7 @@ static int lan966x_ptp_settime64(struct ptp_clock_info *ptp, return 0; } -static int lan966x_ptp_gettime64(struct ptp_clock_info *ptp, - struct timespec64 *ts) +int lan966x_ptp_gettime64(struct ptp_clock_info *ptp, struct timespec64 *ts) { struct lan966x_phc *phc = container_of(ptp, struct lan966x_phc, info); struct lan966x *lan966x = phc->lan966x; @@ -890,3 +889,9 @@ void lan966x_ptp_rxtstamp(struct lan966x *lan966x, struct sk_buff *skb, shhwtstamps = skb_hwtstamps(skb); shhwtstamps->hwtstamp = full_ts_in_ns; } + +u32 lan966x_ptp_get_period_ps(void) +{ + /* This represents the system clock period in picoseconds */ + return 15125; +} diff --git a/drivers/net/ethernet/microchip/lan966x/lan966x_regs.h b/drivers/net/ethernet/microchip/lan966x/lan966x_regs.h index 8265ad89f0bc..1d90b93dd417 100644 --- a/drivers/net/ethernet/microchip/lan966x/lan966x_regs.h +++ b/drivers/net/ethernet/microchip/lan966x/lan966x_regs.h @@ -90,6 +90,24 @@ enum lan966x_target { #define ANA_AUTOAGE_AGE_PERIOD_GET(x)\ FIELD_GET(ANA_AUTOAGE_AGE_PERIOD, x) +/* ANA:ANA:MIRRORPORTS */ +#define ANA_MIRRORPORTS __REG(TARGET_ANA, 0, 1, 29824, 0, 1, 244, 60, 0, 1, 4) + +#define ANA_MIRRORPORTS_MIRRORPORTS GENMASK(8, 0) +#define ANA_MIRRORPORTS_MIRRORPORTS_SET(x)\ + FIELD_PREP(ANA_MIRRORPORTS_MIRRORPORTS, x) +#define ANA_MIRRORPORTS_MIRRORPORTS_GET(x)\ + FIELD_GET(ANA_MIRRORPORTS_MIRRORPORTS, x) + +/* ANA:ANA:EMIRRORPORTS */ +#define ANA_EMIRRORPORTS __REG(TARGET_ANA, 0, 1, 29824, 0, 1, 244, 64, 0, 1, 4) + +#define ANA_EMIRRORPORTS_EMIRRORPORTS GENMASK(8, 0) +#define ANA_EMIRRORPORTS_EMIRRORPORTS_SET(x)\ + FIELD_PREP(ANA_EMIRRORPORTS_EMIRRORPORTS, x) +#define ANA_EMIRRORPORTS_EMIRRORPORTS_GET(x)\ + FIELD_GET(ANA_EMIRRORPORTS_EMIRRORPORTS, x) + /* ANA:ANA:FLOODING */ #define ANA_FLOODING(r) __REG(TARGET_ANA, 0, 1, 29824, 0, 1, 244, 68, r, 8, 4) @@ -330,6 +348,12 @@ enum lan966x_target { /* ANA:PORT:PORT_CFG */ #define ANA_PORT_CFG(g) __REG(TARGET_ANA, 0, 1, 28672, g, 9, 128, 112, 0, 1, 4) +#define ANA_PORT_CFG_SRC_MIRROR_ENA BIT(13) +#define ANA_PORT_CFG_SRC_MIRROR_ENA_SET(x)\ + FIELD_PREP(ANA_PORT_CFG_SRC_MIRROR_ENA, x) +#define ANA_PORT_CFG_SRC_MIRROR_ENA_GET(x)\ + FIELD_GET(ANA_PORT_CFG_SRC_MIRROR_ENA, x) + #define ANA_PORT_CFG_LEARNAUTO BIT(6) #define ANA_PORT_CFG_LEARNAUTO_SET(x)\ FIELD_PREP(ANA_PORT_CFG_LEARNAUTO, x) @@ -354,6 +378,21 @@ enum lan966x_target { #define ANA_PORT_CFG_PORTID_VAL_GET(x)\ FIELD_GET(ANA_PORT_CFG_PORTID_VAL, x) +/* ANA:PORT:POL_CFG */ +#define ANA_POL_CFG(g) __REG(TARGET_ANA, 0, 1, 28672, g, 9, 128, 116, 0, 1, 4) + +#define ANA_POL_CFG_PORT_POL_ENA BIT(17) +#define ANA_POL_CFG_PORT_POL_ENA_SET(x)\ + FIELD_PREP(ANA_POL_CFG_PORT_POL_ENA, x) +#define ANA_POL_CFG_PORT_POL_ENA_GET(x)\ + FIELD_GET(ANA_POL_CFG_PORT_POL_ENA, x) + +#define ANA_POL_CFG_POL_ORDER GENMASK(8, 0) +#define ANA_POL_CFG_POL_ORDER_SET(x)\ + FIELD_PREP(ANA_POL_CFG_POL_ORDER, x) +#define ANA_POL_CFG_POL_ORDER_GET(x)\ + FIELD_GET(ANA_POL_CFG_POL_ORDER, x) + /* ANA:PFC:PFC_CFG */ #define ANA_PFC_CFG(g) __REG(TARGET_ANA, 0, 1, 30720, g, 8, 64, 0, 0, 1, 4) @@ -363,6 +402,108 @@ enum lan966x_target { #define ANA_PFC_CFG_FC_LINK_SPEED_GET(x)\ FIELD_GET(ANA_PFC_CFG_FC_LINK_SPEED, x) +/* ANA:COMMON:AGGR_CFG */ +#define ANA_AGGR_CFG __REG(TARGET_ANA, 0, 1, 31232, 0, 1, 552, 0, 0, 1, 4) + +#define ANA_AGGR_CFG_AC_RND_ENA BIT(6) +#define ANA_AGGR_CFG_AC_RND_ENA_SET(x)\ + FIELD_PREP(ANA_AGGR_CFG_AC_RND_ENA, x) +#define ANA_AGGR_CFG_AC_RND_ENA_GET(x)\ + FIELD_GET(ANA_AGGR_CFG_AC_RND_ENA, x) + +#define ANA_AGGR_CFG_AC_DMAC_ENA BIT(5) +#define ANA_AGGR_CFG_AC_DMAC_ENA_SET(x)\ + FIELD_PREP(ANA_AGGR_CFG_AC_DMAC_ENA, x) +#define ANA_AGGR_CFG_AC_DMAC_ENA_GET(x)\ + FIELD_GET(ANA_AGGR_CFG_AC_DMAC_ENA, x) + +#define ANA_AGGR_CFG_AC_SMAC_ENA BIT(4) +#define ANA_AGGR_CFG_AC_SMAC_ENA_SET(x)\ + FIELD_PREP(ANA_AGGR_CFG_AC_SMAC_ENA, x) +#define ANA_AGGR_CFG_AC_SMAC_ENA_GET(x)\ + FIELD_GET(ANA_AGGR_CFG_AC_SMAC_ENA, x) + +#define ANA_AGGR_CFG_AC_IP6_FLOW_LBL_ENA BIT(3) +#define ANA_AGGR_CFG_AC_IP6_FLOW_LBL_ENA_SET(x)\ + FIELD_PREP(ANA_AGGR_CFG_AC_IP6_FLOW_LBL_ENA, x) +#define ANA_AGGR_CFG_AC_IP6_FLOW_LBL_ENA_GET(x)\ + FIELD_GET(ANA_AGGR_CFG_AC_IP6_FLOW_LBL_ENA, x) + +#define ANA_AGGR_CFG_AC_IP6_TCPUDP_ENA BIT(2) +#define ANA_AGGR_CFG_AC_IP6_TCPUDP_ENA_SET(x)\ + FIELD_PREP(ANA_AGGR_CFG_AC_IP6_TCPUDP_ENA, x) +#define ANA_AGGR_CFG_AC_IP6_TCPUDP_ENA_GET(x)\ + FIELD_GET(ANA_AGGR_CFG_AC_IP6_TCPUDP_ENA, x) + +#define ANA_AGGR_CFG_AC_IP4_SIPDIP_ENA BIT(1) +#define ANA_AGGR_CFG_AC_IP4_SIPDIP_ENA_SET(x)\ + FIELD_PREP(ANA_AGGR_CFG_AC_IP4_SIPDIP_ENA, x) +#define ANA_AGGR_CFG_AC_IP4_SIPDIP_ENA_GET(x)\ + FIELD_GET(ANA_AGGR_CFG_AC_IP4_SIPDIP_ENA, x) + +#define ANA_AGGR_CFG_AC_IP4_TCPUDP_ENA BIT(0) +#define ANA_AGGR_CFG_AC_IP4_TCPUDP_ENA_SET(x)\ + FIELD_PREP(ANA_AGGR_CFG_AC_IP4_TCPUDP_ENA, x) +#define ANA_AGGR_CFG_AC_IP4_TCPUDP_ENA_GET(x)\ + FIELD_GET(ANA_AGGR_CFG_AC_IP4_TCPUDP_ENA, x) + +/* ANA:POL:POL_PIR_CFG */ +#define ANA_POL_PIR_CFG(g) __REG(TARGET_ANA, 0, 1, 16384, g, 345, 32, 0, 0, 1, 4) + +#define ANA_POL_PIR_CFG_PIR_RATE GENMASK(20, 6) +#define ANA_POL_PIR_CFG_PIR_RATE_SET(x)\ + FIELD_PREP(ANA_POL_PIR_CFG_PIR_RATE, x) +#define ANA_POL_PIR_CFG_PIR_RATE_GET(x)\ + FIELD_GET(ANA_POL_PIR_CFG_PIR_RATE, x) + +#define ANA_POL_PIR_CFG_PIR_BURST GENMASK(5, 0) +#define ANA_POL_PIR_CFG_PIR_BURST_SET(x)\ + FIELD_PREP(ANA_POL_PIR_CFG_PIR_BURST, x) +#define ANA_POL_PIR_CFG_PIR_BURST_GET(x)\ + FIELD_GET(ANA_POL_PIR_CFG_PIR_BURST, x) + +/* ANA:POL:POL_MODE_CFG */ +#define ANA_POL_MODE(g) __REG(TARGET_ANA, 0, 1, 16384, g, 345, 32, 8, 0, 1, 4) + +#define ANA_POL_MODE_DROP_ON_YELLOW_ENA BIT(11) +#define ANA_POL_MODE_DROP_ON_YELLOW_ENA_SET(x)\ + FIELD_PREP(ANA_POL_MODE_DROP_ON_YELLOW_ENA, x) +#define ANA_POL_MODE_DROP_ON_YELLOW_ENA_GET(x)\ + FIELD_GET(ANA_POL_MODE_DROP_ON_YELLOW_ENA, x) + +#define ANA_POL_MODE_MARK_ALL_FRMS_RED_ENA BIT(10) +#define ANA_POL_MODE_MARK_ALL_FRMS_RED_ENA_SET(x)\ + FIELD_PREP(ANA_POL_MODE_MARK_ALL_FRMS_RED_ENA, x) +#define ANA_POL_MODE_MARK_ALL_FRMS_RED_ENA_GET(x)\ + FIELD_GET(ANA_POL_MODE_MARK_ALL_FRMS_RED_ENA, x) + +#define ANA_POL_MODE_IPG_SIZE GENMASK(9, 5) +#define ANA_POL_MODE_IPG_SIZE_SET(x)\ + FIELD_PREP(ANA_POL_MODE_IPG_SIZE, x) +#define ANA_POL_MODE_IPG_SIZE_GET(x)\ + FIELD_GET(ANA_POL_MODE_IPG_SIZE, x) + +#define ANA_POL_MODE_FRM_MODE GENMASK(4, 3) +#define ANA_POL_MODE_FRM_MODE_SET(x)\ + FIELD_PREP(ANA_POL_MODE_FRM_MODE, x) +#define ANA_POL_MODE_FRM_MODE_GET(x)\ + FIELD_GET(ANA_POL_MODE_FRM_MODE, x) + +#define ANA_POL_MODE_OVERSHOOT_ENA BIT(0) +#define ANA_POL_MODE_OVERSHOOT_ENA_SET(x)\ + FIELD_PREP(ANA_POL_MODE_OVERSHOOT_ENA, x) +#define ANA_POL_MODE_OVERSHOOT_ENA_GET(x)\ + FIELD_GET(ANA_POL_MODE_OVERSHOOT_ENA, x) + +/* ANA:POL:POL_PIR_STATE */ +#define ANA_POL_PIR_STATE(g) __REG(TARGET_ANA, 0, 1, 16384, g, 345, 32, 12, 0, 1, 4) + +#define ANA_POL_PIR_STATE_PIR_LVL GENMASK(21, 0) +#define ANA_POL_PIR_STATE_PIR_LVL_SET(x)\ + FIELD_PREP(ANA_POL_PIR_STATE_PIR_LVL, x) +#define ANA_POL_PIR_STATE_PIR_LVL_GET(x)\ + FIELD_GET(ANA_POL_PIR_STATE_PIR_LVL, x) + /* CHIP_TOP:CUPHY_CFG:CUPHY_PORT_CFG */ #define CHIP_TOP_CUPHY_PORT_CFG(r) __REG(TARGET_CHIP_TOP, 0, 1, 16, 0, 1, 20, 8, r, 2, 4) @@ -504,6 +645,12 @@ enum lan966x_target { #define DEV_PCS1G_MODE_CFG_SGMII_MODE_ENA_GET(x)\ FIELD_GET(DEV_PCS1G_MODE_CFG_SGMII_MODE_ENA, x) +#define DEV_PCS1G_MODE_CFG_SAVE_PREAMBLE_ENA BIT(1) +#define DEV_PCS1G_MODE_CFG_SAVE_PREAMBLE_ENA_SET(x)\ + FIELD_PREP(DEV_PCS1G_MODE_CFG_SAVE_PREAMBLE_ENA, x) +#define DEV_PCS1G_MODE_CFG_SAVE_PREAMBLE_ENA_GET(x)\ + FIELD_GET(DEV_PCS1G_MODE_CFG_SAVE_PREAMBLE_ENA, x) + /* DEV:PCS1G_CFG_STATUS:PCS1G_SD_CFG */ #define DEV_PCS1G_SD_CFG(t) __REG(TARGET_DEV, t, 8, 72, 0, 1, 68, 8, 0, 1, 4) @@ -967,6 +1114,215 @@ enum lan966x_target { /* QSYS:RES_CTRL:RES_CFG */ #define QSYS_RES_CFG(g) __REG(TARGET_QSYS, 0, 1, 32768, g, 1024, 8, 0, 0, 1, 4) +/* QSYS:HSCH:CIR_CFG */ +#define QSYS_CIR_CFG(g) __REG(TARGET_QSYS, 0, 1, 16384, g, 90, 128, 0, 0, 1, 4) + +#define QSYS_CIR_CFG_CIR_RATE GENMASK(20, 6) +#define QSYS_CIR_CFG_CIR_RATE_SET(x)\ + FIELD_PREP(QSYS_CIR_CFG_CIR_RATE, x) +#define QSYS_CIR_CFG_CIR_RATE_GET(x)\ + FIELD_GET(QSYS_CIR_CFG_CIR_RATE, x) + +#define QSYS_CIR_CFG_CIR_BURST GENMASK(5, 0) +#define QSYS_CIR_CFG_CIR_BURST_SET(x)\ + FIELD_PREP(QSYS_CIR_CFG_CIR_BURST, x) +#define QSYS_CIR_CFG_CIR_BURST_GET(x)\ + FIELD_GET(QSYS_CIR_CFG_CIR_BURST, x) + +/* QSYS:HSCH:SE_CFG */ +#define QSYS_SE_CFG(g) __REG(TARGET_QSYS, 0, 1, 16384, g, 90, 128, 8, 0, 1, 4) + +#define QSYS_SE_CFG_SE_DWRR_CNT GENMASK(9, 6) +#define QSYS_SE_CFG_SE_DWRR_CNT_SET(x)\ + FIELD_PREP(QSYS_SE_CFG_SE_DWRR_CNT, x) +#define QSYS_SE_CFG_SE_DWRR_CNT_GET(x)\ + FIELD_GET(QSYS_SE_CFG_SE_DWRR_CNT, x) + +#define QSYS_SE_CFG_SE_RR_ENA BIT(5) +#define QSYS_SE_CFG_SE_RR_ENA_SET(x)\ + FIELD_PREP(QSYS_SE_CFG_SE_RR_ENA, x) +#define QSYS_SE_CFG_SE_RR_ENA_GET(x)\ + FIELD_GET(QSYS_SE_CFG_SE_RR_ENA, x) + +#define QSYS_SE_CFG_SE_AVB_ENA BIT(4) +#define QSYS_SE_CFG_SE_AVB_ENA_SET(x)\ + FIELD_PREP(QSYS_SE_CFG_SE_AVB_ENA, x) +#define QSYS_SE_CFG_SE_AVB_ENA_GET(x)\ + FIELD_GET(QSYS_SE_CFG_SE_AVB_ENA, x) + +#define QSYS_SE_CFG_SE_FRM_MODE GENMASK(3, 2) +#define QSYS_SE_CFG_SE_FRM_MODE_SET(x)\ + FIELD_PREP(QSYS_SE_CFG_SE_FRM_MODE, x) +#define QSYS_SE_CFG_SE_FRM_MODE_GET(x)\ + FIELD_GET(QSYS_SE_CFG_SE_FRM_MODE, x) + +#define QSYS_SE_DWRR_CFG(g, r) __REG(TARGET_QSYS, 0, 1, 16384, g, 90, 128, 12, r, 12, 4) + +#define QSYS_SE_DWRR_CFG_DWRR_COST GENMASK(4, 0) +#define QSYS_SE_DWRR_CFG_DWRR_COST_SET(x)\ + FIELD_PREP(QSYS_SE_DWRR_CFG_DWRR_COST, x) +#define QSYS_SE_DWRR_CFG_DWRR_COST_GET(x)\ + FIELD_GET(QSYS_SE_DWRR_CFG_DWRR_COST, x) + +/* QSYS:TAS_CONFIG:TAS_CFG_CTRL */ +#define QSYS_TAS_CFG_CTRL __REG(TARGET_QSYS, 0, 1, 57372, 0, 1, 12, 0, 0, 1, 4) + +#define QSYS_TAS_CFG_CTRL_LIST_NUM_MAX GENMASK(27, 23) +#define QSYS_TAS_CFG_CTRL_LIST_NUM_MAX_SET(x)\ + FIELD_PREP(QSYS_TAS_CFG_CTRL_LIST_NUM_MAX, x) +#define QSYS_TAS_CFG_CTRL_LIST_NUM_MAX_GET(x)\ + FIELD_GET(QSYS_TAS_CFG_CTRL_LIST_NUM_MAX, x) + +#define QSYS_TAS_CFG_CTRL_LIST_NUM GENMASK(22, 18) +#define QSYS_TAS_CFG_CTRL_LIST_NUM_SET(x)\ + FIELD_PREP(QSYS_TAS_CFG_CTRL_LIST_NUM, x) +#define QSYS_TAS_CFG_CTRL_LIST_NUM_GET(x)\ + FIELD_GET(QSYS_TAS_CFG_CTRL_LIST_NUM, x) + +#define QSYS_TAS_CFG_CTRL_ALWAYS_GB_SCH_Q BIT(17) +#define QSYS_TAS_CFG_CTRL_ALWAYS_GB_SCH_Q_SET(x)\ + FIELD_PREP(QSYS_TAS_CFG_CTRL_ALWAYS_GB_SCH_Q, x) +#define QSYS_TAS_CFG_CTRL_ALWAYS_GB_SCH_Q_GET(x)\ + FIELD_GET(QSYS_TAS_CFG_CTRL_ALWAYS_GB_SCH_Q, x) + +#define QSYS_TAS_CFG_CTRL_GCL_ENTRY_NUM GENMASK(16, 5) +#define QSYS_TAS_CFG_CTRL_GCL_ENTRY_NUM_SET(x)\ + FIELD_PREP(QSYS_TAS_CFG_CTRL_GCL_ENTRY_NUM, x) +#define QSYS_TAS_CFG_CTRL_GCL_ENTRY_NUM_GET(x)\ + FIELD_GET(QSYS_TAS_CFG_CTRL_GCL_ENTRY_NUM, x) + +/* QSYS:TAS_CONFIG:TAS_GATE_STATE_CTRL */ +#define QSYS_TAS_GS_CTRL __REG(TARGET_QSYS, 0, 1, 57372, 0, 1, 12, 4, 0, 1, 4) + +#define QSYS_TAS_GS_CTRL_HSCH_POS GENMASK(2, 0) +#define QSYS_TAS_GS_CTRL_HSCH_POS_SET(x)\ + FIELD_PREP(QSYS_TAS_GS_CTRL_HSCH_POS, x) +#define QSYS_TAS_GS_CTRL_HSCH_POS_GET(x)\ + FIELD_GET(QSYS_TAS_GS_CTRL_HSCH_POS, x) + +/* QSYS:TAS_CONFIG:TAS_STATEMACHINE_CFG */ +#define QSYS_TAS_STM_CFG __REG(TARGET_QSYS, 0, 1, 57372, 0, 1, 12, 8, 0, 1, 4) + +#define QSYS_TAS_STM_CFG_REVISIT_DLY GENMASK(7, 0) +#define QSYS_TAS_STM_CFG_REVISIT_DLY_SET(x)\ + FIELD_PREP(QSYS_TAS_STM_CFG_REVISIT_DLY, x) +#define QSYS_TAS_STM_CFG_REVISIT_DLY_GET(x)\ + FIELD_GET(QSYS_TAS_STM_CFG_REVISIT_DLY, x) + +/* QSYS:TAS_PROFILE_CFG:TAS_PROFILE_CONFIG */ +#define QSYS_TAS_PROFILE_CFG(g) __REG(TARGET_QSYS, 0, 1, 30720, g, 16, 64, 32, 0, 1, 4) + +#define QSYS_TAS_PROFILE_CFG_PORT_NUM GENMASK(21, 19) +#define QSYS_TAS_PROFILE_CFG_PORT_NUM_SET(x)\ + FIELD_PREP(QSYS_TAS_PROFILE_CFG_PORT_NUM, x) +#define QSYS_TAS_PROFILE_CFG_PORT_NUM_GET(x)\ + FIELD_GET(QSYS_TAS_PROFILE_CFG_PORT_NUM, x) + +#define QSYS_TAS_PROFILE_CFG_LINK_SPEED GENMASK(18, 16) +#define QSYS_TAS_PROFILE_CFG_LINK_SPEED_SET(x)\ + FIELD_PREP(QSYS_TAS_PROFILE_CFG_LINK_SPEED, x) +#define QSYS_TAS_PROFILE_CFG_LINK_SPEED_GET(x)\ + FIELD_GET(QSYS_TAS_PROFILE_CFG_LINK_SPEED, x) + +/* QSYS:TAS_LIST_CFG:TAS_BASE_TIME_NSEC */ +#define QSYS_TAS_BT_NSEC __REG(TARGET_QSYS, 0, 1, 27904, 0, 1, 64, 0, 0, 1, 4) + +#define QSYS_TAS_BT_NSEC_NSEC GENMASK(29, 0) +#define QSYS_TAS_BT_NSEC_NSEC_SET(x)\ + FIELD_PREP(QSYS_TAS_BT_NSEC_NSEC, x) +#define QSYS_TAS_BT_NSEC_NSEC_GET(x)\ + FIELD_GET(QSYS_TAS_BT_NSEC_NSEC, x) + +/* QSYS:TAS_LIST_CFG:TAS_BASE_TIME_SEC_LSB */ +#define QSYS_TAS_BT_SEC_LSB __REG(TARGET_QSYS, 0, 1, 27904, 0, 1, 64, 4, 0, 1, 4) + +/* QSYS:TAS_LIST_CFG:TAS_BASE_TIME_SEC_MSB */ +#define QSYS_TAS_BT_SEC_MSB __REG(TARGET_QSYS, 0, 1, 27904, 0, 1, 64, 8, 0, 1, 4) + +#define QSYS_TAS_BT_SEC_MSB_SEC_MSB GENMASK(15, 0) +#define QSYS_TAS_BT_SEC_MSB_SEC_MSB_SET(x)\ + FIELD_PREP(QSYS_TAS_BT_SEC_MSB_SEC_MSB, x) +#define QSYS_TAS_BT_SEC_MSB_SEC_MSB_GET(x)\ + FIELD_GET(QSYS_TAS_BT_SEC_MSB_SEC_MSB, x) + +/* QSYS:TAS_LIST_CFG:TAS_CYCLE_TIME_CFG */ +#define QSYS_TAS_CT_CFG __REG(TARGET_QSYS, 0, 1, 27904, 0, 1, 64, 24, 0, 1, 4) + +/* QSYS:TAS_LIST_CFG:TAS_STARTUP_CFG */ +#define QSYS_TAS_STARTUP_CFG __REG(TARGET_QSYS, 0, 1, 27904, 0, 1, 64, 28, 0, 1, 4) + +#define QSYS_TAS_STARTUP_CFG_OBSOLETE_IDX GENMASK(27, 23) +#define QSYS_TAS_STARTUP_CFG_OBSOLETE_IDX_SET(x)\ + FIELD_PREP(QSYS_TAS_STARTUP_CFG_OBSOLETE_IDX, x) +#define QSYS_TAS_STARTUP_CFG_OBSOLETE_IDX_GET(x)\ + FIELD_GET(QSYS_TAS_STARTUP_CFG_OBSOLETE_IDX, x) + +/* QSYS:TAS_LIST_CFG:TAS_LIST_CFG */ +#define QSYS_TAS_LIST_CFG __REG(TARGET_QSYS, 0, 1, 27904, 0, 1, 64, 32, 0, 1, 4) + +#define QSYS_TAS_LIST_CFG_LIST_BASE_ADDR GENMASK(11, 0) +#define QSYS_TAS_LIST_CFG_LIST_BASE_ADDR_SET(x)\ + FIELD_PREP(QSYS_TAS_LIST_CFG_LIST_BASE_ADDR, x) +#define QSYS_TAS_LIST_CFG_LIST_BASE_ADDR_GET(x)\ + FIELD_GET(QSYS_TAS_LIST_CFG_LIST_BASE_ADDR, x) + +/* QSYS:TAS_LIST_CFG:TAS_LIST_STATE */ +#define QSYS_TAS_LST __REG(TARGET_QSYS, 0, 1, 27904, 0, 1, 64, 36, 0, 1, 4) + +#define QSYS_TAS_LST_LIST_STATE GENMASK(2, 0) +#define QSYS_TAS_LST_LIST_STATE_SET(x)\ + FIELD_PREP(QSYS_TAS_LST_LIST_STATE, x) +#define QSYS_TAS_LST_LIST_STATE_GET(x)\ + FIELD_GET(QSYS_TAS_LST_LIST_STATE, x) + +/* QSYS:TAS_GCL_CFG:TAS_GCL_CTRL_CFG */ +#define QSYS_TAS_GCL_CT_CFG __REG(TARGET_QSYS, 0, 1, 27968, 0, 1, 16, 0, 0, 1, 4) + +#define QSYS_TAS_GCL_CT_CFG_HSCH_POS GENMASK(12, 10) +#define QSYS_TAS_GCL_CT_CFG_HSCH_POS_SET(x)\ + FIELD_PREP(QSYS_TAS_GCL_CT_CFG_HSCH_POS, x) +#define QSYS_TAS_GCL_CT_CFG_HSCH_POS_GET(x)\ + FIELD_GET(QSYS_TAS_GCL_CT_CFG_HSCH_POS, x) + +#define QSYS_TAS_GCL_CT_CFG_GATE_STATE GENMASK(9, 2) +#define QSYS_TAS_GCL_CT_CFG_GATE_STATE_SET(x)\ + FIELD_PREP(QSYS_TAS_GCL_CT_CFG_GATE_STATE, x) +#define QSYS_TAS_GCL_CT_CFG_GATE_STATE_GET(x)\ + FIELD_GET(QSYS_TAS_GCL_CT_CFG_GATE_STATE, x) + +#define QSYS_TAS_GCL_CT_CFG_OP_TYPE GENMASK(1, 0) +#define QSYS_TAS_GCL_CT_CFG_OP_TYPE_SET(x)\ + FIELD_PREP(QSYS_TAS_GCL_CT_CFG_OP_TYPE, x) +#define QSYS_TAS_GCL_CT_CFG_OP_TYPE_GET(x)\ + FIELD_GET(QSYS_TAS_GCL_CT_CFG_OP_TYPE, x) + +/* QSYS:TAS_GCL_CFG:TAS_GCL_CTRL_CFG2 */ +#define QSYS_TAS_GCL_CT_CFG2 __REG(TARGET_QSYS, 0, 1, 27968, 0, 1, 16, 4, 0, 1, 4) + +#define QSYS_TAS_GCL_CT_CFG2_PORT_PROFILE GENMASK(15, 12) +#define QSYS_TAS_GCL_CT_CFG2_PORT_PROFILE_SET(x)\ + FIELD_PREP(QSYS_TAS_GCL_CT_CFG2_PORT_PROFILE, x) +#define QSYS_TAS_GCL_CT_CFG2_PORT_PROFILE_GET(x)\ + FIELD_GET(QSYS_TAS_GCL_CT_CFG2_PORT_PROFILE, x) + +#define QSYS_TAS_GCL_CT_CFG2_NEXT_GCL GENMASK(11, 0) +#define QSYS_TAS_GCL_CT_CFG2_NEXT_GCL_SET(x)\ + FIELD_PREP(QSYS_TAS_GCL_CT_CFG2_NEXT_GCL, x) +#define QSYS_TAS_GCL_CT_CFG2_NEXT_GCL_GET(x)\ + FIELD_GET(QSYS_TAS_GCL_CT_CFG2_NEXT_GCL, x) + +/* QSYS:TAS_GCL_CFG:TAS_GCL_TIME_CFG */ +#define QSYS_TAS_GCL_TM_CFG __REG(TARGET_QSYS, 0, 1, 27968, 0, 1, 16, 8, 0, 1, 4) + +/* QSYS:HSCH_TAS_STATE:TAS_GATE_STATE */ +#define QSYS_TAS_GATE_STATE __REG(TARGET_QSYS, 0, 1, 28004, 0, 1, 4, 0, 0, 1, 4) + +#define QSYS_TAS_GATE_STATE_TAS_GATE_STATE GENMASK(7, 0) +#define QSYS_TAS_GATE_STATE_TAS_GATE_STATE_SET(x)\ + FIELD_PREP(QSYS_TAS_GATE_STATE_TAS_GATE_STATE, x) +#define QSYS_TAS_GATE_STATE_TAS_GATE_STATE_GET(x)\ + FIELD_GET(QSYS_TAS_GATE_STATE_TAS_GATE_STATE, x) + /* REW:PORT:PORT_VLAN_CFG */ #define REW_PORT_VLAN_CFG(g) __REG(TARGET_REW, 0, 1, 0, g, 10, 128, 0, 0, 1, 4) diff --git a/drivers/net/ethernet/microchip/lan966x/lan966x_switchdev.c b/drivers/net/ethernet/microchip/lan966x/lan966x_switchdev.c index df2bee678559..1c88120eb291 100644 --- a/drivers/net/ethernet/microchip/lan966x/lan966x_switchdev.c +++ b/drivers/net/ethernet/microchip/lan966x/lan966x_switchdev.c @@ -6,8 +6,6 @@ #include "lan966x_main.h" static struct notifier_block lan966x_netdevice_nb __read_mostly; -static struct notifier_block lan966x_switchdev_nb __read_mostly; -static struct notifier_block lan966x_switchdev_blocking_nb __read_mostly; static void lan966x_port_set_mcast_ip_flood(struct lan966x_port *port, u32 pgid_ip) @@ -132,7 +130,7 @@ static int lan966x_port_pre_bridge_flags(struct lan966x_port *port, return 0; } -static void lan966x_update_fwd_mask(struct lan966x *lan966x) +void lan966x_update_fwd_mask(struct lan966x *lan966x) { int i; @@ -140,9 +138,14 @@ static void lan966x_update_fwd_mask(struct lan966x *lan966x) struct lan966x_port *port = lan966x->ports[i]; unsigned long mask = 0; - if (port && lan966x->bridge_fwd_mask & BIT(i)) + if (port && lan966x->bridge_fwd_mask & BIT(i)) { mask = lan966x->bridge_fwd_mask & ~BIT(i); + if (port->bond) + mask &= ~lan966x_lag_get_mask(lan966x, + port->bond); + } + mask |= BIT(CPU_PORT); lan_wr(ANA_PGID_PGID_SET(mask), @@ -150,7 +153,7 @@ static void lan966x_update_fwd_mask(struct lan966x *lan966x) } } -static void lan966x_port_stp_state_set(struct lan966x_port *port, u8 state) +void lan966x_port_stp_state_set(struct lan966x_port *port, u8 state) { struct lan966x *lan966x = port->lan966x; bool learn_ena = false; @@ -171,8 +174,8 @@ static void lan966x_port_stp_state_set(struct lan966x_port *port, u8 state) lan966x_update_fwd_mask(lan966x); } -static void lan966x_port_ageing_set(struct lan966x_port *port, - unsigned long ageing_clock_t) +void lan966x_port_ageing_set(struct lan966x_port *port, + unsigned long ageing_clock_t) { unsigned long ageing_jiffies = clock_t_to_jiffies(ageing_clock_t); u32 ageing_time = jiffies_to_msecs(ageing_jiffies) / 1000; @@ -241,6 +244,7 @@ static int lan966x_port_attr_set(struct net_device *dev, const void *ctx, } static int lan966x_port_bridge_join(struct lan966x_port *port, + struct net_device *brport_dev, struct net_device *bridge, struct netlink_ext_ack *extack) { @@ -258,7 +262,7 @@ static int lan966x_port_bridge_join(struct lan966x_port *port, } } - err = switchdev_bridge_port_offload(dev, dev, port, + err = switchdev_bridge_port_offload(brport_dev, dev, port, &lan966x_switchdev_nb, &lan966x_switchdev_blocking_nb, false, extack); @@ -295,8 +299,9 @@ static void lan966x_port_bridge_leave(struct lan966x_port *port, lan966x_vlan_port_apply(port); } -static int lan966x_port_changeupper(struct net_device *dev, - struct netdev_notifier_changeupper_info *info) +int lan966x_port_changeupper(struct net_device *dev, + struct net_device *brport_dev, + struct netdev_notifier_changeupper_info *info) { struct lan966x_port *port = netdev_priv(dev); struct netlink_ext_ack *extack; @@ -306,44 +311,68 @@ static int lan966x_port_changeupper(struct net_device *dev, if (netif_is_bridge_master(info->upper_dev)) { if (info->linking) - err = lan966x_port_bridge_join(port, info->upper_dev, + err = lan966x_port_bridge_join(port, brport_dev, + info->upper_dev, extack); else lan966x_port_bridge_leave(port, info->upper_dev); } + if (netif_is_lag_master(info->upper_dev)) { + if (info->linking) + err = lan966x_lag_port_join(port, info->upper_dev, + info->upper_dev, + extack); + else + lan966x_lag_port_leave(port, info->upper_dev); + } + return err; } -static int lan966x_port_prechangeupper(struct net_device *dev, - struct netdev_notifier_changeupper_info *info) +int lan966x_port_prechangeupper(struct net_device *dev, + struct net_device *brport_dev, + struct netdev_notifier_changeupper_info *info) { struct lan966x_port *port = netdev_priv(dev); + int err = NOTIFY_DONE; - if (netif_is_bridge_master(info->upper_dev) && !info->linking) - switchdev_bridge_port_unoffload(port->dev, port, - NULL, NULL); + if (netif_is_bridge_master(info->upper_dev) && !info->linking) { + switchdev_bridge_port_unoffload(port->dev, port, NULL, NULL); + lan966x_fdb_flush_workqueue(port->lan966x); + } - return NOTIFY_DONE; + if (netif_is_lag_master(info->upper_dev)) { + err = lan966x_lag_port_prechangeupper(dev, info); + if (err || info->linking) + return err; + + switchdev_bridge_port_unoffload(brport_dev, port, NULL, NULL); + lan966x_fdb_flush_workqueue(port->lan966x); + } + + return err; } -static int lan966x_foreign_bridging_check(struct net_device *bridge, +static int lan966x_foreign_bridging_check(struct net_device *upper, + bool *has_foreign, + bool *seen_lan966x, struct netlink_ext_ack *extack) { struct lan966x *lan966x = NULL; - bool has_foreign = false; struct net_device *dev; struct list_head *iter; - if (!netif_is_bridge_master(bridge)) + if (!netif_is_bridge_master(upper) && + !netif_is_lag_master(upper)) return 0; - netdev_for_each_lower_dev(bridge, dev, iter) { + netdev_for_each_lower_dev(upper, dev, iter) { if (lan966x_netdevice_check(dev)) { struct lan966x_port *port = netdev_priv(dev); if (lan966x) { - /* Bridge already has at least one port of a + /* Upper already has at least one port of a * lan966x switch inside it, check that it's * the same instance of the driver. */ @@ -354,15 +383,24 @@ static int lan966x_foreign_bridging_check(struct net_device *bridge, } } else { /* This is the first lan966x port inside this - * bridge + * upper device */ lan966x = port->lan966x; + *seen_lan966x = true; } + } else if (netif_is_lag_master(dev)) { + /* Allow to have bond interfaces that have only lan966x + * devices + */ + if (lan966x_foreign_bridging_check(dev, has_foreign, + seen_lan966x, + extack)) + return -EINVAL; } else { - has_foreign = true; + *has_foreign = true; } - if (lan966x && has_foreign) { + if (*seen_lan966x && *has_foreign) { NL_SET_ERR_MSG_MOD(extack, "Bridging lan966x ports with foreign interfaces disallowed"); return -EINVAL; @@ -375,7 +413,12 @@ static int lan966x_foreign_bridging_check(struct net_device *bridge, static int lan966x_bridge_check(struct net_device *dev, struct netdev_notifier_changeupper_info *info) { + bool has_foreign = false; + bool seen_lan966x = false; + return lan966x_foreign_bridging_check(info->upper_dev, + &has_foreign, + &seen_lan966x, info->info.extack); } @@ -386,21 +429,44 @@ static int lan966x_netdevice_port_event(struct net_device *dev, int err = 0; if (!lan966x_netdevice_check(dev)) { - if (event == NETDEV_CHANGEUPPER) - return lan966x_bridge_check(dev, ptr); + switch (event) { + case NETDEV_CHANGEUPPER: + case NETDEV_PRECHANGEUPPER: + err = lan966x_bridge_check(dev, ptr); + if (err) + return err; + + if (netif_is_lag_master(dev)) { + if (event == NETDEV_CHANGEUPPER) + err = lan966x_lag_netdev_changeupper(dev, + ptr); + else + err = lan966x_lag_netdev_prechangeupper(dev, + ptr); + + return err; + } + break; + default: + return 0; + } + return 0; } switch (event) { case NETDEV_PRECHANGEUPPER: - err = lan966x_port_prechangeupper(dev, ptr); + err = lan966x_port_prechangeupper(dev, dev, ptr); break; case NETDEV_CHANGEUPPER: err = lan966x_bridge_check(dev, ptr); if (err) return err; - err = lan966x_port_changeupper(dev, ptr); + err = lan966x_port_changeupper(dev, dev, ptr); + break; + case NETDEV_CHANGELOWERSTATE: + err = lan966x_lag_port_changelowerstate(dev, ptr); break; } @@ -418,19 +484,23 @@ static int lan966x_netdevice_event(struct notifier_block *nb, return notifier_from_errno(ret); } -/* We don't offload uppers such as LAG as bridge ports, so every device except - * the bridge itself is foreign. - */ static bool lan966x_foreign_dev_check(const struct net_device *dev, const struct net_device *foreign_dev) { struct lan966x_port *port = netdev_priv(dev); struct lan966x *lan966x = port->lan966x; + int i; if (netif_is_bridge_master(foreign_dev)) if (lan966x->bridge == foreign_dev) return false; + if (netif_is_lag_master(foreign_dev)) + for (i = 0; i < lan966x->num_phys_ports; ++i) + if (lan966x->ports[i] && + lan966x->ports[i]->bond == foreign_dev) + return false; + return true; } @@ -571,11 +641,11 @@ static struct notifier_block lan966x_netdevice_nb __read_mostly = { .notifier_call = lan966x_netdevice_event, }; -static struct notifier_block lan966x_switchdev_nb __read_mostly = { +struct notifier_block lan966x_switchdev_nb __read_mostly = { .notifier_call = lan966x_switchdev_event, }; -static struct notifier_block lan966x_switchdev_blocking_nb __read_mostly = { +struct notifier_block lan966x_switchdev_blocking_nb __read_mostly = { .notifier_call = lan966x_switchdev_blocking_event, }; diff --git a/drivers/net/ethernet/microchip/lan966x/lan966x_taprio.c b/drivers/net/ethernet/microchip/lan966x/lan966x_taprio.c new file mode 100644 index 000000000000..3f5b212066c5 --- /dev/null +++ b/drivers/net/ethernet/microchip/lan966x/lan966x_taprio.c @@ -0,0 +1,528 @@ +// SPDX-License-Identifier: GPL-2.0+ + +#include "lan966x_main.h" + +#define LAN966X_TAPRIO_TIMEOUT_MS 1000 +#define LAN966X_TAPRIO_ENTRIES_PER_PORT 2 + +/* Minimum supported cycle time in nanoseconds */ +#define LAN966X_TAPRIO_MIN_CYCLE_TIME_NS NSEC_PER_USEC + +/* Maximum supported cycle time in nanoseconds */ +#define LAN966X_TAPRIO_MAX_CYCLE_TIME_NS (NSEC_PER_SEC - 1) + +/* Total number of TAS GCL entries */ +#define LAN966X_TAPRIO_NUM_GCL 256 + +/* TAPRIO link speeds for calculation of guard band */ +enum lan966x_taprio_link_speed { + LAN966X_TAPRIO_SPEED_NO_GB, + LAN966X_TAPRIO_SPEED_10, + LAN966X_TAPRIO_SPEED_100, + LAN966X_TAPRIO_SPEED_1000, + LAN966X_TAPRIO_SPEED_2500, +}; + +/* TAPRIO list states */ +enum lan966x_taprio_state { + LAN966X_TAPRIO_STATE_ADMIN, + LAN966X_TAPRIO_STATE_ADVANCING, + LAN966X_TAPRIO_STATE_PENDING, + LAN966X_TAPRIO_STATE_OPERATING, + LAN966X_TAPRIO_STATE_TERMINATING, + LAN966X_TAPRIO_STATE_MAX, +}; + +/* TAPRIO GCL command */ +enum lan966x_taprio_gcl_cmd { + LAN966X_TAPRIO_GCL_CMD_SET_GATE_STATES = 0, +}; + +static u32 lan966x_taprio_list_index(struct lan966x_port *port, u8 entry) +{ + return port->chip_port * LAN966X_TAPRIO_ENTRIES_PER_PORT + entry; +} + +static u32 lan966x_taprio_list_state_get(struct lan966x_port *port) +{ + struct lan966x *lan966x = port->lan966x; + u32 val; + + val = lan_rd(lan966x, QSYS_TAS_LST); + return QSYS_TAS_LST_LIST_STATE_GET(val); +} + +static u32 lan966x_taprio_list_index_state_get(struct lan966x_port *port, + u32 list) +{ + struct lan966x *lan966x = port->lan966x; + + lan_rmw(QSYS_TAS_CFG_CTRL_LIST_NUM_SET(list), + QSYS_TAS_CFG_CTRL_LIST_NUM, + lan966x, QSYS_TAS_CFG_CTRL); + + return lan966x_taprio_list_state_get(port); +} + +static void lan966x_taprio_list_state_set(struct lan966x_port *port, + u32 state) +{ + struct lan966x *lan966x = port->lan966x; + + lan_rmw(QSYS_TAS_LST_LIST_STATE_SET(state), + QSYS_TAS_LST_LIST_STATE, + lan966x, QSYS_TAS_LST); +} + +static int lan966x_taprio_list_shutdown(struct lan966x_port *port, + u32 list) +{ + struct lan966x *lan966x = port->lan966x; + bool pending, operating; + unsigned long end; + u32 state; + + end = jiffies + msecs_to_jiffies(LAN966X_TAPRIO_TIMEOUT_MS); + /* It is required to try multiple times to set the state of list, + * because the HW can overwrite this. + */ + do { + state = lan966x_taprio_list_state_get(port); + + pending = false; + operating = false; + + if (state == LAN966X_TAPRIO_STATE_ADVANCING || + state == LAN966X_TAPRIO_STATE_PENDING) { + lan966x_taprio_list_state_set(port, + LAN966X_TAPRIO_STATE_ADMIN); + pending = true; + } + + if (state == LAN966X_TAPRIO_STATE_OPERATING) { + lan966x_taprio_list_state_set(port, + LAN966X_TAPRIO_STATE_TERMINATING); + operating = true; + } + + /* If the entry was in pending and now gets in admin, then there + * is nothing else to do, so just bail out + */ + state = lan966x_taprio_list_state_get(port); + if (pending && + state == LAN966X_TAPRIO_STATE_ADMIN) + return 0; + + /* If the list was in operating and now is in terminating or + * admin, then is OK to exit but it needs to wait until the list + * will get in admin. It is not required to set the state + * again. + */ + if (operating && + (state == LAN966X_TAPRIO_STATE_TERMINATING || + state == LAN966X_TAPRIO_STATE_ADMIN)) + break; + + } while (!time_after(jiffies, end)); + + end = jiffies + msecs_to_jiffies(LAN966X_TAPRIO_TIMEOUT_MS); + do { + state = lan966x_taprio_list_state_get(port); + if (state == LAN966X_TAPRIO_STATE_ADMIN) + break; + + } while (!time_after(jiffies, end)); + + /* If the list was in operating mode, it could be stopped while some + * queues where closed, so make sure to restore "all-queues-open" + */ + if (operating) { + lan_wr(QSYS_TAS_GS_CTRL_HSCH_POS_SET(port->chip_port), + lan966x, QSYS_TAS_GS_CTRL); + + lan_wr(QSYS_TAS_GATE_STATE_TAS_GATE_STATE_SET(0xff), + lan966x, QSYS_TAS_GATE_STATE); + } + + return 0; +} + +static int lan966x_taprio_shutdown(struct lan966x_port *port) +{ + u32 i, list, state; + int err; + + for (i = 0; i < LAN966X_TAPRIO_ENTRIES_PER_PORT; ++i) { + list = lan966x_taprio_list_index(port, i); + state = lan966x_taprio_list_index_state_get(port, list); + if (state == LAN966X_TAPRIO_STATE_ADMIN) + continue; + + err = lan966x_taprio_list_shutdown(port, list); + if (err) + return err; + } + + return 0; +} + +/* Find a suitable list for a new schedule. First priority is a list in state + * pending. Second priority is a list in state admin. + */ +static int lan966x_taprio_find_list(struct lan966x_port *port, + struct tc_taprio_qopt_offload *qopt, + int *new_list, int *obs_list) +{ + int state[LAN966X_TAPRIO_ENTRIES_PER_PORT]; + int list[LAN966X_TAPRIO_ENTRIES_PER_PORT]; + int err, oper = -1; + u32 i; + + *new_list = -1; + *obs_list = -1; + + /* If there is already an entry in operating mode, return this list in + * obs_list, such that when the new list will get activated the + * operating list will be stopped. In this way is possible to have + * smooth transitions between the lists + */ + for (i = 0; i < LAN966X_TAPRIO_ENTRIES_PER_PORT; ++i) { + list[i] = lan966x_taprio_list_index(port, i); + state[i] = lan966x_taprio_list_index_state_get(port, list[i]); + if (state[i] == LAN966X_TAPRIO_STATE_OPERATING) + oper = list[i]; + } + + for (i = 0; i < LAN966X_TAPRIO_ENTRIES_PER_PORT; ++i) { + if (state[i] == LAN966X_TAPRIO_STATE_PENDING) { + err = lan966x_taprio_shutdown(port); + if (err) + return err; + + *new_list = list[i]; + *obs_list = (oper == -1) ? *new_list : oper; + return 0; + } + } + + for (i = 0; i < LAN966X_TAPRIO_ENTRIES_PER_PORT; ++i) { + if (state[i] == LAN966X_TAPRIO_STATE_ADMIN) { + *new_list = list[i]; + *obs_list = (oper == -1) ? *new_list : oper; + return 0; + } + } + + return -ENOSPC; +} + +static int lan966x_taprio_check(struct tc_taprio_qopt_offload *qopt) +{ + u64 total_time = 0; + u32 i; + + /* This is not supported by th HW */ + if (qopt->cycle_time_extension) + return -EOPNOTSUPP; + + /* There is a limited number of gcl entries that can be used, they are + * shared by all ports + */ + if (qopt->num_entries > LAN966X_TAPRIO_NUM_GCL) + return -EINVAL; + + /* Don't allow cycle times bigger than 1 sec or smaller than 1 usec */ + if (qopt->cycle_time < LAN966X_TAPRIO_MIN_CYCLE_TIME_NS || + qopt->cycle_time > LAN966X_TAPRIO_MAX_CYCLE_TIME_NS) + return -EINVAL; + + for (i = 0; i < qopt->num_entries; ++i) { + struct tc_taprio_sched_entry *entry = &qopt->entries[i]; + + /* Don't allow intervals bigger than 1 sec or smaller than 1 + * usec + */ + if (entry->interval < LAN966X_TAPRIO_MIN_CYCLE_TIME_NS || + entry->interval > LAN966X_TAPRIO_MAX_CYCLE_TIME_NS) + return -EINVAL; + + if (qopt->entries[i].command != TC_TAPRIO_CMD_SET_GATES) + return -EINVAL; + + total_time += qopt->entries[i].interval; + } + + /* Don't allow the total time of intervals be bigger than 1 sec */ + if (total_time > LAN966X_TAPRIO_MAX_CYCLE_TIME_NS) + return -EINVAL; + + /* The HW expects that the cycle time to be at least as big as sum of + * each interval of gcl + */ + if (qopt->cycle_time < total_time) + return -EINVAL; + + return 0; +} + +static int lan966x_taprio_gcl_free_get(struct lan966x_port *port, + unsigned long *free_list) +{ + struct lan966x *lan966x = port->lan966x; + u32 num_free, state, list; + u32 base, next, max_list; + + /* By default everything is free */ + bitmap_fill(free_list, LAN966X_TAPRIO_NUM_GCL); + num_free = LAN966X_TAPRIO_NUM_GCL; + + /* Iterate over all gcl entries and find out which are free. And mark + * those that are not free. + */ + max_list = lan966x->num_phys_ports * LAN966X_TAPRIO_ENTRIES_PER_PORT; + for (list = 0; list < max_list; ++list) { + state = lan966x_taprio_list_index_state_get(port, list); + if (state == LAN966X_TAPRIO_STATE_ADMIN) + continue; + + base = lan_rd(lan966x, QSYS_TAS_LIST_CFG); + base = QSYS_TAS_LIST_CFG_LIST_BASE_ADDR_GET(base); + next = base; + + do { + clear_bit(next, free_list); + num_free--; + + lan_rmw(QSYS_TAS_CFG_CTRL_GCL_ENTRY_NUM_SET(next), + QSYS_TAS_CFG_CTRL_GCL_ENTRY_NUM, + lan966x, QSYS_TAS_CFG_CTRL); + + next = lan_rd(lan966x, QSYS_TAS_GCL_CT_CFG2); + next = QSYS_TAS_GCL_CT_CFG2_NEXT_GCL_GET(next); + } while (base != next); + } + + return num_free; +} + +static void lan966x_taprio_gcl_setup_entry(struct lan966x_port *port, + struct tc_taprio_sched_entry *entry, + u32 next_entry) +{ + struct lan966x *lan966x = port->lan966x; + + /* Setup a single gcl entry */ + lan_wr(QSYS_TAS_GCL_CT_CFG_GATE_STATE_SET(entry->gate_mask) | + QSYS_TAS_GCL_CT_CFG_HSCH_POS_SET(port->chip_port) | + QSYS_TAS_GCL_CT_CFG_OP_TYPE_SET(LAN966X_TAPRIO_GCL_CMD_SET_GATE_STATES), + lan966x, QSYS_TAS_GCL_CT_CFG); + + lan_wr(QSYS_TAS_GCL_CT_CFG2_PORT_PROFILE_SET(port->chip_port) | + QSYS_TAS_GCL_CT_CFG2_NEXT_GCL_SET(next_entry), + lan966x, QSYS_TAS_GCL_CT_CFG2); + + lan_wr(entry->interval, lan966x, QSYS_TAS_GCL_TM_CFG); +} + +static int lan966x_taprio_gcl_setup(struct lan966x_port *port, + struct tc_taprio_qopt_offload *qopt, + int list) +{ + DECLARE_BITMAP(free_list, LAN966X_TAPRIO_NUM_GCL); + struct lan966x *lan966x = port->lan966x; + u32 i, base, next; + + if (lan966x_taprio_gcl_free_get(port, free_list) < qopt->num_entries) + return -ENOSPC; + + /* Select list */ + lan_rmw(QSYS_TAS_CFG_CTRL_LIST_NUM_SET(list), + QSYS_TAS_CFG_CTRL_LIST_NUM, + lan966x, QSYS_TAS_CFG_CTRL); + + /* Setup the address of the first gcl entry */ + base = find_first_bit(free_list, LAN966X_TAPRIO_NUM_GCL); + lan_rmw(QSYS_TAS_LIST_CFG_LIST_BASE_ADDR_SET(base), + QSYS_TAS_LIST_CFG_LIST_BASE_ADDR, + lan966x, QSYS_TAS_LIST_CFG); + + /* Iterate over entries and add them to the gcl list */ + next = base; + for (i = 0; i < qopt->num_entries; ++i) { + lan_rmw(QSYS_TAS_CFG_CTRL_GCL_ENTRY_NUM_SET(next), + QSYS_TAS_CFG_CTRL_GCL_ENTRY_NUM, + lan966x, QSYS_TAS_CFG_CTRL); + + /* If the entry is last, point back to the start of the list */ + if (i == qopt->num_entries - 1) + next = base; + else + next = find_next_bit(free_list, LAN966X_TAPRIO_NUM_GCL, + next + 1); + + lan966x_taprio_gcl_setup_entry(port, &qopt->entries[i], next); + } + + return 0; +} + +/* Calculate new base_time based on cycle_time. The HW recommends to have the + * new base time at least 2 * cycle type + current time + */ +static void lan966x_taprio_new_base_time(struct lan966x *lan966x, + const u32 cycle_time, + const ktime_t org_base_time, + ktime_t *new_base_time) +{ + ktime_t current_time, threshold_time; + struct timespec64 ts; + + /* Get the current time and calculate the threshold_time */ + lan966x_ptp_gettime64(&lan966x->phc[LAN966X_PHC_PORT].info, &ts); + current_time = timespec64_to_ktime(ts); + threshold_time = current_time + (2 * cycle_time); + + /* If the org_base_time is in enough in future just use it */ + if (org_base_time >= threshold_time) { + *new_base_time = org_base_time; + return; + } + + /* If the org_base_time is smaller than current_time, calculate the new + * base time as following. + */ + if (org_base_time <= current_time) { + u64 tmp = current_time - org_base_time; + u32 rem = 0; + + if (tmp > cycle_time) + div_u64_rem(tmp, cycle_time, &rem); + rem = cycle_time - rem; + *new_base_time = threshold_time + rem; + return; + } + + /* The only left place for org_base_time is between current_time and + * threshold_time. In this case the new_base_time is calculated like + * org_base_time + 2 * cycletime + */ + *new_base_time = org_base_time + 2 * cycle_time; +} + +int lan966x_taprio_speed_set(struct lan966x_port *port, int speed) +{ + struct lan966x *lan966x = port->lan966x; + u8 taprio_speed; + + switch (speed) { + case SPEED_10: + taprio_speed = LAN966X_TAPRIO_SPEED_10; + break; + case SPEED_100: + taprio_speed = LAN966X_TAPRIO_SPEED_100; + break; + case SPEED_1000: + taprio_speed = LAN966X_TAPRIO_SPEED_1000; + break; + case SPEED_2500: + taprio_speed = LAN966X_TAPRIO_SPEED_2500; + break; + default: + return -EINVAL; + } + + lan_rmw(QSYS_TAS_PROFILE_CFG_LINK_SPEED_SET(taprio_speed), + QSYS_TAS_PROFILE_CFG_LINK_SPEED, + lan966x, QSYS_TAS_PROFILE_CFG(port->chip_port)); + + return 0; +} + +int lan966x_taprio_add(struct lan966x_port *port, + struct tc_taprio_qopt_offload *qopt) +{ + struct lan966x *lan966x = port->lan966x; + int err, new_list, obs_list; + struct timespec64 ts; + ktime_t base_time; + + err = lan966x_taprio_check(qopt); + if (err) + return err; + + err = lan966x_taprio_find_list(port, qopt, &new_list, &obs_list); + if (err) + return err; + + err = lan966x_taprio_gcl_setup(port, qopt, new_list); + if (err) + return err; + + lan966x_taprio_new_base_time(lan966x, qopt->cycle_time, + qopt->base_time, &base_time); + + ts = ktime_to_timespec64(base_time); + lan_wr(QSYS_TAS_BT_NSEC_NSEC_SET(ts.tv_nsec), + lan966x, QSYS_TAS_BT_NSEC); + + lan_wr(lower_32_bits(ts.tv_sec), + lan966x, QSYS_TAS_BT_SEC_LSB); + + lan_wr(QSYS_TAS_BT_SEC_MSB_SEC_MSB_SET(upper_32_bits(ts.tv_sec)), + lan966x, QSYS_TAS_BT_SEC_MSB); + + lan_wr(qopt->cycle_time, lan966x, QSYS_TAS_CT_CFG); + + lan_rmw(QSYS_TAS_STARTUP_CFG_OBSOLETE_IDX_SET(obs_list), + QSYS_TAS_STARTUP_CFG_OBSOLETE_IDX, + lan966x, QSYS_TAS_STARTUP_CFG); + + /* Start list processing */ + lan_rmw(QSYS_TAS_LST_LIST_STATE_SET(LAN966X_TAPRIO_STATE_ADVANCING), + QSYS_TAS_LST_LIST_STATE, + lan966x, QSYS_TAS_LST); + + return err; +} + +int lan966x_taprio_del(struct lan966x_port *port) +{ + return lan966x_taprio_shutdown(port); +} + +void lan966x_taprio_init(struct lan966x *lan966x) +{ + int num_taprio_lists; + int p; + + lan_wr(QSYS_TAS_STM_CFG_REVISIT_DLY_SET((256 * 1000) / + lan966x_ptp_get_period_ps()), + lan966x, QSYS_TAS_STM_CFG); + + num_taprio_lists = lan966x->num_phys_ports * + LAN966X_TAPRIO_ENTRIES_PER_PORT; + + /* For now we always use guard band on all queues */ + lan_rmw(QSYS_TAS_CFG_CTRL_LIST_NUM_MAX_SET(num_taprio_lists) | + QSYS_TAS_CFG_CTRL_ALWAYS_GB_SCH_Q_SET(1), + QSYS_TAS_CFG_CTRL_LIST_NUM_MAX | + QSYS_TAS_CFG_CTRL_ALWAYS_GB_SCH_Q, + lan966x, QSYS_TAS_CFG_CTRL); + + for (p = 0; p < lan966x->num_phys_ports; p++) + lan_rmw(QSYS_TAS_PROFILE_CFG_PORT_NUM_SET(p), + QSYS_TAS_PROFILE_CFG_PORT_NUM, + lan966x, QSYS_TAS_PROFILE_CFG(p)); +} + +void lan966x_taprio_deinit(struct lan966x *lan966x) +{ + int p; + + for (p = 0; p < lan966x->num_phys_ports; ++p) { + if (!lan966x->ports[p]) + continue; + + lan966x_taprio_del(lan966x->ports[p]); + } +} diff --git a/drivers/net/ethernet/microchip/lan966x/lan966x_tbf.c b/drivers/net/ethernet/microchip/lan966x/lan966x_tbf.c new file mode 100644 index 000000000000..4555a35d0d28 --- /dev/null +++ b/drivers/net/ethernet/microchip/lan966x/lan966x_tbf.c @@ -0,0 +1,85 @@ +// SPDX-License-Identifier: GPL-2.0+ + +#include "lan966x_main.h" + +int lan966x_tbf_add(struct lan966x_port *port, + struct tc_tbf_qopt_offload *qopt) +{ + struct lan966x *lan966x = port->lan966x; + bool root = qopt->parent == TC_H_ROOT; + u32 queue = 0; + u32 cir, cbs; + u32 se_idx; + + if (!root) { + queue = TC_H_MIN(qopt->parent) - 1; + if (queue >= NUM_PRIO_QUEUES) + return -EOPNOTSUPP; + } + + if (root) + se_idx = SE_IDX_PORT + port->chip_port; + else + se_idx = SE_IDX_QUEUE + port->chip_port * NUM_PRIO_QUEUES + queue; + + cir = div_u64(qopt->replace_params.rate.rate_bytes_ps, 1000) * 8; + cbs = qopt->replace_params.max_size; + + /* Rate unit is 100 kbps */ + cir = DIV_ROUND_UP(cir, 100); + /* Avoid using zero rate */ + cir = cir ?: 1; + /* Burst unit is 4kB */ + cbs = DIV_ROUND_UP(cbs, 4096); + /* Avoid using zero burst */ + cbs = cbs ?: 1; + + /* Check that actually the result can be written */ + if (cir > GENMASK(15, 0) || + cbs > GENMASK(6, 0)) + return -EINVAL; + + lan_rmw(QSYS_SE_CFG_SE_AVB_ENA_SET(0) | + QSYS_SE_CFG_SE_FRM_MODE_SET(1), + QSYS_SE_CFG_SE_AVB_ENA | + QSYS_SE_CFG_SE_FRM_MODE, + lan966x, QSYS_SE_CFG(se_idx)); + + lan_wr(QSYS_CIR_CFG_CIR_RATE_SET(cir) | + QSYS_CIR_CFG_CIR_BURST_SET(cbs), + lan966x, QSYS_CIR_CFG(se_idx)); + + return 0; +} + +int lan966x_tbf_del(struct lan966x_port *port, + struct tc_tbf_qopt_offload *qopt) +{ + struct lan966x *lan966x = port->lan966x; + bool root = qopt->parent == TC_H_ROOT; + u32 queue = 0; + u32 se_idx; + + if (!root) { + queue = TC_H_MIN(qopt->parent) - 1; + if (queue >= NUM_PRIO_QUEUES) + return -EOPNOTSUPP; + } + + if (root) + se_idx = SE_IDX_PORT + port->chip_port; + else + se_idx = SE_IDX_QUEUE + port->chip_port * NUM_PRIO_QUEUES + queue; + + lan_rmw(QSYS_SE_CFG_SE_AVB_ENA_SET(0) | + QSYS_SE_CFG_SE_FRM_MODE_SET(0), + QSYS_SE_CFG_SE_AVB_ENA | + QSYS_SE_CFG_SE_FRM_MODE, + lan966x, QSYS_SE_CFG(se_idx)); + + lan_wr(QSYS_CIR_CFG_CIR_RATE_SET(0) | + QSYS_CIR_CFG_CIR_BURST_SET(0), + lan966x, QSYS_CIR_CFG(se_idx)); + + return 0; +} diff --git a/drivers/net/ethernet/microchip/lan966x/lan966x_tc.c b/drivers/net/ethernet/microchip/lan966x/lan966x_tc.c new file mode 100644 index 000000000000..651d5493ae55 --- /dev/null +++ b/drivers/net/ethernet/microchip/lan966x/lan966x_tc.c @@ -0,0 +1,133 @@ +// SPDX-License-Identifier: GPL-2.0+ + +#include <net/pkt_cls.h> + +#include "lan966x_main.h" + +static LIST_HEAD(lan966x_tc_block_cb_list); + +static int lan966x_tc_setup_qdisc_mqprio(struct lan966x_port *port, + struct tc_mqprio_qopt_offload *mqprio) +{ + u8 num_tc = mqprio->qopt.num_tc; + + mqprio->qopt.hw = TC_MQPRIO_HW_OFFLOAD_TCS; + + return num_tc ? lan966x_mqprio_add(port, num_tc) : + lan966x_mqprio_del(port); +} + +static int lan966x_tc_setup_qdisc_taprio(struct lan966x_port *port, + struct tc_taprio_qopt_offload *taprio) +{ + return taprio->enable ? lan966x_taprio_add(port, taprio) : + lan966x_taprio_del(port); +} + +static int lan966x_tc_setup_qdisc_tbf(struct lan966x_port *port, + struct tc_tbf_qopt_offload *qopt) +{ + switch (qopt->command) { + case TC_TBF_REPLACE: + return lan966x_tbf_add(port, qopt); + case TC_TBF_DESTROY: + return lan966x_tbf_del(port, qopt); + default: + return -EOPNOTSUPP; + } + + return -EOPNOTSUPP; +} + +static int lan966x_tc_setup_qdisc_cbs(struct lan966x_port *port, + struct tc_cbs_qopt_offload *qopt) +{ + return qopt->enable ? lan966x_cbs_add(port, qopt) : + lan966x_cbs_del(port, qopt); +} + +static int lan966x_tc_setup_qdisc_ets(struct lan966x_port *port, + struct tc_ets_qopt_offload *qopt) +{ + switch (qopt->command) { + case TC_ETS_REPLACE: + return lan966x_ets_add(port, qopt); + case TC_ETS_DESTROY: + return lan966x_ets_del(port, qopt); + default: + return -EOPNOTSUPP; + }; + + return -EOPNOTSUPP; +} + +static int lan966x_tc_block_cb(enum tc_setup_type type, void *type_data, + void *cb_priv, bool ingress) +{ + struct lan966x_port *port = cb_priv; + + switch (type) { + case TC_SETUP_CLSMATCHALL: + return lan966x_tc_matchall(port, type_data, ingress); + default: + return -EOPNOTSUPP; + } +} + +static int lan966x_tc_block_cb_ingress(enum tc_setup_type type, + void *type_data, void *cb_priv) +{ + return lan966x_tc_block_cb(type, type_data, cb_priv, true); +} + +static int lan966x_tc_block_cb_egress(enum tc_setup_type type, + void *type_data, void *cb_priv) +{ + return lan966x_tc_block_cb(type, type_data, cb_priv, false); +} + +static int lan966x_tc_setup_block(struct lan966x_port *port, + struct flow_block_offload *f) +{ + flow_setup_cb_t *cb; + bool ingress; + + if (f->binder_type == FLOW_BLOCK_BINDER_TYPE_CLSACT_INGRESS) { + cb = lan966x_tc_block_cb_ingress; + port->tc.ingress_shared_block = f->block_shared; + ingress = true; + } else if (f->binder_type == FLOW_BLOCK_BINDER_TYPE_CLSACT_EGRESS) { + cb = lan966x_tc_block_cb_egress; + ingress = false; + } else { + return -EOPNOTSUPP; + } + + return flow_block_cb_setup_simple(f, &lan966x_tc_block_cb_list, + cb, port, port, ingress); +} + +int lan966x_tc_setup(struct net_device *dev, enum tc_setup_type type, + void *type_data) +{ + struct lan966x_port *port = netdev_priv(dev); + + switch (type) { + case TC_SETUP_QDISC_MQPRIO: + return lan966x_tc_setup_qdisc_mqprio(port, type_data); + case TC_SETUP_QDISC_TAPRIO: + return lan966x_tc_setup_qdisc_taprio(port, type_data); + case TC_SETUP_QDISC_TBF: + return lan966x_tc_setup_qdisc_tbf(port, type_data); + case TC_SETUP_QDISC_CBS: + return lan966x_tc_setup_qdisc_cbs(port, type_data); + case TC_SETUP_QDISC_ETS: + return lan966x_tc_setup_qdisc_ets(port, type_data); + case TC_SETUP_BLOCK: + return lan966x_tc_setup_block(port, type_data); + default: + return -EOPNOTSUPP; + } + + return 0; +} diff --git a/drivers/net/ethernet/microchip/lan966x/lan966x_tc_matchall.c b/drivers/net/ethernet/microchip/lan966x/lan966x_tc_matchall.c new file mode 100644 index 000000000000..7368433b9277 --- /dev/null +++ b/drivers/net/ethernet/microchip/lan966x/lan966x_tc_matchall.c @@ -0,0 +1,95 @@ +// SPDX-License-Identifier: GPL-2.0+ + +#include "lan966x_main.h" + +static int lan966x_tc_matchall_add(struct lan966x_port *port, + struct tc_cls_matchall_offload *f, + bool ingress) +{ + struct flow_action_entry *act; + + if (!flow_offload_has_one_action(&f->rule->action)) { + NL_SET_ERR_MSG_MOD(f->common.extack, + "Only once action per filter is supported"); + return -EOPNOTSUPP; + } + + act = &f->rule->action.entries[0]; + switch (act->id) { + case FLOW_ACTION_POLICE: + return lan966x_police_port_add(port, &f->rule->action, act, + f->cookie, ingress, + f->common.extack); + case FLOW_ACTION_MIRRED: + return lan966x_mirror_port_add(port, act, f->cookie, + ingress, f->common.extack); + default: + NL_SET_ERR_MSG_MOD(f->common.extack, + "Unsupported action"); + return -EOPNOTSUPP; + } + + return 0; +} + +static int lan966x_tc_matchall_del(struct lan966x_port *port, + struct tc_cls_matchall_offload *f, + bool ingress) +{ + if (f->cookie == port->tc.police_id) { + return lan966x_police_port_del(port, f->cookie, + f->common.extack); + } else if (f->cookie == port->tc.ingress_mirror_id || + f->cookie == port->tc.egress_mirror_id) { + return lan966x_mirror_port_del(port, ingress, + f->common.extack); + } else { + NL_SET_ERR_MSG_MOD(f->common.extack, + "Unsupported action"); + return -EOPNOTSUPP; + } + + return 0; +} + +static int lan966x_tc_matchall_stats(struct lan966x_port *port, + struct tc_cls_matchall_offload *f, + bool ingress) +{ + if (f->cookie == port->tc.police_id) { + lan966x_police_port_stats(port, &f->stats); + } else if (f->cookie == port->tc.ingress_mirror_id || + f->cookie == port->tc.egress_mirror_id) { + lan966x_mirror_port_stats(port, &f->stats, ingress); + } else { + NL_SET_ERR_MSG_MOD(f->common.extack, + "Unsupported action"); + return -EOPNOTSUPP; + } + + return 0; +} + +int lan966x_tc_matchall(struct lan966x_port *port, + struct tc_cls_matchall_offload *f, + bool ingress) +{ + if (!tc_cls_can_offload_and_chain0(port->dev, &f->common)) { + NL_SET_ERR_MSG_MOD(f->common.extack, + "Only chain zero is supported"); + return -EOPNOTSUPP; + } + + switch (f->command) { + case TC_CLSMATCHALL_REPLACE: + return lan966x_tc_matchall_add(port, f, ingress); + case TC_CLSMATCHALL_DESTROY: + return lan966x_tc_matchall_del(port, f, ingress); + case TC_CLSMATCHALL_STATS: + return lan966x_tc_matchall_stats(port, f, ingress); + default: + return -EOPNOTSUPP; + } + + return 0; +} diff --git a/drivers/net/ethernet/microchip/sparx5/Makefile b/drivers/net/ethernet/microchip/sparx5/Makefile index 4402c3ed1dc5..d1c6ad966747 100644 --- a/drivers/net/ethernet/microchip/sparx5/Makefile +++ b/drivers/net/ethernet/microchip/sparx5/Makefile @@ -8,4 +8,4 @@ obj-$(CONFIG_SPARX5_SWITCH) += sparx5-switch.o sparx5-switch-objs := sparx5_main.o sparx5_packet.o \ sparx5_netdev.o sparx5_phylink.o sparx5_port.o sparx5_mactable.o sparx5_vlan.o \ sparx5_switchdev.o sparx5_calendar.o sparx5_ethtool.o sparx5_fdma.o \ - sparx5_ptp.o sparx5_pgid.o + sparx5_ptp.o sparx5_pgid.o sparx5_tc.o sparx5_qos.o diff --git a/drivers/net/ethernet/microchip/sparx5/sparx5_mactable.c b/drivers/net/ethernet/microchip/sparx5/sparx5_mactable.c index a5837dbe0c7e..4af285918ea2 100644 --- a/drivers/net/ethernet/microchip/sparx5/sparx5_mactable.c +++ b/drivers/net/ethernet/microchip/sparx5/sparx5_mactable.c @@ -186,8 +186,8 @@ bool sparx5_mact_getnext(struct sparx5 *sparx5, return ret == 0; } -bool sparx5_mact_find(struct sparx5 *sparx5, - const unsigned char mac[ETH_ALEN], u16 vid, u32 *pcfg2) +int sparx5_mact_find(struct sparx5 *sparx5, + const unsigned char mac[ETH_ALEN], u16 vid, u32 *pcfg2) { int ret; u32 cfg2; diff --git a/drivers/net/ethernet/microchip/sparx5/sparx5_main.c b/drivers/net/ethernet/microchip/sparx5/sparx5_main.c index 01be7bd84181..62a325e96345 100644 --- a/drivers/net/ethernet/microchip/sparx5/sparx5_main.c +++ b/drivers/net/ethernet/microchip/sparx5/sparx5_main.c @@ -27,6 +27,7 @@ #include "sparx5_main_regs.h" #include "sparx5_main.h" #include "sparx5_port.h" +#include "sparx5_qos.h" #define QLIM_WM(fraction) \ ((SPX5_BUFFER_MEMORY / SPX5_BUFFER_CELL_SZ - 100) * (fraction) / 100) @@ -277,6 +278,7 @@ static int sparx5_create_port(struct sparx5 *sparx5, spx5_port->custom_etype = 0x8880; /* Vitesse */ spx5_port->phylink_pcs.poll = true; spx5_port->phylink_pcs.ops = &sparx5_phylink_pcs_ops; + spx5_port->is_mrouter = false; sparx5->ports[config->portno] = spx5_port; err = sparx5_port_init(sparx5, spx5_port, &config->conf); @@ -661,6 +663,9 @@ static int sparx5_start(struct sparx5 *sparx5) queue_delayed_work(sparx5->mact_queue, &sparx5->mact_work, SPX5_MACT_PULL_DELAY); + mutex_init(&sparx5->mdb_lock); + INIT_LIST_HEAD(&sparx5->mdb_entries); + err = sparx5_register_netdevs(sparx5); if (err) return err; @@ -864,6 +869,12 @@ static int mchp_sparx5_probe(struct platform_device *pdev) goto cleanup_ports; } + err = sparx5_qos_init(sparx5); + if (err) { + dev_err(sparx5->dev, "Failed to initialize QoS\n"); + goto cleanup_ports; + } + err = sparx5_ptp_init(sparx5); if (err) { dev_err(sparx5->dev, "PTP failed\n"); diff --git a/drivers/net/ethernet/microchip/sparx5/sparx5_main.h b/drivers/net/ethernet/microchip/sparx5/sparx5_main.h index b197129044b5..7a83222caa73 100644 --- a/drivers/net/ethernet/microchip/sparx5/sparx5_main.h +++ b/drivers/net/ethernet/microchip/sparx5/sparx5_main.h @@ -190,6 +190,7 @@ struct sparx5_port { u8 ptp_cmd; u16 ts_id; struct sk_buff_head tx_skbs; + bool is_mrouter; }; enum sparx5_core_clockfreq { @@ -215,6 +216,15 @@ struct sparx5_skb_cb { unsigned long jiffies; }; +struct sparx5_mdb_entry { + struct list_head list; + DECLARE_BITMAP(port_mask, SPX5_PORTS); + unsigned char addr[ETH_ALEN]; + bool cpu_copy; + u16 vid; + u16 pgid_idx; +}; + #define SPARX5_PTP_TIMEOUT msecs_to_jiffies(10) #define SPARX5_SKB_CB(skb) \ ((struct sparx5_skb_cb *)((skb)->cb)) @@ -256,6 +266,10 @@ struct sparx5 { struct list_head mact_entries; /* mac table list (mact_entries) mutex */ struct mutex mact_lock; + /* SW MDB table */ + struct list_head mdb_entries; + /* mdb list mutex */ + struct mutex mdb_lock; struct delayed_work mact_work; struct workqueue_struct *mact_queue; /* Board specifics */ @@ -291,7 +305,7 @@ struct frame_info { void sparx5_xtr_flush(struct sparx5 *sparx5, u8 grp); void sparx5_ifh_parse(u32 *ifh, struct frame_info *info); irqreturn_t sparx5_xtr_handler(int irq, void *_priv); -int sparx5_port_xmit_impl(struct sk_buff *skb, struct net_device *dev); +netdev_tx_t sparx5_port_xmit_impl(struct sk_buff *skb, struct net_device *dev); int sparx5_manual_injection_mode(struct sparx5 *sparx5); void sparx5_port_inj_timer_setup(struct sparx5_port *port); @@ -307,8 +321,8 @@ int sparx5_mact_learn(struct sparx5 *sparx5, int port, const unsigned char mac[ETH_ALEN], u16 vid); bool sparx5_mact_getnext(struct sparx5 *sparx5, unsigned char mac[ETH_ALEN], u16 *vid, u32 *pcfg2); -bool sparx5_mact_find(struct sparx5 *sparx5, - const unsigned char mac[ETH_ALEN], u16 vid, u32 *pcfg2); +int sparx5_mact_find(struct sparx5 *sparx5, + const unsigned char mac[ETH_ALEN], u16 vid, u32 *pcfg2); int sparx5_mact_forget(struct sparx5 *sparx5, const unsigned char mac[ETH_ALEN], u16 vid); int sparx5_add_mact_entry(struct sparx5 *sparx5, @@ -325,6 +339,7 @@ void sparx5_mact_init(struct sparx5 *sparx5); /* sparx5_vlan.c */ void sparx5_pgid_update_mask(struct sparx5_port *port, int pgid, bool enable); +void sparx5_pgid_clear(struct sparx5 *spx5, int pgid); void sparx5_pgid_read_mask(struct sparx5 *sparx5, int pgid, u32 portmask[3]); void sparx5_update_fwd(struct sparx5 *sparx5); void sparx5_vlan_init(struct sparx5 *sparx5); diff --git a/drivers/net/ethernet/microchip/sparx5/sparx5_main_regs.h b/drivers/net/ethernet/microchip/sparx5/sparx5_main_regs.h index c94de436b281..fa2eb70f487a 100644 --- a/drivers/net/ethernet/microchip/sparx5/sparx5_main_regs.h +++ b/drivers/net/ethernet/microchip/sparx5/sparx5_main_regs.h @@ -2993,6 +2993,147 @@ enum sparx5_target { #define GCB_SIO_CLOCK_SYS_CLK_PERIOD_GET(x)\ FIELD_GET(GCB_SIO_CLOCK_SYS_CLK_PERIOD, x) +/* HSCH:HSCH_CFG:CIR_CFG */ +#define HSCH_CIR_CFG(g) __REG(TARGET_HSCH, 0, 1, 0, g, 5040, 32, 0, 0, 1, 4) + +#define HSCH_CIR_CFG_CIR_RATE GENMASK(22, 6) +#define HSCH_CIR_CFG_CIR_RATE_SET(x)\ + FIELD_PREP(HSCH_CIR_CFG_CIR_RATE, x) +#define HSCH_CIR_CFG_CIR_RATE_GET(x)\ + FIELD_GET(HSCH_CIR_CFG_CIR_RATE, x) + +#define HSCH_CIR_CFG_CIR_BURST GENMASK(5, 0) +#define HSCH_CIR_CFG_CIR_BURST_SET(x)\ + FIELD_PREP(HSCH_CIR_CFG_CIR_BURST, x) +#define HSCH_CIR_CFG_CIR_BURST_GET(x)\ + FIELD_GET(HSCH_CIR_CFG_CIR_BURST, x) + +/* HSCH:HSCH_CFG:EIR_CFG */ +#define HSCH_EIR_CFG(g) __REG(TARGET_HSCH, 0, 1, 0, g, 5040, 32, 4, 0, 1, 4) + +#define HSCH_EIR_CFG_EIR_RATE GENMASK(22, 6) +#define HSCH_EIR_CFG_EIR_RATE_SET(x)\ + FIELD_PREP(HSCH_EIR_CFG_EIR_RATE, x) +#define HSCH_EIR_CFG_EIR_RATE_GET(x)\ + FIELD_GET(HSCH_EIR_CFG_EIR_RATE, x) + +#define HSCH_EIR_CFG_EIR_BURST GENMASK(5, 0) +#define HSCH_EIR_CFG_EIR_BURST_SET(x)\ + FIELD_PREP(HSCH_EIR_CFG_EIR_BURST, x) +#define HSCH_EIR_CFG_EIR_BURST_GET(x)\ + FIELD_GET(HSCH_EIR_CFG_EIR_BURST, x) + +/* HSCH:HSCH_CFG:SE_CFG */ +#define HSCH_SE_CFG(g) __REG(TARGET_HSCH, 0, 1, 0, g, 5040, 32, 8, 0, 1, 4) + +#define HSCH_SE_CFG_SE_DWRR_CNT GENMASK(12, 6) +#define HSCH_SE_CFG_SE_DWRR_CNT_SET(x)\ + FIELD_PREP(HSCH_SE_CFG_SE_DWRR_CNT, x) +#define HSCH_SE_CFG_SE_DWRR_CNT_GET(x)\ + FIELD_GET(HSCH_SE_CFG_SE_DWRR_CNT, x) + +#define HSCH_SE_CFG_SE_AVB_ENA BIT(5) +#define HSCH_SE_CFG_SE_AVB_ENA_SET(x)\ + FIELD_PREP(HSCH_SE_CFG_SE_AVB_ENA, x) +#define HSCH_SE_CFG_SE_AVB_ENA_GET(x)\ + FIELD_GET(HSCH_SE_CFG_SE_AVB_ENA, x) + +#define HSCH_SE_CFG_SE_FRM_MODE GENMASK(4, 3) +#define HSCH_SE_CFG_SE_FRM_MODE_SET(x)\ + FIELD_PREP(HSCH_SE_CFG_SE_FRM_MODE, x) +#define HSCH_SE_CFG_SE_FRM_MODE_GET(x)\ + FIELD_GET(HSCH_SE_CFG_SE_FRM_MODE, x) + +#define HSCH_SE_CFG_SE_DWRR_FRM_MODE GENMASK(2, 1) +#define HSCH_SE_CFG_SE_DWRR_FRM_MODE_SET(x)\ + FIELD_PREP(HSCH_SE_CFG_SE_DWRR_FRM_MODE, x) +#define HSCH_SE_CFG_SE_DWRR_FRM_MODE_GET(x)\ + FIELD_GET(HSCH_SE_CFG_SE_DWRR_FRM_MODE, x) + +#define HSCH_SE_CFG_SE_STOP BIT(0) +#define HSCH_SE_CFG_SE_STOP_SET(x)\ + FIELD_PREP(HSCH_SE_CFG_SE_STOP, x) +#define HSCH_SE_CFG_SE_STOP_GET(x)\ + FIELD_GET(HSCH_SE_CFG_SE_STOP, x) + +/* HSCH:HSCH_CFG:SE_CONNECT */ +#define HSCH_SE_CONNECT(g) __REG(TARGET_HSCH, 0, 1, 0, g, 5040, 32, 12, 0, 1, 4) + +#define HSCH_SE_CONNECT_SE_LEAK_LINK GENMASK(15, 0) +#define HSCH_SE_CONNECT_SE_LEAK_LINK_SET(x)\ + FIELD_PREP(HSCH_SE_CONNECT_SE_LEAK_LINK, x) +#define HSCH_SE_CONNECT_SE_LEAK_LINK_GET(x)\ + FIELD_GET(HSCH_SE_CONNECT_SE_LEAK_LINK, x) + +/* HSCH:HSCH_CFG:SE_DLB_SENSE */ +#define HSCH_SE_DLB_SENSE(g) __REG(TARGET_HSCH, 0, 1, 0, g, 5040, 32, 16, 0, 1, 4) + +#define HSCH_SE_DLB_SENSE_SE_DLB_PRIO GENMASK(12, 10) +#define HSCH_SE_DLB_SENSE_SE_DLB_PRIO_SET(x)\ + FIELD_PREP(HSCH_SE_DLB_SENSE_SE_DLB_PRIO, x) +#define HSCH_SE_DLB_SENSE_SE_DLB_PRIO_GET(x)\ + FIELD_GET(HSCH_SE_DLB_SENSE_SE_DLB_PRIO, x) + +#define HSCH_SE_DLB_SENSE_SE_DLB_DPORT GENMASK(9, 3) +#define HSCH_SE_DLB_SENSE_SE_DLB_DPORT_SET(x)\ + FIELD_PREP(HSCH_SE_DLB_SENSE_SE_DLB_DPORT, x) +#define HSCH_SE_DLB_SENSE_SE_DLB_DPORT_GET(x)\ + FIELD_GET(HSCH_SE_DLB_SENSE_SE_DLB_DPORT, x) + +#define HSCH_SE_DLB_SENSE_SE_DLB_SE_ENA BIT(2) +#define HSCH_SE_DLB_SENSE_SE_DLB_SE_ENA_SET(x)\ + FIELD_PREP(HSCH_SE_DLB_SENSE_SE_DLB_SE_ENA, x) +#define HSCH_SE_DLB_SENSE_SE_DLB_SE_ENA_GET(x)\ + FIELD_GET(HSCH_SE_DLB_SENSE_SE_DLB_SE_ENA, x) + +#define HSCH_SE_DLB_SENSE_SE_DLB_PRIO_ENA BIT(1) +#define HSCH_SE_DLB_SENSE_SE_DLB_PRIO_ENA_SET(x)\ + FIELD_PREP(HSCH_SE_DLB_SENSE_SE_DLB_PRIO_ENA, x) +#define HSCH_SE_DLB_SENSE_SE_DLB_PRIO_ENA_GET(x)\ + FIELD_GET(HSCH_SE_DLB_SENSE_SE_DLB_PRIO_ENA, x) + +#define HSCH_SE_DLB_SENSE_SE_DLB_DPORT_ENA BIT(0) +#define HSCH_SE_DLB_SENSE_SE_DLB_DPORT_ENA_SET(x)\ + FIELD_PREP(HSCH_SE_DLB_SENSE_SE_DLB_DPORT_ENA, x) +#define HSCH_SE_DLB_SENSE_SE_DLB_DPORT_ENA_GET(x)\ + FIELD_GET(HSCH_SE_DLB_SENSE_SE_DLB_DPORT_ENA, x) + +/* HSCH:HSCH_DWRR:DWRR_ENTRY */ +#define HSCH_DWRR_ENTRY(g) __REG(TARGET_HSCH, 0, 1, 162816, g, 72, 4, 0, 0, 1, 4) + +#define HSCH_DWRR_ENTRY_DWRR_COST GENMASK(24, 20) +#define HSCH_DWRR_ENTRY_DWRR_COST_SET(x)\ + FIELD_PREP(HSCH_DWRR_ENTRY_DWRR_COST, x) +#define HSCH_DWRR_ENTRY_DWRR_COST_GET(x)\ + FIELD_GET(HSCH_DWRR_ENTRY_DWRR_COST, x) + +#define HSCH_DWRR_ENTRY_DWRR_BALANCE GENMASK(19, 0) +#define HSCH_DWRR_ENTRY_DWRR_BALANCE_SET(x)\ + FIELD_PREP(HSCH_DWRR_ENTRY_DWRR_BALANCE, x) +#define HSCH_DWRR_ENTRY_DWRR_BALANCE_GET(x)\ + FIELD_GET(HSCH_DWRR_ENTRY_DWRR_BALANCE, x) + +/* HSCH:HSCH_MISC:HSCH_CFG_CFG */ +#define HSCH_HSCH_CFG_CFG __REG(TARGET_HSCH, 0, 1, 163104, 0, 1, 648, 284, 0, 1, 4) + +#define HSCH_HSCH_CFG_CFG_CFG_SE_IDX GENMASK(26, 14) +#define HSCH_HSCH_CFG_CFG_CFG_SE_IDX_SET(x)\ + FIELD_PREP(HSCH_HSCH_CFG_CFG_CFG_SE_IDX, x) +#define HSCH_HSCH_CFG_CFG_CFG_SE_IDX_GET(x)\ + FIELD_GET(HSCH_HSCH_CFG_CFG_CFG_SE_IDX, x) + +#define HSCH_HSCH_CFG_CFG_HSCH_LAYER GENMASK(13, 12) +#define HSCH_HSCH_CFG_CFG_HSCH_LAYER_SET(x)\ + FIELD_PREP(HSCH_HSCH_CFG_CFG_HSCH_LAYER, x) +#define HSCH_HSCH_CFG_CFG_HSCH_LAYER_GET(x)\ + FIELD_GET(HSCH_HSCH_CFG_CFG_HSCH_LAYER, x) + +#define HSCH_HSCH_CFG_CFG_CSR_GRANT GENMASK(11, 0) +#define HSCH_HSCH_CFG_CFG_CSR_GRANT_SET(x)\ + FIELD_PREP(HSCH_HSCH_CFG_CFG_CSR_GRANT, x) +#define HSCH_HSCH_CFG_CFG_CSR_GRANT_GET(x)\ + FIELD_GET(HSCH_HSCH_CFG_CFG_CSR_GRANT, x) + /* HSCH:HSCH_MISC:SYS_CLK_PER */ #define HSCH_SYS_CLK_PER __REG(TARGET_HSCH, 0, 1, 163104, 0, 1, 648, 640, 0, 1, 4) @@ -3002,6 +3143,30 @@ enum sparx5_target { #define HSCH_SYS_CLK_PER_SYS_CLK_PER_100PS_GET(x)\ FIELD_GET(HSCH_SYS_CLK_PER_SYS_CLK_PER_100PS, x) +/* HSCH:HSCH_LEAK_LISTS:HSCH_TIMER_CFG */ +#define HSCH_HSCH_TIMER_CFG(g, r) __REG(TARGET_HSCH, 0, 1, 161664, g, 4, 32, 0, r, 4, 4) + +#define HSCH_HSCH_TIMER_CFG_LEAK_TIME GENMASK(17, 0) +#define HSCH_HSCH_TIMER_CFG_LEAK_TIME_SET(x)\ + FIELD_PREP(HSCH_HSCH_TIMER_CFG_LEAK_TIME, x) +#define HSCH_HSCH_TIMER_CFG_LEAK_TIME_GET(x)\ + FIELD_GET(HSCH_HSCH_TIMER_CFG_LEAK_TIME, x) + +/* HSCH:HSCH_LEAK_LISTS:HSCH_LEAK_CFG */ +#define HSCH_HSCH_LEAK_CFG(g, r) __REG(TARGET_HSCH, 0, 1, 161664, g, 4, 32, 16, r, 4, 4) + +#define HSCH_HSCH_LEAK_CFG_LEAK_FIRST GENMASK(16, 1) +#define HSCH_HSCH_LEAK_CFG_LEAK_FIRST_SET(x)\ + FIELD_PREP(HSCH_HSCH_LEAK_CFG_LEAK_FIRST, x) +#define HSCH_HSCH_LEAK_CFG_LEAK_FIRST_GET(x)\ + FIELD_GET(HSCH_HSCH_LEAK_CFG_LEAK_FIRST, x) + +#define HSCH_HSCH_LEAK_CFG_LEAK_ERR BIT(0) +#define HSCH_HSCH_LEAK_CFG_LEAK_ERR_SET(x)\ + FIELD_PREP(HSCH_HSCH_LEAK_CFG_LEAK_ERR, x) +#define HSCH_HSCH_LEAK_CFG_LEAK_ERR_GET(x)\ + FIELD_GET(HSCH_HSCH_LEAK_CFG_LEAK_ERR, x) + /* HSCH:SYSTEM:FLUSH_CTRL */ #define HSCH_FLUSH_CTRL __REG(TARGET_HSCH, 0, 1, 184000, 0, 1, 312, 4, 0, 1, 4) diff --git a/drivers/net/ethernet/microchip/sparx5/sparx5_netdev.c b/drivers/net/ethernet/microchip/sparx5/sparx5_netdev.c index af4d3e1f1a6d..19516ccad533 100644 --- a/drivers/net/ethernet/microchip/sparx5/sparx5_netdev.c +++ b/drivers/net/ethernet/microchip/sparx5/sparx5_netdev.c @@ -7,6 +7,7 @@ #include "sparx5_main_regs.h" #include "sparx5_main.h" #include "sparx5_port.h" +#include "sparx5_tc.h" /* The IFH bit position of the first VSTAX bit. This is because the * VSTAX bit positions in Data sheet is starting from zero. @@ -228,6 +229,7 @@ static const struct net_device_ops sparx5_port_netdev_ops = { .ndo_get_stats64 = sparx5_get_stats64, .ndo_get_port_parent_id = sparx5_get_port_parent_id, .ndo_eth_ioctl = sparx5_port_ioctl, + .ndo_setup_tc = sparx5_port_setup_tc, }; bool sparx5_netdevice_check(const struct net_device *dev) @@ -240,10 +242,14 @@ struct net_device *sparx5_create_netdev(struct sparx5 *sparx5, u32 portno) struct sparx5_port *spx5_port; struct net_device *ndev; - ndev = devm_alloc_etherdev(sparx5->dev, sizeof(struct sparx5_port)); + ndev = devm_alloc_etherdev_mqs(sparx5->dev, sizeof(struct sparx5_port), + SPX5_PRIOS, 1); if (!ndev) return ERR_PTR(-ENOMEM); + ndev->hw_features |= NETIF_F_HW_TC; + ndev->features |= NETIF_F_HW_TC; + SET_NETDEV_DEV(ndev, sparx5->dev); spx5_port = netdev_priv(ndev); spx5_port->ndev = ndev; diff --git a/drivers/net/ethernet/microchip/sparx5/sparx5_packet.c b/drivers/net/ethernet/microchip/sparx5/sparx5_packet.c index 21844beba72d..83c16ca5b30f 100644 --- a/drivers/net/ethernet/microchip/sparx5/sparx5_packet.c +++ b/drivers/net/ethernet/microchip/sparx5/sparx5_packet.c @@ -222,13 +222,13 @@ static int sparx5_inject(struct sparx5 *sparx5, return NETDEV_TX_OK; } -int sparx5_port_xmit_impl(struct sk_buff *skb, struct net_device *dev) +netdev_tx_t sparx5_port_xmit_impl(struct sk_buff *skb, struct net_device *dev) { struct net_device_stats *stats = &dev->stats; struct sparx5_port *port = netdev_priv(dev); struct sparx5 *sparx5 = port->sparx5; u32 ifh[IFH_LEN]; - int ret; + netdev_tx_t ret; memset(ifh, 0, IFH_LEN * 4); sparx5_set_port_ifh(ifh, port->portno); diff --git a/drivers/net/ethernet/microchip/sparx5/sparx5_qos.c b/drivers/net/ethernet/microchip/sparx5/sparx5_qos.c new file mode 100644 index 000000000000..1e79d0ef0cb8 --- /dev/null +++ b/drivers/net/ethernet/microchip/sparx5/sparx5_qos.c @@ -0,0 +1,513 @@ +// SPDX-License-Identifier: GPL-2.0+ +/* Microchip Sparx5 Switch driver + * + * Copyright (c) 2022 Microchip Technology Inc. and its subsidiaries. + */ + +#include <net/pkt_cls.h> + +#include "sparx5_main.h" +#include "sparx5_qos.h" + +/* Max rates for leak groups */ +static const u32 spx5_hsch_max_group_rate[SPX5_HSCH_LEAK_GRP_CNT] = { + 1048568, /* 1.049 Gbps */ + 2621420, /* 2.621 Gbps */ + 10485680, /* 10.486 Gbps */ + 26214200 /* 26.214 Gbps */ +}; + +static struct sparx5_layer layers[SPX5_HSCH_LAYER_CNT]; + +static u32 sparx5_lg_get_leak_time(struct sparx5 *sparx5, u32 layer, u32 group) +{ + u32 value; + + value = spx5_rd(sparx5, HSCH_HSCH_TIMER_CFG(layer, group)); + return HSCH_HSCH_TIMER_CFG_LEAK_TIME_GET(value); +} + +static void sparx5_lg_set_leak_time(struct sparx5 *sparx5, u32 layer, u32 group, + u32 leak_time) +{ + spx5_wr(HSCH_HSCH_TIMER_CFG_LEAK_TIME_SET(leak_time), sparx5, + HSCH_HSCH_TIMER_CFG(layer, group)); +} + +static u32 sparx5_lg_get_first(struct sparx5 *sparx5, u32 layer, u32 group) +{ + u32 value; + + value = spx5_rd(sparx5, HSCH_HSCH_LEAK_CFG(layer, group)); + return HSCH_HSCH_LEAK_CFG_LEAK_FIRST_GET(value); +} + +static u32 sparx5_lg_get_next(struct sparx5 *sparx5, u32 layer, u32 group, + u32 idx) + +{ + u32 value; + + value = spx5_rd(sparx5, HSCH_SE_CONNECT(idx)); + return HSCH_SE_CONNECT_SE_LEAK_LINK_GET(value); +} + +static u32 sparx5_lg_get_last(struct sparx5 *sparx5, u32 layer, u32 group) +{ + u32 itr, next; + + itr = sparx5_lg_get_first(sparx5, layer, group); + + for (;;) { + next = sparx5_lg_get_next(sparx5, layer, group, itr); + if (itr == next) + return itr; + + itr = next; + } +} + +static bool sparx5_lg_is_last(struct sparx5 *sparx5, u32 layer, u32 group, + u32 idx) +{ + return idx == sparx5_lg_get_next(sparx5, layer, group, idx); +} + +static bool sparx5_lg_is_first(struct sparx5 *sparx5, u32 layer, u32 group, + u32 idx) +{ + return idx == sparx5_lg_get_first(sparx5, layer, group); +} + +static bool sparx5_lg_is_empty(struct sparx5 *sparx5, u32 layer, u32 group) +{ + return sparx5_lg_get_leak_time(sparx5, layer, group) == 0; +} + +static bool sparx5_lg_is_singular(struct sparx5 *sparx5, u32 layer, u32 group) +{ + if (sparx5_lg_is_empty(sparx5, layer, group)) + return false; + + return sparx5_lg_get_first(sparx5, layer, group) == + sparx5_lg_get_last(sparx5, layer, group); +} + +static void sparx5_lg_enable(struct sparx5 *sparx5, u32 layer, u32 group, + u32 leak_time) +{ + sparx5_lg_set_leak_time(sparx5, layer, group, leak_time); +} + +static void sparx5_lg_disable(struct sparx5 *sparx5, u32 layer, u32 group) +{ + sparx5_lg_set_leak_time(sparx5, layer, group, 0); +} + +static int sparx5_lg_get_group_by_index(struct sparx5 *sparx5, u32 layer, + u32 idx, u32 *group) +{ + u32 itr, next; + int i; + + for (i = 0; i < SPX5_HSCH_LEAK_GRP_CNT; i++) { + if (sparx5_lg_is_empty(sparx5, layer, i)) + continue; + + itr = sparx5_lg_get_first(sparx5, layer, i); + + for (;;) { + next = sparx5_lg_get_next(sparx5, layer, i, itr); + + if (itr == idx) { + *group = i; + return 0; /* Found it */ + } + if (itr == next) + break; /* Was not found */ + + itr = next; + } + } + + return -1; +} + +static int sparx5_lg_get_group_by_rate(u32 layer, u32 rate, u32 *group) +{ + struct sparx5_layer *l = &layers[layer]; + struct sparx5_lg *lg; + u32 i; + + for (i = 0; i < SPX5_HSCH_LEAK_GRP_CNT; i++) { + lg = &l->leak_groups[i]; + if (rate <= lg->max_rate) { + *group = i; + return 0; + } + } + + return -1; +} + +static int sparx5_lg_get_adjacent(struct sparx5 *sparx5, u32 layer, u32 group, + u32 idx, u32 *prev, u32 *next, u32 *first) +{ + u32 itr; + + *first = sparx5_lg_get_first(sparx5, layer, group); + *prev = *first; + *next = *first; + itr = *first; + + for (;;) { + *next = sparx5_lg_get_next(sparx5, layer, group, itr); + + if (itr == idx) + return 0; /* Found it */ + + if (itr == *next) + return -1; /* Was not found */ + + *prev = itr; + itr = *next; + } + + return -1; +} + +static int sparx5_lg_conf_set(struct sparx5 *sparx5, u32 layer, u32 group, + u32 se_first, u32 idx, u32 idx_next, bool empty) +{ + u32 leak_time = layers[layer].leak_groups[group].leak_time; + + /* Stop leaking */ + sparx5_lg_disable(sparx5, layer, group); + + if (empty) + return 0; + + /* Select layer */ + spx5_rmw(HSCH_HSCH_CFG_CFG_HSCH_LAYER_SET(layer), + HSCH_HSCH_CFG_CFG_HSCH_LAYER, sparx5, HSCH_HSCH_CFG_CFG); + + /* Link elements */ + spx5_wr(HSCH_SE_CONNECT_SE_LEAK_LINK_SET(idx_next), sparx5, + HSCH_SE_CONNECT(idx)); + + /* Set the first element. */ + spx5_rmw(HSCH_HSCH_LEAK_CFG_LEAK_FIRST_SET(se_first), + HSCH_HSCH_LEAK_CFG_LEAK_FIRST, sparx5, + HSCH_HSCH_LEAK_CFG(layer, group)); + + /* Start leaking */ + sparx5_lg_enable(sparx5, layer, group, leak_time); + + return 0; +} + +static int sparx5_lg_del(struct sparx5 *sparx5, u32 layer, u32 group, u32 idx) +{ + u32 first, next, prev; + bool empty = false; + + /* idx *must* be present in the leak group */ + WARN_ON(sparx5_lg_get_adjacent(sparx5, layer, group, idx, &prev, &next, + &first) < 0); + + if (sparx5_lg_is_singular(sparx5, layer, group)) { + empty = true; + } else if (sparx5_lg_is_last(sparx5, layer, group, idx)) { + /* idx is removed, prev is now last */ + idx = prev; + next = prev; + } else if (sparx5_lg_is_first(sparx5, layer, group, idx)) { + /* idx is removed and points to itself, first is next */ + first = next; + next = idx; + } else { + /* Next is not touched */ + idx = prev; + } + + return sparx5_lg_conf_set(sparx5, layer, group, first, idx, next, + empty); +} + +static int sparx5_lg_add(struct sparx5 *sparx5, u32 layer, u32 new_group, + u32 idx) +{ + u32 first, next, old_group; + + pr_debug("ADD: layer: %d, new_group: %d, idx: %d", layer, new_group, + idx); + + /* Is this SE already shaping ? */ + if (sparx5_lg_get_group_by_index(sparx5, layer, idx, &old_group) >= 0) { + if (old_group != new_group) { + /* Delete from old group */ + sparx5_lg_del(sparx5, layer, old_group, idx); + } else { + /* Nothing to do here */ + return 0; + } + } + + /* We always add to head of the list */ + first = idx; + + if (sparx5_lg_is_empty(sparx5, layer, new_group)) + next = idx; + else + next = sparx5_lg_get_first(sparx5, layer, new_group); + + return sparx5_lg_conf_set(sparx5, layer, new_group, first, idx, next, + false); +} + +static int sparx5_shaper_conf_set(struct sparx5_port *port, + const struct sparx5_shaper *sh, u32 layer, + u32 idx, u32 group) +{ + int (*sparx5_lg_action)(struct sparx5 *, u32, u32, u32); + struct sparx5 *sparx5 = port->sparx5; + + if (!sh->rate && !sh->burst) + sparx5_lg_action = &sparx5_lg_del; + else + sparx5_lg_action = &sparx5_lg_add; + + /* Select layer */ + spx5_rmw(HSCH_HSCH_CFG_CFG_HSCH_LAYER_SET(layer), + HSCH_HSCH_CFG_CFG_HSCH_LAYER, sparx5, HSCH_HSCH_CFG_CFG); + + /* Set frame mode */ + spx5_rmw(HSCH_SE_CFG_SE_FRM_MODE_SET(sh->mode), HSCH_SE_CFG_SE_FRM_MODE, + sparx5, HSCH_SE_CFG(idx)); + + /* Set committed rate and burst */ + spx5_wr(HSCH_CIR_CFG_CIR_RATE_SET(sh->rate) | + HSCH_CIR_CFG_CIR_BURST_SET(sh->burst), + sparx5, HSCH_CIR_CFG(idx)); + + /* This has to be done after the shaper configuration has been set */ + sparx5_lg_action(sparx5, layer, group, idx); + + return 0; +} + +static u32 sparx5_weight_to_hw_cost(u32 weight_min, u32 weight) +{ + return ((((SPX5_DWRR_COST_MAX << 4) * weight_min / weight) + 8) >> 4) - + 1; +} + +static int sparx5_dwrr_conf_set(struct sparx5_port *port, + struct sparx5_dwrr *dwrr) +{ + int i; + + spx5_rmw(HSCH_HSCH_CFG_CFG_HSCH_LAYER_SET(2) | + HSCH_HSCH_CFG_CFG_CFG_SE_IDX_SET(port->portno), + HSCH_HSCH_CFG_CFG_HSCH_LAYER | HSCH_HSCH_CFG_CFG_CFG_SE_IDX, + port->sparx5, HSCH_HSCH_CFG_CFG); + + /* Number of *lower* indexes that are arbitrated dwrr */ + spx5_rmw(HSCH_SE_CFG_SE_DWRR_CNT_SET(dwrr->count), + HSCH_SE_CFG_SE_DWRR_CNT, port->sparx5, + HSCH_SE_CFG(port->portno)); + + for (i = 0; i < dwrr->count; i++) { + spx5_rmw(HSCH_DWRR_ENTRY_DWRR_COST_SET(dwrr->cost[i]), + HSCH_DWRR_ENTRY_DWRR_COST, port->sparx5, + HSCH_DWRR_ENTRY(i)); + } + + return 0; +} + +static int sparx5_leak_groups_init(struct sparx5 *sparx5) +{ + struct sparx5_layer *layer; + u32 sys_clk_per_100ps; + struct sparx5_lg *lg; + u32 leak_time_us; + int i, ii; + + sys_clk_per_100ps = spx5_rd(sparx5, HSCH_SYS_CLK_PER); + + for (i = 0; i < SPX5_HSCH_LAYER_CNT; i++) { + layer = &layers[i]; + for (ii = 0; ii < SPX5_HSCH_LEAK_GRP_CNT; ii++) { + lg = &layer->leak_groups[ii]; + lg->max_rate = spx5_hsch_max_group_rate[ii]; + + /* Calculate the leak time in us, to serve a maximum + * rate of 'max_rate' for this group + */ + leak_time_us = (SPX5_SE_RATE_MAX * 1000) / lg->max_rate; + + /* Hardware wants leak time in ns */ + lg->leak_time = 1000 * leak_time_us; + + /* Calculate resolution */ + lg->resolution = 1000 / leak_time_us; + + /* Maximum number of shapers that can be served by + * this leak group + */ + lg->max_ses = (1000 * leak_time_us) / sys_clk_per_100ps; + + /* Example: + * Wanted bandwidth is 100Mbit: + * + * 100 mbps can be served by leak group zero. + * + * leak_time is 125000 ns. + * resolution is: 8 + * + * cir = 100000 / 8 = 12500 + * leaks_pr_sec = 125000 / 10^9 = 8000 + * bw = 12500 * 8000 = 10^8 (100 Mbit) + */ + + /* Disable by default - this also indicates an empty + * leak group + */ + sparx5_lg_disable(sparx5, i, ii); + } + } + + return 0; +} + +int sparx5_qos_init(struct sparx5 *sparx5) +{ + int ret; + + ret = sparx5_leak_groups_init(sparx5); + if (ret < 0) + return ret; + + return 0; +} + +int sparx5_tc_mqprio_add(struct net_device *ndev, u8 num_tc) +{ + int i; + + if (num_tc != SPX5_PRIOS) { + netdev_err(ndev, "Only %d traffic classes supported\n", + SPX5_PRIOS); + return -EINVAL; + } + + netdev_set_num_tc(ndev, num_tc); + + for (i = 0; i < num_tc; i++) + netdev_set_tc_queue(ndev, i, 1, i); + + netdev_dbg(ndev, "dev->num_tc %u dev->real_num_tx_queues %u\n", + ndev->num_tc, ndev->real_num_tx_queues); + + return 0; +} + +int sparx5_tc_mqprio_del(struct net_device *ndev) +{ + netdev_reset_tc(ndev); + + netdev_dbg(ndev, "dev->num_tc %u dev->real_num_tx_queues %u\n", + ndev->num_tc, ndev->real_num_tx_queues); + + return 0; +} + +int sparx5_tc_tbf_add(struct sparx5_port *port, + struct tc_tbf_qopt_offload_replace_params *params, + u32 layer, u32 idx) +{ + struct sparx5_shaper sh = { + .mode = SPX5_SE_MODE_DATARATE, + .rate = div_u64(params->rate.rate_bytes_ps, 1000) * 8, + .burst = params->max_size, + }; + struct sparx5_lg *lg; + u32 group; + + /* Find suitable group for this se */ + if (sparx5_lg_get_group_by_rate(layer, sh.rate, &group) < 0) { + pr_debug("Could not find leak group for se with rate: %d", + sh.rate); + return -EINVAL; + } + + lg = &layers[layer].leak_groups[group]; + + pr_debug("Found matching group (speed: %d)\n", lg->max_rate); + + if (sh.rate < SPX5_SE_RATE_MIN || sh.burst < SPX5_SE_BURST_MIN) + return -EINVAL; + + /* Calculate committed rate and burst */ + sh.rate = DIV_ROUND_UP(sh.rate, lg->resolution); + sh.burst = DIV_ROUND_UP(sh.burst, SPX5_SE_BURST_UNIT); + + if (sh.rate > SPX5_SE_RATE_MAX || sh.burst > SPX5_SE_BURST_MAX) + return -EINVAL; + + return sparx5_shaper_conf_set(port, &sh, layer, idx, group); +} + +int sparx5_tc_tbf_del(struct sparx5_port *port, u32 layer, u32 idx) +{ + struct sparx5_shaper sh = {0}; + u32 group; + + sparx5_lg_get_group_by_index(port->sparx5, layer, idx, &group); + + return sparx5_shaper_conf_set(port, &sh, layer, idx, group); +} + +int sparx5_tc_ets_add(struct sparx5_port *port, + struct tc_ets_qopt_offload_replace_params *params) +{ + struct sparx5_dwrr dwrr = {0}; + /* Minimum weight for each iteration */ + unsigned int w_min = 100; + int i; + + /* Find minimum weight for all dwrr bands */ + for (i = 0; i < SPX5_PRIOS; i++) { + if (params->quanta[i] == 0) + continue; + w_min = min(w_min, params->weights[i]); + } + + for (i = 0; i < SPX5_PRIOS; i++) { + /* Strict band; skip */ + if (params->quanta[i] == 0) + continue; + + dwrr.count++; + + /* On the sparx5, bands with higher indexes are preferred and + * arbitrated strict. Strict bands are put in the lower indexes, + * by tc, so we reverse the bands here. + * + * Also convert the weight to something the hardware + * understands. + */ + dwrr.cost[SPX5_PRIOS - i - 1] = + sparx5_weight_to_hw_cost(w_min, params->weights[i]); + } + + return sparx5_dwrr_conf_set(port, &dwrr); +} + +int sparx5_tc_ets_del(struct sparx5_port *port) +{ + struct sparx5_dwrr dwrr = {0}; + + return sparx5_dwrr_conf_set(port, &dwrr); +} diff --git a/drivers/net/ethernet/microchip/sparx5/sparx5_qos.h b/drivers/net/ethernet/microchip/sparx5/sparx5_qos.h new file mode 100644 index 000000000000..ced35033a6c5 --- /dev/null +++ b/drivers/net/ethernet/microchip/sparx5/sparx5_qos.h @@ -0,0 +1,82 @@ +/* SPDX-License-Identifier: GPL-2.0+ */ +/* Microchip Sparx5 Switch driver + * + * Copyright (c) 2022 Microchip Technology Inc. and its subsidiaries. + */ + +#ifndef __SPARX5_QOS_H__ +#define __SPARX5_QOS_H__ + +#include <linux/netdevice.h> + +/* Number of Layers */ +#define SPX5_HSCH_LAYER_CNT 3 + +/* Scheduling elements per layer */ +#define SPX5_HSCH_L0_SE_CNT 5040 +#define SPX5_HSCH_L1_SE_CNT 64 +#define SPX5_HSCH_L2_SE_CNT 64 + +/* Calculate Layer 0 Scheduler Element when using normal hierarchy */ +#define SPX5_HSCH_L0_GET_IDX(port, queue) ((64 * (port)) + (8 * (queue))) + +/* Number of leak groups */ +#define SPX5_HSCH_LEAK_GRP_CNT 4 + +/* Scheduler modes */ +#define SPX5_SE_MODE_LINERATE 0 +#define SPX5_SE_MODE_DATARATE 1 + +/* Rate and burst */ +#define SPX5_SE_RATE_MAX 262143 +#define SPX5_SE_BURST_MAX 127 +#define SPX5_SE_RATE_MIN 1 +#define SPX5_SE_BURST_MIN 1 +#define SPX5_SE_BURST_UNIT 4096 + +/* Dwrr */ +#define SPX5_DWRR_COST_MAX 63 + +struct sparx5_shaper { + u32 mode; + u32 rate; + u32 burst; +}; + +struct sparx5_lg { + u32 max_rate; + u32 resolution; + u32 leak_time; + u32 max_ses; +}; + +struct sparx5_layer { + struct sparx5_lg leak_groups[SPX5_HSCH_LEAK_GRP_CNT]; +}; + +struct sparx5_dwrr { + u32 count; /* Number of inputs running dwrr */ + u8 cost[SPX5_PRIOS]; +}; + +int sparx5_qos_init(struct sparx5 *sparx5); + +/* Multi-Queue Priority */ +int sparx5_tc_mqprio_add(struct net_device *ndev, u8 num_tc); +int sparx5_tc_mqprio_del(struct net_device *ndev); + +/* Token Bucket Filter */ +struct tc_tbf_qopt_offload_replace_params; +int sparx5_tc_tbf_add(struct sparx5_port *port, + struct tc_tbf_qopt_offload_replace_params *params, + u32 layer, u32 idx); +int sparx5_tc_tbf_del(struct sparx5_port *port, u32 layer, u32 idx); + +/* Enhanced Transmission Selection */ +struct tc_ets_qopt_offload_replace_params; +int sparx5_tc_ets_add(struct sparx5_port *port, + struct tc_ets_qopt_offload_replace_params *params); + +int sparx5_tc_ets_del(struct sparx5_port *port); + +#endif /* __SPARX5_QOS_H__ */ diff --git a/drivers/net/ethernet/microchip/sparx5/sparx5_switchdev.c b/drivers/net/ethernet/microchip/sparx5/sparx5_switchdev.c index ec07f7d0528c..4af85d108a06 100644 --- a/drivers/net/ethernet/microchip/sparx5/sparx5_switchdev.c +++ b/drivers/net/ethernet/microchip/sparx5/sparx5_switchdev.c @@ -29,14 +29,23 @@ static int sparx5_port_attr_pre_bridge_flags(struct sparx5_port *port, return 0; } +static void sparx5_port_update_mcast_ip_flood(struct sparx5_port *port, bool flood_flag) +{ + bool should_flood = flood_flag || port->is_mrouter; + int pgid; + + for (pgid = PGID_IPV4_MC_DATA; pgid <= PGID_IPV6_MC_CTRL; pgid++) + sparx5_pgid_update_mask(port, pgid, should_flood); +} + static void sparx5_port_attr_bridge_flags(struct sparx5_port *port, struct switchdev_brport_flags flags) { - int pgid; + if (flags.mask & BR_MCAST_FLOOD) { + sparx5_pgid_update_mask(port, PGID_MC_FLOOD, !!(flags.val & BR_MCAST_FLOOD)); + sparx5_port_update_mcast_ip_flood(port, !!(flags.val & BR_MCAST_FLOOD)); + } - if (flags.mask & BR_MCAST_FLOOD) - for (pgid = PGID_MC_FLOOD; pgid <= PGID_IPV6_MC_CTRL; pgid++) - sparx5_pgid_update_mask(port, pgid, !!(flags.val & BR_MCAST_FLOOD)); if (flags.mask & BR_FLOOD) sparx5_pgid_update_mask(port, PGID_UC_FLOOD, !!(flags.val & BR_FLOOD)); if (flags.mask & BR_BCAST_FLOOD) @@ -82,6 +91,37 @@ static void sparx5_port_attr_ageing_set(struct sparx5_port *port, sparx5_set_ageing(port->sparx5, ageing_time); } +static void sparx5_port_attr_mrouter_set(struct sparx5_port *port, + struct net_device *orig_dev, + bool enable) +{ + struct sparx5 *sparx5 = port->sparx5; + struct sparx5_mdb_entry *e; + bool flood_flag; + + if ((enable && port->is_mrouter) || (!enable && !port->is_mrouter)) + return; + + /* Add/del mrouter port on all active mdb entries in HW. + * Don't change entry port mask, since that represents + * ports that actually joined that group. + */ + mutex_lock(&sparx5->mdb_lock); + list_for_each_entry(e, &sparx5->mdb_entries, list) { + if (!test_bit(port->portno, e->port_mask) && + ether_addr_is_ip_mcast(e->addr)) + sparx5_pgid_update_mask(port, e->pgid_idx, enable); + } + mutex_unlock(&sparx5->mdb_lock); + + /* Enable/disable flooding depending on if port is mrouter port + * or if mcast flood is enabled. + */ + port->is_mrouter = enable; + flood_flag = br_port_flag_is_set(port->ndev, BR_MCAST_FLOOD); + sparx5_port_update_mcast_ip_flood(port, flood_flag); +} + static int sparx5_port_attr_set(struct net_device *dev, const void *ctx, const struct switchdev_attr *attr, struct netlink_ext_ack *extack) @@ -110,6 +150,11 @@ static int sparx5_port_attr_set(struct net_device *dev, const void *ctx, port->vlan_aware = attr->u.vlan_filtering; sparx5_vlan_port_apply(port->sparx5, port); break; + case SWITCHDEV_ATTR_ID_PORT_MROUTER: + sparx5_port_attr_mrouter_set(port, + attr->orig_dev, + attr->u.mrouter); + break; default: return -EOPNOTSUPP; } @@ -386,16 +431,95 @@ static int sparx5_handle_port_vlan_add(struct net_device *dev, v->flags & BRIDGE_VLAN_INFO_UNTAGGED); } +static int sparx5_alloc_mdb_entry(struct sparx5 *sparx5, + const unsigned char *addr, + u16 vid, + struct sparx5_mdb_entry **entry_out) +{ + struct sparx5_mdb_entry *entry; + u16 pgid_idx; + int err; + + entry = kzalloc(sizeof(*entry), GFP_KERNEL); + if (!entry) + return -ENOMEM; + + err = sparx5_pgid_alloc_mcast(sparx5, &pgid_idx); + if (err) { + kfree(entry); + return err; + } + + memcpy(entry->addr, addr, ETH_ALEN); + entry->vid = vid; + entry->pgid_idx = pgid_idx; + + mutex_lock(&sparx5->mdb_lock); + list_add_tail(&entry->list, &sparx5->mdb_entries); + mutex_unlock(&sparx5->mdb_lock); + + *entry_out = entry; + return 0; +} + +static void sparx5_free_mdb_entry(struct sparx5 *sparx5, + const unsigned char *addr, + u16 vid) +{ + struct sparx5_mdb_entry *entry, *tmp; + + mutex_lock(&sparx5->mdb_lock); + list_for_each_entry_safe(entry, tmp, &sparx5->mdb_entries, list) { + if ((vid == 0 || entry->vid == vid) && + ether_addr_equal(addr, entry->addr)) { + list_del(&entry->list); + + sparx5_pgid_free(sparx5, entry->pgid_idx); + kfree(entry); + goto out; + } + } + +out: + mutex_unlock(&sparx5->mdb_lock); +} + +static struct sparx5_mdb_entry *sparx5_mdb_get_entry(struct sparx5 *sparx5, + const unsigned char *addr, + u16 vid) +{ + struct sparx5_mdb_entry *e, *found = NULL; + + mutex_lock(&sparx5->mdb_lock); + list_for_each_entry(e, &sparx5->mdb_entries, list) { + if (ether_addr_equal(e->addr, addr) && e->vid == vid) { + found = e; + goto out; + } + } + +out: + mutex_unlock(&sparx5->mdb_lock); + return found; +} + +static void sparx5_cpu_copy_ena(struct sparx5 *spx5, u16 pgid, bool enable) +{ + spx5_rmw(ANA_AC_PGID_MISC_CFG_PGID_CPU_COPY_ENA_SET(enable), + ANA_AC_PGID_MISC_CFG_PGID_CPU_COPY_ENA, spx5, + ANA_AC_PGID_MISC_CFG(pgid)); +} + static int sparx5_handle_port_mdb_add(struct net_device *dev, struct notifier_block *nb, const struct switchdev_obj_port_mdb *v) { struct sparx5_port *port = netdev_priv(dev); struct sparx5 *spx5 = port->sparx5; - u16 pgid_idx, vid; - u32 mact_entry; - bool is_host; - int res, err; + struct sparx5_mdb_entry *entry; + bool is_host, is_new; + int err, i; + u16 vid; if (!sparx5_netdevice_check(dev)) return -EOPNOTSUPP; @@ -410,66 +534,36 @@ static int sparx5_handle_port_mdb_add(struct net_device *dev, else vid = v->vid; - res = sparx5_mact_find(spx5, v->addr, vid, &mact_entry); - - if (res == 0) { - pgid_idx = LRN_MAC_ACCESS_CFG_2_MAC_ENTRY_ADDR_GET(mact_entry); - - /* MC_IDX starts after the port masks in the PGID table */ - pgid_idx += SPX5_PORTS; - - if (is_host) - spx5_rmw(ANA_AC_PGID_MISC_CFG_PGID_CPU_COPY_ENA_SET(1), - ANA_AC_PGID_MISC_CFG_PGID_CPU_COPY_ENA, spx5, - ANA_AC_PGID_MISC_CFG(pgid_idx)); - else - sparx5_pgid_update_mask(port, pgid_idx, true); - - } else { - err = sparx5_pgid_alloc_mcast(spx5, &pgid_idx); - if (err) { - netdev_warn(dev, "multicast pgid table full\n"); + is_new = false; + entry = sparx5_mdb_get_entry(spx5, v->addr, vid); + if (!entry) { + err = sparx5_alloc_mdb_entry(spx5, v->addr, vid, &entry); + is_new = true; + if (err) return err; - } - - if (is_host) - spx5_rmw(ANA_AC_PGID_MISC_CFG_PGID_CPU_COPY_ENA_SET(1), - ANA_AC_PGID_MISC_CFG_PGID_CPU_COPY_ENA, spx5, - ANA_AC_PGID_MISC_CFG(pgid_idx)); - else - sparx5_pgid_update_mask(port, pgid_idx, true); - - err = sparx5_mact_learn(spx5, pgid_idx, v->addr, vid); - - if (err) { - netdev_warn(dev, "could not learn mac address %pM\n", v->addr); - sparx5_pgid_free(spx5, pgid_idx); - sparx5_pgid_update_mask(port, pgid_idx, false); - return err; - } } - return 0; -} + mutex_lock(&spx5->mdb_lock); + + /* Add any mrouter ports to the new entry */ + if (is_new && ether_addr_is_ip_mcast(v->addr)) + for (i = 0; i < SPX5_PORTS; i++) + if (spx5->ports[i] && spx5->ports[i]->is_mrouter) + sparx5_pgid_update_mask(spx5->ports[i], + entry->pgid_idx, + true); + + if (is_host && !entry->cpu_copy) { + sparx5_cpu_copy_ena(spx5, entry->pgid_idx, true); + entry->cpu_copy = true; + } else if (!is_host) { + sparx5_pgid_update_mask(port, entry->pgid_idx, true); + set_bit(port->portno, entry->port_mask); + } + mutex_unlock(&spx5->mdb_lock); -static int sparx5_mdb_del_entry(struct net_device *dev, - struct sparx5 *spx5, - const unsigned char mac[ETH_ALEN], - const u16 vid, - u16 pgid_idx) -{ - int err; + sparx5_mact_learn(spx5, entry->pgid_idx, entry->addr, entry->vid); - err = sparx5_mact_forget(spx5, mac, vid); - if (err) { - netdev_warn(dev, "could not forget mac address %pM", mac); - return err; - } - err = sparx5_pgid_free(spx5, pgid_idx); - if (err) { - netdev_err(dev, "attempted to free already freed pgid\n"); - return err; - } return 0; } @@ -479,42 +573,45 @@ static int sparx5_handle_port_mdb_del(struct net_device *dev, { struct sparx5_port *port = netdev_priv(dev); struct sparx5 *spx5 = port->sparx5; - u16 pgid_idx, vid; - u32 mact_entry, res, pgid_entry[3], misc_cfg; - bool host_ena; + struct sparx5_mdb_entry *entry; + bool is_host; + u16 vid; if (!sparx5_netdevice_check(dev)) return -EOPNOTSUPP; + is_host = netif_is_bridge_master(v->obj.orig_dev); + if (!br_vlan_enabled(spx5->hw_bridge_dev)) vid = 1; else vid = v->vid; - res = sparx5_mact_find(spx5, v->addr, vid, &mact_entry); - - if (res == 0) { - pgid_idx = LRN_MAC_ACCESS_CFG_2_MAC_ENTRY_ADDR_GET(mact_entry); - - /* MC_IDX starts after the port masks in the PGID table */ - pgid_idx += SPX5_PORTS; - - if (netif_is_bridge_master(v->obj.orig_dev)) - spx5_rmw(ANA_AC_PGID_MISC_CFG_PGID_CPU_COPY_ENA_SET(0), - ANA_AC_PGID_MISC_CFG_PGID_CPU_COPY_ENA, spx5, - ANA_AC_PGID_MISC_CFG(pgid_idx)); - else - sparx5_pgid_update_mask(port, pgid_idx, false); + entry = sparx5_mdb_get_entry(spx5, v->addr, vid); + if (!entry) + return 0; - misc_cfg = spx5_rd(spx5, ANA_AC_PGID_MISC_CFG(pgid_idx)); - host_ena = ANA_AC_PGID_MISC_CFG_PGID_CPU_COPY_ENA_GET(misc_cfg); + mutex_lock(&spx5->mdb_lock); + if (is_host && entry->cpu_copy) { + sparx5_cpu_copy_ena(spx5, entry->pgid_idx, false); + entry->cpu_copy = false; + } else if (!is_host) { + clear_bit(port->portno, entry->port_mask); - sparx5_pgid_read_mask(spx5, pgid_idx, pgid_entry); - if (bitmap_empty((unsigned long *)pgid_entry, SPX5_PORTS) && !host_ena) - /* No ports or CPU are in MC group. Remove entry */ - return sparx5_mdb_del_entry(dev, spx5, v->addr, vid, pgid_idx); + /* Port not mrouter port or addr is L2 mcast, remove port from mask. */ + if (!port->is_mrouter || !ether_addr_is_ip_mcast(v->addr)) + sparx5_pgid_update_mask(port, entry->pgid_idx, false); + } + mutex_unlock(&spx5->mdb_lock); + + if (bitmap_empty(entry->port_mask, SPX5_PORTS) && !entry->cpu_copy) { + /* Clear pgid in case mrouter ports exists + * that are not part of the group. + */ + sparx5_pgid_clear(spx5, entry->pgid_idx); + sparx5_mact_forget(spx5, entry->addr, entry->vid); + sparx5_free_mdb_entry(spx5, entry->addr, entry->vid); } - return 0; } diff --git a/drivers/net/ethernet/microchip/sparx5/sparx5_tc.c b/drivers/net/ethernet/microchip/sparx5/sparx5_tc.c new file mode 100644 index 000000000000..e05429c751ee --- /dev/null +++ b/drivers/net/ethernet/microchip/sparx5/sparx5_tc.c @@ -0,0 +1,125 @@ +// SPDX-License-Identifier: GPL-2.0+ +/* Microchip Sparx5 Switch driver + * + * Copyright (c) 2022 Microchip Technology Inc. and its subsidiaries. + */ + +#include <net/pkt_cls.h> + +#include "sparx5_tc.h" +#include "sparx5_main.h" +#include "sparx5_qos.h" + +static void sparx5_tc_get_layer_and_idx(u32 parent, u32 portno, u32 *layer, + u32 *idx) +{ + if (parent == TC_H_ROOT) { + *layer = 2; + *idx = portno; + } else { + u32 queue = TC_H_MIN(parent) - 1; + *layer = 0; + *idx = SPX5_HSCH_L0_GET_IDX(portno, queue); + } +} + +static int sparx5_tc_setup_qdisc_mqprio(struct net_device *ndev, + struct tc_mqprio_qopt_offload *m) +{ + m->qopt.hw = TC_MQPRIO_HW_OFFLOAD_TCS; + + if (m->qopt.num_tc == 0) + return sparx5_tc_mqprio_del(ndev); + else + return sparx5_tc_mqprio_add(ndev, m->qopt.num_tc); +} + +static int sparx5_tc_setup_qdisc_tbf(struct net_device *ndev, + struct tc_tbf_qopt_offload *qopt) +{ + struct sparx5_port *port = netdev_priv(ndev); + u32 layer, se_idx; + + sparx5_tc_get_layer_and_idx(qopt->parent, port->portno, &layer, + &se_idx); + + switch (qopt->command) { + case TC_TBF_REPLACE: + return sparx5_tc_tbf_add(port, &qopt->replace_params, layer, + se_idx); + case TC_TBF_DESTROY: + return sparx5_tc_tbf_del(port, layer, se_idx); + case TC_TBF_STATS: + return -EOPNOTSUPP; + default: + return -EOPNOTSUPP; + } + + return -EOPNOTSUPP; +} + +static int sparx5_tc_setup_qdisc_ets(struct net_device *ndev, + struct tc_ets_qopt_offload *qopt) +{ + struct tc_ets_qopt_offload_replace_params *params = + &qopt->replace_params; + struct sparx5_port *port = netdev_priv(ndev); + int i; + + /* Only allow ets on ports */ + if (qopt->parent != TC_H_ROOT) + return -EOPNOTSUPP; + + switch (qopt->command) { + case TC_ETS_REPLACE: + + /* We support eight priorities */ + if (params->bands != SPX5_PRIOS) + return -EOPNOTSUPP; + + /* Sanity checks */ + for (i = 0; i < SPX5_PRIOS; ++i) { + /* Priority map is *always* reverse e.g: 7 6 5 .. 0 */ + if (params->priomap[i] != (7 - i)) + return -EOPNOTSUPP; + /* Throw an error if we receive zero weights by tc */ + if (params->quanta[i] && params->weights[i] == 0) { + pr_err("Invalid ets configuration; band %d has weight zero", + i); + return -EINVAL; + } + } + + sparx5_tc_ets_add(port, params); + break; + case TC_ETS_DESTROY: + + sparx5_tc_ets_del(port); + + break; + case TC_ETS_GRAFT: + return -EOPNOTSUPP; + + default: + return -EOPNOTSUPP; + } + + return -EOPNOTSUPP; +} + +int sparx5_port_setup_tc(struct net_device *ndev, enum tc_setup_type type, + void *type_data) +{ + switch (type) { + case TC_SETUP_QDISC_MQPRIO: + return sparx5_tc_setup_qdisc_mqprio(ndev, type_data); + case TC_SETUP_QDISC_TBF: + return sparx5_tc_setup_qdisc_tbf(ndev, type_data); + case TC_SETUP_QDISC_ETS: + return sparx5_tc_setup_qdisc_ets(ndev, type_data); + default: + return -EOPNOTSUPP; + } + + return 0; +} diff --git a/drivers/net/ethernet/microchip/sparx5/sparx5_tc.h b/drivers/net/ethernet/microchip/sparx5/sparx5_tc.h new file mode 100644 index 000000000000..5b55e11b77e1 --- /dev/null +++ b/drivers/net/ethernet/microchip/sparx5/sparx5_tc.h @@ -0,0 +1,15 @@ +/* SPDX-License-Identifier: GPL-2.0+ */ +/* Microchip Sparx5 Switch driver + * + * Copyright (c) 2022 Microchip Technology Inc. and its subsidiaries. + */ + +#ifndef __SPARX5_TC_H__ +#define __SPARX5_TC_H__ + +#include <linux/netdevice.h> + +int sparx5_port_setup_tc(struct net_device *ndev, enum tc_setup_type type, + void *type_data); + +#endif /* __SPARX5_TC_H__ */ diff --git a/drivers/net/ethernet/microchip/sparx5/sparx5_vlan.c b/drivers/net/ethernet/microchip/sparx5/sparx5_vlan.c index 37e4ac965849..34f954bbf815 100644 --- a/drivers/net/ethernet/microchip/sparx5/sparx5_vlan.c +++ b/drivers/net/ethernet/microchip/sparx5/sparx5_vlan.c @@ -138,6 +138,13 @@ void sparx5_pgid_update_mask(struct sparx5_port *port, int pgid, bool enable) } } +void sparx5_pgid_clear(struct sparx5 *spx5, int pgid) +{ + spx5_wr(0, spx5, ANA_AC_PGID_CFG(pgid)); + spx5_wr(0, spx5, ANA_AC_PGID_CFG1(pgid)); + spx5_wr(0, spx5, ANA_AC_PGID_CFG2(pgid)); +} + void sparx5_pgid_read_mask(struct sparx5 *spx5, int pgid, u32 portmask[3]) { portmask[0] = spx5_rd(spx5, ANA_AC_PGID_CFG(pgid)); diff --git a/drivers/net/ethernet/moxa/moxart_ether.c b/drivers/net/ethernet/moxa/moxart_ether.c index 9e57d23e57bf..3da99b62797d 100644 --- a/drivers/net/ethernet/moxa/moxart_ether.c +++ b/drivers/net/ethernet/moxa/moxart_ether.c @@ -29,12 +29,12 @@ #include "moxart_ether.h" -static inline void moxart_desc_write(u32 data, u32 *desc) +static inline void moxart_desc_write(u32 data, __le32 *desc) { *desc = cpu_to_le32(data); } -static inline u32 moxart_desc_read(u32 *desc) +static inline u32 moxart_desc_read(__le32 *desc) { return le32_to_cpu(*desc); } diff --git a/drivers/net/ethernet/mscc/Makefile b/drivers/net/ethernet/mscc/Makefile index 41b34a509308..5d435a565d4c 100644 --- a/drivers/net/ethernet/mscc/Makefile +++ b/drivers/net/ethernet/mscc/Makefile @@ -2,16 +2,17 @@ obj-$(CONFIG_MSCC_OCELOT_SWITCH_LIB) += mscc_ocelot_switch_lib.o mscc_ocelot_switch_lib-y := \ ocelot.o \ + ocelot_devlink.o \ + ocelot_flower.o \ ocelot_io.o \ ocelot_police.o \ - ocelot_vcap.o \ - ocelot_flower.o \ ocelot_ptp.o \ - ocelot_devlink.o \ + ocelot_stats.o \ + ocelot_vcap.o \ vsc7514_regs.o mscc_ocelot_switch_lib-$(CONFIG_BRIDGE_MRP) += ocelot_mrp.o obj-$(CONFIG_MSCC_OCELOT_SWITCH) += mscc_ocelot.o mscc_ocelot-y := \ ocelot_fdma.o \ - ocelot_vsc7514.o \ - ocelot_net.o + ocelot_net.o \ + ocelot_vsc7514.o diff --git a/drivers/net/ethernet/mscc/ocelot.c b/drivers/net/ethernet/mscc/ocelot.c index 8f8005bc6eea..13b14110a060 100644 --- a/drivers/net/ethernet/mscc/ocelot.c +++ b/drivers/net/ethernet/mscc/ocelot.c @@ -6,7 +6,6 @@ */ #include <linux/dsa/ocelot.h> #include <linux/if_bridge.h> -#include <linux/ptp_classify.h> #include <soc/mscc/ocelot_vcap.h> #include "ocelot.h" #include "ocelot_vcap.h" @@ -917,211 +916,6 @@ void ocelot_phylink_mac_link_up(struct ocelot *ocelot, int port, } EXPORT_SYMBOL_GPL(ocelot_phylink_mac_link_up); -static int ocelot_port_add_txtstamp_skb(struct ocelot *ocelot, int port, - struct sk_buff *clone) -{ - struct ocelot_port *ocelot_port = ocelot->ports[port]; - unsigned long flags; - - spin_lock_irqsave(&ocelot->ts_id_lock, flags); - - if (ocelot_port->ptp_skbs_in_flight == OCELOT_MAX_PTP_ID || - ocelot->ptp_skbs_in_flight == OCELOT_PTP_FIFO_SIZE) { - spin_unlock_irqrestore(&ocelot->ts_id_lock, flags); - return -EBUSY; - } - - skb_shinfo(clone)->tx_flags |= SKBTX_IN_PROGRESS; - /* Store timestamp ID in OCELOT_SKB_CB(clone)->ts_id */ - OCELOT_SKB_CB(clone)->ts_id = ocelot_port->ts_id; - - ocelot_port->ts_id++; - if (ocelot_port->ts_id == OCELOT_MAX_PTP_ID) - ocelot_port->ts_id = 0; - - ocelot_port->ptp_skbs_in_flight++; - ocelot->ptp_skbs_in_flight++; - - skb_queue_tail(&ocelot_port->tx_skbs, clone); - - spin_unlock_irqrestore(&ocelot->ts_id_lock, flags); - - return 0; -} - -static bool ocelot_ptp_is_onestep_sync(struct sk_buff *skb, - unsigned int ptp_class) -{ - struct ptp_header *hdr; - u8 msgtype, twostep; - - hdr = ptp_parse_header(skb, ptp_class); - if (!hdr) - return false; - - msgtype = ptp_get_msgtype(hdr, ptp_class); - twostep = hdr->flag_field[0] & 0x2; - - if (msgtype == PTP_MSGTYPE_SYNC && twostep == 0) - return true; - - return false; -} - -int ocelot_port_txtstamp_request(struct ocelot *ocelot, int port, - struct sk_buff *skb, - struct sk_buff **clone) -{ - struct ocelot_port *ocelot_port = ocelot->ports[port]; - u8 ptp_cmd = ocelot_port->ptp_cmd; - unsigned int ptp_class; - int err; - - /* Don't do anything if PTP timestamping not enabled */ - if (!ptp_cmd) - return 0; - - ptp_class = ptp_classify_raw(skb); - if (ptp_class == PTP_CLASS_NONE) - return -EINVAL; - - /* Store ptp_cmd in OCELOT_SKB_CB(skb)->ptp_cmd */ - if (ptp_cmd == IFH_REW_OP_ORIGIN_PTP) { - if (ocelot_ptp_is_onestep_sync(skb, ptp_class)) { - OCELOT_SKB_CB(skb)->ptp_cmd = ptp_cmd; - return 0; - } - - /* Fall back to two-step timestamping */ - ptp_cmd = IFH_REW_OP_TWO_STEP_PTP; - } - - if (ptp_cmd == IFH_REW_OP_TWO_STEP_PTP) { - *clone = skb_clone_sk(skb); - if (!(*clone)) - return -ENOMEM; - - err = ocelot_port_add_txtstamp_skb(ocelot, port, *clone); - if (err) - return err; - - OCELOT_SKB_CB(skb)->ptp_cmd = ptp_cmd; - OCELOT_SKB_CB(*clone)->ptp_class = ptp_class; - } - - return 0; -} -EXPORT_SYMBOL(ocelot_port_txtstamp_request); - -static void ocelot_get_hwtimestamp(struct ocelot *ocelot, - struct timespec64 *ts) -{ - unsigned long flags; - u32 val; - - spin_lock_irqsave(&ocelot->ptp_clock_lock, flags); - - /* Read current PTP time to get seconds */ - val = ocelot_read_rix(ocelot, PTP_PIN_CFG, TOD_ACC_PIN); - - val &= ~(PTP_PIN_CFG_SYNC | PTP_PIN_CFG_ACTION_MASK | PTP_PIN_CFG_DOM); - val |= PTP_PIN_CFG_ACTION(PTP_PIN_ACTION_SAVE); - ocelot_write_rix(ocelot, val, PTP_PIN_CFG, TOD_ACC_PIN); - ts->tv_sec = ocelot_read_rix(ocelot, PTP_PIN_TOD_SEC_LSB, TOD_ACC_PIN); - - /* Read packet HW timestamp from FIFO */ - val = ocelot_read(ocelot, SYS_PTP_TXSTAMP); - ts->tv_nsec = SYS_PTP_TXSTAMP_PTP_TXSTAMP(val); - - /* Sec has incremented since the ts was registered */ - if ((ts->tv_sec & 0x1) != !!(val & SYS_PTP_TXSTAMP_PTP_TXSTAMP_SEC)) - ts->tv_sec--; - - spin_unlock_irqrestore(&ocelot->ptp_clock_lock, flags); -} - -static bool ocelot_validate_ptp_skb(struct sk_buff *clone, u16 seqid) -{ - struct ptp_header *hdr; - - hdr = ptp_parse_header(clone, OCELOT_SKB_CB(clone)->ptp_class); - if (WARN_ON(!hdr)) - return false; - - return seqid == ntohs(hdr->sequence_id); -} - -void ocelot_get_txtstamp(struct ocelot *ocelot) -{ - int budget = OCELOT_PTP_QUEUE_SZ; - - while (budget--) { - struct sk_buff *skb, *skb_tmp, *skb_match = NULL; - struct skb_shared_hwtstamps shhwtstamps; - u32 val, id, seqid, txport; - struct ocelot_port *port; - struct timespec64 ts; - unsigned long flags; - - val = ocelot_read(ocelot, SYS_PTP_STATUS); - - /* Check if a timestamp can be retrieved */ - if (!(val & SYS_PTP_STATUS_PTP_MESS_VLD)) - break; - - WARN_ON(val & SYS_PTP_STATUS_PTP_OVFL); - - /* Retrieve the ts ID and Tx port */ - id = SYS_PTP_STATUS_PTP_MESS_ID_X(val); - txport = SYS_PTP_STATUS_PTP_MESS_TXPORT_X(val); - seqid = SYS_PTP_STATUS_PTP_MESS_SEQ_ID(val); - - port = ocelot->ports[txport]; - - spin_lock(&ocelot->ts_id_lock); - port->ptp_skbs_in_flight--; - ocelot->ptp_skbs_in_flight--; - spin_unlock(&ocelot->ts_id_lock); - - /* Retrieve its associated skb */ -try_again: - spin_lock_irqsave(&port->tx_skbs.lock, flags); - - skb_queue_walk_safe(&port->tx_skbs, skb, skb_tmp) { - if (OCELOT_SKB_CB(skb)->ts_id != id) - continue; - __skb_unlink(skb, &port->tx_skbs); - skb_match = skb; - break; - } - - spin_unlock_irqrestore(&port->tx_skbs.lock, flags); - - if (WARN_ON(!skb_match)) - continue; - - if (!ocelot_validate_ptp_skb(skb_match, seqid)) { - dev_err_ratelimited(ocelot->dev, - "port %d received stale TX timestamp for seqid %d, discarding\n", - txport, seqid); - dev_kfree_skb_any(skb); - goto try_again; - } - - /* Get the h/w timestamp */ - ocelot_get_hwtimestamp(ocelot, &ts); - - /* Set the timestamp into the skb */ - memset(&shhwtstamps, 0, sizeof(shhwtstamps)); - shhwtstamps.hwtstamp = ktime_set(ts.tv_sec, ts.tv_nsec); - skb_complete_tx_timestamp(skb_match, &shhwtstamps); - - /* Next ts */ - ocelot_write(ocelot, SYS_PTP_NXT_PTP_NXT, SYS_PTP_NXT); - } -} -EXPORT_SYMBOL(ocelot_get_txtstamp); - static int ocelot_rx_frame_word(struct ocelot *ocelot, u8 grp, bool ifh, u32 *rval) { @@ -1373,50 +1167,6 @@ int ocelot_fdb_del(struct ocelot *ocelot, int port, const unsigned char *addr, } EXPORT_SYMBOL(ocelot_fdb_del); -int ocelot_port_fdb_do_dump(const unsigned char *addr, u16 vid, - bool is_static, void *data) -{ - struct ocelot_dump_ctx *dump = data; - u32 portid = NETLINK_CB(dump->cb->skb).portid; - u32 seq = dump->cb->nlh->nlmsg_seq; - struct nlmsghdr *nlh; - struct ndmsg *ndm; - - if (dump->idx < dump->cb->args[2]) - goto skip; - - nlh = nlmsg_put(dump->skb, portid, seq, RTM_NEWNEIGH, - sizeof(*ndm), NLM_F_MULTI); - if (!nlh) - return -EMSGSIZE; - - ndm = nlmsg_data(nlh); - ndm->ndm_family = AF_BRIDGE; - ndm->ndm_pad1 = 0; - ndm->ndm_pad2 = 0; - ndm->ndm_flags = NTF_SELF; - ndm->ndm_type = 0; - ndm->ndm_ifindex = dump->dev->ifindex; - ndm->ndm_state = is_static ? NUD_NOARP : NUD_REACHABLE; - - if (nla_put(dump->skb, NDA_LLADDR, ETH_ALEN, addr)) - goto nla_put_failure; - - if (vid && nla_put_u16(dump->skb, NDA_VLAN, vid)) - goto nla_put_failure; - - nlmsg_end(dump->skb, nlh); - -skip: - dump->idx++; - return 0; - -nla_put_failure: - nlmsg_cancel(dump->skb, nlh); - return -EMSGSIZE; -} -EXPORT_SYMBOL(ocelot_port_fdb_do_dump); - /* Caller must hold &ocelot->mact_lock */ static int ocelot_mact_read(struct ocelot *ocelot, int port, int row, int col, struct ocelot_mact_entry *entry) @@ -1548,53 +1298,6 @@ int ocelot_fdb_dump(struct ocelot *ocelot, int port, } EXPORT_SYMBOL(ocelot_fdb_dump); -static void ocelot_populate_l2_ptp_trap_key(struct ocelot_vcap_filter *trap) -{ - trap->key_type = OCELOT_VCAP_KEY_ETYPE; - *(__be16 *)trap->key.etype.etype.value = htons(ETH_P_1588); - *(__be16 *)trap->key.etype.etype.mask = htons(0xffff); -} - -static void -ocelot_populate_ipv4_ptp_event_trap_key(struct ocelot_vcap_filter *trap) -{ - trap->key_type = OCELOT_VCAP_KEY_IPV4; - trap->key.ipv4.proto.value[0] = IPPROTO_UDP; - trap->key.ipv4.proto.mask[0] = 0xff; - trap->key.ipv4.dport.value = PTP_EV_PORT; - trap->key.ipv4.dport.mask = 0xffff; -} - -static void -ocelot_populate_ipv6_ptp_event_trap_key(struct ocelot_vcap_filter *trap) -{ - trap->key_type = OCELOT_VCAP_KEY_IPV6; - trap->key.ipv4.proto.value[0] = IPPROTO_UDP; - trap->key.ipv4.proto.mask[0] = 0xff; - trap->key.ipv6.dport.value = PTP_EV_PORT; - trap->key.ipv6.dport.mask = 0xffff; -} - -static void -ocelot_populate_ipv4_ptp_general_trap_key(struct ocelot_vcap_filter *trap) -{ - trap->key_type = OCELOT_VCAP_KEY_IPV4; - trap->key.ipv4.proto.value[0] = IPPROTO_UDP; - trap->key.ipv4.proto.mask[0] = 0xff; - trap->key.ipv4.dport.value = PTP_GEN_PORT; - trap->key.ipv4.dport.mask = 0xffff; -} - -static void -ocelot_populate_ipv6_ptp_general_trap_key(struct ocelot_vcap_filter *trap) -{ - trap->key_type = OCELOT_VCAP_KEY_IPV6; - trap->key.ipv4.proto.value[0] = IPPROTO_UDP; - trap->key.ipv4.proto.mask[0] = 0xff; - trap->key.ipv6.dport.value = PTP_GEN_PORT; - trap->key.ipv6.dport.mask = 0xffff; -} - int ocelot_trap_add(struct ocelot *ocelot, int port, unsigned long cookie, bool take_ts, void (*populate)(struct ocelot_vcap_filter *f)) @@ -1663,381 +1366,6 @@ int ocelot_trap_del(struct ocelot *ocelot, int port, unsigned long cookie) return ocelot_vcap_filter_replace(ocelot, trap); } -static int ocelot_l2_ptp_trap_add(struct ocelot *ocelot, int port) -{ - unsigned long l2_cookie = OCELOT_VCAP_IS2_L2_PTP_TRAP(ocelot); - - return ocelot_trap_add(ocelot, port, l2_cookie, true, - ocelot_populate_l2_ptp_trap_key); -} - -static int ocelot_l2_ptp_trap_del(struct ocelot *ocelot, int port) -{ - unsigned long l2_cookie = OCELOT_VCAP_IS2_L2_PTP_TRAP(ocelot); - - return ocelot_trap_del(ocelot, port, l2_cookie); -} - -static int ocelot_ipv4_ptp_trap_add(struct ocelot *ocelot, int port) -{ - unsigned long ipv4_gen_cookie = OCELOT_VCAP_IS2_IPV4_GEN_PTP_TRAP(ocelot); - unsigned long ipv4_ev_cookie = OCELOT_VCAP_IS2_IPV4_EV_PTP_TRAP(ocelot); - int err; - - err = ocelot_trap_add(ocelot, port, ipv4_ev_cookie, true, - ocelot_populate_ipv4_ptp_event_trap_key); - if (err) - return err; - - err = ocelot_trap_add(ocelot, port, ipv4_gen_cookie, false, - ocelot_populate_ipv4_ptp_general_trap_key); - if (err) - ocelot_trap_del(ocelot, port, ipv4_ev_cookie); - - return err; -} - -static int ocelot_ipv4_ptp_trap_del(struct ocelot *ocelot, int port) -{ - unsigned long ipv4_gen_cookie = OCELOT_VCAP_IS2_IPV4_GEN_PTP_TRAP(ocelot); - unsigned long ipv4_ev_cookie = OCELOT_VCAP_IS2_IPV4_EV_PTP_TRAP(ocelot); - int err; - - err = ocelot_trap_del(ocelot, port, ipv4_ev_cookie); - err |= ocelot_trap_del(ocelot, port, ipv4_gen_cookie); - return err; -} - -static int ocelot_ipv6_ptp_trap_add(struct ocelot *ocelot, int port) -{ - unsigned long ipv6_gen_cookie = OCELOT_VCAP_IS2_IPV6_GEN_PTP_TRAP(ocelot); - unsigned long ipv6_ev_cookie = OCELOT_VCAP_IS2_IPV6_EV_PTP_TRAP(ocelot); - int err; - - err = ocelot_trap_add(ocelot, port, ipv6_ev_cookie, true, - ocelot_populate_ipv6_ptp_event_trap_key); - if (err) - return err; - - err = ocelot_trap_add(ocelot, port, ipv6_gen_cookie, false, - ocelot_populate_ipv6_ptp_general_trap_key); - if (err) - ocelot_trap_del(ocelot, port, ipv6_ev_cookie); - - return err; -} - -static int ocelot_ipv6_ptp_trap_del(struct ocelot *ocelot, int port) -{ - unsigned long ipv6_gen_cookie = OCELOT_VCAP_IS2_IPV6_GEN_PTP_TRAP(ocelot); - unsigned long ipv6_ev_cookie = OCELOT_VCAP_IS2_IPV6_EV_PTP_TRAP(ocelot); - int err; - - err = ocelot_trap_del(ocelot, port, ipv6_ev_cookie); - err |= ocelot_trap_del(ocelot, port, ipv6_gen_cookie); - return err; -} - -static int ocelot_setup_ptp_traps(struct ocelot *ocelot, int port, - bool l2, bool l4) -{ - int err; - - if (l2) - err = ocelot_l2_ptp_trap_add(ocelot, port); - else - err = ocelot_l2_ptp_trap_del(ocelot, port); - if (err) - return err; - - if (l4) { - err = ocelot_ipv4_ptp_trap_add(ocelot, port); - if (err) - goto err_ipv4; - - err = ocelot_ipv6_ptp_trap_add(ocelot, port); - if (err) - goto err_ipv6; - } else { - err = ocelot_ipv4_ptp_trap_del(ocelot, port); - - err |= ocelot_ipv6_ptp_trap_del(ocelot, port); - } - if (err) - return err; - - return 0; - -err_ipv6: - ocelot_ipv4_ptp_trap_del(ocelot, port); -err_ipv4: - if (l2) - ocelot_l2_ptp_trap_del(ocelot, port); - return err; -} - -int ocelot_hwstamp_get(struct ocelot *ocelot, int port, struct ifreq *ifr) -{ - return copy_to_user(ifr->ifr_data, &ocelot->hwtstamp_config, - sizeof(ocelot->hwtstamp_config)) ? -EFAULT : 0; -} -EXPORT_SYMBOL(ocelot_hwstamp_get); - -int ocelot_hwstamp_set(struct ocelot *ocelot, int port, struct ifreq *ifr) -{ - struct ocelot_port *ocelot_port = ocelot->ports[port]; - bool l2 = false, l4 = false; - struct hwtstamp_config cfg; - int err; - - if (copy_from_user(&cfg, ifr->ifr_data, sizeof(cfg))) - return -EFAULT; - - /* Tx type sanity check */ - switch (cfg.tx_type) { - case HWTSTAMP_TX_ON: - ocelot_port->ptp_cmd = IFH_REW_OP_TWO_STEP_PTP; - break; - case HWTSTAMP_TX_ONESTEP_SYNC: - /* IFH_REW_OP_ONE_STEP_PTP updates the correctional field, we - * need to update the origin time. - */ - ocelot_port->ptp_cmd = IFH_REW_OP_ORIGIN_PTP; - break; - case HWTSTAMP_TX_OFF: - ocelot_port->ptp_cmd = 0; - break; - default: - return -ERANGE; - } - - mutex_lock(&ocelot->ptp_lock); - - switch (cfg.rx_filter) { - case HWTSTAMP_FILTER_NONE: - break; - case HWTSTAMP_FILTER_PTP_V2_L4_EVENT: - case HWTSTAMP_FILTER_PTP_V2_L4_SYNC: - case HWTSTAMP_FILTER_PTP_V2_L4_DELAY_REQ: - l4 = true; - break; - case HWTSTAMP_FILTER_PTP_V2_L2_EVENT: - case HWTSTAMP_FILTER_PTP_V2_L2_SYNC: - case HWTSTAMP_FILTER_PTP_V2_L2_DELAY_REQ: - l2 = true; - break; - case HWTSTAMP_FILTER_PTP_V2_EVENT: - case HWTSTAMP_FILTER_PTP_V2_SYNC: - case HWTSTAMP_FILTER_PTP_V2_DELAY_REQ: - l2 = true; - l4 = true; - break; - default: - mutex_unlock(&ocelot->ptp_lock); - return -ERANGE; - } - - err = ocelot_setup_ptp_traps(ocelot, port, l2, l4); - if (err) { - mutex_unlock(&ocelot->ptp_lock); - return err; - } - - if (l2 && l4) - cfg.rx_filter = HWTSTAMP_FILTER_PTP_V2_EVENT; - else if (l2) - cfg.rx_filter = HWTSTAMP_FILTER_PTP_V2_L2_EVENT; - else if (l4) - cfg.rx_filter = HWTSTAMP_FILTER_PTP_V2_L4_EVENT; - else - cfg.rx_filter = HWTSTAMP_FILTER_NONE; - - /* Commit back the result & save it */ - memcpy(&ocelot->hwtstamp_config, &cfg, sizeof(cfg)); - mutex_unlock(&ocelot->ptp_lock); - - return copy_to_user(ifr->ifr_data, &cfg, sizeof(cfg)) ? -EFAULT : 0; -} -EXPORT_SYMBOL(ocelot_hwstamp_set); - -void ocelot_get_strings(struct ocelot *ocelot, int port, u32 sset, u8 *data) -{ - int i; - - if (sset != ETH_SS_STATS) - return; - - for (i = 0; i < OCELOT_NUM_STATS; i++) { - if (ocelot->stats_layout[i].name[0] == '\0') - continue; - - memcpy(data + i * ETH_GSTRING_LEN, ocelot->stats_layout[i].name, - ETH_GSTRING_LEN); - } -} -EXPORT_SYMBOL(ocelot_get_strings); - -/* Caller must hold &ocelot->stats_lock */ -static int ocelot_port_update_stats(struct ocelot *ocelot, int port) -{ - unsigned int idx = port * OCELOT_NUM_STATS; - struct ocelot_stats_region *region; - int err, j; - - /* Configure the port to read the stats from */ - ocelot_write(ocelot, SYS_STAT_CFG_STAT_VIEW(port), SYS_STAT_CFG); - - list_for_each_entry(region, &ocelot->stats_regions, node) { - err = ocelot_bulk_read(ocelot, region->base, region->buf, - region->count); - if (err) - return err; - - for (j = 0; j < region->count; j++) { - u64 *stat = &ocelot->stats[idx + j]; - u64 val = region->buf[j]; - - if (val < (*stat & U32_MAX)) - *stat += (u64)1 << 32; - - *stat = (*stat & ~(u64)U32_MAX) + val; - } - - idx += region->count; - } - - return err; -} - -static void ocelot_check_stats_work(struct work_struct *work) -{ - struct delayed_work *del_work = to_delayed_work(work); - struct ocelot *ocelot = container_of(del_work, struct ocelot, - stats_work); - int i, err; - - spin_lock(&ocelot->stats_lock); - for (i = 0; i < ocelot->num_phys_ports; i++) { - err = ocelot_port_update_stats(ocelot, i); - if (err) - break; - } - spin_unlock(&ocelot->stats_lock); - - if (err) - dev_err(ocelot->dev, "Error %d updating ethtool stats\n", err); - - queue_delayed_work(ocelot->stats_queue, &ocelot->stats_work, - OCELOT_STATS_CHECK_DELAY); -} - -void ocelot_get_ethtool_stats(struct ocelot *ocelot, int port, u64 *data) -{ - int i, err; - - spin_lock(&ocelot->stats_lock); - - /* check and update now */ - err = ocelot_port_update_stats(ocelot, port); - - /* Copy all supported counters */ - for (i = 0; i < OCELOT_NUM_STATS; i++) { - int index = port * OCELOT_NUM_STATS + i; - - if (ocelot->stats_layout[i].name[0] == '\0') - continue; - - *data++ = ocelot->stats[index]; - } - - spin_unlock(&ocelot->stats_lock); - - if (err) - dev_err(ocelot->dev, "Error %d updating ethtool stats\n", err); -} -EXPORT_SYMBOL(ocelot_get_ethtool_stats); - -int ocelot_get_sset_count(struct ocelot *ocelot, int port, int sset) -{ - int i, num_stats = 0; - - if (sset != ETH_SS_STATS) - return -EOPNOTSUPP; - - for (i = 0; i < OCELOT_NUM_STATS; i++) - if (ocelot->stats_layout[i].name[0] != '\0') - num_stats++; - - return num_stats; -} -EXPORT_SYMBOL(ocelot_get_sset_count); - -static int ocelot_prepare_stats_regions(struct ocelot *ocelot) -{ - struct ocelot_stats_region *region = NULL; - unsigned int last; - int i; - - INIT_LIST_HEAD(&ocelot->stats_regions); - - for (i = 0; i < OCELOT_NUM_STATS; i++) { - if (ocelot->stats_layout[i].name[0] == '\0') - continue; - - if (region && ocelot->stats_layout[i].reg == last + 4) { - region->count++; - } else { - region = devm_kzalloc(ocelot->dev, sizeof(*region), - GFP_KERNEL); - if (!region) - return -ENOMEM; - - region->base = ocelot->stats_layout[i].reg; - region->count = 1; - list_add_tail(®ion->node, &ocelot->stats_regions); - } - - last = ocelot->stats_layout[i].reg; - } - - list_for_each_entry(region, &ocelot->stats_regions, node) { - region->buf = devm_kcalloc(ocelot->dev, region->count, - sizeof(*region->buf), GFP_KERNEL); - if (!region->buf) - return -ENOMEM; - } - - return 0; -} - -int ocelot_get_ts_info(struct ocelot *ocelot, int port, - struct ethtool_ts_info *info) -{ - info->phc_index = ocelot->ptp_clock ? - ptp_clock_index(ocelot->ptp_clock) : -1; - if (info->phc_index == -1) { - info->so_timestamping |= SOF_TIMESTAMPING_TX_SOFTWARE | - SOF_TIMESTAMPING_RX_SOFTWARE | - SOF_TIMESTAMPING_SOFTWARE; - return 0; - } - info->so_timestamping |= SOF_TIMESTAMPING_TX_SOFTWARE | - SOF_TIMESTAMPING_RX_SOFTWARE | - SOF_TIMESTAMPING_SOFTWARE | - SOF_TIMESTAMPING_TX_HARDWARE | - SOF_TIMESTAMPING_RX_HARDWARE | - SOF_TIMESTAMPING_RAW_HARDWARE; - info->tx_types = BIT(HWTSTAMP_TX_OFF) | BIT(HWTSTAMP_TX_ON) | - BIT(HWTSTAMP_TX_ONESTEP_SYNC); - info->rx_filters = BIT(HWTSTAMP_FILTER_NONE) | - BIT(HWTSTAMP_FILTER_PTP_V2_EVENT) | - BIT(HWTSTAMP_FILTER_PTP_V2_L2_EVENT) | - BIT(HWTSTAMP_FILTER_PTP_V2_L4_EVENT); - - return 0; -} -EXPORT_SYMBOL(ocelot_get_ts_info); - static u32 ocelot_get_bond_mask(struct ocelot *ocelot, struct net_device *bond) { u32 mask = 0; @@ -2061,7 +1389,7 @@ static u32 ocelot_get_bond_mask(struct ocelot *ocelot, struct net_device *bond) /* The logical port number of a LAG is equal to the lowest numbered physical * port ID present in that LAG. It may change if that port ever leaves the LAG. */ -static int ocelot_bond_get_id(struct ocelot *ocelot, struct net_device *bond) +int ocelot_bond_get_id(struct ocelot *ocelot, struct net_device *bond) { int bond_mask = ocelot_get_bond_mask(ocelot, bond); @@ -2070,7 +1398,18 @@ static int ocelot_bond_get_id(struct ocelot *ocelot, struct net_device *bond) return __ffs(bond_mask); } +EXPORT_SYMBOL_GPL(ocelot_bond_get_id); +/* Returns the mask of user ports assigned to this DSA tag_8021q CPU port. + * Note that when CPU ports are in a LAG, the user ports are assigned to the + * 'primary' CPU port, the one whose physical port number gives the logical + * port number of the LAG. + * + * We leave PGID_SRC poorly configured for the 'secondary' CPU port in the LAG + * (to which no user port is assigned), but it appears that forwarding from + * this secondary CPU port looks at the PGID_SRC associated with the logical + * port ID that it's assigned to, which *is* configured properly. + */ static u32 ocelot_dsa_8021q_cpu_assigned_ports(struct ocelot *ocelot, struct ocelot_port *cpu) { @@ -2087,9 +1426,15 @@ static u32 ocelot_dsa_8021q_cpu_assigned_ports(struct ocelot *ocelot, mask |= BIT(port); } + if (cpu->bond) + mask &= ~ocelot_get_bond_mask(ocelot, cpu->bond); + return mask; } +/* Returns the DSA tag_8021q CPU port that the given port is assigned to, + * or the bit mask of CPU ports if said CPU port is in a LAG. + */ u32 ocelot_port_assigned_dsa_8021q_cpu_mask(struct ocelot *ocelot, int port) { struct ocelot_port *ocelot_port = ocelot->ports[port]; @@ -2098,6 +1443,9 @@ u32 ocelot_port_assigned_dsa_8021q_cpu_mask(struct ocelot *ocelot, int port) if (!cpu_port) return 0; + if (cpu_port->bond) + return ocelot_get_bond_mask(ocelot, cpu_port->bond); + return BIT(cpu_port->index); } EXPORT_SYMBOL_GPL(ocelot_port_assigned_dsa_8021q_cpu_mask); @@ -2221,61 +1569,61 @@ static void ocelot_update_pgid_cpu(struct ocelot *ocelot) ocelot_write_rix(ocelot, pgid_cpu, ANA_PGID_PGID, PGID_CPU); } -void ocelot_port_assign_dsa_8021q_cpu(struct ocelot *ocelot, int port, - int cpu) +void ocelot_port_setup_dsa_8021q_cpu(struct ocelot *ocelot, int cpu) { struct ocelot_port *cpu_port = ocelot->ports[cpu]; u16 vid; mutex_lock(&ocelot->fwd_domain_lock); - ocelot->ports[port]->dsa_8021q_cpu = cpu_port; - - if (!cpu_port->is_dsa_8021q_cpu) { - cpu_port->is_dsa_8021q_cpu = true; + cpu_port->is_dsa_8021q_cpu = true; - for (vid = OCELOT_RSV_VLAN_RANGE_START; vid < VLAN_N_VID; vid++) - ocelot_vlan_member_add(ocelot, cpu, vid, true); + for (vid = OCELOT_RSV_VLAN_RANGE_START; vid < VLAN_N_VID; vid++) + ocelot_vlan_member_add(ocelot, cpu, vid, true); - ocelot_update_pgid_cpu(ocelot); - } - - ocelot_apply_bridge_fwd_mask(ocelot, true); + ocelot_update_pgid_cpu(ocelot); mutex_unlock(&ocelot->fwd_domain_lock); } -EXPORT_SYMBOL_GPL(ocelot_port_assign_dsa_8021q_cpu); +EXPORT_SYMBOL_GPL(ocelot_port_setup_dsa_8021q_cpu); -void ocelot_port_unassign_dsa_8021q_cpu(struct ocelot *ocelot, int port) +void ocelot_port_teardown_dsa_8021q_cpu(struct ocelot *ocelot, int cpu) { - struct ocelot_port *cpu_port = ocelot->ports[port]->dsa_8021q_cpu; - bool keep = false; + struct ocelot_port *cpu_port = ocelot->ports[cpu]; u16 vid; - int p; mutex_lock(&ocelot->fwd_domain_lock); - ocelot->ports[port]->dsa_8021q_cpu = NULL; + cpu_port->is_dsa_8021q_cpu = false; - for (p = 0; p < ocelot->num_phys_ports; p++) { - if (!ocelot->ports[p]) - continue; + for (vid = OCELOT_RSV_VLAN_RANGE_START; vid < VLAN_N_VID; vid++) + ocelot_vlan_member_del(ocelot, cpu_port->index, vid); - if (ocelot->ports[p]->dsa_8021q_cpu == cpu_port) { - keep = true; - break; - } - } + ocelot_update_pgid_cpu(ocelot); - if (!keep) { - cpu_port->is_dsa_8021q_cpu = false; + mutex_unlock(&ocelot->fwd_domain_lock); +} +EXPORT_SYMBOL_GPL(ocelot_port_teardown_dsa_8021q_cpu); - for (vid = OCELOT_RSV_VLAN_RANGE_START; vid < VLAN_N_VID; vid++) - ocelot_vlan_member_del(ocelot, cpu_port->index, vid); +void ocelot_port_assign_dsa_8021q_cpu(struct ocelot *ocelot, int port, + int cpu) +{ + struct ocelot_port *cpu_port = ocelot->ports[cpu]; - ocelot_update_pgid_cpu(ocelot); - } + mutex_lock(&ocelot->fwd_domain_lock); + + ocelot->ports[port]->dsa_8021q_cpu = cpu_port; + ocelot_apply_bridge_fwd_mask(ocelot, true); + + mutex_unlock(&ocelot->fwd_domain_lock); +} +EXPORT_SYMBOL_GPL(ocelot_port_assign_dsa_8021q_cpu); + +void ocelot_port_unassign_dsa_8021q_cpu(struct ocelot *ocelot, int port) +{ + mutex_lock(&ocelot->fwd_domain_lock); + ocelot->ports[port]->dsa_8021q_cpu = NULL; ocelot_apply_bridge_fwd_mask(ocelot, true); mutex_unlock(&ocelot->fwd_domain_lock); @@ -2792,10 +2140,14 @@ static void ocelot_migrate_lag_fdbs(struct ocelot *ocelot, int ocelot_port_lag_join(struct ocelot *ocelot, int port, struct net_device *bond, - struct netdev_lag_upper_info *info) + struct netdev_lag_upper_info *info, + struct netlink_ext_ack *extack) { - if (info->tx_type != NETDEV_LAG_TX_TYPE_HASH) + if (info->tx_type != NETDEV_LAG_TX_TYPE_HASH) { + NL_SET_ERR_MSG_MOD(extack, + "Can only offload LAG using hash TX type"); return -EOPNOTSUPP; + } mutex_lock(&ocelot->fwd_domain_lock); @@ -3365,7 +2717,6 @@ static void ocelot_detect_features(struct ocelot *ocelot) int ocelot_init(struct ocelot *ocelot) { - char queue_name[32]; int i, ret; u32 port; @@ -3377,29 +2728,21 @@ int ocelot_init(struct ocelot *ocelot) } } - ocelot->stats = devm_kcalloc(ocelot->dev, - ocelot->num_phys_ports * OCELOT_NUM_STATS, - sizeof(u64), GFP_KERNEL); - if (!ocelot->stats) - return -ENOMEM; - - spin_lock_init(&ocelot->stats_lock); mutex_init(&ocelot->ptp_lock); mutex_init(&ocelot->mact_lock); mutex_init(&ocelot->fwd_domain_lock); mutex_init(&ocelot->tas_lock); spin_lock_init(&ocelot->ptp_clock_lock); spin_lock_init(&ocelot->ts_id_lock); - snprintf(queue_name, sizeof(queue_name), "%s-stats", - dev_name(ocelot->dev)); - ocelot->stats_queue = create_singlethread_workqueue(queue_name); - if (!ocelot->stats_queue) - return -ENOMEM; ocelot->owq = alloc_ordered_workqueue("ocelot-owq", 0); - if (!ocelot->owq) { - destroy_workqueue(ocelot->stats_queue); + if (!ocelot->owq) return -ENOMEM; + + ret = ocelot_stats_init(ocelot); + if (ret) { + destroy_workqueue(ocelot->owq); + return ret; } INIT_LIST_HEAD(&ocelot->multicast); @@ -3511,25 +2854,13 @@ int ocelot_init(struct ocelot *ocelot) ANA_CPUQ_8021_CFG_CPUQ_BPDU_VAL(6), ANA_CPUQ_8021_CFG, i); - ret = ocelot_prepare_stats_regions(ocelot); - if (ret) { - destroy_workqueue(ocelot->stats_queue); - destroy_workqueue(ocelot->owq); - return ret; - } - - INIT_DELAYED_WORK(&ocelot->stats_work, ocelot_check_stats_work); - queue_delayed_work(ocelot->stats_queue, &ocelot->stats_work, - OCELOT_STATS_CHECK_DELAY); - return 0; } EXPORT_SYMBOL(ocelot_init); void ocelot_deinit(struct ocelot *ocelot) { - cancel_delayed_work(&ocelot->stats_work); - destroy_workqueue(ocelot->stats_queue); + ocelot_stats_deinit(ocelot); destroy_workqueue(ocelot->owq); } EXPORT_SYMBOL(ocelot_deinit); diff --git a/drivers/net/ethernet/mscc/ocelot.h b/drivers/net/ethernet/mscc/ocelot.h index 6d65cc87d757..70dbd9c4e512 100644 --- a/drivers/net/ethernet/mscc/ocelot.h +++ b/drivers/net/ethernet/mscc/ocelot.h @@ -51,13 +51,6 @@ struct ocelot_port_private { struct ocelot_port_tc tc; }; -struct ocelot_dump_ctx { - struct net_device *dev; - struct sk_buff *skb; - struct netlink_callback *cb; - int idx; -}; - /* A (PGID) port mask structure, encoding the 2^ocelot->num_phys_ports * possibilities of egress port masks for L2 multicast traffic. * For a switch with 9 user ports, there are 512 possible port masks, but the @@ -84,8 +77,6 @@ struct ocelot_multicast { int ocelot_bridge_num_find(struct ocelot *ocelot, const struct net_device *bridge); -int ocelot_port_fdb_do_dump(const unsigned char *addr, u16 vid, - bool is_static, void *data); int ocelot_mact_learn(struct ocelot *ocelot, int port, const unsigned char mac[ETH_ALEN], unsigned int vid, enum macaccess_entry_type type); @@ -115,6 +106,9 @@ struct ocelot_mirror *ocelot_mirror_get(struct ocelot *ocelot, int to, struct netlink_ext_ack *extack); void ocelot_mirror_put(struct ocelot *ocelot); +int ocelot_stats_init(struct ocelot *ocelot); +void ocelot_stats_deinit(struct ocelot *ocelot); + extern struct notifier_block ocelot_netdevice_nb; extern struct notifier_block ocelot_switchdev_nb; extern struct notifier_block ocelot_switchdev_blocking_nb; diff --git a/drivers/net/ethernet/mscc/ocelot_net.c b/drivers/net/ethernet/mscc/ocelot_net.c index 330d30841cdc..50858cc10fef 100644 --- a/drivers/net/ethernet/mscc/ocelot_net.c +++ b/drivers/net/ethernet/mscc/ocelot_net.c @@ -20,6 +20,13 @@ #define OCELOT_MAC_QUIRKS OCELOT_QUIRK_QSGMII_PORTS_MUST_BE_UP +struct ocelot_dump_ctx { + struct net_device *dev; + struct sk_buff *skb; + struct netlink_callback *cb; + int idx; +}; + static bool ocelot_netdevice_dev_check(const struct net_device *dev); static struct ocelot *devlink_port_to_ocelot(struct devlink_port *dlp) @@ -725,42 +732,8 @@ static void ocelot_get_stats64(struct net_device *dev, struct ocelot_port_private *priv = netdev_priv(dev); struct ocelot *ocelot = priv->port.ocelot; int port = priv->port.index; - u64 *s; - - spin_lock(&ocelot->stats_lock); - - s = &ocelot->stats[port * OCELOT_NUM_STATS]; - - /* Get Rx stats */ - stats->rx_bytes = s[OCELOT_STAT_RX_OCTETS]; - stats->rx_packets = s[OCELOT_STAT_RX_SHORTS] + - s[OCELOT_STAT_RX_FRAGMENTS] + - s[OCELOT_STAT_RX_JABBERS] + - s[OCELOT_STAT_RX_LONGS] + - s[OCELOT_STAT_RX_64] + - s[OCELOT_STAT_RX_65_127] + - s[OCELOT_STAT_RX_128_255] + - s[OCELOT_STAT_RX_256_511] + - s[OCELOT_STAT_RX_512_1023] + - s[OCELOT_STAT_RX_1024_1526] + - s[OCELOT_STAT_RX_1527_MAX]; - stats->multicast = s[OCELOT_STAT_RX_MULTICAST]; - stats->rx_dropped = dev->stats.rx_dropped; - - /* Get Tx stats */ - stats->tx_bytes = s[OCELOT_STAT_TX_OCTETS]; - stats->tx_packets = s[OCELOT_STAT_TX_64] + - s[OCELOT_STAT_TX_65_127] + - s[OCELOT_STAT_TX_128_255] + - s[OCELOT_STAT_TX_256_511] + - s[OCELOT_STAT_TX_512_1023] + - s[OCELOT_STAT_TX_1024_1526] + - s[OCELOT_STAT_TX_1527_MAX]; - stats->tx_dropped = s[OCELOT_STAT_TX_DROPS] + - s[OCELOT_STAT_TX_AGED]; - stats->collisions = s[OCELOT_STAT_TX_COLLISION]; - - spin_unlock(&ocelot->stats_lock); + + return ocelot_port_get_stats64(ocelot, port, stats); } static int ocelot_port_fdb_add(struct ndmsg *ndm, struct nlattr *tb[], @@ -790,6 +763,49 @@ static int ocelot_port_fdb_del(struct ndmsg *ndm, struct nlattr *tb[], return ocelot_fdb_del(ocelot, port, addr, vid, ocelot_port->bridge); } +static int ocelot_port_fdb_do_dump(const unsigned char *addr, u16 vid, + bool is_static, void *data) +{ + struct ocelot_dump_ctx *dump = data; + u32 portid = NETLINK_CB(dump->cb->skb).portid; + u32 seq = dump->cb->nlh->nlmsg_seq; + struct nlmsghdr *nlh; + struct ndmsg *ndm; + + if (dump->idx < dump->cb->args[2]) + goto skip; + + nlh = nlmsg_put(dump->skb, portid, seq, RTM_NEWNEIGH, + sizeof(*ndm), NLM_F_MULTI); + if (!nlh) + return -EMSGSIZE; + + ndm = nlmsg_data(nlh); + ndm->ndm_family = AF_BRIDGE; + ndm->ndm_pad1 = 0; + ndm->ndm_pad2 = 0; + ndm->ndm_flags = NTF_SELF; + ndm->ndm_type = 0; + ndm->ndm_ifindex = dump->dev->ifindex; + ndm->ndm_state = is_static ? NUD_NOARP : NUD_REACHABLE; + + if (nla_put(dump->skb, NDA_LLADDR, ETH_ALEN, addr)) + goto nla_put_failure; + + if (vid && nla_put_u16(dump->skb, NDA_VLAN, vid)) + goto nla_put_failure; + + nlmsg_end(dump->skb, nlh); + +skip: + dump->idx++; + return 0; + +nla_put_failure: + nlmsg_cancel(dump->skb, nlh); + return -EMSGSIZE; +} + static int ocelot_port_fdb_dump(struct sk_buff *skb, struct netlink_callback *cb, struct net_device *dev, @@ -1396,11 +1412,10 @@ static int ocelot_netdevice_lag_join(struct net_device *dev, int port = priv->port.index; int err; - err = ocelot_port_lag_join(ocelot, port, bond, info); - if (err == -EOPNOTSUPP) { - NL_SET_ERR_MSG_MOD(extack, "Offloading not supported"); + err = ocelot_port_lag_join(ocelot, port, bond, info, extack); + if (err == -EOPNOTSUPP) + /* Offloading not supported, fall back to software LAG */ return 0; - } bridge_dev = netdev_master_upper_dev_get(bond); if (!bridge_dev || !netif_is_bridge_master(bridge_dev)) diff --git a/drivers/net/ethernet/mscc/ocelot_ptp.c b/drivers/net/ethernet/mscc/ocelot_ptp.c index 09c703efe946..1a82f10c8853 100644 --- a/drivers/net/ethernet/mscc/ocelot_ptp.c +++ b/drivers/net/ethernet/mscc/ocelot_ptp.c @@ -6,9 +6,13 @@ */ #include <linux/time64.h> +#include <linux/dsa/ocelot.h> +#include <linux/ptp_classify.h> #include <soc/mscc/ocelot_ptp.h> #include <soc/mscc/ocelot_sys.h> +#include <soc/mscc/ocelot_vcap.h> #include <soc/mscc/ocelot.h> +#include "ocelot.h" int ocelot_ptp_gettime64(struct ptp_clock_info *ptp, struct timespec64 *ts) { @@ -310,6 +314,483 @@ int ocelot_ptp_enable(struct ptp_clock_info *ptp, } EXPORT_SYMBOL(ocelot_ptp_enable); +static void ocelot_populate_l2_ptp_trap_key(struct ocelot_vcap_filter *trap) +{ + trap->key_type = OCELOT_VCAP_KEY_ETYPE; + *(__be16 *)trap->key.etype.etype.value = htons(ETH_P_1588); + *(__be16 *)trap->key.etype.etype.mask = htons(0xffff); +} + +static void +ocelot_populate_ipv4_ptp_event_trap_key(struct ocelot_vcap_filter *trap) +{ + trap->key_type = OCELOT_VCAP_KEY_IPV4; + trap->key.ipv4.proto.value[0] = IPPROTO_UDP; + trap->key.ipv4.proto.mask[0] = 0xff; + trap->key.ipv4.dport.value = PTP_EV_PORT; + trap->key.ipv4.dport.mask = 0xffff; +} + +static void +ocelot_populate_ipv6_ptp_event_trap_key(struct ocelot_vcap_filter *trap) +{ + trap->key_type = OCELOT_VCAP_KEY_IPV6; + trap->key.ipv4.proto.value[0] = IPPROTO_UDP; + trap->key.ipv4.proto.mask[0] = 0xff; + trap->key.ipv6.dport.value = PTP_EV_PORT; + trap->key.ipv6.dport.mask = 0xffff; +} + +static void +ocelot_populate_ipv4_ptp_general_trap_key(struct ocelot_vcap_filter *trap) +{ + trap->key_type = OCELOT_VCAP_KEY_IPV4; + trap->key.ipv4.proto.value[0] = IPPROTO_UDP; + trap->key.ipv4.proto.mask[0] = 0xff; + trap->key.ipv4.dport.value = PTP_GEN_PORT; + trap->key.ipv4.dport.mask = 0xffff; +} + +static void +ocelot_populate_ipv6_ptp_general_trap_key(struct ocelot_vcap_filter *trap) +{ + trap->key_type = OCELOT_VCAP_KEY_IPV6; + trap->key.ipv4.proto.value[0] = IPPROTO_UDP; + trap->key.ipv4.proto.mask[0] = 0xff; + trap->key.ipv6.dport.value = PTP_GEN_PORT; + trap->key.ipv6.dport.mask = 0xffff; +} + +static int ocelot_l2_ptp_trap_add(struct ocelot *ocelot, int port) +{ + unsigned long l2_cookie = OCELOT_VCAP_IS2_L2_PTP_TRAP(ocelot); + + return ocelot_trap_add(ocelot, port, l2_cookie, true, + ocelot_populate_l2_ptp_trap_key); +} + +static int ocelot_l2_ptp_trap_del(struct ocelot *ocelot, int port) +{ + unsigned long l2_cookie = OCELOT_VCAP_IS2_L2_PTP_TRAP(ocelot); + + return ocelot_trap_del(ocelot, port, l2_cookie); +} + +static int ocelot_ipv4_ptp_trap_add(struct ocelot *ocelot, int port) +{ + unsigned long ipv4_gen_cookie = OCELOT_VCAP_IS2_IPV4_GEN_PTP_TRAP(ocelot); + unsigned long ipv4_ev_cookie = OCELOT_VCAP_IS2_IPV4_EV_PTP_TRAP(ocelot); + int err; + + err = ocelot_trap_add(ocelot, port, ipv4_ev_cookie, true, + ocelot_populate_ipv4_ptp_event_trap_key); + if (err) + return err; + + err = ocelot_trap_add(ocelot, port, ipv4_gen_cookie, false, + ocelot_populate_ipv4_ptp_general_trap_key); + if (err) + ocelot_trap_del(ocelot, port, ipv4_ev_cookie); + + return err; +} + +static int ocelot_ipv4_ptp_trap_del(struct ocelot *ocelot, int port) +{ + unsigned long ipv4_gen_cookie = OCELOT_VCAP_IS2_IPV4_GEN_PTP_TRAP(ocelot); + unsigned long ipv4_ev_cookie = OCELOT_VCAP_IS2_IPV4_EV_PTP_TRAP(ocelot); + int err; + + err = ocelot_trap_del(ocelot, port, ipv4_ev_cookie); + err |= ocelot_trap_del(ocelot, port, ipv4_gen_cookie); + return err; +} + +static int ocelot_ipv6_ptp_trap_add(struct ocelot *ocelot, int port) +{ + unsigned long ipv6_gen_cookie = OCELOT_VCAP_IS2_IPV6_GEN_PTP_TRAP(ocelot); + unsigned long ipv6_ev_cookie = OCELOT_VCAP_IS2_IPV6_EV_PTP_TRAP(ocelot); + int err; + + err = ocelot_trap_add(ocelot, port, ipv6_ev_cookie, true, + ocelot_populate_ipv6_ptp_event_trap_key); + if (err) + return err; + + err = ocelot_trap_add(ocelot, port, ipv6_gen_cookie, false, + ocelot_populate_ipv6_ptp_general_trap_key); + if (err) + ocelot_trap_del(ocelot, port, ipv6_ev_cookie); + + return err; +} + +static int ocelot_ipv6_ptp_trap_del(struct ocelot *ocelot, int port) +{ + unsigned long ipv6_gen_cookie = OCELOT_VCAP_IS2_IPV6_GEN_PTP_TRAP(ocelot); + unsigned long ipv6_ev_cookie = OCELOT_VCAP_IS2_IPV6_EV_PTP_TRAP(ocelot); + int err; + + err = ocelot_trap_del(ocelot, port, ipv6_ev_cookie); + err |= ocelot_trap_del(ocelot, port, ipv6_gen_cookie); + return err; +} + +static int ocelot_setup_ptp_traps(struct ocelot *ocelot, int port, + bool l2, bool l4) +{ + int err; + + if (l2) + err = ocelot_l2_ptp_trap_add(ocelot, port); + else + err = ocelot_l2_ptp_trap_del(ocelot, port); + if (err) + return err; + + if (l4) { + err = ocelot_ipv4_ptp_trap_add(ocelot, port); + if (err) + goto err_ipv4; + + err = ocelot_ipv6_ptp_trap_add(ocelot, port); + if (err) + goto err_ipv6; + } else { + err = ocelot_ipv4_ptp_trap_del(ocelot, port); + + err |= ocelot_ipv6_ptp_trap_del(ocelot, port); + } + if (err) + return err; + + return 0; + +err_ipv6: + ocelot_ipv4_ptp_trap_del(ocelot, port); +err_ipv4: + if (l2) + ocelot_l2_ptp_trap_del(ocelot, port); + return err; +} + +int ocelot_hwstamp_get(struct ocelot *ocelot, int port, struct ifreq *ifr) +{ + return copy_to_user(ifr->ifr_data, &ocelot->hwtstamp_config, + sizeof(ocelot->hwtstamp_config)) ? -EFAULT : 0; +} +EXPORT_SYMBOL(ocelot_hwstamp_get); + +int ocelot_hwstamp_set(struct ocelot *ocelot, int port, struct ifreq *ifr) +{ + struct ocelot_port *ocelot_port = ocelot->ports[port]; + bool l2 = false, l4 = false; + struct hwtstamp_config cfg; + int err; + + if (copy_from_user(&cfg, ifr->ifr_data, sizeof(cfg))) + return -EFAULT; + + /* Tx type sanity check */ + switch (cfg.tx_type) { + case HWTSTAMP_TX_ON: + ocelot_port->ptp_cmd = IFH_REW_OP_TWO_STEP_PTP; + break; + case HWTSTAMP_TX_ONESTEP_SYNC: + /* IFH_REW_OP_ONE_STEP_PTP updates the correctional field, we + * need to update the origin time. + */ + ocelot_port->ptp_cmd = IFH_REW_OP_ORIGIN_PTP; + break; + case HWTSTAMP_TX_OFF: + ocelot_port->ptp_cmd = 0; + break; + default: + return -ERANGE; + } + + mutex_lock(&ocelot->ptp_lock); + + switch (cfg.rx_filter) { + case HWTSTAMP_FILTER_NONE: + break; + case HWTSTAMP_FILTER_PTP_V2_L4_EVENT: + case HWTSTAMP_FILTER_PTP_V2_L4_SYNC: + case HWTSTAMP_FILTER_PTP_V2_L4_DELAY_REQ: + l4 = true; + break; + case HWTSTAMP_FILTER_PTP_V2_L2_EVENT: + case HWTSTAMP_FILTER_PTP_V2_L2_SYNC: + case HWTSTAMP_FILTER_PTP_V2_L2_DELAY_REQ: + l2 = true; + break; + case HWTSTAMP_FILTER_PTP_V2_EVENT: + case HWTSTAMP_FILTER_PTP_V2_SYNC: + case HWTSTAMP_FILTER_PTP_V2_DELAY_REQ: + l2 = true; + l4 = true; + break; + default: + mutex_unlock(&ocelot->ptp_lock); + return -ERANGE; + } + + err = ocelot_setup_ptp_traps(ocelot, port, l2, l4); + if (err) { + mutex_unlock(&ocelot->ptp_lock); + return err; + } + + if (l2 && l4) + cfg.rx_filter = HWTSTAMP_FILTER_PTP_V2_EVENT; + else if (l2) + cfg.rx_filter = HWTSTAMP_FILTER_PTP_V2_L2_EVENT; + else if (l4) + cfg.rx_filter = HWTSTAMP_FILTER_PTP_V2_L4_EVENT; + else + cfg.rx_filter = HWTSTAMP_FILTER_NONE; + + /* Commit back the result & save it */ + memcpy(&ocelot->hwtstamp_config, &cfg, sizeof(cfg)); + mutex_unlock(&ocelot->ptp_lock); + + return copy_to_user(ifr->ifr_data, &cfg, sizeof(cfg)) ? -EFAULT : 0; +} +EXPORT_SYMBOL(ocelot_hwstamp_set); + +int ocelot_get_ts_info(struct ocelot *ocelot, int port, + struct ethtool_ts_info *info) +{ + info->phc_index = ocelot->ptp_clock ? + ptp_clock_index(ocelot->ptp_clock) : -1; + if (info->phc_index == -1) { + info->so_timestamping |= SOF_TIMESTAMPING_TX_SOFTWARE | + SOF_TIMESTAMPING_RX_SOFTWARE | + SOF_TIMESTAMPING_SOFTWARE; + return 0; + } + info->so_timestamping |= SOF_TIMESTAMPING_TX_SOFTWARE | + SOF_TIMESTAMPING_RX_SOFTWARE | + SOF_TIMESTAMPING_SOFTWARE | + SOF_TIMESTAMPING_TX_HARDWARE | + SOF_TIMESTAMPING_RX_HARDWARE | + SOF_TIMESTAMPING_RAW_HARDWARE; + info->tx_types = BIT(HWTSTAMP_TX_OFF) | BIT(HWTSTAMP_TX_ON) | + BIT(HWTSTAMP_TX_ONESTEP_SYNC); + info->rx_filters = BIT(HWTSTAMP_FILTER_NONE) | + BIT(HWTSTAMP_FILTER_PTP_V2_EVENT) | + BIT(HWTSTAMP_FILTER_PTP_V2_L2_EVENT) | + BIT(HWTSTAMP_FILTER_PTP_V2_L4_EVENT); + + return 0; +} +EXPORT_SYMBOL(ocelot_get_ts_info); + +static int ocelot_port_add_txtstamp_skb(struct ocelot *ocelot, int port, + struct sk_buff *clone) +{ + struct ocelot_port *ocelot_port = ocelot->ports[port]; + unsigned long flags; + + spin_lock_irqsave(&ocelot->ts_id_lock, flags); + + if (ocelot_port->ptp_skbs_in_flight == OCELOT_MAX_PTP_ID || + ocelot->ptp_skbs_in_flight == OCELOT_PTP_FIFO_SIZE) { + spin_unlock_irqrestore(&ocelot->ts_id_lock, flags); + return -EBUSY; + } + + skb_shinfo(clone)->tx_flags |= SKBTX_IN_PROGRESS; + /* Store timestamp ID in OCELOT_SKB_CB(clone)->ts_id */ + OCELOT_SKB_CB(clone)->ts_id = ocelot_port->ts_id; + + ocelot_port->ts_id++; + if (ocelot_port->ts_id == OCELOT_MAX_PTP_ID) + ocelot_port->ts_id = 0; + + ocelot_port->ptp_skbs_in_flight++; + ocelot->ptp_skbs_in_flight++; + + skb_queue_tail(&ocelot_port->tx_skbs, clone); + + spin_unlock_irqrestore(&ocelot->ts_id_lock, flags); + + return 0; +} + +static bool ocelot_ptp_is_onestep_sync(struct sk_buff *skb, + unsigned int ptp_class) +{ + struct ptp_header *hdr; + u8 msgtype, twostep; + + hdr = ptp_parse_header(skb, ptp_class); + if (!hdr) + return false; + + msgtype = ptp_get_msgtype(hdr, ptp_class); + twostep = hdr->flag_field[0] & 0x2; + + if (msgtype == PTP_MSGTYPE_SYNC && twostep == 0) + return true; + + return false; +} + +int ocelot_port_txtstamp_request(struct ocelot *ocelot, int port, + struct sk_buff *skb, + struct sk_buff **clone) +{ + struct ocelot_port *ocelot_port = ocelot->ports[port]; + u8 ptp_cmd = ocelot_port->ptp_cmd; + unsigned int ptp_class; + int err; + + /* Don't do anything if PTP timestamping not enabled */ + if (!ptp_cmd) + return 0; + + ptp_class = ptp_classify_raw(skb); + if (ptp_class == PTP_CLASS_NONE) + return -EINVAL; + + /* Store ptp_cmd in OCELOT_SKB_CB(skb)->ptp_cmd */ + if (ptp_cmd == IFH_REW_OP_ORIGIN_PTP) { + if (ocelot_ptp_is_onestep_sync(skb, ptp_class)) { + OCELOT_SKB_CB(skb)->ptp_cmd = ptp_cmd; + return 0; + } + + /* Fall back to two-step timestamping */ + ptp_cmd = IFH_REW_OP_TWO_STEP_PTP; + } + + if (ptp_cmd == IFH_REW_OP_TWO_STEP_PTP) { + *clone = skb_clone_sk(skb); + if (!(*clone)) + return -ENOMEM; + + err = ocelot_port_add_txtstamp_skb(ocelot, port, *clone); + if (err) + return err; + + OCELOT_SKB_CB(skb)->ptp_cmd = ptp_cmd; + OCELOT_SKB_CB(*clone)->ptp_class = ptp_class; + } + + return 0; +} +EXPORT_SYMBOL(ocelot_port_txtstamp_request); + +static void ocelot_get_hwtimestamp(struct ocelot *ocelot, + struct timespec64 *ts) +{ + unsigned long flags; + u32 val; + + spin_lock_irqsave(&ocelot->ptp_clock_lock, flags); + + /* Read current PTP time to get seconds */ + val = ocelot_read_rix(ocelot, PTP_PIN_CFG, TOD_ACC_PIN); + + val &= ~(PTP_PIN_CFG_SYNC | PTP_PIN_CFG_ACTION_MASK | PTP_PIN_CFG_DOM); + val |= PTP_PIN_CFG_ACTION(PTP_PIN_ACTION_SAVE); + ocelot_write_rix(ocelot, val, PTP_PIN_CFG, TOD_ACC_PIN); + ts->tv_sec = ocelot_read_rix(ocelot, PTP_PIN_TOD_SEC_LSB, TOD_ACC_PIN); + + /* Read packet HW timestamp from FIFO */ + val = ocelot_read(ocelot, SYS_PTP_TXSTAMP); + ts->tv_nsec = SYS_PTP_TXSTAMP_PTP_TXSTAMP(val); + + /* Sec has incremented since the ts was registered */ + if ((ts->tv_sec & 0x1) != !!(val & SYS_PTP_TXSTAMP_PTP_TXSTAMP_SEC)) + ts->tv_sec--; + + spin_unlock_irqrestore(&ocelot->ptp_clock_lock, flags); +} + +static bool ocelot_validate_ptp_skb(struct sk_buff *clone, u16 seqid) +{ + struct ptp_header *hdr; + + hdr = ptp_parse_header(clone, OCELOT_SKB_CB(clone)->ptp_class); + if (WARN_ON(!hdr)) + return false; + + return seqid == ntohs(hdr->sequence_id); +} + +void ocelot_get_txtstamp(struct ocelot *ocelot) +{ + int budget = OCELOT_PTP_QUEUE_SZ; + + while (budget--) { + struct sk_buff *skb, *skb_tmp, *skb_match = NULL; + struct skb_shared_hwtstamps shhwtstamps; + u32 val, id, seqid, txport; + struct ocelot_port *port; + struct timespec64 ts; + unsigned long flags; + + val = ocelot_read(ocelot, SYS_PTP_STATUS); + + /* Check if a timestamp can be retrieved */ + if (!(val & SYS_PTP_STATUS_PTP_MESS_VLD)) + break; + + WARN_ON(val & SYS_PTP_STATUS_PTP_OVFL); + + /* Retrieve the ts ID and Tx port */ + id = SYS_PTP_STATUS_PTP_MESS_ID_X(val); + txport = SYS_PTP_STATUS_PTP_MESS_TXPORT_X(val); + seqid = SYS_PTP_STATUS_PTP_MESS_SEQ_ID(val); + + port = ocelot->ports[txport]; + + spin_lock(&ocelot->ts_id_lock); + port->ptp_skbs_in_flight--; + ocelot->ptp_skbs_in_flight--; + spin_unlock(&ocelot->ts_id_lock); + + /* Retrieve its associated skb */ +try_again: + spin_lock_irqsave(&port->tx_skbs.lock, flags); + + skb_queue_walk_safe(&port->tx_skbs, skb, skb_tmp) { + if (OCELOT_SKB_CB(skb)->ts_id != id) + continue; + __skb_unlink(skb, &port->tx_skbs); + skb_match = skb; + break; + } + + spin_unlock_irqrestore(&port->tx_skbs.lock, flags); + + if (WARN_ON(!skb_match)) + continue; + + if (!ocelot_validate_ptp_skb(skb_match, seqid)) { + dev_err_ratelimited(ocelot->dev, + "port %d received stale TX timestamp for seqid %d, discarding\n", + txport, seqid); + dev_kfree_skb_any(skb); + goto try_again; + } + + /* Get the h/w timestamp */ + ocelot_get_hwtimestamp(ocelot, &ts); + + /* Set the timestamp into the skb */ + memset(&shhwtstamps, 0, sizeof(shhwtstamps)); + shhwtstamps.hwtstamp = ktime_set(ts.tv_sec, ts.tv_nsec); + skb_complete_tx_timestamp(skb_match, &shhwtstamps); + + /* Next ts */ + ocelot_write(ocelot, SYS_PTP_NXT_PTP_NXT, SYS_PTP_NXT); + } +} +EXPORT_SYMBOL(ocelot_get_txtstamp); + int ocelot_init_timestamp(struct ocelot *ocelot, const struct ptp_clock_info *info) { diff --git a/drivers/net/ethernet/mscc/ocelot_stats.c b/drivers/net/ethernet/mscc/ocelot_stats.c new file mode 100644 index 000000000000..dbd20b125cea --- /dev/null +++ b/drivers/net/ethernet/mscc/ocelot_stats.c @@ -0,0 +1,458 @@ +// SPDX-License-Identifier: (GPL-2.0 OR MIT) +/* Statistics for Ocelot switch family + * + * Copyright (c) 2017 Microsemi Corporation + * Copyright 2022 NXP + */ +#include <linux/spinlock.h> +#include <linux/mutex.h> +#include <linux/workqueue.h> +#include "ocelot.h" + +/* Read the counters from hardware and keep them in region->buf. + * Caller must hold &ocelot->stat_view_lock. + */ +static int ocelot_port_update_stats(struct ocelot *ocelot, int port) +{ + struct ocelot_stats_region *region; + int err; + + /* Configure the port to read the stats from */ + ocelot_write(ocelot, SYS_STAT_CFG_STAT_VIEW(port), SYS_STAT_CFG); + + list_for_each_entry(region, &ocelot->stats_regions, node) { + err = ocelot_bulk_read(ocelot, region->base, region->buf, + region->count); + if (err) + return err; + } + + return 0; +} + +/* Transfer the counters from region->buf to ocelot->stats. + * Caller must hold &ocelot->stat_view_lock and &ocelot->stats_lock. + */ +static void ocelot_port_transfer_stats(struct ocelot *ocelot, int port) +{ + unsigned int idx = port * OCELOT_NUM_STATS; + struct ocelot_stats_region *region; + int j; + + list_for_each_entry(region, &ocelot->stats_regions, node) { + for (j = 0; j < region->count; j++) { + u64 *stat = &ocelot->stats[idx + j]; + u64 val = region->buf[j]; + + if (val < (*stat & U32_MAX)) + *stat += (u64)1 << 32; + + *stat = (*stat & ~(u64)U32_MAX) + val; + } + + idx += region->count; + } +} + +static void ocelot_check_stats_work(struct work_struct *work) +{ + struct delayed_work *del_work = to_delayed_work(work); + struct ocelot *ocelot = container_of(del_work, struct ocelot, + stats_work); + int port, err; + + mutex_lock(&ocelot->stat_view_lock); + + for (port = 0; port < ocelot->num_phys_ports; port++) { + err = ocelot_port_update_stats(ocelot, port); + if (err) + break; + + spin_lock(&ocelot->stats_lock); + ocelot_port_transfer_stats(ocelot, port); + spin_unlock(&ocelot->stats_lock); + } + + if (!err && ocelot->ops->update_stats) + ocelot->ops->update_stats(ocelot); + + mutex_unlock(&ocelot->stat_view_lock); + + if (err) + dev_err(ocelot->dev, "Error %d updating ethtool stats\n", err); + + queue_delayed_work(ocelot->stats_queue, &ocelot->stats_work, + OCELOT_STATS_CHECK_DELAY); +} + +void ocelot_get_strings(struct ocelot *ocelot, int port, u32 sset, u8 *data) +{ + int i; + + if (sset != ETH_SS_STATS) + return; + + for (i = 0; i < OCELOT_NUM_STATS; i++) { + if (ocelot->stats_layout[i].name[0] == '\0') + continue; + + memcpy(data + i * ETH_GSTRING_LEN, ocelot->stats_layout[i].name, + ETH_GSTRING_LEN); + } +} +EXPORT_SYMBOL(ocelot_get_strings); + +/* Update ocelot->stats for the given port and run the given callback */ +static void ocelot_port_stats_run(struct ocelot *ocelot, int port, void *priv, + void (*cb)(struct ocelot *ocelot, int port, + void *priv)) +{ + int err; + + mutex_lock(&ocelot->stat_view_lock); + + err = ocelot_port_update_stats(ocelot, port); + if (err) { + dev_err(ocelot->dev, "Failed to update port %d stats: %pe\n", + port, ERR_PTR(err)); + goto out_unlock; + } + + spin_lock(&ocelot->stats_lock); + + ocelot_port_transfer_stats(ocelot, port); + cb(ocelot, port, priv); + + spin_unlock(&ocelot->stats_lock); + +out_unlock: + mutex_unlock(&ocelot->stat_view_lock); +} + +int ocelot_get_sset_count(struct ocelot *ocelot, int port, int sset) +{ + int i, num_stats = 0; + + if (sset != ETH_SS_STATS) + return -EOPNOTSUPP; + + for (i = 0; i < OCELOT_NUM_STATS; i++) + if (ocelot->stats_layout[i].name[0] != '\0') + num_stats++; + + return num_stats; +} +EXPORT_SYMBOL(ocelot_get_sset_count); + +static void ocelot_port_ethtool_stats_cb(struct ocelot *ocelot, int port, + void *priv) +{ + u64 *data = priv; + int i; + + /* Copy all supported counters */ + for (i = 0; i < OCELOT_NUM_STATS; i++) { + int index = port * OCELOT_NUM_STATS + i; + + if (ocelot->stats_layout[i].name[0] == '\0') + continue; + + *data++ = ocelot->stats[index]; + } +} + +void ocelot_get_ethtool_stats(struct ocelot *ocelot, int port, u64 *data) +{ + ocelot_port_stats_run(ocelot, port, data, ocelot_port_ethtool_stats_cb); +} +EXPORT_SYMBOL(ocelot_get_ethtool_stats); + +static void ocelot_port_pause_stats_cb(struct ocelot *ocelot, int port, void *priv) +{ + u64 *s = &ocelot->stats[port * OCELOT_NUM_STATS]; + struct ethtool_pause_stats *pause_stats = priv; + + pause_stats->tx_pause_frames = s[OCELOT_STAT_TX_PAUSE]; + pause_stats->rx_pause_frames = s[OCELOT_STAT_RX_PAUSE]; +} + +void ocelot_port_get_pause_stats(struct ocelot *ocelot, int port, + struct ethtool_pause_stats *pause_stats) +{ + ocelot_port_stats_run(ocelot, port, pause_stats, + ocelot_port_pause_stats_cb); +} +EXPORT_SYMBOL_GPL(ocelot_port_get_pause_stats); + +static const struct ethtool_rmon_hist_range ocelot_rmon_ranges[] = { + { 64, 64 }, + { 65, 127 }, + { 128, 255 }, + { 256, 511 }, + { 512, 1023 }, + { 1024, 1526 }, + { 1527, 65535 }, + {}, +}; + +static void ocelot_port_rmon_stats_cb(struct ocelot *ocelot, int port, void *priv) +{ + u64 *s = &ocelot->stats[port * OCELOT_NUM_STATS]; + struct ethtool_rmon_stats *rmon_stats = priv; + + rmon_stats->undersize_pkts = s[OCELOT_STAT_RX_SHORTS]; + rmon_stats->oversize_pkts = s[OCELOT_STAT_RX_LONGS]; + rmon_stats->fragments = s[OCELOT_STAT_RX_FRAGMENTS]; + rmon_stats->jabbers = s[OCELOT_STAT_RX_JABBERS]; + + rmon_stats->hist[0] = s[OCELOT_STAT_RX_64]; + rmon_stats->hist[1] = s[OCELOT_STAT_RX_65_127]; + rmon_stats->hist[2] = s[OCELOT_STAT_RX_128_255]; + rmon_stats->hist[3] = s[OCELOT_STAT_RX_256_511]; + rmon_stats->hist[4] = s[OCELOT_STAT_RX_512_1023]; + rmon_stats->hist[5] = s[OCELOT_STAT_RX_1024_1526]; + rmon_stats->hist[6] = s[OCELOT_STAT_RX_1527_MAX]; + + rmon_stats->hist_tx[0] = s[OCELOT_STAT_TX_64]; + rmon_stats->hist_tx[1] = s[OCELOT_STAT_TX_65_127]; + rmon_stats->hist_tx[2] = s[OCELOT_STAT_TX_128_255]; + rmon_stats->hist_tx[3] = s[OCELOT_STAT_TX_128_255]; + rmon_stats->hist_tx[4] = s[OCELOT_STAT_TX_256_511]; + rmon_stats->hist_tx[5] = s[OCELOT_STAT_TX_512_1023]; + rmon_stats->hist_tx[6] = s[OCELOT_STAT_TX_1024_1526]; +} + +void ocelot_port_get_rmon_stats(struct ocelot *ocelot, int port, + struct ethtool_rmon_stats *rmon_stats, + const struct ethtool_rmon_hist_range **ranges) +{ + *ranges = ocelot_rmon_ranges; + + ocelot_port_stats_run(ocelot, port, rmon_stats, + ocelot_port_rmon_stats_cb); +} +EXPORT_SYMBOL_GPL(ocelot_port_get_rmon_stats); + +static void ocelot_port_ctrl_stats_cb(struct ocelot *ocelot, int port, void *priv) +{ + u64 *s = &ocelot->stats[port * OCELOT_NUM_STATS]; + struct ethtool_eth_ctrl_stats *ctrl_stats = priv; + + ctrl_stats->MACControlFramesReceived = s[OCELOT_STAT_RX_CONTROL]; +} + +void ocelot_port_get_eth_ctrl_stats(struct ocelot *ocelot, int port, + struct ethtool_eth_ctrl_stats *ctrl_stats) +{ + ocelot_port_stats_run(ocelot, port, ctrl_stats, + ocelot_port_ctrl_stats_cb); +} +EXPORT_SYMBOL_GPL(ocelot_port_get_eth_ctrl_stats); + +static void ocelot_port_mac_stats_cb(struct ocelot *ocelot, int port, void *priv) +{ + u64 *s = &ocelot->stats[port * OCELOT_NUM_STATS]; + struct ethtool_eth_mac_stats *mac_stats = priv; + + mac_stats->OctetsTransmittedOK = s[OCELOT_STAT_TX_OCTETS]; + mac_stats->FramesTransmittedOK = s[OCELOT_STAT_TX_64] + + s[OCELOT_STAT_TX_65_127] + + s[OCELOT_STAT_TX_128_255] + + s[OCELOT_STAT_TX_256_511] + + s[OCELOT_STAT_TX_512_1023] + + s[OCELOT_STAT_TX_1024_1526] + + s[OCELOT_STAT_TX_1527_MAX]; + mac_stats->OctetsReceivedOK = s[OCELOT_STAT_RX_OCTETS]; + mac_stats->FramesReceivedOK = s[OCELOT_STAT_RX_GREEN_PRIO_0] + + s[OCELOT_STAT_RX_GREEN_PRIO_1] + + s[OCELOT_STAT_RX_GREEN_PRIO_2] + + s[OCELOT_STAT_RX_GREEN_PRIO_3] + + s[OCELOT_STAT_RX_GREEN_PRIO_4] + + s[OCELOT_STAT_RX_GREEN_PRIO_5] + + s[OCELOT_STAT_RX_GREEN_PRIO_6] + + s[OCELOT_STAT_RX_GREEN_PRIO_7] + + s[OCELOT_STAT_RX_YELLOW_PRIO_0] + + s[OCELOT_STAT_RX_YELLOW_PRIO_1] + + s[OCELOT_STAT_RX_YELLOW_PRIO_2] + + s[OCELOT_STAT_RX_YELLOW_PRIO_3] + + s[OCELOT_STAT_RX_YELLOW_PRIO_4] + + s[OCELOT_STAT_RX_YELLOW_PRIO_5] + + s[OCELOT_STAT_RX_YELLOW_PRIO_6] + + s[OCELOT_STAT_RX_YELLOW_PRIO_7]; + mac_stats->MulticastFramesXmittedOK = s[OCELOT_STAT_TX_MULTICAST]; + mac_stats->BroadcastFramesXmittedOK = s[OCELOT_STAT_TX_BROADCAST]; + mac_stats->MulticastFramesReceivedOK = s[OCELOT_STAT_RX_MULTICAST]; + mac_stats->BroadcastFramesReceivedOK = s[OCELOT_STAT_RX_BROADCAST]; + mac_stats->FrameTooLongErrors = s[OCELOT_STAT_RX_LONGS]; + /* Sadly, C_RX_CRC is the sum of FCS and alignment errors, they are not + * counted individually. + */ + mac_stats->FrameCheckSequenceErrors = s[OCELOT_STAT_RX_CRC_ALIGN_ERRS]; + mac_stats->AlignmentErrors = s[OCELOT_STAT_RX_CRC_ALIGN_ERRS]; +} + +void ocelot_port_get_eth_mac_stats(struct ocelot *ocelot, int port, + struct ethtool_eth_mac_stats *mac_stats) +{ + ocelot_port_stats_run(ocelot, port, mac_stats, + ocelot_port_mac_stats_cb); +} +EXPORT_SYMBOL_GPL(ocelot_port_get_eth_mac_stats); + +static void ocelot_port_phy_stats_cb(struct ocelot *ocelot, int port, void *priv) +{ + u64 *s = &ocelot->stats[port * OCELOT_NUM_STATS]; + struct ethtool_eth_phy_stats *phy_stats = priv; + + phy_stats->SymbolErrorDuringCarrier = s[OCELOT_STAT_RX_SYM_ERRS]; +} + +void ocelot_port_get_eth_phy_stats(struct ocelot *ocelot, int port, + struct ethtool_eth_phy_stats *phy_stats) +{ + ocelot_port_stats_run(ocelot, port, phy_stats, + ocelot_port_phy_stats_cb); +} +EXPORT_SYMBOL_GPL(ocelot_port_get_eth_phy_stats); + +void ocelot_port_get_stats64(struct ocelot *ocelot, int port, + struct rtnl_link_stats64 *stats) +{ + u64 *s = &ocelot->stats[port * OCELOT_NUM_STATS]; + + spin_lock(&ocelot->stats_lock); + + /* Get Rx stats */ + stats->rx_bytes = s[OCELOT_STAT_RX_OCTETS]; + stats->rx_packets = s[OCELOT_STAT_RX_SHORTS] + + s[OCELOT_STAT_RX_FRAGMENTS] + + s[OCELOT_STAT_RX_JABBERS] + + s[OCELOT_STAT_RX_LONGS] + + s[OCELOT_STAT_RX_64] + + s[OCELOT_STAT_RX_65_127] + + s[OCELOT_STAT_RX_128_255] + + s[OCELOT_STAT_RX_256_511] + + s[OCELOT_STAT_RX_512_1023] + + s[OCELOT_STAT_RX_1024_1526] + + s[OCELOT_STAT_RX_1527_MAX]; + stats->multicast = s[OCELOT_STAT_RX_MULTICAST]; + stats->rx_missed_errors = s[OCELOT_STAT_DROP_TAIL]; + stats->rx_dropped = s[OCELOT_STAT_RX_RED_PRIO_0] + + s[OCELOT_STAT_RX_RED_PRIO_1] + + s[OCELOT_STAT_RX_RED_PRIO_2] + + s[OCELOT_STAT_RX_RED_PRIO_3] + + s[OCELOT_STAT_RX_RED_PRIO_4] + + s[OCELOT_STAT_RX_RED_PRIO_5] + + s[OCELOT_STAT_RX_RED_PRIO_6] + + s[OCELOT_STAT_RX_RED_PRIO_7] + + s[OCELOT_STAT_DROP_LOCAL] + + s[OCELOT_STAT_DROP_YELLOW_PRIO_0] + + s[OCELOT_STAT_DROP_YELLOW_PRIO_1] + + s[OCELOT_STAT_DROP_YELLOW_PRIO_2] + + s[OCELOT_STAT_DROP_YELLOW_PRIO_3] + + s[OCELOT_STAT_DROP_YELLOW_PRIO_4] + + s[OCELOT_STAT_DROP_YELLOW_PRIO_5] + + s[OCELOT_STAT_DROP_YELLOW_PRIO_6] + + s[OCELOT_STAT_DROP_YELLOW_PRIO_7] + + s[OCELOT_STAT_DROP_GREEN_PRIO_0] + + s[OCELOT_STAT_DROP_GREEN_PRIO_1] + + s[OCELOT_STAT_DROP_GREEN_PRIO_2] + + s[OCELOT_STAT_DROP_GREEN_PRIO_3] + + s[OCELOT_STAT_DROP_GREEN_PRIO_4] + + s[OCELOT_STAT_DROP_GREEN_PRIO_5] + + s[OCELOT_STAT_DROP_GREEN_PRIO_6] + + s[OCELOT_STAT_DROP_GREEN_PRIO_7]; + + /* Get Tx stats */ + stats->tx_bytes = s[OCELOT_STAT_TX_OCTETS]; + stats->tx_packets = s[OCELOT_STAT_TX_64] + + s[OCELOT_STAT_TX_65_127] + + s[OCELOT_STAT_TX_128_255] + + s[OCELOT_STAT_TX_256_511] + + s[OCELOT_STAT_TX_512_1023] + + s[OCELOT_STAT_TX_1024_1526] + + s[OCELOT_STAT_TX_1527_MAX]; + stats->tx_dropped = s[OCELOT_STAT_TX_DROPS] + + s[OCELOT_STAT_TX_AGED]; + stats->collisions = s[OCELOT_STAT_TX_COLLISION]; + + spin_unlock(&ocelot->stats_lock); +} +EXPORT_SYMBOL(ocelot_port_get_stats64); + +static int ocelot_prepare_stats_regions(struct ocelot *ocelot) +{ + struct ocelot_stats_region *region = NULL; + unsigned int last; + int i; + + INIT_LIST_HEAD(&ocelot->stats_regions); + + for (i = 0; i < OCELOT_NUM_STATS; i++) { + if (!ocelot->stats_layout[i].reg) + continue; + + if (region && ocelot->stats_layout[i].reg == last + 4) { + region->count++; + } else { + region = devm_kzalloc(ocelot->dev, sizeof(*region), + GFP_KERNEL); + if (!region) + return -ENOMEM; + + region->base = ocelot->stats_layout[i].reg; + region->count = 1; + list_add_tail(®ion->node, &ocelot->stats_regions); + } + + last = ocelot->stats_layout[i].reg; + } + + list_for_each_entry(region, &ocelot->stats_regions, node) { + region->buf = devm_kcalloc(ocelot->dev, region->count, + sizeof(*region->buf), GFP_KERNEL); + if (!region->buf) + return -ENOMEM; + } + + return 0; +} + +int ocelot_stats_init(struct ocelot *ocelot) +{ + char queue_name[32]; + int ret; + + ocelot->stats = devm_kcalloc(ocelot->dev, + ocelot->num_phys_ports * OCELOT_NUM_STATS, + sizeof(u64), GFP_KERNEL); + if (!ocelot->stats) + return -ENOMEM; + + snprintf(queue_name, sizeof(queue_name), "%s-stats", + dev_name(ocelot->dev)); + ocelot->stats_queue = create_singlethread_workqueue(queue_name); + if (!ocelot->stats_queue) + return -ENOMEM; + + spin_lock_init(&ocelot->stats_lock); + mutex_init(&ocelot->stat_view_lock); + + ret = ocelot_prepare_stats_regions(ocelot); + if (ret) { + destroy_workqueue(ocelot->stats_queue); + return ret; + } + + INIT_DELAYED_WORK(&ocelot->stats_work, ocelot_check_stats_work); + queue_delayed_work(ocelot->stats_queue, &ocelot->stats_work, + OCELOT_STATS_CHECK_DELAY); + + return 0; +} + +void ocelot_stats_deinit(struct ocelot *ocelot) +{ + cancel_delayed_work(&ocelot->stats_work); + destroy_workqueue(ocelot->stats_queue); +} diff --git a/drivers/net/ethernet/mscc/ocelot_vsc7514.c b/drivers/net/ethernet/mscc/ocelot_vsc7514.c index 9c488953f541..6f22aea08a64 100644 --- a/drivers/net/ethernet/mscc/ocelot_vsc7514.c +++ b/drivers/net/ethernet/mscc/ocelot_vsc7514.c @@ -6,6 +6,7 @@ */ #include <linux/dsa/ocelot.h> #include <linux/interrupt.h> +#include <linux/iopoll.h> #include <linux/module.h> #include <linux/of_net.h> #include <linux/netdevice.h> @@ -25,6 +26,9 @@ #define VSC7514_VCAP_POLICER_BASE 128 #define VSC7514_VCAP_POLICER_MAX 191 +#define MEM_INIT_SLEEP_US 1000 +#define MEM_INIT_TIMEOUT_US 100000 + static const u32 *ocelot_regmap[TARGET_MAX] = { [ANA] = vsc7514_ana_regmap, [QS] = vsc7514_qs_regmap, @@ -97,378 +101,7 @@ static const struct reg_field ocelot_regfields[REGFIELD_MAX] = { }; static const struct ocelot_stat_layout ocelot_stats_layout[OCELOT_NUM_STATS] = { - [OCELOT_STAT_RX_OCTETS] = { - .name = "rx_octets", - .reg = SYS_COUNT_RX_OCTETS, - }, - [OCELOT_STAT_RX_UNICAST] = { - .name = "rx_unicast", - .reg = SYS_COUNT_RX_UNICAST, - }, - [OCELOT_STAT_RX_MULTICAST] = { - .name = "rx_multicast", - .reg = SYS_COUNT_RX_MULTICAST, - }, - [OCELOT_STAT_RX_BROADCAST] = { - .name = "rx_broadcast", - .reg = SYS_COUNT_RX_BROADCAST, - }, - [OCELOT_STAT_RX_SHORTS] = { - .name = "rx_shorts", - .reg = SYS_COUNT_RX_SHORTS, - }, - [OCELOT_STAT_RX_FRAGMENTS] = { - .name = "rx_fragments", - .reg = SYS_COUNT_RX_FRAGMENTS, - }, - [OCELOT_STAT_RX_JABBERS] = { - .name = "rx_jabbers", - .reg = SYS_COUNT_RX_JABBERS, - }, - [OCELOT_STAT_RX_CRC_ALIGN_ERRS] = { - .name = "rx_crc_align_errs", - .reg = SYS_COUNT_RX_CRC_ALIGN_ERRS, - }, - [OCELOT_STAT_RX_SYM_ERRS] = { - .name = "rx_sym_errs", - .reg = SYS_COUNT_RX_SYM_ERRS, - }, - [OCELOT_STAT_RX_64] = { - .name = "rx_frames_below_65_octets", - .reg = SYS_COUNT_RX_64, - }, - [OCELOT_STAT_RX_65_127] = { - .name = "rx_frames_65_to_127_octets", - .reg = SYS_COUNT_RX_65_127, - }, - [OCELOT_STAT_RX_128_255] = { - .name = "rx_frames_128_to_255_octets", - .reg = SYS_COUNT_RX_128_255, - }, - [OCELOT_STAT_RX_256_511] = { - .name = "rx_frames_256_to_511_octets", - .reg = SYS_COUNT_RX_256_511, - }, - [OCELOT_STAT_RX_512_1023] = { - .name = "rx_frames_512_to_1023_octets", - .reg = SYS_COUNT_RX_512_1023, - }, - [OCELOT_STAT_RX_1024_1526] = { - .name = "rx_frames_1024_to_1526_octets", - .reg = SYS_COUNT_RX_1024_1526, - }, - [OCELOT_STAT_RX_1527_MAX] = { - .name = "rx_frames_over_1526_octets", - .reg = SYS_COUNT_RX_1527_MAX, - }, - [OCELOT_STAT_RX_PAUSE] = { - .name = "rx_pause", - .reg = SYS_COUNT_RX_PAUSE, - }, - [OCELOT_STAT_RX_CONTROL] = { - .name = "rx_control", - .reg = SYS_COUNT_RX_CONTROL, - }, - [OCELOT_STAT_RX_LONGS] = { - .name = "rx_longs", - .reg = SYS_COUNT_RX_LONGS, - }, - [OCELOT_STAT_RX_CLASSIFIED_DROPS] = { - .name = "rx_classified_drops", - .reg = SYS_COUNT_RX_CLASSIFIED_DROPS, - }, - [OCELOT_STAT_RX_RED_PRIO_0] = { - .name = "rx_red_prio_0", - .reg = SYS_COUNT_RX_RED_PRIO_0, - }, - [OCELOT_STAT_RX_RED_PRIO_1] = { - .name = "rx_red_prio_1", - .reg = SYS_COUNT_RX_RED_PRIO_1, - }, - [OCELOT_STAT_RX_RED_PRIO_2] = { - .name = "rx_red_prio_2", - .reg = SYS_COUNT_RX_RED_PRIO_2, - }, - [OCELOT_STAT_RX_RED_PRIO_3] = { - .name = "rx_red_prio_3", - .reg = SYS_COUNT_RX_RED_PRIO_3, - }, - [OCELOT_STAT_RX_RED_PRIO_4] = { - .name = "rx_red_prio_4", - .reg = SYS_COUNT_RX_RED_PRIO_4, - }, - [OCELOT_STAT_RX_RED_PRIO_5] = { - .name = "rx_red_prio_5", - .reg = SYS_COUNT_RX_RED_PRIO_5, - }, - [OCELOT_STAT_RX_RED_PRIO_6] = { - .name = "rx_red_prio_6", - .reg = SYS_COUNT_RX_RED_PRIO_6, - }, - [OCELOT_STAT_RX_RED_PRIO_7] = { - .name = "rx_red_prio_7", - .reg = SYS_COUNT_RX_RED_PRIO_7, - }, - [OCELOT_STAT_RX_YELLOW_PRIO_0] = { - .name = "rx_yellow_prio_0", - .reg = SYS_COUNT_RX_YELLOW_PRIO_0, - }, - [OCELOT_STAT_RX_YELLOW_PRIO_1] = { - .name = "rx_yellow_prio_1", - .reg = SYS_COUNT_RX_YELLOW_PRIO_1, - }, - [OCELOT_STAT_RX_YELLOW_PRIO_2] = { - .name = "rx_yellow_prio_2", - .reg = SYS_COUNT_RX_YELLOW_PRIO_2, - }, - [OCELOT_STAT_RX_YELLOW_PRIO_3] = { - .name = "rx_yellow_prio_3", - .reg = SYS_COUNT_RX_YELLOW_PRIO_3, - }, - [OCELOT_STAT_RX_YELLOW_PRIO_4] = { - .name = "rx_yellow_prio_4", - .reg = SYS_COUNT_RX_YELLOW_PRIO_4, - }, - [OCELOT_STAT_RX_YELLOW_PRIO_5] = { - .name = "rx_yellow_prio_5", - .reg = SYS_COUNT_RX_YELLOW_PRIO_5, - }, - [OCELOT_STAT_RX_YELLOW_PRIO_6] = { - .name = "rx_yellow_prio_6", - .reg = SYS_COUNT_RX_YELLOW_PRIO_6, - }, - [OCELOT_STAT_RX_YELLOW_PRIO_7] = { - .name = "rx_yellow_prio_7", - .reg = SYS_COUNT_RX_YELLOW_PRIO_7, - }, - [OCELOT_STAT_RX_GREEN_PRIO_0] = { - .name = "rx_green_prio_0", - .reg = SYS_COUNT_RX_GREEN_PRIO_0, - }, - [OCELOT_STAT_RX_GREEN_PRIO_1] = { - .name = "rx_green_prio_1", - .reg = SYS_COUNT_RX_GREEN_PRIO_1, - }, - [OCELOT_STAT_RX_GREEN_PRIO_2] = { - .name = "rx_green_prio_2", - .reg = SYS_COUNT_RX_GREEN_PRIO_2, - }, - [OCELOT_STAT_RX_GREEN_PRIO_3] = { - .name = "rx_green_prio_3", - .reg = SYS_COUNT_RX_GREEN_PRIO_3, - }, - [OCELOT_STAT_RX_GREEN_PRIO_4] = { - .name = "rx_green_prio_4", - .reg = SYS_COUNT_RX_GREEN_PRIO_4, - }, - [OCELOT_STAT_RX_GREEN_PRIO_5] = { - .name = "rx_green_prio_5", - .reg = SYS_COUNT_RX_GREEN_PRIO_5, - }, - [OCELOT_STAT_RX_GREEN_PRIO_6] = { - .name = "rx_green_prio_6", - .reg = SYS_COUNT_RX_GREEN_PRIO_6, - }, - [OCELOT_STAT_RX_GREEN_PRIO_7] = { - .name = "rx_green_prio_7", - .reg = SYS_COUNT_RX_GREEN_PRIO_7, - }, - [OCELOT_STAT_TX_OCTETS] = { - .name = "tx_octets", - .reg = SYS_COUNT_TX_OCTETS, - }, - [OCELOT_STAT_TX_UNICAST] = { - .name = "tx_unicast", - .reg = SYS_COUNT_TX_UNICAST, - }, - [OCELOT_STAT_TX_MULTICAST] = { - .name = "tx_multicast", - .reg = SYS_COUNT_TX_MULTICAST, - }, - [OCELOT_STAT_TX_BROADCAST] = { - .name = "tx_broadcast", - .reg = SYS_COUNT_TX_BROADCAST, - }, - [OCELOT_STAT_TX_COLLISION] = { - .name = "tx_collision", - .reg = SYS_COUNT_TX_COLLISION, - }, - [OCELOT_STAT_TX_DROPS] = { - .name = "tx_drops", - .reg = SYS_COUNT_TX_DROPS, - }, - [OCELOT_STAT_TX_PAUSE] = { - .name = "tx_pause", - .reg = SYS_COUNT_TX_PAUSE, - }, - [OCELOT_STAT_TX_64] = { - .name = "tx_frames_below_65_octets", - .reg = SYS_COUNT_TX_64, - }, - [OCELOT_STAT_TX_65_127] = { - .name = "tx_frames_65_to_127_octets", - .reg = SYS_COUNT_TX_65_127, - }, - [OCELOT_STAT_TX_128_255] = { - .name = "tx_frames_128_255_octets", - .reg = SYS_COUNT_TX_128_255, - }, - [OCELOT_STAT_TX_256_511] = { - .name = "tx_frames_256_511_octets", - .reg = SYS_COUNT_TX_256_511, - }, - [OCELOT_STAT_TX_512_1023] = { - .name = "tx_frames_512_1023_octets", - .reg = SYS_COUNT_TX_512_1023, - }, - [OCELOT_STAT_TX_1024_1526] = { - .name = "tx_frames_1024_1526_octets", - .reg = SYS_COUNT_TX_1024_1526, - }, - [OCELOT_STAT_TX_1527_MAX] = { - .name = "tx_frames_over_1526_octets", - .reg = SYS_COUNT_TX_1527_MAX, - }, - [OCELOT_STAT_TX_YELLOW_PRIO_0] = { - .name = "tx_yellow_prio_0", - .reg = SYS_COUNT_TX_YELLOW_PRIO_0, - }, - [OCELOT_STAT_TX_YELLOW_PRIO_1] = { - .name = "tx_yellow_prio_1", - .reg = SYS_COUNT_TX_YELLOW_PRIO_1, - }, - [OCELOT_STAT_TX_YELLOW_PRIO_2] = { - .name = "tx_yellow_prio_2", - .reg = SYS_COUNT_TX_YELLOW_PRIO_2, - }, - [OCELOT_STAT_TX_YELLOW_PRIO_3] = { - .name = "tx_yellow_prio_3", - .reg = SYS_COUNT_TX_YELLOW_PRIO_3, - }, - [OCELOT_STAT_TX_YELLOW_PRIO_4] = { - .name = "tx_yellow_prio_4", - .reg = SYS_COUNT_TX_YELLOW_PRIO_4, - }, - [OCELOT_STAT_TX_YELLOW_PRIO_5] = { - .name = "tx_yellow_prio_5", - .reg = SYS_COUNT_TX_YELLOW_PRIO_5, - }, - [OCELOT_STAT_TX_YELLOW_PRIO_6] = { - .name = "tx_yellow_prio_6", - .reg = SYS_COUNT_TX_YELLOW_PRIO_6, - }, - [OCELOT_STAT_TX_YELLOW_PRIO_7] = { - .name = "tx_yellow_prio_7", - .reg = SYS_COUNT_TX_YELLOW_PRIO_7, - }, - [OCELOT_STAT_TX_GREEN_PRIO_0] = { - .name = "tx_green_prio_0", - .reg = SYS_COUNT_TX_GREEN_PRIO_0, - }, - [OCELOT_STAT_TX_GREEN_PRIO_1] = { - .name = "tx_green_prio_1", - .reg = SYS_COUNT_TX_GREEN_PRIO_1, - }, - [OCELOT_STAT_TX_GREEN_PRIO_2] = { - .name = "tx_green_prio_2", - .reg = SYS_COUNT_TX_GREEN_PRIO_2, - }, - [OCELOT_STAT_TX_GREEN_PRIO_3] = { - .name = "tx_green_prio_3", - .reg = SYS_COUNT_TX_GREEN_PRIO_3, - }, - [OCELOT_STAT_TX_GREEN_PRIO_4] = { - .name = "tx_green_prio_4", - .reg = SYS_COUNT_TX_GREEN_PRIO_4, - }, - [OCELOT_STAT_TX_GREEN_PRIO_5] = { - .name = "tx_green_prio_5", - .reg = SYS_COUNT_TX_GREEN_PRIO_5, - }, - [OCELOT_STAT_TX_GREEN_PRIO_6] = { - .name = "tx_green_prio_6", - .reg = SYS_COUNT_TX_GREEN_PRIO_6, - }, - [OCELOT_STAT_TX_GREEN_PRIO_7] = { - .name = "tx_green_prio_7", - .reg = SYS_COUNT_TX_GREEN_PRIO_7, - }, - [OCELOT_STAT_TX_AGED] = { - .name = "tx_aged", - .reg = SYS_COUNT_TX_AGING, - }, - [OCELOT_STAT_DROP_LOCAL] = { - .name = "drop_local", - .reg = SYS_COUNT_DROP_LOCAL, - }, - [OCELOT_STAT_DROP_TAIL] = { - .name = "drop_tail", - .reg = SYS_COUNT_DROP_TAIL, - }, - [OCELOT_STAT_DROP_YELLOW_PRIO_0] = { - .name = "drop_yellow_prio_0", - .reg = SYS_COUNT_DROP_YELLOW_PRIO_0, - }, - [OCELOT_STAT_DROP_YELLOW_PRIO_1] = { - .name = "drop_yellow_prio_1", - .reg = SYS_COUNT_DROP_YELLOW_PRIO_1, - }, - [OCELOT_STAT_DROP_YELLOW_PRIO_2] = { - .name = "drop_yellow_prio_2", - .reg = SYS_COUNT_DROP_YELLOW_PRIO_2, - }, - [OCELOT_STAT_DROP_YELLOW_PRIO_3] = { - .name = "drop_yellow_prio_3", - .reg = SYS_COUNT_DROP_YELLOW_PRIO_3, - }, - [OCELOT_STAT_DROP_YELLOW_PRIO_4] = { - .name = "drop_yellow_prio_4", - .reg = SYS_COUNT_DROP_YELLOW_PRIO_4, - }, - [OCELOT_STAT_DROP_YELLOW_PRIO_5] = { - .name = "drop_yellow_prio_5", - .reg = SYS_COUNT_DROP_YELLOW_PRIO_5, - }, - [OCELOT_STAT_DROP_YELLOW_PRIO_6] = { - .name = "drop_yellow_prio_6", - .reg = SYS_COUNT_DROP_YELLOW_PRIO_6, - }, - [OCELOT_STAT_DROP_YELLOW_PRIO_7] = { - .name = "drop_yellow_prio_7", - .reg = SYS_COUNT_DROP_YELLOW_PRIO_7, - }, - [OCELOT_STAT_DROP_GREEN_PRIO_0] = { - .name = "drop_green_prio_0", - .reg = SYS_COUNT_DROP_GREEN_PRIO_0, - }, - [OCELOT_STAT_DROP_GREEN_PRIO_1] = { - .name = "drop_green_prio_1", - .reg = SYS_COUNT_DROP_GREEN_PRIO_1, - }, - [OCELOT_STAT_DROP_GREEN_PRIO_2] = { - .name = "drop_green_prio_2", - .reg = SYS_COUNT_DROP_GREEN_PRIO_2, - }, - [OCELOT_STAT_DROP_GREEN_PRIO_3] = { - .name = "drop_green_prio_3", - .reg = SYS_COUNT_DROP_GREEN_PRIO_3, - }, - [OCELOT_STAT_DROP_GREEN_PRIO_4] = { - .name = "drop_green_prio_4", - .reg = SYS_COUNT_DROP_GREEN_PRIO_4, - }, - [OCELOT_STAT_DROP_GREEN_PRIO_5] = { - .name = "drop_green_prio_5", - .reg = SYS_COUNT_DROP_GREEN_PRIO_5, - }, - [OCELOT_STAT_DROP_GREEN_PRIO_6] = { - .name = "drop_green_prio_6", - .reg = SYS_COUNT_DROP_GREEN_PRIO_6, - }, - [OCELOT_STAT_DROP_GREEN_PRIO_7] = { - .name = "drop_green_prio_7", - .reg = SYS_COUNT_DROP_GREEN_PRIO_7, - }, + OCELOT_COMMON_STATS, }; static void ocelot_pll5_init(struct ocelot *ocelot) @@ -562,27 +195,43 @@ static const struct of_device_id mscc_ocelot_match[] = { }; MODULE_DEVICE_TABLE(of, mscc_ocelot_match); +static int ocelot_mem_init_status(struct ocelot *ocelot) +{ + unsigned int val; + int err; + + err = regmap_field_read(ocelot->regfields[SYS_RESET_CFG_MEM_INIT], + &val); + + return err ?: val; +} + static int ocelot_reset(struct ocelot *ocelot) { - int retries = 100; + int err; u32 val; - regmap_field_write(ocelot->regfields[SYS_RESET_CFG_MEM_INIT], 1); - regmap_field_write(ocelot->regfields[SYS_RESET_CFG_MEM_ENA], 1); + err = regmap_field_write(ocelot->regfields[SYS_RESET_CFG_MEM_INIT], 1); + if (err) + return err; - do { - msleep(1); - regmap_field_read(ocelot->regfields[SYS_RESET_CFG_MEM_INIT], - &val); - } while (val && --retries); + err = regmap_field_write(ocelot->regfields[SYS_RESET_CFG_MEM_ENA], 1); + if (err) + return err; - if (!retries) - return -ETIMEDOUT; + /* MEM_INIT is a self-clearing bit. Wait for it to be cleared (should be + * 100us) before enabling the switch core. + */ + err = readx_poll_timeout(ocelot_mem_init_status, ocelot, val, !val, + MEM_INIT_SLEEP_US, MEM_INIT_TIMEOUT_US); + if (err) + return err; - regmap_field_write(ocelot->regfields[SYS_RESET_CFG_MEM_ENA], 1); - regmap_field_write(ocelot->regfields[SYS_RESET_CFG_CORE_ENA], 1); + err = regmap_field_write(ocelot->regfields[SYS_RESET_CFG_MEM_ENA], 1); + if (err) + return err; - return 0; + return regmap_field_write(ocelot->regfields[SYS_RESET_CFG_CORE_ENA], 1); } /* Watermark encode diff --git a/drivers/net/ethernet/mscc/vsc7514_regs.c b/drivers/net/ethernet/mscc/vsc7514_regs.c index 9cf82ecf191c..9d2d3e13cacf 100644 --- a/drivers/net/ethernet/mscc/vsc7514_regs.c +++ b/drivers/net/ethernet/mscc/vsc7514_regs.c @@ -242,7 +242,7 @@ const u32 vsc7514_sys_regmap[] = { REG(SYS_COUNT_TX_GREEN_PRIO_5, 0x00016c), REG(SYS_COUNT_TX_GREEN_PRIO_6, 0x000170), REG(SYS_COUNT_TX_GREEN_PRIO_7, 0x000174), - REG(SYS_COUNT_TX_AGING, 0x000178), + REG(SYS_COUNT_TX_AGED, 0x000178), REG(SYS_COUNT_DROP_LOCAL, 0x000200), REG(SYS_COUNT_DROP_TAIL, 0x000204), REG(SYS_COUNT_DROP_YELLOW_PRIO_0, 0x000208), @@ -283,7 +283,6 @@ const u32 vsc7514_sys_regmap[] = { REG(SYS_MMGT_FAST, 0x0006a0), REG(SYS_EVENTS_DIF, 0x0006a4), REG(SYS_EVENTS_CORE, 0x0006b4), - REG(SYS_CNT, 0x000000), REG(SYS_PTP_STATUS, 0x0006b8), REG(SYS_PTP_TXSTAMP, 0x0006bc), REG(SYS_PTP_NXT, 0x0006c0), diff --git a/drivers/net/ethernet/myricom/myri10ge/myri10ge.c b/drivers/net/ethernet/myricom/myri10ge/myri10ge.c index 971dde8c3286..9063e2e22cd5 100644 --- a/drivers/net/ethernet/myricom/myri10ge/myri10ge.c +++ b/drivers/net/ethernet/myricom/myri10ge/myri10ge.c @@ -1647,10 +1647,10 @@ myri10ge_get_drvinfo(struct net_device *netdev, struct ethtool_drvinfo *info) { struct myri10ge_priv *mgp = netdev_priv(netdev); - strlcpy(info->driver, "myri10ge", sizeof(info->driver)); - strlcpy(info->version, MYRI10GE_VERSION_STR, sizeof(info->version)); - strlcpy(info->fw_version, mgp->fw_version, sizeof(info->fw_version)); - strlcpy(info->bus_info, pci_name(mgp->pdev), sizeof(info->bus_info)); + strscpy(info->driver, "myri10ge", sizeof(info->driver)); + strscpy(info->version, MYRI10GE_VERSION_STR, sizeof(info->version)); + strscpy(info->fw_version, mgp->fw_version, sizeof(info->fw_version)); + strscpy(info->bus_info, pci_name(mgp->pdev), sizeof(info->bus_info)); } static int myri10ge_get_coalesce(struct net_device *netdev, diff --git a/drivers/net/ethernet/natsemi/natsemi.c b/drivers/net/ethernet/natsemi/natsemi.c index 9aae7f1eb5d2..650a5a166070 100644 --- a/drivers/net/ethernet/natsemi/natsemi.c +++ b/drivers/net/ethernet/natsemi/natsemi.c @@ -869,7 +869,7 @@ static int natsemi_probe1(struct pci_dev *pdev, const struct pci_device_id *ent) np = netdev_priv(dev); np->ioaddr = ioaddr; - netif_napi_add(dev, &np->napi, natsemi_poll, 64); + netif_napi_add(dev, &np->napi, natsemi_poll); np->dev = dev; np->pci_dev = pdev; @@ -2564,9 +2564,9 @@ static void set_rx_mode(struct net_device *dev) static void get_drvinfo(struct net_device *dev, struct ethtool_drvinfo *info) { struct netdev_private *np = netdev_priv(dev); - strlcpy(info->driver, DRV_NAME, sizeof(info->driver)); - strlcpy(info->version, DRV_VERSION, sizeof(info->version)); - strlcpy(info->bus_info, pci_name(np->pci_dev), sizeof(info->bus_info)); + strscpy(info->driver, DRV_NAME, sizeof(info->driver)); + strscpy(info->version, DRV_VERSION, sizeof(info->version)); + strscpy(info->bus_info, pci_name(np->pci_dev), sizeof(info->bus_info)); } static int get_regs_len(struct net_device *dev) diff --git a/drivers/net/ethernet/natsemi/ns83820.c b/drivers/net/ethernet/natsemi/ns83820.c index 49ea130c9067..998586872599 100644 --- a/drivers/net/ethernet/natsemi/ns83820.c +++ b/drivers/net/ethernet/natsemi/ns83820.c @@ -1351,9 +1351,9 @@ static int ns83820_set_link_ksettings(struct net_device *ndev, static void ns83820_get_drvinfo(struct net_device *ndev, struct ethtool_drvinfo *info) { struct ns83820 *dev = PRIV(ndev); - strlcpy(info->driver, "ns83820", sizeof(info->driver)); - strlcpy(info->version, VERSION, sizeof(info->version)); - strlcpy(info->bus_info, pci_name(dev->pci_dev), sizeof(info->bus_info)); + strscpy(info->driver, "ns83820", sizeof(info->driver)); + strscpy(info->version, VERSION, sizeof(info->version)); + strscpy(info->bus_info, pci_name(dev->pci_dev), sizeof(info->bus_info)); } static u32 ns83820_get_link(struct net_device *ndev) diff --git a/drivers/net/ethernet/neterion/s2io.c b/drivers/net/ethernet/neterion/s2io.c index 30f955efa830..dcf8212119f9 100644 --- a/drivers/net/ethernet/neterion/s2io.c +++ b/drivers/net/ethernet/neterion/s2io.c @@ -5348,9 +5348,9 @@ static void s2io_ethtool_gdrvinfo(struct net_device *dev, { struct s2io_nic *sp = netdev_priv(dev); - strlcpy(info->driver, s2io_driver_name, sizeof(info->driver)); - strlcpy(info->version, s2io_driver_version, sizeof(info->version)); - strlcpy(info->bus_info, pci_name(sp->pdev), sizeof(info->bus_info)); + strscpy(info->driver, s2io_driver_name, sizeof(info->driver)); + strscpy(info->version, s2io_driver_version, sizeof(info->version)); + strscpy(info->bus_info, pci_name(sp->pdev), sizeof(info->bus_info)); } /** @@ -7359,10 +7359,9 @@ static int rx_osm_handler(struct ring_info *ring_data, struct RxD_t * rxdp) int get_off = ring_data->rx_curr_get_info.offset; int buf0_len = RXD_GET_BUFFER0_SIZE_3(rxdp->Control_2); int buf2_len = RXD_GET_BUFFER2_SIZE_3(rxdp->Control_2); - unsigned char *buff = skb_push(skb, buf0_len); struct buffAdd *ba = &ring_data->ba[get_block][get_off]; - memcpy(buff, ba->ba_0, buf0_len); + skb_put_data(skb, ba->ba_0, buf0_len); skb_put(skb, buf2_len); } @@ -7905,10 +7904,10 @@ s2io_init_nic(struct pci_dev *pdev, const struct pci_device_id *pre) for (i = 0; i < config->rx_ring_num ; i++) { struct ring_info *ring = &mac_control->rings[i]; - netif_napi_add(dev, &ring->napi, s2io_poll_msix, 64); + netif_napi_add(dev, &ring->napi, s2io_poll_msix); } } else { - netif_napi_add(dev, &sp->napi, s2io_poll_inta, 64); + netif_napi_add(dev, &sp->napi, s2io_poll_inta); } /* Not needed for Herc */ diff --git a/drivers/net/ethernet/netronome/nfp/crypto/tls.c b/drivers/net/ethernet/netronome/nfp/crypto/tls.c index 78368e71ce83..f80f1a6953fa 100644 --- a/drivers/net/ethernet/netronome/nfp/crypto/tls.c +++ b/drivers/net/ethernet/netronome/nfp/crypto/tls.c @@ -474,6 +474,7 @@ int nfp_net_tls_rx_resync_req(struct net_device *netdev, { struct nfp_net *nn = netdev_priv(netdev); struct nfp_net_tls_offload_ctx *ntls; + struct net *net = dev_net(netdev); struct ipv6hdr *ipv6h; struct tcphdr *th; struct iphdr *iph; @@ -494,13 +495,13 @@ int nfp_net_tls_rx_resync_req(struct net_device *netdev, switch (ipv6h->version) { case 4: - sk = inet_lookup_established(dev_net(netdev), &tcp_hashinfo, + sk = inet_lookup_established(net, net->ipv4.tcp_death_row.hashinfo, iph->saddr, th->source, iph->daddr, th->dest, netdev->ifindex); break; #if IS_ENABLED(CONFIG_IPV6) case 6: - sk = __inet6_lookup_established(dev_net(netdev), &tcp_hashinfo, + sk = __inet6_lookup_established(net, net->ipv4.tcp_death_row.hashinfo, &ipv6h->saddr, th->source, &ipv6h->daddr, ntohs(th->dest), netdev->ifindex, 0); diff --git a/drivers/net/ethernet/netronome/nfp/flower/conntrack.c b/drivers/net/ethernet/netronome/nfp/flower/conntrack.c index b3b2a23b8d89..f693119541d5 100644 --- a/drivers/net/ethernet/netronome/nfp/flower/conntrack.c +++ b/drivers/net/ethernet/netronome/nfp/flower/conntrack.c @@ -1,6 +1,9 @@ // SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) /* Copyright (C) 2021 Corigine, Inc. */ +#include <net/tc_act/tc_csum.h> +#include <net/tc_act/tc_ct.h> + #include "conntrack.h" #include "../nfp_port.h" @@ -56,9 +59,17 @@ bool is_pre_ct_flow(struct flow_cls_offload *flow) int i; flow_action_for_each(i, act, &flow->rule->action) { - if (act->id == FLOW_ACTION_CT && !act->ct.action) - return true; + if (act->id == FLOW_ACTION_CT) { + /* The pre_ct rule only have the ct or ct nat action, cannot + * contains other ct action e.g ct commit and so on. + */ + if ((!act->ct.action || act->ct.action == TCA_CT_ACT_NAT)) + return true; + else + return false; + } } + return false; } @@ -66,13 +77,37 @@ bool is_post_ct_flow(struct flow_cls_offload *flow) { struct flow_rule *rule = flow_cls_offload_flow_rule(flow); struct flow_dissector *dissector = rule->match.dissector; + struct flow_action_entry *act; + bool exist_ct_clear = false; struct flow_match_ct ct; + int i; + + /* post ct entry cannot contains any ct action except ct_clear. */ + flow_action_for_each(i, act, &flow->rule->action) { + if (act->id == FLOW_ACTION_CT) { + /* ignore ct clear action. */ + if (act->ct.action == TCA_CT_ACT_CLEAR) { + exist_ct_clear = true; + continue; + } + + return false; + } + } if (dissector->used_keys & BIT(FLOW_DISSECTOR_KEY_CT)) { flow_rule_match_ct(rule, &ct); if (ct.key->ct_state & TCA_FLOWER_KEY_CT_FLAGS_ESTABLISHED) return true; + } else { + /* when do nat with ct, the post ct entry ignore the ct status, + * will match the nat field(sip/dip) instead. In this situation, + * the flow chain index is not zero and contains ct clear action. + */ + if (flow->common.chain_index && exist_ct_clear) + return true; } + return false; } @@ -168,6 +203,20 @@ static void *get_mangled_tos_ttl(struct flow_rule *rule, void *buf, return buf; } +/* Note entry1 and entry2 are not swappable. only skip ip and + * tport merge check for pre_ct and post_ct when pre_ct do nat. + */ +static bool nfp_ct_merge_check_cannot_skip(struct nfp_fl_ct_flow_entry *entry1, + struct nfp_fl_ct_flow_entry *entry2) +{ + /* only pre_ct have NFP_FL_ACTION_DO_NAT flag. */ + if ((entry1->flags & NFP_FL_ACTION_DO_NAT) && + entry2->type == CT_TYPE_POST_CT) + return false; + + return true; +} + /* Note entry1 and entry2 are not swappable, entry1 should be * the former flow whose mangle action need be taken into account * if existed, and entry2 should be the latter flow whose action @@ -225,7 +274,12 @@ static int nfp_ct_merge_check(struct nfp_fl_ct_flow_entry *entry1, goto check_failed; } - if (ovlp_keys & BIT(FLOW_DISSECTOR_KEY_IPV4_ADDRS)) { + /* if pre ct entry do nat, the nat ip exists in nft entry, + * will be do merge check when do nft and post ct merge, + * so skip this ip merge check here. + */ + if ((ovlp_keys & BIT(FLOW_DISSECTOR_KEY_IPV4_ADDRS)) && + nfp_ct_merge_check_cannot_skip(entry1, entry2)) { struct flow_match_ipv4_addrs match1, match2; flow_rule_match_ipv4_addrs(entry1->rule, &match1); @@ -242,7 +296,12 @@ static int nfp_ct_merge_check(struct nfp_fl_ct_flow_entry *entry1, goto check_failed; } - if (ovlp_keys & BIT(FLOW_DISSECTOR_KEY_IPV6_ADDRS)) { + /* if pre ct entry do nat, the nat ip exists in nft entry, + * will be do merge check when do nft and post ct merge, + * so skip this ip merge check here. + */ + if ((ovlp_keys & BIT(FLOW_DISSECTOR_KEY_IPV6_ADDRS)) && + nfp_ct_merge_check_cannot_skip(entry1, entry2)) { struct flow_match_ipv6_addrs match1, match2; flow_rule_match_ipv6_addrs(entry1->rule, &match1); @@ -259,7 +318,12 @@ static int nfp_ct_merge_check(struct nfp_fl_ct_flow_entry *entry1, goto check_failed; } - if (ovlp_keys & BIT(FLOW_DISSECTOR_KEY_PORTS)) { + /* if pre ct entry do nat, the nat tport exists in nft entry, + * will be do merge check when do nft and post ct merge, + * so skip this tport merge check here. + */ + if ((ovlp_keys & BIT(FLOW_DISSECTOR_KEY_PORTS)) && + nfp_ct_merge_check_cannot_skip(entry1, entry2)) { enum flow_action_mangle_base htype = FLOW_ACT_MANGLE_UNSPEC; struct flow_match_ports match1, match2; @@ -404,12 +468,55 @@ check_failed: return -EINVAL; } +static int nfp_ct_check_vlan_merge(struct flow_action_entry *a_in, + struct flow_rule *rule) +{ + struct flow_match_vlan match; + + if (unlikely(flow_rule_match_key(rule, FLOW_DISSECTOR_KEY_CVLAN))) + return -EOPNOTSUPP; + + /* post_ct does not match VLAN KEY, can be merged. */ + if (likely(!flow_rule_match_key(rule, FLOW_DISSECTOR_KEY_VLAN))) + return 0; + + switch (a_in->id) { + /* pre_ct has pop vlan, post_ct cannot match VLAN KEY, cannot be merged. */ + case FLOW_ACTION_VLAN_POP: + return -EOPNOTSUPP; + + case FLOW_ACTION_VLAN_PUSH: + case FLOW_ACTION_VLAN_MANGLE: + flow_rule_match_vlan(rule, &match); + /* different vlan id, cannot be merged. */ + if ((match.key->vlan_id & match.mask->vlan_id) ^ + (a_in->vlan.vid & match.mask->vlan_id)) + return -EOPNOTSUPP; + + /* different tpid, cannot be merged. */ + if ((match.key->vlan_tpid & match.mask->vlan_tpid) ^ + (a_in->vlan.proto & match.mask->vlan_tpid)) + return -EOPNOTSUPP; + + /* different priority, cannot be merged. */ + if ((match.key->vlan_priority & match.mask->vlan_priority) ^ + (a_in->vlan.prio & match.mask->vlan_priority)) + return -EOPNOTSUPP; + + break; + default: + return -EOPNOTSUPP; + } + + return 0; +} + static int nfp_ct_merge_act_check(struct nfp_fl_ct_flow_entry *pre_ct_entry, struct nfp_fl_ct_flow_entry *post_ct_entry, struct nfp_fl_ct_flow_entry *nft_entry) { struct flow_action_entry *act; - int i; + int i, err; /* Check for pre_ct->action conflicts */ flow_action_for_each(i, act, &pre_ct_entry->rule->action) { @@ -417,6 +524,10 @@ static int nfp_ct_merge_act_check(struct nfp_fl_ct_flow_entry *pre_ct_entry, case FLOW_ACTION_VLAN_PUSH: case FLOW_ACTION_VLAN_POP: case FLOW_ACTION_VLAN_MANGLE: + err = nfp_ct_check_vlan_merge(act, post_ct_entry->rule); + if (err) + return err; + break; case FLOW_ACTION_MPLS_PUSH: case FLOW_ACTION_MPLS_POP: case FLOW_ACTION_MPLS_MANGLE: @@ -468,6 +579,12 @@ static int nfp_ct_check_meta(struct nfp_fl_ct_flow_entry *post_ct_entry, return -EINVAL; return 0; + } else { + /* post_ct with ct clear action will not match the + * ct status when nft is nat entry. + */ + if (nft_entry->flags & NFP_FL_ACTION_DO_MANGLE) + return 0; } return -EINVAL; @@ -537,11 +654,37 @@ nfp_fl_calc_key_layers_sz(struct nfp_fl_key_ls in_key_ls, uint16_t *map) return key_size; } +/* get the csum flag according the ip proto and mangle action. */ +static void nfp_fl_get_csum_flag(struct flow_action_entry *a_in, u8 ip_proto, u32 *csum) +{ + if (a_in->id != FLOW_ACTION_MANGLE) + return; + + switch (a_in->mangle.htype) { + case FLOW_ACT_MANGLE_HDR_TYPE_IP4: + *csum |= TCA_CSUM_UPDATE_FLAG_IPV4HDR; + if (ip_proto == IPPROTO_TCP) + *csum |= TCA_CSUM_UPDATE_FLAG_TCP; + else if (ip_proto == IPPROTO_UDP) + *csum |= TCA_CSUM_UPDATE_FLAG_UDP; + break; + case FLOW_ACT_MANGLE_HDR_TYPE_TCP: + *csum |= TCA_CSUM_UPDATE_FLAG_TCP; + break; + case FLOW_ACT_MANGLE_HDR_TYPE_UDP: + *csum |= TCA_CSUM_UPDATE_FLAG_UDP; + break; + default: + break; + } +} + static int nfp_fl_merge_actions_offload(struct flow_rule **rules, struct nfp_flower_priv *priv, struct net_device *netdev, struct nfp_fl_payload *flow_pay) { + enum flow_action_hw_stats tmp_stats = FLOW_ACTION_HW_STATS_DONT_CARE; struct flow_action_entry *a_in; int i, j, num_actions, id; struct flow_rule *a_rule; @@ -551,15 +694,25 @@ static int nfp_fl_merge_actions_offload(struct flow_rule **rules, rules[CT_TYPE_NFT]->action.num_entries + rules[CT_TYPE_POST_CT]->action.num_entries; - a_rule = flow_rule_alloc(num_actions); + /* Add one action to make sure there is enough room to add an checksum action + * when do nat. + */ + a_rule = flow_rule_alloc(num_actions + 1); if (!a_rule) return -ENOMEM; /* Actions need a BASIC dissector. */ a_rule->match = rules[CT_TYPE_PRE_CT]->match; + /* post_ct entry have one action at least. */ + if (rules[CT_TYPE_POST_CT]->action.num_entries != 0) { + tmp_stats = rules[CT_TYPE_POST_CT]->action.entries[0].hw_stats; + } /* Copy actions */ for (j = 0; j < _CT_TYPE_MAX; j++) { + u32 csum_updated = 0; + u8 ip_proto = 0; + if (flow_rule_match_key(rules[j], FLOW_DISSECTOR_KEY_BASIC)) { struct flow_match_basic match; @@ -571,8 +724,10 @@ static int nfp_fl_merge_actions_offload(struct flow_rule **rules, * through the subflows and assign the proper subflow to a_rule */ flow_rule_match_basic(rules[j], &match); - if (match.mask->ip_proto) + if (match.mask->ip_proto) { a_rule->match = rules[j]->match; + ip_proto = match.key->ip_proto; + } } for (i = 0; i < rules[j]->action.num_entries; i++) { @@ -589,11 +744,32 @@ static int nfp_fl_merge_actions_offload(struct flow_rule **rules, case FLOW_ACTION_CT_METADATA: continue; default: + /* nft entry is generated by tc ct, which mangle action do not care + * the stats, inherit the post entry stats to meet the + * flow_action_hw_stats_check. + */ + if (j == CT_TYPE_NFT) { + if (a_in->hw_stats == FLOW_ACTION_HW_STATS_DONT_CARE) + a_in->hw_stats = tmp_stats; + nfp_fl_get_csum_flag(a_in, ip_proto, &csum_updated); + } memcpy(&a_rule->action.entries[offset++], a_in, sizeof(struct flow_action_entry)); break; } } + /* nft entry have mangle action, but do not have checksum action when do NAT, + * hardware will automatically fix IPv4 and TCP/UDP checksum. so add an csum action + * to meet csum action check. + */ + if (csum_updated) { + struct flow_action_entry *csum_action; + + csum_action = &a_rule->action.entries[offset++]; + csum_action->id = FLOW_ACTION_CSUM; + csum_action->csum_flags = csum_updated; + csum_action->hw_stats = tmp_stats; + } } /* Some actions would have been ignored, so update the num_entries field */ @@ -1191,6 +1367,49 @@ static struct net_device *get_netdev_from_rule(struct flow_rule *rule) return NULL; } +static void nfp_nft_ct_translate_mangle_action(struct flow_action_entry *mangle_action) +{ + if (mangle_action->id != FLOW_ACTION_MANGLE) + return; + + switch (mangle_action->mangle.htype) { + case FLOW_ACT_MANGLE_HDR_TYPE_IP4: + case FLOW_ACT_MANGLE_HDR_TYPE_IP6: + mangle_action->mangle.val = (__force u32)cpu_to_be32(mangle_action->mangle.val); + mangle_action->mangle.mask = (__force u32)cpu_to_be32(mangle_action->mangle.mask); + return; + + case FLOW_ACT_MANGLE_HDR_TYPE_TCP: + case FLOW_ACT_MANGLE_HDR_TYPE_UDP: + mangle_action->mangle.val = (__force u16)cpu_to_be16(mangle_action->mangle.val); + mangle_action->mangle.mask = (__force u16)cpu_to_be16(mangle_action->mangle.mask); + return; + + default: + return; + } +} + +static int nfp_nft_ct_set_flow_flag(struct flow_action_entry *act, + struct nfp_fl_ct_flow_entry *entry) +{ + switch (act->id) { + case FLOW_ACTION_CT: + if (act->ct.action == TCA_CT_ACT_NAT) + entry->flags |= NFP_FL_ACTION_DO_NAT; + break; + + case FLOW_ACTION_MANGLE: + entry->flags |= NFP_FL_ACTION_DO_MANGLE; + break; + + default: + break; + } + + return 0; +} + static struct nfp_fl_ct_flow_entry *nfp_fl_ct_add_flow(struct nfp_fl_ct_zone_entry *zt, struct net_device *netdev, @@ -1257,6 +1476,13 @@ nfp_fl_ct_flow_entry *nfp_fl_ct_add_flow(struct nfp_fl_ct_zone_entry *zt, new_act = &entry->rule->action.entries[i]; memcpy(new_act, act, sizeof(struct flow_action_entry)); + /* nft entry mangle field is host byte order, need translate to + * network byte order. + */ + if (is_nft) + nfp_nft_ct_translate_mangle_action(new_act); + + nfp_nft_ct_set_flow_flag(new_act, entry); /* Entunnel is a special case, need to allocate and copy * tunnel info. */ diff --git a/drivers/net/ethernet/netronome/nfp/flower/conntrack.h b/drivers/net/ethernet/netronome/nfp/flower/conntrack.h index beb6cceff9d8..762c0b36e269 100644 --- a/drivers/net/ethernet/netronome/nfp/flower/conntrack.h +++ b/drivers/net/ethernet/netronome/nfp/flower/conntrack.h @@ -103,6 +103,10 @@ enum nfp_nfp_layer_name { _FLOW_PAY_LAYERS_MAX }; +/* NFP flow entry flags. */ +#define NFP_FL_ACTION_DO_NAT BIT(0) +#define NFP_FL_ACTION_DO_MANGLE BIT(1) + /** * struct nfp_fl_ct_flow_entry - Flow entry containing conntrack flow information * @cookie: Flow cookie, same as original TC flow, used as key @@ -115,6 +119,7 @@ enum nfp_nfp_layer_name { * @rule: Reference to the original TC flow rule * @stats: Used to cache stats for updating * @tun_offset: Used to indicate tunnel action offset in action list + * @flags: Used to indicate flow flag like NAT which used by merge. */ struct nfp_fl_ct_flow_entry { unsigned long cookie; @@ -127,6 +132,7 @@ struct nfp_fl_ct_flow_entry { struct flow_rule *rule; struct flow_stats stats; u8 tun_offset; // Set to NFP_FL_CT_NO_TUN if no tun + u8 flags; }; /** diff --git a/drivers/net/ethernet/netronome/nfp/flower/offload.c b/drivers/net/ethernet/netronome/nfp/flower/offload.c index 83c97154c0c7..3ab3e4536b99 100644 --- a/drivers/net/ethernet/netronome/nfp/flower/offload.c +++ b/drivers/net/ethernet/netronome/nfp/flower/offload.c @@ -1301,9 +1301,14 @@ static bool offload_pre_check(struct flow_cls_offload *flow) { struct flow_rule *rule = flow_cls_offload_flow_rule(flow); struct flow_dissector *dissector = rule->match.dissector; + struct flow_match_ct ct; - if (dissector->used_keys & BIT(FLOW_DISSECTOR_KEY_CT)) - return false; + if (dissector->used_keys & BIT(FLOW_DISSECTOR_KEY_CT)) { + flow_rule_match_ct(rule, &ct); + /* Allow special case where CT match is all 0 */ + if (memchr_inv(ct.key, 0, sizeof(*ct.key))) + return false; + } if (flow->common.chain_index) return false; diff --git a/drivers/net/ethernet/netronome/nfp/flower/qos_conf.c b/drivers/net/ethernet/netronome/nfp/flower/qos_conf.c index 7b92026e1a6f..99052a925d9e 100644 --- a/drivers/net/ethernet/netronome/nfp/flower/qos_conf.c +++ b/drivers/net/ethernet/netronome/nfp/flower/qos_conf.c @@ -119,7 +119,8 @@ int nfp_flower_offload_one_police(struct nfp_app *app, bool ingress, static int nfp_policer_validate(const struct flow_action *action, const struct flow_action_entry *act, - struct netlink_ext_ack *extack) + struct netlink_ext_ack *extack, + bool ingress) { if (act->police.exceed.act_id != FLOW_ACTION_DROP) { NL_SET_ERR_MSG_MOD(extack, @@ -127,12 +128,20 @@ static int nfp_policer_validate(const struct flow_action *action, return -EOPNOTSUPP; } - if (act->police.notexceed.act_id != FLOW_ACTION_CONTINUE && - act->police.notexceed.act_id != FLOW_ACTION_PIPE && - act->police.notexceed.act_id != FLOW_ACTION_ACCEPT) { - NL_SET_ERR_MSG_MOD(extack, - "Offload not supported when conform action is not continue, pipe or ok"); - return -EOPNOTSUPP; + if (ingress) { + if (act->police.notexceed.act_id != FLOW_ACTION_CONTINUE && + act->police.notexceed.act_id != FLOW_ACTION_ACCEPT) { + NL_SET_ERR_MSG_MOD(extack, + "Offload not supported when conform action is not continue or ok"); + return -EOPNOTSUPP; + } + } else { + if (act->police.notexceed.act_id != FLOW_ACTION_PIPE && + act->police.notexceed.act_id != FLOW_ACTION_ACCEPT) { + NL_SET_ERR_MSG_MOD(extack, + "Offload not supported when conform action is not pipe or ok"); + return -EOPNOTSUPP; + } } if (act->police.notexceed.act_id == FLOW_ACTION_ACCEPT && @@ -218,7 +227,7 @@ nfp_flower_install_rate_limiter(struct nfp_app *app, struct net_device *netdev, return -EOPNOTSUPP; } - err = nfp_policer_validate(&flow->rule->action, action, extack); + err = nfp_policer_validate(&flow->rule->action, action, extack, true); if (err) return err; @@ -687,6 +696,7 @@ nfp_act_install_actions(struct nfp_app *app, struct flow_offload_action *fl_act, bool pps_support, pps; bool add = false; u64 rate; + int err; pps_support = !!(fl_priv->flower_ext_feats & NFP_FL_FEATS_QOS_PPS); @@ -698,6 +708,11 @@ nfp_act_install_actions(struct nfp_app *app, struct flow_offload_action *fl_act, "unsupported offload: qos rate limit offload requires police action"); continue; } + + err = nfp_policer_validate(&fl_act->action, action, extack, false); + if (err) + return err; + if (action->police.rate_bytes_ps > 0) { rate = action->police.rate_bytes_ps; burst = action->police.burst; diff --git a/drivers/net/ethernet/netronome/nfp/nfd3/xsk.c b/drivers/net/ethernet/netronome/nfp/nfd3/xsk.c index 65e243168765..5d9db8c2a5b4 100644 --- a/drivers/net/ethernet/netronome/nfp/nfd3/xsk.c +++ b/drivers/net/ethernet/netronome/nfp/nfd3/xsk.c @@ -84,7 +84,7 @@ static void nfp_nfd3_xsk_rx_skb(struct nfp_net_rx_ring *rx_ring, nfp_net_xsk_rx_drop(r_vec, xrxbuf); return; } - memcpy(skb_put(skb, pkt_len), xrxbuf->xdp->data, pkt_len); + skb_put_data(skb, xrxbuf->xdp->data, pkt_len); skb->mark = meta->mark; skb_set_hash(skb, meta->hash, meta->hash_type); diff --git a/drivers/net/ethernet/netronome/nfp/nfp_main.c b/drivers/net/ethernet/netronome/nfp/nfp_main.c index 873429f7a6da..e66e548919d4 100644 --- a/drivers/net/ethernet/netronome/nfp/nfp_main.c +++ b/drivers/net/ethernet/netronome/nfp/nfp_main.c @@ -691,6 +691,71 @@ static int nfp_pf_find_rtsyms(struct nfp_pf *pf) return 0; } +int nfp_net_pf_get_app_id(struct nfp_pf *pf) +{ + return nfp_pf_rtsym_read_optional(pf, "_pf%u_net_app_id", + NFP_APP_CORE_NIC); +} + +static u64 nfp_net_pf_get_app_cap(struct nfp_pf *pf) +{ + char name[32]; + int err = 0; + u64 val; + + snprintf(name, sizeof(name), "_pf%u_net_app_cap", nfp_cppcore_pcie_unit(pf->cpp)); + + val = nfp_rtsym_read_le(pf->rtbl, name, &err); + if (err) { + if (err != -ENOENT) + nfp_err(pf->cpp, "Unable to read symbol %s\n", name); + + return 0; + } + + return val; +} + +static int nfp_pf_cfg_hwinfo(struct nfp_pf *pf, bool sp_indiff) +{ + struct nfp_nsp *nsp; + char hwinfo[32]; + int err; + + nsp = nfp_nsp_open(pf->cpp); + if (IS_ERR(nsp)) + return PTR_ERR(nsp); + + snprintf(hwinfo, sizeof(hwinfo), "sp_indiff=%d", sp_indiff); + err = nfp_nsp_hwinfo_set(nsp, hwinfo, sizeof(hwinfo)); + /* Not a fatal error, no need to return error to stop driver from loading */ + if (err) { + nfp_warn(pf->cpp, "HWinfo(sp_indiff=%d) set failed: %d\n", sp_indiff, err); + } else { + /* Need reinit eth_tbl since the eth table state may change + * after sp_indiff is configured. + */ + kfree(pf->eth_tbl); + pf->eth_tbl = __nfp_eth_read_ports(pf->cpp, nsp); + } + + nfp_nsp_close(nsp); + return 0; +} + +static int nfp_pf_nsp_cfg(struct nfp_pf *pf) +{ + bool sp_indiff = (nfp_net_pf_get_app_id(pf) == NFP_APP_FLOWER_NIC) || + (nfp_net_pf_get_app_cap(pf) & NFP_NET_APP_CAP_SP_INDIFF); + + return nfp_pf_cfg_hwinfo(pf, sp_indiff); +} + +static void nfp_pf_nsp_clean(struct nfp_pf *pf) +{ + nfp_pf_cfg_hwinfo(pf, false); +} + static int nfp_pci_probe(struct pci_dev *pdev, const struct pci_device_id *pci_id) { @@ -791,10 +856,14 @@ static int nfp_pci_probe(struct pci_dev *pdev, goto err_fw_unload; } - err = nfp_net_pci_probe(pf); + err = nfp_pf_nsp_cfg(pf); if (err) goto err_fw_unload; + err = nfp_net_pci_probe(pf); + if (err) + goto err_nsp_clean; + err = nfp_hwmon_register(pf); if (err) { dev_err(&pdev->dev, "Failed to register hwmon info\n"); @@ -805,6 +874,8 @@ static int nfp_pci_probe(struct pci_dev *pdev, err_net_remove: nfp_net_pci_remove(pf); +err_nsp_clean: + nfp_pf_nsp_clean(pf); err_fw_unload: kfree(pf->rtbl); nfp_mip_close(pf->mip); @@ -844,6 +915,7 @@ static void __nfp_pci_shutdown(struct pci_dev *pdev, bool unload_fw) nfp_net_pci_remove(pf); + nfp_pf_nsp_clean(pf); vfree(pf->dumpspec); kfree(pf->rtbl); nfp_mip_close(pf->mip); diff --git a/drivers/net/ethernet/netronome/nfp/nfp_main.h b/drivers/net/ethernet/netronome/nfp/nfp_main.h index f56ca11de134..afd3edfa2428 100644 --- a/drivers/net/ethernet/netronome/nfp/nfp_main.h +++ b/drivers/net/ethernet/netronome/nfp/nfp_main.h @@ -161,6 +161,7 @@ bool nfp_ctrl_tx(struct nfp_net *nn, struct sk_buff *skb); int nfp_pf_rtsym_read_optional(struct nfp_pf *pf, const char *format, unsigned int default_val); +int nfp_net_pf_get_app_id(struct nfp_pf *pf); u8 __iomem * nfp_pf_map_rtsym(struct nfp_pf *pf, const char *name, const char *sym_fmt, unsigned int min_size, struct nfp_cpp_area **area); @@ -190,4 +191,7 @@ int nfp_shared_buf_pool_set(struct nfp_pf *pf, unsigned int sb, int nfp_devlink_params_register(struct nfp_pf *pf); void nfp_devlink_params_unregister(struct nfp_pf *pf); + +unsigned int nfp_net_lr2speed(unsigned int linkrate); +unsigned int nfp_net_speed2lr(unsigned int speed); #endif /* NFP_MAIN_H */ diff --git a/drivers/net/ethernet/netronome/nfp/nfp_net_common.c b/drivers/net/ethernet/netronome/nfp/nfp_net_common.c index 349a2b1a19a2..27f4786ace4f 100644 --- a/drivers/net/ethernet/netronome/nfp/nfp_net_common.c +++ b/drivers/net/ethernet/netronome/nfp/nfp_net_common.c @@ -474,19 +474,22 @@ static void nfp_net_read_link_status(struct nfp_net *nn) { unsigned long flags; bool link_up; - u32 sts; + u16 sts; spin_lock_irqsave(&nn->link_status_lock, flags); - sts = nn_readl(nn, NFP_NET_CFG_STS); + sts = nn_readw(nn, NFP_NET_CFG_STS); link_up = !!(sts & NFP_NET_CFG_STS_LINK); if (nn->link_up == link_up) goto out; nn->link_up = link_up; - if (nn->port) + if (nn->port) { set_bit(NFP_PORT_CHANGED, &nn->port->flags); + if (nn->port->link_cb) + nn->port->link_cb(nn->port); + } if (nn->link_up) { netif_carrier_on(nn->dp.netdev); @@ -768,9 +771,7 @@ nfp_net_napi_add(struct nfp_net_dp *dp, struct nfp_net_r_vector *r_vec, int idx) { if (dp->netdev) netif_napi_add(dp->netdev, &r_vec->napi, - nfp_net_has_xsk_pool_slow(dp, idx) ? - dp->ops->xsk_poll : dp->ops->poll, - NAPI_POLL_WEIGHT); + nfp_net_has_xsk_pool_slow(dp, idx) ? dp->ops->xsk_poll : dp->ops->poll); else tasklet_enable(&r_vec->tasklet); } diff --git a/drivers/net/ethernet/netronome/nfp/nfp_net_ctrl.h b/drivers/net/ethernet/netronome/nfp/nfp_net_ctrl.h index ac05ec34d69e..6714d5e8fdab 100644 --- a/drivers/net/ethernet/netronome/nfp/nfp_net_ctrl.h +++ b/drivers/net/ethernet/netronome/nfp/nfp_net_ctrl.h @@ -14,6 +14,9 @@ #include <linux/types.h> +/* 64-bit per app capabilities */ +#define NFP_NET_APP_CAP_SP_INDIFF BIT_ULL(0) /* indifferent to port speed */ + /* Configuration BAR size. * * The configuration BAR is 8K in size, but due to @@ -193,6 +196,10 @@ #define NFP_NET_CFG_STS_LINK_RATE_40G 5 #define NFP_NET_CFG_STS_LINK_RATE_50G 6 #define NFP_NET_CFG_STS_LINK_RATE_100G 7 +/* NSP Link rate is a 16-bit word. It's determined by NSP and + * written to CFG BAR by NFP driver. + */ +#define NFP_NET_CFG_STS_NSP_LINK_RATE 0x0036 #define NFP_NET_CFG_CAP 0x0038 #define NFP_NET_CFG_MAX_TXRINGS 0x003c #define NFP_NET_CFG_MAX_RXRINGS 0x0040 diff --git a/drivers/net/ethernet/netronome/nfp/nfp_net_ethtool.c b/drivers/net/ethernet/netronome/nfp/nfp_net_ethtool.c index b1b1b648e40c..22a5d2419084 100644 --- a/drivers/net/ethernet/netronome/nfp/nfp_net_ethtool.c +++ b/drivers/net/ethernet/netronome/nfp/nfp_net_ethtool.c @@ -205,7 +205,7 @@ nfp_get_drvinfo(struct nfp_app *app, struct pci_dev *pdev, { char nsp_version[ETHTOOL_FWVERS_LEN] = {}; - strlcpy(drvinfo->driver, dev_driver_string(&pdev->dev), + strscpy(drvinfo->driver, dev_driver_string(&pdev->dev), sizeof(drvinfo->driver)); nfp_net_get_nspinfo(app, nsp_version); snprintf(drvinfo->fw_version, sizeof(drvinfo->fw_version), @@ -222,18 +222,49 @@ nfp_net_get_drvinfo(struct net_device *netdev, struct ethtool_drvinfo *drvinfo) snprintf(vnic_version, sizeof(vnic_version), "%d.%d.%d.%d", nn->fw_ver.extend, nn->fw_ver.class, nn->fw_ver.major, nn->fw_ver.minor); - strlcpy(drvinfo->bus_info, pci_name(nn->pdev), + strscpy(drvinfo->bus_info, pci_name(nn->pdev), sizeof(drvinfo->bus_info)); nfp_get_drvinfo(nn->app, nn->pdev, vnic_version, drvinfo); } +static int +nfp_net_nway_reset(struct net_device *netdev) +{ + struct nfp_eth_table_port *eth_port; + struct nfp_port *port; + int err; + + port = nfp_port_from_netdev(netdev); + eth_port = nfp_port_get_eth_port(port); + if (!eth_port) + return -EOPNOTSUPP; + + if (!netif_running(netdev)) + return 0; + + err = nfp_eth_set_configured(port->app->cpp, eth_port->index, false); + if (err) { + netdev_info(netdev, "Link down failed: %d\n", err); + return err; + } + + err = nfp_eth_set_configured(port->app->cpp, eth_port->index, true); + if (err) { + netdev_info(netdev, "Link up failed: %d\n", err); + return err; + } + + netdev_info(netdev, "Link reset succeeded\n"); + return 0; +} + static void nfp_app_get_drvinfo(struct net_device *netdev, struct ethtool_drvinfo *drvinfo) { struct nfp_app *app = nfp_app_from_netdev(netdev); - strlcpy(drvinfo->bus_info, pci_name(app->pdev), + strscpy(drvinfo->bus_info, pci_name(app->pdev), sizeof(drvinfo->bus_info)); nfp_get_drvinfo(app, app->pdev, "*", drvinfo); } @@ -273,20 +304,11 @@ static int nfp_net_get_link_ksettings(struct net_device *netdev, struct ethtool_link_ksettings *cmd) { - static const u32 ls_to_ethtool[] = { - [NFP_NET_CFG_STS_LINK_RATE_UNSUPPORTED] = 0, - [NFP_NET_CFG_STS_LINK_RATE_UNKNOWN] = SPEED_UNKNOWN, - [NFP_NET_CFG_STS_LINK_RATE_1G] = SPEED_1000, - [NFP_NET_CFG_STS_LINK_RATE_10G] = SPEED_10000, - [NFP_NET_CFG_STS_LINK_RATE_25G] = SPEED_25000, - [NFP_NET_CFG_STS_LINK_RATE_40G] = SPEED_40000, - [NFP_NET_CFG_STS_LINK_RATE_50G] = SPEED_50000, - [NFP_NET_CFG_STS_LINK_RATE_100G] = SPEED_100000, - }; struct nfp_eth_table_port *eth_port; struct nfp_port *port; struct nfp_net *nn; - u32 sts, ls; + unsigned int speed; + u16 sts; /* Init to unknowns */ ethtool_link_ksettings_add_link_mode(cmd, supported, FIBRE); @@ -299,8 +321,13 @@ nfp_net_get_link_ksettings(struct net_device *netdev, if (eth_port) { ethtool_link_ksettings_add_link_mode(cmd, supported, Pause); ethtool_link_ksettings_add_link_mode(cmd, advertising, Pause); - cmd->base.autoneg = eth_port->aneg != NFP_ANEG_DISABLED ? - AUTONEG_ENABLE : AUTONEG_DISABLE; + if (eth_port->supp_aneg) { + ethtool_link_ksettings_add_link_mode(cmd, supported, Autoneg); + if (eth_port->aneg == NFP_ANEG_AUTO) { + ethtool_link_ksettings_add_link_mode(cmd, advertising, Autoneg); + cmd->base.autoneg = AUTONEG_ENABLE; + } + } nfp_net_set_fec_link_mode(eth_port, cmd); } @@ -319,18 +346,15 @@ nfp_net_get_link_ksettings(struct net_device *netdev, return -EOPNOTSUPP; nn = netdev_priv(netdev); - sts = nn_readl(nn, NFP_NET_CFG_STS); - - ls = FIELD_GET(NFP_NET_CFG_STS_LINK_RATE, sts); - if (ls == NFP_NET_CFG_STS_LINK_RATE_UNSUPPORTED) + sts = nn_readw(nn, NFP_NET_CFG_STS); + speed = nfp_net_lr2speed(FIELD_GET(NFP_NET_CFG_STS_LINK_RATE, sts)); + if (!speed) return -EOPNOTSUPP; - if (ls == NFP_NET_CFG_STS_LINK_RATE_UNKNOWN || - ls >= ARRAY_SIZE(ls_to_ethtool)) - return 0; - - cmd->base.speed = ls_to_ethtool[ls]; - cmd->base.duplex = DUPLEX_FULL; + if (speed != SPEED_UNKNOWN) { + cmd->base.speed = speed; + cmd->base.duplex = DUPLEX_FULL; + } return 0; } @@ -339,6 +363,7 @@ static int nfp_net_set_link_ksettings(struct net_device *netdev, const struct ethtool_link_ksettings *cmd) { + bool req_aneg = (cmd->base.autoneg == AUTONEG_ENABLE); struct nfp_eth_table_port *eth_port; struct nfp_port *port; struct nfp_nsp *nsp; @@ -358,13 +383,25 @@ nfp_net_set_link_ksettings(struct net_device *netdev, if (IS_ERR(nsp)) return PTR_ERR(nsp); - err = __nfp_eth_set_aneg(nsp, cmd->base.autoneg == AUTONEG_ENABLE ? - NFP_ANEG_AUTO : NFP_ANEG_DISABLED); + if (req_aneg && !eth_port->supp_aneg) { + netdev_warn(netdev, "Autoneg is not supported.\n"); + err = -EOPNOTSUPP; + goto err_bad_set; + } + + err = __nfp_eth_set_aneg(nsp, req_aneg ? NFP_ANEG_AUTO : NFP_ANEG_DISABLED); if (err) goto err_bad_set; + if (cmd->base.speed != SPEED_UNKNOWN) { u32 speed = cmd->base.speed / eth_port->lanes; + if (req_aneg) { + netdev_err(netdev, "Speed changing is not allowed when working on autoneg mode.\n"); + err = -EINVAL; + goto err_bad_set; + } + err = __nfp_eth_set_speed(nsp, speed); if (err) goto err_bad_set; @@ -1008,7 +1045,7 @@ nfp_port_get_fecparam(struct net_device *netdev, return 0; param->fec = nfp_port_fec_nsp_to_ethtool(eth_port->fec_modes_supported); - param->active_fec = nfp_port_fec_nsp_to_ethtool(eth_port->fec); + param->active_fec = nfp_port_fec_nsp_to_ethtool(BIT(eth_port->act_fec)); return 0; } @@ -1676,11 +1713,166 @@ static int nfp_net_set_phys_id(struct net_device *netdev, return err; } +#define NFP_EEPROM_LEN ETH_ALEN + +static int +nfp_net_get_eeprom_len(struct net_device *netdev) +{ + struct nfp_eth_table_port *eth_port; + struct nfp_port *port; + + port = nfp_port_from_netdev(netdev); + eth_port = __nfp_port_get_eth_port(port); + if (!eth_port) + return 0; + + return NFP_EEPROM_LEN; +} + +static int +nfp_net_get_nsp_hwindex(struct net_device *netdev, + struct nfp_nsp **nspptr, + u32 *index) +{ + struct nfp_eth_table_port *eth_port; + struct nfp_port *port; + struct nfp_nsp *nsp; + int err; + + port = nfp_port_from_netdev(netdev); + eth_port = __nfp_port_get_eth_port(port); + if (!eth_port) + return -EOPNOTSUPP; + + nsp = nfp_nsp_open(port->app->cpp); + if (IS_ERR(nsp)) { + err = PTR_ERR(nsp); + netdev_err(netdev, "Failed to access the NSP: %d\n", err); + return err; + } + + if (!nfp_nsp_has_hwinfo_lookup(nsp)) { + netdev_err(netdev, "NSP doesn't support PF MAC generation\n"); + nfp_nsp_close(nsp); + return -EOPNOTSUPP; + } + + *nspptr = nsp; + *index = eth_port->eth_index; + + return 0; +} + +static int +nfp_net_get_port_mac_by_hwinfo(struct net_device *netdev, + u8 *mac_addr) +{ + char hwinfo[32] = {}; + struct nfp_nsp *nsp; + u32 index; + int err; + + err = nfp_net_get_nsp_hwindex(netdev, &nsp, &index); + if (err) + return err; + + snprintf(hwinfo, sizeof(hwinfo), "eth%u.mac", index); + err = nfp_nsp_hwinfo_lookup(nsp, hwinfo, sizeof(hwinfo)); + nfp_nsp_close(nsp); + if (err) { + netdev_err(netdev, "Reading persistent MAC address failed: %d\n", + err); + return -EOPNOTSUPP; + } + + if (sscanf(hwinfo, "%02hhx:%02hhx:%02hhx:%02hhx:%02hhx:%02hhx", + &mac_addr[0], &mac_addr[1], &mac_addr[2], + &mac_addr[3], &mac_addr[4], &mac_addr[5]) != 6) { + netdev_err(netdev, "Can't parse persistent MAC address (%s)\n", + hwinfo); + return -EOPNOTSUPP; + } + + return 0; +} + +static int +nfp_net_set_port_mac_by_hwinfo(struct net_device *netdev, + u8 *mac_addr) +{ + char hwinfo[32] = {}; + struct nfp_nsp *nsp; + u32 index; + int err; + + err = nfp_net_get_nsp_hwindex(netdev, &nsp, &index); + if (err) + return err; + + snprintf(hwinfo, sizeof(hwinfo), + "eth%u.mac=%02hhx:%02hhx:%02hhx:%02hhx:%02hhx:%02hhx", + index, mac_addr[0], mac_addr[1], mac_addr[2], mac_addr[3], + mac_addr[4], mac_addr[5]); + + err = nfp_nsp_hwinfo_set(nsp, hwinfo, sizeof(hwinfo)); + nfp_nsp_close(nsp); + if (err) { + netdev_err(netdev, "HWinfo set failed: %d, hwinfo: %s\n", + err, hwinfo); + return -EOPNOTSUPP; + } + + return 0; +} + +static int +nfp_net_get_eeprom(struct net_device *netdev, + struct ethtool_eeprom *eeprom, u8 *bytes) +{ + struct nfp_net *nn = netdev_priv(netdev); + u8 buf[NFP_EEPROM_LEN] = {}; + + if (eeprom->len == 0) + return -EINVAL; + + if (nfp_net_get_port_mac_by_hwinfo(netdev, buf)) + return -EOPNOTSUPP; + + eeprom->magic = nn->pdev->vendor | (nn->pdev->device << 16); + memcpy(bytes, buf + eeprom->offset, eeprom->len); + + return 0; +} + +static int +nfp_net_set_eeprom(struct net_device *netdev, + struct ethtool_eeprom *eeprom, u8 *bytes) +{ + struct nfp_net *nn = netdev_priv(netdev); + u8 buf[NFP_EEPROM_LEN] = {}; + + if (eeprom->len == 0) + return -EINVAL; + + if (eeprom->magic != (nn->pdev->vendor | nn->pdev->device << 16)) + return -EINVAL; + + if (nfp_net_get_port_mac_by_hwinfo(netdev, buf)) + return -EOPNOTSUPP; + + memcpy(buf + eeprom->offset, bytes, eeprom->len); + if (nfp_net_set_port_mac_by_hwinfo(netdev, buf)) + return -EOPNOTSUPP; + + return 0; +} + static const struct ethtool_ops nfp_net_ethtool_ops = { .supported_coalesce_params = ETHTOOL_COALESCE_USECS | ETHTOOL_COALESCE_MAX_FRAMES | ETHTOOL_COALESCE_USE_ADAPTIVE, .get_drvinfo = nfp_net_get_drvinfo, + .nway_reset = nfp_net_nway_reset, .get_link = ethtool_op_get_link, .get_ringparam = nfp_net_get_ringparam, .set_ringparam = nfp_net_set_ringparam, @@ -1699,6 +1891,9 @@ static const struct ethtool_ops nfp_net_ethtool_ops = { .set_dump = nfp_app_set_dump, .get_dump_flag = nfp_app_get_dump_flag, .get_dump_data = nfp_app_get_dump_data, + .get_eeprom_len = nfp_net_get_eeprom_len, + .get_eeprom = nfp_net_get_eeprom, + .set_eeprom = nfp_net_set_eeprom, .get_module_info = nfp_port_get_module_info, .get_module_eeprom = nfp_port_get_module_eeprom, .get_coalesce = nfp_net_get_coalesce, @@ -1715,6 +1910,7 @@ static const struct ethtool_ops nfp_net_ethtool_ops = { const struct ethtool_ops nfp_port_ethtool_ops = { .get_drvinfo = nfp_app_get_drvinfo, + .nway_reset = nfp_net_nway_reset, .get_link = ethtool_op_get_link, .get_strings = nfp_port_get_strings, .get_ethtool_stats = nfp_port_get_stats, diff --git a/drivers/net/ethernet/netronome/nfp/nfp_net_main.c b/drivers/net/ethernet/netronome/nfp/nfp_net_main.c index ca4e05650fe6..3bae92dc899e 100644 --- a/drivers/net/ethernet/netronome/nfp/nfp_net_main.c +++ b/drivers/net/ethernet/netronome/nfp/nfp_net_main.c @@ -77,12 +77,6 @@ static int nfp_net_pf_get_num_ports(struct nfp_pf *pf) return nfp_pf_rtsym_read_optional(pf, "nfd_cfg_pf%u_num_ports", 1); } -static int nfp_net_pf_get_app_id(struct nfp_pf *pf) -{ - return nfp_pf_rtsym_read_optional(pf, "_pf%u_net_app_id", - NFP_APP_CORE_NIC); -} - static void nfp_net_pf_free_vnic(struct nfp_pf *pf, struct nfp_net *nn) { if (nfp_net_is_data_vnic(nn)) @@ -202,6 +196,9 @@ nfp_net_pf_alloc_vnics(struct nfp_pf *pf, void __iomem *ctrl_bar, goto err_free_prev; } + if (nn->port) + nn->port->link_cb = nfp_net_refresh_port_table; + ctrl_bar += NFP_PF_CSR_SLICE_SIZE; /* Kill the vNIC if app init marked it as invalid */ @@ -523,6 +520,57 @@ err_unmap_ctrl: return err; } +static const unsigned int lr_to_speed[] = { + [NFP_NET_CFG_STS_LINK_RATE_UNSUPPORTED] = 0, + [NFP_NET_CFG_STS_LINK_RATE_UNKNOWN] = SPEED_UNKNOWN, + [NFP_NET_CFG_STS_LINK_RATE_1G] = SPEED_1000, + [NFP_NET_CFG_STS_LINK_RATE_10G] = SPEED_10000, + [NFP_NET_CFG_STS_LINK_RATE_25G] = SPEED_25000, + [NFP_NET_CFG_STS_LINK_RATE_40G] = SPEED_40000, + [NFP_NET_CFG_STS_LINK_RATE_50G] = SPEED_50000, + [NFP_NET_CFG_STS_LINK_RATE_100G] = SPEED_100000, +}; + +unsigned int nfp_net_lr2speed(unsigned int linkrate) +{ + if (linkrate < ARRAY_SIZE(lr_to_speed)) + return lr_to_speed[linkrate]; + + return SPEED_UNKNOWN; +} + +unsigned int nfp_net_speed2lr(unsigned int speed) +{ + int i; + + for (i = 0; i < ARRAY_SIZE(lr_to_speed); i++) { + if (speed == lr_to_speed[i]) + return i; + } + + return NFP_NET_CFG_STS_LINK_RATE_UNKNOWN; +} + +static void nfp_net_notify_port_speed(struct nfp_port *port) +{ + struct net_device *netdev = port->netdev; + struct nfp_net *nn; + u16 sts; + + if (!nfp_netdev_is_nfp_net(netdev)) + return; + + nn = netdev_priv(netdev); + sts = nn_readw(nn, NFP_NET_CFG_STS); + + if (!(sts & NFP_NET_CFG_STS_LINK)) { + nn_writew(nn, NFP_NET_CFG_STS_NSP_LINK_RATE, NFP_NET_CFG_STS_LINK_RATE_UNKNOWN); + return; + } + + nn_writew(nn, NFP_NET_CFG_STS_NSP_LINK_RATE, nfp_net_speed2lr(port->eth_port->speed)); +} + static int nfp_net_eth_port_update(struct nfp_cpp *cpp, struct nfp_port *port, struct nfp_eth_table *eth_table) @@ -544,6 +592,7 @@ nfp_net_eth_port_update(struct nfp_cpp *cpp, struct nfp_port *port, } memcpy(port->eth_port, eth_port, sizeof(*eth_port)); + nfp_net_notify_port_speed(port); return 0; } diff --git a/drivers/net/ethernet/netronome/nfp/nfp_port.h b/drivers/net/ethernet/netronome/nfp/nfp_port.h index d1ebe6c72f7f..6793cdf9ff11 100644 --- a/drivers/net/ethernet/netronome/nfp/nfp_port.h +++ b/drivers/net/ethernet/netronome/nfp/nfp_port.h @@ -46,6 +46,7 @@ enum nfp_port_flags { * @tc_offload_cnt: number of active TC offloads, how offloads are counted * is not defined, use as a boolean * @app: backpointer to the app structure + * @link_cb: callback when link status changed * @dl_port: devlink port structure * @eth_id: for %NFP_PORT_PHYS_PORT port ID in NFP enumeration scheme * @eth_forced: for %NFP_PORT_PHYS_PORT port is forced UP or DOWN, don't change @@ -66,6 +67,7 @@ struct nfp_port { unsigned long tc_offload_cnt; struct nfp_app *app; + void (*link_cb)(struct nfp_port *port); struct devlink_port dl_port; diff --git a/drivers/net/ethernet/netronome/nfp/nfpcore/nfp_nsp.h b/drivers/net/ethernet/netronome/nfp/nfpcore/nfp_nsp.h index 77d66855be42..992d72ac98d3 100644 --- a/drivers/net/ethernet/netronome/nfp/nfpcore/nfp_nsp.h +++ b/drivers/net/ethernet/netronome/nfp/nfpcore/nfp_nsp.h @@ -132,6 +132,7 @@ enum nfp_eth_fec { * @ports.interface: interface (module) plugged in * @ports.media: media type of the @interface * @ports.fec: forward error correction mode + * @ports.act_fec: active forward error correction mode * @ports.aneg: auto negotiation mode * @ports.mac_addr: interface MAC address * @ports.label_port: port id @@ -162,6 +163,7 @@ struct nfp_eth_table { enum nfp_eth_media media; enum nfp_eth_fec fec; + enum nfp_eth_fec act_fec; enum nfp_eth_aneg aneg; u8 mac_addr[ETH_ALEN]; @@ -172,6 +174,7 @@ struct nfp_eth_table { bool enabled; bool tx_enabled; bool rx_enabled; + bool supp_aneg; bool override_changed; diff --git a/drivers/net/ethernet/netronome/nfp/nfpcore/nfp_nsp_eth.c b/drivers/net/ethernet/netronome/nfp/nfpcore/nfp_nsp_eth.c index 4cc38799eabc..bb64efec4c46 100644 --- a/drivers/net/ethernet/netronome/nfp/nfpcore/nfp_nsp_eth.c +++ b/drivers/net/ethernet/netronome/nfp/nfpcore/nfp_nsp_eth.c @@ -27,6 +27,7 @@ #define NSP_ETH_PORT_PHYLABEL GENMASK_ULL(59, 54) #define NSP_ETH_PORT_FEC_SUPP_BASER BIT_ULL(60) #define NSP_ETH_PORT_FEC_SUPP_RS BIT_ULL(61) +#define NSP_ETH_PORT_SUPP_ANEG BIT_ULL(63) #define NSP_ETH_PORT_LANES_MASK cpu_to_le64(NSP_ETH_PORT_LANES) @@ -40,6 +41,7 @@ #define NSP_ETH_STATE_OVRD_CHNG BIT_ULL(22) #define NSP_ETH_STATE_ANEG GENMASK_ULL(25, 23) #define NSP_ETH_STATE_FEC GENMASK_ULL(27, 26) +#define NSP_ETH_STATE_ACT_FEC GENMASK_ULL(29, 28) #define NSP_ETH_CTRL_CONFIGURED BIT_ULL(0) #define NSP_ETH_CTRL_ENABLED BIT_ULL(1) @@ -170,7 +172,14 @@ nfp_eth_port_translate(struct nfp_nsp *nsp, const union eth_table_entry *src, if (dst->fec_modes_supported) dst->fec_modes_supported |= NFP_FEC_AUTO | NFP_FEC_DISABLED; - dst->fec = 1 << FIELD_GET(NSP_ETH_STATE_FEC, state); + dst->fec = FIELD_GET(NSP_ETH_STATE_FEC, state); + dst->act_fec = dst->fec; + + if (nfp_nsp_get_abi_ver_minor(nsp) < 33) + return; + + dst->act_fec = FIELD_GET(NSP_ETH_STATE_ACT_FEC, state); + dst->supp_aneg = FIELD_GET(NSP_ETH_PORT_SUPP_ANEG, port); } static void diff --git a/drivers/net/ethernet/ni/nixge.c b/drivers/net/ethernet/ni/nixge.c index 4b3482ce90a1..3db4a2431741 100644 --- a/drivers/net/ethernet/ni/nixge.c +++ b/drivers/net/ethernet/ni/nixge.c @@ -990,8 +990,8 @@ static const struct net_device_ops nixge_netdev_ops = { static void nixge_ethtools_get_drvinfo(struct net_device *ndev, struct ethtool_drvinfo *ed) { - strlcpy(ed->driver, "nixge", sizeof(ed->driver)); - strlcpy(ed->bus_info, "platform", sizeof(ed->bus_info)); + strscpy(ed->driver, "nixge", sizeof(ed->driver)); + strscpy(ed->bus_info, "platform", sizeof(ed->bus_info)); } static int @@ -1294,7 +1294,7 @@ static int nixge_probe(struct platform_device *pdev) priv->ndev = ndev; priv->dev = &pdev->dev; - netif_napi_add(ndev, &priv->napi, nixge_poll, NAPI_POLL_WEIGHT); + netif_napi_add(ndev, &priv->napi, nixge_poll); err = nixge_of_get_resources(pdev); if (err) goto free_netdev; diff --git a/drivers/net/ethernet/nvidia/forcedeth.c b/drivers/net/ethernet/nvidia/forcedeth.c index 5116badaf091..daa028729d44 100644 --- a/drivers/net/ethernet/nvidia/forcedeth.c +++ b/drivers/net/ethernet/nvidia/forcedeth.c @@ -4291,9 +4291,9 @@ static void nv_do_stats_poll(struct timer_list *t) static void nv_get_drvinfo(struct net_device *dev, struct ethtool_drvinfo *info) { struct fe_priv *np = netdev_priv(dev); - strlcpy(info->driver, DRV_NAME, sizeof(info->driver)); - strlcpy(info->version, FORCEDETH_VERSION, sizeof(info->version)); - strlcpy(info->bus_info, pci_name(np->pci_dev), sizeof(info->bus_info)); + strscpy(info->driver, DRV_NAME, sizeof(info->driver)); + strscpy(info->version, FORCEDETH_VERSION, sizeof(info->version)); + strscpy(info->bus_info, pci_name(np->pci_dev), sizeof(info->bus_info)); } static void nv_get_wol(struct net_device *dev, struct ethtool_wolinfo *wolinfo) @@ -5876,7 +5876,7 @@ static int nv_probe(struct pci_dev *pci_dev, const struct pci_device_id *id) else dev->netdev_ops = &nv_netdev_ops_optimized; - netif_napi_add(dev, &np->napi, nv_napi_poll, NAPI_POLL_WEIGHT); + netif_napi_add(dev, &np->napi, nv_napi_poll); dev->ethtool_ops = &ops; dev->watchdog_timeo = NV_WATCHDOG_TIMEO; diff --git a/drivers/net/ethernet/nxp/lpc_eth.c b/drivers/net/ethernet/nxp/lpc_eth.c index f606d75b33b4..1a4a272f4c5c 100644 --- a/drivers/net/ethernet/nxp/lpc_eth.c +++ b/drivers/net/ethernet/nxp/lpc_eth.c @@ -1184,9 +1184,9 @@ static int lpc_eth_open(struct net_device *ndev) static void lpc_eth_ethtool_getdrvinfo(struct net_device *ndev, struct ethtool_drvinfo *info) { - strlcpy(info->driver, MODNAME, sizeof(info->driver)); - strlcpy(info->version, DRV_VERSION, sizeof(info->version)); - strlcpy(info->bus_info, dev_name(ndev->dev.parent), + strscpy(info->driver, MODNAME, sizeof(info->driver)); + strscpy(info->version, DRV_VERSION, sizeof(info->version)); + strscpy(info->bus_info, dev_name(ndev->dev.parent), sizeof(info->bus_info)); } diff --git a/drivers/net/ethernet/oki-semi/pch_gbe/pch_gbe_ethtool.c b/drivers/net/ethernet/oki-semi/pch_gbe/pch_gbe_ethtool.c index 84cc79e928c8..541b8bcd3223 100644 --- a/drivers/net/ethernet/oki-semi/pch_gbe/pch_gbe_ethtool.c +++ b/drivers/net/ethernet/oki-semi/pch_gbe/pch_gbe_ethtool.c @@ -169,9 +169,9 @@ static void pch_gbe_get_drvinfo(struct net_device *netdev, { struct pch_gbe_adapter *adapter = netdev_priv(netdev); - strlcpy(drvinfo->driver, KBUILD_MODNAME, sizeof(drvinfo->driver)); - strlcpy(drvinfo->version, pch_driver_version, sizeof(drvinfo->version)); - strlcpy(drvinfo->bus_info, pci_name(adapter->pdev), + strscpy(drvinfo->driver, KBUILD_MODNAME, sizeof(drvinfo->driver)); + strscpy(drvinfo->version, pch_driver_version, sizeof(drvinfo->version)); + strscpy(drvinfo->bus_info, pci_name(adapter->pdev), sizeof(drvinfo->bus_info)); } diff --git a/drivers/net/ethernet/oki-semi/pch_gbe/pch_gbe_main.c b/drivers/net/ethernet/oki-semi/pch_gbe/pch_gbe_main.c index 46da937ad27f..3f2c30184752 100644 --- a/drivers/net/ethernet/oki-semi/pch_gbe/pch_gbe_main.c +++ b/drivers/net/ethernet/oki-semi/pch_gbe/pch_gbe_main.c @@ -2516,8 +2516,7 @@ static int pch_gbe_probe(struct pci_dev *pdev, netdev->netdev_ops = &pch_gbe_netdev_ops; netdev->watchdog_timeo = PCH_GBE_WATCHDOG_PERIOD; - netif_napi_add(netdev, &adapter->napi, - pch_gbe_napi_poll, NAPI_POLL_WEIGHT); + netif_napi_add(netdev, &adapter->napi, pch_gbe_napi_poll); netdev->hw_features = NETIF_F_RXCSUM | NETIF_F_IP_CSUM | NETIF_F_IPV6_CSUM; netdev->features = netdev->hw_features; diff --git a/drivers/net/ethernet/packetengines/hamachi.c b/drivers/net/ethernet/packetengines/hamachi.c index 9c408328be0d..1cc001087193 100644 --- a/drivers/net/ethernet/packetengines/hamachi.c +++ b/drivers/net/ethernet/packetengines/hamachi.c @@ -1819,9 +1819,9 @@ static void hamachi_get_drvinfo(struct net_device *dev, struct ethtool_drvinfo * { struct hamachi_private *np = netdev_priv(dev); - strlcpy(info->driver, DRV_NAME, sizeof(info->driver)); - strlcpy(info->version, DRV_VERSION, sizeof(info->version)); - strlcpy(info->bus_info, pci_name(np->pci_dev), sizeof(info->bus_info)); + strscpy(info->driver, DRV_NAME, sizeof(info->driver)); + strscpy(info->version, DRV_VERSION, sizeof(info->version)); + strscpy(info->bus_info, pci_name(np->pci_dev), sizeof(info->bus_info)); } static int hamachi_get_link_ksettings(struct net_device *dev, diff --git a/drivers/net/ethernet/packetengines/yellowfin.c b/drivers/net/ethernet/packetengines/yellowfin.c index 03650022d444..640ac01689fb 100644 --- a/drivers/net/ethernet/packetengines/yellowfin.c +++ b/drivers/net/ethernet/packetengines/yellowfin.c @@ -1340,9 +1340,9 @@ static void yellowfin_get_drvinfo(struct net_device *dev, struct ethtool_drvinfo { struct yellowfin_private *np = netdev_priv(dev); - strlcpy(info->driver, DRV_NAME, sizeof(info->driver)); - strlcpy(info->version, DRV_VERSION, sizeof(info->version)); - strlcpy(info->bus_info, pci_name(np->pci_dev), sizeof(info->bus_info)); + strscpy(info->driver, DRV_NAME, sizeof(info->driver)); + strscpy(info->version, DRV_VERSION, sizeof(info->version)); + strscpy(info->bus_info, pci_name(np->pci_dev), sizeof(info->bus_info)); } static const struct ethtool_ops ethtool_ops = { diff --git a/drivers/net/ethernet/pasemi/pasemi_mac.c b/drivers/net/ethernet/pasemi/pasemi_mac.c index f0ace3a0e85c..aaab590ef548 100644 --- a/drivers/net/ethernet/pasemi/pasemi_mac.c +++ b/drivers/net/ethernet/pasemi/pasemi_mac.c @@ -1697,7 +1697,7 @@ pasemi_mac_probe(struct pci_dev *pdev, const struct pci_device_id *ent) mac->pdev = pdev; mac->netdev = dev; - netif_napi_add(dev, &mac->napi, pasemi_mac_poll, 64); + netif_napi_add(dev, &mac->napi, pasemi_mac_poll); dev->features = NETIF_F_IP_CSUM | NETIF_F_LLTX | NETIF_F_SG | NETIF_F_HIGHDMA | NETIF_F_GSO; diff --git a/drivers/net/ethernet/pensando/ionic/ionic_bus_pci.c b/drivers/net/ethernet/pensando/ionic/ionic_bus_pci.c index 0a7a757494bc..ce436e97324a 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_bus_pci.c +++ b/drivers/net/ethernet/pensando/ionic/ionic_bus_pci.c @@ -320,16 +320,16 @@ static int ionic_probe(struct pci_dev *pdev, const struct pci_device_id *ent) dev_err(dev, "Cannot enable existing VFs: %d\n", err); } - err = ionic_lif_register(ionic->lif); + err = ionic_devlink_register(ionic); if (err) { - dev_err(dev, "Cannot register LIF: %d, aborting\n", err); + dev_err(dev, "Cannot register devlink: %d\n", err); goto err_out_deinit_lifs; } - err = ionic_devlink_register(ionic); + err = ionic_lif_register(ionic->lif); if (err) { - dev_err(dev, "Cannot register devlink: %d\n", err); - goto err_out_deregister_lifs; + dev_err(dev, "Cannot register LIF: %d, aborting\n", err); + goto err_out_deregister_devlink; } mod_timer(&ionic->watchdog_timer, @@ -337,8 +337,8 @@ static int ionic_probe(struct pci_dev *pdev, const struct pci_device_id *ent) return 0; -err_out_deregister_lifs: - ionic_lif_unregister(ionic->lif); +err_out_deregister_devlink: + ionic_devlink_unregister(ionic); err_out_deinit_lifs: ionic_vf_dealloc(ionic); ionic_lif_deinit(ionic->lif); @@ -380,8 +380,8 @@ static void ionic_remove(struct pci_dev *pdev) del_timer_sync(&ionic->watchdog_timer); if (ionic->lif) { - ionic_devlink_unregister(ionic); ionic_lif_unregister(ionic->lif); + ionic_devlink_unregister(ionic); ionic_lif_deinit(ionic->lif); ionic_lif_free(ionic->lif); ionic->lif = NULL; diff --git a/drivers/net/ethernet/pensando/ionic/ionic_lif.c b/drivers/net/ethernet/pensando/ionic/ionic_lif.c index 0be79c516781..5d58fd99be3c 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_lif.c +++ b/drivers/net/ethernet/pensando/ionic/ionic_lif.c @@ -774,8 +774,7 @@ static int ionic_lif_txq_init(struct ionic_lif *lif, struct ionic_qcq *qcq) dev_dbg(dev, "txq->hw_index %d\n", q->hw_index); if (test_bit(IONIC_LIF_F_SPLIT_INTR, lif->state)) - netif_napi_add(lif->netdev, &qcq->napi, ionic_tx_napi, - NAPI_POLL_WEIGHT); + netif_napi_add(lif->netdev, &qcq->napi, ionic_tx_napi); qcq->flags |= IONIC_QCQ_F_INITED; @@ -830,11 +829,9 @@ static int ionic_lif_rxq_init(struct ionic_lif *lif, struct ionic_qcq *qcq) dev_dbg(dev, "rxq->hw_index %d\n", q->hw_index); if (test_bit(IONIC_LIF_F_SPLIT_INTR, lif->state)) - netif_napi_add(lif->netdev, &qcq->napi, ionic_rx_napi, - NAPI_POLL_WEIGHT); + netif_napi_add(lif->netdev, &qcq->napi, ionic_rx_napi); else - netif_napi_add(lif->netdev, &qcq->napi, ionic_txrx_napi, - NAPI_POLL_WEIGHT); + netif_napi_add(lif->netdev, &qcq->napi, ionic_txrx_napi); qcq->flags |= IONIC_QCQ_F_INITED; @@ -3165,8 +3162,7 @@ static int ionic_lif_adminq_init(struct ionic_lif *lif) dev_dbg(dev, "adminq->hw_type %d\n", q->hw_type); dev_dbg(dev, "adminq->hw_index %d\n", q->hw_index); - netif_napi_add(lif->netdev, &qcq->napi, ionic_adminq_napi, - NAPI_POLL_WEIGHT); + netif_napi_add(lif->netdev, &qcq->napi, ionic_adminq_napi); napi_enable(&qcq->napi); diff --git a/drivers/net/ethernet/qlogic/netxen/netxen_nic_ethtool.c b/drivers/net/ethernet/qlogic/netxen/netxen_nic_ethtool.c index 3c4a84ea6321..8c4cb910e09b 100644 --- a/drivers/net/ethernet/qlogic/netxen/netxen_nic_ethtool.c +++ b/drivers/net/ethernet/qlogic/netxen/netxen_nic_ethtool.c @@ -65,9 +65,9 @@ netxen_nic_get_drvinfo(struct net_device *dev, struct ethtool_drvinfo *drvinfo) u32 fw_minor = 0; u32 fw_build = 0; - strlcpy(drvinfo->driver, netxen_nic_driver_name, + strscpy(drvinfo->driver, netxen_nic_driver_name, sizeof(drvinfo->driver)); - strlcpy(drvinfo->version, NETXEN_NIC_LINUX_VERSIONID, + strscpy(drvinfo->version, NETXEN_NIC_LINUX_VERSIONID, sizeof(drvinfo->version)); fw_major = NXRD32(adapter, NETXEN_FW_VERSION_MAJOR); fw_minor = NXRD32(adapter, NETXEN_FW_VERSION_MINOR); @@ -75,7 +75,7 @@ netxen_nic_get_drvinfo(struct net_device *dev, struct ethtool_drvinfo *drvinfo) snprintf(drvinfo->fw_version, sizeof(drvinfo->fw_version), "%d.%d.%d", fw_major, fw_minor, fw_build); - strlcpy(drvinfo->bus_info, pci_name(adapter->pdev), + strscpy(drvinfo->bus_info, pci_name(adapter->pdev), sizeof(drvinfo->bus_info)); } diff --git a/drivers/net/ethernet/qlogic/netxen/netxen_nic_main.c b/drivers/net/ethernet/qlogic/netxen/netxen_nic_main.c index 4e6f00af17d9..de8d54b23f73 100644 --- a/drivers/net/ethernet/qlogic/netxen/netxen_nic_main.c +++ b/drivers/net/ethernet/qlogic/netxen/netxen_nic_main.c @@ -173,8 +173,7 @@ netxen_napi_add(struct netxen_adapter *adapter, struct net_device *netdev) for (ring = 0; ring < adapter->max_sds_rings; ring++) { sds_ring = &recv_ctx->sds_rings[ring]; - netif_napi_add(netdev, &sds_ring->napi, - netxen_nic_poll, NAPI_POLL_WEIGHT); + netif_napi_add(netdev, &sds_ring->napi, netxen_nic_poll); } return 0; diff --git a/drivers/net/ethernet/qlogic/qed/qed_int.c b/drivers/net/ethernet/qlogic/qed/qed_int.c index d701ecd3ba00..2661c483c67e 100644 --- a/drivers/net/ethernet/qlogic/qed/qed_int.c +++ b/drivers/net/ethernet/qlogic/qed/qed_int.c @@ -1119,7 +1119,7 @@ static int qed_int_deassertion(struct qed_hwfn *p_hwfn, snprintf(bit_name, 30, p_aeu->bit_name, num); else - strlcpy(bit_name, + strscpy(bit_name, p_aeu->bit_name, 30); /* We now need to pass bitmask in its diff --git a/drivers/net/ethernet/qlogic/qede/qede_ethtool.c b/drivers/net/ethernet/qlogic/qede/qede_ethtool.c index 97a7ab0826ed..8034d812d5a0 100644 --- a/drivers/net/ethernet/qlogic/qede/qede_ethtool.c +++ b/drivers/net/ethernet/qlogic/qede/qede_ethtool.c @@ -624,7 +624,7 @@ static void qede_get_drvinfo(struct net_device *ndev, struct qede_dev *edev = netdev_priv(ndev); char mbi[ETHTOOL_FWVERS_LEN]; - strlcpy(info->driver, "qede", sizeof(info->driver)); + strscpy(info->driver, "qede", sizeof(info->driver)); snprintf(storm, ETHTOOL_FWVERS_LEN, "%d.%d.%d.%d", edev->dev_info.common.fw_major, @@ -661,7 +661,7 @@ static void qede_get_drvinfo(struct net_device *ndev, "mfw %s", mfw); } - strlcpy(info->bus_info, pci_name(edev->pdev), sizeof(info->bus_info)); + strscpy(info->bus_info, pci_name(edev->pdev), sizeof(info->bus_info)); } static void qede_get_wol(struct net_device *ndev, struct ethtool_wolinfo *wol) diff --git a/drivers/net/ethernet/qlogic/qede/qede_main.c b/drivers/net/ethernet/qlogic/qede/qede_main.c index f56b679adb4b..953f304b8588 100644 --- a/drivers/net/ethernet/qlogic/qede/qede_main.c +++ b/drivers/net/ethernet/qlogic/qede/qede_main.c @@ -1214,7 +1214,7 @@ static int __qede_probe(struct pci_dev *pdev, u32 dp_module, u8 dp_level, /* Start the Slowpath-process */ memset(&sp_params, 0, sizeof(sp_params)); sp_params.int_mode = QED_INT_MODE_MSIX; - strlcpy(sp_params.name, "qede LAN", QED_DRV_VER_STR_SIZE); + strscpy(sp_params.name, "qede LAN", QED_DRV_VER_STR_SIZE); rc = qed_ops->common->slowpath_start(cdev, &sp_params); if (rc) { pr_notice("Cannot start slowpath\n"); @@ -1904,8 +1904,7 @@ static void qede_napi_add_enable(struct qede_dev *edev) /* Add NAPI objects */ for_each_queue(i) { - netif_napi_add(edev->ndev, &edev->fp_array[i].napi, - qede_poll, NAPI_POLL_WEIGHT); + netif_napi_add(edev->ndev, &edev->fp_array[i].napi, qede_poll); napi_enable(&edev->fp_array[i].napi); } } diff --git a/drivers/net/ethernet/qlogic/qla3xxx.c b/drivers/net/ethernet/qlogic/qla3xxx.c index 06f4d9a9e938..76072f8c3d2f 100644 --- a/drivers/net/ethernet/qlogic/qla3xxx.c +++ b/drivers/net/ethernet/qlogic/qla3xxx.c @@ -1736,10 +1736,10 @@ static void ql_get_drvinfo(struct net_device *ndev, struct ethtool_drvinfo *drvinfo) { struct ql3_adapter *qdev = netdev_priv(ndev); - strlcpy(drvinfo->driver, ql3xxx_driver_name, sizeof(drvinfo->driver)); - strlcpy(drvinfo->version, ql3xxx_driver_version, + strscpy(drvinfo->driver, ql3xxx_driver_name, sizeof(drvinfo->driver)); + strscpy(drvinfo->version, ql3xxx_driver_version, sizeof(drvinfo->version)); - strlcpy(drvinfo->bus_info, pci_name(qdev->pdev), + strscpy(drvinfo->bus_info, pci_name(qdev->pdev), sizeof(drvinfo->bus_info)); } @@ -3813,7 +3813,7 @@ static int ql3xxx_probe(struct pci_dev *pdev, ndev->ethtool_ops = &ql3xxx_ethtool_ops; ndev->watchdog_timeo = 5 * HZ; - netif_napi_add(ndev, &qdev->napi, ql_poll, 64); + netif_napi_add(ndev, &qdev->napi, ql_poll); ndev->irq = pdev->irq; diff --git a/drivers/net/ethernet/qlogic/qlcnic/qlcnic_ethtool.c b/drivers/net/ethernet/qlogic/qlcnic/qlcnic_ethtool.c index 54a2d653be63..1ee491f78c6b 100644 --- a/drivers/net/ethernet/qlogic/qlcnic/qlcnic_ethtool.c +++ b/drivers/net/ethernet/qlogic/qlcnic/qlcnic_ethtool.c @@ -277,10 +277,10 @@ qlcnic_get_drvinfo(struct net_device *dev, struct ethtool_drvinfo *drvinfo) snprintf(drvinfo->fw_version, sizeof(drvinfo->fw_version), "%d.%d.%d", fw_major, fw_minor, fw_build); - strlcpy(drvinfo->bus_info, pci_name(adapter->pdev), + strscpy(drvinfo->bus_info, pci_name(adapter->pdev), sizeof(drvinfo->bus_info)); - strlcpy(drvinfo->driver, qlcnic_driver_name, sizeof(drvinfo->driver)); - strlcpy(drvinfo->version, QLCNIC_LINUX_VERSIONID, + strscpy(drvinfo->driver, qlcnic_driver_name, sizeof(drvinfo->driver)); + strscpy(drvinfo->version, QLCNIC_LINUX_VERSIONID, sizeof(drvinfo->version)); } diff --git a/drivers/net/ethernet/qlogic/qlcnic/qlcnic_io.c b/drivers/net/ethernet/qlogic/qlcnic/qlcnic_io.c index 9da5e97f8a0a..92930a055cbc 100644 --- a/drivers/net/ethernet/qlogic/qlcnic/qlcnic_io.c +++ b/drivers/net/ethernet/qlogic/qlcnic/qlcnic_io.c @@ -1586,17 +1586,15 @@ int qlcnic_82xx_napi_add(struct qlcnic_adapter *adapter, sds_ring = &recv_ctx->sds_rings[ring]; if (qlcnic_check_multi_tx(adapter) && !adapter->ahw->diag_test) { - netif_napi_add(netdev, &sds_ring->napi, qlcnic_rx_poll, - NAPI_POLL_WEIGHT); + netif_napi_add(netdev, &sds_ring->napi, + qlcnic_rx_poll); } else { if (ring == (adapter->drv_sds_rings - 1)) netif_napi_add(netdev, &sds_ring->napi, - qlcnic_poll, - NAPI_POLL_WEIGHT); + qlcnic_poll); else netif_napi_add(netdev, &sds_ring->napi, - qlcnic_rx_poll, - NAPI_POLL_WEIGHT); + qlcnic_rx_poll); } } @@ -2115,17 +2113,14 @@ int qlcnic_83xx_napi_add(struct qlcnic_adapter *adapter, if (adapter->flags & QLCNIC_MSIX_ENABLED) { if (!(adapter->flags & QLCNIC_TX_INTR_SHARED)) netif_napi_add(netdev, &sds_ring->napi, - qlcnic_83xx_rx_poll, - NAPI_POLL_WEIGHT); + qlcnic_83xx_rx_poll); else netif_napi_add(netdev, &sds_ring->napi, - qlcnic_83xx_msix_sriov_vf_poll, - NAPI_POLL_WEIGHT); + qlcnic_83xx_msix_sriov_vf_poll); } else { netif_napi_add(netdev, &sds_ring->napi, - qlcnic_83xx_poll, - NAPI_POLL_WEIGHT); + qlcnic_83xx_poll); } } diff --git a/drivers/net/ethernet/qualcomm/emac/emac.c b/drivers/net/ethernet/qualcomm/emac/emac.c index a55c52696d49..3115b2c12898 100644 --- a/drivers/net/ethernet/qualcomm/emac/emac.c +++ b/drivers/net/ethernet/qualcomm/emac/emac.c @@ -684,8 +684,7 @@ static int emac_probe(struct platform_device *pdev) /* Initialize queues */ emac_mac_rx_tx_ring_init_all(pdev, adpt); - netif_napi_add(netdev, &adpt->rx_q.napi, emac_napi_rtx, - NAPI_POLL_WEIGHT); + netif_napi_add(netdev, &adpt->rx_q.napi, emac_napi_rtx); ret = register_netdev(netdev); if (ret) { diff --git a/drivers/net/ethernet/qualcomm/qca_debug.c b/drivers/net/ethernet/qualcomm/qca_debug.c index 792ce9a323cd..f62c39544e08 100644 --- a/drivers/net/ethernet/qualcomm/qca_debug.c +++ b/drivers/net/ethernet/qualcomm/qca_debug.c @@ -164,10 +164,10 @@ qcaspi_get_drvinfo(struct net_device *dev, struct ethtool_drvinfo *p) { struct qcaspi *qca = netdev_priv(dev); - strlcpy(p->driver, QCASPI_DRV_NAME, sizeof(p->driver)); - strlcpy(p->version, QCASPI_DRV_VERSION, sizeof(p->version)); - strlcpy(p->fw_version, "QCA7000", sizeof(p->fw_version)); - strlcpy(p->bus_info, dev_name(&qca->spi_dev->dev), + strscpy(p->driver, QCASPI_DRV_NAME, sizeof(p->driver)); + strscpy(p->version, QCASPI_DRV_VERSION, sizeof(p->version)); + strscpy(p->fw_version, "QCA7000", sizeof(p->fw_version)); + strscpy(p->bus_info, dev_name(&qca->spi_dev->dev), sizeof(p->bus_info)); } diff --git a/drivers/net/ethernet/qualcomm/rmnet/rmnet_map.h b/drivers/net/ethernet/qualcomm/rmnet/rmnet_map.h index e5a0b38f7dbe..2b033060fc20 100644 --- a/drivers/net/ethernet/qualcomm/rmnet/rmnet_map.h +++ b/drivers/net/ethernet/qualcomm/rmnet/rmnet_map.h @@ -19,7 +19,7 @@ struct rmnet_map_control_command { __be16 flow_control_seq_num; __be32 qos_id; } flow_control; - u8 data[0]; + DECLARE_FLEX_ARRAY(u8, data); }; } __aligned(1); diff --git a/drivers/net/ethernet/rdc/r6040.c b/drivers/net/ethernet/rdc/r6040.c index a6bf7d505178..eecd52ed1ed2 100644 --- a/drivers/net/ethernet/rdc/r6040.c +++ b/drivers/net/ethernet/rdc/r6040.c @@ -939,9 +939,9 @@ static void netdev_get_drvinfo(struct net_device *dev, { struct r6040_private *rp = netdev_priv(dev); - strlcpy(info->driver, DRV_NAME, sizeof(info->driver)); - strlcpy(info->version, DRV_VERSION, sizeof(info->version)); - strlcpy(info->bus_info, pci_name(rp->pdev), sizeof(info->bus_info)); + strscpy(info->driver, DRV_NAME, sizeof(info->driver)); + strscpy(info->version, DRV_VERSION, sizeof(info->version)); + strscpy(info->bus_info, pci_name(rp->pdev), sizeof(info->bus_info)); } static const struct ethtool_ops netdev_ethtool_ops = { @@ -1127,7 +1127,7 @@ static int r6040_init_one(struct pci_dev *pdev, const struct pci_device_id *ent) dev->ethtool_ops = &netdev_ethtool_ops; dev->watchdog_timeo = TX_TIMEOUT; - netif_napi_add(dev, &lp->napi, r6040_poll, 64); + netif_napi_add(dev, &lp->napi, r6040_poll); lp->mii_bus = mdiobus_alloc(); if (!lp->mii_bus) { diff --git a/drivers/net/ethernet/realtek/8139cp.c b/drivers/net/ethernet/realtek/8139cp.c index e0feeec13da6..f5786d78ed23 100644 --- a/drivers/net/ethernet/realtek/8139cp.c +++ b/drivers/net/ethernet/realtek/8139cp.c @@ -1382,9 +1382,9 @@ static void cp_get_drvinfo (struct net_device *dev, struct ethtool_drvinfo *info { struct cp_private *cp = netdev_priv(dev); - strlcpy(info->driver, DRV_NAME, sizeof(info->driver)); - strlcpy(info->version, DRV_VERSION, sizeof(info->version)); - strlcpy(info->bus_info, pci_name(cp->pdev), sizeof(info->bus_info)); + strscpy(info->driver, DRV_NAME, sizeof(info->driver)); + strscpy(info->version, DRV_VERSION, sizeof(info->version)); + strscpy(info->bus_info, pci_name(cp->pdev), sizeof(info->bus_info)); } static void cp_get_ringparam(struct net_device *dev, diff --git a/drivers/net/ethernet/realtek/8139too.c b/drivers/net/ethernet/realtek/8139too.c index 15b40fd93cd2..469e2e229c6e 100644 --- a/drivers/net/ethernet/realtek/8139too.c +++ b/drivers/net/ethernet/realtek/8139too.c @@ -1002,7 +1002,7 @@ static int rtl8139_init_one(struct pci_dev *pdev, dev->netdev_ops = &rtl8139_netdev_ops; dev->ethtool_ops = &rtl8139_ethtool_ops; dev->watchdog_timeo = TX_TIMEOUT; - netif_napi_add(dev, &tp->napi, rtl8139_poll, 64); + netif_napi_add(dev, &tp->napi, rtl8139_poll); /* note: the hardware is not capable of sg/csum/highdma, however * through the use of skb_copy_and_csum_dev we enable these @@ -2380,9 +2380,9 @@ static int rtl8139_set_wol(struct net_device *dev, struct ethtool_wolinfo *wol) static void rtl8139_get_drvinfo(struct net_device *dev, struct ethtool_drvinfo *info) { struct rtl8139_private *tp = netdev_priv(dev); - strlcpy(info->driver, DRV_NAME, sizeof(info->driver)); - strlcpy(info->version, DRV_VERSION, sizeof(info->version)); - strlcpy(info->bus_info, pci_name(tp->pci_dev), sizeof(info->bus_info)); + strscpy(info->driver, DRV_NAME, sizeof(info->driver)); + strscpy(info->version, DRV_VERSION, sizeof(info->version)); + strscpy(info->bus_info, pci_name(tp->pci_dev), sizeof(info->bus_info)); } static int rtl8139_get_link_ksettings(struct net_device *dev, diff --git a/drivers/net/ethernet/realtek/r8169.h b/drivers/net/ethernet/realtek/r8169.h index 8da4b66b71b5..55ef8251feb5 100644 --- a/drivers/net/ethernet/realtek/r8169.h +++ b/drivers/net/ethernet/realtek/r8169.h @@ -23,10 +23,10 @@ enum mac_version { RTL_GIGA_MAC_VER_09, RTL_GIGA_MAC_VER_10, RTL_GIGA_MAC_VER_11, - RTL_GIGA_MAC_VER_12, - RTL_GIGA_MAC_VER_13, + /* RTL_GIGA_MAC_VER_12 was handled the same as VER_17 */ + /* RTL_GIGA_MAC_VER_13 was merged with VER_10 */ RTL_GIGA_MAC_VER_14, - RTL_GIGA_MAC_VER_16, + /* RTL_GIGA_MAC_VER_16 was merged with VER_10 */ RTL_GIGA_MAC_VER_17, RTL_GIGA_MAC_VER_18, RTL_GIGA_MAC_VER_19, @@ -51,20 +51,20 @@ enum mac_version { RTL_GIGA_MAC_VER_38, RTL_GIGA_MAC_VER_39, RTL_GIGA_MAC_VER_40, - RTL_GIGA_MAC_VER_41, + /* support for RTL_GIGA_MAC_VER_41 has been removed */ RTL_GIGA_MAC_VER_42, RTL_GIGA_MAC_VER_43, RTL_GIGA_MAC_VER_44, - RTL_GIGA_MAC_VER_45, + /* support for RTL_GIGA_MAC_VER_45 has been removed */ RTL_GIGA_MAC_VER_46, - RTL_GIGA_MAC_VER_47, + /* support for RTL_GIGA_MAC_VER_47 has been removed */ RTL_GIGA_MAC_VER_48, - RTL_GIGA_MAC_VER_49, - RTL_GIGA_MAC_VER_50, + /* support for RTL_GIGA_MAC_VER_49 has been removed */ + /* support for RTL_GIGA_MAC_VER_50 has been removed */ RTL_GIGA_MAC_VER_51, RTL_GIGA_MAC_VER_52, RTL_GIGA_MAC_VER_53, - RTL_GIGA_MAC_VER_60, + /* support for RTL_GIGA_MAC_VER_60 has been removed */ RTL_GIGA_MAC_VER_61, RTL_GIGA_MAC_VER_63, RTL_GIGA_MAC_NONE diff --git a/drivers/net/ethernet/realtek/r8169_main.c b/drivers/net/ethernet/realtek/r8169_main.c index 1b7fdb4f056b..a73d061d9fcb 100644 --- a/drivers/net/ethernet/realtek/r8169_main.c +++ b/drivers/net/ethernet/realtek/r8169_main.c @@ -49,10 +49,8 @@ #define FIRMWARE_8106E_2 "rtl_nic/rtl8106e-2.fw" #define FIRMWARE_8168G_2 "rtl_nic/rtl8168g-2.fw" #define FIRMWARE_8168G_3 "rtl_nic/rtl8168g-3.fw" -#define FIRMWARE_8168H_1 "rtl_nic/rtl8168h-1.fw" #define FIRMWARE_8168H_2 "rtl_nic/rtl8168h-2.fw" #define FIRMWARE_8168FP_3 "rtl_nic/rtl8168fp-3.fw" -#define FIRMWARE_8107E_1 "rtl_nic/rtl8107e-1.fw" #define FIRMWARE_8107E_2 "rtl_nic/rtl8107e-2.fw" #define FIRMWARE_8125A_3 "rtl_nic/rtl8125a-3.fw" #define FIRMWARE_8125B_2 "rtl_nic/rtl8125b-2.fw" @@ -102,12 +100,9 @@ static const struct { [RTL_GIGA_MAC_VER_07] = {"RTL8102e" }, [RTL_GIGA_MAC_VER_08] = {"RTL8102e" }, [RTL_GIGA_MAC_VER_09] = {"RTL8102e/RTL8103e" }, - [RTL_GIGA_MAC_VER_10] = {"RTL8101e" }, + [RTL_GIGA_MAC_VER_10] = {"RTL8101e/RTL8100e" }, [RTL_GIGA_MAC_VER_11] = {"RTL8168b/8111b" }, - [RTL_GIGA_MAC_VER_12] = {"RTL8168b/8111b" }, - [RTL_GIGA_MAC_VER_13] = {"RTL8101e/RTL8100e" }, [RTL_GIGA_MAC_VER_14] = {"RTL8401" }, - [RTL_GIGA_MAC_VER_16] = {"RTL8101e" }, [RTL_GIGA_MAC_VER_17] = {"RTL8168b/8111b" }, [RTL_GIGA_MAC_VER_18] = {"RTL8168cp/8111cp" }, [RTL_GIGA_MAC_VER_19] = {"RTL8168c/8111c" }, @@ -131,20 +126,14 @@ static const struct { [RTL_GIGA_MAC_VER_38] = {"RTL8411", FIRMWARE_8411_1 }, [RTL_GIGA_MAC_VER_39] = {"RTL8106e", FIRMWARE_8106E_1}, [RTL_GIGA_MAC_VER_40] = {"RTL8168g/8111g", FIRMWARE_8168G_2}, - [RTL_GIGA_MAC_VER_41] = {"RTL8168g/8111g" }, [RTL_GIGA_MAC_VER_42] = {"RTL8168gu/8111gu", FIRMWARE_8168G_3}, [RTL_GIGA_MAC_VER_43] = {"RTL8106eus", FIRMWARE_8106E_2}, [RTL_GIGA_MAC_VER_44] = {"RTL8411b", FIRMWARE_8411_2 }, - [RTL_GIGA_MAC_VER_45] = {"RTL8168h/8111h", FIRMWARE_8168H_1}, [RTL_GIGA_MAC_VER_46] = {"RTL8168h/8111h", FIRMWARE_8168H_2}, - [RTL_GIGA_MAC_VER_47] = {"RTL8107e", FIRMWARE_8107E_1}, [RTL_GIGA_MAC_VER_48] = {"RTL8107e", FIRMWARE_8107E_2}, - [RTL_GIGA_MAC_VER_49] = {"RTL8168ep/8111ep" }, - [RTL_GIGA_MAC_VER_50] = {"RTL8168ep/8111ep" }, [RTL_GIGA_MAC_VER_51] = {"RTL8168ep/8111ep" }, [RTL_GIGA_MAC_VER_52] = {"RTL8168fp/RTL8117", FIRMWARE_8168FP_3}, [RTL_GIGA_MAC_VER_53] = {"RTL8168fp/RTL8117", }, - [RTL_GIGA_MAC_VER_60] = {"RTL8125A" }, [RTL_GIGA_MAC_VER_61] = {"RTL8125A", FIRMWARE_8125A_3}, /* reserve 62 for CFG_METHOD_4 in the vendor driver */ [RTL_GIGA_MAC_VER_63] = {"RTL8125B", FIRMWARE_8125B_2}, @@ -658,10 +647,8 @@ MODULE_FIRMWARE(FIRMWARE_8106E_1); MODULE_FIRMWARE(FIRMWARE_8106E_2); MODULE_FIRMWARE(FIRMWARE_8168G_2); MODULE_FIRMWARE(FIRMWARE_8168G_3); -MODULE_FIRMWARE(FIRMWARE_8168H_1); MODULE_FIRMWARE(FIRMWARE_8168H_2); MODULE_FIRMWARE(FIRMWARE_8168FP_3); -MODULE_FIRMWARE(FIRMWARE_8107E_1); MODULE_FIRMWARE(FIRMWARE_8107E_2); MODULE_FIRMWARE(FIRMWARE_8125A_3); MODULE_FIRMWARE(FIRMWARE_8125B_2); @@ -689,7 +676,7 @@ static void rtl_pci_commit(struct rtl8169_private *tp) static bool rtl_is_8125(struct rtl8169_private *tp) { - return tp->mac_version >= RTL_GIGA_MAC_VER_60; + return tp->mac_version >= RTL_GIGA_MAC_VER_61; } static bool rtl_is_8168evl_up(struct rtl8169_private *tp) @@ -892,8 +879,6 @@ static void rtl8168g_phy_suspend_quirk(struct rtl8169_private *tp, int value) { switch (tp->mac_version) { case RTL_GIGA_MAC_VER_40: - case RTL_GIGA_MAC_VER_41: - case RTL_GIGA_MAC_VER_49: if (value & BMCR_RESET || !(value & BMCR_PDOWN)) rtl_eri_set_bits(tp, 0x1a8, 0xfc000000); else @@ -1207,7 +1192,7 @@ static enum rtl_dash_type rtl_check_dash(struct rtl8169_private *tp) case RTL_GIGA_MAC_VER_28: case RTL_GIGA_MAC_VER_31: return r8168dp_check_dash(tp) ? RTL_DASH_DP : RTL_DASH_NONE; - case RTL_GIGA_MAC_VER_49 ... RTL_GIGA_MAC_VER_53: + case RTL_GIGA_MAC_VER_51 ... RTL_GIGA_MAC_VER_53: return r8168ep_check_dash(tp) ? RTL_DASH_EP : RTL_DASH_NONE; default: return RTL_DASH_NONE; @@ -1423,11 +1408,11 @@ static void rtl8169_get_drvinfo(struct net_device *dev, struct rtl8169_private *tp = netdev_priv(dev); struct rtl_fw *rtl_fw = tp->rtl_fw; - strlcpy(info->driver, KBUILD_MODNAME, sizeof(info->driver)); - strlcpy(info->bus_info, pci_name(tp->pci_dev), sizeof(info->bus_info)); + strscpy(info->driver, KBUILD_MODNAME, sizeof(info->driver)); + strscpy(info->bus_info, pci_name(tp->pci_dev), sizeof(info->bus_info)); BUILD_BUG_ON(sizeof(info->fw_version) < sizeof(rtl_fw->version)); if (rtl_fw) - strlcpy(info->fw_version, rtl_fw->version, + strscpy(info->fw_version, rtl_fw->version, sizeof(info->fw_version)); } @@ -2011,7 +1996,10 @@ static enum mac_version rtl8169_get_mac_version(u16 xid, bool gmii) /* 8168F family. */ { 0x7c8, 0x488, RTL_GIGA_MAC_VER_38 }, - { 0x7cf, 0x481, RTL_GIGA_MAC_VER_36 }, + /* It seems this chip version never made it to + * the wild. Let's disable detection. + * { 0x7cf, 0x481, RTL_GIGA_MAC_VER_36 }, + */ { 0x7cf, 0x480, RTL_GIGA_MAC_VER_35 }, /* 8168E family. */ @@ -2041,7 +2029,6 @@ static enum mac_version rtl8169_get_mac_version(u16 xid, bool gmii) { 0x7c8, 0x3c0, RTL_GIGA_MAC_VER_22 }, /* 8168B family. */ - { 0x7cf, 0x380, RTL_GIGA_MAC_VER_12 }, { 0x7c8, 0x380, RTL_GIGA_MAC_VER_17 }, { 0x7c8, 0x300, RTL_GIGA_MAC_VER_11 }, @@ -2054,19 +2041,10 @@ static enum mac_version rtl8169_get_mac_version(u16 xid, bool gmii) { 0x7cf, 0x249, RTL_GIGA_MAC_VER_08 }, { 0x7cf, 0x348, RTL_GIGA_MAC_VER_07 }, { 0x7cf, 0x248, RTL_GIGA_MAC_VER_07 }, - { 0x7cf, 0x340, RTL_GIGA_MAC_VER_13 }, { 0x7cf, 0x240, RTL_GIGA_MAC_VER_14 }, - { 0x7cf, 0x343, RTL_GIGA_MAC_VER_10 }, - { 0x7cf, 0x342, RTL_GIGA_MAC_VER_16 }, { 0x7c8, 0x348, RTL_GIGA_MAC_VER_09 }, { 0x7c8, 0x248, RTL_GIGA_MAC_VER_09 }, - { 0x7c8, 0x340, RTL_GIGA_MAC_VER_16 }, - /* FIXME: where did these entries come from ? -- FR - * Not even r8101 vendor driver knows these id's, - * so let's disable detection for now. -- HK - * { 0xfc8, 0x388, RTL_GIGA_MAC_VER_13 }, - * { 0xfc8, 0x308, RTL_GIGA_MAC_VER_13 }, - */ + { 0x7c8, 0x340, RTL_GIGA_MAC_VER_10 }, /* 8110 family. */ { 0xfc8, 0x980, RTL_GIGA_MAC_VER_06 }, @@ -2088,8 +2066,6 @@ static enum mac_version rtl8169_get_mac_version(u16 xid, bool gmii) if (ver != RTL_GIGA_MAC_NONE && !gmii) { if (ver == RTL_GIGA_MAC_VER_42) ver = RTL_GIGA_MAC_VER_43; - else if (ver == RTL_GIGA_MAC_VER_45) - ver = RTL_GIGA_MAC_VER_47; else if (ver == RTL_GIGA_MAC_VER_46) ver = RTL_GIGA_MAC_VER_48; } @@ -2271,7 +2247,7 @@ static void rtl_init_rxcfg(struct rtl8169_private *tp) case RTL_GIGA_MAC_VER_40 ... RTL_GIGA_MAC_VER_53: RTL_W32(tp, RxConfig, RX128_INT_EN | RX_MULTI_EN | RX_DMA_BURST | RX_EARLY_OFF); break; - case RTL_GIGA_MAC_VER_60 ... RTL_GIGA_MAC_VER_63: + case RTL_GIGA_MAC_VER_61 ... RTL_GIGA_MAC_VER_63: RTL_W32(tp, RxConfig, RX_FETCH_DFLT_8125 | RX_DMA_BURST); break; default: @@ -2338,7 +2314,6 @@ static void rtl_jumbo_config(struct rtl8169_private *tp) rtl_unlock_config_regs(tp); switch (tp->mac_version) { - case RTL_GIGA_MAC_VER_12: case RTL_GIGA_MAC_VER_17: if (jumbo) { readrq = 512; @@ -2455,7 +2430,7 @@ static void rtl_wait_txrx_fifo_empty(struct rtl8169_private *tp) rtl_loop_wait_high(tp, &rtl_txcfg_empty_cond, 100, 42); rtl_loop_wait_high(tp, &rtl_rxtx_empty_cond, 100, 42); break; - case RTL_GIGA_MAC_VER_60 ... RTL_GIGA_MAC_VER_61: + case RTL_GIGA_MAC_VER_61 ... RTL_GIGA_MAC_VER_61: rtl_loop_wait_high(tp, &rtl_rxtx_empty_cond, 100, 42); break; case RTL_GIGA_MAC_VER_63: @@ -2468,6 +2443,11 @@ static void rtl_wait_txrx_fifo_empty(struct rtl8169_private *tp) } } +static void rtl_disable_rxdvgate(struct rtl8169_private *tp) +{ + RTL_W32(tp, MISC, RTL_R32(tp, MISC) & ~RXDV_GATED_EN); +} + static void rtl_enable_rxdvgate(struct rtl8169_private *tp) { RTL_W32(tp, MISC, RTL_R32(tp, MISC) | RXDV_GATED_EN); @@ -2700,8 +2680,8 @@ static void rtl_hw_aspm_clkreq_enable(struct rtl8169_private *tp, bool enable) RTL_W8(tp, Config2, RTL_R8(tp, Config2) | ClkReqEn); switch (tp->mac_version) { - case RTL_GIGA_MAC_VER_45 ... RTL_GIGA_MAC_VER_48: - case RTL_GIGA_MAC_VER_60 ... RTL_GIGA_MAC_VER_63: + case RTL_GIGA_MAC_VER_46 ... RTL_GIGA_MAC_VER_48: + case RTL_GIGA_MAC_VER_61 ... RTL_GIGA_MAC_VER_63: /* reset ephy tx/rx disable timer */ r8168_mac_ocp_modify(tp, 0xe094, 0xff00, 0); /* chip can trigger L1.2 */ @@ -2712,8 +2692,8 @@ static void rtl_hw_aspm_clkreq_enable(struct rtl8169_private *tp, bool enable) } } else { switch (tp->mac_version) { - case RTL_GIGA_MAC_VER_45 ... RTL_GIGA_MAC_VER_48: - case RTL_GIGA_MAC_VER_60 ... RTL_GIGA_MAC_VER_63: + case RTL_GIGA_MAC_VER_46 ... RTL_GIGA_MAC_VER_48: + case RTL_GIGA_MAC_VER_61 ... RTL_GIGA_MAC_VER_63: r8168_mac_ocp_modify(tp, 0xe092, 0x00ff, 0); break; default: @@ -2985,7 +2965,7 @@ static void rtl_hw_start_8168g(struct rtl8169_private *tp) rtl_reset_packet_filter(tp); rtl_eri_write(tp, 0x2f8, ERIAR_MASK_0011, 0x1d8f); - RTL_W32(tp, MISC, RTL_R32(tp, MISC) & ~RXDV_GATED_EN); + rtl_disable_rxdvgate(tp); rtl_eri_write(tp, 0xc0, ERIAR_MASK_0011, 0x0000); rtl_eri_write(tp, 0xb8, ERIAR_MASK_0011, 0x0000); @@ -3223,7 +3203,7 @@ static void rtl_hw_start_8168h_1(struct rtl8169_private *tp) rtl_eri_write(tp, 0x5f0, ERIAR_MASK_0011, 0x4f87); - RTL_W32(tp, MISC, RTL_R32(tp, MISC) & ~RXDV_GATED_EN); + rtl_disable_rxdvgate(tp); rtl_eri_write(tp, 0xc0, ERIAR_MASK_0011, 0x0000); rtl_eri_write(tp, 0xb8, ERIAR_MASK_0011, 0x0000); @@ -3274,7 +3254,7 @@ static void rtl_hw_start_8168ep(struct rtl8169_private *tp) rtl_eri_write(tp, 0x5f0, ERIAR_MASK_0011, 0x4f87); - RTL_W32(tp, MISC, RTL_R32(tp, MISC) & ~RXDV_GATED_EN); + rtl_disable_rxdvgate(tp); rtl_eri_write(tp, 0xc0, ERIAR_MASK_0011, 0x0000); rtl_eri_write(tp, 0xb8, ERIAR_MASK_0011, 0x0000); @@ -3288,45 +3268,6 @@ static void rtl_hw_start_8168ep(struct rtl8169_private *tp) rtl_pcie_state_l2l3_disable(tp); } -static void rtl_hw_start_8168ep_1(struct rtl8169_private *tp) -{ - static const struct ephy_info e_info_8168ep_1[] = { - { 0x00, 0xffff, 0x10ab }, - { 0x06, 0xffff, 0xf030 }, - { 0x08, 0xffff, 0x2006 }, - { 0x0d, 0xffff, 0x1666 }, - { 0x0c, 0x3ff0, 0x0000 } - }; - - /* disable aspm and clock request before access ephy */ - rtl_hw_aspm_clkreq_enable(tp, false); - rtl_ephy_init(tp, e_info_8168ep_1); - - rtl_hw_start_8168ep(tp); - - rtl_hw_aspm_clkreq_enable(tp, true); -} - -static void rtl_hw_start_8168ep_2(struct rtl8169_private *tp) -{ - static const struct ephy_info e_info_8168ep_2[] = { - { 0x00, 0xffff, 0x10a3 }, - { 0x19, 0xffff, 0xfc00 }, - { 0x1e, 0xffff, 0x20ea } - }; - - /* disable aspm and clock request before access ephy */ - rtl_hw_aspm_clkreq_enable(tp, false); - rtl_ephy_init(tp, e_info_8168ep_2); - - rtl_hw_start_8168ep(tp); - - RTL_W8(tp, DLLPR, RTL_R8(tp, DLLPR) & ~PFM_EN); - RTL_W8(tp, MISC_1, RTL_R8(tp, MISC_1) & ~PFM_D3COLD_EN); - - rtl_hw_aspm_clkreq_enable(tp, true); -} - static void rtl_hw_start_8168ep_3(struct rtl8169_private *tp) { static const struct ephy_info e_info_8168ep_3[] = { @@ -3377,7 +3318,7 @@ static void rtl_hw_start_8117(struct rtl8169_private *tp) rtl_eri_write(tp, 0x5f0, ERIAR_MASK_0011, 0x4f87); - RTL_W32(tp, MISC, RTL_R32(tp, MISC) & ~RXDV_GATED_EN); + rtl_disable_rxdvgate(tp); rtl_eri_write(tp, 0xc0, ERIAR_MASK_0011, 0x0000); rtl_eri_write(tp, 0xb8, ERIAR_MASK_0011, 0x0000); @@ -3621,48 +3562,7 @@ static void rtl_hw_start_8125_common(struct rtl8169_private *tp) else rtl8125a_config_eee_mac(tp); - RTL_W32(tp, MISC, RTL_R32(tp, MISC) & ~RXDV_GATED_EN); - udelay(10); -} - -static void rtl_hw_start_8125a_1(struct rtl8169_private *tp) -{ - static const struct ephy_info e_info_8125a_1[] = { - { 0x01, 0xffff, 0xa812 }, - { 0x09, 0xffff, 0x520c }, - { 0x04, 0xffff, 0xd000 }, - { 0x0d, 0xffff, 0xf702 }, - { 0x0a, 0xffff, 0x8653 }, - { 0x06, 0xffff, 0x001e }, - { 0x08, 0xffff, 0x3595 }, - { 0x20, 0xffff, 0x9455 }, - { 0x21, 0xffff, 0x99ff }, - { 0x02, 0xffff, 0x6046 }, - { 0x29, 0xffff, 0xfe00 }, - { 0x23, 0xffff, 0xab62 }, - - { 0x41, 0xffff, 0xa80c }, - { 0x49, 0xffff, 0x520c }, - { 0x44, 0xffff, 0xd000 }, - { 0x4d, 0xffff, 0xf702 }, - { 0x4a, 0xffff, 0x8653 }, - { 0x46, 0xffff, 0x001e }, - { 0x48, 0xffff, 0x3595 }, - { 0x60, 0xffff, 0x9455 }, - { 0x61, 0xffff, 0x99ff }, - { 0x42, 0xffff, 0x6046 }, - { 0x69, 0xffff, 0xfe00 }, - { 0x63, 0xffff, 0xab62 }, - }; - - rtl_set_def_aspm_entry_latency(tp); - - /* disable aspm and clock request before access ephy */ - rtl_hw_aspm_clkreq_enable(tp, false); - rtl_ephy_init(tp, e_info_8125a_1); - - rtl_hw_start_8125_common(tp); - rtl_hw_aspm_clkreq_enable(tp, true); + rtl_disable_rxdvgate(tp); } static void rtl_hw_start_8125a_2(struct rtl8169_private *tp) @@ -3721,10 +3621,7 @@ static void rtl_hw_config(struct rtl8169_private *tp) [RTL_GIGA_MAC_VER_09] = rtl_hw_start_8102e_2, [RTL_GIGA_MAC_VER_10] = NULL, [RTL_GIGA_MAC_VER_11] = rtl_hw_start_8168b, - [RTL_GIGA_MAC_VER_12] = rtl_hw_start_8168b, - [RTL_GIGA_MAC_VER_13] = NULL, [RTL_GIGA_MAC_VER_14] = rtl_hw_start_8401, - [RTL_GIGA_MAC_VER_16] = NULL, [RTL_GIGA_MAC_VER_17] = rtl_hw_start_8168b, [RTL_GIGA_MAC_VER_18] = rtl_hw_start_8168cp_1, [RTL_GIGA_MAC_VER_19] = rtl_hw_start_8168c_1, @@ -3748,20 +3645,14 @@ static void rtl_hw_config(struct rtl8169_private *tp) [RTL_GIGA_MAC_VER_38] = rtl_hw_start_8411, [RTL_GIGA_MAC_VER_39] = rtl_hw_start_8106, [RTL_GIGA_MAC_VER_40] = rtl_hw_start_8168g_1, - [RTL_GIGA_MAC_VER_41] = rtl_hw_start_8168g_1, [RTL_GIGA_MAC_VER_42] = rtl_hw_start_8168g_2, [RTL_GIGA_MAC_VER_43] = rtl_hw_start_8168g_2, [RTL_GIGA_MAC_VER_44] = rtl_hw_start_8411_2, - [RTL_GIGA_MAC_VER_45] = rtl_hw_start_8168h_1, [RTL_GIGA_MAC_VER_46] = rtl_hw_start_8168h_1, - [RTL_GIGA_MAC_VER_47] = rtl_hw_start_8168h_1, [RTL_GIGA_MAC_VER_48] = rtl_hw_start_8168h_1, - [RTL_GIGA_MAC_VER_49] = rtl_hw_start_8168ep_1, - [RTL_GIGA_MAC_VER_50] = rtl_hw_start_8168ep_2, [RTL_GIGA_MAC_VER_51] = rtl_hw_start_8168ep_3, [RTL_GIGA_MAC_VER_52] = rtl_hw_start_8117, [RTL_GIGA_MAC_VER_53] = rtl_hw_start_8117, - [RTL_GIGA_MAC_VER_60] = rtl_hw_start_8125a_1, [RTL_GIGA_MAC_VER_61] = rtl_hw_start_8125a_2, [RTL_GIGA_MAC_VER_63] = rtl_hw_start_8125b, }; @@ -4156,7 +4047,6 @@ static unsigned int rtl_quirk_packet_padto(struct rtl8169_private *tp, switch (tp->mac_version) { case RTL_GIGA_MAC_VER_34: - case RTL_GIGA_MAC_VER_60: case RTL_GIGA_MAC_VER_61: case RTL_GIGA_MAC_VER_63: padto = max_t(unsigned int, padto, ETH_ZLEN); @@ -4677,8 +4567,7 @@ static void r8169_phylink_handler(struct net_device *ndev) pm_runtime_idle(&tp->pci_dev->dev); } - if (net_ratelimit()) - phy_print_status(tp->phydev); + phy_print_status(tp->phydev); } static int r8169_phy_connect(struct rtl8169_private *tp) @@ -4954,23 +4843,6 @@ static const struct dev_pm_ops rtl8169_pm_ops = { rtl8169_runtime_idle) }; -static void rtl_wol_shutdown_quirk(struct rtl8169_private *tp) -{ - /* WoL fails with 8168b when the receiver is disabled. */ - switch (tp->mac_version) { - case RTL_GIGA_MAC_VER_11: - case RTL_GIGA_MAC_VER_12: - case RTL_GIGA_MAC_VER_17: - pci_clear_master(tp->pci_dev); - - RTL_W8(tp, ChipCmd, CmdRxEnb); - rtl_pci_commit(tp); - break; - default: - break; - } -} - static void rtl_shutdown(struct pci_dev *pdev) { struct rtl8169_private *tp = pci_get_drvdata(pdev); @@ -4984,9 +4856,6 @@ static void rtl_shutdown(struct pci_dev *pdev) if (system_state == SYSTEM_POWER_OFF && tp->dash_type == RTL_DASH_NONE) { - if (tp->saved_wolopts) - rtl_wol_shutdown_quirk(tp); - pci_wake_from_d3(pdev, tp->saved_wolopts); pci_set_power_state(pdev, PCI_D3hot); } @@ -5194,13 +5063,13 @@ static void rtl_hw_init_8125(struct rtl8169_private *tp) static void rtl_hw_initialize(struct rtl8169_private *tp) { switch (tp->mac_version) { - case RTL_GIGA_MAC_VER_49 ... RTL_GIGA_MAC_VER_53: + case RTL_GIGA_MAC_VER_51 ... RTL_GIGA_MAC_VER_53: rtl8168ep_stop_cmac(tp); fallthrough; case RTL_GIGA_MAC_VER_40 ... RTL_GIGA_MAC_VER_48: rtl_hw_init_8168g(tp); break; - case RTL_GIGA_MAC_VER_60 ... RTL_GIGA_MAC_VER_63: + case RTL_GIGA_MAC_VER_61 ... RTL_GIGA_MAC_VER_63: rtl_hw_init_8125(tp); break; default: @@ -5220,7 +5089,6 @@ static int rtl_jumbo_max(struct rtl8169_private *tp) return JUMBO_7K; /* RTL8168b */ case RTL_GIGA_MAC_VER_11: - case RTL_GIGA_MAC_VER_12: case RTL_GIGA_MAC_VER_17: return JUMBO_4K; /* RTL8168c */ @@ -5231,37 +5099,6 @@ static int rtl_jumbo_max(struct rtl8169_private *tp) } } -static void rtl_disable_clk(void *data) -{ - clk_disable_unprepare(data); -} - -static int rtl_get_ether_clk(struct rtl8169_private *tp) -{ - struct device *d = tp_to_dev(tp); - struct clk *clk; - int rc; - - clk = devm_clk_get(d, "ether_clk"); - if (IS_ERR(clk)) { - rc = PTR_ERR(clk); - if (rc == -ENOENT) - /* clk-core allows NULL (for suspend / resume) */ - rc = 0; - else - dev_err_probe(d, rc, "failed to get clk\n"); - } else { - tp->clk = clk; - rc = clk_prepare_enable(clk); - if (rc) - dev_err(d, "failed to enable clk: %d\n", rc); - else - rc = devm_add_action_or_reset(d, rtl_disable_clk, clk); - } - - return rc; -} - static void rtl_init_mac_address(struct rtl8169_private *tp) { u8 mac_addr[ETH_ALEN] __aligned(2) = {}; @@ -5291,7 +5128,7 @@ done: /* register is set if system vendor successfully tested ASPM 1.2 */ static bool rtl_aspm_is_safe(struct rtl8169_private *tp) { - if (tp->mac_version >= RTL_GIGA_MAC_VER_60 && + if (tp->mac_version >= RTL_GIGA_MAC_VER_61 && r8168_mac_ocp_read(tp, 0xc0b2) & 0xf) return true; @@ -5325,9 +5162,9 @@ static int rtl_init_one(struct pci_dev *pdev, const struct pci_device_id *ent) return -ENOMEM; /* Get the *optional* external "ether_clk" used on some boards */ - rc = rtl_get_ether_clk(tp); - if (rc) - return rc; + tp->clk = devm_clk_get_optional_enabled(&pdev->dev, "ether_clk"); + if (IS_ERR(tp->clk)) + return dev_err_probe(&pdev->dev, PTR_ERR(tp->clk), "failed to get ether_clk\n"); /* enable device (incl. PCI PM wakeup and hotplug setup) */ rc = pcim_enable_device(pdev); @@ -5346,12 +5183,6 @@ static int rtl_init_one(struct pci_dev *pdev, const struct pci_device_id *ent) return -ENODEV; } - /* check for weird/broken PCI region reporting */ - if (pci_resource_len(pdev, region) < R8169_REGS_SIZE) { - dev_err(&pdev->dev, "Invalid PCI region size(s), aborting\n"); - return -ENODEV; - } - rc = pcim_iomap_regions(pdev, BIT(region), KBUILD_MODNAME); if (rc < 0) { dev_err(&pdev->dev, "cannot remap MMIO, aborting\n"); @@ -5378,7 +5209,7 @@ static int rtl_init_one(struct pci_dev *pdev, const struct pci_device_id *ent) */ if (rtl_aspm_is_safe(tp)) rc = 0; - else if (tp->mac_version >= RTL_GIGA_MAC_VER_45) + else if (tp->mac_version >= RTL_GIGA_MAC_VER_46) rc = pci_disable_link_state(pdev, PCIE_LINK_STATE_L1_2); else rc = pci_disable_link_state(pdev, PCIE_LINK_STATE_L1); @@ -5413,7 +5244,7 @@ static int rtl_init_one(struct pci_dev *pdev, const struct pci_device_id *ent) dev->ethtool_ops = &rtl8169_ethtool_ops; - netif_napi_add(dev, &tp->napi, rtl8169_poll, NAPI_POLL_WEIGHT); + netif_napi_add(dev, &tp->napi, rtl8169_poll); dev->hw_features = NETIF_F_IP_CSUM | NETIF_F_RXCSUM | NETIF_F_HW_VLAN_CTAG_TX | NETIF_F_HW_VLAN_CTAG_RX; diff --git a/drivers/net/ethernet/realtek/r8169_phy_config.c b/drivers/net/ethernet/realtek/r8169_phy_config.c index 15c295f90196..930496cd34ed 100644 --- a/drivers/net/ethernet/realtek/r8169_phy_config.c +++ b/drivers/net/ethernet/realtek/r8169_phy_config.c @@ -793,71 +793,6 @@ static void rtl8168g_2_hw_phy_config(struct rtl8169_private *tp, rtl8168g_config_eee_phy(phydev); } -static void rtl8168h_1_hw_phy_config(struct rtl8169_private *tp, - struct phy_device *phydev) -{ - u16 dout_tapbin; - u32 data; - - r8169_apply_firmware(tp); - - /* CHN EST parameters adjust - giga master */ - r8168g_phy_param(phydev, 0x809b, 0xf800, 0x8000); - r8168g_phy_param(phydev, 0x80a2, 0xff00, 0x8000); - r8168g_phy_param(phydev, 0x80a4, 0xff00, 0x8500); - r8168g_phy_param(phydev, 0x809c, 0xff00, 0xbd00); - - /* CHN EST parameters adjust - giga slave */ - r8168g_phy_param(phydev, 0x80ad, 0xf800, 0x7000); - r8168g_phy_param(phydev, 0x80b4, 0xff00, 0x5000); - r8168g_phy_param(phydev, 0x80ac, 0xff00, 0x4000); - - /* CHN EST parameters adjust - fnet */ - r8168g_phy_param(phydev, 0x808e, 0xff00, 0x1200); - r8168g_phy_param(phydev, 0x8090, 0xff00, 0xe500); - r8168g_phy_param(phydev, 0x8092, 0xff00, 0x9f00); - - /* enable R-tune & PGA-retune function */ - dout_tapbin = 0; - data = phy_read_paged(phydev, 0x0a46, 0x13); - data &= 3; - data <<= 2; - dout_tapbin |= data; - data = phy_read_paged(phydev, 0x0a46, 0x12); - data &= 0xc000; - data >>= 14; - dout_tapbin |= data; - dout_tapbin = ~(dout_tapbin ^ 0x08); - dout_tapbin <<= 12; - dout_tapbin &= 0xf000; - - r8168g_phy_param(phydev, 0x827a, 0xf000, dout_tapbin); - r8168g_phy_param(phydev, 0x827b, 0xf000, dout_tapbin); - r8168g_phy_param(phydev, 0x827c, 0xf000, dout_tapbin); - r8168g_phy_param(phydev, 0x827d, 0xf000, dout_tapbin); - r8168g_phy_param(phydev, 0x0811, 0x0000, 0x0800); - phy_modify_paged(phydev, 0x0a42, 0x16, 0x0000, 0x0002); - - rtl8168g_enable_gphy_10m(phydev); - - /* SAR ADC performance */ - phy_modify_paged(phydev, 0x0bca, 0x17, BIT(12) | BIT(13), BIT(14)); - - r8168g_phy_param(phydev, 0x803f, 0x3000, 0x0000); - r8168g_phy_param(phydev, 0x8047, 0x3000, 0x0000); - r8168g_phy_param(phydev, 0x804f, 0x3000, 0x0000); - r8168g_phy_param(phydev, 0x8057, 0x3000, 0x0000); - r8168g_phy_param(phydev, 0x805f, 0x3000, 0x0000); - r8168g_phy_param(phydev, 0x8067, 0x3000, 0x0000); - r8168g_phy_param(phydev, 0x806f, 0x3000, 0x0000); - - /* disable phy pfm mode */ - phy_modify_paged(phydev, 0x0a44, 0x11, BIT(7), 0); - - rtl8168g_disable_aldps(phydev); - rtl8168h_config_eee_phy(phydev); -} - static void rtl8168h_2_hw_phy_config(struct rtl8169_private *tp, struct phy_device *phydev) { @@ -895,27 +830,6 @@ static void rtl8168h_2_hw_phy_config(struct rtl8169_private *tp, rtl8168g_config_eee_phy(phydev); } -static void rtl8168ep_1_hw_phy_config(struct rtl8169_private *tp, - struct phy_device *phydev) -{ - /* Enable PHY auto speed down */ - phy_modify_paged(phydev, 0x0a44, 0x11, 0, BIT(3) | BIT(2)); - - rtl8168g_phy_adjust_10m_aldps(phydev); - - /* Enable EEE auto-fallback function */ - phy_modify_paged(phydev, 0x0a4b, 0x11, 0, BIT(2)); - - /* Enable UC LPF tune function */ - r8168g_phy_param(phydev, 0x8012, 0x0000, 0x8000); - - /* set rg_sel_sdm_rate */ - phy_modify_paged(phydev, 0x0c42, 0x11, BIT(13), BIT(14)); - - rtl8168g_disable_aldps(phydev); - rtl8168g_config_eee_phy(phydev); -} - static void rtl8168ep_2_hw_phy_config(struct rtl8169_private *tp, struct phy_device *phydev) { @@ -1081,44 +995,6 @@ static void rtl8125_legacy_force_mode(struct phy_device *phydev) phy_modify_paged(phydev, 0xa5b, 0x12, BIT(15), 0); } -static void rtl8125a_1_hw_phy_config(struct rtl8169_private *tp, - struct phy_device *phydev) -{ - phy_modify_paged(phydev, 0xad4, 0x10, 0x03ff, 0x0084); - phy_modify_paged(phydev, 0xad4, 0x17, 0x0000, 0x0010); - phy_modify_paged(phydev, 0xad1, 0x13, 0x03ff, 0x0006); - phy_modify_paged(phydev, 0xad3, 0x11, 0x003f, 0x0006); - phy_modify_paged(phydev, 0xac0, 0x14, 0x0000, 0x1100); - phy_modify_paged(phydev, 0xac8, 0x15, 0xf000, 0x7000); - phy_modify_paged(phydev, 0xad1, 0x14, 0x0000, 0x0400); - phy_modify_paged(phydev, 0xad1, 0x15, 0x0000, 0x03ff); - phy_modify_paged(phydev, 0xad1, 0x16, 0x0000, 0x03ff); - - r8168g_phy_param(phydev, 0x80ea, 0xff00, 0xc400); - r8168g_phy_param(phydev, 0x80eb, 0x0700, 0x0300); - r8168g_phy_param(phydev, 0x80f8, 0xff00, 0x1c00); - r8168g_phy_param(phydev, 0x80f1, 0xff00, 0x3000); - r8168g_phy_param(phydev, 0x80fe, 0xff00, 0xa500); - r8168g_phy_param(phydev, 0x8102, 0xff00, 0x5000); - r8168g_phy_param(phydev, 0x8105, 0xff00, 0x3300); - r8168g_phy_param(phydev, 0x8100, 0xff00, 0x7000); - r8168g_phy_param(phydev, 0x8104, 0xff00, 0xf000); - r8168g_phy_param(phydev, 0x8106, 0xff00, 0x6500); - r8168g_phy_param(phydev, 0x80dc, 0xff00, 0xed00); - r8168g_phy_param(phydev, 0x80df, 0x0000, 0x0100); - r8168g_phy_param(phydev, 0x80e1, 0x0100, 0x0000); - - phy_modify_paged(phydev, 0xbf0, 0x13, 0x003f, 0x0038); - r8168g_phy_param(phydev, 0x819f, 0xffff, 0xd0b6); - - phy_write_paged(phydev, 0xbc3, 0x12, 0x5555); - phy_modify_paged(phydev, 0xbf0, 0x15, 0x0e00, 0x0a00); - phy_modify_paged(phydev, 0xa5c, 0x10, 0x0400, 0x0000); - rtl8168g_enable_gphy_10m(phydev); - - rtl8125a_config_eee_phy(phydev); -} - static void rtl8125a_2_hw_phy_config(struct rtl8169_private *tp, struct phy_device *phydev) { @@ -1239,10 +1115,7 @@ void r8169_hw_phy_config(struct rtl8169_private *tp, struct phy_device *phydev, [RTL_GIGA_MAC_VER_09] = rtl8102e_hw_phy_config, [RTL_GIGA_MAC_VER_10] = NULL, [RTL_GIGA_MAC_VER_11] = rtl8168bb_hw_phy_config, - [RTL_GIGA_MAC_VER_12] = rtl8168bef_hw_phy_config, - [RTL_GIGA_MAC_VER_13] = NULL, [RTL_GIGA_MAC_VER_14] = rtl8401_hw_phy_config, - [RTL_GIGA_MAC_VER_16] = NULL, [RTL_GIGA_MAC_VER_17] = rtl8168bef_hw_phy_config, [RTL_GIGA_MAC_VER_18] = rtl8168cp_1_hw_phy_config, [RTL_GIGA_MAC_VER_19] = rtl8168c_1_hw_phy_config, @@ -1266,20 +1139,14 @@ void r8169_hw_phy_config(struct rtl8169_private *tp, struct phy_device *phydev, [RTL_GIGA_MAC_VER_38] = rtl8411_hw_phy_config, [RTL_GIGA_MAC_VER_39] = rtl8106e_hw_phy_config, [RTL_GIGA_MAC_VER_40] = rtl8168g_1_hw_phy_config, - [RTL_GIGA_MAC_VER_41] = NULL, [RTL_GIGA_MAC_VER_42] = rtl8168g_2_hw_phy_config, [RTL_GIGA_MAC_VER_43] = rtl8168g_2_hw_phy_config, [RTL_GIGA_MAC_VER_44] = rtl8168g_2_hw_phy_config, - [RTL_GIGA_MAC_VER_45] = rtl8168h_1_hw_phy_config, [RTL_GIGA_MAC_VER_46] = rtl8168h_2_hw_phy_config, - [RTL_GIGA_MAC_VER_47] = rtl8168h_1_hw_phy_config, [RTL_GIGA_MAC_VER_48] = rtl8168h_2_hw_phy_config, - [RTL_GIGA_MAC_VER_49] = rtl8168ep_1_hw_phy_config, - [RTL_GIGA_MAC_VER_50] = rtl8168ep_2_hw_phy_config, [RTL_GIGA_MAC_VER_51] = rtl8168ep_2_hw_phy_config, [RTL_GIGA_MAC_VER_52] = rtl8117_hw_phy_config, [RTL_GIGA_MAC_VER_53] = rtl8117_hw_phy_config, - [RTL_GIGA_MAC_VER_60] = rtl8125a_1_hw_phy_config, [RTL_GIGA_MAC_VER_61] = rtl8125a_2_hw_phy_config, [RTL_GIGA_MAC_VER_63] = rtl8125b_hw_phy_config, }; diff --git a/drivers/net/ethernet/renesas/ravb.h b/drivers/net/ethernet/renesas/ravb.h index b980bce763d3..e0f8276cffed 100644 --- a/drivers/net/ethernet/renesas/ravb.h +++ b/drivers/net/ethernet/renesas/ravb.h @@ -189,6 +189,7 @@ enum ravb_reg { PSR = 0x0528, PIPR = 0x052c, CXR31 = 0x0530, /* RZ/G2L only */ + CXR35 = 0x0540, /* RZ/G2L only */ MPR = 0x0558, PFTCR = 0x055c, PFRCR = 0x0560, @@ -965,6 +966,13 @@ enum CXR31_BIT { CXR31_SEL_LINK1 = 0x00000008, }; +enum CXR35_BIT { + CXR35_SEL_XMII = 0x00000003, + CXR35_SEL_XMII_RGMII = 0x00000000, + CXR35_SEL_XMII_MII = 0x00000002, + CXR35_HALFCYC_CLKSW = 0xffff0000, +}; + enum CSR0_BIT { CSR0_TPE = 0x00000010, CSR0_RPE = 0x00000020, diff --git a/drivers/net/ethernet/renesas/ravb_main.c b/drivers/net/ethernet/renesas/ravb_main.c index 7e32b04eb0c7..36324126db6d 100644 --- a/drivers/net/ethernet/renesas/ravb_main.c +++ b/drivers/net/ethernet/renesas/ravb_main.c @@ -540,7 +540,13 @@ static void ravb_emac_init_gbeth(struct net_device *ndev) /* E-MAC interrupt enable register */ ravb_write(ndev, ECSIPR_ICDIP, ECSIPR); - ravb_modify(ndev, CXR31, CXR31_SEL_LINK0 | CXR31_SEL_LINK1, CXR31_SEL_LINK0); + if (priv->phy_interface == PHY_INTERFACE_MODE_MII) { + ravb_modify(ndev, CXR31, CXR31_SEL_LINK0 | CXR31_SEL_LINK1, 0); + ravb_write(ndev, (1000 << 16) | CXR35_SEL_XMII_MII, CXR35); + } else { + ravb_modify(ndev, CXR31, CXR31_SEL_LINK0 | CXR31_SEL_LINK1, + CXR31_SEL_LINK0); + } } static void ravb_emac_init_rcar(struct net_device *ndev) @@ -2514,6 +2520,7 @@ static const struct of_device_id ravb_match_table[] = { { .compatible = "renesas,etheravb-rcar-gen2", .data = &ravb_gen2_hw_info }, { .compatible = "renesas,etheravb-r8a7795", .data = &ravb_gen3_hw_info }, { .compatible = "renesas,etheravb-rcar-gen3", .data = &ravb_gen3_hw_info }, + { .compatible = "renesas,etheravb-rcar-gen4", .data = &ravb_gen3_hw_info }, { .compatible = "renesas,etheravb-rzv2m", .data = &ravb_rzv2m_hw_info }, { .compatible = "renesas,rzg2l-gbeth", .data = &gbeth_hw_info }, { } @@ -2834,9 +2841,9 @@ static int ravb_probe(struct platform_device *pdev) goto out_dma_free; } - netif_napi_add(ndev, &priv->napi[RAVB_BE], ravb_poll, 64); + netif_napi_add(ndev, &priv->napi[RAVB_BE], ravb_poll); if (info->nc_queues) - netif_napi_add(ndev, &priv->napi[RAVB_NC], ravb_poll, 64); + netif_napi_add(ndev, &priv->napi[RAVB_NC], ravb_poll); /* Network device register */ error = register_netdev(ndev); diff --git a/drivers/net/ethernet/renesas/sh_eth.c b/drivers/net/ethernet/renesas/sh_eth.c index 7fd8828d3a84..71a499113308 100644 --- a/drivers/net/ethernet/renesas/sh_eth.c +++ b/drivers/net/ethernet/renesas/sh_eth.c @@ -3368,7 +3368,7 @@ static int sh_eth_drv_probe(struct platform_device *pdev) goto out_release; } - netif_napi_add(ndev, &mdp->napi, sh_eth_poll, 64); + netif_napi_add(ndev, &mdp->napi, sh_eth_poll); /* network device register */ ret = register_netdev(ndev); diff --git a/drivers/net/ethernet/rocker/rocker_main.c b/drivers/net/ethernet/rocker/rocker_main.c index fc83ec23bd1d..023682cd2768 100644 --- a/drivers/net/ethernet/rocker/rocker_main.c +++ b/drivers/net/ethernet/rocker/rocker_main.c @@ -2226,8 +2226,8 @@ rocker_port_set_link_ksettings(struct net_device *dev, static void rocker_port_get_drvinfo(struct net_device *dev, struct ethtool_drvinfo *drvinfo) { - strlcpy(drvinfo->driver, rocker_driver_name, sizeof(drvinfo->driver)); - strlcpy(drvinfo->version, UTS_RELEASE, sizeof(drvinfo->version)); + strscpy(drvinfo->driver, rocker_driver_name, sizeof(drvinfo->driver)); + strscpy(drvinfo->version, UTS_RELEASE, sizeof(drvinfo->version)); } static struct rocker_port_stats { @@ -2574,8 +2574,7 @@ static int rocker_probe_port(struct rocker *rocker, unsigned int port_number) dev->netdev_ops = &rocker_port_netdev_ops; dev->ethtool_ops = &rocker_port_ethtool_ops; netif_napi_add_tx(dev, &rocker_port->napi_tx, rocker_port_poll_tx); - netif_napi_add(dev, &rocker_port->napi_rx, rocker_port_poll_rx, - NAPI_POLL_WEIGHT); + netif_napi_add(dev, &rocker_port->napi_rx, rocker_port_poll_rx); rocker_carrier_init(rocker_port); dev->features |= NETIF_F_NETNS_LOCAL | NETIF_F_SG; diff --git a/drivers/net/ethernet/samsung/sxgbe/sxgbe_ethtool.c b/drivers/net/ethernet/samsung/sxgbe/sxgbe_ethtool.c index 98edb01024f0..8ba017ec9849 100644 --- a/drivers/net/ethernet/samsung/sxgbe/sxgbe_ethtool.c +++ b/drivers/net/ethernet/samsung/sxgbe/sxgbe_ethtool.c @@ -175,8 +175,8 @@ static int sxgbe_set_eee(struct net_device *dev, static void sxgbe_getdrvinfo(struct net_device *dev, struct ethtool_drvinfo *info) { - strlcpy(info->driver, KBUILD_MODNAME, sizeof(info->driver)); - strlcpy(info->version, DRV_VERSION, sizeof(info->version)); + strscpy(info->driver, KBUILD_MODNAME, sizeof(info->driver)); + strscpy(info->version, DRV_VERSION, sizeof(info->version)); } static u32 sxgbe_getmsglevel(struct net_device *dev) diff --git a/drivers/net/ethernet/samsung/sxgbe/sxgbe_main.c b/drivers/net/ethernet/samsung/sxgbe/sxgbe_main.c index a1c10b61269b..9664f029fa16 100644 --- a/drivers/net/ethernet/samsung/sxgbe/sxgbe_main.c +++ b/drivers/net/ethernet/samsung/sxgbe/sxgbe_main.c @@ -2143,7 +2143,7 @@ struct sxgbe_priv_data *sxgbe_drv_probe(struct device *device, pr_info("Enable RX Mitigation via HW Watchdog Timer\n"); } - netif_napi_add(ndev, &priv->napi, sxgbe_poll, 64); + netif_napi_add(ndev, &priv->napi, sxgbe_poll); spin_lock_init(&priv->stats_lock); diff --git a/drivers/net/ethernet/sfc/Makefile b/drivers/net/ethernet/sfc/Makefile index bb06fa228367..b5e45fc6337e 100644 --- a/drivers/net/ethernet/sfc/Makefile +++ b/drivers/net/ethernet/sfc/Makefile @@ -9,7 +9,7 @@ sfc-y += efx.o efx_common.o efx_channels.o nic.o \ ef100_ethtool.o ef100_rx.o ef100_tx.o sfc-$(CONFIG_SFC_MTD) += mtd.o sfc-$(CONFIG_SFC_SRIOV) += sriov.o ef10_sriov.o ef100_sriov.o ef100_rep.o \ - mae.o tc.o + mae.o tc.o tc_bindings.o obj-$(CONFIG_SFC) += sfc.o diff --git a/drivers/net/ethernet/sfc/ef100_ethtool.c b/drivers/net/ethernet/sfc/ef100_ethtool.c index 702abbe59b76..135ece2f1375 100644 --- a/drivers/net/ethernet/sfc/ef100_ethtool.c +++ b/drivers/net/ethernet/sfc/ef100_ethtool.c @@ -43,6 +43,8 @@ const struct ethtool_ops ef100_ethtool_ops = { .get_pauseparam = efx_ethtool_get_pauseparam, .set_pauseparam = efx_ethtool_set_pauseparam, .get_sset_count = efx_ethtool_get_sset_count, + .get_priv_flags = efx_ethtool_get_priv_flags, + .set_priv_flags = efx_ethtool_set_priv_flags, .self_test = efx_ethtool_self_test, .get_strings = efx_ethtool_get_strings, .get_link_ksettings = efx_ethtool_get_link_ksettings, diff --git a/drivers/net/ethernet/sfc/ef100_netdev.c b/drivers/net/ethernet/sfc/ef100_netdev.c index 17b9d37218cb..88fa29572e23 100644 --- a/drivers/net/ethernet/sfc/ef100_netdev.c +++ b/drivers/net/ethernet/sfc/ef100_netdev.c @@ -23,6 +23,7 @@ #include "mcdi_filters.h" #include "rx_common.h" #include "ef100_sriov.h" +#include "tc_bindings.h" static void ef100_update_name(struct efx_nic *efx) { @@ -246,6 +247,9 @@ static const struct net_device_ops ef100_netdev_ops = { #ifdef CONFIG_RFS_ACCEL .ndo_rx_flow_steer = efx_filter_rfs, #endif +#ifdef CONFIG_SFC_SRIOV + .ndo_setup_tc = efx_tc_setup, +#endif }; /* Netdev registration diff --git a/drivers/net/ethernet/sfc/ef100_nic.c b/drivers/net/ethernet/sfc/ef100_nic.c index 8061efdaf82c..ad686c671ab8 100644 --- a/drivers/net/ethernet/sfc/ef100_nic.c +++ b/drivers/net/ethernet/sfc/ef100_nic.c @@ -1137,6 +1137,9 @@ int ef100_probe_netdev_pf(struct efx_nic *efx) */ netif_warn(efx, probe, net_dev, "Failed to probe MAE rc %d\n", rc); + } else { + net_dev->features |= NETIF_F_HW_TC; + efx->fixed_features |= NETIF_F_HW_TC; } #endif return 0; diff --git a/drivers/net/ethernet/sfc/ef100_rep.c b/drivers/net/ethernet/sfc/ef100_rep.c index 73ae4656a6e7..81ab22c74635 100644 --- a/drivers/net/ethernet/sfc/ef100_rep.c +++ b/drivers/net/ethernet/sfc/ef100_rep.c @@ -14,6 +14,7 @@ #include "ef100_nic.h" #include "mae.h" #include "rx_common.h" +#include "tc_bindings.h" #define EFX_EF100_REP_DRIVER "efx_ef100_rep" @@ -42,8 +43,7 @@ static int efx_ef100_rep_open(struct net_device *net_dev) { struct efx_rep *efv = netdev_priv(net_dev); - netif_napi_add(net_dev, &efv->napi, efx_ef100_rep_poll, - NAPI_POLL_WEIGHT); + netif_napi_add(net_dev, &efv->napi, efx_ef100_rep_poll); napi_enable(&efv->napi); return 0; } @@ -107,6 +107,20 @@ static int efx_ef100_rep_get_phys_port_name(struct net_device *dev, return 0; } +static int efx_ef100_rep_setup_tc(struct net_device *net_dev, + enum tc_setup_type type, void *type_data) +{ + struct efx_rep *efv = netdev_priv(net_dev); + struct efx_nic *efx = efv->parent; + + if (type == TC_SETUP_CLSFLOWER) + return efx_tc_flower(efx, net_dev, type_data, efv); + if (type == TC_SETUP_BLOCK) + return efx_tc_setup_block(net_dev, efx, type_data, efv); + + return -EOPNOTSUPP; +} + static void efx_ef100_rep_get_stats64(struct net_device *dev, struct rtnl_link_stats64 *stats) { @@ -120,13 +134,14 @@ static void efx_ef100_rep_get_stats64(struct net_device *dev, stats->tx_errors = atomic64_read(&efv->stats.tx_errors); } -static const struct net_device_ops efx_ef100_rep_netdev_ops = { +const struct net_device_ops efx_ef100_rep_netdev_ops = { .ndo_open = efx_ef100_rep_open, .ndo_stop = efx_ef100_rep_close, .ndo_start_xmit = efx_ef100_rep_xmit, .ndo_get_port_parent_id = efx_ef100_rep_get_port_parent_id, .ndo_get_phys_port_name = efx_ef100_rep_get_phys_port_name, .ndo_get_stats64 = efx_ef100_rep_get_stats64, + .ndo_setup_tc = efx_ef100_rep_setup_tc, }; static void efx_ef100_rep_get_drvinfo(struct net_device *dev, diff --git a/drivers/net/ethernet/sfc/ef100_rep.h b/drivers/net/ethernet/sfc/ef100_rep.h index 070f700893c1..c21bc716f847 100644 --- a/drivers/net/ethernet/sfc/ef100_rep.h +++ b/drivers/net/ethernet/sfc/ef100_rep.h @@ -66,4 +66,5 @@ void efx_ef100_rep_rx_packet(struct efx_rep *efv, struct efx_rx_buffer *rx_buf); * Caller must hold rcu_read_lock(). */ struct efx_rep *efx_ef100_find_rep_by_mport(struct efx_nic *efx, u16 mport); +extern const struct net_device_ops efx_ef100_rep_netdev_ops; #endif /* EF100_REP_H */ diff --git a/drivers/net/ethernet/sfc/efx.c b/drivers/net/ethernet/sfc/efx.c index 153d68e29b8b..054d5ce6029e 100644 --- a/drivers/net/ethernet/sfc/efx.c +++ b/drivers/net/ethernet/sfc/efx.c @@ -778,7 +778,7 @@ static void efx_unregister_netdev(struct efx_nic *efx) return; if (efx_dev_registered(efx)) { - strlcpy(efx->name, pci_name(efx->pci_dev), sizeof(efx->name)); + strscpy(efx->name, pci_name(efx->pci_dev), sizeof(efx->name)); efx_fini_mcdi_logging(efx); device_remove_file(&efx->pci_dev->dev, &dev_attr_phy_type); unregister_netdev(efx->net_dev); @@ -1175,6 +1175,17 @@ static int efx_pm_freeze(struct device *dev) return 0; } +static void efx_pci_shutdown(struct pci_dev *pci_dev) +{ + struct efx_nic *efx = pci_get_drvdata(pci_dev); + + if (!efx) + return; + + efx_pm_freeze(&pci_dev->dev); + pci_disable_device(pci_dev); +} + static int efx_pm_thaw(struct device *dev) { int rc; @@ -1279,6 +1290,7 @@ static struct pci_driver efx_pci_driver = { .probe = efx_pci_probe, .remove = efx_pci_remove, .driver.pm = &efx_pm_ops, + .shutdown = efx_pci_shutdown, .err_handler = &efx_err_handlers, #ifdef CONFIG_SFC_SRIOV .sriov_configure = efx_pci_sriov_configure, diff --git a/drivers/net/ethernet/sfc/efx_channels.c b/drivers/net/ethernet/sfc/efx_channels.c index 5b4d661ab986..aaa381743bca 100644 --- a/drivers/net/ethernet/sfc/efx_channels.c +++ b/drivers/net/ethernet/sfc/efx_channels.c @@ -1313,7 +1313,7 @@ void efx_init_napi_channel(struct efx_channel *channel) struct efx_nic *efx = channel->efx; channel->napi_dev = efx->net_dev; - netif_napi_add(channel->napi_dev, &channel->napi_str, efx_poll, 64); + netif_napi_add(channel->napi_dev, &channel->napi_str, efx_poll); } void efx_init_napi(struct efx_nic *efx) diff --git a/drivers/net/ethernet/sfc/efx_common.c b/drivers/net/ethernet/sfc/efx_common.c index a929a1aaba92..c2224e41a694 100644 --- a/drivers/net/ethernet/sfc/efx_common.c +++ b/drivers/net/ethernet/sfc/efx_common.c @@ -996,7 +996,7 @@ int efx_init_struct(struct efx_nic *efx, struct pci_dev *pci_dev) efx->pci_dev = pci_dev; efx->msg_enable = debug; efx->state = STATE_UNINIT; - strlcpy(efx->name, pci_name(pci_dev), sizeof(efx->name)); + strscpy(efx->name, pci_name(pci_dev), sizeof(efx->name)); efx->rx_prefix_size = efx->type->rx_prefix_size; efx->rx_ip_align = diff --git a/drivers/net/ethernet/sfc/ethtool_common.c b/drivers/net/ethernet/sfc/ethtool_common.c index bc840ede3053..6649a2327d03 100644 --- a/drivers/net/ethernet/sfc/ethtool_common.c +++ b/drivers/net/ethernet/sfc/ethtool_common.c @@ -101,15 +101,23 @@ static const struct efx_sw_stat_desc efx_sw_stat_desc[] = { #define EFX_ETHTOOL_SW_STAT_COUNT ARRAY_SIZE(efx_sw_stat_desc) +static const char efx_ethtool_priv_flags_strings[][ETH_GSTRING_LEN] = { + "log-tc-errors", +}; + +#define EFX_ETHTOOL_PRIV_FLAGS_LOG_TC_ERRS BIT(0) + +#define EFX_ETHTOOL_PRIV_FLAGS_COUNT ARRAY_SIZE(efx_ethtool_priv_flags_strings) + void efx_ethtool_get_drvinfo(struct net_device *net_dev, struct ethtool_drvinfo *info) { struct efx_nic *efx = efx_netdev_priv(net_dev); - strlcpy(info->driver, KBUILD_MODNAME, sizeof(info->driver)); + strscpy(info->driver, KBUILD_MODNAME, sizeof(info->driver)); efx_mcdi_print_fwver(efx, info->fw_version, sizeof(info->fw_version)); - strlcpy(info->bus_info, pci_name(efx->pci_dev), sizeof(info->bus_info)); + strscpy(info->bus_info, pci_name(efx->pci_dev), sizeof(info->bus_info)); } u32 efx_ethtool_get_msglevel(struct net_device *net_dev) @@ -452,6 +460,8 @@ int efx_ethtool_get_sset_count(struct net_device *net_dev, int string_set) efx_ptp_describe_stats(efx, NULL); case ETH_SS_TEST: return efx_ethtool_fill_self_tests(efx, NULL, NULL, NULL); + case ETH_SS_PRIV_FLAGS: + return EFX_ETHTOOL_PRIV_FLAGS_COUNT; default: return -EINVAL; } @@ -468,7 +478,7 @@ void efx_ethtool_get_strings(struct net_device *net_dev, strings += (efx->type->describe_stats(efx, strings) * ETH_GSTRING_LEN); for (i = 0; i < EFX_ETHTOOL_SW_STAT_COUNT; i++) - strlcpy(strings + i * ETH_GSTRING_LEN, + strscpy(strings + i * ETH_GSTRING_LEN, efx_sw_stat_desc[i].name, ETH_GSTRING_LEN); strings += EFX_ETHTOOL_SW_STAT_COUNT * ETH_GSTRING_LEN; strings += (efx_describe_per_queue_stats(efx, strings) * @@ -478,12 +488,39 @@ void efx_ethtool_get_strings(struct net_device *net_dev, case ETH_SS_TEST: efx_ethtool_fill_self_tests(efx, NULL, strings, NULL); break; + case ETH_SS_PRIV_FLAGS: + for (i = 0; i < EFX_ETHTOOL_PRIV_FLAGS_COUNT; i++) + strscpy(strings + i * ETH_GSTRING_LEN, + efx_ethtool_priv_flags_strings[i], + ETH_GSTRING_LEN); + break; default: /* No other string sets */ break; } } +u32 efx_ethtool_get_priv_flags(struct net_device *net_dev) +{ + struct efx_nic *efx = efx_netdev_priv(net_dev); + u32 ret_flags = 0; + + if (efx->log_tc_errs) + ret_flags |= EFX_ETHTOOL_PRIV_FLAGS_LOG_TC_ERRS; + + return ret_flags; +} + +int efx_ethtool_set_priv_flags(struct net_device *net_dev, u32 flags) +{ + struct efx_nic *efx = efx_netdev_priv(net_dev); + + efx->log_tc_errs = + !!(flags & EFX_ETHTOOL_PRIV_FLAGS_LOG_TC_ERRS); + + return 0; +} + void efx_ethtool_get_stats(struct net_device *net_dev, struct ethtool_stats *stats, u64 *data) diff --git a/drivers/net/ethernet/sfc/ethtool_common.h b/drivers/net/ethernet/sfc/ethtool_common.h index 659491932101..0afc74021a5e 100644 --- a/drivers/net/ethernet/sfc/ethtool_common.h +++ b/drivers/net/ethernet/sfc/ethtool_common.h @@ -27,6 +27,8 @@ int efx_ethtool_fill_self_tests(struct efx_nic *efx, int efx_ethtool_get_sset_count(struct net_device *net_dev, int string_set); void efx_ethtool_get_strings(struct net_device *net_dev, u32 string_set, u8 *strings); +u32 efx_ethtool_get_priv_flags(struct net_device *net_dev); +int efx_ethtool_set_priv_flags(struct net_device *net_dev, u32 flags); void efx_ethtool_get_stats(struct net_device *net_dev, struct ethtool_stats *stats __attribute__ ((unused)), u64 *data); diff --git a/drivers/net/ethernet/sfc/falcon/efx.c b/drivers/net/ethernet/sfc/falcon/efx.c index a63f40b09856..e151b0957751 100644 --- a/drivers/net/ethernet/sfc/falcon/efx.c +++ b/drivers/net/ethernet/sfc/falcon/efx.c @@ -2012,7 +2012,7 @@ static void ef4_init_napi_channel(struct ef4_channel *channel) struct ef4_nic *efx = channel->efx; channel->napi_dev = efx->net_dev; - netif_napi_add(channel->napi_dev, &channel->napi_str, ef4_poll, 64); + netif_napi_add(channel->napi_dev, &channel->napi_str, ef4_poll); } static void ef4_init_napi(struct ef4_nic *efx) @@ -2329,7 +2329,7 @@ static void ef4_unregister_netdev(struct ef4_nic *efx) BUG_ON(netdev_priv(efx->net_dev) != efx); if (ef4_dev_registered(efx)) { - strlcpy(efx->name, pci_name(efx->pci_dev), sizeof(efx->name)); + strscpy(efx->name, pci_name(efx->pci_dev), sizeof(efx->name)); device_remove_file(&efx->pci_dev->dev, &dev_attr_phy_type); unregister_netdev(efx->net_dev); } @@ -2640,7 +2640,7 @@ static int ef4_init_struct(struct ef4_nic *efx, efx->pci_dev = pci_dev; efx->msg_enable = debug; efx->state = STATE_UNINIT; - strlcpy(efx->name, pci_name(pci_dev), sizeof(efx->name)); + strscpy(efx->name, pci_name(pci_dev), sizeof(efx->name)); efx->net_dev = net_dev; efx->rx_prefix_size = efx->type->rx_prefix_size; diff --git a/drivers/net/ethernet/sfc/falcon/ethtool.c b/drivers/net/ethernet/sfc/falcon/ethtool.c index 907254b36663..3976a333f7e3 100644 --- a/drivers/net/ethernet/sfc/falcon/ethtool.c +++ b/drivers/net/ethernet/sfc/falcon/ethtool.c @@ -162,9 +162,9 @@ static void ef4_ethtool_get_drvinfo(struct net_device *net_dev, { struct ef4_nic *efx = netdev_priv(net_dev); - strlcpy(info->driver, KBUILD_MODNAME, sizeof(info->driver)); - strlcpy(info->version, EF4_DRIVER_VERSION, sizeof(info->version)); - strlcpy(info->bus_info, pci_name(efx->pci_dev), sizeof(info->bus_info)); + strscpy(info->driver, KBUILD_MODNAME, sizeof(info->driver)); + strscpy(info->version, EF4_DRIVER_VERSION, sizeof(info->version)); + strscpy(info->bus_info, pci_name(efx->pci_dev), sizeof(info->bus_info)); } static int ef4_ethtool_get_regs_len(struct net_device *net_dev) @@ -412,7 +412,7 @@ static void ef4_ethtool_get_strings(struct net_device *net_dev, strings += (efx->type->describe_stats(efx, strings) * ETH_GSTRING_LEN); for (i = 0; i < EF4_ETHTOOL_SW_STAT_COUNT; i++) - strlcpy(strings + i * ETH_GSTRING_LEN, + strscpy(strings + i * ETH_GSTRING_LEN, ef4_sw_stat_desc[i].name, ETH_GSTRING_LEN); strings += EF4_ETHTOOL_SW_STAT_COUNT * ETH_GSTRING_LEN; strings += (ef4_describe_per_queue_stats(efx, strings) * diff --git a/drivers/net/ethernet/sfc/falcon/falcon.c b/drivers/net/ethernet/sfc/falcon/falcon.c index 3324a6219a09..7a1c9337081b 100644 --- a/drivers/net/ethernet/sfc/falcon/falcon.c +++ b/drivers/net/ethernet/sfc/falcon/falcon.c @@ -2387,7 +2387,7 @@ static int falcon_probe_nic(struct ef4_nic *efx) board->i2c_data.data = efx; board->i2c_adap.algo_data = &board->i2c_data; board->i2c_adap.dev.parent = &efx->pci_dev->dev; - strlcpy(board->i2c_adap.name, "SFC4000 GPIO", + strscpy(board->i2c_adap.name, "SFC4000 GPIO", sizeof(board->i2c_adap.name)); rc = i2c_bit_add_bus(&board->i2c_adap); if (rc) diff --git a/drivers/net/ethernet/sfc/falcon/nic.c b/drivers/net/ethernet/sfc/falcon/nic.c index 156da315ec89..78c851b5a56f 100644 --- a/drivers/net/ethernet/sfc/falcon/nic.c +++ b/drivers/net/ethernet/sfc/falcon/nic.c @@ -452,7 +452,7 @@ size_t ef4_nic_describe_stats(const struct ef4_hw_stat_desc *desc, size_t count, for_each_set_bit(index, mask, count) { if (desc[index].name) { if (names) { - strlcpy(names, desc[index].name, + strscpy(names, desc[index].name, ETH_GSTRING_LEN); names += ETH_GSTRING_LEN; } diff --git a/drivers/net/ethernet/sfc/filter.h b/drivers/net/ethernet/sfc/filter.h index 4d928839d292..be72e71da027 100644 --- a/drivers/net/ethernet/sfc/filter.h +++ b/drivers/net/ethernet/sfc/filter.h @@ -9,6 +9,7 @@ #include <linux/types.h> #include <linux/if_ether.h> +#include <linux/in6.h> #include <asm/byteorder.h> /** @@ -224,6 +225,27 @@ efx_filter_set_ipv4_local(struct efx_filter_spec *spec, u8 proto, } /** + * efx_filter_set_ipv6_local - specify IPv6 host, transport protocol and port + * @spec: Specification to initialise + * @proto: Transport layer protocol number + * @host: Local host address (network byte order) + * @port: Local port (network byte order) + */ +static inline int +efx_filter_set_ipv6_local(struct efx_filter_spec *spec, u8 proto, + const struct in6_addr *host, __be16 port) +{ + spec->match_flags |= + EFX_FILTER_MATCH_ETHER_TYPE | EFX_FILTER_MATCH_IP_PROTO | + EFX_FILTER_MATCH_LOC_HOST | EFX_FILTER_MATCH_LOC_PORT; + spec->ether_type = htons(ETH_P_IPV6); + spec->ip_proto = proto; + memcpy(spec->loc_host, host, sizeof(spec->loc_host)); + spec->loc_port = port; + return 0; +} + +/** * efx_filter_set_ipv4_full - specify IPv4 hosts, transport protocol and ports * @spec: Specification to initialise * @proto: Transport layer protocol number diff --git a/drivers/net/ethernet/sfc/mae.c b/drivers/net/ethernet/sfc/mae.c index 97627f5e3674..874c765b2465 100644 --- a/drivers/net/ethernet/sfc/mae.c +++ b/drivers/net/ethernet/sfc/mae.c @@ -112,6 +112,167 @@ int efx_mae_lookup_mport(struct efx_nic *efx, u32 selector, u32 *id) return 0; } +static int efx_mae_get_basic_caps(struct efx_nic *efx, struct mae_caps *caps) +{ + MCDI_DECLARE_BUF(outbuf, MC_CMD_MAE_GET_CAPS_OUT_LEN); + size_t outlen; + int rc; + + BUILD_BUG_ON(MC_CMD_MAE_GET_CAPS_IN_LEN); + + rc = efx_mcdi_rpc(efx, MC_CMD_MAE_GET_CAPS, NULL, 0, outbuf, + sizeof(outbuf), &outlen); + if (rc) + return rc; + if (outlen < sizeof(outbuf)) + return -EIO; + caps->match_field_count = MCDI_DWORD(outbuf, MAE_GET_CAPS_OUT_MATCH_FIELD_COUNT); + caps->action_prios = MCDI_DWORD(outbuf, MAE_GET_CAPS_OUT_ACTION_PRIOS); + return 0; +} + +static int efx_mae_get_rule_fields(struct efx_nic *efx, u32 cmd, + u8 *field_support) +{ + MCDI_DECLARE_BUF(outbuf, MC_CMD_MAE_GET_AR_CAPS_OUT_LEN(MAE_NUM_FIELDS)); + MCDI_DECLARE_STRUCT_PTR(caps); + unsigned int count; + size_t outlen; + int rc, i; + + BUILD_BUG_ON(MC_CMD_MAE_GET_AR_CAPS_IN_LEN); + + rc = efx_mcdi_rpc(efx, cmd, NULL, 0, outbuf, sizeof(outbuf), &outlen); + if (rc) + return rc; + count = MCDI_DWORD(outbuf, MAE_GET_AR_CAPS_OUT_COUNT); + memset(field_support, MAE_FIELD_UNSUPPORTED, MAE_NUM_FIELDS); + caps = _MCDI_DWORD(outbuf, MAE_GET_AR_CAPS_OUT_FIELD_FLAGS); + /* We're only interested in the support status enum, not any other + * flags, so just extract that from each entry. + */ + for (i = 0; i < count; i++) + if (i * sizeof(*outbuf) + MC_CMD_MAE_GET_AR_CAPS_OUT_FIELD_FLAGS_OFST < outlen) + field_support[i] = EFX_DWORD_FIELD(caps[i], MAE_FIELD_FLAGS_SUPPORT_STATUS); + return 0; +} + +int efx_mae_get_caps(struct efx_nic *efx, struct mae_caps *caps) +{ + int rc; + + rc = efx_mae_get_basic_caps(efx, caps); + if (rc) + return rc; + return efx_mae_get_rule_fields(efx, MC_CMD_MAE_GET_AR_CAPS, + caps->action_rule_fields); +} + +/* Bit twiddling: + * Prefix: 1...110...0 + * ~: 0...001...1 + * + 1: 0...010...0 is power of two + * so (~x) & ((~x) + 1) == 0. Converse holds also. + */ +#define is_prefix_byte(_x) !(((_x) ^ 0xff) & (((_x) ^ 0xff) + 1)) + +enum mask_type { MASK_ONES, MASK_ZEROES, MASK_PREFIX, MASK_OTHER }; + +static const char *mask_type_name(enum mask_type typ) +{ + switch (typ) { + case MASK_ONES: + return "all-1s"; + case MASK_ZEROES: + return "all-0s"; + case MASK_PREFIX: + return "prefix"; + case MASK_OTHER: + return "arbitrary"; + default: /* can't happen */ + return "unknown"; + } +} + +/* Checks a (big-endian) bytestring is a bit prefix */ +static enum mask_type classify_mask(const u8 *mask, size_t len) +{ + bool zeroes = true; /* All bits seen so far are zeroes */ + bool ones = true; /* All bits seen so far are ones */ + bool prefix = true; /* Valid prefix so far */ + size_t i; + + for (i = 0; i < len; i++) { + if (ones) { + if (!is_prefix_byte(mask[i])) + prefix = false; + } else if (mask[i]) { + prefix = false; + } + if (mask[i] != 0xff) + ones = false; + if (mask[i]) + zeroes = false; + } + if (ones) + return MASK_ONES; + if (zeroes) + return MASK_ZEROES; + if (prefix) + return MASK_PREFIX; + return MASK_OTHER; +} + +static int efx_mae_match_check_cap_typ(u8 support, enum mask_type typ) +{ + switch (support) { + case MAE_FIELD_UNSUPPORTED: + case MAE_FIELD_SUPPORTED_MATCH_NEVER: + if (typ == MASK_ZEROES) + return 0; + return -EOPNOTSUPP; + case MAE_FIELD_SUPPORTED_MATCH_OPTIONAL: + if (typ == MASK_ZEROES) + return 0; + fallthrough; + case MAE_FIELD_SUPPORTED_MATCH_ALWAYS: + if (typ == MASK_ONES) + return 0; + return -EINVAL; + case MAE_FIELD_SUPPORTED_MATCH_PREFIX: + if (typ == MASK_OTHER) + return -EOPNOTSUPP; + return 0; + case MAE_FIELD_SUPPORTED_MATCH_MASK: + return 0; + default: + return -EIO; + } +} + +int efx_mae_match_check_caps(struct efx_nic *efx, + const struct efx_tc_match_fields *mask, + struct netlink_ext_ack *extack) +{ + const u8 *supported_fields = efx->tc->caps->action_rule_fields; + __be32 ingress_port = cpu_to_be32(mask->ingress_port); + enum mask_type ingress_port_mask_type; + int rc; + + /* Check for _PREFIX assumes big-endian, so we need to convert */ + ingress_port_mask_type = classify_mask((const u8 *)&ingress_port, + sizeof(ingress_port)); + rc = efx_mae_match_check_cap_typ(supported_fields[MAE_FIELD_INGRESS_PORT], + ingress_port_mask_type); + if (rc) { + efx_tc_err(efx, "No support for %s mask in field ingress_port\n", + mask_type_name(ingress_port_mask_type)); + NL_SET_ERR_MSG_MOD(extack, "Unsupported mask type for ingress_port"); + return rc; + } + return 0; +} + static bool efx_mae_asl_id(u32 id) { return !!(id & BIT(31)); @@ -279,6 +440,10 @@ static int efx_mae_populate_match_criteria(MCDI_DECLARE_STRUCT_PTR(match_crit), } MCDI_STRUCT_SET_DWORD(match_crit, MAE_FIELD_MASK_VALUE_PAIRS_V2_INGRESS_MPORT_SELECTOR_MASK, match->mask.ingress_port); + MCDI_STRUCT_SET_BYTE(match_crit, MAE_FIELD_MASK_VALUE_PAIRS_V2_RECIRC_ID, + match->value.recirc_id); + MCDI_STRUCT_SET_BYTE(match_crit, MAE_FIELD_MASK_VALUE_PAIRS_V2_RECIRC_ID_MASK, + match->mask.recirc_id); return 0; } diff --git a/drivers/net/ethernet/sfc/mae.h b/drivers/net/ethernet/sfc/mae.h index 0369be4d8983..3e0cd238d523 100644 --- a/drivers/net/ethernet/sfc/mae.h +++ b/drivers/net/ethernet/sfc/mae.h @@ -27,6 +27,20 @@ void efx_mae_mport_mport(struct efx_nic *efx, u32 mport_id, u32 *out); int efx_mae_lookup_mport(struct efx_nic *efx, u32 selector, u32 *id); +#define MAE_NUM_FIELDS (MAE_FIELD_ENC_VNET_ID + 1) + +struct mae_caps { + u32 match_field_count; + u32 action_prios; + u8 action_rule_fields[MAE_NUM_FIELDS]; +}; + +int efx_mae_get_caps(struct efx_nic *efx, struct mae_caps *caps); + +int efx_mae_match_check_caps(struct efx_nic *efx, + const struct efx_tc_match_fields *mask, + struct netlink_ext_ack *extack); + int efx_mae_alloc_action_set(struct efx_nic *efx, struct efx_tc_action_set *act); int efx_mae_free_action_set(struct efx_nic *efx, u32 fw_id); diff --git a/drivers/net/ethernet/sfc/mcdi.h b/drivers/net/ethernet/sfc/mcdi.h index 26bc69f76801..1f18e9dc62e8 100644 --- a/drivers/net/ethernet/sfc/mcdi.h +++ b/drivers/net/ethernet/sfc/mcdi.h @@ -201,6 +201,12 @@ void efx_mcdi_sensor_event(struct efx_nic *efx, efx_qword_t *ev); ((u8 *)(_buf) + (_offset)) #define MCDI_PTR(_buf, _field) \ _MCDI_PTR(_buf, MC_CMD_ ## _field ## _OFST) +/* Use MCDI_STRUCT_ functions to access members of MCDI structuredefs. + * _buf should point to the start of the structure, typically obtained with + * MCDI_DECLARE_STRUCT_PTR(structure) = _MCDI_DWORD(mcdi_buf, FIELD_WHICH_IS_STRUCT); + */ +#define MCDI_STRUCT_PTR(_buf, _field) \ + _MCDI_PTR(_buf, _field ## _OFST) #define _MCDI_CHECK_ALIGN(_ofst, _align) \ ((_ofst) + BUILD_BUG_ON_ZERO((_ofst) & (_align - 1))) #define _MCDI_DWORD(_buf, _field) \ @@ -208,6 +214,10 @@ void efx_mcdi_sensor_event(struct efx_nic *efx, efx_qword_t *ev); #define _MCDI_STRUCT_DWORD(_buf, _field) \ ((_buf) + (_MCDI_CHECK_ALIGN(_field ## _OFST, 4) >> 2)) +#define MCDI_STRUCT_SET_BYTE(_buf, _field, _value) do { \ + BUILD_BUG_ON(_field ## _LEN != 1); \ + *(u8 *)MCDI_STRUCT_PTR(_buf, _field) = _value; \ + } while (0) #define MCDI_BYTE(_buf, _field) \ ((void)BUILD_BUG_ON_ZERO(MC_CMD_ ## _field ## _LEN != 1), \ *MCDI_PTR(_buf, _field)) diff --git a/drivers/net/ethernet/sfc/mcdi_mon.c b/drivers/net/ethernet/sfc/mcdi_mon.c index 5954fcfee2b1..f5128db7c7e7 100644 --- a/drivers/net/ethernet/sfc/mcdi_mon.c +++ b/drivers/net/ethernet/sfc/mcdi_mon.c @@ -285,7 +285,7 @@ efx_mcdi_mon_add_attr(struct efx_nic *efx, const char *name, struct efx_mcdi_mon *hwmon = efx_mcdi_mon(efx); struct efx_mcdi_mon_attribute *attr = &hwmon->attrs[hwmon->n_attrs]; - strlcpy(attr->name, name, sizeof(attr->name)); + strscpy(attr->name, name, sizeof(attr->name)); attr->index = index; attr->type = type; if (type < ARRAY_SIZE(efx_mcdi_sensor_type)) diff --git a/drivers/net/ethernet/sfc/net_driver.h b/drivers/net/ethernet/sfc/net_driver.h index 7ef823d7a89a..2e9ba0cfe848 100644 --- a/drivers/net/ethernet/sfc/net_driver.h +++ b/drivers/net/ethernet/sfc/net_driver.h @@ -855,6 +855,7 @@ enum efx_xdp_tx_queues_mode { * @timer_max_ns: Interrupt timer maximum value, in nanoseconds * @irq_rx_adaptive: Adaptive IRQ moderation enabled for RX event queues * @irqs_hooked: Channel interrupts are hooked + * @log_tc_errs: Error logging for TC filter insertion is enabled * @irq_rx_mod_step_us: Step size for IRQ moderation for RX event queues * @irq_rx_moderation_us: IRQ moderation time for RX event queues * @msg_enable: Log message enable flags @@ -1017,6 +1018,7 @@ struct efx_nic { unsigned int timer_max_ns; bool irq_rx_adaptive; bool irqs_hooked; + bool log_tc_errs; unsigned int irq_mod_step_us; unsigned int irq_rx_moderation_us; u32 msg_enable; diff --git a/drivers/net/ethernet/sfc/nic.c b/drivers/net/ethernet/sfc/nic.c index 22fbb0ae77fb..63e2394382bb 100644 --- a/drivers/net/ethernet/sfc/nic.c +++ b/drivers/net/ethernet/sfc/nic.c @@ -465,7 +465,7 @@ size_t efx_nic_describe_stats(const struct efx_hw_stat_desc *desc, size_t count, for_each_set_bit(index, mask, count) { if (desc[index].name) { if (names) { - strlcpy(names, desc[index].name, + strscpy(names, desc[index].name, ETH_GSTRING_LEN); names += ETH_GSTRING_LEN; } diff --git a/drivers/net/ethernet/sfc/ptp.c b/drivers/net/ethernet/sfc/ptp.c index 10ad0b93d283..eaef4a15008a 100644 --- a/drivers/net/ethernet/sfc/ptp.c +++ b/drivers/net/ethernet/sfc/ptp.c @@ -118,9 +118,14 @@ #define PTP_MIN_LENGTH 63 -#define PTP_ADDRESS 0xe0000181 /* 224.0.1.129 */ +#define PTP_RXFILTERS_LEN 5 + +#define PTP_ADDR_IPV4 0xe0000181 /* 224.0.1.129 */ +#define PTP_ADDR_IPV6 {0xff, 0x0e, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, \ + 0, 0x01, 0x81} /* ff0e::181 */ #define PTP_EVENT_PORT 319 #define PTP_GENERAL_PORT 320 +#define PTP_ADDR_ETHER {0x01, 0x1b, 0x19, 0, 0, 0} /* 01-1B-19-00-00-00 */ /* Annoyingly the format of the version numbers are different between * versions 1 and 2 so it isn't possible to simply look for 1 or 2. @@ -224,9 +229,8 @@ struct efx_ptp_timeset { * @work: Work task * @reset_required: A serious error has occurred and the PTP task needs to be * reset (disable, enable). - * @rxfilter_event: Receive filter when operating - * @rxfilter_general: Receive filter when operating - * @rxfilter_installed: Receive filter installed + * @rxfilters: Receive filters when operating + * @rxfilters_count: Num of installed rxfilters, should be == PTP_RXFILTERS_LEN * @config: Current timestamp configuration * @enabled: PTP operation enabled * @mode: Mode in which PTP operating (PTP version) @@ -295,9 +299,8 @@ struct efx_ptp_data { struct workqueue_struct *workwq; struct work_struct work; bool reset_required; - u32 rxfilter_event; - u32 rxfilter_general; - bool rxfilter_installed; + u32 rxfilters[PTP_RXFILTERS_LEN]; + size_t rxfilters_count; struct hwtstamp_config config; bool enabled; unsigned int mode; @@ -1290,61 +1293,108 @@ static void efx_ptp_remove_multicast_filters(struct efx_nic *efx) { struct efx_ptp_data *ptp = efx->ptp_data; - if (ptp->rxfilter_installed) { - efx_filter_remove_id_safe(efx, EFX_FILTER_PRI_REQUIRED, - ptp->rxfilter_general); + while (ptp->rxfilters_count) { + ptp->rxfilters_count--; efx_filter_remove_id_safe(efx, EFX_FILTER_PRI_REQUIRED, - ptp->rxfilter_event); - ptp->rxfilter_installed = false; + ptp->rxfilters[ptp->rxfilters_count]); } } -static int efx_ptp_insert_multicast_filters(struct efx_nic *efx) +static void efx_ptp_init_filter(struct efx_nic *efx, + struct efx_filter_spec *rxfilter) +{ + struct efx_channel *channel = efx->ptp_data->channel; + struct efx_rx_queue *queue = efx_channel_get_rx_queue(channel); + + efx_filter_init_rx(rxfilter, EFX_FILTER_PRI_REQUIRED, 0, + efx_rx_queue_index(queue)); +} + +static int efx_ptp_insert_filter(struct efx_nic *efx, + struct efx_filter_spec *rxfilter) { struct efx_ptp_data *ptp = efx->ptp_data; + + int rc = efx_filter_insert_filter(efx, rxfilter, true); + if (rc < 0) + return rc; + ptp->rxfilters[ptp->rxfilters_count] = rc; + ptp->rxfilters_count++; + return 0; +} + +static int efx_ptp_insert_ipv4_filter(struct efx_nic *efx, u16 port) +{ struct efx_filter_spec rxfilter; + + efx_ptp_init_filter(efx, &rxfilter); + efx_filter_set_ipv4_local(&rxfilter, IPPROTO_UDP, htonl(PTP_ADDR_IPV4), + htons(port)); + return efx_ptp_insert_filter(efx, &rxfilter); +} + +static int efx_ptp_insert_ipv6_filter(struct efx_nic *efx, u16 port) +{ + const struct in6_addr addr = {{PTP_ADDR_IPV6}}; + struct efx_filter_spec rxfilter; + + efx_ptp_init_filter(efx, &rxfilter); + efx_filter_set_ipv6_local(&rxfilter, IPPROTO_UDP, &addr, htons(port)); + return efx_ptp_insert_filter(efx, &rxfilter); +} + +static int efx_ptp_insert_eth_filter(struct efx_nic *efx) +{ + const u8 addr[ETH_ALEN] = PTP_ADDR_ETHER; + struct efx_filter_spec rxfilter; + + efx_ptp_init_filter(efx, &rxfilter); + efx_filter_set_eth_local(&rxfilter, EFX_FILTER_VID_UNSPEC, addr); + rxfilter.match_flags |= EFX_FILTER_MATCH_ETHER_TYPE; + rxfilter.ether_type = htons(ETH_P_1588); + return efx_ptp_insert_filter(efx, &rxfilter); +} + +static int efx_ptp_insert_multicast_filters(struct efx_nic *efx) +{ + struct efx_ptp_data *ptp = efx->ptp_data; int rc; - if (!ptp->channel || ptp->rxfilter_installed) + if (!ptp->channel || ptp->rxfilters_count) return 0; /* Must filter on both event and general ports to ensure * that there is no packet re-ordering. */ - efx_filter_init_rx(&rxfilter, EFX_FILTER_PRI_REQUIRED, 0, - efx_rx_queue_index( - efx_channel_get_rx_queue(ptp->channel))); - rc = efx_filter_set_ipv4_local(&rxfilter, IPPROTO_UDP, - htonl(PTP_ADDRESS), - htons(PTP_EVENT_PORT)); - if (rc != 0) - return rc; - - rc = efx_filter_insert_filter(efx, &rxfilter, true); + rc = efx_ptp_insert_ipv4_filter(efx, PTP_EVENT_PORT); if (rc < 0) - return rc; - ptp->rxfilter_event = rc; - - efx_filter_init_rx(&rxfilter, EFX_FILTER_PRI_REQUIRED, 0, - efx_rx_queue_index( - efx_channel_get_rx_queue(ptp->channel))); - rc = efx_filter_set_ipv4_local(&rxfilter, IPPROTO_UDP, - htonl(PTP_ADDRESS), - htons(PTP_GENERAL_PORT)); - if (rc != 0) goto fail; - rc = efx_filter_insert_filter(efx, &rxfilter, true); + rc = efx_ptp_insert_ipv4_filter(efx, PTP_GENERAL_PORT); if (rc < 0) goto fail; - ptp->rxfilter_general = rc; - ptp->rxfilter_installed = true; + /* if the NIC supports hw timestamps by the MAC, we can support + * PTP over IPv6 and Ethernet + */ + if (efx_ptp_use_mac_tx_timestamps(efx)) { + rc = efx_ptp_insert_ipv6_filter(efx, PTP_EVENT_PORT); + if (rc < 0) + goto fail; + + rc = efx_ptp_insert_ipv6_filter(efx, PTP_GENERAL_PORT); + if (rc < 0) + goto fail; + + rc = efx_ptp_insert_eth_filter(efx); + if (rc < 0) + goto fail; + } + return 0; fail: - efx_filter_remove_id_safe(efx, EFX_FILTER_PRI_REQUIRED, - ptp->rxfilter_event); + efx_ptp_remove_multicast_filters(efx); return rc; } diff --git a/drivers/net/ethernet/sfc/siena/efx.c b/drivers/net/ethernet/sfc/siena/efx.c index 63d999e63960..60e5b7c8ccf9 100644 --- a/drivers/net/ethernet/sfc/siena/efx.c +++ b/drivers/net/ethernet/sfc/siena/efx.c @@ -775,7 +775,7 @@ static void efx_unregister_netdev(struct efx_nic *efx) BUG_ON(netdev_priv(efx->net_dev) != efx); if (efx_dev_registered(efx)) { - strlcpy(efx->name, pci_name(efx->pci_dev), sizeof(efx->name)); + strscpy(efx->name, pci_name(efx->pci_dev), sizeof(efx->name)); efx_siena_fini_mcdi_logging(efx); device_remove_file(&efx->pci_dev->dev, &dev_attr_phy_type); unregister_netdev(efx->net_dev); @@ -1148,6 +1148,17 @@ static int efx_pm_freeze(struct device *dev) return 0; } +static void efx_pci_shutdown(struct pci_dev *pci_dev) +{ + struct efx_nic *efx = pci_get_drvdata(pci_dev); + + if (!efx) + return; + + efx_pm_freeze(&pci_dev->dev); + pci_disable_device(pci_dev); +} + static int efx_pm_thaw(struct device *dev) { int rc; @@ -1252,6 +1263,7 @@ static struct pci_driver efx_pci_driver = { .probe = efx_pci_probe, .remove = efx_pci_remove, .driver.pm = &efx_pm_ops, + .shutdown = efx_pci_shutdown, .err_handler = &efx_siena_err_handlers, #ifdef CONFIG_SFC_SIENA_SRIOV .sriov_configure = efx_pci_sriov_configure, diff --git a/drivers/net/ethernet/sfc/siena/efx_channels.c b/drivers/net/ethernet/sfc/siena/efx_channels.c index f54ebd007286..06ed74994e36 100644 --- a/drivers/net/ethernet/sfc/siena/efx_channels.c +++ b/drivers/net/ethernet/sfc/siena/efx_channels.c @@ -1317,7 +1317,7 @@ static void efx_init_napi_channel(struct efx_channel *channel) struct efx_nic *efx = channel->efx; channel->napi_dev = efx->net_dev; - netif_napi_add(channel->napi_dev, &channel->napi_str, efx_poll, 64); + netif_napi_add(channel->napi_dev, &channel->napi_str, efx_poll); } void efx_siena_init_napi(struct efx_nic *efx) diff --git a/drivers/net/ethernet/sfc/siena/efx_common.c b/drivers/net/ethernet/sfc/siena/efx_common.c index 954daf464abb..1fd396b00bfb 100644 --- a/drivers/net/ethernet/sfc/siena/efx_common.c +++ b/drivers/net/ethernet/sfc/siena/efx_common.c @@ -1006,7 +1006,7 @@ int efx_siena_init_struct(struct efx_nic *efx, efx->pci_dev = pci_dev; efx->msg_enable = debug; efx->state = STATE_UNINIT; - strlcpy(efx->name, pci_name(pci_dev), sizeof(efx->name)); + strscpy(efx->name, pci_name(pci_dev), sizeof(efx->name)); efx->net_dev = net_dev; efx->rx_prefix_size = efx->type->rx_prefix_size; diff --git a/drivers/net/ethernet/sfc/siena/ethtool_common.c b/drivers/net/ethernet/sfc/siena/ethtool_common.c index 0207d07f54e3..f590e87e5a23 100644 --- a/drivers/net/ethernet/sfc/siena/ethtool_common.c +++ b/drivers/net/ethernet/sfc/siena/ethtool_common.c @@ -105,10 +105,10 @@ void efx_siena_ethtool_get_drvinfo(struct net_device *net_dev, { struct efx_nic *efx = netdev_priv(net_dev); - strlcpy(info->driver, KBUILD_MODNAME, sizeof(info->driver)); + strscpy(info->driver, KBUILD_MODNAME, sizeof(info->driver)); efx_siena_mcdi_print_fwver(efx, info->fw_version, sizeof(info->fw_version)); - strlcpy(info->bus_info, pci_name(efx->pci_dev), sizeof(info->bus_info)); + strscpy(info->bus_info, pci_name(efx->pci_dev), sizeof(info->bus_info)); } u32 efx_siena_ethtool_get_msglevel(struct net_device *net_dev) @@ -467,7 +467,7 @@ void efx_siena_ethtool_get_strings(struct net_device *net_dev, strings += (efx->type->describe_stats(efx, strings) * ETH_GSTRING_LEN); for (i = 0; i < EFX_ETHTOOL_SW_STAT_COUNT; i++) - strlcpy(strings + i * ETH_GSTRING_LEN, + strscpy(strings + i * ETH_GSTRING_LEN, efx_sw_stat_desc[i].name, ETH_GSTRING_LEN); strings += EFX_ETHTOOL_SW_STAT_COUNT * ETH_GSTRING_LEN; strings += (efx_describe_per_queue_stats(efx, strings) * diff --git a/drivers/net/ethernet/sfc/siena/mcdi_mon.c b/drivers/net/ethernet/sfc/siena/mcdi_mon.c index c7ea703c5d7a..56a9c56ed9e3 100644 --- a/drivers/net/ethernet/sfc/siena/mcdi_mon.c +++ b/drivers/net/ethernet/sfc/siena/mcdi_mon.c @@ -285,7 +285,7 @@ efx_mcdi_mon_add_attr(struct efx_nic *efx, const char *name, struct efx_mcdi_mon *hwmon = efx_mcdi_mon(efx); struct efx_mcdi_mon_attribute *attr = &hwmon->attrs[hwmon->n_attrs]; - strlcpy(attr->name, name, sizeof(attr->name)); + strscpy(attr->name, name, sizeof(attr->name)); attr->index = index; attr->type = type; if (type < ARRAY_SIZE(efx_mcdi_sensor_type)) diff --git a/drivers/net/ethernet/sfc/siena/nic.c b/drivers/net/ethernet/sfc/siena/nic.c index abf9a4adf139..0ea0433a6230 100644 --- a/drivers/net/ethernet/sfc/siena/nic.c +++ b/drivers/net/ethernet/sfc/siena/nic.c @@ -458,7 +458,7 @@ size_t efx_siena_describe_stats(const struct efx_hw_stat_desc *desc, size_t coun for_each_set_bit(index, mask, count) { if (desc[index].name) { if (names) { - strlcpy(names, desc[index].name, + strscpy(names, desc[index].name, ETH_GSTRING_LEN); names += ETH_GSTRING_LEN; } diff --git a/drivers/net/ethernet/sfc/tc.c b/drivers/net/ethernet/sfc/tc.c index 0c0aeb91f500..3478860d4023 100644 --- a/drivers/net/ethernet/sfc/tc.c +++ b/drivers/net/ethernet/sfc/tc.c @@ -9,11 +9,60 @@ * by the Free Software Foundation, incorporated herein by reference. */ +#include <net/pkt_cls.h> #include "tc.h" +#include "tc_bindings.h" #include "mae.h" #include "ef100_rep.h" #include "efx.h" +#define EFX_EFV_PF NULL +/* Look up the representor information (efv) for a device. + * May return NULL for the PF (us), or an error pointer for a device that + * isn't supported as a TC offload endpoint + */ +static struct efx_rep *efx_tc_flower_lookup_efv(struct efx_nic *efx, + struct net_device *dev) +{ + struct efx_rep *efv; + + if (!dev) + return ERR_PTR(-EOPNOTSUPP); + /* Is it us (the PF)? */ + if (dev == efx->net_dev) + return EFX_EFV_PF; + /* Is it an efx vfrep at all? */ + if (dev->netdev_ops != &efx_ef100_rep_netdev_ops) + return ERR_PTR(-EOPNOTSUPP); + /* Is it ours? We don't support TC rules that include another + * EF100's netdevices (not even on another port of the same NIC). + */ + efv = netdev_priv(dev); + if (efv->parent != efx) + return ERR_PTR(-EOPNOTSUPP); + return efv; +} + +/* Convert a driver-internal vport ID into an external device (wire or VF) */ +static s64 efx_tc_flower_external_mport(struct efx_nic *efx, struct efx_rep *efv) +{ + u32 mport; + + if (IS_ERR(efv)) + return PTR_ERR(efv); + if (!efv) /* device is PF (us) */ + efx_mae_mport_wire(efx, &mport); + else /* device is repr */ + efx_mae_mport_mport(efx, efv->mport, &mport); + return mport; +} + +static const struct rhashtable_params efx_tc_match_action_ht_params = { + .key_len = sizeof(unsigned long), + .key_offset = offsetof(struct efx_tc_flow_rule, cookie), + .head_offset = offsetof(struct efx_tc_flow_rule, linkage), +}; + static void efx_tc_free_action_set(struct efx_nic *efx, struct efx_tc_action_set *act, bool in_hw) { @@ -58,6 +107,333 @@ static void efx_tc_delete_rule(struct efx_nic *efx, struct efx_tc_flow_rule *rul rule->fw_id = MC_CMD_MAE_ACTION_RULE_INSERT_OUT_ACTION_RULE_ID_NULL; } +static void efx_tc_flow_free(void *ptr, void *arg) +{ + struct efx_tc_flow_rule *rule = ptr; + struct efx_nic *efx = arg; + + netif_err(efx, drv, efx->net_dev, + "tc rule %lx still present at teardown, removing\n", + rule->cookie); + + efx_mae_delete_rule(efx, rule->fw_id); + + /* Release entries in subsidiary tables */ + efx_tc_free_action_set_list(efx, &rule->acts, true); + + kfree(rule); +} + +static int efx_tc_flower_parse_match(struct efx_nic *efx, + struct flow_rule *rule, + struct efx_tc_match *match, + struct netlink_ext_ack *extack) +{ + struct flow_dissector *dissector = rule->match.dissector; + + if (flow_rule_match_key(rule, FLOW_DISSECTOR_KEY_CONTROL)) { + struct flow_match_control fm; + + flow_rule_match_control(rule, &fm); + + if (fm.mask->flags) { + efx_tc_err(efx, "Unsupported match on control.flags %#x\n", + fm.mask->flags); + NL_SET_ERR_MSG_MOD(extack, "Unsupported match on control.flags"); + return -EOPNOTSUPP; + } + } + if (dissector->used_keys & + ~(BIT(FLOW_DISSECTOR_KEY_CONTROL) | + BIT(FLOW_DISSECTOR_KEY_BASIC))) { + efx_tc_err(efx, "Unsupported flower keys %#x\n", dissector->used_keys); + NL_SET_ERR_MSG_MOD(extack, "Unsupported flower keys encountered"); + return -EOPNOTSUPP; + } + + if (flow_rule_match_key(rule, FLOW_DISSECTOR_KEY_BASIC)) { + struct flow_match_basic fm; + + flow_rule_match_basic(rule, &fm); + if (fm.mask->n_proto) { + EFX_TC_ERR_MSG(efx, extack, "Unsupported eth_proto match\n"); + return -EOPNOTSUPP; + } + if (fm.mask->ip_proto) { + EFX_TC_ERR_MSG(efx, extack, "Unsupported ip_proto match\n"); + return -EOPNOTSUPP; + } + } + + return 0; +} + +static int efx_tc_flower_replace(struct efx_nic *efx, + struct net_device *net_dev, + struct flow_cls_offload *tc, + struct efx_rep *efv) +{ + struct flow_rule *fr = flow_cls_offload_flow_rule(tc); + struct netlink_ext_ack *extack = tc->common.extack; + struct efx_tc_flow_rule *rule = NULL, *old; + struct efx_tc_action_set *act = NULL; + const struct flow_action_entry *fa; + struct efx_rep *from_efv, *to_efv; + struct efx_tc_match match; + s64 rc; + int i; + + if (!tc_can_offload_extack(efx->net_dev, extack)) + return -EOPNOTSUPP; + if (WARN_ON(!efx->tc)) + return -ENETDOWN; + if (WARN_ON(!efx->tc->up)) + return -ENETDOWN; + + from_efv = efx_tc_flower_lookup_efv(efx, net_dev); + if (IS_ERR(from_efv)) { + /* Might be a tunnel decap rule from an indirect block. + * Support for those not implemented yet. + */ + return -EOPNOTSUPP; + } + + if (efv != from_efv) { + /* can't happen */ + efx_tc_err(efx, "for %s efv is %snull but from_efv is %snull\n", + netdev_name(net_dev), efv ? "non-" : "", + from_efv ? "non-" : ""); + if (efv) + NL_SET_ERR_MSG_MOD(extack, "vfrep filter has PF net_dev (can't happen)"); + else + NL_SET_ERR_MSG_MOD(extack, "PF filter has vfrep net_dev (can't happen)"); + return -EINVAL; + } + + /* Parse match */ + memset(&match, 0, sizeof(match)); + rc = efx_tc_flower_external_mport(efx, from_efv); + if (rc < 0) { + EFX_TC_ERR_MSG(efx, extack, "Failed to identify ingress m-port"); + return rc; + } + match.value.ingress_port = rc; + match.mask.ingress_port = ~0; + rc = efx_tc_flower_parse_match(efx, fr, &match, extack); + if (rc) + return rc; + + if (tc->common.chain_index) { + EFX_TC_ERR_MSG(efx, extack, "No support for nonzero chain_index"); + return -EOPNOTSUPP; + } + match.mask.recirc_id = 0xff; + + rc = efx_mae_match_check_caps(efx, &match.mask, extack); + if (rc) + return rc; + + rule = kzalloc(sizeof(*rule), GFP_USER); + if (!rule) + return -ENOMEM; + INIT_LIST_HEAD(&rule->acts.list); + rule->cookie = tc->cookie; + old = rhashtable_lookup_get_insert_fast(&efx->tc->match_action_ht, + &rule->linkage, + efx_tc_match_action_ht_params); + if (old) { + netif_dbg(efx, drv, efx->net_dev, + "Already offloaded rule (cookie %lx)\n", tc->cookie); + rc = -EEXIST; + NL_SET_ERR_MSG_MOD(extack, "Rule already offloaded"); + goto release; + } + + /* Parse actions */ + act = kzalloc(sizeof(*act), GFP_USER); + if (!act) { + rc = -ENOMEM; + goto release; + } + + flow_action_for_each(i, fa, &fr->action) { + struct efx_tc_action_set save; + + if (!act) { + /* more actions after a non-pipe action */ + EFX_TC_ERR_MSG(efx, extack, "Action follows non-pipe action"); + rc = -EINVAL; + goto release; + } + + switch (fa->id) { + case FLOW_ACTION_DROP: + rc = efx_mae_alloc_action_set(efx, act); + if (rc) { + EFX_TC_ERR_MSG(efx, extack, "Failed to write action set to hw (drop)"); + goto release; + } + list_add_tail(&act->list, &rule->acts.list); + act = NULL; /* end of the line */ + break; + case FLOW_ACTION_REDIRECT: + case FLOW_ACTION_MIRRED: + save = *act; + to_efv = efx_tc_flower_lookup_efv(efx, fa->dev); + if (IS_ERR(to_efv)) { + EFX_TC_ERR_MSG(efx, extack, "Mirred egress device not on switch"); + rc = PTR_ERR(to_efv); + goto release; + } + rc = efx_tc_flower_external_mport(efx, to_efv); + if (rc < 0) { + EFX_TC_ERR_MSG(efx, extack, "Failed to identify egress m-port"); + goto release; + } + act->dest_mport = rc; + act->deliver = 1; + rc = efx_mae_alloc_action_set(efx, act); + if (rc) { + EFX_TC_ERR_MSG(efx, extack, "Failed to write action set to hw (mirred)"); + goto release; + } + list_add_tail(&act->list, &rule->acts.list); + act = NULL; + if (fa->id == FLOW_ACTION_REDIRECT) + break; /* end of the line */ + /* Mirror, so continue on with saved act */ + act = kzalloc(sizeof(*act), GFP_USER); + if (!act) { + rc = -ENOMEM; + goto release; + } + *act = save; + break; + default: + efx_tc_err(efx, "Unhandled action %u\n", fa->id); + rc = -EOPNOTSUPP; + NL_SET_ERR_MSG_MOD(extack, "Unsupported action"); + goto release; + } + } + + if (act) { + /* Not shot/redirected, so deliver to default dest */ + if (from_efv == EFX_EFV_PF) + /* Rule applies to traffic from the wire, + * and default dest is thus the PF + */ + efx_mae_mport_uplink(efx, &act->dest_mport); + else + /* Representor, so rule applies to traffic from + * representee, and default dest is thus the rep. + * All reps use the same mport for delivery + */ + efx_mae_mport_mport(efx, efx->tc->reps_mport_id, + &act->dest_mport); + act->deliver = 1; + rc = efx_mae_alloc_action_set(efx, act); + if (rc) { + EFX_TC_ERR_MSG(efx, extack, "Failed to write action set to hw (deliver)"); + goto release; + } + list_add_tail(&act->list, &rule->acts.list); + act = NULL; /* Prevent double-free in error path */ + } + + netif_dbg(efx, drv, efx->net_dev, + "Successfully parsed filter (cookie %lx)\n", + tc->cookie); + + rule->match = match; + + rc = efx_mae_alloc_action_set_list(efx, &rule->acts); + if (rc) { + EFX_TC_ERR_MSG(efx, extack, "Failed to write action set list to hw"); + goto release; + } + rc = efx_mae_insert_rule(efx, &rule->match, EFX_TC_PRIO_TC, + rule->acts.fw_id, &rule->fw_id); + if (rc) { + EFX_TC_ERR_MSG(efx, extack, "Failed to insert rule in hw"); + goto release_acts; + } + return 0; + +release_acts: + efx_mae_free_action_set_list(efx, &rule->acts); +release: + /* We failed to insert the rule, so free up any entries we created in + * subsidiary tables. + */ + if (act) + efx_tc_free_action_set(efx, act, false); + if (rule) { + rhashtable_remove_fast(&efx->tc->match_action_ht, + &rule->linkage, + efx_tc_match_action_ht_params); + efx_tc_free_action_set_list(efx, &rule->acts, false); + } + kfree(rule); + return rc; +} + +static int efx_tc_flower_destroy(struct efx_nic *efx, + struct net_device *net_dev, + struct flow_cls_offload *tc) +{ + struct netlink_ext_ack *extack = tc->common.extack; + struct efx_tc_flow_rule *rule; + + rule = rhashtable_lookup_fast(&efx->tc->match_action_ht, &tc->cookie, + efx_tc_match_action_ht_params); + if (!rule) { + /* Only log a message if we're the ingress device. Otherwise + * it's a foreign filter and we might just not have been + * interested (e.g. we might not have been the egress device + * either). + */ + if (!IS_ERR(efx_tc_flower_lookup_efv(efx, net_dev))) + netif_warn(efx, drv, efx->net_dev, + "Filter %lx not found to remove\n", tc->cookie); + NL_SET_ERR_MSG_MOD(extack, "Flow cookie not found in offloaded rules"); + return -ENOENT; + } + + /* Remove it from HW */ + efx_tc_delete_rule(efx, rule); + /* Delete it from SW */ + rhashtable_remove_fast(&efx->tc->match_action_ht, &rule->linkage, + efx_tc_match_action_ht_params); + netif_dbg(efx, drv, efx->net_dev, "Removed filter %lx\n", rule->cookie); + kfree(rule); + return 0; +} + +int efx_tc_flower(struct efx_nic *efx, struct net_device *net_dev, + struct flow_cls_offload *tc, struct efx_rep *efv) +{ + int rc; + + if (!efx->tc) + return -EOPNOTSUPP; + + mutex_lock(&efx->tc->mutex); + switch (tc->command) { + case FLOW_CLS_REPLACE: + rc = efx_tc_flower_replace(efx, net_dev, tc, efv); + break; + case FLOW_CLS_DESTROY: + rc = efx_tc_flower_destroy(efx, net_dev, tc); + break; + default: + rc = -EOPNOTSUPP; + break; + } + mutex_unlock(&efx->tc->mutex); + return rc; +} + static int efx_tc_configure_default_rule(struct efx_nic *efx, u32 ing_port, u32 eg_port, struct efx_tc_flow_rule *rule) { @@ -201,13 +577,37 @@ int efx_init_tc(struct efx_nic *efx) { int rc; + rc = efx_mae_get_caps(efx, efx->tc->caps); + if (rc) + return rc; + if (efx->tc->caps->match_field_count > MAE_NUM_FIELDS) + /* Firmware supports some match fields the driver doesn't know + * about. Not fatal, unless any of those fields are required + * (MAE_FIELD_SUPPORTED_MATCH_ALWAYS) but if so we don't know. + */ + netif_warn(efx, probe, efx->net_dev, + "FW reports additional match fields %u\n", + efx->tc->caps->match_field_count); + if (efx->tc->caps->action_prios < EFX_TC_PRIO__NUM) { + netif_err(efx, probe, efx->net_dev, + "Too few action prios supported (have %u, need %u)\n", + efx->tc->caps->action_prios, EFX_TC_PRIO__NUM); + return -EIO; + } rc = efx_tc_configure_default_rule_pf(efx); if (rc) return rc; rc = efx_tc_configure_default_rule_wire(efx); if (rc) return rc; - return efx_tc_configure_rep_mport(efx); + rc = efx_tc_configure_rep_mport(efx); + if (rc) + return rc; + efx->tc->up = true; + rc = flow_indr_dev_register(efx_tc_indr_setup_cb, efx); + if (rc) + return rc; + return 0; } void efx_fini_tc(struct efx_nic *efx) @@ -215,20 +615,35 @@ void efx_fini_tc(struct efx_nic *efx) /* We can get called even if efx_init_struct_tc() failed */ if (!efx->tc) return; + if (efx->tc->up) + flow_indr_dev_unregister(efx_tc_indr_setup_cb, efx, efx_tc_block_unbind); efx_tc_deconfigure_rep_mport(efx); efx_tc_deconfigure_default_rule(efx, &efx->tc->dflt.pf); efx_tc_deconfigure_default_rule(efx, &efx->tc->dflt.wire); + efx->tc->up = false; } int efx_init_struct_tc(struct efx_nic *efx) { + int rc; + if (efx->type->is_vf) return 0; efx->tc = kzalloc(sizeof(*efx->tc), GFP_KERNEL); if (!efx->tc) return -ENOMEM; + efx->tc->caps = kzalloc(sizeof(struct mae_caps), GFP_KERNEL); + if (!efx->tc->caps) { + rc = -ENOMEM; + goto fail_alloc_caps; + } + INIT_LIST_HEAD(&efx->tc->block_list); + mutex_init(&efx->tc->mutex); + rc = rhashtable_init(&efx->tc->match_action_ht, &efx_tc_match_action_ht_params); + if (rc < 0) + goto fail_match_action_ht; efx->tc->reps_filter_uc = -1; efx->tc->reps_filter_mc = -1; INIT_LIST_HEAD(&efx->tc->dflt.pf.acts.list); @@ -236,6 +651,13 @@ int efx_init_struct_tc(struct efx_nic *efx) INIT_LIST_HEAD(&efx->tc->dflt.wire.acts.list); efx->tc->dflt.wire.fw_id = MC_CMD_MAE_ACTION_RULE_INSERT_OUT_ACTION_RULE_ID_NULL; return 0; +fail_match_action_ht: + mutex_destroy(&efx->tc->mutex); + kfree(efx->tc->caps); +fail_alloc_caps: + kfree(efx->tc); + efx->tc = NULL; + return rc; } void efx_fini_struct_tc(struct efx_nic *efx) @@ -243,10 +665,16 @@ void efx_fini_struct_tc(struct efx_nic *efx) if (!efx->tc) return; + mutex_lock(&efx->tc->mutex); EFX_WARN_ON_PARANOID(efx->tc->dflt.pf.fw_id != MC_CMD_MAE_ACTION_RULE_INSERT_OUT_ACTION_RULE_ID_NULL); EFX_WARN_ON_PARANOID(efx->tc->dflt.wire.fw_id != MC_CMD_MAE_ACTION_RULE_INSERT_OUT_ACTION_RULE_ID_NULL); + rhashtable_free_and_destroy(&efx->tc->match_action_ht, efx_tc_flow_free, + efx); + mutex_unlock(&efx->tc->mutex); + mutex_destroy(&efx->tc->mutex); + kfree(efx->tc->caps); kfree(efx->tc); efx->tc = NULL; } diff --git a/drivers/net/ethernet/sfc/tc.h b/drivers/net/ethernet/sfc/tc.h index 309123c6b386..196fd74ed973 100644 --- a/drivers/net/ethernet/sfc/tc.h +++ b/drivers/net/ethernet/sfc/tc.h @@ -11,8 +11,28 @@ #ifndef EFX_TC_H #define EFX_TC_H +#include <net/flow_offload.h> +#include <linux/rhashtable.h> #include "net_driver.h" +/* Error reporting: convenience macros. For indicating why a given filter + * insertion is not supported; errors in internal operation or in the + * hardware should be netif_err()s instead. + */ +/* Used when error message is constant. */ +#define EFX_TC_ERR_MSG(efx, extack, message) do { \ + NL_SET_ERR_MSG_MOD(extack, message); \ + if (efx->log_tc_errs) \ + netif_info(efx, drv, efx->net_dev, "%s\n", message); \ +} while (0) +/* Used when error message is not constant; caller should also supply a + * constant extack message with NL_SET_ERR_MSG_MOD(). + */ +#define efx_tc_err(efx, fmt, args...) do { \ +if (efx->log_tc_errs) \ + netif_info(efx, drv, efx->net_dev, fmt, ##args);\ +} while (0) + struct efx_tc_action_set { u16 deliver:1; u32 dest_mport; @@ -23,6 +43,7 @@ struct efx_tc_action_set { struct efx_tc_match_fields { /* L1 */ u32 ingress_port; + u8 recirc_id; }; struct efx_tc_match { @@ -36,12 +57,15 @@ struct efx_tc_action_set_list { }; struct efx_tc_flow_rule { + unsigned long cookie; + struct rhash_head linkage; struct efx_tc_match match; struct efx_tc_action_set_list acts; u32 fw_id; }; enum efx_tc_rule_prios { + EFX_TC_PRIO_TC, /* Rule inserted by TC */ EFX_TC_PRIO_DFLT, /* Default switch rule; one of efx_tc_default_rules */ EFX_TC_PRIO__NUM }; @@ -49,6 +73,10 @@ enum efx_tc_rule_prios { /** * struct efx_tc_state - control plane data for TC offload * + * @caps: MAE capabilities reported by MCDI + * @block_list: List of &struct efx_tc_block_binding + * @mutex: Used to serialise operations on TC hashtables + * @match_action_ht: Hashtable of TC match-action rules * @reps_mport_id: MAE port allocated for representor RX * @reps_filter_uc: VNIC filter for representor unicast RX (promisc) * @reps_filter_mc: VNIC filter for representor multicast RX (allmulti) @@ -57,14 +85,20 @@ enum efx_tc_rule_prios { * %EFX_TC_PRIO_DFLT. Named by *ingress* port * @dflt.pf: rule for traffic ingressing from PF (egresses to wire) * @dflt.wire: rule for traffic ingressing from wire (egresses to PF) + * @up: have TC datastructures been set up? */ struct efx_tc_state { + struct mae_caps *caps; + struct list_head block_list; + struct mutex mutex; + struct rhashtable match_action_ht; u32 reps_mport_id, reps_mport_vport_id; s32 reps_filter_uc, reps_filter_mc; struct { struct efx_tc_flow_rule pf; struct efx_tc_flow_rule wire; } dflt; + bool up; }; struct efx_rep; @@ -72,6 +106,8 @@ struct efx_rep; int efx_tc_configure_default_rule_rep(struct efx_rep *efv); void efx_tc_deconfigure_default_rule(struct efx_nic *efx, struct efx_tc_flow_rule *rule); +int efx_tc_flower(struct efx_nic *efx, struct net_device *net_dev, + struct flow_cls_offload *tc, struct efx_rep *efv); int efx_tc_insert_rep_filters(struct efx_nic *efx); void efx_tc_remove_rep_filters(struct efx_nic *efx); diff --git a/drivers/net/ethernet/sfc/tc_bindings.c b/drivers/net/ethernet/sfc/tc_bindings.c new file mode 100644 index 000000000000..c18d64519c2d --- /dev/null +++ b/drivers/net/ethernet/sfc/tc_bindings.c @@ -0,0 +1,228 @@ +// SPDX-License-Identifier: GPL-2.0-only +/**************************************************************************** + * Driver for Solarflare network controllers and boards + * Copyright 2022 Xilinx Inc. + * + * This program is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 as published + * by the Free Software Foundation, incorporated herein by reference. + */ + +#include "tc_bindings.h" +#include "tc.h" + +struct efx_tc_block_binding { + struct list_head list; + struct efx_nic *efx; + struct efx_rep *efv; + struct net_device *otherdev; /* may actually be us */ + struct flow_block *block; +}; + +static struct efx_tc_block_binding *efx_tc_find_binding(struct efx_nic *efx, + struct net_device *otherdev) +{ + struct efx_tc_block_binding *binding; + + ASSERT_RTNL(); + list_for_each_entry(binding, &efx->tc->block_list, list) + if (binding->otherdev == otherdev) + return binding; + return NULL; +} + +static int efx_tc_block_cb(enum tc_setup_type type, void *type_data, + void *cb_priv) +{ + struct efx_tc_block_binding *binding = cb_priv; + struct flow_cls_offload *tcf = type_data; + + switch (type) { + case TC_SETUP_CLSFLOWER: + return efx_tc_flower(binding->efx, binding->otherdev, + tcf, binding->efv); + default: + return -EOPNOTSUPP; + } +} + +void efx_tc_block_unbind(void *cb_priv) +{ + struct efx_tc_block_binding *binding = cb_priv; + + list_del(&binding->list); + kfree(binding); +} + +static struct efx_tc_block_binding *efx_tc_create_binding( + struct efx_nic *efx, struct efx_rep *efv, + struct net_device *otherdev, struct flow_block *block) +{ + struct efx_tc_block_binding *binding = kmalloc(sizeof(*binding), GFP_KERNEL); + + if (!binding) + return ERR_PTR(-ENOMEM); + binding->efx = efx; + binding->efv = efv; + binding->otherdev = otherdev; + binding->block = block; + list_add(&binding->list, &efx->tc->block_list); + return binding; +} + +int efx_tc_setup_block(struct net_device *net_dev, struct efx_nic *efx, + struct flow_block_offload *tcb, struct efx_rep *efv) +{ + struct efx_tc_block_binding *binding; + struct flow_block_cb *block_cb; + int rc; + + if (tcb->binder_type != FLOW_BLOCK_BINDER_TYPE_CLSACT_INGRESS) + return -EOPNOTSUPP; + + if (WARN_ON(!efx->tc)) + return -ENETDOWN; + + switch (tcb->command) { + case FLOW_BLOCK_BIND: + binding = efx_tc_create_binding(efx, efv, net_dev, tcb->block); + if (IS_ERR(binding)) + return PTR_ERR(binding); + block_cb = flow_block_cb_alloc(efx_tc_block_cb, binding, + binding, efx_tc_block_unbind); + rc = PTR_ERR_OR_ZERO(block_cb); + netif_dbg(efx, drv, efx->net_dev, + "bind %sdirect block for device %s, rc %d\n", + net_dev == efx->net_dev ? "" : + efv ? "semi" : "in", + net_dev ? net_dev->name : NULL, rc); + if (rc) { + list_del(&binding->list); + kfree(binding); + } else { + flow_block_cb_add(block_cb, tcb); + } + return rc; + case FLOW_BLOCK_UNBIND: + binding = efx_tc_find_binding(efx, net_dev); + if (binding) { + block_cb = flow_block_cb_lookup(tcb->block, + efx_tc_block_cb, + binding); + if (block_cb) { + flow_block_cb_remove(block_cb, tcb); + netif_dbg(efx, drv, efx->net_dev, + "unbound %sdirect block for device %s\n", + net_dev == efx->net_dev ? "" : + binding->efv ? "semi" : "in", + net_dev ? net_dev->name : NULL); + return 0; + } + } + /* If we're in driver teardown, then we expect to have + * already unbound all our blocks (we did it early while + * we still had MCDI to remove the filters), so getting + * unbind callbacks now isn't a problem. + */ + netif_cond_dbg(efx, drv, efx->net_dev, + !efx->tc->up, warn, + "%sdirect block unbind for device %s, was never bound\n", + net_dev == efx->net_dev ? "" : "in", + net_dev ? net_dev->name : NULL); + return -ENOENT; + default: + return -EOPNOTSUPP; + } +} + +int efx_tc_indr_setup_cb(struct net_device *net_dev, struct Qdisc *sch, + void *cb_priv, enum tc_setup_type type, + void *type_data, void *data, + void (*cleanup)(struct flow_block_cb *block_cb)) +{ + struct flow_block_offload *tcb = type_data; + struct efx_tc_block_binding *binding; + struct flow_block_cb *block_cb; + struct efx_nic *efx = cb_priv; + bool is_ovs_int_port; + int rc; + + if (!net_dev) + return -EOPNOTSUPP; + + if (tcb->binder_type != FLOW_BLOCK_BINDER_TYPE_CLSACT_INGRESS && + tcb->binder_type != FLOW_BLOCK_BINDER_TYPE_CLSACT_EGRESS) + return -EOPNOTSUPP; + + is_ovs_int_port = netif_is_ovs_master(net_dev); + if (tcb->binder_type == FLOW_BLOCK_BINDER_TYPE_CLSACT_EGRESS && + !is_ovs_int_port) + return -EOPNOTSUPP; + + if (is_ovs_int_port) + return -EOPNOTSUPP; + + switch (type) { + case TC_SETUP_BLOCK: + switch (tcb->command) { + case FLOW_BLOCK_BIND: + binding = efx_tc_create_binding(efx, NULL, net_dev, tcb->block); + if (IS_ERR(binding)) + return PTR_ERR(binding); + block_cb = flow_indr_block_cb_alloc(efx_tc_block_cb, binding, + binding, efx_tc_block_unbind, + tcb, net_dev, sch, data, binding, + cleanup); + rc = PTR_ERR_OR_ZERO(block_cb); + netif_dbg(efx, drv, efx->net_dev, + "bind indr block for device %s, rc %d\n", + net_dev ? net_dev->name : NULL, rc); + if (rc) { + list_del(&binding->list); + kfree(binding); + } else { + flow_block_cb_add(block_cb, tcb); + } + return rc; + case FLOW_BLOCK_UNBIND: + binding = efx_tc_find_binding(efx, net_dev); + if (!binding) + return -ENOENT; + block_cb = flow_block_cb_lookup(tcb->block, + efx_tc_block_cb, + binding); + if (!block_cb) + return -ENOENT; + flow_indr_block_cb_remove(block_cb, tcb); + netif_dbg(efx, drv, efx->net_dev, + "unbind indr block for device %s\n", + net_dev ? net_dev->name : NULL); + return 0; + default: + return -EOPNOTSUPP; + } + default: + return -EOPNOTSUPP; + } +} + +/* .ndo_setup_tc implementation + * Entry point for flower block and filter management. + */ +int efx_tc_setup(struct net_device *net_dev, enum tc_setup_type type, + void *type_data) +{ + struct efx_nic *efx = efx_netdev_priv(net_dev); + + if (efx->type->is_vf) + return -EOPNOTSUPP; + if (!efx->tc) + return -EOPNOTSUPP; + + if (type == TC_SETUP_CLSFLOWER) + return efx_tc_flower(efx, net_dev, type_data, NULL); + if (type == TC_SETUP_BLOCK) + return efx_tc_setup_block(net_dev, efx, type_data, NULL); + + return -EOPNOTSUPP; +} diff --git a/drivers/net/ethernet/sfc/tc_bindings.h b/drivers/net/ethernet/sfc/tc_bindings.h new file mode 100644 index 000000000000..c210bb09150e --- /dev/null +++ b/drivers/net/ethernet/sfc/tc_bindings.h @@ -0,0 +1,29 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/**************************************************************************** + * Driver for Solarflare network controllers and boards + * Copyright 2022 Xilinx Inc. + * + * This program is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 as published + * by the Free Software Foundation, incorporated herein by reference. + */ + +#ifndef EFX_TC_BINDINGS_H +#define EFX_TC_BINDINGS_H +#include "net_driver.h" + +#include <net/sch_generic.h> + +struct efx_rep; + +void efx_tc_block_unbind(void *cb_priv); +int efx_tc_setup_block(struct net_device *net_dev, struct efx_nic *efx, + struct flow_block_offload *tcb, struct efx_rep *efv); +int efx_tc_setup(struct net_device *net_dev, enum tc_setup_type type, + void *type_data); + +int efx_tc_indr_setup_cb(struct net_device *net_dev, struct Qdisc *sch, + void *cb_priv, enum tc_setup_type type, + void *type_data, void *data, + void (*cleanup)(struct flow_block_cb *block_cb)); +#endif /* EFX_TC_BINDINGS_H */ diff --git a/drivers/net/ethernet/sgi/ioc3-eth.c b/drivers/net/ethernet/sgi/ioc3-eth.c index e2d009866a7b..8fc3f5272fa7 100644 --- a/drivers/net/ethernet/sgi/ioc3-eth.c +++ b/drivers/net/ethernet/sgi/ioc3-eth.c @@ -1158,9 +1158,9 @@ static inline unsigned int ioc3_hash(const unsigned char *addr) static void ioc3_get_drvinfo(struct net_device *dev, struct ethtool_drvinfo *info) { - strlcpy(info->driver, IOC3_NAME, sizeof(info->driver)); - strlcpy(info->version, IOC3_VERSION, sizeof(info->version)); - strlcpy(info->bus_info, pci_name(to_pci_dev(dev->dev.parent)), + strscpy(info->driver, IOC3_NAME, sizeof(info->driver)); + strscpy(info->version, IOC3_VERSION, sizeof(info->version)); + strscpy(info->bus_info, pci_name(to_pci_dev(dev->dev.parent)), sizeof(info->bus_info)); } diff --git a/drivers/net/ethernet/sis/sis190.c b/drivers/net/ethernet/sis/sis190.c index 216bb2d34d7c..dda4e488c77a 100644 --- a/drivers/net/ethernet/sis/sis190.c +++ b/drivers/net/ethernet/sis/sis190.c @@ -1769,9 +1769,9 @@ static void sis190_get_drvinfo(struct net_device *dev, { struct sis190_private *tp = netdev_priv(dev); - strlcpy(info->driver, DRV_NAME, sizeof(info->driver)); - strlcpy(info->version, DRV_VERSION, sizeof(info->version)); - strlcpy(info->bus_info, pci_name(tp->pci_dev), + strscpy(info->driver, DRV_NAME, sizeof(info->driver)); + strscpy(info->version, DRV_VERSION, sizeof(info->version)); + strscpy(info->bus_info, pci_name(tp->pci_dev), sizeof(info->bus_info)); } diff --git a/drivers/net/ethernet/sis/sis900.c b/drivers/net/ethernet/sis/sis900.c index 23a336c5096e..cb7fec226cab 100644 --- a/drivers/net/ethernet/sis/sis900.c +++ b/drivers/net/ethernet/sis/sis900.c @@ -2027,9 +2027,9 @@ static void sis900_get_drvinfo(struct net_device *net_dev, { struct sis900_private *sis_priv = netdev_priv(net_dev); - strlcpy(info->driver, SIS900_MODULE_NAME, sizeof(info->driver)); - strlcpy(info->version, SIS900_DRV_VERSION, sizeof(info->version)); - strlcpy(info->bus_info, pci_name(sis_priv->pci_dev), + strscpy(info->driver, SIS900_MODULE_NAME, sizeof(info->driver)); + strscpy(info->version, SIS900_DRV_VERSION, sizeof(info->version)); + strscpy(info->bus_info, pci_name(sis_priv->pci_dev), sizeof(info->bus_info)); } diff --git a/drivers/net/ethernet/smsc/epic100.c b/drivers/net/ethernet/smsc/epic100.c index 0329caf63279..013e90d69182 100644 --- a/drivers/net/ethernet/smsc/epic100.c +++ b/drivers/net/ethernet/smsc/epic100.c @@ -482,7 +482,7 @@ static int epic_init_one(struct pci_dev *pdev, const struct pci_device_id *ent) dev->netdev_ops = &epic_netdev_ops; dev->ethtool_ops = &netdev_ethtool_ops; dev->watchdog_timeo = TX_TIMEOUT; - netif_napi_add(dev, &ep->napi, epic_poll, 64); + netif_napi_add(dev, &ep->napi, epic_poll); ret = register_netdev(dev); if (ret < 0) @@ -1392,9 +1392,9 @@ static void netdev_get_drvinfo (struct net_device *dev, struct ethtool_drvinfo * { struct epic_private *np = netdev_priv(dev); - strlcpy(info->driver, DRV_NAME, sizeof(info->driver)); - strlcpy(info->version, DRV_VERSION, sizeof(info->version)); - strlcpy(info->bus_info, pci_name(np->pci_dev), sizeof(info->bus_info)); + strscpy(info->driver, DRV_NAME, sizeof(info->driver)); + strscpy(info->version, DRV_VERSION, sizeof(info->version)); + strscpy(info->bus_info, pci_name(np->pci_dev), sizeof(info->bus_info)); } static int netdev_get_link_ksettings(struct net_device *dev, diff --git a/drivers/net/ethernet/smsc/smc911x.c b/drivers/net/ethernet/smsc/smc911x.c index 24d66af797d4..52ecfb461c41 100644 --- a/drivers/net/ethernet/smsc/smc911x.c +++ b/drivers/net/ethernet/smsc/smc911x.c @@ -1509,9 +1509,9 @@ smc911x_ethtool_set_link_ksettings(struct net_device *dev, static void smc911x_ethtool_getdrvinfo(struct net_device *dev, struct ethtool_drvinfo *info) { - strlcpy(info->driver, CARDNAME, sizeof(info->driver)); - strlcpy(info->version, version, sizeof(info->version)); - strlcpy(info->bus_info, dev_name(dev->dev.parent), + strscpy(info->driver, CARDNAME, sizeof(info->driver)); + strscpy(info->version, version, sizeof(info->version)); + strscpy(info->bus_info, dev_name(dev->dev.parent), sizeof(info->bus_info)); } diff --git a/drivers/net/ethernet/smsc/smc91c92_cs.c b/drivers/net/ethernet/smsc/smc91c92_cs.c index 37c822e27207..29bb19f42de9 100644 --- a/drivers/net/ethernet/smsc/smc91c92_cs.c +++ b/drivers/net/ethernet/smsc/smc91c92_cs.c @@ -1909,8 +1909,8 @@ static int check_if_running(struct net_device *dev) static void smc_get_drvinfo(struct net_device *dev, struct ethtool_drvinfo *info) { - strlcpy(info->driver, DRV_NAME, sizeof(info->driver)); - strlcpy(info->version, DRV_VERSION, sizeof(info->version)); + strscpy(info->driver, DRV_NAME, sizeof(info->driver)); + strscpy(info->version, DRV_VERSION, sizeof(info->version)); } static int smc_get_link_ksettings(struct net_device *dev, diff --git a/drivers/net/ethernet/smsc/smc91x.c b/drivers/net/ethernet/smsc/smc91x.c index a31c159e96ea..35e99bf0c401 100644 --- a/drivers/net/ethernet/smsc/smc91x.c +++ b/drivers/net/ethernet/smsc/smc91x.c @@ -1588,9 +1588,9 @@ smc_ethtool_set_link_ksettings(struct net_device *dev, static void smc_ethtool_getdrvinfo(struct net_device *dev, struct ethtool_drvinfo *info) { - strlcpy(info->driver, CARDNAME, sizeof(info->driver)); - strlcpy(info->version, version, sizeof(info->version)); - strlcpy(info->bus_info, dev_name(dev->dev.parent), + strscpy(info->driver, CARDNAME, sizeof(info->driver)); + strscpy(info->version, version, sizeof(info->version)); + strscpy(info->bus_info, dev_name(dev->dev.parent), sizeof(info->bus_info)); } diff --git a/drivers/net/ethernet/smsc/smsc911x.c b/drivers/net/ethernet/smsc/smsc911x.c index 3829c2805b16..a2e511912e6a 100644 --- a/drivers/net/ethernet/smsc/smsc911x.c +++ b/drivers/net/ethernet/smsc/smsc911x.c @@ -1955,9 +1955,9 @@ static int smsc911x_set_mac_address(struct net_device *dev, void *p) static void smsc911x_ethtool_getdrvinfo(struct net_device *dev, struct ethtool_drvinfo *info) { - strlcpy(info->driver, SMSC_CHIPNAME, sizeof(info->driver)); - strlcpy(info->version, SMSC_DRV_VERSION, sizeof(info->version)); - strlcpy(info->bus_info, dev_name(dev->dev.parent), + strscpy(info->driver, SMSC_CHIPNAME, sizeof(info->driver)); + strscpy(info->version, SMSC_DRV_VERSION, sizeof(info->version)); + strscpy(info->bus_info, dev_name(dev->dev.parent), sizeof(info->bus_info)); } diff --git a/drivers/net/ethernet/smsc/smsc9420.c b/drivers/net/ethernet/smsc/smsc9420.c index 0c68c7f8056d..71fbb358bb7d 100644 --- a/drivers/net/ethernet/smsc/smsc9420.c +++ b/drivers/net/ethernet/smsc/smsc9420.c @@ -215,10 +215,10 @@ static void smsc9420_ethtool_get_drvinfo(struct net_device *netdev, { struct smsc9420_pdata *pd = netdev_priv(netdev); - strlcpy(drvinfo->driver, DRV_NAME, sizeof(drvinfo->driver)); - strlcpy(drvinfo->bus_info, pci_name(pd->pdev), + strscpy(drvinfo->driver, DRV_NAME, sizeof(drvinfo->driver)); + strscpy(drvinfo->bus_info, pci_name(pd->pdev), sizeof(drvinfo->bus_info)); - strlcpy(drvinfo->version, DRV_VERSION, sizeof(drvinfo->version)); + strscpy(drvinfo->version, DRV_VERSION, sizeof(drvinfo->version)); } static u32 smsc9420_ethtool_get_msglevel(struct net_device *netdev) @@ -1585,7 +1585,7 @@ smsc9420_probe(struct pci_dev *pdev, const struct pci_device_id *id) dev->netdev_ops = &smsc9420_netdev_ops; dev->ethtool_ops = &smsc9420_ethtool_ops; - netif_napi_add(dev, &pd->napi, smsc9420_rx_poll, NAPI_POLL_WEIGHT); + netif_napi_add(dev, &pd->napi, smsc9420_rx_poll); result = register_netdev(dev); if (result) { diff --git a/drivers/net/ethernet/socionext/netsec.c b/drivers/net/ethernet/socionext/netsec.c index b0c5a44785fa..2240f6d0b89b 100644 --- a/drivers/net/ethernet/socionext/netsec.c +++ b/drivers/net/ethernet/socionext/netsec.c @@ -526,8 +526,8 @@ static int netsec_phy_read(struct mii_bus *bus, int phy_addr, int reg_addr) static void netsec_et_get_drvinfo(struct net_device *net_device, struct ethtool_drvinfo *info) { - strlcpy(info->driver, "netsec", sizeof(info->driver)); - strlcpy(info->bus_info, dev_name(net_device->dev.parent), + strscpy(info->driver, "netsec", sizeof(info->driver)); + strscpy(info->bus_info, dev_name(net_device->dev.parent), sizeof(info->bus_info)); } @@ -2093,7 +2093,7 @@ static int netsec_probe(struct platform_device *pdev) dev_info(&pdev->dev, "hardware revision %d.%d\n", hw_ver >> 16, hw_ver & 0xffff); - netif_napi_add(ndev, &priv->napi, netsec_napi_poll, NAPI_POLL_WEIGHT); + netif_napi_add(ndev, &priv->napi, netsec_napi_poll); ndev->netdev_ops = &netsec_netdev_ops; ndev->ethtool_ops = &netsec_ethtool_ops; diff --git a/drivers/net/ethernet/socionext/sni_ave.c b/drivers/net/ethernet/socionext/sni_ave.c index f0c8de2c6075..1fa09b49ba7f 100644 --- a/drivers/net/ethernet/socionext/sni_ave.c +++ b/drivers/net/ethernet/socionext/sni_ave.c @@ -395,8 +395,8 @@ static void ave_ethtool_get_drvinfo(struct net_device *ndev, { struct device *dev = ndev->dev.parent; - strlcpy(info->driver, dev->driver->name, sizeof(info->driver)); - strlcpy(info->bus_info, dev_name(dev), sizeof(info->bus_info)); + strscpy(info->driver, dev->driver->name, sizeof(info->driver)); + strscpy(info->bus_info, dev_name(dev), sizeof(info->bus_info)); ave_hw_read_version(ndev, info->fw_version, sizeof(info->fw_version)); } @@ -1687,8 +1687,7 @@ static int ave_probe(struct platform_device *pdev) pdev->name, pdev->id); /* Register as a NAPI supported driver */ - netif_napi_add(ndev, &priv->napi_rx, ave_napi_poll_rx, - NAPI_POLL_WEIGHT); + netif_napi_add(ndev, &priv->napi_rx, ave_napi_poll_rx); netif_napi_add_tx(ndev, &priv->napi_tx, ave_napi_poll_tx); platform_set_drvdata(pdev, ndev); diff --git a/drivers/net/ethernet/stmicro/stmmac/dwmac-dwc-qos-eth.c b/drivers/net/ethernet/stmicro/stmmac/dwmac-dwc-qos-eth.c index 358fc26f8d1f..80efdeeb0b59 100644 --- a/drivers/net/ethernet/stmicro/stmmac/dwmac-dwc-qos-eth.c +++ b/drivers/net/ethernet/stmicro/stmmac/dwmac-dwc-qos-eth.c @@ -445,9 +445,7 @@ static int dwc_eth_dwmac_probe(struct platform_device *pdev) ret = data->probe(pdev, plat_dat, &stmmac_res); if (ret < 0) { - if (ret != -EPROBE_DEFER) - dev_err(&pdev->dev, "failed to probe subdriver: %d\n", - ret); + dev_err_probe(&pdev->dev, ret, "failed to probe subdriver\n"); goto remove_config; } diff --git a/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.c b/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.c index 9af25be42401..0a2afc1a3124 100644 --- a/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.c +++ b/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.c @@ -610,7 +610,6 @@ static int intel_mgbe_common_data(struct pci_dev *pdev, plat->int_snapshot_num = AUX_SNAPSHOT1; plat->ext_snapshot_num = AUX_SNAPSHOT0; - plat->has_crossts = true; plat->crosststamp = intel_crosststamp; plat->int_snapshot_en = 0; diff --git a/drivers/net/ethernet/stmicro/stmmac/dwmac-rk.c b/drivers/net/ethernet/stmicro/stmmac/dwmac-rk.c index c469abc91fa1..f7269d79a385 100644 --- a/drivers/net/ethernet/stmicro/stmmac/dwmac-rk.c +++ b/drivers/net/ethernet/stmicro/stmmac/dwmac-rk.c @@ -32,6 +32,8 @@ struct rk_gmac_ops { void (*set_to_rmii)(struct rk_priv_data *bsp_priv); void (*set_rgmii_speed)(struct rk_priv_data *bsp_priv, int speed); void (*set_rmii_speed)(struct rk_priv_data *bsp_priv, int speed); + void (*set_clock_selection)(struct rk_priv_data *bsp_priv, bool input, + bool enable); void (*integrated_phy_powerup)(struct rk_priv_data *bsp_priv); bool regs_valid; u32 regs[]; @@ -66,6 +68,7 @@ struct rk_priv_data { int rx_delay; struct regmap *grf; + struct regmap *php_grf; }; #define HIWORD_UPDATE(val, mask, shift) \ @@ -1101,6 +1104,147 @@ static const struct rk_gmac_ops rk3568_ops = { }, }; +/* sys_grf */ +#define RK3588_GRF_GMAC_CON7 0X031c +#define RK3588_GRF_GMAC_CON8 0X0320 +#define RK3588_GRF_GMAC_CON9 0X0324 + +#define RK3588_GMAC_RXCLK_DLY_ENABLE(id) GRF_BIT(2 * (id) + 3) +#define RK3588_GMAC_RXCLK_DLY_DISABLE(id) GRF_CLR_BIT(2 * (id) + 3) +#define RK3588_GMAC_TXCLK_DLY_ENABLE(id) GRF_BIT(2 * (id) + 2) +#define RK3588_GMAC_TXCLK_DLY_DISABLE(id) GRF_CLR_BIT(2 * (id) + 2) + +#define RK3588_GMAC_CLK_RX_DL_CFG(val) HIWORD_UPDATE(val, 0xFF, 8) +#define RK3588_GMAC_CLK_TX_DL_CFG(val) HIWORD_UPDATE(val, 0xFF, 0) + +/* php_grf */ +#define RK3588_GRF_GMAC_CON0 0X0008 +#define RK3588_GRF_CLK_CON1 0X0070 + +#define RK3588_GMAC_PHY_INTF_SEL_RGMII(id) \ + (GRF_BIT(3 + (id) * 6) | GRF_CLR_BIT(4 + (id) * 6) | GRF_CLR_BIT(5 + (id) * 6)) +#define RK3588_GMAC_PHY_INTF_SEL_RMII(id) \ + (GRF_CLR_BIT(3 + (id) * 6) | GRF_CLR_BIT(4 + (id) * 6) | GRF_BIT(5 + (id) * 6)) + +#define RK3588_GMAC_CLK_RMII_MODE(id) GRF_BIT(5 * (id)) +#define RK3588_GMAC_CLK_RGMII_MODE(id) GRF_CLR_BIT(5 * (id)) + +#define RK3588_GMAC_CLK_SELET_CRU(id) GRF_BIT(5 * (id) + 4) +#define RK3588_GMAC_CLK_SELET_IO(id) GRF_CLR_BIT(5 * (id) + 4) + +#define RK3588_GMA_CLK_RMII_DIV2(id) GRF_BIT(5 * (id) + 2) +#define RK3588_GMA_CLK_RMII_DIV20(id) GRF_CLR_BIT(5 * (id) + 2) + +#define RK3588_GMAC_CLK_RGMII_DIV1(id) \ + (GRF_CLR_BIT(5 * (id) + 2) | GRF_CLR_BIT(5 * (id) + 3)) +#define RK3588_GMAC_CLK_RGMII_DIV5(id) \ + (GRF_BIT(5 * (id) + 2) | GRF_BIT(5 * (id) + 3)) +#define RK3588_GMAC_CLK_RGMII_DIV50(id) \ + (GRF_CLR_BIT(5 * (id) + 2) | GRF_BIT(5 * (id) + 3)) + +#define RK3588_GMAC_CLK_RMII_GATE(id) GRF_BIT(5 * (id) + 1) +#define RK3588_GMAC_CLK_RMII_NOGATE(id) GRF_CLR_BIT(5 * (id) + 1) + +static void rk3588_set_to_rgmii(struct rk_priv_data *bsp_priv, + int tx_delay, int rx_delay) +{ + struct device *dev = &bsp_priv->pdev->dev; + u32 offset_con, id = bsp_priv->id; + + if (IS_ERR(bsp_priv->grf) || IS_ERR(bsp_priv->php_grf)) { + dev_err(dev, "Missing rockchip,grf or rockchip,php_grf property\n"); + return; + } + + offset_con = bsp_priv->id == 1 ? RK3588_GRF_GMAC_CON9 : + RK3588_GRF_GMAC_CON8; + + regmap_write(bsp_priv->php_grf, RK3588_GRF_GMAC_CON0, + RK3588_GMAC_PHY_INTF_SEL_RGMII(id)); + + regmap_write(bsp_priv->php_grf, RK3588_GRF_CLK_CON1, + RK3588_GMAC_CLK_RGMII_MODE(id)); + + regmap_write(bsp_priv->grf, RK3588_GRF_GMAC_CON7, + RK3588_GMAC_RXCLK_DLY_ENABLE(id) | + RK3588_GMAC_TXCLK_DLY_ENABLE(id)); + + regmap_write(bsp_priv->grf, offset_con, + RK3588_GMAC_CLK_RX_DL_CFG(rx_delay) | + RK3588_GMAC_CLK_TX_DL_CFG(tx_delay)); +} + +static void rk3588_set_to_rmii(struct rk_priv_data *bsp_priv) +{ + struct device *dev = &bsp_priv->pdev->dev; + + if (IS_ERR(bsp_priv->php_grf)) { + dev_err(dev, "%s: Missing rockchip,php_grf property\n", __func__); + return; + } + + regmap_write(bsp_priv->php_grf, RK3588_GRF_GMAC_CON0, + RK3588_GMAC_PHY_INTF_SEL_RMII(bsp_priv->id)); + + regmap_write(bsp_priv->php_grf, RK3588_GRF_CLK_CON1, + RK3588_GMAC_CLK_RMII_MODE(bsp_priv->id)); +} + +static void rk3588_set_gmac_speed(struct rk_priv_data *bsp_priv, int speed) +{ + struct device *dev = &bsp_priv->pdev->dev; + unsigned int val = 0, id = bsp_priv->id; + + switch (speed) { + case 10: + if (bsp_priv->phy_iface == PHY_INTERFACE_MODE_RMII) + val = RK3588_GMA_CLK_RMII_DIV20(id); + else + val = RK3588_GMAC_CLK_RGMII_DIV50(id); + break; + case 100: + if (bsp_priv->phy_iface == PHY_INTERFACE_MODE_RMII) + val = RK3588_GMA_CLK_RMII_DIV2(id); + else + val = RK3588_GMAC_CLK_RGMII_DIV5(id); + break; + case 1000: + if (bsp_priv->phy_iface != PHY_INTERFACE_MODE_RMII) + val = RK3588_GMAC_CLK_RGMII_DIV1(id); + else + goto err; + break; + default: + goto err; + } + + regmap_write(bsp_priv->php_grf, RK3588_GRF_CLK_CON1, val); + + return; +err: + dev_err(dev, "unknown speed value for GMAC speed=%d", speed); +} + +static void rk3588_set_clock_selection(struct rk_priv_data *bsp_priv, bool input, + bool enable) +{ + unsigned int val = input ? RK3588_GMAC_CLK_SELET_IO(bsp_priv->id) : + RK3588_GMAC_CLK_SELET_CRU(bsp_priv->id); + + val |= enable ? RK3588_GMAC_CLK_RMII_NOGATE(bsp_priv->id) : + RK3588_GMAC_CLK_RMII_GATE(bsp_priv->id); + + regmap_write(bsp_priv->php_grf, RK3588_GRF_CLK_CON1, val); +} + +static const struct rk_gmac_ops rk3588_ops = { + .set_to_rgmii = rk3588_set_to_rgmii, + .set_to_rmii = rk3588_set_to_rmii, + .set_rgmii_speed = rk3588_set_gmac_speed, + .set_rmii_speed = rk3588_set_gmac_speed, + .set_clock_selection = rk3588_set_clock_selection, +}; + #define RV1108_GRF_GMAC_CON0 0X0900 /* RV1108_GRF_GMAC_CON0 */ @@ -1153,6 +1297,130 @@ static const struct rk_gmac_ops rv1108_ops = { .set_rmii_speed = rv1108_set_rmii_speed, }; +#define RV1126_GRF_GMAC_CON0 0X0070 +#define RV1126_GRF_GMAC_CON1 0X0074 +#define RV1126_GRF_GMAC_CON2 0X0078 + +/* RV1126_GRF_GMAC_CON0 */ +#define RV1126_GMAC_PHY_INTF_SEL_RGMII \ + (GRF_BIT(4) | GRF_CLR_BIT(5) | GRF_CLR_BIT(6)) +#define RV1126_GMAC_PHY_INTF_SEL_RMII \ + (GRF_CLR_BIT(4) | GRF_CLR_BIT(5) | GRF_BIT(6)) +#define RV1126_GMAC_FLOW_CTRL GRF_BIT(7) +#define RV1126_GMAC_FLOW_CTRL_CLR GRF_CLR_BIT(7) +#define RV1126_GMAC_M0_RXCLK_DLY_ENABLE GRF_BIT(1) +#define RV1126_GMAC_M0_RXCLK_DLY_DISABLE GRF_CLR_BIT(1) +#define RV1126_GMAC_M0_TXCLK_DLY_ENABLE GRF_BIT(0) +#define RV1126_GMAC_M0_TXCLK_DLY_DISABLE GRF_CLR_BIT(0) +#define RV1126_GMAC_M1_RXCLK_DLY_ENABLE GRF_BIT(3) +#define RV1126_GMAC_M1_RXCLK_DLY_DISABLE GRF_CLR_BIT(3) +#define RV1126_GMAC_M1_TXCLK_DLY_ENABLE GRF_BIT(2) +#define RV1126_GMAC_M1_TXCLK_DLY_DISABLE GRF_CLR_BIT(2) + +/* RV1126_GRF_GMAC_CON1 */ +#define RV1126_GMAC_M0_CLK_RX_DL_CFG(val) HIWORD_UPDATE(val, 0x7F, 8) +#define RV1126_GMAC_M0_CLK_TX_DL_CFG(val) HIWORD_UPDATE(val, 0x7F, 0) +/* RV1126_GRF_GMAC_CON2 */ +#define RV1126_GMAC_M1_CLK_RX_DL_CFG(val) HIWORD_UPDATE(val, 0x7F, 8) +#define RV1126_GMAC_M1_CLK_TX_DL_CFG(val) HIWORD_UPDATE(val, 0x7F, 0) + +static void rv1126_set_to_rgmii(struct rk_priv_data *bsp_priv, + int tx_delay, int rx_delay) +{ + struct device *dev = &bsp_priv->pdev->dev; + + if (IS_ERR(bsp_priv->grf)) { + dev_err(dev, "Missing rockchip,grf property\n"); + return; + } + + regmap_write(bsp_priv->grf, RV1126_GRF_GMAC_CON0, + RV1126_GMAC_PHY_INTF_SEL_RGMII | + RV1126_GMAC_M0_RXCLK_DLY_ENABLE | + RV1126_GMAC_M0_TXCLK_DLY_ENABLE | + RV1126_GMAC_M1_RXCLK_DLY_ENABLE | + RV1126_GMAC_M1_TXCLK_DLY_ENABLE); + + regmap_write(bsp_priv->grf, RV1126_GRF_GMAC_CON1, + RV1126_GMAC_M0_CLK_RX_DL_CFG(rx_delay) | + RV1126_GMAC_M0_CLK_TX_DL_CFG(tx_delay)); + + regmap_write(bsp_priv->grf, RV1126_GRF_GMAC_CON2, + RV1126_GMAC_M1_CLK_RX_DL_CFG(rx_delay) | + RV1126_GMAC_M1_CLK_TX_DL_CFG(tx_delay)); +} + +static void rv1126_set_to_rmii(struct rk_priv_data *bsp_priv) +{ + struct device *dev = &bsp_priv->pdev->dev; + + if (IS_ERR(bsp_priv->grf)) { + dev_err(dev, "%s: Missing rockchip,grf property\n", __func__); + return; + } + + regmap_write(bsp_priv->grf, RV1126_GRF_GMAC_CON0, + RV1126_GMAC_PHY_INTF_SEL_RMII); +} + +static void rv1126_set_rgmii_speed(struct rk_priv_data *bsp_priv, int speed) +{ + struct device *dev = &bsp_priv->pdev->dev; + unsigned long rate; + int ret; + + switch (speed) { + case 10: + rate = 2500000; + break; + case 100: + rate = 25000000; + break; + case 1000: + rate = 125000000; + break; + default: + dev_err(dev, "unknown speed value for RGMII speed=%d", speed); + return; + } + + ret = clk_set_rate(bsp_priv->clk_mac_speed, rate); + if (ret) + dev_err(dev, "%s: set clk_mac_speed rate %ld failed %d\n", + __func__, rate, ret); +} + +static void rv1126_set_rmii_speed(struct rk_priv_data *bsp_priv, int speed) +{ + struct device *dev = &bsp_priv->pdev->dev; + unsigned long rate; + int ret; + + switch (speed) { + case 10: + rate = 2500000; + break; + case 100: + rate = 25000000; + break; + default: + dev_err(dev, "unknown speed value for RGMII speed=%d", speed); + return; + } + + ret = clk_set_rate(bsp_priv->clk_mac_speed, rate); + if (ret) + dev_err(dev, "%s: set clk_mac_speed rate %ld failed %d\n", + __func__, rate, ret); +} + +static const struct rk_gmac_ops rv1126_ops = { + .set_to_rgmii = rv1126_set_to_rgmii, + .set_to_rmii = rv1126_set_to_rmii, + .set_rgmii_speed = rv1126_set_rgmii_speed, + .set_rmii_speed = rv1126_set_rmii_speed, +}; + #define RK_GRF_MACPHY_CON0 0xb00 #define RK_GRF_MACPHY_CON1 0xb04 #define RK_GRF_MACPHY_CON2 0xb08 @@ -1304,6 +1572,10 @@ static int gmac_clk_enable(struct rk_priv_data *bsp_priv, bool enable) if (!IS_ERR(bsp_priv->clk_mac_speed)) clk_prepare_enable(bsp_priv->clk_mac_speed); + if (bsp_priv->ops && bsp_priv->ops->set_clock_selection) + bsp_priv->ops->set_clock_selection(bsp_priv, + bsp_priv->clock_input, true); + /** * if (!IS_ERR(bsp_priv->clk_mac)) * clk_prepare_enable(bsp_priv->clk_mac); @@ -1330,6 +1602,10 @@ static int gmac_clk_enable(struct rk_priv_data *bsp_priv, bool enable) clk_disable_unprepare(bsp_priv->mac_clk_tx); clk_disable_unprepare(bsp_priv->clk_mac_speed); + + if (bsp_priv->ops && bsp_priv->ops->set_clock_selection) + bsp_priv->ops->set_clock_selection(bsp_priv, + bsp_priv->clock_input, false); /** * if (!IS_ERR(bsp_priv->clk_mac)) * clk_disable_unprepare(bsp_priv->clk_mac); @@ -1444,6 +1720,8 @@ static struct rk_priv_data *rk_gmac_setup(struct platform_device *pdev, bsp_priv->grf = syscon_regmap_lookup_by_phandle(dev->of_node, "rockchip,grf"); + bsp_priv->php_grf = syscon_regmap_lookup_by_phandle(dev->of_node, + "rockchip,php-grf"); if (plat->phy_node) { bsp_priv->integrated_phy = of_property_read_bool(plat->phy_node, @@ -1680,7 +1958,9 @@ static const struct of_device_id rk_gmac_dwmac_match[] = { { .compatible = "rockchip,rk3368-gmac", .data = &rk3368_ops }, { .compatible = "rockchip,rk3399-gmac", .data = &rk3399_ops }, { .compatible = "rockchip,rk3568-gmac", .data = &rk3568_ops }, + { .compatible = "rockchip,rk3588-gmac", .data = &rk3588_ops }, { .compatible = "rockchip,rv1108-gmac", .data = &rv1108_ops }, + { .compatible = "rockchip,rv1126-gmac", .data = &rv1126_ops }, { } }; MODULE_DEVICE_TABLE(of, rk_gmac_dwmac_match); diff --git a/drivers/net/ethernet/stmicro/stmmac/dwmac100.h b/drivers/net/ethernet/stmicro/stmmac/dwmac100.h index 35ab8d0bdce7..7ab791c8d355 100644 --- a/drivers/net/ethernet/stmicro/stmmac/dwmac100.h +++ b/drivers/net/ethernet/stmicro/stmmac/dwmac100.h @@ -56,7 +56,7 @@ #define MAC_CONTROL_TE 0x00000008 /* Transmitter Enable */ #define MAC_CONTROL_RE 0x00000004 /* Receiver Enable */ -#define MAC_CORE_INIT (MAC_CONTROL_HBD | MAC_CONTROL_ASTP) +#define MAC_CORE_INIT (MAC_CONTROL_HBD) /* MAC FLOW CTRL defines */ #define MAC_FLOW_CTRL_PT_MASK 0xffff0000 /* Pause Time Mask */ diff --git a/drivers/net/ethernet/stmicro/stmmac/dwmac1000.h b/drivers/net/ethernet/stmicro/stmmac/dwmac1000.h index 3c73453725f9..4296ddda8aaa 100644 --- a/drivers/net/ethernet/stmicro/stmmac/dwmac1000.h +++ b/drivers/net/ethernet/stmicro/stmmac/dwmac1000.h @@ -126,7 +126,7 @@ enum inter_frame_gap { #define GMAC_CONTROL_TE 0x00000008 /* Transmitter Enable */ #define GMAC_CONTROL_RE 0x00000004 /* Receiver Enable */ -#define GMAC_CORE_INIT (GMAC_CONTROL_JD | GMAC_CONTROL_PS | GMAC_CONTROL_ACS | \ +#define GMAC_CORE_INIT (GMAC_CONTROL_JD | GMAC_CONTROL_PS | \ GMAC_CONTROL_BE | GMAC_CONTROL_DCRS) /* GMAC Frame Filter defines */ diff --git a/drivers/net/ethernet/stmicro/stmmac/dwmac1000_core.c b/drivers/net/ethernet/stmicro/stmmac/dwmac1000_core.c index 76edb9b72675..0e00dd83d027 100644 --- a/drivers/net/ethernet/stmicro/stmmac/dwmac1000_core.c +++ b/drivers/net/ethernet/stmicro/stmmac/dwmac1000_core.c @@ -15,7 +15,6 @@ #include <linux/crc32.h> #include <linux/slab.h> #include <linux/ethtool.h> -#include <net/dsa.h> #include <asm/io.h> #include "stmmac.h" #include "stmmac_pcs.h" @@ -24,7 +23,6 @@ static void dwmac1000_core_init(struct mac_device_info *hw, struct net_device *dev) { - struct stmmac_priv *priv = netdev_priv(dev); void __iomem *ioaddr = hw->pcsr; u32 value = readl(ioaddr + GMAC_CONTROL); int mtu = dev->mtu; @@ -32,13 +30,6 @@ static void dwmac1000_core_init(struct mac_device_info *hw, /* Configure GMAC core */ value |= GMAC_CORE_INIT; - /* Clear ACS bit because Ethernet switch tagging formats such as - * Broadcom tags can look like invalid LLC/SNAP packets and cause the - * hardware to truncate packets on reception. - */ - if (netdev_uses_dsa(dev) || !priv->plat->enh_desc) - value &= ~GMAC_CONTROL_ACS; - if (mtu > 1500) value |= GMAC_CONTROL_2K; if (mtu > 2000) diff --git a/drivers/net/ethernet/stmicro/stmmac/dwmac100_core.c b/drivers/net/ethernet/stmicro/stmmac/dwmac100_core.c index 75071a7d551a..a6e8d7bd9588 100644 --- a/drivers/net/ethernet/stmicro/stmmac/dwmac100_core.c +++ b/drivers/net/ethernet/stmicro/stmmac/dwmac100_core.c @@ -15,7 +15,6 @@ *******************************************************************************/ #include <linux/crc32.h> -#include <net/dsa.h> #include <asm/io.h> #include "stmmac.h" #include "dwmac100.h" @@ -28,13 +27,6 @@ static void dwmac100_core_init(struct mac_device_info *hw, value |= MAC_CORE_INIT; - /* Clear ASTP bit because Ethernet switch tagging formats such as - * Broadcom tags can look like invalid LLC/SNAP packets and cause the - * hardware to truncate packets on reception. - */ - if (netdev_uses_dsa(dev)) - value &= ~MAC_CONTROL_ASTP; - writel(value, ioaddr + MAC_CONTROL); #ifdef STMMAC_VLAN_TAG_USED diff --git a/drivers/net/ethernet/stmicro/stmmac/dwmac4_core.c b/drivers/net/ethernet/stmicro/stmmac/dwmac4_core.c index d8f1fbc25bdd..c25bfecb4a2d 100644 --- a/drivers/net/ethernet/stmicro/stmmac/dwmac4_core.c +++ b/drivers/net/ethernet/stmicro/stmmac/dwmac4_core.c @@ -14,7 +14,6 @@ #include <linux/slab.h> #include <linux/ethtool.h> #include <linux/io.h> -#include <net/dsa.h> #include "stmmac.h" #include "stmmac_pcs.h" #include "dwmac4.h" diff --git a/drivers/net/ethernet/stmicro/stmmac/stmmac_ethtool.c b/drivers/net/ethernet/stmicro/stmmac/stmmac_ethtool.c index d6a44d53fe08..f453b0d09366 100644 --- a/drivers/net/ethernet/stmicro/stmmac/stmmac_ethtool.c +++ b/drivers/net/ethernet/stmicro/stmmac/stmmac_ethtool.c @@ -287,15 +287,15 @@ static void stmmac_ethtool_getdrvinfo(struct net_device *dev, struct stmmac_priv *priv = netdev_priv(dev); if (priv->plat->has_gmac || priv->plat->has_gmac4) - strlcpy(info->driver, GMAC_ETHTOOL_NAME, sizeof(info->driver)); + strscpy(info->driver, GMAC_ETHTOOL_NAME, sizeof(info->driver)); else if (priv->plat->has_xgmac) - strlcpy(info->driver, XGMAC_ETHTOOL_NAME, sizeof(info->driver)); + strscpy(info->driver, XGMAC_ETHTOOL_NAME, sizeof(info->driver)); else - strlcpy(info->driver, MAC100_ETHTOOL_NAME, + strscpy(info->driver, MAC100_ETHTOOL_NAME, sizeof(info->driver)); if (priv->plat->pdev) { - strlcpy(info->bus_info, pci_name(priv->plat->pdev), + strscpy(info->bus_info, pci_name(priv->plat->pdev), sizeof(info->bus_info)); } } diff --git a/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c b/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c index 9083159b93f1..65c96773c6d2 100644 --- a/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c +++ b/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c @@ -5089,16 +5089,8 @@ read_again: buf1_len = stmmac_rx_buf1_len(priv, p, status, len); len += buf1_len; - /* ACS is set; GMAC core strips PAD/FCS for IEEE 802.3 - * Type frames (LLC/LLC-SNAP) - * - * llc_snap is never checked in GMAC >= 4, so this ACS - * feature is always disabled and packets need to be - * stripped manually. - */ - if (likely(!(status & rx_not_ls)) && - (likely(priv->synopsys_id >= DWMAC_CORE_4_00) || - unlikely(status != llc_snap))) { + /* ACS is disabled; strip manually. */ + if (likely(!(status & rx_not_ls))) { buf1_len -= ETH_FCS_LEN; len -= ETH_FCS_LEN; } @@ -5275,16 +5267,8 @@ read_again: buf2_len = stmmac_rx_buf2_len(priv, p, status, len); len += buf2_len; - /* ACS is set; GMAC core strips PAD/FCS for IEEE 802.3 - * Type frames (LLC/LLC-SNAP) - * - * llc_snap is never checked in GMAC >= 4, so this ACS - * feature is always disabled and packets need to be - * stripped manually. - */ - if (likely(!(status & rx_not_ls)) && - (likely(priv->synopsys_id >= DWMAC_CORE_4_00) || - unlikely(status != llc_snap))) { + /* ACS is disabled; strip manually. */ + if (likely(!(status & rx_not_ls))) { if (buf2_len) { buf2_len -= ETH_FCS_LEN; len -= ETH_FCS_LEN; @@ -6903,8 +6887,7 @@ static void stmmac_napi_add(struct net_device *dev) spin_lock_init(&ch->lock); if (queue < priv->plat->rx_queues_to_use) { - netif_napi_add(dev, &ch->rx_napi, stmmac_napi_poll_rx, - NAPI_POLL_WEIGHT); + netif_napi_add(dev, &ch->rx_napi, stmmac_napi_poll_rx); } if (queue < priv->plat->tx_queues_to_use) { netif_napi_add_tx(dev, &ch->tx_napi, @@ -6913,8 +6896,7 @@ static void stmmac_napi_add(struct net_device *dev) if (queue < priv->plat->rx_queues_to_use && queue < priv->plat->tx_queues_to_use) { netif_napi_add(dev, &ch->rxtx_napi, - stmmac_napi_poll_rxtx, - NAPI_POLL_WEIGHT); + stmmac_napi_poll_rxtx); } } } diff --git a/drivers/net/ethernet/stmicro/stmmac/stmmac_platform.c b/drivers/net/ethernet/stmicro/stmmac/stmmac_platform.c index 9f5cac4000da..50f6b4a14be4 100644 --- a/drivers/net/ethernet/stmicro/stmmac/stmmac_platform.c +++ b/drivers/net/ethernet/stmicro/stmmac/stmmac_platform.c @@ -440,11 +440,12 @@ stmmac_probe_config_dt(struct platform_device *pdev, u8 *mac) /* Default to phy auto-detection */ plat->phy_addr = -1; - /* Default to get clk_csr from stmmac_clk_crs_set(), + /* Default to get clk_csr from stmmac_clk_csr_set(), * or get clk_csr from device tree. */ plat->clk_csr = -1; - of_property_read_u32(np, "clk_csr", &plat->clk_csr); + if (of_property_read_u32(np, "snps,clk-csr", &plat->clk_csr)) + of_property_read_u32(np, "clk_csr", &plat->clk_csr); /* "snps,phy-addr" is not a standard property. Mark it as deprecated * and warn of its use. Remove this when phy node support is added. diff --git a/drivers/net/ethernet/sun/cassini.c b/drivers/net/ethernet/sun/cassini.c index 0b08b0e085e8..0aca193d9550 100644 --- a/drivers/net/ethernet/sun/cassini.c +++ b/drivers/net/ethernet/sun/cassini.c @@ -4484,9 +4484,9 @@ static void cas_set_multicast(struct net_device *dev) static void cas_get_drvinfo(struct net_device *dev, struct ethtool_drvinfo *info) { struct cas *cp = netdev_priv(dev); - strlcpy(info->driver, DRV_MODULE_NAME, sizeof(info->driver)); - strlcpy(info->version, DRV_MODULE_VERSION, sizeof(info->version)); - strlcpy(info->bus_info, pci_name(cp->pdev), sizeof(info->bus_info)); + strscpy(info->driver, DRV_MODULE_NAME, sizeof(info->driver)); + strscpy(info->version, DRV_MODULE_VERSION, sizeof(info->version)); + strscpy(info->bus_info, pci_name(cp->pdev), sizeof(info->bus_info)); } static int cas_get_link_ksettings(struct net_device *dev, @@ -5050,7 +5050,7 @@ static int cas_init_one(struct pci_dev *pdev, const struct pci_device_id *ent) dev->watchdog_timeo = CAS_TX_TIMEOUT; #ifdef USE_NAPI - netif_napi_add(dev, &cp->napi, cas_poll, 64); + netif_napi_add(dev, &cp->napi, cas_poll); #endif dev->irq = pdev->irq; dev->dma = 0; diff --git a/drivers/net/ethernet/sun/ldmvsw.c b/drivers/net/ethernet/sun/ldmvsw.c index 0cd8493b810f..8addee6d04bd 100644 --- a/drivers/net/ethernet/sun/ldmvsw.c +++ b/drivers/net/ethernet/sun/ldmvsw.c @@ -63,8 +63,8 @@ static struct vio_version vsw_versions[] = { static void vsw_get_drvinfo(struct net_device *dev, struct ethtool_drvinfo *info) { - strlcpy(info->driver, DRV_MODULE_NAME, sizeof(info->driver)); - strlcpy(info->version, DRV_MODULE_VERSION, sizeof(info->version)); + strscpy(info->driver, DRV_MODULE_NAME, sizeof(info->driver)); + strscpy(info->version, DRV_MODULE_VERSION, sizeof(info->version)); } static u32 vsw_get_msglevel(struct net_device *dev) @@ -354,8 +354,7 @@ static int vsw_port_probe(struct vio_dev *vdev, const struct vio_device_id *id) dev_set_drvdata(&vdev->dev, port); - netif_napi_add(dev, &port->napi, sunvnet_poll_common, - NAPI_POLL_WEIGHT); + netif_napi_add(dev, &port->napi, sunvnet_poll_common); spin_lock_irqsave(&vp->lock, flags); list_add_rcu(&port->list, &vp->port_list); diff --git a/drivers/net/ethernet/sun/niu.c b/drivers/net/ethernet/sun/niu.c index df70df29deea..e6144d963eaa 100644 --- a/drivers/net/ethernet/sun/niu.c +++ b/drivers/net/ethernet/sun/niu.c @@ -6798,12 +6798,12 @@ static void niu_get_drvinfo(struct net_device *dev, struct niu *np = netdev_priv(dev); struct niu_vpd *vpd = &np->vpd; - strlcpy(info->driver, DRV_MODULE_NAME, sizeof(info->driver)); - strlcpy(info->version, DRV_MODULE_VERSION, sizeof(info->version)); + strscpy(info->driver, DRV_MODULE_NAME, sizeof(info->driver)); + strscpy(info->version, DRV_MODULE_VERSION, sizeof(info->version)); snprintf(info->fw_version, sizeof(info->fw_version), "%d.%d", vpd->fcode_major, vpd->fcode_minor); if (np->parent->plat_type != PLAT_TYPE_NIU) - strlcpy(info->bus_info, pci_name(np->pdev), + strscpy(info->bus_info, pci_name(np->pdev), sizeof(info->bus_info)); } @@ -9115,7 +9115,7 @@ static int niu_ldg_init(struct niu *np) for (i = 0; i < np->num_ldg; i++) { struct niu_ldg *lp = &np->ldg[i]; - netif_napi_add(np->dev, &lp->napi, niu_poll, 64); + netif_napi_add(np->dev, &lp->napi, niu_poll); lp->np = np; lp->ldg_num = ldg_num_map[i]; diff --git a/drivers/net/ethernet/sun/sunbmac.c b/drivers/net/ethernet/sun/sunbmac.c index 531a6f449afa..34b94153bf0c 100644 --- a/drivers/net/ethernet/sun/sunbmac.c +++ b/drivers/net/ethernet/sun/sunbmac.c @@ -1038,8 +1038,8 @@ static void bigmac_set_multicast(struct net_device *dev) /* Ethtool support... */ static void bigmac_get_drvinfo(struct net_device *dev, struct ethtool_drvinfo *info) { - strlcpy(info->driver, "sunbmac", sizeof(info->driver)); - strlcpy(info->version, "2.0", sizeof(info->version)); + strscpy(info->driver, "sunbmac", sizeof(info->driver)); + strscpy(info->version, "2.0", sizeof(info->version)); } static u32 bigmac_get_link(struct net_device *dev) diff --git a/drivers/net/ethernet/sun/sungem.c b/drivers/net/ethernet/sun/sungem.c index a14591b41acb..4154e68639ac 100644 --- a/drivers/net/ethernet/sun/sungem.c +++ b/drivers/net/ethernet/sun/sungem.c @@ -2521,9 +2521,9 @@ static void gem_get_drvinfo(struct net_device *dev, struct ethtool_drvinfo *info { struct gem *gp = netdev_priv(dev); - strlcpy(info->driver, DRV_NAME, sizeof(info->driver)); - strlcpy(info->version, DRV_VERSION, sizeof(info->version)); - strlcpy(info->bus_info, pci_name(gp->pdev), sizeof(info->bus_info)); + strscpy(info->driver, DRV_NAME, sizeof(info->driver)); + strscpy(info->version, DRV_VERSION, sizeof(info->version)); + strscpy(info->bus_info, pci_name(gp->pdev), sizeof(info->bus_info)); } static int gem_get_link_ksettings(struct net_device *dev, @@ -2980,7 +2980,7 @@ static int gem_init_one(struct pci_dev *pdev, const struct pci_device_id *ent) goto err_out_free_consistent; dev->netdev_ops = &gem_netdev_ops; - netif_napi_add(dev, &gp->napi, gem_poll, 64); + netif_napi_add(dev, &gp->napi, gem_poll); dev->ethtool_ops = &gem_ethtool_ops; dev->watchdog_timeo = 5 * HZ; dev->dma = 0; diff --git a/drivers/net/ethernet/sun/sunhme.c b/drivers/net/ethernet/sun/sunhme.c index 88aa0d310aee..62deed210a95 100644 --- a/drivers/net/ethernet/sun/sunhme.c +++ b/drivers/net/ethernet/sun/sunhme.c @@ -61,15 +61,8 @@ #include "sunhme.h" #define DRV_NAME "sunhme" -#define DRV_VERSION "3.10" -#define DRV_RELDATE "August 26, 2008" -#define DRV_AUTHOR "David S. Miller (davem@davemloft.net)" -static char version[] = - DRV_NAME ".c:v" DRV_VERSION " " DRV_RELDATE " " DRV_AUTHOR "\n"; - -MODULE_VERSION(DRV_VERSION); -MODULE_AUTHOR(DRV_AUTHOR); +MODULE_AUTHOR("David S. Miller (davem@davemloft.net)"); MODULE_DESCRIPTION("Sun HappyMealEthernet(HME) 10/100baseT ethernet driver"); MODULE_LICENSE("GPL"); @@ -87,13 +80,17 @@ static struct quattro *qfe_sbus_list; static struct quattro *qfe_pci_list; #endif -#undef HMEDEBUG -#undef SXDEBUG -#undef RXDEBUG -#undef TXDEBUG -#undef TXLOGGING +#define hme_debug(fmt, ...) pr_debug("%s: " fmt, __func__, ##__VA_ARGS__) +#define HMD hme_debug + +/* "Auto Switch Debug" aka phy debug */ +#if 1 +#define ASD hme_debug +#else +#define ASD(...) +#endif -#ifdef TXLOGGING +#if 0 struct hme_tx_logent { unsigned int tstamp; int tx_new, tx_old; @@ -128,46 +125,16 @@ static __inline__ void tx_dump_log(void) this = txlog_cur_entry; for (i = 0; i < TX_LOG_LEN; i++) { - printk("TXLOG[%d]: j[%08x] tx[N(%d)O(%d)] action[%08x] stat[%08x]\n", i, + pr_err("TXLOG[%d]: j[%08x] tx[N(%d)O(%d)] action[%08x] stat[%08x]\n", i, tx_log[this].tstamp, tx_log[this].tx_new, tx_log[this].tx_old, tx_log[this].action, tx_log[this].status); this = (this + 1) & (TX_LOG_LEN - 1); } } -static __inline__ void tx_dump_ring(struct happy_meal *hp) -{ - struct hmeal_init_block *hb = hp->happy_block; - struct happy_meal_txd *tp = &hb->happy_meal_txd[0]; - int i; - - for (i = 0; i < TX_RING_SIZE; i+=4) { - printk("TXD[%d..%d]: [%08x:%08x] [%08x:%08x] [%08x:%08x] [%08x:%08x]\n", - i, i + 4, - le32_to_cpu(tp[i].tx_flags), le32_to_cpu(tp[i].tx_addr), - le32_to_cpu(tp[i + 1].tx_flags), le32_to_cpu(tp[i + 1].tx_addr), - le32_to_cpu(tp[i + 2].tx_flags), le32_to_cpu(tp[i + 2].tx_addr), - le32_to_cpu(tp[i + 3].tx_flags), le32_to_cpu(tp[i + 3].tx_addr)); - } -} -#else -#define tx_add_log(hp, a, s) do { } while(0) -#define tx_dump_log() do { } while(0) -#define tx_dump_ring(hp) do { } while(0) -#endif - -#ifdef HMEDEBUG -#define HMD(x) printk x -#else -#define HMD(x) -#endif - -/* #define AUTO_SWITCH_DEBUG */ - -#ifdef AUTO_SWITCH_DEBUG -#define ASD(x) printk x #else -#define ASD(x) +#define tx_add_log(hp, a, s) +#define tx_dump_log() #endif #define DEFAULT_IPG0 16 /* For lance-mode only */ @@ -343,8 +310,6 @@ static int happy_meal_bb_read(struct happy_meal *hp, int retval = 0; int i; - ASD(("happy_meal_bb_read: reg=%d ", reg)); - /* Enable the MIF BitBang outputs. */ hme_write32(hp, tregs + TCVR_BBOENAB, 1); @@ -378,7 +343,7 @@ static int happy_meal_bb_read(struct happy_meal *hp, (void) BB_GET_BIT2(hp, tregs, (hp->tcvr_type == internal)); (void) BB_GET_BIT2(hp, tregs, (hp->tcvr_type == internal)); (void) BB_GET_BIT2(hp, tregs, (hp->tcvr_type == internal)); - ASD(("value=%x\n", retval)); + ASD("reg=%d value=%x\n", reg, retval); return retval; } @@ -389,7 +354,7 @@ static void happy_meal_bb_write(struct happy_meal *hp, u32 tmp; int i; - ASD(("happy_meal_bb_write: reg=%d value=%x\n", reg, value)); + ASD("reg=%d value=%x\n", reg, value); /* Enable the MIF BitBang outputs. */ hme_write32(hp, tregs + TCVR_BBOENAB, 1); @@ -433,14 +398,13 @@ static int happy_meal_tcvr_read(struct happy_meal *hp, int tries = TCVR_READ_TRIES; int retval; - ASD(("happy_meal_tcvr_read: reg=0x%02x ", reg)); if (hp->tcvr_type == none) { - ASD(("no transceiver, value=TCVR_FAILURE\n")); + ASD("no transceiver, value=TCVR_FAILURE\n"); return TCVR_FAILURE; } if (!(hp->happy_flags & HFLAG_FENABLE)) { - ASD(("doing bit bang\n")); + ASD("doing bit bang\n"); return happy_meal_bb_read(hp, tregs, reg); } @@ -449,11 +413,11 @@ static int happy_meal_tcvr_read(struct happy_meal *hp, while (!(hme_read32(hp, tregs + TCVR_FRAME) & 0x10000) && --tries) udelay(20); if (!tries) { - printk(KERN_ERR "happy meal: Aieee, transceiver MIF read bolixed\n"); + netdev_err(hp->dev, "Aieee, transceiver MIF read bolixed\n"); return TCVR_FAILURE; } retval = hme_read32(hp, tregs + TCVR_FRAME) & 0xffff; - ASD(("value=%04x\n", retval)); + ASD("reg=0x%02x value=%04x\n", reg, retval); return retval; } @@ -465,7 +429,7 @@ static void happy_meal_tcvr_write(struct happy_meal *hp, { int tries = TCVR_WRITE_TRIES; - ASD(("happy_meal_tcvr_write: reg=0x%02x value=%04x\n", reg, value)); + ASD("reg=0x%02x value=%04x\n", reg, value); /* Welcome to Sun Microsystems, can I take your order please? */ if (!(hp->happy_flags & HFLAG_FENABLE)) { @@ -482,7 +446,7 @@ static void happy_meal_tcvr_write(struct happy_meal *hp, /* Anything else? */ if (!tries) - printk(KERN_ERR "happy meal: Aieee, transceiver MIF write bolixed\n"); + netdev_err(hp->dev, "Aieee, transceiver MIF write bolixed\n"); /* Fifty-two cents is your change, have a nice day. */ } @@ -660,8 +624,8 @@ static void happy_meal_timer(struct timer_list *t) /* Enter force mode. */ do_force_mode: hp->sw_bmcr = happy_meal_tcvr_read(hp, tregs, MII_BMCR); - printk(KERN_NOTICE "%s: Auto-Negotiation unsuccessful, trying force link mode\n", - hp->dev->name); + netdev_notice(hp->dev, + "Auto-Negotiation unsuccessful, trying force link mode\n"); hp->sw_bmcr = BMCR_SPEED100; happy_meal_tcvr_write(hp, tregs, MII_BMCR, hp->sw_bmcr); @@ -720,8 +684,8 @@ static void happy_meal_timer(struct timer_list *t) restart_timer = 0; } else { if (hp->timer_ticks >= 10) { - printk(KERN_NOTICE "%s: Auto negotiation successful, link still " - "not completely up.\n", hp->dev->name); + netdev_notice(hp->dev, + "Auto negotiation successful, link still not completely up.\n"); hp->timer_ticks = 0; restart_timer = 1; } else { @@ -776,14 +740,14 @@ static void happy_meal_timer(struct timer_list *t) */ /* Let the user know... */ - printk(KERN_NOTICE "%s: Link down, cable problem?\n", - hp->dev->name); + netdev_notice(hp->dev, + "Link down, cable problem?\n"); ret = happy_meal_init(hp); if (ret) { /* ho hum... */ - printk(KERN_ERR "%s: Error, cannot re-init the " - "Happy Meal.\n", hp->dev->name); + netdev_err(hp->dev, + "Error, cannot re-init the Happy Meal.\n"); } goto out; } @@ -805,8 +769,8 @@ static void happy_meal_timer(struct timer_list *t) case asleep: default: /* Can't happens.... */ - printk(KERN_ERR "%s: Aieee, link timer is asleep but we got one anyways!\n", - hp->dev->name); + netdev_err(hp->dev, + "Aieee, link timer is asleep but we got one anyways!\n"); restart_timer = 0; hp->timer_ticks = 0; hp->timer_state = asleep; /* foo on you */ @@ -830,7 +794,7 @@ static void happy_meal_tx_reset(struct happy_meal *hp, void __iomem *bregs) { int tries = TX_RESET_TRIES; - HMD(("happy_meal_tx_reset: reset, ")); + HMD("reset...\n"); /* Would you like to try our SMCC Delux? */ hme_write32(hp, bregs + BMAC_TXSWRESET, 0); @@ -839,10 +803,10 @@ static void happy_meal_tx_reset(struct happy_meal *hp, void __iomem *bregs) /* Lettuce, tomato, buggy hardware (no extra charge)? */ if (!tries) - printk(KERN_ERR "happy meal: Transceiver BigMac ATTACK!"); + netdev_err(hp->dev, "Transceiver BigMac ATTACK!"); /* Take care. */ - HMD(("done\n")); + HMD("done\n"); } /* hp->happy_lock must be held */ @@ -850,7 +814,7 @@ static void happy_meal_rx_reset(struct happy_meal *hp, void __iomem *bregs) { int tries = RX_RESET_TRIES; - HMD(("happy_meal_rx_reset: reset, ")); + HMD("reset...\n"); /* We have a special on GNU/Viking hardware bugs today. */ hme_write32(hp, bregs + BMAC_RXSWRESET, 0); @@ -859,10 +823,10 @@ static void happy_meal_rx_reset(struct happy_meal *hp, void __iomem *bregs) /* Will that be all? */ if (!tries) - printk(KERN_ERR "happy meal: Receiver BigMac ATTACK!"); + netdev_err(hp->dev, "Receiver BigMac ATTACK!\n"); /* Don't forget your vik_1137125_wa. Have a nice day. */ - HMD(("done\n")); + HMD("done\n"); } #define STOP_TRIES 16 @@ -872,7 +836,7 @@ static void happy_meal_stop(struct happy_meal *hp, void __iomem *gregs) { int tries = STOP_TRIES; - HMD(("happy_meal_stop: reset, ")); + HMD("reset...\n"); /* We're consolidating our STB products, it's your lucky day. */ hme_write32(hp, gregs + GREG_SWRESET, GREG_RESET_ALL); @@ -881,10 +845,10 @@ static void happy_meal_stop(struct happy_meal *hp, void __iomem *gregs) /* Come back next week when we are "Sun Microelectronics". */ if (!tries) - printk(KERN_ERR "happy meal: Fry guys."); + netdev_err(hp->dev, "Fry guys.\n"); /* Remember: "Different name, same old buggy as shit hardware." */ - HMD(("done\n")); + HMD("done\n"); } /* hp->happy_lock must be held */ @@ -913,21 +877,18 @@ static void happy_meal_get_counters(struct happy_meal *hp, void __iomem *bregs) /* hp->happy_lock must be held */ static void happy_meal_poll_stop(struct happy_meal *hp, void __iomem *tregs) { - ASD(("happy_meal_poll_stop: ")); - /* If polling disabled or not polling already, nothing to do. */ if ((hp->happy_flags & (HFLAG_POLLENABLE | HFLAG_POLL)) != (HFLAG_POLLENABLE | HFLAG_POLL)) { - HMD(("not polling, return\n")); + ASD("not polling, return\n"); return; } /* Shut up the MIF. */ - ASD(("were polling, mif ints off, ")); + ASD("were polling, mif ints off, polling off\n"); hme_write32(hp, tregs + TCVR_IMASK, 0xffff); /* Turn off polling. */ - ASD(("polling off, ")); hme_write32(hp, tregs + TCVR_CFG, hme_read32(hp, tregs + TCVR_CFG) & ~(TCV_CFG_PENABLE)); @@ -936,7 +897,7 @@ static void happy_meal_poll_stop(struct happy_meal *hp, void __iomem *tregs) /* Let the bits set. */ udelay(200); - ASD(("done\n")); + ASD("done\n"); } /* Only Sun can take such nice parts and fuck up the programming interface @@ -952,44 +913,40 @@ static int happy_meal_tcvr_reset(struct happy_meal *hp, void __iomem *tregs) int result, tries = TCVR_RESET_TRIES; tconfig = hme_read32(hp, tregs + TCVR_CFG); - ASD(("happy_meal_tcvr_reset: tcfg<%08lx> ", tconfig)); + ASD("tcfg=%08x\n", tconfig); if (hp->tcvr_type == external) { - ASD(("external<")); hme_write32(hp, tregs + TCVR_CFG, tconfig & ~(TCV_CFG_PSELECT)); hp->tcvr_type = internal; hp->paddr = TCV_PADDR_ITX; - ASD(("ISOLATE,")); happy_meal_tcvr_write(hp, tregs, MII_BMCR, (BMCR_LOOPBACK|BMCR_PDOWN|BMCR_ISOLATE)); result = happy_meal_tcvr_read(hp, tregs, MII_BMCR); if (result == TCVR_FAILURE) { - ASD(("phyread_fail>\n")); + ASD("phyread_fail\n"); return -1; } - ASD(("phyread_ok,PSELECT>")); + ASD("external: ISOLATE, phyread_ok, PSELECT\n"); hme_write32(hp, tregs + TCVR_CFG, tconfig | TCV_CFG_PSELECT); hp->tcvr_type = external; hp->paddr = TCV_PADDR_ETX; } else { if (tconfig & TCV_CFG_MDIO1) { - ASD(("internal<PSELECT,")); hme_write32(hp, tregs + TCVR_CFG, (tconfig | TCV_CFG_PSELECT)); - ASD(("ISOLATE,")); happy_meal_tcvr_write(hp, tregs, MII_BMCR, (BMCR_LOOPBACK|BMCR_PDOWN|BMCR_ISOLATE)); result = happy_meal_tcvr_read(hp, tregs, MII_BMCR); if (result == TCVR_FAILURE) { - ASD(("phyread_fail>\n")); + ASD("phyread_fail>\n"); return -1; } - ASD(("phyread_ok,~PSELECT>")); + ASD("internal: PSELECT, ISOLATE, phyread_ok, ~PSELECT\n"); hme_write32(hp, tregs + TCVR_CFG, (tconfig & ~(TCV_CFG_PSELECT))); hp->tcvr_type = internal; hp->paddr = TCV_PADDR_ITX; } } - ASD(("BMCR_RESET ")); + ASD("BMCR_RESET...\n"); happy_meal_tcvr_write(hp, tregs, MII_BMCR, BMCR_RESET); while (--tries) { @@ -1002,10 +959,10 @@ static int happy_meal_tcvr_reset(struct happy_meal *hp, void __iomem *tregs) udelay(20); } if (!tries) { - ASD(("BMCR RESET FAILED!\n")); + ASD("BMCR RESET FAILED!\n"); return -1; } - ASD(("RESET_OK\n")); + ASD("RESET_OK\n"); /* Get fresh copies of the PHY registers. */ hp->sw_bmsr = happy_meal_tcvr_read(hp, tregs, MII_BMSR); @@ -1013,7 +970,7 @@ static int happy_meal_tcvr_reset(struct happy_meal *hp, void __iomem *tregs) hp->sw_physid2 = happy_meal_tcvr_read(hp, tregs, MII_PHYSID2); hp->sw_advertise = happy_meal_tcvr_read(hp, tregs, MII_ADVERTISE); - ASD(("UNISOLATE")); + ASD("UNISOLATE...\n"); hp->sw_bmcr &= ~(BMCR_ISOLATE); happy_meal_tcvr_write(hp, tregs, MII_BMCR, hp->sw_bmcr); @@ -1027,10 +984,10 @@ static int happy_meal_tcvr_reset(struct happy_meal *hp, void __iomem *tregs) udelay(20); } if (!tries) { - ASD((" FAILED!\n")); + ASD("UNISOLATE FAILED!\n"); return -1; } - ASD((" SUCCESS and CSCONFIG_DFBYPASS\n")); + ASD("SUCCESS and CSCONFIG_DFBYPASS\n"); if (!is_lucent_phy(hp)) { result = happy_meal_tcvr_read(hp, tregs, DP83840_CSCONFIG); @@ -1048,60 +1005,55 @@ static void happy_meal_transceiver_check(struct happy_meal *hp, void __iomem *tr { unsigned long tconfig = hme_read32(hp, tregs + TCVR_CFG); - ASD(("happy_meal_transceiver_check: tcfg=%08lx ", tconfig)); + ASD("tcfg=%08lx\n", tconfig); if (hp->happy_flags & HFLAG_POLL) { /* If we are polling, we must stop to get the transceiver type. */ - ASD(("<polling> ")); if (hp->tcvr_type == internal) { if (tconfig & TCV_CFG_MDIO1) { - ASD(("<internal> <poll stop> ")); happy_meal_poll_stop(hp, tregs); hp->paddr = TCV_PADDR_ETX; hp->tcvr_type = external; - ASD(("<external>\n")); tconfig &= ~(TCV_CFG_PENABLE); tconfig |= TCV_CFG_PSELECT; hme_write32(hp, tregs + TCVR_CFG, tconfig); + ASD("poll stop, internal->external\n"); } } else { if (hp->tcvr_type == external) { - ASD(("<external> ")); if (!(hme_read32(hp, tregs + TCVR_STATUS) >> 16)) { - ASD(("<poll stop> ")); happy_meal_poll_stop(hp, tregs); hp->paddr = TCV_PADDR_ITX; hp->tcvr_type = internal; - ASD(("<internal>\n")); hme_write32(hp, tregs + TCVR_CFG, hme_read32(hp, tregs + TCVR_CFG) & ~(TCV_CFG_PSELECT)); + ASD("poll stop, external->internal\n"); } - ASD(("\n")); } else { - ASD(("<none>\n")); + ASD("polling, none\n"); } } } else { u32 reread = hme_read32(hp, tregs + TCVR_CFG); /* Else we can just work off of the MDIO bits. */ - ASD(("<not polling> ")); if (reread & TCV_CFG_MDIO1) { hme_write32(hp, tregs + TCVR_CFG, tconfig | TCV_CFG_PSELECT); hp->paddr = TCV_PADDR_ETX; hp->tcvr_type = external; - ASD(("<external>\n")); + ASD("not polling, external\n"); } else { if (reread & TCV_CFG_MDIO0) { hme_write32(hp, tregs + TCVR_CFG, tconfig & ~(TCV_CFG_PSELECT)); hp->paddr = TCV_PADDR_ITX; hp->tcvr_type = internal; - ASD(("<internal>\n")); + ASD("not polling, internal\n"); } else { - printk(KERN_ERR "happy meal: Transceiver and a coke please."); + netdev_err(hp->dev, + "Transceiver and a coke please."); hp->tcvr_type = none; /* Grrr... */ - ASD(("<none>\n")); + ASD("not polling, none\n"); } } } @@ -1208,15 +1160,14 @@ static void happy_meal_init_rings(struct happy_meal *hp) struct hmeal_init_block *hb = hp->happy_block; int i; - HMD(("happy_meal_init_rings: counters to zero, ")); + HMD("counters to zero\n"); hp->rx_new = hp->rx_old = hp->tx_new = hp->tx_old = 0; /* Free any skippy bufs left around in the rings. */ - HMD(("clean, ")); happy_meal_clean_rings(hp); /* Now get new skippy bufs for the receive ring. */ - HMD(("init rxring, ")); + HMD("init rxring\n"); for (i = 0; i < RX_RING_SIZE; i++) { struct sk_buff *skb; u32 mapping; @@ -1243,11 +1194,11 @@ static void happy_meal_init_rings(struct happy_meal *hp) skb_reserve(skb, RX_OFFSET); } - HMD(("init txring, ")); + HMD("init txring\n"); for (i = 0; i < TX_RING_SIZE; i++) hme_write_txd(hp, &hb->happy_meal_txd[i], 0, 0); - HMD(("done\n")); + HMD("done\n"); } /* hp->happy_lock must be held */ @@ -1294,17 +1245,11 @@ happy_meal_begin_auto_negotiation(struct happy_meal *hp, * XXX so I completely skip checking for it in the BMSR for now. */ -#ifdef AUTO_SWITCH_DEBUG - ASD(("%s: Advertising [ ", hp->dev->name)); - if (hp->sw_advertise & ADVERTISE_10HALF) - ASD(("10H ")); - if (hp->sw_advertise & ADVERTISE_10FULL) - ASD(("10F ")); - if (hp->sw_advertise & ADVERTISE_100HALF) - ASD(("100H ")); - if (hp->sw_advertise & ADVERTISE_100FULL) - ASD(("100F ")); -#endif + ASD("Advertising [ %s%s%s%s]\n", + hp->sw_advertise & ADVERTISE_10HALF ? "10H " : "", + hp->sw_advertise & ADVERTISE_10FULL ? "10F " : "", + hp->sw_advertise & ADVERTISE_100HALF ? "100H " : "", + hp->sw_advertise & ADVERTISE_100FULL ? "100F " : ""); /* Enable Auto-Negotiation, this is usually on already... */ hp->sw_bmcr |= BMCR_ANENABLE; @@ -1324,10 +1269,11 @@ happy_meal_begin_auto_negotiation(struct happy_meal *hp, udelay(10); } if (!timeout) { - printk(KERN_ERR "%s: Happy Meal would not start auto negotiation " - "BMCR=0x%04x\n", hp->dev->name, hp->sw_bmcr); - printk(KERN_NOTICE "%s: Performing force link detection.\n", - hp->dev->name); + netdev_err(hp->dev, + "Happy Meal would not start auto negotiation BMCR=0x%04x\n", + hp->sw_bmcr); + netdev_notice(hp->dev, + "Performing force link detection.\n"); goto force_link; } else { hp->timer_state = arbwait; @@ -1382,70 +1328,69 @@ static int happy_meal_init(struct happy_meal *hp) void __iomem *erxregs = hp->erxregs; void __iomem *bregs = hp->bigmacregs; void __iomem *tregs = hp->tcvregs; + const char *bursts; u32 regtmp, rxcfg; /* If auto-negotiation timer is running, kill it. */ del_timer(&hp->happy_timer); - HMD(("happy_meal_init: happy_flags[%08x] ", - hp->happy_flags)); + HMD("happy_flags[%08x]\n", hp->happy_flags); if (!(hp->happy_flags & HFLAG_INIT)) { - HMD(("set HFLAG_INIT, ")); + HMD("set HFLAG_INIT\n"); hp->happy_flags |= HFLAG_INIT; happy_meal_get_counters(hp, bregs); } /* Stop polling. */ - HMD(("to happy_meal_poll_stop\n")); + HMD("to happy_meal_poll_stop\n"); happy_meal_poll_stop(hp, tregs); /* Stop transmitter and receiver. */ - HMD(("happy_meal_init: to happy_meal_stop\n")); + HMD("to happy_meal_stop\n"); happy_meal_stop(hp, gregs); /* Alloc and reset the tx/rx descriptor chains. */ - HMD(("happy_meal_init: to happy_meal_init_rings\n")); + HMD("to happy_meal_init_rings\n"); happy_meal_init_rings(hp); /* Shut up the MIF. */ - HMD(("happy_meal_init: Disable all MIF irqs (old[%08x]), ", - hme_read32(hp, tregs + TCVR_IMASK))); + HMD("Disable all MIF irqs (old[%08x])\n", + hme_read32(hp, tregs + TCVR_IMASK)); hme_write32(hp, tregs + TCVR_IMASK, 0xffff); /* See if we can enable the MIF frame on this card to speak to the DP83840. */ if (hp->happy_flags & HFLAG_FENABLE) { - HMD(("use frame old[%08x], ", - hme_read32(hp, tregs + TCVR_CFG))); + HMD("use frame old[%08x]\n", + hme_read32(hp, tregs + TCVR_CFG)); hme_write32(hp, tregs + TCVR_CFG, hme_read32(hp, tregs + TCVR_CFG) & ~(TCV_CFG_BENABLE)); } else { - HMD(("use bitbang old[%08x], ", - hme_read32(hp, tregs + TCVR_CFG))); + HMD("use bitbang old[%08x]\n", + hme_read32(hp, tregs + TCVR_CFG)); hme_write32(hp, tregs + TCVR_CFG, hme_read32(hp, tregs + TCVR_CFG) | TCV_CFG_BENABLE); } /* Check the state of the transceiver. */ - HMD(("to happy_meal_transceiver_check\n")); + HMD("to happy_meal_transceiver_check\n"); happy_meal_transceiver_check(hp, tregs); /* Put the Big Mac into a sane state. */ - HMD(("happy_meal_init: ")); switch(hp->tcvr_type) { case none: /* Cannot operate if we don't know the transceiver type! */ - HMD(("AAIEEE no transceiver type, EAGAIN")); + HMD("AAIEEE no transceiver type, EAGAIN\n"); return -EAGAIN; case internal: /* Using the MII buffers. */ - HMD(("internal, using MII, ")); + HMD("internal, using MII\n"); hme_write32(hp, bregs + BMAC_XIFCFG, 0); break; case external: /* Not using the MII, disable it. */ - HMD(("external, disable MII, ")); + HMD("external, disable MII\n"); hme_write32(hp, bregs + BMAC_XIFCFG, BIGMAC_XCFG_MIIDISAB); break; } @@ -1454,18 +1399,16 @@ static int happy_meal_init(struct happy_meal *hp) return -EAGAIN; /* Reset the Happy Meal Big Mac transceiver and the receiver. */ - HMD(("tx/rx reset, ")); + HMD("tx/rx reset\n"); happy_meal_tx_reset(hp, bregs); happy_meal_rx_reset(hp, bregs); /* Set jam size and inter-packet gaps to reasonable defaults. */ - HMD(("jsize/ipg1/ipg2, ")); hme_write32(hp, bregs + BMAC_JSIZE, DEFAULT_JAMSIZE); hme_write32(hp, bregs + BMAC_IGAP1, DEFAULT_IPG1); hme_write32(hp, bregs + BMAC_IGAP2, DEFAULT_IPG2); /* Load up the MAC address and random seed. */ - HMD(("rseed/macaddr, ")); /* The docs recommend to use the 10LSB of our MAC here. */ hme_write32(hp, bregs + BMAC_RSEED, ((e[5] | e[4]<<8)&0x3ff)); @@ -1474,7 +1417,6 @@ static int happy_meal_init(struct happy_meal *hp) hme_write32(hp, bregs + BMAC_MACADDR1, ((e[2] << 8) | e[3])); hme_write32(hp, bregs + BMAC_MACADDR0, ((e[0] << 8) | e[1])); - HMD(("htable, ")); if ((hp->dev->flags & IFF_ALLMULTI) || (netdev_mc_count(hp->dev) > 64)) { hme_write32(hp, bregs + BMAC_HTABLE0, 0xffff); @@ -1504,9 +1446,9 @@ static int happy_meal_init(struct happy_meal *hp) } /* Set the RX and TX ring ptrs. */ - HMD(("ring ptrs rxr[%08x] txr[%08x]\n", - ((__u32)hp->hblock_dvma + hblock_offset(happy_meal_rxd, 0)), - ((__u32)hp->hblock_dvma + hblock_offset(happy_meal_txd, 0)))); + HMD("ring ptrs rxr[%08x] txr[%08x]\n", + ((__u32)hp->hblock_dvma + hblock_offset(happy_meal_rxd, 0)), + ((__u32)hp->hblock_dvma + hblock_offset(happy_meal_txd, 0))); hme_write32(hp, erxregs + ERX_RING, ((__u32)hp->hblock_dvma + hblock_offset(happy_meal_rxd, 0))); hme_write32(hp, etxregs + ETX_RING, @@ -1524,9 +1466,6 @@ static int happy_meal_init(struct happy_meal *hp) | 0x4); /* Set the supported burst sizes. */ - HMD(("happy_meal_init: old[%08x] bursts<", - hme_read32(hp, gregs + GREG_CFG))); - #ifndef CONFIG_SPARC /* It is always PCI and can handle 64byte bursts. */ hme_write32(hp, gregs + GREG_CFG, GREG_CFG_BURST64); @@ -1554,34 +1493,35 @@ static int happy_meal_init(struct happy_meal *hp) } #endif - HMD(("64>")); + bursts = "64"; hme_write32(hp, gregs + GREG_CFG, gcfg); } else if (hp->happy_bursts & DMA_BURST32) { - HMD(("32>")); + bursts = "32"; hme_write32(hp, gregs + GREG_CFG, GREG_CFG_BURST32); } else if (hp->happy_bursts & DMA_BURST16) { - HMD(("16>")); + bursts = "16"; hme_write32(hp, gregs + GREG_CFG, GREG_CFG_BURST16); } else { - HMD(("XXX>")); + bursts = "XXX"; hme_write32(hp, gregs + GREG_CFG, 0); } #endif /* CONFIG_SPARC */ + HMD("old[%08x] bursts<%s>\n", + hme_read32(hp, gregs + GREG_CFG), bursts); + /* Turn off interrupts we do not want to hear. */ - HMD((", enable global interrupts, ")); hme_write32(hp, gregs + GREG_IMASK, (GREG_IMASK_GOTFRAME | GREG_IMASK_RCNTEXP | GREG_IMASK_SENTFRAME | GREG_IMASK_TXPERR)); /* Set the transmit ring buffer size. */ - HMD(("tx rsize=%d oreg[%08x], ", (int)TX_RING_SIZE, - hme_read32(hp, etxregs + ETX_RSIZE))); + HMD("tx rsize=%d oreg[%08x]\n", (int)TX_RING_SIZE, + hme_read32(hp, etxregs + ETX_RSIZE)); hme_write32(hp, etxregs + ETX_RSIZE, (TX_RING_SIZE >> ETX_RSIZE_SHIFT) - 1); /* Enable transmitter DVMA. */ - HMD(("tx dma enable old[%08x], ", - hme_read32(hp, etxregs + ETX_CFG))); + HMD("tx dma enable old[%08x]\n", hme_read32(hp, etxregs + ETX_CFG)); hme_write32(hp, etxregs + ETX_CFG, hme_read32(hp, etxregs + ETX_CFG) | ETX_CFG_DMAENABLE); @@ -1590,21 +1530,23 @@ static int happy_meal_init(struct happy_meal *hp) * properly. I cannot think of a sane way to provide complete * coverage for this hardware bug yet. */ - HMD(("erx regs bug old[%08x]\n", - hme_read32(hp, erxregs + ERX_CFG))); + HMD("erx regs bug old[%08x]\n", + hme_read32(hp, erxregs + ERX_CFG)); hme_write32(hp, erxregs + ERX_CFG, ERX_CFG_DEFAULT(RX_OFFSET)); regtmp = hme_read32(hp, erxregs + ERX_CFG); hme_write32(hp, erxregs + ERX_CFG, ERX_CFG_DEFAULT(RX_OFFSET)); if (hme_read32(hp, erxregs + ERX_CFG) != ERX_CFG_DEFAULT(RX_OFFSET)) { - printk(KERN_ERR "happy meal: Eieee, rx config register gets greasy fries.\n"); - printk(KERN_ERR "happy meal: Trying to set %08x, reread gives %08x\n", - ERX_CFG_DEFAULT(RX_OFFSET), regtmp); + netdev_err(hp->dev, + "Eieee, rx config register gets greasy fries.\n"); + netdev_err(hp->dev, + "Trying to set %08x, reread gives %08x\n", + ERX_CFG_DEFAULT(RX_OFFSET), regtmp); /* XXX Should return failure here... */ } /* Enable Big Mac hash table filter. */ - HMD(("happy_meal_init: enable hash rx_cfg_old[%08x], ", - hme_read32(hp, bregs + BMAC_RXCFG))); + HMD("enable hash rx_cfg_old[%08x]\n", + hme_read32(hp, bregs + BMAC_RXCFG)); rxcfg = BIGMAC_RXCFG_HENABLE | BIGMAC_RXCFG_REJME; if (hp->dev->flags & IFF_PROMISC) rxcfg |= BIGMAC_RXCFG_PMISC; @@ -1614,7 +1556,7 @@ static int happy_meal_init(struct happy_meal *hp) udelay(10); /* Ok, configure the Big Mac transmitter. */ - HMD(("BIGMAC init, ")); + HMD("BIGMAC init\n"); regtmp = 0; if (hp->happy_flags & HFLAG_FULL) regtmp |= BIGMAC_TXCFG_FULLDPLX; @@ -1638,14 +1580,13 @@ static int happy_meal_init(struct happy_meal *hp) if (hp->tcvr_type == external) regtmp |= BIGMAC_XCFG_MIIDISAB; - HMD(("XIF config old[%08x], ", - hme_read32(hp, bregs + BMAC_XIFCFG))); + HMD("XIF config old[%08x]\n", hme_read32(hp, bregs + BMAC_XIFCFG)); hme_write32(hp, bregs + BMAC_XIFCFG, regtmp); /* Start things up. */ - HMD(("tx old[%08x] and rx [%08x] ON!\n", - hme_read32(hp, bregs + BMAC_TXCFG), - hme_read32(hp, bregs + BMAC_RXCFG))); + HMD("tx old[%08x] and rx [%08x] ON!\n", + hme_read32(hp, bregs + BMAC_TXCFG), + hme_read32(hp, bregs + BMAC_RXCFG)); /* Set larger TX/RX size to allow for 802.1q */ hme_write32(hp, bregs + BMAC_TXMAX, ETH_FRAME_LEN + 8); @@ -1735,25 +1676,26 @@ static int happy_meal_is_not_so_happy(struct happy_meal *hp, u32 status) GREG_STAT_MIFIRQ | GREG_STAT_TXEACK | GREG_STAT_TXLERR | GREG_STAT_TXPERR | GREG_STAT_TXTERR | GREG_STAT_SLVERR | GREG_STAT_SLVPERR)) - printk(KERN_ERR "%s: Error interrupt for happy meal, status = %08x\n", - hp->dev->name, status); + netdev_err(hp->dev, + "Error interrupt for happy meal, status = %08x\n", + status); if (status & GREG_STAT_RFIFOVF) { /* Receive FIFO overflow is harmless and the hardware will take care of it, just some packets are lost. Who cares. */ - printk(KERN_DEBUG "%s: Happy Meal receive FIFO overflow.\n", hp->dev->name); + netdev_dbg(hp->dev, "Happy Meal receive FIFO overflow.\n"); } if (status & GREG_STAT_STSTERR) { /* BigMAC SQE link test failed. */ - printk(KERN_ERR "%s: Happy Meal BigMAC SQE test failed.\n", hp->dev->name); + netdev_err(hp->dev, "Happy Meal BigMAC SQE test failed.\n"); reset = 1; } if (status & GREG_STAT_TFIFO_UND) { /* Transmit FIFO underrun, again DMA error likely. */ - printk(KERN_ERR "%s: Happy Meal transmitter FIFO underrun, DMA error.\n", - hp->dev->name); + netdev_err(hp->dev, + "Happy Meal transmitter FIFO underrun, DMA error.\n"); reset = 1; } @@ -1761,7 +1703,7 @@ static int happy_meal_is_not_so_happy(struct happy_meal *hp, u32 status) /* Driver error, tried to transmit something larger * than ethernet max mtu. */ - printk(KERN_ERR "%s: Happy Meal MAX Packet size error.\n", hp->dev->name); + netdev_err(hp->dev, "Happy Meal MAX Packet size error.\n"); reset = 1; } @@ -1771,21 +1713,16 @@ static int happy_meal_is_not_so_happy(struct happy_meal *hp, u32 status) * faster than the interrupt handler could keep up * with. */ - printk(KERN_INFO "%s: Happy Meal out of receive " - "descriptors, packet dropped.\n", - hp->dev->name); + netdev_info(hp->dev, + "Happy Meal out of receive descriptors, packet dropped.\n"); } if (status & (GREG_STAT_RXERR|GREG_STAT_RXPERR|GREG_STAT_RXTERR)) { /* All sorts of DMA receive errors. */ - printk(KERN_ERR "%s: Happy Meal rx DMA errors [ ", hp->dev->name); - if (status & GREG_STAT_RXERR) - printk("GenericError "); - if (status & GREG_STAT_RXPERR) - printk("ParityError "); - if (status & GREG_STAT_RXTERR) - printk("RxTagBotch "); - printk("]\n"); + netdev_err(hp->dev, "Happy Meal rx DMA errors [ %s%s%s]\n", + status & GREG_STAT_RXERR ? "GenericError " : "", + status & GREG_STAT_RXPERR ? "ParityError " : "", + status & GREG_STAT_RXTERR ? "RxTagBotch " : ""); reset = 1; } @@ -1793,29 +1730,24 @@ static int happy_meal_is_not_so_happy(struct happy_meal *hp, u32 status) /* Driver bug, didn't set EOP bit in tx descriptor given * to the happy meal. */ - printk(KERN_ERR "%s: EOP not set in happy meal transmit descriptor!\n", - hp->dev->name); + netdev_err(hp->dev, + "EOP not set in happy meal transmit descriptor!\n"); reset = 1; } if (status & GREG_STAT_MIFIRQ) { /* MIF signalled an interrupt, were we polling it? */ - printk(KERN_ERR "%s: Happy Meal MIF interrupt.\n", hp->dev->name); + netdev_err(hp->dev, "Happy Meal MIF interrupt.\n"); } if (status & (GREG_STAT_TXEACK|GREG_STAT_TXLERR|GREG_STAT_TXPERR|GREG_STAT_TXTERR)) { /* All sorts of transmit DMA errors. */ - printk(KERN_ERR "%s: Happy Meal tx DMA errors [ ", hp->dev->name); - if (status & GREG_STAT_TXEACK) - printk("GenericError "); - if (status & GREG_STAT_TXLERR) - printk("LateError "); - if (status & GREG_STAT_TXPERR) - printk("ParityError "); - if (status & GREG_STAT_TXTERR) - printk("TagBotch "); - printk("]\n"); + netdev_err(hp->dev, "Happy Meal tx DMA errors [ %s%s%s%s]\n", + status & GREG_STAT_TXEACK ? "GenericError " : "", + status & GREG_STAT_TXLERR ? "LateError " : "", + status & GREG_STAT_TXPERR ? "ParityError " : "", + status & GREG_STAT_TXTERR ? "TagBotch " : ""); reset = 1; } @@ -1823,14 +1755,14 @@ static int happy_meal_is_not_so_happy(struct happy_meal *hp, u32 status) /* Bus or parity error when cpu accessed happy meal registers * or it's internal FIFO's. Should never see this. */ - printk(KERN_ERR "%s: Happy Meal register access SBUS slave (%s) error.\n", - hp->dev->name, - (status & GREG_STAT_SLVPERR) ? "parity" : "generic"); + netdev_err(hp->dev, + "Happy Meal register access SBUS slave (%s) error.\n", + (status & GREG_STAT_SLVPERR) ? "parity" : "generic"); reset = 1; } if (reset) { - printk(KERN_NOTICE "%s: Resetting...\n", hp->dev->name); + netdev_notice(hp->dev, "Resetting...\n"); happy_meal_init(hp); return 1; } @@ -1842,22 +1774,22 @@ static void happy_meal_mif_interrupt(struct happy_meal *hp) { void __iomem *tregs = hp->tcvregs; - printk(KERN_INFO "%s: Link status change.\n", hp->dev->name); + netdev_info(hp->dev, "Link status change.\n"); hp->sw_bmcr = happy_meal_tcvr_read(hp, tregs, MII_BMCR); hp->sw_lpa = happy_meal_tcvr_read(hp, tregs, MII_LPA); /* Use the fastest transmission protocol possible. */ if (hp->sw_lpa & LPA_100FULL) { - printk(KERN_INFO "%s: Switching to 100Mbps at full duplex.", hp->dev->name); + netdev_info(hp->dev, "Switching to 100Mbps at full duplex.\n"); hp->sw_bmcr |= (BMCR_FULLDPLX | BMCR_SPEED100); } else if (hp->sw_lpa & LPA_100HALF) { - printk(KERN_INFO "%s: Switching to 100MBps at half duplex.", hp->dev->name); + netdev_info(hp->dev, "Switching to 100MBps at half duplex.\n"); hp->sw_bmcr |= BMCR_SPEED100; } else if (hp->sw_lpa & LPA_10FULL) { - printk(KERN_INFO "%s: Switching to 10MBps at full duplex.", hp->dev->name); + netdev_info(hp->dev, "Switching to 10MBps at full duplex.\n"); hp->sw_bmcr |= BMCR_FULLDPLX; } else { - printk(KERN_INFO "%s: Using 10Mbps at half duplex.", hp->dev->name); + netdev_info(hp->dev, "Using 10Mbps at half duplex.\n"); } happy_meal_tcvr_write(hp, tregs, MII_BMCR, hp->sw_bmcr); @@ -1865,12 +1797,6 @@ static void happy_meal_mif_interrupt(struct happy_meal *hp) happy_meal_poll_stop(hp, tregs); } -#ifdef TXDEBUG -#define TXD(x) printk x -#else -#define TXD(x) -#endif - /* hp->happy_lock must be held */ static void happy_meal_tx(struct happy_meal *hp) { @@ -1880,13 +1806,12 @@ static void happy_meal_tx(struct happy_meal *hp) int elem; elem = hp->tx_old; - TXD(("TX<")); while (elem != hp->tx_new) { struct sk_buff *skb; u32 flags, dma_addr, dma_len; int frag; - TXD(("[%d]", elem)); + netdev_vdbg(hp->dev, "TX[%d]\n", elem); this = &txbase[elem]; flags = hme_read_desc32(hp, &this->tx_flags); if (flags & TXFLAG_OWN) @@ -1922,19 +1847,12 @@ static void happy_meal_tx(struct happy_meal *hp) dev->stats.tx_packets++; } hp->tx_old = elem; - TXD((">")); if (netif_queue_stopped(dev) && TX_BUFFS_AVAIL(hp) > (MAX_SKB_FRAGS + 1)) netif_wake_queue(dev); } -#ifdef RXDEBUG -#define RXD(x) printk x -#else -#define RXD(x) -#endif - /* Originally I used to handle the allocation failure by just giving back just * that one ring buffer to the happy meal. Problem is that usually when that * condition is triggered, the happy meal expects you to do something reasonable @@ -1951,7 +1869,6 @@ static void happy_meal_rx(struct happy_meal *hp, struct net_device *dev) int elem = hp->rx_new, drops = 0; u32 flags; - RXD(("RX<")); this = &rxbase[elem]; while (!((flags = hme_read_desc32(hp, &this->rx_flags)) & RXFLAG_OWN)) { struct sk_buff *skb; @@ -1959,11 +1876,9 @@ static void happy_meal_rx(struct happy_meal *hp, struct net_device *dev) u16 csum = flags & RXFLAG_CSUM; u32 dma_addr = hme_read_desc32(hp, &this->rx_addr); - RXD(("[%d ", elem)); - /* Check for errors. */ if ((len < ETH_ZLEN) || (flags & RXFLAG_OVERFLOW)) { - RXD(("ERR(%08x)]", flags)); + netdev_vdbg(dev, "RX[%d ERR(%08x)]", elem, flags); dev->stats.rx_errors++; if (len < ETH_ZLEN) dev->stats.rx_length_errors++; @@ -2035,7 +1950,7 @@ static void happy_meal_rx(struct happy_meal *hp, struct net_device *dev) skb->csum = csum_unfold(~(__force __sum16)htons(csum)); skb->ip_summed = CHECKSUM_COMPLETE; - RXD(("len=%d csum=%4x]", len, csum)); + netdev_vdbg(dev, "RX[%d len=%d csum=%4x]", elem, len, csum); skb->protocol = eth_type_trans(skb, dev); netif_rx(skb); @@ -2047,8 +1962,7 @@ static void happy_meal_rx(struct happy_meal *hp, struct net_device *dev) } hp->rx_new = elem; if (drops) - printk(KERN_INFO "%s: Memory squeeze, deferring packet.\n", hp->dev->name); - RXD((">")); + netdev_info(hp->dev, "Memory squeeze, deferring packet.\n"); } static irqreturn_t happy_meal_interrupt(int irq, void *dev_id) @@ -2057,32 +1971,25 @@ static irqreturn_t happy_meal_interrupt(int irq, void *dev_id) struct happy_meal *hp = netdev_priv(dev); u32 happy_status = hme_read32(hp, hp->gregs + GREG_STAT); - HMD(("happy_meal_interrupt: status=%08x ", happy_status)); + HMD("status=%08x\n", happy_status); spin_lock(&hp->happy_lock); if (happy_status & GREG_STAT_ERRORS) { - HMD(("ERRORS ")); if (happy_meal_is_not_so_happy(hp, /* un- */ happy_status)) goto out; } - if (happy_status & GREG_STAT_MIFIRQ) { - HMD(("MIFIRQ ")); + if (happy_status & GREG_STAT_MIFIRQ) happy_meal_mif_interrupt(hp); - } - if (happy_status & GREG_STAT_TXALL) { - HMD(("TXALL ")); + if (happy_status & GREG_STAT_TXALL) happy_meal_tx(hp); - } - if (happy_status & GREG_STAT_RXTOHOST) { - HMD(("RXTOHOST ")); + if (happy_status & GREG_STAT_RXTOHOST) happy_meal_rx(hp, dev); - } - HMD(("done\n")); + HMD("done\n"); out: spin_unlock(&hp->happy_lock); @@ -2100,7 +2007,7 @@ static irqreturn_t quattro_sbus_interrupt(int irq, void *cookie) struct happy_meal *hp = netdev_priv(dev); u32 happy_status = hme_read32(hp, hp->gregs + GREG_STAT); - HMD(("quattro_interrupt: status=%08x ", happy_status)); + HMD("status=%08x\n", happy_status); if (!(happy_status & (GREG_STAT_ERRORS | GREG_STAT_MIFIRQ | @@ -2110,31 +2017,23 @@ static irqreturn_t quattro_sbus_interrupt(int irq, void *cookie) spin_lock(&hp->happy_lock); - if (happy_status & GREG_STAT_ERRORS) { - HMD(("ERRORS ")); + if (happy_status & GREG_STAT_ERRORS) if (happy_meal_is_not_so_happy(hp, happy_status)) goto next; - } - if (happy_status & GREG_STAT_MIFIRQ) { - HMD(("MIFIRQ ")); + if (happy_status & GREG_STAT_MIFIRQ) happy_meal_mif_interrupt(hp); - } - if (happy_status & GREG_STAT_TXALL) { - HMD(("TXALL ")); + if (happy_status & GREG_STAT_TXALL) happy_meal_tx(hp); - } - if (happy_status & GREG_STAT_RXTOHOST) { - HMD(("RXTOHOST ")); + if (happy_status & GREG_STAT_RXTOHOST) happy_meal_rx(hp, dev); - } next: spin_unlock(&hp->happy_lock); } - HMD(("done\n")); + HMD("done\n"); return IRQ_HANDLED; } @@ -2145,8 +2044,6 @@ static int happy_meal_open(struct net_device *dev) struct happy_meal *hp = netdev_priv(dev); int res; - HMD(("happy_meal_open: ")); - /* On SBUS Quattro QFE cards, all hme interrupts are concentrated * into a single source which we register handling at probe time. */ @@ -2154,15 +2051,14 @@ static int happy_meal_open(struct net_device *dev) res = request_irq(hp->irq, happy_meal_interrupt, IRQF_SHARED, dev->name, dev); if (res) { - HMD(("EAGAIN\n")); - printk(KERN_ERR "happy_meal(SBUS): Can't order irq %d to go.\n", - hp->irq); + HMD("EAGAIN\n"); + netdev_err(dev, "Can't order irq %d to go.\n", hp->irq); return -EAGAIN; } } - HMD(("to happy_meal_init\n")); + HMD("to happy_meal_init\n"); spin_lock_irq(&hp->happy_lock); res = happy_meal_init(hp); @@ -2196,22 +2092,16 @@ static int happy_meal_close(struct net_device *dev) return 0; } -#ifdef SXDEBUG -#define SXD(x) printk x -#else -#define SXD(x) -#endif - static void happy_meal_tx_timeout(struct net_device *dev, unsigned int txqueue) { struct happy_meal *hp = netdev_priv(dev); - printk (KERN_ERR "%s: transmit timed out, resetting\n", dev->name); + netdev_err(dev, "transmit timed out, resetting\n"); tx_dump_log(); - printk (KERN_ERR "%s: Happy Status %08x TX[%08x:%08x]\n", dev->name, - hme_read32(hp, hp->gregs + GREG_STAT), - hme_read32(hp, hp->etxregs + ETX_CFG), - hme_read32(hp, hp->bigmacregs + BMAC_TXCFG)); + netdev_err(dev, "Happy Status %08x TX[%08x:%08x]\n", + hme_read32(hp, hp->gregs + GREG_STAT), + hme_read32(hp, hp->etxregs + ETX_CFG), + hme_read32(hp, hp->bigmacregs + BMAC_TXCFG)); spin_lock_irq(&hp->happy_lock); happy_meal_init(hp); @@ -2261,13 +2151,12 @@ static netdev_tx_t happy_meal_start_xmit(struct sk_buff *skb, if (TX_BUFFS_AVAIL(hp) <= (skb_shinfo(skb)->nr_frags + 1)) { netif_stop_queue(dev); spin_unlock_irq(&hp->happy_lock); - printk(KERN_ERR "%s: BUG! Tx Ring full when queue awake!\n", - dev->name); + netdev_err(dev, "BUG! Tx Ring full when queue awake!\n"); return NETDEV_TX_BUSY; } entry = hp->tx_new; - SXD(("SX<l[%d]e[%d]>", len, entry)); + netdev_vdbg(dev, "SX<l[%d]e[%d]>\n", skb->len, entry); hp->tx_skbs[entry] = skb; if (skb_shinfo(skb)->nr_frags == 0) { @@ -2467,11 +2356,10 @@ static void hme_get_drvinfo(struct net_device *dev, struct ethtool_drvinfo *info { struct happy_meal *hp = netdev_priv(dev); - strlcpy(info->driver, "sunhme", sizeof(info->driver)); - strlcpy(info->version, "2.02", sizeof(info->version)); + strscpy(info->driver, DRV_NAME, sizeof(info->driver)); if (hp->happy_flags & HFLAG_PCI) { struct pci_dev *pdev = hp->happy_dev; - strlcpy(info->bus_info, pci_name(pdev), sizeof(info->bus_info)); + strscpy(info->bus_info, pci_name(pdev), sizeof(info->bus_info)); } #ifdef CONFIG_SBUS else { @@ -2504,8 +2392,6 @@ static const struct ethtool_ops hme_ethtool_ops = { .set_link_ksettings = hme_set_link_ksettings, }; -static int hme_version_printed; - #ifdef CONFIG_SBUS /* Given a happy meal sbus device, find it's quattro parent. * If none exist, allocate and return a new one. @@ -2523,19 +2409,15 @@ static struct quattro *quattro_sbus_find(struct platform_device *child) if (qp) return qp; - qp = kmalloc(sizeof(struct quattro), GFP_KERNEL); - if (qp != NULL) { - int i; - - for (i = 0; i < 4; i++) - qp->happy_meals[i] = NULL; + qp = kzalloc(sizeof(*qp), GFP_KERNEL); + if (!qp) + return NULL; - qp->quattro_dev = child; - qp->next = qfe_sbus_list; - qfe_sbus_list = qp; + qp->quattro_dev = child; + qp->next = qfe_sbus_list; + qfe_sbus_list = qp; - platform_set_drvdata(op, qp); - } + platform_set_drvdata(op, qp); return qp; } @@ -2563,8 +2445,9 @@ static int __init quattro_sbus_register_irqs(void) IRQF_SHARED, "Quattro", qp); if (err != 0) { - printk(KERN_ERR "Quattro HME: IRQ registration " - "error %d.\n", err); + dev_err(&op->dev, + "Quattro HME: IRQ registration error %d.\n", + err); return err; } } @@ -2595,30 +2478,33 @@ static void quattro_sbus_free_irqs(void) #ifdef CONFIG_PCI static struct quattro *quattro_pci_find(struct pci_dev *pdev) { + int i; struct pci_dev *bdev = pdev->bus->self; struct quattro *qp; - if (!bdev) return NULL; + if (!bdev) + return ERR_PTR(-ENODEV); + for (qp = qfe_pci_list; qp != NULL; qp = qp->next) { struct pci_dev *qpdev = qp->quattro_dev; if (qpdev == bdev) return qp; } + qp = kmalloc(sizeof(struct quattro), GFP_KERNEL); - if (qp != NULL) { - int i; + if (!qp) + return ERR_PTR(-ENOMEM); - for (i = 0; i < 4; i++) - qp->happy_meals[i] = NULL; + for (i = 0; i < 4; i++) + qp->happy_meals[i] = NULL; - qp->quattro_dev = bdev; - qp->next = qfe_pci_list; - qfe_pci_list = qp; + qp->quattro_dev = bdev; + qp->next = qfe_pci_list; + qfe_pci_list = qp; - /* No range tricks necessary on PCI. */ - qp->nranges = 0; - } + /* No range tricks necessary on PCI. */ + qp->nranges = 0; return qp; } #endif /* CONFIG_PCI */ @@ -2668,9 +2554,6 @@ static int happy_meal_sbus_probe_one(struct platform_device *op, int is_qfe) goto err_out; SET_NETDEV_DEV(dev, &op->dev); - if (hme_version_printed++ == 0) - printk(KERN_INFO "%s", version); - /* If user did not specify a MAC address specifically, use * the Quattro local-mac-address property... */ @@ -2712,35 +2595,35 @@ static int happy_meal_sbus_probe_one(struct platform_device *op, int is_qfe) hp->gregs = of_ioremap(&op->resource[0], 0, GREG_REG_SIZE, "HME Global Regs"); if (!hp->gregs) { - printk(KERN_ERR "happymeal: Cannot map global registers.\n"); + dev_err(&op->dev, "Cannot map global registers.\n"); goto err_out_free_netdev; } hp->etxregs = of_ioremap(&op->resource[1], 0, ETX_REG_SIZE, "HME TX Regs"); if (!hp->etxregs) { - printk(KERN_ERR "happymeal: Cannot map MAC TX registers.\n"); + dev_err(&op->dev, "Cannot map MAC TX registers.\n"); goto err_out_iounmap; } hp->erxregs = of_ioremap(&op->resource[2], 0, ERX_REG_SIZE, "HME RX Regs"); if (!hp->erxregs) { - printk(KERN_ERR "happymeal: Cannot map MAC RX registers.\n"); + dev_err(&op->dev, "Cannot map MAC RX registers.\n"); goto err_out_iounmap; } hp->bigmacregs = of_ioremap(&op->resource[3], 0, BMAC_REG_SIZE, "HME BIGMAC Regs"); if (!hp->bigmacregs) { - printk(KERN_ERR "happymeal: Cannot map BIGMAC registers.\n"); + dev_err(&op->dev, "Cannot map BIGMAC registers.\n"); goto err_out_iounmap; } hp->tcvregs = of_ioremap(&op->resource[4], 0, TCVR_REG_SIZE, "HME Tranceiver Regs"); if (!hp->tcvregs) { - printk(KERN_ERR "happymeal: Cannot map TCVR registers.\n"); + dev_err(&op->dev, "Cannot map TCVR registers.\n"); goto err_out_iounmap; } @@ -2807,21 +2690,19 @@ static int happy_meal_sbus_probe_one(struct platform_device *op, int is_qfe) err = register_netdev(hp->dev); if (err) { - printk(KERN_ERR "happymeal: Cannot register net device, " - "aborting.\n"); + dev_err(&op->dev, "Cannot register net device, aborting.\n"); goto err_out_free_coherent; } platform_set_drvdata(op, hp); if (qfe_slot != -1) - printk(KERN_INFO "%s: Quattro HME slot %d (SBUS) 10/100baseT Ethernet ", - dev->name, qfe_slot); + netdev_info(dev, + "Quattro HME slot %d (SBUS) 10/100baseT Ethernet %pM\n", + qfe_slot, dev->dev_addr); else - printk(KERN_INFO "%s: HAPPY MEAL (SBUS) 10/100baseT Ethernet ", - dev->name); - - printk("%pM\n", dev->dev_addr); + netdev_info(dev, "HAPPY MEAL (SBUS) 10/100baseT Ethernet %pM\n", + dev->dev_addr); return 0; @@ -2949,7 +2830,7 @@ static int happy_meal_pci_probe(struct pci_dev *pdev, struct happy_meal *hp; struct net_device *dev; void __iomem *hpreg_base; - unsigned long hpreg_res; + struct resource *hpreg_res; int i, qfe_slot = -1; char prom_name[64]; u8 addr[ETH_ALEN]; @@ -2966,32 +2847,33 @@ static int happy_meal_pci_probe(struct pci_dev *pdev, strcpy(prom_name, "SUNW,hme"); #endif - err = -ENODEV; - - if (pci_enable_device(pdev)) + err = pcim_enable_device(pdev); + if (err) goto err_out; pci_set_master(pdev); if (!strcmp(prom_name, "SUNW,qfe") || !strcmp(prom_name, "qfe")) { qp = quattro_pci_find(pdev); - if (qp == NULL) + if (IS_ERR(qp)) { + err = PTR_ERR(qp); goto err_out; + } + for (qfe_slot = 0; qfe_slot < 4; qfe_slot++) - if (qp->happy_meals[qfe_slot] == NULL) + if (!qp->happy_meals[qfe_slot]) break; + if (qfe_slot == 4) goto err_out; } - dev = alloc_etherdev(sizeof(struct happy_meal)); - err = -ENOMEM; - if (!dev) + dev = devm_alloc_etherdev(&pdev->dev, sizeof(struct happy_meal)); + if (!dev) { + err = -ENOMEM; goto err_out; + } SET_NETDEV_DEV(dev, &pdev->dev); - if (hme_version_printed++ == 0) - printk(KERN_INFO "%s", version); - hp = netdev_priv(dev); hp->happy_dev = pdev; @@ -3005,21 +2887,26 @@ static int happy_meal_pci_probe(struct pci_dev *pdev, qp->happy_meals[qfe_slot] = dev; } - hpreg_res = pci_resource_start(pdev, 0); - err = -ENODEV; + err = -EINVAL; if ((pci_resource_flags(pdev, 0) & IORESOURCE_IO) != 0) { - printk(KERN_ERR "happymeal(PCI): Cannot find proper PCI device base address.\n"); + dev_err(&pdev->dev, + "Cannot find proper PCI device base address.\n"); goto err_out_clear_quattro; } - if (pci_request_regions(pdev, DRV_NAME)) { - printk(KERN_ERR "happymeal(PCI): Cannot obtain PCI resources, " - "aborting.\n"); + + hpreg_res = devm_request_region(&pdev->dev, pci_resource_start(pdev, 0), + pci_resource_len(pdev, 0), DRV_NAME); + if (IS_ERR(hpreg_res)) { + err = PTR_ERR(hpreg_res); + dev_err(&pdev->dev, "Cannot obtain PCI resources, aborting.\n"); goto err_out_clear_quattro; } - if ((hpreg_base = ioremap(hpreg_res, 0x8000)) == NULL) { - printk(KERN_ERR "happymeal(PCI): Unable to remap card memory.\n"); - goto err_out_free_res; + hpreg_base = pcim_iomap(pdev, 0, 0x8000); + if (!hpreg_base) { + err = -ENOMEM; + dev_err(&pdev->dev, "Unable to remap card memory.\n"); + goto err_out_clear_quattro; } for (i = 0; i < 6; i++) { @@ -3085,11 +2972,12 @@ static int happy_meal_pci_probe(struct pci_dev *pdev, hp->happy_bursts = DMA_BURSTBITS; #endif - hp->happy_block = dma_alloc_coherent(&pdev->dev, PAGE_SIZE, - &hp->hblock_dvma, GFP_KERNEL); - err = -ENODEV; - if (!hp->happy_block) - goto err_out_iounmap; + hp->happy_block = dmam_alloc_coherent(&pdev->dev, PAGE_SIZE, + &hp->hblock_dvma, GFP_KERNEL); + if (!hp->happy_block) { + err = -ENOMEM; + goto err_out_clear_quattro; + } hp->linkcheck = 0; hp->timer_state = asleep; @@ -3123,11 +3011,10 @@ static int happy_meal_pci_probe(struct pci_dev *pdev, happy_meal_set_initial_advertisement(hp); spin_unlock_irq(&hp->happy_lock); - err = register_netdev(hp->dev); + err = devm_register_netdev(&pdev->dev, dev); if (err) { - printk(KERN_ERR "happymeal(PCI): Cannot register net device, " - "aborting.\n"); - goto err_out_free_coherent; + dev_err(&pdev->dev, "Cannot register net device, aborting.\n"); + goto err_out_clear_quattro; } pci_set_drvdata(pdev, hp); @@ -3140,61 +3027,30 @@ static int happy_meal_pci_probe(struct pci_dev *pdev, int i = simple_strtoul(dev->name + 3, NULL, 10); sprintf(prom_name, "-%d", i + 3); } - printk(KERN_INFO "%s%s: Quattro HME (PCI/CheerIO) 10/100baseT Ethernet ", dev->name, prom_name); - if (qpdev->vendor == PCI_VENDOR_ID_DEC && - qpdev->device == PCI_DEVICE_ID_DEC_21153) - printk("DEC 21153 PCI Bridge\n"); - else - printk("unknown bridge %04x.%04x\n", - qpdev->vendor, qpdev->device); + netdev_info(dev, + "%s: Quattro HME (PCI/CheerIO) 10/100baseT Ethernet bridge %04x.%04x\n", + prom_name, qpdev->vendor, qpdev->device); } if (qfe_slot != -1) - printk(KERN_INFO "%s: Quattro HME slot %d (PCI/CheerIO) 10/100baseT Ethernet ", - dev->name, qfe_slot); + netdev_info(dev, + "Quattro HME slot %d (PCI/CheerIO) 10/100baseT Ethernet %pM\n", + qfe_slot, dev->dev_addr); else - printk(KERN_INFO "%s: HAPPY MEAL (PCI/CheerIO) 10/100BaseT Ethernet ", - dev->name); - - printk("%pM\n", dev->dev_addr); + netdev_info(dev, + "HAPPY MEAL (PCI/CheerIO) 10/100BaseT Ethernet %pM\n", + dev->dev_addr); return 0; -err_out_free_coherent: - dma_free_coherent(hp->dma_dev, PAGE_SIZE, - hp->happy_block, hp->hblock_dvma); - -err_out_iounmap: - iounmap(hp->gregs); - -err_out_free_res: - pci_release_regions(pdev); - err_out_clear_quattro: if (qp != NULL) qp->happy_meals[qfe_slot] = NULL; - free_netdev(dev); - err_out: return err; } -static void happy_meal_pci_remove(struct pci_dev *pdev) -{ - struct happy_meal *hp = pci_get_drvdata(pdev); - struct net_device *net_dev = hp->dev; - - unregister_netdev(net_dev); - - dma_free_coherent(hp->dma_dev, PAGE_SIZE, - hp->happy_block, hp->hblock_dvma); - iounmap(hp->gregs); - pci_release_regions(hp->happy_dev); - - free_netdev(net_dev); -} - static const struct pci_device_id happymeal_pci_ids[] = { { PCI_DEVICE(PCI_VENDOR_ID_SUN, PCI_DEVICE_ID_SUN_HAPPYMEAL) }, { } /* Terminating entry */ @@ -3206,7 +3062,6 @@ static struct pci_driver hme_pci_driver = { .name = "hme", .id_table = happymeal_pci_ids, .probe = happy_meal_pci_probe, - .remove = happy_meal_pci_remove, }; static int __init happy_meal_pci_init(void) diff --git a/drivers/net/ethernet/sun/sunqe.c b/drivers/net/ethernet/sun/sunqe.c index efe0d33f6024..6418fcc3139f 100644 --- a/drivers/net/ethernet/sun/sunqe.c +++ b/drivers/net/ethernet/sun/sunqe.c @@ -684,8 +684,8 @@ static void qe_get_drvinfo(struct net_device *dev, struct ethtool_drvinfo *info) struct sunqe *qep = netdev_priv(dev); struct platform_device *op; - strlcpy(info->driver, "sunqe", sizeof(info->driver)); - strlcpy(info->version, "3.0", sizeof(info->version)); + strscpy(info->driver, "sunqe", sizeof(info->driver)); + strscpy(info->version, "3.0", sizeof(info->version)); op = qep->op; regs = of_get_property(op->dev.of_node, "reg", NULL); diff --git a/drivers/net/ethernet/sun/sunvnet.c b/drivers/net/ethernet/sun/sunvnet.c index da8119625cf3..acda6cbd0238 100644 --- a/drivers/net/ethernet/sun/sunvnet.c +++ b/drivers/net/ethernet/sun/sunvnet.c @@ -60,8 +60,8 @@ static struct vio_version vnet_versions[] = { static void vnet_get_drvinfo(struct net_device *dev, struct ethtool_drvinfo *info) { - strlcpy(info->driver, DRV_MODULE_NAME, sizeof(info->driver)); - strlcpy(info->version, DRV_MODULE_VERSION, sizeof(info->version)); + strscpy(info->driver, DRV_MODULE_NAME, sizeof(info->driver)); + strscpy(info->version, DRV_MODULE_VERSION, sizeof(info->version)); } static u32 vnet_get_msglevel(struct net_device *dev) @@ -467,8 +467,7 @@ static int vnet_port_probe(struct vio_dev *vdev, const struct vio_device_id *id) if (err) goto err_out_free_port; - netif_napi_add(port->vp->dev, &port->napi, sunvnet_poll_common, - NAPI_POLL_WEIGHT); + netif_napi_add(port->vp->dev, &port->napi, sunvnet_poll_common); INIT_HLIST_NODE(&port->hash); INIT_LIST_HEAD(&port->list); diff --git a/drivers/net/ethernet/sunplus/spl2sw_driver.c b/drivers/net/ethernet/sunplus/spl2sw_driver.c index 546206640492..9be585237277 100644 --- a/drivers/net/ethernet/sunplus/spl2sw_driver.c +++ b/drivers/net/ethernet/sunplus/spl2sw_driver.c @@ -62,7 +62,8 @@ static int spl2sw_ethernet_stop(struct net_device *ndev) return 0; } -static int spl2sw_ethernet_start_xmit(struct sk_buff *skb, struct net_device *ndev) +static netdev_tx_t spl2sw_ethernet_start_xmit(struct sk_buff *skb, + struct net_device *ndev) { struct spl2sw_mac *mac = netdev_priv(ndev); struct spl2sw_common *comm = mac->comm; @@ -248,8 +249,8 @@ static int spl2sw_nvmem_get_mac_address(struct device *dev, struct device_node * /* Check if mac address is valid */ if (!is_valid_ether_addr(mac)) { - kfree(mac); dev_info(dev, "Invalid mac address in nvmem (%pM)!\n", mac); + kfree(mac); return -EINVAL; } @@ -492,7 +493,7 @@ static int spl2sw_probe(struct platform_device *pdev) } /* Add and enable napi. */ - netif_napi_add(ndev, &comm->rx_napi, spl2sw_rx_poll, NAPI_POLL_WEIGHT); + netif_napi_add(ndev, &comm->rx_napi, spl2sw_rx_poll); napi_enable(&comm->rx_napi); netif_napi_add_tx(ndev, &comm->tx_napi, spl2sw_tx_poll); napi_enable(&comm->tx_napi); diff --git a/drivers/net/ethernet/synopsys/dwc-xlgmac-common.c b/drivers/net/ethernet/synopsys/dwc-xlgmac-common.c index 5c9b6c90942b..f8e133604146 100644 --- a/drivers/net/ethernet/synopsys/dwc-xlgmac-common.c +++ b/drivers/net/ethernet/synopsys/dwc-xlgmac-common.c @@ -54,8 +54,8 @@ static void xlgmac_default_config(struct xlgmac_pdata *pdata) pdata->phy_speed = SPEED_25000; pdata->sysclk_rate = XLGMAC_SYSCLOCK; - strlcpy(pdata->drv_name, XLGMAC_DRV_NAME, sizeof(pdata->drv_name)); - strlcpy(pdata->drv_ver, XLGMAC_DRV_VERSION, sizeof(pdata->drv_ver)); + strscpy(pdata->drv_name, XLGMAC_DRV_NAME, sizeof(pdata->drv_name)); + strscpy(pdata->drv_ver, XLGMAC_DRV_VERSION, sizeof(pdata->drv_ver)); } static void xlgmac_init_all_ops(struct xlgmac_pdata *pdata) diff --git a/drivers/net/ethernet/synopsys/dwc-xlgmac-ethtool.c b/drivers/net/ethernet/synopsys/dwc-xlgmac-ethtool.c index 49f8c6be9459..e794da727fe0 100644 --- a/drivers/net/ethernet/synopsys/dwc-xlgmac-ethtool.c +++ b/drivers/net/ethernet/synopsys/dwc-xlgmac-ethtool.c @@ -102,9 +102,9 @@ static void xlgmac_ethtool_get_drvinfo(struct net_device *netdev, u32 ver = pdata->hw_feat.version; u32 snpsver, devid, userver; - strlcpy(drvinfo->driver, pdata->drv_name, sizeof(drvinfo->driver)); - strlcpy(drvinfo->version, pdata->drv_ver, sizeof(drvinfo->version)); - strlcpy(drvinfo->bus_info, dev_name(pdata->dev), + strscpy(drvinfo->driver, pdata->drv_name, sizeof(drvinfo->driver)); + strscpy(drvinfo->version, pdata->drv_ver, sizeof(drvinfo->version)); + strscpy(drvinfo->bus_info, dev_name(pdata->dev), sizeof(drvinfo->bus_info)); /* S|SNPSVER: Synopsys-defined Version * D|DEVID: Indicates the Device family diff --git a/drivers/net/ethernet/synopsys/dwc-xlgmac-net.c b/drivers/net/ethernet/synopsys/dwc-xlgmac-net.c index e54ce73396ee..36b948820c1e 100644 --- a/drivers/net/ethernet/synopsys/dwc-xlgmac-net.c +++ b/drivers/net/ethernet/synopsys/dwc-xlgmac-net.c @@ -419,15 +419,14 @@ static void xlgmac_napi_enable(struct xlgmac_pdata *pdata, unsigned int add) for (i = 0; i < pdata->channel_count; i++, channel++) { if (add) netif_napi_add(pdata->netdev, &channel->napi, - xlgmac_one_poll, - NAPI_POLL_WEIGHT); + xlgmac_one_poll); napi_enable(&channel->napi); } } else { if (add) netif_napi_add(pdata->netdev, &pdata->napi, - xlgmac_all_poll, NAPI_POLL_WEIGHT); + xlgmac_all_poll); napi_enable(&pdata->napi); } diff --git a/drivers/net/ethernet/tehuti/tehuti.c b/drivers/net/ethernet/tehuti/tehuti.c index 985073eba3bd..ca409515ead5 100644 --- a/drivers/net/ethernet/tehuti/tehuti.c +++ b/drivers/net/ethernet/tehuti/tehuti.c @@ -1994,7 +1994,7 @@ bdx_probe(struct pci_dev *pdev, const struct pci_device_id *ent) priv->nic = nic; priv->msg_enable = BDX_DEF_MSG_ENABLE; - netif_napi_add(ndev, &priv->napi, bdx_poll, 64); + netif_napi_add(ndev, &priv->napi, bdx_poll); if ((readl(nic->regs + FPGA_VER) & 0xFFF) == 308) { DBG("HW statistics not supported\n"); @@ -2133,10 +2133,10 @@ bdx_get_drvinfo(struct net_device *netdev, struct ethtool_drvinfo *drvinfo) { struct bdx_priv *priv = netdev_priv(netdev); - strlcpy(drvinfo->driver, BDX_DRV_NAME, sizeof(drvinfo->driver)); - strlcpy(drvinfo->version, BDX_DRV_VERSION, sizeof(drvinfo->version)); - strlcpy(drvinfo->fw_version, "N/A", sizeof(drvinfo->fw_version)); - strlcpy(drvinfo->bus_info, pci_name(priv->pdev), + strscpy(drvinfo->driver, BDX_DRV_NAME, sizeof(drvinfo->driver)); + strscpy(drvinfo->version, BDX_DRV_VERSION, sizeof(drvinfo->version)); + strscpy(drvinfo->fw_version, "N/A", sizeof(drvinfo->fw_version)); + strscpy(drvinfo->bus_info, pci_name(priv->pdev), sizeof(drvinfo->bus_info)); } diff --git a/drivers/net/ethernet/ti/Kconfig b/drivers/net/ethernet/ti/Kconfig index fb30bc5d56cb..fce06663e1e1 100644 --- a/drivers/net/ethernet/ti/Kconfig +++ b/drivers/net/ethernet/ti/Kconfig @@ -33,6 +33,7 @@ config TI_DAVINCI_MDIO tristate "TI DaVinci MDIO Support" depends on ARCH_DAVINCI || ARCH_OMAP2PLUS || ARCH_KEYSTONE || ARCH_K3 || COMPILE_TEST select PHYLIB + select MDIO_BITBANG help This driver supports TI's DaVinci MDIO module. diff --git a/drivers/net/ethernet/ti/am65-cpsw-ethtool.c b/drivers/net/ethernet/ti/am65-cpsw-ethtool.c index abc1e4276cf0..c51e2af91f69 100644 --- a/drivers/net/ethernet/ti/am65-cpsw-ethtool.c +++ b/drivers/net/ethernet/ti/am65-cpsw-ethtool.c @@ -402,9 +402,9 @@ static void am65_cpsw_get_drvinfo(struct net_device *ndev, { struct am65_cpsw_common *common = am65_ndev_to_common(ndev); - strlcpy(info->driver, dev_driver_string(common->dev), + strscpy(info->driver, dev_driver_string(common->dev), sizeof(info->driver)); - strlcpy(info->bus_info, dev_name(common->dev), sizeof(info->bus_info)); + strscpy(info->bus_info, dev_name(common->dev), sizeof(info->bus_info)); } static u32 am65_cpsw_get_msglevel(struct net_device *ndev) diff --git a/drivers/net/ethernet/ti/am65-cpsw-nuss.c b/drivers/net/ethernet/ti/am65-cpsw-nuss.c index f4a6b590a1e3..3cbe4ec46234 100644 --- a/drivers/net/ethernet/ti/am65-cpsw-nuss.c +++ b/drivers/net/ethernet/ti/am65-cpsw-nuss.c @@ -74,6 +74,9 @@ #define AM65_CPSW_PORTN_REG_TS_VLAN_LTYPE_REG 0x318 #define AM65_CPSW_PORTN_REG_TS_CTL_LTYPE2 0x31C +#define AM65_CPSW_SGMII_CONTROL_REG 0x010 +#define AM65_CPSW_SGMII_CONTROL_MR_AN_ENABLE BIT(0) + #define AM65_CPSW_CTL_VLAN_AWARE BIT(1) #define AM65_CPSW_CTL_P0_ENABLE BIT(2) #define AM65_CPSW_CTL_P0_TX_CRC_REMOVE BIT(13) @@ -360,8 +363,7 @@ static void am65_cpsw_init_host_port_emac(struct am65_cpsw_common *common); static void am65_cpsw_init_port_switch_ale(struct am65_cpsw_port *port); static void am65_cpsw_init_port_emac_ale(struct am65_cpsw_port *port); -static int am65_cpsw_nuss_common_open(struct am65_cpsw_common *common, - netdev_features_t features) +static int am65_cpsw_nuss_common_open(struct am65_cpsw_common *common) { struct am65_cpsw_host *host_p = am65_common_get_host(common); int port_idx, i, ret; @@ -574,7 +576,7 @@ static int am65_cpsw_nuss_ndo_slave_open(struct net_device *ndev) for (i = 0; i < common->tx_ch_num; i++) netdev_tx_reset_queue(netdev_get_tx_queue(ndev, i)); - ret = am65_cpsw_nuss_common_open(common, ndev->features); + ret = am65_cpsw_nuss_common_open(common); if (ret) return ret; @@ -590,11 +592,6 @@ static int am65_cpsw_nuss_ndo_slave_open(struct net_device *ndev) /* mac_sl should be configured via phy-link interface */ am65_cpsw_sl_ctl_reset(port); - ret = phy_set_mode_ext(port->slave.ifphy, PHY_MODE_ETHERNET, - port->slave.phy_if); - if (ret) - goto error_cleanup; - ret = phylink_of_phy_connect(port->slave.phylink, port->slave.phy_node, 0); if (ret) goto error_cleanup; @@ -1409,7 +1406,14 @@ static const struct net_device_ops am65_cpsw_nuss_netdev_ops = { static void am65_cpsw_nuss_mac_config(struct phylink_config *config, unsigned int mode, const struct phylink_link_state *state) { - /* Currently not used */ + struct am65_cpsw_slave_data *slave = container_of(config, struct am65_cpsw_slave_data, + phylink_config); + struct am65_cpsw_port *port = container_of(slave, struct am65_cpsw_port, slave); + struct am65_cpsw_common *common = port->common; + + if (common->pdata.extra_modes & BIT(state->interface)) + writel(AM65_CPSW_SGMII_CONTROL_MR_AN_ENABLE, + port->sgmii_base + AM65_CPSW_SGMII_CONTROL_REG); } static void am65_cpsw_nuss_mac_link_down(struct phylink_config *config, unsigned int mode, @@ -1847,6 +1851,8 @@ static int am65_cpsw_nuss_init_slave_ports(struct am65_cpsw_common *common) port->common = common; port->port_base = common->cpsw_base + AM65_CPSW_NU_PORTS_BASE + AM65_CPSW_NU_PORTS_OFFSET * (port_id); + if (common->pdata.extra_modes) + port->sgmii_base = common->ss_base + AM65_CPSW_SGMII_BASE * (port_id); port->stat_base = common->cpsw_base + AM65_CPSW_NU_STATS_BASE + (AM65_CPSW_NU_STATS_PORT_OFFSET * port_id); port->name = of_get_property(port_np, "label", NULL); @@ -1886,6 +1892,10 @@ static int am65_cpsw_nuss_init_slave_ports(struct am65_cpsw_common *common) goto of_node_put; } + ret = phy_set_mode_ext(port->slave.ifphy, PHY_MODE_ETHERNET, port->slave.phy_if); + if (ret) + goto of_node_put; + ret = of_get_mac_address(port_np, port->slave.mac_addr); if (ret) { am65_cpsw_am654_get_efuse_macid(port_np, @@ -1981,7 +1991,18 @@ am65_cpsw_nuss_init_port_ndev(struct am65_cpsw_common *common, u32 port_idx) port->slave.phylink_config.type = PHYLINK_NETDEV; port->slave.phylink_config.mac_capabilities = MAC_SYM_PAUSE | MAC_10 | MAC_100 | MAC_1000FD; - phy_interface_set_rgmii(port->slave.phylink_config.supported_interfaces); + if (phy_interface_mode_is_rgmii(port->slave.phy_if)) { + phy_interface_set_rgmii(port->slave.phylink_config.supported_interfaces); + } else if (port->slave.phy_if == PHY_INTERFACE_MODE_RMII) { + __set_bit(PHY_INTERFACE_MODE_RMII, + port->slave.phylink_config.supported_interfaces); + } else if (common->pdata.extra_modes & BIT(port->slave.phy_if)) { + __set_bit(PHY_INTERFACE_MODE_QSGMII, + port->slave.phylink_config.supported_interfaces); + } else { + dev_err(dev, "selected phy-mode is not supported\n"); + return -EOPNOTSUPP; + } phylink = phylink_create(&port->slave.phylink_config, of_node_to_fwnode(port->slave.phy_node), @@ -2023,7 +2044,7 @@ static int am65_cpsw_nuss_init_ndevs(struct am65_cpsw_common *common) } netif_napi_add(common->dma_ndev, &common->napi_rx, - am65_cpsw_nuss_rx_poll, NAPI_POLL_WEIGHT); + am65_cpsw_nuss_rx_poll); return ret; } @@ -2611,10 +2632,18 @@ static const struct am65_cpsw_pdata am64x_cpswxg_pdata = { .fdqring_mode = K3_RINGACC_RING_MODE_RING, }; +static const struct am65_cpsw_pdata j7200_cpswxg_pdata = { + .quirks = 0, + .ale_dev_id = "am64-cpswxg", + .fdqring_mode = K3_RINGACC_RING_MODE_RING, + .extra_modes = BIT(PHY_INTERFACE_MODE_QSGMII), +}; + static const struct of_device_id am65_cpsw_nuss_of_mtable[] = { { .compatible = "ti,am654-cpsw-nuss", .data = &am65x_sr1_0}, { .compatible = "ti,j721e-cpsw-nuss", .data = &j721e_pdata}, { .compatible = "ti,am642-cpsw-nuss", .data = &am64x_cpswxg_pdata}, + { .compatible = "ti,j7200-cpswxg-nuss", .data = &j7200_cpswxg_pdata}, { /* sentinel */ }, }; MODULE_DEVICE_TABLE(of, am65_cpsw_nuss_of_mtable); diff --git a/drivers/net/ethernet/ti/am65-cpsw-nuss.h b/drivers/net/ethernet/ti/am65-cpsw-nuss.h index ac945631bf2f..2c9850fdfcb6 100644 --- a/drivers/net/ethernet/ti/am65-cpsw-nuss.h +++ b/drivers/net/ethernet/ti/am65-cpsw-nuss.h @@ -46,6 +46,7 @@ struct am65_cpsw_port { const char *name; u32 port_id; void __iomem *port_base; + void __iomem *sgmii_base; void __iomem *stat_base; void __iomem *fetch_ram_base; bool disabled; @@ -88,6 +89,7 @@ struct am65_cpsw_rx_chn { struct am65_cpsw_pdata { u32 quirks; + u64 extra_modes; enum k3_ring_mode fdqring_mode; const char *ale_dev_id; }; diff --git a/drivers/net/ethernet/ti/am65-cpts.c b/drivers/net/ethernet/ti/am65-cpts.c index c30a6e510aa3..e2f0fb286143 100644 --- a/drivers/net/ethernet/ti/am65-cpts.c +++ b/drivers/net/ethernet/ti/am65-cpts.c @@ -943,9 +943,7 @@ struct am65_cpts *am65_cpts_create(struct device *dev, void __iomem *regs, cpts->irq = of_irq_get_byname(node, "cpts"); if (cpts->irq <= 0) { ret = cpts->irq ?: -ENXIO; - if (ret != -EPROBE_DEFER) - dev_err(dev, "Failed to get IRQ number (err = %d)\n", - ret); + dev_err_probe(dev, ret, "Failed to get IRQ number\n"); return ERR_PTR(ret); } @@ -965,8 +963,7 @@ struct am65_cpts *am65_cpts_create(struct device *dev, void __iomem *regs, cpts->refclk = devm_get_clk_from_child(dev, node, "cpts"); if (IS_ERR(cpts->refclk)) { ret = PTR_ERR(cpts->refclk); - if (ret != -EPROBE_DEFER) - dev_err(dev, "Failed to get refclk %d\n", ret); + dev_err_probe(dev, ret, "Failed to get refclk\n"); return ERR_PTR(ret); } diff --git a/drivers/net/ethernet/ti/cpmac.c b/drivers/net/ethernet/ti/cpmac.c index bef5e68dac31..80eeeb463c4f 100644 --- a/drivers/net/ethernet/ti/cpmac.c +++ b/drivers/net/ethernet/ti/cpmac.c @@ -851,8 +851,8 @@ static int cpmac_set_ringparam(struct net_device *dev, static void cpmac_get_drvinfo(struct net_device *dev, struct ethtool_drvinfo *info) { - strlcpy(info->driver, "cpmac", sizeof(info->driver)); - strlcpy(info->version, CPMAC_VERSION, sizeof(info->version)); + strscpy(info->driver, "cpmac", sizeof(info->driver)); + strscpy(info->version, CPMAC_VERSION, sizeof(info->version)); snprintf(info->bus_info, sizeof(info->bus_info), "%s", "cpmac"); } @@ -1109,7 +1109,7 @@ static int cpmac_probe(struct platform_device *pdev) dev->netdev_ops = &cpmac_netdev_ops; dev->ethtool_ops = &cpmac_ethtool_ops; - netif_napi_add(dev, &priv->napi, cpmac_poll, 64); + netif_napi_add(dev, &priv->napi, cpmac_poll); spin_lock_init(&priv->lock); spin_lock_init(&priv->rx_lock); @@ -1169,7 +1169,7 @@ static struct platform_driver cpmac_driver = { .remove = cpmac_remove, }; -int cpmac_init(void) +int __init cpmac_init(void) { u32 mask; int i, res; @@ -1239,7 +1239,7 @@ fail_alloc: return res; } -void cpmac_exit(void) +void __exit cpmac_exit(void) { platform_driver_unregister(&cpmac_driver); mdiobus_unregister(cpmac_mii); diff --git a/drivers/net/ethernet/ti/cpsw.c b/drivers/net/ethernet/ti/cpsw.c index ed66c4d4d830..709ca6dd6ecb 100644 --- a/drivers/net/ethernet/ti/cpsw.c +++ b/drivers/net/ethernet/ti/cpsw.c @@ -1172,9 +1172,9 @@ static void cpsw_get_drvinfo(struct net_device *ndev, struct cpsw_common *cpsw = ndev_to_cpsw(ndev); struct platform_device *pdev = to_platform_device(cpsw->dev); - strlcpy(info->driver, "cpsw", sizeof(info->driver)); - strlcpy(info->version, "1.0", sizeof(info->version)); - strlcpy(info->bus_info, pdev->name, sizeof(info->bus_info)); + strscpy(info->driver, "cpsw", sizeof(info->driver)); + strscpy(info->version, "1.0", sizeof(info->version)); + strscpy(info->bus_info, pdev->name, sizeof(info->bus_info)); } static int cpsw_set_pauseparam(struct net_device *ndev, @@ -1319,8 +1319,7 @@ static int cpsw_probe_dt(struct cpsw_platform_data *data, */ ret = of_phy_register_fixed_link(slave_node); if (ret) { - if (ret != -EPROBE_DEFER) - dev_err(&pdev->dev, "failed to register fixed-link phy: %d\n", ret); + dev_err_probe(&pdev->dev, ret, "failed to register fixed-link phy\n"); goto err_node_put; } slave_data->phy_node = of_node_get(slave_node); @@ -1638,8 +1637,7 @@ static int cpsw_probe(struct platform_device *pdev) ndev->netdev_ops = &cpsw_netdev_ops; ndev->ethtool_ops = &cpsw_ethtool_ops; netif_napi_add(ndev, &cpsw->napi_rx, - cpsw->quirk_irq ? cpsw_rx_poll : cpsw_rx_mq_poll, - NAPI_POLL_WEIGHT); + cpsw->quirk_irq ? cpsw_rx_poll : cpsw_rx_mq_poll); netif_napi_add_tx(ndev, &cpsw->napi_tx, cpsw->quirk_irq ? cpsw_tx_poll : cpsw_tx_mq_poll); diff --git a/drivers/net/ethernet/ti/cpsw_new.c b/drivers/net/ethernet/ti/cpsw_new.c index 353e58b22c51..83596ec0c7cb 100644 --- a/drivers/net/ethernet/ti/cpsw_new.c +++ b/drivers/net/ethernet/ti/cpsw_new.c @@ -1146,9 +1146,9 @@ static void cpsw_get_drvinfo(struct net_device *ndev, struct platform_device *pdev; pdev = to_platform_device(cpsw->dev); - strlcpy(info->driver, "cpsw-switch", sizeof(info->driver)); - strlcpy(info->version, "2.0", sizeof(info->version)); - strlcpy(info->bus_info, pdev->name, sizeof(info->bus_info)); + strscpy(info->driver, "cpsw-switch", sizeof(info->driver)); + strscpy(info->version, "2.0", sizeof(info->version)); + strscpy(info->bus_info, pdev->name, sizeof(info->bus_info)); } static int cpsw_set_pauseparam(struct net_device *ndev, @@ -1288,9 +1288,8 @@ static int cpsw_probe_dt(struct cpsw_common *cpsw) if (of_phy_is_fixed_link(port_np)) { ret = of_phy_register_fixed_link(port_np); if (ret) { - if (ret != -EPROBE_DEFER) - dev_err(dev, "%pOF failed to register fixed-link phy: %d\n", - port_np, ret); + dev_err_probe(dev, ret, "%pOF failed to register fixed-link phy\n", + port_np); goto err_node_put; } slave_data->phy_node = of_node_get(port_np); @@ -1417,9 +1416,7 @@ static int cpsw_create_ports(struct cpsw_common *cpsw) * accordingly. */ netif_napi_add(ndev, &cpsw->napi_rx, - cpsw->quirk_irq ? - cpsw_rx_poll : cpsw_rx_mq_poll, - NAPI_POLL_WEIGHT); + cpsw->quirk_irq ? cpsw_rx_poll : cpsw_rx_mq_poll); netif_napi_add_tx(ndev, &cpsw->napi_tx, cpsw->quirk_irq ? cpsw_tx_poll : cpsw_tx_mq_poll); diff --git a/drivers/net/ethernet/ti/davinci_emac.c b/drivers/net/ethernet/ti/davinci_emac.c index 2a3e4e842fa5..2eb9d5a32588 100644 --- a/drivers/net/ethernet/ti/davinci_emac.c +++ b/drivers/net/ethernet/ti/davinci_emac.c @@ -374,8 +374,8 @@ static char *emac_rxhost_errcodes[16] = { static void emac_get_drvinfo(struct net_device *ndev, struct ethtool_drvinfo *info) { - strlcpy(info->driver, emac_version_string, sizeof(info->driver)); - strlcpy(info->version, EMAC_MODULE_VERSION, sizeof(info->version)); + strscpy(info->driver, emac_version_string, sizeof(info->driver)); + strscpy(info->version, EMAC_MODULE_VERSION, sizeof(info->version)); } /** @@ -949,7 +949,7 @@ static void emac_tx_handler(void *token, int len, int status) * * Returns success(NETDEV_TX_OK) or error code (typically out of desc's) */ -static int emac_dev_xmit(struct sk_buff *skb, struct net_device *ndev) +static netdev_tx_t emac_dev_xmit(struct sk_buff *skb, struct net_device *ndev) { struct device *emac_dev = &ndev->dev; int ret_code; @@ -1948,7 +1948,7 @@ static int davinci_emac_probe(struct platform_device *pdev) ndev->netdev_ops = &emac_netdev_ops; ndev->ethtool_ops = ðtool_ops; - netif_napi_add(ndev, &priv->napi, emac_poll, NAPI_POLL_WEIGHT); + netif_napi_add(ndev, &priv->napi, emac_poll); pm_runtime_enable(&pdev->dev); rc = pm_runtime_resume_and_get(&pdev->dev); diff --git a/drivers/net/ethernet/ti/davinci_mdio.c b/drivers/net/ethernet/ti/davinci_mdio.c index ea3772618043..946b9753ccfb 100644 --- a/drivers/net/ethernet/ti/davinci_mdio.c +++ b/drivers/net/ethernet/ti/davinci_mdio.c @@ -26,6 +26,8 @@ #include <linux/of_device.h> #include <linux/of_mdio.h> #include <linux/pinctrl/consumer.h> +#include <linux/mdio-bitbang.h> +#include <linux/sys_soc.h> /* * This timeout definition is a worst-case ultra defensive measure against @@ -41,6 +43,7 @@ struct davinci_mdio_of_param { int autosuspend_delay_ms; + bool manual_mode; }; struct davinci_mdio_regs { @@ -49,6 +52,15 @@ struct davinci_mdio_regs { #define CONTROL_IDLE BIT(31) #define CONTROL_ENABLE BIT(30) #define CONTROL_MAX_DIV (0xffff) +#define CONTROL_CLKDIV GENMASK(15, 0) + +#define MDIO_MAN_MDCLK_O BIT(2) +#define MDIO_MAN_OE BIT(1) +#define MDIO_MAN_PIN BIT(0) +#define MDIO_MANUALMODE BIT(31) + +#define MDIO_PIN 0 + u32 alive; u32 link; @@ -59,7 +71,9 @@ struct davinci_mdio_regs { u32 userintmasked; u32 userintmaskset; u32 userintmaskclr; - u32 __reserved_1[20]; + u32 manualif; + u32 poll; + u32 __reserved_1[18]; struct { u32 access; @@ -79,6 +93,7 @@ static const struct mdio_platform_data default_pdata = { struct davinci_mdio_data { struct mdio_platform_data pdata; + struct mdiobb_ctrl bb_ctrl; struct davinci_mdio_regs __iomem *regs; struct clk *clk; struct device *dev; @@ -90,6 +105,7 @@ struct davinci_mdio_data { */ bool skip_scan; u32 clk_div; + bool manual_mode; }; static void davinci_mdio_init_clk(struct davinci_mdio_data *data) @@ -128,9 +144,122 @@ static void davinci_mdio_enable(struct davinci_mdio_data *data) writel(data->clk_div | CONTROL_ENABLE, &data->regs->control); } -static int davinci_mdio_reset(struct mii_bus *bus) +static void davinci_mdio_disable(struct davinci_mdio_data *data) +{ + u32 reg; + + /* Disable MDIO state machine */ + reg = readl(&data->regs->control); + + reg &= ~CONTROL_CLKDIV; + reg |= data->clk_div; + + reg &= ~CONTROL_ENABLE; + writel(reg, &data->regs->control); +} + +static void davinci_mdio_enable_manual_mode(struct davinci_mdio_data *data) +{ + u32 reg; + /* set manual mode */ + reg = readl(&data->regs->poll); + reg |= MDIO_MANUALMODE; + writel(reg, &data->regs->poll); +} + +static void davinci_set_mdc(struct mdiobb_ctrl *ctrl, int level) +{ + struct davinci_mdio_data *data; + u32 reg; + + data = container_of(ctrl, struct davinci_mdio_data, bb_ctrl); + reg = readl(&data->regs->manualif); + + if (level) + reg |= MDIO_MAN_MDCLK_O; + else + reg &= ~MDIO_MAN_MDCLK_O; + + writel(reg, &data->regs->manualif); +} + +static void davinci_set_mdio_dir(struct mdiobb_ctrl *ctrl, int output) +{ + struct davinci_mdio_data *data; + u32 reg; + + data = container_of(ctrl, struct davinci_mdio_data, bb_ctrl); + reg = readl(&data->regs->manualif); + + if (output) + reg |= MDIO_MAN_OE; + else + reg &= ~MDIO_MAN_OE; + + writel(reg, &data->regs->manualif); +} + +static void davinci_set_mdio_data(struct mdiobb_ctrl *ctrl, int value) +{ + struct davinci_mdio_data *data; + u32 reg; + + data = container_of(ctrl, struct davinci_mdio_data, bb_ctrl); + reg = readl(&data->regs->manualif); + + if (value) + reg |= MDIO_MAN_PIN; + else + reg &= ~MDIO_MAN_PIN; + + writel(reg, &data->regs->manualif); +} + +static int davinci_get_mdio_data(struct mdiobb_ctrl *ctrl) +{ + struct davinci_mdio_data *data; + unsigned long reg; + + data = container_of(ctrl, struct davinci_mdio_data, bb_ctrl); + reg = readl(&data->regs->manualif); + return test_bit(MDIO_PIN, ®); +} + +static int davinci_mdiobb_read(struct mii_bus *bus, int phy, int reg) +{ + int ret; + + ret = pm_runtime_resume_and_get(bus->parent); + if (ret < 0) + return ret; + + ret = mdiobb_read(bus, phy, reg); + + pm_runtime_mark_last_busy(bus->parent); + pm_runtime_put_autosuspend(bus->parent); + + return ret; +} + +static int davinci_mdiobb_write(struct mii_bus *bus, int phy, int reg, + u16 val) +{ + int ret; + + ret = pm_runtime_resume_and_get(bus->parent); + if (ret < 0) + return ret; + + ret = mdiobb_write(bus, phy, reg, val); + + pm_runtime_mark_last_busy(bus->parent); + pm_runtime_put_autosuspend(bus->parent); + + return ret; +} + +static int davinci_mdio_common_reset(struct davinci_mdio_data *data) { - struct davinci_mdio_data *data = bus->priv; u32 phy_mask, ver; int ret; @@ -138,6 +267,11 @@ static int davinci_mdio_reset(struct mii_bus *bus) if (ret < 0) return ret; + if (data->manual_mode) { + davinci_mdio_disable(data); + davinci_mdio_enable_manual_mode(data); + } + /* wait for scan logic to settle */ msleep(PHY_MAX_ADDR * data->access_time); @@ -171,6 +305,23 @@ done: return 0; } +static int davinci_mdio_reset(struct mii_bus *bus) +{ + struct davinci_mdio_data *data = bus->priv; + + return davinci_mdio_common_reset(data); +} + +static int davinci_mdiobb_reset(struct mii_bus *bus) +{ + struct mdiobb_ctrl *ctrl = bus->priv; + struct davinci_mdio_data *data; + + data = container_of(ctrl, struct davinci_mdio_data, bb_ctrl); + + return davinci_mdio_common_reset(data); +} + /* wait until hardware is ready for another user access */ static inline int wait_for_user_access(struct davinci_mdio_data *data) { @@ -318,6 +469,28 @@ static int davinci_mdio_probe_dt(struct mdio_platform_data *data, return 0; } +struct k3_mdio_soc_data { + bool manual_mode; +}; + +static const struct k3_mdio_soc_data am65_mdio_soc_data = { + .manual_mode = true, +}; + +static const struct soc_device_attribute k3_mdio_socinfo[] = { + { .family = "AM62X", .revision = "SR1.0", .data = &am65_mdio_soc_data }, + { .family = "AM64X", .revision = "SR1.0", .data = &am65_mdio_soc_data }, + { .family = "AM64X", .revision = "SR2.0", .data = &am65_mdio_soc_data }, + { .family = "AM65X", .revision = "SR1.0", .data = &am65_mdio_soc_data }, + { .family = "AM65X", .revision = "SR2.0", .data = &am65_mdio_soc_data }, + { .family = "J7200", .revision = "SR1.0", .data = &am65_mdio_soc_data }, + { .family = "J7200", .revision = "SR2.0", .data = &am65_mdio_soc_data }, + { .family = "J721E", .revision = "SR1.0", .data = &am65_mdio_soc_data }, + { .family = "J721E", .revision = "SR2.0", .data = &am65_mdio_soc_data }, + { .family = "J721S2", .revision = "SR1.0", .data = &am65_mdio_soc_data}, + { /* sentinel */ }, +}; + #if IS_ENABLED(CONFIG_OF) static const struct davinci_mdio_of_param of_cpsw_mdio_data = { .autosuspend_delay_ms = 100, @@ -331,6 +504,14 @@ static const struct of_device_id davinci_mdio_of_mtable[] = { MODULE_DEVICE_TABLE(of, davinci_mdio_of_mtable); #endif +static const struct mdiobb_ops davinci_mdiobb_ops = { + .owner = THIS_MODULE, + .set_mdc = davinci_set_mdc, + .set_mdio_dir = davinci_set_mdio_dir, + .set_mdio_data = davinci_set_mdio_data, + .get_mdio_data = davinci_get_mdio_data, +}; + static int davinci_mdio_probe(struct platform_device *pdev) { struct mdio_platform_data *pdata = dev_get_platdata(&pdev->dev); @@ -345,7 +526,26 @@ static int davinci_mdio_probe(struct platform_device *pdev) if (!data) return -ENOMEM; - data->bus = devm_mdiobus_alloc(dev); + data->manual_mode = false; + data->bb_ctrl.ops = &davinci_mdiobb_ops; + + if (IS_ENABLED(CONFIG_OF) && dev->of_node) { + const struct soc_device_attribute *soc_match_data; + + soc_match_data = soc_device_match(k3_mdio_socinfo); + if (soc_match_data && soc_match_data->data) { + const struct k3_mdio_soc_data *socdata = + soc_match_data->data; + + data->manual_mode = socdata->manual_mode; + } + } + + if (data->manual_mode) + data->bus = alloc_mdio_bitbang(&data->bb_ctrl); + else + data->bus = devm_mdiobus_alloc(dev); + if (!data->bus) { dev_err(dev, "failed to alloc mii bus\n"); return -ENOMEM; @@ -371,11 +571,20 @@ static int davinci_mdio_probe(struct platform_device *pdev) } data->bus->name = dev_name(dev); - data->bus->read = davinci_mdio_read; - data->bus->write = davinci_mdio_write; - data->bus->reset = davinci_mdio_reset; + + if (data->manual_mode) { + data->bus->read = davinci_mdiobb_read; + data->bus->write = davinci_mdiobb_write; + data->bus->reset = davinci_mdiobb_reset; + + dev_info(dev, "Configuring MDIO in manual mode\n"); + } else { + data->bus->read = davinci_mdio_read; + data->bus->write = davinci_mdio_write; + data->bus->reset = davinci_mdio_reset; + data->bus->priv = data; + } data->bus->parent = dev; - data->bus->priv = data; data->clk = devm_clk_get(dev, "fck"); if (IS_ERR(data->clk)) { @@ -433,9 +642,13 @@ static int davinci_mdio_remove(struct platform_device *pdev) { struct davinci_mdio_data *data = platform_get_drvdata(pdev); - if (data->bus) + if (data->bus) { mdiobus_unregister(data->bus); + if (data->manual_mode) + free_mdio_bitbang(data->bus); + } + pm_runtime_dont_use_autosuspend(&pdev->dev); pm_runtime_disable(&pdev->dev); @@ -452,7 +665,9 @@ static int davinci_mdio_runtime_suspend(struct device *dev) ctrl = readl(&data->regs->control); ctrl &= ~CONTROL_ENABLE; writel(ctrl, &data->regs->control); - wait_for_idle(data); + + if (!data->manual_mode) + wait_for_idle(data); return 0; } @@ -461,7 +676,12 @@ static int davinci_mdio_runtime_resume(struct device *dev) { struct davinci_mdio_data *data = dev_get_drvdata(dev); - davinci_mdio_enable(data); + if (data->manual_mode) { + davinci_mdio_disable(data); + davinci_mdio_enable_manual_mode(data); + } else { + davinci_mdio_enable(data); + } return 0; } #endif diff --git a/drivers/net/ethernet/ti/netcp_core.c b/drivers/net/ethernet/ti/netcp_core.c index b15d44261e76..aba70bef4894 100644 --- a/drivers/net/ethernet/ti/netcp_core.c +++ b/drivers/net/ethernet/ti/netcp_core.c @@ -2095,7 +2095,7 @@ static int netcp_create_interface(struct netcp_device *netcp_device, } /* NAPI register */ - netif_napi_add(ndev, &netcp->rx_napi, netcp_rx_poll, NAPI_POLL_WEIGHT); + netif_napi_add(ndev, &netcp->rx_napi, netcp_rx_poll); netif_napi_add_tx(ndev, &netcp->tx_napi, netcp_tx_poll); /* Register the network device */ diff --git a/drivers/net/ethernet/ti/tlan.c b/drivers/net/ethernet/ti/tlan.c index 741c42c6a417..b3da76efa8f5 100644 --- a/drivers/net/ethernet/ti/tlan.c +++ b/drivers/net/ethernet/ti/tlan.c @@ -762,12 +762,12 @@ static void tlan_get_drvinfo(struct net_device *dev, { struct tlan_priv *priv = netdev_priv(dev); - strlcpy(info->driver, KBUILD_MODNAME, sizeof(info->driver)); + strscpy(info->driver, KBUILD_MODNAME, sizeof(info->driver)); if (priv->pci_dev) - strlcpy(info->bus_info, pci_name(priv->pci_dev), + strscpy(info->bus_info, pci_name(priv->pci_dev), sizeof(info->bus_info)); else - strlcpy(info->bus_info, "EISA", sizeof(info->bus_info)); + strscpy(info->bus_info, "EISA", sizeof(info->bus_info)); } static int tlan_get_eeprom_len(struct net_device *dev) diff --git a/drivers/net/ethernet/toshiba/ps3_gelic_net.c b/drivers/net/ethernet/toshiba/ps3_gelic_net.c index 3dbfb1b20649..cf8de8a7a8a1 100644 --- a/drivers/net/ethernet/toshiba/ps3_gelic_net.c +++ b/drivers/net/ethernet/toshiba/ps3_gelic_net.c @@ -1187,8 +1187,8 @@ int gelic_net_open(struct net_device *netdev) void gelic_net_get_drvinfo(struct net_device *netdev, struct ethtool_drvinfo *info) { - strlcpy(info->driver, DRV_NAME, sizeof(info->driver)); - strlcpy(info->version, DRV_VERSION, sizeof(info->version)); + strscpy(info->driver, DRV_NAME, sizeof(info->driver)); + strscpy(info->version, DRV_VERSION, sizeof(info->version)); } static int gelic_ether_get_link_ksettings(struct net_device *netdev, @@ -1441,7 +1441,7 @@ static void gelic_ether_setup_netdev_ops(struct net_device *netdev, { netdev->watchdog_timeo = GELIC_NET_WATCHDOG_TIMEOUT; /* NAPI */ - netif_napi_add(netdev, napi, gelic_net_poll, NAPI_POLL_WEIGHT); + netif_napi_add(netdev, napi, gelic_net_poll); netdev->ethtool_ops = &gelic_ether_ethtool_ops; netdev->netdev_ops = &gelic_netdevice_ops; } diff --git a/drivers/net/ethernet/toshiba/spider_net.c b/drivers/net/ethernet/toshiba/spider_net.c index bc4914c758ad..50d7eacfec58 100644 --- a/drivers/net/ethernet/toshiba/spider_net.c +++ b/drivers/net/ethernet/toshiba/spider_net.c @@ -2270,8 +2270,7 @@ spider_net_setup_netdev(struct spider_net_card *card) card->aneg_count = 0; timer_setup(&card->aneg_timer, spider_net_link_phy, 0); - netif_napi_add(netdev, &card->napi, - spider_net_poll, NAPI_POLL_WEIGHT); + netif_napi_add(netdev, &card->napi, spider_net_poll); spider_net_setup_netdev_ops(netdev); diff --git a/drivers/net/ethernet/toshiba/spider_net_ethtool.c b/drivers/net/ethernet/toshiba/spider_net_ethtool.c index 93110dba0bfa..fef9fd127b5e 100644 --- a/drivers/net/ethernet/toshiba/spider_net_ethtool.c +++ b/drivers/net/ethernet/toshiba/spider_net_ethtool.c @@ -63,12 +63,12 @@ spider_net_ethtool_get_drvinfo(struct net_device *netdev, card = netdev_priv(netdev); /* clear and fill out info */ - strlcpy(drvinfo->driver, spider_net_driver_name, + strscpy(drvinfo->driver, spider_net_driver_name, sizeof(drvinfo->driver)); - strlcpy(drvinfo->version, VERSION, sizeof(drvinfo->version)); - strlcpy(drvinfo->fw_version, "no information", + strscpy(drvinfo->version, VERSION, sizeof(drvinfo->version)); + strscpy(drvinfo->fw_version, "no information", sizeof(drvinfo->fw_version)); - strlcpy(drvinfo->bus_info, pci_name(card->pdev), + strscpy(drvinfo->bus_info, pci_name(card->pdev), sizeof(drvinfo->bus_info)); } diff --git a/drivers/net/ethernet/toshiba/tc35815.c b/drivers/net/ethernet/toshiba/tc35815.c index 47aab9c132c8..b50be67b398b 100644 --- a/drivers/net/ethernet/toshiba/tc35815.c +++ b/drivers/net/ethernet/toshiba/tc35815.c @@ -1956,9 +1956,9 @@ static void tc35815_get_drvinfo(struct net_device *dev, struct ethtool_drvinfo * { struct tc35815_local *lp = netdev_priv(dev); - strlcpy(info->driver, MODNAME, sizeof(info->driver)); - strlcpy(info->version, DRV_VERSION, sizeof(info->version)); - strlcpy(info->bus_info, pci_name(lp->pci_dev), sizeof(info->bus_info)); + strscpy(info->driver, MODNAME, sizeof(info->driver)); + strscpy(info->version, DRV_VERSION, sizeof(info->version)); + strscpy(info->bus_info, pci_name(lp->pci_dev), sizeof(info->bus_info)); } static u32 tc35815_get_msglevel(struct net_device *dev) diff --git a/drivers/net/ethernet/tundra/tsi108_eth.c b/drivers/net/ethernet/tundra/tsi108_eth.c index 5251fc324221..2cd2afc3fff0 100644 --- a/drivers/net/ethernet/tundra/tsi108_eth.c +++ b/drivers/net/ethernet/tundra/tsi108_eth.c @@ -59,9 +59,6 @@ /* Check the phy status every half a second. */ #define CHECK_PHY_INTERVAL (HZ/2) -static int tsi108_init_one(struct platform_device *pdev); -static int tsi108_ether_remove(struct platform_device *pdev); - struct tsi108_prv_data { void __iomem *regs; /* Base of normal regs */ void __iomem *phyregs; /* Base of register bank used for PHY access */ @@ -144,16 +141,6 @@ struct tsi108_prv_data { struct platform_device *pdev; }; -/* Structure for a device driver */ - -static struct platform_driver tsi_eth_driver = { - .probe = tsi108_init_one, - .remove = tsi108_ether_remove, - .driver = { - .name = "tsi-ethernet", - }, -}; - static void tsi108_timed_checker(struct timer_list *t); #ifdef DEBUG @@ -1598,7 +1585,7 @@ tsi108_init_one(struct platform_device *pdev) data->phy_type = einfo->phy_type; data->irq_num = einfo->irq_num; data->id = pdev->id; - netif_napi_add(dev, &data->napi, tsi108_poll, 64); + netif_napi_add(dev, &data->napi, tsi108_poll); dev->netdev_ops = &tsi108_netdev_ops; dev->ethtool_ops = &tsi108_ethtool_ops; @@ -1683,6 +1670,16 @@ static int tsi108_ether_remove(struct platform_device *pdev) return 0; } + +/* Structure for a device driver */ + +static struct platform_driver tsi_eth_driver = { + .probe = tsi108_init_one, + .remove = tsi108_ether_remove, + .driver = { + .name = "tsi-ethernet", + }, +}; module_platform_driver(tsi_eth_driver); MODULE_AUTHOR("Tundra Semiconductor Corporation"); diff --git a/drivers/net/ethernet/vertexcom/mse102x.c b/drivers/net/ethernet/vertexcom/mse102x.c index eb39a45de012..aeed2a093e34 100644 --- a/drivers/net/ethernet/vertexcom/mse102x.c +++ b/drivers/net/ethernet/vertexcom/mse102x.c @@ -750,6 +750,13 @@ static const struct of_device_id mse102x_match_table[] = { }; MODULE_DEVICE_TABLE(of, mse102x_match_table); +static const struct spi_device_id mse102x_ids[] = { + { "mse1021" }, + { "mse1022" }, + { } +}; +MODULE_DEVICE_TABLE(spi, mse102x_ids); + static struct spi_driver mse102x_driver = { .driver = { .name = DRV_NAME, @@ -758,10 +765,11 @@ static struct spi_driver mse102x_driver = { }, .probe = mse102x_probe_spi, .remove = mse102x_remove_spi, + .id_table = mse102x_ids, }; module_spi_driver(mse102x_driver); MODULE_DESCRIPTION("MSE102x Network driver"); -MODULE_AUTHOR("Stefan Wahren <stefan.wahren@in-tech.com>"); +MODULE_AUTHOR("Stefan Wahren <stefan.wahren@chargebyte.com>"); MODULE_LICENSE("GPL"); MODULE_ALIAS("spi:" DRV_NAME); diff --git a/drivers/net/ethernet/via/via-rhine.c b/drivers/net/ethernet/via/via-rhine.c index 509c5e9b29df..0fb15a17b547 100644 --- a/drivers/net/ethernet/via/via-rhine.c +++ b/drivers/net/ethernet/via/via-rhine.c @@ -965,7 +965,7 @@ static int rhine_init_one_common(struct device *hwdev, u32 quirks, dev->ethtool_ops = &netdev_ethtool_ops; dev->watchdog_timeo = TX_TIMEOUT; - netif_napi_add(dev, &rp->napi, rhine_napipoll, 64); + netif_napi_add(dev, &rp->napi, rhine_napipoll); if (rp->quirks & rqRhineI) dev->features |= NETIF_F_SG|NETIF_F_HW_CSUM; @@ -2281,8 +2281,8 @@ static void netdev_get_drvinfo(struct net_device *dev, struct ethtool_drvinfo *i { struct device *hwdev = dev->dev.parent; - strlcpy(info->driver, DRV_NAME, sizeof(info->driver)); - strlcpy(info->bus_info, dev_name(hwdev), sizeof(info->bus_info)); + strscpy(info->driver, DRV_NAME, sizeof(info->driver)); + strscpy(info->bus_info, dev_name(hwdev), sizeof(info->bus_info)); } static int netdev_get_link_ksettings(struct net_device *dev, diff --git a/drivers/net/ethernet/via/via-velocity.c b/drivers/net/ethernet/via/via-velocity.c index ff0c102cb578..a502812ac418 100644 --- a/drivers/net/ethernet/via/via-velocity.c +++ b/drivers/net/ethernet/via/via-velocity.c @@ -2846,7 +2846,7 @@ static int velocity_probe(struct device *dev, int irq, netdev->netdev_ops = &velocity_netdev_ops; netdev->ethtool_ops = &velocity_ethtool_ops; - netif_napi_add(netdev, &vptr->napi, velocity_poll, NAPI_POLL_WEIGHT); + netif_napi_add(netdev, &vptr->napi, velocity_poll); netdev->hw_features = NETIF_F_IP_CSUM | NETIF_F_SG | NETIF_F_HW_VLAN_CTAG_TX; @@ -3419,13 +3419,13 @@ static void velocity_get_drvinfo(struct net_device *dev, struct ethtool_drvinfo { struct velocity_info *vptr = netdev_priv(dev); - strlcpy(info->driver, VELOCITY_NAME, sizeof(info->driver)); - strlcpy(info->version, VELOCITY_VERSION, sizeof(info->version)); + strscpy(info->driver, VELOCITY_NAME, sizeof(info->driver)); + strscpy(info->version, VELOCITY_VERSION, sizeof(info->version)); if (vptr->pdev) - strlcpy(info->bus_info, pci_name(vptr->pdev), + strscpy(info->bus_info, pci_name(vptr->pdev), sizeof(info->bus_info)); else - strlcpy(info->bus_info, "platform", sizeof(info->bus_info)); + strscpy(info->bus_info, "platform", sizeof(info->bus_info)); } static void velocity_ethtool_get_wol(struct net_device *dev, struct ethtool_wolinfo *wol) diff --git a/drivers/net/ethernet/wangxun/Kconfig b/drivers/net/ethernet/wangxun/Kconfig index b4a4fa0a58f8..f5d43d8c9629 100644 --- a/drivers/net/ethernet/wangxun/Kconfig +++ b/drivers/net/ethernet/wangxun/Kconfig @@ -16,6 +16,19 @@ config NET_VENDOR_WANGXUN if NET_VENDOR_WANGXUN +config NGBE + tristate "Wangxun(R) GbE PCI Express adapters support" + depends on PCI + help + This driver supports Wangxun(R) GbE PCI Express family of + adapters. + + More specific information on configuring the driver is in + <file:Documentation/networking/device_drivers/ethernet/wangxun/ngbe.rst>. + + To compile this driver as a module, choose M here. The module + will be called ngbe. + config TXGBE tristate "Wangxun(R) 10GbE PCI Express adapters support" depends on PCI diff --git a/drivers/net/ethernet/wangxun/Makefile b/drivers/net/ethernet/wangxun/Makefile index c34db1bead25..ac3fb06b233c 100644 --- a/drivers/net/ethernet/wangxun/Makefile +++ b/drivers/net/ethernet/wangxun/Makefile @@ -4,3 +4,4 @@ # obj-$(CONFIG_TXGBE) += txgbe/ +obj-$(CONFIG_NGBE) += ngbe/ diff --git a/drivers/net/ethernet/wangxun/ngbe/Makefile b/drivers/net/ethernet/wangxun/ngbe/Makefile new file mode 100644 index 000000000000..0baf75907496 --- /dev/null +++ b/drivers/net/ethernet/wangxun/ngbe/Makefile @@ -0,0 +1,9 @@ +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2019 - 2022 Beijing WangXun Technology Co., Ltd. +# +# Makefile for the Wangxun(R) GbE PCI Express ethernet driver +# + +obj-$(CONFIG_NGBE) += ngbe.o + +ngbe-objs := ngbe_main.o diff --git a/drivers/net/ethernet/wangxun/ngbe/ngbe.h b/drivers/net/ethernet/wangxun/ngbe/ngbe.h new file mode 100644 index 000000000000..f5fa6e5238cc --- /dev/null +++ b/drivers/net/ethernet/wangxun/ngbe/ngbe.h @@ -0,0 +1,24 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright (c) 2019 - 2022 Beijing WangXun Technology Co., Ltd. */ + +#ifndef _NGBE_H_ +#define _NGBE_H_ + +#include "ngbe_type.h" + +#define NGBE_MAX_FDIR_INDICES 7 + +#define NGBE_MAX_RX_QUEUES (NGBE_MAX_FDIR_INDICES + 1) +#define NGBE_MAX_TX_QUEUES (NGBE_MAX_FDIR_INDICES + 1) + +/* board specific private data structure */ +struct ngbe_adapter { + u8 __iomem *io_addr; /* Mainly for iounmap use */ + /* OS defined structs */ + struct net_device *netdev; + struct pci_dev *pdev; +}; + +extern char ngbe_driver_name[]; + +#endif /* _NGBE_H_ */ diff --git a/drivers/net/ethernet/wangxun/ngbe/ngbe_main.c b/drivers/net/ethernet/wangxun/ngbe/ngbe_main.c new file mode 100644 index 000000000000..7674cb6e5700 --- /dev/null +++ b/drivers/net/ethernet/wangxun/ngbe/ngbe_main.c @@ -0,0 +1,170 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2019 - 2022 Beijing WangXun Technology Co., Ltd. */ + +#include <linux/types.h> +#include <linux/module.h> +#include <linux/pci.h> +#include <linux/netdevice.h> +#include <linux/string.h> +#include <linux/aer.h> +#include <linux/etherdevice.h> + +#include "ngbe.h" +char ngbe_driver_name[] = "ngbe"; + +/* ngbe_pci_tbl - PCI Device ID Table + * + * { Vendor ID, Device ID, SubVendor ID, SubDevice ID, + * Class, Class Mask, private data (not used) } + */ +static const struct pci_device_id ngbe_pci_tbl[] = { + { PCI_VDEVICE(WANGXUN, NGBE_DEV_ID_EM_WX1860AL_W), 0}, + { PCI_VDEVICE(WANGXUN, NGBE_DEV_ID_EM_WX1860A2), 0}, + { PCI_VDEVICE(WANGXUN, NGBE_DEV_ID_EM_WX1860A2S), 0}, + { PCI_VDEVICE(WANGXUN, NGBE_DEV_ID_EM_WX1860A4), 0}, + { PCI_VDEVICE(WANGXUN, NGBE_DEV_ID_EM_WX1860A4S), 0}, + { PCI_VDEVICE(WANGXUN, NGBE_DEV_ID_EM_WX1860AL2), 0}, + { PCI_VDEVICE(WANGXUN, NGBE_DEV_ID_EM_WX1860AL2S), 0}, + { PCI_VDEVICE(WANGXUN, NGBE_DEV_ID_EM_WX1860AL4), 0}, + { PCI_VDEVICE(WANGXUN, NGBE_DEV_ID_EM_WX1860AL4S), 0}, + { PCI_VDEVICE(WANGXUN, NGBE_DEV_ID_EM_WX1860LC), 0}, + { PCI_VDEVICE(WANGXUN, NGBE_DEV_ID_EM_WX1860A1), 0}, + { PCI_VDEVICE(WANGXUN, NGBE_DEV_ID_EM_WX1860A1L), 0}, + /* required last entry */ + { .device = 0 } +}; + +static void ngbe_dev_shutdown(struct pci_dev *pdev, bool *enable_wake) +{ + struct ngbe_adapter *adapter = pci_get_drvdata(pdev); + struct net_device *netdev = adapter->netdev; + + netif_device_detach(netdev); + + pci_disable_device(pdev); +} + +static void ngbe_shutdown(struct pci_dev *pdev) +{ + bool wake; + + ngbe_dev_shutdown(pdev, &wake); + + if (system_state == SYSTEM_POWER_OFF) { + pci_wake_from_d3(pdev, wake); + pci_set_power_state(pdev, PCI_D3hot); + } +} + +/** + * ngbe_probe - Device Initialization Routine + * @pdev: PCI device information struct + * @ent: entry in ngbe_pci_tbl + * + * Returns 0 on success, negative on failure + * + * ngbe_probe initializes an adapter identified by a pci_dev structure. + * The OS initialization, configuring of the adapter private structure, + * and a hardware reset occur. + **/ +static int ngbe_probe(struct pci_dev *pdev, + const struct pci_device_id __always_unused *ent) +{ + struct ngbe_adapter *adapter = NULL; + struct net_device *netdev; + int err; + + err = pci_enable_device_mem(pdev); + if (err) + return err; + + err = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64)); + if (err) { + dev_err(&pdev->dev, + "No usable DMA configuration, aborting\n"); + goto err_pci_disable_dev; + } + + err = pci_request_selected_regions(pdev, + pci_select_bars(pdev, IORESOURCE_MEM), + ngbe_driver_name); + if (err) { + dev_err(&pdev->dev, + "pci_request_selected_regions failed %d\n", err); + goto err_pci_disable_dev; + } + + pci_enable_pcie_error_reporting(pdev); + pci_set_master(pdev); + + netdev = devm_alloc_etherdev_mqs(&pdev->dev, + sizeof(struct ngbe_adapter), + NGBE_MAX_TX_QUEUES, + NGBE_MAX_RX_QUEUES); + if (!netdev) { + err = -ENOMEM; + goto err_pci_release_regions; + } + + SET_NETDEV_DEV(netdev, &pdev->dev); + + adapter = netdev_priv(netdev); + adapter->netdev = netdev; + adapter->pdev = pdev; + + adapter->io_addr = devm_ioremap(&pdev->dev, + pci_resource_start(pdev, 0), + pci_resource_len(pdev, 0)); + if (!adapter->io_addr) { + err = -EIO; + goto err_pci_release_regions; + } + + netdev->features |= NETIF_F_HIGHDMA; + + pci_set_drvdata(pdev, adapter); + + return 0; + +err_pci_release_regions: + pci_disable_pcie_error_reporting(pdev); + pci_release_selected_regions(pdev, + pci_select_bars(pdev, IORESOURCE_MEM)); +err_pci_disable_dev: + pci_disable_device(pdev); + return err; +} + +/** + * ngbe_remove - Device Removal Routine + * @pdev: PCI device information struct + * + * ngbe_remove is called by the PCI subsystem to alert the driver + * that it should release a PCI device. The could be caused by a + * Hot-Plug event, or because the driver is going to be removed from + * memory. + **/ +static void ngbe_remove(struct pci_dev *pdev) +{ + pci_release_selected_regions(pdev, + pci_select_bars(pdev, IORESOURCE_MEM)); + + pci_disable_pcie_error_reporting(pdev); + + pci_disable_device(pdev); +} + +static struct pci_driver ngbe_driver = { + .name = ngbe_driver_name, + .id_table = ngbe_pci_tbl, + .probe = ngbe_probe, + .remove = ngbe_remove, + .shutdown = ngbe_shutdown, +}; + +module_pci_driver(ngbe_driver); + +MODULE_DEVICE_TABLE(pci, ngbe_pci_tbl); +MODULE_AUTHOR("Beijing WangXun Technology Co., Ltd, <software@net-swift.com>"); +MODULE_DESCRIPTION("WangXun(R) Gigabit PCI Express Network Driver"); +MODULE_LICENSE("GPL"); diff --git a/drivers/net/ethernet/wangxun/ngbe/ngbe_type.h b/drivers/net/ethernet/wangxun/ngbe/ngbe_type.h new file mode 100644 index 000000000000..26e776c3539a --- /dev/null +++ b/drivers/net/ethernet/wangxun/ngbe/ngbe_type.h @@ -0,0 +1,50 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright (c) 2019 - 2022 Beijing WangXun Technology Co., Ltd. */ + +#ifndef _NGBE_TYPE_H_ +#define _NGBE_TYPE_H_ + +#include <linux/types.h> +#include <linux/netdevice.h> + +/************ NGBE_register.h ************/ +/* Vendor ID */ +#ifndef PCI_VENDOR_ID_WANGXUN +#define PCI_VENDOR_ID_WANGXUN 0x8088 +#endif + +/* Device IDs */ +#define NGBE_DEV_ID_EM_WX1860AL_W 0x0100 +#define NGBE_DEV_ID_EM_WX1860A2 0x0101 +#define NGBE_DEV_ID_EM_WX1860A2S 0x0102 +#define NGBE_DEV_ID_EM_WX1860A4 0x0103 +#define NGBE_DEV_ID_EM_WX1860A4S 0x0104 +#define NGBE_DEV_ID_EM_WX1860AL2 0x0105 +#define NGBE_DEV_ID_EM_WX1860AL2S 0x0106 +#define NGBE_DEV_ID_EM_WX1860AL4 0x0107 +#define NGBE_DEV_ID_EM_WX1860AL4S 0x0108 +#define NGBE_DEV_ID_EM_WX1860LC 0x0109 +#define NGBE_DEV_ID_EM_WX1860A1 0x010a +#define NGBE_DEV_ID_EM_WX1860A1L 0x010b + +/* Subsystem ID */ +#define NGBE_SUBID_M88E1512_SFP 0x0003 +#define NGBE_SUBID_OCP_CARD 0x0040 +#define NGBE_SUBID_LY_M88E1512_SFP 0x0050 +#define NGBE_SUBID_M88E1512_RJ45 0x0051 +#define NGBE_SUBID_M88E1512_MIX 0x0052 +#define NGBE_SUBID_YT8521S_SFP 0x0060 +#define NGBE_SUBID_INTERNAL_YT8521S_SFP 0x0061 +#define NGBE_SUBID_YT8521S_SFP_GPIO 0x0062 +#define NGBE_SUBID_INTERNAL_YT8521S_SFP_GPIO 0x0064 +#define NGBE_SUBID_LY_YT8521S_SFP 0x0070 +#define NGBE_SUBID_RGMII_FPGA 0x0080 + +#define NGBE_OEM_MASK 0x00FF + +#define NGBE_NCSI_SUP 0x8000 +#define NGBE_NCSI_MASK 0x8000 +#define NGBE_WOL_SUP 0x4000 +#define NGBE_WOL_MASK 0x4000 + +#endif /* _NGBE_TYPE_H_ */ diff --git a/drivers/net/ethernet/wiznet/w5100.c b/drivers/net/ethernet/wiznet/w5100.c index acd78120e53c..634946e87e5f 100644 --- a/drivers/net/ethernet/wiznet/w5100.c +++ b/drivers/net/ethernet/wiznet/w5100.c @@ -719,9 +719,9 @@ static void w5100_hw_close(struct w5100_priv *priv) static void w5100_get_drvinfo(struct net_device *ndev, struct ethtool_drvinfo *info) { - strlcpy(info->driver, DRV_NAME, sizeof(info->driver)); - strlcpy(info->version, DRV_VERSION, sizeof(info->version)); - strlcpy(info->bus_info, dev_name(ndev->dev.parent), + strscpy(info->driver, DRV_NAME, sizeof(info->driver)); + strscpy(info->version, DRV_VERSION, sizeof(info->version)); + strscpy(info->bus_info, dev_name(ndev->dev.parent), sizeof(info->bus_info)); } diff --git a/drivers/net/ethernet/wiznet/w5300.c b/drivers/net/ethernet/wiznet/w5300.c index 773f8c77909a..b0958fe8111e 100644 --- a/drivers/net/ethernet/wiznet/w5300.c +++ b/drivers/net/ethernet/wiznet/w5300.c @@ -282,9 +282,9 @@ static void w5300_hw_close(struct w5300_priv *priv) static void w5300_get_drvinfo(struct net_device *ndev, struct ethtool_drvinfo *info) { - strlcpy(info->driver, DRV_NAME, sizeof(info->driver)); - strlcpy(info->version, DRV_VERSION, sizeof(info->version)); - strlcpy(info->bus_info, dev_name(ndev->dev.parent), + strscpy(info->driver, DRV_NAME, sizeof(info->driver)); + strscpy(info->version, DRV_VERSION, sizeof(info->version)); + strscpy(info->bus_info, dev_name(ndev->dev.parent), sizeof(info->bus_info)); } diff --git a/drivers/net/ethernet/xilinx/ll_temac.h b/drivers/net/ethernet/xilinx/ll_temac.h index c6395c406418..6668d1b760d8 100644 --- a/drivers/net/ethernet/xilinx/ll_temac.h +++ b/drivers/net/ethernet/xilinx/ll_temac.h @@ -21,36 +21,45 @@ /* Configuration options */ /* Accept all incoming packets. - * This option defaults to disabled (cleared) */ + * This option defaults to disabled (cleared) + */ #define XTE_OPTION_PROMISC (1 << 0) /* Jumbo frame support for Tx & Rx. - * This option defaults to disabled (cleared) */ + * This option defaults to disabled (cleared) + */ #define XTE_OPTION_JUMBO (1 << 1) /* VLAN Rx & Tx frame support. - * This option defaults to disabled (cleared) */ + * This option defaults to disabled (cleared) + */ #define XTE_OPTION_VLAN (1 << 2) /* Enable recognition of flow control frames on Rx - * This option defaults to enabled (set) */ + * This option defaults to enabled (set) + */ #define XTE_OPTION_FLOW_CONTROL (1 << 4) /* Strip FCS and PAD from incoming frames. * Note: PAD from VLAN frames is not stripped. - * This option defaults to disabled (set) */ + * This option defaults to disabled (set) + */ #define XTE_OPTION_FCS_STRIP (1 << 5) /* Generate FCS field and add PAD automatically for outgoing frames. - * This option defaults to enabled (set) */ + * This option defaults to enabled (set) + */ #define XTE_OPTION_FCS_INSERT (1 << 6) /* Enable Length/Type error checking for incoming frames. When this option is -set, the MAC will filter frames that have a mismatched type/length field -and if XTE_OPTION_REPORT_RXERR is set, the user is notified when these -types of frames are encountered. When this option is cleared, the MAC will -allow these types of frames to be received. -This option defaults to enabled (set) */ + * set, the MAC will filter frames that have a mismatched type/length field + * and if XTE_OPTION_REPORT_RXERR is set, the user is notified when these + * types of frames are encountered. When this option is cleared, the MAC will + * allow these types of frames to be received. + * This option defaults to enabled (set) + */ #define XTE_OPTION_LENTYPE_ERR (1 << 7) /* Enable the transmitter. - * This option defaults to enabled (set) */ + * This option defaults to enabled (set) + */ #define XTE_OPTION_TXEN (1 << 11) /* Enable the receiver -* This option defaults to enabled (set) */ + * This option defaults to enabled (set) + */ #define XTE_OPTION_RXEN (1 << 12) /* Default options set when device is initialized or reset */ @@ -68,18 +77,18 @@ This option defaults to enabled (set) */ #define TX_TAILDESC_PTR 0x04 /* rw */ #define TX_CHNL_CTRL 0x05 /* rw */ /* - 0:7 24:31 IRQTimeout - 8:15 16:23 IRQCount - 16:20 11:15 Reserved - 21 10 0 - 22 9 UseIntOnEnd - 23 8 LdIRQCnt - 24 7 IRQEn - 25:28 3:6 Reserved - 29 2 IrqErrEn - 30 1 IrqDlyEn - 31 0 IrqCoalEn -*/ + * 0:7 24:31 IRQTimeout + * 8:15 16:23 IRQCount + * 16:20 11:15 Reserved + * 21 10 0 + * 22 9 UseIntOnEnd + * 23 8 LdIRQCnt + * 24 7 IRQEn + * 25:28 3:6 Reserved + * 29 2 IrqErrEn + * 30 1 IrqDlyEn + * 31 0 IrqCoalEn + */ #define CHNL_CTRL_IRQ_IOE (1 << 9) #define CHNL_CTRL_IRQ_EN (1 << 7) #define CHNL_CTRL_IRQ_ERR_EN (1 << 2) @@ -87,35 +96,35 @@ This option defaults to enabled (set) */ #define CHNL_CTRL_IRQ_COAL_EN (1 << 0) #define TX_IRQ_REG 0x06 /* rw */ /* - 0:7 24:31 DltTmrValue - 8:15 16:23 ClscCntrValue - 16:17 14:15 Reserved - 18:21 10:13 ClscCnt - 22:23 8:9 DlyCnt - 24:28 3::7 Reserved - 29 2 ErrIrq - 30 1 DlyIrq - 31 0 CoalIrq + * 0:7 24:31 DltTmrValue + * 8:15 16:23 ClscCntrValue + * 16:17 14:15 Reserved + * 18:21 10:13 ClscCnt + * 22:23 8:9 DlyCnt + * 24:28 3::7 Reserved + * 29 2 ErrIrq + * 30 1 DlyIrq + * 31 0 CoalIrq */ #define TX_CHNL_STS 0x07 /* r */ /* - 0:9 22:31 Reserved - 10 21 TailPErr - 11 20 CmpErr - 12 19 AddrErr - 13 18 NxtPErr - 14 17 CurPErr - 15 16 BsyWr - 16:23 8:15 Reserved - 24 7 Error - 25 6 IOE - 26 5 SOE - 27 4 Cmplt - 28 3 SOP - 29 2 EOP - 30 1 EngBusy - 31 0 Reserved -*/ + * 0:9 22:31 Reserved + * 10 21 TailPErr + * 11 20 CmpErr + * 12 19 AddrErr + * 13 18 NxtPErr + * 14 17 CurPErr + * 15 16 BsyWr + * 16:23 8:15 Reserved + * 24 7 Error + * 25 6 IOE + * 26 5 SOE + * 27 4 Cmplt + * 28 3 SOP + * 29 2 EOP + * 30 1 EngBusy + * 31 0 Reserved + */ #define RX_NXTDESC_PTR 0x08 /* r */ #define RX_CURBUF_ADDR 0x09 /* r */ @@ -124,17 +133,17 @@ This option defaults to enabled (set) */ #define RX_TAILDESC_PTR 0x0c /* rw */ #define RX_CHNL_CTRL 0x0d /* rw */ /* - 0:7 24:31 IRQTimeout - 8:15 16:23 IRQCount - 16:20 11:15 Reserved - 21 10 0 - 22 9 UseIntOnEnd - 23 8 LdIRQCnt - 24 7 IRQEn - 25:28 3:6 Reserved - 29 2 IrqErrEn - 30 1 IrqDlyEn - 31 0 IrqCoalEn + * 0:7 24:31 IRQTimeout + * 8:15 16:23 IRQCount + * 16:20 11:15 Reserved + * 21 10 0 + * 22 9 UseIntOnEnd + * 23 8 LdIRQCnt + * 24 7 IRQEn + * 25:28 3:6 Reserved + * 29 2 IrqErrEn + * 30 1 IrqDlyEn + * 31 0 IrqCoalEn */ #define RX_IRQ_REG 0x0e /* rw */ #define IRQ_COAL (1 << 0) @@ -142,13 +151,13 @@ This option defaults to enabled (set) */ #define IRQ_ERR (1 << 2) #define IRQ_DMAERR (1 << 7) /* this is not documented ??? */ /* - 0:7 24:31 DltTmrValue - 8:15 16:23 ClscCntrValue - 16:17 14:15 Reserved - 18:21 10:13 ClscCnt - 22:23 8:9 DlyCnt - 24:28 3::7 Reserved -*/ + * 0:7 24:31 DltTmrValue + * 8:15 16:23 ClscCntrValue + * 16:17 14:15 Reserved + * 18:21 10:13 ClscCnt + * 22:23 8:9 DlyCnt + * 24:28 3::7 Reserved + */ #define RX_CHNL_STS 0x0f /* r */ #define CHNL_STS_ENGBUSY (1 << 1) #define CHNL_STS_EOP (1 << 2) @@ -165,23 +174,23 @@ This option defaults to enabled (set) */ #define CHNL_STS_CMPERR (1 << 20) #define CHNL_STS_TAILERR (1 << 21) /* - 0:9 22:31 Reserved - 10 21 TailPErr - 11 20 CmpErr - 12 19 AddrErr - 13 18 NxtPErr - 14 17 CurPErr - 15 16 BsyWr - 16:23 8:15 Reserved - 24 7 Error - 25 6 IOE - 26 5 SOE - 27 4 Cmplt - 28 3 SOP - 29 2 EOP - 30 1 EngBusy - 31 0 Reserved -*/ + * 0:9 22:31 Reserved + * 10 21 TailPErr + * 11 20 CmpErr + * 12 19 AddrErr + * 13 18 NxtPErr + * 14 17 CurPErr + * 15 16 BsyWr + * 16:23 8:15 Reserved + * 24 7 Error + * 25 6 IOE + * 26 5 SOE + * 27 4 Cmplt + * 28 3 SOP + * 29 2 EOP + * 30 1 EngBusy + * 31 0 Reserved + */ #define DMA_CONTROL_REG 0x10 /* rw */ #define DMA_CONTROL_RST (1 << 0) diff --git a/drivers/net/ethernet/xilinx/ll_temac_main.c b/drivers/net/ethernet/xilinx/ll_temac_main.c index 3f6b9dfca095..1066420d6a83 100644 --- a/drivers/net/ethernet/xilinx/ll_temac_main.c +++ b/drivers/net/ethernet/xilinx/ll_temac_main.c @@ -117,8 +117,8 @@ int temac_indirect_busywait(struct temac_local *lp) spin_until_cond(hard_acs_rdy_or_timeout(lp, timeout)); if (WARN_ON(!hard_acs_rdy(lp))) return -ETIMEDOUT; - else - return 0; + + return 0; } /* @@ -261,7 +261,7 @@ static void temac_dma_dcr_out(struct temac_local *lp, int reg, u32 value) * I/O functions */ static int temac_dcr_setup(struct temac_local *lp, struct platform_device *op, - struct device_node *np) + struct device_node *np) { unsigned int dcrs; @@ -286,7 +286,7 @@ static int temac_dcr_setup(struct temac_local *lp, struct platform_device *op, * such as with MicroBlaze and x86 */ static int temac_dcr_setup(struct temac_local *lp, struct platform_device *op, - struct device_node *np) + struct device_node *np) { return -1; } @@ -307,11 +307,9 @@ static void temac_dma_bd_release(struct net_device *ndev) for (i = 0; i < lp->rx_bd_num; i++) { if (!lp->rx_skb[i]) break; - else { - dma_unmap_single(ndev->dev.parent, lp->rx_bd_v[i].phys, - XTE_MAX_JUMBO_FRAME_SIZE, DMA_FROM_DEVICE); - dev_kfree_skb(lp->rx_skb[i]); - } + dma_unmap_single(ndev->dev.parent, lp->rx_bd_v[i].phys, + XTE_MAX_JUMBO_FRAME_SIZE, DMA_FROM_DEVICE); + dev_kfree_skb(lp->rx_skb[i]); } if (lp->rx_bd_v) dma_free_coherent(ndev->dev.parent, @@ -430,7 +428,8 @@ static void temac_do_set_mac_address(struct net_device *ndev) (ndev->dev_addr[2] << 16) | (ndev->dev_addr[3] << 24)); /* There are reserved bits in EUAW1 - * so don't affect them Set MAC bits [47:32] in EUAW1 */ + * so don't affect them Set MAC bits [47:32] in EUAW1 + */ temac_indirect_out32_locked(lp, XTE_UAW1_OFFSET, (ndev->dev_addr[4] & 0x000000ff) | (ndev->dev_addr[5] << 8)); @@ -530,66 +529,66 @@ static struct temac_option { { .opt = XTE_OPTION_JUMBO, .reg = XTE_RXC1_OFFSET, - .m_or =XTE_RXC1_RXJMBO_MASK, + .m_or = XTE_RXC1_RXJMBO_MASK, }, /* Turn on VLAN packet support for both Rx and Tx */ { .opt = XTE_OPTION_VLAN, .reg = XTE_TXC_OFFSET, - .m_or =XTE_TXC_TXVLAN_MASK, + .m_or = XTE_TXC_TXVLAN_MASK, }, { .opt = XTE_OPTION_VLAN, .reg = XTE_RXC1_OFFSET, - .m_or =XTE_RXC1_RXVLAN_MASK, + .m_or = XTE_RXC1_RXVLAN_MASK, }, /* Turn on FCS stripping on receive packets */ { .opt = XTE_OPTION_FCS_STRIP, .reg = XTE_RXC1_OFFSET, - .m_or =XTE_RXC1_RXFCS_MASK, + .m_or = XTE_RXC1_RXFCS_MASK, }, /* Turn on FCS insertion on transmit packets */ { .opt = XTE_OPTION_FCS_INSERT, .reg = XTE_TXC_OFFSET, - .m_or =XTE_TXC_TXFCS_MASK, + .m_or = XTE_TXC_TXFCS_MASK, }, /* Turn on length/type field checking on receive packets */ { .opt = XTE_OPTION_LENTYPE_ERR, .reg = XTE_RXC1_OFFSET, - .m_or =XTE_RXC1_RXLT_MASK, + .m_or = XTE_RXC1_RXLT_MASK, }, /* Turn on flow control */ { .opt = XTE_OPTION_FLOW_CONTROL, .reg = XTE_FCC_OFFSET, - .m_or =XTE_FCC_RXFLO_MASK, + .m_or = XTE_FCC_RXFLO_MASK, }, /* Turn on flow control */ { .opt = XTE_OPTION_FLOW_CONTROL, .reg = XTE_FCC_OFFSET, - .m_or =XTE_FCC_TXFLO_MASK, + .m_or = XTE_FCC_TXFLO_MASK, }, /* Turn on promiscuous frame filtering (all frames are received ) */ { .opt = XTE_OPTION_PROMISC, .reg = XTE_AFM_OFFSET, - .m_or =XTE_AFM_EPPRM_MASK, + .m_or = XTE_AFM_EPPRM_MASK, }, /* Enable transmitter if not already enabled */ { .opt = XTE_OPTION_TXEN, .reg = XTE_TXC_OFFSET, - .m_or =XTE_TXC_TXEN_MASK, + .m_or = XTE_TXC_TXEN_MASK, }, /* Enable receiver? */ { .opt = XTE_OPTION_RXEN, .reg = XTE_RXC1_OFFSET, - .m_or =XTE_RXC1_RXEN_MASK, + .m_or = XTE_RXC1_RXEN_MASK, }, {} }; @@ -641,7 +640,7 @@ static void temac_device_reset(struct net_device *ndev) udelay(1); if (--timeout == 0) { dev_err(&ndev->dev, - "temac_device_reset RX reset timeout!!\n"); + "%s RX reset timeout!!\n", __func__); break; } } @@ -653,7 +652,7 @@ static void temac_device_reset(struct net_device *ndev) udelay(1); if (--timeout == 0) { dev_err(&ndev->dev, - "temac_device_reset TX reset timeout!!\n"); + "%s TX reset timeout!!\n", __func__); break; } } @@ -672,7 +671,7 @@ static void temac_device_reset(struct net_device *ndev) udelay(1); if (--timeout == 0) { dev_err(&ndev->dev, - "temac_device_reset DMA reset timeout!!\n"); + "%s DMA reset timeout!!\n", __func__); break; } } @@ -680,7 +679,7 @@ static void temac_device_reset(struct net_device *ndev) if (temac_dma_bd_init(ndev)) { dev_err(&ndev->dev, - "temac_device_reset descriptor allocation failed\n"); + "%s descriptor allocation failed\n", __func__); } spin_lock_irqsave(lp->indirect_lock, flags); @@ -691,7 +690,8 @@ static void temac_device_reset(struct net_device *ndev) spin_unlock_irqrestore(lp->indirect_lock, flags); /* Sync default options with HW - * but leave receiver and transmitter disabled. */ + * but leave receiver and transmitter disabled. + */ temac_setoptions(ndev, lp->options & ~(XTE_OPTION_TXEN | XTE_OPTION_RXEN)); @@ -723,9 +723,15 @@ static void temac_adjust_link(struct net_device *ndev) mii_speed &= ~XTE_EMCFG_LINKSPD_MASK; switch (phy->speed) { - case SPEED_1000: mii_speed |= XTE_EMCFG_LINKSPD_1000; break; - case SPEED_100: mii_speed |= XTE_EMCFG_LINKSPD_100; break; - case SPEED_10: mii_speed |= XTE_EMCFG_LINKSPD_10; break; + case SPEED_1000: + mii_speed |= XTE_EMCFG_LINKSPD_1000; + break; + case SPEED_100: + mii_speed |= XTE_EMCFG_LINKSPD_100; + break; + case SPEED_10: + mii_speed |= XTE_EMCFG_LINKSPD_10; + break; } /* Write new speed setting out to TEMAC */ @@ -1007,7 +1013,6 @@ static void ll_temac_recv(struct net_device *ndev) if (((lp->temac_features & TEMAC_FEATURE_RX_CSUM) != 0) && (skb->protocol == htons(ETH_P_IP)) && (skb->len > 64)) { - /* Convert from device endianness (be32) to cpu * endianness, and if necessary swap the bytes * (back) for proper IP checksum byte order @@ -1563,16 +1568,12 @@ static int temac_probe(struct platform_device *pdev) } /* Error handle returned DMA RX and TX interrupts */ - if (lp->rx_irq < 0) { - if (lp->rx_irq != -EPROBE_DEFER) - dev_err(&pdev->dev, "could not get DMA RX irq\n"); - return lp->rx_irq; - } - if (lp->tx_irq < 0) { - if (lp->tx_irq != -EPROBE_DEFER) - dev_err(&pdev->dev, "could not get DMA TX irq\n"); - return lp->tx_irq; - } + if (lp->rx_irq < 0) + return dev_err_probe(&pdev->dev, lp->rx_irq, + "could not get DMA RX irq\n"); + if (lp->tx_irq < 0) + return dev_err_probe(&pdev->dev, lp->tx_irq, + "could not get DMA TX irq\n"); if (temac_np) { /* Retrieve the MAC address */ diff --git a/drivers/net/ethernet/xilinx/ll_temac_mdio.c b/drivers/net/ethernet/xilinx/ll_temac_mdio.c index 6fd2dea4e60f..2371c072b53f 100644 --- a/drivers/net/ethernet/xilinx/ll_temac_mdio.c +++ b/drivers/net/ethernet/xilinx/ll_temac_mdio.c @@ -29,7 +29,8 @@ static int temac_mdio_read(struct mii_bus *bus, int phy_id, int reg) /* Write the PHY address to the MIIM Access Initiator register. * When the transfer completes, the PHY register value will appear - * in the LSW0 register */ + * in the LSW0 register + */ spin_lock_irqsave(lp->indirect_lock, flags); temac_iow(lp, XTE_LSW0_OFFSET, (phy_id << 5) | reg); rc = temac_indirect_in32_locked(lp, XTE_MIIMAI_OFFSET); @@ -88,7 +89,8 @@ int temac_mdio_setup(struct temac_local *lp, struct platform_device *pdev) } /* Enable the MDIO bus by asserting the enable bit and writing - * in the clock config */ + * in the clock config + */ temac_indirect_out32(lp, XTE_MC_OFFSET, 1 << 6 | clk_div); bus = devm_mdiobus_alloc(&pdev->dev); diff --git a/drivers/net/ethernet/xilinx/xilinx_axienet.h b/drivers/net/ethernet/xilinx/xilinx_axienet.h index f2e2261b4b7d..6370c447ac5c 100644 --- a/drivers/net/ethernet/xilinx/xilinx_axienet.h +++ b/drivers/net/ethernet/xilinx/xilinx_axienet.h @@ -402,6 +402,9 @@ struct axidma_bd { * @rx_bd_num: Size of RX buffer descriptor ring * @rx_bd_ci: Stores the index of the Rx buffer descriptor in the ring being * accessed currently. + * @rx_packets: RX packet count for statistics + * @rx_bytes: RX byte count for statistics + * @rx_stat_sync: Synchronization object for RX stats * @napi_tx: NAPI TX control structure * @tx_dma_cr: Nominal content of TX DMA control register * @tx_bd_v: Virtual address of the TX buffer descriptor ring @@ -411,6 +414,9 @@ struct axidma_bd { * complete. Only updated at runtime by TX NAPI poll. * @tx_bd_tail: Stores the index of the next Tx buffer descriptor in the ring * to be populated. + * @tx_packets: TX packet count for statistics + * @tx_bytes: TX byte count for statistics + * @tx_stat_sync: Synchronization object for TX stats * @dma_err_task: Work structure to process Axi DMA errors * @tx_irq: Axidma TX IRQ number * @rx_irq: Axidma RX IRQ number @@ -458,6 +464,9 @@ struct axienet_local { dma_addr_t rx_bd_p; u32 rx_bd_num; u32 rx_bd_ci; + u64_stats_t rx_packets; + u64_stats_t rx_bytes; + struct u64_stats_sync rx_stat_sync; struct napi_struct napi_tx; u32 tx_dma_cr; @@ -466,6 +475,9 @@ struct axienet_local { u32 tx_bd_num; u32 tx_bd_ci; u32 tx_bd_tail; + u64_stats_t tx_packets; + u64_stats_t tx_bytes; + struct u64_stats_sync tx_stat_sync; struct work_struct dma_err_task; @@ -591,7 +603,7 @@ static inline void axienet_dma_out_addr(struct axienet_local *lp, off_t reg, #else /* CONFIG_64BIT */ static inline void axienet_dma_out_addr(struct axienet_local *lp, off_t reg, - dma_addr_t addr) + dma_addr_t addr) { axienet_dma_out32(lp, reg, lower_32_bits(addr)); } diff --git a/drivers/net/ethernet/xilinx/xilinx_axienet_main.c b/drivers/net/ethernet/xilinx/xilinx_axienet_main.c index 1760930ec0c4..d1d772580da9 100644 --- a/drivers/net/ethernet/xilinx/xilinx_axienet_main.c +++ b/drivers/net/ethernet/xilinx/xilinx_axienet_main.c @@ -597,7 +597,7 @@ static int axienet_device_reset(struct net_device *ndev) lp->options &= (~XAE_OPTION_JUMBO); if ((ndev->mtu > XAE_MTU) && - (ndev->mtu <= XAE_JUMBO_MTU)) { + (ndev->mtu <= XAE_JUMBO_MTU)) { lp->max_frm_size = ndev->mtu + VLAN_ETH_HLEN + XAE_TRL_SIZE; @@ -645,7 +645,7 @@ static int axienet_device_reset(struct net_device *ndev) * @nr_bds: Max number of descriptors to clean up * @force: Whether to clean descriptors even if not complete * @sizep: Pointer to a u32 filled with the total sum of all bytes - * in all cleaned-up descriptors. Ignored if NULL. + * in all cleaned-up descriptors. Ignored if NULL. * @budget: NAPI budget (use 0 when not called from NAPI poll) * * Would either be called after a successful transmit operation, or after @@ -752,8 +752,10 @@ static int axienet_tx_poll(struct napi_struct *napi, int budget) if (lp->tx_bd_ci >= lp->tx_bd_num) lp->tx_bd_ci %= lp->tx_bd_num; - ndev->stats.tx_packets += packets; - ndev->stats.tx_bytes += size; + u64_stats_update_begin(&lp->tx_stat_sync); + u64_stats_add(&lp->tx_packets, packets); + u64_stats_add(&lp->tx_bytes, size); + u64_stats_update_end(&lp->tx_stat_sync); /* Matches barrier in axienet_start_xmit */ smp_mb(); @@ -984,8 +986,10 @@ static int axienet_rx_poll(struct napi_struct *napi, int budget) cur_p = &lp->rx_bd_v[lp->rx_bd_ci]; } - lp->ndev->stats.rx_packets += packets; - lp->ndev->stats.rx_bytes += size; + u64_stats_update_begin(&lp->rx_stat_sync); + u64_stats_add(&lp->rx_packets, packets); + u64_stats_add(&lp->rx_bytes, size); + u64_stats_update_end(&lp->rx_stat_sync); if (tail_p) axienet_dma_out_addr(lp, XAXIDMA_RX_TDESC_OFFSET, tail_p); @@ -1292,10 +1296,32 @@ static int axienet_ioctl(struct net_device *dev, struct ifreq *rq, int cmd) return phylink_mii_ioctl(lp->phylink, rq, cmd); } +static void +axienet_get_stats64(struct net_device *dev, struct rtnl_link_stats64 *stats) +{ + struct axienet_local *lp = netdev_priv(dev); + unsigned int start; + + netdev_stats_to_stats64(stats, &dev->stats); + + do { + start = u64_stats_fetch_begin_irq(&lp->rx_stat_sync); + stats->rx_packets = u64_stats_read(&lp->rx_packets); + stats->rx_bytes = u64_stats_read(&lp->rx_bytes); + } while (u64_stats_fetch_retry_irq(&lp->rx_stat_sync, start)); + + do { + start = u64_stats_fetch_begin_irq(&lp->tx_stat_sync); + stats->tx_packets = u64_stats_read(&lp->tx_packets); + stats->tx_bytes = u64_stats_read(&lp->tx_bytes); + } while (u64_stats_fetch_retry_irq(&lp->tx_stat_sync, start)); +} + static const struct net_device_ops axienet_netdev_ops = { .ndo_open = axienet_open, .ndo_stop = axienet_stop, .ndo_start_xmit = axienet_start_xmit, + .ndo_get_stats64 = axienet_get_stats64, .ndo_change_mtu = axienet_change_mtu, .ndo_set_mac_address = netdev_set_mac_address, .ndo_validate_addr = eth_validate_addr, @@ -1317,8 +1343,8 @@ static const struct net_device_ops axienet_netdev_ops = { static void axienet_ethtools_get_drvinfo(struct net_device *ndev, struct ethtool_drvinfo *ed) { - strlcpy(ed->driver, DRIVER_NAME, sizeof(ed->driver)); - strlcpy(ed->version, DRIVER_VERSION, sizeof(ed->version)); + strscpy(ed->driver, DRIVER_NAME, sizeof(ed->driver)); + strscpy(ed->version, DRIVER_VERSION, sizeof(ed->version)); } /** @@ -1349,7 +1375,7 @@ static int axienet_ethtools_get_regs_len(struct net_device *ndev) static void axienet_ethtools_get_regs(struct net_device *ndev, struct ethtool_regs *regs, void *ret) { - u32 *data = (u32 *) ret; + u32 *data = (u32 *)ret; size_t len = sizeof(u32) * AXIENET_REGS_N; struct axienet_local *lp = netdev_priv(ndev); @@ -1850,8 +1876,11 @@ static int axienet_probe(struct platform_device *pdev) lp->rx_bd_num = RX_BD_NUM_DEFAULT; lp->tx_bd_num = TX_BD_NUM_DEFAULT; - netif_napi_add(ndev, &lp->napi_rx, axienet_rx_poll, NAPI_POLL_WEIGHT); - netif_napi_add(ndev, &lp->napi_tx, axienet_tx_poll, NAPI_POLL_WEIGHT); + u64_stats_init(&lp->rx_stat_sync); + u64_stats_init(&lp->tx_stat_sync); + + netif_napi_add(ndev, &lp->napi_rx, axienet_rx_poll); + netif_napi_add(ndev, &lp->napi_tx, axienet_tx_poll); lp->axi_clk = devm_clk_get_optional(&pdev->dev, "s_axi_lite_clk"); if (!lp->axi_clk) { diff --git a/drivers/net/ethernet/xilinx/xilinx_axienet_mdio.c b/drivers/net/ethernet/xilinx/xilinx_axienet_mdio.c index 2772a79cd3ed..0b3b6935c558 100644 --- a/drivers/net/ethernet/xilinx/xilinx_axienet_mdio.c +++ b/drivers/net/ethernet/xilinx/xilinx_axienet_mdio.c @@ -126,7 +126,7 @@ static int axienet_mdio_write(struct mii_bus *bus, int phy_id, int reg, return ret; } - axienet_iow(lp, XAE_MDIO_MWD_OFFSET, (u32) val); + axienet_iow(lp, XAE_MDIO_MWD_OFFSET, (u32)val); axienet_iow(lp, XAE_MDIO_MCR_OFFSET, (((phy_id << XAE_MDIO_MCR_PHYAD_SHIFT) & XAE_MDIO_MCR_PHYAD_MASK) | diff --git a/drivers/net/ethernet/xilinx/xilinx_emaclite.c b/drivers/net/ethernet/xilinx/xilinx_emaclite.c index 016a9c4f2c6c..05848ff15fb5 100644 --- a/drivers/net/ethernet/xilinx/xilinx_emaclite.c +++ b/drivers/net/ethernet/xilinx/xilinx_emaclite.c @@ -1060,7 +1060,7 @@ static bool get_bool(struct platform_device *ofdev, const char *s) static void xemaclite_ethtools_get_drvinfo(struct net_device *ndev, struct ethtool_drvinfo *ed) { - strlcpy(ed->driver, DRIVER_NAME, sizeof(ed->driver)); + strscpy(ed->driver, DRIVER_NAME, sizeof(ed->driver)); } static const struct ethtool_ops xemaclite_ethtool_ops = { diff --git a/drivers/net/ethernet/xircom/xirc2ps_cs.c b/drivers/net/ethernet/xircom/xirc2ps_cs.c index f9587e55b842..894e92ef415b 100644 --- a/drivers/net/ethernet/xircom/xirc2ps_cs.c +++ b/drivers/net/ethernet/xircom/xirc2ps_cs.c @@ -1402,7 +1402,7 @@ do_open(struct net_device *dev) static void netdev_get_drvinfo(struct net_device *dev, struct ethtool_drvinfo *info) { - strlcpy(info->driver, "xirc2ps_cs", sizeof(info->driver)); + strscpy(info->driver, "xirc2ps_cs", sizeof(info->driver)); snprintf(info->bus_info, sizeof(info->bus_info), "PCMCIA 0x%lx", dev->base_addr); } diff --git a/drivers/net/ethernet/xscale/ixp4xx_eth.c b/drivers/net/ethernet/xscale/ixp4xx_eth.c index 3591b9edc9a1..3b0c5f177447 100644 --- a/drivers/net/ethernet/xscale/ixp4xx_eth.c +++ b/drivers/net/ethernet/xscale/ixp4xx_eth.c @@ -841,7 +841,7 @@ static void eth_txdone_irq(void *unused) } } -static int eth_xmit(struct sk_buff *skb, struct net_device *dev) +static netdev_tx_t eth_xmit(struct sk_buff *skb, struct net_device *dev) { struct port *port = netdev_priv(dev); unsigned int txreadyq = port->plat->txreadyq; @@ -999,11 +999,11 @@ static void ixp4xx_get_drvinfo(struct net_device *dev, { struct port *port = netdev_priv(dev); - strlcpy(info->driver, DRV_NAME, sizeof(info->driver)); + strscpy(info->driver, DRV_NAME, sizeof(info->driver)); snprintf(info->fw_version, sizeof(info->fw_version), "%u:%u:%u:%u", port->firmware[0], port->firmware[1], port->firmware[2], port->firmware[3]); - strlcpy(info->bus_info, "internal", sizeof(info->bus_info)); + strscpy(info->bus_info, "internal", sizeof(info->bus_info)); } static int ixp4xx_get_ts_info(struct net_device *dev, diff --git a/drivers/net/fjes/fjes_ethtool.c b/drivers/net/fjes/fjes_ethtool.c index 746736c83873..19c99529566b 100644 --- a/drivers/net/fjes/fjes_ethtool.c +++ b/drivers/net/fjes/fjes_ethtool.c @@ -151,11 +151,11 @@ static void fjes_get_drvinfo(struct net_device *netdev, plat_dev = adapter->plat_dev; - strlcpy(drvinfo->driver, fjes_driver_name, sizeof(drvinfo->driver)); - strlcpy(drvinfo->version, fjes_driver_version, + strscpy(drvinfo->driver, fjes_driver_name, sizeof(drvinfo->driver)); + strscpy(drvinfo->version, fjes_driver_version, sizeof(drvinfo->version)); - strlcpy(drvinfo->fw_version, "none", sizeof(drvinfo->fw_version)); + strscpy(drvinfo->fw_version, "none", sizeof(drvinfo->fw_version)); snprintf(drvinfo->bus_info, sizeof(drvinfo->bus_info), "platform:%s", plat_dev->name); } diff --git a/drivers/net/fjes/fjes_main.c b/drivers/net/fjes/fjes_main.c index 5805e4a56385..1eff202f6a1f 100644 --- a/drivers/net/fjes/fjes_main.c +++ b/drivers/net/fjes/fjes_main.c @@ -32,68 +32,12 @@ MODULE_VERSION(DRV_VERSION); #define ACPI_MOTHERBOARD_RESOURCE_HID "PNP0C02" -static int fjes_request_irq(struct fjes_adapter *); -static void fjes_free_irq(struct fjes_adapter *); - -static int fjes_open(struct net_device *); -static int fjes_close(struct net_device *); -static int fjes_setup_resources(struct fjes_adapter *); -static void fjes_free_resources(struct fjes_adapter *); -static netdev_tx_t fjes_xmit_frame(struct sk_buff *, struct net_device *); -static void fjes_raise_intr_rxdata_task(struct work_struct *); -static void fjes_tx_stall_task(struct work_struct *); -static void fjes_force_close_task(struct work_struct *); -static irqreturn_t fjes_intr(int, void*); -static void fjes_get_stats64(struct net_device *, struct rtnl_link_stats64 *); -static int fjes_change_mtu(struct net_device *, int); -static int fjes_vlan_rx_add_vid(struct net_device *, __be16 proto, u16); -static int fjes_vlan_rx_kill_vid(struct net_device *, __be16 proto, u16); -static void fjes_tx_retry(struct net_device *, unsigned int txqueue); - -static int fjes_acpi_add(struct acpi_device *); -static int fjes_acpi_remove(struct acpi_device *); -static acpi_status fjes_get_acpi_resource(struct acpi_resource *, void*); - -static int fjes_probe(struct platform_device *); -static int fjes_remove(struct platform_device *); - -static int fjes_sw_init(struct fjes_adapter *); -static void fjes_netdev_setup(struct net_device *); -static void fjes_irq_watch_task(struct work_struct *); -static void fjes_watch_unshare_task(struct work_struct *); -static void fjes_rx_irq(struct fjes_adapter *, int); -static int fjes_poll(struct napi_struct *, int); - static const struct acpi_device_id fjes_acpi_ids[] = { {ACPI_MOTHERBOARD_RESOURCE_HID, 0}, {"", 0}, }; MODULE_DEVICE_TABLE(acpi, fjes_acpi_ids); -static struct acpi_driver fjes_acpi_driver = { - .name = DRV_NAME, - .class = DRV_NAME, - .owner = THIS_MODULE, - .ids = fjes_acpi_ids, - .ops = { - .add = fjes_acpi_add, - .remove = fjes_acpi_remove, - }, -}; - -static struct platform_driver fjes_driver = { - .driver = { - .name = DRV_NAME, - }, - .probe = fjes_probe, - .remove = fjes_remove, -}; - -static struct resource fjes_resource[] = { - DEFINE_RES_MEM(0, 1), - DEFINE_RES_IRQ(0) -}; - static bool is_extended_socket_device(struct acpi_device *device) { struct acpi_buffer buffer = { ACPI_ALLOCATE_BUFFER, NULL}; @@ -139,43 +83,6 @@ static int acpi_check_extended_socket_status(struct acpi_device *device) return 0; } -static int fjes_acpi_add(struct acpi_device *device) -{ - struct platform_device *plat_dev; - acpi_status status; - - if (!is_extended_socket_device(device)) - return -ENODEV; - - if (acpi_check_extended_socket_status(device)) - return -ENODEV; - - status = acpi_walk_resources(device->handle, METHOD_NAME__CRS, - fjes_get_acpi_resource, fjes_resource); - if (ACPI_FAILURE(status)) - return -ENODEV; - - /* create platform_device */ - plat_dev = platform_device_register_simple(DRV_NAME, 0, fjes_resource, - ARRAY_SIZE(fjes_resource)); - if (IS_ERR(plat_dev)) - return PTR_ERR(plat_dev); - - device->driver_data = plat_dev; - - return 0; -} - -static int fjes_acpi_remove(struct acpi_device *device) -{ - struct platform_device *plat_dev; - - plat_dev = (struct platform_device *)acpi_driver_data(device); - platform_device_unregister(plat_dev); - - return 0; -} - static acpi_status fjes_get_acpi_resource(struct acpi_resource *acpi_res, void *data) { @@ -206,143 +113,59 @@ fjes_get_acpi_resource(struct acpi_resource *acpi_res, void *data) return AE_OK; } -static int fjes_request_irq(struct fjes_adapter *adapter) -{ - struct net_device *netdev = adapter->netdev; - int result = -1; - - adapter->interrupt_watch_enable = true; - if (!delayed_work_pending(&adapter->interrupt_watch_task)) { - queue_delayed_work(adapter->control_wq, - &adapter->interrupt_watch_task, - FJES_IRQ_WATCH_DELAY); - } - - if (!adapter->irq_registered) { - result = request_irq(adapter->hw.hw_res.irq, fjes_intr, - IRQF_SHARED, netdev->name, adapter); - if (result) - adapter->irq_registered = false; - else - adapter->irq_registered = true; - } - - return result; -} - -static void fjes_free_irq(struct fjes_adapter *adapter) -{ - struct fjes_hw *hw = &adapter->hw; - - adapter->interrupt_watch_enable = false; - cancel_delayed_work_sync(&adapter->interrupt_watch_task); - - fjes_hw_set_irqmask(hw, REG_ICTL_MASK_ALL, true); - - if (adapter->irq_registered) { - free_irq(adapter->hw.hw_res.irq, adapter); - adapter->irq_registered = false; - } -} - -static const struct net_device_ops fjes_netdev_ops = { - .ndo_open = fjes_open, - .ndo_stop = fjes_close, - .ndo_start_xmit = fjes_xmit_frame, - .ndo_get_stats64 = fjes_get_stats64, - .ndo_change_mtu = fjes_change_mtu, - .ndo_tx_timeout = fjes_tx_retry, - .ndo_vlan_rx_add_vid = fjes_vlan_rx_add_vid, - .ndo_vlan_rx_kill_vid = fjes_vlan_rx_kill_vid, +static struct resource fjes_resource[] = { + DEFINE_RES_MEM(0, 1), + DEFINE_RES_IRQ(0) }; -/* fjes_open - Called when a network interface is made active */ -static int fjes_open(struct net_device *netdev) +static int fjes_acpi_add(struct acpi_device *device) { - struct fjes_adapter *adapter = netdev_priv(netdev); - struct fjes_hw *hw = &adapter->hw; - int result; - - if (adapter->open_guard) - return -ENXIO; - - result = fjes_setup_resources(adapter); - if (result) - goto err_setup_res; - - hw->txrx_stop_req_bit = 0; - hw->epstop_req_bit = 0; + struct platform_device *plat_dev; + acpi_status status; - napi_enable(&adapter->napi); + if (!is_extended_socket_device(device)) + return -ENODEV; - fjes_hw_capture_interrupt_status(hw); + if (acpi_check_extended_socket_status(device)) + return -ENODEV; - result = fjes_request_irq(adapter); - if (result) - goto err_req_irq; + status = acpi_walk_resources(device->handle, METHOD_NAME__CRS, + fjes_get_acpi_resource, fjes_resource); + if (ACPI_FAILURE(status)) + return -ENODEV; - fjes_hw_set_irqmask(hw, REG_ICTL_MASK_ALL, false); + /* create platform_device */ + plat_dev = platform_device_register_simple(DRV_NAME, 0, fjes_resource, + ARRAY_SIZE(fjes_resource)); + if (IS_ERR(plat_dev)) + return PTR_ERR(plat_dev); - netif_tx_start_all_queues(netdev); - netif_carrier_on(netdev); + device->driver_data = plat_dev; return 0; - -err_req_irq: - fjes_free_irq(adapter); - napi_disable(&adapter->napi); - -err_setup_res: - fjes_free_resources(adapter); - return result; } -/* fjes_close - Disables a network interface */ -static int fjes_close(struct net_device *netdev) +static int fjes_acpi_remove(struct acpi_device *device) { - struct fjes_adapter *adapter = netdev_priv(netdev); - struct fjes_hw *hw = &adapter->hw; - unsigned long flags; - int epidx; - - netif_tx_stop_all_queues(netdev); - netif_carrier_off(netdev); - - fjes_hw_raise_epstop(hw); - - napi_disable(&adapter->napi); - - spin_lock_irqsave(&hw->rx_status_lock, flags); - for (epidx = 0; epidx < hw->max_epid; epidx++) { - if (epidx == hw->my_epid) - continue; - - if (fjes_hw_get_partner_ep_status(hw, epidx) == - EP_PARTNER_SHARED) - adapter->hw.ep_shm_info[epidx] - .tx.info->v1i.rx_status &= - ~FJES_RX_POLL_WORK; - } - spin_unlock_irqrestore(&hw->rx_status_lock, flags); - - fjes_free_irq(adapter); - - cancel_delayed_work_sync(&adapter->interrupt_watch_task); - cancel_work_sync(&adapter->unshare_watch_task); - adapter->unshare_watch_bitmask = 0; - cancel_work_sync(&adapter->raise_intr_rxdata_task); - cancel_work_sync(&adapter->tx_stall_task); - - cancel_work_sync(&hw->update_zone_task); - cancel_work_sync(&hw->epstop_task); - - fjes_hw_wait_epstop(hw); + struct platform_device *plat_dev; - fjes_free_resources(adapter); + plat_dev = (struct platform_device *)acpi_driver_data(device); + platform_device_unregister(plat_dev); return 0; } +static struct acpi_driver fjes_acpi_driver = { + .name = DRV_NAME, + .class = DRV_NAME, + .owner = THIS_MODULE, + .ids = fjes_acpi_ids, + .ops = { + .add = fjes_acpi_add, + .remove = fjes_acpi_remove, + }, +}; + static int fjes_setup_resources(struct fjes_adapter *adapter) { struct net_device *netdev = adapter->netdev; @@ -421,6 +244,188 @@ static int fjes_setup_resources(struct fjes_adapter *adapter) return 0; } +static void fjes_rx_irq(struct fjes_adapter *adapter, int src_epid) +{ + struct fjes_hw *hw = &adapter->hw; + + fjes_hw_set_irqmask(hw, REG_ICTL_MASK_RX_DATA, true); + + adapter->unset_rx_last = true; + napi_schedule(&adapter->napi); +} + +static void fjes_stop_req_irq(struct fjes_adapter *adapter, int src_epid) +{ + struct fjes_hw *hw = &adapter->hw; + enum ep_partner_status status; + unsigned long flags; + + set_bit(src_epid, &hw->hw_info.buffer_unshare_reserve_bit); + + status = fjes_hw_get_partner_ep_status(hw, src_epid); + trace_fjes_stop_req_irq_pre(hw, src_epid, status); + switch (status) { + case EP_PARTNER_WAITING: + spin_lock_irqsave(&hw->rx_status_lock, flags); + hw->ep_shm_info[src_epid].tx.info->v1i.rx_status |= + FJES_RX_STOP_REQ_DONE; + spin_unlock_irqrestore(&hw->rx_status_lock, flags); + clear_bit(src_epid, &hw->txrx_stop_req_bit); + fallthrough; + case EP_PARTNER_UNSHARE: + case EP_PARTNER_COMPLETE: + default: + set_bit(src_epid, &adapter->unshare_watch_bitmask); + if (!work_pending(&adapter->unshare_watch_task)) + queue_work(adapter->control_wq, + &adapter->unshare_watch_task); + break; + case EP_PARTNER_SHARED: + set_bit(src_epid, &hw->epstop_req_bit); + + if (!work_pending(&hw->epstop_task)) + queue_work(adapter->control_wq, &hw->epstop_task); + break; + } + trace_fjes_stop_req_irq_post(hw, src_epid); +} + +static void fjes_txrx_stop_req_irq(struct fjes_adapter *adapter, + int src_epid) +{ + struct fjes_hw *hw = &adapter->hw; + enum ep_partner_status status; + unsigned long flags; + + status = fjes_hw_get_partner_ep_status(hw, src_epid); + trace_fjes_txrx_stop_req_irq_pre(hw, src_epid, status); + switch (status) { + case EP_PARTNER_UNSHARE: + case EP_PARTNER_COMPLETE: + default: + break; + case EP_PARTNER_WAITING: + if (src_epid < hw->my_epid) { + spin_lock_irqsave(&hw->rx_status_lock, flags); + hw->ep_shm_info[src_epid].tx.info->v1i.rx_status |= + FJES_RX_STOP_REQ_DONE; + spin_unlock_irqrestore(&hw->rx_status_lock, flags); + + clear_bit(src_epid, &hw->txrx_stop_req_bit); + set_bit(src_epid, &adapter->unshare_watch_bitmask); + + if (!work_pending(&adapter->unshare_watch_task)) + queue_work(adapter->control_wq, + &adapter->unshare_watch_task); + } + break; + case EP_PARTNER_SHARED: + if (hw->ep_shm_info[src_epid].rx.info->v1i.rx_status & + FJES_RX_STOP_REQ_REQUEST) { + set_bit(src_epid, &hw->epstop_req_bit); + if (!work_pending(&hw->epstop_task)) + queue_work(adapter->control_wq, + &hw->epstop_task); + } + break; + } + trace_fjes_txrx_stop_req_irq_post(hw, src_epid); +} + +static void fjes_update_zone_irq(struct fjes_adapter *adapter, + int src_epid) +{ + struct fjes_hw *hw = &adapter->hw; + + if (!work_pending(&hw->update_zone_task)) + queue_work(adapter->control_wq, &hw->update_zone_task); +} + +static irqreturn_t fjes_intr(int irq, void *data) +{ + struct fjes_adapter *adapter = data; + struct fjes_hw *hw = &adapter->hw; + irqreturn_t ret; + u32 icr; + + icr = fjes_hw_capture_interrupt_status(hw); + + if (icr & REG_IS_MASK_IS_ASSERT) { + if (icr & REG_ICTL_MASK_RX_DATA) { + fjes_rx_irq(adapter, icr & REG_IS_MASK_EPID); + hw->ep_shm_info[icr & REG_IS_MASK_EPID].ep_stats + .recv_intr_rx += 1; + } + + if (icr & REG_ICTL_MASK_DEV_STOP_REQ) { + fjes_stop_req_irq(adapter, icr & REG_IS_MASK_EPID); + hw->ep_shm_info[icr & REG_IS_MASK_EPID].ep_stats + .recv_intr_stop += 1; + } + + if (icr & REG_ICTL_MASK_TXRX_STOP_REQ) { + fjes_txrx_stop_req_irq(adapter, icr & REG_IS_MASK_EPID); + hw->ep_shm_info[icr & REG_IS_MASK_EPID].ep_stats + .recv_intr_unshare += 1; + } + + if (icr & REG_ICTL_MASK_TXRX_STOP_DONE) + fjes_hw_set_irqmask(hw, + REG_ICTL_MASK_TXRX_STOP_DONE, true); + + if (icr & REG_ICTL_MASK_INFO_UPDATE) { + fjes_update_zone_irq(adapter, icr & REG_IS_MASK_EPID); + hw->ep_shm_info[icr & REG_IS_MASK_EPID].ep_stats + .recv_intr_zoneupdate += 1; + } + + ret = IRQ_HANDLED; + } else { + ret = IRQ_NONE; + } + + return ret; +} + +static int fjes_request_irq(struct fjes_adapter *adapter) +{ + struct net_device *netdev = adapter->netdev; + int result = -1; + + adapter->interrupt_watch_enable = true; + if (!delayed_work_pending(&adapter->interrupt_watch_task)) { + queue_delayed_work(adapter->control_wq, + &adapter->interrupt_watch_task, + FJES_IRQ_WATCH_DELAY); + } + + if (!adapter->irq_registered) { + result = request_irq(adapter->hw.hw_res.irq, fjes_intr, + IRQF_SHARED, netdev->name, adapter); + if (result) + adapter->irq_registered = false; + else + adapter->irq_registered = true; + } + + return result; +} + +static void fjes_free_irq(struct fjes_adapter *adapter) +{ + struct fjes_hw *hw = &adapter->hw; + + adapter->interrupt_watch_enable = false; + cancel_delayed_work_sync(&adapter->interrupt_watch_task); + + fjes_hw_set_irqmask(hw, REG_ICTL_MASK_ALL, true); + + if (adapter->irq_registered) { + free_irq(adapter->hw.hw_res.irq, adapter); + adapter->irq_registered = false; + } +} + static void fjes_free_resources(struct fjes_adapter *adapter) { struct net_device *netdev = adapter->netdev; @@ -477,121 +482,91 @@ static void fjes_free_resources(struct fjes_adapter *adapter) } } -static void fjes_tx_stall_task(struct work_struct *work) +/* fjes_open - Called when a network interface is made active */ +static int fjes_open(struct net_device *netdev) { - struct fjes_adapter *adapter = container_of(work, - struct fjes_adapter, tx_stall_task); - struct net_device *netdev = adapter->netdev; + struct fjes_adapter *adapter = netdev_priv(netdev); struct fjes_hw *hw = &adapter->hw; - int all_queue_available, sendable; - enum ep_partner_status pstatus; - int max_epid, my_epid, epid; - union ep_buffer_info *info; - int i; - - if (((long)jiffies - - dev_trans_start(netdev)) > FJES_TX_TX_STALL_TIMEOUT) { - netif_wake_queue(netdev); - return; - } - - my_epid = hw->my_epid; - max_epid = hw->max_epid; + int result; - for (i = 0; i < 5; i++) { - all_queue_available = 1; + if (adapter->open_guard) + return -ENXIO; - for (epid = 0; epid < max_epid; epid++) { - if (my_epid == epid) - continue; + result = fjes_setup_resources(adapter); + if (result) + goto err_setup_res; - pstatus = fjes_hw_get_partner_ep_status(hw, epid); - sendable = (pstatus == EP_PARTNER_SHARED); - if (!sendable) - continue; + hw->txrx_stop_req_bit = 0; + hw->epstop_req_bit = 0; - info = adapter->hw.ep_shm_info[epid].tx.info; + napi_enable(&adapter->napi); - if (!(info->v1i.rx_status & FJES_RX_MTU_CHANGING_DONE)) - return; + fjes_hw_capture_interrupt_status(hw); - if (EP_RING_FULL(info->v1i.head, info->v1i.tail, - info->v1i.count_max)) { - all_queue_available = 0; - break; - } - } + result = fjes_request_irq(adapter); + if (result) + goto err_req_irq; - if (all_queue_available) { - netif_wake_queue(netdev); - return; - } - } + fjes_hw_set_irqmask(hw, REG_ICTL_MASK_ALL, false); - usleep_range(50, 100); + netif_tx_start_all_queues(netdev); + netif_carrier_on(netdev); - queue_work(adapter->txrx_wq, &adapter->tx_stall_task); -} + return 0; -static void fjes_force_close_task(struct work_struct *work) -{ - struct fjes_adapter *adapter = container_of(work, - struct fjes_adapter, force_close_task); - struct net_device *netdev = adapter->netdev; +err_req_irq: + fjes_free_irq(adapter); + napi_disable(&adapter->napi); - rtnl_lock(); - dev_close(netdev); - rtnl_unlock(); +err_setup_res: + fjes_free_resources(adapter); + return result; } -static void fjes_raise_intr_rxdata_task(struct work_struct *work) +/* fjes_close - Disables a network interface */ +static int fjes_close(struct net_device *netdev) { - struct fjes_adapter *adapter = container_of(work, - struct fjes_adapter, raise_intr_rxdata_task); + struct fjes_adapter *adapter = netdev_priv(netdev); struct fjes_hw *hw = &adapter->hw; - enum ep_partner_status pstatus; - int max_epid, my_epid, epid; + unsigned long flags; + int epidx; - my_epid = hw->my_epid; - max_epid = hw->max_epid; + netif_tx_stop_all_queues(netdev); + netif_carrier_off(netdev); - for (epid = 0; epid < max_epid; epid++) - hw->ep_shm_info[epid].tx_status_work = 0; + fjes_hw_raise_epstop(hw); - for (epid = 0; epid < max_epid; epid++) { - if (epid == my_epid) - continue; + napi_disable(&adapter->napi); - pstatus = fjes_hw_get_partner_ep_status(hw, epid); - if (pstatus == EP_PARTNER_SHARED) { - hw->ep_shm_info[epid].tx_status_work = - hw->ep_shm_info[epid].tx.info->v1i.tx_status; + spin_lock_irqsave(&hw->rx_status_lock, flags); + for (epidx = 0; epidx < hw->max_epid; epidx++) { + if (epidx == hw->my_epid) + continue; - if (hw->ep_shm_info[epid].tx_status_work == - FJES_TX_DELAY_SEND_PENDING) { - hw->ep_shm_info[epid].tx.info->v1i.tx_status = - FJES_TX_DELAY_SEND_NONE; - } - } + if (fjes_hw_get_partner_ep_status(hw, epidx) == + EP_PARTNER_SHARED) + adapter->hw.ep_shm_info[epidx] + .tx.info->v1i.rx_status &= + ~FJES_RX_POLL_WORK; } + spin_unlock_irqrestore(&hw->rx_status_lock, flags); - for (epid = 0; epid < max_epid; epid++) { - if (epid == my_epid) - continue; + fjes_free_irq(adapter); - pstatus = fjes_hw_get_partner_ep_status(hw, epid); - if ((hw->ep_shm_info[epid].tx_status_work == - FJES_TX_DELAY_SEND_PENDING) && - (pstatus == EP_PARTNER_SHARED) && - !(hw->ep_shm_info[epid].rx.info->v1i.rx_status & - FJES_RX_POLL_WORK)) { - fjes_hw_raise_interrupt(hw, epid, - REG_ICTL_MASK_RX_DATA); - hw->ep_shm_info[epid].ep_stats.send_intr_rx += 1; - } - } + cancel_delayed_work_sync(&adapter->interrupt_watch_task); + cancel_work_sync(&adapter->unshare_watch_task); + adapter->unshare_watch_bitmask = 0; + cancel_work_sync(&adapter->raise_intr_rxdata_task); + cancel_work_sync(&adapter->tx_stall_task); - usleep_range(500, 1000); + cancel_work_sync(&hw->update_zone_task); + cancel_work_sync(&hw->epstop_task); + + fjes_hw_wait_epstop(hw); + + fjes_free_resources(adapter); + + return 0; } static int fjes_tx_send(struct fjes_adapter *adapter, int dest, @@ -787,13 +762,6 @@ fjes_xmit_frame(struct sk_buff *skb, struct net_device *netdev) return ret; } -static void fjes_tx_retry(struct net_device *netdev, unsigned int txqueue) -{ - struct netdev_queue *queue = netdev_get_tx_queue(netdev, 0); - - netif_tx_wake_queue(queue); -} - static void fjes_get_stats64(struct net_device *netdev, struct rtnl_link_stats64 *stats) { @@ -871,6 +839,13 @@ static int fjes_change_mtu(struct net_device *netdev, int new_mtu) return ret; } +static void fjes_tx_retry(struct net_device *netdev, unsigned int txqueue) +{ + struct netdev_queue *queue = netdev_get_tx_queue(netdev, 0); + + netif_tx_wake_queue(queue); +} + static int fjes_vlan_rx_add_vid(struct net_device *netdev, __be16 proto, u16 vid) { @@ -907,137 +882,29 @@ static int fjes_vlan_rx_kill_vid(struct net_device *netdev, return 0; } -static void fjes_txrx_stop_req_irq(struct fjes_adapter *adapter, - int src_epid) -{ - struct fjes_hw *hw = &adapter->hw; - enum ep_partner_status status; - unsigned long flags; - - status = fjes_hw_get_partner_ep_status(hw, src_epid); - trace_fjes_txrx_stop_req_irq_pre(hw, src_epid, status); - switch (status) { - case EP_PARTNER_UNSHARE: - case EP_PARTNER_COMPLETE: - default: - break; - case EP_PARTNER_WAITING: - if (src_epid < hw->my_epid) { - spin_lock_irqsave(&hw->rx_status_lock, flags); - hw->ep_shm_info[src_epid].tx.info->v1i.rx_status |= - FJES_RX_STOP_REQ_DONE; - spin_unlock_irqrestore(&hw->rx_status_lock, flags); - - clear_bit(src_epid, &hw->txrx_stop_req_bit); - set_bit(src_epid, &adapter->unshare_watch_bitmask); - - if (!work_pending(&adapter->unshare_watch_task)) - queue_work(adapter->control_wq, - &adapter->unshare_watch_task); - } - break; - case EP_PARTNER_SHARED: - if (hw->ep_shm_info[src_epid].rx.info->v1i.rx_status & - FJES_RX_STOP_REQ_REQUEST) { - set_bit(src_epid, &hw->epstop_req_bit); - if (!work_pending(&hw->epstop_task)) - queue_work(adapter->control_wq, - &hw->epstop_task); - } - break; - } - trace_fjes_txrx_stop_req_irq_post(hw, src_epid); -} - -static void fjes_stop_req_irq(struct fjes_adapter *adapter, int src_epid) -{ - struct fjes_hw *hw = &adapter->hw; - enum ep_partner_status status; - unsigned long flags; - - set_bit(src_epid, &hw->hw_info.buffer_unshare_reserve_bit); - - status = fjes_hw_get_partner_ep_status(hw, src_epid); - trace_fjes_stop_req_irq_pre(hw, src_epid, status); - switch (status) { - case EP_PARTNER_WAITING: - spin_lock_irqsave(&hw->rx_status_lock, flags); - hw->ep_shm_info[src_epid].tx.info->v1i.rx_status |= - FJES_RX_STOP_REQ_DONE; - spin_unlock_irqrestore(&hw->rx_status_lock, flags); - clear_bit(src_epid, &hw->txrx_stop_req_bit); - fallthrough; - case EP_PARTNER_UNSHARE: - case EP_PARTNER_COMPLETE: - default: - set_bit(src_epid, &adapter->unshare_watch_bitmask); - if (!work_pending(&adapter->unshare_watch_task)) - queue_work(adapter->control_wq, - &adapter->unshare_watch_task); - break; - case EP_PARTNER_SHARED: - set_bit(src_epid, &hw->epstop_req_bit); - - if (!work_pending(&hw->epstop_task)) - queue_work(adapter->control_wq, &hw->epstop_task); - break; - } - trace_fjes_stop_req_irq_post(hw, src_epid); -} - -static void fjes_update_zone_irq(struct fjes_adapter *adapter, - int src_epid) -{ - struct fjes_hw *hw = &adapter->hw; - - if (!work_pending(&hw->update_zone_task)) - queue_work(adapter->control_wq, &hw->update_zone_task); -} +static const struct net_device_ops fjes_netdev_ops = { + .ndo_open = fjes_open, + .ndo_stop = fjes_close, + .ndo_start_xmit = fjes_xmit_frame, + .ndo_get_stats64 = fjes_get_stats64, + .ndo_change_mtu = fjes_change_mtu, + .ndo_tx_timeout = fjes_tx_retry, + .ndo_vlan_rx_add_vid = fjes_vlan_rx_add_vid, + .ndo_vlan_rx_kill_vid = fjes_vlan_rx_kill_vid, +}; -static irqreturn_t fjes_intr(int irq, void *data) +/* fjes_netdev_setup - netdevice initialization routine */ +static void fjes_netdev_setup(struct net_device *netdev) { - struct fjes_adapter *adapter = data; - struct fjes_hw *hw = &adapter->hw; - irqreturn_t ret; - u32 icr; - - icr = fjes_hw_capture_interrupt_status(hw); - - if (icr & REG_IS_MASK_IS_ASSERT) { - if (icr & REG_ICTL_MASK_RX_DATA) { - fjes_rx_irq(adapter, icr & REG_IS_MASK_EPID); - hw->ep_shm_info[icr & REG_IS_MASK_EPID].ep_stats - .recv_intr_rx += 1; - } - - if (icr & REG_ICTL_MASK_DEV_STOP_REQ) { - fjes_stop_req_irq(adapter, icr & REG_IS_MASK_EPID); - hw->ep_shm_info[icr & REG_IS_MASK_EPID].ep_stats - .recv_intr_stop += 1; - } - - if (icr & REG_ICTL_MASK_TXRX_STOP_REQ) { - fjes_txrx_stop_req_irq(adapter, icr & REG_IS_MASK_EPID); - hw->ep_shm_info[icr & REG_IS_MASK_EPID].ep_stats - .recv_intr_unshare += 1; - } - - if (icr & REG_ICTL_MASK_TXRX_STOP_DONE) - fjes_hw_set_irqmask(hw, - REG_ICTL_MASK_TXRX_STOP_DONE, true); - - if (icr & REG_ICTL_MASK_INFO_UPDATE) { - fjes_update_zone_irq(adapter, icr & REG_IS_MASK_EPID); - hw->ep_shm_info[icr & REG_IS_MASK_EPID].ep_stats - .recv_intr_zoneupdate += 1; - } - - ret = IRQ_HANDLED; - } else { - ret = IRQ_NONE; - } + ether_setup(netdev); - return ret; + netdev->watchdog_timeo = FJES_TX_RETRY_INTERVAL; + netdev->netdev_ops = &fjes_netdev_ops; + fjes_set_ethtool_ops(netdev); + netdev->mtu = fjes_support_mtu[3]; + netdev->min_mtu = fjes_support_mtu[0]; + netdev->max_mtu = fjes_support_mtu[3]; + netdev->features |= NETIF_F_HW_VLAN_CTAG_FILTER; } static int fjes_rxframe_search_exist(struct fjes_adapter *adapter, @@ -1087,16 +954,6 @@ static void fjes_rxframe_release(struct fjes_adapter *adapter, int cur_epid) fjes_hw_epbuf_rx_curpkt_drop(&adapter->hw.ep_shm_info[cur_epid].rx); } -static void fjes_rx_irq(struct fjes_adapter *adapter, int src_epid) -{ - struct fjes_hw *hw = &adapter->hw; - - fjes_hw_set_irqmask(hw, REG_ICTL_MASK_RX_DATA, true); - - adapter->unset_rx_last = true; - napi_schedule(&adapter->napi); -} - static int fjes_poll(struct napi_struct *napi, int budget) { struct fjes_adapter *adapter = @@ -1196,182 +1053,130 @@ static int fjes_poll(struct napi_struct *napi, int budget) return work_done; } -/* fjes_probe - Device Initialization Routine */ -static int fjes_probe(struct platform_device *plat_dev) +static int fjes_sw_init(struct fjes_adapter *adapter) { - struct fjes_adapter *adapter; - struct net_device *netdev; - struct resource *res; - struct fjes_hw *hw; - u8 addr[ETH_ALEN]; - int err; - - err = -ENOMEM; - netdev = alloc_netdev_mq(sizeof(struct fjes_adapter), "es%d", - NET_NAME_UNKNOWN, fjes_netdev_setup, - FJES_MAX_QUEUES); - - if (!netdev) - goto err_out; + struct net_device *netdev = adapter->netdev; - SET_NETDEV_DEV(netdev, &plat_dev->dev); + netif_napi_add(netdev, &adapter->napi, fjes_poll); - dev_set_drvdata(&plat_dev->dev, netdev); - adapter = netdev_priv(netdev); - adapter->netdev = netdev; - adapter->plat_dev = plat_dev; - hw = &adapter->hw; - hw->back = adapter; + return 0; +} - /* setup the private structure */ - err = fjes_sw_init(adapter); - if (err) - goto err_free_netdev; +static void fjes_force_close_task(struct work_struct *work) +{ + struct fjes_adapter *adapter = container_of(work, + struct fjes_adapter, force_close_task); + struct net_device *netdev = adapter->netdev; - INIT_WORK(&adapter->force_close_task, fjes_force_close_task); - adapter->force_reset = false; - adapter->open_guard = false; + rtnl_lock(); + dev_close(netdev); + rtnl_unlock(); +} - adapter->txrx_wq = alloc_workqueue(DRV_NAME "/txrx", WQ_MEM_RECLAIM, 0); - if (unlikely(!adapter->txrx_wq)) { - err = -ENOMEM; - goto err_free_netdev; - } +static void fjes_tx_stall_task(struct work_struct *work) +{ + struct fjes_adapter *adapter = container_of(work, + struct fjes_adapter, tx_stall_task); + struct net_device *netdev = adapter->netdev; + struct fjes_hw *hw = &adapter->hw; + int all_queue_available, sendable; + enum ep_partner_status pstatus; + int max_epid, my_epid, epid; + union ep_buffer_info *info; + int i; - adapter->control_wq = alloc_workqueue(DRV_NAME "/control", - WQ_MEM_RECLAIM, 0); - if (unlikely(!adapter->control_wq)) { - err = -ENOMEM; - goto err_free_txrx_wq; + if (((long)jiffies - + dev_trans_start(netdev)) > FJES_TX_TX_STALL_TIMEOUT) { + netif_wake_queue(netdev); + return; } - INIT_WORK(&adapter->tx_stall_task, fjes_tx_stall_task); - INIT_WORK(&adapter->raise_intr_rxdata_task, - fjes_raise_intr_rxdata_task); - INIT_WORK(&adapter->unshare_watch_task, fjes_watch_unshare_task); - adapter->unshare_watch_bitmask = 0; + my_epid = hw->my_epid; + max_epid = hw->max_epid; - INIT_DELAYED_WORK(&adapter->interrupt_watch_task, fjes_irq_watch_task); - adapter->interrupt_watch_enable = false; + for (i = 0; i < 5; i++) { + all_queue_available = 1; - res = platform_get_resource(plat_dev, IORESOURCE_MEM, 0); - if (!res) { - err = -EINVAL; - goto err_free_control_wq; - } - hw->hw_res.start = res->start; - hw->hw_res.size = resource_size(res); - hw->hw_res.irq = platform_get_irq(plat_dev, 0); - if (hw->hw_res.irq < 0) { - err = hw->hw_res.irq; - goto err_free_control_wq; - } + for (epid = 0; epid < max_epid; epid++) { + if (my_epid == epid) + continue; - err = fjes_hw_init(&adapter->hw); - if (err) - goto err_free_control_wq; + pstatus = fjes_hw_get_partner_ep_status(hw, epid); + sendable = (pstatus == EP_PARTNER_SHARED); + if (!sendable) + continue; - /* setup MAC address (02:00:00:00:00:[epid])*/ - addr[0] = 2; - addr[1] = 0; - addr[2] = 0; - addr[3] = 0; - addr[4] = 0; - addr[5] = hw->my_epid; /* EPID */ - eth_hw_addr_set(netdev, addr); + info = adapter->hw.ep_shm_info[epid].tx.info; - err = register_netdev(netdev); - if (err) - goto err_hw_exit; + if (!(info->v1i.rx_status & FJES_RX_MTU_CHANGING_DONE)) + return; - netif_carrier_off(netdev); + if (EP_RING_FULL(info->v1i.head, info->v1i.tail, + info->v1i.count_max)) { + all_queue_available = 0; + break; + } + } - fjes_dbg_adapter_init(adapter); + if (all_queue_available) { + netif_wake_queue(netdev); + return; + } + } - return 0; + usleep_range(50, 100); -err_hw_exit: - fjes_hw_exit(&adapter->hw); -err_free_control_wq: - destroy_workqueue(adapter->control_wq); -err_free_txrx_wq: - destroy_workqueue(adapter->txrx_wq); -err_free_netdev: - free_netdev(netdev); -err_out: - return err; + queue_work(adapter->txrx_wq, &adapter->tx_stall_task); } -/* fjes_remove - Device Removal Routine */ -static int fjes_remove(struct platform_device *plat_dev) +static void fjes_raise_intr_rxdata_task(struct work_struct *work) { - struct net_device *netdev = dev_get_drvdata(&plat_dev->dev); - struct fjes_adapter *adapter = netdev_priv(netdev); + struct fjes_adapter *adapter = container_of(work, + struct fjes_adapter, raise_intr_rxdata_task); struct fjes_hw *hw = &adapter->hw; + enum ep_partner_status pstatus; + int max_epid, my_epid, epid; - fjes_dbg_adapter_exit(adapter); - - cancel_delayed_work_sync(&adapter->interrupt_watch_task); - cancel_work_sync(&adapter->unshare_watch_task); - cancel_work_sync(&adapter->raise_intr_rxdata_task); - cancel_work_sync(&adapter->tx_stall_task); - if (adapter->control_wq) - destroy_workqueue(adapter->control_wq); - if (adapter->txrx_wq) - destroy_workqueue(adapter->txrx_wq); - - unregister_netdev(netdev); - - fjes_hw_exit(hw); - - netif_napi_del(&adapter->napi); - - free_netdev(netdev); - - return 0; -} - -static int fjes_sw_init(struct fjes_adapter *adapter) -{ - struct net_device *netdev = adapter->netdev; - - netif_napi_add(netdev, &adapter->napi, fjes_poll, 64); - - return 0; -} + my_epid = hw->my_epid; + max_epid = hw->max_epid; -/* fjes_netdev_setup - netdevice initialization routine */ -static void fjes_netdev_setup(struct net_device *netdev) -{ - ether_setup(netdev); + for (epid = 0; epid < max_epid; epid++) + hw->ep_shm_info[epid].tx_status_work = 0; - netdev->watchdog_timeo = FJES_TX_RETRY_INTERVAL; - netdev->netdev_ops = &fjes_netdev_ops; - fjes_set_ethtool_ops(netdev); - netdev->mtu = fjes_support_mtu[3]; - netdev->min_mtu = fjes_support_mtu[0]; - netdev->max_mtu = fjes_support_mtu[3]; - netdev->features |= NETIF_F_HW_VLAN_CTAG_FILTER; -} + for (epid = 0; epid < max_epid; epid++) { + if (epid == my_epid) + continue; -static void fjes_irq_watch_task(struct work_struct *work) -{ - struct fjes_adapter *adapter = container_of(to_delayed_work(work), - struct fjes_adapter, interrupt_watch_task); + pstatus = fjes_hw_get_partner_ep_status(hw, epid); + if (pstatus == EP_PARTNER_SHARED) { + hw->ep_shm_info[epid].tx_status_work = + hw->ep_shm_info[epid].tx.info->v1i.tx_status; - local_irq_disable(); - fjes_intr(adapter->hw.hw_res.irq, adapter); - local_irq_enable(); + if (hw->ep_shm_info[epid].tx_status_work == + FJES_TX_DELAY_SEND_PENDING) { + hw->ep_shm_info[epid].tx.info->v1i.tx_status = + FJES_TX_DELAY_SEND_NONE; + } + } + } - if (fjes_rxframe_search_exist(adapter, 0) >= 0) - napi_schedule(&adapter->napi); + for (epid = 0; epid < max_epid; epid++) { + if (epid == my_epid) + continue; - if (adapter->interrupt_watch_enable) { - if (!delayed_work_pending(&adapter->interrupt_watch_task)) - queue_delayed_work(adapter->control_wq, - &adapter->interrupt_watch_task, - FJES_IRQ_WATCH_DELAY); + pstatus = fjes_hw_get_partner_ep_status(hw, epid); + if ((hw->ep_shm_info[epid].tx_status_work == + FJES_TX_DELAY_SEND_PENDING) && + (pstatus == EP_PARTNER_SHARED) && + !(hw->ep_shm_info[epid].rx.info->v1i.rx_status & + FJES_RX_POLL_WORK)) { + fjes_hw_raise_interrupt(hw, epid, + REG_ICTL_MASK_RX_DATA); + hw->ep_shm_info[epid].ep_stats.send_intr_rx += 1; + } } + + usleep_range(500, 1000); } static void fjes_watch_unshare_task(struct work_struct *work) @@ -1508,6 +1313,169 @@ static void fjes_watch_unshare_task(struct work_struct *work) } } +static void fjes_irq_watch_task(struct work_struct *work) +{ + struct fjes_adapter *adapter = container_of(to_delayed_work(work), + struct fjes_adapter, interrupt_watch_task); + + local_irq_disable(); + fjes_intr(adapter->hw.hw_res.irq, adapter); + local_irq_enable(); + + if (fjes_rxframe_search_exist(adapter, 0) >= 0) + napi_schedule(&adapter->napi); + + if (adapter->interrupt_watch_enable) { + if (!delayed_work_pending(&adapter->interrupt_watch_task)) + queue_delayed_work(adapter->control_wq, + &adapter->interrupt_watch_task, + FJES_IRQ_WATCH_DELAY); + } +} + +/* fjes_probe - Device Initialization Routine */ +static int fjes_probe(struct platform_device *plat_dev) +{ + struct fjes_adapter *adapter; + struct net_device *netdev; + struct resource *res; + struct fjes_hw *hw; + u8 addr[ETH_ALEN]; + int err; + + err = -ENOMEM; + netdev = alloc_netdev_mq(sizeof(struct fjes_adapter), "es%d", + NET_NAME_UNKNOWN, fjes_netdev_setup, + FJES_MAX_QUEUES); + + if (!netdev) + goto err_out; + + SET_NETDEV_DEV(netdev, &plat_dev->dev); + + dev_set_drvdata(&plat_dev->dev, netdev); + adapter = netdev_priv(netdev); + adapter->netdev = netdev; + adapter->plat_dev = plat_dev; + hw = &adapter->hw; + hw->back = adapter; + + /* setup the private structure */ + err = fjes_sw_init(adapter); + if (err) + goto err_free_netdev; + + INIT_WORK(&adapter->force_close_task, fjes_force_close_task); + adapter->force_reset = false; + adapter->open_guard = false; + + adapter->txrx_wq = alloc_workqueue(DRV_NAME "/txrx", WQ_MEM_RECLAIM, 0); + if (unlikely(!adapter->txrx_wq)) { + err = -ENOMEM; + goto err_free_netdev; + } + + adapter->control_wq = alloc_workqueue(DRV_NAME "/control", + WQ_MEM_RECLAIM, 0); + if (unlikely(!adapter->control_wq)) { + err = -ENOMEM; + goto err_free_txrx_wq; + } + + INIT_WORK(&adapter->tx_stall_task, fjes_tx_stall_task); + INIT_WORK(&adapter->raise_intr_rxdata_task, + fjes_raise_intr_rxdata_task); + INIT_WORK(&adapter->unshare_watch_task, fjes_watch_unshare_task); + adapter->unshare_watch_bitmask = 0; + + INIT_DELAYED_WORK(&adapter->interrupt_watch_task, fjes_irq_watch_task); + adapter->interrupt_watch_enable = false; + + res = platform_get_resource(plat_dev, IORESOURCE_MEM, 0); + if (!res) { + err = -EINVAL; + goto err_free_control_wq; + } + hw->hw_res.start = res->start; + hw->hw_res.size = resource_size(res); + hw->hw_res.irq = platform_get_irq(plat_dev, 0); + if (hw->hw_res.irq < 0) { + err = hw->hw_res.irq; + goto err_free_control_wq; + } + + err = fjes_hw_init(&adapter->hw); + if (err) + goto err_free_control_wq; + + /* setup MAC address (02:00:00:00:00:[epid])*/ + addr[0] = 2; + addr[1] = 0; + addr[2] = 0; + addr[3] = 0; + addr[4] = 0; + addr[5] = hw->my_epid; /* EPID */ + eth_hw_addr_set(netdev, addr); + + err = register_netdev(netdev); + if (err) + goto err_hw_exit; + + netif_carrier_off(netdev); + + fjes_dbg_adapter_init(adapter); + + return 0; + +err_hw_exit: + fjes_hw_exit(&adapter->hw); +err_free_control_wq: + destroy_workqueue(adapter->control_wq); +err_free_txrx_wq: + destroy_workqueue(adapter->txrx_wq); +err_free_netdev: + free_netdev(netdev); +err_out: + return err; +} + +/* fjes_remove - Device Removal Routine */ +static int fjes_remove(struct platform_device *plat_dev) +{ + struct net_device *netdev = dev_get_drvdata(&plat_dev->dev); + struct fjes_adapter *adapter = netdev_priv(netdev); + struct fjes_hw *hw = &adapter->hw; + + fjes_dbg_adapter_exit(adapter); + + cancel_delayed_work_sync(&adapter->interrupt_watch_task); + cancel_work_sync(&adapter->unshare_watch_task); + cancel_work_sync(&adapter->raise_intr_rxdata_task); + cancel_work_sync(&adapter->tx_stall_task); + if (adapter->control_wq) + destroy_workqueue(adapter->control_wq); + if (adapter->txrx_wq) + destroy_workqueue(adapter->txrx_wq); + + unregister_netdev(netdev); + + fjes_hw_exit(hw); + + netif_napi_del(&adapter->napi); + + free_netdev(netdev); + + return 0; +} + +static struct platform_driver fjes_driver = { + .driver = { + .name = DRV_NAME, + }, + .probe = fjes_probe, + .remove = fjes_remove, +}; + static acpi_status acpi_find_extended_socket_device(acpi_handle obj_handle, u32 level, void *context, void **return_value) diff --git a/drivers/net/geneve.c b/drivers/net/geneve.c index 7962c37b3f14..f393e454f45c 100644 --- a/drivers/net/geneve.c +++ b/drivers/net/geneve.c @@ -503,12 +503,9 @@ static struct sk_buff *geneve_gro_receive(struct sock *sk, off_gnv = skb_gro_offset(skb); hlen = off_gnv + sizeof(*gh); - gh = skb_gro_header_fast(skb, off_gnv); - if (skb_gro_header_hard(skb, hlen)) { - gh = skb_gro_header_slow(skb, hlen, off_gnv); - if (unlikely(!gh)) - goto out; - } + gh = skb_gro_header(skb, hlen, off_gnv); + if (unlikely(!gh)) + goto out; if (gh->ver != GENEVE_VER || gh->oam) goto out; @@ -1200,8 +1197,8 @@ static const struct net_device_ops geneve_netdev_ops = { static void geneve_get_drvinfo(struct net_device *dev, struct ethtool_drvinfo *drvinfo) { - strlcpy(drvinfo->version, GENEVE_NETDEV_VER, sizeof(drvinfo->version)); - strlcpy(drvinfo->driver, "geneve", sizeof(drvinfo->driver)); + strscpy(drvinfo->version, GENEVE_NETDEV_VER, sizeof(drvinfo->version)); + strscpy(drvinfo->driver, "geneve", sizeof(drvinfo->driver)); } static const struct ethtool_ops geneve_ethtool_ops = { diff --git a/drivers/net/gtp.c b/drivers/net/gtp.c index a208e2b1a9af..15c7dc82107f 100644 --- a/drivers/net/gtp.c +++ b/drivers/net/gtp.c @@ -1859,6 +1859,7 @@ static struct genl_family gtp_genl_family __ro_after_init = { .module = THIS_MODULE, .small_ops = gtp_genl_ops, .n_small_ops = ARRAY_SIZE(gtp_genl_ops), + .resv_start_op = GTP_CMD_ECHOREQ + 1, .mcgrps = gtp_genl_mcgrps, .n_mcgrps = ARRAY_SIZE(gtp_genl_mcgrps), }; diff --git a/drivers/net/hamradio/hdlcdrv.c b/drivers/net/hamradio/hdlcdrv.c index 8297411e87ea..a6184d6c7b15 100644 --- a/drivers/net/hamradio/hdlcdrv.c +++ b/drivers/net/hamradio/hdlcdrv.c @@ -600,7 +600,7 @@ static int hdlcdrv_siocdevprivate(struct net_device *dev, struct ifreq *ifr, case HDLCDRVCTL_DRIVERNAME: if (s->ops && s->ops->drvname) { - strlcpy(bi.data.drivername, s->ops->drvname, + strscpy(bi.data.drivername, s->ops->drvname, sizeof(bi.data.drivername)); break; } diff --git a/drivers/net/hyperv/netvsc.c b/drivers/net/hyperv/netvsc.c index 6e42cb03e226..f066de0da492 100644 --- a/drivers/net/hyperv/netvsc.c +++ b/drivers/net/hyperv/netvsc.c @@ -1779,8 +1779,7 @@ struct netvsc_device *netvsc_device_add(struct hv_device *device, } /* Enable NAPI handler before init callbacks */ - netif_napi_add(ndev, &net_device->chan_table[0].napi, - netvsc_poll, NAPI_POLL_WEIGHT); + netif_napi_add(ndev, &net_device->chan_table[0].napi, netvsc_poll); /* Open the channel */ device->channel->next_request_id_callback = vmbus_next_request_id; diff --git a/drivers/net/hyperv/netvsc_drv.c b/drivers/net/hyperv/netvsc_drv.c index 15ebd5426604..5f08482065ca 100644 --- a/drivers/net/hyperv/netvsc_drv.c +++ b/drivers/net/hyperv/netvsc_drv.c @@ -935,8 +935,8 @@ int netvsc_recv_callback(struct net_device *net, static void netvsc_get_drvinfo(struct net_device *net, struct ethtool_drvinfo *info) { - strlcpy(info->driver, KBUILD_MODNAME, sizeof(info->driver)); - strlcpy(info->fw_version, "N/A", sizeof(info->fw_version)); + strscpy(info->driver, KBUILD_MODNAME, sizeof(info->driver)); + strscpy(info->fw_version, "N/A", sizeof(info->fw_version)); } static void netvsc_get_channels(struct net_device *net, diff --git a/drivers/net/hyperv/rndis_filter.c b/drivers/net/hyperv/rndis_filter.c index 6da36cb8af80..11f767a20444 100644 --- a/drivers/net/hyperv/rndis_filter.c +++ b/drivers/net/hyperv/rndis_filter.c @@ -1575,7 +1575,7 @@ struct netvsc_device *rndis_filter_device_add(struct hv_device *dev, for (i = 1; i < net_device->num_chn; i++) netif_napi_add(net, &net_device->chan_table[i].napi, - netvsc_poll, NAPI_POLL_WEIGHT); + netvsc_poll); return net_device; diff --git a/drivers/net/ieee802154/mac802154_hwsim.c b/drivers/net/ieee802154/mac802154_hwsim.c index 38c217bd7c82..2f0544dd7c2a 100644 --- a/drivers/net/ieee802154/mac802154_hwsim.c +++ b/drivers/net/ieee802154/mac802154_hwsim.c @@ -630,6 +630,7 @@ static struct genl_family hwsim_genl_family __ro_after_init = { .module = THIS_MODULE, .small_ops = hwsim_nl_ops, .n_small_ops = ARRAY_SIZE(hwsim_nl_ops), + .resv_start_op = MAC802154_HWSIM_CMD_NEW_EDGE + 1, .mcgrps = hwsim_mcgrps, .n_mcgrps = ARRAY_SIZE(hwsim_mcgrps), }; diff --git a/drivers/net/ipa/Makefile b/drivers/net/ipa/Makefile index 8b2220eb6b92..48255fc4b25c 100644 --- a/drivers/net/ipa/Makefile +++ b/drivers/net/ipa/Makefile @@ -13,4 +13,6 @@ ipa-y := ipa_main.o ipa_power.o ipa_reg.o ipa_mem.o \ ipa_resource.o ipa_qmi.o ipa_qmi_msg.o \ ipa_sysfs.o +ipa-y += $(IPA_VERSIONS:%=reg/ipa_reg-v%.o) + ipa-y += $(IPA_VERSIONS:%=data/ipa_data-v%.o) diff --git a/drivers/net/ipa/data/ipa_data-v3.1.c b/drivers/net/ipa/data/ipa_data-v3.1.c index 1c1895aea811..e0d71f609272 100644 --- a/drivers/net/ipa/data/ipa_data-v3.1.c +++ b/drivers/net/ipa/data/ipa_data-v3.1.c @@ -526,7 +526,7 @@ static const struct ipa_power_data ipa_power_data = { /* Configuration data for an SoC having IPA v3.1 */ const struct ipa_data ipa_data_v3_1 = { .version = IPA_VERSION_3_1, - .backward_compat = BCR_CMDQ_L_LACK_ONE_ENTRY_FMASK, + .backward_compat = BIT(BCR_CMDQ_L_LACK_ONE_ENTRY), .qsb_count = ARRAY_SIZE(ipa_qsb_data), .qsb_data = ipa_qsb_data, .endpoint_count = ARRAY_SIZE(ipa_gsi_endpoint_data), diff --git a/drivers/net/ipa/data/ipa_data-v3.5.1.c b/drivers/net/ipa/data/ipa_data-v3.5.1.c index 58b708d2fc75..383ef1890065 100644 --- a/drivers/net/ipa/data/ipa_data-v3.5.1.c +++ b/drivers/net/ipa/data/ipa_data-v3.5.1.c @@ -407,11 +407,11 @@ static const struct ipa_power_data ipa_power_data = { /* Configuration data for an SoC having IPA v3.5.1 */ const struct ipa_data ipa_data_v3_5_1 = { .version = IPA_VERSION_3_5_1, - .backward_compat = BCR_CMDQ_L_LACK_ONE_ENTRY_FMASK | - BCR_TX_NOT_USING_BRESP_FMASK | - BCR_SUSPEND_L2_IRQ_FMASK | - BCR_HOLB_DROP_L2_IRQ_FMASK | - BCR_DUAL_TX_FMASK, + .backward_compat = BIT(BCR_CMDQ_L_LACK_ONE_ENTRY) | + BIT(BCR_TX_NOT_USING_BRESP) | + BIT(BCR_SUSPEND_L2_IRQ) | + BIT(BCR_HOLB_DROP_L2_IRQ) | + BIT(BCR_DUAL_TX), .qsb_count = ARRAY_SIZE(ipa_qsb_data), .qsb_data = ipa_qsb_data, .endpoint_count = ARRAY_SIZE(ipa_gsi_endpoint_data), diff --git a/drivers/net/ipa/gsi.c b/drivers/net/ipa/gsi.c index 9e307eebd33f..bea2da1c4c51 100644 --- a/drivers/net/ipa/gsi.c +++ b/drivers/net/ipa/gsi.c @@ -1,7 +1,7 @@ // SPDX-License-Identifier: GPL-2.0 /* Copyright (c) 2015-2018, The Linux Foundation. All rights reserved. - * Copyright (C) 2018-2021 Linaro Ltd. + * Copyright (C) 2018-2022 Linaro Ltd. */ #include <linux/types.h> @@ -56,9 +56,9 @@ * element can also contain an immediate command, requesting the IPA perform * actions other than data transfer. * - * Each TRE refers to a block of data--also located DRAM. After writing one - * or more TREs to a channel, the writer (either the IPA or an EE) writes a - * doorbell register to inform the receiving side how many elements have + * Each TRE refers to a block of data--also located in DRAM. After writing + * one or more TREs to a channel, the writer (either the IPA or an EE) writes + * a doorbell register to inform the receiving side how many elements have * been written. * * Each channel has a GSI "event ring" associated with it. An event ring @@ -710,43 +710,32 @@ static void gsi_evt_ring_program(struct gsi *gsi, u32 evt_ring_id) static struct gsi_trans *gsi_channel_trans_last(struct gsi_channel *channel) { struct gsi_trans_info *trans_info = &channel->trans_info; - const struct list_head *list; + u32 pending_id = trans_info->pending_id; struct gsi_trans *trans; - - spin_lock_bh(&trans_info->spinlock); - - /* There is a small chance a TX transaction got allocated just - * before we disabled transmits, so check for that. - */ - if (channel->toward_ipa) { - list = &trans_info->alloc; - if (!list_empty(list)) - goto done; - list = &trans_info->committed; - if (!list_empty(list)) - goto done; - list = &trans_info->pending; - if (!list_empty(list)) - goto done; + u16 trans_id; + + if (channel->toward_ipa && pending_id != trans_info->free_id) { + /* There is a small chance a TX transaction got allocated + * just before we disabled transmits, so check for that. + * The last allocated, committed, or pending transaction + * precedes the first free transaction. + */ + trans_id = trans_info->free_id - 1; + } else if (trans_info->polled_id != pending_id) { + /* Otherwise (TX or RX) we want to wait for anything that + * has completed, or has been polled but not released yet. + * + * The last completed or polled transaction precedes the + * first pending transaction. + */ + trans_id = pending_id - 1; + } else { + return NULL; } - /* Otherwise (TX or RX) we want to wait for anything that - * has completed, or has been polled but not released yet. - */ - list = &trans_info->complete; - if (!list_empty(list)) - goto done; - list = &trans_info->polled; - if (list_empty(list)) - list = NULL; -done: - trans = list ? list_last_entry(list, struct gsi_trans, links) : NULL; - /* Caller will wait for this, so take a reference */ - if (trans) - refcount_inc(&trans->refcount); - - spin_unlock_bh(&trans_info->spinlock); + trans = &trans_info->trans[trans_id % channel->tre_count]; + refcount_inc(&trans->refcount); return trans; } @@ -1358,8 +1347,8 @@ gsi_event_trans(struct gsi *gsi, struct gsi_event *event) * we update transactions to record their actual received lengths. * * When an event for a TX channel arrives we use information in the - * transaction to report the number of requests and bytes have been - * transferred. + * transaction to report the number of requests and bytes that have + * been transferred. * * This function is called whenever we learn that the GSI hardware has filled * new events since the last time we checked. The ring's index field tells @@ -1485,8 +1474,8 @@ void gsi_channel_doorbell(struct gsi_channel *channel) iowrite32(val, gsi->virt + GSI_CH_C_DOORBELL_0_OFFSET(channel_id)); } -/* Consult hardware, move any newly completed transactions to completed list */ -static struct gsi_trans *gsi_channel_update(struct gsi_channel *channel) +/* Consult hardware, move newly completed transactions to completed state */ +void gsi_channel_update(struct gsi_channel *channel) { u32 evt_ring_id = channel->evt_ring_id; struct gsi *gsi = channel->gsi; @@ -1505,12 +1494,12 @@ static struct gsi_trans *gsi_channel_update(struct gsi_channel *channel) offset = GSI_EV_CH_E_CNTXT_4_OFFSET(evt_ring_id); index = gsi_ring_index(ring, ioread32(gsi->virt + offset)); if (index == ring->index % ring->count) - return NULL; + return; /* Get the transaction for the latest completed event. */ trans = gsi_event_trans(gsi, gsi_ring_virt(ring, index - 1)); if (!trans) - return NULL; + return; /* For RX channels, update each completed transaction with the number * of bytes that were actually received. For TX channels, report @@ -1518,8 +1507,6 @@ static struct gsi_trans *gsi_channel_update(struct gsi_channel *channel) * up the network stack. */ gsi_evt_ring_update(gsi, evt_ring_id, index); - - return gsi_channel_trans_complete(channel); } /** @@ -1528,21 +1515,18 @@ static struct gsi_trans *gsi_channel_update(struct gsi_channel *channel) * * Return: Transaction pointer, or null if none are available * - * This function returns the first entry on a channel's completed transaction - * list. If that list is empty, the hardware is consulted to determine - * whether any new transactions have completed. If so, they're moved to the - * completed list and the new first entry is returned. If there are no more - * completed transactions, a null pointer is returned. + * This function returns the first of a channel's completed transactions. + * If no transactions are in completed state, the hardware is consulted to + * determine whether any new transactions have completed. If so, they're + * moved to completed state and the first such transaction is returned. + * If there are no more completed transactions, a null pointer is returned. */ static struct gsi_trans *gsi_channel_poll_one(struct gsi_channel *channel) { struct gsi_trans *trans; - /* Get the first transaction from the completed list */ + /* Get the first completed transaction */ trans = gsi_channel_trans_complete(channel); - if (!trans) /* List is empty; see if there's more to do */ - trans = gsi_channel_update(channel); - if (trans) gsi_trans_move_polled(trans); @@ -1623,7 +1607,7 @@ static int gsi_channel_setup_one(struct gsi *gsi, u32 channel_id) gsi_channel_poll); else netif_napi_add(&gsi->dummy_dev, &channel->napi, - gsi_channel_poll, NAPI_POLL_WEIGHT); + gsi_channel_poll); return 0; diff --git a/drivers/net/ipa/gsi.h b/drivers/net/ipa/gsi.h index 23de5f67374c..49dcadba4e0b 100644 --- a/drivers/net/ipa/gsi.h +++ b/drivers/net/ipa/gsi.h @@ -1,7 +1,7 @@ /* SPDX-License-Identifier: GPL-2.0 */ /* Copyright (c) 2015-2018, The Linux Foundation. All rights reserved. - * Copyright (C) 2018-2021 Linaro Ltd. + * Copyright (C) 2018-2022 Linaro Ltd. */ #ifndef _GSI_H_ #define _GSI_H_ @@ -31,14 +31,6 @@ struct gsi_trans; struct gsi_channel_data; struct ipa_gsi_endpoint_data; -/* Execution environment IDs */ -enum gsi_ee_id { - GSI_EE_AP = 0x0, - GSI_EE_MODEM = 0x1, - GSI_EE_UC = 0x2, - GSI_EE_TZ = 0x3, -}; - struct gsi_ring { void *virt; /* ring array base address */ dma_addr_t addr; /* primarily low 32 bits used */ @@ -82,18 +74,18 @@ struct gsi_trans_pool { struct gsi_trans_info { atomic_t tre_avail; /* TREs available for allocation */ - struct gsi_trans_pool pool; /* transaction pool */ + + u16 free_id; /* first free trans in array */ + u16 allocated_id; /* first allocated transaction */ + u16 committed_id; /* first committed transaction */ + u16 pending_id; /* first pending transaction */ + u16 completed_id; /* first completed transaction */ + u16 polled_id; /* first polled transaction */ + struct gsi_trans *trans; /* transaction array */ struct gsi_trans **map; /* TRE -> transaction map */ struct gsi_trans_pool sg_pool; /* scatterlist pool */ struct gsi_trans_pool cmd_pool; /* command payload DMA pool */ - - spinlock_t spinlock; /* protects updates to the lists */ - struct list_head alloc; /* allocated, not committed */ - struct list_head committed; /* committed, awaiting doorbell */ - struct list_head pending; /* pending, awaiting completion */ - struct list_head complete; /* completed, awaiting poll */ - struct list_head polled; /* returned by gsi_channel_poll_one() */ }; /* Hardware values signifying the state of a channel */ diff --git a/drivers/net/ipa/gsi_private.h b/drivers/net/ipa/gsi_private.h index 0b2516fa21b5..c65f7c5cdc8d 100644 --- a/drivers/net/ipa/gsi_private.h +++ b/drivers/net/ipa/gsi_private.h @@ -1,7 +1,7 @@ /* SPDX-License-Identifier: GPL-2.0 */ /* Copyright (c) 2015-2018, The Linux Foundation. All rights reserved. - * Copyright (C) 2018-2020 Linaro Ltd. + * Copyright (C) 2018-2022 Linaro Ltd. */ #ifndef _GSI_PRIVATE_H_ #define _GSI_PRIVATE_H_ @@ -18,13 +18,13 @@ struct gsi_channel; /** * gsi_trans_move_complete() - Mark a GSI transaction completed - * @trans: Transaction to commit + * @trans: Transaction whose state is to be updated */ void gsi_trans_move_complete(struct gsi_trans *trans); /** * gsi_trans_move_polled() - Mark a transaction polled - * @trans: Transaction to update + * @trans: Transaction whose state is to be updated */ void gsi_trans_move_polled(struct gsi_trans *trans); @@ -94,6 +94,14 @@ void gsi_channel_trans_exit(struct gsi_channel *channel); */ void gsi_channel_doorbell(struct gsi_channel *channel); +/* gsi_channel_update() - Update knowledge of channel hardware state + * @channel: Channel to be updated + * + * Consult hardware, change the state of any newly-completed transactions + * on a channel. + */ +void gsi_channel_update(struct gsi_channel *channel); + /** * gsi_ring_virt() - Return virtual address for a ring entry * @ring: Ring whose address is to be translated diff --git a/drivers/net/ipa/gsi_reg.h b/drivers/net/ipa/gsi_reg.h index 5bd8b31656d3..3763359f208f 100644 --- a/drivers/net/ipa/gsi_reg.h +++ b/drivers/net/ipa/gsi_reg.h @@ -1,7 +1,7 @@ /* SPDX-License-Identifier: GPL-2.0 */ /* Copyright (c) 2015-2018, The Linux Foundation. All rights reserved. - * Copyright (C) 2018-2021 Linaro Ltd. + * Copyright (C) 2018-2022 Linaro Ltd. */ #ifndef _GSI_REG_H_ #define _GSI_REG_H_ @@ -55,14 +55,10 @@ /* The inter-EE IRQ registers are relative to gsi->virt_raw (IPA v3.5+) */ #define GSI_INTER_EE_SRC_CH_IRQ_MSK_OFFSET \ - GSI_INTER_EE_N_SRC_CH_IRQ_MSK_OFFSET(GSI_EE_AP) -#define GSI_INTER_EE_N_SRC_CH_IRQ_MSK_OFFSET(ee) \ - (0x0000c020 + 0x1000 * (ee)) + (0x0000c020 + 0x1000 * GSI_EE_AP) #define GSI_INTER_EE_SRC_EV_CH_IRQ_MSK_OFFSET \ - GSI_INTER_EE_N_SRC_EV_CH_IRQ_MSK_OFFSET(GSI_EE_AP) -#define GSI_INTER_EE_N_SRC_EV_CH_IRQ_MSK_OFFSET(ee) \ - (0x0000c024 + 0x1000 * (ee)) + (0x0000c024 + 0x1000 * GSI_EE_AP) /* All other register offsets are relative to gsi->virt */ @@ -81,9 +77,7 @@ enum gsi_channel_type { }; #define GSI_CH_C_CNTXT_0_OFFSET(ch) \ - GSI_EE_N_CH_C_CNTXT_0_OFFSET((ch), GSI_EE_AP) -#define GSI_EE_N_CH_C_CNTXT_0_OFFSET(ch, ee) \ - (0x0001c000 + 0x4000 * (ee) + 0x80 * (ch)) + (0x0001c000 + 0x4000 * GSI_EE_AP + 0x80 * (ch)) #define CHTYPE_PROTOCOL_FMASK GENMASK(2, 0) #define CHTYPE_DIR_FMASK GENMASK(3, 3) #define EE_FMASK GENMASK(7, 4) @@ -112,9 +106,7 @@ chtype_protocol_encoded(enum ipa_version version, enum gsi_channel_type type) } #define GSI_CH_C_CNTXT_1_OFFSET(ch) \ - GSI_EE_N_CH_C_CNTXT_1_OFFSET((ch), GSI_EE_AP) -#define GSI_EE_N_CH_C_CNTXT_1_OFFSET(ch, ee) \ - (0x0001c004 + 0x4000 * (ee) + 0x80 * (ch)) + (0x0001c004 + 0x4000 * GSI_EE_AP + 0x80 * (ch)) /* Encoded value for CH_C_CNTXT_1 register R_LENGTH field */ static inline u32 r_length_encoded(enum ipa_version version, u32 length) @@ -125,19 +117,13 @@ static inline u32 r_length_encoded(enum ipa_version version, u32 length) } #define GSI_CH_C_CNTXT_2_OFFSET(ch) \ - GSI_EE_N_CH_C_CNTXT_2_OFFSET((ch), GSI_EE_AP) -#define GSI_EE_N_CH_C_CNTXT_2_OFFSET(ch, ee) \ - (0x0001c008 + 0x4000 * (ee) + 0x80 * (ch)) + (0x0001c008 + 0x4000 * GSI_EE_AP + 0x80 * (ch)) #define GSI_CH_C_CNTXT_3_OFFSET(ch) \ - GSI_EE_N_CH_C_CNTXT_3_OFFSET((ch), GSI_EE_AP) -#define GSI_EE_N_CH_C_CNTXT_3_OFFSET(ch, ee) \ - (0x0001c00c + 0x4000 * (ee) + 0x80 * (ch)) + (0x0001c00c + 0x4000 * GSI_EE_AP + 0x80 * (ch)) #define GSI_CH_C_QOS_OFFSET(ch) \ - GSI_EE_N_CH_C_QOS_OFFSET((ch), GSI_EE_AP) -#define GSI_EE_N_CH_C_QOS_OFFSET(ch, ee) \ - (0x0001c05c + 0x4000 * (ee) + 0x80 * (ch)) + (0x0001c05c + 0x4000 * GSI_EE_AP + 0x80 * (ch)) #define WRR_WEIGHT_FMASK GENMASK(3, 0) #define MAX_PREFETCH_FMASK GENMASK(8, 8) #define USE_DB_ENG_FMASK GENMASK(9, 9) @@ -158,29 +144,19 @@ enum gsi_prefetch_mode { }; #define GSI_CH_C_SCRATCH_0_OFFSET(ch) \ - GSI_EE_N_CH_C_SCRATCH_0_OFFSET((ch), GSI_EE_AP) -#define GSI_EE_N_CH_C_SCRATCH_0_OFFSET(ch, ee) \ - (0x0001c060 + 0x4000 * (ee) + 0x80 * (ch)) + (0x0001c060 + 0x4000 * GSI_EE_AP + 0x80 * (ch)) #define GSI_CH_C_SCRATCH_1_OFFSET(ch) \ - GSI_EE_N_CH_C_SCRATCH_1_OFFSET((ch), GSI_EE_AP) -#define GSI_EE_N_CH_C_SCRATCH_1_OFFSET(ch, ee) \ - (0x0001c064 + 0x4000 * (ee) + 0x80 * (ch)) + (0x0001c064 + 0x4000 * GSI_EE_AP + 0x80 * (ch)) #define GSI_CH_C_SCRATCH_2_OFFSET(ch) \ - GSI_EE_N_CH_C_SCRATCH_2_OFFSET((ch), GSI_EE_AP) -#define GSI_EE_N_CH_C_SCRATCH_2_OFFSET(ch, ee) \ - (0x0001c068 + 0x4000 * (ee) + 0x80 * (ch)) + (0x0001c068 + 0x4000 * GSI_EE_AP + 0x80 * (ch)) #define GSI_CH_C_SCRATCH_3_OFFSET(ch) \ - GSI_EE_N_CH_C_SCRATCH_3_OFFSET((ch), GSI_EE_AP) -#define GSI_EE_N_CH_C_SCRATCH_3_OFFSET(ch, ee) \ - (0x0001c06c + 0x4000 * (ee) + 0x80 * (ch)) + (0x0001c06c + 0x4000 * GSI_EE_AP + 0x80 * (ch)) #define GSI_EV_CH_E_CNTXT_0_OFFSET(ev) \ - GSI_EE_N_EV_CH_E_CNTXT_0_OFFSET((ev), GSI_EE_AP) -#define GSI_EE_N_EV_CH_E_CNTXT_0_OFFSET(ev, ee) \ - (0x0001d000 + 0x4000 * (ee) + 0x80 * (ev)) + (0x0001d000 + 0x4000 * GSI_EE_AP + 0x80 * (ev)) /* enum gsi_channel_type defines EV_CHTYPE field values in EV_CH_E_CNTXT_0 */ #define EV_CHTYPE_FMASK GENMASK(3, 0) #define EV_EE_FMASK GENMASK(7, 4) @@ -190,9 +166,7 @@ enum gsi_prefetch_mode { #define EV_ELEMENT_SIZE_FMASK GENMASK(31, 24) #define GSI_EV_CH_E_CNTXT_1_OFFSET(ev) \ - GSI_EE_N_EV_CH_E_CNTXT_1_OFFSET((ev), GSI_EE_AP) -#define GSI_EE_N_EV_CH_E_CNTXT_1_OFFSET(ev, ee) \ - (0x0001d004 + 0x4000 * (ee) + 0x80 * (ev)) + (0x0001d004 + 0x4000 * GSI_EE_AP + 0x80 * (ev)) /* Encoded value for EV_CH_C_CNTXT_1 register EV_R_LENGTH field */ static inline u32 ev_r_length_encoded(enum ipa_version version, u32 length) { @@ -202,83 +176,53 @@ static inline u32 ev_r_length_encoded(enum ipa_version version, u32 length) } #define GSI_EV_CH_E_CNTXT_2_OFFSET(ev) \ - GSI_EE_N_EV_CH_E_CNTXT_2_OFFSET((ev), GSI_EE_AP) -#define GSI_EE_N_EV_CH_E_CNTXT_2_OFFSET(ev, ee) \ - (0x0001d008 + 0x4000 * (ee) + 0x80 * (ev)) + (0x0001d008 + 0x4000 * GSI_EE_AP + 0x80 * (ev)) #define GSI_EV_CH_E_CNTXT_3_OFFSET(ev) \ - GSI_EE_N_EV_CH_E_CNTXT_3_OFFSET((ev), GSI_EE_AP) -#define GSI_EE_N_EV_CH_E_CNTXT_3_OFFSET(ev, ee) \ - (0x0001d00c + 0x4000 * (ee) + 0x80 * (ev)) + (0x0001d00c + 0x4000 * GSI_EE_AP + 0x80 * (ev)) #define GSI_EV_CH_E_CNTXT_4_OFFSET(ev) \ - GSI_EE_N_EV_CH_E_CNTXT_4_OFFSET((ev), GSI_EE_AP) -#define GSI_EE_N_EV_CH_E_CNTXT_4_OFFSET(ev, ee) \ - (0x0001d010 + 0x4000 * (ee) + 0x80 * (ev)) + (0x0001d010 + 0x4000 * GSI_EE_AP + 0x80 * (ev)) #define GSI_EV_CH_E_CNTXT_8_OFFSET(ev) \ - GSI_EE_N_EV_CH_E_CNTXT_8_OFFSET((ev), GSI_EE_AP) -#define GSI_EE_N_EV_CH_E_CNTXT_8_OFFSET(ev, ee) \ - (0x0001d020 + 0x4000 * (ee) + 0x80 * (ev)) + (0x0001d020 + 0x4000 * GSI_EE_AP + 0x80 * (ev)) #define MODT_FMASK GENMASK(15, 0) #define MODC_FMASK GENMASK(23, 16) #define MOD_CNT_FMASK GENMASK(31, 24) #define GSI_EV_CH_E_CNTXT_9_OFFSET(ev) \ - GSI_EE_N_EV_CH_E_CNTXT_9_OFFSET((ev), GSI_EE_AP) -#define GSI_EE_N_EV_CH_E_CNTXT_9_OFFSET(ev, ee) \ - (0x0001d024 + 0x4000 * (ee) + 0x80 * (ev)) + (0x0001d024 + 0x4000 * GSI_EE_AP + 0x80 * (ev)) #define GSI_EV_CH_E_CNTXT_10_OFFSET(ev) \ - GSI_EE_N_EV_CH_E_CNTXT_10_OFFSET((ev), GSI_EE_AP) -#define GSI_EE_N_EV_CH_E_CNTXT_10_OFFSET(ev, ee) \ - (0x0001d028 + 0x4000 * (ee) + 0x80 * (ev)) + (0x0001d028 + 0x4000 * GSI_EE_AP + 0x80 * (ev)) #define GSI_EV_CH_E_CNTXT_11_OFFSET(ev) \ - GSI_EE_N_EV_CH_E_CNTXT_11_OFFSET((ev), GSI_EE_AP) -#define GSI_EE_N_EV_CH_E_CNTXT_11_OFFSET(ev, ee) \ - (0x0001d02c + 0x4000 * (ee) + 0x80 * (ev)) + (0x0001d02c + 0x4000 * GSI_EE_AP + 0x80 * (ev)) #define GSI_EV_CH_E_CNTXT_12_OFFSET(ev) \ - GSI_EE_N_EV_CH_E_CNTXT_12_OFFSET((ev), GSI_EE_AP) -#define GSI_EE_N_EV_CH_E_CNTXT_12_OFFSET(ev, ee) \ - (0x0001d030 + 0x4000 * (ee) + 0x80 * (ev)) + (0x0001d030 + 0x4000 * GSI_EE_AP + 0x80 * (ev)) #define GSI_EV_CH_E_CNTXT_13_OFFSET(ev) \ - GSI_EE_N_EV_CH_E_CNTXT_13_OFFSET((ev), GSI_EE_AP) -#define GSI_EE_N_EV_CH_E_CNTXT_13_OFFSET(ev, ee) \ - (0x0001d034 + 0x4000 * (ee) + 0x80 * (ev)) + (0x0001d034 + 0x4000 * GSI_EE_AP + 0x80 * (ev)) #define GSI_EV_CH_E_SCRATCH_0_OFFSET(ev) \ - GSI_EE_N_EV_CH_E_SCRATCH_0_OFFSET((ev), GSI_EE_AP) -#define GSI_EE_N_EV_CH_E_SCRATCH_0_OFFSET(ev, ee) \ - (0x0001d048 + 0x4000 * (ee) + 0x80 * (ev)) + (0x0001d048 + 0x4000 * GSI_EE_AP + 0x80 * (ev)) #define GSI_EV_CH_E_SCRATCH_1_OFFSET(ev) \ - GSI_EE_N_EV_CH_E_SCRATCH_1_OFFSET((ev), GSI_EE_AP) -#define GSI_EE_N_EV_CH_E_SCRATCH_1_OFFSET(ev, ee) \ - (0x0001d04c + 0x4000 * (ee) + 0x80 * (ev)) + (0x0001d04c + 0x4000 * GSI_EE_AP + 0x80 * (ev)) #define GSI_CH_C_DOORBELL_0_OFFSET(ch) \ - GSI_EE_N_CH_C_DOORBELL_0_OFFSET((ch), GSI_EE_AP) -#define GSI_EE_N_CH_C_DOORBELL_0_OFFSET(ch, ee) \ - (0x0001e000 + 0x4000 * (ee) + 0x08 * (ch)) + (0x0001e000 + 0x4000 * GSI_EE_AP + 0x08 * (ch)) #define GSI_EV_CH_E_DOORBELL_0_OFFSET(ev) \ - GSI_EE_N_EV_CH_E_DOORBELL_0_OFFSET((ev), GSI_EE_AP) -#define GSI_EE_N_EV_CH_E_DOORBELL_0_OFFSET(ev, ee) \ - (0x0001e100 + 0x4000 * (ee) + 0x08 * (ev)) + (0x0001e100 + 0x4000 * GSI_EE_AP + 0x08 * (ev)) #define GSI_GSI_STATUS_OFFSET \ - GSI_EE_N_GSI_STATUS_OFFSET(GSI_EE_AP) -#define GSI_EE_N_GSI_STATUS_OFFSET(ee) \ - (0x0001f000 + 0x4000 * (ee)) + (0x0001f000 + 0x4000 * GSI_EE_AP) #define ENABLED_FMASK GENMASK(0, 0) #define GSI_CH_CMD_OFFSET \ - GSI_EE_N_CH_CMD_OFFSET(GSI_EE_AP) -#define GSI_EE_N_CH_CMD_OFFSET(ee) \ - (0x0001f008 + 0x4000 * (ee)) + (0x0001f008 + 0x4000 * GSI_EE_AP) #define CH_CHID_FMASK GENMASK(7, 0) #define CH_OPCODE_FMASK GENMASK(31, 24) @@ -293,9 +237,7 @@ enum gsi_ch_cmd_opcode { }; #define GSI_EV_CH_CMD_OFFSET \ - GSI_EE_N_EV_CH_CMD_OFFSET(GSI_EE_AP) -#define GSI_EE_N_EV_CH_CMD_OFFSET(ee) \ - (0x0001f010 + 0x4000 * (ee)) + (0x0001f010 + 0x4000 * GSI_EE_AP) #define EV_CHID_FMASK GENMASK(7, 0) #define EV_OPCODE_FMASK GENMASK(31, 24) @@ -307,9 +249,7 @@ enum gsi_evt_cmd_opcode { }; #define GSI_GENERIC_CMD_OFFSET \ - GSI_EE_N_GENERIC_CMD_OFFSET(GSI_EE_AP) -#define GSI_EE_N_GENERIC_CMD_OFFSET(ee) \ - (0x0001f018 + 0x4000 * (ee)) + (0x0001f018 + 0x4000 * GSI_EE_AP) #define GENERIC_OPCODE_FMASK GENMASK(4, 0) #define GENERIC_CHID_FMASK GENMASK(9, 5) #define GENERIC_EE_FMASK GENMASK(13, 10) @@ -326,9 +266,7 @@ enum gsi_generic_cmd_opcode { /* The next register is present for IPA v3.5.1 and above */ #define GSI_GSI_HW_PARAM_2_OFFSET \ - GSI_EE_N_GSI_HW_PARAM_2_OFFSET(GSI_EE_AP) -#define GSI_EE_N_GSI_HW_PARAM_2_OFFSET(ee) \ - (0x0001f040 + 0x4000 * (ee)) + (0x0001f040 + 0x4000 * GSI_EE_AP) #define IRAM_SIZE_FMASK GENMASK(2, 0) #define NUM_CH_PER_EE_FMASK GENMASK(7, 3) #define NUM_EV_PER_EE_FMASK GENMASK(12, 8) @@ -357,13 +295,9 @@ enum gsi_iram_size { /* IRQ condition for each type is cleared by writing type-specific register */ #define GSI_CNTXT_TYPE_IRQ_OFFSET \ - GSI_EE_N_CNTXT_TYPE_IRQ_OFFSET(GSI_EE_AP) -#define GSI_EE_N_CNTXT_TYPE_IRQ_OFFSET(ee) \ - (0x0001f080 + 0x4000 * (ee)) + (0x0001f080 + 0x4000 * GSI_EE_AP) #define GSI_CNTXT_TYPE_IRQ_MSK_OFFSET \ - GSI_EE_N_CNTXT_TYPE_IRQ_MSK_OFFSET(GSI_EE_AP) -#define GSI_EE_N_CNTXT_TYPE_IRQ_MSK_OFFSET(ee) \ - (0x0001f088 + 0x4000 * (ee)) + (0x0001f088 + 0x4000 * GSI_EE_AP) /* Values here are bit positions in the TYPE_IRQ and TYPE_IRQ_MSK registers */ enum gsi_irq_type_id { @@ -377,62 +311,38 @@ enum gsi_irq_type_id { }; #define GSI_CNTXT_SRC_CH_IRQ_OFFSET \ - GSI_EE_N_CNTXT_SRC_CH_IRQ_OFFSET(GSI_EE_AP) -#define GSI_EE_N_CNTXT_SRC_CH_IRQ_OFFSET(ee) \ - (0x0001f090 + 0x4000 * (ee)) + (0x0001f090 + 0x4000 * GSI_EE_AP) #define GSI_CNTXT_SRC_EV_CH_IRQ_OFFSET \ - GSI_EE_N_CNTXT_SRC_EV_CH_IRQ_OFFSET(GSI_EE_AP) -#define GSI_EE_N_CNTXT_SRC_EV_CH_IRQ_OFFSET(ee) \ - (0x0001f094 + 0x4000 * (ee)) + (0x0001f094 + 0x4000 * GSI_EE_AP) #define GSI_CNTXT_SRC_CH_IRQ_MSK_OFFSET \ - GSI_EE_N_CNTXT_SRC_CH_IRQ_MSK_OFFSET(GSI_EE_AP) -#define GSI_EE_N_CNTXT_SRC_CH_IRQ_MSK_OFFSET(ee) \ - (0x0001f098 + 0x4000 * (ee)) + (0x0001f098 + 0x4000 * GSI_EE_AP) #define GSI_CNTXT_SRC_EV_CH_IRQ_MSK_OFFSET \ - GSI_EE_N_CNTXT_SRC_EV_CH_IRQ_MSK_OFFSET(GSI_EE_AP) -#define GSI_EE_N_CNTXT_SRC_EV_CH_IRQ_MSK_OFFSET(ee) \ - (0x0001f09c + 0x4000 * (ee)) + (0x0001f09c + 0x4000 * GSI_EE_AP) #define GSI_CNTXT_SRC_CH_IRQ_CLR_OFFSET \ - GSI_EE_N_CNTXT_SRC_CH_IRQ_CLR_OFFSET(GSI_EE_AP) -#define GSI_EE_N_CNTXT_SRC_CH_IRQ_CLR_OFFSET(ee) \ - (0x0001f0a0 + 0x4000 * (ee)) + (0x0001f0a0 + 0x4000 * GSI_EE_AP) #define GSI_CNTXT_SRC_EV_CH_IRQ_CLR_OFFSET \ - GSI_EE_N_CNTXT_SRC_EV_CH_IRQ_CLR_OFFSET(GSI_EE_AP) -#define GSI_EE_N_CNTXT_SRC_EV_CH_IRQ_CLR_OFFSET(ee) \ - (0x0001f0a4 + 0x4000 * (ee)) + (0x0001f0a4 + 0x4000 * GSI_EE_AP) #define GSI_CNTXT_SRC_IEOB_IRQ_OFFSET \ - GSI_EE_N_CNTXT_SRC_IEOB_IRQ_OFFSET(GSI_EE_AP) -#define GSI_EE_N_CNTXT_SRC_IEOB_IRQ_OFFSET(ee) \ - (0x0001f0b0 + 0x4000 * (ee)) + (0x0001f0b0 + 0x4000 * GSI_EE_AP) #define GSI_CNTXT_SRC_IEOB_IRQ_MSK_OFFSET \ - GSI_EE_N_CNTXT_SRC_IEOB_IRQ_MSK_OFFSET(GSI_EE_AP) -#define GSI_EE_N_CNTXT_SRC_IEOB_IRQ_MSK_OFFSET(ee) \ - (0x0001f0b8 + 0x4000 * (ee)) + (0x0001f0b8 + 0x4000 * GSI_EE_AP) #define GSI_CNTXT_SRC_IEOB_IRQ_CLR_OFFSET \ - GSI_EE_N_CNTXT_SRC_IEOB_IRQ_CLR_OFFSET(GSI_EE_AP) -#define GSI_EE_N_CNTXT_SRC_IEOB_IRQ_CLR_OFFSET(ee) \ - (0x0001f0c0 + 0x4000 * (ee)) + (0x0001f0c0 + 0x4000 * GSI_EE_AP) #define GSI_CNTXT_GLOB_IRQ_STTS_OFFSET \ - GSI_EE_N_CNTXT_GLOB_IRQ_STTS_OFFSET(GSI_EE_AP) -#define GSI_EE_N_CNTXT_GLOB_IRQ_STTS_OFFSET(ee) \ - (0x0001f100 + 0x4000 * (ee)) + (0x0001f100 + 0x4000 * GSI_EE_AP) #define GSI_CNTXT_GLOB_IRQ_EN_OFFSET \ - GSI_EE_N_CNTXT_GLOB_IRQ_EN_OFFSET(GSI_EE_AP) -#define GSI_EE_N_CNTXT_GLOB_IRQ_EN_OFFSET(ee) \ - (0x0001f108 + 0x4000 * (ee)) + (0x0001f108 + 0x4000 * GSI_EE_AP) #define GSI_CNTXT_GLOB_IRQ_CLR_OFFSET \ - GSI_EE_N_CNTXT_GLOB_IRQ_CLR_OFFSET(GSI_EE_AP) -#define GSI_EE_N_CNTXT_GLOB_IRQ_CLR_OFFSET(ee) \ - (0x0001f110 + 0x4000 * (ee)) + (0x0001f110 + 0x4000 * GSI_EE_AP) /* Values here are bit positions in the GLOB_IRQ_* registers */ enum gsi_global_irq_id { ERROR_INT = 0x0, @@ -442,17 +352,11 @@ enum gsi_global_irq_id { }; #define GSI_CNTXT_GSI_IRQ_STTS_OFFSET \ - GSI_EE_N_CNTXT_GSI_IRQ_STTS_OFFSET(GSI_EE_AP) -#define GSI_EE_N_CNTXT_GSI_IRQ_STTS_OFFSET(ee) \ - (0x0001f118 + 0x4000 * (ee)) + (0x0001f118 + 0x4000 * GSI_EE_AP) #define GSI_CNTXT_GSI_IRQ_EN_OFFSET \ - GSI_EE_N_CNTXT_GSI_IRQ_EN_OFFSET(GSI_EE_AP) -#define GSI_EE_N_CNTXT_GSI_IRQ_EN_OFFSET(ee) \ - (0x0001f120 + 0x4000 * (ee)) + (0x0001f120 + 0x4000 * GSI_EE_AP) #define GSI_CNTXT_GSI_IRQ_CLR_OFFSET \ - GSI_EE_N_CNTXT_GSI_IRQ_CLR_OFFSET(GSI_EE_AP) -#define GSI_EE_N_CNTXT_GSI_IRQ_CLR_OFFSET(ee) \ - (0x0001f128 + 0x4000 * (ee)) + (0x0001f128 + 0x4000 * GSI_EE_AP) /* Values here are bit positions in the (general) GSI_IRQ_* registers */ enum gsi_general_id { BREAK_POINT = 0x0, @@ -462,15 +366,11 @@ enum gsi_general_id { }; #define GSI_CNTXT_INTSET_OFFSET \ - GSI_EE_N_CNTXT_INTSET_OFFSET(GSI_EE_AP) -#define GSI_EE_N_CNTXT_INTSET_OFFSET(ee) \ - (0x0001f180 + 0x4000 * (ee)) + (0x0001f180 + 0x4000 * GSI_EE_AP) #define INTYPE_FMASK GENMASK(0, 0) #define GSI_ERROR_LOG_OFFSET \ - GSI_EE_N_ERROR_LOG_OFFSET(GSI_EE_AP) -#define GSI_EE_N_ERROR_LOG_OFFSET(ee) \ - (0x0001f200 + 0x4000 * (ee)) + (0x0001f200 + 0x4000 * GSI_EE_AP) /* Fields below are present for IPA v3.5.1 and above */ #define ERR_ARG3_FMASK GENMASK(3, 0) @@ -501,14 +401,10 @@ enum gsi_err_type { }; #define GSI_ERROR_LOG_CLR_OFFSET \ - GSI_EE_N_ERROR_LOG_CLR_OFFSET(GSI_EE_AP) -#define GSI_EE_N_ERROR_LOG_CLR_OFFSET(ee) \ - (0x0001f210 + 0x4000 * (ee)) + (0x0001f210 + 0x4000 * GSI_EE_AP) #define GSI_CNTXT_SCRATCH_0_OFFSET \ - GSI_EE_N_CNTXT_SCRATCH_0_OFFSET(GSI_EE_AP) -#define GSI_EE_N_CNTXT_SCRATCH_0_OFFSET(ee) \ - (0x0001f400 + 0x4000 * (ee)) + (0x0001f400 + 0x4000 * GSI_EE_AP) #define INTER_EE_RESULT_FMASK GENMASK(2, 0) #define GENERIC_EE_RESULT_FMASK GENMASK(7, 5) diff --git a/drivers/net/ipa/gsi_trans.c b/drivers/net/ipa/gsi_trans.c index 18e7e8c405be..26b7f683a3e1 100644 --- a/drivers/net/ipa/gsi_trans.c +++ b/drivers/net/ipa/gsi_trans.c @@ -1,7 +1,7 @@ // SPDX-License-Identifier: GPL-2.0 /* Copyright (c) 2012-2018, The Linux Foundation. All rights reserved. - * Copyright (C) 2019-2020 Linaro Ltd. + * Copyright (C) 2019-2022 Linaro Ltd. */ #include <linux/types.h> @@ -22,37 +22,36 @@ * DOC: GSI Transactions * * A GSI transaction abstracts the behavior of a GSI channel by representing - * everything about a related group of IPA commands in a single structure. - * (A "command" in this sense is either a data transfer or an IPA immediate + * everything about a related group of IPA operations in a single structure. + * (A "operation" in this sense is either a data transfer or an IPA immediate * command.) Most details of interaction with the GSI hardware are managed - * by the GSI transaction core, allowing users to simply describe commands + * by the GSI transaction core, allowing users to simply describe operations * to be performed. When a transaction has completed a callback function * (dependent on the type of endpoint associated with the channel) allows * cleanup of resources associated with the transaction. * - * To perform a command (or set of them), a user of the GSI transaction + * To perform an operation (or set of them), a user of the GSI transaction * interface allocates a transaction, indicating the number of TREs required - * (one per command). If sufficient TREs are available, they are reserved + * (one per operation). If sufficient TREs are available, they are reserved * for use in the transaction and the allocation succeeds. This way - * exhaustion of the available TREs in a channel ring is detected - * as early as possible. All resources required to complete a transaction - * are allocated at transaction allocation time. + * exhaustion of the available TREs in a channel ring is detected as early + * as possible. Any other resources that might be needed to complete a + * transaction are also allocated when the transaction is allocated. * - * Commands performed as part of a transaction are represented in an array - * of Linux scatterlist structures. This array is allocated with the - * transaction, and its entries are initialized using standard scatterlist - * functions (such as sg_set_buf() or skb_to_sgvec()). + * Operations performed as part of a transaction are represented in an array + * of Linux scatterlist structures, allocated with the transaction. These + * scatterlist structures are initialized by "adding" operations to the + * transaction. If a buffer in an operation must be mapped for DMA, this is + * done at the time it is added to the transaction. It is possible for a + * mapping error to occur when an operation is added. In this case the + * transaction should simply be freed; this correctly releases resources + * associated with the transaction. * - * Once a transaction's scatterlist structures have been initialized, the - * transaction is committed. The caller is responsible for mapping buffers - * for DMA if necessary, and this should be done *before* allocating - * the transaction. Between a successful allocation and commit of a - * transaction no errors should occur. - * - * Committing transfers ownership of the entire transaction to the GSI - * transaction core. The GSI transaction code formats the content of - * the scatterlist array into the channel ring buffer and informs the - * hardware that new TREs are available to process. + * Once all operations have been successfully added to a transaction, the + * transaction is committed. Committing transfers ownership of the entire + * transaction to the GSI transaction core. The GSI transaction code + * formats the content of the scatterlist array into the channel ring + * buffer and informs the hardware that new TREs are available to process. * * The last TRE in each transaction is marked to interrupt the AP when the * GSI hardware has completed it. Because transfers described by TREs are @@ -125,11 +124,10 @@ void gsi_trans_pool_exit(struct gsi_trans_pool *pool) memset(pool, 0, sizeof(*pool)); } -/* Allocate the requested number of (zeroed) entries from the pool */ -/* Home-grown DMA pool. This way we can preallocate and use the tre_count - * to guarantee allocations will succeed. Even though we specify max_alloc - * (and it can be more than one), we only allow allocation of a single - * element from a DMA pool. +/* Home-grown DMA pool. This way we can preallocate the pool, and guarantee + * allocations will succeed. The immediate commands in a transaction can + * require up to max_alloc elements from the pool. But we only allow + * allocation of a single element from a DMA pool at a time. */ int gsi_trans_pool_init_dma(struct device *dev, struct gsi_trans_pool *pool, size_t size, u32 count, u32 max_alloc) @@ -237,68 +235,63 @@ gsi_channel_trans_mapped(struct gsi_channel *channel, u32 index) /* Return the oldest completed transaction for a channel (or null) */ struct gsi_trans *gsi_channel_trans_complete(struct gsi_channel *channel) { - return list_first_entry_or_null(&channel->trans_info.complete, - struct gsi_trans, links); + struct gsi_trans_info *trans_info = &channel->trans_info; + u16 trans_id = trans_info->completed_id; + + if (trans_id == trans_info->pending_id) { + gsi_channel_update(channel); + if (trans_id == trans_info->pending_id) + return NULL; + } + + return &trans_info->trans[trans_id %= channel->tre_count]; } -/* Move a transaction from the allocated list to the committed list */ +/* Move a transaction from allocated to committed state */ static void gsi_trans_move_committed(struct gsi_trans *trans) { struct gsi_channel *channel = &trans->gsi->channel[trans->channel_id]; struct gsi_trans_info *trans_info = &channel->trans_info; - spin_lock_bh(&trans_info->spinlock); - - list_move_tail(&trans->links, &trans_info->committed); - - spin_unlock_bh(&trans_info->spinlock); + /* This allocated transaction is now committed */ + trans_info->allocated_id++; } -/* Move transactions from the committed list to the pending list */ +/* Move committed transactions to pending state */ static void gsi_trans_move_pending(struct gsi_trans *trans) { struct gsi_channel *channel = &trans->gsi->channel[trans->channel_id]; struct gsi_trans_info *trans_info = &channel->trans_info; - struct list_head list; - - spin_lock_bh(&trans_info->spinlock); + u16 trans_index = trans - &trans_info->trans[0]; + u16 delta; - /* Move this transaction and all predecessors to the pending list */ - list_cut_position(&list, &trans_info->committed, &trans->links); - list_splice_tail(&list, &trans_info->pending); - - spin_unlock_bh(&trans_info->spinlock); + /* These committed transactions are now pending */ + delta = trans_index - trans_info->committed_id + 1; + trans_info->committed_id += delta % channel->tre_count; } -/* Move a transaction and all of its predecessors from the pending list - * to the completed list. - */ +/* Move pending transactions to completed state */ void gsi_trans_move_complete(struct gsi_trans *trans) { struct gsi_channel *channel = &trans->gsi->channel[trans->channel_id]; struct gsi_trans_info *trans_info = &channel->trans_info; - struct list_head list; + u16 trans_index = trans - trans_info->trans; + u16 delta; - spin_lock_bh(&trans_info->spinlock); - - /* Move this transaction and all predecessors to completed list */ - list_cut_position(&list, &trans_info->pending, &trans->links); - list_splice_tail(&list, &trans_info->complete); - - spin_unlock_bh(&trans_info->spinlock); + /* These pending transactions are now completed */ + delta = trans_index - trans_info->pending_id + 1; + delta %= channel->tre_count; + trans_info->pending_id += delta; } -/* Move a transaction from the completed list to the polled list */ +/* Move a transaction from completed to polled state */ void gsi_trans_move_polled(struct gsi_trans *trans) { struct gsi_channel *channel = &trans->gsi->channel[trans->channel_id]; struct gsi_trans_info *trans_info = &channel->trans_info; - spin_lock_bh(&trans_info->spinlock); - - list_move_tail(&trans->links, &trans_info->polled); - - spin_unlock_bh(&trans_info->spinlock); + /* This completed transaction is now polled */ + trans_info->completed_id++; } /* Reserve some number of TREs on a channel. Returns true if successful */ @@ -343,20 +336,22 @@ struct gsi_trans *gsi_channel_trans_alloc(struct gsi *gsi, u32 channel_id, struct gsi_channel *channel = &gsi->channel[channel_id]; struct gsi_trans_info *trans_info; struct gsi_trans *trans; + u16 trans_index; if (WARN_ON(tre_count > channel->trans_tre_max)) return NULL; trans_info = &channel->trans_info; - /* We reserve the TREs now, but consume them at commit time. - * If there aren't enough available, we're done. - */ + /* If we can't reserve the TREs for the transaction, we're done */ if (!gsi_trans_tre_reserve(trans_info, tre_count)) return NULL; - /* Allocate and initialize non-zero fields in the transaction */ - trans = gsi_trans_pool_alloc(&trans_info->pool, 1); + trans_index = trans_info->free_id % channel->tre_count; + trans = &trans_info->trans[trans_index]; + memset(trans, 0, sizeof(*trans)); + + /* Initialize non-zero fields in the transaction */ trans->gsi = gsi; trans->channel_id = channel_id; trans->rsvd_count = tre_count; @@ -367,45 +362,37 @@ struct gsi_trans *gsi_channel_trans_alloc(struct gsi *gsi, u32 channel_id, sg_init_marker(trans->sgl, tre_count); trans->direction = direction; - - spin_lock_bh(&trans_info->spinlock); - - list_add_tail(&trans->links, &trans_info->alloc); - - spin_unlock_bh(&trans_info->spinlock); - refcount_set(&trans->refcount, 1); + /* This free transaction is now allocated */ + trans_info->free_id++; + return trans; } /* Free a previously-allocated transaction */ void gsi_trans_free(struct gsi_trans *trans) { - refcount_t *refcount = &trans->refcount; struct gsi_trans_info *trans_info; - bool last; - /* We must hold the lock to release the last reference */ - if (refcount_dec_not_one(refcount)) + if (!refcount_dec_and_test(&trans->refcount)) return; + /* Unused transactions are allocated but never committed, pending, + * completed, or polled. + */ trans_info = &trans->gsi->channel[trans->channel_id].trans_info; - - spin_lock_bh(&trans_info->spinlock); - - /* Reference might have been added before we got the lock */ - last = refcount_dec_and_test(refcount); - if (last) - list_del(&trans->links); - - spin_unlock_bh(&trans_info->spinlock); - - if (!last) - return; - - if (trans->used_count) + if (!trans->used_count) { + trans_info->allocated_id++; + trans_info->committed_id++; + trans_info->pending_id++; + trans_info->completed_id++; + } else { ipa_gsi_trans_release(trans); + } + + /* This transaction is now free */ + trans_info->polled_id++; /* Releasing the reserved TREs implicitly frees the sgl[] and * (if present) info[] arrays, plus the transaction itself. @@ -548,8 +535,8 @@ static void gsi_trans_tre_fill(struct gsi_tre *dest_tre, dma_addr_t addr, * * Formats channel ring TRE entries based on the content of the scatterlist. * Maps a transaction pointer to the last ring entry used for the transaction, - * so it can be recovered when it completes. Moves the transaction to the - * pending list. Finally, updates the channel ring pointer and optionally + * so it can be recovered when it completes. Moves the transaction to + * pending state. Finally, updates the channel ring pointer and optionally * rings the doorbell. */ static void __gsi_trans_commit(struct gsi_trans *trans, bool ring_db) @@ -654,23 +641,27 @@ void gsi_trans_complete(struct gsi_trans *trans) void gsi_channel_trans_cancel_pending(struct gsi_channel *channel) { struct gsi_trans_info *trans_info = &channel->trans_info; - struct gsi_trans *trans; - bool cancelled; + u16 trans_id = trans_info->pending_id; /* channel->gsi->mutex is held by caller */ - spin_lock_bh(&trans_info->spinlock); - cancelled = !list_empty(&trans_info->pending); - list_for_each_entry(trans, &trans_info->pending, links) - trans->cancelled = true; + /* If there are no pending transactions, we're done */ + if (trans_id == trans_info->committed_id) + return; - list_splice_tail_init(&trans_info->pending, &trans_info->complete); + /* Mark all pending transactions cancelled */ + do { + struct gsi_trans *trans; + + trans = &trans_info->trans[trans_id % channel->tre_count]; + trans->cancelled = true; + } while (++trans_id != trans_info->committed_id); - spin_unlock_bh(&trans_info->spinlock); + /* All pending transactions are now completed */ + trans_info->pending_id = trans_info->committed_id; /* Schedule NAPI polling to complete the cancelled transactions */ - if (cancelled) - napi_schedule(&channel->napi); + napi_schedule(&channel->napi); } /* Issue a command to read a single byte from a channel */ @@ -736,10 +727,16 @@ int gsi_channel_trans_init(struct gsi *gsi, u32 channel_id) * modulo that number to determine the next one that's free. * Transactions are allocated one at a time. */ - ret = gsi_trans_pool_init(&trans_info->pool, sizeof(struct gsi_trans), - tre_max, 1); - if (ret) + trans_info->trans = kcalloc(tre_count, sizeof(*trans_info->trans), + GFP_KERNEL); + if (!trans_info->trans) return -ENOMEM; + trans_info->free_id = 0; /* all modulo channel->tre_count */ + trans_info->allocated_id = 0; + trans_info->committed_id = 0; + trans_info->pending_id = 0; + trans_info->completed_id = 0; + trans_info->polled_id = 0; /* A completion event contains a pointer to the TRE that caused * the event (which will be the last one used by the transaction). @@ -765,19 +762,13 @@ int gsi_channel_trans_init(struct gsi *gsi, u32 channel_id) if (ret) goto err_map_free; - spin_lock_init(&trans_info->spinlock); - INIT_LIST_HEAD(&trans_info->alloc); - INIT_LIST_HEAD(&trans_info->committed); - INIT_LIST_HEAD(&trans_info->pending); - INIT_LIST_HEAD(&trans_info->complete); - INIT_LIST_HEAD(&trans_info->polled); return 0; err_map_free: kfree(trans_info->map); err_trans_free: - gsi_trans_pool_exit(&trans_info->pool); + kfree(trans_info->trans); dev_err(gsi->dev, "error %d initializing channel %u transactions\n", ret, channel_id); @@ -791,6 +782,6 @@ void gsi_channel_trans_exit(struct gsi_channel *channel) struct gsi_trans_info *trans_info = &channel->trans_info; gsi_trans_pool_exit(&trans_info->sg_pool); - gsi_trans_pool_exit(&trans_info->pool); + kfree(trans_info->trans); kfree(trans_info->map); } diff --git a/drivers/net/ipa/gsi_trans.h b/drivers/net/ipa/gsi_trans.h index 7084507830c2..30c1c2dc77c6 100644 --- a/drivers/net/ipa/gsi_trans.h +++ b/drivers/net/ipa/gsi_trans.h @@ -1,7 +1,7 @@ /* SPDX-License-Identifier: GPL-2.0 */ /* Copyright (c) 2012-2018, The Linux Foundation. All rights reserved. - * Copyright (C) 2019-2020 Linaro Ltd. + * Copyright (C) 2019-2022 Linaro Ltd. */ #ifndef _GSI_TRANS_H_ #define _GSI_TRANS_H_ @@ -29,7 +29,6 @@ struct gsi_trans_pool; * struct gsi_trans - a GSI transaction * * Most fields in this structure for internal use by the transaction core code: - * @links: Links for channel transaction lists by state * @gsi: GSI pointer * @channel_id: Channel number transaction is associated with * @cancelled: If set by the core code, transaction was cancelled @@ -50,8 +49,6 @@ struct gsi_trans_pool; * received. */ struct gsi_trans { - struct list_head links; /* gsi_channel lists */ - struct gsi *gsi; u8 channel_id; @@ -77,7 +74,7 @@ struct gsi_trans { /** * gsi_trans_pool_init() - Initialize a pool of structures for transactions - * @pool: GSI transaction poll pointer + * @pool: GSI transaction pool pointer * @size: Size of elements in the pool * @count: Minimum number of elements in the pool * @max_alloc: Maximum number of elements allocated at a time from pool diff --git a/drivers/net/ipa/ipa.h b/drivers/net/ipa/ipa.h index 4fc3c72359f5..09ead433ec38 100644 --- a/drivers/net/ipa/ipa.h +++ b/drivers/net/ipa/ipa.h @@ -1,7 +1,7 @@ /* SPDX-License-Identifier: GPL-2.0 */ /* Copyright (c) 2012-2018, The Linux Foundation. All rights reserved. - * Copyright (C) 2018-2020 Linaro Ltd. + * Copyright (C) 2018-2022 Linaro Ltd. */ #ifndef _IPA_H_ #define _IPA_H_ @@ -44,6 +44,7 @@ struct ipa_interrupt; * @uc_loaded: true after microcontroller has reported it's ready * @reg_addr: DMA address used for IPA register access * @reg_virt: Virtual address used for IPA register access + * @regs: IPA register definitions * @mem_addr: DMA address of IPA-local memory space * @mem_virt: Virtual address of IPA-local memory space * @mem_offset: Offset from @mem_virt used for access to IPA memory @@ -90,6 +91,7 @@ struct ipa { dma_addr_t reg_addr; void __iomem *reg_virt; + const struct ipa_regs *regs; dma_addr_t mem_addr; void *mem_virt; diff --git a/drivers/net/ipa/ipa_cmd.c b/drivers/net/ipa/ipa_cmd.c index 6dea40259b60..26c3db9f52b1 100644 --- a/drivers/net/ipa/ipa_cmd.c +++ b/drivers/net/ipa/ipa_cmd.c @@ -1,7 +1,7 @@ // SPDX-License-Identifier: GPL-2.0 /* Copyright (c) 2012-2018, The Linux Foundation. All rights reserved. - * Copyright (C) 2019-2021 Linaro Ltd. + * Copyright (C) 2019-2022 Linaro Ltd. */ #include <linux/types.h> @@ -32,7 +32,7 @@ * immediate command's opcode. The payload for a command resides in AP * memory and is described by a single scatterlist entry in its transaction. * Commands do not require a transaction completion callback, and are - * (currently) always issued using gsi_trans_commit_wait(). + * always issued using gsi_trans_commit_wait(). */ /* Some commands can wait until indicated pipeline stages are clear */ @@ -305,6 +305,7 @@ static bool ipa_cmd_register_write_offset_valid(struct ipa *ipa, /* Check whether offsets passed to register_write are valid */ static bool ipa_cmd_register_write_valid(struct ipa *ipa) { + const struct ipa_reg *reg; const char *name; u32 offset; @@ -312,7 +313,8 @@ static bool ipa_cmd_register_write_valid(struct ipa *ipa) * offset will fit in a register write IPA immediate command. */ if (ipa_table_hash_support(ipa)) { - offset = ipa_reg_filt_rout_hash_flush_offset(ipa->version); + reg = ipa_reg(ipa, FILT_ROUT_HASH_FLUSH); + offset = ipa_reg_offset(reg); name = "filter/route hash flush"; if (!ipa_cmd_register_write_offset_valid(ipa, name, offset)) return false; @@ -325,7 +327,8 @@ static bool ipa_cmd_register_write_valid(struct ipa *ipa) * worst case (highest endpoint number) offset of that endpoint * fits in the register write command field(s) that must hold it. */ - offset = IPA_REG_ENDP_STATUS_N_OFFSET(IPA_ENDPOINT_COUNT - 1); + reg = ipa_reg(ipa, ENDP_STATUS); + offset = ipa_reg_n_offset(reg, IPA_ENDPOINT_COUNT - 1); name = "maximal endpoint status"; if (!ipa_cmd_register_write_offset_valid(ipa, name, offset)) return false; diff --git a/drivers/net/ipa/ipa_cmd.h b/drivers/net/ipa/ipa_cmd.h index 9215ddad1010..8e4243c1f0bb 100644 --- a/drivers/net/ipa/ipa_cmd.h +++ b/drivers/net/ipa/ipa_cmd.h @@ -1,7 +1,7 @@ /* SPDX-License-Identifier: GPL-2.0 */ /* Copyright (c) 2012-2018, The Linux Foundation. All rights reserved. - * Copyright (C) 2019-2020 Linaro Ltd. + * Copyright (C) 2019-2022 Linaro Ltd. */ #ifndef _IPA_CMD_H_ #define _IPA_CMD_H_ diff --git a/drivers/net/ipa/ipa_data.h b/drivers/net/ipa/ipa_data.h index e15eb3cd3e33..e5a6ce75c7dd 100644 --- a/drivers/net/ipa/ipa_data.h +++ b/drivers/net/ipa/ipa_data.h @@ -1,7 +1,7 @@ /* SPDX-License-Identifier: GPL-2.0 */ /* Copyright (c) 2012-2018, The Linux Foundation. All rights reserved. - * Copyright (C) 2019-2021 Linaro Ltd. + * Copyright (C) 2019-2022 Linaro Ltd. */ #ifndef _IPA_DATA_H_ #define _IPA_DATA_H_ @@ -31,7 +31,7 @@ * communication path between the IPA and a particular execution environment * (EE), such as the AP or Modem. Each EE has a set of channels associated * with it, and each channel has an ID unique for that EE. For the most part - * the only GSI channels of concern to this driver belong to the AP + * the only GSI channels of concern to this driver belong to the AP. * * An endpoint is an IPA construct representing a single channel anywhere * in the system. An IPA endpoint ID maps directly to an (EE, channel_id) diff --git a/drivers/net/ipa/ipa_endpoint.c b/drivers/net/ipa/ipa_endpoint.c index 66d2bfdf9e42..093e11ec7c2d 100644 --- a/drivers/net/ipa/ipa_endpoint.c +++ b/drivers/net/ipa/ipa_endpoint.c @@ -1,7 +1,7 @@ // SPDX-License-Identifier: GPL-2.0 /* Copyright (c) 2012-2018, The Linux Foundation. All rights reserved. - * Copyright (C) 2019-2021 Linaro Ltd. + * Copyright (C) 2019-2022 Linaro Ltd. */ #include <linux/types.h> @@ -23,8 +23,6 @@ #include "ipa_gsi.h" #include "ipa_power.h" -#define atomic_dec_not_zero(v) atomic_add_unless((v), -1, 0) - /* Hardware is told about receive buffers once a "batch" has been queued */ #define IPA_REPLENISH_BATCH 16 /* Must be non-zero */ @@ -72,14 +70,6 @@ struct ipa_status { #define IPA_STATUS_FLAGS1_RT_RULE_ID_FMASK GENMASK(31, 22) #define IPA_STATUS_FLAGS2_TAG_FMASK GENMASK_ULL(63, 16) -static u32 aggr_byte_limit_max(enum ipa_version version) -{ - if (version < IPA_VERSION_4_5) - return field_max(aggr_byte_limit_fmask(true)); - - return field_max(aggr_byte_limit_fmask(false)); -} - /* Compute the aggregation size value to use for a given buffer size */ static u32 ipa_aggr_size_kb(u32 rx_buffer_size, bool aggr_hard_limit) { @@ -111,6 +101,7 @@ static bool ipa_endpoint_data_valid_one(struct ipa *ipa, u32 count, if (!data->toward_ipa) { const struct ipa_endpoint_rx *rx_config; + const struct ipa_reg *reg; u32 buffer_size; u32 aggr_size; u32 limit; @@ -171,7 +162,9 @@ static bool ipa_endpoint_data_valid_one(struct ipa *ipa, u32 count, */ aggr_size = ipa_aggr_size_kb(buffer_size - NET_SKB_PAD, rx_config->aggr_hard_limit); - limit = aggr_byte_limit_max(ipa->version); + reg = ipa_reg(ipa, ENDP_INIT_AGGR); + + limit = ipa_reg_field_max(reg, BYTE_LIMIT); if (aggr_size > limit) { dev_err(dev, "aggregated size too large for RX endpoint %u (%u KB > %u KB)\n", data->endpoint_id, aggr_size, limit); @@ -182,6 +175,15 @@ static bool ipa_endpoint_data_valid_one(struct ipa *ipa, u32 count, return true; /* Nothing more to check for RX */ } + /* Starting with IPA v4.5 sequencer replication is obsolete */ + if (ipa->version >= IPA_VERSION_4_5) { + if (data->endpoint.config.tx.seq_rep_type) { + dev_err(dev, "no-zero seq_rep_type TX endpoint %u\n", + data->endpoint_id); + return false; + } + } + if (data->endpoint.config.status_enable) { other_name = data->endpoint.config.tx.status_endpoint; if (other_name >= count) { @@ -299,8 +301,10 @@ static struct gsi_trans *ipa_endpoint_trans_alloc(struct ipa_endpoint *endpoint, static bool ipa_endpoint_init_ctrl(struct ipa_endpoint *endpoint, bool suspend_delay) { - u32 offset = IPA_REG_ENDP_INIT_CTRL_N_OFFSET(endpoint->endpoint_id); struct ipa *ipa = endpoint->ipa; + const struct ipa_reg *reg; + u32 field_id; + u32 offset; bool state; u32 mask; u32 val; @@ -310,9 +314,13 @@ ipa_endpoint_init_ctrl(struct ipa_endpoint *endpoint, bool suspend_delay) else WARN_ON(ipa->version >= IPA_VERSION_4_0); - mask = endpoint->toward_ipa ? ENDP_DELAY_FMASK : ENDP_SUSPEND_FMASK; - + reg = ipa_reg(ipa, ENDP_INIT_CTRL); + offset = ipa_reg_n_offset(reg, endpoint->endpoint_id); val = ioread32(ipa->reg_virt + offset); + + field_id = endpoint->toward_ipa ? ENDP_DELAY : ENDP_SUSPEND; + mask = ipa_reg_bit(reg, field_id); + state = !!(val & mask); /* Don't bother if it's already in the requested state */ @@ -339,13 +347,13 @@ static bool ipa_endpoint_aggr_active(struct ipa_endpoint *endpoint) { u32 mask = BIT(endpoint->endpoint_id); struct ipa *ipa = endpoint->ipa; - u32 offset; + const struct ipa_reg *reg; u32 val; WARN_ON(!(mask & ipa->available)); - offset = ipa_reg_state_aggr_active_offset(ipa->version); - val = ioread32(ipa->reg_virt + offset); + reg = ipa_reg(ipa, STATE_AGGR_ACTIVE); + val = ioread32(ipa->reg_virt + ipa_reg_offset(reg)); return !!(val & mask); } @@ -354,10 +362,12 @@ static void ipa_endpoint_force_close(struct ipa_endpoint *endpoint) { u32 mask = BIT(endpoint->endpoint_id); struct ipa *ipa = endpoint->ipa; + const struct ipa_reg *reg; WARN_ON(!(mask & ipa->available)); - iowrite32(mask, ipa->reg_virt + IPA_REG_AGGR_FORCE_CLOSE_OFFSET); + reg = ipa_reg(ipa, AGGR_FORCE_CLOSE); + iowrite32(mask, ipa->reg_virt + ipa_reg_offset(reg)); } /** @@ -456,6 +466,7 @@ int ipa_endpoint_modem_exception_reset_all(struct ipa *ipa) while (initialized) { u32 endpoint_id = __ffs(initialized); struct ipa_endpoint *endpoint; + const struct ipa_reg *reg; u32 offset; initialized ^= BIT(endpoint_id); @@ -465,7 +476,8 @@ int ipa_endpoint_modem_exception_reset_all(struct ipa *ipa) if (!(endpoint->ee_id == GSI_EE_MODEM && endpoint->toward_ipa)) continue; - offset = IPA_REG_ENDP_STATUS_N_OFFSET(endpoint_id); + reg = ipa_reg(ipa, ENDP_STATUS); + offset = ipa_reg_n_offset(reg, endpoint_id); /* Value written is 0, and all bits are updated. That * means status is disabled on the endpoint, and as a @@ -485,22 +497,23 @@ int ipa_endpoint_modem_exception_reset_all(struct ipa *ipa) static void ipa_endpoint_init_cfg(struct ipa_endpoint *endpoint) { - u32 offset = IPA_REG_ENDP_INIT_CFG_N_OFFSET(endpoint->endpoint_id); + u32 endpoint_id = endpoint->endpoint_id; + struct ipa *ipa = endpoint->ipa; enum ipa_cs_offload_en enabled; + const struct ipa_reg *reg; u32 val = 0; + reg = ipa_reg(ipa, ENDP_INIT_CFG); /* FRAG_OFFLOAD_EN is 0 */ if (endpoint->config.checksum) { - enum ipa_version version = endpoint->ipa->version; + enum ipa_version version = ipa->version; if (endpoint->toward_ipa) { - u32 checksum_offset; + u32 off; /* Checksum header offset is in 4-byte units */ - checksum_offset = sizeof(struct rmnet_map_header); - checksum_offset /= sizeof(u32); - val |= u32_encode_bits(checksum_offset, - CS_METADATA_HDR_OFFSET_FMASK); + off = sizeof(struct rmnet_map_header) / sizeof(u32); + val |= ipa_reg_encode(reg, CS_METADATA_HDR_OFFSET, off); enabled = version < IPA_VERSION_4_5 ? IPA_CS_OFFLOAD_UL @@ -513,24 +526,26 @@ static void ipa_endpoint_init_cfg(struct ipa_endpoint *endpoint) } else { enabled = IPA_CS_OFFLOAD_NONE; } - val |= u32_encode_bits(enabled, CS_OFFLOAD_EN_FMASK); + val |= ipa_reg_encode(reg, CS_OFFLOAD_EN, enabled); /* CS_GEN_QMB_MASTER_SEL is 0 */ - iowrite32(val, endpoint->ipa->reg_virt + offset); + iowrite32(val, ipa->reg_virt + ipa_reg_n_offset(reg, endpoint_id)); } static void ipa_endpoint_init_nat(struct ipa_endpoint *endpoint) { - u32 offset; + u32 endpoint_id = endpoint->endpoint_id; + struct ipa *ipa = endpoint->ipa; + const struct ipa_reg *reg; u32 val; if (!endpoint->toward_ipa) return; - offset = IPA_REG_ENDP_INIT_NAT_N_OFFSET(endpoint->endpoint_id); - val = u32_encode_bits(IPA_NAT_BYPASS, NAT_EN_FMASK); + reg = ipa_reg(ipa, ENDP_INIT_NAT); + val = ipa_reg_encode(reg, NAT_EN, IPA_NAT_BYPASS); - iowrite32(val, endpoint->ipa->reg_virt + offset); + iowrite32(val, ipa->reg_virt + ipa_reg_n_offset(reg, endpoint_id)); } static u32 @@ -554,6 +569,50 @@ ipa_qmap_header_size(enum ipa_version version, struct ipa_endpoint *endpoint) return header_size; } +/* Encoded value for ENDP_INIT_HDR register HDR_LEN* field(s) */ +static u32 ipa_header_size_encode(enum ipa_version version, + const struct ipa_reg *reg, u32 header_size) +{ + u32 field_max = ipa_reg_field_max(reg, HDR_LEN); + u32 val; + + /* We know field_max can be used as a mask (2^n - 1) */ + val = ipa_reg_encode(reg, HDR_LEN, header_size & field_max); + if (version < IPA_VERSION_4_5) { + WARN_ON(header_size > field_max); + return val; + } + + /* IPA v4.5 adds a few more most-significant bits */ + header_size >>= hweight32(field_max); + WARN_ON(header_size > ipa_reg_field_max(reg, HDR_LEN_MSB)); + val |= ipa_reg_encode(reg, HDR_LEN_MSB, header_size); + + return val; +} + +/* Encoded value for ENDP_INIT_HDR register OFST_METADATA* field(s) */ +static u32 ipa_metadata_offset_encode(enum ipa_version version, + const struct ipa_reg *reg, u32 offset) +{ + u32 field_max = ipa_reg_field_max(reg, HDR_OFST_METADATA); + u32 val; + + /* We know field_max can be used as a mask (2^n - 1) */ + val = ipa_reg_encode(reg, HDR_OFST_METADATA, offset); + if (version < IPA_VERSION_4_5) { + WARN_ON(offset > field_max); + return val; + } + + /* IPA v4.5 adds a few more most-significant bits */ + offset >>= hweight32(field_max); + WARN_ON(offset > ipa_reg_field_max(reg, HDR_OFST_METADATA_MSB)); + val |= ipa_reg_encode(reg, HDR_OFST_METADATA_MSB, offset); + + return val; +} + /** * ipa_endpoint_init_hdr() - Initialize HDR endpoint configuration register * @endpoint: Endpoint pointer @@ -577,36 +636,38 @@ ipa_qmap_header_size(enum ipa_version version, struct ipa_endpoint *endpoint) */ static void ipa_endpoint_init_hdr(struct ipa_endpoint *endpoint) { - u32 offset = IPA_REG_ENDP_INIT_HDR_N_OFFSET(endpoint->endpoint_id); + u32 endpoint_id = endpoint->endpoint_id; struct ipa *ipa = endpoint->ipa; + const struct ipa_reg *reg; u32 val = 0; + reg = ipa_reg(ipa, ENDP_INIT_HDR); if (endpoint->config.qmap) { enum ipa_version version = ipa->version; size_t header_size; header_size = ipa_qmap_header_size(version, endpoint); - val = ipa_header_size_encoded(version, header_size); + val = ipa_header_size_encode(version, reg, header_size); /* Define how to fill fields in a received QMAP header */ if (!endpoint->toward_ipa) { - u32 offset; /* Field offset within header */ + u32 off; /* Field offset within header */ /* Where IPA will write the metadata value */ - offset = offsetof(struct rmnet_map_header, mux_id); - val |= ipa_metadata_offset_encoded(version, offset); + off = offsetof(struct rmnet_map_header, mux_id); + val |= ipa_metadata_offset_encode(version, reg, off); /* Where IPA will write the length */ - offset = offsetof(struct rmnet_map_header, pkt_len); + off = offsetof(struct rmnet_map_header, pkt_len); /* Upper bits are stored in HDR_EXT with IPA v4.5 */ if (version >= IPA_VERSION_4_5) - offset &= field_mask(HDR_OFST_PKT_SIZE_FMASK); + off &= ipa_reg_field_max(reg, HDR_OFST_PKT_SIZE); - val |= HDR_OFST_PKT_SIZE_VALID_FMASK; - val |= u32_encode_bits(offset, HDR_OFST_PKT_SIZE_FMASK); + val |= ipa_reg_bit(reg, HDR_OFST_PKT_SIZE_VALID); + val |= ipa_reg_encode(reg, HDR_OFST_PKT_SIZE, off); } /* For QMAP TX, metadata offset is 0 (modem assumes this) */ - val |= HDR_OFST_METADATA_VALID_FMASK; + val |= ipa_reg_bit(reg, HDR_OFST_METADATA_VALID); /* HDR_ADDITIONAL_CONST_LEN is 0; (RX only) */ /* HDR_A5_MUX is 0 */ @@ -614,19 +675,21 @@ static void ipa_endpoint_init_hdr(struct ipa_endpoint *endpoint) /* HDR_METADATA_REG_VALID is 0 (TX only, version < v4.5) */ } - iowrite32(val, ipa->reg_virt + offset); + iowrite32(val, ipa->reg_virt + ipa_reg_n_offset(reg, endpoint_id)); } static void ipa_endpoint_init_hdr_ext(struct ipa_endpoint *endpoint) { - u32 offset = IPA_REG_ENDP_INIT_HDR_EXT_N_OFFSET(endpoint->endpoint_id); u32 pad_align = endpoint->config.rx.pad_align; + u32 endpoint_id = endpoint->endpoint_id; struct ipa *ipa = endpoint->ipa; + const struct ipa_reg *reg; u32 val = 0; + reg = ipa_reg(ipa, ENDP_INIT_HDR_EXT); if (endpoint->config.qmap) { /* We have a header, so we must specify its endianness */ - val |= HDR_ENDIANNESS_FMASK; /* big endian */ + val |= ipa_reg_bit(reg, HDR_ENDIANNESS); /* big endian */ /* A QMAP header contains a 6 bit pad field at offset 0. * The RMNet driver assumes this field is meaningful in @@ -636,16 +699,16 @@ static void ipa_endpoint_init_hdr_ext(struct ipa_endpoint *endpoint) * (although 0) should be ignored. */ if (!endpoint->toward_ipa) { - val |= HDR_TOTAL_LEN_OR_PAD_VALID_FMASK; + val |= ipa_reg_bit(reg, HDR_TOTAL_LEN_OR_PAD_VALID); /* HDR_TOTAL_LEN_OR_PAD is 0 (pad, not total_len) */ - val |= HDR_PAYLOAD_LEN_INC_PADDING_FMASK; + val |= ipa_reg_bit(reg, HDR_PAYLOAD_LEN_INC_PADDING); /* HDR_TOTAL_LEN_OR_PAD_OFFSET is 0 */ } } /* HDR_PAYLOAD_LEN_INC_PADDING is 0 */ if (!endpoint->toward_ipa) - val |= u32_encode_bits(pad_align, HDR_PAD_TO_ALIGNMENT_FMASK); + val |= ipa_reg_encode(reg, HDR_PAD_TO_ALIGNMENT, pad_align); /* IPA v4.5 adds some most-significant bits to a few fields, * two of which are defined in the HDR (not HDR_EXT) register. @@ -653,191 +716,170 @@ static void ipa_endpoint_init_hdr_ext(struct ipa_endpoint *endpoint) if (ipa->version >= IPA_VERSION_4_5) { /* HDR_TOTAL_LEN_OR_PAD_OFFSET is 0, so MSB is 0 */ if (endpoint->config.qmap && !endpoint->toward_ipa) { - u32 offset; + u32 mask = ipa_reg_field_max(reg, HDR_OFST_PKT_SIZE); + u32 off; /* Field offset within header */ - offset = offsetof(struct rmnet_map_header, pkt_len); - offset >>= hweight32(HDR_OFST_PKT_SIZE_FMASK); - val |= u32_encode_bits(offset, - HDR_OFST_PKT_SIZE_MSB_FMASK); + off = offsetof(struct rmnet_map_header, pkt_len); + /* Low bits are in the ENDP_INIT_HDR register */ + off >>= hweight32(mask); + val |= ipa_reg_encode(reg, HDR_OFST_PKT_SIZE_MSB, off); /* HDR_ADDITIONAL_CONST_LEN is 0 so MSB is 0 */ } } - iowrite32(val, ipa->reg_virt + offset); + + iowrite32(val, ipa->reg_virt + ipa_reg_n_offset(reg, endpoint_id)); } static void ipa_endpoint_init_hdr_metadata_mask(struct ipa_endpoint *endpoint) { u32 endpoint_id = endpoint->endpoint_id; + struct ipa *ipa = endpoint->ipa; + const struct ipa_reg *reg; u32 val = 0; u32 offset; if (endpoint->toward_ipa) return; /* Register not valid for TX endpoints */ - offset = IPA_REG_ENDP_INIT_HDR_METADATA_MASK_N_OFFSET(endpoint_id); + reg = ipa_reg(ipa, ENDP_INIT_HDR_METADATA_MASK); + offset = ipa_reg_n_offset(reg, endpoint_id); /* Note that HDR_ENDIANNESS indicates big endian header fields */ if (endpoint->config.qmap) val = (__force u32)cpu_to_be32(IPA_ENDPOINT_QMAP_METADATA_MASK); - iowrite32(val, endpoint->ipa->reg_virt + offset); + iowrite32(val, ipa->reg_virt + offset); } static void ipa_endpoint_init_mode(struct ipa_endpoint *endpoint) { - u32 offset = IPA_REG_ENDP_INIT_MODE_N_OFFSET(endpoint->endpoint_id); + struct ipa *ipa = endpoint->ipa; + const struct ipa_reg *reg; + u32 offset; u32 val; if (!endpoint->toward_ipa) return; /* Register not valid for RX endpoints */ + reg = ipa_reg(ipa, ENDP_INIT_MODE); if (endpoint->config.dma_mode) { enum ipa_endpoint_name name = endpoint->config.dma_endpoint; - u32 dma_endpoint_id; + u32 dma_endpoint_id = ipa->name_map[name]->endpoint_id; - dma_endpoint_id = endpoint->ipa->name_map[name]->endpoint_id; - - val = u32_encode_bits(IPA_DMA, MODE_FMASK); - val |= u32_encode_bits(dma_endpoint_id, DEST_PIPE_INDEX_FMASK); + val = ipa_reg_encode(reg, ENDP_MODE, IPA_DMA); + val |= ipa_reg_encode(reg, DEST_PIPE_INDEX, dma_endpoint_id); } else { - val = u32_encode_bits(IPA_BASIC, MODE_FMASK); + val = ipa_reg_encode(reg, ENDP_MODE, IPA_BASIC); } /* All other bits unspecified (and 0) */ - iowrite32(val, endpoint->ipa->reg_virt + offset); + offset = ipa_reg_n_offset(reg, endpoint->endpoint_id); + iowrite32(val, ipa->reg_virt + offset); } -/* Encoded values for AGGR endpoint register fields */ -static u32 aggr_byte_limit_encoded(enum ipa_version version, u32 limit) +/* For IPA v4.5+, times are expressed using Qtime. The AP uses one of two + * pulse generators (0 and 1) to measure elapsed time. In ipa_qtime_config() + * they're configured to have granularity 100 usec and 1 msec, respectively. + * + * The return value is the positive or negative Qtime value to use to + * express the (microsecond) time provided. A positive return value + * means pulse generator 0 can be used; otherwise use pulse generator 1. + */ +static int ipa_qtime_val(u32 microseconds, u32 max) { - if (version < IPA_VERSION_4_5) - return u32_encode_bits(limit, aggr_byte_limit_fmask(true)); + u32 val; - return u32_encode_bits(limit, aggr_byte_limit_fmask(false)); + /* Use 100 microsecond granularity if possible */ + val = DIV_ROUND_CLOSEST(microseconds, 100); + if (val <= max) + return (int)val; + + /* Have to use pulse generator 1 (millisecond granularity) */ + val = DIV_ROUND_CLOSEST(microseconds, 1000); + WARN_ON(val > max); + + return (int)-val; } /* Encode the aggregation timer limit (microseconds) based on IPA version */ -static u32 aggr_time_limit_encoded(enum ipa_version version, u32 limit) +static u32 aggr_time_limit_encode(struct ipa *ipa, const struct ipa_reg *reg, + u32 microseconds) { - u32 gran_sel; - u32 fmask; + u32 max; u32 val; - if (version < IPA_VERSION_4_5) { - /* We set aggregation granularity in ipa_hardware_config() */ - fmask = aggr_time_limit_fmask(true); - val = DIV_ROUND_CLOSEST(limit, IPA_AGGR_GRANULARITY); - WARN(val > field_max(fmask), - "aggr_time_limit too large (%u > %u usec)\n", - val, field_max(fmask) * IPA_AGGR_GRANULARITY); - - return u32_encode_bits(val, fmask); - } - - /* IPA v4.5 expresses the time limit using Qtime. The AP has - * pulse generators 0 and 1 available, which were configured - * in ipa_qtime_config() to have granularity 100 usec and - * 1 msec, respectively. Use pulse generator 0 if possible, - * otherwise fall back to pulse generator 1. - */ - fmask = aggr_time_limit_fmask(false); - val = DIV_ROUND_CLOSEST(limit, 100); - if (val > field_max(fmask)) { - /* Have to use pulse generator 1 (millisecond granularity) */ - gran_sel = AGGR_GRAN_SEL_FMASK; - val = DIV_ROUND_CLOSEST(limit, 1000); - WARN(val > field_max(fmask), - "aggr_time_limit too large (%u > %u usec)\n", - limit, field_max(fmask) * 1000); - } else { - /* We can use pulse generator 0 (100 usec granularity) */ - gran_sel = 0; - } + if (!microseconds) + return 0; /* Nothing to compute if time limit is 0 */ - return gran_sel | u32_encode_bits(val, fmask); -} + max = ipa_reg_field_max(reg, TIME_LIMIT); + if (ipa->version >= IPA_VERSION_4_5) { + u32 gran_sel; + int ret; + + /* Compute the Qtime limit value to use */ + ret = ipa_qtime_val(microseconds, max); + if (ret < 0) { + val = -ret; + gran_sel = ipa_reg_bit(reg, AGGR_GRAN_SEL); + } else { + val = ret; + gran_sel = 0; + } -static u32 aggr_sw_eof_active_encoded(enum ipa_version version, bool enabled) -{ - u32 val = enabled ? 1 : 0; + return gran_sel | ipa_reg_encode(reg, TIME_LIMIT, val); + } - if (version < IPA_VERSION_4_5) - return u32_encode_bits(val, aggr_sw_eof_active_fmask(true)); + /* We program aggregation granularity in ipa_hardware_config() */ + val = DIV_ROUND_CLOSEST(microseconds, IPA_AGGR_GRANULARITY); + WARN(val > max, "aggr_time_limit too large (%u > %u usec)\n", + microseconds, max * IPA_AGGR_GRANULARITY); - return u32_encode_bits(val, aggr_sw_eof_active_fmask(false)); + return ipa_reg_encode(reg, TIME_LIMIT, val); } static void ipa_endpoint_init_aggr(struct ipa_endpoint *endpoint) { - u32 offset = IPA_REG_ENDP_INIT_AGGR_N_OFFSET(endpoint->endpoint_id); - enum ipa_version version = endpoint->ipa->version; + u32 endpoint_id = endpoint->endpoint_id; + struct ipa *ipa = endpoint->ipa; + const struct ipa_reg *reg; u32 val = 0; + reg = ipa_reg(ipa, ENDP_INIT_AGGR); if (endpoint->config.aggregation) { if (!endpoint->toward_ipa) { const struct ipa_endpoint_rx *rx_config; u32 buffer_size; - bool close_eof; u32 limit; rx_config = &endpoint->config.rx; - val |= u32_encode_bits(IPA_ENABLE_AGGR, AGGR_EN_FMASK); - val |= u32_encode_bits(IPA_GENERIC, AGGR_TYPE_FMASK); + val |= ipa_reg_encode(reg, AGGR_EN, IPA_ENABLE_AGGR); + val |= ipa_reg_encode(reg, AGGR_TYPE, IPA_GENERIC); buffer_size = rx_config->buffer_size; limit = ipa_aggr_size_kb(buffer_size - NET_SKB_PAD, rx_config->aggr_hard_limit); - val |= aggr_byte_limit_encoded(version, limit); + val |= ipa_reg_encode(reg, BYTE_LIMIT, limit); limit = rx_config->aggr_time_limit; - val |= aggr_time_limit_encoded(version, limit); + val |= aggr_time_limit_encode(ipa, reg, limit); /* AGGR_PKT_LIMIT is 0 (unlimited) */ - close_eof = rx_config->aggr_close_eof; - val |= aggr_sw_eof_active_encoded(version, close_eof); + if (rx_config->aggr_close_eof) + val |= ipa_reg_bit(reg, SW_EOF_ACTIVE); } else { - val |= u32_encode_bits(IPA_ENABLE_DEAGGR, - AGGR_EN_FMASK); - val |= u32_encode_bits(IPA_QCMAP, AGGR_TYPE_FMASK); + val |= ipa_reg_encode(reg, AGGR_EN, IPA_ENABLE_DEAGGR); + val |= ipa_reg_encode(reg, AGGR_TYPE, IPA_QCMAP); /* other fields ignored */ } /* AGGR_FORCE_CLOSE is 0 */ /* AGGR_GRAN_SEL is 0 for IPA v4.5 */ } else { - val |= u32_encode_bits(IPA_BYPASS_AGGR, AGGR_EN_FMASK); + val |= ipa_reg_encode(reg, AGGR_EN, IPA_BYPASS_AGGR); /* other fields ignored */ } - iowrite32(val, endpoint->ipa->reg_virt + offset); -} - -/* Return the Qtime-based head-of-line blocking timer value that - * represents the given number of microseconds. The result - * includes both the timer value and the selected timer granularity. - */ -static u32 hol_block_timer_qtime_val(struct ipa *ipa, u32 microseconds) -{ - u32 gran_sel; - u32 val; - - /* IPA v4.5 expresses time limits using Qtime. The AP has - * pulse generators 0 and 1 available, which were configured - * in ipa_qtime_config() to have granularity 100 usec and - * 1 msec, respectively. Use pulse generator 0 if possible, - * otherwise fall back to pulse generator 1. - */ - val = DIV_ROUND_CLOSEST(microseconds, 100); - if (val > field_max(TIME_LIMIT_FMASK)) { - /* Have to use pulse generator 1 (millisecond granularity) */ - gran_sel = GRAN_SEL_FMASK; - val = DIV_ROUND_CLOSEST(microseconds, 1000); - } else { - /* We can use pulse generator 0 (100 usec granularity) */ - gran_sel = 0; - } - - return gran_sel | u32_encode_bits(val, TIME_LIMIT_FMASK); + iowrite32(val, ipa->reg_virt + ipa_reg_n_offset(reg, endpoint_id)); } /* The head-of-line blocking timer is defined as a tick count. For @@ -845,12 +887,11 @@ static u32 hol_block_timer_qtime_val(struct ipa *ipa, u32 microseconds) * derived from the 19.2 MHz SoC XO clock. For older IPA versions * each tick represents 128 cycles of the IPA core clock. * - * Return the encoded value that should be written to that register - * that represents the timeout period provided. For IPA v4.2 this - * encodes a base and scale value, while for earlier versions the - * value is a simple tick count. + * Return the encoded value representing the timeout period provided + * that should be written to the ENDP_INIT_HOL_BLOCK_TIMER register. */ -static u32 hol_block_timer_val(struct ipa *ipa, u32 microseconds) +static u32 hol_block_timer_encode(struct ipa *ipa, const struct ipa_reg *reg, + u32 microseconds) { u32 width; u32 scale; @@ -862,18 +903,34 @@ static u32 hol_block_timer_val(struct ipa *ipa, u32 microseconds) if (!microseconds) return 0; /* Nothing to compute if timer period is 0 */ - if (ipa->version >= IPA_VERSION_4_5) - return hol_block_timer_qtime_val(ipa, microseconds); + if (ipa->version >= IPA_VERSION_4_5) { + u32 max = ipa_reg_field_max(reg, TIMER_LIMIT); + u32 gran_sel; + int ret; + + /* Compute the Qtime limit value to use */ + ret = ipa_qtime_val(microseconds, max); + if (ret < 0) { + val = -ret; + gran_sel = ipa_reg_bit(reg, TIMER_GRAN_SEL); + } else { + val = ret; + gran_sel = 0; + } - /* Use 64 bit arithmetic to avoid overflow... */ + return gran_sel | ipa_reg_encode(reg, TIMER_LIMIT, val); + } + + /* Use 64 bit arithmetic to avoid overflow */ rate = ipa_core_clock_rate(ipa); ticks = DIV_ROUND_CLOSEST(microseconds * rate, 128 * USEC_PER_SEC); - /* ...but we still need to fit into a 32-bit register */ - WARN_ON(ticks > U32_MAX); + + /* We still need the result to fit into the field */ + WARN_ON(ticks > ipa_reg_field_max(reg, TIMER_BASE_VALUE)); /* IPA v3.5.1 through v4.1 just record the tick count */ if (ipa->version < IPA_VERSION_4_2) - return (u32)ticks; + return ipa_reg_encode(reg, TIMER_BASE_VALUE, (u32)ticks); /* For IPA v4.2, the tick count is represented by base and * scale fields within the 32-bit timer register, where: @@ -883,8 +940,8 @@ static u32 hol_block_timer_val(struct ipa *ipa, u32 microseconds) * count, and extract the number of bits in the base field * such that high bit is included. */ - high = fls(ticks); /* 1..32 */ - width = HWEIGHT32(BASE_VALUE_FMASK); + high = fls(ticks); /* 1..32 (or warning above) */ + width = hweight32(ipa_reg_fmask(reg, TIMER_BASE_VALUE)); scale = high > width ? high - width : 0; if (scale) { /* If we're scaling, round up to get a closer result */ @@ -894,8 +951,8 @@ static u32 hol_block_timer_val(struct ipa *ipa, u32 microseconds) scale++; } - val = u32_encode_bits(scale, SCALE_FMASK); - val |= u32_encode_bits(ticks >> scale, BASE_VALUE_FMASK); + val = ipa_reg_encode(reg, TIMER_SCALE, scale); + val |= ipa_reg_encode(reg, TIMER_BASE_VALUE, (u32)ticks >> scale); return val; } @@ -906,28 +963,34 @@ static void ipa_endpoint_init_hol_block_timer(struct ipa_endpoint *endpoint, { u32 endpoint_id = endpoint->endpoint_id; struct ipa *ipa = endpoint->ipa; - u32 offset; + const struct ipa_reg *reg; u32 val; /* This should only be changed when HOL_BLOCK_EN is disabled */ - offset = IPA_REG_ENDP_INIT_HOL_BLOCK_TIMER_N_OFFSET(endpoint_id); - val = hol_block_timer_val(ipa, microseconds); - iowrite32(val, ipa->reg_virt + offset); + reg = ipa_reg(ipa, ENDP_INIT_HOL_BLOCK_TIMER); + val = hol_block_timer_encode(ipa, reg, microseconds); + + iowrite32(val, ipa->reg_virt + ipa_reg_n_offset(reg, endpoint_id)); } static void ipa_endpoint_init_hol_block_en(struct ipa_endpoint *endpoint, bool enable) { u32 endpoint_id = endpoint->endpoint_id; + struct ipa *ipa = endpoint->ipa; + const struct ipa_reg *reg; u32 offset; u32 val; - val = enable ? HOL_BLOCK_EN_FMASK : 0; - offset = IPA_REG_ENDP_INIT_HOL_BLOCK_EN_N_OFFSET(endpoint_id); - iowrite32(val, endpoint->ipa->reg_virt + offset); + reg = ipa_reg(ipa, ENDP_INIT_HOL_BLOCK_EN); + offset = ipa_reg_n_offset(reg, endpoint_id); + val = enable ? ipa_reg_bit(reg, HOL_BLOCK_EN) : 0; + + iowrite32(val, ipa->reg_virt + offset); + /* When enabling, the register must be written twice for IPA v4.5+ */ - if (enable && endpoint->ipa->version >= IPA_VERSION_4_5) - iowrite32(val, endpoint->ipa->reg_virt + offset); + if (enable && ipa->version >= IPA_VERSION_4_5) + iowrite32(val, ipa->reg_virt + offset); } /* Assumes HOL_BLOCK is in disabled state */ @@ -960,46 +1023,58 @@ void ipa_endpoint_modem_hol_block_clear_all(struct ipa *ipa) static void ipa_endpoint_init_deaggr(struct ipa_endpoint *endpoint) { - u32 offset = IPA_REG_ENDP_INIT_DEAGGR_N_OFFSET(endpoint->endpoint_id); + u32 endpoint_id = endpoint->endpoint_id; + struct ipa *ipa = endpoint->ipa; + const struct ipa_reg *reg; u32 val = 0; if (!endpoint->toward_ipa) return; /* Register not valid for RX endpoints */ + reg = ipa_reg(ipa, ENDP_INIT_DEAGGR); /* DEAGGR_HDR_LEN is 0 */ /* PACKET_OFFSET_VALID is 0 */ /* PACKET_OFFSET_LOCATION is ignored (not valid) */ /* MAX_PACKET_LEN is 0 (not enforced) */ - iowrite32(val, endpoint->ipa->reg_virt + offset); + iowrite32(val, ipa->reg_virt + ipa_reg_n_offset(reg, endpoint_id)); } static void ipa_endpoint_init_rsrc_grp(struct ipa_endpoint *endpoint) { - u32 offset = IPA_REG_ENDP_INIT_RSRC_GRP_N_OFFSET(endpoint->endpoint_id); + u32 resource_group = endpoint->config.resource_group; + u32 endpoint_id = endpoint->endpoint_id; struct ipa *ipa = endpoint->ipa; + const struct ipa_reg *reg; u32 val; - val = rsrc_grp_encoded(ipa->version, endpoint->config.resource_group); - iowrite32(val, ipa->reg_virt + offset); + reg = ipa_reg(ipa, ENDP_INIT_RSRC_GRP); + val = ipa_reg_encode(reg, ENDP_RSRC_GRP, resource_group); + + iowrite32(val, ipa->reg_virt + ipa_reg_n_offset(reg, endpoint_id)); } static void ipa_endpoint_init_seq(struct ipa_endpoint *endpoint) { - u32 offset = IPA_REG_ENDP_INIT_SEQ_N_OFFSET(endpoint->endpoint_id); - u32 val = 0; + u32 endpoint_id = endpoint->endpoint_id; + struct ipa *ipa = endpoint->ipa; + const struct ipa_reg *reg; + u32 val; if (!endpoint->toward_ipa) return; /* Register not valid for RX endpoints */ + reg = ipa_reg(ipa, ENDP_INIT_SEQ); + /* Low-order byte configures primary packet processing */ - val |= u32_encode_bits(endpoint->config.tx.seq_type, SEQ_TYPE_FMASK); + val = ipa_reg_encode(reg, SEQ_TYPE, endpoint->config.tx.seq_type); - /* Second byte configures replicated packet processing */ - val |= u32_encode_bits(endpoint->config.tx.seq_rep_type, - SEQ_REP_TYPE_FMASK); + /* Second byte (if supported) configures replicated packet processing */ + if (ipa->version < IPA_VERSION_4_5) + val |= ipa_reg_encode(reg, SEQ_REP_TYPE, + endpoint->config.tx.seq_rep_type); - iowrite32(val, endpoint->ipa->reg_virt + offset); + iowrite32(val, ipa->reg_virt + ipa_reg_n_offset(reg, endpoint_id)); } /** @@ -1049,13 +1124,12 @@ static void ipa_endpoint_status(struct ipa_endpoint *endpoint) { u32 endpoint_id = endpoint->endpoint_id; struct ipa *ipa = endpoint->ipa; + const struct ipa_reg *reg; u32 val = 0; - u32 offset; - - offset = IPA_REG_ENDP_STATUS_N_OFFSET(endpoint_id); + reg = ipa_reg(ipa, ENDP_STATUS); if (endpoint->config.status_enable) { - val |= STATUS_EN_FMASK; + val |= ipa_reg_bit(reg, STATUS_EN); if (endpoint->toward_ipa) { enum ipa_endpoint_name name; u32 status_endpoint_id; @@ -1063,16 +1137,16 @@ static void ipa_endpoint_status(struct ipa_endpoint *endpoint) name = endpoint->config.tx.status_endpoint; status_endpoint_id = ipa->name_map[name]->endpoint_id; - val |= u32_encode_bits(status_endpoint_id, - STATUS_ENDP_FMASK); + val |= ipa_reg_encode(reg, STATUS_ENDP, + status_endpoint_id); } /* STATUS_LOCATION is 0, meaning status element precedes - * packet (not present for IPA v4.5) + * packet (not present for IPA v4.5+) */ - /* STATUS_PKT_SUPPRESS_FMASK is 0 (not present for v3.5.1) */ + /* STATUS_PKT_SUPPRESS_FMASK is 0 (not present for v4.0+) */ } - iowrite32(val, ipa->reg_virt + offset); + iowrite32(val, ipa->reg_virt + ipa_reg_n_offset(reg, endpoint_id)); } static int ipa_endpoint_replenish_one(struct ipa_endpoint *endpoint, @@ -1412,16 +1486,18 @@ void ipa_endpoint_trans_release(struct ipa_endpoint *endpoint, void ipa_endpoint_default_route_set(struct ipa *ipa, u32 endpoint_id) { + const struct ipa_reg *reg; u32 val; + reg = ipa_reg(ipa, ROUTE); /* ROUTE_DIS is 0 */ - val = u32_encode_bits(endpoint_id, ROUTE_DEF_PIPE_FMASK); - val |= ROUTE_DEF_HDR_TABLE_FMASK; - val |= u32_encode_bits(0, ROUTE_DEF_HDR_OFST_FMASK); - val |= u32_encode_bits(endpoint_id, ROUTE_FRAG_DEF_PIPE_FMASK); - val |= ROUTE_DEF_RETAIN_HDR_FMASK; + val = ipa_reg_encode(reg, ROUTE_DEF_PIPE, endpoint_id); + val |= ipa_reg_bit(reg, ROUTE_DEF_HDR_TABLE); + /* ROUTE_DEF_HDR_OFST is 0 */ + val |= ipa_reg_encode(reg, ROUTE_FRAG_DEF_PIPE, endpoint_id); + val |= ipa_reg_bit(reg, ROUTE_DEF_RETAIN_HDR); - iowrite32(val, ipa->reg_virt + IPA_REG_ROUTE_OFFSET); + iowrite32(val, ipa->reg_virt + ipa_reg_offset(reg)); } void ipa_endpoint_default_route_clear(struct ipa *ipa) @@ -1765,6 +1841,7 @@ void ipa_endpoint_teardown(struct ipa *ipa) int ipa_endpoint_config(struct ipa *ipa) { struct device *dev = &ipa->pdev->dev; + const struct ipa_reg *reg; u32 initialized; u32 rx_base; u32 rx_mask; @@ -1791,11 +1868,12 @@ int ipa_endpoint_config(struct ipa *ipa) /* Find out about the endpoints supplied by the hardware, and ensure * the highest one doesn't exceed the number we support. */ - val = ioread32(ipa->reg_virt + IPA_REG_FLAVOR_0_OFFSET); + reg = ipa_reg(ipa, FLAVOR_0); + val = ioread32(ipa->reg_virt + ipa_reg_offset(reg)); /* Our RX is an IPA producer */ - rx_base = u32_get_bits(val, IPA_PROD_LOWEST_FMASK); - max = rx_base + u32_get_bits(val, IPA_MAX_PROD_PIPES_FMASK); + rx_base = ipa_reg_decode(reg, PROD_LOWEST, val); + max = rx_base + ipa_reg_decode(reg, MAX_PROD_PIPES, val); if (max > IPA_ENDPOINT_MAX) { dev_err(dev, "too many endpoints (%u > %u)\n", max, IPA_ENDPOINT_MAX); @@ -1804,7 +1882,7 @@ int ipa_endpoint_config(struct ipa *ipa) rx_mask = GENMASK(max - 1, rx_base); /* Our TX is an IPA consumer */ - max = u32_get_bits(val, IPA_MAX_CONS_PIPES_FMASK); + max = ipa_reg_decode(reg, MAX_CONS_PIPES, val); tx_mask = GENMASK(max - 1, 0); ipa->available = rx_mask | tx_mask; diff --git a/drivers/net/ipa/ipa_endpoint.h b/drivers/net/ipa/ipa_endpoint.h index 28e0a7386fd7..d8dfa24f5214 100644 --- a/drivers/net/ipa/ipa_endpoint.h +++ b/drivers/net/ipa/ipa_endpoint.h @@ -1,7 +1,7 @@ /* SPDX-License-Identifier: GPL-2.0 */ /* Copyright (c) 2012-2018, The Linux Foundation. All rights reserved. - * Copyright (C) 2019-2020 Linaro Ltd. + * Copyright (C) 2019-2022 Linaro Ltd. */ #ifndef _IPA_ENDPOINT_H_ #define _IPA_ENDPOINT_H_ diff --git a/drivers/net/ipa/ipa_interrupt.c b/drivers/net/ipa/ipa_interrupt.c index 307bed2ee707..c269432f9c2e 100644 --- a/drivers/net/ipa/ipa_interrupt.c +++ b/drivers/net/ipa/ipa_interrupt.c @@ -1,7 +1,7 @@ // SPDX-License-Identifier: GPL-2.0 /* Copyright (c) 2014-2018, The Linux Foundation. All rights reserved. - * Copyright (C) 2018-2020 Linaro Ltd. + * Copyright (C) 2018-2022 Linaro Ltd. */ /* DOC: IPA Interrupts @@ -53,13 +53,15 @@ static void ipa_interrupt_process(struct ipa_interrupt *interrupt, u32 irq_id) { bool uc_irq = ipa_interrupt_uc(interrupt, irq_id); struct ipa *ipa = interrupt->ipa; + const struct ipa_reg *reg; u32 mask = BIT(irq_id); u32 offset; /* For microcontroller interrupts, clear the interrupt right away, * "to avoid clearing unhandled interrupts." */ - offset = ipa_reg_irq_clr_offset(ipa->version); + reg = ipa_reg(ipa, IPA_IRQ_CLR); + offset = ipa_reg_offset(reg); if (uc_irq) iowrite32(mask, ipa->reg_virt + offset); @@ -80,6 +82,7 @@ static irqreturn_t ipa_isr_thread(int irq, void *dev_id) struct ipa_interrupt *interrupt = dev_id; struct ipa *ipa = interrupt->ipa; u32 enabled = interrupt->enabled; + const struct ipa_reg *reg; struct device *dev; u32 pending; u32 offset; @@ -95,7 +98,8 @@ static irqreturn_t ipa_isr_thread(int irq, void *dev_id) * including conditions whose interrupt is not enabled. Handle * only the enabled ones. */ - offset = ipa_reg_irq_stts_offset(ipa->version); + reg = ipa_reg(ipa, IPA_IRQ_STTS); + offset = ipa_reg_offset(reg); pending = ioread32(ipa->reg_virt + offset); while ((mask = pending & enabled)) { do { @@ -112,7 +116,8 @@ static irqreturn_t ipa_isr_thread(int irq, void *dev_id) if (pending) { dev_dbg(dev, "clearing disabled IPA interrupts 0x%08x\n", pending); - offset = ipa_reg_irq_clr_offset(ipa->version); + reg = ipa_reg(ipa, IPA_IRQ_CLR); + offset = ipa_reg_offset(reg); iowrite32(pending, ipa->reg_virt + offset); } out_power_put: @@ -128,6 +133,7 @@ static void ipa_interrupt_suspend_control(struct ipa_interrupt *interrupt, { struct ipa *ipa = interrupt->ipa; u32 mask = BIT(endpoint_id); + const struct ipa_reg *reg; u32 offset; u32 val; @@ -137,7 +143,8 @@ static void ipa_interrupt_suspend_control(struct ipa_interrupt *interrupt, if (ipa->version == IPA_VERSION_3_0) return; - offset = ipa_reg_irq_suspend_en_offset(ipa->version); + reg = ipa_reg(ipa, IRQ_SUSPEND_EN); + offset = ipa_reg_offset(reg); val = ioread32(ipa->reg_virt + offset); if (enable) val |= mask; @@ -164,18 +171,18 @@ ipa_interrupt_suspend_disable(struct ipa_interrupt *interrupt, u32 endpoint_id) void ipa_interrupt_suspend_clear_all(struct ipa_interrupt *interrupt) { struct ipa *ipa = interrupt->ipa; - u32 offset; + const struct ipa_reg *reg; u32 val; - offset = ipa_reg_irq_suspend_info_offset(ipa->version); - val = ioread32(ipa->reg_virt + offset); + reg = ipa_reg(ipa, IRQ_SUSPEND_INFO); + val = ioread32(ipa->reg_virt + ipa_reg_offset(reg)); /* SUSPEND interrupt status isn't cleared on IPA version 3.0 */ if (ipa->version == IPA_VERSION_3_0) return; - offset = ipa_reg_irq_suspend_clr_offset(ipa->version); - iowrite32(val, ipa->reg_virt + offset); + reg = ipa_reg(ipa, IRQ_SUSPEND_CLR); + iowrite32(val, ipa->reg_virt + ipa_reg_offset(reg)); } /* Simulate arrival of an IPA TX_SUSPEND interrupt */ @@ -189,7 +196,7 @@ void ipa_interrupt_add(struct ipa_interrupt *interrupt, enum ipa_irq_id ipa_irq, ipa_irq_handler_t handler) { struct ipa *ipa = interrupt->ipa; - u32 offset; + const struct ipa_reg *reg; if (WARN_ON(ipa_irq >= IPA_IRQ_COUNT)) return; @@ -198,8 +205,9 @@ void ipa_interrupt_add(struct ipa_interrupt *interrupt, /* Update the IPA interrupt mask to enable it */ interrupt->enabled |= BIT(ipa_irq); - offset = ipa_reg_irq_en_offset(ipa->version); - iowrite32(interrupt->enabled, ipa->reg_virt + offset); + + reg = ipa_reg(ipa, IPA_IRQ_EN); + iowrite32(interrupt->enabled, ipa->reg_virt + ipa_reg_offset(reg)); } /* Remove the handler for an IPA interrupt type */ @@ -207,15 +215,16 @@ void ipa_interrupt_remove(struct ipa_interrupt *interrupt, enum ipa_irq_id ipa_irq) { struct ipa *ipa = interrupt->ipa; - u32 offset; + const struct ipa_reg *reg; if (WARN_ON(ipa_irq >= IPA_IRQ_COUNT)) return; /* Update the IPA interrupt mask to disable it */ interrupt->enabled &= ~BIT(ipa_irq); - offset = ipa_reg_irq_en_offset(ipa->version); - iowrite32(interrupt->enabled, ipa->reg_virt + offset); + + reg = ipa_reg(ipa, IPA_IRQ_EN); + iowrite32(interrupt->enabled, ipa->reg_virt + ipa_reg_offset(reg)); interrupt->handler[ipa_irq] = NULL; } @@ -225,8 +234,8 @@ struct ipa_interrupt *ipa_interrupt_config(struct ipa *ipa) { struct device *dev = &ipa->pdev->dev; struct ipa_interrupt *interrupt; + const struct ipa_reg *reg; unsigned int irq; - u32 offset; int ret; ret = platform_get_irq_byname(ipa->pdev, "ipa"); @@ -244,8 +253,8 @@ struct ipa_interrupt *ipa_interrupt_config(struct ipa *ipa) interrupt->irq = irq; /* Start with all IPA interrupts disabled */ - offset = ipa_reg_irq_en_offset(ipa->version); - iowrite32(0, ipa->reg_virt + offset); + reg = ipa_reg(ipa, IPA_IRQ_EN); + iowrite32(0, ipa->reg_virt + ipa_reg_offset(reg)); ret = request_threaded_irq(irq, NULL, ipa_isr_thread, IRQF_ONESHOT, "ipa", interrupt); diff --git a/drivers/net/ipa/ipa_interrupt.h b/drivers/net/ipa/ipa_interrupt.h index 231390cea52a..f31fd9965fdc 100644 --- a/drivers/net/ipa/ipa_interrupt.h +++ b/drivers/net/ipa/ipa_interrupt.h @@ -1,7 +1,7 @@ /* SPDX-License-Identifier: GPL-2.0 */ /* Copyright (c) 2012-2018, The Linux Foundation. All rights reserved. - * Copyright (C) 2018-2020 Linaro Ltd. + * Copyright (C) 2018-2022 Linaro Ltd. */ #ifndef _IPA_INTERRUPT_H_ #define _IPA_INTERRUPT_H_ diff --git a/drivers/net/ipa/ipa_main.c b/drivers/net/ipa/ipa_main.c index 32962d885acd..3461ad3029ab 100644 --- a/drivers/net/ipa/ipa_main.c +++ b/drivers/net/ipa/ipa_main.c @@ -1,7 +1,7 @@ // SPDX-License-Identifier: GPL-2.0 /* Copyright (c) 2012-2018, The Linux Foundation. All rights reserved. - * Copyright (C) 2018-2021 Linaro Ltd. + * Copyright (C) 2018-2022 Linaro Ltd. */ #include <linux/types.h> @@ -183,31 +183,97 @@ static void ipa_teardown(struct ipa *ipa) gsi_teardown(&ipa->gsi); } +static void +ipa_hardware_config_bcr(struct ipa *ipa, const struct ipa_data *data) +{ + const struct ipa_reg *reg; + u32 val; + + /* IPA v4.5+ has no backward compatibility register */ + if (ipa->version >= IPA_VERSION_4_5) + return; + + reg = ipa_reg(ipa, IPA_BCR); + val = data->backward_compat; + iowrite32(val, ipa->reg_virt + ipa_reg_offset(reg)); +} + +static void ipa_hardware_config_tx(struct ipa *ipa) +{ + enum ipa_version version = ipa->version; + const struct ipa_reg *reg; + u32 offset; + u32 val; + + if (version <= IPA_VERSION_4_0 || version >= IPA_VERSION_4_5) + return; + + /* Disable PA mask to allow HOLB drop */ + reg = ipa_reg(ipa, IPA_TX_CFG); + offset = ipa_reg_offset(reg); + + val = ioread32(ipa->reg_virt + offset); + + val &= ~ipa_reg_bit(reg, PA_MASK_EN); + + iowrite32(val, ipa->reg_virt + offset); +} + +static void ipa_hardware_config_clkon(struct ipa *ipa) +{ + enum ipa_version version = ipa->version; + const struct ipa_reg *reg; + u32 val; + + if (version >= IPA_VERSION_4_5) + return; + + if (version < IPA_VERSION_4_0 && version != IPA_VERSION_3_1) + return; + + /* Implement some hardware workarounds */ + reg = ipa_reg(ipa, CLKON_CFG); + if (version == IPA_VERSION_3_1) { + /* Disable MISC clock gating */ + val = ipa_reg_bit(reg, CLKON_MISC); + } else { /* IPA v4.0+ */ + /* Enable open global clocks in the CLKON configuration */ + val = ipa_reg_bit(reg, CLKON_GLOBAL); + val |= ipa_reg_bit(reg, GLOBAL_2X_CLK); + } + + iowrite32(val, ipa->reg_virt + ipa_reg_offset(reg)); +} + /* Configure bus access behavior for IPA components */ static void ipa_hardware_config_comp(struct ipa *ipa) { + const struct ipa_reg *reg; + u32 offset; u32 val; /* Nothing to configure prior to IPA v4.0 */ if (ipa->version < IPA_VERSION_4_0) return; - val = ioread32(ipa->reg_virt + IPA_REG_COMP_CFG_OFFSET); + reg = ipa_reg(ipa, COMP_CFG); + offset = ipa_reg_offset(reg); + val = ioread32(ipa->reg_virt + offset); if (ipa->version == IPA_VERSION_4_0) { - val &= ~IPA_QMB_SELECT_CONS_EN_FMASK; - val &= ~IPA_QMB_SELECT_PROD_EN_FMASK; - val &= ~IPA_QMB_SELECT_GLOBAL_EN_FMASK; + val &= ~ipa_reg_bit(reg, IPA_QMB_SELECT_CONS_EN); + val &= ~ipa_reg_bit(reg, IPA_QMB_SELECT_PROD_EN); + val &= ~ipa_reg_bit(reg, IPA_QMB_SELECT_GLOBAL_EN); } else if (ipa->version < IPA_VERSION_4_5) { - val |= GSI_MULTI_AXI_MASTERS_DIS_FMASK; + val |= ipa_reg_bit(reg, GSI_MULTI_AXI_MASTERS_DIS); } else { - /* For IPA v4.5 IPA_FULL_FLUSH_WAIT_RSC_CLOSE_EN is 0 */ + /* For IPA v4.5 FULL_FLUSH_WAIT_RS_CLOSURE_EN is 0 */ } - val |= GSI_MULTI_INORDER_RD_DIS_FMASK; - val |= GSI_MULTI_INORDER_WR_DIS_FMASK; + val |= ipa_reg_bit(reg, GSI_MULTI_INORDER_RD_DIS); + val |= ipa_reg_bit(reg, GSI_MULTI_INORDER_WR_DIS); - iowrite32(val, ipa->reg_virt + IPA_REG_COMP_CFG_OFFSET); + iowrite32(val, ipa->reg_virt + offset); } /* Configure DDR and (possibly) PCIe max read/write QSB values */ @@ -216,6 +282,7 @@ ipa_hardware_config_qsb(struct ipa *ipa, const struct ipa_data *data) { const struct ipa_qsb_data *data0; const struct ipa_qsb_data *data1; + const struct ipa_reg *reg; u32 val; /* QMB 0 represents DDR; QMB 1 (if present) represents PCIe */ @@ -224,25 +291,31 @@ ipa_hardware_config_qsb(struct ipa *ipa, const struct ipa_data *data) data1 = &data->qsb_data[IPA_QSB_MASTER_PCIE]; /* Max outstanding write accesses for QSB masters */ - val = u32_encode_bits(data0->max_writes, GEN_QMB_0_MAX_WRITES_FMASK); + reg = ipa_reg(ipa, QSB_MAX_WRITES); + + val = ipa_reg_encode(reg, GEN_QMB_0_MAX_WRITES, data0->max_writes); if (data->qsb_count > 1) - val |= u32_encode_bits(data1->max_writes, - GEN_QMB_1_MAX_WRITES_FMASK); - iowrite32(val, ipa->reg_virt + IPA_REG_QSB_MAX_WRITES_OFFSET); + val |= ipa_reg_encode(reg, GEN_QMB_1_MAX_WRITES, + data1->max_writes); + + iowrite32(val, ipa->reg_virt + ipa_reg_offset(reg)); /* Max outstanding read accesses for QSB masters */ - val = u32_encode_bits(data0->max_reads, GEN_QMB_0_MAX_READS_FMASK); + reg = ipa_reg(ipa, QSB_MAX_READS); + + val = ipa_reg_encode(reg, GEN_QMB_0_MAX_READS, data0->max_reads); if (ipa->version >= IPA_VERSION_4_0) - val |= u32_encode_bits(data0->max_reads_beats, - GEN_QMB_0_MAX_READS_BEATS_FMASK); + val |= ipa_reg_encode(reg, GEN_QMB_0_MAX_READS_BEATS, + data0->max_reads_beats); if (data->qsb_count > 1) { - val |= u32_encode_bits(data1->max_reads, - GEN_QMB_1_MAX_READS_FMASK); + val = ipa_reg_encode(reg, GEN_QMB_1_MAX_READS, + data1->max_reads); if (ipa->version >= IPA_VERSION_4_0) - val |= u32_encode_bits(data1->max_reads_beats, - GEN_QMB_1_MAX_READS_BEATS_FMASK); + val |= ipa_reg_encode(reg, GEN_QMB_1_MAX_READS_BEATS, + data1->max_reads_beats); } - iowrite32(val, ipa->reg_virt + IPA_REG_QSB_MAX_READS_OFFSET); + + iowrite32(val, ipa->reg_virt + ipa_reg_offset(reg)); } /* The internal inactivity timer clock is used for the aggregation timer */ @@ -278,48 +351,96 @@ static __always_inline u32 ipa_aggr_granularity_val(u32 usec) */ static void ipa_qtime_config(struct ipa *ipa) { + const struct ipa_reg *reg; + u32 offset; u32 val; /* Timer clock divider must be disabled when we change the rate */ - iowrite32(0, ipa->reg_virt + IPA_REG_TIMERS_XO_CLK_DIV_CFG_OFFSET); + reg = ipa_reg(ipa, TIMERS_XO_CLK_DIV_CFG); + iowrite32(0, ipa->reg_virt + ipa_reg_offset(reg)); + reg = ipa_reg(ipa, QTIME_TIMESTAMP_CFG); /* Set DPL time stamp resolution to use Qtime (instead of 1 msec) */ - val = u32_encode_bits(DPL_TIMESTAMP_SHIFT, DPL_TIMESTAMP_LSB_FMASK); - val |= u32_encode_bits(1, DPL_TIMESTAMP_SEL_FMASK); + val = ipa_reg_encode(reg, DPL_TIMESTAMP_LSB, DPL_TIMESTAMP_SHIFT); + val |= ipa_reg_bit(reg, DPL_TIMESTAMP_SEL); /* Configure tag and NAT Qtime timestamp resolution as well */ - val |= u32_encode_bits(TAG_TIMESTAMP_SHIFT, TAG_TIMESTAMP_LSB_FMASK); - val |= u32_encode_bits(NAT_TIMESTAMP_SHIFT, NAT_TIMESTAMP_LSB_FMASK); - iowrite32(val, ipa->reg_virt + IPA_REG_QTIME_TIMESTAMP_CFG_OFFSET); + val = ipa_reg_encode(reg, TAG_TIMESTAMP_LSB, TAG_TIMESTAMP_SHIFT); + val = ipa_reg_encode(reg, NAT_TIMESTAMP_LSB, NAT_TIMESTAMP_SHIFT); + + iowrite32(val, ipa->reg_virt + ipa_reg_offset(reg)); /* Set granularity of pulse generators used for other timers */ - val = u32_encode_bits(IPA_GRAN_100_US, GRAN_0_FMASK); - val |= u32_encode_bits(IPA_GRAN_1_MS, GRAN_1_FMASK); - val |= u32_encode_bits(IPA_GRAN_1_MS, GRAN_2_FMASK); - iowrite32(val, ipa->reg_virt + IPA_REG_TIMERS_PULSE_GRAN_CFG_OFFSET); + reg = ipa_reg(ipa, TIMERS_PULSE_GRAN_CFG); + val = ipa_reg_encode(reg, PULSE_GRAN_0, IPA_GRAN_100_US); + val |= ipa_reg_encode(reg, PULSE_GRAN_1, IPA_GRAN_1_MS); + val |= ipa_reg_encode(reg, PULSE_GRAN_2, IPA_GRAN_1_MS); + + iowrite32(val, ipa->reg_virt + ipa_reg_offset(reg)); /* Actual divider is 1 more than value supplied here */ - val = u32_encode_bits(IPA_XO_CLOCK_DIVIDER - 1, DIV_VALUE_FMASK); - iowrite32(val, ipa->reg_virt + IPA_REG_TIMERS_XO_CLK_DIV_CFG_OFFSET); + reg = ipa_reg(ipa, TIMERS_XO_CLK_DIV_CFG); + offset = ipa_reg_offset(reg); + val = ipa_reg_encode(reg, DIV_VALUE, IPA_XO_CLOCK_DIVIDER - 1); + + iowrite32(val, ipa->reg_virt + offset); /* Divider value is set; re-enable the common timer clock divider */ - val |= u32_encode_bits(1, DIV_ENABLE_FMASK); - iowrite32(val, ipa->reg_virt + IPA_REG_TIMERS_XO_CLK_DIV_CFG_OFFSET); + val |= ipa_reg_bit(reg, DIV_ENABLE); + + iowrite32(val, ipa->reg_virt + offset); +} + +/* Before IPA v4.5 timing is controlled by a counter register */ +static void ipa_hardware_config_counter(struct ipa *ipa) +{ + u32 granularity = ipa_aggr_granularity_val(IPA_AGGR_GRANULARITY); + const struct ipa_reg *reg; + u32 val; + + reg = ipa_reg(ipa, COUNTER_CFG); + /* If defined, EOT_COAL_GRANULARITY is 0 */ + val = ipa_reg_encode(reg, AGGR_GRANULARITY, granularity); + iowrite32(val, ipa->reg_virt + ipa_reg_offset(reg)); +} + +static void ipa_hardware_config_timing(struct ipa *ipa) +{ + if (ipa->version < IPA_VERSION_4_5) + ipa_hardware_config_counter(ipa); + else + ipa_qtime_config(ipa); +} + +static void ipa_hardware_config_hashing(struct ipa *ipa) +{ + const struct ipa_reg *reg; + + if (ipa->version != IPA_VERSION_4_2) + return; + + /* IPA v4.2 does not support hashed tables, so disable them */ + reg = ipa_reg(ipa, FILT_ROUT_HASH_EN); + + /* IPV6_ROUTER_HASH, IPV6_FILTER_HASH, IPV4_ROUTER_HASH, + * IPV4_FILTER_HASH are all zero. + */ + iowrite32(0, ipa->reg_virt + ipa_reg_offset(reg)); } static void ipa_idle_indication_cfg(struct ipa *ipa, u32 enter_idle_debounce_thresh, bool const_non_idle_enable) { - u32 offset; + const struct ipa_reg *reg; u32 val; - val = u32_encode_bits(enter_idle_debounce_thresh, - ENTER_IDLE_DEBOUNCE_THRESH_FMASK); + reg = ipa_reg(ipa, IDLE_INDICATION_CFG); + val = ipa_reg_encode(reg, ENTER_IDLE_DEBOUNCE_THRESH, + enter_idle_debounce_thresh); if (const_non_idle_enable) - val |= CONST_NON_IDLE_ENABLE_FMASK; + val |= ipa_reg_bit(reg, CONST_NON_IDLE_ENABLE); - offset = ipa_reg_idle_indication_cfg_offset(ipa->version); - iowrite32(val, ipa->reg_virt + offset); + iowrite32(val, ipa->reg_virt + ipa_reg_offset(reg)); } /** @@ -349,55 +470,13 @@ static void ipa_hardware_dcd_deconfig(struct ipa *ipa) */ static void ipa_hardware_config(struct ipa *ipa, const struct ipa_data *data) { - enum ipa_version version = ipa->version; - u32 granularity; - u32 val; - - /* IPA v4.5+ has no backward compatibility register */ - if (version < IPA_VERSION_4_5) { - val = data->backward_compat; - iowrite32(val, ipa->reg_virt + IPA_REG_BCR_OFFSET); - } - - /* Implement some hardware workarounds */ - if (version >= IPA_VERSION_4_0 && version < IPA_VERSION_4_5) { - /* Disable PA mask to allow HOLB drop */ - val = ioread32(ipa->reg_virt + IPA_REG_TX_CFG_OFFSET); - val &= ~PA_MASK_EN_FMASK; - iowrite32(val, ipa->reg_virt + IPA_REG_TX_CFG_OFFSET); - - /* Enable open global clocks in the CLKON configuration */ - val = GLOBAL_FMASK | GLOBAL_2X_CLK_FMASK; - } else if (version == IPA_VERSION_3_1) { - val = MISC_FMASK; /* Disable MISC clock gating */ - } else { - val = 0; /* No CLKON configuration needed */ - } - if (val) - iowrite32(val, ipa->reg_virt + IPA_REG_CLKON_CFG_OFFSET); - + ipa_hardware_config_bcr(ipa, data); + ipa_hardware_config_tx(ipa); + ipa_hardware_config_clkon(ipa); ipa_hardware_config_comp(ipa); - - /* Configure system bus limits */ ipa_hardware_config_qsb(ipa, data); - - if (version < IPA_VERSION_4_5) { - /* Configure aggregation timer granularity */ - granularity = ipa_aggr_granularity_val(IPA_AGGR_GRANULARITY); - val = u32_encode_bits(granularity, AGGR_GRANULARITY_FMASK); - iowrite32(val, ipa->reg_virt + IPA_REG_COUNTER_CFG_OFFSET); - } else { - ipa_qtime_config(ipa); - } - - /* IPA v4.2 does not support hashed tables, so disable them */ - if (version == IPA_VERSION_4_2) { - u32 offset = ipa_reg_filt_rout_hash_en_offset(version); - - iowrite32(0, ipa->reg_virt + offset); - } - - /* Enable dynamic clock division */ + ipa_hardware_config_timing(ipa); + ipa_hardware_config_hashing(ipa); ipa_hardware_dcd_config(ipa); } @@ -612,29 +691,6 @@ static void ipa_validate_build(void) /* Aggregation granularity value can't be 0, and must fit */ BUILD_BUG_ON(!ipa_aggr_granularity_val(IPA_AGGR_GRANULARITY)); - BUILD_BUG_ON(ipa_aggr_granularity_val(IPA_AGGR_GRANULARITY) > - field_max(AGGR_GRANULARITY_FMASK)); -} - -static bool ipa_version_valid(enum ipa_version version) -{ - switch (version) { - case IPA_VERSION_3_0: - case IPA_VERSION_3_1: - case IPA_VERSION_3_5: - case IPA_VERSION_3_5_1: - case IPA_VERSION_4_0: - case IPA_VERSION_4_1: - case IPA_VERSION_4_2: - case IPA_VERSION_4_5: - case IPA_VERSION_4_7: - case IPA_VERSION_4_9: - case IPA_VERSION_4_11: - return true; - - default: - return false; - } } /** @@ -678,8 +734,8 @@ static int ipa_probe(struct platform_device *pdev) return -ENODEV; } - if (!ipa_version_valid(data->version)) { - dev_err(dev, "invalid IPA version\n"); + if (!ipa_version_supported(data->version)) { + dev_err(dev, "unsupported IPA version %u\n", data->version); return -EINVAL; } diff --git a/drivers/net/ipa/ipa_mem.c b/drivers/net/ipa/ipa_mem.c index 53a1dbeaffa6..f84c6830495a 100644 --- a/drivers/net/ipa/ipa_mem.c +++ b/drivers/net/ipa/ipa_mem.c @@ -1,7 +1,7 @@ // SPDX-License-Identifier: GPL-2.0 /* Copyright (c) 2012-2018, The Linux Foundation. All rights reserved. - * Copyright (C) 2019-2021 Linaro Ltd. + * Copyright (C) 2019-2022 Linaro Ltd. */ #include <linux/types.h> @@ -75,6 +75,7 @@ ipa_mem_zero_region_add(struct gsi_trans *trans, enum ipa_mem_id mem_id) int ipa_mem_setup(struct ipa *ipa) { dma_addr_t addr = ipa->zero_addr; + const struct ipa_reg *reg; const struct ipa_mem *mem; struct gsi_trans *trans; u32 offset; @@ -112,8 +113,10 @@ int ipa_mem_setup(struct ipa *ipa) /* Tell the hardware where the processing context area is located */ mem = ipa_mem_find(ipa, IPA_MEM_MODEM_PROC_CTX); offset = ipa->mem_offset + mem->offset; - val = proc_cntxt_base_addr_encoded(ipa->version, offset); - iowrite32(val, ipa->reg_virt + IPA_REG_LOCAL_PKT_PROC_CNTXT_OFFSET); + + reg = ipa_reg(ipa, LOCAL_PKT_PROC_CNTXT); + val = ipa_reg_encode(reg, IPA_BASE_ADDR, offset); + iowrite32(val, ipa->reg_virt + ipa_reg_offset(reg)); return 0; } @@ -306,6 +309,7 @@ static bool ipa_mem_size_valid(struct ipa *ipa) int ipa_mem_config(struct ipa *ipa) { struct device *dev = &ipa->pdev->dev; + const struct ipa_reg *reg; const struct ipa_mem *mem; dma_addr_t addr; u32 mem_size; @@ -314,12 +318,14 @@ int ipa_mem_config(struct ipa *ipa) u32 i; /* Check the advertised location and size of the shared memory area */ - val = ioread32(ipa->reg_virt + IPA_REG_SHARED_MEM_SIZE_OFFSET); + reg = ipa_reg(ipa, SHARED_MEM_SIZE); + val = ioread32(ipa->reg_virt + ipa_reg_offset(reg)); /* The fields in the register are in 8 byte units */ - ipa->mem_offset = 8 * u32_get_bits(val, SHARED_MEM_BADDR_FMASK); + ipa->mem_offset = 8 * ipa_reg_decode(reg, MEM_BADDR, val); + /* Make sure the end is within the region's mapped space */ - mem_size = 8 * u32_get_bits(val, SHARED_MEM_SIZE_FMASK); + mem_size = 8 * ipa_reg_decode(reg, MEM_SIZE, val); /* If the sizes don't match, issue a warning */ if (ipa->mem_offset + mem_size < ipa->mem_size) { diff --git a/drivers/net/ipa/ipa_modem.c b/drivers/net/ipa/ipa_modem.c index c8b1c4d9c507..423422a2a445 100644 --- a/drivers/net/ipa/ipa_modem.c +++ b/drivers/net/ipa/ipa_modem.c @@ -1,7 +1,7 @@ // SPDX-License-Identifier: GPL-2.0 /* Copyright (c) 2014-2018, The Linux Foundation. All rights reserved. - * Copyright (C) 2018-2021 Linaro Ltd. + * Copyright (C) 2018-2022 Linaro Ltd. */ #include <linux/errno.h> diff --git a/drivers/net/ipa/ipa_modem.h b/drivers/net/ipa/ipa_modem.h index e64ccc2402e9..d85718db9a57 100644 --- a/drivers/net/ipa/ipa_modem.h +++ b/drivers/net/ipa/ipa_modem.h @@ -1,7 +1,7 @@ /* SPDX-License-Identifier: GPL-2.0 */ /* Copyright (c) 2012-2018, The Linux Foundation. All rights reserved. - * Copyright (C) 2018-2020 Linaro Ltd. + * Copyright (C) 2018-2022 Linaro Ltd. */ #ifndef _IPA_MODEM_H_ #define _IPA_MODEM_H_ diff --git a/drivers/net/ipa/ipa_power.c b/drivers/net/ipa/ipa_power.c index db5ac7552286..8420f93128a2 100644 --- a/drivers/net/ipa/ipa_power.c +++ b/drivers/net/ipa/ipa_power.c @@ -1,7 +1,7 @@ // SPDX-License-Identifier: GPL-2.0 /* Copyright (c) 2012-2018, The Linux Foundation. All rights reserved. - * Copyright (C) 2018-2021 Linaro Ltd. + * Copyright (C) 2018-2022 Linaro Ltd. */ #include <linux/clk.h> diff --git a/drivers/net/ipa/ipa_power.h b/drivers/net/ipa/ipa_power.h index 6f84f057a209..896f052e51a1 100644 --- a/drivers/net/ipa/ipa_power.h +++ b/drivers/net/ipa/ipa_power.h @@ -1,7 +1,7 @@ /* SPDX-License-Identifier: GPL-2.0 */ /* Copyright (c) 2012-2018, The Linux Foundation. All rights reserved. - * Copyright (C) 2018-2020 Linaro Ltd. + * Copyright (C) 2018-2022 Linaro Ltd. */ #ifndef _IPA_POWER_H_ #define _IPA_POWER_H_ diff --git a/drivers/net/ipa/ipa_qmi.c b/drivers/net/ipa/ipa_qmi.c index 6f874f99b910..8295fd4b70d1 100644 --- a/drivers/net/ipa/ipa_qmi.c +++ b/drivers/net/ipa/ipa_qmi.c @@ -1,7 +1,7 @@ // SPDX-License-Identifier: GPL-2.0 /* Copyright (c) 2013-2018, The Linux Foundation. All rights reserved. - * Copyright (C) 2018-2020 Linaro Ltd. + * Copyright (C) 2018-2022 Linaro Ltd. */ #include <linux/types.h> diff --git a/drivers/net/ipa/ipa_qmi.h b/drivers/net/ipa/ipa_qmi.h index 856ef629ccc8..1c236826c17a 100644 --- a/drivers/net/ipa/ipa_qmi.h +++ b/drivers/net/ipa/ipa_qmi.h @@ -1,7 +1,7 @@ /* SPDX-License-Identifier: GPL-2.0 */ /* Copyright (c) 2018, The Linux Foundation. All rights reserved. - * Copyright (C) 2018-2020 Linaro Ltd. + * Copyright (C) 2018-2022 Linaro Ltd. */ #ifndef _IPA_QMI_H_ #define _IPA_QMI_H_ diff --git a/drivers/net/ipa/ipa_qmi_msg.c b/drivers/net/ipa/ipa_qmi_msg.c index 75d3fc0092e9..97c0befe8d86 100644 --- a/drivers/net/ipa/ipa_qmi_msg.c +++ b/drivers/net/ipa/ipa_qmi_msg.c @@ -1,7 +1,7 @@ // SPDX-License-Identifier: GPL-2.0 /* Copyright (c) 2018, The Linux Foundation. All rights reserved. - * Copyright (C) 2018-2020 Linaro Ltd. + * Copyright (C) 2018-2022 Linaro Ltd. */ #include <linux/stddef.h> #include <linux/soc/qcom/qmi.h> diff --git a/drivers/net/ipa/ipa_qmi_msg.h b/drivers/net/ipa/ipa_qmi_msg.h index 9651aa59b596..e29663965f43 100644 --- a/drivers/net/ipa/ipa_qmi_msg.h +++ b/drivers/net/ipa/ipa_qmi_msg.h @@ -1,7 +1,7 @@ /* SPDX-License-Identifier: GPL-2.0 */ /* Copyright (c) 2018, The Linux Foundation. All rights reserved. - * Copyright (C) 2018-2020 Linaro Ltd. + * Copyright (C) 2018-2022 Linaro Ltd. */ #ifndef _IPA_QMI_MSG_H_ #define _IPA_QMI_MSG_H_ diff --git a/drivers/net/ipa/ipa_reg.c b/drivers/net/ipa/ipa_reg.c index e6147a1cd787..22f067741d9b 100644 --- a/drivers/net/ipa/ipa_reg.c +++ b/drivers/net/ipa/ipa_reg.c @@ -1,7 +1,7 @@ // SPDX-License-Identifier: GPL-2.0 /* Copyright (c) 2012-2018, The Linux Foundation. All rights reserved. - * Copyright (C) 2019-2020 Linaro Ltd. + * Copyright (C) 2019-2022 Linaro Ltd. */ #include <linux/io.h> @@ -9,11 +9,105 @@ #include "ipa.h" #include "ipa_reg.h" +/* Is this register valid and defined for the current IPA version? */ +static bool ipa_reg_valid(struct ipa *ipa, enum ipa_reg_id reg_id) +{ + enum ipa_version version = ipa->version; + bool valid; + + /* Check for bogus (out of range) register IDs */ + if ((u32)reg_id >= ipa->regs->reg_count) + return false; + + switch (reg_id) { + case IPA_BCR: + case COUNTER_CFG: + valid = version < IPA_VERSION_4_5; + break; + + case IPA_TX_CFG: + case FLAVOR_0: + case IDLE_INDICATION_CFG: + valid = version >= IPA_VERSION_3_5; + break; + + case QTIME_TIMESTAMP_CFG: + case TIMERS_XO_CLK_DIV_CFG: + case TIMERS_PULSE_GRAN_CFG: + valid = version >= IPA_VERSION_4_5; + break; + + case SRC_RSRC_GRP_45_RSRC_TYPE: + case DST_RSRC_GRP_45_RSRC_TYPE: + valid = version <= IPA_VERSION_3_1 || + version == IPA_VERSION_4_5; + break; + + case SRC_RSRC_GRP_67_RSRC_TYPE: + case DST_RSRC_GRP_67_RSRC_TYPE: + valid = version <= IPA_VERSION_3_1; + break; + + case ENDP_FILTER_ROUTER_HSH_CFG: + valid = version != IPA_VERSION_4_2; + break; + + case IRQ_SUSPEND_EN: + case IRQ_SUSPEND_CLR: + valid = version >= IPA_VERSION_3_1; + break; + + default: + valid = true; /* Others should be defined for all versions */ + break; + } + + /* To be valid, it must be defined */ + + return valid && ipa->regs->reg[reg_id]; +} + +const struct ipa_reg *ipa_reg(struct ipa *ipa, enum ipa_reg_id reg_id) +{ + if (WARN_ON(!ipa_reg_valid(ipa, reg_id))) + return NULL; + + return ipa->regs->reg[reg_id]; +} + +static const struct ipa_regs *ipa_regs(enum ipa_version version) +{ + switch (version) { + case IPA_VERSION_3_1: + return &ipa_regs_v3_1; + case IPA_VERSION_3_5_1: + return &ipa_regs_v3_5_1; + case IPA_VERSION_4_2: + return &ipa_regs_v4_2; + case IPA_VERSION_4_5: + return &ipa_regs_v4_5; + case IPA_VERSION_4_9: + return &ipa_regs_v4_9; + case IPA_VERSION_4_11: + return &ipa_regs_v4_11; + default: + return NULL; + } +} + int ipa_reg_init(struct ipa *ipa) { struct device *dev = &ipa->pdev->dev; + const struct ipa_regs *regs; struct resource *res; + regs = ipa_regs(ipa->version); + if (!regs) + return -EINVAL; + + if (WARN_ON(regs->reg_count > IPA_REG_ID_COUNT)) + return -EINVAL; + /* Setup IPA register memory */ res = platform_get_resource_byname(ipa->pdev, IORESOURCE_MEM, "ipa-reg"); @@ -28,6 +122,7 @@ int ipa_reg_init(struct ipa *ipa) return -ENOMEM; } ipa->reg_addr = res->start; + ipa->regs = regs; return 0; } diff --git a/drivers/net/ipa/ipa_reg.h b/drivers/net/ipa/ipa_reg.h index 6f35438cda89..7bf70f70f63f 100644 --- a/drivers/net/ipa/ipa_reg.h +++ b/drivers/net/ipa/ipa_reg.h @@ -1,12 +1,13 @@ /* SPDX-License-Identifier: GPL-2.0 */ /* Copyright (c) 2012-2018, The Linux Foundation. All rights reserved. - * Copyright (C) 2018-2021 Linaro Ltd. + * Copyright (C) 2018-2022 Linaro Ltd. */ #ifndef _IPA_REG_H_ #define _IPA_REG_H_ #include <linux/bitfield.h> +#include <linux/bug.h> #include "ipa_version.h" @@ -16,304 +17,325 @@ struct ipa; * DOC: IPA Registers * * IPA registers are located within the "ipa-reg" address space defined by - * Device Tree. The offset of each register within that space is specified - * by symbols defined below. The address space is mapped to virtual memory - * space in ipa_mem_init(). All IPA registers are 32 bits wide. + * Device Tree. Each register has a specified offset within that space, + * which is mapped into virtual memory space in ipa_mem_init(). Each + * has a unique identifer, taken from the ipa_reg_id enumerated type. + * All IPA registers are 32 bits wide. * - * Certain register types are duplicated for a number of instances of - * something. For example, each IPA endpoint has an set of registers - * defining its configuration. The offset to an endpoint's set of registers - * is computed based on an "base" offset, plus an endpoint's ID multiplied - * and a "stride" value for the register. For such registers, the offset is - * computed by a function-like macro that takes a parameter used in the - * computation. + * Certain "parameterized" register types are duplicated for a number of + * instances of something. For example, each IPA endpoint has an set of + * registers defining its configuration. The offset to an endpoint's set + * of registers is computed based on an "base" offset, plus an endpoint's + * ID multiplied and a "stride" value for the register. Similarly, some + * registers have an offset that depends on execution environment. In + * this case, the stride is multiplied by a member of the gsi_ee_id + * enumerated type. * - * Some register offsets depend on execution environment. For these an "ee" - * parameter is supplied to the offset macro. The "ee" value is a member of - * the gsi_ee enumerated type. + * Each version of IPA implements an array of ipa_reg structures indexed + * by register ID. Each entry in the array specifies the base offset and + * (for parameterized registers) a non-zero stride value. Not all versions + * of IPA define all registers. The offset for a register is returned by + * ipa_reg_offset() when the register's ipa_reg structure is supplied; + * zero is returned for an undefined register (this should never happen). * - * The offset of a register dependent on endpoint ID is computed by a macro - * that is supplied a parameter "ep", "txep", or "rxep". A register with an - * "ep" parameter is valid for any endpoint; a register with a "txep" or - * "rxep" parameter is valid only for TX or RX endpoints, respectively. The - * "*ep" value is assumed to be less than the maximum valid endpoint ID - * for the current hardware, and that will not exceed IPA_ENDPOINT_MAX. - * - * The offset of registers related to filter and route tables is computed - * by a macro that is supplied a parameter "er". The "er" represents an - * endpoint ID for filters, or a route ID for routes. For filters, the - * endpoint ID must be less than IPA_ENDPOINT_MAX, but is further restricted - * because not all endpoints support filtering. For routes, the route ID - * must be less than IPA_ROUTE_MAX. - * - * The offset of registers related to resource types is computed by a macro - * that is supplied a parameter "rt". The "rt" represents a resource type, - * which is a member of the ipa_resource_type_src enumerated type for - * source endpoint resources or the ipa_resource_type_dst enumerated type - * for destination endpoint resources. - * - * Some registers encode multiple fields within them. For these, each field - * has a symbol below defining a field mask that encodes both the position - * and width of the field within its register. - * - * In some cases, different versions of IPA hardware use different offset or - * field mask values. In such cases an inline_function(ipa) is used rather - * than a MACRO to define the offset or field mask to use. - * - * Finally, some registers hold bitmasks representing endpoints. In such - * cases the @available field in the @ipa structure defines the "full" set - * of valid bits for the register. + * Some registers encode multiple fields within them. Each field in + * such a register has a unique identifier (from an enumerated type). + * The position and width of the fields in a register are defined by + * an array of field masks, indexed by field ID. Two functions are + * used to access register fields; both take an ipa_reg structure as + * argument. To encode a value to be represented in a register field, + * the value and field ID are passed to ipa_reg_encode(). To extract + * a value encoded in a register field, the field ID is passed to + * ipa_reg_decode(). In addition, for single-bit fields, ipa_reg_bit() + * can be used to either encode the bit value, or to generate a mask + * used to extract the bit value. */ -#define IPA_REG_COMP_CFG_OFFSET 0x0000003c -/* The next field is not supported for IPA v4.0+, not present for IPA v4.5+ */ -#define ENABLE_FMASK GENMASK(0, 0) -/* The next field is present for IPA v4.7+ */ -#define RAM_ARB_PRI_CLIENT_SAMP_FIX_DIS_FMASK GENMASK(0, 0) -#define GSI_SNOC_BYPASS_DIS_FMASK GENMASK(1, 1) -#define GEN_QMB_0_SNOC_BYPASS_DIS_FMASK GENMASK(2, 2) -#define GEN_QMB_1_SNOC_BYPASS_DIS_FMASK GENMASK(3, 3) -/* The next field is not present for IPA v4.5+ */ -#define IPA_DCMP_FAST_CLK_EN_FMASK GENMASK(4, 4) -/* The next twelve fields are present for IPA v4.0+ */ -#define IPA_QMB_SELECT_CONS_EN_FMASK GENMASK(5, 5) -#define IPA_QMB_SELECT_PROD_EN_FMASK GENMASK(6, 6) -#define GSI_MULTI_INORDER_RD_DIS_FMASK GENMASK(7, 7) -#define GSI_MULTI_INORDER_WR_DIS_FMASK GENMASK(8, 8) -#define GEN_QMB_0_MULTI_INORDER_RD_DIS_FMASK GENMASK(9, 9) -#define GEN_QMB_1_MULTI_INORDER_RD_DIS_FMASK GENMASK(10, 10) -#define GEN_QMB_0_MULTI_INORDER_WR_DIS_FMASK GENMASK(11, 11) -#define GEN_QMB_1_MULTI_INORDER_WR_DIS_FMASK GENMASK(12, 12) -#define GEN_QMB_0_SNOC_CNOC_LOOP_PROT_DIS_FMASK GENMASK(13, 13) -#define GSI_SNOC_CNOC_LOOP_PROT_DISABLE_FMASK GENMASK(14, 14) -#define GSI_MULTI_AXI_MASTERS_DIS_FMASK GENMASK(15, 15) -#define IPA_QMB_SELECT_GLOBAL_EN_FMASK GENMASK(16, 16) -/* The next five fields are present for IPA v4.9+ */ -#define QMB_RAM_RD_CACHE_DISABLE_FMASK GENMASK(19, 19) -#define GENQMB_AOOOWR_FMASK GENMASK(20, 20) -#define IF_OUT_OF_BUF_STOP_RESET_MASK_EN_FMASK GENMASK(21, 21) -#define GEN_QMB_1_DYNAMIC_ASIZE_FMASK GENMASK(30, 30) -#define GEN_QMB_0_DYNAMIC_ASIZE_FMASK GENMASK(31, 31) - -/* Encoded value for COMP_CFG register ATOMIC_FETCHER_ARB_LOCK_DIS field */ -static inline u32 arbitration_lock_disable_encoded(enum ipa_version version, - u32 mask) -{ - WARN_ON(version < IPA_VERSION_4_0); +/* enum ipa_reg_id - IPA register IDs */ +enum ipa_reg_id { + COMP_CFG, + CLKON_CFG, + ROUTE, + SHARED_MEM_SIZE, + QSB_MAX_WRITES, + QSB_MAX_READS, + FILT_ROUT_HASH_EN, + FILT_ROUT_HASH_FLUSH, + STATE_AGGR_ACTIVE, + IPA_BCR, /* Not IPA v4.5+ */ + LOCAL_PKT_PROC_CNTXT, + AGGR_FORCE_CLOSE, + COUNTER_CFG, /* Not IPA v4.5+ */ + IPA_TX_CFG, /* IPA v3.5+ */ + FLAVOR_0, /* IPA v3.5+ */ + IDLE_INDICATION_CFG, /* IPA v3.5+ */ + QTIME_TIMESTAMP_CFG, /* IPA v4.5+ */ + TIMERS_XO_CLK_DIV_CFG, /* IPA v4.5+ */ + TIMERS_PULSE_GRAN_CFG, /* IPA v4.5+ */ + SRC_RSRC_GRP_01_RSRC_TYPE, + SRC_RSRC_GRP_23_RSRC_TYPE, + SRC_RSRC_GRP_45_RSRC_TYPE, /* Not IPA v3.5+, IPA v4.5 */ + SRC_RSRC_GRP_67_RSRC_TYPE, /* Not IPA v3.5+ */ + DST_RSRC_GRP_01_RSRC_TYPE, + DST_RSRC_GRP_23_RSRC_TYPE, + DST_RSRC_GRP_45_RSRC_TYPE, /* Not IPA v3.5+, IPA v4.5 */ + DST_RSRC_GRP_67_RSRC_TYPE, /* Not IPA v3.5+ */ + ENDP_INIT_CTRL, /* Not IPA v4.2+ for TX, not IPA v4.0+ for RX */ + ENDP_INIT_CFG, + ENDP_INIT_NAT, /* TX only */ + ENDP_INIT_HDR, + ENDP_INIT_HDR_EXT, + ENDP_INIT_HDR_METADATA_MASK, /* RX only */ + ENDP_INIT_MODE, /* TX only */ + ENDP_INIT_AGGR, + ENDP_INIT_HOL_BLOCK_EN, /* RX only */ + ENDP_INIT_HOL_BLOCK_TIMER, /* RX only */ + ENDP_INIT_DEAGGR, /* TX only */ + ENDP_INIT_RSRC_GRP, + ENDP_INIT_SEQ, /* TX only */ + ENDP_STATUS, + ENDP_FILTER_ROUTER_HSH_CFG, /* Not IPA v4.2 */ + /* The IRQ registers are only used for GSI_EE_AP */ + IPA_IRQ_STTS, + IPA_IRQ_EN, + IPA_IRQ_CLR, + IPA_IRQ_UC, + IRQ_SUSPEND_INFO, + IRQ_SUSPEND_EN, /* IPA v3.1+ */ + IRQ_SUSPEND_CLR, /* IPA v3.1+ */ + IPA_REG_ID_COUNT, /* Last; not an ID */ +}; - if (version < IPA_VERSION_4_9) - return u32_encode_bits(mask, GENMASK(20, 17)); +/** + * struct ipa_reg - An IPA register descriptor + * @offset: Register offset relative to base of the "ipa-reg" memory + * @stride: Distance between two instances, if parameterized + * @fcount: Number of entries in the @fmask array + * @fmask: Array of mask values defining position and width of fields + * @name: Upper-case name of the IPA register + */ +struct ipa_reg { + u32 offset; + u32 stride; + u32 fcount; + const u32 *fmask; /* BIT(nr) or GENMASK(h, l) */ + const char *name; +}; - if (version == IPA_VERSION_4_9) - return u32_encode_bits(mask, GENMASK(24, 22)); +/* Helper macro for defining "simple" (non-parameterized) registers */ +#define IPA_REG(__NAME, __reg_id, __offset) \ + IPA_REG_STRIDE(__NAME, __reg_id, __offset, 0) - return u32_encode_bits(mask, GENMASK(23, 22)); -} +/* Helper macro for defining parameterized registers, specifying stride */ +#define IPA_REG_STRIDE(__NAME, __reg_id, __offset, __stride) \ + static const struct ipa_reg ipa_reg_ ## __reg_id = { \ + .name = #__NAME, \ + .offset = __offset, \ + .stride = __stride, \ + } -/* Encoded value for COMP_CFG register FULL_FLUSH_WAIT_RS_CLOSURE_EN field */ -static inline u32 full_flush_rsc_closure_en_encoded(enum ipa_version version, - bool enable) -{ - u32 val = enable ? 1 : 0; +#define IPA_REG_FIELDS(__NAME, __name, __offset) \ + IPA_REG_STRIDE_FIELDS(__NAME, __name, __offset, 0) - WARN_ON(version < IPA_VERSION_4_5); +#define IPA_REG_STRIDE_FIELDS(__NAME, __name, __offset, __stride) \ + static const struct ipa_reg ipa_reg_ ## __name = { \ + .name = #__NAME, \ + .offset = __offset, \ + .stride = __stride, \ + .fcount = ARRAY_SIZE(ipa_reg_ ## __name ## _fmask), \ + .fmask = ipa_reg_ ## __name ## _fmask, \ + } - if (version == IPA_VERSION_4_5 || version == IPA_VERSION_4_7) - return u32_encode_bits(val, GENMASK(21, 21)); +/** + * struct ipa_regs - Description of registers supported by hardware + * @reg_count: Number of registers in the @reg[] array + * @reg: Array of register descriptors + */ +struct ipa_regs { + u32 reg_count; + const struct ipa_reg **reg; +}; - return u32_encode_bits(val, GENMASK(17, 17)); -} +/* COMP_CFG register */ +enum ipa_reg_comp_cfg_field_id { + COMP_CFG_ENABLE, /* Not IPA v4.0+ */ + RAM_ARB_PRI_CLIENT_SAMP_FIX_DIS, /* IPA v4.7+ */ + GSI_SNOC_BYPASS_DIS, + GEN_QMB_0_SNOC_BYPASS_DIS, + GEN_QMB_1_SNOC_BYPASS_DIS, + IPA_DCMP_FAST_CLK_EN, /* Not IPA v4.5+ */ + IPA_QMB_SELECT_CONS_EN, /* IPA v4.0+ */ + IPA_QMB_SELECT_PROD_EN, /* IPA v4.0+ */ + GSI_MULTI_INORDER_RD_DIS, /* IPA v4.0+ */ + GSI_MULTI_INORDER_WR_DIS, /* IPA v4.0+ */ + GEN_QMB_0_MULTI_INORDER_RD_DIS, /* IPA v4.0+ */ + GEN_QMB_1_MULTI_INORDER_RD_DIS, /* IPA v4.0+ */ + GEN_QMB_0_MULTI_INORDER_WR_DIS, /* IPA v4.0+ */ + GEN_QMB_1_MULTI_INORDER_WR_DIS, /* IPA v4.0+ */ + GEN_QMB_0_SNOC_CNOC_LOOP_PROT_DIS, /* IPA v4.0+ */ + GSI_SNOC_CNOC_LOOP_PROT_DISABLE, /* IPA v4.0+ */ + GSI_MULTI_AXI_MASTERS_DIS, /* IPA v4.0+ */ + IPA_QMB_SELECT_GLOBAL_EN, /* IPA v4.0+ */ + QMB_RAM_RD_CACHE_DISABLE, /* IPA v4.9+ */ + GENQMB_AOOOWR, /* IPA v4.9+ */ + IF_OUT_OF_BUF_STOP_RESET_MASK_EN, /* IPA v4.9+ */ + GEN_QMB_1_DYNAMIC_ASIZE, /* IPA v4.9+ */ + GEN_QMB_0_DYNAMIC_ASIZE, /* IPA v4.9+ */ + ATOMIC_FETCHER_ARB_LOCK_DIS, /* IPA v4.0+ */ + FULL_FLUSH_WAIT_RS_CLOSURE_EN, /* IPA v4.5+ */ +}; -#define IPA_REG_CLKON_CFG_OFFSET 0x00000044 -#define RX_FMASK GENMASK(0, 0) -#define PROC_FMASK GENMASK(1, 1) -#define TX_WRAPPER_FMASK GENMASK(2, 2) -#define MISC_FMASK GENMASK(3, 3) -#define RAM_ARB_FMASK GENMASK(4, 4) -#define FTCH_HPS_FMASK GENMASK(5, 5) -#define FTCH_DPS_FMASK GENMASK(6, 6) -#define HPS_FMASK GENMASK(7, 7) -#define DPS_FMASK GENMASK(8, 8) -#define RX_HPS_CMDQS_FMASK GENMASK(9, 9) -#define HPS_DPS_CMDQS_FMASK GENMASK(10, 10) -#define DPS_TX_CMDQS_FMASK GENMASK(11, 11) -#define RSRC_MNGR_FMASK GENMASK(12, 12) -#define CTX_HANDLER_FMASK GENMASK(13, 13) -#define ACK_MNGR_FMASK GENMASK(14, 14) -#define D_DCPH_FMASK GENMASK(15, 15) -#define H_DCPH_FMASK GENMASK(16, 16) -/* The next field is not present for IPA v4.5+ */ -#define DCMP_FMASK GENMASK(17, 17) -/* The next three fields are present for IPA v3.5+ */ -#define NTF_TX_CMDQS_FMASK GENMASK(18, 18) -#define TX_0_FMASK GENMASK(19, 19) -#define TX_1_FMASK GENMASK(20, 20) -/* The next field is present for IPA v3.5.1+ */ -#define FNR_FMASK GENMASK(21, 21) -/* The next eight fields are present for IPA v4.0+ */ -#define QSB2AXI_CMDQ_L_FMASK GENMASK(22, 22) -#define AGGR_WRAPPER_FMASK GENMASK(23, 23) -#define RAM_SLAVEWAY_FMASK GENMASK(24, 24) -#define QMB_FMASK GENMASK(25, 25) -#define WEIGHT_ARB_FMASK GENMASK(26, 26) -#define GSI_IF_FMASK GENMASK(27, 27) -#define GLOBAL_FMASK GENMASK(28, 28) -#define GLOBAL_2X_CLK_FMASK GENMASK(29, 29) -/* The next field is present for IPA v4.5+ */ -#define DPL_FIFO_FMASK GENMASK(30, 30) -/* The next field is present for IPA v4.7+ */ -#define DRBIP_FMASK GENMASK(31, 31) - -#define IPA_REG_ROUTE_OFFSET 0x00000048 -#define ROUTE_DIS_FMASK GENMASK(0, 0) -#define ROUTE_DEF_PIPE_FMASK GENMASK(5, 1) -#define ROUTE_DEF_HDR_TABLE_FMASK GENMASK(6, 6) -#define ROUTE_DEF_HDR_OFST_FMASK GENMASK(16, 7) -#define ROUTE_FRAG_DEF_PIPE_FMASK GENMASK(21, 17) -#define ROUTE_DEF_RETAIN_HDR_FMASK GENMASK(24, 24) - -#define IPA_REG_SHARED_MEM_SIZE_OFFSET 0x00000054 -#define SHARED_MEM_SIZE_FMASK GENMASK(15, 0) -#define SHARED_MEM_BADDR_FMASK GENMASK(31, 16) - -#define IPA_REG_QSB_MAX_WRITES_OFFSET 0x00000074 -#define GEN_QMB_0_MAX_WRITES_FMASK GENMASK(3, 0) -#define GEN_QMB_1_MAX_WRITES_FMASK GENMASK(7, 4) - -#define IPA_REG_QSB_MAX_READS_OFFSET 0x00000078 -#define GEN_QMB_0_MAX_READS_FMASK GENMASK(3, 0) -#define GEN_QMB_1_MAX_READS_FMASK GENMASK(7, 4) -/* The next two fields are present for IPA v4.0+ */ -#define GEN_QMB_0_MAX_READS_BEATS_FMASK GENMASK(23, 16) -#define GEN_QMB_1_MAX_READS_BEATS_FMASK GENMASK(31, 24) - -static inline u32 ipa_reg_filt_rout_hash_en_offset(enum ipa_version version) -{ - if (version < IPA_VERSION_4_0) - return 0x000008c; +/* CLKON_CFG register */ +enum ipa_reg_clkon_cfg_field_id { + CLKON_RX, + CLKON_PROC, + TX_WRAPPER, + CLKON_MISC, + RAM_ARB, + FTCH_HPS, + FTCH_DPS, + CLKON_HPS, + CLKON_DPS, + RX_HPS_CMDQS, + HPS_DPS_CMDQS, + DPS_TX_CMDQS, + RSRC_MNGR, + CTX_HANDLER, + ACK_MNGR, + D_DCPH, + H_DCPH, + CLKON_DCMP, /* IPA v4.5+ */ + NTF_TX_CMDQS, /* IPA v3.5+ */ + CLKON_TX_0, /* IPA v3.5+ */ + CLKON_TX_1, /* IPA v3.5+ */ + CLKON_FNR, /* IPA v3.5.1+ */ + QSB2AXI_CMDQ_L, /* IPA v4.0+ */ + AGGR_WRAPPER, /* IPA v4.0+ */ + RAM_SLAVEWAY, /* IPA v4.0+ */ + CLKON_QMB, /* IPA v4.0+ */ + WEIGHT_ARB, /* IPA v4.0+ */ + GSI_IF, /* IPA v4.0+ */ + CLKON_GLOBAL, /* IPA v4.0+ */ + GLOBAL_2X_CLK, /* IPA v4.0+ */ + DPL_FIFO, /* IPA v4.5+ */ + DRBIP, /* IPA v4.7+ */ +}; - return 0x0000148; -} +/* ROUTE register */ +enum ipa_reg_route_field_id { + ROUTE_DIS, + ROUTE_DEF_PIPE, + ROUTE_DEF_HDR_TABLE, + ROUTE_DEF_HDR_OFST, + ROUTE_FRAG_DEF_PIPE, + ROUTE_DEF_RETAIN_HDR, +}; -static inline u32 ipa_reg_filt_rout_hash_flush_offset(enum ipa_version version) -{ - if (version < IPA_VERSION_4_0) - return 0x0000090; +/* SHARED_MEM_SIZE register */ +enum ipa_reg_shared_mem_size_field_id { + MEM_SIZE, + MEM_BADDR, +}; - return 0x000014c; -} +/* QSB_MAX_WRITES register */ +enum ipa_reg_qsb_max_writes_field_id { + GEN_QMB_0_MAX_WRITES, + GEN_QMB_1_MAX_WRITES, +}; -/* The next four fields are used for the hash enable and flush registers */ -#define IPV6_ROUTER_HASH_FMASK GENMASK(0, 0) -#define IPV6_FILTER_HASH_FMASK GENMASK(4, 4) -#define IPV4_ROUTER_HASH_FMASK GENMASK(8, 8) -#define IPV4_FILTER_HASH_FMASK GENMASK(12, 12) +/* QSB_MAX_READS register */ +enum ipa_reg_qsb_max_reads_field_id { + GEN_QMB_0_MAX_READS, + GEN_QMB_1_MAX_READS, + GEN_QMB_0_MAX_READS_BEATS, /* IPA v4.0+ */ + GEN_QMB_1_MAX_READS_BEATS, /* IPA v4.0+ */ +}; -/* ipa->available defines the valid bits in the STATE_AGGR_ACTIVE register */ -static inline u32 ipa_reg_state_aggr_active_offset(enum ipa_version version) -{ - if (version < IPA_VERSION_4_0) - return 0x0000010c; +/* FILT_ROUT_HASH_EN and FILT_ROUT_HASH_FLUSH registers */ +enum ipa_reg_rout_hash_field_id { + IPV6_ROUTER_HASH, + IPV6_FILTER_HASH, + IPV4_ROUTER_HASH, + IPV4_FILTER_HASH, +}; - return 0x000000b4; -} +/* BCR register */ +enum ipa_bcr_compat { + BCR_CMDQ_L_LACK_ONE_ENTRY = 0x0, /* Not IPA v4.2+ */ + BCR_TX_NOT_USING_BRESP = 0x1, /* Not IPA v4.2+ */ + BCR_TX_SUSPEND_IRQ_ASSERT_ONCE = 0x2, /* Not IPA v4.0+ */ + BCR_SUSPEND_L2_IRQ = 0x3, /* Not IPA v4.2+ */ + BCR_HOLB_DROP_L2_IRQ = 0x4, /* Not IPA v4.2+ */ + BCR_DUAL_TX = 0x5, /* IPA v3.5+ */ + BCR_ENABLE_FILTER_DATA_CACHE = 0x6, /* IPA v3.5+ */ + BCR_NOTIF_PRIORITY_OVER_ZLT = 0x7, /* IPA v3.5+ */ + BCR_FILTER_PREFETCH_EN = 0x8, /* IPA v3.5+ */ + BCR_ROUTER_PREFETCH_EN = 0x9, /* IPA v3.5+ */ +}; -/* The next register is not present for IPA v4.5+ */ -#define IPA_REG_BCR_OFFSET 0x000001d0 -/* The next two fields are not present for IPA v4.2+ */ -#define BCR_CMDQ_L_LACK_ONE_ENTRY_FMASK GENMASK(0, 0) -#define BCR_TX_NOT_USING_BRESP_FMASK GENMASK(1, 1) -/* The next field is invalid for IPA v4.0+ */ -#define BCR_TX_SUSPEND_IRQ_ASSERT_ONCE_FMASK GENMASK(2, 2) -/* The next two fields are not present for IPA v4.2+ */ -#define BCR_SUSPEND_L2_IRQ_FMASK GENMASK(3, 3) -#define BCR_HOLB_DROP_L2_IRQ_FMASK GENMASK(4, 4) -/* The next five fields are present for IPA v3.5+ */ -#define BCR_DUAL_TX_FMASK GENMASK(5, 5) -#define BCR_ENABLE_FILTER_DATA_CACHE_FMASK GENMASK(6, 6) -#define BCR_NOTIF_PRIORITY_OVER_ZLT_FMASK GENMASK(7, 7) -#define BCR_FILTER_PREFETCH_EN_FMASK GENMASK(8, 8) -#define BCR_ROUTER_PREFETCH_EN_FMASK GENMASK(9, 9) - -/* The value of the next register must be a multiple of 8 (bottom 3 bits 0) */ -#define IPA_REG_LOCAL_PKT_PROC_CNTXT_OFFSET 0x000001e8 - -/* Encoded value for LOCAL_PKT_PROC_CNTXT register BASE_ADDR field */ -static inline u32 proc_cntxt_base_addr_encoded(enum ipa_version version, - u32 addr) -{ - if (version < IPA_VERSION_4_5) - return u32_encode_bits(addr, GENMASK(16, 0)); +/* LOCAL_PKT_PROC_CNTXT register */ +enum ipa_reg_local_pkt_proc_cntxt_field_id { + IPA_BASE_ADDR, +}; - return u32_encode_bits(addr, GENMASK(17, 0)); -} +/* COUNTER_CFG register */ +enum ipa_reg_counter_cfg_field_id { + EOT_COAL_GRANULARITY, /* Not v3.5+ */ + AGGR_GRANULARITY, +}; -/* ipa->available defines the valid bits in the AGGR_FORCE_CLOSE register */ -#define IPA_REG_AGGR_FORCE_CLOSE_OFFSET 0x000001ec - -/* The next register is not present for IPA v4.5+ */ -#define IPA_REG_COUNTER_CFG_OFFSET 0x000001f0 -/* The next field is not present for IPA v3.5+ */ -#define EOT_COAL_GRANULARITY GENMASK(3, 0) -#define AGGR_GRANULARITY_FMASK GENMASK(8, 4) - -/* The next register is present for IPA v3.5+ */ -#define IPA_REG_TX_CFG_OFFSET 0x000001fc -/* The next three fields are not present for IPA v4.0+ */ -#define TX0_PREFETCH_DISABLE_FMASK GENMASK(0, 0) -#define TX1_PREFETCH_DISABLE_FMASK GENMASK(1, 1) -#define PREFETCH_ALMOST_EMPTY_SIZE_FMASK GENMASK(4, 2) -/* The next six fields are present for IPA v4.0+ */ -#define PREFETCH_ALMOST_EMPTY_SIZE_TX0_FMASK GENMASK(5, 2) -#define DMAW_SCND_OUTSD_PRED_THRESHOLD_FMASK GENMASK(9, 6) -#define DMAW_SCND_OUTSD_PRED_EN_FMASK GENMASK(10, 10) -#define DMAW_MAX_BEATS_256_DIS_FMASK GENMASK(11, 11) -#define PA_MASK_EN_FMASK GENMASK(12, 12) -#define PREFETCH_ALMOST_EMPTY_SIZE_TX1_FMASK GENMASK(16, 13) -/* The next field is present for IPA v4.5+ */ -#define DUAL_TX_ENABLE_FMASK GENMASK(17, 17) -/* The next field is present for IPA v4.2+, but not IPA v4.5 */ -#define SSPND_PA_NO_START_STATE_FMASK GENMASK(18, 18) -/* The next field is present for IPA v4.2 only */ -#define SSPND_PA_NO_BQ_STATE_FMASK GENMASK(19, 19) - -/* The next register is present for IPA v3.5+ */ -#define IPA_REG_FLAVOR_0_OFFSET 0x00000210 -#define IPA_MAX_PIPES_FMASK GENMASK(3, 0) -#define IPA_MAX_CONS_PIPES_FMASK GENMASK(12, 8) -#define IPA_MAX_PROD_PIPES_FMASK GENMASK(20, 16) -#define IPA_PROD_LOWEST_FMASK GENMASK(27, 24) - -/* The next register is present for IPA v3.5+ */ -static inline u32 ipa_reg_idle_indication_cfg_offset(enum ipa_version version) -{ - if (version >= IPA_VERSION_4_2) - return 0x00000240; +/* IPA_TX_CFG register */ +enum ipa_reg_ipa_tx_cfg_field_id { + TX0_PREFETCH_DISABLE, /* Not v4.0+ */ + TX1_PREFETCH_DISABLE, /* Not v4.0+ */ + PREFETCH_ALMOST_EMPTY_SIZE, /* Not v4.0+ */ + PREFETCH_ALMOST_EMPTY_SIZE_TX0, /* v4.0+ */ + DMAW_SCND_OUTSD_PRED_THRESHOLD, /* v4.0+ */ + DMAW_SCND_OUTSD_PRED_EN, /* v4.0+ */ + DMAW_MAX_BEATS_256_DIS, /* v4.0+ */ + PA_MASK_EN, /* v4.0+ */ + PREFETCH_ALMOST_EMPTY_SIZE_TX1, /* v4.0+ */ + DUAL_TX_ENABLE, /* v4.5+ */ + SSPND_PA_NO_START_STATE, /* v4,2+, not v4.5 */ + SSPND_PA_NO_BQ_STATE, /* v4.2 only */ +}; - return 0x00000220; -} +/* FLAVOR_0 register */ +enum ipa_reg_flavor_0_field_id { + MAX_PIPES, + MAX_CONS_PIPES, + MAX_PROD_PIPES, + PROD_LOWEST, +}; + +/* IDLE_INDICATION_CFG register */ +enum ipa_reg_idle_indication_cfg_field_id { + ENTER_IDLE_DEBOUNCE_THRESH, + CONST_NON_IDLE_ENABLE, +}; + +/* QTIME_TIMESTAMP_CFG register */ +enum ipa_reg_qtime_timestamp_cfg_field_id { + DPL_TIMESTAMP_LSB, + DPL_TIMESTAMP_SEL, + TAG_TIMESTAMP_LSB, + NAT_TIMESTAMP_LSB, +}; -#define ENTER_IDLE_DEBOUNCE_THRESH_FMASK GENMASK(15, 0) -#define CONST_NON_IDLE_ENABLE_FMASK GENMASK(16, 16) - -/* The next register is present for IPA v4.5+ */ -#define IPA_REG_QTIME_TIMESTAMP_CFG_OFFSET 0x0000024c -#define DPL_TIMESTAMP_LSB_FMASK GENMASK(4, 0) -#define DPL_TIMESTAMP_SEL_FMASK GENMASK(7, 7) -#define TAG_TIMESTAMP_LSB_FMASK GENMASK(12, 8) -#define NAT_TIMESTAMP_LSB_FMASK GENMASK(20, 16) - -/* The next register is present for IPA v4.5+ */ -#define IPA_REG_TIMERS_XO_CLK_DIV_CFG_OFFSET 0x00000250 -#define DIV_VALUE_FMASK GENMASK(8, 0) -#define DIV_ENABLE_FMASK GENMASK(31, 31) - -/* The next register is present for IPA v4.5+ */ -#define IPA_REG_TIMERS_PULSE_GRAN_CFG_OFFSET 0x00000254 -#define GRAN_0_FMASK GENMASK(2, 0) -#define GRAN_1_FMASK GENMASK(5, 3) -#define GRAN_2_FMASK GENMASK(8, 6) -/* Values for GRAN_x fields of TIMERS_PULSE_GRAN_CFG */ +/* TIMERS_XO_CLK_DIV_CFG register */ +enum ipa_reg_timers_xo_clk_div_cfg_field_id { + DIV_VALUE, + DIV_ENABLE, +}; + +/* TIMERS_PULSE_GRAN_CFG register */ +enum ipa_reg_timers_pulse_gran_cfg_field_id { + PULSE_GRAN_0, + PULSE_GRAN_1, + PULSE_GRAN_2, +}; + +/* Values for IPA_GRAN_x fields of TIMERS_PULSE_GRAN_CFG */ enum ipa_pulse_gran { IPA_GRAN_10_US = 0x0, IPA_GRAN_20_US = 0x1, @@ -325,267 +347,160 @@ enum ipa_pulse_gran { IPA_GRAN_655350_US = 0x7, }; -/* Not all of the following are present (depends on IPA version) */ -#define IPA_REG_SRC_RSRC_GRP_01_RSRC_TYPE_N_OFFSET(rt) \ - (0x00000400 + 0x0020 * (rt)) -#define IPA_REG_SRC_RSRC_GRP_23_RSRC_TYPE_N_OFFSET(rt) \ - (0x00000404 + 0x0020 * (rt)) -#define IPA_REG_SRC_RSRC_GRP_45_RSRC_TYPE_N_OFFSET(rt) \ - (0x00000408 + 0x0020 * (rt)) -#define IPA_REG_SRC_RSRC_GRP_67_RSRC_TYPE_N_OFFSET(rt) \ - (0x0000040c + 0x0020 * (rt)) -#define IPA_REG_DST_RSRC_GRP_01_RSRC_TYPE_N_OFFSET(rt) \ - (0x00000500 + 0x0020 * (rt)) -#define IPA_REG_DST_RSRC_GRP_23_RSRC_TYPE_N_OFFSET(rt) \ - (0x00000504 + 0x0020 * (rt)) -#define IPA_REG_DST_RSRC_GRP_45_RSRC_TYPE_N_OFFSET(rt) \ - (0x00000508 + 0x0020 * (rt)) -#define IPA_REG_DST_RSRC_GRP_67_RSRC_TYPE_N_OFFSET(rt) \ - (0x0000050c + 0x0020 * (rt)) -/* The next four fields are used for all resource group registers */ -#define X_MIN_LIM_FMASK GENMASK(5, 0) -#define X_MAX_LIM_FMASK GENMASK(13, 8) -/* The next two fields are not always present (if resource count is odd) */ -#define Y_MIN_LIM_FMASK GENMASK(21, 16) -#define Y_MAX_LIM_FMASK GENMASK(29, 24) - -#define IPA_REG_ENDP_INIT_CTRL_N_OFFSET(ep) \ - (0x00000800 + 0x0070 * (ep)) -/* Valid only for RX (IPA producer) endpoints (do not use for IPA v4.0+) */ -#define ENDP_SUSPEND_FMASK GENMASK(0, 0) -/* Valid only for TX (IPA consumer) endpoints */ -#define ENDP_DELAY_FMASK GENMASK(1, 1) - -#define IPA_REG_ENDP_INIT_CFG_N_OFFSET(ep) \ - (0x00000808 + 0x0070 * (ep)) -#define FRAG_OFFLOAD_EN_FMASK GENMASK(0, 0) -#define CS_OFFLOAD_EN_FMASK GENMASK(2, 1) -#define CS_METADATA_HDR_OFFSET_FMASK GENMASK(6, 3) -#define CS_GEN_QMB_MASTER_SEL_FMASK GENMASK(8, 8) +/* {SRC,DST}_RSRC_GRP_{01,23,45,67}_RSRC_TYPE registers */ +enum ipa_reg_rsrc_grp_rsrc_type_field_id { + X_MIN_LIM, + X_MAX_LIM, + Y_MIN_LIM, + Y_MAX_LIM, +}; + +/* ENDP_INIT_CTRL register */ +enum ipa_reg_endp_init_ctrl_field_id { + ENDP_SUSPEND, /* Not v4.0+ */ + ENDP_DELAY, /* Not v4.2+ */ +}; + +/* ENDP_INIT_CFG register */ +enum ipa_reg_endp_init_cfg_field_id { + FRAG_OFFLOAD_EN, + CS_OFFLOAD_EN, + CS_METADATA_HDR_OFFSET, + CS_GEN_QMB_MASTER_SEL, +}; /** enum ipa_cs_offload_en - ENDP_INIT_CFG register CS_OFFLOAD_EN field value */ enum ipa_cs_offload_en { - IPA_CS_OFFLOAD_NONE = 0x0, - IPA_CS_OFFLOAD_UL = 0x1, /* Before IPA v4.5 (TX) */ - IPA_CS_OFFLOAD_DL = 0x2, /* Before IPA v4.5 (RX) */ - IPA_CS_OFFLOAD_INLINE = 0x1, /* IPA v4.5 (TX and RX) */ + IPA_CS_OFFLOAD_NONE = 0x0, + IPA_CS_OFFLOAD_UL /* TX */ = 0x1, /* Not IPA v4.5+ */ + IPA_CS_OFFLOAD_DL /* RX */ = 0x2, /* Not IPA v4.5+ */ + IPA_CS_OFFLOAD_INLINE /* TX and RX */ = 0x1, /* IPA v4.5+ */ }; -/* Valid only for TX (IPA consumer) endpoints */ -#define IPA_REG_ENDP_INIT_NAT_N_OFFSET(ep) \ - (0x0000080c + 0x0070 * (ep)) -#define NAT_EN_FMASK GENMASK(1, 0) +/* ENDP_INIT_NAT register */ +enum ipa_reg_endp_init_nat_field_id { + NAT_EN, +}; /** enum ipa_nat_en - ENDP_INIT_NAT register NAT_EN field value */ enum ipa_nat_en { - IPA_NAT_BYPASS = 0x0, - IPA_NAT_SRC = 0x1, - IPA_NAT_DST = 0x2, -}; - -#define IPA_REG_ENDP_INIT_HDR_N_OFFSET(ep) \ - (0x00000810 + 0x0070 * (ep)) -#define HDR_LEN_FMASK GENMASK(5, 0) -#define HDR_OFST_METADATA_VALID_FMASK GENMASK(6, 6) -#define HDR_OFST_METADATA_FMASK GENMASK(12, 7) -#define HDR_ADDITIONAL_CONST_LEN_FMASK GENMASK(18, 13) -#define HDR_OFST_PKT_SIZE_VALID_FMASK GENMASK(19, 19) -#define HDR_OFST_PKT_SIZE_FMASK GENMASK(25, 20) -/* The next field is not present for IPA v4.9+ */ -#define HDR_A5_MUX_FMASK GENMASK(26, 26) -#define HDR_LEN_INC_DEAGG_HDR_FMASK GENMASK(27, 27) -/* The next field is not present for IPA v4.5+ */ -#define HDR_METADATA_REG_VALID_FMASK GENMASK(28, 28) -/* The next two fields are present for IPA v4.5+ */ -#define HDR_LEN_MSB_FMASK GENMASK(29, 28) -#define HDR_OFST_METADATA_MSB_FMASK GENMASK(31, 30) - -/* Encoded value for ENDP_INIT_HDR register HDR_LEN* field(s) */ -static inline u32 ipa_header_size_encoded(enum ipa_version version, - u32 header_size) -{ - u32 size = header_size & field_mask(HDR_LEN_FMASK); - u32 val; - - val = u32_encode_bits(size, HDR_LEN_FMASK); - if (version < IPA_VERSION_4_5) { - WARN_ON(header_size != size); - return val; - } - - /* IPA v4.5 adds a few more most-significant bits */ - size = header_size >> hweight32(HDR_LEN_FMASK); - val |= u32_encode_bits(size, HDR_LEN_MSB_FMASK); - - return val; -} - -/* Encoded value for ENDP_INIT_HDR register OFST_METADATA* field(s) */ -static inline u32 ipa_metadata_offset_encoded(enum ipa_version version, - u32 offset) -{ - u32 off = offset & field_mask(HDR_OFST_METADATA_FMASK); - u32 val; - - val = u32_encode_bits(off, HDR_OFST_METADATA_FMASK); - if (version < IPA_VERSION_4_5) { - WARN_ON(offset != off); - return val; - } + IPA_NAT_BYPASS = 0x0, + IPA_NAT_SRC = 0x1, + IPA_NAT_DST = 0x2, +}; - /* IPA v4.5 adds a few more most-significant bits */ - off = offset >> hweight32(HDR_OFST_METADATA_FMASK); - val |= u32_encode_bits(off, HDR_OFST_METADATA_MSB_FMASK); +/* ENDP_INIT_HDR register */ +enum ipa_reg_endp_init_hdr_field_id { + HDR_LEN, + HDR_OFST_METADATA_VALID, + HDR_OFST_METADATA, + HDR_ADDITIONAL_CONST_LEN, + HDR_OFST_PKT_SIZE_VALID, + HDR_OFST_PKT_SIZE, + HDR_A5_MUX, /* Not v4.9+ */ + HDR_LEN_INC_DEAGG_HDR, + HDR_METADATA_REG_VALID, /* Not v4.5+ */ + HDR_LEN_MSB, /* v4.5+ */ + HDR_OFST_METADATA_MSB, /* v4.5+ */ +}; - return val; -} +/* ENDP_INIT_HDR_EXT register */ +enum ipa_reg_endp_init_hdr_ext_field_id { + HDR_ENDIANNESS, + HDR_TOTAL_LEN_OR_PAD_VALID, + HDR_TOTAL_LEN_OR_PAD, + HDR_PAYLOAD_LEN_INC_PADDING, + HDR_TOTAL_LEN_OR_PAD_OFFSET, + HDR_PAD_TO_ALIGNMENT, + HDR_TOTAL_LEN_OR_PAD_OFFSET_MSB, /* v4.5+ */ + HDR_OFST_PKT_SIZE_MSB, /* v4.5+ */ + HDR_ADDITIONAL_CONST_LEN_MSB, /* v4.5+ */ +}; -#define IPA_REG_ENDP_INIT_HDR_EXT_N_OFFSET(ep) \ - (0x00000814 + 0x0070 * (ep)) -#define HDR_ENDIANNESS_FMASK GENMASK(0, 0) -#define HDR_TOTAL_LEN_OR_PAD_VALID_FMASK GENMASK(1, 1) -#define HDR_TOTAL_LEN_OR_PAD_FMASK GENMASK(2, 2) -#define HDR_PAYLOAD_LEN_INC_PADDING_FMASK GENMASK(3, 3) -#define HDR_TOTAL_LEN_OR_PAD_OFFSET_FMASK GENMASK(9, 4) -#define HDR_PAD_TO_ALIGNMENT_FMASK GENMASK(13, 10) -/* The next three fields are present for IPA v4.5+ */ -#define HDR_TOTAL_LEN_OR_PAD_OFFSET_MSB_FMASK GENMASK(17, 16) -#define HDR_OFST_PKT_SIZE_MSB_FMASK GENMASK(19, 18) -#define HDR_ADDITIONAL_CONST_LEN_MSB_FMASK GENMASK(21, 20) - -/* Valid only for RX (IPA producer) endpoints */ -#define IPA_REG_ENDP_INIT_HDR_METADATA_MASK_N_OFFSET(rxep) \ - (0x00000818 + 0x0070 * (rxep)) - -/* Valid only for TX (IPA consumer) endpoints */ -#define IPA_REG_ENDP_INIT_MODE_N_OFFSET(txep) \ - (0x00000820 + 0x0070 * (txep)) -#define MODE_FMASK GENMASK(2, 0) -/* The next field is present for IPA v4.5+ */ -#define DCPH_ENABLE_FMASK GENMASK(3, 3) -#define DEST_PIPE_INDEX_FMASK GENMASK(8, 4) -#define BYTE_THRESHOLD_FMASK GENMASK(27, 12) -#define PIPE_REPLICATION_EN_FMASK GENMASK(28, 28) -#define PAD_EN_FMASK GENMASK(29, 29) -/* The next field is not present for IPA v4.5+ */ -#define HDR_FTCH_DISABLE_FMASK GENMASK(30, 30) -/* The next field is present for IPA v4.9+ */ -#define DRBIP_ACL_ENABLE GENMASK(30, 30) +/* ENDP_INIT_MODE register */ +enum ipa_reg_endp_init_mode_field_id { + ENDP_MODE, + DCPH_ENABLE, /* v4.5+ */ + DEST_PIPE_INDEX, + BYTE_THRESHOLD, + PIPE_REPLICATION_EN, + PAD_EN, + HDR_FTCH_DISABLE, /* v4.5+ */ + DRBIP_ACL_ENABLE, /* v4.9+ */ +}; /** enum ipa_mode - ENDP_INIT_MODE register MODE field value */ enum ipa_mode { - IPA_BASIC = 0x0, - IPA_ENABLE_FRAMING_HDLC = 0x1, - IPA_ENABLE_DEFRAMING_HDLC = 0x2, - IPA_DMA = 0x3, + IPA_BASIC = 0x0, + IPA_ENABLE_FRAMING_HDLC = 0x1, + IPA_ENABLE_DEFRAMING_HDLC = 0x2, + IPA_DMA = 0x3, }; -#define IPA_REG_ENDP_INIT_AGGR_N_OFFSET(ep) \ - (0x00000824 + 0x0070 * (ep)) -#define AGGR_EN_FMASK GENMASK(1, 0) -#define AGGR_TYPE_FMASK GENMASK(4, 2) - -/* The legacy value is used for IPA hardware before IPA v4.5 */ -static inline u32 aggr_byte_limit_fmask(bool legacy) -{ - return legacy ? GENMASK(9, 5) : GENMASK(10, 5); -} - -/* The legacy value is used for IPA hardware before IPA v4.5 */ -static inline u32 aggr_time_limit_fmask(bool legacy) -{ - return legacy ? GENMASK(14, 10) : GENMASK(16, 12); -} - -/* The legacy value is used for IPA hardware before IPA v4.5 */ -static inline u32 aggr_pkt_limit_fmask(bool legacy) -{ - return legacy ? GENMASK(20, 15) : GENMASK(22, 17); -} - -/* The legacy value is used for IPA hardware before IPA v4.5 */ -static inline u32 aggr_sw_eof_active_fmask(bool legacy) -{ - return legacy ? GENMASK(21, 21) : GENMASK(23, 23); -} - -/* The legacy value is used for IPA hardware before IPA v4.5 */ -static inline u32 aggr_force_close_fmask(bool legacy) -{ - return legacy ? GENMASK(22, 22) : GENMASK(24, 24); -} - -/* The legacy value is used for IPA hardware before IPA v4.5 */ -static inline u32 aggr_hard_byte_limit_enable_fmask(bool legacy) -{ - return legacy ? GENMASK(24, 24) : GENMASK(26, 26); -} - -/* The next field is present for IPA v4.5+ */ -#define AGGR_GRAN_SEL_FMASK GENMASK(27, 27) +/* ENDP_INIT_AGGR register */ +enum ipa_reg_endp_init_aggr_field_id { + AGGR_EN, + AGGR_TYPE, + BYTE_LIMIT, + TIME_LIMIT, + PKT_LIMIT, + SW_EOF_ACTIVE, + FORCE_CLOSE, + HARD_BYTE_LIMIT_EN, + AGGR_GRAN_SEL, +}; /** enum ipa_aggr_en - ENDP_INIT_AGGR register AGGR_EN field value */ enum ipa_aggr_en { - IPA_BYPASS_AGGR = 0x0, /* (TX, RX) */ - IPA_ENABLE_AGGR = 0x1, /* (RX) */ - IPA_ENABLE_DEAGGR = 0x2, /* (TX) */ + IPA_BYPASS_AGGR /* TX and RX */ = 0x0, + IPA_ENABLE_AGGR /* RX */ = 0x1, + IPA_ENABLE_DEAGGR /* TX */ = 0x2, }; /** enum ipa_aggr_type - ENDP_INIT_AGGR register AGGR_TYPE field value */ enum ipa_aggr_type { - IPA_MBIM_16 = 0x0, - IPA_HDLC = 0x1, - IPA_TLP = 0x2, - IPA_RNDIS = 0x3, - IPA_GENERIC = 0x4, - IPA_COALESCE = 0x5, - IPA_QCMAP = 0x6, -}; - -/* Valid only for RX (IPA producer) endpoints */ -#define IPA_REG_ENDP_INIT_HOL_BLOCK_EN_N_OFFSET(rxep) \ - (0x0000082c + 0x0070 * (rxep)) -#define HOL_BLOCK_EN_FMASK GENMASK(0, 0) - -/* Valid only for RX (IPA producer) endpoints */ -#define IPA_REG_ENDP_INIT_HOL_BLOCK_TIMER_N_OFFSET(rxep) \ - (0x00000830 + 0x0070 * (rxep)) -/* The next two fields are present for IPA v4.2 only */ -#define BASE_VALUE_FMASK GENMASK(4, 0) -#define SCALE_FMASK GENMASK(12, 8) -/* The next two fields are present for IPA v4.5 */ -#define TIME_LIMIT_FMASK GENMASK(4, 0) -#define GRAN_SEL_FMASK GENMASK(8, 8) - -/* Valid only for TX (IPA consumer) endpoints */ -#define IPA_REG_ENDP_INIT_DEAGGR_N_OFFSET(txep) \ - (0x00000834 + 0x0070 * (txep)) -#define DEAGGR_HDR_LEN_FMASK GENMASK(5, 0) -#define SYSPIPE_ERR_DETECTION_FMASK GENMASK(6, 6) -#define PACKET_OFFSET_VALID_FMASK GENMASK(7, 7) -#define PACKET_OFFSET_LOCATION_FMASK GENMASK(13, 8) -#define IGNORE_MIN_PKT_ERR_FMASK GENMASK(14, 14) -#define MAX_PACKET_LEN_FMASK GENMASK(31, 16) - -#define IPA_REG_ENDP_INIT_RSRC_GRP_N_OFFSET(ep) \ - (0x00000838 + 0x0070 * (ep)) -/* Encoded value for ENDP_INIT_RSRC_GRP register RSRC_GRP field */ -static inline u32 rsrc_grp_encoded(enum ipa_version version, u32 rsrc_grp) -{ - if (version < IPA_VERSION_3_5 || version == IPA_VERSION_4_5) - return u32_encode_bits(rsrc_grp, GENMASK(2, 0)); + IPA_MBIM_16 = 0x0, + IPA_HDLC = 0x1, + IPA_TLP = 0x2, + IPA_RNDIS = 0x3, + IPA_GENERIC = 0x4, + IPA_COALESCE = 0x5, + IPA_QCMAP = 0x6, +}; - if (version == IPA_VERSION_4_2 || version == IPA_VERSION_4_7) - return u32_encode_bits(rsrc_grp, GENMASK(0, 0)); +/* ENDP_INIT_HOL_BLOCK_EN register */ +enum ipa_reg_endp_init_hol_block_en_field_id { + HOL_BLOCK_EN, +}; - return u32_encode_bits(rsrc_grp, GENMASK(1, 0)); -} +/* ENDP_INIT_HOL_BLOCK_TIMER register */ +enum ipa_reg_endp_init_hol_block_timer_field_id { + TIMER_BASE_VALUE, /* Not v4.5+ */ + TIMER_SCALE, /* v4.2 only */ + TIMER_LIMIT, /* v4.5+ */ + TIMER_GRAN_SEL, /* v4.5+ */ +}; -/* Valid only for TX (IPA consumer) endpoints */ -#define IPA_REG_ENDP_INIT_SEQ_N_OFFSET(txep) \ - (0x0000083c + 0x0070 * (txep)) -#define SEQ_TYPE_FMASK GENMASK(7, 0) -#define SEQ_REP_TYPE_FMASK GENMASK(15, 8) +/* ENDP_INIT_DEAGGR register */ +enum ipa_reg_endp_deaggr_field_id { + DEAGGR_HDR_LEN, + SYSPIPE_ERR_DETECTION, + PACKET_OFFSET_VALID, + PACKET_OFFSET_LOCATION, + IGNORE_MIN_PKT_ERR, + MAX_PACKET_LEN, +}; + +/* ENDP_INIT_RSRC_GRP register */ +enum ipa_reg_endp_init_rsrc_grp_field_id { + ENDP_RSRC_GRP, +}; + +/* ENDP_INIT_SEQ register */ +enum ipa_reg_endp_init_seq_field_id { + SEQ_TYPE, + SEQ_REP_TYPE, /* Not v4.5+ */ +}; /** * enum ipa_seq_type - HPS and DPS sequencer type @@ -629,76 +544,36 @@ enum ipa_seq_rep_type { IPA_SEQ_REP_DMA_PARSER = 0x08, }; -#define IPA_REG_ENDP_STATUS_N_OFFSET(ep) \ - (0x00000840 + 0x0070 * (ep)) -#define STATUS_EN_FMASK GENMASK(0, 0) -#define STATUS_ENDP_FMASK GENMASK(5, 1) -/* The next field is not present for IPA v4.5+ */ -#define STATUS_LOCATION_FMASK GENMASK(8, 8) -/* The next field is present for IPA v4.0+ */ -#define STATUS_PKT_SUPPRESS_FMASK GENMASK(9, 9) - -/* The next register is not present for IPA v4.2 (which no hashing support) */ -#define IPA_REG_ENDP_FILTER_ROUTER_HSH_CFG_N_OFFSET(er) \ - (0x0000085c + 0x0070 * (er)) -#define FILTER_HASH_MSK_SRC_ID_FMASK GENMASK(0, 0) -#define FILTER_HASH_MSK_SRC_IP_FMASK GENMASK(1, 1) -#define FILTER_HASH_MSK_DST_IP_FMASK GENMASK(2, 2) -#define FILTER_HASH_MSK_SRC_PORT_FMASK GENMASK(3, 3) -#define FILTER_HASH_MSK_DST_PORT_FMASK GENMASK(4, 4) -#define FILTER_HASH_MSK_PROTOCOL_FMASK GENMASK(5, 5) -#define FILTER_HASH_MSK_METADATA_FMASK GENMASK(6, 6) -#define IPA_REG_ENDP_FILTER_HASH_MSK_ALL GENMASK(6, 0) - -#define ROUTER_HASH_MSK_SRC_ID_FMASK GENMASK(16, 16) -#define ROUTER_HASH_MSK_SRC_IP_FMASK GENMASK(17, 17) -#define ROUTER_HASH_MSK_DST_IP_FMASK GENMASK(18, 18) -#define ROUTER_HASH_MSK_SRC_PORT_FMASK GENMASK(19, 19) -#define ROUTER_HASH_MSK_DST_PORT_FMASK GENMASK(20, 20) -#define ROUTER_HASH_MSK_PROTOCOL_FMASK GENMASK(21, 21) -#define ROUTER_HASH_MSK_METADATA_FMASK GENMASK(22, 22) -#define IPA_REG_ENDP_ROUTER_HASH_MSK_ALL GENMASK(22, 16) - -static inline u32 ipa_reg_irq_stts_ee_n_offset(enum ipa_version version, - u32 ee) -{ - if (version < IPA_VERSION_4_9) - return 0x00003008 + 0x1000 * ee; - - return 0x00004008 + 0x1000 * ee; -} - -static inline u32 ipa_reg_irq_stts_offset(enum ipa_version version) -{ - return ipa_reg_irq_stts_ee_n_offset(version, GSI_EE_AP); -} - -static inline u32 ipa_reg_irq_en_ee_n_offset(enum ipa_version version, u32 ee) -{ - if (version < IPA_VERSION_4_9) - return 0x0000300c + 0x1000 * ee; - - return 0x0000400c + 0x1000 * ee; -} - -static inline u32 ipa_reg_irq_en_offset(enum ipa_version version) -{ - return ipa_reg_irq_en_ee_n_offset(version, GSI_EE_AP); -} - -static inline u32 ipa_reg_irq_clr_ee_n_offset(enum ipa_version version, u32 ee) -{ - if (version < IPA_VERSION_4_9) - return 0x00003010 + 0x1000 * ee; - - return 0x00004010 + 0x1000 * ee; -} +/* ENDP_STATUS register */ +enum ipa_reg_endp_status_field_id { + STATUS_EN, + STATUS_ENDP, + STATUS_LOCATION, /* Not v4.5+ */ + STATUS_PKT_SUPPRESS, /* v4.0+ */ +}; -static inline u32 ipa_reg_irq_clr_offset(enum ipa_version version) -{ - return ipa_reg_irq_clr_ee_n_offset(version, GSI_EE_AP); -} +/* ENDP_FILTER_ROUTER_HSH_CFG register */ +enum ipa_reg_endp_filter_router_hsh_cfg_field_id { + FILTER_HASH_MSK_SRC_ID, + FILTER_HASH_MSK_SRC_IP, + FILTER_HASH_MSK_DST_IP, + FILTER_HASH_MSK_SRC_PORT, + FILTER_HASH_MSK_DST_PORT, + FILTER_HASH_MSK_PROTOCOL, + FILTER_HASH_MSK_METADATA, + FILTER_HASH_MSK_ALL, /* Bitwise OR of the above 6 fields */ + + ROUTER_HASH_MSK_SRC_ID, + ROUTER_HASH_MSK_SRC_IP, + ROUTER_HASH_MSK_DST_IP, + ROUTER_HASH_MSK_SRC_PORT, + ROUTER_HASH_MSK_DST_PORT, + ROUTER_HASH_MSK_PROTOCOL, + ROUTER_HASH_MSK_METADATA, + ROUTER_HASH_MSK_ALL, /* Bitwise OR of the above 6 fields */ +}; +/* IPA_IRQ_STTS, IPA_IRQ_EN, and IPA_IRQ_CLR registers */ /** * enum ipa_irq_id - Bit positions representing type of IPA IRQ * @IPA_IRQ_UC_0: Microcontroller event interrupt @@ -774,74 +649,82 @@ enum ipa_irq_id { IPA_IRQ_COUNT, /* Last; not an id */ }; -static inline u32 ipa_reg_irq_uc_ee_n_offset(enum ipa_version version, u32 ee) -{ - if (version < IPA_VERSION_4_9) - return 0x0000301c + 0x1000 * ee; +/* IPA_IRQ_UC register */ +enum ipa_reg_ipa_irq_uc_field_id { + UC_INTR, +}; - return 0x0000401c + 0x1000 * ee; -} +extern const struct ipa_regs ipa_regs_v3_1; +extern const struct ipa_regs ipa_regs_v3_5_1; +extern const struct ipa_regs ipa_regs_v4_2; +extern const struct ipa_regs ipa_regs_v4_5; +extern const struct ipa_regs ipa_regs_v4_9; +extern const struct ipa_regs ipa_regs_v4_11; -static inline u32 ipa_reg_irq_uc_offset(enum ipa_version version) +/* Return the field mask for a field in a register */ +static inline u32 ipa_reg_fmask(const struct ipa_reg *reg, u32 field_id) { - return ipa_reg_irq_uc_ee_n_offset(version, GSI_EE_AP); -} + if (!reg || WARN_ON(field_id >= reg->fcount)) + return 0; -#define UC_INTR_FMASK GENMASK(0, 0) + return reg->fmask[field_id]; +} -/* ipa->available defines the valid bits in the SUSPEND_INFO register */ -static inline u32 -ipa_reg_irq_suspend_info_ee_n_offset(enum ipa_version version, u32 ee) +/* Return the mask for a single-bit field in a register */ +static inline u32 ipa_reg_bit(const struct ipa_reg *reg, u32 field_id) { - if (version == IPA_VERSION_3_0) - return 0x00003098 + 0x1000 * ee; + u32 fmask = ipa_reg_fmask(reg, field_id); - if (version < IPA_VERSION_4_9) - return 0x00003030 + 0x1000 * ee; + WARN_ON(!is_power_of_2(fmask)); - return 0x00004030 + 0x1000 * ee; + return fmask; } +/* Encode a value into the given field of a register */ static inline u32 -ipa_reg_irq_suspend_info_offset(enum ipa_version version) +ipa_reg_encode(const struct ipa_reg *reg, u32 field_id, u32 val) { - return ipa_reg_irq_suspend_info_ee_n_offset(version, GSI_EE_AP); -} + u32 fmask = ipa_reg_fmask(reg, field_id); -/* ipa->available defines the valid bits in the SUSPEND_EN register */ -static inline u32 -ipa_reg_irq_suspend_en_ee_n_offset(enum ipa_version version, u32 ee) -{ - WARN_ON(version == IPA_VERSION_3_0); + if (!fmask) + return 0; - if (version < IPA_VERSION_4_9) - return 0x00003034 + 0x1000 * ee; + val <<= __ffs(fmask); + if (WARN_ON(val & ~fmask)) + return 0; - return 0x00004034 + 0x1000 * ee; + return val; } +/* Given a register value, decode (extract) the value in the given field */ static inline u32 -ipa_reg_irq_suspend_en_offset(enum ipa_version version) +ipa_reg_decode(const struct ipa_reg *reg, u32 field_id, u32 val) { - return ipa_reg_irq_suspend_en_ee_n_offset(version, GSI_EE_AP); + u32 fmask = ipa_reg_fmask(reg, field_id); + + return fmask ? (val & fmask) >> __ffs(fmask) : 0; } -/* ipa->available defines the valid bits in the SUSPEND_CLR register */ -static inline u32 -ipa_reg_irq_suspend_clr_ee_n_offset(enum ipa_version version, u32 ee) +/* Return the maximum value representable by the given field; always 2^n - 1 */ +static inline u32 ipa_reg_field_max(const struct ipa_reg *reg, u32 field_id) { - WARN_ON(version == IPA_VERSION_3_0); + u32 fmask = ipa_reg_fmask(reg, field_id); - if (version < IPA_VERSION_4_9) - return 0x00003038 + 0x1000 * ee; + return fmask ? fmask >> __ffs(fmask) : 0; +} + +const struct ipa_reg *ipa_reg(struct ipa *ipa, enum ipa_reg_id reg_id); - return 0x00004038 + 0x1000 * ee; +/* Returns 0 for NULL reg; warning will have already been issued */ +static inline u32 ipa_reg_offset(const struct ipa_reg *reg) +{ + return reg ? reg->offset : 0; } -static inline u32 -ipa_reg_irq_suspend_clr_offset(enum ipa_version version) +/* Returns 0 for NULL reg; warning will have already been issued */ +static inline u32 ipa_reg_n_offset(const struct ipa_reg *reg, u32 n) { - return ipa_reg_irq_suspend_clr_ee_n_offset(version, GSI_EE_AP); + return reg ? reg->offset + n * reg->stride : 0; } int ipa_reg_init(struct ipa *ipa); diff --git a/drivers/net/ipa/ipa_resource.c b/drivers/net/ipa/ipa_resource.c index 06cec7199382..a257f0e5e361 100644 --- a/drivers/net/ipa/ipa_resource.c +++ b/drivers/net/ipa/ipa_resource.c @@ -1,7 +1,7 @@ // SPDX-License-Identifier: GPL-2.0 /* Copyright (c) 2012-2018, The Linux Foundation. All rights reserved. - * Copyright (C) 2018-2021 Linaro Ltd. + * Copyright (C) 2018-2022 Linaro Ltd. */ #include <linux/types.h> @@ -69,20 +69,21 @@ static bool ipa_resource_limits_valid(struct ipa *ipa, } static void -ipa_resource_config_common(struct ipa *ipa, u32 offset, +ipa_resource_config_common(struct ipa *ipa, u32 resource_type, + const struct ipa_reg *reg, const struct ipa_resource_limits *xlimits, const struct ipa_resource_limits *ylimits) { u32 val; - val = u32_encode_bits(xlimits->min, X_MIN_LIM_FMASK); - val |= u32_encode_bits(xlimits->max, X_MAX_LIM_FMASK); + val = ipa_reg_encode(reg, X_MIN_LIM, xlimits->min); + val |= ipa_reg_encode(reg, X_MAX_LIM, xlimits->max); if (ylimits) { - val |= u32_encode_bits(ylimits->min, Y_MIN_LIM_FMASK); - val |= u32_encode_bits(ylimits->max, Y_MAX_LIM_FMASK); + val |= ipa_reg_encode(reg, Y_MIN_LIM, ylimits->min); + val |= ipa_reg_encode(reg, Y_MAX_LIM, ylimits->max); } - iowrite32(val, ipa->reg_virt + offset); + iowrite32(val, ipa->reg_virt + ipa_reg_n_offset(reg, resource_type)); } static void ipa_resource_config_src(struct ipa *ipa, u32 resource_type, @@ -91,34 +92,35 @@ static void ipa_resource_config_src(struct ipa *ipa, u32 resource_type, u32 group_count = data->rsrc_group_src_count; const struct ipa_resource_limits *ylimits; const struct ipa_resource *resource; - u32 offset; + const struct ipa_reg *reg; resource = &data->resource_src[resource_type]; - offset = IPA_REG_SRC_RSRC_GRP_01_RSRC_TYPE_N_OFFSET(resource_type); + reg = ipa_reg(ipa, SRC_RSRC_GRP_01_RSRC_TYPE); ylimits = group_count == 1 ? NULL : &resource->limits[1]; - ipa_resource_config_common(ipa, offset, &resource->limits[0], ylimits); - + ipa_resource_config_common(ipa, resource_type, reg, + &resource->limits[0], ylimits); if (group_count < 3) return; - offset = IPA_REG_SRC_RSRC_GRP_23_RSRC_TYPE_N_OFFSET(resource_type); + reg = ipa_reg(ipa, SRC_RSRC_GRP_23_RSRC_TYPE); ylimits = group_count == 3 ? NULL : &resource->limits[3]; - ipa_resource_config_common(ipa, offset, &resource->limits[2], ylimits); - + ipa_resource_config_common(ipa, resource_type, reg, + &resource->limits[2], ylimits); if (group_count < 5) return; - offset = IPA_REG_SRC_RSRC_GRP_45_RSRC_TYPE_N_OFFSET(resource_type); + reg = ipa_reg(ipa, SRC_RSRC_GRP_45_RSRC_TYPE); ylimits = group_count == 5 ? NULL : &resource->limits[5]; - ipa_resource_config_common(ipa, offset, &resource->limits[4], ylimits); - + ipa_resource_config_common(ipa, resource_type, reg, + &resource->limits[4], ylimits); if (group_count < 7) return; - offset = IPA_REG_SRC_RSRC_GRP_67_RSRC_TYPE_N_OFFSET(resource_type); + reg = ipa_reg(ipa, SRC_RSRC_GRP_67_RSRC_TYPE); ylimits = group_count == 7 ? NULL : &resource->limits[7]; - ipa_resource_config_common(ipa, offset, &resource->limits[6], ylimits); + ipa_resource_config_common(ipa, resource_type, reg, + &resource->limits[6], ylimits); } static void ipa_resource_config_dst(struct ipa *ipa, u32 resource_type, @@ -127,34 +129,35 @@ static void ipa_resource_config_dst(struct ipa *ipa, u32 resource_type, u32 group_count = data->rsrc_group_dst_count; const struct ipa_resource_limits *ylimits; const struct ipa_resource *resource; - u32 offset; + const struct ipa_reg *reg; resource = &data->resource_dst[resource_type]; - offset = IPA_REG_DST_RSRC_GRP_01_RSRC_TYPE_N_OFFSET(resource_type); + reg = ipa_reg(ipa, DST_RSRC_GRP_01_RSRC_TYPE); ylimits = group_count == 1 ? NULL : &resource->limits[1]; - ipa_resource_config_common(ipa, offset, &resource->limits[0], ylimits); - + ipa_resource_config_common(ipa, resource_type, reg, + &resource->limits[0], ylimits); if (group_count < 3) return; - offset = IPA_REG_DST_RSRC_GRP_23_RSRC_TYPE_N_OFFSET(resource_type); + reg = ipa_reg(ipa, DST_RSRC_GRP_23_RSRC_TYPE); ylimits = group_count == 3 ? NULL : &resource->limits[3]; - ipa_resource_config_common(ipa, offset, &resource->limits[2], ylimits); - + ipa_resource_config_common(ipa, resource_type, reg, + &resource->limits[2], ylimits); if (group_count < 5) return; - offset = IPA_REG_DST_RSRC_GRP_45_RSRC_TYPE_N_OFFSET(resource_type); + reg = ipa_reg(ipa, DST_RSRC_GRP_45_RSRC_TYPE); ylimits = group_count == 5 ? NULL : &resource->limits[5]; - ipa_resource_config_common(ipa, offset, &resource->limits[4], ylimits); - + ipa_resource_config_common(ipa, resource_type, reg, + &resource->limits[4], ylimits); if (group_count < 7) return; - offset = IPA_REG_DST_RSRC_GRP_67_RSRC_TYPE_N_OFFSET(resource_type); + reg = ipa_reg(ipa, DST_RSRC_GRP_67_RSRC_TYPE); ylimits = group_count == 7 ? NULL : &resource->limits[7]; - ipa_resource_config_common(ipa, offset, &resource->limits[6], ylimits); + ipa_resource_config_common(ipa, resource_type, reg, + &resource->limits[6], ylimits); } /* Configure resources; there is no ipa_resource_deconfig() */ diff --git a/drivers/net/ipa/ipa_smp2p.c b/drivers/net/ipa/ipa_smp2p.c index 211233612039..5620dc271fac 100644 --- a/drivers/net/ipa/ipa_smp2p.c +++ b/drivers/net/ipa/ipa_smp2p.c @@ -1,7 +1,7 @@ // SPDX-License-Identifier: GPL-2.0 /* Copyright (c) 2012-2018, The Linux Foundation. All rights reserved. - * Copyright (C) 2019-2020 Linaro Ltd. + * Copyright (C) 2019-2022 Linaro Ltd. */ #include <linux/types.h> diff --git a/drivers/net/ipa/ipa_smp2p.h b/drivers/net/ipa/ipa_smp2p.h index 59cee31a7383..9b969b03d1a4 100644 --- a/drivers/net/ipa/ipa_smp2p.h +++ b/drivers/net/ipa/ipa_smp2p.h @@ -1,7 +1,7 @@ /* SPDX-License-Identifier: GPL-2.0 */ /* Copyright (c) 2012-2018, The Linux Foundation. All rights reserved. - * Copyright (C) 2019-2020 Linaro Ltd. + * Copyright (C) 2019-2022 Linaro Ltd. */ #ifndef _IPA_SMP2P_H_ #define _IPA_SMP2P_H_ diff --git a/drivers/net/ipa/ipa_sysfs.c b/drivers/net/ipa/ipa_sysfs.c index c0c8641cdd14..5cbc15a971f9 100644 --- a/drivers/net/ipa/ipa_sysfs.c +++ b/drivers/net/ipa/ipa_sysfs.c @@ -1,6 +1,6 @@ // SPDX-License-Identifier: GPL-2.0 -/* Copyright (C) 2021 Linaro Ltd. */ +/* Copyright (C) 2021-2022 Linaro Ltd. */ #include <linux/kernel.h> #include <linux/types.h> diff --git a/drivers/net/ipa/ipa_sysfs.h b/drivers/net/ipa/ipa_sysfs.h index 4a3ffd1e4e3f..58ba22810bab 100644 --- a/drivers/net/ipa/ipa_sysfs.h +++ b/drivers/net/ipa/ipa_sysfs.h @@ -1,7 +1,7 @@ /* SPDX-License-Identifier: GPL-2.0 */ /* Copyright (c) 2012-2018, The Linux Foundation. All rights reserved. - * Copyright (C) 2019-2021 Linaro Ltd. + * Copyright (C) 2019-2022 Linaro Ltd. */ #ifndef _IPA_SYSFS_H_ #define _IPA_SYSFS_H_ diff --git a/drivers/net/ipa/ipa_table.c b/drivers/net/ipa/ipa_table.c index 69efe672ca52..510ff2dc8999 100644 --- a/drivers/net/ipa/ipa_table.c +++ b/drivers/net/ipa/ipa_table.c @@ -1,7 +1,7 @@ // SPDX-License-Identifier: GPL-2.0 /* Copyright (c) 2012-2018, The Linux Foundation. All rights reserved. - * Copyright (C) 2018-2021 Linaro Ltd. + * Copyright (C) 2018-2022 Linaro Ltd. */ #include <linux/types.h> @@ -384,8 +384,9 @@ void ipa_table_reset(struct ipa *ipa, bool modem) int ipa_table_hash_flush(struct ipa *ipa) { - u32 offset = ipa_reg_filt_rout_hash_flush_offset(ipa->version); + const struct ipa_reg *reg; struct gsi_trans *trans; + u32 offset; u32 val; if (!ipa_table_hash_support(ipa)) @@ -397,8 +398,13 @@ int ipa_table_hash_flush(struct ipa *ipa) return -EBUSY; } - val = IPV4_FILTER_HASH_FMASK | IPV6_FILTER_HASH_FMASK; - val |= IPV6_ROUTER_HASH_FMASK | IPV4_ROUTER_HASH_FMASK; + reg = ipa_reg(ipa, FILT_ROUT_HASH_FLUSH); + offset = ipa_reg_offset(reg); + + val = ipa_reg_bit(reg, IPV6_ROUTER_HASH); + val |= ipa_reg_bit(reg, IPV6_FILTER_HASH); + val |= ipa_reg_bit(reg, IPV4_ROUTER_HASH); + val |= ipa_reg_bit(reg, IPV4_FILTER_HASH); ipa_cmd_register_write_add(trans, offset, val, val, false); @@ -516,15 +522,18 @@ int ipa_table_setup(struct ipa *ipa) static void ipa_filter_tuple_zero(struct ipa_endpoint *endpoint) { u32 endpoint_id = endpoint->endpoint_id; + struct ipa *ipa = endpoint->ipa; + const struct ipa_reg *reg; u32 offset; u32 val; - offset = IPA_REG_ENDP_FILTER_ROUTER_HSH_CFG_N_OFFSET(endpoint_id); + reg = ipa_reg(ipa, ENDP_FILTER_ROUTER_HSH_CFG); + offset = ipa_reg_n_offset(reg, endpoint_id); val = ioread32(endpoint->ipa->reg_virt + offset); /* Zero all filter-related fields, preserving the rest */ - u32p_replace_bits(&val, 0, IPA_REG_ENDP_FILTER_HASH_MSK_ALL); + val &= ~ipa_reg_fmask(reg, FILTER_HASH_MSK_ALL); iowrite32(val, endpoint->ipa->reg_virt + offset); } @@ -565,13 +574,17 @@ static bool ipa_route_id_modem(u32 route_id) */ static void ipa_route_tuple_zero(struct ipa *ipa, u32 route_id) { - u32 offset = IPA_REG_ENDP_FILTER_ROUTER_HSH_CFG_N_OFFSET(route_id); + const struct ipa_reg *reg; + u32 offset; u32 val; + reg = ipa_reg(ipa, ENDP_FILTER_ROUTER_HSH_CFG); + offset = ipa_reg_n_offset(reg, route_id); + val = ioread32(ipa->reg_virt + offset); /* Zero all route-related fields, preserving the rest */ - u32p_replace_bits(&val, 0, IPA_REG_ENDP_ROUTER_HASH_MSK_ALL); + val &= ~ipa_reg_fmask(reg, ROUTER_HASH_MSK_ALL); iowrite32(val, ipa->reg_virt + offset); } diff --git a/drivers/net/ipa/ipa_table.h b/drivers/net/ipa/ipa_table.h index 1538e2e1732f..395189f75d78 100644 --- a/drivers/net/ipa/ipa_table.h +++ b/drivers/net/ipa/ipa_table.h @@ -1,7 +1,7 @@ /* SPDX-License-Identifier: GPL-2.0 */ /* Copyright (c) 2012-2018, The Linux Foundation. All rights reserved. - * Copyright (C) 2019-2021 Linaro Ltd. + * Copyright (C) 2019-2022 Linaro Ltd. */ #ifndef _IPA_TABLE_H_ #define _IPA_TABLE_H_ diff --git a/drivers/net/ipa/ipa_uc.c b/drivers/net/ipa/ipa_uc.c index fe11910518d9..f0ee47281015 100644 --- a/drivers/net/ipa/ipa_uc.c +++ b/drivers/net/ipa/ipa_uc.c @@ -1,7 +1,7 @@ // SPDX-License-Identifier: GPL-2.0 /* Copyright (c) 2012-2018, The Linux Foundation. All rights reserved. - * Copyright (C) 2018-2020 Linaro Ltd. + * Copyright (C) 2018-2022 Linaro Ltd. */ #include <linux/types.h> @@ -222,7 +222,7 @@ void ipa_uc_power(struct ipa *ipa) static void send_uc_command(struct ipa *ipa, u32 command, u32 command_param) { struct ipa_uc_mem_area *shared = ipa_uc_shared(ipa); - u32 offset; + const struct ipa_reg *reg; u32 val; /* Fill in the command data */ @@ -233,9 +233,10 @@ static void send_uc_command(struct ipa *ipa, u32 command, u32 command_param) shared->response_param = 0; /* Use an interrupt to tell the microcontroller the command is ready */ - val = u32_encode_bits(1, UC_INTR_FMASK); - offset = ipa_reg_irq_uc_offset(ipa->version); - iowrite32(val, ipa->reg_virt + offset); + reg = ipa_reg(ipa, IPA_IRQ_UC); + val = ipa_reg_bit(reg, UC_INTR); + + iowrite32(val, ipa->reg_virt + ipa_reg_offset(reg)); } /* Tell the microcontroller the AP is shutting down */ diff --git a/drivers/net/ipa/ipa_uc.h b/drivers/net/ipa/ipa_uc.h index 23847f934d64..8514096e6f36 100644 --- a/drivers/net/ipa/ipa_uc.h +++ b/drivers/net/ipa/ipa_uc.h @@ -1,7 +1,7 @@ /* SPDX-License-Identifier: GPL-2.0 */ /* Copyright (c) 2012-2018, The Linux Foundation. All rights reserved. - * Copyright (C) 2019-2020 Linaro Ltd. + * Copyright (C) 2019-2022 Linaro Ltd. */ #ifndef _IPA_UC_H_ #define _IPA_UC_H_ diff --git a/drivers/net/ipa/ipa_version.h b/drivers/net/ipa/ipa_version.h index 6c16c895d842..7870e0cc3d7c 100644 --- a/drivers/net/ipa/ipa_version.h +++ b/drivers/net/ipa/ipa_version.h @@ -1,7 +1,7 @@ /* SPDX-License-Identifier: GPL-2.0 */ /* Copyright (c) 2012-2018, The Linux Foundation. All rights reserved. - * Copyright (C) 2019-2020 Linaro Ltd. + * Copyright (C) 2019-2022 Linaro Ltd. */ #ifndef _IPA_VERSION_H_ #define _IPA_VERSION_H_ @@ -19,10 +19,10 @@ * @IPA_VERSION_4_7: IPA version 4.7/GSI version 2.7 * @IPA_VERSION_4_9: IPA version 4.9/GSI version 2.9 * @IPA_VERSION_4_11: IPA version 4.11/GSI version 2.11 (2.1.1) + * @IPA_VERSION_COUNT: Number of defined IPA versions * * Defines the version of IPA (and GSI) hardware present on the platform. - * Please update ipa_version_valid() and ipa_version_string() whenever a - * new version is added. + * Please update ipa_version_string() whenever a new version is added. */ enum ipa_version { IPA_VERSION_3_0, @@ -36,6 +36,30 @@ enum ipa_version { IPA_VERSION_4_7, IPA_VERSION_4_9, IPA_VERSION_4_11, + IPA_VERSION_COUNT, /* Last; not a version */ +}; + +static inline bool ipa_version_supported(enum ipa_version version) +{ + switch (version) { + case IPA_VERSION_3_1: + case IPA_VERSION_3_5_1: + case IPA_VERSION_4_2: + case IPA_VERSION_4_5: + case IPA_VERSION_4_9: + case IPA_VERSION_4_11: + return true; + default: + return false; + } +} + +/* Execution environment IDs */ +enum gsi_ee_id { + GSI_EE_AP = 0x0, + GSI_EE_MODEM = 0x1, + GSI_EE_UC = 0x2, + GSI_EE_TZ = 0x3, }; #endif /* _IPA_VERSION_H_ */ diff --git a/drivers/net/ipa/reg/ipa_reg-v3.1.c b/drivers/net/ipa/reg/ipa_reg-v3.1.c new file mode 100644 index 000000000000..116b27717e3d --- /dev/null +++ b/drivers/net/ipa/reg/ipa_reg-v3.1.c @@ -0,0 +1,478 @@ +// SPDX-License-Identifier: GPL-2.0 + +/* Copyright (C) 2022 Linaro Ltd. */ + +#include <linux/types.h> + +#include "../ipa.h" +#include "../ipa_reg.h" + +static const u32 ipa_reg_comp_cfg_fmask[] = { + [COMP_CFG_ENABLE] = BIT(0), + [GSI_SNOC_BYPASS_DIS] = BIT(1), + [GEN_QMB_0_SNOC_BYPASS_DIS] = BIT(2), + [GEN_QMB_1_SNOC_BYPASS_DIS] = BIT(3), + [IPA_DCMP_FAST_CLK_EN] = BIT(4), + /* Bits 5-31 reserved */ +}; + +IPA_REG_FIELDS(COMP_CFG, comp_cfg, 0x0000003c); + +static const u32 ipa_reg_clkon_cfg_fmask[] = { + [CLKON_RX] = BIT(0), + [CLKON_PROC] = BIT(1), + [TX_WRAPPER] = BIT(2), + [CLKON_MISC] = BIT(3), + [RAM_ARB] = BIT(4), + [FTCH_HPS] = BIT(5), + [FTCH_DPS] = BIT(6), + [CLKON_HPS] = BIT(7), + [CLKON_DPS] = BIT(8), + [RX_HPS_CMDQS] = BIT(9), + [HPS_DPS_CMDQS] = BIT(10), + [DPS_TX_CMDQS] = BIT(11), + [RSRC_MNGR] = BIT(12), + [CTX_HANDLER] = BIT(13), + [ACK_MNGR] = BIT(14), + [D_DCPH] = BIT(15), + [H_DCPH] = BIT(16), + /* Bits 17-31 reserved */ +}; + +IPA_REG_FIELDS(CLKON_CFG, clkon_cfg, 0x00000044); + +static const u32 ipa_reg_route_fmask[] = { + [ROUTE_DIS] = BIT(0), + [ROUTE_DEF_PIPE] = GENMASK(5, 1), + [ROUTE_DEF_HDR_TABLE] = BIT(6), + [ROUTE_DEF_HDR_OFST] = GENMASK(16, 7), + [ROUTE_FRAG_DEF_PIPE] = GENMASK(21, 17), + /* Bits 22-23 reserved */ + [ROUTE_DEF_RETAIN_HDR] = BIT(24), + /* Bits 25-31 reserved */ +}; + +IPA_REG_FIELDS(ROUTE, route, 0x00000048); + +static const u32 ipa_reg_shared_mem_size_fmask[] = { + [MEM_SIZE] = GENMASK(15, 0), + [MEM_BADDR] = GENMASK(31, 16), +}; + +IPA_REG_FIELDS(SHARED_MEM_SIZE, shared_mem_size, 0x00000054); + +static const u32 ipa_reg_qsb_max_writes_fmask[] = { + [GEN_QMB_0_MAX_WRITES] = GENMASK(3, 0), + [GEN_QMB_1_MAX_WRITES] = GENMASK(7, 4), + /* Bits 8-31 reserved */ +}; + +IPA_REG_FIELDS(QSB_MAX_WRITES, qsb_max_writes, 0x00000074); + +static const u32 ipa_reg_qsb_max_reads_fmask[] = { + [GEN_QMB_0_MAX_READS] = GENMASK(3, 0), + [GEN_QMB_1_MAX_READS] = GENMASK(7, 4), +}; + +IPA_REG_FIELDS(QSB_MAX_READS, qsb_max_reads, 0x00000078); + +static const u32 ipa_reg_filt_rout_hash_en_fmask[] = { + [IPV6_ROUTER_HASH] = BIT(0), + /* Bits 1-3 reserved */ + [IPV6_FILTER_HASH] = BIT(4), + /* Bits 5-7 reserved */ + [IPV4_ROUTER_HASH] = BIT(8), + /* Bits 9-11 reserved */ + [IPV4_FILTER_HASH] = BIT(12), + /* Bits 13-31 reserved */ +}; + +IPA_REG_FIELDS(FILT_ROUT_HASH_EN, filt_rout_hash_en, 0x000008c); + +static const u32 ipa_reg_filt_rout_hash_flush_fmask[] = { + [IPV6_ROUTER_HASH] = BIT(0), + /* Bits 1-3 reserved */ + [IPV6_FILTER_HASH] = BIT(4), + /* Bits 5-7 reserved */ + [IPV4_ROUTER_HASH] = BIT(8), + /* Bits 9-11 reserved */ + [IPV4_FILTER_HASH] = BIT(12), + /* Bits 13-31 reserved */ +}; + +IPA_REG_FIELDS(FILT_ROUT_HASH_FLUSH, filt_rout_hash_flush, 0x0000090); + +/* Valid bits defined by ipa->available */ +IPA_REG(STATE_AGGR_ACTIVE, state_aggr_active, 0x0000010c); + +IPA_REG(IPA_BCR, ipa_bcr, 0x000001d0); + +static const u32 ipa_reg_local_pkt_proc_cntxt_fmask[] = { + [IPA_BASE_ADDR] = GENMASK(16, 0), + /* Bits 17-31 reserved */ +}; + +/* Offset must be a multiple of 8 */ +IPA_REG_FIELDS(LOCAL_PKT_PROC_CNTXT, local_pkt_proc_cntxt, 0x000001e8); + +/* Valid bits defined by ipa->available */ +IPA_REG(AGGR_FORCE_CLOSE, aggr_force_close, 0x000001ec); + +static const u32 ipa_reg_counter_cfg_fmask[] = { + [EOT_COAL_GRANULARITY] = GENMASK(3, 0), + [AGGR_GRANULARITY] = GENMASK(8, 4), + /* Bits 5-31 reserved */ +}; + +IPA_REG_FIELDS(COUNTER_CFG, counter_cfg, 0x000001f0); + +static const u32 ipa_reg_src_rsrc_grp_01_rsrc_type_fmask[] = { + [X_MIN_LIM] = GENMASK(5, 0), + /* Bits 6-7 reserved */ + [X_MAX_LIM] = GENMASK(13, 8), + /* Bits 14-15 reserved */ + [Y_MIN_LIM] = GENMASK(21, 16), + /* Bits 22-23 reserved */ + [Y_MAX_LIM] = GENMASK(29, 24), + /* Bits 30-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(SRC_RSRC_GRP_01_RSRC_TYPE, src_rsrc_grp_01_rsrc_type, + 0x00000400, 0x0020); + +static const u32 ipa_reg_src_rsrc_grp_23_rsrc_type_fmask[] = { + [X_MIN_LIM] = GENMASK(5, 0), + /* Bits 6-7 reserved */ + [X_MAX_LIM] = GENMASK(13, 8), + /* Bits 14-15 reserved */ + [Y_MIN_LIM] = GENMASK(21, 16), + /* Bits 22-23 reserved */ + [Y_MAX_LIM] = GENMASK(29, 24), + /* Bits 30-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(SRC_RSRC_GRP_23_RSRC_TYPE, src_rsrc_grp_23_rsrc_type, + 0x00000404, 0x0020); + +static const u32 ipa_reg_src_rsrc_grp_45_rsrc_type_fmask[] = { + [X_MIN_LIM] = GENMASK(5, 0), + /* Bits 6-7 reserved */ + [X_MAX_LIM] = GENMASK(13, 8), + /* Bits 14-15 reserved */ + [Y_MIN_LIM] = GENMASK(21, 16), + /* Bits 22-23 reserved */ + [Y_MAX_LIM] = GENMASK(29, 24), + /* Bits 30-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(SRC_RSRC_GRP_45_RSRC_TYPE, src_rsrc_grp_45_rsrc_type, + 0x00000408, 0x0020); + +static const u32 ipa_reg_src_rsrc_grp_67_rsrc_type_fmask[] = { + [X_MIN_LIM] = GENMASK(5, 0), + /* Bits 6-7 reserved */ + [X_MAX_LIM] = GENMASK(13, 8), + /* Bits 14-15 reserved */ + [Y_MIN_LIM] = GENMASK(21, 16), + /* Bits 22-23 reserved */ + [Y_MAX_LIM] = GENMASK(29, 24), + /* Bits 30-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(SRC_RSRC_GRP_67_RSRC_TYPE, src_rsrc_grp_67_rsrc_type, + 0x0000040c, 0x0020); + +static const u32 ipa_reg_dst_rsrc_grp_01_rsrc_type_fmask[] = { + [X_MIN_LIM] = GENMASK(5, 0), + /* Bits 6-7 reserved */ + [X_MAX_LIM] = GENMASK(13, 8), + /* Bits 14-15 reserved */ + [Y_MIN_LIM] = GENMASK(21, 16), + /* Bits 22-23 reserved */ + [Y_MAX_LIM] = GENMASK(29, 24), + /* Bits 30-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(DST_RSRC_GRP_01_RSRC_TYPE, dst_rsrc_grp_01_rsrc_type, + 0x00000500, 0x0020); + +static const u32 ipa_reg_dst_rsrc_grp_23_rsrc_type_fmask[] = { + [X_MIN_LIM] = GENMASK(5, 0), + /* Bits 6-7 reserved */ + [X_MAX_LIM] = GENMASK(13, 8), + /* Bits 14-15 reserved */ + [Y_MIN_LIM] = GENMASK(21, 16), + /* Bits 22-23 reserved */ + [Y_MAX_LIM] = GENMASK(29, 24), + /* Bits 30-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(DST_RSRC_GRP_23_RSRC_TYPE, dst_rsrc_grp_23_rsrc_type, + 0x00000504, 0x0020); + +static const u32 ipa_reg_dst_rsrc_grp_45_rsrc_type_fmask[] = { + [X_MIN_LIM] = GENMASK(5, 0), + /* Bits 6-7 reserved */ + [X_MAX_LIM] = GENMASK(13, 8), + /* Bits 14-15 reserved */ + [Y_MIN_LIM] = GENMASK(21, 16), + /* Bits 22-23 reserved */ + [Y_MAX_LIM] = GENMASK(29, 24), + /* Bits 30-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(DST_RSRC_GRP_45_RSRC_TYPE, dst_rsrc_grp_45_rsrc_type, + 0x00000508, 0x0020); + +static const u32 ipa_reg_dst_rsrc_grp_67_rsrc_type_fmask[] = { + [X_MIN_LIM] = GENMASK(5, 0), + /* Bits 6-7 reserved */ + [X_MAX_LIM] = GENMASK(13, 8), + /* Bits 14-15 reserved */ + [Y_MIN_LIM] = GENMASK(21, 16), + /* Bits 22-23 reserved */ + [Y_MAX_LIM] = GENMASK(29, 24), + /* Bits 30-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(DST_RSRC_GRP_67_RSRC_TYPE, dst_rsrc_grp_67_rsrc_type, + 0x0000050c, 0x0020); + +static const u32 ipa_reg_endp_init_ctrl_fmask[] = { + [ENDP_SUSPEND] = BIT(0), + [ENDP_DELAY] = BIT(1), + /* Bits 2-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(ENDP_INIT_CTRL, endp_init_ctrl, 0x00000800, 0x0070); + +static const u32 ipa_reg_endp_init_cfg_fmask[] = { + [FRAG_OFFLOAD_EN] = BIT(0), + [CS_OFFLOAD_EN] = GENMASK(2, 1), + [CS_METADATA_HDR_OFFSET] = GENMASK(6, 3), + /* Bit 7 reserved */ + [CS_GEN_QMB_MASTER_SEL] = BIT(8), + /* Bits 9-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(ENDP_INIT_CFG, endp_init_cfg, 0x00000808, 0x0070); + +static const u32 ipa_reg_endp_init_nat_fmask[] = { + [NAT_EN] = GENMASK(1, 0), + /* Bits 2-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(ENDP_INIT_NAT, endp_init_nat, 0x0000080c, 0x0070); + +static const u32 ipa_reg_endp_init_hdr_fmask[] = { + [HDR_LEN] = GENMASK(5, 0), + [HDR_OFST_METADATA_VALID] = BIT(6), + [HDR_OFST_METADATA] = GENMASK(12, 7), + [HDR_ADDITIONAL_CONST_LEN] = GENMASK(18, 13), + [HDR_OFST_PKT_SIZE_VALID] = BIT(19), + [HDR_OFST_PKT_SIZE] = GENMASK(25, 20), + [HDR_A5_MUX] = BIT(26), + [HDR_LEN_INC_DEAGG_HDR] = BIT(27), + [HDR_METADATA_REG_VALID] = BIT(28), + /* Bits 29-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(ENDP_INIT_HDR, endp_init_hdr, 0x00000810, 0x0070); + +static const u32 ipa_reg_endp_init_hdr_ext_fmask[] = { + [HDR_ENDIANNESS] = BIT(0), + [HDR_TOTAL_LEN_OR_PAD_VALID] = BIT(1), + [HDR_TOTAL_LEN_OR_PAD] = BIT(2), + [HDR_PAYLOAD_LEN_INC_PADDING] = BIT(3), + [HDR_TOTAL_LEN_OR_PAD_OFFSET] = GENMASK(9, 4), + [HDR_PAD_TO_ALIGNMENT] = GENMASK(13, 10), + /* Bits 14-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(ENDP_INIT_HDR_EXT, endp_init_hdr_ext, 0x00000814, 0x0070); + +IPA_REG_STRIDE(ENDP_INIT_HDR_METADATA_MASK, endp_init_hdr_metadata_mask, + 0x00000818, 0x0070); + +static const u32 ipa_reg_endp_init_mode_fmask[] = { + [ENDP_MODE] = GENMASK(2, 0), + /* Bit 3 reserved */ + [DEST_PIPE_INDEX] = GENMASK(8, 4), + /* Bits 9-11 reserved */ + [BYTE_THRESHOLD] = GENMASK(27, 12), + [PIPE_REPLICATION_EN] = BIT(28), + [PAD_EN] = BIT(29), + [HDR_FTCH_DISABLE] = BIT(30), + /* Bit 31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(ENDP_INIT_MODE, endp_init_mode, 0x00000820, 0x0070); + +static const u32 ipa_reg_endp_init_aggr_fmask[] = { + [AGGR_EN] = GENMASK(1, 0), + [AGGR_TYPE] = GENMASK(4, 2), + [BYTE_LIMIT] = GENMASK(9, 5), + [TIME_LIMIT] = GENMASK(14, 10), + [PKT_LIMIT] = GENMASK(20, 15), + [SW_EOF_ACTIVE] = BIT(21), + [FORCE_CLOSE] = BIT(22), + /* Bit 23 reserved */ + [HARD_BYTE_LIMIT_EN] = BIT(24), + /* Bits 25-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(ENDP_INIT_AGGR, endp_init_aggr, 0x00000824, 0x0070); + +static const u32 ipa_reg_endp_init_hol_block_en_fmask[] = { + [HOL_BLOCK_EN] = BIT(0), + /* Bits 1-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(ENDP_INIT_HOL_BLOCK_EN, endp_init_hol_block_en, + 0x0000082c, 0x0070); + +/* Entire register is a tick count */ +static const u32 ipa_reg_endp_init_hol_block_timer_fmask[] = { + [TIMER_BASE_VALUE] = GENMASK(31, 0), +}; + +IPA_REG_STRIDE_FIELDS(ENDP_INIT_HOL_BLOCK_TIMER, endp_init_hol_block_timer, + 0x00000830, 0x0070); + +static const u32 ipa_reg_endp_init_deaggr_fmask[] = { + [DEAGGR_HDR_LEN] = GENMASK(5, 0), + [SYSPIPE_ERR_DETECTION] = BIT(6), + [PACKET_OFFSET_VALID] = BIT(7), + [PACKET_OFFSET_LOCATION] = GENMASK(13, 8), + [IGNORE_MIN_PKT_ERR] = BIT(14), + /* Bit 15 reserved */ + [MAX_PACKET_LEN] = GENMASK(31, 16), +}; + +IPA_REG_STRIDE_FIELDS(ENDP_INIT_DEAGGR, endp_init_deaggr, 0x00000834, 0x0070); + +static const u32 ipa_reg_endp_init_rsrc_grp_fmask[] = { + [ENDP_RSRC_GRP] = GENMASK(2, 0), + /* Bits 3-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(ENDP_INIT_RSRC_GRP, endp_init_rsrc_grp, + 0x00000838, 0x0070); + +static const u32 ipa_reg_endp_init_seq_fmask[] = { + [SEQ_TYPE] = GENMASK(7, 0), + [SEQ_REP_TYPE] = GENMASK(15, 8), + /* Bits 16-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(ENDP_INIT_SEQ, endp_init_seq, 0x0000083c, 0x0070); + +static const u32 ipa_reg_endp_status_fmask[] = { + [STATUS_EN] = BIT(0), + [STATUS_ENDP] = GENMASK(5, 1), + /* Bits 6-7 reserved */ + [STATUS_LOCATION] = BIT(8), + /* Bits 9-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(ENDP_STATUS, endp_status, 0x00000840, 0x0070); + +static const u32 ipa_reg_endp_filter_router_hsh_cfg_fmask[] = { + [FILTER_HASH_MSK_SRC_ID] = BIT(0), + [FILTER_HASH_MSK_SRC_IP] = BIT(1), + [FILTER_HASH_MSK_DST_IP] = BIT(2), + [FILTER_HASH_MSK_SRC_PORT] = BIT(3), + [FILTER_HASH_MSK_DST_PORT] = BIT(4), + [FILTER_HASH_MSK_PROTOCOL] = BIT(5), + [FILTER_HASH_MSK_METADATA] = BIT(6), + [FILTER_HASH_MSK_ALL] = GENMASK(6, 0), + /* Bits 7-15 reserved */ + [ROUTER_HASH_MSK_SRC_ID] = BIT(16), + [ROUTER_HASH_MSK_SRC_IP] = BIT(17), + [ROUTER_HASH_MSK_DST_IP] = BIT(18), + [ROUTER_HASH_MSK_SRC_PORT] = BIT(19), + [ROUTER_HASH_MSK_DST_PORT] = BIT(20), + [ROUTER_HASH_MSK_PROTOCOL] = BIT(21), + [ROUTER_HASH_MSK_METADATA] = BIT(22), + [ROUTER_HASH_MSK_ALL] = GENMASK(22, 16), + /* Bits 23-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(ENDP_FILTER_ROUTER_HSH_CFG, endp_filter_router_hsh_cfg, + 0x0000085c, 0x0070); + +/* Valid bits defined by enum ipa_irq_id; only used for GSI_EE_AP */ +IPA_REG(IPA_IRQ_STTS, ipa_irq_stts, 0x00003008 + 0x1000 * GSI_EE_AP); + +/* Valid bits defined by enum ipa_irq_id; only used for GSI_EE_AP */ +IPA_REG(IPA_IRQ_EN, ipa_irq_en, 0x0000300c + 0x1000 * GSI_EE_AP); + +/* Valid bits defined by enum ipa_irq_id; only used for GSI_EE_AP */ +IPA_REG(IPA_IRQ_CLR, ipa_irq_clr, 0x00003010 + 0x1000 * GSI_EE_AP); + +static const u32 ipa_reg_ipa_irq_uc_fmask[] = { + [UC_INTR] = BIT(0), + /* Bits 1-31 reserved */ +}; + +IPA_REG_FIELDS(IPA_IRQ_UC, ipa_irq_uc, 0x0000301c + 0x1000 * GSI_EE_AP); + +/* Valid bits defined by ipa->available */ +IPA_REG(IRQ_SUSPEND_INFO, irq_suspend_info, 0x00003030 + 0x1000 * GSI_EE_AP); + +/* Valid bits defined by ipa->available */ +IPA_REG(IRQ_SUSPEND_EN, irq_suspend_en, 0x00003034 + 0x1000 * GSI_EE_AP); + +/* Valid bits defined by ipa->available */ +IPA_REG(IRQ_SUSPEND_CLR, irq_suspend_clr, 0x00003038 + 0x1000 * GSI_EE_AP); + +static const struct ipa_reg *ipa_reg_array[] = { + [COMP_CFG] = &ipa_reg_comp_cfg, + [CLKON_CFG] = &ipa_reg_clkon_cfg, + [ROUTE] = &ipa_reg_route, + [SHARED_MEM_SIZE] = &ipa_reg_shared_mem_size, + [QSB_MAX_WRITES] = &ipa_reg_qsb_max_writes, + [QSB_MAX_READS] = &ipa_reg_qsb_max_reads, + [FILT_ROUT_HASH_EN] = &ipa_reg_filt_rout_hash_en, + [FILT_ROUT_HASH_FLUSH] = &ipa_reg_filt_rout_hash_flush, + [STATE_AGGR_ACTIVE] = &ipa_reg_state_aggr_active, + [IPA_BCR] = &ipa_reg_ipa_bcr, + [LOCAL_PKT_PROC_CNTXT] = &ipa_reg_local_pkt_proc_cntxt, + [AGGR_FORCE_CLOSE] = &ipa_reg_aggr_force_close, + [COUNTER_CFG] = &ipa_reg_counter_cfg, + [SRC_RSRC_GRP_01_RSRC_TYPE] = &ipa_reg_src_rsrc_grp_01_rsrc_type, + [SRC_RSRC_GRP_23_RSRC_TYPE] = &ipa_reg_src_rsrc_grp_23_rsrc_type, + [SRC_RSRC_GRP_45_RSRC_TYPE] = &ipa_reg_src_rsrc_grp_45_rsrc_type, + [SRC_RSRC_GRP_67_RSRC_TYPE] = &ipa_reg_src_rsrc_grp_67_rsrc_type, + [DST_RSRC_GRP_01_RSRC_TYPE] = &ipa_reg_dst_rsrc_grp_01_rsrc_type, + [DST_RSRC_GRP_23_RSRC_TYPE] = &ipa_reg_dst_rsrc_grp_23_rsrc_type, + [DST_RSRC_GRP_45_RSRC_TYPE] = &ipa_reg_dst_rsrc_grp_45_rsrc_type, + [DST_RSRC_GRP_67_RSRC_TYPE] = &ipa_reg_dst_rsrc_grp_67_rsrc_type, + [ENDP_INIT_CTRL] = &ipa_reg_endp_init_ctrl, + [ENDP_INIT_CFG] = &ipa_reg_endp_init_cfg, + [ENDP_INIT_NAT] = &ipa_reg_endp_init_nat, + [ENDP_INIT_HDR] = &ipa_reg_endp_init_hdr, + [ENDP_INIT_HDR_EXT] = &ipa_reg_endp_init_hdr_ext, + [ENDP_INIT_HDR_METADATA_MASK] = &ipa_reg_endp_init_hdr_metadata_mask, + [ENDP_INIT_MODE] = &ipa_reg_endp_init_mode, + [ENDP_INIT_AGGR] = &ipa_reg_endp_init_aggr, + [ENDP_INIT_HOL_BLOCK_EN] = &ipa_reg_endp_init_hol_block_en, + [ENDP_INIT_HOL_BLOCK_TIMER] = &ipa_reg_endp_init_hol_block_timer, + [ENDP_INIT_DEAGGR] = &ipa_reg_endp_init_deaggr, + [ENDP_INIT_RSRC_GRP] = &ipa_reg_endp_init_rsrc_grp, + [ENDP_INIT_SEQ] = &ipa_reg_endp_init_seq, + [ENDP_STATUS] = &ipa_reg_endp_status, + [ENDP_FILTER_ROUTER_HSH_CFG] = &ipa_reg_endp_filter_router_hsh_cfg, + [IPA_IRQ_STTS] = &ipa_reg_ipa_irq_stts, + [IPA_IRQ_EN] = &ipa_reg_ipa_irq_en, + [IPA_IRQ_CLR] = &ipa_reg_ipa_irq_clr, + [IPA_IRQ_UC] = &ipa_reg_ipa_irq_uc, + [IRQ_SUSPEND_INFO] = &ipa_reg_irq_suspend_info, + [IRQ_SUSPEND_EN] = &ipa_reg_irq_suspend_en, + [IRQ_SUSPEND_CLR] = &ipa_reg_irq_suspend_clr, +}; + +const struct ipa_regs ipa_regs_v3_1 = { + .reg_count = ARRAY_SIZE(ipa_reg_array), + .reg = ipa_reg_array, +}; diff --git a/drivers/net/ipa/reg/ipa_reg-v3.5.1.c b/drivers/net/ipa/reg/ipa_reg-v3.5.1.c new file mode 100644 index 000000000000..6e2f939b18f1 --- /dev/null +++ b/drivers/net/ipa/reg/ipa_reg-v3.5.1.c @@ -0,0 +1,456 @@ +// SPDX-License-Identifier: GPL-2.0 + +/* Copyright (C) 2022 Linaro Ltd. */ + +#include <linux/types.h> + +#include "../ipa.h" +#include "../ipa_reg.h" + +static const u32 ipa_reg_comp_cfg_fmask[] = { + [COMP_CFG_ENABLE] = BIT(0), + [GSI_SNOC_BYPASS_DIS] = BIT(1), + [GEN_QMB_0_SNOC_BYPASS_DIS] = BIT(2), + [GEN_QMB_1_SNOC_BYPASS_DIS] = BIT(3), + [IPA_DCMP_FAST_CLK_EN] = BIT(4), + /* Bits 5-31 reserved */ +}; + +IPA_REG_FIELDS(COMP_CFG, comp_cfg, 0x0000003c); + +static const u32 ipa_reg_clkon_cfg_fmask[] = { + [CLKON_RX] = BIT(0), + [CLKON_PROC] = BIT(1), + [TX_WRAPPER] = BIT(2), + [CLKON_MISC] = BIT(3), + [RAM_ARB] = BIT(4), + [FTCH_HPS] = BIT(5), + [FTCH_DPS] = BIT(6), + [CLKON_HPS] = BIT(7), + [CLKON_DPS] = BIT(8), + [RX_HPS_CMDQS] = BIT(9), + [HPS_DPS_CMDQS] = BIT(10), + [DPS_TX_CMDQS] = BIT(11), + [RSRC_MNGR] = BIT(12), + [CTX_HANDLER] = BIT(13), + [ACK_MNGR] = BIT(14), + [D_DCPH] = BIT(15), + [H_DCPH] = BIT(16), + /* Bit 17 reserved */ + [NTF_TX_CMDQS] = BIT(18), + [CLKON_TX_0] = BIT(19), + [CLKON_TX_1] = BIT(20), + [CLKON_FNR] = BIT(21), + /* Bits 22-31 reserved */ +}; + +IPA_REG_FIELDS(CLKON_CFG, clkon_cfg, 0x00000044); + +static const u32 ipa_reg_route_fmask[] = { + [ROUTE_DIS] = BIT(0), + [ROUTE_DEF_PIPE] = GENMASK(5, 1), + [ROUTE_DEF_HDR_TABLE] = BIT(6), + [ROUTE_DEF_HDR_OFST] = GENMASK(16, 7), + [ROUTE_FRAG_DEF_PIPE] = GENMASK(21, 17), + /* Bits 22-23 reserved */ + [ROUTE_DEF_RETAIN_HDR] = BIT(24), + /* Bits 25-31 reserved */ +}; + +IPA_REG_FIELDS(ROUTE, route, 0x00000048); + +static const u32 ipa_reg_shared_mem_size_fmask[] = { + [MEM_SIZE] = GENMASK(15, 0), + [MEM_BADDR] = GENMASK(31, 16), +}; + +IPA_REG_FIELDS(SHARED_MEM_SIZE, shared_mem_size, 0x00000054); + +static const u32 ipa_reg_qsb_max_writes_fmask[] = { + [GEN_QMB_0_MAX_WRITES] = GENMASK(3, 0), + [GEN_QMB_1_MAX_WRITES] = GENMASK(7, 4), + /* Bits 8-31 reserved */ +}; + +IPA_REG_FIELDS(QSB_MAX_WRITES, qsb_max_writes, 0x00000074); + +static const u32 ipa_reg_qsb_max_reads_fmask[] = { + [GEN_QMB_0_MAX_READS] = GENMASK(3, 0), + [GEN_QMB_1_MAX_READS] = GENMASK(7, 4), +}; + +IPA_REG_FIELDS(QSB_MAX_READS, qsb_max_reads, 0x00000078); + +static const u32 ipa_reg_filt_rout_hash_en_fmask[] = { + [IPV6_ROUTER_HASH] = BIT(0), + /* Bits 1-3 reserved */ + [IPV6_FILTER_HASH] = BIT(4), + /* Bits 5-7 reserved */ + [IPV4_ROUTER_HASH] = BIT(8), + /* Bits 9-11 reserved */ + [IPV4_FILTER_HASH] = BIT(12), + /* Bits 13-31 reserved */ +}; + +IPA_REG_FIELDS(FILT_ROUT_HASH_EN, filt_rout_hash_en, 0x000008c); + +static const u32 ipa_reg_filt_rout_hash_flush_fmask[] = { + [IPV6_ROUTER_HASH] = BIT(0), + /* Bits 1-3 reserved */ + [IPV6_FILTER_HASH] = BIT(4), + /* Bits 5-7 reserved */ + [IPV4_ROUTER_HASH] = BIT(8), + /* Bits 9-11 reserved */ + [IPV4_FILTER_HASH] = BIT(12), + /* Bits 13-31 reserved */ +}; + +IPA_REG_FIELDS(FILT_ROUT_HASH_FLUSH, filt_rout_hash_flush, 0x0000090); + +/* Valid bits defined by ipa->available */ +IPA_REG(STATE_AGGR_ACTIVE, state_aggr_active, 0x0000010c); + +IPA_REG(IPA_BCR, ipa_bcr, 0x000001d0); + +static const u32 ipa_reg_local_pkt_proc_cntxt_fmask[] = { + [IPA_BASE_ADDR] = GENMASK(16, 0), + /* Bits 17-31 reserved */ +}; + +/* Offset must be a multiple of 8 */ +IPA_REG_FIELDS(LOCAL_PKT_PROC_CNTXT, local_pkt_proc_cntxt, 0x000001e8); + +/* Valid bits defined by ipa->available */ +IPA_REG(AGGR_FORCE_CLOSE, aggr_force_close, 0x000001ec); + +static const u32 ipa_reg_counter_cfg_fmask[] = { + /* Bits 0-3 reserved */ + [AGGR_GRANULARITY] = GENMASK(8, 4), + /* Bits 5-31 reserved */ +}; + +IPA_REG_FIELDS(COUNTER_CFG, counter_cfg, 0x000001f0); + +static const u32 ipa_reg_ipa_tx_cfg_fmask[] = { + [TX0_PREFETCH_DISABLE] = BIT(0), + [TX1_PREFETCH_DISABLE] = BIT(1), + [PREFETCH_ALMOST_EMPTY_SIZE] = GENMASK(4, 2), + /* Bits 5-31 reserved */ +}; + +IPA_REG_FIELDS(IPA_TX_CFG, ipa_tx_cfg, 0x000001fc); + +static const u32 ipa_reg_flavor_0_fmask[] = { + [MAX_PIPES] = GENMASK(3, 0), + /* Bits 4-7 reserved */ + [MAX_CONS_PIPES] = GENMASK(12, 8), + /* Bits 13-15 reserved */ + [MAX_PROD_PIPES] = GENMASK(20, 16), + /* Bits 21-23 reserved */ + [PROD_LOWEST] = GENMASK(27, 24), + /* Bits 28-31 reserved */ +}; + +IPA_REG_FIELDS(FLAVOR_0, flavor_0, 0x00000210); + +static const u32 ipa_reg_idle_indication_cfg_fmask[] = { + [ENTER_IDLE_DEBOUNCE_THRESH] = GENMASK(15, 0), + [CONST_NON_IDLE_ENABLE] = BIT(16), + /* Bits 17-31 reserved */ +}; + +IPA_REG_FIELDS(IDLE_INDICATION_CFG, idle_indication_cfg, 0x00000220); + +static const u32 ipa_reg_src_rsrc_grp_01_rsrc_type_fmask[] = { + [X_MIN_LIM] = GENMASK(5, 0), + /* Bits 6-7 reserved */ + [X_MAX_LIM] = GENMASK(13, 8), + /* Bits 14-15 reserved */ + [Y_MIN_LIM] = GENMASK(21, 16), + /* Bits 22-23 reserved */ + [Y_MAX_LIM] = GENMASK(29, 24), + /* Bits 30-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(SRC_RSRC_GRP_01_RSRC_TYPE, src_rsrc_grp_01_rsrc_type, + 0x00000400, 0x0020); + +static const u32 ipa_reg_src_rsrc_grp_23_rsrc_type_fmask[] = { + [X_MIN_LIM] = GENMASK(5, 0), + /* Bits 6-7 reserved */ + [X_MAX_LIM] = GENMASK(13, 8), + /* Bits 14-15 reserved */ + [Y_MIN_LIM] = GENMASK(21, 16), + /* Bits 22-23 reserved */ + [Y_MAX_LIM] = GENMASK(29, 24), + /* Bits 30-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(SRC_RSRC_GRP_23_RSRC_TYPE, src_rsrc_grp_23_rsrc_type, + 0x00000404, 0x0020); + +static const u32 ipa_reg_dst_rsrc_grp_01_rsrc_type_fmask[] = { + [X_MIN_LIM] = GENMASK(5, 0), + /* Bits 6-7 reserved */ + [X_MAX_LIM] = GENMASK(13, 8), + /* Bits 14-15 reserved */ + [Y_MIN_LIM] = GENMASK(21, 16), + /* Bits 22-23 reserved */ + [Y_MAX_LIM] = GENMASK(29, 24), + /* Bits 30-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(DST_RSRC_GRP_01_RSRC_TYPE, dst_rsrc_grp_01_rsrc_type, + 0x00000500, 0x0020); + +static const u32 ipa_reg_dst_rsrc_grp_23_rsrc_type_fmask[] = { + [X_MIN_LIM] = GENMASK(5, 0), + /* Bits 6-7 reserved */ + [X_MAX_LIM] = GENMASK(13, 8), + /* Bits 14-15 reserved */ + [Y_MIN_LIM] = GENMASK(21, 16), + /* Bits 22-23 reserved */ + [Y_MAX_LIM] = GENMASK(29, 24), + /* Bits 30-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(DST_RSRC_GRP_23_RSRC_TYPE, dst_rsrc_grp_23_rsrc_type, + 0x00000504, 0x0020); + +static const u32 ipa_reg_endp_init_ctrl_fmask[] = { + [ENDP_SUSPEND] = BIT(0), + [ENDP_DELAY] = BIT(1), + /* Bits 2-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(ENDP_INIT_CTRL, endp_init_ctrl, 0x00000800, 0x0070); + +static const u32 ipa_reg_endp_init_cfg_fmask[] = { + [FRAG_OFFLOAD_EN] = BIT(0), + [CS_OFFLOAD_EN] = GENMASK(2, 1), + [CS_METADATA_HDR_OFFSET] = GENMASK(6, 3), + /* Bit 7 reserved */ + [CS_GEN_QMB_MASTER_SEL] = BIT(8), + /* Bits 9-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(ENDP_INIT_CFG, endp_init_cfg, 0x00000808, 0x0070); + +static const u32 ipa_reg_endp_init_nat_fmask[] = { + [NAT_EN] = GENMASK(1, 0), + /* Bits 2-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(ENDP_INIT_NAT, endp_init_nat, 0x0000080c, 0x0070); + +static const u32 ipa_reg_endp_init_hdr_fmask[] = { + [HDR_LEN] = GENMASK(5, 0), + [HDR_OFST_METADATA_VALID] = BIT(6), + [HDR_OFST_METADATA] = GENMASK(12, 7), + [HDR_ADDITIONAL_CONST_LEN] = GENMASK(18, 13), + [HDR_OFST_PKT_SIZE_VALID] = BIT(19), + [HDR_OFST_PKT_SIZE] = GENMASK(25, 20), + [HDR_A5_MUX] = BIT(26), + [HDR_LEN_INC_DEAGG_HDR] = BIT(27), + [HDR_METADATA_REG_VALID] = BIT(28), + /* Bits 29-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(ENDP_INIT_HDR, endp_init_hdr, 0x00000810, 0x0070); + +static const u32 ipa_reg_endp_init_hdr_ext_fmask[] = { + [HDR_ENDIANNESS] = BIT(0), + [HDR_TOTAL_LEN_OR_PAD_VALID] = BIT(1), + [HDR_TOTAL_LEN_OR_PAD] = BIT(2), + [HDR_PAYLOAD_LEN_INC_PADDING] = BIT(3), + [HDR_TOTAL_LEN_OR_PAD_OFFSET] = GENMASK(9, 4), + [HDR_PAD_TO_ALIGNMENT] = GENMASK(13, 10), + /* Bits 14-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(ENDP_INIT_HDR_EXT, endp_init_hdr_ext, 0x00000814, 0x0070); + +IPA_REG_STRIDE(ENDP_INIT_HDR_METADATA_MASK, endp_init_hdr_metadata_mask, + 0x00000818, 0x0070); + +static const u32 ipa_reg_endp_init_mode_fmask[] = { + [ENDP_MODE] = GENMASK(2, 0), + /* Bit 3 reserved */ + [DEST_PIPE_INDEX] = GENMASK(8, 4), + /* Bits 9-11 reserved */ + [BYTE_THRESHOLD] = GENMASK(27, 12), + [PIPE_REPLICATION_EN] = BIT(28), + [PAD_EN] = BIT(29), + [HDR_FTCH_DISABLE] = BIT(30), + /* Bit 31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(ENDP_INIT_MODE, endp_init_mode, 0x00000820, 0x0070); + +static const u32 ipa_reg_endp_init_aggr_fmask[] = { + [AGGR_EN] = GENMASK(1, 0), + [AGGR_TYPE] = GENMASK(4, 2), + [BYTE_LIMIT] = GENMASK(9, 5), + [TIME_LIMIT] = GENMASK(14, 10), + [PKT_LIMIT] = GENMASK(20, 15), + [SW_EOF_ACTIVE] = BIT(21), + [FORCE_CLOSE] = BIT(22), + /* Bit 23 reserved */ + [HARD_BYTE_LIMIT_EN] = BIT(24), + /* Bits 25-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(ENDP_INIT_AGGR, endp_init_aggr, 0x00000824, 0x0070); + +static const u32 ipa_reg_endp_init_hol_block_en_fmask[] = { + [HOL_BLOCK_EN] = BIT(0), + /* Bits 1-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(ENDP_INIT_HOL_BLOCK_EN, endp_init_hol_block_en, + 0x0000082c, 0x0070); + +/* Entire register is a tick count */ +static const u32 ipa_reg_endp_init_hol_block_timer_fmask[] = { + [TIMER_BASE_VALUE] = GENMASK(31, 0), +}; + +IPA_REG_STRIDE_FIELDS(ENDP_INIT_HOL_BLOCK_TIMER, endp_init_hol_block_timer, + 0x00000830, 0x0070); + +static const u32 ipa_reg_endp_init_deaggr_fmask[] = { + [DEAGGR_HDR_LEN] = GENMASK(5, 0), + [SYSPIPE_ERR_DETECTION] = BIT(6), + [PACKET_OFFSET_VALID] = BIT(7), + [PACKET_OFFSET_LOCATION] = GENMASK(13, 8), + [IGNORE_MIN_PKT_ERR] = BIT(14), + /* Bit 15 reserved */ + [MAX_PACKET_LEN] = GENMASK(31, 16), +}; + +IPA_REG_STRIDE_FIELDS(ENDP_INIT_DEAGGR, endp_init_deaggr, 0x00000834, 0x0070); + +static const u32 ipa_reg_endp_init_rsrc_grp_fmask[] = { + [ENDP_RSRC_GRP] = GENMASK(1, 0), + /* Bits 2-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(ENDP_INIT_RSRC_GRP, endp_init_rsrc_grp, + 0x00000838, 0x0070); + +static const u32 ipa_reg_endp_init_seq_fmask[] = { + [SEQ_TYPE] = GENMASK(7, 0), + [SEQ_REP_TYPE] = GENMASK(15, 8), + /* Bits 16-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(ENDP_INIT_SEQ, endp_init_seq, 0x0000083c, 0x0070); + +static const u32 ipa_reg_endp_status_fmask[] = { + [STATUS_EN] = BIT(0), + [STATUS_ENDP] = GENMASK(5, 1), + /* Bits 6-7 reserved */ + [STATUS_LOCATION] = BIT(8), + /* Bits 9-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(ENDP_STATUS, endp_status, 0x00000840, 0x0070); + +static const u32 ipa_reg_endp_filter_router_hsh_cfg_fmask[] = { + [FILTER_HASH_MSK_SRC_ID] = BIT(0), + [FILTER_HASH_MSK_SRC_IP] = BIT(1), + [FILTER_HASH_MSK_DST_IP] = BIT(2), + [FILTER_HASH_MSK_SRC_PORT] = BIT(3), + [FILTER_HASH_MSK_DST_PORT] = BIT(4), + [FILTER_HASH_MSK_PROTOCOL] = BIT(5), + [FILTER_HASH_MSK_METADATA] = BIT(6), + [FILTER_HASH_MSK_ALL] = GENMASK(6, 0), + /* Bits 7-15 reserved */ + [ROUTER_HASH_MSK_SRC_ID] = BIT(16), + [ROUTER_HASH_MSK_SRC_IP] = BIT(17), + [ROUTER_HASH_MSK_DST_IP] = BIT(18), + [ROUTER_HASH_MSK_SRC_PORT] = BIT(19), + [ROUTER_HASH_MSK_DST_PORT] = BIT(20), + [ROUTER_HASH_MSK_PROTOCOL] = BIT(21), + [ROUTER_HASH_MSK_METADATA] = BIT(22), + [ROUTER_HASH_MSK_ALL] = GENMASK(22, 16), + /* Bits 23-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(ENDP_FILTER_ROUTER_HSH_CFG, endp_filter_router_hsh_cfg, + 0x0000085c, 0x0070); + +/* Valid bits defined by enum ipa_irq_id; only used for GSI_EE_AP */ +IPA_REG(IPA_IRQ_STTS, ipa_irq_stts, 0x00003008 + 0x1000 * GSI_EE_AP); + +/* Valid bits defined by enum ipa_irq_id; only used for GSI_EE_AP */ +IPA_REG(IPA_IRQ_EN, ipa_irq_en, 0x0000300c + 0x1000 * GSI_EE_AP); + +/* Valid bits defined by enum ipa_irq_id; only used for GSI_EE_AP */ +IPA_REG(IPA_IRQ_CLR, ipa_irq_clr, 0x00003010 + 0x1000 * GSI_EE_AP); + +static const u32 ipa_reg_ipa_irq_uc_fmask[] = { + [UC_INTR] = BIT(0), + /* Bits 1-31 reserved */ +}; + +IPA_REG_FIELDS(IPA_IRQ_UC, ipa_irq_uc, 0x0000301c + 0x1000 * GSI_EE_AP); + +/* Valid bits defined by ipa->available */ +IPA_REG(IRQ_SUSPEND_INFO, irq_suspend_info, 0x00003030 + 0x1000 * GSI_EE_AP); + +/* Valid bits defined by ipa->available */ +IPA_REG(IRQ_SUSPEND_EN, irq_suspend_en, 0x00003034 + 0x1000 * GSI_EE_AP); + +/* Valid bits defined by ipa->available */ +IPA_REG(IRQ_SUSPEND_CLR, irq_suspend_clr, 0x00003038 + 0x1000 * GSI_EE_AP); + +static const struct ipa_reg *ipa_reg_array[] = { + [COMP_CFG] = &ipa_reg_comp_cfg, + [CLKON_CFG] = &ipa_reg_clkon_cfg, + [ROUTE] = &ipa_reg_route, + [SHARED_MEM_SIZE] = &ipa_reg_shared_mem_size, + [QSB_MAX_WRITES] = &ipa_reg_qsb_max_writes, + [QSB_MAX_READS] = &ipa_reg_qsb_max_reads, + [FILT_ROUT_HASH_EN] = &ipa_reg_filt_rout_hash_en, + [FILT_ROUT_HASH_FLUSH] = &ipa_reg_filt_rout_hash_flush, + [STATE_AGGR_ACTIVE] = &ipa_reg_state_aggr_active, + [IPA_BCR] = &ipa_reg_ipa_bcr, + [LOCAL_PKT_PROC_CNTXT] = &ipa_reg_local_pkt_proc_cntxt, + [AGGR_FORCE_CLOSE] = &ipa_reg_aggr_force_close, + [COUNTER_CFG] = &ipa_reg_counter_cfg, + [IPA_TX_CFG] = &ipa_reg_ipa_tx_cfg, + [FLAVOR_0] = &ipa_reg_flavor_0, + [IDLE_INDICATION_CFG] = &ipa_reg_idle_indication_cfg, + [SRC_RSRC_GRP_01_RSRC_TYPE] = &ipa_reg_src_rsrc_grp_01_rsrc_type, + [SRC_RSRC_GRP_23_RSRC_TYPE] = &ipa_reg_src_rsrc_grp_23_rsrc_type, + [DST_RSRC_GRP_01_RSRC_TYPE] = &ipa_reg_dst_rsrc_grp_01_rsrc_type, + [DST_RSRC_GRP_23_RSRC_TYPE] = &ipa_reg_dst_rsrc_grp_23_rsrc_type, + [ENDP_INIT_CTRL] = &ipa_reg_endp_init_ctrl, + [ENDP_INIT_CFG] = &ipa_reg_endp_init_cfg, + [ENDP_INIT_NAT] = &ipa_reg_endp_init_nat, + [ENDP_INIT_HDR] = &ipa_reg_endp_init_hdr, + [ENDP_INIT_HDR_EXT] = &ipa_reg_endp_init_hdr_ext, + [ENDP_INIT_HDR_METADATA_MASK] = &ipa_reg_endp_init_hdr_metadata_mask, + [ENDP_INIT_MODE] = &ipa_reg_endp_init_mode, + [ENDP_INIT_AGGR] = &ipa_reg_endp_init_aggr, + [ENDP_INIT_HOL_BLOCK_EN] = &ipa_reg_endp_init_hol_block_en, + [ENDP_INIT_HOL_BLOCK_TIMER] = &ipa_reg_endp_init_hol_block_timer, + [ENDP_INIT_DEAGGR] = &ipa_reg_endp_init_deaggr, + [ENDP_INIT_RSRC_GRP] = &ipa_reg_endp_init_rsrc_grp, + [ENDP_INIT_SEQ] = &ipa_reg_endp_init_seq, + [ENDP_STATUS] = &ipa_reg_endp_status, + [ENDP_FILTER_ROUTER_HSH_CFG] = &ipa_reg_endp_filter_router_hsh_cfg, + [IPA_IRQ_STTS] = &ipa_reg_ipa_irq_stts, + [IPA_IRQ_EN] = &ipa_reg_ipa_irq_en, + [IPA_IRQ_CLR] = &ipa_reg_ipa_irq_clr, + [IPA_IRQ_UC] = &ipa_reg_ipa_irq_uc, + [IRQ_SUSPEND_INFO] = &ipa_reg_irq_suspend_info, + [IRQ_SUSPEND_EN] = &ipa_reg_irq_suspend_en, + [IRQ_SUSPEND_CLR] = &ipa_reg_irq_suspend_clr, +}; + +const struct ipa_regs ipa_regs_v3_5_1 = { + .reg_count = ARRAY_SIZE(ipa_reg_array), + .reg = ipa_reg_array, +}; diff --git a/drivers/net/ipa/reg/ipa_reg-v4.11.c b/drivers/net/ipa/reg/ipa_reg-v4.11.c new file mode 100644 index 000000000000..8fd36569bb9f --- /dev/null +++ b/drivers/net/ipa/reg/ipa_reg-v4.11.c @@ -0,0 +1,512 @@ +// SPDX-License-Identifier: GPL-2.0 + +/* Copyright (C) 2022 Linaro Ltd. */ + +#include <linux/types.h> + +#include "../ipa.h" +#include "../ipa_reg.h" + +static const u32 ipa_reg_comp_cfg_fmask[] = { + [RAM_ARB_PRI_CLIENT_SAMP_FIX_DIS] = BIT(0), + [GSI_SNOC_BYPASS_DIS] = BIT(1), + [GEN_QMB_0_SNOC_BYPASS_DIS] = BIT(2), + [GEN_QMB_1_SNOC_BYPASS_DIS] = BIT(3), + /* Bit 4 reserved */ + [IPA_QMB_SELECT_CONS_EN] = BIT(5), + [IPA_QMB_SELECT_PROD_EN] = BIT(6), + [GSI_MULTI_INORDER_RD_DIS] = BIT(7), + [GSI_MULTI_INORDER_WR_DIS] = BIT(8), + [GEN_QMB_0_MULTI_INORDER_RD_DIS] = BIT(9), + [GEN_QMB_1_MULTI_INORDER_RD_DIS] = BIT(10), + [GEN_QMB_0_MULTI_INORDER_WR_DIS] = BIT(11), + [GEN_QMB_1_MULTI_INORDER_WR_DIS] = BIT(12), + [GEN_QMB_0_SNOC_CNOC_LOOP_PROT_DIS] = BIT(13), + [GSI_SNOC_CNOC_LOOP_PROT_DISABLE] = BIT(14), + [GSI_MULTI_AXI_MASTERS_DIS] = BIT(15), + [IPA_QMB_SELECT_GLOBAL_EN] = BIT(16), + [FULL_FLUSH_WAIT_RS_CLOSURE_EN] = BIT(17), + /* Bit 18 reserved */ + [QMB_RAM_RD_CACHE_DISABLE] = BIT(19), + [GENQMB_AOOOWR] = BIT(20), + [IF_OUT_OF_BUF_STOP_RESET_MASK_EN] = BIT(21), + [ATOMIC_FETCHER_ARB_LOCK_DIS] = GENMASK(23, 22), + /* Bits 24-29 reserved */ + [GEN_QMB_1_DYNAMIC_ASIZE] = BIT(30), + [GEN_QMB_0_DYNAMIC_ASIZE] = BIT(31), +}; + +IPA_REG_FIELDS(COMP_CFG, comp_cfg, 0x0000003c); + +static const u32 ipa_reg_clkon_cfg_fmask[] = { + [CLKON_RX] = BIT(0), + [CLKON_PROC] = BIT(1), + [TX_WRAPPER] = BIT(2), + [CLKON_MISC] = BIT(3), + [RAM_ARB] = BIT(4), + [FTCH_HPS] = BIT(5), + [FTCH_DPS] = BIT(6), + [CLKON_HPS] = BIT(7), + [CLKON_DPS] = BIT(8), + [RX_HPS_CMDQS] = BIT(9), + [HPS_DPS_CMDQS] = BIT(10), + [DPS_TX_CMDQS] = BIT(11), + [RSRC_MNGR] = BIT(12), + [CTX_HANDLER] = BIT(13), + [ACK_MNGR] = BIT(14), + [D_DCPH] = BIT(15), + [H_DCPH] = BIT(16), + /* Bit 17 reserved */ + [NTF_TX_CMDQS] = BIT(18), + [CLKON_TX_0] = BIT(19), + [CLKON_TX_1] = BIT(20), + [CLKON_FNR] = BIT(21), + [QSB2AXI_CMDQ_L] = BIT(22), + [AGGR_WRAPPER] = BIT(23), + [RAM_SLAVEWAY] = BIT(24), + [CLKON_QMB] = BIT(25), + [WEIGHT_ARB] = BIT(26), + [GSI_IF] = BIT(27), + [CLKON_GLOBAL] = BIT(28), + [GLOBAL_2X_CLK] = BIT(29), + [DPL_FIFO] = BIT(30), + [DRBIP] = BIT(31), +}; + +IPA_REG_FIELDS(CLKON_CFG, clkon_cfg, 0x00000044); + +static const u32 ipa_reg_route_fmask[] = { + [ROUTE_DIS] = BIT(0), + [ROUTE_DEF_PIPE] = GENMASK(5, 1), + [ROUTE_DEF_HDR_TABLE] = BIT(6), + [ROUTE_DEF_HDR_OFST] = GENMASK(16, 7), + [ROUTE_FRAG_DEF_PIPE] = GENMASK(21, 17), + /* Bits 22-23 reserved */ + [ROUTE_DEF_RETAIN_HDR] = BIT(24), + /* Bits 25-31 reserved */ +}; + +IPA_REG_FIELDS(ROUTE, route, 0x00000048); + +static const u32 ipa_reg_shared_mem_size_fmask[] = { + [MEM_SIZE] = GENMASK(15, 0), + [MEM_BADDR] = GENMASK(31, 16), +}; + +IPA_REG_FIELDS(SHARED_MEM_SIZE, shared_mem_size, 0x00000054); + +static const u32 ipa_reg_qsb_max_writes_fmask[] = { + [GEN_QMB_0_MAX_WRITES] = GENMASK(3, 0), + [GEN_QMB_1_MAX_WRITES] = GENMASK(7, 4), + /* Bits 8-31 reserved */ +}; + +IPA_REG_FIELDS(QSB_MAX_WRITES, qsb_max_writes, 0x00000074); + +static const u32 ipa_reg_qsb_max_reads_fmask[] = { + [GEN_QMB_0_MAX_READS] = GENMASK(3, 0), + [GEN_QMB_1_MAX_READS] = GENMASK(7, 4), + /* Bits 8-15 reserved */ + [GEN_QMB_0_MAX_READS_BEATS] = GENMASK(23, 16), + [GEN_QMB_1_MAX_READS_BEATS] = GENMASK(31, 24), +}; + +IPA_REG_FIELDS(QSB_MAX_READS, qsb_max_reads, 0x00000078); + +static const u32 ipa_reg_filt_rout_hash_en_fmask[] = { + [IPV6_ROUTER_HASH] = BIT(0), + /* Bits 1-3 reserved */ + [IPV6_FILTER_HASH] = BIT(4), + /* Bits 5-7 reserved */ + [IPV4_ROUTER_HASH] = BIT(8), + /* Bits 9-11 reserved */ + [IPV4_FILTER_HASH] = BIT(12), + /* Bits 13-31 reserved */ +}; + +IPA_REG_FIELDS(FILT_ROUT_HASH_EN, filt_rout_hash_en, 0x0000148); + +static const u32 ipa_reg_filt_rout_hash_flush_fmask[] = { + [IPV6_ROUTER_HASH] = BIT(0), + /* Bits 1-3 reserved */ + [IPV6_FILTER_HASH] = BIT(4), + /* Bits 5-7 reserved */ + [IPV4_ROUTER_HASH] = BIT(8), + /* Bits 9-11 reserved */ + [IPV4_FILTER_HASH] = BIT(12), + /* Bits 13-31 reserved */ +}; + +IPA_REG_FIELDS(FILT_ROUT_HASH_FLUSH, filt_rout_hash_flush, 0x000014c); + +/* Valid bits defined by ipa->available */ +IPA_REG(STATE_AGGR_ACTIVE, state_aggr_active, 0x000000b4); + +static const u32 ipa_reg_local_pkt_proc_cntxt_fmask[] = { + [IPA_BASE_ADDR] = GENMASK(17, 0), + /* Bits 18-31 reserved */ +}; + +/* Offset must be a multiple of 8 */ +IPA_REG_FIELDS(LOCAL_PKT_PROC_CNTXT, local_pkt_proc_cntxt, 0x000001e8); + +/* Valid bits defined by ipa->available */ +IPA_REG(AGGR_FORCE_CLOSE, aggr_force_close, 0x000001ec); + +static const u32 ipa_reg_ipa_tx_cfg_fmask[] = { + /* Bits 0-1 reserved */ + [PREFETCH_ALMOST_EMPTY_SIZE_TX0] = GENMASK(5, 2), + [DMAW_SCND_OUTSD_PRED_THRESHOLD] = GENMASK(9, 6), + [DMAW_SCND_OUTSD_PRED_EN] = BIT(10), + [DMAW_MAX_BEATS_256_DIS] = BIT(11), + [PA_MASK_EN] = BIT(12), + [PREFETCH_ALMOST_EMPTY_SIZE_TX1] = GENMASK(16, 13), + [DUAL_TX_ENABLE] = BIT(17), + [SSPND_PA_NO_START_STATE] = BIT(18), + /* Bits 19-31 reserved */ +}; + +IPA_REG_FIELDS(IPA_TX_CFG, ipa_tx_cfg, 0x000001fc); + +static const u32 ipa_reg_flavor_0_fmask[] = { + [MAX_PIPES] = GENMASK(4, 0), + /* Bits 5-7 reserved */ + [MAX_CONS_PIPES] = GENMASK(12, 8), + /* Bits 13-15 reserved */ + [MAX_PROD_PIPES] = GENMASK(20, 16), + /* Bits 21-23 reserved */ + [PROD_LOWEST] = GENMASK(27, 24), + /* Bits 28-31 reserved */ +}; + +IPA_REG_FIELDS(FLAVOR_0, flavor_0, 0x00000210); + +static const u32 ipa_reg_idle_indication_cfg_fmask[] = { + [ENTER_IDLE_DEBOUNCE_THRESH] = GENMASK(15, 0), + [CONST_NON_IDLE_ENABLE] = BIT(16), + /* Bits 17-31 reserved */ +}; + +IPA_REG_FIELDS(IDLE_INDICATION_CFG, idle_indication_cfg, 0x00000240); + +static const u32 ipa_reg_qtime_timestamp_cfg_fmask[] = { + [DPL_TIMESTAMP_LSB] = GENMASK(4, 0), + /* Bits 5-6 reserved */ + [DPL_TIMESTAMP_SEL] = BIT(7), + [TAG_TIMESTAMP_LSB] = GENMASK(12, 8), + /* Bits 13-15 reserved */ + [NAT_TIMESTAMP_LSB] = GENMASK(20, 16), + /* Bits 21-31 reserved */ +}; + +IPA_REG_FIELDS(QTIME_TIMESTAMP_CFG, qtime_timestamp_cfg, 0x0000024c); + +static const u32 ipa_reg_timers_xo_clk_div_cfg_fmask[] = { + [DIV_VALUE] = GENMASK(8, 0), + /* Bits 9-30 reserved */ + [DIV_ENABLE] = BIT(31), +}; + +IPA_REG_FIELDS(TIMERS_XO_CLK_DIV_CFG, timers_xo_clk_div_cfg, 0x00000250); + +static const u32 ipa_reg_timers_pulse_gran_cfg_fmask[] = { + [PULSE_GRAN_0] = GENMASK(2, 0), + [PULSE_GRAN_1] = GENMASK(5, 3), + [PULSE_GRAN_2] = GENMASK(8, 6), + /* Bits 9-31 reserved */ +}; + +IPA_REG_FIELDS(TIMERS_PULSE_GRAN_CFG, timers_pulse_gran_cfg, 0x00000254); + +static const u32 ipa_reg_src_rsrc_grp_01_rsrc_type_fmask[] = { + [X_MIN_LIM] = GENMASK(5, 0), + /* Bits 6-7 reserved */ + [X_MAX_LIM] = GENMASK(13, 8), + /* Bits 14-15 reserved */ + [Y_MIN_LIM] = GENMASK(21, 16), + /* Bits 22-23 reserved */ + [Y_MAX_LIM] = GENMASK(29, 24), + /* Bits 30-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(SRC_RSRC_GRP_01_RSRC_TYPE, src_rsrc_grp_01_rsrc_type, + 0x00000400, 0x0020); + +static const u32 ipa_reg_src_rsrc_grp_23_rsrc_type_fmask[] = { + [X_MIN_LIM] = GENMASK(5, 0), + /* Bits 6-7 reserved */ + [X_MAX_LIM] = GENMASK(13, 8), + /* Bits 14-15 reserved */ + [Y_MIN_LIM] = GENMASK(21, 16), + /* Bits 22-23 reserved */ + [Y_MAX_LIM] = GENMASK(29, 24), + /* Bits 30-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(SRC_RSRC_GRP_23_RSRC_TYPE, src_rsrc_grp_23_rsrc_type, + 0x00000404, 0x0020); + +static const u32 ipa_reg_dst_rsrc_grp_01_rsrc_type_fmask[] = { + [X_MIN_LIM] = GENMASK(5, 0), + /* Bits 6-7 reserved */ + [X_MAX_LIM] = GENMASK(13, 8), + /* Bits 14-15 reserved */ + [Y_MIN_LIM] = GENMASK(21, 16), + /* Bits 22-23 reserved */ + [Y_MAX_LIM] = GENMASK(29, 24), + /* Bits 30-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(DST_RSRC_GRP_01_RSRC_TYPE, dst_rsrc_grp_01_rsrc_type, + 0x00000500, 0x0020); + +static const u32 ipa_reg_dst_rsrc_grp_23_rsrc_type_fmask[] = { + [X_MIN_LIM] = GENMASK(5, 0), + /* Bits 6-7 reserved */ + [X_MAX_LIM] = GENMASK(13, 8), + /* Bits 14-15 reserved */ + [Y_MIN_LIM] = GENMASK(21, 16), + /* Bits 22-23 reserved */ + [Y_MAX_LIM] = GENMASK(29, 24), + /* Bits 30-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(DST_RSRC_GRP_23_RSRC_TYPE, dst_rsrc_grp_23_rsrc_type, + 0x00000504, 0x0020); + +static const u32 ipa_reg_endp_init_cfg_fmask[] = { + [FRAG_OFFLOAD_EN] = BIT(0), + [CS_OFFLOAD_EN] = GENMASK(2, 1), + [CS_METADATA_HDR_OFFSET] = GENMASK(6, 3), + /* Bit 7 reserved */ + [CS_GEN_QMB_MASTER_SEL] = BIT(8), + /* Bits 9-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(ENDP_INIT_CFG, endp_init_cfg, 0x00000808, 0x0070); + +static const u32 ipa_reg_endp_init_nat_fmask[] = { + [NAT_EN] = GENMASK(1, 0), + /* Bits 2-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(ENDP_INIT_NAT, endp_init_nat, 0x0000080c, 0x0070); + +static const u32 ipa_reg_endp_init_hdr_fmask[] = { + [HDR_LEN] = GENMASK(5, 0), + [HDR_OFST_METADATA_VALID] = BIT(6), + [HDR_OFST_METADATA] = GENMASK(12, 7), + [HDR_ADDITIONAL_CONST_LEN] = GENMASK(18, 13), + [HDR_OFST_PKT_SIZE_VALID] = BIT(19), + [HDR_OFST_PKT_SIZE] = GENMASK(25, 20), + /* Bit 26 reserved */ + [HDR_LEN_INC_DEAGG_HDR] = BIT(27), + [HDR_LEN_MSB] = GENMASK(29, 28), + [HDR_OFST_METADATA_MSB] = GENMASK(31, 30), +}; + +IPA_REG_STRIDE_FIELDS(ENDP_INIT_HDR, endp_init_hdr, 0x00000810, 0x0070); + +static const u32 ipa_reg_endp_init_hdr_ext_fmask[] = { + [HDR_ENDIANNESS] = BIT(0), + [HDR_TOTAL_LEN_OR_PAD_VALID] = BIT(1), + [HDR_TOTAL_LEN_OR_PAD] = BIT(2), + [HDR_PAYLOAD_LEN_INC_PADDING] = BIT(3), + [HDR_TOTAL_LEN_OR_PAD_OFFSET] = GENMASK(9, 4), + [HDR_PAD_TO_ALIGNMENT] = GENMASK(13, 10), + /* Bits 14-15 reserved */ + [HDR_TOTAL_LEN_OR_PAD_OFFSET_MSB] = GENMASK(17, 16), + [HDR_OFST_PKT_SIZE_MSB] = GENMASK(19, 18), + [HDR_ADDITIONAL_CONST_LEN_MSB] = GENMASK(21, 20), + /* Bits 22-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(ENDP_INIT_HDR_EXT, endp_init_hdr_ext, 0x00000814, 0x0070); + +IPA_REG_STRIDE(ENDP_INIT_HDR_METADATA_MASK, endp_init_hdr_metadata_mask, + 0x00000818, 0x0070); + +static const u32 ipa_reg_endp_init_mode_fmask[] = { + [ENDP_MODE] = GENMASK(2, 0), + [DCPH_ENABLE] = BIT(3), + [DEST_PIPE_INDEX] = GENMASK(8, 4), + /* Bits 9-11 reserved */ + [BYTE_THRESHOLD] = GENMASK(27, 12), + [PIPE_REPLICATION_EN] = BIT(28), + [PAD_EN] = BIT(29), + [DRBIP_ACL_ENABLE] = BIT(30), + /* Bit 31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(ENDP_INIT_MODE, endp_init_mode, 0x00000820, 0x0070); + +static const u32 ipa_reg_endp_init_aggr_fmask[] = { + [AGGR_EN] = GENMASK(1, 0), + [AGGR_TYPE] = GENMASK(4, 2), + [BYTE_LIMIT] = GENMASK(10, 5), + /* Bit 11 reserved */ + [TIME_LIMIT] = GENMASK(16, 12), + [PKT_LIMIT] = GENMASK(22, 17), + [SW_EOF_ACTIVE] = BIT(23), + [FORCE_CLOSE] = BIT(24), + /* Bit 25 reserved */ + [HARD_BYTE_LIMIT_EN] = BIT(26), + [AGGR_GRAN_SEL] = BIT(27), + /* Bits 28-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(ENDP_INIT_AGGR, endp_init_aggr, 0x00000824, 0x0070); + +static const u32 ipa_reg_endp_init_hol_block_en_fmask[] = { + [HOL_BLOCK_EN] = BIT(0), + /* Bits 1-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(ENDP_INIT_HOL_BLOCK_EN, endp_init_hol_block_en, + 0x0000082c, 0x0070); + +static const u32 ipa_reg_endp_init_hol_block_timer_fmask[] = { + [TIMER_LIMIT] = GENMASK(4, 0), + /* Bits 5-7 reserved */ + [TIMER_GRAN_SEL] = BIT(8), + /* Bits 9-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(ENDP_INIT_HOL_BLOCK_TIMER, endp_init_hol_block_timer, + 0x00000830, 0x0070); + +static const u32 ipa_reg_endp_init_deaggr_fmask[] = { + [DEAGGR_HDR_LEN] = GENMASK(5, 0), + [SYSPIPE_ERR_DETECTION] = BIT(6), + [PACKET_OFFSET_VALID] = BIT(7), + [PACKET_OFFSET_LOCATION] = GENMASK(13, 8), + [IGNORE_MIN_PKT_ERR] = BIT(14), + /* Bit 15 reserved */ + [MAX_PACKET_LEN] = GENMASK(31, 16), +}; + +IPA_REG_STRIDE_FIELDS(ENDP_INIT_DEAGGR, endp_init_deaggr, 0x00000834, 0x0070); + +static const u32 ipa_reg_endp_init_rsrc_grp_fmask[] = { + [ENDP_RSRC_GRP] = GENMASK(1, 0), + /* Bits 2-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(ENDP_INIT_RSRC_GRP, endp_init_rsrc_grp, + 0x00000838, 0x0070); + +static const u32 ipa_reg_endp_init_seq_fmask[] = { + [SEQ_TYPE] = GENMASK(7, 0), + /* Bits 8-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(ENDP_INIT_SEQ, endp_init_seq, 0x0000083c, 0x0070); + +static const u32 ipa_reg_endp_status_fmask[] = { + [STATUS_EN] = BIT(0), + [STATUS_ENDP] = GENMASK(5, 1), + /* Bits 6-8 reserved */ + [STATUS_PKT_SUPPRESS] = BIT(9), + /* Bits 10-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(ENDP_STATUS, endp_status, 0x00000840, 0x0070); + +static const u32 ipa_reg_endp_filter_router_hsh_cfg_fmask[] = { + [FILTER_HASH_MSK_SRC_ID] = BIT(0), + [FILTER_HASH_MSK_SRC_IP] = BIT(1), + [FILTER_HASH_MSK_DST_IP] = BIT(2), + [FILTER_HASH_MSK_SRC_PORT] = BIT(3), + [FILTER_HASH_MSK_DST_PORT] = BIT(4), + [FILTER_HASH_MSK_PROTOCOL] = BIT(5), + [FILTER_HASH_MSK_METADATA] = BIT(6), + [FILTER_HASH_MSK_ALL] = GENMASK(6, 0), + /* Bits 7-15 reserved */ + [ROUTER_HASH_MSK_SRC_ID] = BIT(16), + [ROUTER_HASH_MSK_SRC_IP] = BIT(17), + [ROUTER_HASH_MSK_DST_IP] = BIT(18), + [ROUTER_HASH_MSK_SRC_PORT] = BIT(19), + [ROUTER_HASH_MSK_DST_PORT] = BIT(20), + [ROUTER_HASH_MSK_PROTOCOL] = BIT(21), + [ROUTER_HASH_MSK_METADATA] = BIT(22), + [ROUTER_HASH_MSK_ALL] = GENMASK(22, 16), + /* Bits 23-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(ENDP_FILTER_ROUTER_HSH_CFG, endp_filter_router_hsh_cfg, + 0x0000085c, 0x0070); + +/* Valid bits defined by enum ipa_irq_id; only used for GSI_EE_AP */ +IPA_REG(IPA_IRQ_STTS, ipa_irq_stts, 0x00004008 + 0x1000 * GSI_EE_AP); + +/* Valid bits defined by enum ipa_irq_id; only used for GSI_EE_AP */ +IPA_REG(IPA_IRQ_EN, ipa_irq_en, 0x0000400c + 0x1000 * GSI_EE_AP); + +/* Valid bits defined by enum ipa_irq_id; only used for GSI_EE_AP */ +IPA_REG(IPA_IRQ_CLR, ipa_irq_clr, 0x00004010 + 0x1000 * GSI_EE_AP); + +static const u32 ipa_reg_ipa_irq_uc_fmask[] = { + [UC_INTR] = BIT(0), + /* Bits 1-31 reserved */ +}; + +IPA_REG_FIELDS(IPA_IRQ_UC, ipa_irq_uc, 0x0000401c + 0x1000 * GSI_EE_AP); + +/* Valid bits defined by ipa->available */ +IPA_REG(IRQ_SUSPEND_INFO, irq_suspend_info, 0x00004030 + 0x1000 * GSI_EE_AP); + +/* Valid bits defined by ipa->available */ +IPA_REG(IRQ_SUSPEND_EN, irq_suspend_en, 0x00004034 + 0x1000 * GSI_EE_AP); + +/* Valid bits defined by ipa->available */ +IPA_REG(IRQ_SUSPEND_CLR, irq_suspend_clr, 0x00004038 + 0x1000 * GSI_EE_AP); + +static const struct ipa_reg *ipa_reg_array[] = { + [COMP_CFG] = &ipa_reg_comp_cfg, + [CLKON_CFG] = &ipa_reg_clkon_cfg, + [ROUTE] = &ipa_reg_route, + [SHARED_MEM_SIZE] = &ipa_reg_shared_mem_size, + [QSB_MAX_WRITES] = &ipa_reg_qsb_max_writes, + [QSB_MAX_READS] = &ipa_reg_qsb_max_reads, + [FILT_ROUT_HASH_EN] = &ipa_reg_filt_rout_hash_en, + [FILT_ROUT_HASH_FLUSH] = &ipa_reg_filt_rout_hash_flush, + [STATE_AGGR_ACTIVE] = &ipa_reg_state_aggr_active, + [LOCAL_PKT_PROC_CNTXT] = &ipa_reg_local_pkt_proc_cntxt, + [AGGR_FORCE_CLOSE] = &ipa_reg_aggr_force_close, + [IPA_TX_CFG] = &ipa_reg_ipa_tx_cfg, + [FLAVOR_0] = &ipa_reg_flavor_0, + [IDLE_INDICATION_CFG] = &ipa_reg_idle_indication_cfg, + [QTIME_TIMESTAMP_CFG] = &ipa_reg_qtime_timestamp_cfg, + [TIMERS_XO_CLK_DIV_CFG] = &ipa_reg_timers_xo_clk_div_cfg, + [TIMERS_PULSE_GRAN_CFG] = &ipa_reg_timers_pulse_gran_cfg, + [SRC_RSRC_GRP_01_RSRC_TYPE] = &ipa_reg_src_rsrc_grp_01_rsrc_type, + [SRC_RSRC_GRP_23_RSRC_TYPE] = &ipa_reg_src_rsrc_grp_23_rsrc_type, + [DST_RSRC_GRP_01_RSRC_TYPE] = &ipa_reg_dst_rsrc_grp_01_rsrc_type, + [DST_RSRC_GRP_23_RSRC_TYPE] = &ipa_reg_dst_rsrc_grp_23_rsrc_type, + [ENDP_INIT_CFG] = &ipa_reg_endp_init_cfg, + [ENDP_INIT_NAT] = &ipa_reg_endp_init_nat, + [ENDP_INIT_HDR] = &ipa_reg_endp_init_hdr, + [ENDP_INIT_HDR_EXT] = &ipa_reg_endp_init_hdr_ext, + [ENDP_INIT_HDR_METADATA_MASK] = &ipa_reg_endp_init_hdr_metadata_mask, + [ENDP_INIT_MODE] = &ipa_reg_endp_init_mode, + [ENDP_INIT_AGGR] = &ipa_reg_endp_init_aggr, + [ENDP_INIT_HOL_BLOCK_EN] = &ipa_reg_endp_init_hol_block_en, + [ENDP_INIT_HOL_BLOCK_TIMER] = &ipa_reg_endp_init_hol_block_timer, + [ENDP_INIT_DEAGGR] = &ipa_reg_endp_init_deaggr, + [ENDP_INIT_RSRC_GRP] = &ipa_reg_endp_init_rsrc_grp, + [ENDP_INIT_SEQ] = &ipa_reg_endp_init_seq, + [ENDP_STATUS] = &ipa_reg_endp_status, + [ENDP_FILTER_ROUTER_HSH_CFG] = &ipa_reg_endp_filter_router_hsh_cfg, + [IPA_IRQ_STTS] = &ipa_reg_ipa_irq_stts, + [IPA_IRQ_EN] = &ipa_reg_ipa_irq_en, + [IPA_IRQ_CLR] = &ipa_reg_ipa_irq_clr, + [IPA_IRQ_UC] = &ipa_reg_ipa_irq_uc, + [IRQ_SUSPEND_INFO] = &ipa_reg_irq_suspend_info, + [IRQ_SUSPEND_EN] = &ipa_reg_irq_suspend_en, + [IRQ_SUSPEND_CLR] = &ipa_reg_irq_suspend_clr, +}; + +const struct ipa_regs ipa_regs_v4_11 = { + .reg_count = ARRAY_SIZE(ipa_reg_array), + .reg = ipa_reg_array, +}; diff --git a/drivers/net/ipa/reg/ipa_reg-v4.2.c b/drivers/net/ipa/reg/ipa_reg-v4.2.c new file mode 100644 index 000000000000..f8e78e1907c8 --- /dev/null +++ b/drivers/net/ipa/reg/ipa_reg-v4.2.c @@ -0,0 +1,456 @@ +// SPDX-License-Identifier: GPL-2.0 + +/* Copyright (C) 2022 Linaro Ltd. */ + +#include <linux/types.h> + +#include "../ipa.h" +#include "../ipa_reg.h" + +static const u32 ipa_reg_comp_cfg_fmask[] = { + /* Bit 0 reserved */ + [GSI_SNOC_BYPASS_DIS] = BIT(1), + [GEN_QMB_0_SNOC_BYPASS_DIS] = BIT(2), + [GEN_QMB_1_SNOC_BYPASS_DIS] = BIT(3), + [IPA_DCMP_FAST_CLK_EN] = BIT(4), + [IPA_QMB_SELECT_CONS_EN] = BIT(5), + [IPA_QMB_SELECT_PROD_EN] = BIT(6), + [GSI_MULTI_INORDER_RD_DIS] = BIT(7), + [GSI_MULTI_INORDER_WR_DIS] = BIT(8), + [GEN_QMB_0_MULTI_INORDER_RD_DIS] = BIT(9), + [GEN_QMB_1_MULTI_INORDER_RD_DIS] = BIT(10), + [GEN_QMB_0_MULTI_INORDER_WR_DIS] = BIT(11), + [GEN_QMB_1_MULTI_INORDER_WR_DIS] = BIT(12), + [GEN_QMB_0_SNOC_CNOC_LOOP_PROT_DIS] = BIT(13), + [GSI_SNOC_CNOC_LOOP_PROT_DISABLE] = BIT(14), + [GSI_MULTI_AXI_MASTERS_DIS] = BIT(15), + [IPA_QMB_SELECT_GLOBAL_EN] = BIT(16), + [ATOMIC_FETCHER_ARB_LOCK_DIS] = GENMASK(20, 17), + /* Bits 21-31 reserved */ +}; + +IPA_REG_FIELDS(COMP_CFG, comp_cfg, 0x0000003c); + +static const u32 ipa_reg_clkon_cfg_fmask[] = { + [CLKON_RX] = BIT(0), + [CLKON_PROC] = BIT(1), + [TX_WRAPPER] = BIT(2), + [CLKON_MISC] = BIT(3), + [RAM_ARB] = BIT(4), + [FTCH_HPS] = BIT(5), + [FTCH_DPS] = BIT(6), + [CLKON_HPS] = BIT(7), + [CLKON_DPS] = BIT(8), + [RX_HPS_CMDQS] = BIT(9), + [HPS_DPS_CMDQS] = BIT(10), + [DPS_TX_CMDQS] = BIT(11), + [RSRC_MNGR] = BIT(12), + [CTX_HANDLER] = BIT(13), + [ACK_MNGR] = BIT(14), + [D_DCPH] = BIT(15), + [H_DCPH] = BIT(16), + /* Bit 17 reserved */ + [NTF_TX_CMDQS] = BIT(18), + [CLKON_TX_0] = BIT(19), + [CLKON_TX_1] = BIT(20), + [CLKON_FNR] = BIT(21), + [QSB2AXI_CMDQ_L] = BIT(22), + [AGGR_WRAPPER] = BIT(23), + [RAM_SLAVEWAY] = BIT(24), + [CLKON_QMB] = BIT(25), + [WEIGHT_ARB] = BIT(26), + [GSI_IF] = BIT(27), + [CLKON_GLOBAL] = BIT(28), + [GLOBAL_2X_CLK] = BIT(29), + /* Bits 30-31 reserved */ +}; + +IPA_REG_FIELDS(CLKON_CFG, clkon_cfg, 0x00000044); + +static const u32 ipa_reg_route_fmask[] = { + [ROUTE_DIS] = BIT(0), + [ROUTE_DEF_PIPE] = GENMASK(5, 1), + [ROUTE_DEF_HDR_TABLE] = BIT(6), + [ROUTE_DEF_HDR_OFST] = GENMASK(16, 7), + [ROUTE_FRAG_DEF_PIPE] = GENMASK(21, 17), + /* Bits 22-23 reserved */ + [ROUTE_DEF_RETAIN_HDR] = BIT(24), + /* Bits 25-31 reserved */ +}; + +IPA_REG_FIELDS(ROUTE, route, 0x00000048); + +static const u32 ipa_reg_shared_mem_size_fmask[] = { + [MEM_SIZE] = GENMASK(15, 0), + [MEM_BADDR] = GENMASK(31, 16), +}; + +IPA_REG_FIELDS(SHARED_MEM_SIZE, shared_mem_size, 0x00000054); + +static const u32 ipa_reg_qsb_max_writes_fmask[] = { + [GEN_QMB_0_MAX_WRITES] = GENMASK(3, 0), + [GEN_QMB_1_MAX_WRITES] = GENMASK(7, 4), + /* Bits 8-31 reserved */ +}; + +IPA_REG_FIELDS(QSB_MAX_WRITES, qsb_max_writes, 0x00000074); + +static const u32 ipa_reg_qsb_max_reads_fmask[] = { + [GEN_QMB_0_MAX_READS] = GENMASK(3, 0), + [GEN_QMB_1_MAX_READS] = GENMASK(7, 4), + /* Bits 8-15 reserved */ + [GEN_QMB_0_MAX_READS_BEATS] = GENMASK(23, 16), + [GEN_QMB_1_MAX_READS_BEATS] = GENMASK(31, 24), +}; + +IPA_REG_FIELDS(QSB_MAX_READS, qsb_max_reads, 0x00000078); + +static const u32 ipa_reg_filt_rout_hash_en_fmask[] = { + [IPV6_ROUTER_HASH] = BIT(0), + /* Bits 1-3 reserved */ + [IPV6_FILTER_HASH] = BIT(4), + /* Bits 5-7 reserved */ + [IPV4_ROUTER_HASH] = BIT(8), + /* Bits 9-11 reserved */ + [IPV4_FILTER_HASH] = BIT(12), + /* Bits 13-31 reserved */ +}; + +IPA_REG_FIELDS(FILT_ROUT_HASH_EN, filt_rout_hash_en, 0x0000148); + +static const u32 ipa_reg_filt_rout_hash_flush_fmask[] = { + [IPV6_ROUTER_HASH] = BIT(0), + /* Bits 1-3 reserved */ + [IPV6_FILTER_HASH] = BIT(4), + /* Bits 5-7 reserved */ + [IPV4_ROUTER_HASH] = BIT(8), + /* Bits 9-11 reserved */ + [IPV4_FILTER_HASH] = BIT(12), + /* Bits 13-31 reserved */ +}; + +IPA_REG_FIELDS(FILT_ROUT_HASH_FLUSH, filt_rout_hash_flush, 0x000014c); + +/* Valid bits defined by ipa->available */ +IPA_REG(STATE_AGGR_ACTIVE, state_aggr_active, 0x000000b4); + +IPA_REG(IPA_BCR, ipa_bcr, 0x000001d0); + +static const u32 ipa_reg_local_pkt_proc_cntxt_fmask[] = { + [IPA_BASE_ADDR] = GENMASK(16, 0), + /* Bits 17-31 reserved */ +}; + +/* Offset must be a multiple of 8 */ +IPA_REG_FIELDS(LOCAL_PKT_PROC_CNTXT, local_pkt_proc_cntxt, 0x000001e8); + +/* Valid bits defined by ipa->available */ +IPA_REG(AGGR_FORCE_CLOSE, aggr_force_close, 0x000001ec); + +static const u32 ipa_reg_counter_cfg_fmask[] = { + /* Bits 0-3 reserved */ + [AGGR_GRANULARITY] = GENMASK(8, 4), + /* Bits 9-31 reserved */ +}; + +IPA_REG_FIELDS(COUNTER_CFG, counter_cfg, 0x000001f0); + +static const u32 ipa_reg_ipa_tx_cfg_fmask[] = { + /* Bits 0-1 reserved */ + [PREFETCH_ALMOST_EMPTY_SIZE_TX0] = GENMASK(5, 2), + [DMAW_SCND_OUTSD_PRED_THRESHOLD] = GENMASK(9, 6), + [DMAW_SCND_OUTSD_PRED_EN] = BIT(10), + [DMAW_MAX_BEATS_256_DIS] = BIT(11), + [PA_MASK_EN] = BIT(12), + [PREFETCH_ALMOST_EMPTY_SIZE_TX1] = GENMASK(16, 13), + /* Bit 17 reserved */ + [SSPND_PA_NO_START_STATE] = BIT(18), + [SSPND_PA_NO_BQ_STATE] = BIT(19), + /* Bits 20-31 reserved */ +}; + +IPA_REG_FIELDS(IPA_TX_CFG, ipa_tx_cfg, 0x000001fc); + +static const u32 ipa_reg_flavor_0_fmask[] = { + [MAX_PIPES] = GENMASK(3, 0), + /* Bits 4-7 reserved */ + [MAX_CONS_PIPES] = GENMASK(12, 8), + /* Bits 13-15 reserved */ + [MAX_PROD_PIPES] = GENMASK(20, 16), + /* Bits 21-23 reserved */ + [PROD_LOWEST] = GENMASK(27, 24), + /* Bits 28-31 reserved */ +}; + +IPA_REG_FIELDS(FLAVOR_0, flavor_0, 0x00000210); + +static const u32 ipa_reg_idle_indication_cfg_fmask[] = { + [ENTER_IDLE_DEBOUNCE_THRESH] = GENMASK(15, 0), + [CONST_NON_IDLE_ENABLE] = BIT(16), + /* Bits 17-31 reserved */ +}; + +IPA_REG_FIELDS(IDLE_INDICATION_CFG, idle_indication_cfg, 0x00000240); + +static const u32 ipa_reg_src_rsrc_grp_01_rsrc_type_fmask[] = { + [X_MIN_LIM] = GENMASK(5, 0), + /* Bits 6-7 reserved */ + [X_MAX_LIM] = GENMASK(13, 8), + /* Bits 14-15 reserved */ + [Y_MIN_LIM] = GENMASK(21, 16), + /* Bits 22-23 reserved */ + [Y_MAX_LIM] = GENMASK(29, 24), + /* Bits 30-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(SRC_RSRC_GRP_01_RSRC_TYPE, src_rsrc_grp_01_rsrc_type, + 0x00000400, 0x0020); + +static const u32 ipa_reg_src_rsrc_grp_23_rsrc_type_fmask[] = { + [X_MIN_LIM] = GENMASK(5, 0), + /* Bits 6-7 reserved */ + [X_MAX_LIM] = GENMASK(13, 8), + /* Bits 14-15 reserved */ + [Y_MIN_LIM] = GENMASK(21, 16), + /* Bits 22-23 reserved */ + [Y_MAX_LIM] = GENMASK(29, 24), + /* Bits 30-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(SRC_RSRC_GRP_23_RSRC_TYPE, src_rsrc_grp_23_rsrc_type, + 0x00000404, 0x0020); + +static const u32 ipa_reg_dst_rsrc_grp_01_rsrc_type_fmask[] = { + [X_MIN_LIM] = GENMASK(5, 0), + /* Bits 6-7 reserved */ + [X_MAX_LIM] = GENMASK(13, 8), + /* Bits 14-15 reserved */ + [Y_MIN_LIM] = GENMASK(21, 16), + /* Bits 22-23 reserved */ + [Y_MAX_LIM] = GENMASK(29, 24), + /* Bits 30-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(DST_RSRC_GRP_01_RSRC_TYPE, dst_rsrc_grp_01_rsrc_type, + 0x00000500, 0x0020); + +static const u32 ipa_reg_dst_rsrc_grp_23_rsrc_type_fmask[] = { + [X_MIN_LIM] = GENMASK(5, 0), + /* Bits 6-7 reserved */ + [X_MAX_LIM] = GENMASK(13, 8), + /* Bits 14-15 reserved */ + [Y_MIN_LIM] = GENMASK(21, 16), + /* Bits 22-23 reserved */ + [Y_MAX_LIM] = GENMASK(29, 24), + /* Bits 30-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(DST_RSRC_GRP_23_RSRC_TYPE, dst_rsrc_grp_23_rsrc_type, + 0x00000504, 0x0020); + +static const u32 ipa_reg_endp_init_cfg_fmask[] = { + [FRAG_OFFLOAD_EN] = BIT(0), + [CS_OFFLOAD_EN] = GENMASK(2, 1), + [CS_METADATA_HDR_OFFSET] = GENMASK(6, 3), + /* Bit 7 reserved */ + [CS_GEN_QMB_MASTER_SEL] = BIT(8), + /* Bits 9-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(ENDP_INIT_CFG, endp_init_cfg, 0x00000808, 0x0070); + +static const u32 ipa_reg_endp_init_nat_fmask[] = { + [NAT_EN] = GENMASK(1, 0), + /* Bits 2-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(ENDP_INIT_NAT, endp_init_nat, 0x0000080c, 0x0070); + +static const u32 ipa_reg_endp_init_hdr_fmask[] = { + [HDR_LEN] = GENMASK(5, 0), + [HDR_OFST_METADATA_VALID] = BIT(6), + [HDR_OFST_METADATA] = GENMASK(12, 7), + [HDR_ADDITIONAL_CONST_LEN] = GENMASK(18, 13), + [HDR_OFST_PKT_SIZE_VALID] = BIT(19), + [HDR_OFST_PKT_SIZE] = GENMASK(25, 20), + [HDR_A5_MUX] = BIT(26), + [HDR_LEN_INC_DEAGG_HDR] = BIT(27), + [HDR_METADATA_REG_VALID] = BIT(28), + /* Bits 29-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(ENDP_INIT_HDR, endp_init_hdr, 0x00000810, 0x0070); + +static const u32 ipa_reg_endp_init_hdr_ext_fmask[] = { + [HDR_ENDIANNESS] = BIT(0), + [HDR_TOTAL_LEN_OR_PAD_VALID] = BIT(1), + [HDR_TOTAL_LEN_OR_PAD] = BIT(2), + [HDR_PAYLOAD_LEN_INC_PADDING] = BIT(3), + [HDR_TOTAL_LEN_OR_PAD_OFFSET] = GENMASK(9, 4), + [HDR_PAD_TO_ALIGNMENT] = GENMASK(13, 10), + /* Bits 14-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(ENDP_INIT_HDR_EXT, endp_init_hdr_ext, 0x00000814, 0x0070); + +IPA_REG_STRIDE(ENDP_INIT_HDR_METADATA_MASK, endp_init_hdr_metadata_mask, + 0x00000818, 0x0070); + +static const u32 ipa_reg_endp_init_mode_fmask[] = { + [ENDP_MODE] = GENMASK(2, 0), + /* Bit 3 reserved */ + [DEST_PIPE_INDEX] = GENMASK(8, 4), + /* Bits 9-11 reserved */ + [BYTE_THRESHOLD] = GENMASK(27, 12), + [PIPE_REPLICATION_EN] = BIT(28), + [PAD_EN] = BIT(29), + [HDR_FTCH_DISABLE] = BIT(30), + /* Bit 31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(ENDP_INIT_MODE, endp_init_mode, 0x00000820, 0x0070); + +static const u32 ipa_reg_endp_init_aggr_fmask[] = { + [AGGR_EN] = GENMASK(1, 0), + [AGGR_TYPE] = GENMASK(4, 2), + [BYTE_LIMIT] = GENMASK(9, 5), + [TIME_LIMIT] = GENMASK(14, 10), + [PKT_LIMIT] = GENMASK(20, 15), + [SW_EOF_ACTIVE] = BIT(21), + [FORCE_CLOSE] = BIT(22), + /* Bit 23 reserved */ + [HARD_BYTE_LIMIT_EN] = BIT(24), + /* Bits 25-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(ENDP_INIT_AGGR, endp_init_aggr, 0x00000824, 0x0070); + +static const u32 ipa_reg_endp_init_hol_block_en_fmask[] = { + [HOL_BLOCK_EN] = BIT(0), + /* Bits 1-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(ENDP_INIT_HOL_BLOCK_EN, endp_init_hol_block_en, + 0x0000082c, 0x0070); + +static const u32 ipa_reg_endp_init_hol_block_timer_fmask[] = { + [TIMER_BASE_VALUE] = GENMASK(4, 0), + /* Bits 5-7 reserved */ + [TIMER_SCALE] = GENMASK(12, 8), + /* Bits 9-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(ENDP_INIT_HOL_BLOCK_TIMER, endp_init_hol_block_timer, + 0x00000830, 0x0070); + +static const u32 ipa_reg_endp_init_deaggr_fmask[] = { + [DEAGGR_HDR_LEN] = GENMASK(5, 0), + [SYSPIPE_ERR_DETECTION] = BIT(6), + [PACKET_OFFSET_VALID] = BIT(7), + [PACKET_OFFSET_LOCATION] = GENMASK(13, 8), + [IGNORE_MIN_PKT_ERR] = BIT(14), + /* Bit 15 reserved */ + [MAX_PACKET_LEN] = GENMASK(31, 16), +}; + +IPA_REG_STRIDE_FIELDS(ENDP_INIT_DEAGGR, endp_init_deaggr, 0x00000834, 0x0070); + +static const u32 ipa_reg_endp_init_rsrc_grp_fmask[] = { + [ENDP_RSRC_GRP] = BIT(0), + /* Bits 1-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(ENDP_INIT_RSRC_GRP, endp_init_rsrc_grp, + 0x00000838, 0x0070); + +static const u32 ipa_reg_endp_init_seq_fmask[] = { + [SEQ_TYPE] = GENMASK(7, 0), + [SEQ_REP_TYPE] = GENMASK(15, 8), + /* Bits 16-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(ENDP_INIT_SEQ, endp_init_seq, 0x0000083c, 0x0070); + +static const u32 ipa_reg_endp_status_fmask[] = { + [STATUS_EN] = BIT(0), + [STATUS_ENDP] = GENMASK(5, 1), + /* Bits 6-7 reserved */ + [STATUS_LOCATION] = BIT(8), + [STATUS_PKT_SUPPRESS] = BIT(9), + /* Bits 10-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(ENDP_STATUS, endp_status, 0x00000840, 0x0070); + +/* Valid bits defined by enum ipa_irq_id; only used for GSI_EE_AP */ +IPA_REG(IPA_IRQ_STTS, ipa_irq_stts, 0x00003008 + 0x1000 * GSI_EE_AP); + +/* Valid bits defined by enum ipa_irq_id; only used for GSI_EE_AP */ +IPA_REG(IPA_IRQ_EN, ipa_irq_en, 0x0000300c + 0x1000 * GSI_EE_AP); + +/* Valid bits defined by enum ipa_irq_id; only used for GSI_EE_AP */ +IPA_REG(IPA_IRQ_CLR, ipa_irq_clr, 0x00003010 + 0x1000 * GSI_EE_AP); + +static const u32 ipa_reg_ipa_irq_uc_fmask[] = { + [UC_INTR] = BIT(0), + /* Bits 1-31 reserved */ +}; + +IPA_REG_FIELDS(IPA_IRQ_UC, ipa_irq_uc, 0x0000301c + 0x1000 * GSI_EE_AP); + +/* Valid bits defined by ipa->available */ +IPA_REG(IRQ_SUSPEND_INFO, irq_suspend_info, 0x00003030 + 0x1000 * GSI_EE_AP); + +/* Valid bits defined by ipa->available */ +IPA_REG(IRQ_SUSPEND_EN, irq_suspend_en, 0x00003034 + 0x1000 * GSI_EE_AP); + +/* Valid bits defined by ipa->available */ +IPA_REG(IRQ_SUSPEND_CLR, irq_suspend_clr, 0x00003038 + 0x1000 * GSI_EE_AP); + +static const struct ipa_reg *ipa_reg_array[] = { + [COMP_CFG] = &ipa_reg_comp_cfg, + [CLKON_CFG] = &ipa_reg_clkon_cfg, + [ROUTE] = &ipa_reg_route, + [SHARED_MEM_SIZE] = &ipa_reg_shared_mem_size, + [QSB_MAX_WRITES] = &ipa_reg_qsb_max_writes, + [QSB_MAX_READS] = &ipa_reg_qsb_max_reads, + [FILT_ROUT_HASH_EN] = &ipa_reg_filt_rout_hash_en, + [FILT_ROUT_HASH_FLUSH] = &ipa_reg_filt_rout_hash_flush, + [STATE_AGGR_ACTIVE] = &ipa_reg_state_aggr_active, + [IPA_BCR] = &ipa_reg_ipa_bcr, + [LOCAL_PKT_PROC_CNTXT] = &ipa_reg_local_pkt_proc_cntxt, + [AGGR_FORCE_CLOSE] = &ipa_reg_aggr_force_close, + [COUNTER_CFG] = &ipa_reg_counter_cfg, + [IPA_TX_CFG] = &ipa_reg_ipa_tx_cfg, + [FLAVOR_0] = &ipa_reg_flavor_0, + [IDLE_INDICATION_CFG] = &ipa_reg_idle_indication_cfg, + [SRC_RSRC_GRP_01_RSRC_TYPE] = &ipa_reg_src_rsrc_grp_01_rsrc_type, + [SRC_RSRC_GRP_23_RSRC_TYPE] = &ipa_reg_src_rsrc_grp_23_rsrc_type, + [DST_RSRC_GRP_01_RSRC_TYPE] = &ipa_reg_dst_rsrc_grp_01_rsrc_type, + [DST_RSRC_GRP_23_RSRC_TYPE] = &ipa_reg_dst_rsrc_grp_23_rsrc_type, + [ENDP_INIT_CFG] = &ipa_reg_endp_init_cfg, + [ENDP_INIT_NAT] = &ipa_reg_endp_init_nat, + [ENDP_INIT_HDR] = &ipa_reg_endp_init_hdr, + [ENDP_INIT_HDR_EXT] = &ipa_reg_endp_init_hdr_ext, + [ENDP_INIT_HDR_METADATA_MASK] = &ipa_reg_endp_init_hdr_metadata_mask, + [ENDP_INIT_MODE] = &ipa_reg_endp_init_mode, + [ENDP_INIT_AGGR] = &ipa_reg_endp_init_aggr, + [ENDP_INIT_HOL_BLOCK_EN] = &ipa_reg_endp_init_hol_block_en, + [ENDP_INIT_HOL_BLOCK_TIMER] = &ipa_reg_endp_init_hol_block_timer, + [ENDP_INIT_DEAGGR] = &ipa_reg_endp_init_deaggr, + [ENDP_INIT_RSRC_GRP] = &ipa_reg_endp_init_rsrc_grp, + [ENDP_INIT_SEQ] = &ipa_reg_endp_init_seq, + [ENDP_STATUS] = &ipa_reg_endp_status, + [IPA_IRQ_STTS] = &ipa_reg_ipa_irq_stts, + [IPA_IRQ_EN] = &ipa_reg_ipa_irq_en, + [IPA_IRQ_CLR] = &ipa_reg_ipa_irq_clr, + [IPA_IRQ_UC] = &ipa_reg_ipa_irq_uc, + [IRQ_SUSPEND_INFO] = &ipa_reg_irq_suspend_info, + [IRQ_SUSPEND_EN] = &ipa_reg_irq_suspend_en, + [IRQ_SUSPEND_CLR] = &ipa_reg_irq_suspend_clr, +}; + +const struct ipa_regs ipa_regs_v4_2 = { + .reg_count = ARRAY_SIZE(ipa_reg_array), + .reg = ipa_reg_array, +}; diff --git a/drivers/net/ipa/reg/ipa_reg-v4.5.c b/drivers/net/ipa/reg/ipa_reg-v4.5.c new file mode 100644 index 000000000000..d32b805abb11 --- /dev/null +++ b/drivers/net/ipa/reg/ipa_reg-v4.5.c @@ -0,0 +1,533 @@ +// SPDX-License-Identifier: GPL-2.0 + +/* Copyright (C) 2022 Linaro Ltd. */ + +#include <linux/types.h> + +#include "../ipa.h" +#include "../ipa_reg.h" + +static const u32 ipa_reg_comp_cfg_fmask[] = { + /* Bit 0 reserved */ + [GSI_SNOC_BYPASS_DIS] = BIT(1), + [GEN_QMB_0_SNOC_BYPASS_DIS] = BIT(2), + [GEN_QMB_1_SNOC_BYPASS_DIS] = BIT(3), + /* Bit 4 reserved */ + [IPA_QMB_SELECT_CONS_EN] = BIT(5), + [IPA_QMB_SELECT_PROD_EN] = BIT(6), + [GSI_MULTI_INORDER_RD_DIS] = BIT(7), + [GSI_MULTI_INORDER_WR_DIS] = BIT(8), + [GEN_QMB_0_MULTI_INORDER_RD_DIS] = BIT(9), + [GEN_QMB_1_MULTI_INORDER_RD_DIS] = BIT(10), + [GEN_QMB_0_MULTI_INORDER_WR_DIS] = BIT(11), + [GEN_QMB_1_MULTI_INORDER_WR_DIS] = BIT(12), + [GEN_QMB_0_SNOC_CNOC_LOOP_PROT_DIS] = BIT(13), + [GSI_SNOC_CNOC_LOOP_PROT_DISABLE] = BIT(14), + [GSI_MULTI_AXI_MASTERS_DIS] = BIT(15), + [IPA_QMB_SELECT_GLOBAL_EN] = BIT(16), + [ATOMIC_FETCHER_ARB_LOCK_DIS] = GENMASK(20, 17), + [FULL_FLUSH_WAIT_RS_CLOSURE_EN] = BIT(21), + /* Bits 22-31 reserved */ +}; + +IPA_REG_FIELDS(COMP_CFG, comp_cfg, 0x0000003c); + +static const u32 ipa_reg_clkon_cfg_fmask[] = { + [CLKON_RX] = BIT(0), + [CLKON_PROC] = BIT(1), + [TX_WRAPPER] = BIT(2), + [CLKON_MISC] = BIT(3), + [RAM_ARB] = BIT(4), + [FTCH_HPS] = BIT(5), + [FTCH_DPS] = BIT(6), + [CLKON_HPS] = BIT(7), + [CLKON_DPS] = BIT(8), + [RX_HPS_CMDQS] = BIT(9), + [HPS_DPS_CMDQS] = BIT(10), + [DPS_TX_CMDQS] = BIT(11), + [RSRC_MNGR] = BIT(12), + [CTX_HANDLER] = BIT(13), + [ACK_MNGR] = BIT(14), + [D_DCPH] = BIT(15), + [H_DCPH] = BIT(16), + [CLKON_DCMP] = BIT(17), + [NTF_TX_CMDQS] = BIT(18), + [CLKON_TX_0] = BIT(19), + [CLKON_TX_1] = BIT(20), + [CLKON_FNR] = BIT(21), + [QSB2AXI_CMDQ_L] = BIT(22), + [AGGR_WRAPPER] = BIT(23), + [RAM_SLAVEWAY] = BIT(24), + [CLKON_QMB] = BIT(25), + [WEIGHT_ARB] = BIT(26), + [GSI_IF] = BIT(27), + [CLKON_GLOBAL] = BIT(28), + [GLOBAL_2X_CLK] = BIT(29), + [DPL_FIFO] = BIT(30), + /* Bit 31 reserved */ +}; + +IPA_REG_FIELDS(CLKON_CFG, clkon_cfg, 0x00000044); + +static const u32 ipa_reg_route_fmask[] = { + [ROUTE_DIS] = BIT(0), + [ROUTE_DEF_PIPE] = GENMASK(5, 1), + [ROUTE_DEF_HDR_TABLE] = BIT(6), + [ROUTE_DEF_HDR_OFST] = GENMASK(16, 7), + [ROUTE_FRAG_DEF_PIPE] = GENMASK(21, 17), + /* Bits 22-23 reserved */ + [ROUTE_DEF_RETAIN_HDR] = BIT(24), + /* Bits 25-31 reserved */ +}; + +IPA_REG_FIELDS(ROUTE, route, 0x00000048); + +static const u32 ipa_reg_shared_mem_size_fmask[] = { + [MEM_SIZE] = GENMASK(15, 0), + [MEM_BADDR] = GENMASK(31, 16), +}; + +IPA_REG_FIELDS(SHARED_MEM_SIZE, shared_mem_size, 0x00000054); + +static const u32 ipa_reg_qsb_max_writes_fmask[] = { + [GEN_QMB_0_MAX_WRITES] = GENMASK(3, 0), + [GEN_QMB_1_MAX_WRITES] = GENMASK(7, 4), + /* Bits 8-31 reserved */ +}; + +IPA_REG_FIELDS(QSB_MAX_WRITES, qsb_max_writes, 0x00000074); + +static const u32 ipa_reg_qsb_max_reads_fmask[] = { + [GEN_QMB_0_MAX_READS] = GENMASK(3, 0), + [GEN_QMB_1_MAX_READS] = GENMASK(7, 4), + /* Bits 8-15 reserved */ + [GEN_QMB_0_MAX_READS_BEATS] = GENMASK(23, 16), + [GEN_QMB_1_MAX_READS_BEATS] = GENMASK(31, 24), +}; + +IPA_REG_FIELDS(QSB_MAX_READS, qsb_max_reads, 0x00000078); + +static const u32 ipa_reg_filt_rout_hash_en_fmask[] = { + [IPV6_ROUTER_HASH] = BIT(0), + /* Bits 1-3 reserved */ + [IPV6_FILTER_HASH] = BIT(4), + /* Bits 5-7 reserved */ + [IPV4_ROUTER_HASH] = BIT(8), + /* Bits 9-11 reserved */ + [IPV4_FILTER_HASH] = BIT(12), + /* Bits 13-31 reserved */ +}; + +IPA_REG_FIELDS(FILT_ROUT_HASH_EN, filt_rout_hash_en, 0x0000148); + +static const u32 ipa_reg_filt_rout_hash_flush_fmask[] = { + [IPV6_ROUTER_HASH] = BIT(0), + /* Bits 1-3 reserved */ + [IPV6_FILTER_HASH] = BIT(4), + /* Bits 5-7 reserved */ + [IPV4_ROUTER_HASH] = BIT(8), + /* Bits 9-11 reserved */ + [IPV4_FILTER_HASH] = BIT(12), + /* Bits 13-31 reserved */ +}; + +IPA_REG_FIELDS(FILT_ROUT_HASH_FLUSH, filt_rout_hash_flush, 0x000014c); + +/* Valid bits defined by ipa->available */ +IPA_REG(STATE_AGGR_ACTIVE, state_aggr_active, 0x000000b4); + +static const u32 ipa_reg_local_pkt_proc_cntxt_fmask[] = { + [IPA_BASE_ADDR] = GENMASK(17, 0), + /* Bits 18-31 reserved */ +}; + +/* Offset must be a multiple of 8 */ +IPA_REG_FIELDS(LOCAL_PKT_PROC_CNTXT, local_pkt_proc_cntxt, 0x000001e8); + +/* Valid bits defined by ipa->available */ +IPA_REG(AGGR_FORCE_CLOSE, aggr_force_close, 0x000001ec); + +static const u32 ipa_reg_ipa_tx_cfg_fmask[] = { + /* Bits 0-1 reserved */ + [PREFETCH_ALMOST_EMPTY_SIZE_TX0] = GENMASK(5, 2), + [DMAW_SCND_OUTSD_PRED_THRESHOLD] = GENMASK(9, 6), + [DMAW_SCND_OUTSD_PRED_EN] = BIT(10), + [DMAW_MAX_BEATS_256_DIS] = BIT(11), + [PA_MASK_EN] = BIT(12), + [PREFETCH_ALMOST_EMPTY_SIZE_TX1] = GENMASK(16, 13), + [DUAL_TX_ENABLE] = BIT(17), + /* Bits 18-31 reserved */ +}; + +IPA_REG_FIELDS(IPA_TX_CFG, ipa_tx_cfg, 0x000001fc); + +static const u32 ipa_reg_flavor_0_fmask[] = { + [MAX_PIPES] = GENMASK(3, 0), + /* Bits 4-7 reserved */ + [MAX_CONS_PIPES] = GENMASK(12, 8), + /* Bits 13-15 reserved */ + [MAX_PROD_PIPES] = GENMASK(20, 16), + /* Bits 21-23 reserved */ + [PROD_LOWEST] = GENMASK(27, 24), + /* Bits 28-31 reserved */ +}; + +IPA_REG_FIELDS(FLAVOR_0, flavor_0, 0x00000210); + +static const u32 ipa_reg_idle_indication_cfg_fmask[] = { + [ENTER_IDLE_DEBOUNCE_THRESH] = GENMASK(15, 0), + [CONST_NON_IDLE_ENABLE] = BIT(16), + /* Bits 17-31 reserved */ +}; + +IPA_REG_FIELDS(IDLE_INDICATION_CFG, idle_indication_cfg, 0x00000240); + +static const u32 ipa_reg_qtime_timestamp_cfg_fmask[] = { + [DPL_TIMESTAMP_LSB] = GENMASK(4, 0), + /* Bits 5-6 reserved */ + [DPL_TIMESTAMP_SEL] = BIT(7), + [TAG_TIMESTAMP_LSB] = GENMASK(12, 8), + /* Bits 13-15 reserved */ + [NAT_TIMESTAMP_LSB] = GENMASK(20, 16), + /* Bits 21-31 reserved */ +}; + +IPA_REG_FIELDS(QTIME_TIMESTAMP_CFG, qtime_timestamp_cfg, 0x0000024c); + +static const u32 ipa_reg_timers_xo_clk_div_cfg_fmask[] = { + [DIV_VALUE] = GENMASK(8, 0), + /* Bits 9-30 reserved */ + [DIV_ENABLE] = BIT(31), +}; + +IPA_REG_FIELDS(TIMERS_XO_CLK_DIV_CFG, timers_xo_clk_div_cfg, 0x00000250); + +static const u32 ipa_reg_timers_pulse_gran_cfg_fmask[] = { + [PULSE_GRAN_0] = GENMASK(2, 0), + [PULSE_GRAN_1] = GENMASK(5, 3), + [PULSE_GRAN_2] = GENMASK(8, 6), +}; + +IPA_REG_FIELDS(TIMERS_PULSE_GRAN_CFG, timers_pulse_gran_cfg, 0x00000254); + +static const u32 ipa_reg_src_rsrc_grp_01_rsrc_type_fmask[] = { + [X_MIN_LIM] = GENMASK(5, 0), + /* Bits 6-7 reserved */ + [X_MAX_LIM] = GENMASK(13, 8), + /* Bits 14-15 reserved */ + [Y_MIN_LIM] = GENMASK(21, 16), + /* Bits 22-23 reserved */ + [Y_MAX_LIM] = GENMASK(29, 24), + /* Bits 30-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(SRC_RSRC_GRP_01_RSRC_TYPE, src_rsrc_grp_01_rsrc_type, + 0x00000400, 0x0020); + +static const u32 ipa_reg_src_rsrc_grp_23_rsrc_type_fmask[] = { + [X_MIN_LIM] = GENMASK(5, 0), + /* Bits 6-7 reserved */ + [X_MAX_LIM] = GENMASK(13, 8), + /* Bits 14-15 reserved */ + [Y_MIN_LIM] = GENMASK(21, 16), + /* Bits 22-23 reserved */ + [Y_MAX_LIM] = GENMASK(29, 24), + /* Bits 30-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(SRC_RSRC_GRP_23_RSRC_TYPE, src_rsrc_grp_23_rsrc_type, + 0x00000404, 0x0020); + +static const u32 ipa_reg_src_rsrc_grp_45_rsrc_type_fmask[] = { + [X_MIN_LIM] = GENMASK(5, 0), + /* Bits 6-7 reserved */ + [X_MAX_LIM] = GENMASK(13, 8), + /* Bits 14-15 reserved */ + [Y_MIN_LIM] = GENMASK(21, 16), + /* Bits 22-23 reserved */ + [Y_MAX_LIM] = GENMASK(29, 24), + /* Bits 30-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(SRC_RSRC_GRP_45_RSRC_TYPE, src_rsrc_grp_45_rsrc_type, + 0x00000408, 0x0020); + +static const u32 ipa_reg_dst_rsrc_grp_01_rsrc_type_fmask[] = { + [X_MIN_LIM] = GENMASK(5, 0), + /* Bits 6-7 reserved */ + [X_MAX_LIM] = GENMASK(13, 8), + /* Bits 14-15 reserved */ + [Y_MIN_LIM] = GENMASK(21, 16), + /* Bits 22-23 reserved */ + [Y_MAX_LIM] = GENMASK(29, 24), + /* Bits 30-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(DST_RSRC_GRP_01_RSRC_TYPE, dst_rsrc_grp_01_rsrc_type, + 0x00000500, 0x0020); + +static const u32 ipa_reg_dst_rsrc_grp_23_rsrc_type_fmask[] = { + [X_MIN_LIM] = GENMASK(5, 0), + /* Bits 6-7 reserved */ + [X_MAX_LIM] = GENMASK(13, 8), + /* Bits 14-15 reserved */ + [Y_MIN_LIM] = GENMASK(21, 16), + /* Bits 22-23 reserved */ + [Y_MAX_LIM] = GENMASK(29, 24), + /* Bits 30-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(DST_RSRC_GRP_23_RSRC_TYPE, dst_rsrc_grp_23_rsrc_type, + 0x00000504, 0x0020); + +static const u32 ipa_reg_dst_rsrc_grp_45_rsrc_type_fmask[] = { + [X_MIN_LIM] = GENMASK(5, 0), + /* Bits 6-7 reserved */ + [X_MAX_LIM] = GENMASK(13, 8), + /* Bits 14-15 reserved */ + [Y_MIN_LIM] = GENMASK(21, 16), + /* Bits 22-23 reserved */ + [Y_MAX_LIM] = GENMASK(29, 24), + /* Bits 30-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(DST_RSRC_GRP_45_RSRC_TYPE, dst_rsrc_grp_45_rsrc_type, + 0x00000508, 0x0020); + +static const u32 ipa_reg_endp_init_cfg_fmask[] = { + [FRAG_OFFLOAD_EN] = BIT(0), + [CS_OFFLOAD_EN] = GENMASK(2, 1), + [CS_METADATA_HDR_OFFSET] = GENMASK(6, 3), + /* Bit 7 reserved */ + [CS_GEN_QMB_MASTER_SEL] = BIT(8), + /* Bits 9-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(ENDP_INIT_CFG, endp_init_cfg, 0x00000808, 0x0070); + +static const u32 ipa_reg_endp_init_nat_fmask[] = { + [NAT_EN] = GENMASK(1, 0), + /* Bits 2-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(ENDP_INIT_NAT, endp_init_nat, 0x0000080c, 0x0070); + +static const u32 ipa_reg_endp_init_hdr_fmask[] = { + [HDR_LEN] = GENMASK(5, 0), + [HDR_OFST_METADATA_VALID] = BIT(6), + [HDR_OFST_METADATA] = GENMASK(12, 7), + [HDR_ADDITIONAL_CONST_LEN] = GENMASK(18, 13), + [HDR_OFST_PKT_SIZE_VALID] = BIT(19), + [HDR_OFST_PKT_SIZE] = GENMASK(25, 20), + [HDR_A5_MUX] = BIT(26), + [HDR_LEN_INC_DEAGG_HDR] = BIT(27), + [HDR_LEN_MSB] = GENMASK(29, 28), + [HDR_OFST_METADATA_MSB] = GENMASK(31, 30), +}; + +IPA_REG_STRIDE_FIELDS(ENDP_INIT_HDR, endp_init_hdr, 0x00000810, 0x0070); + +static const u32 ipa_reg_endp_init_hdr_ext_fmask[] = { + [HDR_ENDIANNESS] = BIT(0), + [HDR_TOTAL_LEN_OR_PAD_VALID] = BIT(1), + [HDR_TOTAL_LEN_OR_PAD] = BIT(2), + [HDR_PAYLOAD_LEN_INC_PADDING] = BIT(3), + [HDR_TOTAL_LEN_OR_PAD_OFFSET] = GENMASK(9, 4), + [HDR_PAD_TO_ALIGNMENT] = GENMASK(13, 10), + /* Bits 14-15 reserved */ + [HDR_TOTAL_LEN_OR_PAD_OFFSET_MSB] = GENMASK(17, 16), + [HDR_OFST_PKT_SIZE_MSB] = GENMASK(19, 18), + [HDR_ADDITIONAL_CONST_LEN_MSB] = GENMASK(21, 20), + /* Bits 22-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(ENDP_INIT_HDR_EXT, endp_init_hdr_ext, 0x00000814, 0x0070); + +IPA_REG_STRIDE(ENDP_INIT_HDR_METADATA_MASK, endp_init_hdr_metadata_mask, + 0x00000818, 0x0070); + +static const u32 ipa_reg_endp_init_mode_fmask[] = { + [ENDP_MODE] = GENMASK(2, 0), + [DCPH_ENABLE] = BIT(3), + [DEST_PIPE_INDEX] = GENMASK(8, 4), + /* Bits 9-11 reserved */ + [BYTE_THRESHOLD] = GENMASK(27, 12), + [PIPE_REPLICATION_EN] = BIT(28), + [PAD_EN] = BIT(29), + /* Bits 30-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(ENDP_INIT_MODE, endp_init_mode, 0x00000820, 0x0070); + +static const u32 ipa_reg_endp_init_aggr_fmask[] = { + [AGGR_EN] = GENMASK(1, 0), + [AGGR_TYPE] = GENMASK(4, 2), + [BYTE_LIMIT] = GENMASK(10, 5), + /* Bit 11 reserved */ + [TIME_LIMIT] = GENMASK(16, 12), + [PKT_LIMIT] = GENMASK(22, 17), + [SW_EOF_ACTIVE] = BIT(23), + [FORCE_CLOSE] = BIT(24), + /* Bit 25 reserved */ + [HARD_BYTE_LIMIT_EN] = BIT(26), + [AGGR_GRAN_SEL] = BIT(27), + /* Bits 28-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(ENDP_INIT_AGGR, endp_init_aggr, 0x00000824, 0x0070); + +static const u32 ipa_reg_endp_init_hol_block_en_fmask[] = { + [HOL_BLOCK_EN] = BIT(0), + /* Bits 1-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(ENDP_INIT_HOL_BLOCK_EN, endp_init_hol_block_en, + 0x0000082c, 0x0070); + +static const u32 ipa_reg_endp_init_hol_block_timer_fmask[] = { + [TIMER_LIMIT] = GENMASK(4, 0), + /* Bits 5-7 reserved */ + [TIMER_GRAN_SEL] = BIT(8), + /* Bits 9-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(ENDP_INIT_HOL_BLOCK_TIMER, endp_init_hol_block_timer, + 0x00000830, 0x0070); + +static const u32 ipa_reg_endp_init_deaggr_fmask[] = { + [DEAGGR_HDR_LEN] = GENMASK(5, 0), + [SYSPIPE_ERR_DETECTION] = BIT(6), + [PACKET_OFFSET_VALID] = BIT(7), + [PACKET_OFFSET_LOCATION] = GENMASK(13, 8), + [IGNORE_MIN_PKT_ERR] = BIT(14), + /* Bit 15 reserved */ + [MAX_PACKET_LEN] = GENMASK(31, 16), +}; + +IPA_REG_STRIDE_FIELDS(ENDP_INIT_DEAGGR, endp_init_deaggr, 0x00000834, 0x0070); + +static const u32 ipa_reg_endp_init_rsrc_grp_fmask[] = { + [ENDP_RSRC_GRP] = GENMASK(2, 0), + /* Bits 3-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(ENDP_INIT_RSRC_GRP, endp_init_rsrc_grp, + 0x00000838, 0x0070); + +static const u32 ipa_reg_endp_init_seq_fmask[] = { + [SEQ_TYPE] = GENMASK(7, 0), + /* Bits 8-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(ENDP_INIT_SEQ, endp_init_seq, 0x0000083c, 0x0070); + +static const u32 ipa_reg_endp_status_fmask[] = { + [STATUS_EN] = BIT(0), + [STATUS_ENDP] = GENMASK(5, 1), + /* Bits 6-8 reserved */ + [STATUS_PKT_SUPPRESS] = BIT(9), + /* Bits 10-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(ENDP_STATUS, endp_status, 0x00000840, 0x0070); + +static const u32 ipa_reg_endp_filter_router_hsh_cfg_fmask[] = { + [FILTER_HASH_MSK_SRC_ID] = BIT(0), + [FILTER_HASH_MSK_SRC_IP] = BIT(1), + [FILTER_HASH_MSK_DST_IP] = BIT(2), + [FILTER_HASH_MSK_SRC_PORT] = BIT(3), + [FILTER_HASH_MSK_DST_PORT] = BIT(4), + [FILTER_HASH_MSK_PROTOCOL] = BIT(5), + [FILTER_HASH_MSK_METADATA] = BIT(6), + [FILTER_HASH_MSK_ALL] = GENMASK(6, 0), + /* Bits 7-15 reserved */ + [ROUTER_HASH_MSK_SRC_ID] = BIT(16), + [ROUTER_HASH_MSK_SRC_IP] = BIT(17), + [ROUTER_HASH_MSK_DST_IP] = BIT(18), + [ROUTER_HASH_MSK_SRC_PORT] = BIT(19), + [ROUTER_HASH_MSK_DST_PORT] = BIT(20), + [ROUTER_HASH_MSK_PROTOCOL] = BIT(21), + [ROUTER_HASH_MSK_METADATA] = BIT(22), + [ROUTER_HASH_MSK_ALL] = GENMASK(22, 16), + /* Bits 23-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(ENDP_FILTER_ROUTER_HSH_CFG, endp_filter_router_hsh_cfg, + 0x0000085c, 0x0070); + +/* Valid bits defined by enum ipa_irq_id; only used for GSI_EE_AP */ +IPA_REG(IPA_IRQ_STTS, ipa_irq_stts, 0x00003008 + 0x1000 * GSI_EE_AP); + +/* Valid bits defined by enum ipa_irq_id; only used for GSI_EE_AP */ +IPA_REG(IPA_IRQ_EN, ipa_irq_en, 0x0000300c + 0x1000 * GSI_EE_AP); + +/* Valid bits defined by enum ipa_irq_id; only used for GSI_EE_AP */ +IPA_REG(IPA_IRQ_CLR, ipa_irq_clr, 0x00003010 + 0x1000 * GSI_EE_AP); + +static const u32 ipa_reg_ipa_irq_uc_fmask[] = { + [UC_INTR] = BIT(0), + /* Bits 1-31 reserved */ +}; + +IPA_REG_FIELDS(IPA_IRQ_UC, ipa_irq_uc, 0x0000301c + 0x1000 * GSI_EE_AP); + +/* Valid bits defined by ipa->available */ +IPA_REG(IRQ_SUSPEND_INFO, irq_suspend_info, 0x00003030 + 0x1000 * GSI_EE_AP); + +/* Valid bits defined by ipa->available */ +IPA_REG(IRQ_SUSPEND_EN, irq_suspend_en, 0x00003034 + 0x1000 * GSI_EE_AP); + +/* Valid bits defined by ipa->available */ +IPA_REG(IRQ_SUSPEND_CLR, irq_suspend_clr, 0x00003038 + 0x1000 * GSI_EE_AP); + +static const struct ipa_reg *ipa_reg_array[] = { + [COMP_CFG] = &ipa_reg_comp_cfg, + [CLKON_CFG] = &ipa_reg_clkon_cfg, + [ROUTE] = &ipa_reg_route, + [SHARED_MEM_SIZE] = &ipa_reg_shared_mem_size, + [QSB_MAX_WRITES] = &ipa_reg_qsb_max_writes, + [QSB_MAX_READS] = &ipa_reg_qsb_max_reads, + [FILT_ROUT_HASH_EN] = &ipa_reg_filt_rout_hash_en, + [FILT_ROUT_HASH_FLUSH] = &ipa_reg_filt_rout_hash_flush, + [STATE_AGGR_ACTIVE] = &ipa_reg_state_aggr_active, + [LOCAL_PKT_PROC_CNTXT] = &ipa_reg_local_pkt_proc_cntxt, + [AGGR_FORCE_CLOSE] = &ipa_reg_aggr_force_close, + [IPA_TX_CFG] = &ipa_reg_ipa_tx_cfg, + [FLAVOR_0] = &ipa_reg_flavor_0, + [IDLE_INDICATION_CFG] = &ipa_reg_idle_indication_cfg, + [QTIME_TIMESTAMP_CFG] = &ipa_reg_qtime_timestamp_cfg, + [TIMERS_XO_CLK_DIV_CFG] = &ipa_reg_timers_xo_clk_div_cfg, + [TIMERS_PULSE_GRAN_CFG] = &ipa_reg_timers_pulse_gran_cfg, + [SRC_RSRC_GRP_01_RSRC_TYPE] = &ipa_reg_src_rsrc_grp_01_rsrc_type, + [SRC_RSRC_GRP_23_RSRC_TYPE] = &ipa_reg_src_rsrc_grp_23_rsrc_type, + [SRC_RSRC_GRP_45_RSRC_TYPE] = &ipa_reg_src_rsrc_grp_45_rsrc_type, + [DST_RSRC_GRP_01_RSRC_TYPE] = &ipa_reg_dst_rsrc_grp_01_rsrc_type, + [DST_RSRC_GRP_23_RSRC_TYPE] = &ipa_reg_dst_rsrc_grp_23_rsrc_type, + [DST_RSRC_GRP_45_RSRC_TYPE] = &ipa_reg_dst_rsrc_grp_45_rsrc_type, + [ENDP_INIT_CFG] = &ipa_reg_endp_init_cfg, + [ENDP_INIT_NAT] = &ipa_reg_endp_init_nat, + [ENDP_INIT_HDR] = &ipa_reg_endp_init_hdr, + [ENDP_INIT_HDR_EXT] = &ipa_reg_endp_init_hdr_ext, + [ENDP_INIT_HDR_METADATA_MASK] = &ipa_reg_endp_init_hdr_metadata_mask, + [ENDP_INIT_MODE] = &ipa_reg_endp_init_mode, + [ENDP_INIT_AGGR] = &ipa_reg_endp_init_aggr, + [ENDP_INIT_HOL_BLOCK_EN] = &ipa_reg_endp_init_hol_block_en, + [ENDP_INIT_HOL_BLOCK_TIMER] = &ipa_reg_endp_init_hol_block_timer, + [ENDP_INIT_DEAGGR] = &ipa_reg_endp_init_deaggr, + [ENDP_INIT_RSRC_GRP] = &ipa_reg_endp_init_rsrc_grp, + [ENDP_INIT_SEQ] = &ipa_reg_endp_init_seq, + [ENDP_STATUS] = &ipa_reg_endp_status, + [ENDP_FILTER_ROUTER_HSH_CFG] = &ipa_reg_endp_filter_router_hsh_cfg, + [IPA_IRQ_STTS] = &ipa_reg_ipa_irq_stts, + [IPA_IRQ_EN] = &ipa_reg_ipa_irq_en, + [IPA_IRQ_CLR] = &ipa_reg_ipa_irq_clr, + [IPA_IRQ_UC] = &ipa_reg_ipa_irq_uc, + [IRQ_SUSPEND_INFO] = &ipa_reg_irq_suspend_info, + [IRQ_SUSPEND_EN] = &ipa_reg_irq_suspend_en, + [IRQ_SUSPEND_CLR] = &ipa_reg_irq_suspend_clr, +}; + +const struct ipa_regs ipa_regs_v4_5 = { + .reg_count = ARRAY_SIZE(ipa_reg_array), + .reg = ipa_reg_array, +}; diff --git a/drivers/net/ipa/reg/ipa_reg-v4.9.c b/drivers/net/ipa/reg/ipa_reg-v4.9.c new file mode 100644 index 000000000000..eabbc5451937 --- /dev/null +++ b/drivers/net/ipa/reg/ipa_reg-v4.9.c @@ -0,0 +1,509 @@ +// SPDX-License-Identifier: GPL-2.0 + +/* Copyright (C) 2022 Linaro Ltd. */ + +#include <linux/types.h> + +#include "../ipa.h" +#include "../ipa_reg.h" + +static const u32 ipa_reg_comp_cfg_fmask[] = { + [RAM_ARB_PRI_CLIENT_SAMP_FIX_DIS] = BIT(0), + [GSI_SNOC_BYPASS_DIS] = BIT(1), + [GEN_QMB_0_SNOC_BYPASS_DIS] = BIT(2), + [GEN_QMB_1_SNOC_BYPASS_DIS] = BIT(3), + /* Bit 4 reserved */ + [IPA_QMB_SELECT_CONS_EN] = BIT(5), + [IPA_QMB_SELECT_PROD_EN] = BIT(6), + [GSI_MULTI_INORDER_RD_DIS] = BIT(7), + [GSI_MULTI_INORDER_WR_DIS] = BIT(8), + [GEN_QMB_0_MULTI_INORDER_RD_DIS] = BIT(9), + [GEN_QMB_1_MULTI_INORDER_RD_DIS] = BIT(10), + [GEN_QMB_0_MULTI_INORDER_WR_DIS] = BIT(11), + [GEN_QMB_1_MULTI_INORDER_WR_DIS] = BIT(12), + [GEN_QMB_0_SNOC_CNOC_LOOP_PROT_DIS] = BIT(13), + [GSI_SNOC_CNOC_LOOP_PROT_DISABLE] = BIT(14), + [GSI_MULTI_AXI_MASTERS_DIS] = BIT(15), + [IPA_QMB_SELECT_GLOBAL_EN] = BIT(16), + [FULL_FLUSH_WAIT_RS_CLOSURE_EN] = BIT(17), + [QMB_RAM_RD_CACHE_DISABLE] = BIT(19), + [GENQMB_AOOOWR] = BIT(20), + [IF_OUT_OF_BUF_STOP_RESET_MASK_EN] = BIT(21), + [ATOMIC_FETCHER_ARB_LOCK_DIS] = GENMASK(24, 22), + /* Bits 25-29 reserved */ + [GEN_QMB_1_DYNAMIC_ASIZE] = BIT(30), + [GEN_QMB_0_DYNAMIC_ASIZE] = BIT(31), +}; + +IPA_REG_FIELDS(COMP_CFG, comp_cfg, 0x0000003c); + +static const u32 ipa_reg_clkon_cfg_fmask[] = { + [CLKON_RX] = BIT(0), + [CLKON_PROC] = BIT(1), + [TX_WRAPPER] = BIT(2), + [CLKON_MISC] = BIT(3), + [RAM_ARB] = BIT(4), + [FTCH_HPS] = BIT(5), + [FTCH_DPS] = BIT(6), + [CLKON_HPS] = BIT(7), + [CLKON_DPS] = BIT(8), + [RX_HPS_CMDQS] = BIT(9), + [HPS_DPS_CMDQS] = BIT(10), + [DPS_TX_CMDQS] = BIT(11), + [RSRC_MNGR] = BIT(12), + [CTX_HANDLER] = BIT(13), + [ACK_MNGR] = BIT(14), + [D_DCPH] = BIT(15), + [H_DCPH] = BIT(16), + [CLKON_DCMP] = BIT(17), + [NTF_TX_CMDQS] = BIT(18), + [CLKON_TX_0] = BIT(19), + [CLKON_TX_1] = BIT(20), + [CLKON_FNR] = BIT(21), + [QSB2AXI_CMDQ_L] = BIT(22), + [AGGR_WRAPPER] = BIT(23), + [RAM_SLAVEWAY] = BIT(24), + [CLKON_QMB] = BIT(25), + [WEIGHT_ARB] = BIT(26), + [GSI_IF] = BIT(27), + [CLKON_GLOBAL] = BIT(28), + [GLOBAL_2X_CLK] = BIT(29), + [DPL_FIFO] = BIT(30), + [DRBIP] = BIT(31), +}; + +IPA_REG_FIELDS(CLKON_CFG, clkon_cfg, 0x00000044); + +static const u32 ipa_reg_route_fmask[] = { + [ROUTE_DIS] = BIT(0), + [ROUTE_DEF_PIPE] = GENMASK(5, 1), + [ROUTE_DEF_HDR_TABLE] = BIT(6), + [ROUTE_DEF_HDR_OFST] = GENMASK(16, 7), + [ROUTE_FRAG_DEF_PIPE] = GENMASK(21, 17), + /* Bits 22-23 reserved */ + [ROUTE_DEF_RETAIN_HDR] = BIT(24), + /* Bits 25-31 reserved */ +}; + +IPA_REG_FIELDS(ROUTE, route, 0x00000048); + +static const u32 ipa_reg_shared_mem_size_fmask[] = { + [MEM_SIZE] = GENMASK(15, 0), + [MEM_BADDR] = GENMASK(31, 16), +}; + +IPA_REG_FIELDS(SHARED_MEM_SIZE, shared_mem_size, 0x00000054); + +static const u32 ipa_reg_qsb_max_writes_fmask[] = { + [GEN_QMB_0_MAX_WRITES] = GENMASK(3, 0), + [GEN_QMB_1_MAX_WRITES] = GENMASK(7, 4), + /* Bits 8-31 reserved */ +}; + +IPA_REG_FIELDS(QSB_MAX_WRITES, qsb_max_writes, 0x00000074); + +static const u32 ipa_reg_qsb_max_reads_fmask[] = { + [GEN_QMB_0_MAX_READS] = GENMASK(3, 0), + [GEN_QMB_1_MAX_READS] = GENMASK(7, 4), + /* Bits 8-15 reserved */ + [GEN_QMB_0_MAX_READS_BEATS] = GENMASK(23, 16), + [GEN_QMB_1_MAX_READS_BEATS] = GENMASK(31, 24), +}; + +IPA_REG_FIELDS(QSB_MAX_READS, qsb_max_reads, 0x00000078); + +static const u32 ipa_reg_filt_rout_hash_en_fmask[] = { + [IPV6_ROUTER_HASH] = BIT(0), + /* Bits 1-3 reserved */ + [IPV6_FILTER_HASH] = BIT(4), + /* Bits 5-7 reserved */ + [IPV4_ROUTER_HASH] = BIT(8), + /* Bits 9-11 reserved */ + [IPV4_FILTER_HASH] = BIT(12), + /* Bits 13-31 reserved */ +}; + +IPA_REG_FIELDS(FILT_ROUT_HASH_EN, filt_rout_hash_en, 0x0000148); + +static const u32 ipa_reg_filt_rout_hash_flush_fmask[] = { + [IPV6_ROUTER_HASH] = BIT(0), + /* Bits 1-3 reserved */ + [IPV6_FILTER_HASH] = BIT(4), + /* Bits 5-7 reserved */ + [IPV4_ROUTER_HASH] = BIT(8), + /* Bits 9-11 reserved */ + [IPV4_FILTER_HASH] = BIT(12), + /* Bits 13-31 reserved */ +}; + +IPA_REG_FIELDS(FILT_ROUT_HASH_FLUSH, filt_rout_hash_flush, 0x000014c); + +/* Valid bits defined by ipa->available */ +IPA_REG(STATE_AGGR_ACTIVE, state_aggr_active, 0x000000b4); + +static const u32 ipa_reg_local_pkt_proc_cntxt_fmask[] = { + [IPA_BASE_ADDR] = GENMASK(17, 0), + /* Bits 18-31 reserved */ +}; + +/* Offset must be a multiple of 8 */ +IPA_REG_FIELDS(LOCAL_PKT_PROC_CNTXT, local_pkt_proc_cntxt, 0x000001e8); + +/* Valid bits defined by ipa->available */ +IPA_REG(AGGR_FORCE_CLOSE, aggr_force_close, 0x000001ec); + +static const u32 ipa_reg_ipa_tx_cfg_fmask[] = { + /* Bits 0-1 reserved */ + [PREFETCH_ALMOST_EMPTY_SIZE_TX0] = GENMASK(5, 2), + [DMAW_SCND_OUTSD_PRED_THRESHOLD] = GENMASK(9, 6), + [DMAW_SCND_OUTSD_PRED_EN] = BIT(10), + [DMAW_MAX_BEATS_256_DIS] = BIT(11), + [PA_MASK_EN] = BIT(12), + [PREFETCH_ALMOST_EMPTY_SIZE_TX1] = GENMASK(16, 13), + [DUAL_TX_ENABLE] = BIT(17), + [SSPND_PA_NO_START_STATE] = BIT(18), + /* Bits 19-31 reserved */ +}; + +IPA_REG_FIELDS(IPA_TX_CFG, ipa_tx_cfg, 0x000001fc); + +static const u32 ipa_reg_flavor_0_fmask[] = { + [MAX_PIPES] = GENMASK(3, 0), + /* Bits 4-7 reserved */ + [MAX_CONS_PIPES] = GENMASK(12, 8), + /* Bits 13-15 reserved */ + [MAX_PROD_PIPES] = GENMASK(20, 16), + /* Bits 21-23 reserved */ + [PROD_LOWEST] = GENMASK(27, 24), + /* Bits 28-31 reserved */ +}; + +IPA_REG_FIELDS(FLAVOR_0, flavor_0, 0x00000210); + +static const u32 ipa_reg_idle_indication_cfg_fmask[] = { + [ENTER_IDLE_DEBOUNCE_THRESH] = GENMASK(15, 0), + [CONST_NON_IDLE_ENABLE] = BIT(16), + /* Bits 17-31 reserved */ +}; + +IPA_REG_FIELDS(IDLE_INDICATION_CFG, idle_indication_cfg, 0x00000240); + +static const u32 ipa_reg_qtime_timestamp_cfg_fmask[] = { + [DPL_TIMESTAMP_LSB] = GENMASK(4, 0), + /* Bits 5-6 reserved */ + [DPL_TIMESTAMP_SEL] = BIT(7), + [TAG_TIMESTAMP_LSB] = GENMASK(12, 8), + /* Bits 13-15 reserved */ + [NAT_TIMESTAMP_LSB] = GENMASK(20, 16), + /* Bits 21-31 reserved */ +}; + +IPA_REG_FIELDS(QTIME_TIMESTAMP_CFG, qtime_timestamp_cfg, 0x0000024c); + +static const u32 ipa_reg_timers_xo_clk_div_cfg_fmask[] = { + [DIV_VALUE] = GENMASK(8, 0), + /* Bits 9-30 reserved */ + [DIV_ENABLE] = BIT(31), +}; + +IPA_REG_FIELDS(TIMERS_XO_CLK_DIV_CFG, timers_xo_clk_div_cfg, 0x00000250); + +static const u32 ipa_reg_timers_pulse_gran_cfg_fmask[] = { + [PULSE_GRAN_0] = GENMASK(2, 0), + [PULSE_GRAN_1] = GENMASK(5, 3), + [PULSE_GRAN_2] = GENMASK(8, 6), +}; + +IPA_REG_FIELDS(TIMERS_PULSE_GRAN_CFG, timers_pulse_gran_cfg, 0x00000254); + +static const u32 ipa_reg_src_rsrc_grp_01_rsrc_type_fmask[] = { + [X_MIN_LIM] = GENMASK(5, 0), + /* Bits 6-7 reserved */ + [X_MAX_LIM] = GENMASK(13, 8), + /* Bits 14-15 reserved */ + [Y_MIN_LIM] = GENMASK(21, 16), + /* Bits 22-23 reserved */ + [Y_MAX_LIM] = GENMASK(29, 24), + /* Bits 30-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(SRC_RSRC_GRP_01_RSRC_TYPE, src_rsrc_grp_01_rsrc_type, + 0x00000400, 0x0020); + +static const u32 ipa_reg_src_rsrc_grp_23_rsrc_type_fmask[] = { + [X_MIN_LIM] = GENMASK(5, 0), + /* Bits 6-7 reserved */ + [X_MAX_LIM] = GENMASK(13, 8), + /* Bits 14-15 reserved */ + [Y_MIN_LIM] = GENMASK(21, 16), + /* Bits 22-23 reserved */ + [Y_MAX_LIM] = GENMASK(29, 24), + /* Bits 30-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(SRC_RSRC_GRP_23_RSRC_TYPE, src_rsrc_grp_23_rsrc_type, + 0x00000404, 0x0020); + +static const u32 ipa_reg_dst_rsrc_grp_01_rsrc_type_fmask[] = { + [X_MIN_LIM] = GENMASK(5, 0), + /* Bits 6-7 reserved */ + [X_MAX_LIM] = GENMASK(13, 8), + /* Bits 14-15 reserved */ + [Y_MIN_LIM] = GENMASK(21, 16), + /* Bits 22-23 reserved */ + [Y_MAX_LIM] = GENMASK(29, 24), + /* Bits 30-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(DST_RSRC_GRP_01_RSRC_TYPE, dst_rsrc_grp_01_rsrc_type, + 0x00000500, 0x0020); + +static const u32 ipa_reg_dst_rsrc_grp_23_rsrc_type_fmask[] = { + [X_MIN_LIM] = GENMASK(5, 0), + /* Bits 6-7 reserved */ + [X_MAX_LIM] = GENMASK(13, 8), + /* Bits 14-15 reserved */ + [Y_MIN_LIM] = GENMASK(21, 16), + /* Bits 22-23 reserved */ + [Y_MAX_LIM] = GENMASK(29, 24), + /* Bits 30-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(DST_RSRC_GRP_23_RSRC_TYPE, dst_rsrc_grp_23_rsrc_type, + 0x00000504, 0x0020); + +static const u32 ipa_reg_endp_init_cfg_fmask[] = { + [FRAG_OFFLOAD_EN] = BIT(0), + [CS_OFFLOAD_EN] = GENMASK(2, 1), + [CS_METADATA_HDR_OFFSET] = GENMASK(6, 3), + /* Bit 7 reserved */ + [CS_GEN_QMB_MASTER_SEL] = BIT(8), + /* Bits 9-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(ENDP_INIT_CFG, endp_init_cfg, 0x00000808, 0x0070); + +static const u32 ipa_reg_endp_init_nat_fmask[] = { + [NAT_EN] = GENMASK(1, 0), + /* Bits 2-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(ENDP_INIT_NAT, endp_init_nat, 0x0000080c, 0x0070); + +static const u32 ipa_reg_endp_init_hdr_fmask[] = { + [HDR_LEN] = GENMASK(5, 0), + [HDR_OFST_METADATA_VALID] = BIT(6), + [HDR_OFST_METADATA] = GENMASK(12, 7), + [HDR_ADDITIONAL_CONST_LEN] = GENMASK(18, 13), + [HDR_OFST_PKT_SIZE_VALID] = BIT(19), + [HDR_OFST_PKT_SIZE] = GENMASK(25, 20), + [HDR_LEN_INC_DEAGG_HDR] = BIT(27), + [HDR_LEN_MSB] = GENMASK(29, 28), + [HDR_OFST_METADATA_MSB] = GENMASK(31, 30), +}; + +IPA_REG_STRIDE_FIELDS(ENDP_INIT_HDR, endp_init_hdr, 0x00000810, 0x0070); + +static const u32 ipa_reg_endp_init_hdr_ext_fmask[] = { + [HDR_ENDIANNESS] = BIT(0), + [HDR_TOTAL_LEN_OR_PAD_VALID] = BIT(1), + [HDR_TOTAL_LEN_OR_PAD] = BIT(2), + [HDR_PAYLOAD_LEN_INC_PADDING] = BIT(3), + [HDR_TOTAL_LEN_OR_PAD_OFFSET] = GENMASK(9, 4), + [HDR_PAD_TO_ALIGNMENT] = GENMASK(13, 10), + /* Bits 14-15 reserved */ + [HDR_TOTAL_LEN_OR_PAD_OFFSET_MSB] = GENMASK(17, 16), + [HDR_OFST_PKT_SIZE_MSB] = GENMASK(19, 18), + [HDR_ADDITIONAL_CONST_LEN_MSB] = GENMASK(21, 20), + /* Bits 22-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(ENDP_INIT_HDR_EXT, endp_init_hdr_ext, 0x00000814, 0x0070); + +IPA_REG_STRIDE(ENDP_INIT_HDR_METADATA_MASK, endp_init_hdr_metadata_mask, + 0x00000818, 0x0070); + +static const u32 ipa_reg_endp_init_mode_fmask[] = { + [ENDP_MODE] = GENMASK(2, 0), + [DCPH_ENABLE] = BIT(3), + [DEST_PIPE_INDEX] = GENMASK(8, 4), + /* Bits 9-11 reserved */ + [BYTE_THRESHOLD] = GENMASK(27, 12), + [PIPE_REPLICATION_EN] = BIT(28), + [PAD_EN] = BIT(29), + [DRBIP_ACL_ENABLE] = BIT(30), + /* Bit 31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(ENDP_INIT_MODE, endp_init_mode, 0x00000820, 0x0070); + +static const u32 ipa_reg_endp_init_aggr_fmask[] = { + [AGGR_EN] = GENMASK(1, 0), + [AGGR_TYPE] = GENMASK(4, 2), + [BYTE_LIMIT] = GENMASK(10, 5), + /* Bit 11 reserved */ + [TIME_LIMIT] = GENMASK(16, 12), + [PKT_LIMIT] = GENMASK(22, 17), + [SW_EOF_ACTIVE] = BIT(23), + [FORCE_CLOSE] = BIT(24), + /* Bit 25 reserved */ + [HARD_BYTE_LIMIT_EN] = BIT(26), + [AGGR_GRAN_SEL] = BIT(27), + /* Bits 28-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(ENDP_INIT_AGGR, endp_init_aggr, 0x00000824, 0x0070); + +static const u32 ipa_reg_endp_init_hol_block_en_fmask[] = { + [HOL_BLOCK_EN] = BIT(0), + /* Bits 1-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(ENDP_INIT_HOL_BLOCK_EN, endp_init_hol_block_en, + 0x0000082c, 0x0070); + +static const u32 ipa_reg_endp_init_hol_block_timer_fmask[] = { + [TIMER_LIMIT] = GENMASK(4, 0), + /* Bits 5-7 reserved */ + [TIMER_GRAN_SEL] = BIT(8), + /* Bits 9-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(ENDP_INIT_HOL_BLOCK_TIMER, endp_init_hol_block_timer, + 0x00000830, 0x0070); + +static const u32 ipa_reg_endp_init_deaggr_fmask[] = { + [DEAGGR_HDR_LEN] = GENMASK(5, 0), + [SYSPIPE_ERR_DETECTION] = BIT(6), + [PACKET_OFFSET_VALID] = BIT(7), + [PACKET_OFFSET_LOCATION] = GENMASK(13, 8), + [IGNORE_MIN_PKT_ERR] = BIT(14), + /* Bit 15 reserved */ + [MAX_PACKET_LEN] = GENMASK(31, 16), +}; + +IPA_REG_STRIDE_FIELDS(ENDP_INIT_DEAGGR, endp_init_deaggr, 0x00000834, 0x0070); + +static const u32 ipa_reg_endp_init_rsrc_grp_fmask[] = { + [ENDP_RSRC_GRP] = GENMASK(1, 0), + /* Bits 2-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(ENDP_INIT_RSRC_GRP, endp_init_rsrc_grp, + 0x00000838, 0x0070); + +static const u32 ipa_reg_endp_init_seq_fmask[] = { + [SEQ_TYPE] = GENMASK(7, 0), + /* Bits 8-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(ENDP_INIT_SEQ, endp_init_seq, 0x0000083c, 0x0070); + +static const u32 ipa_reg_endp_status_fmask[] = { + [STATUS_EN] = BIT(0), + [STATUS_ENDP] = GENMASK(5, 1), + /* Bits 6-8 reserved */ + [STATUS_PKT_SUPPRESS] = BIT(9), + /* Bits 10-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(ENDP_STATUS, endp_status, 0x00000840, 0x0070); + +static const u32 ipa_reg_endp_filter_router_hsh_cfg_fmask[] = { + [FILTER_HASH_MSK_SRC_ID] = BIT(0), + [FILTER_HASH_MSK_SRC_IP] = BIT(1), + [FILTER_HASH_MSK_DST_IP] = BIT(2), + [FILTER_HASH_MSK_SRC_PORT] = BIT(3), + [FILTER_HASH_MSK_DST_PORT] = BIT(4), + [FILTER_HASH_MSK_PROTOCOL] = BIT(5), + [FILTER_HASH_MSK_METADATA] = BIT(6), + [FILTER_HASH_MSK_ALL] = GENMASK(6, 0), + /* Bits 7-15 reserved */ + [ROUTER_HASH_MSK_SRC_ID] = BIT(16), + [ROUTER_HASH_MSK_SRC_IP] = BIT(17), + [ROUTER_HASH_MSK_DST_IP] = BIT(18), + [ROUTER_HASH_MSK_SRC_PORT] = BIT(19), + [ROUTER_HASH_MSK_DST_PORT] = BIT(20), + [ROUTER_HASH_MSK_PROTOCOL] = BIT(21), + [ROUTER_HASH_MSK_METADATA] = BIT(22), + [ROUTER_HASH_MSK_ALL] = GENMASK(22, 16), + /* Bits 23-31 reserved */ +}; + +IPA_REG_STRIDE_FIELDS(ENDP_FILTER_ROUTER_HSH_CFG, endp_filter_router_hsh_cfg, + 0x0000085c, 0x0070); + +/* Valid bits defined by enum ipa_irq_id; only used for GSI_EE_AP */ +IPA_REG(IPA_IRQ_STTS, ipa_irq_stts, 0x00004008 + 0x1000 * GSI_EE_AP); + +/* Valid bits defined by enum ipa_irq_id; only used for GSI_EE_AP */ +IPA_REG(IPA_IRQ_EN, ipa_irq_en, 0x0000400c + 0x1000 * GSI_EE_AP); + +/* Valid bits defined by enum ipa_irq_id; only used for GSI_EE_AP */ +IPA_REG(IPA_IRQ_CLR, ipa_irq_clr, 0x00004010 + 0x1000 * GSI_EE_AP); + +static const u32 ipa_reg_ipa_irq_uc_fmask[] = { + [UC_INTR] = BIT(0), + /* Bits 1-31 reserved */ +}; + +IPA_REG_FIELDS(IPA_IRQ_UC, ipa_irq_uc, 0x0000401c + 0x1000 * GSI_EE_AP); + +/* Valid bits defined by ipa->available */ +IPA_REG(IRQ_SUSPEND_INFO, irq_suspend_info, 0x00004030 + 0x1000 * GSI_EE_AP); + +/* Valid bits defined by ipa->available */ +IPA_REG(IRQ_SUSPEND_EN, irq_suspend_en, 0x00004034 + 0x1000 * GSI_EE_AP); + +/* Valid bits defined by ipa->available */ +IPA_REG(IRQ_SUSPEND_CLR, irq_suspend_clr, 0x00004038 + 0x1000 * GSI_EE_AP); + +static const struct ipa_reg *ipa_reg_array[] = { + [COMP_CFG] = &ipa_reg_comp_cfg, + [CLKON_CFG] = &ipa_reg_clkon_cfg, + [ROUTE] = &ipa_reg_route, + [SHARED_MEM_SIZE] = &ipa_reg_shared_mem_size, + [QSB_MAX_WRITES] = &ipa_reg_qsb_max_writes, + [QSB_MAX_READS] = &ipa_reg_qsb_max_reads, + [FILT_ROUT_HASH_EN] = &ipa_reg_filt_rout_hash_en, + [FILT_ROUT_HASH_FLUSH] = &ipa_reg_filt_rout_hash_flush, + [STATE_AGGR_ACTIVE] = &ipa_reg_state_aggr_active, + [LOCAL_PKT_PROC_CNTXT] = &ipa_reg_local_pkt_proc_cntxt, + [AGGR_FORCE_CLOSE] = &ipa_reg_aggr_force_close, + [IPA_TX_CFG] = &ipa_reg_ipa_tx_cfg, + [FLAVOR_0] = &ipa_reg_flavor_0, + [IDLE_INDICATION_CFG] = &ipa_reg_idle_indication_cfg, + [QTIME_TIMESTAMP_CFG] = &ipa_reg_qtime_timestamp_cfg, + [TIMERS_XO_CLK_DIV_CFG] = &ipa_reg_timers_xo_clk_div_cfg, + [TIMERS_PULSE_GRAN_CFG] = &ipa_reg_timers_pulse_gran_cfg, + [SRC_RSRC_GRP_01_RSRC_TYPE] = &ipa_reg_src_rsrc_grp_01_rsrc_type, + [SRC_RSRC_GRP_23_RSRC_TYPE] = &ipa_reg_src_rsrc_grp_23_rsrc_type, + [DST_RSRC_GRP_01_RSRC_TYPE] = &ipa_reg_dst_rsrc_grp_01_rsrc_type, + [DST_RSRC_GRP_23_RSRC_TYPE] = &ipa_reg_dst_rsrc_grp_23_rsrc_type, + [ENDP_INIT_CFG] = &ipa_reg_endp_init_cfg, + [ENDP_INIT_NAT] = &ipa_reg_endp_init_nat, + [ENDP_INIT_HDR] = &ipa_reg_endp_init_hdr, + [ENDP_INIT_HDR_EXT] = &ipa_reg_endp_init_hdr_ext, + [ENDP_INIT_HDR_METADATA_MASK] = &ipa_reg_endp_init_hdr_metadata_mask, + [ENDP_INIT_MODE] = &ipa_reg_endp_init_mode, + [ENDP_INIT_AGGR] = &ipa_reg_endp_init_aggr, + [ENDP_INIT_HOL_BLOCK_EN] = &ipa_reg_endp_init_hol_block_en, + [ENDP_INIT_HOL_BLOCK_TIMER] = &ipa_reg_endp_init_hol_block_timer, + [ENDP_INIT_DEAGGR] = &ipa_reg_endp_init_deaggr, + [ENDP_INIT_RSRC_GRP] = &ipa_reg_endp_init_rsrc_grp, + [ENDP_INIT_SEQ] = &ipa_reg_endp_init_seq, + [ENDP_STATUS] = &ipa_reg_endp_status, + [ENDP_FILTER_ROUTER_HSH_CFG] = &ipa_reg_endp_filter_router_hsh_cfg, + [IPA_IRQ_STTS] = &ipa_reg_ipa_irq_stts, + [IPA_IRQ_EN] = &ipa_reg_ipa_irq_en, + [IPA_IRQ_CLR] = &ipa_reg_ipa_irq_clr, + [IPA_IRQ_UC] = &ipa_reg_ipa_irq_uc, + [IRQ_SUSPEND_INFO] = &ipa_reg_irq_suspend_info, + [IRQ_SUSPEND_EN] = &ipa_reg_irq_suspend_en, + [IRQ_SUSPEND_CLR] = &ipa_reg_irq_suspend_clr, +}; + +const struct ipa_regs ipa_regs_v4_9 = { + .reg_count = ARRAY_SIZE(ipa_reg_array), + .reg = ipa_reg_array, +}; diff --git a/drivers/net/ipvlan/ipvlan_main.c b/drivers/net/ipvlan/ipvlan_main.c index 49ba8a50dfb1..54c94a69c2bb 100644 --- a/drivers/net/ipvlan/ipvlan_main.c +++ b/drivers/net/ipvlan/ipvlan_main.c @@ -408,8 +408,8 @@ static int ipvlan_ethtool_get_link_ksettings(struct net_device *dev, static void ipvlan_ethtool_get_drvinfo(struct net_device *dev, struct ethtool_drvinfo *drvinfo) { - strlcpy(drvinfo->driver, IPVLAN_DRV, sizeof(drvinfo->driver)); - strlcpy(drvinfo->version, IPV_DRV_VER, sizeof(drvinfo->version)); + strscpy(drvinfo->driver, IPVLAN_DRV, sizeof(drvinfo->driver)); + strscpy(drvinfo->version, IPV_DRV_VER, sizeof(drvinfo->version)); } static u32 ipvlan_ethtool_get_msglevel(struct net_device *dev) diff --git a/drivers/net/macsec.c b/drivers/net/macsec.c index c6d271e5687e..c891b60937a7 100644 --- a/drivers/net/macsec.c +++ b/drivers/net/macsec.c @@ -18,14 +18,13 @@ #include <net/sock.h> #include <net/gro_cells.h> #include <net/macsec.h> +#include <net/dst_metadata.h> #include <linux/phy.h> #include <linux/byteorder/generic.h> #include <linux/if_arp.h> #include <uapi/linux/if_macsec.h> -#define MACSEC_SCI_LEN 8 - /* SecTAG length = macsec_eth_header without the optional SCI */ #define MACSEC_TAG_LEN 6 @@ -46,20 +45,10 @@ struct macsec_eth_header { u8 secure_channel_id[8]; /* optional */ } __packed; -#define MACSEC_TCI_VERSION 0x80 -#define MACSEC_TCI_ES 0x40 /* end station */ -#define MACSEC_TCI_SC 0x20 /* SCI present */ -#define MACSEC_TCI_SCB 0x10 /* epon */ -#define MACSEC_TCI_E 0x08 /* encryption */ -#define MACSEC_TCI_C 0x04 /* changed text */ -#define MACSEC_AN_MASK 0x03 /* association number */ -#define MACSEC_TCI_CONFID (MACSEC_TCI_E | MACSEC_TCI_C) - /* minimum secure data length deemed "not short", see IEEE 802.1AE-2006 9.7 */ #define MIN_NON_SHORT_LEN 48 #define GCM_AES_IV_LEN 12 -#define DEFAULT_ICV_LEN 16 #define for_each_rxsc(secy, sc) \ for (sc = rcu_dereference_bh(secy->rx_sc); \ @@ -243,7 +232,6 @@ static struct macsec_cb *macsec_skb_cb(struct sk_buff *skb) return (struct macsec_cb *)skb->cb; } -#define MACSEC_PORT_ES (htons(0x0001)) #define MACSEC_PORT_SCB (0x0000) #define MACSEC_UNDEF_SCI ((__force sci_t)0xffffffffffffffffULL) #define MACSEC_UNDEF_SSCI ((__force ssci_t)0xffffffff) @@ -258,14 +246,6 @@ static struct macsec_cb *macsec_skb_cb(struct sk_buff *skb) #define DEFAULT_ENCODING_SA 0 #define MACSEC_XPN_MAX_REPLAY_WINDOW (((1 << 30) - 1)) -static bool send_sci(const struct macsec_secy *secy) -{ - const struct macsec_tx_sc *tx_sc = &secy->tx_sc; - - return tx_sc->send_sci || - (secy->n_rx_sc > 1 && !tx_sc->end_station && !tx_sc->scb); -} - static sci_t make_sci(const u8 *addr, __be16 port) { sci_t sci; @@ -330,7 +310,7 @@ static void macsec_fill_sectag(struct macsec_eth_header *h, /* with GCM, C/E clear for !encrypt, both set for encrypt */ if (tx_sc->encrypt) h->tci_an |= MACSEC_TCI_CONFID; - else if (secy->icv_len != DEFAULT_ICV_LEN) + else if (secy->icv_len != MACSEC_DEFAULT_ICV_LEN) h->tci_an |= MACSEC_TCI_C; h->tci_an |= tx_sc->encoding_sa; @@ -654,7 +634,7 @@ static struct sk_buff *macsec_encrypt(struct sk_buff *skb, unprotected_len = skb->len; eth = eth_hdr(skb); - sci_present = send_sci(secy); + sci_present = macsec_send_sci(secy); hh = skb_push(skb, macsec_extra_len(sci_present)); memmove(hh, eth, 2 * ETH_ALEN); @@ -1024,11 +1004,13 @@ static enum rx_handler_result handle_not_macsec(struct sk_buff *skb) /* Deliver to the uncontrolled port by default */ enum rx_handler_result ret = RX_HANDLER_PASS; struct ethhdr *hdr = eth_hdr(skb); + struct metadata_dst *md_dst; struct macsec_rxh_data *rxd; struct macsec_dev *macsec; rcu_read_lock(); rxd = macsec_data_rcu(skb->dev); + md_dst = skb_metadata_dst(skb); list_for_each_entry_rcu(macsec, &rxd->secys, secys) { struct sk_buff *nskb; @@ -1039,6 +1021,10 @@ static enum rx_handler_result handle_not_macsec(struct sk_buff *skb) * the SecTAG, so we have to deduce which port to deliver to. */ if (macsec_is_offloaded(macsec) && netif_running(ndev)) { + if (md_dst && md_dst->type == METADATA_MACSEC && + (!find_rx_sc(&macsec->secy, md_dst->u.macsec_info.sci))) + continue; + if (ether_addr_equal_64bits(hdr->h_dest, ndev->dev_addr)) { /* exact match, divert skb to this port */ @@ -1296,7 +1282,7 @@ nosci: /* 10.6.1 if the SC is not found */ cbit = !!(hdr->tci_an & MACSEC_TCI_C); if (!cbit) - macsec_finalize_skb(skb, DEFAULT_ICV_LEN, + macsec_finalize_skb(skb, MACSEC_DEFAULT_ICV_LEN, macsec_extra_len(macsec_skb_cb(skb)->has_sci)); list_for_each_entry_rcu(macsec, &rxd->secys, secys) { @@ -1677,22 +1663,8 @@ static int macsec_offload(int (* const func)(struct macsec_context *), if (ctx->offload == MACSEC_OFFLOAD_PHY) mutex_lock(&ctx->phydev->lock); - /* Phase I: prepare. The drive should fail here if there are going to be - * issues in the commit phase. - */ - ctx->prepare = true; ret = (*func)(ctx); - if (ret) - goto phy_unlock; - /* Phase II: commit. This step cannot fail. */ - ctx->prepare = false; - ret = (*func)(ctx); - /* This should never happen: commit is not allowed to fail */ - if (unlikely(ret)) - WARN(1, "MACsec offloading commit failed (%d)\n", ret); - -phy_unlock: if (ctx->offload == MACSEC_OFFLOAD_PHY) mutex_unlock(&ctx->phydev->lock); @@ -1842,6 +1814,12 @@ static int macsec_add_rxsa(struct sk_buff *skb, struct genl_info *info) rx_sa->sc = rx_sc; + if (secy->xpn) { + rx_sa->ssci = nla_get_ssci(tb_sa[MACSEC_SA_ATTR_SSCI]); + nla_memcpy(rx_sa->key.salt.bytes, tb_sa[MACSEC_SA_ATTR_SALT], + MACSEC_SALT_LEN); + } + /* If h/w offloading is available, propagate to the device */ if (macsec_is_offloaded(netdev_priv(dev))) { const struct macsec_ops *ops; @@ -1864,12 +1842,6 @@ static int macsec_add_rxsa(struct sk_buff *skb, struct genl_info *info) goto cleanup; } - if (secy->xpn) { - rx_sa->ssci = nla_get_ssci(tb_sa[MACSEC_SA_ATTR_SSCI]); - nla_memcpy(rx_sa->key.salt.bytes, tb_sa[MACSEC_SA_ATTR_SALT], - MACSEC_SALT_LEN); - } - nla_memcpy(rx_sa->key.id, tb_sa[MACSEC_SA_ATTR_KEYID], MACSEC_KEYID_LEN); rcu_assign_pointer(rx_sc->sa[assoc_num], rx_sa); @@ -2084,6 +2056,12 @@ static int macsec_add_txsa(struct sk_buff *skb, struct genl_info *info) if (assoc_num == tx_sc->encoding_sa && tx_sa->active) secy->operational = true; + if (secy->xpn) { + tx_sa->ssci = nla_get_ssci(tb_sa[MACSEC_SA_ATTR_SSCI]); + nla_memcpy(tx_sa->key.salt.bytes, tb_sa[MACSEC_SA_ATTR_SALT], + MACSEC_SALT_LEN); + } + /* If h/w offloading is available, propagate to the device */ if (macsec_is_offloaded(netdev_priv(dev))) { const struct macsec_ops *ops; @@ -2106,12 +2084,6 @@ static int macsec_add_txsa(struct sk_buff *skb, struct genl_info *info) goto cleanup; } - if (secy->xpn) { - tx_sa->ssci = nla_get_ssci(tb_sa[MACSEC_SA_ATTR_SSCI]); - nla_memcpy(tx_sa->key.salt.bytes, tb_sa[MACSEC_SA_ATTR_SALT], - MACSEC_SALT_LEN); - } - nla_memcpy(tx_sa->key.id, tb_sa[MACSEC_SA_ATTR_KEYID], MACSEC_KEYID_LEN); rcu_assign_pointer(tx_sc->sa[assoc_num], tx_sa); @@ -3404,6 +3376,7 @@ static struct genl_family macsec_fam __ro_after_init = { .module = THIS_MODULE, .small_ops = macsec_genl_ops, .n_small_ops = ARRAY_SIZE(macsec_genl_ops), + .resv_start_op = MACSEC_CMD_UPD_OFFLOAD + 1, }; static netdev_tx_t macsec_start_xmit(struct sk_buff *skb, @@ -3415,6 +3388,11 @@ static netdev_tx_t macsec_start_xmit(struct sk_buff *skb, int ret, len; if (macsec_is_offloaded(netdev_priv(dev))) { + struct metadata_dst *md_dst = secy->tx_sc.md_dst; + + skb_dst_drop(skb); + dst_hold(&md_dst->dst); + skb_dst_set(skb, &md_dst->dst); skb->dev = macsec->real_dev; return dev_queue_xmit(skb); } @@ -3742,6 +3720,8 @@ static void macsec_free_netdev(struct net_device *dev) { struct macsec_dev *macsec = macsec_priv(dev); + if (macsec->secy.tx_sc.md_dst) + metadata_dst_free(macsec->secy.tx_sc.md_dst); free_percpu(macsec->stats); free_percpu(macsec->secy.tx_sc.stats); @@ -4014,6 +3994,13 @@ static int macsec_add_dev(struct net_device *dev, sci_t sci, u8 icv_len) return -ENOMEM; } + secy->tx_sc.md_dst = metadata_dst_alloc(0, METADATA_MACSEC, GFP_KERNEL); + if (!secy->tx_sc.md_dst) { + free_percpu(secy->tx_sc.stats); + free_percpu(macsec->stats); + return -ENOMEM; + } + if (sci == MACSEC_UNDEF_SCI) sci = dev_to_sci(dev, MACSEC_PORT_ES); @@ -4027,6 +4014,7 @@ static int macsec_add_dev(struct net_device *dev, sci_t sci, u8 icv_len) secy->xpn = DEFAULT_XPN; secy->sci = sci; + secy->tx_sc.md_dst->u.macsec_info.sci = sci; secy->tx_sc.active = true; secy->tx_sc.encoding_sa = DEFAULT_ENCODING_SA; secy->tx_sc.encrypt = DEFAULT_ENCRYPT; @@ -4045,7 +4033,7 @@ static int macsec_newlink(struct net *net, struct net_device *dev, { struct macsec_dev *macsec = macsec_priv(dev); rx_handler_func_t *rx_handler; - u8 icv_len = DEFAULT_ICV_LEN; + u8 icv_len = MACSEC_DEFAULT_ICV_LEN; struct net_device *real_dev; int err, mtu; sci_t sci; @@ -4169,7 +4157,7 @@ static int macsec_validate_attr(struct nlattr *tb[], struct nlattr *data[], struct netlink_ext_ack *extack) { u64 csid = MACSEC_DEFAULT_CIPHER_ID; - u8 icv_len = DEFAULT_ICV_LEN; + u8 icv_len = MACSEC_DEFAULT_ICV_LEN; int flag; bool es, scb, sci; @@ -4181,7 +4169,7 @@ static int macsec_validate_attr(struct nlattr *tb[], struct nlattr *data[], if (data[IFLA_MACSEC_ICV_LEN]) { icv_len = nla_get_u8(data[IFLA_MACSEC_ICV_LEN]); - if (icv_len != DEFAULT_ICV_LEN) { + if (icv_len != MACSEC_DEFAULT_ICV_LEN) { char dummy_key[DEFAULT_SAK_LEN] = { 0 }; struct crypto_aead *dummy_tfm; diff --git a/drivers/net/macvlan.c b/drivers/net/macvlan.c index 1080d6ebff63..713e3354cb2e 100644 --- a/drivers/net/macvlan.c +++ b/drivers/net/macvlan.c @@ -1043,8 +1043,8 @@ static int macvlan_fdb_del(struct ndmsg *ndm, struct nlattr *tb[], static void macvlan_ethtool_get_drvinfo(struct net_device *dev, struct ethtool_drvinfo *drvinfo) { - strlcpy(drvinfo->driver, "macvlan", sizeof(drvinfo->driver)); - strlcpy(drvinfo->version, "0.1", sizeof(drvinfo->version)); + strscpy(drvinfo->driver, "macvlan", sizeof(drvinfo->driver)); + strscpy(drvinfo->version, "0.1", sizeof(drvinfo->version)); } static int macvlan_ethtool_get_link_ksettings(struct net_device *dev, diff --git a/drivers/net/macvtap.c b/drivers/net/macvtap.c index cecf8c63096c..d1f435788e90 100644 --- a/drivers/net/macvtap.c +++ b/drivers/net/macvtap.c @@ -207,7 +207,7 @@ static struct notifier_block macvtap_notifier_block __read_mostly = { .notifier_call = macvtap_device_event, }; -static int macvtap_init(void) +static int __init macvtap_init(void) { int err; @@ -241,7 +241,7 @@ out1: } module_init(macvtap_init); -static void macvtap_exit(void) +static void __exit macvtap_exit(void) { rtnl_link_unregister(&macvtap_link_ops); unregister_netdevice_notifier(&macvtap_notifier_block); diff --git a/drivers/net/mdio/fwnode_mdio.c b/drivers/net/mdio/fwnode_mdio.c index 1c1584fca632..689e728345ce 100644 --- a/drivers/net/mdio/fwnode_mdio.c +++ b/drivers/net/mdio/fwnode_mdio.c @@ -10,10 +10,31 @@ #include <linux/fwnode_mdio.h> #include <linux/of.h> #include <linux/phy.h> +#include <linux/pse-pd/pse.h> MODULE_AUTHOR("Calvin Johnson <calvin.johnson@oss.nxp.com>"); MODULE_LICENSE("GPL"); +static struct pse_control * +fwnode_find_pse_control(struct fwnode_handle *fwnode) +{ + struct pse_control *psec; + struct device_node *np; + + if (!IS_ENABLED(CONFIG_PSE_CONTROLLER)) + return NULL; + + np = to_of_node(fwnode); + if (!np) + return NULL; + + psec = of_pse_control_get(np); + if (PTR_ERR(psec) == -ENOENT) + return NULL; + + return psec; +} + static struct mii_timestamper * fwnode_find_mii_timestamper(struct fwnode_handle *fwnode) { @@ -91,14 +112,21 @@ int fwnode_mdiobus_register_phy(struct mii_bus *bus, struct fwnode_handle *child, u32 addr) { struct mii_timestamper *mii_ts = NULL; + struct pse_control *psec = NULL; struct phy_device *phy; bool is_c45 = false; u32 phy_id; int rc; + psec = fwnode_find_pse_control(child); + if (IS_ERR(psec)) + return PTR_ERR(psec); + mii_ts = fwnode_find_mii_timestamper(child); - if (IS_ERR(mii_ts)) - return PTR_ERR(mii_ts); + if (IS_ERR(mii_ts)) { + rc = PTR_ERR(mii_ts); + goto clean_pse; + } rc = fwnode_property_match_string(child, "compatible", "ethernet-phy-ieee802.3-c45"); @@ -110,8 +138,8 @@ int fwnode_mdiobus_register_phy(struct mii_bus *bus, else phy = phy_device_create(bus, addr, phy_id, 0, NULL); if (IS_ERR(phy)) { - unregister_mii_timestamper(mii_ts); - return PTR_ERR(phy); + rc = PTR_ERR(phy); + goto clean_mii_ts; } if (is_acpi_node(child)) { @@ -125,25 +153,33 @@ int fwnode_mdiobus_register_phy(struct mii_bus *bus, /* All data is now stored in the phy struct, so register it */ rc = phy_device_register(phy); if (rc) { - phy_device_free(phy); fwnode_handle_put(phy->mdio.dev.fwnode); - return rc; + goto clean_phy; } } else if (is_of_node(child)) { rc = fwnode_mdiobus_phy_device_register(bus, phy, child, addr); - if (rc) { - unregister_mii_timestamper(mii_ts); - phy_device_free(phy); - return rc; - } + if (rc) + goto clean_phy; } + phy->psec = psec; + /* phy->mii_ts may already be defined by the PHY driver. A * mii_timestamper probed via the device tree will still have * precedence. */ if (mii_ts) phy->mii_ts = mii_ts; + return 0; + +clean_phy: + phy_device_free(phy); +clean_mii_ts: + unregister_mii_timestamper(mii_ts); +clean_pse: + pse_control_put(psec); + + return rc; } EXPORT_SYMBOL(fwnode_mdiobus_register_phy); diff --git a/drivers/net/mdio/mdio-i2c.c b/drivers/net/mdio/mdio-i2c.c index 09200a70b315..bf8bf5e20faf 100644 --- a/drivers/net/mdio/mdio-i2c.c +++ b/drivers/net/mdio/mdio-i2c.c @@ -3,6 +3,7 @@ * MDIO I2C bridge * * Copyright (C) 2015-2016 Russell King + * Copyright (C) 2021 Marek Behun * * Network PHYs can appear on I2C buses when they are part of SFP module. * This driver exposes these PHYs to the networking PHY code, allowing @@ -12,6 +13,7 @@ #include <linux/i2c.h> #include <linux/mdio/mdio-i2c.h> #include <linux/phy.h> +#include <linux/sfp.h> /* * I2C bus addresses 0x50 and 0x51 are normally an EEPROM, which is @@ -28,7 +30,7 @@ static unsigned int i2c_mii_phy_addr(int phy_id) return phy_id + 0x40; } -static int i2c_mii_read(struct mii_bus *bus, int phy_id, int reg) +static int i2c_mii_read_default(struct mii_bus *bus, int phy_id, int reg) { struct i2c_adapter *i2c = bus->priv; struct i2c_msg msgs[2]; @@ -62,7 +64,8 @@ static int i2c_mii_read(struct mii_bus *bus, int phy_id, int reg) return data[0] << 8 | data[1]; } -static int i2c_mii_write(struct mii_bus *bus, int phy_id, int reg, u16 val) +static int i2c_mii_write_default(struct mii_bus *bus, int phy_id, int reg, + u16 val) { struct i2c_adapter *i2c = bus->priv; struct i2c_msg msg; @@ -91,9 +94,288 @@ static int i2c_mii_write(struct mii_bus *bus, int phy_id, int reg, u16 val) return ret < 0 ? ret : 0; } -struct mii_bus *mdio_i2c_alloc(struct device *parent, struct i2c_adapter *i2c) +/* RollBall SFPs do not access internal PHY via I2C address 0x56, but + * instead via address 0x51, when SFP page is set to 0x03 and password to + * 0xffffffff. + * + * address size contents description + * ------- ---- -------- ----------- + * 0x80 1 CMD 0x01/0x02/0x04 for write/read/done + * 0x81 1 DEV Clause 45 device + * 0x82 2 REG Clause 45 register + * 0x84 2 VAL Register value + */ +#define ROLLBALL_PHY_I2C_ADDR 0x51 + +#define ROLLBALL_PASSWORD (SFP_VSL + 3) + +#define ROLLBALL_CMD_ADDR 0x80 +#define ROLLBALL_DATA_ADDR 0x81 + +#define ROLLBALL_CMD_WRITE 0x01 +#define ROLLBALL_CMD_READ 0x02 +#define ROLLBALL_CMD_DONE 0x04 + +#define SFP_PAGE_ROLLBALL_MDIO 3 + +static int __i2c_transfer_err(struct i2c_adapter *i2c, struct i2c_msg *msgs, + int num) +{ + int ret; + + ret = __i2c_transfer(i2c, msgs, num); + if (ret < 0) + return ret; + else if (ret != num) + return -EIO; + else + return 0; +} + +static int __i2c_rollball_get_page(struct i2c_adapter *i2c, int bus_addr, + u8 *page) +{ + struct i2c_msg msgs[2]; + u8 addr = SFP_PAGE; + + msgs[0].addr = bus_addr; + msgs[0].flags = 0; + msgs[0].len = 1; + msgs[0].buf = &addr; + + msgs[1].addr = bus_addr; + msgs[1].flags = I2C_M_RD; + msgs[1].len = 1; + msgs[1].buf = page; + + return __i2c_transfer_err(i2c, msgs, 2); +} + +static int __i2c_rollball_set_page(struct i2c_adapter *i2c, int bus_addr, + u8 page) +{ + struct i2c_msg msg; + u8 buf[2]; + + buf[0] = SFP_PAGE; + buf[1] = page; + + msg.addr = bus_addr; + msg.flags = 0; + msg.len = 2; + msg.buf = buf; + + return __i2c_transfer_err(i2c, &msg, 1); +} + +/* In order to not interfere with other SFP code (which possibly may manipulate + * SFP_PAGE), for every transfer we do this: + * 1. lock the bus + * 2. save content of SFP_PAGE + * 3. set SFP_PAGE to 3 + * 4. do the transfer + * 5. restore original SFP_PAGE + * 6. unlock the bus + * Note that one might think that steps 2 to 5 could be theoretically done all + * in one call to i2c_transfer (by constructing msgs array in such a way), but + * unfortunately tests show that this does not work :-( Changed SFP_PAGE does + * not take into account until i2c_transfer() is done. + */ +static int i2c_transfer_rollball(struct i2c_adapter *i2c, + struct i2c_msg *msgs, int num) +{ + int ret, main_err = 0; + u8 saved_page; + + i2c_lock_bus(i2c, I2C_LOCK_SEGMENT); + + /* save original page */ + ret = __i2c_rollball_get_page(i2c, msgs->addr, &saved_page); + if (ret) + goto unlock; + + /* change to RollBall MDIO page */ + ret = __i2c_rollball_set_page(i2c, msgs->addr, SFP_PAGE_ROLLBALL_MDIO); + if (ret) + goto unlock; + + /* do the transfer; we try to restore original page if this fails */ + ret = __i2c_transfer_err(i2c, msgs, num); + if (ret) + main_err = ret; + + /* restore original page */ + ret = __i2c_rollball_set_page(i2c, msgs->addr, saved_page); + +unlock: + i2c_unlock_bus(i2c, I2C_LOCK_SEGMENT); + + return main_err ? : ret; +} + +static int i2c_rollball_mii_poll(struct mii_bus *bus, int bus_addr, u8 *buf, + size_t len) +{ + struct i2c_adapter *i2c = bus->priv; + struct i2c_msg msgs[2]; + u8 cmd_addr, tmp, *res; + int i, ret; + + cmd_addr = ROLLBALL_CMD_ADDR; + + res = buf ? buf : &tmp; + len = buf ? len : 1; + + msgs[0].addr = bus_addr; + msgs[0].flags = 0; + msgs[0].len = 1; + msgs[0].buf = &cmd_addr; + + msgs[1].addr = bus_addr; + msgs[1].flags = I2C_M_RD; + msgs[1].len = len; + msgs[1].buf = res; + + /* By experiment it takes up to 70 ms to access a register for these + * SFPs. Sleep 20ms between iterations and try 10 times. + */ + i = 10; + do { + msleep(20); + + ret = i2c_transfer_rollball(i2c, msgs, ARRAY_SIZE(msgs)); + if (ret) + return ret; + + if (*res == ROLLBALL_CMD_DONE) + return 0; + } while (i-- > 0); + + dev_dbg(&bus->dev, "poll timed out\n"); + + return -ETIMEDOUT; +} + +static int i2c_rollball_mii_cmd(struct mii_bus *bus, int bus_addr, u8 cmd, + u8 *data, size_t len) +{ + struct i2c_adapter *i2c = bus->priv; + struct i2c_msg msgs[2]; + u8 cmdbuf[2]; + + cmdbuf[0] = ROLLBALL_CMD_ADDR; + cmdbuf[1] = cmd; + + msgs[0].addr = bus_addr; + msgs[0].flags = 0; + msgs[0].len = len; + msgs[0].buf = data; + + msgs[1].addr = bus_addr; + msgs[1].flags = 0; + msgs[1].len = sizeof(cmdbuf); + msgs[1].buf = cmdbuf; + + return i2c_transfer_rollball(i2c, msgs, ARRAY_SIZE(msgs)); +} + +static int i2c_mii_read_rollball(struct mii_bus *bus, int phy_id, int reg) +{ + u8 buf[4], res[6]; + int bus_addr, ret; + u16 val; + + if (!(reg & MII_ADDR_C45)) + return -EOPNOTSUPP; + + bus_addr = i2c_mii_phy_addr(phy_id); + if (bus_addr != ROLLBALL_PHY_I2C_ADDR) + return 0xffff; + + buf[0] = ROLLBALL_DATA_ADDR; + buf[1] = (reg >> 16) & 0x1f; + buf[2] = (reg >> 8) & 0xff; + buf[3] = reg & 0xff; + + ret = i2c_rollball_mii_cmd(bus, bus_addr, ROLLBALL_CMD_READ, buf, + sizeof(buf)); + if (ret < 0) + return ret; + + ret = i2c_rollball_mii_poll(bus, bus_addr, res, sizeof(res)); + if (ret == -ETIMEDOUT) + return 0xffff; + else if (ret < 0) + return ret; + + val = res[4] << 8 | res[5]; + + return val; +} + +static int i2c_mii_write_rollball(struct mii_bus *bus, int phy_id, int reg, + u16 val) +{ + int bus_addr, ret; + u8 buf[6]; + + if (!(reg & MII_ADDR_C45)) + return -EOPNOTSUPP; + + bus_addr = i2c_mii_phy_addr(phy_id); + if (bus_addr != ROLLBALL_PHY_I2C_ADDR) + return 0; + + buf[0] = ROLLBALL_DATA_ADDR; + buf[1] = (reg >> 16) & 0x1f; + buf[2] = (reg >> 8) & 0xff; + buf[3] = reg & 0xff; + buf[4] = val >> 8; + buf[5] = val & 0xff; + + ret = i2c_rollball_mii_cmd(bus, bus_addr, ROLLBALL_CMD_WRITE, buf, + sizeof(buf)); + if (ret < 0) + return ret; + + ret = i2c_rollball_mii_poll(bus, bus_addr, NULL, 0); + if (ret < 0) + return ret; + + return 0; +} + +static int i2c_mii_init_rollball(struct i2c_adapter *i2c) +{ + struct i2c_msg msg; + u8 pw[5]; + int ret; + + pw[0] = ROLLBALL_PASSWORD; + pw[1] = 0xff; + pw[2] = 0xff; + pw[3] = 0xff; + pw[4] = 0xff; + + msg.addr = ROLLBALL_PHY_I2C_ADDR; + msg.flags = 0; + msg.len = sizeof(pw); + msg.buf = pw; + + ret = i2c_transfer(i2c, &msg, 1); + if (ret < 0) + return ret; + else if (ret != 1) + return -EIO; + else + return 0; +} + +struct mii_bus *mdio_i2c_alloc(struct device *parent, struct i2c_adapter *i2c, + enum mdio_i2c_proto protocol) { struct mii_bus *mii; + int ret; if (!i2c_check_functionality(i2c, I2C_FUNC_I2C)) return ERR_PTR(-EINVAL); @@ -104,10 +386,28 @@ struct mii_bus *mdio_i2c_alloc(struct device *parent, struct i2c_adapter *i2c) snprintf(mii->id, MII_BUS_ID_SIZE, "i2c:%s", dev_name(parent)); mii->parent = parent; - mii->read = i2c_mii_read; - mii->write = i2c_mii_write; mii->priv = i2c; + switch (protocol) { + case MDIO_I2C_ROLLBALL: + ret = i2c_mii_init_rollball(i2c); + if (ret < 0) { + dev_err(parent, + "Cannot initialize RollBall MDIO I2C protocol: %d\n", + ret); + mdiobus_free(mii); + return ERR_PTR(ret); + } + + mii->read = i2c_mii_read_rollball; + mii->write = i2c_mii_write_rollball; + break; + default: + mii->read = i2c_mii_read_default; + mii->write = i2c_mii_write_default; + break; + } + return mii; } EXPORT_SYMBOL_GPL(mdio_i2c_alloc); diff --git a/drivers/net/mdio/mdio-mscc-miim.c b/drivers/net/mdio/mdio-mscc-miim.c index 08541007b18a..51f68daac152 100644 --- a/drivers/net/mdio/mdio-mscc-miim.c +++ b/drivers/net/mdio/mdio-mscc-miim.c @@ -12,6 +12,7 @@ #include <linux/iopoll.h> #include <linux/kernel.h> #include <linux/mdio/mdio-mscc-miim.h> +#include <linux/mfd/ocelot.h> #include <linux/module.h> #include <linux/of_mdio.h> #include <linux/phy.h> @@ -270,44 +271,25 @@ static int mscc_miim_clk_set(struct mii_bus *bus) static int mscc_miim_probe(struct platform_device *pdev) { - struct regmap *mii_regmap, *phy_regmap = NULL; struct device_node *np = pdev->dev.of_node; + struct regmap *mii_regmap, *phy_regmap; struct device *dev = &pdev->dev; - void __iomem *regs, *phy_regs; struct mscc_miim_dev *miim; - struct resource *res; struct mii_bus *bus; int ret; - regs = devm_platform_get_and_ioremap_resource(pdev, 0, NULL); - if (IS_ERR(regs)) { - dev_err(dev, "Unable to map MIIM registers\n"); - return PTR_ERR(regs); - } - - mii_regmap = devm_regmap_init_mmio(dev, regs, &mscc_miim_regmap_config); - - if (IS_ERR(mii_regmap)) { - dev_err(dev, "Unable to create MIIM regmap\n"); - return PTR_ERR(mii_regmap); - } + mii_regmap = ocelot_regmap_from_resource(pdev, 0, + &mscc_miim_regmap_config); + if (IS_ERR(mii_regmap)) + return dev_err_probe(dev, PTR_ERR(mii_regmap), + "Unable to create MIIM regmap\n"); /* This resource is optional */ - res = platform_get_resource(pdev, IORESOURCE_MEM, 1); - if (res) { - phy_regs = devm_ioremap_resource(dev, res); - if (IS_ERR(phy_regs)) { - dev_err(dev, "Unable to map internal phy registers\n"); - return PTR_ERR(phy_regs); - } - - phy_regmap = devm_regmap_init_mmio(dev, phy_regs, - &mscc_miim_phy_regmap_config); - if (IS_ERR(phy_regmap)) { - dev_err(dev, "Unable to create phy register regmap\n"); - return PTR_ERR(phy_regmap); - } - } + phy_regmap = ocelot_regmap_from_resource_optional(pdev, 1, + &mscc_miim_phy_regmap_config); + if (IS_ERR(phy_regmap)) + return dev_err_probe(dev, PTR_ERR(phy_regmap), + "Unable to create phy register regmap\n"); ret = mscc_miim_setup(dev, &bus, "mscc_miim", mii_regmap, 0); if (ret < 0) { diff --git a/drivers/net/mdio/mdio-mux-meson-g12a.c b/drivers/net/mdio/mdio-mux-meson-g12a.c index b8866bc3f2e8..4a2e94faf57e 100644 --- a/drivers/net/mdio/mdio-mux-meson-g12a.c +++ b/drivers/net/mdio/mdio-mux-meson-g12a.c @@ -233,11 +233,9 @@ static int g12a_ephy_glue_clk_register(struct device *dev) snprintf(in_name, sizeof(in_name), "clkin%d", i); clk = devm_clk_get(dev, in_name); - if (IS_ERR(clk)) { - if (PTR_ERR(clk) != -EPROBE_DEFER) - dev_err(dev, "Missing clock %s\n", in_name); - return PTR_ERR(clk); - } + if (IS_ERR(clk)) + return dev_err_probe(dev, PTR_ERR(clk), + "Missing clock %s\n", in_name); parent_names[i] = __clk_get_name(clk); } @@ -317,12 +315,9 @@ static int g12a_mdio_mux_probe(struct platform_device *pdev) return PTR_ERR(priv->regs); priv->pclk = devm_clk_get(dev, "pclk"); - if (IS_ERR(priv->pclk)) { - ret = PTR_ERR(priv->pclk); - if (ret != -EPROBE_DEFER) - dev_err(dev, "failed to get peripheral clock\n"); - return ret; - } + if (IS_ERR(priv->pclk)) + return dev_err_probe(dev, PTR_ERR(priv->pclk), + "failed to get peripheral clock\n"); /* Make sure the device registers are clocked */ ret = clk_prepare_enable(priv->pclk); @@ -339,8 +334,7 @@ static int g12a_mdio_mux_probe(struct platform_device *pdev) ret = mdio_mux_init(dev, dev->of_node, g12a_mdio_switch_fn, &priv->mux_handle, dev, NULL); if (ret) { - if (ret != -EPROBE_DEFER) - dev_err(dev, "mdio multiplexer init failed: %d", ret); + dev_err_probe(dev, ret, "mdio multiplexer init failed\n"); goto err; } diff --git a/drivers/net/mdio/mdio-mux-mmioreg.c b/drivers/net/mdio/mdio-mux-mmioreg.c index c02fb2a067ee..c02c9c660016 100644 --- a/drivers/net/mdio/mdio-mux-mmioreg.c +++ b/drivers/net/mdio/mdio-mux-mmioreg.c @@ -159,12 +159,9 @@ static int mdio_mux_mmioreg_probe(struct platform_device *pdev) ret = mdio_mux_init(&pdev->dev, pdev->dev.of_node, mdio_mux_mmioreg_switch_fn, &s->mux_handle, s, NULL); - if (ret) { - if (ret != -EPROBE_DEFER) - dev_err(&pdev->dev, - "failed to register mdio-mux bus %pOF\n", np); - return ret; - } + if (ret) + return dev_err_probe(&pdev->dev, ret, + "failed to register mdio-mux bus %pOF\n", np); pdev->dev.platform_data = s; diff --git a/drivers/net/mdio/mdio-mux-multiplexer.c b/drivers/net/mdio/mdio-mux-multiplexer.c index 527acfc3c045..bfa5af577b0a 100644 --- a/drivers/net/mdio/mdio-mux-multiplexer.c +++ b/drivers/net/mdio/mdio-mux-multiplexer.c @@ -72,12 +72,9 @@ static int mdio_mux_multiplexer_probe(struct platform_device *pdev) return -ENOMEM; s->muxc = devm_mux_control_get(dev, NULL); - if (IS_ERR(s->muxc)) { - ret = PTR_ERR(s->muxc); - if (ret != -EPROBE_DEFER) - dev_err(&pdev->dev, "Failed to get mux: %d\n", ret); - return ret; - } + if (IS_ERR(s->muxc)) + return dev_err_probe(&pdev->dev, PTR_ERR(s->muxc), + "Failed to get mux\n"); platform_set_drvdata(pdev, s); diff --git a/drivers/net/net_failover.c b/drivers/net/net_failover.c index 21a0435c02de..7a28e082436e 100644 --- a/drivers/net/net_failover.c +++ b/drivers/net/net_failover.c @@ -324,8 +324,8 @@ static const struct net_device_ops failover_dev_ops = { static void nfo_ethtool_get_drvinfo(struct net_device *dev, struct ethtool_drvinfo *drvinfo) { - strlcpy(drvinfo->driver, FAILOVER_NAME, sizeof(drvinfo->driver)); - strlcpy(drvinfo->version, FAILOVER_VERSION, sizeof(drvinfo->version)); + strscpy(drvinfo->driver, FAILOVER_NAME, sizeof(drvinfo->driver)); + strscpy(drvinfo->version, FAILOVER_VERSION, sizeof(drvinfo->version)); } static int nfo_ethtool_get_link_ksettings(struct net_device *dev, diff --git a/drivers/net/netconsole.c b/drivers/net/netconsole.c index ddac61d79145..bdff9ac5056d 100644 --- a/drivers/net/netconsole.c +++ b/drivers/net/netconsole.c @@ -55,7 +55,7 @@ MODULE_PARM_DESC(oops_only, "Only log oops messages"); #ifndef MODULE static int __init option_setup(char *opt) { - strlcpy(config, opt, MAX_PARAM_LENGTH); + strscpy(config, opt, MAX_PARAM_LENGTH); return 1; } __setup("netconsole=", option_setup); @@ -178,7 +178,7 @@ static struct netconsole_target *alloc_param_target(char *target_config) goto fail; nt->np.name = "netconsole"; - strlcpy(nt->np.dev_name, "eth0", IFNAMSIZ); + strscpy(nt->np.dev_name, "eth0", IFNAMSIZ); nt->np.local_port = 6665; nt->np.remote_port = 6666; eth_broadcast_addr(nt->np.remote_mac); @@ -414,7 +414,7 @@ static ssize_t dev_name_store(struct config_item *item, const char *buf, return -EINVAL; } - strlcpy(nt->np.dev_name, buf, IFNAMSIZ); + strscpy(nt->np.dev_name, buf, IFNAMSIZ); /* Get rid of possible trailing newline from echo(1) */ len = strnlen(nt->np.dev_name, IFNAMSIZ); @@ -630,7 +630,7 @@ static struct config_item *make_netconsole_target(struct config_group *group, return ERR_PTR(-ENOMEM); nt->np.name = "netconsole"; - strlcpy(nt->np.dev_name, "eth0", IFNAMSIZ); + strscpy(nt->np.dev_name, "eth0", IFNAMSIZ); nt->np.local_port = 6665; nt->np.remote_port = 6666; eth_broadcast_addr(nt->np.remote_mac); @@ -708,7 +708,7 @@ restart: if (nt->np.dev == dev) { switch (event) { case NETDEV_CHANGENAME: - strlcpy(nt->np.dev_name, dev->name, IFNAMSIZ); + strscpy(nt->np.dev_name, dev->name, IFNAMSIZ); break; case NETDEV_RELEASE: case NETDEV_JOIN: diff --git a/drivers/net/netdevsim/dev.c b/drivers/net/netdevsim/dev.c index e88f783c297e..794fc0cc73b8 100644 --- a/drivers/net/netdevsim/dev.c +++ b/drivers/net/netdevsim/dev.c @@ -965,7 +965,6 @@ static int nsim_dev_reload_up(struct devlink *devlink, enum devlink_reload_actio struct netlink_ext_ack *extack) { struct nsim_dev *nsim_dev = devlink_priv(devlink); - int ret; if (nsim_dev->fail_reload) { /* For testing purposes, user set debugfs fail_reload @@ -976,15 +975,25 @@ static int nsim_dev_reload_up(struct devlink *devlink, enum devlink_reload_actio } *actions_performed = BIT(DEVLINK_RELOAD_ACTION_DRIVER_REINIT); - ret = nsim_dev_reload_create(nsim_dev, extack); - return ret; + + return nsim_dev_reload_create(nsim_dev, extack); } static int nsim_dev_info_get(struct devlink *devlink, struct devlink_info_req *req, struct netlink_ext_ack *extack) { - return devlink_info_driver_name_put(req, DRV_NAME); + int err; + + err = devlink_info_driver_name_put(req, DRV_NAME); + if (err) + return err; + err = devlink_info_version_stored_put_ext(req, "fw.mgmt", "10.20.30", + DEVLINK_INFO_VERSION_TYPE_COMPONENT); + if (err) + return err; + return devlink_info_version_running_put_ext(req, "fw.mgmt", "10.20.30", + DEVLINK_INFO_VERSION_TYPE_COMPONENT); } #define NSIM_DEV_FLASH_SIZE 500000 @@ -1312,8 +1321,7 @@ nsim_dev_devlink_trap_drop_counter_get(struct devlink *devlink, static const struct devlink_ops nsim_dev_devlink_ops = { .eswitch_mode_set = nsim_devlink_eswitch_mode_set, .eswitch_mode_get = nsim_devlink_eswitch_mode_get, - .supported_flash_update_params = DEVLINK_SUPPORT_FLASH_UPDATE_COMPONENT | - DEVLINK_SUPPORT_FLASH_UPDATE_OVERWRITE_MASK, + .supported_flash_update_params = DEVLINK_SUPPORT_FLASH_UPDATE_OVERWRITE_MASK, .reload_actions = BIT(DEVLINK_RELOAD_ACTION_DRIVER_REINIT), .reload_down = nsim_dev_reload_down, .reload_up = nsim_dev_reload_up, diff --git a/drivers/net/ntb_netdev.c b/drivers/net/ntb_netdev.c index 80bdc07f2cd3..464d88ca8ab0 100644 --- a/drivers/net/ntb_netdev.c +++ b/drivers/net/ntb_netdev.c @@ -364,9 +364,9 @@ static void ntb_get_drvinfo(struct net_device *ndev, { struct ntb_netdev *dev = netdev_priv(ndev); - strlcpy(info->driver, KBUILD_MODNAME, sizeof(info->driver)); - strlcpy(info->version, NTB_NETDEV_VER, sizeof(info->version)); - strlcpy(info->bus_info, pci_name(dev->pdev), sizeof(info->bus_info)); + strscpy(info->driver, KBUILD_MODNAME, sizeof(info->driver)); + strscpy(info->version, NTB_NETDEV_VER, sizeof(info->version)); + strscpy(info->bus_info, pci_name(dev->pdev), sizeof(info->bus_info)); } static int ntb_get_link_ksettings(struct net_device *dev, diff --git a/drivers/net/pcs/Kconfig b/drivers/net/pcs/Kconfig index 6289b7c765f1..6e7e6c346a3e 100644 --- a/drivers/net/pcs/Kconfig +++ b/drivers/net/pcs/Kconfig @@ -26,4 +26,10 @@ config PCS_RZN1_MIIC on RZ/N1 SoCs. This PCS converts MII to RMII/RGMII or can be set in pass-through mode for MII. +config PCS_ALTERA_TSE + tristate + help + This module provides helper functions for the Altera Triple Speed + Ethernet SGMII PCS, that can be found on the Intel Socfpga family. + endmenu diff --git a/drivers/net/pcs/Makefile b/drivers/net/pcs/Makefile index 0ff5388fcdea..4c780d8f2e98 100644 --- a/drivers/net/pcs/Makefile +++ b/drivers/net/pcs/Makefile @@ -6,3 +6,4 @@ pcs_xpcs-$(CONFIG_PCS_XPCS) := pcs-xpcs.o pcs-xpcs-nxp.o obj-$(CONFIG_PCS_XPCS) += pcs_xpcs.o obj-$(CONFIG_PCS_LYNX) += pcs-lynx.o obj-$(CONFIG_PCS_RZN1_MIIC) += pcs-rzn1-miic.o +obj-$(CONFIG_PCS_ALTERA_TSE) += pcs-altera-tse.o diff --git a/drivers/net/pcs/pcs-altera-tse.c b/drivers/net/pcs/pcs-altera-tse.c new file mode 100644 index 000000000000..97a7cabff962 --- /dev/null +++ b/drivers/net/pcs/pcs-altera-tse.c @@ -0,0 +1,175 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (C) 2022 Bootlin + * + * Maxime Chevallier <maxime.chevallier@bootlin.com> + */ + +#include <linux/netdevice.h> +#include <linux/phy.h> +#include <linux/phylink.h> +#include <linux/pcs-altera-tse.h> + +/* SGMII PCS register addresses + */ +#define SGMII_PCS_SCRATCH 0x10 +#define SGMII_PCS_REV 0x11 +#define SGMII_PCS_LINK_TIMER_0 0x12 +#define SGMII_PCS_LINK_TIMER_REG(x) (0x12 + (x)) +#define SGMII_PCS_LINK_TIMER_1 0x13 +#define SGMII_PCS_IF_MODE 0x14 +#define PCS_IF_MODE_SGMII_ENA BIT(0) +#define PCS_IF_MODE_USE_SGMII_AN BIT(1) +#define PCS_IF_MODE_SGMI_SPEED_MASK GENMASK(3, 2) +#define PCS_IF_MODE_SGMI_SPEED_10 (0 << 2) +#define PCS_IF_MODE_SGMI_SPEED_100 (1 << 2) +#define PCS_IF_MODE_SGMI_SPEED_1000 (2 << 2) +#define PCS_IF_MODE_SGMI_HALF_DUPLEX BIT(4) +#define PCS_IF_MODE_SGMI_PHY_AN BIT(5) +#define SGMII_PCS_DIS_READ_TO 0x15 +#define SGMII_PCS_READ_TO 0x16 +#define SGMII_PCS_SW_RESET_TIMEOUT 100 /* usecs */ + +struct altera_tse_pcs { + struct phylink_pcs pcs; + void __iomem *base; + int reg_width; +}; + +static struct altera_tse_pcs *phylink_pcs_to_tse_pcs(struct phylink_pcs *pcs) +{ + return container_of(pcs, struct altera_tse_pcs, pcs); +} + +static u16 tse_pcs_read(struct altera_tse_pcs *tse_pcs, int regnum) +{ + if (tse_pcs->reg_width == 4) + return readl(tse_pcs->base + regnum * 4); + else + return readw(tse_pcs->base + regnum * 2); +} + +static void tse_pcs_write(struct altera_tse_pcs *tse_pcs, int regnum, + u16 value) +{ + if (tse_pcs->reg_width == 4) + writel(value, tse_pcs->base + regnum * 4); + else + writew(value, tse_pcs->base + regnum * 2); +} + +static int tse_pcs_reset(struct altera_tse_pcs *tse_pcs) +{ + int i = 0; + u16 bmcr; + + /* Reset PCS block */ + bmcr = tse_pcs_read(tse_pcs, MII_BMCR); + bmcr |= BMCR_RESET; + tse_pcs_write(tse_pcs, MII_BMCR, bmcr); + + for (i = 0; i < SGMII_PCS_SW_RESET_TIMEOUT; i++) { + if (!(tse_pcs_read(tse_pcs, MII_BMCR) & BMCR_RESET)) + return 0; + udelay(1); + } + + return -ETIMEDOUT; +} + +static int alt_tse_pcs_validate(struct phylink_pcs *pcs, + unsigned long *supported, + const struct phylink_link_state *state) +{ + if (state->interface == PHY_INTERFACE_MODE_SGMII || + state->interface == PHY_INTERFACE_MODE_1000BASEX) + return 1; + + return -EINVAL; +} + +static int alt_tse_pcs_config(struct phylink_pcs *pcs, unsigned int mode, + phy_interface_t interface, + const unsigned long *advertising, + bool permit_pause_to_mac) +{ + struct altera_tse_pcs *tse_pcs = phylink_pcs_to_tse_pcs(pcs); + u32 ctrl, if_mode; + + ctrl = tse_pcs_read(tse_pcs, MII_BMCR); + if_mode = tse_pcs_read(tse_pcs, SGMII_PCS_IF_MODE); + + /* Set link timer to 1.6ms, as per the MegaCore Function User Guide */ + tse_pcs_write(tse_pcs, SGMII_PCS_LINK_TIMER_0, 0x0D40); + tse_pcs_write(tse_pcs, SGMII_PCS_LINK_TIMER_1, 0x03); + + if (interface == PHY_INTERFACE_MODE_SGMII) { + if_mode |= PCS_IF_MODE_USE_SGMII_AN | PCS_IF_MODE_SGMII_ENA; + } else if (interface == PHY_INTERFACE_MODE_1000BASEX) { + if_mode &= ~(PCS_IF_MODE_USE_SGMII_AN | PCS_IF_MODE_SGMII_ENA); + if_mode |= PCS_IF_MODE_SGMI_SPEED_1000; + } + + ctrl |= (BMCR_SPEED1000 | BMCR_FULLDPLX | BMCR_ANENABLE); + + tse_pcs_write(tse_pcs, MII_BMCR, ctrl); + tse_pcs_write(tse_pcs, SGMII_PCS_IF_MODE, if_mode); + + return tse_pcs_reset(tse_pcs); +} + +static void alt_tse_pcs_get_state(struct phylink_pcs *pcs, + struct phylink_link_state *state) +{ + struct altera_tse_pcs *tse_pcs = phylink_pcs_to_tse_pcs(pcs); + u16 bmsr, lpa; + + bmsr = tse_pcs_read(tse_pcs, MII_BMSR); + lpa = tse_pcs_read(tse_pcs, MII_LPA); + + phylink_mii_c22_pcs_decode_state(state, bmsr, lpa); +} + +static void alt_tse_pcs_an_restart(struct phylink_pcs *pcs) +{ + struct altera_tse_pcs *tse_pcs = phylink_pcs_to_tse_pcs(pcs); + u16 bmcr; + + bmcr = tse_pcs_read(tse_pcs, MII_BMCR); + bmcr |= BMCR_ANRESTART; + tse_pcs_write(tse_pcs, MII_BMCR, bmcr); + + /* This PCS seems to require a soft reset to re-sync the AN logic */ + tse_pcs_reset(tse_pcs); +} + +static const struct phylink_pcs_ops alt_tse_pcs_ops = { + .pcs_validate = alt_tse_pcs_validate, + .pcs_get_state = alt_tse_pcs_get_state, + .pcs_config = alt_tse_pcs_config, + .pcs_an_restart = alt_tse_pcs_an_restart, +}; + +struct phylink_pcs *alt_tse_pcs_create(struct net_device *ndev, + void __iomem *pcs_base, int reg_width) +{ + struct altera_tse_pcs *tse_pcs; + + if (reg_width != 4 && reg_width != 2) + return ERR_PTR(-EINVAL); + + tse_pcs = devm_kzalloc(&ndev->dev, sizeof(*tse_pcs), GFP_KERNEL); + if (!tse_pcs) + return ERR_PTR(-ENOMEM); + + tse_pcs->pcs.ops = &alt_tse_pcs_ops; + tse_pcs->base = pcs_base; + tse_pcs->reg_width = reg_width; + + return &tse_pcs->pcs; +} +EXPORT_SYMBOL_GPL(alt_tse_pcs_create); + +MODULE_LICENSE("GPL"); +MODULE_DESCRIPTION("Altera TSE PCS driver"); +MODULE_AUTHOR("Maxime Chevallier <maxime.chevallier@bootlin.com>"); diff --git a/drivers/net/phy/adin.c b/drivers/net/phy/adin.c index ee374a85544a..134637584a83 100644 --- a/drivers/net/phy/adin.c +++ b/drivers/net/phy/adin.c @@ -749,7 +749,7 @@ static void adin_get_strings(struct phy_device *phydev, u8 *data) int i; for (i = 0; i < ARRAY_SIZE(adin_hw_stats); i++) { - strlcpy(&data[i * ETH_GSTRING_LEN], + strscpy(&data[i * ETH_GSTRING_LEN], adin_hw_stats[i].string, ETH_GSTRING_LEN); } } diff --git a/drivers/net/phy/adin1100.c b/drivers/net/phy/adin1100.c index b6d139501199..7619d6185801 100644 --- a/drivers/net/phy/adin1100.c +++ b/drivers/net/phy/adin1100.c @@ -15,6 +15,8 @@ #include <linux/property.h> #define PHY_ID_ADIN1100 0x0283bc81 +#define PHY_ID_ADIN1110 0x0283bc91 +#define PHY_ID_ADIN2111 0x0283bca1 #define ADIN_FORCED_MODE 0x8000 #define ADIN_FORCED_MODE_EN BIT(0) @@ -265,7 +267,8 @@ static int adin_probe(struct phy_device *phydev) static struct phy_driver adin_driver[] = { { - PHY_ID_MATCH_MODEL(PHY_ID_ADIN1100), + .phy_id = PHY_ID_ADIN1100, + .phy_id_mask = 0xffffffcf, .name = "ADIN1100", .get_features = adin_get_features, .soft_reset = adin_soft_reset, @@ -284,6 +287,8 @@ module_phy_driver(adin_driver); static struct mdio_device_id __maybe_unused adin_tbl[] = { { PHY_ID_MATCH_MODEL(PHY_ID_ADIN1100) }, + { PHY_ID_MATCH_MODEL(PHY_ID_ADIN1110) }, + { PHY_ID_MATCH_MODEL(PHY_ID_ADIN2111) }, { } }; diff --git a/drivers/net/phy/aquantia_main.c b/drivers/net/phy/aquantia_main.c index 7111e2e958e9..47a76df36b74 100644 --- a/drivers/net/phy/aquantia_main.c +++ b/drivers/net/phy/aquantia_main.c @@ -27,9 +27,12 @@ #define MDIO_PHYXS_VEND_IF_STATUS 0xe812 #define MDIO_PHYXS_VEND_IF_STATUS_TYPE_MASK GENMASK(7, 3) #define MDIO_PHYXS_VEND_IF_STATUS_TYPE_KR 0 +#define MDIO_PHYXS_VEND_IF_STATUS_TYPE_KX 1 #define MDIO_PHYXS_VEND_IF_STATUS_TYPE_XFI 2 #define MDIO_PHYXS_VEND_IF_STATUS_TYPE_USXGMII 3 +#define MDIO_PHYXS_VEND_IF_STATUS_TYPE_XAUI 4 #define MDIO_PHYXS_VEND_IF_STATUS_TYPE_SGMII 6 +#define MDIO_PHYXS_VEND_IF_STATUS_TYPE_RXAUI 7 #define MDIO_PHYXS_VEND_IF_STATUS_TYPE_OCSGMII 10 #define MDIO_AN_VEND_PROV 0xc400 @@ -94,6 +97,19 @@ #define VEND1_GLOBAL_GEN_STAT2 0xc831 #define VEND1_GLOBAL_GEN_STAT2_OP_IN_PROG BIT(15) +/* The following registers all have similar layouts; first the registers... */ +#define VEND1_GLOBAL_CFG_10M 0x0310 +#define VEND1_GLOBAL_CFG_100M 0x031b +#define VEND1_GLOBAL_CFG_1G 0x031c +#define VEND1_GLOBAL_CFG_2_5G 0x031d +#define VEND1_GLOBAL_CFG_5G 0x031e +#define VEND1_GLOBAL_CFG_10G 0x031f +/* ...and now the fields */ +#define VEND1_GLOBAL_CFG_RATE_ADAPT GENMASK(8, 7) +#define VEND1_GLOBAL_CFG_RATE_ADAPT_NONE 0 +#define VEND1_GLOBAL_CFG_RATE_ADAPT_USX 1 +#define VEND1_GLOBAL_CFG_RATE_ADAPT_PAUSE 2 + #define VEND1_GLOBAL_RSVD_STAT1 0xc885 #define VEND1_GLOBAL_RSVD_STAT1_FW_BUILD_ID GENMASK(7, 4) #define VEND1_GLOBAL_RSVD_STAT1_PROV_ID GENMASK(3, 0) @@ -344,40 +360,57 @@ static int aqr_read_status(struct phy_device *phydev) static int aqr107_read_rate(struct phy_device *phydev) { + u32 config_reg; int val; val = phy_read_mmd(phydev, MDIO_MMD_AN, MDIO_AN_TX_VEND_STATUS1); if (val < 0) return val; + if (val & MDIO_AN_TX_VEND_STATUS1_FULL_DUPLEX) + phydev->duplex = DUPLEX_FULL; + else + phydev->duplex = DUPLEX_HALF; + switch (FIELD_GET(MDIO_AN_TX_VEND_STATUS1_RATE_MASK, val)) { case MDIO_AN_TX_VEND_STATUS1_10BASET: phydev->speed = SPEED_10; + config_reg = VEND1_GLOBAL_CFG_10M; break; case MDIO_AN_TX_VEND_STATUS1_100BASETX: phydev->speed = SPEED_100; + config_reg = VEND1_GLOBAL_CFG_100M; break; case MDIO_AN_TX_VEND_STATUS1_1000BASET: phydev->speed = SPEED_1000; + config_reg = VEND1_GLOBAL_CFG_1G; break; case MDIO_AN_TX_VEND_STATUS1_2500BASET: phydev->speed = SPEED_2500; + config_reg = VEND1_GLOBAL_CFG_2_5G; break; case MDIO_AN_TX_VEND_STATUS1_5000BASET: phydev->speed = SPEED_5000; + config_reg = VEND1_GLOBAL_CFG_5G; break; case MDIO_AN_TX_VEND_STATUS1_10GBASET: phydev->speed = SPEED_10000; + config_reg = VEND1_GLOBAL_CFG_10G; break; default: phydev->speed = SPEED_UNKNOWN; - break; + return 0; } - if (val & MDIO_AN_TX_VEND_STATUS1_FULL_DUPLEX) - phydev->duplex = DUPLEX_FULL; + val = phy_read_mmd(phydev, MDIO_MMD_VEND1, config_reg); + if (val < 0) + return val; + + if (FIELD_GET(VEND1_GLOBAL_CFG_RATE_ADAPT, val) == + VEND1_GLOBAL_CFG_RATE_ADAPT_PAUSE) + phydev->rate_matching = RATE_MATCH_PAUSE; else - phydev->duplex = DUPLEX_HALF; + phydev->rate_matching = RATE_MATCH_NONE; return 0; } @@ -401,15 +434,24 @@ static int aqr107_read_status(struct phy_device *phydev) case MDIO_PHYXS_VEND_IF_STATUS_TYPE_KR: phydev->interface = PHY_INTERFACE_MODE_10GKR; break; + case MDIO_PHYXS_VEND_IF_STATUS_TYPE_KX: + phydev->interface = PHY_INTERFACE_MODE_1000BASEKX; + break; case MDIO_PHYXS_VEND_IF_STATUS_TYPE_XFI: phydev->interface = PHY_INTERFACE_MODE_10GBASER; break; case MDIO_PHYXS_VEND_IF_STATUS_TYPE_USXGMII: phydev->interface = PHY_INTERFACE_MODE_USXGMII; break; + case MDIO_PHYXS_VEND_IF_STATUS_TYPE_XAUI: + phydev->interface = PHY_INTERFACE_MODE_XAUI; + break; case MDIO_PHYXS_VEND_IF_STATUS_TYPE_SGMII: phydev->interface = PHY_INTERFACE_MODE_SGMII; break; + case MDIO_PHYXS_VEND_IF_STATUS_TYPE_RXAUI: + phydev->interface = PHY_INTERFACE_MODE_RXAUI; + break; case MDIO_PHYXS_VEND_IF_STATUS_TYPE_OCSGMII: phydev->interface = PHY_INTERFACE_MODE_2500BASEX; break; @@ -522,11 +564,14 @@ static int aqr107_config_init(struct phy_device *phydev) /* Check that the PHY interface type is compatible */ if (phydev->interface != PHY_INTERFACE_MODE_SGMII && + phydev->interface != PHY_INTERFACE_MODE_1000BASEKX && phydev->interface != PHY_INTERFACE_MODE_2500BASEX && phydev->interface != PHY_INTERFACE_MODE_XGMII && phydev->interface != PHY_INTERFACE_MODE_USXGMII && phydev->interface != PHY_INTERFACE_MODE_10GKR && - phydev->interface != PHY_INTERFACE_MODE_10GBASER) + phydev->interface != PHY_INTERFACE_MODE_10GBASER && + phydev->interface != PHY_INTERFACE_MODE_XAUI && + phydev->interface != PHY_INTERFACE_MODE_RXAUI) return -ENODEV; WARN(phydev->interface == PHY_INTERFACE_MODE_XGMII, @@ -630,6 +675,16 @@ static int aqr107_wait_processor_intensive_op(struct phy_device *phydev) return 0; } +static int aqr107_get_rate_matching(struct phy_device *phydev, + phy_interface_t iface) +{ + if (iface == PHY_INTERFACE_MODE_10GBASER || + iface == PHY_INTERFACE_MODE_2500BASEX || + iface == PHY_INTERFACE_MODE_NA) + return RATE_MATCH_PAUSE; + return RATE_MATCH_NONE; +} + static int aqr107_suspend(struct phy_device *phydev) { int err; @@ -703,6 +758,7 @@ static struct phy_driver aqr_driver[] = { PHY_ID_MATCH_MODEL(PHY_ID_AQR107), .name = "Aquantia AQR107", .probe = aqr107_probe, + .get_rate_matching = aqr107_get_rate_matching, .config_init = aqr107_config_init, .config_aneg = aqr_config_aneg, .config_intr = aqr_config_intr, @@ -721,6 +777,7 @@ static struct phy_driver aqr_driver[] = { PHY_ID_MATCH_MODEL(PHY_ID_AQCS109), .name = "Aquantia AQCS109", .probe = aqr107_probe, + .get_rate_matching = aqr107_get_rate_matching, .config_init = aqcs109_config_init, .config_aneg = aqr_config_aneg, .config_intr = aqr_config_intr, @@ -747,6 +804,7 @@ static struct phy_driver aqr_driver[] = { PHY_ID_MATCH_MODEL(PHY_ID_AQR113C), .name = "Aquantia AQR113C", .probe = aqr107_probe, + .get_rate_matching = aqr107_get_rate_matching, .config_init = aqr107_config_init, .config_aneg = aqr_config_aneg, .config_intr = aqr_config_intr, diff --git a/drivers/net/phy/at803x.c b/drivers/net/phy/at803x.c index 59fe356942b5..9e9adde335c8 100644 --- a/drivers/net/phy/at803x.c +++ b/drivers/net/phy/at803x.c @@ -115,6 +115,7 @@ #define AT803X_DEBUG_REG_HIB_CTRL 0x0b #define AT803X_DEBUG_HIB_CTRL_SEL_RST_80U BIT(10) #define AT803X_DEBUG_HIB_CTRL_EN_ANY_CHANGE BIT(13) +#define AT803X_DEBUG_HIB_CTRL_PS_HIB_EN BIT(15) #define AT803X_DEBUG_REG_3C 0x3C @@ -192,6 +193,9 @@ #define AT803X_KEEP_PLL_ENABLED BIT(0) #define AT803X_DISABLE_SMARTEEE BIT(1) +/* disable hibernation mode */ +#define AT803X_DISABLE_HIBERNATION_MODE BIT(2) + /* ADC threshold */ #define QCA808X_PHY_DEBUG_ADC_THRESHOLD 0x2c80 #define QCA808X_ADC_THRESHOLD_MASK GENMASK(7, 0) @@ -672,6 +676,7 @@ static int at803x_sfp_insert(void *upstream, const struct sfp_eeprom_id *id) struct phy_device *phydev = upstream; __ETHTOOL_DECLARE_LINK_MODE_MASK(phy_support); __ETHTOOL_DECLARE_LINK_MODE_MASK(sfp_support); + DECLARE_PHY_INTERFACE_MASK(interfaces); phy_interface_t iface; linkmode_zero(phy_support); @@ -682,7 +687,7 @@ static int at803x_sfp_insert(void *upstream, const struct sfp_eeprom_id *id) phylink_set(phy_support, Asym_Pause); linkmode_zero(sfp_support); - sfp_parse_support(phydev->sfp_bus, id, sfp_support); + sfp_parse_support(phydev->sfp_bus, id, sfp_support, interfaces); /* Some modules support 10G modes as well as others we support. * Mask out non-supported modes so the correct interface is picked. */ @@ -730,6 +735,9 @@ static int at803x_parse_dt(struct phy_device *phydev) if (of_property_read_bool(node, "qca,disable-smarteee")) priv->flags |= AT803X_DISABLE_SMARTEEE; + if (of_property_read_bool(node, "qca,disable-hibernation-mode")) + priv->flags |= AT803X_DISABLE_HIBERNATION_MODE; + if (!of_property_read_u32(node, "qca,smarteee-tw-us-1g", &tw)) { if (!tw || tw > 255) { phydev_err(phydev, "invalid qca,smarteee-tw-us-1g\n"); @@ -999,6 +1007,20 @@ static int at8031_pll_config(struct phy_device *phydev) AT803X_DEBUG_PLL_ON, 0); } +static int at803x_hibernation_mode_config(struct phy_device *phydev) +{ + struct at803x_priv *priv = phydev->priv; + + /* The default after hardware reset is hibernation mode enabled. After + * software reset, the value is retained. + */ + if (!(priv->flags & AT803X_DISABLE_HIBERNATION_MODE)) + return 0; + + return at803x_debug_reg_mask(phydev, AT803X_DEBUG_REG_HIB_CTRL, + AT803X_DEBUG_HIB_CTRL_PS_HIB_EN, 0); +} + static int at803x_config_init(struct phy_device *phydev) { struct at803x_priv *priv = phydev->priv; @@ -1051,6 +1073,10 @@ static int at803x_config_init(struct phy_device *phydev) if (ret < 0) return ret; + ret = at803x_hibernation_mode_config(phydev); + if (ret < 0) + return ret; + /* Ar803x extended next page bit is enabled by default. Cisco * multigig switches read this bit and attempt to negotiate 10Gbps * rates even if the next page bit is disabled. This is incorrect diff --git a/drivers/net/phy/bcm-phy-lib.c b/drivers/net/phy/bcm-phy-lib.c index 287cccf8f7f4..b2c0baa51f39 100644 --- a/drivers/net/phy/bcm-phy-lib.c +++ b/drivers/net/phy/bcm-phy-lib.c @@ -519,7 +519,7 @@ void bcm_phy_get_strings(struct phy_device *phydev, u8 *data) unsigned int i; for (i = 0; i < ARRAY_SIZE(bcm_phy_hw_stats); i++) - strlcpy(data + i * ETH_GSTRING_LEN, + strscpy(data + i * ETH_GSTRING_LEN, bcm_phy_hw_stats[i].string, ETH_GSTRING_LEN); } EXPORT_SYMBOL_GPL(bcm_phy_get_strings); diff --git a/drivers/net/phy/broadcom.c b/drivers/net/phy/broadcom.c index 31fbcdddc9ad..ad71c88c87e7 100644 --- a/drivers/net/phy/broadcom.c +++ b/drivers/net/phy/broadcom.c @@ -766,6 +766,41 @@ static irqreturn_t brcm_fet_handle_interrupt(struct phy_device *phydev) return IRQ_HANDLED; } +static int brcm_fet_suspend(struct phy_device *phydev) +{ + int reg, err, err2, brcmtest; + + /* We cannot use a read/modify/write here otherwise the PHY continues + * to drive LEDs which defeats the purpose of low power mode. + */ + err = phy_write(phydev, MII_BMCR, BMCR_PDOWN); + if (err < 0) + return err; + + /* Enable shadow register access */ + brcmtest = phy_read(phydev, MII_BRCM_FET_BRCMTEST); + if (brcmtest < 0) + return brcmtest; + + reg = brcmtest | MII_BRCM_FET_BT_SRE; + + err = phy_write(phydev, MII_BRCM_FET_BRCMTEST, reg); + if (err < 0) + return err; + + /* Set standby mode */ + err = phy_modify(phydev, MII_BRCM_FET_SHDW_AUXMODE4, + MII_BRCM_FET_SHDW_AM4_STANDBY, + MII_BRCM_FET_SHDW_AM4_STANDBY); + + /* Disable shadow register access */ + err2 = phy_write(phydev, MII_BRCM_FET_BRCMTEST, brcmtest); + if (!err) + err = err2; + + return err; +} + static int bcm54xx_phy_probe(struct phy_device *phydev) { struct bcm54xx_phy_priv *priv; @@ -1033,6 +1068,8 @@ static struct phy_driver broadcom_drivers[] = { .config_init = brcm_fet_config_init, .config_intr = brcm_fet_config_intr, .handle_interrupt = brcm_fet_handle_interrupt, + .suspend = brcm_fet_suspend, + .resume = brcm_fet_config_init, }, { .phy_id = PHY_ID_BCM5241, .phy_id_mask = 0xfffffff0, @@ -1041,6 +1078,8 @@ static struct phy_driver broadcom_drivers[] = { .config_init = brcm_fet_config_init, .config_intr = brcm_fet_config_intr, .handle_interrupt = brcm_fet_handle_interrupt, + .suspend = brcm_fet_suspend, + .resume = brcm_fet_config_init, }, { .phy_id = PHY_ID_BCM5395, .phy_id_mask = 0xfffffff0, diff --git a/drivers/net/phy/marvell-88x2222.c b/drivers/net/phy/marvell-88x2222.c index f070776ca904..fd9ad4820192 100644 --- a/drivers/net/phy/marvell-88x2222.c +++ b/drivers/net/phy/marvell-88x2222.c @@ -478,6 +478,7 @@ static int mv2222_config_init(struct phy_device *phydev) static int mv2222_sfp_insert(void *upstream, const struct sfp_eeprom_id *id) { + DECLARE_PHY_INTERFACE_MASK(interfaces); struct phy_device *phydev = upstream; phy_interface_t sfp_interface; struct mv2222_data *priv; @@ -489,7 +490,7 @@ static int mv2222_sfp_insert(void *upstream, const struct sfp_eeprom_id *id) priv = (struct mv2222_data *)phydev->priv; dev = &phydev->mdio.dev; - sfp_parse_support(phydev->sfp_bus, id, sfp_supported); + sfp_parse_support(phydev->sfp_bus, id, sfp_supported, interfaces); phydev->port = sfp_parse_port(phydev->sfp_bus, id, sfp_supported); sfp_interface = sfp_select_interface(phydev->sfp_bus, sfp_supported); diff --git a/drivers/net/phy/marvell.c b/drivers/net/phy/marvell.c index a714150f5e8c..2810f4f9da0c 100644 --- a/drivers/net/phy/marvell.c +++ b/drivers/net/phy/marvell.c @@ -1952,7 +1952,7 @@ static void marvell_get_strings(struct phy_device *phydev, u8 *data) int i; for (i = 0; i < count; i++) { - strlcpy(data + i * ETH_GSTRING_LEN, + strscpy(data + i * ETH_GSTRING_LEN, marvell_hw_stats[i].string, ETH_GSTRING_LEN); } } @@ -2845,6 +2845,7 @@ static int marvell_probe(struct phy_device *phydev) static int m88e1510_sfp_insert(void *upstream, const struct sfp_eeprom_id *id) { + DECLARE_PHY_INTERFACE_MASK(interfaces); struct phy_device *phydev = upstream; phy_interface_t interface; struct device *dev; @@ -2856,7 +2857,7 @@ static int m88e1510_sfp_insert(void *upstream, const struct sfp_eeprom_id *id) dev = &phydev->mdio.dev; - sfp_parse_support(phydev->sfp_bus, id, supported); + sfp_parse_support(phydev->sfp_bus, id, supported, interfaces); interface = sfp_select_interface(phydev->sfp_bus, supported); dev_info(dev, "%s SFP module inserted\n", phy_modes(interface)); diff --git a/drivers/net/phy/marvell10g.c b/drivers/net/phy/marvell10g.c index 2b7d0720720b..383a9c9f36e5 100644 --- a/drivers/net/phy/marvell10g.c +++ b/drivers/net/phy/marvell10g.c @@ -96,6 +96,11 @@ enum { MV_PCS_PORT_INFO_NPORTS_MASK = 0x0380, MV_PCS_PORT_INFO_NPORTS_SHIFT = 7, + /* SerDes reinitialization 88E21X0 */ + MV_AN_21X0_SERDES_CTRL2 = 0x800f, + MV_AN_21X0_SERDES_CTRL2_AUTO_INIT_DIS = BIT(13), + MV_AN_21X0_SERDES_CTRL2_RUN_INIT = BIT(15), + /* These registers appear at 0x800X and 0xa00X - the 0xa00X control * registers appear to set themselves to the 0x800X when AN is * restarted, but status registers appear readable from either. @@ -117,16 +122,16 @@ enum { MV_V2_33X0_PORT_CTRL_MACTYPE_10GBASER_NO_SGMII_AN = 0x5, MV_V2_33X0_PORT_CTRL_MACTYPE_10GBASER_RATE_MATCH = 0x6, MV_V2_33X0_PORT_CTRL_MACTYPE_USXGMII = 0x7, - MV_V2_PORT_INTR_STS = 0xf040, - MV_V2_PORT_INTR_MASK = 0xf043, - MV_V2_PORT_INTR_STS_WOL_EN = BIT(8), - MV_V2_MAGIC_PKT_WORD0 = 0xf06b, - MV_V2_MAGIC_PKT_WORD1 = 0xf06c, - MV_V2_MAGIC_PKT_WORD2 = 0xf06d, + MV_V2_PORT_INTR_STS = 0xf040, + MV_V2_PORT_INTR_MASK = 0xf043, + MV_V2_PORT_INTR_STS_WOL_EN = BIT(8), + MV_V2_MAGIC_PKT_WORD0 = 0xf06b, + MV_V2_MAGIC_PKT_WORD1 = 0xf06c, + MV_V2_MAGIC_PKT_WORD2 = 0xf06d, /* Wake on LAN registers */ - MV_V2_WOL_CTRL = 0xf06e, - MV_V2_WOL_CTRL_CLEAR_STS = BIT(15), - MV_V2_WOL_CTRL_MAGIC_PKT_EN = BIT(0), + MV_V2_WOL_CTRL = 0xf06e, + MV_V2_WOL_CTRL_CLEAR_STS = BIT(15), + MV_V2_WOL_CTRL_MAGIC_PKT_EN = BIT(0), /* Temperature control/read registers (88X3310 only) */ MV_V2_TEMP_CTRL = 0xf08a, MV_V2_TEMP_CTRL_MASK = 0xc000, @@ -140,6 +145,8 @@ struct mv3310_chip { bool (*has_downshift)(struct phy_device *phydev); void (*init_supported_interfaces)(unsigned long *mask); int (*get_mactype)(struct phy_device *phydev); + int (*set_mactype)(struct phy_device *phydev, int mactype); + int (*select_mactype)(unsigned long *interfaces); int (*init_interface)(struct phy_device *phydev, int mactype); #ifdef CONFIG_HWMON @@ -466,9 +473,10 @@ static int mv3310_sfp_insert(void *upstream, const struct sfp_eeprom_id *id) { struct phy_device *phydev = upstream; __ETHTOOL_DECLARE_LINK_MODE_MASK(support) = { 0, }; + DECLARE_PHY_INTERFACE_MASK(interfaces); phy_interface_t iface; - sfp_parse_support(phydev->sfp_bus, id, support); + sfp_parse_support(phydev->sfp_bus, id, support, interfaces); iface = sfp_select_interface(phydev->sfp_bus, support); if (iface != PHY_INTERFACE_MODE_10GBASER) { @@ -593,6 +601,49 @@ static int mv2110_get_mactype(struct phy_device *phydev) return mactype & MV_PMA_21X0_PORT_CTRL_MACTYPE_MASK; } +static int mv2110_set_mactype(struct phy_device *phydev, int mactype) +{ + int err, val; + + mactype &= MV_PMA_21X0_PORT_CTRL_MACTYPE_MASK; + err = phy_modify_mmd(phydev, MDIO_MMD_PMAPMD, MV_PMA_21X0_PORT_CTRL, + MV_PMA_21X0_PORT_CTRL_SWRST | + MV_PMA_21X0_PORT_CTRL_MACTYPE_MASK, + MV_PMA_21X0_PORT_CTRL_SWRST | mactype); + if (err) + return err; + + err = phy_set_bits_mmd(phydev, MDIO_MMD_AN, MV_AN_21X0_SERDES_CTRL2, + MV_AN_21X0_SERDES_CTRL2_AUTO_INIT_DIS | + MV_AN_21X0_SERDES_CTRL2_RUN_INIT); + if (err) + return err; + + err = phy_read_mmd_poll_timeout(phydev, MDIO_MMD_AN, + MV_AN_21X0_SERDES_CTRL2, val, + !(val & + MV_AN_21X0_SERDES_CTRL2_RUN_INIT), + 5000, 100000, true); + if (err) + return err; + + return phy_clear_bits_mmd(phydev, MDIO_MMD_AN, MV_AN_21X0_SERDES_CTRL2, + MV_AN_21X0_SERDES_CTRL2_AUTO_INIT_DIS); +} + +static int mv2110_select_mactype(unsigned long *interfaces) +{ + if (test_bit(PHY_INTERFACE_MODE_USXGMII, interfaces)) + return MV_PMA_21X0_PORT_CTRL_MACTYPE_USXGMII; + else if (test_bit(PHY_INTERFACE_MODE_SGMII, interfaces) && + !test_bit(PHY_INTERFACE_MODE_10GBASER, interfaces)) + return MV_PMA_21X0_PORT_CTRL_MACTYPE_5GBASER; + else if (test_bit(PHY_INTERFACE_MODE_10GBASER, interfaces)) + return MV_PMA_21X0_PORT_CTRL_MACTYPE_10GBASER_RATE_MATCH; + else + return -1; +} + static int mv3310_get_mactype(struct phy_device *phydev) { int mactype; @@ -604,6 +655,46 @@ static int mv3310_get_mactype(struct phy_device *phydev) return mactype & MV_V2_33X0_PORT_CTRL_MACTYPE_MASK; } +static int mv3310_set_mactype(struct phy_device *phydev, int mactype) +{ + int ret; + + mactype &= MV_V2_33X0_PORT_CTRL_MACTYPE_MASK; + ret = phy_modify_mmd_changed(phydev, MDIO_MMD_VEND2, MV_V2_PORT_CTRL, + MV_V2_33X0_PORT_CTRL_MACTYPE_MASK, + mactype); + if (ret <= 0) + return ret; + + return phy_set_bits_mmd(phydev, MDIO_MMD_VEND2, MV_V2_PORT_CTRL, + MV_V2_33X0_PORT_CTRL_SWRST); +} + +static int mv3310_select_mactype(unsigned long *interfaces) +{ + if (test_bit(PHY_INTERFACE_MODE_USXGMII, interfaces)) + return MV_V2_33X0_PORT_CTRL_MACTYPE_USXGMII; + else if (test_bit(PHY_INTERFACE_MODE_SGMII, interfaces) && + test_bit(PHY_INTERFACE_MODE_10GBASER, interfaces)) + return MV_V2_33X0_PORT_CTRL_MACTYPE_10GBASER; + else if (test_bit(PHY_INTERFACE_MODE_SGMII, interfaces) && + test_bit(PHY_INTERFACE_MODE_RXAUI, interfaces)) + return MV_V2_33X0_PORT_CTRL_MACTYPE_RXAUI; + else if (test_bit(PHY_INTERFACE_MODE_SGMII, interfaces) && + test_bit(PHY_INTERFACE_MODE_XAUI, interfaces)) + return MV_V2_3310_PORT_CTRL_MACTYPE_XAUI; + else if (test_bit(PHY_INTERFACE_MODE_10GBASER, interfaces)) + return MV_V2_33X0_PORT_CTRL_MACTYPE_10GBASER_RATE_MATCH; + else if (test_bit(PHY_INTERFACE_MODE_RXAUI, interfaces)) + return MV_V2_33X0_PORT_CTRL_MACTYPE_RXAUI_RATE_MATCH; + else if (test_bit(PHY_INTERFACE_MODE_XAUI, interfaces)) + return MV_V2_3310_PORT_CTRL_MACTYPE_XAUI_RATE_MATCH; + else if (test_bit(PHY_INTERFACE_MODE_SGMII, interfaces)) + return MV_V2_33X0_PORT_CTRL_MACTYPE_10GBASER; + else + return -1; +} + static int mv2110_init_interface(struct phy_device *phydev, int mactype) { struct mv3310_priv *priv = dev_get_drvdata(&phydev->mdio.dev); @@ -687,6 +778,20 @@ static int mv3310_config_init(struct phy_device *phydev) if (err) return err; + /* If host provided host supported interface modes, try to select the + * best one + */ + if (!phy_interface_empty(phydev->host_interfaces)) { + mactype = chip->select_mactype(phydev->host_interfaces); + if (mactype >= 0) { + phydev_info(phydev, "Changing MACTYPE to %i\n", + mactype); + err = chip->set_mactype(phydev, mactype); + if (err) + return err; + } + } + mactype = chip->get_mactype(phydev); if (mactype < 0) return mactype; @@ -1049,6 +1154,8 @@ static const struct mv3310_chip mv3310_type = { .has_downshift = mv3310_has_downshift, .init_supported_interfaces = mv3310_init_supported_interfaces, .get_mactype = mv3310_get_mactype, + .set_mactype = mv3310_set_mactype, + .select_mactype = mv3310_select_mactype, .init_interface = mv3310_init_interface, #ifdef CONFIG_HWMON @@ -1060,6 +1167,8 @@ static const struct mv3310_chip mv3340_type = { .has_downshift = mv3310_has_downshift, .init_supported_interfaces = mv3340_init_supported_interfaces, .get_mactype = mv3310_get_mactype, + .set_mactype = mv3310_set_mactype, + .select_mactype = mv3310_select_mactype, .init_interface = mv3340_init_interface, #ifdef CONFIG_HWMON @@ -1070,6 +1179,8 @@ static const struct mv3310_chip mv3340_type = { static const struct mv3310_chip mv2110_type = { .init_supported_interfaces = mv2110_init_supported_interfaces, .get_mactype = mv2110_get_mactype, + .set_mactype = mv2110_set_mactype, + .select_mactype = mv2110_select_mactype, .init_interface = mv2110_init_interface, #ifdef CONFIG_HWMON @@ -1080,6 +1191,8 @@ static const struct mv3310_chip mv2110_type = { static const struct mv3310_chip mv2111_type = { .init_supported_interfaces = mv2111_init_supported_interfaces, .get_mactype = mv2110_get_mactype, + .set_mactype = mv2110_set_mactype, + .select_mactype = mv2110_select_mactype, .init_interface = mv2110_init_interface, #ifdef CONFIG_HWMON diff --git a/drivers/net/phy/mdio_bus.c b/drivers/net/phy/mdio_bus.c index 8a2dbe849866..f82090bdf7ab 100644 --- a/drivers/net/phy/mdio_bus.c +++ b/drivers/net/phy/mdio_bus.c @@ -232,7 +232,7 @@ static ssize_t mdio_bus_stat_field_show(struct device *dev, val = mdio_bus_get_stat(&bus->stats[sattr->addr], sattr->field_offset); - return sprintf(buf, "%llu\n", val); + return sysfs_emit(buf, "%llu\n", val); } static ssize_t mdio_bus_device_stat_field_show(struct device *dev, @@ -251,7 +251,7 @@ static ssize_t mdio_bus_device_stat_field_show(struct device *dev, val = mdio_bus_get_stat(&bus->stats[addr], sattr->field_offset); - return sprintf(buf, "%llu\n", val); + return sysfs_emit(buf, "%llu\n", val); } #define MDIO_BUS_STATS_ATTR_DECL(field, file) \ diff --git a/drivers/net/phy/micrel.c b/drivers/net/phy/micrel.c index 38234d7e14c5..3757e069c486 100644 --- a/drivers/net/phy/micrel.c +++ b/drivers/net/phy/micrel.c @@ -92,6 +92,15 @@ #define KSZ9x31_LMD_VCT_DATA_HI_PULSE_MASK GENMASK(1, 0) #define KSZ9x31_LMD_VCT_DATA_MASK GENMASK(7, 0) +#define KSZPHY_WIRE_PAIR_MASK 0x3 + +#define LAN8814_CABLE_DIAG 0x12 +#define LAN8814_CABLE_DIAG_STAT_MASK GENMASK(9, 8) +#define LAN8814_CABLE_DIAG_VCT_DATA_MASK GENMASK(7, 0) +#define LAN8814_PAIR_BIT_SHIFT 12 + +#define LAN8814_WIRE_PAIR_MASK 0xF + /* Lan8814 general Interrupt control/status reg in GPHY specific block. */ #define LAN8814_INTC 0x18 #define LAN8814_INTS 0x1B @@ -257,6 +266,8 @@ static struct kszphy_hw_stat kszphy_hw_stats[] = { struct kszphy_type { u32 led_mode_reg; u16 interrupt_level_mask; + u16 cable_diag_reg; + unsigned long pair_mask; bool has_broadcast_disable; bool has_nand_tree_disable; bool has_rmii_ref_clk_sel; @@ -313,6 +324,13 @@ struct kszphy_priv { static const struct kszphy_type lan8814_type = { .led_mode_reg = ~LAN8814_LED_CTRL_1, + .cable_diag_reg = LAN8814_CABLE_DIAG, + .pair_mask = LAN8814_WIRE_PAIR_MASK, +}; + +static const struct kszphy_type ksz886x_type = { + .cable_diag_reg = KSZ8081_LMD, + .pair_mask = KSZPHY_WIRE_PAIR_MASK, }; static const struct kszphy_type ksz8021_type = { @@ -1650,7 +1668,7 @@ static void kszphy_get_strings(struct phy_device *phydev, u8 *data) int i; for (i = 0; i < ARRAY_SIZE(kszphy_hw_stats); i++) { - strlcpy(data + i * ETH_GSTRING_LEN, + strscpy(data + i * ETH_GSTRING_LEN, kszphy_hw_stats[i].string, ETH_GSTRING_LEN); } } @@ -1796,6 +1814,17 @@ static int kszphy_probe(struct phy_device *phydev) return 0; } +static int lan8814_cable_test_start(struct phy_device *phydev) +{ + /* If autoneg is enabled, we won't be able to test cross pair + * short. In this case, the PHY will "detect" a link and + * confuse the internal state machine - disable auto neg here. + * Set the speed to 1000mbit and full duplex. + */ + return phy_modify(phydev, MII_BMCR, BMCR_ANENABLE | BMCR_SPEED100, + BMCR_SPEED1000 | BMCR_FULLDPLX); +} + static int ksz886x_cable_test_start(struct phy_device *phydev) { if (phydev->dev_flags & MICREL_KSZ8_P1_ERRATA) @@ -1809,9 +1838,9 @@ static int ksz886x_cable_test_start(struct phy_device *phydev) return phy_clear_bits(phydev, MII_BMCR, BMCR_ANENABLE | BMCR_SPEED100); } -static int ksz886x_cable_test_result_trans(u16 status) +static int ksz886x_cable_test_result_trans(u16 status, u16 mask) { - switch (FIELD_GET(KSZ8081_LMD_STAT_MASK, status)) { + switch (FIELD_GET(mask, status)) { case KSZ8081_LMD_STAT_NORMAL: return ETHTOOL_A_CABLE_RESULT_CODE_OK; case KSZ8081_LMD_STAT_SHORT: @@ -1825,15 +1854,15 @@ static int ksz886x_cable_test_result_trans(u16 status) } } -static bool ksz886x_cable_test_failed(u16 status) +static bool ksz886x_cable_test_failed(u16 status, u16 mask) { - return FIELD_GET(KSZ8081_LMD_STAT_MASK, status) == + return FIELD_GET(mask, status) == KSZ8081_LMD_STAT_FAIL; } -static bool ksz886x_cable_test_fault_length_valid(u16 status) +static bool ksz886x_cable_test_fault_length_valid(u16 status, u16 mask) { - switch (FIELD_GET(KSZ8081_LMD_STAT_MASK, status)) { + switch (FIELD_GET(mask, status)) { case KSZ8081_LMD_STAT_OPEN: fallthrough; case KSZ8081_LMD_STAT_SHORT: @@ -1842,29 +1871,79 @@ static bool ksz886x_cable_test_fault_length_valid(u16 status) return false; } -static int ksz886x_cable_test_fault_length(u16 status) +static int ksz886x_cable_test_fault_length(struct phy_device *phydev, u16 status, u16 data_mask) { int dt; /* According to the data sheet the distance to the fault is - * DELTA_TIME * 0.4 meters. + * DELTA_TIME * 0.4 meters for ksz phys. + * (DELTA_TIME - 22) * 0.8 for lan8814 phy. */ - dt = FIELD_GET(KSZ8081_LMD_DELTA_TIME_MASK, status); + dt = FIELD_GET(data_mask, status); - return (dt * 400) / 10; + if ((phydev->phy_id & MICREL_PHY_ID_MASK) == PHY_ID_LAN8814) + return ((dt - 22) * 800) / 10; + else + return (dt * 400) / 10; } static int ksz886x_cable_test_wait_for_completion(struct phy_device *phydev) { + const struct kszphy_type *type = phydev->drv->driver_data; int val, ret; - ret = phy_read_poll_timeout(phydev, KSZ8081_LMD, val, + ret = phy_read_poll_timeout(phydev, type->cable_diag_reg, val, !(val & KSZ8081_LMD_ENABLE_TEST), 30000, 100000, true); return ret < 0 ? ret : 0; } +static int lan8814_cable_test_one_pair(struct phy_device *phydev, int pair) +{ + static const int ethtool_pair[] = { ETHTOOL_A_CABLE_PAIR_A, + ETHTOOL_A_CABLE_PAIR_B, + ETHTOOL_A_CABLE_PAIR_C, + ETHTOOL_A_CABLE_PAIR_D, + }; + u32 fault_length; + int ret; + int val; + + val = KSZ8081_LMD_ENABLE_TEST; + val = val | (pair << LAN8814_PAIR_BIT_SHIFT); + + ret = phy_write(phydev, LAN8814_CABLE_DIAG, val); + if (ret < 0) + return ret; + + ret = ksz886x_cable_test_wait_for_completion(phydev); + if (ret) + return ret; + + val = phy_read(phydev, LAN8814_CABLE_DIAG); + if (val < 0) + return val; + + if (ksz886x_cable_test_failed(val, LAN8814_CABLE_DIAG_STAT_MASK)) + return -EAGAIN; + + ret = ethnl_cable_test_result(phydev, ethtool_pair[pair], + ksz886x_cable_test_result_trans(val, + LAN8814_CABLE_DIAG_STAT_MASK + )); + if (ret) + return ret; + + if (!ksz886x_cable_test_fault_length_valid(val, LAN8814_CABLE_DIAG_STAT_MASK)) + return 0; + + fault_length = ksz886x_cable_test_fault_length(phydev, val, + LAN8814_CABLE_DIAG_VCT_DATA_MASK); + + return ethnl_cable_test_fault_length(phydev, ethtool_pair[pair], fault_length); +} + static int ksz886x_cable_test_one_pair(struct phy_device *phydev, int pair) { static const int ethtool_pair[] = { @@ -1872,6 +1951,7 @@ static int ksz886x_cable_test_one_pair(struct phy_device *phydev, int pair) ETHTOOL_A_CABLE_PAIR_B, }; int ret, val, mdix; + u32 fault_length; /* There is no way to choice the pair, like we do one ksz9031. * We can workaround this limitation by using the MDI-X functionality. @@ -1910,25 +1990,27 @@ static int ksz886x_cable_test_one_pair(struct phy_device *phydev, int pair) if (val < 0) return val; - if (ksz886x_cable_test_failed(val)) + if (ksz886x_cable_test_failed(val, KSZ8081_LMD_STAT_MASK)) return -EAGAIN; ret = ethnl_cable_test_result(phydev, ethtool_pair[pair], - ksz886x_cable_test_result_trans(val)); + ksz886x_cable_test_result_trans(val, KSZ8081_LMD_STAT_MASK)); if (ret) return ret; - if (!ksz886x_cable_test_fault_length_valid(val)) + if (!ksz886x_cable_test_fault_length_valid(val, KSZ8081_LMD_STAT_MASK)) return 0; - return ethnl_cable_test_fault_length(phydev, ethtool_pair[pair], - ksz886x_cable_test_fault_length(val)); + fault_length = ksz886x_cable_test_fault_length(phydev, val, KSZ8081_LMD_DELTA_TIME_MASK); + + return ethnl_cable_test_fault_length(phydev, ethtool_pair[pair], fault_length); } static int ksz886x_cable_test_get_status(struct phy_device *phydev, bool *finished) { - unsigned long pair_mask = 0x3; + const struct kszphy_type *type = phydev->drv->driver_data; + unsigned long pair_mask = type->pair_mask; int retries = 20; int pair, ret; @@ -1937,7 +2019,10 @@ static int ksz886x_cable_test_get_status(struct phy_device *phydev, /* Try harder if link partner is active */ while (pair_mask && retries--) { for_each_set_bit(pair, &pair_mask, 4) { - ret = ksz886x_cable_test_one_pair(phydev, pair); + if (type->cable_diag_reg == LAN8814_CABLE_DIAG) + ret = lan8814_cable_test_one_pair(phydev, pair); + else + ret = ksz886x_cable_test_one_pair(phydev, pair); if (ret == -EAGAIN) continue; if (ret < 0) @@ -2676,6 +2761,66 @@ static int lan8804_config_init(struct phy_device *phydev) return 0; } +static irqreturn_t lan8804_handle_interrupt(struct phy_device *phydev) +{ + int status; + + status = phy_read(phydev, LAN8814_INTS); + if (status < 0) { + phy_error(phydev); + return IRQ_NONE; + } + + if (status > 0) + phy_trigger_machine(phydev); + + return IRQ_HANDLED; +} + +#define LAN8804_OUTPUT_CONTROL 25 +#define LAN8804_OUTPUT_CONTROL_INTR_BUFFER BIT(14) +#define LAN8804_CONTROL 31 +#define LAN8804_CONTROL_INTR_POLARITY BIT(14) + +static int lan8804_config_intr(struct phy_device *phydev) +{ + int err; + + /* This is an internal PHY of lan966x and is not possible to change the + * polarity on the GIC found in lan966x, therefore change the polarity + * of the interrupt in the PHY from being active low instead of active + * high. + */ + phy_write(phydev, LAN8804_CONTROL, LAN8804_CONTROL_INTR_POLARITY); + + /* By default interrupt buffer is open-drain in which case the interrupt + * can be active only low. Therefore change the interrupt buffer to be + * push-pull to be able to change interrupt polarity + */ + phy_write(phydev, LAN8804_OUTPUT_CONTROL, + LAN8804_OUTPUT_CONTROL_INTR_BUFFER); + + if (phydev->interrupts == PHY_INTERRUPT_ENABLED) { + err = phy_read(phydev, LAN8814_INTS); + if (err < 0) + return err; + + err = phy_write(phydev, LAN8814_INTC, LAN8814_INT_LINK); + if (err) + return err; + } else { + err = phy_write(phydev, LAN8814_INTC, 0); + if (err) + return err; + + err = phy_read(phydev, LAN8814_INTS); + if (err < 0) + return err; + } + + return 0; +} + static irqreturn_t lan8814_handle_interrupt(struct phy_device *phydev) { int irq_status, tsu_irq_status; @@ -2735,9 +2880,9 @@ static int lan8814_config_intr(struct phy_device *phydev) if (err) return err; - err = phy_write(phydev, LAN8814_INTC, LAN8814_INT_LINK); + err = phy_write(phydev, LAN8814_INTC, LAN8814_INT_LINK); } else { - err = phy_write(phydev, LAN8814_INTC, 0); + err = phy_write(phydev, LAN8814_INTC, 0); if (err) return err; @@ -3117,6 +3262,7 @@ static struct phy_driver ksphy_driver[] = { .phy_id = PHY_ID_LAN8814, .phy_id_mask = MICREL_PHY_ID_MASK, .name = "Microchip INDY Gigabit Quad PHY", + .flags = PHY_POLL_CABLE_TEST, .config_init = lan8814_config_init, .driver_data = &lan8814_type, .probe = lan8814_probe, @@ -3129,6 +3275,8 @@ static struct phy_driver ksphy_driver[] = { .resume = kszphy_resume, .config_intr = lan8814_config_intr, .handle_interrupt = lan8814_handle_interrupt, + .cable_test_start = lan8814_cable_test_start, + .cable_test_get_status = ksz886x_cable_test_get_status, }, { .phy_id = PHY_ID_LAN8804, .phy_id_mask = MICREL_PHY_ID_MASK, @@ -3143,6 +3291,8 @@ static struct phy_driver ksphy_driver[] = { .get_stats = kszphy_get_stats, .suspend = genphy_suspend, .resume = kszphy_resume, + .config_intr = lan8804_config_intr, + .handle_interrupt = lan8804_handle_interrupt, }, { .phy_id = PHY_ID_KSZ9131, .phy_id_mask = MICREL_PHY_ID_MASK, @@ -3175,6 +3325,7 @@ static struct phy_driver ksphy_driver[] = { .phy_id = PHY_ID_KSZ886X, .phy_id_mask = MICREL_PHY_ID_MASK, .name = "Micrel KSZ8851 Ethernet MAC or KSZ886X Switch", + .driver_data = &ksz886x_type, /* PHY_BASIC_FEATURES */ .flags = PHY_POLL_CABLE_TEST, .config_init = kszphy_config_init, @@ -3197,6 +3348,8 @@ static struct phy_driver ksphy_driver[] = { .name = "Microchip KSZ9477", /* PHY_GBIT_FEATURES */ .config_init = kszphy_config_init, + .config_intr = kszphy_config_intr, + .handle_interrupt = kszphy_handle_interrupt, .suspend = genphy_suspend, .resume = genphy_resume, } }; diff --git a/drivers/net/phy/mscc/mscc_macsec.c b/drivers/net/phy/mscc/mscc_macsec.c index b7b2521c73fb..ee5b17edca39 100644 --- a/drivers/net/phy/mscc/mscc_macsec.c +++ b/drivers/net/phy/mscc/mscc_macsec.c @@ -706,14 +706,6 @@ static int __vsc8584_macsec_add_rxsa(struct macsec_context *ctx, struct phy_device *phydev = ctx->phydev; struct vsc8531_private *priv = phydev->priv; - if (!flow) { - flow = vsc8584_macsec_alloc_flow(priv, MACSEC_INGR); - if (IS_ERR(flow)) - return PTR_ERR(flow); - - memcpy(flow->key, ctx->sa.key, priv->secy->key_len); - } - flow->assoc_num = ctx->sa.assoc_num; flow->rx_sa = ctx->sa.rx_sa; @@ -730,24 +722,13 @@ static int __vsc8584_macsec_add_rxsa(struct macsec_context *ctx, static int __vsc8584_macsec_add_txsa(struct macsec_context *ctx, struct macsec_flow *flow, bool update) { - struct phy_device *phydev = ctx->phydev; - struct vsc8531_private *priv = phydev->priv; - - if (!flow) { - flow = vsc8584_macsec_alloc_flow(priv, MACSEC_EGR); - if (IS_ERR(flow)) - return PTR_ERR(flow); - - memcpy(flow->key, ctx->sa.key, priv->secy->key_len); - } - flow->assoc_num = ctx->sa.assoc_num; flow->tx_sa = ctx->sa.tx_sa; /* Always match untagged packets on egress */ flow->match.untagged = 1; - return vsc8584_macsec_add_flow(phydev, flow, update); + return vsc8584_macsec_add_flow(ctx->phydev, flow, update); } static int vsc8584_macsec_dev_open(struct macsec_context *ctx) @@ -755,10 +736,6 @@ static int vsc8584_macsec_dev_open(struct macsec_context *ctx) struct vsc8531_private *priv = ctx->phydev->priv; struct macsec_flow *flow, *tmp; - /* No operation to perform before the commit step */ - if (ctx->prepare) - return 0; - list_for_each_entry_safe(flow, tmp, &priv->macsec_flows, list) vsc8584_macsec_flow_enable(ctx->phydev, flow); @@ -770,10 +747,6 @@ static int vsc8584_macsec_dev_stop(struct macsec_context *ctx) struct vsc8531_private *priv = ctx->phydev->priv; struct macsec_flow *flow, *tmp; - /* No operation to perform before the commit step */ - if (ctx->prepare) - return 0; - list_for_each_entry_safe(flow, tmp, &priv->macsec_flows, list) vsc8584_macsec_flow_disable(ctx->phydev, flow); @@ -785,12 +758,8 @@ static int vsc8584_macsec_add_secy(struct macsec_context *ctx) struct vsc8531_private *priv = ctx->phydev->priv; struct macsec_secy *secy = ctx->secy; - if (ctx->prepare) { - if (priv->secy) - return -EEXIST; - - return 0; - } + if (priv->secy) + return -EEXIST; priv->secy = secy; @@ -807,10 +776,6 @@ static int vsc8584_macsec_del_secy(struct macsec_context *ctx) struct vsc8531_private *priv = ctx->phydev->priv; struct macsec_flow *flow, *tmp; - /* No operation to perform before the commit step */ - if (ctx->prepare) - return 0; - list_for_each_entry_safe(flow, tmp, &priv->macsec_flows, list) vsc8584_macsec_del_flow(ctx->phydev, flow); @@ -823,10 +788,6 @@ static int vsc8584_macsec_del_secy(struct macsec_context *ctx) static int vsc8584_macsec_upd_secy(struct macsec_context *ctx) { - /* No operation to perform before the commit step */ - if (ctx->prepare) - return 0; - vsc8584_macsec_del_secy(ctx); return vsc8584_macsec_add_secy(ctx); } @@ -847,10 +808,6 @@ static int vsc8584_macsec_del_rxsc(struct macsec_context *ctx) struct vsc8531_private *priv = ctx->phydev->priv; struct macsec_flow *flow, *tmp; - /* No operation to perform before the commit step */ - if (ctx->prepare) - return 0; - list_for_each_entry_safe(flow, tmp, &priv->macsec_flows, list) { if (flow->bank == MACSEC_INGR && flow->rx_sa && flow->rx_sa->sc->sci == ctx->rx_sc->sci) @@ -862,33 +819,40 @@ static int vsc8584_macsec_del_rxsc(struct macsec_context *ctx) static int vsc8584_macsec_add_rxsa(struct macsec_context *ctx) { - struct macsec_flow *flow = NULL; - - if (ctx->prepare) - return __vsc8584_macsec_add_rxsa(ctx, flow, false); + struct phy_device *phydev = ctx->phydev; + struct vsc8531_private *priv = phydev->priv; + struct macsec_flow *flow; + int ret; - flow = vsc8584_macsec_find_flow(ctx, MACSEC_INGR); + flow = vsc8584_macsec_alloc_flow(priv, MACSEC_INGR); if (IS_ERR(flow)) return PTR_ERR(flow); - vsc8584_macsec_flow_enable(ctx->phydev, flow); + memcpy(flow->key, ctx->sa.key, priv->secy->key_len); + + ret = __vsc8584_macsec_add_rxsa(ctx, flow, false); + if (ret) + return ret; + + vsc8584_macsec_flow_enable(phydev, flow); return 0; } static int vsc8584_macsec_upd_rxsa(struct macsec_context *ctx) { struct macsec_flow *flow; + int ret; flow = vsc8584_macsec_find_flow(ctx, MACSEC_INGR); if (IS_ERR(flow)) return PTR_ERR(flow); - if (ctx->prepare) { - /* Make sure the flow is disabled before updating it */ - vsc8584_macsec_flow_disable(ctx->phydev, flow); + /* Make sure the flow is disabled before updating it */ + vsc8584_macsec_flow_disable(ctx->phydev, flow); - return __vsc8584_macsec_add_rxsa(ctx, flow, true); - } + ret = __vsc8584_macsec_add_rxsa(ctx, flow, true); + if (ret) + return ret; vsc8584_macsec_flow_enable(ctx->phydev, flow); return 0; @@ -899,11 +863,8 @@ static int vsc8584_macsec_del_rxsa(struct macsec_context *ctx) struct macsec_flow *flow; flow = vsc8584_macsec_find_flow(ctx, MACSEC_INGR); - if (IS_ERR(flow)) return PTR_ERR(flow); - if (ctx->prepare) - return 0; vsc8584_macsec_del_flow(ctx->phydev, flow); return 0; @@ -911,33 +872,40 @@ static int vsc8584_macsec_del_rxsa(struct macsec_context *ctx) static int vsc8584_macsec_add_txsa(struct macsec_context *ctx) { - struct macsec_flow *flow = NULL; - - if (ctx->prepare) - return __vsc8584_macsec_add_txsa(ctx, flow, false); + struct phy_device *phydev = ctx->phydev; + struct vsc8531_private *priv = phydev->priv; + struct macsec_flow *flow; + int ret; - flow = vsc8584_macsec_find_flow(ctx, MACSEC_EGR); + flow = vsc8584_macsec_alloc_flow(priv, MACSEC_EGR); if (IS_ERR(flow)) return PTR_ERR(flow); - vsc8584_macsec_flow_enable(ctx->phydev, flow); + memcpy(flow->key, ctx->sa.key, priv->secy->key_len); + + ret = __vsc8584_macsec_add_txsa(ctx, flow, false); + if (ret) + return ret; + + vsc8584_macsec_flow_enable(phydev, flow); return 0; } static int vsc8584_macsec_upd_txsa(struct macsec_context *ctx) { struct macsec_flow *flow; + int ret; flow = vsc8584_macsec_find_flow(ctx, MACSEC_EGR); if (IS_ERR(flow)) return PTR_ERR(flow); - if (ctx->prepare) { - /* Make sure the flow is disabled before updating it */ - vsc8584_macsec_flow_disable(ctx->phydev, flow); + /* Make sure the flow is disabled before updating it */ + vsc8584_macsec_flow_disable(ctx->phydev, flow); - return __vsc8584_macsec_add_txsa(ctx, flow, true); - } + ret = __vsc8584_macsec_add_txsa(ctx, flow, true); + if (ret) + return ret; vsc8584_macsec_flow_enable(ctx->phydev, flow); return 0; @@ -948,11 +916,8 @@ static int vsc8584_macsec_del_txsa(struct macsec_context *ctx) struct macsec_flow *flow; flow = vsc8584_macsec_find_flow(ctx, MACSEC_EGR); - if (IS_ERR(flow)) return PTR_ERR(flow); - if (ctx->prepare) - return 0; vsc8584_macsec_del_flow(ctx->phydev, flow); return 0; diff --git a/drivers/net/phy/mscc/mscc_main.c b/drivers/net/phy/mscc/mscc_main.c index 7e3017e7a1c0..8a13b1ad9a33 100644 --- a/drivers/net/phy/mscc/mscc_main.c +++ b/drivers/net/phy/mscc/mscc_main.c @@ -136,7 +136,7 @@ static void vsc85xx_get_strings(struct phy_device *phydev, u8 *data) return; for (i = 0; i < priv->nstats; i++) - strlcpy(data + i * ETH_GSTRING_LEN, priv->hw_stats[i].string, + strscpy(data + i * ETH_GSTRING_LEN, priv->hw_stats[i].string, ETH_GSTRING_LEN); } diff --git a/drivers/net/phy/nxp-tja11xx.c b/drivers/net/phy/nxp-tja11xx.c index 2a8195c50d14..ec91e671f8aa 100644 --- a/drivers/net/phy/nxp-tja11xx.c +++ b/drivers/net/phy/nxp-tja11xx.c @@ -10,6 +10,7 @@ #include <linux/mdio.h> #include <linux/mii.h> #include <linux/module.h> +#include <linux/of.h> #include <linux/phy.h> #include <linux/hwmon.h> #include <linux/bitfield.h> @@ -34,6 +35,11 @@ #define MII_CFG1 18 #define MII_CFG1_MASTER_SLAVE BIT(15) #define MII_CFG1_AUTO_OP BIT(14) +#define MII_CFG1_INTERFACE_MODE_MASK GENMASK(9, 8) +#define MII_CFG1_MII_MODE (0x0 << 8) +#define MII_CFG1_RMII_MODE_REFCLK_IN BIT(8) +#define MII_CFG1_RMII_MODE_REFCLK_OUT BIT(9) +#define MII_CFG1_REVMII_MODE GENMASK(9, 8) #define MII_CFG1_SLEEP_CONFIRM BIT(6) #define MII_CFG1_LED_MODE_MASK GENMASK(5, 4) #define MII_CFG1_LED_MODE_LINKUP 0 @@ -72,11 +78,15 @@ #define MII_COMMCFG 27 #define MII_COMMCFG_AUTO_OP BIT(15) +/* Configure REF_CLK as input in RMII mode */ +#define TJA110X_RMII_MODE_REFCLK_IN BIT(0) + struct tja11xx_priv { char *hwmon_name; struct device *hwmon_dev; struct phy_device *phydev; struct work_struct phy_register_work; + u32 flags; }; struct tja11xx_phy_stats { @@ -251,8 +261,34 @@ do_test: return __genphy_config_aneg(phydev, changed); } +static int tja11xx_get_interface_mode(struct phy_device *phydev) +{ + struct tja11xx_priv *priv = phydev->priv; + int mii_mode; + + switch (phydev->interface) { + case PHY_INTERFACE_MODE_MII: + mii_mode = MII_CFG1_MII_MODE; + break; + case PHY_INTERFACE_MODE_REVMII: + mii_mode = MII_CFG1_REVMII_MODE; + break; + case PHY_INTERFACE_MODE_RMII: + if (priv->flags & TJA110X_RMII_MODE_REFCLK_IN) + mii_mode = MII_CFG1_RMII_MODE_REFCLK_IN; + else + mii_mode = MII_CFG1_RMII_MODE_REFCLK_OUT; + break; + default: + return -EINVAL; + } + + return mii_mode; +} + static int tja11xx_config_init(struct phy_device *phydev) { + u16 reg_mask, reg_val; int ret; ret = tja11xx_enable_reg_write(phydev); @@ -265,15 +301,32 @@ static int tja11xx_config_init(struct phy_device *phydev) switch (phydev->phy_id & PHY_ID_MASK) { case PHY_ID_TJA1100: - ret = phy_modify(phydev, MII_CFG1, - MII_CFG1_AUTO_OP | MII_CFG1_LED_MODE_MASK | - MII_CFG1_LED_ENABLE, - MII_CFG1_AUTO_OP | MII_CFG1_LED_MODE_LINKUP | - MII_CFG1_LED_ENABLE); + reg_mask = MII_CFG1_AUTO_OP | MII_CFG1_LED_MODE_MASK | + MII_CFG1_LED_ENABLE; + reg_val = MII_CFG1_AUTO_OP | MII_CFG1_LED_MODE_LINKUP | + MII_CFG1_LED_ENABLE; + + reg_mask |= MII_CFG1_INTERFACE_MODE_MASK; + ret = tja11xx_get_interface_mode(phydev); + if (ret < 0) + return ret; + + reg_val |= (ret & 0xffff); + ret = phy_modify(phydev, MII_CFG1, reg_mask, reg_val); if (ret) return ret; break; case PHY_ID_TJA1101: + reg_mask = MII_CFG1_INTERFACE_MODE_MASK; + ret = tja11xx_get_interface_mode(phydev); + if (ret < 0) + return ret; + + reg_val = ret & 0xffff; + ret = phy_modify(phydev, MII_CFG1, reg_mask, reg_val); + if (ret) + return ret; + fallthrough; case PHY_ID_TJA1102: ret = phy_set_bits(phydev, MII_COMMCFG, MII_COMMCFG_AUTO_OP); if (ret) @@ -458,16 +511,36 @@ static int tja11xx_hwmon_register(struct phy_device *phydev, return PTR_ERR_OR_ZERO(priv->hwmon_dev); } +static int tja11xx_parse_dt(struct phy_device *phydev) +{ + struct device_node *node = phydev->mdio.dev.of_node; + struct tja11xx_priv *priv = phydev->priv; + + if (!IS_ENABLED(CONFIG_OF_MDIO)) + return 0; + + if (of_property_read_bool(node, "nxp,rmii-refclk-in")) + priv->flags |= TJA110X_RMII_MODE_REFCLK_IN; + + return 0; +} + static int tja11xx_probe(struct phy_device *phydev) { struct device *dev = &phydev->mdio.dev; struct tja11xx_priv *priv; + int ret; priv = devm_kzalloc(dev, sizeof(*priv), GFP_KERNEL); if (!priv) return -ENOMEM; priv->phydev = phydev; + phydev->priv = priv; + + ret = tja11xx_parse_dt(phydev); + if (ret) + return ret; return tja11xx_hwmon_register(phydev, priv); } diff --git a/drivers/net/phy/phy-core.c b/drivers/net/phy/phy-core.c index 1f2531a1a876..2c8bf438ea61 100644 --- a/drivers/net/phy/phy-core.c +++ b/drivers/net/phy/phy-core.c @@ -74,6 +74,80 @@ const char *phy_duplex_to_str(unsigned int duplex) } EXPORT_SYMBOL_GPL(phy_duplex_to_str); +/** + * phy_rate_matching_to_str - Return a string describing the rate matching + * + * @rate_matching: Type of rate matching to describe + */ +const char *phy_rate_matching_to_str(int rate_matching) +{ + switch (rate_matching) { + case RATE_MATCH_NONE: + return "none"; + case RATE_MATCH_PAUSE: + return "pause"; + case RATE_MATCH_CRS: + return "crs"; + case RATE_MATCH_OPEN_LOOP: + return "open-loop"; + } + return "Unsupported (update phy-core.c)"; +} +EXPORT_SYMBOL_GPL(phy_rate_matching_to_str); + +/** + * phy_interface_num_ports - Return the number of links that can be carried by + * a given MAC-PHY physical link. Returns 0 if this is + * unknown, the number of links else. + * + * @interface: The interface mode we want to get the number of ports + */ +int phy_interface_num_ports(phy_interface_t interface) +{ + switch (interface) { + case PHY_INTERFACE_MODE_NA: + return 0; + case PHY_INTERFACE_MODE_INTERNAL: + case PHY_INTERFACE_MODE_MII: + case PHY_INTERFACE_MODE_GMII: + case PHY_INTERFACE_MODE_TBI: + case PHY_INTERFACE_MODE_REVMII: + case PHY_INTERFACE_MODE_RMII: + case PHY_INTERFACE_MODE_REVRMII: + case PHY_INTERFACE_MODE_RGMII: + case PHY_INTERFACE_MODE_RGMII_ID: + case PHY_INTERFACE_MODE_RGMII_RXID: + case PHY_INTERFACE_MODE_RGMII_TXID: + case PHY_INTERFACE_MODE_RTBI: + case PHY_INTERFACE_MODE_XGMII: + case PHY_INTERFACE_MODE_XLGMII: + case PHY_INTERFACE_MODE_MOCA: + case PHY_INTERFACE_MODE_TRGMII: + case PHY_INTERFACE_MODE_USXGMII: + case PHY_INTERFACE_MODE_SGMII: + case PHY_INTERFACE_MODE_SMII: + case PHY_INTERFACE_MODE_1000BASEX: + case PHY_INTERFACE_MODE_2500BASEX: + case PHY_INTERFACE_MODE_5GBASER: + case PHY_INTERFACE_MODE_10GBASER: + case PHY_INTERFACE_MODE_25GBASER: + case PHY_INTERFACE_MODE_10GKR: + case PHY_INTERFACE_MODE_100BASEX: + case PHY_INTERFACE_MODE_RXAUI: + case PHY_INTERFACE_MODE_XAUI: + case PHY_INTERFACE_MODE_1000BASEKX: + return 1; + case PHY_INTERFACE_MODE_QSGMII: + case PHY_INTERFACE_MODE_QUSGMII: + return 4; + case PHY_INTERFACE_MODE_MAX: + WARN_ONCE(1, "PHY_INTERFACE_MODE_MAX isn't a valid interface mode"); + return 0; + } + return 0; +} +EXPORT_SYMBOL_GPL(phy_interface_num_ports); + /* A mapping of all SUPPORTED settings to speed/duplex. This table * must be grouped by speed and sorted in descending match priority * - iow, descending speed. diff --git a/drivers/net/phy/phy.c b/drivers/net/phy/phy.c index 8d3ee3a6495b..e741d8aebffe 100644 --- a/drivers/net/phy/phy.c +++ b/drivers/net/phy/phy.c @@ -115,6 +115,33 @@ void phy_print_status(struct phy_device *phydev) EXPORT_SYMBOL(phy_print_status); /** + * phy_get_rate_matching - determine if rate matching is supported + * @phydev: The phy device to return rate matching for + * @iface: The interface mode to use + * + * This determines the type of rate matching (if any) that @phy supports + * using @iface. @iface may be %PHY_INTERFACE_MODE_NA to determine if any + * interface supports rate matching. + * + * Return: The type of rate matching @phy supports for @iface, or + * %RATE_MATCH_NONE. + */ +int phy_get_rate_matching(struct phy_device *phydev, + phy_interface_t iface) +{ + int ret = RATE_MATCH_NONE; + + if (phydev->drv->get_rate_matching) { + mutex_lock(&phydev->lock); + ret = phydev->drv->get_rate_matching(phydev, iface); + mutex_unlock(&phydev->lock); + } + + return ret; +} +EXPORT_SYMBOL_GPL(phy_get_rate_matching); + +/** * phy_config_interrupt - configure the PHY device for the requested interrupts * @phydev: the phy_device struct * @interrupts: interrupt flags to configure for this @phydev @@ -256,6 +283,7 @@ void phy_ethtool_ksettings_get(struct phy_device *phydev, cmd->base.duplex = phydev->duplex; cmd->base.master_slave_cfg = phydev->master_slave_get; cmd->base.master_slave_state = phydev->master_slave_state; + cmd->base.rate_matching = phydev->rate_matching; if (phydev->interface == PHY_INTERFACE_MODE_MOCA) cmd->base.port = PORT_BNC; else diff --git a/drivers/net/phy/phy_device.c b/drivers/net/phy/phy_device.c index 4df8c337221b..57849ac0384e 100644 --- a/drivers/net/phy/phy_device.c +++ b/drivers/net/phy/phy_device.c @@ -26,6 +26,7 @@ #include <linux/netdevice.h> #include <linux/phy.h> #include <linux/phy_led_triggers.h> +#include <linux/pse-pd/pse.h> #include <linux/property.h> #include <linux/sfp.h> #include <linux/skbuff.h> @@ -372,7 +373,7 @@ int phy_register_fixup(const char *bus_id, u32 phy_uid, u32 phy_uid_mask, if (!fixup) return -ENOMEM; - strlcpy(fixup->bus_id, bus_id, sizeof(fixup->bus_id)); + strscpy(fixup->bus_id, bus_id, sizeof(fixup->bus_id)); fixup->phy_uid = phy_uid; fixup->phy_uid_mask = phy_uid_mask; fixup->run = run; @@ -522,7 +523,7 @@ phy_id_show(struct device *dev, struct device_attribute *attr, char *buf) { struct phy_device *phydev = to_phy_device(dev); - return sprintf(buf, "0x%.8lx\n", (unsigned long)phydev->phy_id); + return sysfs_emit(buf, "0x%.8lx\n", (unsigned long)phydev->phy_id); } static DEVICE_ATTR_RO(phy_id); @@ -537,7 +538,7 @@ phy_interface_show(struct device *dev, struct device_attribute *attr, char *buf) else mode = phy_modes(phydev->interface); - return sprintf(buf, "%s\n", mode); + return sysfs_emit(buf, "%s\n", mode); } static DEVICE_ATTR_RO(phy_interface); @@ -547,7 +548,7 @@ phy_has_fixups_show(struct device *dev, struct device_attribute *attr, { struct phy_device *phydev = to_phy_device(dev); - return sprintf(buf, "%d\n", phydev->has_fixups); + return sysfs_emit(buf, "%d\n", phydev->has_fixups); } static DEVICE_ATTR_RO(phy_has_fixups); @@ -557,7 +558,7 @@ static ssize_t phy_dev_flags_show(struct device *dev, { struct phy_device *phydev = to_phy_device(dev); - return sprintf(buf, "0x%08x\n", phydev->dev_flags); + return sysfs_emit(buf, "0x%08x\n", phydev->dev_flags); } static DEVICE_ATTR_RO(phy_dev_flags); @@ -991,6 +992,7 @@ EXPORT_SYMBOL(phy_device_register); void phy_device_remove(struct phy_device *phydev) { unregister_mii_timestamper(phydev->mii_ts); + pse_control_put(phydev->psec); device_del(&phydev->mdio.dev); @@ -1312,7 +1314,7 @@ phy_standalone_show(struct device *dev, struct device_attribute *attr, { struct phy_device *phydev = to_phy_device(dev); - return sprintf(buf, "%d\n", !phydev->attached_dev); + return sysfs_emit(buf, "%d\n", !phydev->attached_dev); } static DEVICE_ATTR_RO(phy_standalone); diff --git a/drivers/net/phy/phylink.c b/drivers/net/phy/phylink.c index 9bd69328dc4d..75464df191ef 100644 --- a/drivers/net/phy/phylink.c +++ b/drivers/net/phy/phylink.c @@ -77,6 +77,7 @@ struct phylink { struct sfp_bus *sfp_bus; bool sfp_may_have_phy; + DECLARE_PHY_INTERFACE_MASK(sfp_interfaces); __ETHTOOL_DECLARE_LINK_MODE_MASK(sfp_support); u8 sfp_port; }; @@ -155,8 +156,84 @@ static const char *phylink_an_mode_str(unsigned int mode) return mode < ARRAY_SIZE(modestr) ? modestr[mode] : "unknown"; } -static void phylink_caps_to_linkmodes(unsigned long *linkmodes, - unsigned long caps) +/** + * phylink_interface_max_speed() - get the maximum speed of a phy interface + * @interface: phy interface mode defined by &typedef phy_interface_t + * + * Determine the maximum speed of a phy interface. This is intended to help + * determine the correct speed to pass to the MAC when the phy is performing + * rate matching. + * + * Return: The maximum speed of @interface + */ +static int phylink_interface_max_speed(phy_interface_t interface) +{ + switch (interface) { + case PHY_INTERFACE_MODE_100BASEX: + case PHY_INTERFACE_MODE_REVRMII: + case PHY_INTERFACE_MODE_RMII: + case PHY_INTERFACE_MODE_SMII: + case PHY_INTERFACE_MODE_REVMII: + case PHY_INTERFACE_MODE_MII: + return SPEED_100; + + case PHY_INTERFACE_MODE_TBI: + case PHY_INTERFACE_MODE_MOCA: + case PHY_INTERFACE_MODE_RTBI: + case PHY_INTERFACE_MODE_1000BASEX: + case PHY_INTERFACE_MODE_1000BASEKX: + case PHY_INTERFACE_MODE_TRGMII: + case PHY_INTERFACE_MODE_RGMII_TXID: + case PHY_INTERFACE_MODE_RGMII_RXID: + case PHY_INTERFACE_MODE_RGMII_ID: + case PHY_INTERFACE_MODE_RGMII: + case PHY_INTERFACE_MODE_QSGMII: + case PHY_INTERFACE_MODE_SGMII: + case PHY_INTERFACE_MODE_GMII: + return SPEED_1000; + + case PHY_INTERFACE_MODE_2500BASEX: + return SPEED_2500; + + case PHY_INTERFACE_MODE_5GBASER: + return SPEED_5000; + + case PHY_INTERFACE_MODE_XGMII: + case PHY_INTERFACE_MODE_RXAUI: + case PHY_INTERFACE_MODE_XAUI: + case PHY_INTERFACE_MODE_10GBASER: + case PHY_INTERFACE_MODE_10GKR: + case PHY_INTERFACE_MODE_USXGMII: + case PHY_INTERFACE_MODE_QUSGMII: + return SPEED_10000; + + case PHY_INTERFACE_MODE_25GBASER: + return SPEED_25000; + + case PHY_INTERFACE_MODE_XLGMII: + return SPEED_40000; + + case PHY_INTERFACE_MODE_INTERNAL: + case PHY_INTERFACE_MODE_NA: + case PHY_INTERFACE_MODE_MAX: + /* No idea! Garbage in, unknown out */ + return SPEED_UNKNOWN; + } + + /* If we get here, someone forgot to add an interface mode above */ + WARN_ON_ONCE(1); + return SPEED_UNKNOWN; +} + +/** + * phylink_caps_to_linkmodes() - Convert capabilities to ethtool link modes + * @linkmodes: ethtool linkmode mask (must be already initialised) + * @caps: bitmask of MAC capabilities + * + * Set all possible pause, speed and duplex linkmodes in @linkmodes that are + * supported by the @caps. @linkmodes must have been initialised previously. + */ +void phylink_caps_to_linkmodes(unsigned long *linkmodes, unsigned long caps) { if (caps & MAC_SYM_PAUSE) __set_bit(ETHTOOL_LINK_MODE_Pause_BIT, linkmodes); @@ -295,21 +372,72 @@ static void phylink_caps_to_linkmodes(unsigned long *linkmodes, __set_bit(ETHTOOL_LINK_MODE_400000baseCR4_Full_BIT, linkmodes); } } +EXPORT_SYMBOL_GPL(phylink_caps_to_linkmodes); + +static struct { + unsigned long mask; + int speed; + unsigned int duplex; +} phylink_caps_params[] = { + { MAC_400000FD, SPEED_400000, DUPLEX_FULL }, + { MAC_200000FD, SPEED_200000, DUPLEX_FULL }, + { MAC_100000FD, SPEED_100000, DUPLEX_FULL }, + { MAC_56000FD, SPEED_56000, DUPLEX_FULL }, + { MAC_50000FD, SPEED_50000, DUPLEX_FULL }, + { MAC_40000FD, SPEED_40000, DUPLEX_FULL }, + { MAC_25000FD, SPEED_25000, DUPLEX_FULL }, + { MAC_20000FD, SPEED_20000, DUPLEX_FULL }, + { MAC_10000FD, SPEED_10000, DUPLEX_FULL }, + { MAC_5000FD, SPEED_5000, DUPLEX_FULL }, + { MAC_2500FD, SPEED_2500, DUPLEX_FULL }, + { MAC_1000FD, SPEED_1000, DUPLEX_FULL }, + { MAC_1000HD, SPEED_1000, DUPLEX_HALF }, + { MAC_100FD, SPEED_100, DUPLEX_FULL }, + { MAC_100HD, SPEED_100, DUPLEX_HALF }, + { MAC_10FD, SPEED_10, DUPLEX_FULL }, + { MAC_10HD, SPEED_10, DUPLEX_HALF }, +}; /** - * phylink_get_linkmodes() - get acceptable link modes - * @linkmodes: ethtool linkmode mask (must be already initialised) + * phylink_cap_from_speed_duplex - Get mac capability from speed/duplex + * @speed: the speed to search for + * @duplex: the duplex to search for + * + * Find the mac capability for a given speed and duplex. + * + * Return: A mask with the mac capability patching @speed and @duplex, or 0 if + * there were no matches. + */ +static unsigned long phylink_cap_from_speed_duplex(int speed, + unsigned int duplex) +{ + int i; + + for (i = 0; i < ARRAY_SIZE(phylink_caps_params); i++) { + if (speed == phylink_caps_params[i].speed && + duplex == phylink_caps_params[i].duplex) + return phylink_caps_params[i].mask; + } + + return 0; +} + +/** + * phylink_get_capabilities() - get capabilities for a given MAC * @interface: phy interface mode defined by &typedef phy_interface_t * @mac_capabilities: bitmask of MAC capabilities + * @rate_matching: type of rate matching being performed * - * Set all possible pause, speed and duplex linkmodes in @linkmodes that - * are supported by the @interface mode and @mac_capabilities. @linkmodes - * must have been initialised previously. + * Get the MAC capabilities that are supported by the @interface mode and + * @mac_capabilities. */ -void phylink_get_linkmodes(unsigned long *linkmodes, phy_interface_t interface, - unsigned long mac_capabilities) +unsigned long phylink_get_capabilities(phy_interface_t interface, + unsigned long mac_capabilities, + int rate_matching) { + int max_speed = phylink_interface_max_speed(interface); unsigned long caps = MAC_SYM_PAUSE | MAC_ASYM_PAUSE; + unsigned long matched_caps = 0; switch (interface) { case PHY_INTERFACE_MODE_USXGMII: @@ -321,6 +449,7 @@ void phylink_get_linkmodes(unsigned long *linkmodes, phy_interface_t interface, case PHY_INTERFACE_MODE_RGMII_ID: case PHY_INTERFACE_MODE_RGMII: case PHY_INTERFACE_MODE_QSGMII: + case PHY_INTERFACE_MODE_QUSGMII: case PHY_INTERFACE_MODE_SGMII: case PHY_INTERFACE_MODE_GMII: caps |= MAC_1000HD | MAC_1000FD; @@ -344,6 +473,7 @@ void phylink_get_linkmodes(unsigned long *linkmodes, phy_interface_t interface, case PHY_INTERFACE_MODE_1000BASEX: caps |= MAC_1000HD; fallthrough; + case PHY_INTERFACE_MODE_1000BASEKX: case PHY_INTERFACE_MODE_TRGMII: caps |= MAC_1000FD; break; @@ -381,9 +511,55 @@ void phylink_get_linkmodes(unsigned long *linkmodes, phy_interface_t interface, break; } - phylink_caps_to_linkmodes(linkmodes, caps & mac_capabilities); + switch (rate_matching) { + case RATE_MATCH_OPEN_LOOP: + /* TODO */ + fallthrough; + case RATE_MATCH_NONE: + matched_caps = 0; + break; + case RATE_MATCH_PAUSE: { + /* The MAC must support asymmetric pause towards the local + * device for this. We could allow just symmetric pause, but + * then we might have to renegotiate if the link partner + * doesn't support pause. This is because there's no way to + * accept pause frames without transmitting them if we only + * support symmetric pause. + */ + if (!(mac_capabilities & MAC_SYM_PAUSE) || + !(mac_capabilities & MAC_ASYM_PAUSE)) + break; + + /* We can't adapt if the MAC doesn't support the interface's + * max speed at full duplex. + */ + if (mac_capabilities & + phylink_cap_from_speed_duplex(max_speed, DUPLEX_FULL)) { + /* Although a duplex-matching phy might exist, we + * conservatively remove these modes because the MAC + * will not be aware of the half-duplex nature of the + * link. + */ + matched_caps = GENMASK(__fls(caps), __fls(MAC_10HD)); + matched_caps &= ~(MAC_1000HD | MAC_100HD | MAC_10HD); + } + break; + } + case RATE_MATCH_CRS: + /* The MAC must support half duplex at the interface's max + * speed. + */ + if (mac_capabilities & + phylink_cap_from_speed_duplex(max_speed, DUPLEX_HALF)) { + matched_caps = GENMASK(__fls(caps), __fls(MAC_10HD)); + matched_caps &= mac_capabilities; + } + break; + } + + return (caps & mac_capabilities) | matched_caps; } -EXPORT_SYMBOL_GPL(phylink_get_linkmodes); +EXPORT_SYMBOL_GPL(phylink_get_capabilities); /** * phylink_generic_validate() - generic validate() callback implementation @@ -400,10 +576,14 @@ void phylink_generic_validate(struct phylink_config *config, struct phylink_link_state *state) { __ETHTOOL_DECLARE_LINK_MODE_MASK(mask) = { 0, }; + unsigned long caps; phylink_set_port_modes(mask); phylink_set(mask, Autoneg); - phylink_get_linkmodes(mask, state->interface, config->mac_capabilities); + caps = phylink_get_capabilities(state->interface, + config->mac_capabilities, + state->rate_matching); + phylink_caps_to_linkmodes(mask, caps); linkmode_and(supported, supported, mask); linkmode_and(state->advertising, state->advertising, mask); @@ -458,8 +638,9 @@ static int phylink_validate_mac_and_pcs(struct phylink *pl, return phylink_is_empty_linkmode(supported) ? -EINVAL : 0; } -static int phylink_validate_any(struct phylink *pl, unsigned long *supported, - struct phylink_link_state *state) +static int phylink_validate_mask(struct phylink *pl, unsigned long *supported, + struct phylink_link_state *state, + const unsigned long *interfaces) { __ETHTOOL_DECLARE_LINK_MODE_MASK(all_adv) = { 0, }; __ETHTOOL_DECLARE_LINK_MODE_MASK(all_s) = { 0, }; @@ -468,7 +649,7 @@ static int phylink_validate_any(struct phylink *pl, unsigned long *supported, int intf; for (intf = 0; intf < PHY_INTERFACE_MODE_MAX; intf++) { - if (test_bit(intf, pl->config->supported_interfaces)) { + if (test_bit(intf, interfaces)) { linkmode_copy(s, supported); t = *state; @@ -489,12 +670,14 @@ static int phylink_validate_any(struct phylink *pl, unsigned long *supported, static int phylink_validate(struct phylink *pl, unsigned long *supported, struct phylink_link_state *state) { - if (!phy_interface_empty(pl->config->supported_interfaces)) { + const unsigned long *interfaces = pl->config->supported_interfaces; + + if (!phy_interface_empty(interfaces)) { if (state->interface == PHY_INTERFACE_MODE_NA) - return phylink_validate_any(pl, supported, state); + return phylink_validate_mask(pl, supported, state, + interfaces); - if (!test_bit(state->interface, - pl->config->supported_interfaces)) + if (!test_bit(state->interface, interfaces)) return -EINVAL; } @@ -632,6 +815,12 @@ static int phylink_parse_mode(struct phylink *pl, struct fwnode_handle *fwnode) switch (pl->link_config.interface) { case PHY_INTERFACE_MODE_SGMII: case PHY_INTERFACE_MODE_QSGMII: + case PHY_INTERFACE_MODE_QUSGMII: + case PHY_INTERFACE_MODE_RGMII: + case PHY_INTERFACE_MODE_RGMII_ID: + case PHY_INTERFACE_MODE_RGMII_RXID: + case PHY_INTERFACE_MODE_RGMII_TXID: + case PHY_INTERFACE_MODE_RTBI: phylink_set(pl->supported, 10baseT_Half); phylink_set(pl->supported, 10baseT_Full); phylink_set(pl->supported, 100baseT_Half); @@ -774,11 +963,12 @@ static void phylink_mac_config(struct phylink *pl, const struct phylink_link_state *state) { phylink_dbg(pl, - "%s: mode=%s/%s/%s/%s adv=%*pb pause=%02x link=%u an=%u\n", + "%s: mode=%s/%s/%s/%s/%s adv=%*pb pause=%02x link=%u an=%u\n", __func__, phylink_an_mode_str(pl->cur_link_an_mode), phy_modes(state->interface), phy_speed_to_str(state->speed), phy_duplex_to_str(state->duplex), + phy_rate_matching_to_str(state->rate_matching), __ETHTOOL_LINK_MODE_MASK_NBITS, state->advertising, state->pause, state->link, state->an_enabled); @@ -915,7 +1105,8 @@ static void phylink_mac_pcs_get_state(struct phylink *pl, linkmode_zero(state->lp_advertising); state->interface = pl->link_config.interface; state->an_enabled = pl->link_config.an_enabled; - if (state->an_enabled) { + state->rate_matching = pl->link_config.rate_matching; + if (state->an_enabled) { state->speed = SPEED_UNKNOWN; state->duplex = DUPLEX_UNKNOWN; state->pause = MLO_PAUSE_NONE; @@ -998,19 +1189,43 @@ static void phylink_link_up(struct phylink *pl, struct phylink_link_state link_state) { struct net_device *ndev = pl->netdev; + int speed, duplex; + bool rx_pause; + + speed = link_state.speed; + duplex = link_state.duplex; + rx_pause = !!(link_state.pause & MLO_PAUSE_RX); + + switch (link_state.rate_matching) { + case RATE_MATCH_PAUSE: + /* The PHY is doing rate matchion from the media rate (in + * the link_state) to the interface speed, and will send + * pause frames to the MAC to limit its transmission speed. + */ + speed = phylink_interface_max_speed(link_state.interface); + duplex = DUPLEX_FULL; + rx_pause = true; + break; + + case RATE_MATCH_CRS: + /* The PHY is doing rate matchion from the media rate (in + * the link_state) to the interface speed, and will cause + * collisions to the MAC to limit its transmission speed. + */ + speed = phylink_interface_max_speed(link_state.interface); + duplex = DUPLEX_HALF; + break; + } pl->cur_interface = link_state.interface; if (pl->pcs && pl->pcs->ops->pcs_link_up) pl->pcs->ops->pcs_link_up(pl->pcs, pl->cur_link_an_mode, - pl->cur_interface, - link_state.speed, link_state.duplex); + pl->cur_interface, speed, duplex); - pl->mac_ops->mac_link_up(pl->config, pl->phydev, - pl->cur_link_an_mode, pl->cur_interface, - link_state.speed, link_state.duplex, - !!(link_state.pause & MLO_PAUSE_TX), - !!(link_state.pause & MLO_PAUSE_RX)); + pl->mac_ops->mac_link_up(pl->config, pl->phydev, pl->cur_link_an_mode, + pl->cur_interface, speed, duplex, + !!(link_state.pause & MLO_PAUSE_TX), rx_pause); if (ndev) netif_carrier_on(ndev); @@ -1102,6 +1317,17 @@ static void phylink_resolve(struct work_struct *w) } link_state.interface = pl->phy_state.interface; + /* If we are doing rate matching, then the + * link speed/duplex comes from the PHY + */ + if (pl->phy_state.rate_matching) { + link_state.rate_matching = + pl->phy_state.rate_matching; + link_state.speed = pl->phy_state.speed; + link_state.duplex = + pl->phy_state.duplex; + } + /* If we have a PHY, we need to update with * the PHY flow control bits. */ @@ -1336,6 +1562,7 @@ static void phylink_phy_change(struct phy_device *phydev, bool up) mutex_lock(&pl->state_mutex); pl->phy_state.speed = phydev->speed; pl->phy_state.duplex = phydev->duplex; + pl->phy_state.rate_matching = phydev->rate_matching; pl->phy_state.pause = MLO_PAUSE_NONE; if (tx_pause) pl->phy_state.pause |= MLO_PAUSE_TX; @@ -1347,10 +1574,11 @@ static void phylink_phy_change(struct phy_device *phydev, bool up) phylink_run_resolve(pl); - phylink_dbg(pl, "phy link %s %s/%s/%s/%s\n", up ? "up" : "down", + phylink_dbg(pl, "phy link %s %s/%s/%s/%s/%s\n", up ? "up" : "down", phy_modes(phydev->interface), phy_speed_to_str(phydev->speed), phy_duplex_to_str(phydev->duplex), + phy_rate_matching_to_str(phydev->rate_matching), phylink_pause_to_str(pl->phy_state.pause)); } @@ -1387,6 +1615,7 @@ static int phylink_bringup_phy(struct phylink *pl, struct phy_device *phy, config.interface = PHY_INTERFACE_MODE_NA; else config.interface = interface; + config.rate_matching = phy_get_rate_matching(phy, config.interface); ret = phylink_validate(pl, supported, &config); if (ret) { @@ -1414,6 +1643,7 @@ static int phylink_bringup_phy(struct phylink *pl, struct phy_device *phy, pl->phy_state.pause = MLO_PAUSE_NONE; pl->phy_state.speed = SPEED_UNKNOWN; pl->phy_state.duplex = DUPLEX_UNKNOWN; + pl->phy_state.rate_matching = RATE_MATCH_NONE; linkmode_copy(pl->supported, supported); linkmode_copy(pl->link_config.advertising, config.advertising); @@ -1439,7 +1669,7 @@ static int phylink_attach_phy(struct phylink *pl, struct phy_device *phy, { if (WARN_ON(pl->cfg_link_an_mode == MLO_AN_FIXED || (pl->cfg_link_an_mode == MLO_AN_INBAND && - phy_interface_mode_is_8023z(interface)))) + phy_interface_mode_is_8023z(interface) && !pl->sfp_bus))) return -EINVAL; if (pl->phydev) @@ -1856,8 +2086,10 @@ static void phylink_get_ksettings(const struct phylink_link_state *state, { phylink_merge_link_mode(kset->link_modes.advertising, state->advertising); linkmode_copy(kset->link_modes.lp_advertising, state->lp_advertising); - kset->base.speed = state->speed; - kset->base.duplex = state->duplex; + if (kset->base.rate_matching == RATE_MATCH_NONE) { + kset->base.speed = state->speed; + kset->base.duplex = state->duplex; + } kset->base.autoneg = state->an_enabled ? AUTONEG_ENABLE : AUTONEG_DISABLE; } @@ -2571,21 +2803,85 @@ static void phylink_sfp_detach(void *upstream, struct sfp_bus *bus) pl->netdev->sfp_bus = NULL; } -static int phylink_sfp_config(struct phylink *pl, u8 mode, - const unsigned long *supported, - const unsigned long *advertising) +static const phy_interface_t phylink_sfp_interface_preference[] = { + PHY_INTERFACE_MODE_25GBASER, + PHY_INTERFACE_MODE_USXGMII, + PHY_INTERFACE_MODE_10GBASER, + PHY_INTERFACE_MODE_5GBASER, + PHY_INTERFACE_MODE_2500BASEX, + PHY_INTERFACE_MODE_SGMII, + PHY_INTERFACE_MODE_1000BASEX, + PHY_INTERFACE_MODE_100BASEX, +}; + +static DECLARE_PHY_INTERFACE_MASK(phylink_sfp_interfaces); + +static phy_interface_t phylink_choose_sfp_interface(struct phylink *pl, + const unsigned long *intf) +{ + phy_interface_t interface; + size_t i; + + interface = PHY_INTERFACE_MODE_NA; + for (i = 0; i < ARRAY_SIZE(phylink_sfp_interface_preference); i++) + if (test_bit(phylink_sfp_interface_preference[i], intf)) { + interface = phylink_sfp_interface_preference[i]; + break; + } + + return interface; +} + +static void phylink_sfp_set_config(struct phylink *pl, u8 mode, + unsigned long *supported, + struct phylink_link_state *state) +{ + bool changed = false; + + phylink_dbg(pl, "requesting link mode %s/%s with support %*pb\n", + phylink_an_mode_str(mode), phy_modes(state->interface), + __ETHTOOL_LINK_MODE_MASK_NBITS, supported); + + if (!linkmode_equal(pl->supported, supported)) { + linkmode_copy(pl->supported, supported); + changed = true; + } + + if (!linkmode_equal(pl->link_config.advertising, state->advertising)) { + linkmode_copy(pl->link_config.advertising, state->advertising); + changed = true; + } + + if (pl->cur_link_an_mode != mode || + pl->link_config.interface != state->interface) { + pl->cur_link_an_mode = mode; + pl->link_config.interface = state->interface; + + changed = true; + + phylink_info(pl, "switched to %s/%s link mode\n", + phylink_an_mode_str(mode), + phy_modes(state->interface)); + } + + if (changed && !test_bit(PHYLINK_DISABLE_STOPPED, + &pl->phylink_disable_state)) + phylink_mac_initial_config(pl, false); +} + +static int phylink_sfp_config_phy(struct phylink *pl, u8 mode, + struct phy_device *phy) { __ETHTOOL_DECLARE_LINK_MODE_MASK(support1); __ETHTOOL_DECLARE_LINK_MODE_MASK(support); struct phylink_link_state config; phy_interface_t iface; - bool changed; int ret; - linkmode_copy(support, supported); + linkmode_copy(support, phy->supported); memset(&config, 0, sizeof(config)); - linkmode_copy(config.advertising, advertising); + linkmode_copy(config.advertising, phy->advertising); config.interface = PHY_INTERFACE_MODE_NA; config.speed = SPEED_UNKNOWN; config.duplex = DUPLEX_UNKNOWN; @@ -2622,60 +2918,100 @@ static int phylink_sfp_config(struct phylink *pl, u8 mode, return ret; } - phylink_dbg(pl, "requesting link mode %s/%s with support %*pb\n", - phylink_an_mode_str(mode), phy_modes(config.interface), - __ETHTOOL_LINK_MODE_MASK_NBITS, support); + pl->link_port = pl->sfp_port; + + phylink_sfp_set_config(pl, mode, support, &config); + + return 0; +} + +static int phylink_sfp_config_optical(struct phylink *pl) +{ + __ETHTOOL_DECLARE_LINK_MODE_MASK(support); + DECLARE_PHY_INTERFACE_MASK(interfaces); + struct phylink_link_state config; + phy_interface_t interface; + int ret; + + phylink_dbg(pl, "optical SFP: interfaces=[mac=%*pbl, sfp=%*pbl]\n", + (int)PHY_INTERFACE_MODE_MAX, + pl->config->supported_interfaces, + (int)PHY_INTERFACE_MODE_MAX, + pl->sfp_interfaces); - if (phy_interface_mode_is_8023z(iface) && pl->phydev) + /* Find the union of the supported interfaces by the PCS/MAC and + * the SFP module. + */ + phy_interface_and(interfaces, pl->config->supported_interfaces, + pl->sfp_interfaces); + if (phy_interface_empty(interfaces)) { + phylink_err(pl, "unsupported SFP module: no common interface modes\n"); return -EINVAL; + } - changed = !linkmode_equal(pl->supported, support) || - !linkmode_equal(pl->link_config.advertising, - config.advertising); - if (changed) { - linkmode_copy(pl->supported, support); - linkmode_copy(pl->link_config.advertising, config.advertising); + memset(&config, 0, sizeof(config)); + linkmode_copy(support, pl->sfp_support); + linkmode_copy(config.advertising, pl->sfp_support); + config.speed = SPEED_UNKNOWN; + config.duplex = DUPLEX_UNKNOWN; + config.pause = MLO_PAUSE_AN; + config.an_enabled = true; + + /* For all the interfaces that are supported, reduce the sfp_support + * mask to only those link modes that can be supported. + */ + ret = phylink_validate_mask(pl, pl->sfp_support, &config, interfaces); + if (ret) { + phylink_err(pl, "unsupported SFP module: validation with support %*pb failed\n", + __ETHTOOL_LINK_MODE_MASK_NBITS, support); + return ret; } - if (pl->cur_link_an_mode != mode || - pl->link_config.interface != config.interface) { - pl->link_config.interface = config.interface; - pl->cur_link_an_mode = mode; + interface = phylink_choose_sfp_interface(pl, interfaces); + if (interface == PHY_INTERFACE_MODE_NA) { + phylink_err(pl, "failed to select SFP interface\n"); + return -EINVAL; + } - changed = true; + phylink_dbg(pl, "optical SFP: chosen %s interface\n", + phy_modes(interface)); - phylink_info(pl, "switched to %s/%s link mode\n", - phylink_an_mode_str(mode), - phy_modes(config.interface)); + config.interface = interface; + + /* Ignore errors if we're expecting a PHY to attach later */ + ret = phylink_validate(pl, support, &config); + if (ret) { + phylink_err(pl, "validation with support %*pb failed: %pe\n", + __ETHTOOL_LINK_MODE_MASK_NBITS, support, + ERR_PTR(ret)); + return ret; } pl->link_port = pl->sfp_port; - if (changed && !test_bit(PHYLINK_DISABLE_STOPPED, - &pl->phylink_disable_state)) - phylink_mac_initial_config(pl, false); + phylink_sfp_set_config(pl, MLO_AN_INBAND, pl->sfp_support, &config); - return ret; + return 0; } static int phylink_sfp_module_insert(void *upstream, const struct sfp_eeprom_id *id) { struct phylink *pl = upstream; - unsigned long *support = pl->sfp_support; ASSERT_RTNL(); - linkmode_zero(support); - sfp_parse_support(pl->sfp_bus, id, support); - pl->sfp_port = sfp_parse_port(pl->sfp_bus, id, support); + linkmode_zero(pl->sfp_support); + phy_interface_zero(pl->sfp_interfaces); + sfp_parse_support(pl->sfp_bus, id, pl->sfp_support, pl->sfp_interfaces); + pl->sfp_port = sfp_parse_port(pl->sfp_bus, id, pl->sfp_support); /* If this module may have a PHY connecting later, defer until later */ pl->sfp_may_have_phy = sfp_may_have_phy(pl->sfp_bus, id); if (pl->sfp_may_have_phy) return 0; - return phylink_sfp_config(pl, MLO_AN_INBAND, support, support); + return phylink_sfp_config_optical(pl); } static int phylink_sfp_module_start(void *upstream) @@ -2694,8 +3030,7 @@ static int phylink_sfp_module_start(void *upstream) if (!pl->sfp_may_have_phy) return 0; - return phylink_sfp_config(pl, MLO_AN_INBAND, - pl->sfp_support, pl->sfp_support); + return phylink_sfp_config_optical(pl); } static void phylink_sfp_module_stop(void *upstream) @@ -2755,8 +3090,12 @@ static int phylink_sfp_connect_phy(void *upstream, struct phy_device *phy) else mode = MLO_AN_INBAND; + /* Set the PHY's host supported interfaces */ + phy_interface_and(phy->host_interfaces, phylink_sfp_interfaces, + pl->config->supported_interfaces); + /* Do the initial configuration */ - ret = phylink_sfp_config(pl, mode, phy->supported, phy->advertising); + ret = phylink_sfp_config_phy(pl, mode, phy); if (ret < 0) return ret; @@ -2929,6 +3268,7 @@ void phylink_mii_c22_pcs_decode_state(struct phylink_link_state *state, case PHY_INTERFACE_MODE_SGMII: case PHY_INTERFACE_MODE_QSGMII: + case PHY_INTERFACE_MODE_QUSGMII: phylink_decode_sgmii_word(state, lpa); break; @@ -3107,4 +3447,15 @@ void phylink_mii_c45_pcs_get_state(struct mdio_device *pcs, } EXPORT_SYMBOL_GPL(phylink_mii_c45_pcs_get_state); +static int __init phylink_init(void) +{ + for (int i = 0; i < ARRAY_SIZE(phylink_sfp_interface_preference); ++i) + __set_bit(phylink_sfp_interface_preference[i], + phylink_sfp_interfaces); + + return 0; +} + +module_init(phylink_init); + MODULE_LICENSE("GPL v2"); diff --git a/drivers/net/phy/realtek.c b/drivers/net/phy/realtek.c index a5671ab896b3..3d99fd6664d7 100644 --- a/drivers/net/phy/realtek.c +++ b/drivers/net/phy/realtek.c @@ -70,6 +70,7 @@ #define RTLGEN_SPEED_MASK 0x0630 #define RTL_GENERIC_PHYID 0x001cc800 +#define RTL_8211FVD_PHYID 0x001cc878 MODULE_DESCRIPTION("Realtek PHY driver"); MODULE_AUTHOR("Johnson Leung"); @@ -78,6 +79,7 @@ MODULE_LICENSE("GPL"); struct rtl821x_priv { u16 phycr1; u16 phycr2; + bool has_phycr2; }; static int rtl821x_read_page(struct phy_device *phydev) @@ -94,6 +96,7 @@ static int rtl821x_probe(struct phy_device *phydev) { struct device *dev = &phydev->mdio.dev; struct rtl821x_priv *priv; + u32 phy_id = phydev->drv->phy_id; int ret; priv = devm_kzalloc(dev, sizeof(*priv), GFP_KERNEL); @@ -108,13 +111,16 @@ static int rtl821x_probe(struct phy_device *phydev) if (of_property_read_bool(dev->of_node, "realtek,aldps-enable")) priv->phycr1 |= RTL8211F_ALDPS_PLL_OFF | RTL8211F_ALDPS_ENABLE | RTL8211F_ALDPS_XTAL_OFF; - ret = phy_read_paged(phydev, 0xa43, RTL8211F_PHYCR2); - if (ret < 0) - return ret; + priv->has_phycr2 = !(phy_id == RTL_8211FVD_PHYID); + if (priv->has_phycr2) { + ret = phy_read_paged(phydev, 0xa43, RTL8211F_PHYCR2); + if (ret < 0) + return ret; - priv->phycr2 = ret & RTL8211F_CLKOUT_EN; - if (of_property_read_bool(dev->of_node, "realtek,clkout-disable")) - priv->phycr2 &= ~RTL8211F_CLKOUT_EN; + priv->phycr2 = ret & RTL8211F_CLKOUT_EN; + if (of_property_read_bool(dev->of_node, "realtek,clkout-disable")) + priv->phycr2 &= ~RTL8211F_CLKOUT_EN; + } phydev->priv = priv; @@ -400,12 +406,14 @@ static int rtl8211f_config_init(struct phy_device *phydev) val_rxdly ? "enabled" : "disabled"); } - ret = phy_modify_paged(phydev, 0xa43, RTL8211F_PHYCR2, - RTL8211F_CLKOUT_EN, priv->phycr2); - if (ret < 0) { - dev_err(dev, "clkout configuration failed: %pe\n", - ERR_PTR(ret)); - return ret; + if (priv->has_phycr2) { + ret = phy_modify_paged(phydev, 0xa43, RTL8211F_PHYCR2, + RTL8211F_CLKOUT_EN, priv->phycr2); + if (ret < 0) { + dev_err(dev, "clkout configuration failed: %pe\n", + ERR_PTR(ret)); + return ret; + } } return genphy_soft_reset(phydev); @@ -924,6 +932,18 @@ static struct phy_driver realtek_drvs[] = { .read_page = rtl821x_read_page, .write_page = rtl821x_write_page, }, { + PHY_ID_MATCH_EXACT(RTL_8211FVD_PHYID), + .name = "RTL8211F-VD Gigabit Ethernet", + .probe = rtl821x_probe, + .config_init = &rtl8211f_config_init, + .read_status = rtlgen_read_status, + .config_intr = &rtl8211f_config_intr, + .handle_interrupt = rtl8211f_handle_interrupt, + .suspend = genphy_suspend, + .resume = rtl821x_resume, + .read_page = rtl821x_read_page, + .write_page = rtl821x_write_page, + }, { .name = "Generic FE-GE Realtek PHY", .match_phy_device = rtlgen_match_phy_device, .read_status = rtlgen_read_status, diff --git a/drivers/net/phy/sfp-bus.c b/drivers/net/phy/sfp-bus.c index 15aa5ac1ff49..29e3fa86bac3 100644 --- a/drivers/net/phy/sfp-bus.c +++ b/drivers/net/phy/sfp-bus.c @@ -10,12 +10,6 @@ #include "sfp.h" -struct sfp_quirk { - const char *vendor; - const char *part; - void (*modes)(const struct sfp_eeprom_id *id, unsigned long *modes); -}; - /** * struct sfp_bus - internal representation of a sfp bus */ @@ -38,93 +32,6 @@ struct sfp_bus { bool started; }; -static void sfp_quirk_2500basex(const struct sfp_eeprom_id *id, - unsigned long *modes) -{ - phylink_set(modes, 2500baseX_Full); -} - -static void sfp_quirk_ubnt_uf_instant(const struct sfp_eeprom_id *id, - unsigned long *modes) -{ - /* Ubiquiti U-Fiber Instant module claims that support all transceiver - * types including 10G Ethernet which is not truth. So clear all claimed - * modes and set only one mode which module supports: 1000baseX_Full. - */ - phylink_zero(modes); - phylink_set(modes, 1000baseX_Full); -} - -static const struct sfp_quirk sfp_quirks[] = { - { - // Alcatel Lucent G-010S-P can operate at 2500base-X, but - // incorrectly report 2500MBd NRZ in their EEPROM - .vendor = "ALCATELLUCENT", - .part = "G010SP", - .modes = sfp_quirk_2500basex, - }, { - // Alcatel Lucent G-010S-A can operate at 2500base-X, but - // report 3.2GBd NRZ in their EEPROM - .vendor = "ALCATELLUCENT", - .part = "3FE46541AA", - .modes = sfp_quirk_2500basex, - }, { - // Huawei MA5671A can operate at 2500base-X, but report 1.2GBd - // NRZ in their EEPROM - .vendor = "HUAWEI", - .part = "MA5671A", - .modes = sfp_quirk_2500basex, - }, { - // Lantech 8330-262D-E can operate at 2500base-X, but - // incorrectly report 2500MBd NRZ in their EEPROM - .vendor = "Lantech", - .part = "8330-262D-E", - .modes = sfp_quirk_2500basex, - }, { - .vendor = "UBNT", - .part = "UF-INSTANT", - .modes = sfp_quirk_ubnt_uf_instant, - }, -}; - -static size_t sfp_strlen(const char *str, size_t maxlen) -{ - size_t size, i; - - /* Trailing characters should be filled with space chars */ - for (i = 0, size = 0; i < maxlen; i++) - if (str[i] != ' ') - size = i + 1; - - return size; -} - -static bool sfp_match(const char *qs, const char *str, size_t len) -{ - if (!qs) - return true; - if (strlen(qs) != len) - return false; - return !strncmp(qs, str, len); -} - -static const struct sfp_quirk *sfp_lookup_quirk(const struct sfp_eeprom_id *id) -{ - const struct sfp_quirk *q; - unsigned int i; - size_t vs, ps; - - vs = sfp_strlen(id->base.vendor_name, ARRAY_SIZE(id->base.vendor_name)); - ps = sfp_strlen(id->base.vendor_pn, ARRAY_SIZE(id->base.vendor_pn)); - - for (i = 0, q = sfp_quirks; i < ARRAY_SIZE(sfp_quirks); i++, q++) - if (sfp_match(q->vendor, id->base.vendor_name, vs) && - sfp_match(q->part, id->base.vendor_pn, ps)) - return q; - - return NULL; -} - /** * sfp_parse_port() - Parse the EEPROM base ID, setting the port type * @bus: a pointer to the &struct sfp_bus structure for the sfp module @@ -232,12 +139,14 @@ EXPORT_SYMBOL_GPL(sfp_may_have_phy); * @bus: a pointer to the &struct sfp_bus structure for the sfp module * @id: a pointer to the module's &struct sfp_eeprom_id * @support: pointer to an array of unsigned long for the ethtool support mask + * @interfaces: pointer to an array of unsigned long for phy interface modes + * mask * * Parse the EEPROM identification information and derive the supported * ethtool link modes for the module. */ void sfp_parse_support(struct sfp_bus *bus, const struct sfp_eeprom_id *id, - unsigned long *support) + unsigned long *support, unsigned long *interfaces) { unsigned int br_min, br_nom, br_max; __ETHTOOL_DECLARE_LINK_MODE_MASK(modes) = { 0, }; @@ -264,54 +173,81 @@ void sfp_parse_support(struct sfp_bus *bus, const struct sfp_eeprom_id *id, } /* Set ethtool support from the compliance fields. */ - if (id->base.e10g_base_sr) + if (id->base.e10g_base_sr) { phylink_set(modes, 10000baseSR_Full); - if (id->base.e10g_base_lr) + __set_bit(PHY_INTERFACE_MODE_10GBASER, interfaces); + } + if (id->base.e10g_base_lr) { phylink_set(modes, 10000baseLR_Full); - if (id->base.e10g_base_lrm) + __set_bit(PHY_INTERFACE_MODE_10GBASER, interfaces); + } + if (id->base.e10g_base_lrm) { phylink_set(modes, 10000baseLRM_Full); - if (id->base.e10g_base_er) + __set_bit(PHY_INTERFACE_MODE_10GBASER, interfaces); + } + if (id->base.e10g_base_er) { phylink_set(modes, 10000baseER_Full); + __set_bit(PHY_INTERFACE_MODE_10GBASER, interfaces); + } if (id->base.e1000_base_sx || id->base.e1000_base_lx || - id->base.e1000_base_cx) + id->base.e1000_base_cx) { phylink_set(modes, 1000baseX_Full); + __set_bit(PHY_INTERFACE_MODE_1000BASEX, interfaces); + } if (id->base.e1000_base_t) { phylink_set(modes, 1000baseT_Half); phylink_set(modes, 1000baseT_Full); + __set_bit(PHY_INTERFACE_MODE_1000BASEX, interfaces); + __set_bit(PHY_INTERFACE_MODE_SGMII, interfaces); } /* 1000Base-PX or 1000Base-BX10 */ if ((id->base.e_base_px || id->base.e_base_bx10) && - br_min <= 1300 && br_max >= 1200) + br_min <= 1300 && br_max >= 1200) { phylink_set(modes, 1000baseX_Full); + __set_bit(PHY_INTERFACE_MODE_1000BASEX, interfaces); + } /* 100Base-FX, 100Base-LX, 100Base-PX, 100Base-BX10 */ - if (id->base.e100_base_fx || id->base.e100_base_lx) + if (id->base.e100_base_fx || id->base.e100_base_lx) { phylink_set(modes, 100baseFX_Full); - if ((id->base.e_base_px || id->base.e_base_bx10) && br_nom == 100) + __set_bit(PHY_INTERFACE_MODE_100BASEX, interfaces); + } + if ((id->base.e_base_px || id->base.e_base_bx10) && br_nom == 100) { phylink_set(modes, 100baseFX_Full); + __set_bit(PHY_INTERFACE_MODE_100BASEX, interfaces); + } /* For active or passive cables, select the link modes * based on the bit rates and the cable compliance bytes. */ if ((id->base.sfp_ct_passive || id->base.sfp_ct_active) && br_nom) { /* This may look odd, but some manufacturers use 12000MBd */ - if (br_min <= 12000 && br_max >= 10300) + if (br_min <= 12000 && br_max >= 10300) { phylink_set(modes, 10000baseCR_Full); - if (br_min <= 3200 && br_max >= 3100) + __set_bit(PHY_INTERFACE_MODE_10GBASER, interfaces); + } + if (br_min <= 3200 && br_max >= 3100) { phylink_set(modes, 2500baseX_Full); - if (br_min <= 1300 && br_max >= 1200) + __set_bit(PHY_INTERFACE_MODE_2500BASEX, interfaces); + } + if (br_min <= 1300 && br_max >= 1200) { phylink_set(modes, 1000baseX_Full); + __set_bit(PHY_INTERFACE_MODE_1000BASEX, interfaces); + } } if (id->base.sfp_ct_passive) { - if (id->base.passive.sff8431_app_e) + if (id->base.passive.sff8431_app_e) { phylink_set(modes, 10000baseCR_Full); + __set_bit(PHY_INTERFACE_MODE_10GBASER, interfaces); + } } if (id->base.sfp_ct_active) { if (id->base.active.sff8431_app_e || id->base.active.sff8431_lim) { phylink_set(modes, 10000baseCR_Full); + __set_bit(PHY_INTERFACE_MODE_10GBASER, interfaces); } } @@ -336,12 +272,14 @@ void sfp_parse_support(struct sfp_bus *bus, const struct sfp_eeprom_id *id, case SFF8024_ECC_10GBASE_T_SFI: case SFF8024_ECC_10GBASE_T_SR: phylink_set(modes, 10000baseT_Full); + __set_bit(PHY_INTERFACE_MODE_10GBASER, interfaces); break; case SFF8024_ECC_5GBASE_T: phylink_set(modes, 5000baseT_Full); break; case SFF8024_ECC_2_5GBASE_T: phylink_set(modes, 2500baseT_Full); + __set_bit(PHY_INTERFACE_MODE_2500BASEX, interfaces); break; default: dev_warn(bus->sfp_dev, @@ -354,10 +292,14 @@ void sfp_parse_support(struct sfp_bus *bus, const struct sfp_eeprom_id *id, if (id->base.fc_speed_100 || id->base.fc_speed_200 || id->base.fc_speed_400) { - if (id->base.br_nominal >= 31) + if (id->base.br_nominal >= 31) { phylink_set(modes, 2500baseX_Full); - if (id->base.br_nominal >= 12) + __set_bit(PHY_INTERFACE_MODE_2500BASEX, interfaces); + } + if (id->base.br_nominal >= 12) { phylink_set(modes, 1000baseX_Full); + __set_bit(PHY_INTERFACE_MODE_1000BASEX, interfaces); + } } /* If we haven't discovered any modes that this module supports, try @@ -370,14 +312,18 @@ void sfp_parse_support(struct sfp_bus *bus, const struct sfp_eeprom_id *id, * 2500BASE-X, so we allow some slack here. */ if (bitmap_empty(modes, __ETHTOOL_LINK_MODE_MASK_NBITS) && br_nom) { - if (br_min <= 1300 && br_max >= 1200) + if (br_min <= 1300 && br_max >= 1200) { phylink_set(modes, 1000baseX_Full); - if (br_min <= 3200 && br_max >= 2500) + __set_bit(PHY_INTERFACE_MODE_1000BASEX, interfaces); + } + if (br_min <= 3200 && br_max >= 2500) { phylink_set(modes, 2500baseX_Full); + __set_bit(PHY_INTERFACE_MODE_2500BASEX, interfaces); + } } - if (bus->sfp_quirk) - bus->sfp_quirk->modes(id, modes); + if (bus->sfp_quirk && bus->sfp_quirk->modes) + bus->sfp_quirk->modes(id, modes, interfaces); linkmode_or(support, support, modes); @@ -786,12 +732,13 @@ void sfp_link_down(struct sfp_bus *bus) } EXPORT_SYMBOL_GPL(sfp_link_down); -int sfp_module_insert(struct sfp_bus *bus, const struct sfp_eeprom_id *id) +int sfp_module_insert(struct sfp_bus *bus, const struct sfp_eeprom_id *id, + const struct sfp_quirk *quirk) { const struct sfp_upstream_ops *ops = sfp_get_upstream_ops(bus); int ret = 0; - bus->sfp_quirk = sfp_lookup_quirk(id); + bus->sfp_quirk = quirk; if (ops && ops->module_insert) ret = ops->module_insert(bus->upstream, id); diff --git a/drivers/net/phy/sfp.c b/drivers/net/phy/sfp.c index 63f90fe9a4d2..40c9a64c5e30 100644 --- a/drivers/net/phy/sfp.c +++ b/drivers/net/phy/sfp.c @@ -166,6 +166,7 @@ static const enum gpiod_flags gpio_flags[] = { * on board (for a copper SFP) time to initialise. */ #define T_WAIT msecs_to_jiffies(50) +#define T_WAIT_ROLLBALL msecs_to_jiffies(25000) #define T_START_UP msecs_to_jiffies(300) #define T_START_UP_BAD_GPON msecs_to_jiffies(60000) @@ -205,8 +206,11 @@ static const enum gpiod_flags gpio_flags[] = { /* SFP modules appear to always have their PHY configured for bus address * 0x56 (which with mdio-i2c, translates to a PHY address of 22). + * RollBall SFPs access phy via SFP Enhanced Digital Diagnostic Interface + * via address 0x51 (mdio-i2c will use RollBall protocol on this address). */ -#define SFP_PHY_ADDR 22 +#define SFP_PHY_ADDR 22 +#define SFP_PHY_ADDR_ROLLBALL 17 struct sff_data { unsigned int gpios; @@ -218,6 +222,7 @@ struct sfp { struct i2c_adapter *i2c; struct mii_bus *i2c_mii; struct sfp_bus *sfp_bus; + enum mdio_i2c_proto mdio_protocol; struct phy_device *mod_phy; const struct sff_data *type; size_t i2c_block_size; @@ -234,6 +239,7 @@ struct sfp { bool need_poll; struct mutex st_mutex; /* Protects state */ + unsigned int state_hw_mask; unsigned int state_soft_mask; unsigned int state; struct delayed_work poll; @@ -250,8 +256,11 @@ struct sfp { struct sfp_eeprom_id id; unsigned int module_power_mW; unsigned int module_t_start_up; + unsigned int module_t_wait; bool tx_fault_ignore; + const struct sfp_quirk *quirk; + #if IS_ENABLED(CONFIG_HWMON) struct sfp_diag diag; struct delayed_work hwmon_probe; @@ -308,6 +317,136 @@ static const struct of_device_id sfp_of_match[] = { }; MODULE_DEVICE_TABLE(of, sfp_of_match); +static void sfp_fixup_long_startup(struct sfp *sfp) +{ + sfp->module_t_start_up = T_START_UP_BAD_GPON; +} + +static void sfp_fixup_ignore_tx_fault(struct sfp *sfp) +{ + sfp->tx_fault_ignore = true; +} + +static void sfp_fixup_halny_gsfp(struct sfp *sfp) +{ + /* Ignore the TX_FAULT and LOS signals on this module. + * these are possibly used for other purposes on this + * module, e.g. a serial port. + */ + sfp->state_hw_mask &= ~(SFP_F_TX_FAULT | SFP_F_LOS); +} + +static void sfp_fixup_rollball(struct sfp *sfp) +{ + sfp->mdio_protocol = MDIO_I2C_ROLLBALL; + sfp->module_t_wait = T_WAIT_ROLLBALL; +} + +static void sfp_fixup_rollball_cc(struct sfp *sfp) +{ + sfp_fixup_rollball(sfp); + + /* Some RollBall SFPs may have wrong (zero) extended compliance code + * burned in EEPROM. For PHY probing we need the correct one. + */ + sfp->id.base.extended_cc = SFF8024_ECC_10GBASE_T_SFI; +} + +static void sfp_quirk_2500basex(const struct sfp_eeprom_id *id, + unsigned long *modes, + unsigned long *interfaces) +{ + linkmode_set_bit(ETHTOOL_LINK_MODE_2500baseX_Full_BIT, modes); + __set_bit(PHY_INTERFACE_MODE_2500BASEX, interfaces); +} + +static void sfp_quirk_ubnt_uf_instant(const struct sfp_eeprom_id *id, + unsigned long *modes, + unsigned long *interfaces) +{ + /* Ubiquiti U-Fiber Instant module claims that support all transceiver + * types including 10G Ethernet which is not truth. So clear all claimed + * modes and set only one mode which module supports: 1000baseX_Full. + */ + linkmode_zero(modes); + linkmode_set_bit(ETHTOOL_LINK_MODE_1000baseX_Full_BIT, modes); +} + +#define SFP_QUIRK(_v, _p, _m, _f) \ + { .vendor = _v, .part = _p, .modes = _m, .fixup = _f, } +#define SFP_QUIRK_M(_v, _p, _m) SFP_QUIRK(_v, _p, _m, NULL) +#define SFP_QUIRK_F(_v, _p, _f) SFP_QUIRK(_v, _p, NULL, _f) + +static const struct sfp_quirk sfp_quirks[] = { + // Alcatel Lucent G-010S-P can operate at 2500base-X, but incorrectly + // report 2500MBd NRZ in their EEPROM + SFP_QUIRK_M("ALCATELLUCENT", "G010SP", sfp_quirk_2500basex), + + // Alcatel Lucent G-010S-A can operate at 2500base-X, but report 3.2GBd + // NRZ in their EEPROM + SFP_QUIRK("ALCATELLUCENT", "3FE46541AA", sfp_quirk_2500basex, + sfp_fixup_long_startup), + + SFP_QUIRK_F("HALNy", "HL-GSFP", sfp_fixup_halny_gsfp), + + // Huawei MA5671A can operate at 2500base-X, but report 1.2GBd NRZ in + // their EEPROM + SFP_QUIRK("HUAWEI", "MA5671A", sfp_quirk_2500basex, + sfp_fixup_ignore_tx_fault), + + // Lantech 8330-262D-E can operate at 2500base-X, but incorrectly report + // 2500MBd NRZ in their EEPROM + SFP_QUIRK_M("Lantech", "8330-262D-E", sfp_quirk_2500basex), + + SFP_QUIRK_M("UBNT", "UF-INSTANT", sfp_quirk_ubnt_uf_instant), + + SFP_QUIRK_F("OEM", "SFP-10G-T", sfp_fixup_rollball_cc), + SFP_QUIRK_F("OEM", "RTSFP-10", sfp_fixup_rollball_cc), + SFP_QUIRK_F("OEM", "RTSFP-10G", sfp_fixup_rollball_cc), + SFP_QUIRK_F("Turris", "RTSFP-10", sfp_fixup_rollball), + SFP_QUIRK_F("Turris", "RTSFP-10G", sfp_fixup_rollball), +}; + +static size_t sfp_strlen(const char *str, size_t maxlen) +{ + size_t size, i; + + /* Trailing characters should be filled with space chars, but + * some manufacturers can't read SFF-8472 and use NUL. + */ + for (i = 0, size = 0; i < maxlen; i++) + if (str[i] != ' ' && str[i] != '\0') + size = i + 1; + + return size; +} + +static bool sfp_match(const char *qs, const char *str, size_t len) +{ + if (!qs) + return true; + if (strlen(qs) != len) + return false; + return !strncmp(qs, str, len); +} + +static const struct sfp_quirk *sfp_lookup_quirk(const struct sfp_eeprom_id *id) +{ + const struct sfp_quirk *q; + unsigned int i; + size_t vs, ps; + + vs = sfp_strlen(id->base.vendor_name, ARRAY_SIZE(id->base.vendor_name)); + ps = sfp_strlen(id->base.vendor_pn, ARRAY_SIZE(id->base.vendor_pn)); + + for (i = 0, q = sfp_quirks; i < ARRAY_SIZE(sfp_quirks); i++, q++) + if (sfp_match(q->vendor, id->base.vendor_name, vs) && + sfp_match(q->part, id->base.vendor_pn, ps)) + return q; + + return NULL; +} + static unsigned long poll_jiffies; static unsigned int sfp_gpio_get_state(struct sfp *sfp) @@ -419,9 +558,6 @@ static int sfp_i2c_write(struct sfp *sfp, bool a2, u8 dev_addr, void *buf, static int sfp_i2c_configure(struct sfp *sfp, struct i2c_adapter *i2c) { - struct mii_bus *i2c_mii; - int ret; - if (!i2c_check_functionality(i2c, I2C_FUNC_I2C)) return -EINVAL; @@ -429,7 +565,15 @@ static int sfp_i2c_configure(struct sfp *sfp, struct i2c_adapter *i2c) sfp->read = sfp_i2c_read; sfp->write = sfp_i2c_write; - i2c_mii = mdio_i2c_alloc(sfp->dev, i2c); + return 0; +} + +static int sfp_i2c_mdiobus_create(struct sfp *sfp) +{ + struct mii_bus *i2c_mii; + int ret; + + i2c_mii = mdio_i2c_alloc(sfp->dev, sfp->i2c, sfp->mdio_protocol); if (IS_ERR(i2c_mii)) return PTR_ERR(i2c_mii); @@ -447,6 +591,12 @@ static int sfp_i2c_configure(struct sfp *sfp, struct i2c_adapter *i2c) return 0; } +static void sfp_i2c_mdiobus_destroy(struct sfp *sfp) +{ + mdiobus_unregister(sfp->i2c_mii); + sfp->i2c_mii = NULL; +} + /* Interface */ static int sfp_read(struct sfp *sfp, bool a2, u8 addr, void *buf, size_t len) { @@ -499,17 +649,18 @@ static void sfp_soft_set_state(struct sfp *sfp, unsigned int state) static void sfp_soft_start_poll(struct sfp *sfp) { const struct sfp_eeprom_id *id = &sfp->id; + unsigned int mask = 0; sfp->state_soft_mask = 0; - if (id->ext.enhopts & SFP_ENHOPTS_SOFT_TX_DISABLE && - !sfp->gpio[GPIO_TX_DISABLE]) - sfp->state_soft_mask |= SFP_F_TX_DISABLE; - if (id->ext.enhopts & SFP_ENHOPTS_SOFT_TX_FAULT && - !sfp->gpio[GPIO_TX_FAULT]) - sfp->state_soft_mask |= SFP_F_TX_FAULT; - if (id->ext.enhopts & SFP_ENHOPTS_SOFT_RX_LOS && - !sfp->gpio[GPIO_LOS]) - sfp->state_soft_mask |= SFP_F_LOS; + if (id->ext.enhopts & SFP_ENHOPTS_SOFT_TX_DISABLE) + mask |= SFP_F_TX_DISABLE; + if (id->ext.enhopts & SFP_ENHOPTS_SOFT_TX_FAULT) + mask |= SFP_F_TX_FAULT; + if (id->ext.enhopts & SFP_ENHOPTS_SOFT_RX_LOS) + mask |= SFP_F_LOS; + + // Poll the soft state for hardware pins we want to ignore + sfp->state_soft_mask = ~sfp->state_hw_mask & mask; if (sfp->state_soft_mask & (SFP_F_LOS | SFP_F_TX_FAULT) && !sfp->need_poll) @@ -523,10 +674,11 @@ static void sfp_soft_stop_poll(struct sfp *sfp) static unsigned int sfp_get_state(struct sfp *sfp) { - unsigned int state = sfp->get_state(sfp); + unsigned int soft = sfp->state_soft_mask & (SFP_F_LOS | SFP_F_TX_FAULT); + unsigned int state; - if (state & SFP_F_PRESENT && - sfp->state_soft_mask & (SFP_F_LOS | SFP_F_TX_FAULT)) + state = sfp->get_state(sfp) & sfp->state_hw_mask; + if (state & SFP_F_PRESENT && soft) state |= sfp_soft_get_state(sfp); return state; @@ -1195,90 +1347,45 @@ static const struct hwmon_ops sfp_hwmon_ops = { .read_string = sfp_hwmon_read_string, }; -static u32 sfp_hwmon_chip_config[] = { - HWMON_C_REGISTER_TZ, - 0, -}; - -static const struct hwmon_channel_info sfp_hwmon_chip = { - .type = hwmon_chip, - .config = sfp_hwmon_chip_config, -}; - -static u32 sfp_hwmon_temp_config[] = { - HWMON_T_INPUT | - HWMON_T_MAX | HWMON_T_MIN | - HWMON_T_MAX_ALARM | HWMON_T_MIN_ALARM | - HWMON_T_CRIT | HWMON_T_LCRIT | - HWMON_T_CRIT_ALARM | HWMON_T_LCRIT_ALARM | - HWMON_T_LABEL, - 0, -}; - -static const struct hwmon_channel_info sfp_hwmon_temp_channel_info = { - .type = hwmon_temp, - .config = sfp_hwmon_temp_config, -}; - -static u32 sfp_hwmon_vcc_config[] = { - HWMON_I_INPUT | - HWMON_I_MAX | HWMON_I_MIN | - HWMON_I_MAX_ALARM | HWMON_I_MIN_ALARM | - HWMON_I_CRIT | HWMON_I_LCRIT | - HWMON_I_CRIT_ALARM | HWMON_I_LCRIT_ALARM | - HWMON_I_LABEL, - 0, -}; - -static const struct hwmon_channel_info sfp_hwmon_vcc_channel_info = { - .type = hwmon_in, - .config = sfp_hwmon_vcc_config, -}; - -static u32 sfp_hwmon_bias_config[] = { - HWMON_C_INPUT | - HWMON_C_MAX | HWMON_C_MIN | - HWMON_C_MAX_ALARM | HWMON_C_MIN_ALARM | - HWMON_C_CRIT | HWMON_C_LCRIT | - HWMON_C_CRIT_ALARM | HWMON_C_LCRIT_ALARM | - HWMON_C_LABEL, - 0, -}; - -static const struct hwmon_channel_info sfp_hwmon_bias_channel_info = { - .type = hwmon_curr, - .config = sfp_hwmon_bias_config, -}; - -static u32 sfp_hwmon_power_config[] = { - /* Transmit power */ - HWMON_P_INPUT | - HWMON_P_MAX | HWMON_P_MIN | - HWMON_P_MAX_ALARM | HWMON_P_MIN_ALARM | - HWMON_P_CRIT | HWMON_P_LCRIT | - HWMON_P_CRIT_ALARM | HWMON_P_LCRIT_ALARM | - HWMON_P_LABEL, - /* Receive power */ - HWMON_P_INPUT | - HWMON_P_MAX | HWMON_P_MIN | - HWMON_P_MAX_ALARM | HWMON_P_MIN_ALARM | - HWMON_P_CRIT | HWMON_P_LCRIT | - HWMON_P_CRIT_ALARM | HWMON_P_LCRIT_ALARM | - HWMON_P_LABEL, - 0, -}; - -static const struct hwmon_channel_info sfp_hwmon_power_channel_info = { - .type = hwmon_power, - .config = sfp_hwmon_power_config, -}; - static const struct hwmon_channel_info *sfp_hwmon_info[] = { - &sfp_hwmon_chip, - &sfp_hwmon_vcc_channel_info, - &sfp_hwmon_temp_channel_info, - &sfp_hwmon_bias_channel_info, - &sfp_hwmon_power_channel_info, + HWMON_CHANNEL_INFO(chip, + HWMON_C_REGISTER_TZ), + HWMON_CHANNEL_INFO(in, + HWMON_I_INPUT | + HWMON_I_MAX | HWMON_I_MIN | + HWMON_I_MAX_ALARM | HWMON_I_MIN_ALARM | + HWMON_I_CRIT | HWMON_I_LCRIT | + HWMON_I_CRIT_ALARM | HWMON_I_LCRIT_ALARM | + HWMON_I_LABEL), + HWMON_CHANNEL_INFO(temp, + HWMON_T_INPUT | + HWMON_T_MAX | HWMON_T_MIN | + HWMON_T_MAX_ALARM | HWMON_T_MIN_ALARM | + HWMON_T_CRIT | HWMON_T_LCRIT | + HWMON_T_CRIT_ALARM | HWMON_T_LCRIT_ALARM | + HWMON_T_LABEL), + HWMON_CHANNEL_INFO(curr, + HWMON_C_INPUT | + HWMON_C_MAX | HWMON_C_MIN | + HWMON_C_MAX_ALARM | HWMON_C_MIN_ALARM | + HWMON_C_CRIT | HWMON_C_LCRIT | + HWMON_C_CRIT_ALARM | HWMON_C_LCRIT_ALARM | + HWMON_C_LABEL), + HWMON_CHANNEL_INFO(power, + /* Transmit power */ + HWMON_P_INPUT | + HWMON_P_MAX | HWMON_P_MIN | + HWMON_P_MAX_ALARM | HWMON_P_MIN_ALARM | + HWMON_P_CRIT | HWMON_P_LCRIT | + HWMON_P_CRIT_ALARM | HWMON_P_LCRIT_ALARM | + HWMON_P_LABEL, + /* Receive power */ + HWMON_P_INPUT | + HWMON_P_MAX | HWMON_P_MIN | + HWMON_P_MAX_ALARM | HWMON_P_MIN_ALARM | + HWMON_P_CRIT | HWMON_P_LCRIT | + HWMON_P_CRIT_ALARM | HWMON_P_LCRIT_ALARM | + HWMON_P_LABEL), NULL, }; @@ -1505,12 +1612,12 @@ static void sfp_sm_phy_detach(struct sfp *sfp) sfp->mod_phy = NULL; } -static int sfp_sm_probe_phy(struct sfp *sfp, bool is_c45) +static int sfp_sm_probe_phy(struct sfp *sfp, int addr, bool is_c45) { struct phy_device *phy; int err; - phy = get_phy_device(sfp->i2c_mii, SFP_PHY_ADDR, is_c45); + phy = get_phy_device(sfp->i2c_mii, addr, is_c45); if (phy == ERR_PTR(-ENODEV)) return PTR_ERR(phy); if (IS_ERR(phy)) { @@ -1606,6 +1713,14 @@ static void sfp_sm_fault(struct sfp *sfp, unsigned int next_state, bool warn) } } +static int sfp_sm_add_mdio_bus(struct sfp *sfp) +{ + if (sfp->mdio_protocol != MDIO_I2C_NONE) + return sfp_i2c_mdiobus_create(sfp); + + return 0; +} + /* Probe a SFP for a PHY device if the module supports copper - the PHY * normally sits at I2C bus address 0x56, and may either be a clause 22 * or clause 45 PHY. @@ -1621,19 +1736,23 @@ static int sfp_sm_probe_for_phy(struct sfp *sfp) { int err = 0; - switch (sfp->id.base.extended_cc) { - case SFF8024_ECC_10GBASE_T_SFI: - case SFF8024_ECC_10GBASE_T_SR: - case SFF8024_ECC_5GBASE_T: - case SFF8024_ECC_2_5GBASE_T: - err = sfp_sm_probe_phy(sfp, true); + switch (sfp->mdio_protocol) { + case MDIO_I2C_NONE: break; - default: - if (sfp->id.base.e1000_base_t) - err = sfp_sm_probe_phy(sfp, false); + case MDIO_I2C_MARVELL_C22: + err = sfp_sm_probe_phy(sfp, SFP_PHY_ADDR, false); + break; + + case MDIO_I2C_C45: + err = sfp_sm_probe_phy(sfp, SFP_PHY_ADDR, true); + break; + + case MDIO_I2C_ROLLBALL: + err = sfp_sm_probe_phy(sfp, SFP_PHY_ADDR_ROLLBALL, true); break; } + return err; } @@ -1947,17 +2066,33 @@ static int sfp_sm_mod_probe(struct sfp *sfp, bool report) if (ret < 0) return ret; - if (!memcmp(id.base.vendor_name, "ALCATELLUCENT ", 16) && - !memcmp(id.base.vendor_pn, "3FE46541AA ", 16)) - sfp->module_t_start_up = T_START_UP_BAD_GPON; + /* Initialise state bits to use from hardware */ + sfp->state_hw_mask = SFP_F_PRESENT; + if (sfp->gpio[GPIO_TX_DISABLE]) + sfp->state_hw_mask |= SFP_F_TX_DISABLE; + if (sfp->gpio[GPIO_TX_FAULT]) + sfp->state_hw_mask |= SFP_F_TX_FAULT; + if (sfp->gpio[GPIO_LOS]) + sfp->state_hw_mask |= SFP_F_LOS; + + sfp->module_t_start_up = T_START_UP; + sfp->module_t_wait = T_WAIT; + + sfp->tx_fault_ignore = false; + + if (sfp->id.base.extended_cc == SFF8024_ECC_10GBASE_T_SFI || + sfp->id.base.extended_cc == SFF8024_ECC_10GBASE_T_SR || + sfp->id.base.extended_cc == SFF8024_ECC_5GBASE_T || + sfp->id.base.extended_cc == SFF8024_ECC_2_5GBASE_T) + sfp->mdio_protocol = MDIO_I2C_C45; + else if (sfp->id.base.e1000_base_t) + sfp->mdio_protocol = MDIO_I2C_MARVELL_C22; else - sfp->module_t_start_up = T_START_UP; + sfp->mdio_protocol = MDIO_I2C_NONE; - if (!memcmp(id.base.vendor_name, "HUAWEI ", 16) && - !memcmp(id.base.vendor_pn, "MA5671A ", 16)) - sfp->tx_fault_ignore = true; - else - sfp->tx_fault_ignore = false; + sfp->quirk = sfp_lookup_quirk(&id); + if (sfp->quirk && sfp->quirk->fixup) + sfp->quirk->fixup(sfp); return 0; } @@ -2071,7 +2206,8 @@ static void sfp_sm_module(struct sfp *sfp, unsigned int event) break; /* Report the module insertion to the upstream device */ - err = sfp_module_insert(sfp->sfp_bus, &sfp->id); + err = sfp_module_insert(sfp->sfp_bus, &sfp->id, + sfp->quirk); if (err < 0) { sfp_sm_mod_next(sfp, SFP_MOD_ERROR, 0); break; @@ -2130,6 +2266,8 @@ static void sfp_sm_main(struct sfp *sfp, unsigned int event) sfp_module_stop(sfp->sfp_bus); if (sfp->mod_phy) sfp_sm_phy_detach(sfp); + if (sfp->i2c_mii) + sfp_i2c_mdiobus_destroy(sfp); sfp_module_tx_disable(sfp); sfp_soft_stop_poll(sfp); sfp_sm_next(sfp, SFP_S_DOWN, 0); @@ -2153,9 +2291,10 @@ static void sfp_sm_main(struct sfp *sfp, unsigned int event) /* We need to check the TX_FAULT state, which is not defined * while TX_DISABLE is asserted. The earliest we want to do - * anything (such as probe for a PHY) is 50ms. + * anything (such as probe for a PHY) is 50ms (or more on + * specific modules). */ - sfp_sm_next(sfp, SFP_S_WAIT, T_WAIT); + sfp_sm_next(sfp, SFP_S_WAIT, sfp->module_t_wait); break; case SFP_S_WAIT: @@ -2169,8 +2308,8 @@ static void sfp_sm_main(struct sfp *sfp, unsigned int event) * deasserting. */ timeout = sfp->module_t_start_up; - if (timeout > T_WAIT) - timeout -= T_WAIT; + if (timeout > sfp->module_t_wait) + timeout -= sfp->module_t_wait; else timeout = 1; @@ -2192,6 +2331,12 @@ static void sfp_sm_main(struct sfp *sfp, unsigned int event) sfp->sm_fault_retries == N_FAULT_INIT); } else if (event == SFP_E_TIMEOUT || event == SFP_E_TX_CLEAR) { init_done: + /* Create mdiobus and start trying for PHY */ + ret = sfp_sm_add_mdio_bus(sfp); + if (ret < 0) { + sfp_sm_next(sfp, SFP_S_FAIL, 0); + break; + } sfp->sm_phy_retries = R_PHY_RETRY; goto phy_probe; } @@ -2573,6 +2718,8 @@ static int sfp_probe(struct platform_device *pdev) return PTR_ERR(sfp->gpio[i]); } + sfp->state_hw_mask = SFP_F_PRESENT; + sfp->get_state = sfp_gpio_get_state; sfp->set_state = sfp_gpio_set_state; diff --git a/drivers/net/phy/sfp.h b/drivers/net/phy/sfp.h index 27226535c72b..6cf1643214d3 100644 --- a/drivers/net/phy/sfp.h +++ b/drivers/net/phy/sfp.h @@ -6,6 +6,14 @@ struct sfp; +struct sfp_quirk { + const char *vendor; + const char *part; + void (*modes)(const struct sfp_eeprom_id *id, unsigned long *modes, + unsigned long *interfaces); + void (*fixup)(struct sfp *sfp); +}; + struct sfp_socket_ops { void (*attach)(struct sfp *sfp); void (*detach)(struct sfp *sfp); @@ -23,7 +31,8 @@ int sfp_add_phy(struct sfp_bus *bus, struct phy_device *phydev); void sfp_remove_phy(struct sfp_bus *bus); void sfp_link_up(struct sfp_bus *bus); void sfp_link_down(struct sfp_bus *bus); -int sfp_module_insert(struct sfp_bus *bus, const struct sfp_eeprom_id *id); +int sfp_module_insert(struct sfp_bus *bus, const struct sfp_eeprom_id *id, + const struct sfp_quirk *quirk); void sfp_module_remove(struct sfp_bus *bus); int sfp_module_start(struct sfp_bus *bus); void sfp_module_stop(struct sfp_bus *bus); diff --git a/drivers/net/phy/smsc.c b/drivers/net/phy/smsc.c index 69423b8965b3..ac7481ce2fc1 100644 --- a/drivers/net/phy/smsc.c +++ b/drivers/net/phy/smsc.c @@ -46,7 +46,6 @@ static struct smsc_hw_stat smsc_hw_stats[] = { struct smsc_phy_priv { u16 intmask; bool energy_enable; - struct clk *refclk; }; static int smsc_phy_ack_interrupt(struct phy_device *phydev) @@ -285,20 +284,12 @@ static void smsc_get_stats(struct phy_device *phydev, data[i] = smsc_get_stat(phydev, i); } -static void smsc_phy_remove(struct phy_device *phydev) -{ - struct smsc_phy_priv *priv = phydev->priv; - - clk_disable_unprepare(priv->refclk); - clk_put(priv->refclk); -} - static int smsc_phy_probe(struct phy_device *phydev) { struct device *dev = &phydev->mdio.dev; struct device_node *of_node = dev->of_node; struct smsc_phy_priv *priv; - int ret; + struct clk *refclk; priv = devm_kzalloc(dev, sizeof(*priv), GFP_KERNEL); if (!priv) @@ -312,22 +303,12 @@ static int smsc_phy_probe(struct phy_device *phydev) phydev->priv = priv; /* Make clk optional to keep DTB backward compatibility. */ - priv->refclk = clk_get_optional(dev, NULL); - if (IS_ERR(priv->refclk)) - return dev_err_probe(dev, PTR_ERR(priv->refclk), + refclk = devm_clk_get_optional_enabled(dev, NULL); + if (IS_ERR(refclk)) + return dev_err_probe(dev, PTR_ERR(refclk), "Failed to request clock\n"); - ret = clk_prepare_enable(priv->refclk); - if (ret) - return ret; - - ret = clk_set_rate(priv->refclk, 50 * 1000 * 1000); - if (ret) { - clk_disable_unprepare(priv->refclk); - return ret; - } - - return 0; + return clk_set_rate(refclk, 50 * 1000 * 1000); } static struct phy_driver smsc_phy_driver[] = { @@ -429,7 +410,6 @@ static struct phy_driver smsc_phy_driver[] = { /* PHY_BASIC_FEATURES */ .probe = smsc_phy_probe, - .remove = smsc_phy_remove, /* basic functions */ .read_status = lan87xx_read_status, diff --git a/drivers/net/phy/spi_ks8995.c b/drivers/net/phy/spi_ks8995.c index ff37f8ba6758..d4202d40d47a 100644 --- a/drivers/net/phy/spi_ks8995.c +++ b/drivers/net/phy/spi_ks8995.c @@ -17,7 +17,6 @@ #include <linux/device.h> #include <linux/gpio/consumer.h> #include <linux/of.h> -#include <linux/of_gpio.h> #include <linux/spi/spi.h> @@ -137,15 +136,10 @@ static const struct ks8995_chip_params ks8995_chip[] = { }, }; -struct ks8995_pdata { - int reset_gpio; - enum of_gpio_flags reset_gpio_flags; -}; - struct ks8995_switch { struct spi_device *spi; struct mutex lock; - struct ks8995_pdata *pdata; + struct gpio_desc *reset_gpio; struct bin_attribute regs_attr; const struct ks8995_chip_params *chip; int revision_id; @@ -401,24 +395,6 @@ err_out: return err; } -/* ks8995_parse_dt - setup platform data from devicetree - * @ks: pointer to switch instance - * - * Parses supported DT properties and sets up platform data - * accordingly. - */ -static void ks8995_parse_dt(struct ks8995_switch *ks) -{ - struct device_node *np = ks->spi->dev.of_node; - struct ks8995_pdata *pdata = ks->pdata; - - if (!np) - return; - - pdata->reset_gpio = of_get_named_gpio_flags(np, "reset-gpios", 0, - &pdata->reset_gpio_flags); -} - static const struct bin_attribute ks8995_registers_attr = { .attr = { .name = "registers", @@ -449,38 +425,22 @@ static int ks8995_probe(struct spi_device *spi) ks->spi = spi; ks->chip = &ks8995_chip[variant]; - if (ks->spi->dev.of_node) { - ks->pdata = devm_kzalloc(&spi->dev, sizeof(*ks->pdata), - GFP_KERNEL); - if (!ks->pdata) - return -ENOMEM; - - ks->pdata->reset_gpio = -1; - - ks8995_parse_dt(ks); + ks->reset_gpio = devm_gpiod_get_optional(&spi->dev, "reset", + GPIOD_OUT_HIGH); + err = PTR_ERR_OR_ZERO(ks->reset_gpio); + if (err) { + dev_err(&spi->dev, + "failed to get reset gpio: %d\n", err); + return err; } - if (!ks->pdata) - ks->pdata = spi->dev.platform_data; + err = gpiod_set_consumer_name(ks->reset_gpio, "switch-reset"); + if (err) + return err; /* de-assert switch reset */ - if (ks->pdata && gpio_is_valid(ks->pdata->reset_gpio)) { - unsigned long flags; - - flags = (ks->pdata->reset_gpio_flags == OF_GPIO_ACTIVE_LOW ? - GPIOF_ACTIVE_LOW : 0); - - err = devm_gpio_request_one(&spi->dev, - ks->pdata->reset_gpio, - flags, "switch-reset"); - if (err) { - dev_err(&spi->dev, - "failed to get reset-gpios: %d\n", err); - return -EIO; - } - - gpiod_set_value(gpio_to_desc(ks->pdata->reset_gpio), 0); - } + /* FIXME: this likely requires a delay */ + gpiod_set_value_cansleep(ks->reset_gpio, 0); spi_set_drvdata(spi, ks); @@ -524,8 +484,7 @@ static void ks8995_remove(struct spi_device *spi) sysfs_remove_bin_file(&spi->dev.kobj, &ks->regs_attr); /* assert reset */ - if (ks->pdata && gpio_is_valid(ks->pdata->reset_gpio)) - gpiod_set_value(gpio_to_desc(ks->pdata->reset_gpio), 1); + gpiod_set_value_cansleep(ks->reset_gpio, 1); } /* ------------------------------------------------------------------------ */ diff --git a/drivers/net/pse-pd/Kconfig b/drivers/net/pse-pd/Kconfig new file mode 100644 index 000000000000..73d163704068 --- /dev/null +++ b/drivers/net/pse-pd/Kconfig @@ -0,0 +1,22 @@ +# SPDX-License-Identifier: GPL-2.0-only +# +# Ethernet Power Sourcing Equipment drivers +# + +menuconfig PSE_CONTROLLER + bool "Ethernet Power Sourcing Equipment Support" + help + Generic Power Sourcing Equipment Controller support. + + If unsure, say no. + +if PSE_CONTROLLER + +config PSE_REGULATOR + tristate "Regulator based PSE controller" + help + This module provides support for simple regulator based Ethernet Power + Sourcing Equipment without automatic classification support. For + example for basic implementation of PoDL (802.3bu) specification. + +endif diff --git a/drivers/net/pse-pd/Makefile b/drivers/net/pse-pd/Makefile new file mode 100644 index 000000000000..1b8aa4c70f0b --- /dev/null +++ b/drivers/net/pse-pd/Makefile @@ -0,0 +1,6 @@ +# SPDX-License-Identifier: GPL-2.0-only +# Makefile for Linux PSE drivers + +obj-$(CONFIG_PSE_CONTROLLER) += pse_core.o + +obj-$(CONFIG_PSE_REGULATOR) += pse_regulator.o diff --git a/drivers/net/pse-pd/pse_core.c b/drivers/net/pse-pd/pse_core.c new file mode 100644 index 000000000000..146b81f08a89 --- /dev/null +++ b/drivers/net/pse-pd/pse_core.c @@ -0,0 +1,314 @@ +// SPDX-License-Identifier: GPL-2.0-only +// +// Framework for Ethernet Power Sourcing Equipment +// +// Copyright (c) 2022 Pengutronix, Oleksij Rempel <kernel@pengutronix.de> +// + +#include <linux/device.h> +#include <linux/of.h> +#include <linux/pse-pd/pse.h> + +static DEFINE_MUTEX(pse_list_mutex); +static LIST_HEAD(pse_controller_list); + +/** + * struct pse_control - a PSE control + * @pcdev: a pointer to the PSE controller device + * this PSE control belongs to + * @list: list entry for the pcdev's PSE controller list + * @id: ID of the PSE line in the PSE controller device + * @refcnt: Number of gets of this pse_control + */ +struct pse_control { + struct pse_controller_dev *pcdev; + struct list_head list; + unsigned int id; + struct kref refcnt; +}; + +/** + * of_pse_zero_xlate - dummy function for controllers with one only control + * @pcdev: a pointer to the PSE controller device + * @pse_spec: PSE line specifier as found in the device tree + * + * This static translation function is used by default if of_xlate in + * :c:type:`pse_controller_dev` is not set. It is useful for all PSE + * controllers with #pse-cells = <0>. + */ +static int of_pse_zero_xlate(struct pse_controller_dev *pcdev, + const struct of_phandle_args *pse_spec) +{ + return 0; +} + +/** + * of_pse_simple_xlate - translate pse_spec to the PSE line number + * @pcdev: a pointer to the PSE controller device + * @pse_spec: PSE line specifier as found in the device tree + * + * This static translation function is used by default if of_xlate in + * :c:type:`pse_controller_dev` is not set. It is useful for all PSE + * controllers with 1:1 mapping, where PSE lines can be indexed by number + * without gaps. + */ +static int of_pse_simple_xlate(struct pse_controller_dev *pcdev, + const struct of_phandle_args *pse_spec) +{ + if (pse_spec->args[0] >= pcdev->nr_lines) + return -EINVAL; + + return pse_spec->args[0]; +} + +/** + * pse_controller_register - register a PSE controller device + * @pcdev: a pointer to the initialized PSE controller device + */ +int pse_controller_register(struct pse_controller_dev *pcdev) +{ + if (!pcdev->of_xlate) { + if (pcdev->of_pse_n_cells == 0) + pcdev->of_xlate = of_pse_zero_xlate; + else if (pcdev->of_pse_n_cells == 1) + pcdev->of_xlate = of_pse_simple_xlate; + } + + mutex_init(&pcdev->lock); + INIT_LIST_HEAD(&pcdev->pse_control_head); + + mutex_lock(&pse_list_mutex); + list_add(&pcdev->list, &pse_controller_list); + mutex_unlock(&pse_list_mutex); + + return 0; +} +EXPORT_SYMBOL_GPL(pse_controller_register); + +/** + * pse_controller_unregister - unregister a PSE controller device + * @pcdev: a pointer to the PSE controller device + */ +void pse_controller_unregister(struct pse_controller_dev *pcdev) +{ + mutex_lock(&pse_list_mutex); + list_del(&pcdev->list); + mutex_unlock(&pse_list_mutex); +} +EXPORT_SYMBOL_GPL(pse_controller_unregister); + +static void devm_pse_controller_release(struct device *dev, void *res) +{ + pse_controller_unregister(*(struct pse_controller_dev **)res); +} + +/** + * devm_pse_controller_register - resource managed pse_controller_register() + * @dev: device that is registering this PSE controller + * @pcdev: a pointer to the initialized PSE controller device + * + * Managed pse_controller_register(). For PSE controllers registered by + * this function, pse_controller_unregister() is automatically called on + * driver detach. See pse_controller_register() for more information. + */ +int devm_pse_controller_register(struct device *dev, + struct pse_controller_dev *pcdev) +{ + struct pse_controller_dev **pcdevp; + int ret; + + pcdevp = devres_alloc(devm_pse_controller_release, sizeof(*pcdevp), + GFP_KERNEL); + if (!pcdevp) + return -ENOMEM; + + ret = pse_controller_register(pcdev); + if (ret) { + devres_free(pcdevp); + return ret; + } + + *pcdevp = pcdev; + devres_add(dev, pcdevp); + + return 0; +} +EXPORT_SYMBOL_GPL(devm_pse_controller_register); + +/* PSE control section */ + +static void __pse_control_release(struct kref *kref) +{ + struct pse_control *psec = container_of(kref, struct pse_control, + refcnt); + + lockdep_assert_held(&pse_list_mutex); + + module_put(psec->pcdev->owner); + + list_del(&psec->list); + kfree(psec); +} + +static void __pse_control_put_internal(struct pse_control *psec) +{ + lockdep_assert_held(&pse_list_mutex); + + kref_put(&psec->refcnt, __pse_control_release); +} + +/** + * pse_control_put - free the PSE control + * @psec: PSE control pointer + */ +void pse_control_put(struct pse_control *psec) +{ + if (IS_ERR_OR_NULL(psec)) + return; + + mutex_lock(&pse_list_mutex); + __pse_control_put_internal(psec); + mutex_unlock(&pse_list_mutex); +} +EXPORT_SYMBOL_GPL(pse_control_put); + +static struct pse_control * +pse_control_get_internal(struct pse_controller_dev *pcdev, unsigned int index) +{ + struct pse_control *psec; + + lockdep_assert_held(&pse_list_mutex); + + list_for_each_entry(psec, &pcdev->pse_control_head, list) { + if (psec->id == index) { + kref_get(&psec->refcnt); + return psec; + } + } + + psec = kzalloc(sizeof(*psec), GFP_KERNEL); + if (!psec) + return ERR_PTR(-ENOMEM); + + if (!try_module_get(pcdev->owner)) { + kfree(psec); + return ERR_PTR(-ENODEV); + } + + psec->pcdev = pcdev; + list_add(&psec->list, &pcdev->pse_control_head); + psec->id = index; + kref_init(&psec->refcnt); + + return psec; +} + +struct pse_control * +of_pse_control_get(struct device_node *node) +{ + struct pse_controller_dev *r, *pcdev; + struct of_phandle_args args; + struct pse_control *psec; + int psec_id; + int ret; + + if (!node) + return ERR_PTR(-EINVAL); + + ret = of_parse_phandle_with_args(node, "pses", "#pse-cells", 0, &args); + if (ret) + return ERR_PTR(ret); + + mutex_lock(&pse_list_mutex); + pcdev = NULL; + list_for_each_entry(r, &pse_controller_list, list) { + if (args.np == r->dev->of_node) { + pcdev = r; + break; + } + } + + if (!pcdev) { + psec = ERR_PTR(-EPROBE_DEFER); + goto out; + } + + if (WARN_ON(args.args_count != pcdev->of_pse_n_cells)) { + psec = ERR_PTR(-EINVAL); + goto out; + } + + psec_id = pcdev->of_xlate(pcdev, &args); + if (psec_id < 0) { + psec = ERR_PTR(psec_id); + goto out; + } + + /* pse_list_mutex also protects the pcdev's pse_control list */ + psec = pse_control_get_internal(pcdev, psec_id); + +out: + mutex_unlock(&pse_list_mutex); + of_node_put(args.np); + + return psec; +} +EXPORT_SYMBOL_GPL(of_pse_control_get); + +/** + * pse_ethtool_get_status - get status of PSE control + * @psec: PSE control pointer + * @extack: extack for reporting useful error messages + * @status: struct to store PSE status + */ +int pse_ethtool_get_status(struct pse_control *psec, + struct netlink_ext_ack *extack, + struct pse_control_status *status) +{ + const struct pse_controller_ops *ops; + int err; + + ops = psec->pcdev->ops; + + if (!ops->ethtool_get_status) { + NL_SET_ERR_MSG(extack, + "PSE driver does not support status report"); + return -EOPNOTSUPP; + } + + mutex_lock(&psec->pcdev->lock); + err = ops->ethtool_get_status(psec->pcdev, psec->id, extack, status); + mutex_unlock(&psec->pcdev->lock); + + return err; +} +EXPORT_SYMBOL_GPL(pse_ethtool_get_status); + +/** + * pse_ethtool_set_config - set PSE control configuration + * @psec: PSE control pointer + * @extack: extack for reporting useful error messages + * @config: Configuration of the test to run + */ +int pse_ethtool_set_config(struct pse_control *psec, + struct netlink_ext_ack *extack, + const struct pse_control_config *config) +{ + const struct pse_controller_ops *ops; + int err; + + ops = psec->pcdev->ops; + + if (!ops->ethtool_set_config) { + NL_SET_ERR_MSG(extack, + "PSE driver does not configuration"); + return -EOPNOTSUPP; + } + + mutex_lock(&psec->pcdev->lock); + err = ops->ethtool_set_config(psec->pcdev, psec->id, extack, config); + mutex_unlock(&psec->pcdev->lock); + + return err; +} +EXPORT_SYMBOL_GPL(pse_ethtool_set_config); diff --git a/drivers/net/pse-pd/pse_regulator.c b/drivers/net/pse-pd/pse_regulator.c new file mode 100644 index 000000000000..e2bf8306ca90 --- /dev/null +++ b/drivers/net/pse-pd/pse_regulator.c @@ -0,0 +1,147 @@ +// SPDX-License-Identifier: GPL-2.0-only +// +// Driver for the regulator based Ethernet Power Sourcing Equipment, without +// auto classification support. +// +// Copyright (c) 2022 Pengutronix, Oleksij Rempel <kernel@pengutronix.de> +// + +#include <linux/module.h> +#include <linux/of.h> +#include <linux/platform_device.h> +#include <linux/pse-pd/pse.h> +#include <linux/regulator/consumer.h> + +struct pse_reg_priv { + struct pse_controller_dev pcdev; + struct regulator *ps; /*power source */ + enum ethtool_podl_pse_admin_state admin_state; +}; + +static struct pse_reg_priv *to_pse_reg(struct pse_controller_dev *pcdev) +{ + return container_of(pcdev, struct pse_reg_priv, pcdev); +} + +static int +pse_reg_ethtool_set_config(struct pse_controller_dev *pcdev, unsigned long id, + struct netlink_ext_ack *extack, + const struct pse_control_config *config) +{ + struct pse_reg_priv *priv = to_pse_reg(pcdev); + int ret; + + if (priv->admin_state == config->admin_cotrol) + return 0; + + switch (config->admin_cotrol) { + case ETHTOOL_PODL_PSE_ADMIN_STATE_ENABLED: + ret = regulator_enable(priv->ps); + break; + case ETHTOOL_PODL_PSE_ADMIN_STATE_DISABLED: + ret = regulator_disable(priv->ps); + break; + default: + dev_err(pcdev->dev, "Unknown admin state %i\n", + config->admin_cotrol); + ret = -ENOTSUPP; + } + + if (ret) + return ret; + + priv->admin_state = config->admin_cotrol; + + return 0; +} + +static int +pse_reg_ethtool_get_status(struct pse_controller_dev *pcdev, unsigned long id, + struct netlink_ext_ack *extack, + struct pse_control_status *status) +{ + struct pse_reg_priv *priv = to_pse_reg(pcdev); + int ret; + + ret = regulator_is_enabled(priv->ps); + if (ret < 0) + return ret; + + if (!ret) + status->podl_pw_status = ETHTOOL_PODL_PSE_PW_D_STATUS_DISABLED; + else + status->podl_pw_status = + ETHTOOL_PODL_PSE_PW_D_STATUS_DELIVERING; + + status->podl_admin_state = priv->admin_state; + + return 0; +} + +static const struct pse_controller_ops pse_reg_ops = { + .ethtool_get_status = pse_reg_ethtool_get_status, + .ethtool_set_config = pse_reg_ethtool_set_config, +}; + +static int +pse_reg_probe(struct platform_device *pdev) +{ + struct device *dev = &pdev->dev; + struct pse_reg_priv *priv; + int ret; + + priv = devm_kzalloc(&pdev->dev, sizeof(*priv), GFP_KERNEL); + if (!priv) + return -ENOMEM; + + if (!pdev->dev.of_node) + return -ENOENT; + + priv->ps = devm_regulator_get_exclusive(dev, "pse"); + if (IS_ERR(priv->ps)) + return dev_err_probe(dev, PTR_ERR(priv->ps), + "failed to get PSE regulator.\n"); + + platform_set_drvdata(pdev, priv); + + ret = regulator_is_enabled(priv->ps); + if (ret < 0) + return ret; + + if (ret) + priv->admin_state = ETHTOOL_PODL_PSE_ADMIN_STATE_ENABLED; + else + priv->admin_state = ETHTOOL_PODL_PSE_ADMIN_STATE_DISABLED; + + priv->pcdev.owner = THIS_MODULE; + priv->pcdev.ops = &pse_reg_ops; + priv->pcdev.dev = dev; + ret = devm_pse_controller_register(dev, &priv->pcdev); + if (ret) { + dev_err(dev, "failed to register PSE controller (%pe)\n", + ERR_PTR(ret)); + return ret; + } + + return 0; +} + +static const __maybe_unused struct of_device_id pse_reg_of_match[] = { + { .compatible = "podl-pse-regulator", }, + { }, +}; +MODULE_DEVICE_TABLE(of, pse_reg_of_match); + +static struct platform_driver pse_reg_driver = { + .probe = pse_reg_probe, + .driver = { + .name = "PSE regulator", + .of_match_table = of_match_ptr(pse_reg_of_match), + }, +}; +module_platform_driver(pse_reg_driver); + +MODULE_AUTHOR("Oleksij Rempel <kernel@pengutronix.de>"); +MODULE_DESCRIPTION("regulator based Ethernet Power Sourcing Equipment"); +MODULE_LICENSE("GPL v2"); +MODULE_ALIAS("platform:pse-regulator"); diff --git a/drivers/net/rionet.c b/drivers/net/rionet.c index 39e61e07e489..fbcb9d05da64 100644 --- a/drivers/net/rionet.c +++ b/drivers/net/rionet.c @@ -443,10 +443,10 @@ static void rionet_get_drvinfo(struct net_device *ndev, { struct rionet_private *rnet = netdev_priv(ndev); - strlcpy(info->driver, DRV_NAME, sizeof(info->driver)); - strlcpy(info->version, DRV_VERSION, sizeof(info->version)); - strlcpy(info->fw_version, "n/a", sizeof(info->fw_version)); - strlcpy(info->bus_info, rnet->mport->name, sizeof(info->bus_info)); + strscpy(info->driver, DRV_NAME, sizeof(info->driver)); + strscpy(info->version, DRV_VERSION, sizeof(info->version)); + strscpy(info->fw_version, "n/a", sizeof(info->fw_version)); + strscpy(info->bus_info, rnet->mport->name, sizeof(info->bus_info)); } static u32 rionet_get_msglevel(struct net_device *ndev) diff --git a/drivers/net/team/team.c b/drivers/net/team/team.c index 154a3c0a6dfd..62ade69295a9 100644 --- a/drivers/net/team/team.c +++ b/drivers/net/team/team.c @@ -2082,8 +2082,8 @@ static const struct net_device_ops team_netdev_ops = { static void team_ethtool_get_drvinfo(struct net_device *dev, struct ethtool_drvinfo *drvinfo) { - strlcpy(drvinfo->driver, DRV_NAME, sizeof(drvinfo->driver)); - strlcpy(drvinfo->version, UTS_RELEASE, sizeof(drvinfo->version)); + strscpy(drvinfo->driver, DRV_NAME, sizeof(drvinfo->driver)); + strscpy(drvinfo->version, UTS_RELEASE, sizeof(drvinfo->version)); } static int team_ethtool_get_link_ksettings(struct net_device *dev, @@ -2852,6 +2852,7 @@ static struct genl_family team_nl_family __ro_after_init = { .module = THIS_MODULE, .small_ops = team_nl_ops, .n_small_ops = ARRAY_SIZE(team_nl_ops), + .resv_start_op = TEAM_CMD_PORT_LIST_GET + 1, .mcgrps = team_nl_mcgrps, .n_mcgrps = ARRAY_SIZE(team_nl_mcgrps), }; diff --git a/drivers/net/thunderbolt.c b/drivers/net/thunderbolt.c index ff5d0e98a088..83fcaeb2ac5e 100644 --- a/drivers/net/thunderbolt.c +++ b/drivers/net/thunderbolt.c @@ -1,6 +1,7 @@ // SPDX-License-Identifier: GPL-2.0 /* - * Networking over Thunderbolt cable using Apple ThunderboltIP protocol + * Networking over Thunderbolt/USB4 cables using USB4NET protocol + * (formerly Apple ThunderboltIP). * * Copyright (C) 2017, Intel Corporation * Authors: Amir Levy <amir.jer.levy@intel.com> @@ -30,6 +31,7 @@ #define TBNET_RING_SIZE 256 #define TBNET_LOGIN_RETRIES 60 #define TBNET_LOGOUT_RETRIES 10 +#define TBNET_E2E BIT(0) #define TBNET_MATCH_FRAGS_ID BIT(1) #define TBNET_64K_FRAMES BIT(2) #define TBNET_MAX_MTU SZ_64K @@ -209,6 +211,10 @@ static const uuid_t tbnet_svc_uuid = static struct tb_property_dir *tbnet_dir; +static bool tbnet_e2e = true; +module_param_named(e2e, tbnet_e2e, bool, 0444); +MODULE_PARM_DESC(e2e, "USB4NET full end-to-end flow control (default: true)"); + static void tbnet_fill_header(struct thunderbolt_ip_header *hdr, u64 route, u8 sequence, const uuid_t *initiator_uuid, const uuid_t *target_uuid, enum thunderbolt_ip_type type, size_t size, u32 command_id) @@ -612,18 +618,13 @@ static void tbnet_connected_work(struct work_struct *work) return; } - /* Both logins successful so enable the high-speed DMA paths and - * start the network device queue. + /* Both logins successful so enable the rings, high-speed DMA + * paths and start the network device queue. + * + * Note we enable the DMA paths last to make sure we have primed + * the Rx ring before any incoming packets are allowed to + * arrive. */ - ret = tb_xdomain_enable_paths(net->xd, net->local_transmit_path, - net->rx_ring.ring->hop, - net->remote_transmit_path, - net->tx_ring.ring->hop); - if (ret) { - netdev_err(net->dev, "failed to enable DMA paths\n"); - return; - } - tb_ring_start(net->tx_ring.ring); tb_ring_start(net->rx_ring.ring); @@ -635,10 +636,21 @@ static void tbnet_connected_work(struct work_struct *work) if (ret) goto err_free_rx_buffers; + ret = tb_xdomain_enable_paths(net->xd, net->local_transmit_path, + net->rx_ring.ring->hop, + net->remote_transmit_path, + net->tx_ring.ring->hop); + if (ret) { + netdev_err(net->dev, "failed to enable DMA paths\n"); + goto err_free_tx_buffers; + } + netif_carrier_on(net->dev); netif_start_queue(net->dev); return; +err_free_tx_buffers: + tbnet_free_buffers(&net->tx_ring); err_free_rx_buffers: tbnet_free_buffers(&net->rx_ring); err_stop_rings: @@ -867,6 +879,7 @@ static int tbnet_open(struct net_device *dev) struct tb_xdomain *xd = net->xd; u16 sof_mask, eof_mask; struct tb_ring *ring; + unsigned int flags; int hopid; netif_carrier_off(dev); @@ -891,9 +904,14 @@ static int tbnet_open(struct net_device *dev) sof_mask = BIT(TBIP_PDF_FRAME_START); eof_mask = BIT(TBIP_PDF_FRAME_END); - ring = tb_ring_alloc_rx(xd->tb->nhi, -1, TBNET_RING_SIZE, - RING_FLAG_FRAME, 0, sof_mask, eof_mask, - tbnet_start_poll, net); + flags = RING_FLAG_FRAME; + /* Only enable full E2E if the other end supports it too */ + if (tbnet_e2e && net->svc->prtcstns & TBNET_E2E) + flags |= RING_FLAG_E2E; + + ring = tb_ring_alloc_rx(xd->tb->nhi, -1, TBNET_RING_SIZE, flags, + net->tx_ring.ring->hop, sof_mask, + eof_mask, tbnet_start_poll, net); if (!ring) { netdev_err(dev, "failed to allocate Rx ring\n"); tb_ring_free(net->tx_ring.ring); @@ -1264,7 +1282,7 @@ static int tbnet_probe(struct tb_service *svc, const struct tb_service_id *id) dev->features = dev->hw_features | NETIF_F_HIGHDMA; dev->hard_header_len += sizeof(struct thunderbolt_ip_frame_header); - netif_napi_add(dev, &net->napi, tbnet_poll, NAPI_POLL_WEIGHT); + netif_napi_add(dev, &net->napi, tbnet_poll); /* MTU range: 68 - 65522 */ dev->min_mtu = ETH_MIN_MTU; @@ -1356,6 +1374,7 @@ static struct tb_service_driver tbnet_driver = { static int __init tbnet_init(void) { + unsigned int flags; int ret; tbnet_dir = tb_property_create_dir(&tbnet_dir_uuid); @@ -1365,12 +1384,11 @@ static int __init tbnet_init(void) tb_property_add_immediate(tbnet_dir, "prtcid", 1); tb_property_add_immediate(tbnet_dir, "prtcvers", 1); tb_property_add_immediate(tbnet_dir, "prtcrevs", 1); - /* Currently only announce support for match frags ID (bit 1). Bit 0 - * is reserved for full E2E flow control which we do not support at - * the moment. - */ - tb_property_add_immediate(tbnet_dir, "prtcstns", - TBNET_MATCH_FRAGS_ID | TBNET_64K_FRAMES); + + flags = TBNET_MATCH_FRAGS_ID | TBNET_64K_FRAMES; + if (tbnet_e2e) + flags |= TBNET_E2E; + tb_property_add_immediate(tbnet_dir, "prtcstns", flags); ret = tb_register_property_dir("network", tbnet_dir); if (ret) { @@ -1393,5 +1411,5 @@ module_exit(tbnet_exit); MODULE_AUTHOR("Amir Levy <amir.jer.levy@intel.com>"); MODULE_AUTHOR("Michael Jamet <michael.jamet@intel.com>"); MODULE_AUTHOR("Mika Westerberg <mika.westerberg@linux.intel.com>"); -MODULE_DESCRIPTION("Thunderbolt network driver"); +MODULE_DESCRIPTION("Thunderbolt/USB4 network driver"); MODULE_LICENSE("GPL v2"); diff --git a/drivers/net/tun.c b/drivers/net/tun.c index db736b944016..27c6d235cbda 100644 --- a/drivers/net/tun.c +++ b/drivers/net/tun.c @@ -2664,7 +2664,7 @@ static ssize_t tun_flags_show(struct device *dev, struct device_attribute *attr, char *buf) { struct tun_struct *tun = netdev_priv(to_net_dev(dev)); - return sprintf(buf, "0x%x\n", tun_flags(tun)); + return sysfs_emit(buf, "0x%x\n", tun_flags(tun)); } static ssize_t owner_show(struct device *dev, struct device_attribute *attr, @@ -2672,9 +2672,9 @@ static ssize_t owner_show(struct device *dev, struct device_attribute *attr, { struct tun_struct *tun = netdev_priv(to_net_dev(dev)); return uid_valid(tun->owner)? - sprintf(buf, "%u\n", - from_kuid_munged(current_user_ns(), tun->owner)): - sprintf(buf, "-1\n"); + sysfs_emit(buf, "%u\n", + from_kuid_munged(current_user_ns(), tun->owner)) : + sysfs_emit(buf, "-1\n"); } static ssize_t group_show(struct device *dev, struct device_attribute *attr, @@ -2682,9 +2682,9 @@ static ssize_t group_show(struct device *dev, struct device_attribute *attr, { struct tun_struct *tun = netdev_priv(to_net_dev(dev)); return gid_valid(tun->group) ? - sprintf(buf, "%u\n", - from_kgid_munged(current_user_ns(), tun->group)): - sprintf(buf, "-1\n"); + sysfs_emit(buf, "%u\n", + from_kgid_munged(current_user_ns(), tun->group)) : + sysfs_emit(buf, "-1\n"); } static DEVICE_ATTR_RO(tun_flags); @@ -3543,15 +3543,15 @@ static void tun_get_drvinfo(struct net_device *dev, struct ethtool_drvinfo *info { struct tun_struct *tun = netdev_priv(dev); - strlcpy(info->driver, DRV_NAME, sizeof(info->driver)); - strlcpy(info->version, DRV_VERSION, sizeof(info->version)); + strscpy(info->driver, DRV_NAME, sizeof(info->driver)); + strscpy(info->version, DRV_VERSION, sizeof(info->version)); switch (tun->flags & TUN_TYPE_MASK) { case IFF_TUN: - strlcpy(info->bus_info, "tun", sizeof(info->bus_info)); + strscpy(info->bus_info, "tun", sizeof(info->bus_info)); break; case IFF_TAP: - strlcpy(info->bus_info, "tap", sizeof(info->bus_info)); + strscpy(info->bus_info, "tap", sizeof(info->bus_info)); break; } } diff --git a/drivers/net/usb/Kconfig b/drivers/net/usb/Kconfig index 76659c1c525a..4402eedb3d1a 100644 --- a/drivers/net/usb/Kconfig +++ b/drivers/net/usb/Kconfig @@ -168,7 +168,7 @@ config USB_NET_AX8817X tristate "ASIX AX88xxx Based USB 2.0 Ethernet Adapters" depends on USB_USBNET select CRC32 - select PHYLIB + select PHYLINK select AX88796B_PHY imply NET_SELFTESTS default y diff --git a/drivers/net/usb/aqc111.c b/drivers/net/usb/aqc111.c index 3020e81159d0..a017e9de2119 100644 --- a/drivers/net/usb/aqc111.c +++ b/drivers/net/usb/aqc111.c @@ -201,7 +201,7 @@ static void aqc111_get_drvinfo(struct net_device *net, /* Inherit standard device info */ usbnet_get_drvinfo(net, info); - strlcpy(info->driver, DRIVER_NAME, sizeof(info->driver)); + strscpy(info->driver, DRIVER_NAME, sizeof(info->driver)); snprintf(info->fw_version, sizeof(info->fw_version), "%u.%u.%u", aqc111_data->fw_ver.major, aqc111_data->fw_ver.minor, diff --git a/drivers/net/usb/asix.h b/drivers/net/usb/asix.h index 21c1ca275cc4..74162190bccc 100644 --- a/drivers/net/usb/asix.h +++ b/drivers/net/usb/asix.h @@ -27,6 +27,7 @@ #include <linux/if_vlan.h> #include <linux/phy.h> #include <net/selftests.h> +#include <linux/phylink.h> #define DRIVER_VERSION "22-Dec-2011" #define DRIVER_NAME "asix" @@ -185,6 +186,8 @@ struct asix_common_private { struct mii_bus *mdio; struct phy_device *phydev; struct phy_device *phydev_int; + struct phylink *phylink; + struct phylink_config phylink_config; u16 phy_addr; bool embd_phy; u8 chipcode; diff --git a/drivers/net/usb/asix_common.c b/drivers/net/usb/asix_common.c index 9ea91c3ff045..72ffc89b477a 100644 --- a/drivers/net/usb/asix_common.c +++ b/drivers/net/usb/asix_common.c @@ -752,8 +752,8 @@ void asix_get_drvinfo(struct net_device *net, struct ethtool_drvinfo *info) { /* Inherit standard device info */ usbnet_get_drvinfo(net, info); - strlcpy(info->driver, DRIVER_NAME, sizeof(info->driver)); - strlcpy(info->version, DRIVER_VERSION, sizeof(info->version)); + strscpy(info->driver, DRIVER_NAME, sizeof(info->driver)); + strscpy(info->version, DRIVER_VERSION, sizeof(info->version)); } int asix_set_mac_address(struct net_device *net, void *p) diff --git a/drivers/net/usb/asix_devices.c b/drivers/net/usb/asix_devices.c index 5b5eb630c4b7..11f60d32be82 100644 --- a/drivers/net/usb/asix_devices.c +++ b/drivers/net/usb/asix_devices.c @@ -303,6 +303,24 @@ static int ax88772_ethtool_get_sset_count(struct net_device *ndev, int sset) } } +static void ax88772_ethtool_get_pauseparam(struct net_device *ndev, + struct ethtool_pauseparam *pause) +{ + struct usbnet *dev = netdev_priv(ndev); + struct asix_common_private *priv = dev->driver_priv; + + phylink_ethtool_get_pauseparam(priv->phylink, pause); +} + +static int ax88772_ethtool_set_pauseparam(struct net_device *ndev, + struct ethtool_pauseparam *pause) +{ + struct usbnet *dev = netdev_priv(ndev); + struct asix_common_private *priv = dev->driver_priv; + + return phylink_ethtool_set_pauseparam(priv->phylink, pause); +} + static const struct ethtool_ops ax88772_ethtool_ops = { .get_drvinfo = asix_get_drvinfo, .get_link = usbnet_get_link, @@ -319,6 +337,8 @@ static const struct ethtool_ops ax88772_ethtool_ops = { .self_test = net_selftest, .get_strings = ax88772_ethtool_get_strings, .get_sset_count = ax88772_ethtool_get_sset_count, + .get_pauseparam = ax88772_ethtool_get_pauseparam, + .set_pauseparam = ax88772_ethtool_set_pauseparam, }; static int ax88772_reset(struct usbnet *dev) @@ -343,7 +363,7 @@ static int ax88772_reset(struct usbnet *dev) if (ret < 0) goto out; - phy_start(priv->phydev); + phylink_start(priv->phylink); return 0; @@ -590,8 +610,11 @@ static void ax88772_suspend(struct usbnet *dev) struct asix_common_private *priv = dev->driver_priv; u16 medium; - if (netif_running(dev->net)) - phy_stop(priv->phydev); + if (netif_running(dev->net)) { + rtnl_lock(); + phylink_suspend(priv->phylink, false); + rtnl_unlock(); + } /* Stop MAC operation */ medium = asix_read_medium_status(dev, 1); @@ -622,8 +645,11 @@ static void ax88772_resume(struct usbnet *dev) if (!priv->reset(dev, 1)) break; - if (netif_running(dev->net)) - phy_start(priv->phydev); + if (netif_running(dev->net)) { + rtnl_lock(); + phylink_resume(priv->phylink); + rtnl_unlock(); + } } static int asix_resume(struct usb_interface *intf) @@ -667,8 +693,7 @@ static int ax88772_init_phy(struct usbnet *dev) return -ENODEV; } - ret = phy_connect_direct(dev->net, priv->phydev, &asix_adjust_link, - PHY_INTERFACE_MODE_INTERNAL); + ret = phylink_connect_phy(priv->phylink, priv->phydev); if (ret) { netdev_err(dev->net, "Could not connect PHY\n"); return ret; @@ -688,6 +713,9 @@ static int ax88772_init_phy(struct usbnet *dev) */ priv->phydev_int = mdiobus_get_phy(priv->mdio, AX_EMBD_PHY_ADDR); if (!priv->phydev_int) { + rtnl_lock(); + phylink_disconnect_phy(priv->phylink); + rtnl_unlock(); netdev_err(dev->net, "Could not find internal PHY\n"); return -ENODEV; } @@ -698,6 +726,89 @@ static int ax88772_init_phy(struct usbnet *dev) return 0; } +static void ax88772_mac_config(struct phylink_config *config, unsigned int mode, + const struct phylink_link_state *state) +{ + /* Nothing to do */ +} + +static void ax88772_mac_link_down(struct phylink_config *config, + unsigned int mode, phy_interface_t interface) +{ + struct usbnet *dev = netdev_priv(to_net_dev(config->dev)); + + asix_write_medium_mode(dev, 0, 0); + usbnet_link_change(dev, false, false); +} + +static void ax88772_mac_link_up(struct phylink_config *config, + struct phy_device *phy, + unsigned int mode, phy_interface_t interface, + int speed, int duplex, + bool tx_pause, bool rx_pause) +{ + struct usbnet *dev = netdev_priv(to_net_dev(config->dev)); + u16 m = AX_MEDIUM_AC | AX_MEDIUM_RE; + + m |= duplex ? AX_MEDIUM_FD : 0; + + switch (speed) { + case SPEED_100: + m |= AX_MEDIUM_PS; + break; + case SPEED_10: + break; + default: + return; + } + + if (tx_pause) + m |= AX_MEDIUM_TFC; + + if (rx_pause) + m |= AX_MEDIUM_RFC; + + asix_write_medium_mode(dev, m, 0); + usbnet_link_change(dev, true, false); +} + +static const struct phylink_mac_ops ax88772_phylink_mac_ops = { + .validate = phylink_generic_validate, + .mac_config = ax88772_mac_config, + .mac_link_down = ax88772_mac_link_down, + .mac_link_up = ax88772_mac_link_up, +}; + +static int ax88772_phylink_setup(struct usbnet *dev) +{ + struct asix_common_private *priv = dev->driver_priv; + phy_interface_t phy_if_mode; + struct phylink *phylink; + + priv->phylink_config.dev = &dev->net->dev; + priv->phylink_config.type = PHYLINK_NETDEV; + priv->phylink_config.mac_capabilities = MAC_SYM_PAUSE | MAC_ASYM_PAUSE | + MAC_10 | MAC_100; + + __set_bit(PHY_INTERFACE_MODE_INTERNAL, + priv->phylink_config.supported_interfaces); + __set_bit(PHY_INTERFACE_MODE_RMII, + priv->phylink_config.supported_interfaces); + + if (priv->embd_phy) + phy_if_mode = PHY_INTERFACE_MODE_INTERNAL; + else + phy_if_mode = PHY_INTERFACE_MODE_RMII; + + phylink = phylink_create(&priv->phylink_config, dev->net->dev.fwnode, + phy_if_mode, &ax88772_phylink_mac_ops); + if (IS_ERR(phylink)) + return PTR_ERR(phylink); + + priv->phylink = phylink; + return 0; +} + static int ax88772_bind(struct usbnet *dev, struct usb_interface *intf) { struct asix_common_private *priv; @@ -788,14 +899,22 @@ static int ax88772_bind(struct usbnet *dev, struct usb_interface *intf) if (ret) return ret; - return ax88772_init_phy(dev); + ret = ax88772_phylink_setup(dev); + if (ret) + return ret; + + ret = ax88772_init_phy(dev); + if (ret) + phylink_destroy(priv->phylink); + + return ret; } static int ax88772_stop(struct usbnet *dev) { struct asix_common_private *priv = dev->driver_priv; - phy_stop(priv->phydev); + phylink_stop(priv->phylink); return 0; } @@ -804,7 +923,10 @@ static void ax88772_unbind(struct usbnet *dev, struct usb_interface *intf) { struct asix_common_private *priv = dev->driver_priv; - phy_disconnect(priv->phydev); + rtnl_lock(); + phylink_disconnect_phy(priv->phylink); + rtnl_unlock(); + phylink_destroy(priv->phylink); asix_rx_fixup_common_free(dev->driver_priv); } diff --git a/drivers/net/usb/catc.c b/drivers/net/usb/catc.c index 843893482abd..ff439ef535ac 100644 --- a/drivers/net/usb/catc.c +++ b/drivers/net/usb/catc.c @@ -672,8 +672,8 @@ static void catc_get_drvinfo(struct net_device *dev, struct ethtool_drvinfo *info) { struct catc *catc = netdev_priv(dev); - strlcpy(info->driver, driver_name, sizeof(info->driver)); - strlcpy(info->version, DRIVER_VERSION, sizeof(info->version)); + strscpy(info->driver, driver_name, sizeof(info->driver)); + strscpy(info->version, DRIVER_VERSION, sizeof(info->version)); usb_make_path(catc->usbdev, info->bus_info, sizeof(info->bus_info)); } diff --git a/drivers/net/usb/lan78xx.c b/drivers/net/usb/lan78xx.c index 3226ab33afae..f18ab8e220db 100644 --- a/drivers/net/usb/lan78xx.c +++ b/drivers/net/usb/lan78xx.c @@ -4374,7 +4374,7 @@ static int lan78xx_probe(struct usb_interface *intf, netif_set_tso_max_size(netdev, LAN78XX_TSO_SIZE(dev)); - netif_napi_add(netdev, &dev->napi, lan78xx_poll, NAPI_POLL_WEIGHT); + netif_napi_add(netdev, &dev->napi, lan78xx_poll); INIT_DELAYED_WORK(&dev->wq, lan78xx_delayedwork); init_usb_anchor(&dev->deferred); diff --git a/drivers/net/usb/pegasus.c b/drivers/net/usb/pegasus.c index feb247e355f7..81ca64debc5b 100644 --- a/drivers/net/usb/pegasus.c +++ b/drivers/net/usb/pegasus.c @@ -894,7 +894,7 @@ static void pegasus_get_drvinfo(struct net_device *dev, { pegasus_t *pegasus = netdev_priv(dev); - strlcpy(info->driver, driver_name, sizeof(info->driver)); + strscpy(info->driver, driver_name, sizeof(info->driver)); usb_make_path(pegasus->usb, info->bus_info, sizeof(info->bus_info)); } diff --git a/drivers/net/usb/r8152.c b/drivers/net/usb/r8152.c index 688905ea0a6d..a481a1d831e2 100644 --- a/drivers/net/usb/r8152.c +++ b/drivers/net/usb/r8152.c @@ -770,6 +770,7 @@ enum rtl8152_flags { RX_EPROTO, }; +#define DEVICE_ID_LENOVO_USB_C_TRAVEL_HUB 0x721e #define DEVICE_ID_THINKPAD_ONELINK_PLUS_DOCK 0x3054 #define DEVICE_ID_THINKPAD_THUNDERBOLT3_DOCK_GEN2 0x3082 #define DEVICE_ID_THINKPAD_USB_C_DONGLE 0x720c @@ -1874,7 +1875,9 @@ static void intr_callback(struct urb *urb) "Stop submitting intr, status %d\n", status); return; case -EOVERFLOW: - netif_info(tp, intr, tp->netdev, "intr status -EOVERFLOW\n"); + if (net_ratelimit()) + netif_info(tp, intr, tp->netdev, + "intr status -EOVERFLOW\n"); goto resubmit; /* -EPIPE: should clear the halt */ default: @@ -2727,22 +2730,26 @@ static void _rtl8152_set_rx_mode(struct net_device *netdev) ocp_data |= RCR_AM | RCR_AAP; mc_filter[1] = 0xffffffff; mc_filter[0] = 0xffffffff; - } else if ((netdev_mc_count(netdev) > multicast_filter_limit) || - (netdev->flags & IFF_ALLMULTI)) { + } else if ((netdev->flags & IFF_MULTICAST && + netdev_mc_count(netdev) > multicast_filter_limit) || + (netdev->flags & IFF_ALLMULTI)) { /* Too many to filter perfectly -- accept all multicasts. */ ocp_data |= RCR_AM; mc_filter[1] = 0xffffffff; mc_filter[0] = 0xffffffff; } else { - struct netdev_hw_addr *ha; - mc_filter[1] = 0; mc_filter[0] = 0; - netdev_for_each_mc_addr(ha, netdev) { - int bit_nr = ether_crc(ETH_ALEN, ha->addr) >> 26; - mc_filter[bit_nr >> 5] |= 1 << (bit_nr & 31); - ocp_data |= RCR_AM; + if (netdev->flags & IFF_MULTICAST) { + struct netdev_hw_addr *ha; + + netdev_for_each_mc_addr(ha, netdev) { + int bit_nr = ether_crc(ETH_ALEN, ha->addr) >> 26; + + mc_filter[bit_nr >> 5] |= 1 << (bit_nr & 31); + ocp_data |= RCR_AM; + } } } @@ -8602,11 +8609,11 @@ static void rtl8152_get_drvinfo(struct net_device *netdev, { struct r8152 *tp = netdev_priv(netdev); - strlcpy(info->driver, MODULENAME, sizeof(info->driver)); - strlcpy(info->version, DRIVER_VERSION, sizeof(info->version)); + strscpy(info->driver, MODULENAME, sizeof(info->driver)); + strscpy(info->version, DRIVER_VERSION, sizeof(info->version)); usb_make_path(tp->udev, info->bus_info, sizeof(info->bus_info)); if (!IS_ERR_OR_NULL(tp->rtl_fw.fw)) - strlcpy(info->fw_version, tp->rtl_fw.version, + strscpy(info->fw_version, tp->rtl_fw.version, sizeof(info->fw_version)); } @@ -9582,6 +9589,7 @@ static bool rtl8152_supports_lenovo_macpassthru(struct usb_device *udev) if (vendor_id == VENDOR_ID_LENOVO) { switch (product_id) { + case DEVICE_ID_LENOVO_USB_C_TRAVEL_HUB: case DEVICE_ID_THINKPAD_ONELINK_PLUS_DOCK: case DEVICE_ID_THINKPAD_THUNDERBOLT3_DOCK_GEN2: case DEVICE_ID_THINKPAD_USB_C_DOCK_GEN2: diff --git a/drivers/net/usb/rtl8150.c b/drivers/net/usb/rtl8150.c index 3d2bf2acca94..97afd7335d86 100644 --- a/drivers/net/usb/rtl8150.c +++ b/drivers/net/usb/rtl8150.c @@ -769,8 +769,8 @@ static void rtl8150_get_drvinfo(struct net_device *netdev, struct ethtool_drvinf { rtl8150_t *dev = netdev_priv(netdev); - strlcpy(info->driver, driver_name, sizeof(info->driver)); - strlcpy(info->version, DRIVER_VERSION, sizeof(info->version)); + strscpy(info->driver, driver_name, sizeof(info->driver)); + strscpy(info->version, DRIVER_VERSION, sizeof(info->version)); usb_make_path(dev->udev, info->bus_info, sizeof(info->bus_info)); } diff --git a/drivers/net/usb/sierra_net.c b/drivers/net/usb/sierra_net.c index bb4cbe8fc846..b3ae949e6f1c 100644 --- a/drivers/net/usb/sierra_net.c +++ b/drivers/net/usb/sierra_net.c @@ -612,8 +612,8 @@ static void sierra_net_get_drvinfo(struct net_device *net, { /* Inherit standard device info */ usbnet_get_drvinfo(net, info); - strlcpy(info->driver, driver_name, sizeof(info->driver)); - strlcpy(info->version, DRIVER_VERSION, sizeof(info->version)); + strscpy(info->driver, driver_name, sizeof(info->driver)); + strscpy(info->version, DRIVER_VERSION, sizeof(info->version)); } static u32 sierra_net_get_link(struct net_device *net) diff --git a/drivers/net/usb/usbnet.c b/drivers/net/usb/usbnet.c index e368b0780753..64a9a80b2309 100644 --- a/drivers/net/usb/usbnet.c +++ b/drivers/net/usb/usbnet.c @@ -1050,9 +1050,9 @@ void usbnet_get_drvinfo (struct net_device *net, struct ethtool_drvinfo *info) { struct usbnet *dev = netdev_priv(net); - strlcpy (info->driver, dev->driver_name, sizeof info->driver); - strlcpy (info->fw_version, dev->driver_info->description, - sizeof info->fw_version); + strscpy(info->driver, dev->driver_name, sizeof(info->driver)); + strscpy(info->fw_version, dev->driver_info->description, + sizeof(info->fw_version)); usb_make_path (dev->udev, info->bus_info, sizeof info->bus_info); } EXPORT_SYMBOL_GPL(usbnet_get_drvinfo); diff --git a/drivers/net/veth.c b/drivers/net/veth.c index 466da01ba2e3..09682ea3354e 100644 --- a/drivers/net/veth.c +++ b/drivers/net/veth.c @@ -128,8 +128,8 @@ static int veth_get_link_ksettings(struct net_device *dev, static void veth_get_drvinfo(struct net_device *dev, struct ethtool_drvinfo *info) { - strlcpy(info->driver, DRV_NAME, sizeof(info->driver)); - strlcpy(info->version, DRV_VERSION, sizeof(info->version)); + strscpy(info->driver, DRV_NAME, sizeof(info->driver)); + strscpy(info->version, DRV_VERSION, sizeof(info->version)); } static void veth_get_strings(struct net_device *dev, u32 stringset, u8 *buf) @@ -1070,7 +1070,7 @@ static int veth_enable_xdp_range(struct net_device *dev, int start, int end, struct veth_rq *rq = &priv->rq[i]; if (!napi_already_on) - netif_napi_add(dev, &rq->xdp_napi, veth_poll, NAPI_POLL_WEIGHT); + netif_napi_add(dev, &rq->xdp_napi, veth_poll); err = xdp_rxq_info_reg(&rq->xdp_rxq, dev, i, rq->xdp_napi.napi_id); if (err < 0) goto err_rxq_reg; @@ -1184,7 +1184,7 @@ static int veth_napi_enable_range(struct net_device *dev, int start, int end) for (i = start; i < end; i++) { struct veth_rq *rq = &priv->rq[i]; - netif_napi_add(dev, &rq->xdp_napi, veth_poll, NAPI_POLL_WEIGHT); + netif_napi_add(dev, &rq->xdp_napi, veth_poll); } err = __veth_napi_enable_range(dev, start, end); diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c index 9cce7dec7366..e0e57083d442 100644 --- a/drivers/net/virtio_net.c +++ b/drivers/net/virtio_net.c @@ -2594,9 +2594,9 @@ static void virtnet_get_drvinfo(struct net_device *dev, struct virtnet_info *vi = netdev_priv(dev); struct virtio_device *vdev = vi->vdev; - strlcpy(info->driver, KBUILD_MODNAME, sizeof(info->driver)); - strlcpy(info->version, VIRTNET_DRIVER_VERSION, sizeof(info->version)); - strlcpy(info->bus_info, virtio_bus_name(vdev), sizeof(info->bus_info)); + strscpy(info->driver, KBUILD_MODNAME, sizeof(info->driver)); + strscpy(info->version, VIRTNET_DRIVER_VERSION, sizeof(info->version)); + strscpy(info->bus_info, virtio_bus_name(vdev), sizeof(info->bus_info)); } diff --git a/drivers/net/vmxnet3/vmxnet3_drv.c b/drivers/net/vmxnet3/vmxnet3_drv.c index 53b3b241e027..d3e7b27eb933 100644 --- a/drivers/net/vmxnet3/vmxnet3_drv.c +++ b/drivers/net/vmxnet3/vmxnet3_drv.c @@ -3882,11 +3882,11 @@ vmxnet3_probe_device(struct pci_dev *pdev, for (i = 0; i < adapter->num_rx_queues; i++) { netif_napi_add(adapter->netdev, &adapter->rx_queue[i].napi, - vmxnet3_poll_rx_only, 64); + vmxnet3_poll_rx_only); } } else { netif_napi_add(adapter->netdev, &adapter->rx_queue[0].napi, - vmxnet3_poll, 64); + vmxnet3_poll); } netif_set_real_num_tx_queues(adapter->netdev, adapter->num_tx_queues); diff --git a/drivers/net/vmxnet3/vmxnet3_ethtool.c b/drivers/net/vmxnet3/vmxnet3_ethtool.c index e2034adc3a1a..18cf7c723201 100644 --- a/drivers/net/vmxnet3/vmxnet3_ethtool.c +++ b/drivers/net/vmxnet3/vmxnet3_ethtool.c @@ -209,12 +209,12 @@ vmxnet3_get_drvinfo(struct net_device *netdev, struct ethtool_drvinfo *drvinfo) { struct vmxnet3_adapter *adapter = netdev_priv(netdev); - strlcpy(drvinfo->driver, vmxnet3_driver_name, sizeof(drvinfo->driver)); + strscpy(drvinfo->driver, vmxnet3_driver_name, sizeof(drvinfo->driver)); - strlcpy(drvinfo->version, VMXNET3_DRIVER_VERSION_REPORT, + strscpy(drvinfo->version, VMXNET3_DRIVER_VERSION_REPORT, sizeof(drvinfo->version)); - strlcpy(drvinfo->bus_info, pci_name(adapter->pdev), + strscpy(drvinfo->bus_info, pci_name(adapter->pdev), sizeof(drvinfo->bus_info)); } diff --git a/drivers/net/vrf.c b/drivers/net/vrf.c index 5df7a0abc39d..badf6f09ae51 100644 --- a/drivers/net/vrf.c +++ b/drivers/net/vrf.c @@ -1541,8 +1541,8 @@ static const struct l3mdev_ops vrf_l3mdev_ops = { static void vrf_get_drvinfo(struct net_device *dev, struct ethtool_drvinfo *info) { - strlcpy(info->driver, DRV_NAME, sizeof(info->driver)); - strlcpy(info->version, DRV_VERSION, sizeof(info->version)); + strscpy(info->driver, DRV_NAME, sizeof(info->driver)); + strscpy(info->version, DRV_VERSION, sizeof(info->version)); } static const struct ethtool_ops vrf_ethtool_ops = { diff --git a/drivers/net/vxlan/vxlan_core.c b/drivers/net/vxlan/vxlan_core.c index c3285242f74f..6ab669dcd1c6 100644 --- a/drivers/net/vxlan/vxlan_core.c +++ b/drivers/net/vxlan/vxlan_core.c @@ -713,12 +713,9 @@ static struct sk_buff *vxlan_gro_receive(struct sock *sk, off_vx = skb_gro_offset(skb); hlen = off_vx + sizeof(*vh); - vh = skb_gro_header_fast(skb, off_vx); - if (skb_gro_header_hard(skb, hlen)) { - vh = skb_gro_header_slow(skb, hlen, off_vx); - if (unlikely(!vh)) - goto out; - } + vh = skb_gro_header(skb, hlen, off_vx); + if (unlikely(!vh)) + goto out; skb_gro_postpull_rcsum(skb, vh, sizeof(struct vxlanhdr)); @@ -3313,8 +3310,8 @@ static int vxlan_validate(struct nlattr *tb[], struct nlattr *data[], static void vxlan_get_drvinfo(struct net_device *netdev, struct ethtool_drvinfo *drvinfo) { - strlcpy(drvinfo->version, VXLAN_VERSION, sizeof(drvinfo->version)); - strlcpy(drvinfo->driver, "vxlan", sizeof(drvinfo->driver)); + strscpy(drvinfo->version, VXLAN_VERSION, sizeof(drvinfo->version)); + strscpy(drvinfo->driver, "vxlan", sizeof(drvinfo->driver)); } static int vxlan_get_link_ksettings(struct net_device *dev, diff --git a/drivers/net/wireguard/netlink.c b/drivers/net/wireguard/netlink.c index 5c804bcabfe6..43c8c84e7ea8 100644 --- a/drivers/net/wireguard/netlink.c +++ b/drivers/net/wireguard/netlink.c @@ -620,6 +620,7 @@ static const struct genl_ops genl_ops[] = { static struct genl_family genl_family __ro_after_init = { .ops = genl_ops, .n_ops = ARRAY_SIZE(genl_ops), + .resv_start_op = WG_CMD_SET_DEVICE + 1, .name = WG_GENL_NAME, .version = WG_GENL_VERSION, .maxattr = WGDEVICE_A_MAX, diff --git a/drivers/net/wireguard/peer.c b/drivers/net/wireguard/peer.c index 1acd00ab2fbc..1cb502a932e0 100644 --- a/drivers/net/wireguard/peer.c +++ b/drivers/net/wireguard/peer.c @@ -54,8 +54,7 @@ struct wg_peer *wg_peer_create(struct wg_device *wg, skb_queue_head_init(&peer->staged_packet_queue); wg_noise_reset_last_sent_handshake(&peer->last_sent_handshake); set_bit(NAPI_STATE_NO_BUSY_POLL, &peer->napi.state); - netif_napi_add(wg->dev, &peer->napi, wg_packet_rx_poll, - NAPI_POLL_WEIGHT); + netif_napi_add(wg->dev, &peer->napi, wg_packet_rx_poll); napi_enable(&peer->napi); list_add_tail(&peer->peer_list, &wg->peer_list); INIT_LIST_HEAD(&peer->allowedips_list); diff --git a/drivers/net/wireless/ath/ath10k/bmi.c b/drivers/net/wireless/ath/ath10k/bmi.c index 4481ed375f55..af6546572df2 100644 --- a/drivers/net/wireless/ath/ath10k/bmi.c +++ b/drivers/net/wireless/ath/ath10k/bmi.c @@ -101,7 +101,7 @@ int ath10k_bmi_get_target_info_sdio(struct ath10k *ar, cmd.id = __cpu_to_le32(BMI_GET_TARGET_INFO); /* Step 1: Read 4 bytes of the target info and check if it is - * the special sentinal version word or the first word in the + * the special sentinel version word or the first word in the * version response. */ resplen = sizeof(u32); @@ -111,7 +111,7 @@ int ath10k_bmi_get_target_info_sdio(struct ath10k *ar, return ret; } - /* Some SDIO boards have a special sentinal byte before the real + /* Some SDIO boards have a special sentinel byte before the real * version response. */ if (__le32_to_cpu(tmp) == TARGET_VERSION_SENTINAL) { diff --git a/drivers/net/wireless/ath/ath10k/ce.c b/drivers/net/wireless/ath/ath10k/ce.c index c45c814fd122..59926227bd49 100644 --- a/drivers/net/wireless/ath/ath10k/ce.c +++ b/drivers/net/wireless/ath/ath10k/ce.c @@ -1323,7 +1323,7 @@ EXPORT_SYMBOL(ath10k_ce_per_engine_service); /* * Handler for per-engine interrupts on ALL active CEs. * This is used in cases where the system is sharing a - * single interrput for all CEs + * single interrupt for all CEs */ void ath10k_ce_per_engine_service_any(struct ath10k *ar) diff --git a/drivers/net/wireless/ath/ath10k/core.c b/drivers/net/wireless/ath/ath10k/core.c index 276954b70d63..400f332a7ff0 100644 --- a/drivers/net/wireless/ath/ath10k/core.c +++ b/drivers/net/wireless/ath/ath10k/core.c @@ -98,6 +98,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = { .tx_stats_over_pktlog = true, .dynamic_sar_support = false, .hw_restart_disconnect = false, + .use_fw_tx_credits = true, }, { .id = QCA988X_HW_2_0_VERSION, @@ -136,6 +137,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = { .tx_stats_over_pktlog = true, .dynamic_sar_support = false, .hw_restart_disconnect = false, + .use_fw_tx_credits = true, }, { .id = QCA9887_HW_1_0_VERSION, @@ -175,6 +177,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = { .tx_stats_over_pktlog = false, .dynamic_sar_support = false, .hw_restart_disconnect = false, + .use_fw_tx_credits = true, }, { .id = QCA6174_HW_3_2_VERSION, @@ -209,6 +212,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = { .supports_peer_stats_info = true, .dynamic_sar_support = true, .hw_restart_disconnect = false, + .use_fw_tx_credits = true, }, { .id = QCA6174_HW_2_1_VERSION, @@ -247,6 +251,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = { .tx_stats_over_pktlog = false, .dynamic_sar_support = false, .hw_restart_disconnect = false, + .use_fw_tx_credits = true, }, { .id = QCA6174_HW_2_1_VERSION, @@ -285,6 +290,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = { .tx_stats_over_pktlog = false, .dynamic_sar_support = false, .hw_restart_disconnect = false, + .use_fw_tx_credits = true, }, { .id = QCA6174_HW_3_0_VERSION, @@ -323,6 +329,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = { .tx_stats_over_pktlog = false, .dynamic_sar_support = false, .hw_restart_disconnect = false, + .use_fw_tx_credits = true, }, { .id = QCA6174_HW_3_2_VERSION, @@ -365,6 +372,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = { .supports_peer_stats_info = true, .dynamic_sar_support = true, .hw_restart_disconnect = false, + .use_fw_tx_credits = true, }, { .id = QCA99X0_HW_2_0_DEV_VERSION, @@ -409,6 +417,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = { .tx_stats_over_pktlog = false, .dynamic_sar_support = false, .hw_restart_disconnect = false, + .use_fw_tx_credits = true, }, { .id = QCA9984_HW_1_0_DEV_VERSION, @@ -460,6 +469,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = { .tx_stats_over_pktlog = false, .dynamic_sar_support = false, .hw_restart_disconnect = false, + .use_fw_tx_credits = true, }, { .id = QCA9888_HW_2_0_DEV_VERSION, @@ -508,6 +518,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = { .tx_stats_over_pktlog = false, .dynamic_sar_support = false, .hw_restart_disconnect = false, + .use_fw_tx_credits = true, }, { .id = QCA9377_HW_1_0_DEV_VERSION, @@ -546,6 +557,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = { .tx_stats_over_pktlog = false, .dynamic_sar_support = false, .hw_restart_disconnect = false, + .use_fw_tx_credits = true, }, { .id = QCA9377_HW_1_1_DEV_VERSION, @@ -586,6 +598,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = { .tx_stats_over_pktlog = false, .dynamic_sar_support = false, .hw_restart_disconnect = false, + .use_fw_tx_credits = true, }, { .id = QCA9377_HW_1_1_DEV_VERSION, @@ -617,6 +630,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = { .credit_size_workaround = true, .dynamic_sar_support = false, .hw_restart_disconnect = false, + .use_fw_tx_credits = true, }, { .id = QCA4019_HW_1_0_DEV_VERSION, @@ -662,6 +676,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = { .tx_stats_over_pktlog = false, .dynamic_sar_support = false, .hw_restart_disconnect = false, + .use_fw_tx_credits = true, }, { .id = WCN3990_HW_1_0_DEV_VERSION, @@ -693,6 +708,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = { .tx_stats_over_pktlog = false, .dynamic_sar_support = true, .hw_restart_disconnect = true, + .use_fw_tx_credits = false, }, }; @@ -3080,7 +3096,7 @@ int ath10k_core_start(struct ath10k *ar, enum ath10k_firmware_mode mode, * enabled always. * * We can still enable BTCOEX if firmware has the support - * eventhough btceox_support value is + * even though btceox_support value is * ATH10K_DT_BTCOEX_NOT_FOUND */ diff --git a/drivers/net/wireless/ath/ath10k/core.h b/drivers/net/wireless/ath/ath10k/core.h index d70d7d088a2b..f5de8ce8fb45 100644 --- a/drivers/net/wireless/ath/ath10k/core.h +++ b/drivers/net/wireless/ath/ath10k/core.h @@ -76,7 +76,7 @@ /* The magic used by QCA spec */ #define ATH10K_SMBIOS_BDF_EXT_MAGIC "BDF_" -/* Default Airtime weight multipler (Tuned for multiclient performance) */ +/* Default Airtime weight multiplier (Tuned for multiclient performance) */ #define ATH10K_AIRTIME_WEIGHT_MULTIPLIER 4 #define ATH10K_MAX_RETRY_COUNT 30 @@ -857,7 +857,7 @@ enum ath10k_dev_flags { /* Disable HW crypto engine */ ATH10K_FLAG_HW_CRYPTO_DISABLED, - /* Bluetooth coexistance enabled */ + /* Bluetooth coexistence enabled */ ATH10K_FLAG_BTCOEX, /* Per Station statistics service */ diff --git a/drivers/net/wireless/ath/ath10k/coredump.c b/drivers/net/wireless/ath/ath10k/coredump.c index fe6b6f97a916..2d1634a890dd 100644 --- a/drivers/net/wireless/ath/ath10k/coredump.c +++ b/drivers/net/wireless/ath/ath10k/coredump.c @@ -531,7 +531,7 @@ static const struct ath10k_mem_section qca6174_hw30_sdio_register_sections[] = { {0x40000, 0x400A4}, - /* SI register is skiped here. + /* SI register is skipped here. * Because it will cause bus hang * * {0x50000, 0x50018}, diff --git a/drivers/net/wireless/ath/ath10k/coredump.h b/drivers/net/wireless/ath/ath10k/coredump.h index 240d70515088..437b9759f05d 100644 --- a/drivers/net/wireless/ath/ath10k/coredump.h +++ b/drivers/net/wireless/ath/ath10k/coredump.h @@ -125,7 +125,7 @@ enum ath10k_mem_region_type { * To minimize the size of the array, the list must obey the format: * '{start0,stop0},{start1,stop1},{start2,stop2}....' The values below must * also obey to 'start0 < stop0 < start1 < stop1 < start2 < ...', otherwise - * we may encouter error in the dump processing. + * we may encounter error in the dump processing. */ struct ath10k_mem_section { u32 start; diff --git a/drivers/net/wireless/ath/ath10k/debug.c b/drivers/net/wireless/ath/ath10k/debug.c index 39378e3f9b2b..c861e66ef6bc 100644 --- a/drivers/net/wireless/ath/ath10k/debug.c +++ b/drivers/net/wireless/ath/ath10k/debug.c @@ -1081,7 +1081,7 @@ exit: * struct available.. */ -/* This generally cooresponds to the debugfs fw_stats file */ +/* This generally corresponds to the debugfs fw_stats file */ static const char ath10k_gstrings_stats[][ETH_GSTRING_LEN] = { "tx_pkts_nic", "tx_bytes_nic", diff --git a/drivers/net/wireless/ath/ath10k/debugfs_sta.c b/drivers/net/wireless/ath/ath10k/debugfs_sta.c index 367539f2c370..87a3365330ff 100644 --- a/drivers/net/wireless/ath/ath10k/debugfs_sta.c +++ b/drivers/net/wireless/ath/ath10k/debugfs_sta.c @@ -498,7 +498,7 @@ static char *get_num_ampdu_subfrm_str(enum ath10k_ampdu_subfrm_num i) { switch (i) { case ATH10K_AMPDU_SUBFRM_NUM_10: - return "upto 10"; + return "up to 10"; case ATH10K_AMPDU_SUBFRM_NUM_20: return "11-20"; case ATH10K_AMPDU_SUBFRM_NUM_30: diff --git a/drivers/net/wireless/ath/ath10k/htc.c b/drivers/net/wireless/ath/ath10k/htc.c index fab398046a3f..6d1784f74bea 100644 --- a/drivers/net/wireless/ath/ath10k/htc.c +++ b/drivers/net/wireless/ath/ath10k/htc.c @@ -947,13 +947,18 @@ int ath10k_htc_wait_target(struct ath10k_htc *htc) return -ECOMM; } - htc->total_transmit_credits = __le16_to_cpu(msg->ready.credit_count); + if (ar->hw_params.use_fw_tx_credits) + htc->total_transmit_credits = __le16_to_cpu(msg->ready.credit_count); + else + htc->total_transmit_credits = 1; + htc->target_credit_size = __le16_to_cpu(msg->ready.credit_size); ath10k_dbg(ar, ATH10K_DBG_HTC, - "Target ready! transmit resources: %d size:%d\n", + "Target ready! transmit resources: %d size:%d actual credits:%d\n", htc->total_transmit_credits, - htc->target_credit_size); + htc->target_credit_size, + msg->ready.credit_count); if ((htc->total_transmit_credits == 0) || (htc->target_credit_size == 0)) { diff --git a/drivers/net/wireless/ath/ath10k/htt_rx.c b/drivers/net/wireless/ath/ath10k/htt_rx.c index 8a075a711b71..e76aab973320 100644 --- a/drivers/net/wireless/ath/ath10k/htt_rx.c +++ b/drivers/net/wireless/ath/ath10k/htt_rx.c @@ -301,12 +301,16 @@ void ath10k_htt_rx_free(struct ath10k_htt *htt) ath10k_htt_get_vaddr_ring(htt), htt->rx_ring.base_paddr); + ath10k_htt_config_paddrs_ring(htt, NULL); + dma_free_coherent(htt->ar->dev, sizeof(*htt->rx_ring.alloc_idx.vaddr), htt->rx_ring.alloc_idx.vaddr, htt->rx_ring.alloc_idx.paddr); + htt->rx_ring.alloc_idx.vaddr = NULL; kfree(htt->rx_ring.netbufs_ring); + htt->rx_ring.netbufs_ring = NULL; } static inline struct sk_buff *ath10k_htt_rx_netbuf_pop(struct ath10k_htt *htt) @@ -846,8 +850,10 @@ err_dma_idx: ath10k_htt_get_rx_ring_size(htt), vaddr_ring, htt->rx_ring.base_paddr); + ath10k_htt_config_paddrs_ring(htt, NULL); err_dma_ring: kfree(htt->rx_ring.netbufs_ring); + htt->rx_ring.netbufs_ring = NULL; err_netbuf: return -ENOMEM; } @@ -2496,7 +2502,7 @@ static bool ath10k_htt_rx_proc_rx_ind_hl(struct ath10k_htt *htt, /* I have not yet seen any case where num_mpdu_ranges > 1. * qcacld does not seem handle that case either, so we introduce the - * same limitiation here as well. + * same limitation here as well. */ if (num_mpdu_ranges > 1) ath10k_warn(ar, diff --git a/drivers/net/wireless/ath/ath10k/htt_tx.c b/drivers/net/wireless/ath/ath10k/htt_tx.c index a19b0795c86d..bd603feb7953 100644 --- a/drivers/net/wireless/ath/ath10k/htt_tx.c +++ b/drivers/net/wireless/ath/ath10k/htt_tx.c @@ -1112,7 +1112,7 @@ int ath10k_htt_tx_fetch_resp(struct ath10k *ar, int len = 0; int ret; - /* Response IDs are echo-ed back only for host driver convienence + /* Response IDs are echo-ed back only for host driver convenience * purposes. They aren't used for anything in the driver yet so use 0. */ diff --git a/drivers/net/wireless/ath/ath10k/hw.c b/drivers/net/wireless/ath/ath10k/hw.c index e52e41a70321..6d32b43a4da6 100644 --- a/drivers/net/wireless/ath/ath10k/hw.c +++ b/drivers/net/wireless/ath/ath10k/hw.c @@ -84,7 +84,7 @@ const struct ath10k_hw_regs qca99x0_regs = { .ce5_base_address = 0x0004b400, .ce6_base_address = 0x0004b800, .ce7_base_address = 0x0004bc00, - /* Note: qca99x0 supports upto 12 Copy Engines. Other than address of + /* Note: qca99x0 supports up to 12 Copy Engines. Other than address of * CE0 and CE1 no other copy engine is directly referred in the code. * It is not really necessary to assign address for newly supported * CEs in this address table. @@ -120,7 +120,7 @@ const struct ath10k_hw_regs qca4019_regs = { .ce5_base_address = 0x0004b400, .ce6_base_address = 0x0004b800, .ce7_base_address = 0x0004bc00, - /* qca4019 supports upto 12 copy engines. Since base address + /* qca4019 supports up to 12 copy engines. Since base address * of ce8 to ce11 are not directly referred in the code, * no need have them in separate members in this table. * Copy Engine Address @@ -924,7 +924,7 @@ static void ath10k_hw_map_target_mem(struct ath10k *ar, u32 msb) ath10k_hif_write32(ar, address, msb); } -/* 1. Write to memory region of target, such as IRAM adn DRAM. +/* 1. Write to memory region of target, such as IRAM and DRAM. * 2. Target address( 0 ~ 00100000 & 0x00400000~0x00500000) * can be written directly. See ath10k_pci_targ_cpu_to_ce_addr() too. * 3. In order to access the region other than the above, diff --git a/drivers/net/wireless/ath/ath10k/hw.h b/drivers/net/wireless/ath/ath10k/hw.h index 93acf0dd580a..1b99f3a39a11 100644 --- a/drivers/net/wireless/ath/ath10k/hw.h +++ b/drivers/net/wireless/ath/ath10k/hw.h @@ -635,6 +635,8 @@ struct ath10k_hw_params { bool dynamic_sar_support; bool hw_restart_disconnect; + + bool use_fw_tx_credits; }; struct htt_resp; diff --git a/drivers/net/wireless/ath/ath10k/mac.c b/drivers/net/wireless/ath/ath10k/mac.c index 9dd3b8fba4b0..ec8d5b29bc72 100644 --- a/drivers/net/wireless/ath/ath10k/mac.c +++ b/drivers/net/wireless/ath/ath10k/mac.c @@ -864,11 +864,36 @@ static int ath10k_peer_delete(struct ath10k *ar, u32 vdev_id, const u8 *addr) return 0; } +static void ath10k_peer_map_cleanup(struct ath10k *ar, struct ath10k_peer *peer) +{ + int peer_id, i; + + lockdep_assert_held(&ar->conf_mutex); + + for_each_set_bit(peer_id, peer->peer_ids, + ATH10K_MAX_NUM_PEER_IDS) { + ar->peer_map[peer_id] = NULL; + } + + /* Double check that peer is properly un-referenced from + * the peer_map + */ + for (i = 0; i < ARRAY_SIZE(ar->peer_map); i++) { + if (ar->peer_map[i] == peer) { + ath10k_warn(ar, "removing stale peer_map entry for %pM (ptr %pK idx %d)\n", + peer->addr, peer, i); + ar->peer_map[i] = NULL; + } + } + + list_del(&peer->list); + kfree(peer); + ar->num_peers--; +} + static void ath10k_peer_cleanup(struct ath10k *ar, u32 vdev_id) { struct ath10k_peer *peer, *tmp; - int peer_id; - int i; lockdep_assert_held(&ar->conf_mutex); @@ -880,25 +905,7 @@ static void ath10k_peer_cleanup(struct ath10k *ar, u32 vdev_id) ath10k_warn(ar, "removing stale peer %pM from vdev_id %d\n", peer->addr, vdev_id); - for_each_set_bit(peer_id, peer->peer_ids, - ATH10K_MAX_NUM_PEER_IDS) { - ar->peer_map[peer_id] = NULL; - } - - /* Double check that peer is properly un-referenced from - * the peer_map - */ - for (i = 0; i < ARRAY_SIZE(ar->peer_map); i++) { - if (ar->peer_map[i] == peer) { - ath10k_warn(ar, "removing stale peer_map entry for %pM (ptr %pK idx %d)\n", - peer->addr, peer, i); - ar->peer_map[i] = NULL; - } - } - - list_del(&peer->list); - kfree(peer); - ar->num_peers--; + ath10k_peer_map_cleanup(ar, peer); } spin_unlock_bh(&ar->data_lock); } @@ -4044,7 +4051,7 @@ static int ath10k_mac_tx(struct ath10k *ar, ath10k_tx_h_seq_no(vif, skb); break; case ATH10K_HW_TXRX_ETHERNET: - /* Convert 802.11->802.3 header only if the frame was erlier + /* Convert 802.11->802.3 header only if the frame was earlier * encapsulated to 802.11 by mac80211. Otherwise pass it as is. */ if (!(info->flags & IEEE80211_TX_CTL_HW_80211_ENCAP)) @@ -7621,10 +7628,7 @@ static int ath10k_sta_state(struct ieee80211_hw *hw, /* Clean up the peer object as well since we * must have failed to do this above. */ - list_del(&peer->list); - ar->peer_map[i] = NULL; - kfree(peer); - ar->num_peers--; + ath10k_peer_map_cleanup(ar, peer); } } spin_unlock_bh(&ar->data_lock); @@ -8093,7 +8097,7 @@ static void ath10k_flush(struct ieee80211_hw *hw, struct ieee80211_vif *vif, /* TODO: Implement this function properly * For now it is needed to reply to Probe Requests in IBSS mode. - * Propably we need this information from FW. + * Probably we need this information from FW. */ static int ath10k_tx_last_beacon(struct ieee80211_hw *hw) { @@ -8516,7 +8520,7 @@ static void ath10k_sta_rc_update(struct ieee80211_hw *hw, "mac sta rc update for %pM changed %08x bw %d nss %d smps %d\n", sta->addr, changed, sta->deflink.bandwidth, sta->deflink.rx_nss, - sta->smps_mode); + sta->deflink.smps_mode); if (changed & IEEE80211_RC_BW_CHANGED) { bw = WMI_PEER_CHWIDTH_20MHZ; @@ -8550,7 +8554,7 @@ static void ath10k_sta_rc_update(struct ieee80211_hw *hw, if (changed & IEEE80211_RC_SMPS_CHANGED) { smps = WMI_PEER_SMPS_PS_NONE; - switch (sta->smps_mode) { + switch (sta->deflink.smps_mode) { case IEEE80211_SMPS_AUTOMATIC: case IEEE80211_SMPS_OFF: smps = WMI_PEER_SMPS_PS_NONE; @@ -8563,7 +8567,7 @@ static void ath10k_sta_rc_update(struct ieee80211_hw *hw, break; case IEEE80211_SMPS_NUM_MODES: ath10k_warn(ar, "Invalid smps %d in sta rc update for %pM\n", - sta->smps_mode, sta->addr); + sta->deflink.smps_mode, sta->addr); smps = WMI_PEER_SMPS_PS_NONE; break; } @@ -9682,7 +9686,7 @@ static const struct ieee80211_iface_limit ath10k_tlv_if_limit_ibss[] = { }, }; -/* FIXME: This is not thouroughly tested. These combinations may over- or +/* FIXME: This is not thoroughly tested. These combinations may over- or * underestimate hw/fw capabilities. */ static struct ieee80211_iface_combination ath10k_tlv_if_comb[] = { @@ -9922,7 +9926,7 @@ int ath10k_mac_register(struct ath10k *ar) WLAN_CIPHER_SUITE_BIP_GMAC_128, WLAN_CIPHER_SUITE_BIP_GMAC_256, - /* Only QCA99x0 and QCA4019 varients support GCMP-128, GCMP-256 + /* Only QCA99x0 and QCA4019 variants support GCMP-128, GCMP-256 * and CCMP-256 in hardware. */ WLAN_CIPHER_SUITE_GCMP, diff --git a/drivers/net/wireless/ath/ath10k/pci.c b/drivers/net/wireless/ath/ath10k/pci.c index bf1c938be7d0..e56c6a6b1379 100644 --- a/drivers/net/wireless/ath/ath10k/pci.c +++ b/drivers/net/wireless/ath/ath10k/pci.c @@ -1244,7 +1244,7 @@ static void ath10k_pci_process_htt_rx_cb(struct ath10k_ce_pipe *ce_state, unsigned int nbytes, max_nbytes, nentries; int orig_len; - /* No need to aquire ce_lock for CE5, since this is the only place CE5 + /* No need to acquire ce_lock for CE5, since this is the only place CE5 * is processed other than init and deinit. Before releasing CE5 * buffers, interrupts are disabled. Thus CE5 access is serialized. */ @@ -3215,8 +3215,7 @@ static void ath10k_pci_free_irq(struct ath10k *ar) void ath10k_pci_init_napi(struct ath10k *ar) { - netif_napi_add(&ar->napi_dev, &ar->napi, ath10k_pci_napi_poll, - NAPI_POLL_WEIGHT); + netif_napi_add(&ar->napi_dev, &ar->napi, ath10k_pci_napi_poll); } static int ath10k_pci_init_irq(struct ath10k *ar) diff --git a/drivers/net/wireless/ath/ath10k/pci.h b/drivers/net/wireless/ath/ath10k/pci.h index cf64898b9447..480cd97ab739 100644 --- a/drivers/net/wireless/ath/ath10k/pci.h +++ b/drivers/net/wireless/ath/ath10k/pci.h @@ -81,7 +81,7 @@ struct ath10k_pci_pipe { /* Handle of underlying Copy Engine */ struct ath10k_ce_pipe *ce_hdl; - /* Our pipe number; facilitiates use of pipe_info ptrs. */ + /* Our pipe number; facilitates use of pipe_info ptrs. */ u8 pipe_num; /* Convenience back pointer to hif_ce_state. */ diff --git a/drivers/net/wireless/ath/ath10k/qmi.c b/drivers/net/wireless/ath/ath10k/qmi.c index d7e406916bc8..66cb7a1e628a 100644 --- a/drivers/net/wireless/ath/ath10k/qmi.c +++ b/drivers/net/wireless/ath/ath10k/qmi.c @@ -792,7 +792,7 @@ static void ath10k_qmi_event_server_arrive(struct ath10k_qmi *qmi) return; /* - * HACK: sleep for a while inbetween receiving the msa info response + * HACK: sleep for a while between receiving the msa info response * and the XPU update to prevent SDM845 from crashing due to a security * violation, when running MPSS.AT.4.0.c2-01184-SDM845_GEN_PACK-1. */ diff --git a/drivers/net/wireless/ath/ath10k/rx_desc.h b/drivers/net/wireless/ath/ath10k/rx_desc.h index 6ce2a8b1060d..777e53aa69dc 100644 --- a/drivers/net/wireless/ath/ath10k/rx_desc.h +++ b/drivers/net/wireless/ath/ath10k/rx_desc.h @@ -448,7 +448,7 @@ struct rx_mpdu_end { * - 4 bytes for WEP * - 8 bytes for TKIP, AES * [padding to 4 bytes] - * c) A-MSDU subframe header (14 bytes) if appliable + * c) A-MSDU subframe header (14 bytes) if applicable * d) LLC/SNAP (RFC1042, 8 bytes) * * In case of A-MSDU only first frame in sequence contains (a) and (b). diff --git a/drivers/net/wireless/ath/ath10k/sdio.c b/drivers/net/wireless/ath/ath10k/sdio.c index 24283c02a5ef..79e09c7a82b3 100644 --- a/drivers/net/wireless/ath/ath10k/sdio.c +++ b/drivers/net/wireless/ath/ath10k/sdio.c @@ -1057,7 +1057,7 @@ static int ath10k_sdio_mbox_proc_pending_irqs(struct ath10k *ar, out: /* An optimization to bypass reading the IRQ status registers - * unecessarily which can re-wake the target, if upper layers + * unnecessarily which can re-wake the target, if upper layers * determine that we are in a low-throughput mode, we can rely on * taking another interrupt rather than re-checking the status * registers which can re-wake the target. @@ -2531,8 +2531,7 @@ static int ath10k_sdio_probe(struct sdio_func *func, return -ENOMEM; } - netif_napi_add(&ar->napi_dev, &ar->napi, ath10k_sdio_napi_poll, - NAPI_POLL_WEIGHT); + netif_napi_add(&ar->napi_dev, &ar->napi, ath10k_sdio_napi_poll); ath10k_dbg(ar, ATH10K_DBG_BOOT, "sdio new func %d vendor 0x%x device 0x%x block 0x%x/0x%x\n", diff --git a/drivers/net/wireless/ath/ath10k/snoc.c b/drivers/net/wireless/ath/ath10k/snoc.c index 5576ad9fd116..cfcb759a87de 100644 --- a/drivers/net/wireless/ath/ath10k/snoc.c +++ b/drivers/net/wireless/ath/ath10k/snoc.c @@ -1242,8 +1242,7 @@ static int ath10k_snoc_napi_poll(struct napi_struct *ctx, int budget) static void ath10k_snoc_init_napi(struct ath10k *ar) { - netif_napi_add(&ar->napi_dev, &ar->napi, ath10k_snoc_napi_poll, - NAPI_POLL_WEIGHT); + netif_napi_add(&ar->napi_dev, &ar->napi, ath10k_snoc_napi_poll); } static int ath10k_snoc_request_irq(struct ath10k *ar) diff --git a/drivers/net/wireless/ath/ath10k/thermal.c b/drivers/net/wireless/ath/ath10k/thermal.c index 36c9a1364253..cefd97323dfe 100644 --- a/drivers/net/wireless/ath/ath10k/thermal.c +++ b/drivers/net/wireless/ath/ath10k/thermal.c @@ -98,7 +98,7 @@ static ssize_t ath10k_thermal_show_temp(struct device *dev, temperature = ar->thermal.temperature; spin_unlock_bh(&ar->data_lock); - /* display in millidegree celcius */ + /* display in millidegree celsius */ ret = snprintf(buf, PAGE_SIZE, "%d\n", temperature * 1000); out: mutex_unlock(&ar->conf_mutex); diff --git a/drivers/net/wireless/ath/ath10k/thermal.h b/drivers/net/wireless/ath/ath10k/thermal.h index 5fdb020f4da3..1f4de9fbf2b3 100644 --- a/drivers/net/wireless/ath/ath10k/thermal.h +++ b/drivers/net/wireless/ath/ath10k/thermal.h @@ -19,7 +19,7 @@ struct ath10k_thermal { /* protected by conf_mutex */ u32 throttle_state; u32 quiet_period; - /* temperature value in Celcius degree + /* temperature value in Celsius degree * protected by data_lock */ int temperature; diff --git a/drivers/net/wireless/ath/ath10k/usb.c b/drivers/net/wireless/ath/ath10k/usb.c index ad6471b21796..b0067af685b1 100644 --- a/drivers/net/wireless/ath/ath10k/usb.c +++ b/drivers/net/wireless/ath/ath10k/usb.c @@ -1014,8 +1014,7 @@ static int ath10k_usb_probe(struct usb_interface *interface, return -ENOMEM; } - netif_napi_add(&ar->napi_dev, &ar->napi, ath10k_usb_napi_poll, - NAPI_POLL_WEIGHT); + netif_napi_add(&ar->napi_dev, &ar->napi, ath10k_usb_napi_poll); usb_get_dev(dev); vendor_id = le16_to_cpu(dev->descriptor.idVendor); diff --git a/drivers/net/wireless/ath/ath10k/usb.h b/drivers/net/wireless/ath/ath10k/usb.h index 34d683e8fc18..48e066ba8162 100644 --- a/drivers/net/wireless/ath/ath10k/usb.h +++ b/drivers/net/wireless/ath/ath10k/usb.h @@ -26,7 +26,7 @@ #define ATH10K_USB_EP_ADDR_APP_DATA_MP_OUT 0x03 #define ATH10K_USB_EP_ADDR_APP_DATA_HP_OUT 0x04 -/* diagnostic command defnitions */ +/* diagnostic command definitions */ #define ATH10K_USB_CONTROL_REQ_SEND_BMI_CMD 1 #define ATH10K_USB_CONTROL_REQ_RECV_BMI_RESP 2 #define ATH10K_USB_CONTROL_REQ_DIAG_CMD 3 diff --git a/drivers/net/wireless/ath/ath10k/wmi-tlv.h b/drivers/net/wireless/ath/ath10k/wmi-tlv.h index b39c9b78b32b..dbb48d70f2e9 100644 --- a/drivers/net/wireless/ath/ath10k/wmi-tlv.h +++ b/drivers/net/wireless/ath/ath10k/wmi-tlv.h @@ -1813,7 +1813,7 @@ struct wmi_tlv_pdev_get_temp_cmd { struct wmi_tlv_pdev_temperature_event { __le32 tlv_hdr; - /* temperature value in Celcius degree */ + /* temperature value in Celsius degree */ __le32 temperature; __le32 pdev_id; } __packed; @@ -2548,7 +2548,7 @@ struct nlo_channel_prediction_cfg { /* Preconfigured stationary threshold. * Lesser value means more conservative. Bigger value means more aggressive. - * Maximum is 100 and mininum is 0. + * Maximum is 100 and minimum is 0. */ __le32 stationary_threshold; diff --git a/drivers/net/wireless/ath/ath10k/wmi.c b/drivers/net/wireless/ath/ath10k/wmi.c index 074d8ba5072a..980d4124fa28 100644 --- a/drivers/net/wireless/ath/ath10k/wmi.c +++ b/drivers/net/wireless/ath/ath10k/wmi.c @@ -3555,7 +3555,7 @@ static void ath10k_wmi_update_tim(struct ath10k *ar, __le32 t; u32 v, tim_len; - /* When FW reports 0 in tim_len, ensure atleast first byte + /* When FW reports 0 in tim_len, ensure at least first byte * in tim_bitmap is considered for pvm calculation. */ tim_len = tim_info->tim_len ? __le32_to_cpu(tim_info->tim_len) : 1; diff --git a/drivers/net/wireless/ath/ath10k/wmi.h b/drivers/net/wireless/ath/ath10k/wmi.h index 4abd12e78028..6de3cc4640a0 100644 --- a/drivers/net/wireless/ath/ath10k/wmi.h +++ b/drivers/net/wireless/ath/ath10k/wmi.h @@ -3170,7 +3170,7 @@ struct wmi_start_scan_common { /* dwell time in msec on passive channels */ __le32 dwell_time_passive; /* - * min time in msec on the BSS channel,only valid if atleast one + * min time in msec on the BSS channel,only valid if at least one * VDEV is active */ __le32 min_rest_time; @@ -3196,7 +3196,7 @@ struct wmi_start_scan_common { * and bssid_list */ __le32 repeat_probe_time; - /* time in msec between 2 consequetive probe requests with in a set. */ + /* time in msec between 2 consecutive probe requests with in a set. */ __le32 probe_spacing_time; /* * data inactivity time in msec on bss channel that will be used by @@ -4397,7 +4397,7 @@ struct wmi_pdev_stats_tx { /* wal pdev continuous xretry */ __le32 pdev_cont_xretry; - /* wal pdev continous xretry */ + /* wal pdev continuous xretry */ __le32 pdev_tx_timeout; /* wal pdev resets */ @@ -5240,7 +5240,7 @@ enum wmi_vdev_param { * scheduler. */ WMI_VDEV_OC_SCHEDULER_AIR_TIME_LIMIT, - /* enable/dsiable WDS for this VDEV */ + /* enable/disable WDS for this VDEV */ WMI_VDEV_PARAM_WDS, /* ATIM Window */ WMI_VDEV_PARAM_ATIM_WINDOW, @@ -5372,7 +5372,7 @@ enum wmi_10x_vdev_param { * scheduler. */ WMI_10X_VDEV_OC_SCHEDULER_AIR_TIME_LIMIT, - /* enable/dsiable WDS for this VDEV */ + /* enable/disable WDS for this VDEV */ WMI_10X_VDEV_PARAM_WDS, /* ATIM Window */ WMI_10X_VDEV_PARAM_ATIM_WINDOW, @@ -5904,7 +5904,7 @@ enum wmi_sta_ps_param_tx_wake_threshold { enum wmi_sta_ps_param_pspoll_count { WMI_STA_PS_PSPOLL_COUNT_NO_MAX = 0, /* - * Values greater than 0 indicate the maximum numer of PS-Poll frames + * Values greater than 0 indicate the maximum number of PS-Poll frames * FW will send before waking up. */ @@ -6947,7 +6947,7 @@ struct wmi_echo_ev_arg { }; struct wmi_pdev_temperature_event { - /* temperature value in Celcius degree */ + /* temperature value in Celsius degree */ __le32 temperature; } __packed; diff --git a/drivers/net/wireless/ath/ath11k/ahb.c b/drivers/net/wireless/ath/ath11k/ahb.c index c47414710138..d34a4d6325b2 100644 --- a/drivers/net/wireless/ath/ath11k/ahb.c +++ b/drivers/net/wireless/ath/ath11k/ahb.c @@ -16,6 +16,8 @@ #include "hif.h" #include <linux/remoteproc.h> #include "pcic.h" +#include <linux/soc/qcom/smem.h> +#include <linux/soc/qcom/smem_state.h> static const struct of_device_id ath11k_ahb_of_match[] = { /* TODO: Should we change the compatible string to something similar @@ -359,6 +361,7 @@ static void ath11k_ahb_ext_irq_enable(struct ath11k_base *ab) struct ath11k_ext_irq_grp *irq_grp = &ab->ext_irq_grp[i]; if (!irq_grp->napi_enabled) { + dev_set_threaded(&irq_grp->napi_ndev, true); napi_enable(&irq_grp->napi); irq_grp->napi_enabled = true; } @@ -406,7 +409,8 @@ static int ath11k_ahb_fwreset_from_cold_boot(struct ath11k_base *ab) int timeout; if (ath11k_cold_boot_cal == 0 || ab->qmi.cal_done || - ab->hw_params.cold_boot_calib == 0) + ab->hw_params.cold_boot_calib == 0 || + ab->hw_params.cbcal_restart_fw == 0) return 0; ath11k_dbg(ab, ATH11K_DBG_AHB, "wait for cold boot done\n"); @@ -541,7 +545,7 @@ static int ath11k_ahb_config_ext_irq(struct ath11k_base *ab) irq_grp->grp_id = i; init_dummy_netdev(&irq_grp->napi_ndev); netif_napi_add(&irq_grp->napi_ndev, &irq_grp->napi, - ath11k_ahb_ext_grp_napi_poll, NAPI_POLL_WEIGHT); + ath11k_ahb_ext_grp_napi_poll); for (j = 0; j < ATH11K_EXT_IRQ_NUM_MAX; j++) { if (ab->hw_params.ring_mask->tx[i] & BIT(j)) { @@ -685,11 +689,90 @@ static int ath11k_ahb_map_service_to_pipe(struct ath11k_base *ab, u16 service_id return 0; } +static int ath11k_ahb_hif_suspend(struct ath11k_base *ab) +{ + struct ath11k_ahb *ab_ahb = ath11k_ahb_priv(ab); + u32 wake_irq; + u32 value = 0; + int ret; + + if (!device_may_wakeup(ab->dev)) + return -EPERM; + + wake_irq = ab->irq_num[ATH11K_PCI_IRQ_CE0_OFFSET + ATH11K_PCI_CE_WAKE_IRQ]; + + ret = enable_irq_wake(wake_irq); + if (ret) { + ath11k_err(ab, "failed to enable wakeup irq :%d\n", ret); + return ret; + } + + value = u32_encode_bits(ab_ahb->smp2p_info.seq_no++, + ATH11K_AHB_SMP2P_SMEM_SEQ_NO); + value |= u32_encode_bits(ATH11K_AHB_POWER_SAVE_ENTER, + ATH11K_AHB_SMP2P_SMEM_MSG); + + ret = qcom_smem_state_update_bits(ab_ahb->smp2p_info.smem_state, + ATH11K_AHB_SMP2P_SMEM_VALUE_MASK, value); + if (ret) { + ath11k_err(ab, "failed to send smp2p power save enter cmd :%d\n", ret); + return ret; + } + + ath11k_dbg(ab, ATH11K_DBG_AHB, "ahb device suspended\n"); + + return ret; +} + +static int ath11k_ahb_hif_resume(struct ath11k_base *ab) +{ + struct ath11k_ahb *ab_ahb = ath11k_ahb_priv(ab); + u32 wake_irq; + u32 value = 0; + int ret; + + if (!device_may_wakeup(ab->dev)) + return -EPERM; + + wake_irq = ab->irq_num[ATH11K_PCI_IRQ_CE0_OFFSET + ATH11K_PCI_CE_WAKE_IRQ]; + + ret = disable_irq_wake(wake_irq); + if (ret) { + ath11k_err(ab, "failed to disable wakeup irq: %d\n", ret); + return ret; + } + + reinit_completion(&ab->wow.wakeup_completed); + + value = u32_encode_bits(ab_ahb->smp2p_info.seq_no++, + ATH11K_AHB_SMP2P_SMEM_SEQ_NO); + value |= u32_encode_bits(ATH11K_AHB_POWER_SAVE_EXIT, + ATH11K_AHB_SMP2P_SMEM_MSG); + + ret = qcom_smem_state_update_bits(ab_ahb->smp2p_info.smem_state, + ATH11K_AHB_SMP2P_SMEM_VALUE_MASK, value); + if (ret) { + ath11k_err(ab, "failed to send smp2p power save enter cmd :%d\n", ret); + return ret; + } + + ret = wait_for_completion_timeout(&ab->wow.wakeup_completed, 3 * HZ); + if (ret == 0) { + ath11k_warn(ab, "timed out while waiting for wow wakeup completion\n"); + return -ETIMEDOUT; + } + + ath11k_dbg(ab, ATH11K_DBG_AHB, "ahb device resumed\n"); + + return 0; +} + static const struct ath11k_hif_ops ath11k_ahb_hif_ops_ipq8074 = { .start = ath11k_ahb_start, .stop = ath11k_ahb_stop, .read32 = ath11k_ahb_read32, .write32 = ath11k_ahb_write32, + .read = NULL, .irq_enable = ath11k_ahb_ext_irq_enable, .irq_disable = ath11k_ahb_ext_irq_disable, .map_service_to_pipe = ath11k_ahb_map_service_to_pipe, @@ -702,6 +785,7 @@ static const struct ath11k_hif_ops ath11k_ahb_hif_ops_wcn6750 = { .stop = ath11k_pcic_stop, .read32 = ath11k_pcic_read32, .write32 = ath11k_pcic_write32, + .read = NULL, .irq_enable = ath11k_pcic_ext_irq_enable, .irq_disable = ath11k_pcic_ext_irq_disable, .get_msi_address = ath11k_pcic_get_msi_address, @@ -709,6 +793,10 @@ static const struct ath11k_hif_ops ath11k_ahb_hif_ops_wcn6750 = { .map_service_to_pipe = ath11k_pcic_map_service_to_pipe, .power_down = ath11k_ahb_power_down, .power_up = ath11k_ahb_power_up, + .suspend = ath11k_ahb_hif_suspend, + .resume = ath11k_ahb_hif_resume, + .ce_irq_enable = ath11k_pci_enable_ce_irqs_except_wake_irq, + .ce_irq_disable = ath11k_pci_disable_ce_irqs_except_wake_irq, }; static int ath11k_core_get_rproc(struct ath11k_base *ab) @@ -783,6 +871,34 @@ static int ath11k_ahb_setup_msi_resources(struct ath11k_base *ab) return 0; } +static int ath11k_ahb_setup_smp2p_handle(struct ath11k_base *ab) +{ + struct ath11k_ahb *ab_ahb = ath11k_ahb_priv(ab); + + if (!ab->hw_params.smp2p_wow_exit) + return 0; + + ab_ahb->smp2p_info.smem_state = qcom_smem_state_get(ab->dev, "wlan-smp2p-out", + &ab_ahb->smp2p_info.smem_bit); + if (IS_ERR(ab_ahb->smp2p_info.smem_state)) { + ath11k_err(ab, "failed to fetch smem state: %ld\n", + PTR_ERR(ab_ahb->smp2p_info.smem_state)); + return PTR_ERR(ab_ahb->smp2p_info.smem_state); + } + + return 0; +} + +static void ath11k_ahb_release_smp2p_handle(struct ath11k_base *ab) +{ + struct ath11k_ahb *ab_ahb = ath11k_ahb_priv(ab); + + if (!ab->hw_params.smp2p_wow_exit) + return; + + qcom_smem_state_put(ab_ahb->smp2p_info.smem_state); +} + static int ath11k_ahb_setup_resources(struct ath11k_base *ab) { struct platform_device *pdev = ab->pdev; @@ -1038,10 +1154,14 @@ static int ath11k_ahb_probe(struct platform_device *pdev) if (ret) goto err_core_free; - ret = ath11k_hal_srng_init(ab); + ret = ath11k_ahb_setup_smp2p_handle(ab); if (ret) goto err_fw_deinit; + ret = ath11k_hal_srng_init(ab); + if (ret) + goto err_release_smp2p_handle; + ret = ath11k_ce_alloc_pipes(ab); if (ret) { ath11k_err(ab, "failed to allocate ce pipes: %d\n", ret); @@ -1078,6 +1198,9 @@ err_ce_free: err_hal_srng_deinit: ath11k_hal_srng_deinit(ab); +err_release_smp2p_handle: + ath11k_ahb_release_smp2p_handle(ab); + err_fw_deinit: ath11k_ahb_fw_resource_deinit(ab); @@ -1088,20 +1211,10 @@ err_core_free: return ret; } -static int ath11k_ahb_remove(struct platform_device *pdev) +static void ath11k_ahb_remove_prepare(struct ath11k_base *ab) { - struct ath11k_base *ab = platform_get_drvdata(pdev); unsigned long left; - if (test_bit(ATH11K_FLAG_QMI_FAIL, &ab->dev_flags)) { - ath11k_ahb_power_down(ab); - ath11k_debugfs_soc_destroy(ab); - ath11k_qmi_deinit_service(ab); - goto qmi_fail; - } - - reinit_completion(&ab->driver_recovery); - if (test_bit(ATH11K_FLAG_RECOVERY, &ab->dev_flags)) { left = wait_for_completion_timeout(&ab->driver_recovery, ATH11K_AHB_RECOVERY_TIMEOUT); @@ -1111,19 +1224,61 @@ static int ath11k_ahb_remove(struct platform_device *pdev) set_bit(ATH11K_FLAG_UNREGISTERING, &ab->dev_flags); cancel_work_sync(&ab->restart_work); + cancel_work_sync(&ab->qmi.event_work); +} + +static void ath11k_ahb_free_resources(struct ath11k_base *ab) +{ + struct platform_device *pdev = ab->pdev; - ath11k_core_deinit(ab); -qmi_fail: ath11k_ahb_free_irq(ab); ath11k_hal_srng_deinit(ab); + ath11k_ahb_release_smp2p_handle(ab); ath11k_ahb_fw_resource_deinit(ab); ath11k_ce_free_pipes(ab); ath11k_core_free(ab); platform_set_drvdata(pdev, NULL); +} + +static int ath11k_ahb_remove(struct platform_device *pdev) +{ + struct ath11k_base *ab = platform_get_drvdata(pdev); + + if (test_bit(ATH11K_FLAG_QMI_FAIL, &ab->dev_flags)) { + ath11k_ahb_power_down(ab); + ath11k_debugfs_soc_destroy(ab); + ath11k_qmi_deinit_service(ab); + goto qmi_fail; + } + + ath11k_ahb_remove_prepare(ab); + ath11k_core_deinit(ab); + +qmi_fail: + ath11k_ahb_free_resources(ab); return 0; } +static void ath11k_ahb_shutdown(struct platform_device *pdev) +{ + struct ath11k_base *ab = platform_get_drvdata(pdev); + + /* platform shutdown() & remove() are mutually exclusive. + * remove() is invoked during rmmod & shutdown() during + * system reboot/shutdown. + */ + ath11k_ahb_remove_prepare(ab); + + if (!(test_bit(ATH11K_FLAG_REGISTERED, &ab->dev_flags))) + goto free_resources; + + ath11k_core_deinit(ab); + +free_resources: + ath11k_ahb_free_resources(ab); +} + static struct platform_driver ath11k_ahb_driver = { .driver = { .name = "ath11k", @@ -1131,6 +1286,7 @@ static struct platform_driver ath11k_ahb_driver = { }, .probe = ath11k_ahb_probe, .remove = ath11k_ahb_remove, + .shutdown = ath11k_ahb_shutdown, }; static int ath11k_ahb_init(void) diff --git a/drivers/net/wireless/ath/ath11k/ahb.h b/drivers/net/wireless/ath/ath11k/ahb.h index 58a945411c5b..415ddfd26654 100644 --- a/drivers/net/wireless/ath/ath11k/ahb.h +++ b/drivers/net/wireless/ath/ath11k/ahb.h @@ -1,6 +1,7 @@ /* SPDX-License-Identifier: BSD-3-Clause-Clear */ /* * Copyright (c) 2018-2019 The Linux Foundation. All rights reserved. + * Copyright (c) 2022, Qualcomm Innovation Center, Inc. All rights reserved. */ #ifndef ATH11K_AHB_H #define ATH11K_AHB_H @@ -8,6 +9,16 @@ #include "core.h" #define ATH11K_AHB_RECOVERY_TIMEOUT (3 * HZ) + +#define ATH11K_AHB_SMP2P_SMEM_MSG GENMASK(15, 0) +#define ATH11K_AHB_SMP2P_SMEM_SEQ_NO GENMASK(31, 16) +#define ATH11K_AHB_SMP2P_SMEM_VALUE_MASK 0xFFFFFFFF + +enum ath11k_ahb_smp2p_msg_id { + ATH11K_AHB_POWER_SAVE_ENTER = 1, + ATH11K_AHB_POWER_SAVE_EXIT, +}; + struct ath11k_base; struct ath11k_ahb { @@ -21,6 +32,11 @@ struct ath11k_ahb { u32 ce_size; bool use_tz; } fw; + struct { + unsigned short seq_no; + unsigned int smem_bit; + struct qcom_smem_state *smem_state; + } smp2p_info; }; static inline struct ath11k_ahb *ath11k_ahb_priv(struct ath11k_base *ab) diff --git a/drivers/net/wireless/ath/ath11k/ce.c b/drivers/net/wireless/ath/ath11k/ce.c index c14c51f38709..f2da95fd4253 100644 --- a/drivers/net/wireless/ath/ath11k/ce.c +++ b/drivers/net/wireless/ath/ath11k/ce.c @@ -250,7 +250,7 @@ const struct ce_attr ath11k_host_ce_config_qcn9074[] = { static bool ath11k_ce_need_shadow_fix(int ce_id) { - /* only ce4 needs shadow workaroud*/ + /* only ce4 needs shadow workaround */ if (ce_id == 4) return true; return false; @@ -1042,7 +1042,7 @@ int ath11k_ce_alloc_pipes(struct ath11k_base *ab) ret = ath11k_ce_alloc_pipe(ab, i); if (ret) { - /* Free any parial successful allocation */ + /* Free any partial successful allocation */ ath11k_ce_free_pipes(ab); return ret; } diff --git a/drivers/net/wireless/ath/ath11k/core.c b/drivers/net/wireless/ath/ath11k/core.c index c3e9e4f7bc24..b99180bc8172 100644 --- a/drivers/net/wireless/ath/ath11k/core.c +++ b/drivers/net/wireless/ath/ath11k/core.c @@ -70,6 +70,7 @@ static const struct ath11k_hw_params ath11k_hw_params[] = { .summary_pad_sz = 0, .fft_hdr_len = 16, .max_fft_bins = 512, + .fragment_160mhz = true, }, .interface_modes = BIT(NL80211_IFTYPE_STATION) | @@ -81,6 +82,7 @@ static const struct ath11k_hw_params ath11k_hw_params[] = { .idle_ps = false, .supports_sta_ps = false, .cold_boot_calib = true, + .cbcal_restart_fw = true, .fw_mem_mode = 0, .num_vdevs = 16 + 1, .num_peers = 512, @@ -106,6 +108,13 @@ static const struct ath11k_hw_params ath11k_hw_params[] = { .hybrid_bus_type = false, .fixed_fw_mem = false, .support_off_channel_tx = false, + .supports_multi_bssid = false, + + .sram_dump = {}, + + .tcl_ring_retry = true, + .tx_ring_size = DP_TCL_DATA_RING_SIZE, + .smp2p_wow_exit = false, }, { .hw_rev = ATH11K_HW_IPQ6018_HW10, @@ -141,6 +150,7 @@ static const struct ath11k_hw_params ath11k_hw_params[] = { .summary_pad_sz = 0, .fft_hdr_len = 16, .max_fft_bins = 512, + .fragment_160mhz = true, }, .interface_modes = BIT(NL80211_IFTYPE_STATION) | @@ -152,6 +162,7 @@ static const struct ath11k_hw_params ath11k_hw_params[] = { .idle_ps = false, .supports_sta_ps = false, .cold_boot_calib = true, + .cbcal_restart_fw = true, .fw_mem_mode = 0, .num_vdevs = 16 + 1, .num_peers = 512, @@ -177,6 +188,13 @@ static const struct ath11k_hw_params ath11k_hw_params[] = { .hybrid_bus_type = false, .fixed_fw_mem = false, .support_off_channel_tx = false, + .supports_multi_bssid = false, + + .sram_dump = {}, + + .tcl_ring_retry = true, + .tx_ring_size = DP_TCL_DATA_RING_SIZE, + .smp2p_wow_exit = false, }, { .name = "qca6390 hw2.0", @@ -212,6 +230,7 @@ static const struct ath11k_hw_params ath11k_hw_params[] = { .summary_pad_sz = 0, .fft_hdr_len = 0, .max_fft_bins = 0, + .fragment_160mhz = false, }, .interface_modes = BIT(NL80211_IFTYPE_STATION) | @@ -222,6 +241,7 @@ static const struct ath11k_hw_params ath11k_hw_params[] = { .idle_ps = true, .supports_sta_ps = true, .cold_boot_calib = false, + .cbcal_restart_fw = false, .fw_mem_mode = 0, .num_vdevs = 16 + 1, .num_peers = 512, @@ -247,6 +267,16 @@ static const struct ath11k_hw_params ath11k_hw_params[] = { .hybrid_bus_type = false, .fixed_fw_mem = false, .support_off_channel_tx = true, + .supports_multi_bssid = true, + + .sram_dump = { + .start = 0x01400000, + .end = 0x0171ffff, + }, + + .tcl_ring_retry = true, + .tx_ring_size = DP_TCL_DATA_RING_SIZE, + .smp2p_wow_exit = false, }, { .name = "qcn9074 hw1.0", @@ -281,6 +311,7 @@ static const struct ath11k_hw_params ath11k_hw_params[] = { .summary_pad_sz = 16, .fft_hdr_len = 24, .max_fft_bins = 1024, + .fragment_160mhz = false, }, .interface_modes = BIT(NL80211_IFTYPE_STATION) | @@ -292,6 +323,7 @@ static const struct ath11k_hw_params ath11k_hw_params[] = { .idle_ps = false, .supports_sta_ps = false, .cold_boot_calib = false, + .cbcal_restart_fw = false, .fw_mem_mode = 2, .num_vdevs = 8, .num_peers = 128, @@ -317,6 +349,13 @@ static const struct ath11k_hw_params ath11k_hw_params[] = { .hybrid_bus_type = false, .fixed_fw_mem = false, .support_off_channel_tx = false, + .supports_multi_bssid = false, + + .sram_dump = {}, + + .tcl_ring_retry = true, + .tx_ring_size = DP_TCL_DATA_RING_SIZE, + .smp2p_wow_exit = false, }, { .name = "wcn6855 hw2.0", @@ -352,6 +391,7 @@ static const struct ath11k_hw_params ath11k_hw_params[] = { .summary_pad_sz = 0, .fft_hdr_len = 0, .max_fft_bins = 0, + .fragment_160mhz = false, }, .interface_modes = BIT(NL80211_IFTYPE_STATION) | @@ -362,6 +402,7 @@ static const struct ath11k_hw_params ath11k_hw_params[] = { .idle_ps = true, .supports_sta_ps = true, .cold_boot_calib = false, + .cbcal_restart_fw = false, .fw_mem_mode = 0, .num_vdevs = 16 + 1, .num_peers = 512, @@ -387,6 +428,16 @@ static const struct ath11k_hw_params ath11k_hw_params[] = { .hybrid_bus_type = false, .fixed_fw_mem = false, .support_off_channel_tx = true, + .supports_multi_bssid = true, + + .sram_dump = { + .start = 0x01400000, + .end = 0x0177ffff, + }, + + .tcl_ring_retry = true, + .tx_ring_size = DP_TCL_DATA_RING_SIZE, + .smp2p_wow_exit = false, }, { .name = "wcn6855 hw2.1", @@ -422,6 +473,7 @@ static const struct ath11k_hw_params ath11k_hw_params[] = { .summary_pad_sz = 0, .fft_hdr_len = 0, .max_fft_bins = 0, + .fragment_160mhz = false, }, .interface_modes = BIT(NL80211_IFTYPE_STATION) | @@ -431,6 +483,7 @@ static const struct ath11k_hw_params ath11k_hw_params[] = { .idle_ps = true, .supports_sta_ps = true, .cold_boot_calib = false, + .cbcal_restart_fw = false, .fw_mem_mode = 0, .num_vdevs = 16 + 1, .num_peers = 512, @@ -456,6 +509,16 @@ static const struct ath11k_hw_params ath11k_hw_params[] = { .hybrid_bus_type = false, .fixed_fw_mem = false, .support_off_channel_tx = true, + .supports_multi_bssid = true, + + .sram_dump = { + .start = 0x01400000, + .end = 0x0177ffff, + }, + + .tcl_ring_retry = true, + .tx_ring_size = DP_TCL_DATA_RING_SIZE, + .smp2p_wow_exit = false, }, { .name = "wcn6750 hw1.0", @@ -468,7 +531,7 @@ static const struct ath11k_hw_params ath11k_hw_params[] = { .max_radios = 1, .bdf_addr = 0x4B0C0000, .hw_ops = &wcn6750_ops, - .ring_mask = &ath11k_hw_ring_mask_qca6390, + .ring_mask = &ath11k_hw_ring_mask_wcn6750, .internal_sleep_clock = false, .regs = &wcn6750_regs, .qmi_service_ins_id = ATH11K_QMI_WLFW_SERVICE_INS_ID_V01_WCN6750, @@ -491,6 +554,7 @@ static const struct ath11k_hw_params ath11k_hw_params[] = { .summary_pad_sz = 0, .fft_hdr_len = 0, .max_fft_bins = 0, + .fragment_160mhz = false, }, .interface_modes = BIT(NL80211_IFTYPE_STATION) | @@ -499,7 +563,8 @@ static const struct ath11k_hw_params ath11k_hw_params[] = { .supports_shadow_regs = true, .idle_ps = true, .supports_sta_ps = true, - .cold_boot_calib = false, + .cold_boot_calib = true, + .cbcal_restart_fw = false, .fw_mem_mode = 0, .num_vdevs = 16 + 1, .num_peers = 512, @@ -508,8 +573,8 @@ static const struct ath11k_hw_params ath11k_hw_params[] = { .supports_regdb = true, .fix_l1ss = false, .credit_flow = true, - .max_tx_ring = DP_TCL_NUM_RING_MAX_QCA6390, - .hal_params = &ath11k_hw_hal_params_qca6390, + .max_tx_ring = DP_TCL_NUM_RING_MAX, + .hal_params = &ath11k_hw_hal_params_wcn6750, .supports_dynamic_smps_6ghz = false, .alloc_cacheable_memory = false, .supports_rssi_stats = true, @@ -524,7 +589,14 @@ static const struct ath11k_hw_params ath11k_hw_params[] = { .static_window_map = true, .hybrid_bus_type = true, .fixed_fw_mem = true, - .support_off_channel_tx = false, + .support_off_channel_tx = true, + .supports_multi_bssid = true, + + .sram_dump = {}, + + .tcl_ring_retry = false, + .tx_ring_size = DP_TCL_DATA_RING_SIZE_WCN6750, + .smp2p_wow_exit = true, }, }; @@ -535,6 +607,52 @@ static inline struct ath11k_pdev *ath11k_core_get_single_pdev(struct ath11k_base return &ab->pdevs[0]; } +void ath11k_fw_stats_pdevs_free(struct list_head *head) +{ + struct ath11k_fw_stats_pdev *i, *tmp; + + list_for_each_entry_safe(i, tmp, head, list) { + list_del(&i->list); + kfree(i); + } +} + +void ath11k_fw_stats_vdevs_free(struct list_head *head) +{ + struct ath11k_fw_stats_vdev *i, *tmp; + + list_for_each_entry_safe(i, tmp, head, list) { + list_del(&i->list); + kfree(i); + } +} + +void ath11k_fw_stats_bcn_free(struct list_head *head) +{ + struct ath11k_fw_stats_bcn *i, *tmp; + + list_for_each_entry_safe(i, tmp, head, list) { + list_del(&i->list); + kfree(i); + } +} + +void ath11k_fw_stats_init(struct ath11k *ar) +{ + INIT_LIST_HEAD(&ar->fw_stats.pdevs); + INIT_LIST_HEAD(&ar->fw_stats.vdevs); + INIT_LIST_HEAD(&ar->fw_stats.bcn); + + init_completion(&ar->fw_stats_complete); +} + +void ath11k_fw_stats_free(struct ath11k_fw_stats *stats) +{ + ath11k_fw_stats_pdevs_free(&stats->pdevs); + ath11k_fw_stats_vdevs_free(&stats->vdevs); + ath11k_fw_stats_bcn_free(&stats->bcn); +} + int ath11k_core_suspend(struct ath11k_base *ab) { int ret; @@ -1544,7 +1662,7 @@ static void ath11k_core_pre_reconfigure_recovery(struct ath11k_base *ab) ar->state_11d = ATH11K_11D_IDLE; complete(&ar->completed_11d_scan); complete(&ar->scan.started); - complete(&ar->scan.completed); + complete_all(&ar->scan.completed); complete(&ar->scan.on_channel); complete(&ar->peer_assoc_done); complete(&ar->peer_delete_done); @@ -1563,6 +1681,8 @@ static void ath11k_core_pre_reconfigure_recovery(struct ath11k_base *ab) wake_up(&ab->wmi_ab.tx_credits_wq); wake_up(&ab->peer_mapping_wq); + + reinit_completion(&ab->driver_recovery); } static void ath11k_core_post_reconfigure_recovery(struct ath11k_base *ab) diff --git a/drivers/net/wireless/ath/ath11k/core.h b/drivers/net/wireless/ath/ath11k/core.h index afad8f55e433..cf2f52cc4e30 100644 --- a/drivers/net/wireless/ath/ath11k/core.h +++ b/drivers/net/wireless/ath/ath11k/core.h @@ -498,6 +498,13 @@ struct ath11k_sta { bool use_4addr_set; u16 tcl_metadata; + + /* Protected with ar->data_lock */ + enum ath11k_wmi_peer_ps_state peer_ps_state; + u64 ps_start_time; + u64 ps_start_jiffies; + u64 ps_total_duration; + bool peer_current_ps_valid; }; #define ATH11K_MIN_5G_FREQ 4150 @@ -545,9 +552,6 @@ struct ath11k_debug { struct dentry *debugfs_pdev; struct ath11k_dbg_htt_stats htt_stats; u32 extd_tx_stats; - struct ath11k_fw_stats fw_stats; - struct completion fw_stats_complete; - bool fw_stats_done; u32 extd_rx_stats; u32 pktlog_filter; u32 pktlog_mode; @@ -710,6 +714,13 @@ struct ath11k { u8 twt_enabled; bool nlo_enabled; u8 alpha2[REG_ALPHA2_LEN + 1]; + struct ath11k_fw_stats fw_stats; + struct completion fw_stats_complete; + bool fw_stats_done; + + /* protected by conf_mutex */ + bool ps_state_enable; + bool ps_timekeeper_enable; }; struct ath11k_band_cap { @@ -887,7 +898,7 @@ struct ath11k_base { /* Below regd's are protected by ab->data_lock */ /* This is the regd set for every radio - * by the firmware during initializatin + * by the firmware during initialization */ struct ieee80211_regdomain *default_regd[MAX_RADIOS]; /* This regd is set during dynamic country setting @@ -1112,6 +1123,12 @@ struct ath11k_fw_stats_bcn { u32 tx_bcn_outage_cnt; }; +void ath11k_fw_stats_init(struct ath11k *ar); +void ath11k_fw_stats_pdevs_free(struct list_head *head); +void ath11k_fw_stats_vdevs_free(struct list_head *head); +void ath11k_fw_stats_bcn_free(struct list_head *head); +void ath11k_fw_stats_free(struct ath11k_fw_stats *stats); + extern const struct ce_pipe_config ath11k_target_ce_config_wlan_ipq8074[]; extern const struct service_to_pipe ath11k_target_service_to_ce_map_wlan_ipq8074[]; extern const struct service_to_pipe ath11k_target_service_to_ce_map_wlan_ipq6018[]; diff --git a/drivers/net/wireless/ath/ath11k/debugfs.c b/drivers/net/wireless/ath/ath11k/debugfs.c index 9648e0017393..ccdf3d5ba1ab 100644 --- a/drivers/net/wireless/ath/ath11k/debugfs.c +++ b/drivers/net/wireless/ath/ath11k/debugfs.c @@ -14,6 +14,7 @@ #include "dp_tx.h" #include "debugfs_htt_stats.h" #include "peer.h" +#include "hif.h" static const char *htt_bp_umac_ring[HTT_SW_UMAC_RING_IDX_MAX] = { "REO2SW1_RING", @@ -91,91 +92,35 @@ void ath11k_debugfs_add_dbring_entry(struct ath11k *ar, spin_unlock_bh(&dbr_data->lock); } -static void ath11k_fw_stats_pdevs_free(struct list_head *head) -{ - struct ath11k_fw_stats_pdev *i, *tmp; - - list_for_each_entry_safe(i, tmp, head, list) { - list_del(&i->list); - kfree(i); - } -} - -static void ath11k_fw_stats_vdevs_free(struct list_head *head) -{ - struct ath11k_fw_stats_vdev *i, *tmp; - - list_for_each_entry_safe(i, tmp, head, list) { - list_del(&i->list); - kfree(i); - } -} - -static void ath11k_fw_stats_bcn_free(struct list_head *head) -{ - struct ath11k_fw_stats_bcn *i, *tmp; - - list_for_each_entry_safe(i, tmp, head, list) { - list_del(&i->list); - kfree(i); - } -} - static void ath11k_debugfs_fw_stats_reset(struct ath11k *ar) { spin_lock_bh(&ar->data_lock); - ar->debug.fw_stats_done = false; - ath11k_fw_stats_pdevs_free(&ar->debug.fw_stats.pdevs); - ath11k_fw_stats_vdevs_free(&ar->debug.fw_stats.vdevs); + ar->fw_stats_done = false; + ath11k_fw_stats_pdevs_free(&ar->fw_stats.pdevs); + ath11k_fw_stats_vdevs_free(&ar->fw_stats.vdevs); spin_unlock_bh(&ar->data_lock); } -void ath11k_debugfs_fw_stats_process(struct ath11k_base *ab, struct sk_buff *skb) +void ath11k_debugfs_fw_stats_process(struct ath11k *ar, struct ath11k_fw_stats *stats) { - struct ath11k_fw_stats stats = {}; - struct ath11k *ar; + struct ath11k_base *ab = ar->ab; struct ath11k_pdev *pdev; bool is_end; static unsigned int num_vdev, num_bcn; size_t total_vdevs_started = 0; - int i, ret; - - INIT_LIST_HEAD(&stats.pdevs); - INIT_LIST_HEAD(&stats.vdevs); - INIT_LIST_HEAD(&stats.bcn); - - ret = ath11k_wmi_pull_fw_stats(ab, skb, &stats); - if (ret) { - ath11k_warn(ab, "failed to pull fw stats: %d\n", ret); - goto free; - } - - rcu_read_lock(); - ar = ath11k_mac_get_ar_by_pdev_id(ab, stats.pdev_id); - if (!ar) { - rcu_read_unlock(); - ath11k_warn(ab, "failed to get ar for pdev_id %d: %d\n", - stats.pdev_id, ret); - goto free; - } + int i; - spin_lock_bh(&ar->data_lock); + /* WMI_REQUEST_PDEV_STAT request has been already processed */ - if (stats.stats_id == WMI_REQUEST_PDEV_STAT) { - list_splice_tail_init(&stats.pdevs, &ar->debug.fw_stats.pdevs); - ar->debug.fw_stats_done = true; - goto complete; - } - - if (stats.stats_id == WMI_REQUEST_RSSI_PER_CHAIN_STAT) { - ar->debug.fw_stats_done = true; - goto complete; + if (stats->stats_id == WMI_REQUEST_RSSI_PER_CHAIN_STAT) { + ar->fw_stats_done = true; + return; } - if (stats.stats_id == WMI_REQUEST_VDEV_STAT) { - if (list_empty(&stats.vdevs)) { + if (stats->stats_id == WMI_REQUEST_VDEV_STAT) { + if (list_empty(&stats->vdevs)) { ath11k_warn(ab, "empty vdev stats"); - goto complete; + return; } /* FW sends all the active VDEV stats irrespective of PDEV, * hence limit until the count of all VDEVs started @@ -188,43 +133,34 @@ void ath11k_debugfs_fw_stats_process(struct ath11k_base *ab, struct sk_buff *skb is_end = ((++num_vdev) == total_vdevs_started); - list_splice_tail_init(&stats.vdevs, - &ar->debug.fw_stats.vdevs); + list_splice_tail_init(&stats->vdevs, + &ar->fw_stats.vdevs); if (is_end) { - ar->debug.fw_stats_done = true; + ar->fw_stats_done = true; num_vdev = 0; } - goto complete; + return; } - if (stats.stats_id == WMI_REQUEST_BCN_STAT) { - if (list_empty(&stats.bcn)) { + if (stats->stats_id == WMI_REQUEST_BCN_STAT) { + if (list_empty(&stats->bcn)) { ath11k_warn(ab, "empty bcn stats"); - goto complete; + return; } /* Mark end until we reached the count of all started VDEVs * within the PDEV */ is_end = ((++num_bcn) == ar->num_started_vdevs); - list_splice_tail_init(&stats.bcn, - &ar->debug.fw_stats.bcn); + list_splice_tail_init(&stats->bcn, + &ar->fw_stats.bcn); if (is_end) { - ar->debug.fw_stats_done = true; + ar->fw_stats_done = true; num_bcn = 0; } } -complete: - complete(&ar->debug.fw_stats_complete); - rcu_read_unlock(); - spin_unlock_bh(&ar->data_lock); - -free: - ath11k_fw_stats_pdevs_free(&stats.pdevs); - ath11k_fw_stats_vdevs_free(&stats.vdevs); - ath11k_fw_stats_bcn_free(&stats.bcn); } static int ath11k_debugfs_fw_stats_request(struct ath11k *ar, @@ -245,7 +181,7 @@ static int ath11k_debugfs_fw_stats_request(struct ath11k *ar, ath11k_debugfs_fw_stats_reset(ar); - reinit_completion(&ar->debug.fw_stats_complete); + reinit_completion(&ar->fw_stats_complete); ret = ath11k_wmi_send_stats_request_cmd(ar, req_param); @@ -255,9 +191,8 @@ static int ath11k_debugfs_fw_stats_request(struct ath11k *ar, return ret; } - time_left = - wait_for_completion_timeout(&ar->debug.fw_stats_complete, - 1 * HZ); + time_left = wait_for_completion_timeout(&ar->fw_stats_complete, 1 * HZ); + if (!time_left) return -ETIMEDOUT; @@ -266,7 +201,7 @@ static int ath11k_debugfs_fw_stats_request(struct ath11k *ar, break; spin_lock_bh(&ar->data_lock); - if (ar->debug.fw_stats_done) { + if (ar->fw_stats_done) { spin_unlock_bh(&ar->data_lock); break; } @@ -338,8 +273,7 @@ static int ath11k_open_pdev_stats(struct inode *inode, struct file *file) goto err_free; } - ath11k_wmi_fw_stats_fill(ar, &ar->debug.fw_stats, req_param.stats_id, - buf); + ath11k_wmi_fw_stats_fill(ar, &ar->fw_stats, req_param.stats_id, buf); file->private_data = buf; @@ -410,8 +344,7 @@ static int ath11k_open_vdev_stats(struct inode *inode, struct file *file) goto err_free; } - ath11k_wmi_fw_stats_fill(ar, &ar->debug.fw_stats, req_param.stats_id, - buf); + ath11k_wmi_fw_stats_fill(ar, &ar->fw_stats, req_param.stats_id, buf); file->private_data = buf; @@ -488,14 +421,13 @@ static int ath11k_open_bcn_stats(struct inode *inode, struct file *file) } } - ath11k_wmi_fw_stats_fill(ar, &ar->debug.fw_stats, req_param.stats_id, - buf); + ath11k_wmi_fw_stats_fill(ar, &ar->fw_stats, req_param.stats_id, buf); /* since beacon stats request is looped for all active VDEVs, saved fw * stats is not freed for each request until done for all active VDEVs */ spin_lock_bh(&ar->data_lock); - ath11k_fw_stats_bcn_free(&ar->debug.fw_stats.bcn); + ath11k_fw_stats_bcn_free(&ar->fw_stats.bcn); spin_unlock_bh(&ar->data_lock); file->private_data = buf; @@ -982,6 +914,63 @@ static const struct file_operations fops_fw_dbglog = { .llseek = default_llseek, }; +static int ath11k_open_sram_dump(struct inode *inode, struct file *file) +{ + struct ath11k_base *ab = inode->i_private; + u8 *buf; + u32 start, end; + int ret; + + start = ab->hw_params.sram_dump.start; + end = ab->hw_params.sram_dump.end; + + buf = vmalloc(end - start + 1); + if (!buf) + return -ENOMEM; + + ret = ath11k_hif_read(ab, buf, start, end); + if (ret) { + ath11k_warn(ab, "failed to dump sram: %d\n", ret); + vfree(buf); + return ret; + } + + file->private_data = buf; + return 0; +} + +static ssize_t ath11k_read_sram_dump(struct file *file, + char __user *user_buf, + size_t count, loff_t *ppos) +{ + struct ath11k_base *ab = file->f_inode->i_private; + const char *buf = file->private_data; + int len; + u32 start, end; + + start = ab->hw_params.sram_dump.start; + end = ab->hw_params.sram_dump.end; + len = end - start + 1; + + return simple_read_from_buffer(user_buf, count, ppos, buf, len); +} + +static int ath11k_release_sram_dump(struct inode *inode, struct file *file) +{ + vfree(file->private_data); + file->private_data = NULL; + + return 0; +} + +static const struct file_operations fops_sram_dump = { + .open = ath11k_open_sram_dump, + .read = ath11k_read_sram_dump, + .release = ath11k_release_sram_dump, + .owner = THIS_MODULE, + .llseek = default_llseek, +}; + int ath11k_debugfs_pdev_create(struct ath11k_base *ab) { if (test_bit(ATH11K_FLAG_REGISTERED, &ab->dev_flags)) @@ -997,6 +986,10 @@ int ath11k_debugfs_pdev_create(struct ath11k_base *ab) debugfs_create_file("soc_dp_stats", 0600, ab->debugfs_soc, ab, &fops_soc_dp_stats); + if (ab->hw_params.sram_dump.start != 0) + debugfs_create_file("sram", 0400, ab->debugfs_soc, ab, + &fops_sram_dump); + return 0; } @@ -1025,7 +1018,7 @@ void ath11k_debugfs_fw_stats_init(struct ath11k *ar) struct dentry *fwstats_dir = debugfs_create_dir("fw_stats", ar->debug.debugfs_pdev); - ar->debug.fw_stats.debugfs_fwstats = fwstats_dir; + ar->fw_stats.debugfs_fwstats = fwstats_dir; /* all stats debugfs files created are under "fw_stats" directory * created per PDEV @@ -1036,12 +1029,6 @@ void ath11k_debugfs_fw_stats_init(struct ath11k *ar) &fops_vdev_stats); debugfs_create_file("beacon_stats", 0600, fwstats_dir, ar, &fops_bcn_stats); - - INIT_LIST_HEAD(&ar->debug.fw_stats.pdevs); - INIT_LIST_HEAD(&ar->debug.fw_stats.vdevs); - INIT_LIST_HEAD(&ar->debug.fw_stats.bcn); - - init_completion(&ar->debug.fw_stats_complete); } static ssize_t ath11k_write_pktlog_filter(struct file *file, @@ -1382,6 +1369,193 @@ static const struct file_operations fops_dbr_debug = { .llseek = default_llseek, }; +static ssize_t ath11k_write_ps_timekeeper_enable(struct file *file, + const char __user *user_buf, + size_t count, loff_t *ppos) +{ + struct ath11k *ar = file->private_data; + ssize_t ret; + u8 ps_timekeeper_enable; + + if (kstrtou8_from_user(user_buf, count, 0, &ps_timekeeper_enable)) + return -EINVAL; + + mutex_lock(&ar->conf_mutex); + + if (ar->state != ATH11K_STATE_ON) { + ret = -ENETDOWN; + goto exit; + } + + if (!ar->ps_state_enable) { + ret = -EINVAL; + goto exit; + } + + ar->ps_timekeeper_enable = !!ps_timekeeper_enable; + ret = count; +exit: + mutex_unlock(&ar->conf_mutex); + + return ret; +} + +static ssize_t ath11k_read_ps_timekeeper_enable(struct file *file, + char __user *user_buf, + size_t count, loff_t *ppos) +{ + struct ath11k *ar = file->private_data; + char buf[32]; + int len; + + mutex_lock(&ar->conf_mutex); + len = scnprintf(buf, sizeof(buf), "%d\n", ar->ps_timekeeper_enable); + mutex_unlock(&ar->conf_mutex); + + return simple_read_from_buffer(user_buf, count, ppos, buf, len); +} + +static const struct file_operations fops_ps_timekeeper_enable = { + .read = ath11k_read_ps_timekeeper_enable, + .write = ath11k_write_ps_timekeeper_enable, + .open = simple_open, + .owner = THIS_MODULE, + .llseek = default_llseek, +}; + +static void ath11k_reset_peer_ps_duration(void *data, + struct ieee80211_sta *sta) +{ + struct ath11k *ar = data; + struct ath11k_sta *arsta = (struct ath11k_sta *)sta->drv_priv; + + spin_lock_bh(&ar->data_lock); + arsta->ps_total_duration = 0; + spin_unlock_bh(&ar->data_lock); +} + +static ssize_t ath11k_write_reset_ps_duration(struct file *file, + const char __user *user_buf, + size_t count, loff_t *ppos) +{ + struct ath11k *ar = file->private_data; + int ret; + u8 reset_ps_duration; + + if (kstrtou8_from_user(user_buf, count, 0, &reset_ps_duration)) + return -EINVAL; + + mutex_lock(&ar->conf_mutex); + + if (ar->state != ATH11K_STATE_ON) { + ret = -ENETDOWN; + goto exit; + } + + if (!ar->ps_state_enable) { + ret = -EINVAL; + goto exit; + } + + ieee80211_iterate_stations_atomic(ar->hw, + ath11k_reset_peer_ps_duration, + ar); + + ret = count; +exit: + mutex_unlock(&ar->conf_mutex); + return ret; +} + +static const struct file_operations fops_reset_ps_duration = { + .write = ath11k_write_reset_ps_duration, + .open = simple_open, + .owner = THIS_MODULE, + .llseek = default_llseek, +}; + +static void ath11k_peer_ps_state_disable(void *data, + struct ieee80211_sta *sta) +{ + struct ath11k *ar = data; + struct ath11k_sta *arsta = (struct ath11k_sta *)sta->drv_priv; + + spin_lock_bh(&ar->data_lock); + arsta->peer_ps_state = WMI_PEER_PS_STATE_DISABLED; + arsta->ps_start_time = 0; + arsta->ps_total_duration = 0; + spin_unlock_bh(&ar->data_lock); +} + +static ssize_t ath11k_write_ps_state_enable(struct file *file, + const char __user *user_buf, + size_t count, loff_t *ppos) +{ + struct ath11k *ar = file->private_data; + struct ath11k_pdev *pdev = ar->pdev; + int ret; + u32 param; + u8 ps_state_enable; + + if (kstrtou8_from_user(user_buf, count, 0, &ps_state_enable)) + return -EINVAL; + + mutex_lock(&ar->conf_mutex); + + ps_state_enable = !!ps_state_enable; + + if (ar->ps_state_enable == ps_state_enable) { + ret = count; + goto exit; + } + + param = WMI_PDEV_PEER_STA_PS_STATECHG_ENABLE; + ret = ath11k_wmi_pdev_set_param(ar, param, ps_state_enable, pdev->pdev_id); + if (ret) { + ath11k_warn(ar->ab, "failed to enable ps_state_enable: %d\n", + ret); + goto exit; + } + ar->ps_state_enable = ps_state_enable; + + if (!ar->ps_state_enable) { + ar->ps_timekeeper_enable = false; + ieee80211_iterate_stations_atomic(ar->hw, + ath11k_peer_ps_state_disable, + ar); + } + + ret = count; + +exit: + mutex_unlock(&ar->conf_mutex); + + return ret; +} + +static ssize_t ath11k_read_ps_state_enable(struct file *file, + char __user *user_buf, + size_t count, loff_t *ppos) +{ + struct ath11k *ar = file->private_data; + char buf[32]; + int len; + + mutex_lock(&ar->conf_mutex); + len = scnprintf(buf, sizeof(buf), "%d\n", ar->ps_state_enable); + mutex_unlock(&ar->conf_mutex); + + return simple_read_from_buffer(user_buf, count, ppos, buf, len); +} + +static const struct file_operations fops_ps_state_enable = { + .read = ath11k_read_ps_state_enable, + .write = ath11k_write_ps_state_enable, + .open = simple_open, + .owner = THIS_MODULE, + .llseek = default_llseek, +}; + int ath11k_debugfs_register(struct ath11k *ar) { struct ath11k_base *ab = ar->ab; @@ -1428,6 +1602,20 @@ int ath11k_debugfs_register(struct ath11k *ar) debugfs_create_file("enable_dbr_debug", 0200, ar->debug.debugfs_pdev, ar, &fops_dbr_debug); + debugfs_create_file("ps_state_enable", 0600, ar->debug.debugfs_pdev, ar, + &fops_ps_state_enable); + + if (test_bit(WMI_TLV_SERVICE_PEER_POWER_SAVE_DURATION_SUPPORT, + ar->ab->wmi_ab.svc_map)) { + debugfs_create_file("ps_timekeeper_enable", 0600, + ar->debug.debugfs_pdev, ar, + &fops_ps_timekeeper_enable); + + debugfs_create_file("reset_ps_duration", 0200, + ar->debug.debugfs_pdev, ar, + &fops_reset_ps_duration); + } + return 0; } @@ -1456,11 +1644,13 @@ static ssize_t ath11k_write_twt_add_dialog(struct file *file, { struct ath11k_vif *arvif = file->private_data; struct wmi_twt_add_dialog_params params = { 0 }; + struct wmi_twt_enable_params twt_params = {0}; + struct ath11k *ar = arvif->ar; u8 buf[128] = {0}; int ret; - if (arvif->ar->twt_enabled == 0) { - ath11k_err(arvif->ar->ab, "twt support is not enabled\n"); + if (ar->twt_enabled == 0) { + ath11k_err(ar->ab, "twt support is not enabled\n"); return -EOPNOTSUPP; } @@ -1490,13 +1680,38 @@ static ssize_t ath11k_write_twt_add_dialog(struct file *file, if (ret != 16) return -EINVAL; + /* In the case of station vif, TWT is entirely handled by + * the firmware based on the input parameters in the TWT enable + * WMI command that is sent to the target during assoc. + * For manually testing the TWT feature, we need to first disable + * TWT and send enable command again with TWT input parameter + * sta_cong_timer_ms set to 0. + */ + if (arvif->vif->type == NL80211_IFTYPE_STATION) { + ath11k_wmi_send_twt_disable_cmd(ar, ar->pdev->pdev_id); + + ath11k_wmi_fill_default_twt_params(&twt_params); + twt_params.sta_cong_timer_ms = 0; + + ath11k_wmi_send_twt_enable_cmd(ar, ar->pdev->pdev_id, &twt_params); + } + params.vdev_id = arvif->vdev_id; ret = ath11k_wmi_send_twt_add_dialog_cmd(arvif->ar, ¶ms); if (ret) - return ret; + goto err_twt_add_dialog; return count; + +err_twt_add_dialog: + if (arvif->vif->type == NL80211_IFTYPE_STATION) { + ath11k_wmi_send_twt_disable_cmd(ar, ar->pdev->pdev_id); + ath11k_wmi_fill_default_twt_params(&twt_params); + ath11k_wmi_send_twt_enable_cmd(ar, ar->pdev->pdev_id, &twt_params); + } + + return ret; } static ssize_t ath11k_write_twt_del_dialog(struct file *file, @@ -1505,11 +1720,13 @@ static ssize_t ath11k_write_twt_del_dialog(struct file *file, { struct ath11k_vif *arvif = file->private_data; struct wmi_twt_del_dialog_params params = { 0 }; + struct wmi_twt_enable_params twt_params = {0}; + struct ath11k *ar = arvif->ar; u8 buf[64] = {0}; int ret; - if (arvif->ar->twt_enabled == 0) { - ath11k_err(arvif->ar->ab, "twt support is not enabled\n"); + if (ar->twt_enabled == 0) { + ath11k_err(ar->ab, "twt support is not enabled\n"); return -EOPNOTSUPP; } @@ -1535,6 +1752,12 @@ static ssize_t ath11k_write_twt_del_dialog(struct file *file, if (ret) return ret; + if (arvif->vif->type == NL80211_IFTYPE_STATION) { + ath11k_wmi_send_twt_disable_cmd(ar, ar->pdev->pdev_id); + ath11k_wmi_fill_default_twt_params(&twt_params); + ath11k_wmi_send_twt_enable_cmd(ar, ar->pdev->pdev_id, &twt_params); + } + return count; } @@ -1638,36 +1861,35 @@ static const struct file_operations ath11k_fops_twt_resume_dialog = { .open = simple_open }; -int ath11k_debugfs_add_interface(struct ath11k_vif *arvif) +void ath11k_debugfs_add_interface(struct ath11k_vif *arvif) { - if (arvif->vif->type == NL80211_IFTYPE_AP && !arvif->debugfs_twt) { - arvif->debugfs_twt = debugfs_create_dir("twt", - arvif->vif->debugfs_dir); - if (!arvif->debugfs_twt || IS_ERR(arvif->debugfs_twt)) { - ath11k_warn(arvif->ar->ab, - "failed to create directory %p\n", - arvif->debugfs_twt); - arvif->debugfs_twt = NULL; - return -1; - } + struct ath11k_base *ab = arvif->ar->ab; - debugfs_create_file("add_dialog", 0200, arvif->debugfs_twt, - arvif, &ath11k_fops_twt_add_dialog); + if (arvif->vif->type != NL80211_IFTYPE_AP && + !(arvif->vif->type == NL80211_IFTYPE_STATION && + test_bit(WMI_TLV_SERVICE_STA_TWT, ab->wmi_ab.svc_map))) + return; - debugfs_create_file("del_dialog", 0200, arvif->debugfs_twt, - arvif, &ath11k_fops_twt_del_dialog); + arvif->debugfs_twt = debugfs_create_dir("twt", + arvif->vif->debugfs_dir); + debugfs_create_file("add_dialog", 0200, arvif->debugfs_twt, + arvif, &ath11k_fops_twt_add_dialog); - debugfs_create_file("pause_dialog", 0200, arvif->debugfs_twt, - arvif, &ath11k_fops_twt_pause_dialog); + debugfs_create_file("del_dialog", 0200, arvif->debugfs_twt, + arvif, &ath11k_fops_twt_del_dialog); - debugfs_create_file("resume_dialog", 0200, arvif->debugfs_twt, - arvif, &ath11k_fops_twt_resume_dialog); - } - return 0; + debugfs_create_file("pause_dialog", 0200, arvif->debugfs_twt, + arvif, &ath11k_fops_twt_pause_dialog); + + debugfs_create_file("resume_dialog", 0200, arvif->debugfs_twt, + arvif, &ath11k_fops_twt_resume_dialog); } void ath11k_debugfs_remove_interface(struct ath11k_vif *arvif) { + if (!arvif->debugfs_twt) + return; + debugfs_remove_recursive(arvif->debugfs_twt); arvif->debugfs_twt = NULL; } diff --git a/drivers/net/wireless/ath/ath11k/debugfs.h b/drivers/net/wireless/ath/ath11k/debugfs.h index 30c00cb28311..3af0169f6cf2 100644 --- a/drivers/net/wireless/ath/ath11k/debugfs.h +++ b/drivers/net/wireless/ath/ath11k/debugfs.h @@ -269,7 +269,7 @@ int ath11k_debugfs_pdev_create(struct ath11k_base *ab); void ath11k_debugfs_pdev_destroy(struct ath11k_base *ab); int ath11k_debugfs_register(struct ath11k *ar); void ath11k_debugfs_unregister(struct ath11k *ar); -void ath11k_debugfs_fw_stats_process(struct ath11k_base *ab, struct sk_buff *skb); +void ath11k_debugfs_fw_stats_process(struct ath11k *ar, struct ath11k_fw_stats *stats); void ath11k_debugfs_fw_stats_init(struct ath11k *ar); int ath11k_debugfs_get_fw_stats(struct ath11k *ar, u32 pdev_id, @@ -306,7 +306,7 @@ static inline int ath11k_debugfs_rx_filter(struct ath11k *ar) return ar->debug.rx_filter; } -int ath11k_debugfs_add_interface(struct ath11k_vif *arvif); +void ath11k_debugfs_add_interface(struct ath11k_vif *arvif); void ath11k_debugfs_remove_interface(struct ath11k_vif *arvif); void ath11k_debugfs_add_dbring_entry(struct ath11k *ar, enum wmi_direct_buffer_module id, @@ -341,8 +341,8 @@ static inline void ath11k_debugfs_unregister(struct ath11k *ar) { } -static inline void ath11k_debugfs_fw_stats_process(struct ath11k_base *ab, - struct sk_buff *skb) +static inline void ath11k_debugfs_fw_stats_process(struct ath11k *ar, + struct ath11k_fw_stats *stats) { } @@ -386,9 +386,8 @@ static inline int ath11k_debugfs_get_fw_stats(struct ath11k *ar, return 0; } -static inline int ath11k_debugfs_add_interface(struct ath11k_vif *arvif) +static inline void ath11k_debugfs_add_interface(struct ath11k_vif *arvif) { - return 0; } static inline void ath11k_debugfs_remove_interface(struct ath11k_vif *arvif) diff --git a/drivers/net/wireless/ath/ath11k/debugfs_htt_stats.h b/drivers/net/wireless/ath/ath11k/debugfs_htt_stats.h index 5d722b51b125..2b97cbbd28cb 100644 --- a/drivers/net/wireless/ath/ath11k/debugfs_htt_stats.h +++ b/drivers/net/wireless/ath/ath11k/debugfs_htt_stats.h @@ -630,7 +630,7 @@ struct htt_tx_hwq_tried_mpdu_cnt_hist_tlv_v { * completing the burst, we identify the txop used in the burst and * incr the corresponding bin. * Each bin represents 1ms & we have 10 bins in this histogram. - * they are deined in FW using the following macros + * they are defined in FW using the following macros * #define WAL_MAX_TXOP_USED_CNT_HISTOGRAM 10 * #define WAL_TXOP_USED_HISTOGRAM_INTERVAL 1000 ( 1 ms ) */ @@ -1897,7 +1897,7 @@ struct htt_phy_counters_tlv { u32 phytx_abort_cnt; /* number of times rx abort initiated by phy */ u32 phyrx_abort_cnt; - /* number of rx defered count initiated by phy */ + /* number of rx deferred count initiated by phy */ u32 phyrx_defer_abort_cnt; /* number of sizing events generated at LSTF */ u32 rx_gain_adj_lstf_event_cnt; diff --git a/drivers/net/wireless/ath/ath11k/debugfs_sta.c b/drivers/net/wireless/ath/ath11k/debugfs_sta.c index 1b1acbdf837a..9cc4ef28e751 100644 --- a/drivers/net/wireless/ath/ath11k/debugfs_sta.c +++ b/drivers/net/wireless/ath/ath11k/debugfs_sta.c @@ -751,6 +751,102 @@ static const struct file_operations fops_htt_peer_stats_reset = { .llseek = default_llseek, }; +static ssize_t ath11k_dbg_sta_read_peer_ps_state(struct file *file, + char __user *user_buf, + size_t count, loff_t *ppos) +{ + struct ieee80211_sta *sta = file->private_data; + struct ath11k_sta *arsta = (struct ath11k_sta *)sta->drv_priv; + struct ath11k *ar = arsta->arvif->ar; + char buf[20]; + int len; + + spin_lock_bh(&ar->data_lock); + + len = scnprintf(buf, sizeof(buf), "%d\n", arsta->peer_ps_state); + + spin_unlock_bh(&ar->data_lock); + + return simple_read_from_buffer(user_buf, count, ppos, buf, len); +} + +static const struct file_operations fops_peer_ps_state = { + .open = simple_open, + .read = ath11k_dbg_sta_read_peer_ps_state, + .owner = THIS_MODULE, + .llseek = default_llseek, +}; + +static ssize_t ath11k_dbg_sta_read_current_ps_duration(struct file *file, + char __user *user_buf, + size_t count, + loff_t *ppos) +{ + struct ieee80211_sta *sta = file->private_data; + struct ath11k_sta *arsta = (struct ath11k_sta *)sta->drv_priv; + struct ath11k *ar = arsta->arvif->ar; + u64 time_since_station_in_power_save; + char buf[20]; + int len; + + spin_lock_bh(&ar->data_lock); + + if (arsta->peer_ps_state == WMI_PEER_PS_STATE_ON && + arsta->peer_current_ps_valid) + time_since_station_in_power_save = jiffies_to_msecs(jiffies + - arsta->ps_start_jiffies); + else + time_since_station_in_power_save = 0; + + len = scnprintf(buf, sizeof(buf), "%llu\n", + time_since_station_in_power_save); + spin_unlock_bh(&ar->data_lock); + + return simple_read_from_buffer(user_buf, count, ppos, buf, len); +} + +static const struct file_operations fops_current_ps_duration = { + .open = simple_open, + .read = ath11k_dbg_sta_read_current_ps_duration, + .owner = THIS_MODULE, + .llseek = default_llseek, +}; + +static ssize_t ath11k_dbg_sta_read_total_ps_duration(struct file *file, + char __user *user_buf, + size_t count, loff_t *ppos) +{ + struct ieee80211_sta *sta = file->private_data; + struct ath11k_sta *arsta = (struct ath11k_sta *)sta->drv_priv; + struct ath11k *ar = arsta->arvif->ar; + char buf[20]; + u64 power_save_duration; + int len; + + spin_lock_bh(&ar->data_lock); + + if (arsta->peer_ps_state == WMI_PEER_PS_STATE_ON && + arsta->peer_current_ps_valid) + power_save_duration = jiffies_to_msecs(jiffies + - arsta->ps_start_jiffies) + + arsta->ps_total_duration; + else + power_save_duration = arsta->ps_total_duration; + + len = scnprintf(buf, sizeof(buf), "%llu\n", power_save_duration); + + spin_unlock_bh(&ar->data_lock); + + return simple_read_from_buffer(user_buf, count, ppos, buf, len); +} + +static const struct file_operations fops_total_ps_duration = { + .open = simple_open, + .read = ath11k_dbg_sta_read_total_ps_duration, + .owner = THIS_MODULE, + .llseek = default_llseek, +}; + void ath11k_debugfs_sta_op_add(struct ieee80211_hw *hw, struct ieee80211_vif *vif, struct ieee80211_sta *sta, struct dentry *dir) { @@ -778,4 +874,15 @@ void ath11k_debugfs_sta_op_add(struct ieee80211_hw *hw, struct ieee80211_vif *vi ar->ab->wmi_ab.svc_map)) debugfs_create_file("htt_peer_stats_reset", 0600, dir, sta, &fops_htt_peer_stats_reset); + + debugfs_create_file("peer_ps_state", 0400, dir, sta, + &fops_peer_ps_state); + + if (test_bit(WMI_TLV_SERVICE_PEER_POWER_SAVE_DURATION_SUPPORT, + ar->ab->wmi_ab.svc_map)) { + debugfs_create_file("current_ps_duration", 0440, dir, sta, + &fops_current_ps_duration); + debugfs_create_file("total_ps_duration", 0440, dir, sta, + &fops_total_ps_duration); + } } diff --git a/drivers/net/wireless/ath/ath11k/dp.c b/drivers/net/wireless/ath/ath11k/dp.c index 8b790ce72e5d..f5156a7fbdd7 100644 --- a/drivers/net/wireless/ath/ath11k/dp.c +++ b/drivers/net/wireless/ath/ath11k/dp.c @@ -1,6 +1,7 @@ // SPDX-License-Identifier: BSD-3-Clause-Clear /* * Copyright (c) 2018-2019 The Linux Foundation. All rights reserved. + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved. */ #include <crypto/hash.h> @@ -131,13 +132,11 @@ static int ath11k_dp_srng_calculate_msi_group(struct ath11k_base *ab, switch (type) { case HAL_WBM2SW_RELEASE: - if (ring_num < 3) { - grp_mask = &ab->hw_params.ring_mask->tx[0]; - } else if (ring_num == 3) { + if (ring_num == DP_RX_RELEASE_RING_NUM) { grp_mask = &ab->hw_params.ring_mask->rx_wbm_rel[0]; ring_num = 0; } else { - return -ENOENT; + grp_mask = &ab->hw_params.ring_mask->tx[0]; } break; case HAL_REO_EXCEPTION: @@ -371,6 +370,7 @@ static int ath11k_dp_srng_common_setup(struct ath11k_base *ab) struct ath11k_dp *dp = &ab->dp; struct hal_srng *srng; int i, ret; + u8 tcl_num, wbm_num; ret = ath11k_dp_srng_setup(ab, &dp->wbm_desc_rel_ring, HAL_SW2WBM_RELEASE, 0, 0, @@ -396,9 +396,12 @@ static int ath11k_dp_srng_common_setup(struct ath11k_base *ab) } for (i = 0; i < ab->hw_params.max_tx_ring; i++) { + tcl_num = ab->hw_params.hal_params->tcl2wbm_rbm_map[i].tcl_ring_num; + wbm_num = ab->hw_params.hal_params->tcl2wbm_rbm_map[i].wbm_ring_num; + ret = ath11k_dp_srng_setup(ab, &dp->tx_ring[i].tcl_data_ring, - HAL_TCL_DATA, i, 0, - DP_TCL_DATA_RING_SIZE); + HAL_TCL_DATA, tcl_num, 0, + ab->hw_params.tx_ring_size); if (ret) { ath11k_warn(ab, "failed to set up tcl_data ring (%d) :%d\n", i, ret); @@ -406,7 +409,7 @@ static int ath11k_dp_srng_common_setup(struct ath11k_base *ab) } ret = ath11k_dp_srng_setup(ab, &dp->tx_ring[i].tcl_comp_ring, - HAL_WBM2SW_RELEASE, i, 0, + HAL_WBM2SW_RELEASE, wbm_num, 0, DP_TX_COMP_RING_SIZE); if (ret) { ath11k_warn(ab, "failed to set up tcl_comp ring (%d) :%d\n", @@ -431,7 +434,7 @@ static int ath11k_dp_srng_common_setup(struct ath11k_base *ab) } ret = ath11k_dp_srng_setup(ab, &dp->rx_rel_ring, HAL_WBM2SW_RELEASE, - 3, 0, DP_RX_RELEASE_RING_SIZE); + DP_RX_RELEASE_RING_NUM, 0, DP_RX_RELEASE_RING_SIZE); if (ret) { ath11k_warn(ab, "failed to set up rx_rel ring :%d\n", ret); goto err; @@ -774,9 +777,10 @@ int ath11k_dp_service_srng(struct ath11k_base *ab, int i, j; int tot_work_done = 0; - if (ab->hw_params.ring_mask->tx[grp_id]) { - i = __fls(ab->hw_params.ring_mask->tx[grp_id]); - ath11k_dp_tx_completion_handler(ab, i); + for (i = 0; i < ab->hw_params.max_tx_ring; i++) { + if (BIT(ab->hw_params.hal_params->tcl2wbm_rbm_map[i].wbm_ring_num) & + ab->hw_params.ring_mask->tx[grp_id]) + ath11k_dp_tx_completion_handler(ab, i); } if (ab->hw_params.ring_mask->rx_err[grp_id]) { @@ -963,7 +967,7 @@ static void ath11k_dp_update_vdev_search(struct ath11k_vif *arvif) { /* When v2_map_support is true:for STA mode, enable address * search index, tcl uses ast_hash value in the descriptor. - * When v2_map_support is false: for STA mode, dont' enable + * When v2_map_support is false: for STA mode, don't enable * address search index. */ switch (arvif->vdev_type) { diff --git a/drivers/net/wireless/ath/ath11k/dp.h b/drivers/net/wireless/ath/ath11k/dp.h index e9dfa209098b..be9eafc872b3 100644 --- a/drivers/net/wireless/ath/ath11k/dp.h +++ b/drivers/net/wireless/ath/ath11k/dp.h @@ -1,6 +1,7 @@ /* SPDX-License-Identifier: BSD-3-Clause-Clear */ /* * Copyright (c) 2018-2019 The Linux Foundation. All rights reserved. + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved. */ #ifndef ATH11K_DP_H @@ -203,6 +204,7 @@ struct ath11k_pdev_dp { #define DP_WBM_RELEASE_RING_SIZE 64 #define DP_TCL_DATA_RING_SIZE 512 +#define DP_TCL_DATA_RING_SIZE_WCN6750 2048 #define DP_TX_COMP_RING_SIZE 32768 #define DP_TX_IDR_SIZE DP_TX_COMP_RING_SIZE #define DP_TCL_CMD_RING_SIZE 32 @@ -222,6 +224,8 @@ struct ath11k_pdev_dp { #define DP_RXDMA_MONITOR_DST_RING_SIZE 2048 #define DP_RXDMA_MONITOR_DESC_RING_SIZE 4096 +#define DP_RX_RELEASE_RING_NUM 3 + #define DP_RX_BUFFER_SIZE 2048 #define DP_RX_BUFFER_SIZE_LITE 1024 #define DP_RX_BUFFER_ALIGN_SIZE 128 @@ -299,7 +303,7 @@ struct ath11k_dp { #define HTT_TX_WBM_COMP_STATUS_OFFSET 8 -/* HTT tx completion is overlayed in wbm_release_ring */ +/* HTT tx completion is overlaid in wbm_release_ring */ #define HTT_TX_WBM_COMP_INFO0_STATUS GENMASK(12, 9) #define HTT_TX_WBM_COMP_INFO0_REINJECT_REASON GENMASK(16, 13) #define HTT_TX_WBM_COMP_INFO0_REINJECT_REASON GENMASK(16, 13) @@ -466,7 +470,7 @@ enum htt_srng_ring_id { * 3'b010: 4 usec * 3'b011: 8 usec (default) * 3'b100: 16 usec - * Others: Reserverd + * Others: Reserved * b'19 - response_required: * Host needs HTT_T2H_MSG_TYPE_SRING_SETUP_DONE as response * b'20:31 - reserved: reserved for future use @@ -993,8 +997,7 @@ struct htt_rx_ring_tlv_filter { #define HTT_RX_FULL_MON_MODE_CFG_CMD_CFG_NON_ZERO_MPDUS_END BIT(2) #define HTT_RX_FULL_MON_MODE_CFG_CMD_CFG_RELEASE_RING GENMASK(10, 3) -/** - * Enumeration for full monitor mode destination ring select +/* Enumeration for full monitor mode destination ring select * 0 - REO destination ring select * 1 - FW destination ring select * 2 - SW destination ring select @@ -1391,8 +1394,7 @@ struct htt_ppdu_stats_info { struct list_head list; }; -/** - * @brief target -> host packet log message +/* @brief target -> host packet log message * * @details * The following field definitions describe the format of the packet log @@ -1430,8 +1432,7 @@ struct htt_pktlog_msg { u8 payload[]; }; -/** - * @brief host -> target FW extended statistics retrieve +/* @brief host -> target FW extended statistics retrieve * * @details * The following field definitions describe the format of the HTT host @@ -1566,8 +1567,7 @@ struct htt_ext_stats_cfg_params { u32 cfg3; }; -/** - * @brief target -> host extended statistics upload +/* @brief target -> host extended statistics upload * * @details * The following field definitions describe the format of the HTT target diff --git a/drivers/net/wireless/ath/ath11k/dp_rx.c b/drivers/net/wireless/ath/ath11k/dp_rx.c index 2148acf37071..c5a4c34d7749 100644 --- a/drivers/net/wireless/ath/ath11k/dp_rx.c +++ b/drivers/net/wireless/ath/ath11k/dp_rx.c @@ -2499,7 +2499,7 @@ static void ath11k_dp_rx_deliver_msdu(struct ath11k *ar, struct napi_struct *nap /* PN for multicast packets are not validate in HW, * so skip 802.3 rx path - * Also, fast_rx expectes the STA to be authorized, hence + * Also, fast_rx expects the STA to be authorized, hence * eapol packets are sent in slow path. */ if (decap == DP_RX_DECAP_TYPE_ETHERNET2_DIX && !is_eapol && @@ -5197,7 +5197,8 @@ int ath11k_dp_rx_process_mon_status(struct ath11k_base *ab, int mac_id, if (log_type != ATH11K_PKTLOG_TYPE_INVALID) trace_ath11k_htt_rxdesc(ar, skb->data, log_type, rx_buf_sz); - memset(ppdu_info, 0, sizeof(struct hal_rx_mon_ppdu_info)); + memset(ppdu_info, 0, sizeof(*ppdu_info)); + ppdu_info->peer_id = HAL_INVALID_PEERID; hal_status = ath11k_hal_rx_parse_mon_status(ab, ppdu_info, skb); if (test_bit(ATH11K_FLAG_MONITOR_STARTED, &ar->monitor_flags) && diff --git a/drivers/net/wireless/ath/ath11k/dp_tx.c b/drivers/net/wireless/ath/ath11k/dp_tx.c index c17a2620aad7..8afbba236935 100644 --- a/drivers/net/wireless/ath/ath11k/dp_tx.c +++ b/drivers/net/wireless/ath/ath11k/dp_tx.c @@ -1,6 +1,7 @@ // SPDX-License-Identifier: BSD-3-Clause-Clear /* * Copyright (c) 2018-2019 The Linux Foundation. All rights reserved. + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved. */ #include "core.h" @@ -93,7 +94,8 @@ int ath11k_dp_tx(struct ath11k *ar, struct ath11k_vif *arvif, u8 pool_id; u8 hal_ring_id; int ret; - u8 ring_selector = 0, ring_map = 0; + u32 ring_selector = 0; + u8 ring_map = 0; bool tcl_ring_retry; if (unlikely(test_bit(ATH11K_FLAG_CRASH_FLUSH, &ar->ab->dev_flags))) @@ -105,19 +107,13 @@ int ath11k_dp_tx(struct ath11k *ar, struct ath11k_vif *arvif, pool_id = skb_get_queue_mapping(skb) & (ATH11K_HW_MAX_QUEUES - 1); - /* Let the default ring selection be based on current processor - * number, where one of the 3 tcl rings are selected based on - * the smp_processor_id(). In case that ring - * is full/busy, we resort to other available rings. - * If all rings are full, we drop the packet. - * //TODO Add throttling logic when all rings are full - */ - ring_selector = smp_processor_id(); + ring_selector = ab->hw_params.hw_ops->get_ring_selector(skb); tcl_ring_sel: tcl_ring_retry = false; ti.ring_id = ring_selector % ab->hw_params.max_tx_ring; + ti.rbm_id = ab->hw_params.hal_params->tcl2wbm_rbm_map[ti.ring_id].rbm_id; ring_map |= BIT(ti.ring_id); @@ -129,7 +125,8 @@ tcl_ring_sel: spin_unlock_bh(&tx_ring->tx_idr_lock); if (unlikely(ret < 0)) { - if (ring_map == (BIT(ab->hw_params.max_tx_ring) - 1)) { + if (ring_map == (BIT(ab->hw_params.max_tx_ring) - 1) || + !ab->hw_params.tcl_ring_retry) { atomic_inc(&ab->soc_stats.tx_err.misc_fail); return -ENOSPC; } @@ -247,7 +244,7 @@ tcl_ring_sel: * Restart ring selection if some rings are not checked yet. */ if (unlikely(ring_map != (BIT(ab->hw_params.max_tx_ring)) - 1) && - ab->hw_params.max_tx_ring > 1) { + ab->hw_params.tcl_ring_retry && ab->hw_params.max_tx_ring > 1) { tcl_ring_retry = true; ring_selector++; } @@ -755,7 +752,7 @@ int ath11k_dp_tx_send_reo_cmd(struct ath11k_base *ab, struct dp_rx_tid *rx_tid, return 0; /* Can this be optimized so that we keep the pending command list only - * for tid delete command to free up the resoruce on the command status + * for tid delete command to free up the resource on the command status * indication? */ dp_cmd = kzalloc(sizeof(*dp_cmd), GFP_ATOMIC); diff --git a/drivers/net/wireless/ath/ath11k/hal.c b/drivers/net/wireless/ath/ath11k/hal.c index bda71ab5a1f2..2fd224480d45 100644 --- a/drivers/net/wireless/ath/ath11k/hal.c +++ b/drivers/net/wireless/ath/ath11k/hal.c @@ -126,7 +126,7 @@ static const struct hal_srng_config hw_srng_config_template[] = { }, { /* WBM2SW_RELEASE */ .start_ring_id = HAL_SRNG_RING_ID_WBM2SW0_RELEASE, - .max_rings = 4, + .max_rings = 5, .entry_size = sizeof(struct hal_wbm_release_ring) >> 2, .lmac_ring = false, .ring_dir = HAL_SRNG_DIR_DST, @@ -1164,7 +1164,7 @@ void ath11k_hal_srng_shadow_update_hp_tp(struct ath11k_base *ab, { lockdep_assert_held(&srng->lock); - /* check whether the ring is emptry. Update the shadow + /* check whether the ring is empty. Update the shadow * HP only when then ring isn't empty. */ if (srng->ring_dir == HAL_SRNG_DIR_SRC && diff --git a/drivers/net/wireless/ath/ath11k/hal.h b/drivers/net/wireless/ath/ath11k/hal.h index 110c337ddf33..6a1f78ee6eb6 100644 --- a/drivers/net/wireless/ath/ath11k/hal.h +++ b/drivers/net/wireless/ath/ath11k/hal.h @@ -243,7 +243,7 @@ struct ath11k_base; #define HAL_WBM0_RELEASE_RING_HP 0x000030c0 #define HAL_WBM1_RELEASE_RING_HP 0x000030c8 -/* TCL ring feild mask and offset */ +/* TCL ring field mask and offset */ #define HAL_TCL1_RING_BASE_MSB_RING_SIZE GENMASK(27, 8) #define HAL_TCL1_RING_BASE_MSB_RING_BASE_ADDR_MSB GENMASK(7, 0) #define HAL_TCL1_RING_ID_ENTRY_SIZE GENMASK(7, 0) @@ -268,7 +268,7 @@ struct ath11k_base; #define HAL_TCL1_RING_FIELD_DSCP_TID_MAP6 GENMASK(20, 18) #define HAL_TCL1_RING_FIELD_DSCP_TID_MAP7 GENMASK(23, 21) -/* REO ring feild mask and offset */ +/* REO ring field mask and offset */ #define HAL_REO1_RING_BASE_MSB_RING_SIZE GENMASK(27, 8) #define HAL_REO1_RING_BASE_MSB_RING_BASE_ADDR_MSB GENMASK(7, 0) #define HAL_REO1_RING_ID_RING_ID GENMASK(15, 8) @@ -389,6 +389,7 @@ enum hal_srng_ring_id { HAL_SRNG_RING_ID_WBM2SW1_RELEASE, HAL_SRNG_RING_ID_WBM2SW2_RELEASE, HAL_SRNG_RING_ID_WBM2SW3_RELEASE, + HAL_SRNG_RING_ID_WBM2SW4_RELEASE, HAL_SRNG_RING_ID_UMAC_ID_END = 127, HAL_SRNG_RING_ID_LMAC1_ID_START, @@ -450,13 +451,13 @@ enum hal_ring_type { /** * enum hal_reo_cmd_type: Enum for REO command type - * @CMD_GET_QUEUE_STATS: Get REO queue status/stats - * @CMD_FLUSH_QUEUE: Flush all frames in REO queue - * @CMD_FLUSH_CACHE: Flush descriptor entries in the cache - * @CMD_UNBLOCK_CACHE: Unblock a descriptor's address that was blocked + * @HAL_REO_CMD_GET_QUEUE_STATS: Get REO queue status/stats + * @HAL_REO_CMD_FLUSH_QUEUE: Flush all frames in REO queue + * @HAL_REO_CMD_FLUSH_CACHE: Flush descriptor entries in the cache + * @HAL_REO_CMD_UNBLOCK_CACHE: Unblock a descriptor's address that was blocked * earlier with a 'REO_FLUSH_CACHE' command - * @CMD_FLUSH_TIMEOUT_LIST: Flush buffers/descriptors from timeout list - * @CMD_UPDATE_RX_REO_QUEUE: Update REO queue settings + * @HAL_REO_CMD_FLUSH_TIMEOUT_LIST: Flush buffers/descriptors from timeout list + * @HAL_REO_CMD_UPDATE_RX_QUEUE: Update REO queue settings */ enum hal_reo_cmd_type { HAL_REO_CMD_GET_QUEUE_STATS = 0, @@ -635,7 +636,7 @@ struct hal_srng { } u; }; -/* Interrupt mitigation - Batch threshold in terms of numer of frames */ +/* Interrupt mitigation - Batch threshold in terms of number of frames */ #define HAL_SRNG_INT_BATCH_THRESHOLD_TX 256 #define HAL_SRNG_INT_BATCH_THRESHOLD_RX 128 #define HAL_SRNG_INT_BATCH_THRESHOLD_OTHER 1 @@ -678,6 +679,7 @@ enum hal_rx_buf_return_buf_manager { HAL_RX_BUF_RBM_SW1_BM, HAL_RX_BUF_RBM_SW2_BM, HAL_RX_BUF_RBM_SW3_BM, + HAL_RX_BUF_RBM_SW4_BM, }; #define HAL_SRNG_DESC_LOOP_CNT 0xf0000000 @@ -873,8 +875,7 @@ struct hal_reo_status { } u; }; -/** - * HAL context to be used to access SRNG APIs (currently used by data path +/* HAL context to be used to access SRNG APIs (currently used by data path * and transport (CE) modules) */ struct ath11k_hal { diff --git a/drivers/net/wireless/ath/ath11k/hal_desc.h b/drivers/net/wireless/ath/ath11k/hal_desc.h index 24e72e75a8c7..d895ea878d9f 100644 --- a/drivers/net/wireless/ath/ath11k/hal_desc.h +++ b/drivers/net/wireless/ath/ath11k/hal_desc.h @@ -607,7 +607,7 @@ struct rx_msdu_desc { * * msdu_continuation * When set, this MSDU buffer was not able to hold the entire MSDU. - * The next buffer will therefor contain additional information + * The next buffer will therefore contain additional information * related to this MSDU. * * msdu_length @@ -643,7 +643,7 @@ struct rx_msdu_desc { * * da_idx_timeout * Indicates, an unsuccessful MAC destination address search due - * to the expiration of search timer fot this MSDU. + * to the expiration of search timer for this MSDU. */ enum hal_reo_dest_ring_buffer_type { @@ -1678,7 +1678,7 @@ struct hal_wbm_release_ring { * Producer: SW/TQM/RXDMA/REO/SWITCH * Consumer: WBM/SW/FW * - * HTT tx status is overlayed on wbm_release ring on 4-byte words 2, 3, 4 and 5 + * HTT tx status is overlaid on wbm_release ring on 4-byte words 2, 3, 4 and 5 * for software based completions. * * buf_addr_info @@ -2159,7 +2159,7 @@ struct hal_reo_status_hdr { * commands. * * execution_time (in us) - * The amount of time REO took to excecute the command. Note that + * The amount of time REO took to execute the command. Note that * this time does not include the duration of the command waiting * in the command ring, before the execution started. * diff --git a/drivers/net/wireless/ath/ath11k/hal_tx.c b/drivers/net/wireless/ath/ath11k/hal_tx.c index c8929de8ce6c..d1b0e36e04a9 100644 --- a/drivers/net/wireless/ath/ath11k/hal_tx.c +++ b/drivers/net/wireless/ath/ath11k/hal_tx.c @@ -1,6 +1,7 @@ // SPDX-License-Identifier: BSD-3-Clause-Clear /* * Copyright (c) 2018-2019 The Linux Foundation. All rights reserved. + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved. */ #include "hal_desc.h" @@ -44,8 +45,7 @@ void ath11k_hal_tx_cmd_desc_setup(struct ath11k_base *ab, void *cmd, FIELD_PREP(BUFFER_ADDR_INFO1_ADDR, ((uint64_t)ti->paddr >> HAL_ADDR_MSB_REG_SHIFT)); tcl_cmd->buf_addr_info.info1 |= - FIELD_PREP(BUFFER_ADDR_INFO1_RET_BUF_MGR, - (ti->ring_id + HAL_RX_BUF_RBM_SW0_BM)) | + FIELD_PREP(BUFFER_ADDR_INFO1_RET_BUF_MGR, ti->rbm_id) | FIELD_PREP(BUFFER_ADDR_INFO1_SW_COOKIE, ti->desc_id); tcl_cmd->info0 = diff --git a/drivers/net/wireless/ath/ath11k/hal_tx.h b/drivers/net/wireless/ath/ath11k/hal_tx.h index 36f4f6f6cbc2..c5e88364afe5 100644 --- a/drivers/net/wireless/ath/ath11k/hal_tx.h +++ b/drivers/net/wireless/ath/ath11k/hal_tx.h @@ -1,6 +1,7 @@ /* SPDX-License-Identifier: BSD-3-Clause-Clear */ /* * Copyright (c) 2018-2019 The Linux Foundation. All rights reserved. + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved. */ #ifndef ATH11K_HAL_TX_H @@ -35,6 +36,7 @@ struct hal_tx_info { u8 lmac_id; u8 dscp_tid_tbl_idx; bool enable_mesh; + u8 rbm_id; }; /* TODO: Check if the actual desc macros can be used instead */ diff --git a/drivers/net/wireless/ath/ath11k/hif.h b/drivers/net/wireless/ath/ath11k/hif.h index e9366f786fbb..659b80d2abd4 100644 --- a/drivers/net/wireless/ath/ath11k/hif.h +++ b/drivers/net/wireless/ath/ath11k/hif.h @@ -11,6 +11,7 @@ struct ath11k_hif_ops { u32 (*read32)(struct ath11k_base *sc, u32 address); void (*write32)(struct ath11k_base *sc, u32 address, u32 data); + int (*read)(struct ath11k_base *ab, void *buf, u32 start, u32 end); void (*irq_enable)(struct ath11k_base *sc); void (*irq_disable)(struct ath11k_base *sc); int (*start)(struct ath11k_base *sc); @@ -99,6 +100,15 @@ static inline void ath11k_hif_write32(struct ath11k_base *sc, u32 address, u32 d sc->hif.ops->write32(sc, address, data); } +static inline int ath11k_hif_read(struct ath11k_base *ab, void *buf, + u32 start, u32 end) +{ + if (!ab->hif.ops->read) + return -EOPNOTSUPP; + + return ab->hif.ops->read(ab, buf, start, end); +} + static inline int ath11k_hif_map_service_to_pipe(struct ath11k_base *sc, u16 service_id, u8 *ul_pipe, u8 *dl_pipe) { @@ -134,4 +144,5 @@ static inline void ath11k_get_ce_msi_idx(struct ath11k_base *ab, u32 ce_id, else *msi_data_idx = ce_id; } + #endif /* _HIF_H_ */ diff --git a/drivers/net/wireless/ath/ath11k/hw.c b/drivers/net/wireless/ath/ath11k/hw.c index 96db85c55585..dbcc0c4035b6 100644 --- a/drivers/net/wireless/ath/ath11k/hw.c +++ b/drivers/net/wireless/ath/ath11k/hw.c @@ -820,6 +820,30 @@ static bool ath11k_hw_wcn6855_rx_desc_get_ldpc_support(struct hal_rx_desc *desc) __le32_to_cpu(desc->u.wcn6855.msdu_start.info2)); } +static u32 ath11k_hw_ipq8074_get_tcl_ring_selector(struct sk_buff *skb) +{ + /* Let the default ring selection be based on current processor + * number, where one of the 3 tcl rings are selected based on + * the smp_processor_id(). In case that ring + * is full/busy, we resort to other available rings. + * If all rings are full, we drop the packet. + * + * TODO: Add throttling logic when all rings are full + */ + return smp_processor_id(); +} + +static u32 ath11k_hw_wcn6750_get_tcl_ring_selector(struct sk_buff *skb) +{ + /* Select the TCL ring based on the flow hash of the SKB instead + * of CPU ID. Since applications pumping the traffic can be scheduled + * on multiple CPUs, there is a chance that packets of the same flow + * could end on different TCL rings, this could sometimes results in + * an out of order arrival of the packets at the receiver. + */ + return skb_get_hash(skb); +} + const struct ath11k_hw_ops ipq8074_ops = { .get_hw_mac_from_pdev_id = ath11k_hw_ipq8074_mac_from_pdev_id, .wmi_init_config = ath11k_init_wmi_config_ipq8074, @@ -857,6 +881,7 @@ const struct ath11k_hw_ops ipq8074_ops = { .mpdu_info_get_peerid = ath11k_hw_ipq8074_mpdu_info_get_peerid, .rx_desc_mac_addr2_valid = ath11k_hw_ipq8074_rx_desc_mac_addr2_valid, .rx_desc_mpdu_start_addr2 = ath11k_hw_ipq8074_rx_desc_mpdu_start_addr2, + .get_ring_selector = ath11k_hw_ipq8074_get_tcl_ring_selector, }; const struct ath11k_hw_ops ipq6018_ops = { @@ -896,6 +921,7 @@ const struct ath11k_hw_ops ipq6018_ops = { .mpdu_info_get_peerid = ath11k_hw_ipq8074_mpdu_info_get_peerid, .rx_desc_mac_addr2_valid = ath11k_hw_ipq8074_rx_desc_mac_addr2_valid, .rx_desc_mpdu_start_addr2 = ath11k_hw_ipq8074_rx_desc_mpdu_start_addr2, + .get_ring_selector = ath11k_hw_ipq8074_get_tcl_ring_selector, }; const struct ath11k_hw_ops qca6390_ops = { @@ -935,6 +961,7 @@ const struct ath11k_hw_ops qca6390_ops = { .mpdu_info_get_peerid = ath11k_hw_ipq8074_mpdu_info_get_peerid, .rx_desc_mac_addr2_valid = ath11k_hw_ipq8074_rx_desc_mac_addr2_valid, .rx_desc_mpdu_start_addr2 = ath11k_hw_ipq8074_rx_desc_mpdu_start_addr2, + .get_ring_selector = ath11k_hw_ipq8074_get_tcl_ring_selector, }; const struct ath11k_hw_ops qcn9074_ops = { @@ -974,6 +1001,7 @@ const struct ath11k_hw_ops qcn9074_ops = { .mpdu_info_get_peerid = ath11k_hw_ipq8074_mpdu_info_get_peerid, .rx_desc_mac_addr2_valid = ath11k_hw_ipq9074_rx_desc_mac_addr2_valid, .rx_desc_mpdu_start_addr2 = ath11k_hw_ipq9074_rx_desc_mpdu_start_addr2, + .get_ring_selector = ath11k_hw_ipq8074_get_tcl_ring_selector, }; const struct ath11k_hw_ops wcn6855_ops = { @@ -1013,6 +1041,7 @@ const struct ath11k_hw_ops wcn6855_ops = { .mpdu_info_get_peerid = ath11k_hw_wcn6855_mpdu_info_get_peerid, .rx_desc_mac_addr2_valid = ath11k_hw_wcn6855_rx_desc_mac_addr2_valid, .rx_desc_mpdu_start_addr2 = ath11k_hw_wcn6855_rx_desc_mpdu_start_addr2, + .get_ring_selector = ath11k_hw_ipq8074_get_tcl_ring_selector, }; const struct ath11k_hw_ops wcn6750_ops = { @@ -1052,11 +1081,14 @@ const struct ath11k_hw_ops wcn6750_ops = { .mpdu_info_get_peerid = ath11k_hw_ipq8074_mpdu_info_get_peerid, .rx_desc_mac_addr2_valid = ath11k_hw_ipq9074_rx_desc_mac_addr2_valid, .rx_desc_mpdu_start_addr2 = ath11k_hw_ipq9074_rx_desc_mpdu_start_addr2, + .get_ring_selector = ath11k_hw_wcn6750_get_tcl_ring_selector, }; -#define ATH11K_TX_RING_MASK_0 0x1 -#define ATH11K_TX_RING_MASK_1 0x2 -#define ATH11K_TX_RING_MASK_2 0x4 +#define ATH11K_TX_RING_MASK_0 BIT(0) +#define ATH11K_TX_RING_MASK_1 BIT(1) +#define ATH11K_TX_RING_MASK_2 BIT(2) +#define ATH11K_TX_RING_MASK_3 BIT(3) +#define ATH11K_TX_RING_MASK_4 BIT(4) #define ATH11K_RX_RING_MASK_0 0x1 #define ATH11K_RX_RING_MASK_1 0x2 @@ -1903,6 +1935,43 @@ const struct ath11k_hw_ring_mask ath11k_hw_ring_mask_qcn9074 = { }, }; +const struct ath11k_hw_ring_mask ath11k_hw_ring_mask_wcn6750 = { + .tx = { + ATH11K_TX_RING_MASK_0, + 0, + ATH11K_TX_RING_MASK_2, + 0, + ATH11K_TX_RING_MASK_4, + }, + .rx_mon_status = { + 0, 0, 0, 0, 0, 0, + ATH11K_RX_MON_STATUS_RING_MASK_0, + }, + .rx = { + 0, 0, 0, 0, 0, 0, 0, + ATH11K_RX_RING_MASK_0, + ATH11K_RX_RING_MASK_1, + ATH11K_RX_RING_MASK_2, + ATH11K_RX_RING_MASK_3, + }, + .rx_err = { + 0, ATH11K_RX_ERR_RING_MASK_0, + }, + .rx_wbm_rel = { + 0, ATH11K_RX_WBM_REL_RING_MASK_0, + }, + .reo_status = { + 0, ATH11K_REO_STATUS_RING_MASK_0, + }, + .rxdma2host = { + ATH11K_RXDMA2HOST_RING_MASK_0, + ATH11K_RXDMA2HOST_RING_MASK_1, + ATH11K_RXDMA2HOST_RING_MASK_2, + }, + .host2rxdma = { + }, +}; + const struct ath11k_hw_regs ipq8074_regs = { /* SW2TCL(x) R0 ring configuration address */ .hal_tcl1_ring_base_lsb = 0x00000510, @@ -2332,12 +2401,55 @@ const struct ath11k_hw_regs wcn6750_regs = { .hal_reo1_misc_ctl = 0x000005d8, }; +static const struct ath11k_hw_tcl2wbm_rbm_map ath11k_hw_tcl2wbm_rbm_map_ipq8074[] = { + { + .tcl_ring_num = 0, + .wbm_ring_num = 0, + .rbm_id = HAL_RX_BUF_RBM_SW0_BM, + }, + { + .tcl_ring_num = 1, + .wbm_ring_num = 1, + .rbm_id = HAL_RX_BUF_RBM_SW1_BM, + }, + { + .tcl_ring_num = 2, + .wbm_ring_num = 2, + .rbm_id = HAL_RX_BUF_RBM_SW2_BM, + }, +}; + +static const struct ath11k_hw_tcl2wbm_rbm_map ath11k_hw_tcl2wbm_rbm_map_wcn6750[] = { + { + .tcl_ring_num = 0, + .wbm_ring_num = 0, + .rbm_id = HAL_RX_BUF_RBM_SW0_BM, + }, + { + .tcl_ring_num = 1, + .wbm_ring_num = 4, + .rbm_id = HAL_RX_BUF_RBM_SW4_BM, + }, + { + .tcl_ring_num = 2, + .wbm_ring_num = 2, + .rbm_id = HAL_RX_BUF_RBM_SW2_BM, + }, +}; + const struct ath11k_hw_hal_params ath11k_hw_hal_params_ipq8074 = { .rx_buf_rbm = HAL_RX_BUF_RBM_SW3_BM, + .tcl2wbm_rbm_map = ath11k_hw_tcl2wbm_rbm_map_ipq8074, }; const struct ath11k_hw_hal_params ath11k_hw_hal_params_qca6390 = { .rx_buf_rbm = HAL_RX_BUF_RBM_SW1_BM, + .tcl2wbm_rbm_map = ath11k_hw_tcl2wbm_rbm_map_ipq8074, +}; + +const struct ath11k_hw_hal_params ath11k_hw_hal_params_wcn6750 = { + .rx_buf_rbm = HAL_RX_BUF_RBM_SW1_BM, + .tcl2wbm_rbm_map = ath11k_hw_tcl2wbm_rbm_map_wcn6750, }; static const struct cfg80211_sar_freq_ranges ath11k_hw_sar_freq_ranges_wcn6855[] = { diff --git a/drivers/net/wireless/ath/ath11k/hw.h b/drivers/net/wireless/ath/ath11k/hw.h index bb5ac940e470..8a3f24862edc 100644 --- a/drivers/net/wireless/ath/ath11k/hw.h +++ b/drivers/net/wireless/ath/ath11k/hw.h @@ -122,8 +122,15 @@ struct ath11k_hw_ring_mask { u8 host2rxdma[ATH11K_EXT_IRQ_GRP_NUM_MAX]; }; +struct ath11k_hw_tcl2wbm_rbm_map { + u8 tcl_ring_num; + u8 wbm_ring_num; + u8 rbm_id; +}; + struct ath11k_hw_hal_params { enum hal_rx_buf_return_buf_manager rx_buf_rbm; + const struct ath11k_hw_tcl2wbm_rbm_map *tcl2wbm_rbm_map; }; struct ath11k_hw_params { @@ -166,6 +173,7 @@ struct ath11k_hw_params { u8 summary_pad_sz; u8 fft_hdr_len; u16 max_fft_bins; + bool fragment_160mhz; } spectral; u16 interface_modes; @@ -175,6 +183,7 @@ struct ath11k_hw_params { bool idle_ps; bool supports_sta_ps; bool cold_boot_calib; + bool cbcal_restart_fw; int fw_mem_mode; u32 num_vdevs; u32 num_peers; @@ -200,6 +209,16 @@ struct ath11k_hw_params { bool hybrid_bus_type; bool fixed_fw_mem; bool support_off_channel_tx; + bool supports_multi_bssid; + + struct { + u32 start; + u32 end; + } sram_dump; + + bool tcl_ring_retry; + u32 tx_ring_size; + bool smp2p_wow_exit; }; struct ath11k_hw_ops { @@ -242,6 +261,7 @@ struct ath11k_hw_ops { u16 (*mpdu_info_get_peerid)(u8 *tlv_data); bool (*rx_desc_mac_addr2_valid)(struct hal_rx_desc *desc); u8* (*rx_desc_mpdu_start_addr2)(struct hal_rx_desc *desc); + u32 (*get_ring_selector)(struct sk_buff *skb); }; extern const struct ath11k_hw_ops ipq8074_ops; @@ -254,9 +274,11 @@ extern const struct ath11k_hw_ops wcn6750_ops; extern const struct ath11k_hw_ring_mask ath11k_hw_ring_mask_ipq8074; extern const struct ath11k_hw_ring_mask ath11k_hw_ring_mask_qca6390; extern const struct ath11k_hw_ring_mask ath11k_hw_ring_mask_qcn9074; +extern const struct ath11k_hw_ring_mask ath11k_hw_ring_mask_wcn6750; extern const struct ath11k_hw_hal_params ath11k_hw_hal_params_ipq8074; extern const struct ath11k_hw_hal_params ath11k_hw_hal_params_qca6390; +extern const struct ath11k_hw_hal_params ath11k_hw_hal_params_wcn6750; static inline int ath11k_hw_get_mac_from_pdev_id(struct ath11k_hw_params *hw, @@ -397,4 +419,5 @@ static inline const char *ath11k_bd_ie_type_str(enum ath11k_bd_ie_type type) } extern const struct cfg80211_sar_capa ath11k_hw_sar_capa_wcn6855; + #endif diff --git a/drivers/net/wireless/ath/ath11k/mac.c b/drivers/net/wireless/ath/ath11k/mac.c index 7e91e347c9ff..84d956ad4093 100644 --- a/drivers/net/wireless/ath/ath11k/mac.c +++ b/drivers/net/wireless/ath/ath11k/mac.c @@ -3059,7 +3059,7 @@ static int ath11k_mac_config_obss_pd(struct ath11k *ar, return ret; } - /* Enable all patial BSSID mask for SRG */ + /* Enable all partial BSSID mask for SRG */ ret = ath11k_wmi_pdev_srg_obss_bssid_enable_bitmap(ar, bitmap); if (ret) { ath11k_warn(ar->ab, @@ -3077,7 +3077,7 @@ static int ath11k_mac_config_obss_pd(struct ath11k *ar, return ret; } - /* Enable all patial BSSID mask for non-SRG */ + /* Enable all partial BSSID mask for non-SRG */ ret = ath11k_wmi_pdev_non_srg_obss_bssid_enable_bitmap(ar, bitmap); if (ret) { ath11k_warn(ar->ab, @@ -3350,10 +3350,15 @@ static void ath11k_mac_op_bss_info_changed(struct ieee80211_hw *hw, ath11k_recalculate_mgmt_rate(ar, vif, &def); if (changed & BSS_CHANGED_TWT) { - if (info->twt_requester || info->twt_responder) - ath11k_wmi_send_twt_enable_cmd(ar, ar->pdev->pdev_id); - else + struct wmi_twt_enable_params twt_params = {0}; + + if (info->twt_requester || info->twt_responder) { + ath11k_wmi_fill_default_twt_params(&twt_params); + ath11k_wmi_send_twt_enable_cmd(ar, ar->pdev->pdev_id, + &twt_params); + } else { ath11k_wmi_send_twt_disable_cmd(ar, ar->pdev->pdev_id); + } } if (changed & BSS_CHANGED_HE_OBSS_PD) @@ -3451,7 +3456,7 @@ void __ath11k_mac_scan_finish(struct ath11k *ar) ar->scan_channel = NULL; ar->scan.roc_freq = 0; cancel_delayed_work(&ar->scan.timeout); - complete(&ar->scan.completed); + complete_all(&ar->scan.completed); break; } } @@ -4524,6 +4529,7 @@ static int ath11k_mac_op_sta_state(struct ieee80211_hw *hw, new_state == IEEE80211_STA_NONE) { memset(arsta, 0, sizeof(*arsta)); arsta->arvif = arvif; + arsta->peer_ps_state = WMI_PEER_PS_STATE_DISABLED; INIT_WORK(&arsta->update_wk, ath11k_sta_rc_update_wk); INIT_WORK(&arsta->set_4addr_wk, ath11k_sta_set_4addr_wk); @@ -4701,7 +4707,7 @@ static void ath11k_mac_op_sta_rc_update(struct ieee80211_hw *hw, "mac sta rc update for %pM changed %08x bw %d nss %d smps %d\n", sta->addr, changed, sta->deflink.bandwidth, sta->deflink.rx_nss, - sta->smps_mode); + sta->deflink.smps_mode); spin_lock_bh(&ar->data_lock); @@ -4737,7 +4743,7 @@ static void ath11k_mac_op_sta_rc_update(struct ieee80211_hw *hw, if (changed & IEEE80211_RC_SMPS_CHANGED) { smps = WMI_PEER_SMPS_PS_NONE; - switch (sta->smps_mode) { + switch (sta->deflink.smps_mode) { case IEEE80211_SMPS_AUTOMATIC: case IEEE80211_SMPS_OFF: smps = WMI_PEER_SMPS_PS_NONE; @@ -4750,7 +4756,7 @@ static void ath11k_mac_op_sta_rc_update(struct ieee80211_hw *hw, break; default: ath11k_warn(ar->ab, "Invalid smps %d in sta rc update for %pM\n", - sta->smps_mode, sta->addr); + sta->deflink.smps_mode, sta->addr); smps = WMI_PEER_SMPS_PS_NONE; break; } @@ -4954,6 +4960,8 @@ static int ath11k_mac_set_txbf_conf(struct ath11k_vif *arvif) if (vht_cap & (IEEE80211_VHT_CAP_SU_BEAMFORMEE_CAPABLE)) { nsts = vht_cap & IEEE80211_VHT_CAP_BEAMFORMEE_STS_MASK; nsts >>= IEEE80211_VHT_CAP_BEAMFORMEE_STS_SHIFT; + if (nsts > (ar->num_rx_chains - 1)) + nsts = ar->num_rx_chains - 1; value |= SM(nsts, WMI_TXBF_STS_CAP_OFFSET); } @@ -4994,7 +5002,7 @@ static int ath11k_mac_set_txbf_conf(struct ath11k_vif *arvif) static void ath11k_set_vht_txbf_cap(struct ath11k *ar, u32 *vht_cap) { bool subfer, subfee; - int sound_dim = 0; + int sound_dim = 0, nsts = 0; subfer = !!(*vht_cap & (IEEE80211_VHT_CAP_SU_BEAMFORMER_CAPABLE)); subfee = !!(*vht_cap & (IEEE80211_VHT_CAP_SU_BEAMFORMEE_CAPABLE)); @@ -5004,6 +5012,11 @@ static void ath11k_set_vht_txbf_cap(struct ath11k *ar, u32 *vht_cap) subfer = false; } + if (ar->num_rx_chains < 2) { + *vht_cap &= ~(IEEE80211_VHT_CAP_SU_BEAMFORMEE_CAPABLE); + subfee = false; + } + /* If SU Beaformer is not set, then disable MU Beamformer Capability */ if (!subfer) *vht_cap &= ~(IEEE80211_VHT_CAP_MU_BEAMFORMER_CAPABLE); @@ -5016,7 +5029,9 @@ static void ath11k_set_vht_txbf_cap(struct ath11k *ar, u32 *vht_cap) sound_dim >>= IEEE80211_VHT_CAP_SOUNDING_DIMENSIONS_SHIFT; *vht_cap &= ~IEEE80211_VHT_CAP_SOUNDING_DIMENSIONS_MASK; - /* TODO: Need to check invalid STS and Sound_dim values set by FW? */ + nsts = (*vht_cap & IEEE80211_VHT_CAP_BEAMFORMEE_STS_MASK); + nsts >>= IEEE80211_VHT_CAP_BEAMFORMEE_STS_SHIFT; + *vht_cap &= ~IEEE80211_VHT_CAP_BEAMFORMEE_STS_MASK; /* Enable Sounding Dimension Field only if SU BF is enabled */ if (subfer) { @@ -5028,9 +5043,15 @@ static void ath11k_set_vht_txbf_cap(struct ath11k *ar, u32 *vht_cap) *vht_cap |= sound_dim; } - /* Use the STS advertised by FW unless SU Beamformee is not supported*/ - if (!subfee) - *vht_cap &= ~(IEEE80211_VHT_CAP_BEAMFORMEE_STS_MASK); + /* Enable Beamformee STS Field only if SU BF is enabled */ + if (subfee) { + if (nsts > (ar->num_rx_chains - 1)) + nsts = ar->num_rx_chains - 1; + + nsts <<= IEEE80211_VHT_CAP_BEAMFORMEE_STS_SHIFT; + nsts &= IEEE80211_VHT_CAP_BEAMFORMEE_STS_MASK; + *vht_cap |= nsts; + } } static struct ieee80211_sta_vht_cap @@ -6173,6 +6194,13 @@ static int ath11k_mac_op_add_interface(struct ieee80211_hw *hw, goto err; } + /* In the case of hardware recovery, debugfs files are + * not deleted since ieee80211_ops.remove_interface() is + * not invoked. In such cases, try to delete the files. + * These will be re-created later. + */ + ath11k_debugfs_remove_interface(arvif); + memset(arvif, 0, sizeof(*arvif)); arvif->ar = ar; @@ -6354,9 +6382,7 @@ static int ath11k_mac_op_add_interface(struct ieee80211_hw *hw, } } - ret = ath11k_debugfs_add_interface(arvif); - if (ret) - goto err_peer_del; + ath11k_debugfs_add_interface(arvif); mutex_unlock(&ar->conf_mutex); @@ -8421,6 +8447,95 @@ exit: return ret; } +static int ath11k_fw_stats_request(struct ath11k *ar, + struct stats_request_params *req_param) +{ + struct ath11k_base *ab = ar->ab; + unsigned long time_left; + int ret; + + lockdep_assert_held(&ar->conf_mutex); + + spin_lock_bh(&ar->data_lock); + ar->fw_stats_done = false; + ath11k_fw_stats_pdevs_free(&ar->fw_stats.pdevs); + spin_unlock_bh(&ar->data_lock); + + reinit_completion(&ar->fw_stats_complete); + + ret = ath11k_wmi_send_stats_request_cmd(ar, req_param); + if (ret) { + ath11k_warn(ab, "could not request fw stats (%d)\n", + ret); + return ret; + } + + time_left = wait_for_completion_timeout(&ar->fw_stats_complete, + 1 * HZ); + + if (!time_left) + return -ETIMEDOUT; + + return 0; +} + +static int ath11k_mac_op_get_txpower(struct ieee80211_hw *hw, + struct ieee80211_vif *vif, + int *dbm) +{ + struct ath11k *ar = hw->priv; + struct ath11k_base *ab = ar->ab; + struct stats_request_params req_param = {0}; + struct ath11k_fw_stats_pdev *pdev; + int ret; + + /* Final Tx power is minimum of Target Power, CTL power, Regulatory + * Power, PSD EIRP Power. We just know the Regulatory power from the + * regulatory rules obtained. FW knows all these power and sets the min + * of these. Hence, we request the FW pdev stats in which FW reports + * the minimum of all vdev's channel Tx power. + */ + mutex_lock(&ar->conf_mutex); + + if (ar->state != ATH11K_STATE_ON) + goto err_fallback; + + req_param.pdev_id = ar->pdev->pdev_id; + req_param.stats_id = WMI_REQUEST_PDEV_STAT; + + ret = ath11k_fw_stats_request(ar, &req_param); + if (ret) { + ath11k_warn(ab, "failed to request fw pdev stats: %d\n", ret); + goto err_fallback; + } + + spin_lock_bh(&ar->data_lock); + pdev = list_first_entry_or_null(&ar->fw_stats.pdevs, + struct ath11k_fw_stats_pdev, list); + if (!pdev) { + spin_unlock_bh(&ar->data_lock); + goto err_fallback; + } + + /* tx power is set as 2 units per dBm in FW. */ + *dbm = pdev->chan_tx_power / 2; + + spin_unlock_bh(&ar->data_lock); + mutex_unlock(&ar->conf_mutex); + + ath11k_dbg(ar->ab, ATH11K_DBG_MAC, "txpower from firmware %d, reported %d dBm\n", + pdev->chan_tx_power, *dbm); + return 0; + +err_fallback: + mutex_unlock(&ar->conf_mutex); + /* We didn't get txpower from FW. Hence, relying on vif->bss_conf.txpower */ + *dbm = vif->bss_conf.txpower; + ath11k_dbg(ar->ab, ATH11K_DBG_MAC, "txpower from firmware NaN, reported %d dBm\n", + *dbm); + return 0; +} + static const struct ieee80211_ops ath11k_ops = { .tx = ath11k_mac_op_tx, .start = ath11k_mac_op_start, @@ -8471,6 +8586,7 @@ static const struct ieee80211_ops ath11k_ops = { #if IS_ENABLED(CONFIG_IPV6) .ipv6_addr_change = ath11k_mac_op_ipv6_changed, #endif + .get_txpower = ath11k_mac_op_get_txpower, .set_sar_specs = ath11k_mac_op_set_bios_sar_specs, .remain_on_channel = ath11k_mac_op_remain_on_channel, @@ -8777,6 +8893,11 @@ static int __ath11k_mac_register(struct ath11k *ar) if (ab->hw_params.single_pdev_only && ar->supports_6ghz) ieee80211_hw_set(ar->hw, SINGLE_SCAN_ON_ALL_BANDS); + if (ab->hw_params.supports_multi_bssid) { + ieee80211_hw_set(ar->hw, SUPPORTS_MULTI_BSSID); + ieee80211_hw_set(ar->hw, SUPPORTS_ONLY_HE_MULTI_BSSID); + } + ieee80211_hw_set(ar->hw, SIGNAL_DBM); ieee80211_hw_set(ar->hw, SUPPORTS_PS); ieee80211_hw_set(ar->hw, SUPPORTS_DYNAMIC_PS); @@ -8967,6 +9088,7 @@ int ath11k_mac_register(struct ath11k_base *ab) struct ath11k_pdev *pdev; int i; int ret; + u8 mac_addr[ETH_ALEN] = {0}; if (test_bit(ATH11K_FLAG_REGISTERED, &ab->dev_flags)) return 0; @@ -8979,13 +9101,18 @@ int ath11k_mac_register(struct ath11k_base *ab) if (ret) return ret; + device_get_mac_address(ab->dev, mac_addr); + for (i = 0; i < ab->num_radios; i++) { pdev = &ab->pdevs[i]; ar = pdev->ar; if (ab->pdevs_macaddr_valid) { ether_addr_copy(ar->mac_addr, pdev->mac_addr); } else { - ether_addr_copy(ar->mac_addr, ab->mac_addr); + if (is_zero_ether_addr(mac_addr)) + ether_addr_copy(ar->mac_addr, ab->mac_addr); + else + ether_addr_copy(ar->mac_addr, mac_addr); ar->mac_addr[4] += i; } @@ -9079,6 +9206,8 @@ int ath11k_mac_allocate(struct ath11k_base *ab) clear_bit(ATH11K_FLAG_MONITOR_VDEV_CREATED, &ar->monitor_flags); ar->vdev_id_11d_scan = ATH11K_11D_INVALID_VDEV_ID; init_completion(&ar->completed_11d_scan); + + ath11k_fw_stats_init(ar); } return 0; diff --git a/drivers/net/wireless/ath/ath11k/mhi.c b/drivers/net/wireless/ath/ath11k/mhi.c index c44df17719f6..86995e8dc913 100644 --- a/drivers/net/wireless/ath/ath11k/mhi.c +++ b/drivers/net/wireless/ath/ath11k/mhi.c @@ -402,8 +402,7 @@ int ath11k_mhi_register(struct ath11k_pci *ab_pci) ret = ath11k_mhi_get_msi(ab_pci); if (ret) { ath11k_err(ab, "failed to get msi for mhi\n"); - mhi_free_controller(mhi_ctrl); - return ret; + goto free_controller; } if (!test_bit(ATH11K_FLAG_MULTI_MSI_VECTORS, &ab->dev_flags)) @@ -412,7 +411,7 @@ int ath11k_mhi_register(struct ath11k_pci *ab_pci) if (test_bit(ATH11K_FLAG_FIXED_MEM_RGN, &ab->dev_flags)) { ret = ath11k_mhi_read_addr_from_dt(mhi_ctrl); if (ret < 0) - return ret; + goto free_controller; } else { mhi_ctrl->iova_start = 0; mhi_ctrl->iova_stop = 0xFFFFFFFF; @@ -440,18 +439,22 @@ int ath11k_mhi_register(struct ath11k_pci *ab_pci) default: ath11k_err(ab, "failed assign mhi_config for unknown hw rev %d\n", ab->hw_rev); - mhi_free_controller(mhi_ctrl); - return -EINVAL; + ret = -EINVAL; + goto free_controller; } ret = mhi_register_controller(mhi_ctrl, ath11k_mhi_config); if (ret) { ath11k_err(ab, "failed to register to mhi bus, err = %d\n", ret); - mhi_free_controller(mhi_ctrl); - return ret; + goto free_controller; } return 0; + +free_controller: + mhi_free_controller(mhi_ctrl); + ab_pci->mhi_ctrl = NULL; + return ret; } void ath11k_mhi_unregister(struct ath11k_pci *ab_pci) diff --git a/drivers/net/wireless/ath/ath11k/pci.c b/drivers/net/wireless/ath/ath11k/pci.c index 5bd34a6273d9..99cf3357c66e 100644 --- a/drivers/net/wireless/ath/ath11k/pci.c +++ b/drivers/net/wireless/ath/ath11k/pci.c @@ -685,6 +685,7 @@ static const struct ath11k_hif_ops ath11k_pci_hif_ops = { .stop = ath11k_pcic_stop, .read32 = ath11k_pcic_read32, .write32 = ath11k_pcic_write32, + .read = ath11k_pcic_read, .power_down = ath11k_pci_power_down, .power_up = ath11k_pci_power_up, .suspend = ath11k_pci_hif_suspend, diff --git a/drivers/net/wireless/ath/ath11k/pcic.c b/drivers/net/wireless/ath/ath11k/pcic.c index 1adf20ebef27..380f9d37b644 100644 --- a/drivers/net/wireless/ath/ath11k/pcic.c +++ b/drivers/net/wireless/ath/ath11k/pcic.c @@ -140,55 +140,100 @@ int ath11k_pcic_init_msi_config(struct ath11k_base *ab) } EXPORT_SYMBOL(ath11k_pcic_init_msi_config); +static void __ath11k_pcic_write32(struct ath11k_base *ab, u32 offset, u32 value) +{ + if (offset < ATH11K_PCI_WINDOW_START) + iowrite32(value, ab->mem + offset); + else + ab->pci.ops->window_write32(ab, offset, value); +} + void ath11k_pcic_write32(struct ath11k_base *ab, u32 offset, u32 value) { int ret = 0; + bool wakeup_required; /* for offset beyond BAR + 4K - 32, may * need to wakeup the device to access. */ - if (test_bit(ATH11K_FLAG_DEVICE_INIT_DONE, &ab->dev_flags) && - offset >= ATH11K_PCI_ACCESS_ALWAYS_OFF && ab->pci.ops->wakeup) + wakeup_required = test_bit(ATH11K_FLAG_DEVICE_INIT_DONE, &ab->dev_flags) && + offset >= ATH11K_PCI_ACCESS_ALWAYS_OFF; + if (wakeup_required && ab->pci.ops->wakeup) ret = ab->pci.ops->wakeup(ab); - if (offset < ATH11K_PCI_WINDOW_START) - iowrite32(value, ab->mem + offset); - else - ab->pci.ops->window_write32(ab, offset, value); + __ath11k_pcic_write32(ab, offset, value); - if (test_bit(ATH11K_FLAG_DEVICE_INIT_DONE, &ab->dev_flags) && - offset >= ATH11K_PCI_ACCESS_ALWAYS_OFF && ab->pci.ops->release && - !ret) + if (wakeup_required && !ret && ab->pci.ops->release) ab->pci.ops->release(ab); } EXPORT_SYMBOL(ath11k_pcic_write32); +static u32 __ath11k_pcic_read32(struct ath11k_base *ab, u32 offset) +{ + u32 val; + + if (offset < ATH11K_PCI_WINDOW_START) + val = ioread32(ab->mem + offset); + else + val = ab->pci.ops->window_read32(ab, offset); + + return val; +} + u32 ath11k_pcic_read32(struct ath11k_base *ab, u32 offset) { int ret = 0; u32 val; + bool wakeup_required; /* for offset beyond BAR + 4K - 32, may * need to wakeup the device to access. */ - if (test_bit(ATH11K_FLAG_DEVICE_INIT_DONE, &ab->dev_flags) && - offset >= ATH11K_PCI_ACCESS_ALWAYS_OFF && ab->pci.ops->wakeup) + wakeup_required = test_bit(ATH11K_FLAG_DEVICE_INIT_DONE, &ab->dev_flags) && + offset >= ATH11K_PCI_ACCESS_ALWAYS_OFF; + if (wakeup_required && ab->pci.ops->wakeup) ret = ab->pci.ops->wakeup(ab); - if (offset < ATH11K_PCI_WINDOW_START) - val = ioread32(ab->mem + offset); - else - val = ab->pci.ops->window_read32(ab, offset); + val = __ath11k_pcic_read32(ab, offset); - if (test_bit(ATH11K_FLAG_DEVICE_INIT_DONE, &ab->dev_flags) && - offset >= ATH11K_PCI_ACCESS_ALWAYS_OFF && ab->pci.ops->release && - !ret) + if (wakeup_required && !ret && ab->pci.ops->release) ab->pci.ops->release(ab); return val; } EXPORT_SYMBOL(ath11k_pcic_read32); +int ath11k_pcic_read(struct ath11k_base *ab, void *buf, u32 start, u32 end) +{ + int ret = 0; + bool wakeup_required; + u32 *data = buf; + u32 i; + + /* for offset beyond BAR + 4K - 32, may + * need to wakeup the device to access. + */ + wakeup_required = test_bit(ATH11K_FLAG_DEVICE_INIT_DONE, &ab->dev_flags) && + end >= ATH11K_PCI_ACCESS_ALWAYS_OFF; + if (wakeup_required && ab->pci.ops->wakeup) { + ret = ab->pci.ops->wakeup(ab); + if (ret) { + ath11k_warn(ab, "failed to wakeup for read from 0x%x: %d\n", + start, ret); + return ret; + } + } + + for (i = start; i < end + 1; i += 4) + *data++ = __ath11k_pcic_read32(ab, i); + + if (wakeup_required && ab->pci.ops->release) + ab->pci.ops->release(ab); + + return 0; +} +EXPORT_SYMBOL(ath11k_pcic_read); + void ath11k_pcic_get_msi_address(struct ath11k_base *ab, u32 *msi_addr_lo, u32 *msi_addr_hi) { @@ -414,6 +459,7 @@ void ath11k_pcic_ext_irq_enable(struct ath11k_base *ab) struct ath11k_ext_irq_grp *irq_grp = &ab->ext_irq_grp[i]; if (!irq_grp->napi_enabled) { + dev_set_threaded(&irq_grp->napi_ndev, true); napi_enable(&irq_grp->napi); irq_grp->napi_enabled = true; } @@ -517,7 +563,7 @@ static int ath11k_pcic_ext_irq_config(struct ath11k_base *ab) irq_grp->grp_id = i; init_dummy_netdev(&irq_grp->napi_ndev); netif_napi_add(&irq_grp->napi_ndev, &irq_grp->napi, - ath11k_pcic_ext_grp_napi_poll, NAPI_POLL_WEIGHT); + ath11k_pcic_ext_grp_napi_poll); if (ab->hw_params.ring_mask->tx[i] || ab->hw_params.ring_mask->rx[i] || @@ -731,3 +777,37 @@ int ath11k_pcic_register_pci_ops(struct ath11k_base *ab, return 0; } EXPORT_SYMBOL(ath11k_pcic_register_pci_ops); + +void ath11k_pci_enable_ce_irqs_except_wake_irq(struct ath11k_base *ab) +{ + int i; + + for (i = 0; i < ab->hw_params.ce_count; i++) { + if (ath11k_ce_get_attr_flags(ab, i) & CE_ATTR_DIS_INTR || + i == ATH11K_PCI_CE_WAKE_IRQ) + continue; + ath11k_pcic_ce_irq_enable(ab, i); + } +} +EXPORT_SYMBOL(ath11k_pci_enable_ce_irqs_except_wake_irq); + +void ath11k_pci_disable_ce_irqs_except_wake_irq(struct ath11k_base *ab) +{ + int i; + int irq_idx; + struct ath11k_ce_pipe *ce_pipe; + + for (i = 0; i < ab->hw_params.ce_count; i++) { + ce_pipe = &ab->ce.ce_pipe[i]; + irq_idx = ATH11K_PCI_IRQ_CE0_OFFSET + i; + + if (ath11k_ce_get_attr_flags(ab, i) & CE_ATTR_DIS_INTR || + i == ATH11K_PCI_CE_WAKE_IRQ) + continue; + + disable_irq_nosync(ab->irq_num[irq_idx]); + synchronize_irq(ab->irq_num[irq_idx]); + tasklet_kill(&ce_pipe->intr_tq); + } +} +EXPORT_SYMBOL(ath11k_pci_disable_ce_irqs_except_wake_irq); diff --git a/drivers/net/wireless/ath/ath11k/pcic.h b/drivers/net/wireless/ath/ath11k/pcic.h index 0afbb34510db..ac012e88bf6d 100644 --- a/drivers/net/wireless/ath/ath11k/pcic.h +++ b/drivers/net/wireless/ath/ath11k/pcic.h @@ -12,6 +12,8 @@ #define ATH11K_PCI_IRQ_CE0_OFFSET 3 #define ATH11K_PCI_IRQ_DP_OFFSET 14 +#define ATH11K_PCI_CE_WAKE_IRQ 2 + #define ATH11K_PCI_WINDOW_ENABLE_BIT 0x40000000 #define ATH11K_PCI_WINDOW_REG_ADDRESS 0x310c #define ATH11K_PCI_WINDOW_VALUE_MASK GENMASK(24, 19) @@ -45,4 +47,8 @@ void ath11k_pcic_ce_irq_disable_sync(struct ath11k_base *ab); int ath11k_pcic_init_msi_config(struct ath11k_base *ab); int ath11k_pcic_register_pci_ops(struct ath11k_base *ab, const struct ath11k_pci_ops *pci_ops); +int ath11k_pcic_read(struct ath11k_base *ab, void *buf, u32 start, u32 end); +void ath11k_pci_enable_ce_irqs_except_wake_irq(struct ath11k_base *ab); +void ath11k_pci_disable_ce_irqs_except_wake_irq(struct ath11k_base *ab); + #endif diff --git a/drivers/net/wireless/ath/ath11k/peer.c b/drivers/net/wireless/ath/ath11k/peer.c index 9e22aaf34b88..1ae7af02c364 100644 --- a/drivers/net/wireless/ath/ath11k/peer.c +++ b/drivers/net/wireless/ath/ath11k/peer.c @@ -302,6 +302,21 @@ static int __ath11k_peer_delete(struct ath11k *ar, u32 vdev_id, const u8 *addr) spin_lock_bh(&ab->base_lock); peer = ath11k_peer_find_by_addr(ab, addr); + /* Check if the found peer is what we want to remove. + * While the sta is transitioning to another band we may + * have 2 peer with the same addr assigned to different + * vdev_id. Make sure we are deleting the correct peer. + */ + if (peer && peer->vdev_id == vdev_id) + ath11k_peer_rhash_delete(ab, peer); + + /* Fallback to peer list search if the correct peer can't be found. + * Skip the deletion of the peer from the rhash since it has already + * been deleted in peer add. + */ + if (!peer) + peer = ath11k_peer_find(ab, vdev_id, addr); + if (!peer) { spin_unlock_bh(&ab->base_lock); mutex_unlock(&ab->tbl_mtx_lock); @@ -312,8 +327,6 @@ static int __ath11k_peer_delete(struct ath11k *ar, u32 vdev_id, const u8 *addr) return -EINVAL; } - ath11k_peer_rhash_delete(ab, peer); - spin_unlock_bh(&ab->base_lock); mutex_unlock(&ab->tbl_mtx_lock); @@ -372,8 +385,17 @@ int ath11k_peer_create(struct ath11k *ar, struct ath11k_vif *arvif, spin_lock_bh(&ar->ab->base_lock); peer = ath11k_peer_find_by_addr(ar->ab, param->peer_addr); if (peer) { - spin_unlock_bh(&ar->ab->base_lock); - return -EINVAL; + if (peer->vdev_id == param->vdev_id) { + spin_unlock_bh(&ar->ab->base_lock); + return -EINVAL; + } + + /* Assume sta is transitioning to another band. + * Remove here the peer from rhash. + */ + mutex_lock(&ar->ab->tbl_mtx_lock); + ath11k_peer_rhash_delete(ar->ab, peer); + mutex_unlock(&ar->ab->tbl_mtx_lock); } spin_unlock_bh(&ar->ab->base_lock); diff --git a/drivers/net/wireless/ath/ath11k/qmi.c b/drivers/net/wireless/ath/ath11k/qmi.c index 00136601cb7d..51de2208b789 100644 --- a/drivers/net/wireless/ath/ath11k/qmi.c +++ b/drivers/net/wireless/ath/ath11k/qmi.c @@ -1696,6 +1696,13 @@ static struct qmi_elem_info qmi_wlanfw_wlan_ini_resp_msg_v01_ei[] = { }, }; +static struct qmi_elem_info qmi_wlfw_fw_init_done_ind_msg_v01_ei[] = { + { + .data_type = QMI_EOTI, + .array_type = NO_ARRAY, + }, +}; + static int ath11k_qmi_host_cap_send(struct ath11k_base *ab) { struct qmi_wlanfw_host_cap_req_msg_v01 req; @@ -1872,7 +1879,7 @@ static int ath11k_qmi_respond_fw_mem_request(struct ath11k_base *ab) /* For QCA6390 by default FW requests a block of ~4M contiguous * DMA memory, it's hard to allocate from OS. So host returns - * failure to FW and FW will then request mulitple blocks of small + * failure to FW and FW will then request multiple blocks of small * chunk size memory. */ if (!(ab->hw_params.fixed_mem_region || @@ -3006,6 +3013,12 @@ static void ath11k_qmi_msg_fw_ready_cb(struct qmi_handle *qmi_hdl, struct ath11k_base *ab = qmi->ab; ath11k_dbg(ab, ATH11K_DBG_QMI, "qmi firmware ready\n"); + + if (!ab->qmi.cal_done) { + ab->qmi.cal_done = 1; + wake_up(&ab->qmi.cold_boot_waitq); + } + ath11k_qmi_driver_event_post(qmi, ATH11K_QMI_EVENT_FW_READY, NULL); } @@ -3023,6 +3036,19 @@ static void ath11k_qmi_msg_cold_boot_cal_done_cb(struct qmi_handle *qmi_hdl, ath11k_dbg(ab, ATH11K_DBG_QMI, "qmi cold boot calibration done\n"); } +static void ath11k_qmi_msg_fw_init_done_cb(struct qmi_handle *qmi_hdl, + struct sockaddr_qrtr *sq, + struct qmi_txn *txn, + const void *decoded) +{ + struct ath11k_qmi *qmi = container_of(qmi_hdl, + struct ath11k_qmi, handle); + struct ath11k_base *ab = qmi->ab; + + ath11k_qmi_driver_event_post(qmi, ATH11K_QMI_EVENT_FW_INIT_DONE, NULL); + ath11k_dbg(ab, ATH11K_DBG_QMI, "qmi firmware init done\n"); +} + static const struct qmi_msg_handler ath11k_qmi_msg_handlers[] = { { .type = QMI_INDICATION, @@ -3053,6 +3079,14 @@ static const struct qmi_msg_handler ath11k_qmi_msg_handlers[] = { sizeof(struct qmi_wlanfw_fw_cold_cal_done_ind_msg_v01), .fn = ath11k_qmi_msg_cold_boot_cal_done_cb, }, + { + .type = QMI_INDICATION, + .msg_id = QMI_WLFW_FW_INIT_DONE_IND_V01, + .ei = qmi_wlfw_fw_init_done_ind_msg_v01_ei, + .decoded_size = + sizeof(struct qmi_wlfw_fw_init_done_ind_msg_v01), + .fn = ath11k_qmi_msg_fw_init_done_cb, + }, }; static int ath11k_qmi_ops_new_server(struct qmi_handle *qmi_hdl, @@ -3145,7 +3179,7 @@ static void ath11k_qmi_driver_event_work(struct work_struct *work) } break; - case ATH11K_QMI_EVENT_FW_READY: + case ATH11K_QMI_EVENT_FW_INIT_DONE: clear_bit(ATH11K_FLAG_QMI_FAIL, &ab->dev_flags); if (test_bit(ATH11K_FLAG_REGISTERED, &ab->dev_flags)) { ath11k_hal_dump_srng_stats(ab); @@ -3169,6 +3203,22 @@ static void ath11k_qmi_driver_event_work(struct work_struct *work) } break; + case ATH11K_QMI_EVENT_FW_READY: + /* For targets requiring a FW restart upon cold + * boot completion, there is no need to process + * FW ready; such targets will receive FW init + * done message after FW restart. + */ + if (ab->hw_params.cbcal_restart_fw) + break; + + clear_bit(ATH11K_FLAG_CRASH_FLUSH, + &ab->dev_flags); + clear_bit(ATH11K_FLAG_RECOVERY, &ab->dev_flags); + ath11k_core_qmi_firmware_ready(ab); + set_bit(ATH11K_FLAG_REGISTERED, &ab->dev_flags); + + break; case ATH11K_QMI_EVENT_COLD_BOOT_CAL_DONE: break; default: diff --git a/drivers/net/wireless/ath/ath11k/qmi.h b/drivers/net/wireless/ath/ath11k/qmi.h index c83cf822be81..2ec56a34fa81 100644 --- a/drivers/net/wireless/ath/ath11k/qmi.h +++ b/drivers/net/wireless/ath/ath11k/qmi.h @@ -31,8 +31,9 @@ #define QMI_WLFW_REQUEST_MEM_IND_V01 0x0035 #define QMI_WLFW_FW_MEM_READY_IND_V01 0x0037 -#define QMI_WLFW_COLD_BOOT_CAL_DONE_IND_V01 0x0021 -#define QMI_WLFW_FW_READY_IND_V01 0x0038 +#define QMI_WLFW_COLD_BOOT_CAL_DONE_IND_V01 0x003E +#define QMI_WLFW_FW_READY_IND_V01 0x0021 +#define QMI_WLFW_FW_INIT_DONE_IND_V01 0x0038 #define QMI_WLANFW_MAX_DATA_SIZE_V01 6144 #define ATH11K_FIRMWARE_MODE_OFF 4 @@ -69,6 +70,7 @@ enum ath11k_qmi_event_type { ATH11K_QMI_EVENT_FORCE_FW_ASSERT, ATH11K_QMI_EVENT_POWER_UP, ATH11K_QMI_EVENT_POWER_DOWN, + ATH11K_QMI_EVENT_FW_INIT_DONE, ATH11K_QMI_EVENT_MAX, }; @@ -291,6 +293,10 @@ struct qmi_wlanfw_fw_cold_cal_done_ind_msg_v01 { char placeholder; }; +struct qmi_wlfw_fw_init_done_ind_msg_v01 { + char placeholder; +}; + #define QMI_WLANFW_CAP_REQ_MSG_V01_MAX_LEN 0 #define QMI_WLANFW_CAP_RESP_MSG_V01_MAX_LEN 235 #define QMI_WLANFW_CAP_REQ_V01 0x0024 diff --git a/drivers/net/wireless/ath/ath11k/rx_desc.h b/drivers/net/wireless/ath/ath11k/rx_desc.h index 26ecc1bcd9d5..786d5f36f5e5 100644 --- a/drivers/net/wireless/ath/ath11k/rx_desc.h +++ b/drivers/net/wireless/ath/ath11k/rx_desc.h @@ -877,7 +877,7 @@ struct rx_msdu_start_wcn6855 { * * l4_offset * Depending upon mode bit, this field either indicates the - * L4 offset nin bytes from the start of RX_HEADER (only valid + * L4 offset in bytes from the start of RX_HEADER (only valid * if either ipv4_proto or ipv6_proto is set to 1) or indicates * the offset in bytes to the start of TCP or UDP header from * the start of the IP header after decapsulation (Only valid if diff --git a/drivers/net/wireless/ath/ath11k/spectral.c b/drivers/net/wireless/ath/ath11k/spectral.c index 516a7b4cd180..705868198df4 100644 --- a/drivers/net/wireless/ath/ath11k/spectral.c +++ b/drivers/net/wireless/ath/ath11k/spectral.c @@ -30,6 +30,7 @@ #define ATH11K_SPECTRAL_20MHZ 20 #define ATH11K_SPECTRAL_40MHZ 40 #define ATH11K_SPECTRAL_80MHZ 80 +#define ATH11K_SPECTRAL_160MHZ 160 #define ATH11K_SPECTRAL_SIGNATURE 0xFA @@ -183,6 +184,8 @@ static int ath11k_spectral_scan_trigger(struct ath11k *ar) if (ar->spectral.mode == ATH11K_SPECTRAL_DISABLED) return 0; + ar->spectral.is_primary = true; + ret = ath11k_wmi_vdev_spectral_enable(ar, arvif->vdev_id, ATH11K_WMI_SPECTRAL_TRIGGER_CMD_CLEAR, ATH11K_WMI_SPECTRAL_ENABLE_CMD_ENABLE); @@ -585,6 +588,7 @@ int ath11k_spectral_process_fft(struct ath11k *ar, u8 chan_width_mhz, bin_sz; int ret; u32 check_length; + bool fragment_sample = false; lockdep_assert_held(&ar->spectral.lock); @@ -639,6 +643,13 @@ int ath11k_spectral_process_fft(struct ath11k *ar, case ATH11K_SPECTRAL_80MHZ: fft_sample->chan_width_mhz = chan_width_mhz; break; + case ATH11K_SPECTRAL_160MHZ: + if (ab->hw_params.spectral.fragment_160mhz) { + chan_width_mhz /= 2; + fragment_sample = true; + } + fft_sample->chan_width_mhz = chan_width_mhz; + break; default: ath11k_warn(ab, "invalid channel width %d\n", chan_width_mhz); return -EINVAL; @@ -663,6 +674,17 @@ int ath11k_spectral_process_fft(struct ath11k *ar, freq = summary->meta.freq2; fft_sample->freq2 = __cpu_to_be16(freq); + /* If freq2 is available then the spectral scan results are fragmented + * as primary and secondary + */ + if (fragment_sample && freq) { + if (!ar->spectral.is_primary) + fft_sample->freq1 = cpu_to_be16(freq); + + /* We have to toggle the is_primary to handle the next report */ + ar->spectral.is_primary = !ar->spectral.is_primary; + } + ath11k_spectral_parse_fft(fft_sample->data, fft_report->bins, num_bins, ab->hw_params.spectral.fft_sz); diff --git a/drivers/net/wireless/ath/ath11k/spectral.h b/drivers/net/wireless/ath/ath11k/spectral.h index 081744265f2a..96bfa16e18e9 100644 --- a/drivers/net/wireless/ath/ath11k/spectral.h +++ b/drivers/net/wireless/ath/ath11k/spectral.h @@ -35,6 +35,7 @@ struct ath11k_spectral { u16 count; u8 fft_size; bool enabled; + bool is_primary; }; #ifdef CONFIG_ATH11K_SPECTRAL diff --git a/drivers/net/wireless/ath/ath11k/thermal.c b/drivers/net/wireless/ath/ath11k/thermal.c index c96b26f39a25..23ed01bd44f9 100644 --- a/drivers/net/wireless/ath/ath11k/thermal.c +++ b/drivers/net/wireless/ath/ath11k/thermal.c @@ -99,7 +99,7 @@ static ssize_t ath11k_thermal_show_temp(struct device *dev, temperature = ar->thermal.temperature; spin_unlock_bh(&ar->data_lock); - /* display in millidegree celcius */ + /* display in millidegree Celsius */ ret = snprintf(buf, PAGE_SIZE, "%d\n", temperature * 1000); out: mutex_unlock(&ar->conf_mutex); diff --git a/drivers/net/wireless/ath/ath11k/thermal.h b/drivers/net/wireless/ath/ath11k/thermal.h index f9af55f3682d..3e39675ef7f5 100644 --- a/drivers/net/wireless/ath/ath11k/thermal.h +++ b/drivers/net/wireless/ath/ath11k/thermal.h @@ -19,7 +19,7 @@ struct ath11k_thermal { /* protected by conf_mutex */ u32 throttle_state; - /* temperature value in Celcius degree + /* temperature value in Celsius degree * protected by data_lock */ int temperature; diff --git a/drivers/net/wireless/ath/ath11k/trace.h b/drivers/net/wireless/ath/ath11k/trace.h index 76560587bea0..9535745fe026 100644 --- a/drivers/net/wireless/ath/ath11k/trace.h +++ b/drivers/net/wireless/ath/ath11k/trace.h @@ -305,6 +305,34 @@ TRACE_EVENT(ath11k_wmi_diag, ) ); +TRACE_EVENT(ath11k_ps_timekeeper, + TP_PROTO(struct ath11k *ar, const void *peer_addr, + u32 peer_ps_timestamp, u8 peer_ps_state), + TP_ARGS(ar, peer_addr, peer_ps_timestamp, peer_ps_state), + + TP_STRUCT__entry(__string(device, dev_name(ar->ab->dev)) + __string(driver, dev_driver_string(ar->ab->dev)) + __dynamic_array(u8, peer_addr, ETH_ALEN) + __field(u8, peer_ps_state) + __field(u32, peer_ps_timestamp) + ), + + TP_fast_assign(__assign_str(device, dev_name(ar->ab->dev)); + __assign_str(driver, dev_driver_string(ar->ab->dev)); + memcpy(__get_dynamic_array(peer_addr), peer_addr, + ETH_ALEN); + __entry->peer_ps_state = peer_ps_state; + __entry->peer_ps_timestamp = peer_ps_timestamp; + ), + + TP_printk("%s %s %u %u", + __get_str(driver), + __get_str(device), + __entry->peer_ps_state, + __entry->peer_ps_timestamp + ) +); + #endif /* _TRACE_H_ || TRACE_HEADER_MULTI_READ*/ /* we don't want to use include/trace/events */ diff --git a/drivers/net/wireless/ath/ath11k/wmi.c b/drivers/net/wireless/ath/ath11k/wmi.c index 88ee4f9d19da..fad9f8d308a2 100644 --- a/drivers/net/wireless/ath/ath11k/wmi.c +++ b/drivers/net/wireless/ath/ath11k/wmi.c @@ -416,7 +416,7 @@ ath11k_pull_mac_phy_cap_svc_ready_ext(struct ath11k_pdev_wmi *wmi_handle, /* tx/rx chainmask reported from fw depends on the actual hw chains used, * For example, for 4x4 capable macphys, first 4 chains can be used for first - * mac and the remaing 4 chains can be used for the second mac or vice-versa. + * mac and the remaining 4 chains can be used for the second mac or vice-versa. * In this case, tx/rx chainmask 0xf will be advertised for first mac and 0xf0 * will be advertised for second mac or vice-versa. Compute the shift value * for tx/rx chainmask which will be used to advertise supported ht/vht rates to @@ -991,9 +991,13 @@ int ath11k_wmi_vdev_up(struct ath11k *ar, u32 vdev_id, u32 aid, const u8 *bssid) { struct ath11k_pdev_wmi *wmi = ar->wmi; struct wmi_vdev_up_cmd *cmd; + struct ieee80211_bss_conf *bss_conf; + struct ath11k_vif *arvif; struct sk_buff *skb; int ret; + arvif = ath11k_mac_get_arvif(ar, vdev_id); + skb = ath11k_wmi_alloc_skb(wmi->wmi_ab, sizeof(*cmd)); if (!skb) return -ENOMEM; @@ -1007,6 +1011,17 @@ int ath11k_wmi_vdev_up(struct ath11k *ar, u32 vdev_id, u32 aid, const u8 *bssid) ether_addr_copy(cmd->vdev_bssid.addr, bssid); + if (arvif && arvif->vif->type == NL80211_IFTYPE_STATION) { + bss_conf = &arvif->vif->bss_conf; + + if (bss_conf->nontransmitted) { + ether_addr_copy(cmd->trans_bssid.addr, + bss_conf->transmitter_bssid); + cmd->profile_idx = bss_conf->bssid_index; + cmd->profile_num = bss_conf->bssid_indicator; + } + } + ret = ath11k_wmi_cmd_send(wmi, skb, WMI_VDEV_UP_CMDID); if (ret) { ath11k_warn(ar->ab, "failed to submit WMI_VDEV_UP cmd\n"); @@ -3064,8 +3079,34 @@ int ath11k_wmi_pdev_pktlog_disable(struct ath11k *ar) return ret; } -int -ath11k_wmi_send_twt_enable_cmd(struct ath11k *ar, u32 pdev_id) +void ath11k_wmi_fill_default_twt_params(struct wmi_twt_enable_params *twt_params) +{ + twt_params->sta_cong_timer_ms = ATH11K_TWT_DEF_STA_CONG_TIMER_MS; + twt_params->default_slot_size = ATH11K_TWT_DEF_DEFAULT_SLOT_SIZE; + twt_params->congestion_thresh_setup = ATH11K_TWT_DEF_CONGESTION_THRESH_SETUP; + twt_params->congestion_thresh_teardown = + ATH11K_TWT_DEF_CONGESTION_THRESH_TEARDOWN; + twt_params->congestion_thresh_critical = + ATH11K_TWT_DEF_CONGESTION_THRESH_CRITICAL; + twt_params->interference_thresh_teardown = + ATH11K_TWT_DEF_INTERFERENCE_THRESH_TEARDOWN; + twt_params->interference_thresh_setup = + ATH11K_TWT_DEF_INTERFERENCE_THRESH_SETUP; + twt_params->min_no_sta_setup = ATH11K_TWT_DEF_MIN_NO_STA_SETUP; + twt_params->min_no_sta_teardown = ATH11K_TWT_DEF_MIN_NO_STA_TEARDOWN; + twt_params->no_of_bcast_mcast_slots = ATH11K_TWT_DEF_NO_OF_BCAST_MCAST_SLOTS; + twt_params->min_no_twt_slots = ATH11K_TWT_DEF_MIN_NO_TWT_SLOTS; + twt_params->max_no_sta_twt = ATH11K_TWT_DEF_MAX_NO_STA_TWT; + twt_params->mode_check_interval = ATH11K_TWT_DEF_MODE_CHECK_INTERVAL; + twt_params->add_sta_slot_interval = ATH11K_TWT_DEF_ADD_STA_SLOT_INTERVAL; + twt_params->remove_sta_slot_interval = + ATH11K_TWT_DEF_REMOVE_STA_SLOT_INTERVAL; + /* TODO add MBSSID support */ + twt_params->mbss_support = 0; +} + +int ath11k_wmi_send_twt_enable_cmd(struct ath11k *ar, u32 pdev_id, + struct wmi_twt_enable_params *params) { struct ath11k_pdev_wmi *wmi = ar->wmi; struct ath11k_base *ab = wmi->wmi_ab->ab; @@ -3083,28 +3124,22 @@ ath11k_wmi_send_twt_enable_cmd(struct ath11k *ar, u32 pdev_id) cmd->tlv_header = FIELD_PREP(WMI_TLV_TAG, WMI_TAG_TWT_ENABLE_CMD) | FIELD_PREP(WMI_TLV_LEN, len - TLV_HDR_SIZE); cmd->pdev_id = pdev_id; - cmd->sta_cong_timer_ms = ATH11K_TWT_DEF_STA_CONG_TIMER_MS; - cmd->default_slot_size = ATH11K_TWT_DEF_DEFAULT_SLOT_SIZE; - cmd->congestion_thresh_setup = ATH11K_TWT_DEF_CONGESTION_THRESH_SETUP; - cmd->congestion_thresh_teardown = - ATH11K_TWT_DEF_CONGESTION_THRESH_TEARDOWN; - cmd->congestion_thresh_critical = - ATH11K_TWT_DEF_CONGESTION_THRESH_CRITICAL; - cmd->interference_thresh_teardown = - ATH11K_TWT_DEF_INTERFERENCE_THRESH_TEARDOWN; - cmd->interference_thresh_setup = - ATH11K_TWT_DEF_INTERFERENCE_THRESH_SETUP; - cmd->min_no_sta_setup = ATH11K_TWT_DEF_MIN_NO_STA_SETUP; - cmd->min_no_sta_teardown = ATH11K_TWT_DEF_MIN_NO_STA_TEARDOWN; - cmd->no_of_bcast_mcast_slots = ATH11K_TWT_DEF_NO_OF_BCAST_MCAST_SLOTS; - cmd->min_no_twt_slots = ATH11K_TWT_DEF_MIN_NO_TWT_SLOTS; - cmd->max_no_sta_twt = ATH11K_TWT_DEF_MAX_NO_STA_TWT; - cmd->mode_check_interval = ATH11K_TWT_DEF_MODE_CHECK_INTERVAL; - cmd->add_sta_slot_interval = ATH11K_TWT_DEF_ADD_STA_SLOT_INTERVAL; - cmd->remove_sta_slot_interval = - ATH11K_TWT_DEF_REMOVE_STA_SLOT_INTERVAL; - /* TODO add MBSSID support */ - cmd->mbss_support = 0; + cmd->sta_cong_timer_ms = params->sta_cong_timer_ms; + cmd->default_slot_size = params->default_slot_size; + cmd->congestion_thresh_setup = params->congestion_thresh_setup; + cmd->congestion_thresh_teardown = params->congestion_thresh_teardown; + cmd->congestion_thresh_critical = params->congestion_thresh_critical; + cmd->interference_thresh_teardown = params->interference_thresh_teardown; + cmd->interference_thresh_setup = params->interference_thresh_setup; + cmd->min_no_sta_setup = params->min_no_sta_setup; + cmd->min_no_sta_teardown = params->min_no_sta_teardown; + cmd->no_of_bcast_mcast_slots = params->no_of_bcast_mcast_slots; + cmd->min_no_twt_slots = params->min_no_twt_slots; + cmd->max_no_sta_twt = params->max_no_sta_twt; + cmd->mode_check_interval = params->mode_check_interval; + cmd->add_sta_slot_interval = params->add_sta_slot_interval; + cmd->remove_sta_slot_interval = params->remove_sta_slot_interval; + cmd->mbss_support = params->mbss_support; ret = ath11k_wmi_cmd_send(wmi, skb, WMI_TWT_ENABLE_CMDID); if (ret) { @@ -6767,6 +6802,107 @@ static void ath11k_bcn_tx_status_event(struct ath11k_base *ab, struct sk_buff *s rcu_read_unlock(); } +static void ath11k_wmi_event_peer_sta_ps_state_chg(struct ath11k_base *ab, + struct sk_buff *skb) +{ + const struct wmi_peer_sta_ps_state_chg_event *ev; + struct ieee80211_sta *sta; + struct ath11k_peer *peer; + struct ath11k *ar; + struct ath11k_sta *arsta; + const void **tb; + enum ath11k_wmi_peer_ps_state peer_previous_ps_state; + int ret; + + tb = ath11k_wmi_tlv_parse_alloc(ab, skb->data, skb->len, GFP_ATOMIC); + if (IS_ERR(tb)) { + ret = PTR_ERR(tb); + ath11k_warn(ab, "failed to parse tlv: %d\n", ret); + return; + } + + ev = tb[WMI_TAG_PEER_STA_PS_STATECHANGE_EVENT]; + if (!ev) { + ath11k_warn(ab, "failed to fetch sta ps change ev"); + kfree(tb); + return; + } + + ath11k_dbg(ab, ATH11K_DBG_WMI, + "peer sta ps chnange ev addr %pM state %u sup_bitmap %x ps_valid %u ts %u\n", + ev->peer_macaddr.addr, ev->peer_ps_state, + ev->ps_supported_bitmap, ev->peer_ps_valid, + ev->peer_ps_timestamp); + + rcu_read_lock(); + + spin_lock_bh(&ab->base_lock); + + peer = ath11k_peer_find_by_addr(ab, ev->peer_macaddr.addr); + + if (!peer) { + spin_unlock_bh(&ab->base_lock); + ath11k_warn(ab, "peer not found %pM\n", ev->peer_macaddr.addr); + goto exit; + } + + ar = ath11k_mac_get_ar_by_vdev_id(ab, peer->vdev_id); + + if (!ar) { + spin_unlock_bh(&ab->base_lock); + ath11k_warn(ab, "invalid vdev id in peer sta ps state change ev %d", + peer->vdev_id); + + goto exit; + } + + sta = peer->sta; + + spin_unlock_bh(&ab->base_lock); + + if (!sta) { + ath11k_warn(ab, "failed to find station entry %pM\n", + ev->peer_macaddr.addr); + goto exit; + } + + arsta = (struct ath11k_sta *)sta->drv_priv; + + spin_lock_bh(&ar->data_lock); + + peer_previous_ps_state = arsta->peer_ps_state; + arsta->peer_ps_state = ev->peer_ps_state; + arsta->peer_current_ps_valid = !!ev->peer_ps_valid; + + if (test_bit(WMI_TLV_SERVICE_PEER_POWER_SAVE_DURATION_SUPPORT, + ar->ab->wmi_ab.svc_map)) { + if (!(ev->ps_supported_bitmap & WMI_PEER_PS_VALID) || + !(ev->ps_supported_bitmap & WMI_PEER_PS_STATE_TIMESTAMP) || + !ev->peer_ps_valid) + goto out; + + if (arsta->peer_ps_state == WMI_PEER_PS_STATE_ON) { + arsta->ps_start_time = ev->peer_ps_timestamp; + arsta->ps_start_jiffies = jiffies; + } else if (arsta->peer_ps_state == WMI_PEER_PS_STATE_OFF && + peer_previous_ps_state == WMI_PEER_PS_STATE_ON) { + arsta->ps_total_duration = arsta->ps_total_duration + + (ev->peer_ps_timestamp - arsta->ps_start_time); + } + + if (ar->ps_timekeeper_enable) + trace_ath11k_ps_timekeeper(ar, ev->peer_macaddr.addr, + ev->peer_ps_timestamp, + arsta->peer_ps_state); + } + +out: + spin_unlock_bh(&ar->data_lock); +exit: + rcu_read_unlock(); + kfree(tb); +} + static void ath11k_vdev_stopped_event(struct ath11k_base *ab, struct sk_buff *skb) { struct ath11k *ar; @@ -7409,7 +7545,53 @@ static void ath11k_peer_assoc_conf_event(struct ath11k_base *ab, struct sk_buff static void ath11k_update_stats_event(struct ath11k_base *ab, struct sk_buff *skb) { - ath11k_debugfs_fw_stats_process(ab, skb); + struct ath11k_fw_stats stats = {}; + struct ath11k *ar; + int ret; + + INIT_LIST_HEAD(&stats.pdevs); + INIT_LIST_HEAD(&stats.vdevs); + INIT_LIST_HEAD(&stats.bcn); + + ret = ath11k_wmi_pull_fw_stats(ab, skb, &stats); + if (ret) { + ath11k_warn(ab, "failed to pull fw stats: %d\n", ret); + goto free; + } + + rcu_read_lock(); + ar = ath11k_mac_get_ar_by_pdev_id(ab, stats.pdev_id); + if (!ar) { + rcu_read_unlock(); + ath11k_warn(ab, "failed to get ar for pdev_id %d: %d\n", + stats.pdev_id, ret); + goto free; + } + + spin_lock_bh(&ar->data_lock); + + /* WMI_REQUEST_PDEV_STAT can be requested via .get_txpower mac ops or via + * debugfs fw stats. Therefore, processing it separately. + */ + if (stats.stats_id == WMI_REQUEST_PDEV_STAT) { + list_splice_tail_init(&stats.pdevs, &ar->fw_stats.pdevs); + ar->fw_stats_done = true; + goto complete; + } + + /* WMI_REQUEST_VDEV_STAT, WMI_REQUEST_BCN_STAT and WMI_REQUEST_RSSI_PER_CHAIN_STAT + * are currently requested only via debugfs fw stats. Hence, processing these + * in debugfs context + */ + ath11k_debugfs_fw_stats_process(ar, &stats); + +complete: + complete(&ar->fw_stats_complete); + rcu_read_unlock(); + spin_unlock_bh(&ar->data_lock); + +free: + ath11k_fw_stats_free(&stats); } /* PDEV_CTL_FAILSAFE_CHECK_EVENT is received from FW when the frequency scanned @@ -7960,6 +8142,9 @@ static void ath11k_wmi_tlv_op_rx(struct ath11k_base *ab, struct sk_buff *skb) case WMI_DIAG_EVENTID: ath11k_wmi_diag_event(ab, skb); break; + case WMI_PEER_STA_PS_STATECHG_EVENTID: + ath11k_wmi_event_peer_sta_ps_state_chg(ab, skb); + break; case WMI_GTK_OFFLOAD_STATUS_EVENTID: ath11k_wmi_gtk_offload_status_event(ab, skb); break; @@ -8962,12 +9147,13 @@ int ath11k_wmi_sta_keepalive(struct ath11k *ar, cmd->interval = arg->interval; cmd->method = arg->method; + arp = (struct wmi_sta_keepalive_arp_resp *)(cmd + 1); + arp->tlv_header = FIELD_PREP(WMI_TLV_TAG, + WMI_TAG_STA_KEEPALIVE_ARP_RESPONSE) | + FIELD_PREP(WMI_TLV_LEN, sizeof(*arp) - TLV_HDR_SIZE); + if (arg->method == WMI_STA_KEEPALIVE_METHOD_UNSOLICITED_ARP_RESPONSE || arg->method == WMI_STA_KEEPALIVE_METHOD_GRATUITOUS_ARP_REQUEST) { - arp = (struct wmi_sta_keepalive_arp_resp *)(cmd + 1); - arp->tlv_header = FIELD_PREP(WMI_TLV_TAG, - WMI_TAG_STA_KEEPALVE_ARP_RESPONSE) | - FIELD_PREP(WMI_TLV_LEN, sizeof(*arp) - TLV_HDR_SIZE); arp->src_ip4_addr = arg->src_ip4_addr; arp->dest_ip4_addr = arg->dest_ip4_addr; ether_addr_copy(arp->dest_mac_addr.addr, arg->dest_mac_addr); diff --git a/drivers/net/wireless/ath/ath11k/wmi.h b/drivers/net/wireless/ath/ath11k/wmi.h index 4da248ffa318..8f2c07d70a4a 100644 --- a/drivers/net/wireless/ath/ath11k/wmi.h +++ b/drivers/net/wireless/ath/ath11k/wmi.h @@ -17,7 +17,7 @@ struct ath11k_vif; #define PSOC_HOST_MAX_NUM_SS (8) -/* defines to set Packet extension values whic can be 0 us, 8 usec or 16 usec */ +/* defines to set Packet extension values which can be 0 us, 8 usec or 16 usec */ #define MAX_HE_NSS 8 #define MAX_HE_MODULATION 8 #define MAX_HE_RU 4 @@ -1214,7 +1214,7 @@ enum wmi_tlv_tag { WMI_TAG_NS_OFFLOAD_TUPLE, WMI_TAG_FTM_INTG_CMD, WMI_TAG_STA_KEEPALIVE_CMD, - WMI_TAG_STA_KEEPALVE_ARP_RESPONSE, + WMI_TAG_STA_KEEPALIVE_ARP_RESPONSE, WMI_TAG_P2P_SET_VENDOR_IE_DATA_CMD, WMI_TAG_AP_PS_PEER_CMD, WMI_TAG_PEER_RATE_RETRY_SCHED_CMD, @@ -2090,6 +2090,7 @@ enum wmi_tlv_service { WMI_TLV_SERVICE_PER_PEER_HTT_STATS_RESET = 213, WMI_TLV_SERVICE_FREQINFO_IN_METADATA = 219, WMI_TLV_SERVICE_EXT2_MSG = 220, + WMI_TLV_SERVICE_PEER_POWER_SAVE_DURATION_SUPPORT = 246, WMI_TLV_SERVICE_SRG_SRP_SPATIAL_REUSE_SUPPORT = 249, /* The second 128 bits */ @@ -4482,7 +4483,7 @@ struct wmi_pdev_radar_ev { } __packed; struct wmi_pdev_temperature_event { - /* temperature value in Celcius degree */ + /* temperature value in Celsius degree */ s32 temp; u32 pdev_id; } __packed; @@ -4708,7 +4709,7 @@ enum wmi_sta_ps_param_tx_wake_threshold { */ enum wmi_sta_ps_param_pspoll_count { WMI_STA_PS_PSPOLL_COUNT_NO_MAX = 0, - /* Values greater than 0 indicate the maximum numer of PS-Poll frames + /* Values greater than 0 indicate the maximum number of PS-Poll frames * FW will send before waking up. */ }; @@ -4820,9 +4821,9 @@ enum wmi_rate_preamble { /** * enum wmi_rtscts_prot_mode - Enable/Disable RTS/CTS and CTS2Self Protection. - * @WMI_RTS_CTS_DISABLED : RTS/CTS protection is disabled. - * @WMI_USE_RTS_CTS : RTS/CTS Enabled. - * @WMI_USE_CTS2SELF : CTS to self protection Enabled. + * @WMI_RTS_CTS_DISABLED: RTS/CTS protection is disabled. + * @WMI_USE_RTS_CTS: RTS/CTS Enabled. + * @WMI_USE_CTS2SELF: CTS to self protection Enabled. */ enum wmi_rtscts_prot_mode { WMI_RTS_CTS_DISABLED = 0, @@ -4833,13 +4834,13 @@ enum wmi_rtscts_prot_mode { /** * enum wmi_rtscts_profile - Selection of RTS CTS profile along with enabling * protection mode. - * @WMI_RTSCTS_FOR_NO_RATESERIES - Neither of rate-series should use RTS-CTS - * @WMI_RTSCTS_FOR_SECOND_RATESERIES - Only second rate-series will use RTS-CTS - * @WMI_RTSCTS_ACROSS_SW_RETRIES - Only the second rate-series will use RTS-CTS, - * but if there's a sw retry, both the rate - * series will use RTS-CTS. - * @WMI_RTSCTS_ERP - RTS/CTS used for ERP protection for every PPDU. - * @WMI_RTSCTS_FOR_ALL_RATESERIES - Enable RTS-CTS for all rate series. + * @WMI_RTSCTS_FOR_NO_RATESERIES: Neither of rate-series should use RTS-CTS + * @WMI_RTSCTS_FOR_SECOND_RATESERIES: Only second rate-series will use RTS-CTS + * @WMI_RTSCTS_ACROSS_SW_RETRIES: Only the second rate-series will use RTS-CTS, + * but if there's a sw retry, both the rate + * series will use RTS-CTS. + * @WMI_RTSCTS_ERP: RTS/CTS used for ERP protection for every PPDU. + * @WMI_RTSCTS_FOR_ALL_RATESERIES: Enable RTS-CTS for all rate series. */ enum wmi_rtscts_profile { WMI_RTSCTS_FOR_NO_RATESERIES = 0, @@ -4933,6 +4934,25 @@ struct wmi_wmm_params_all_arg { #define ATH11K_TWT_DEF_ADD_STA_SLOT_INTERVAL 1000 #define ATH11K_TWT_DEF_REMOVE_STA_SLOT_INTERVAL 5000 +struct wmi_twt_enable_params { + u32 sta_cong_timer_ms; + u32 mbss_support; + u32 default_slot_size; + u32 congestion_thresh_setup; + u32 congestion_thresh_teardown; + u32 congestion_thresh_critical; + u32 interference_thresh_teardown; + u32 interference_thresh_setup; + u32 min_no_sta_setup; + u32 min_no_sta_teardown; + u32 no_of_bcast_mcast_slots; + u32 min_no_twt_slots; + u32 max_no_sta_twt; + u32 mode_check_interval; + u32 add_sta_slot_interval; + u32 remove_sta_slot_interval; +}; + struct wmi_twt_enable_params_cmd { u32 tlv_header; u32 pdev_id; @@ -5350,6 +5370,26 @@ struct wmi_debug_log_config_cmd_fixed_param { #define WMI_SERVICE_READY_TIMEOUT_HZ (5 * HZ) #define WMI_SEND_TIMEOUT_HZ (3 * HZ) +enum ath11k_wmi_peer_ps_state { + WMI_PEER_PS_STATE_OFF, + WMI_PEER_PS_STATE_ON, + WMI_PEER_PS_STATE_DISABLED, +}; + +enum wmi_peer_ps_supported_bitmap { + /* Used to indicate that power save state change is valid */ + WMI_PEER_PS_VALID = 0x1, + WMI_PEER_PS_STATE_TIMESTAMP = 0x2, +}; + +struct wmi_peer_sta_ps_state_chg_event { + struct wmi_mac_addr peer_macaddr; + u32 peer_ps_state; + u32 ps_supported_bitmap; + u32 peer_ps_valid; + u32 peer_ps_timestamp; +} __packed; + struct ath11k_wmi_base { struct ath11k_base *ab; struct ath11k_pdev_wmi wmi[MAX_RADIOS]; @@ -6039,7 +6079,9 @@ void ath11k_wmi_fw_stats_fill(struct ath11k *ar, struct ath11k_fw_stats *fw_stats, u32 stats_id, char *buf); int ath11k_wmi_simulate_radar(struct ath11k *ar); -int ath11k_wmi_send_twt_enable_cmd(struct ath11k *ar, u32 pdev_id); +void ath11k_wmi_fill_default_twt_params(struct wmi_twt_enable_params *twt_params); +int ath11k_wmi_send_twt_enable_cmd(struct ath11k *ar, u32 pdev_id, + struct wmi_twt_enable_params *params); int ath11k_wmi_send_twt_disable_cmd(struct ath11k *ar, u32 pdev_id); int ath11k_wmi_send_twt_add_dialog_cmd(struct ath11k *ar, struct wmi_twt_add_dialog_params *params); diff --git a/drivers/net/wireless/ath/ath11k/wow.c b/drivers/net/wireless/ath/ath11k/wow.c index b3e65cd13d83..1dec23b0699c 100644 --- a/drivers/net/wireless/ath/ath11k/wow.c +++ b/drivers/net/wireless/ath/ath11k/wow.c @@ -1,6 +1,7 @@ // SPDX-License-Identifier: BSD-3-Clause-Clear /* * Copyright (c) 2020 The Linux Foundation. All rights reserved. + * Copyright (c) 2022, Qualcomm Innovation Center, Inc. All rights reserved. */ #include <linux/delay.h> @@ -67,6 +68,13 @@ int ath11k_wow_wakeup(struct ath11k_base *ab) struct ath11k *ar = ath11k_ab_to_ar(ab, 0); int ret; + /* In the case of WCN6750, WoW wakeup is done + * by sending SMP2P power save exit message + * to the target processor. + */ + if (ab->hw_params.smp2p_wow_exit) + return 0; + reinit_completion(&ab->wow.wakeup_completed); ret = ath11k_wmi_wow_host_wakeup_ind(ar); @@ -664,6 +672,12 @@ int ath11k_wow_op_suspend(struct ieee80211_hw *hw, struct ath11k *ar = hw->priv; int ret; + ret = ath11k_mac_wait_tx_complete(ar); + if (ret) { + ath11k_warn(ar->ab, "failed to wait tx complete: %d\n", ret); + return ret; + } + mutex_lock(&ar->conf_mutex); ret = ath11k_dp_rx_pktlog_stop(ar->ab, true); @@ -695,13 +709,6 @@ int ath11k_wow_op_suspend(struct ieee80211_hw *hw, goto cleanup; } - ath11k_mac_drain_tx(ar); - ret = ath11k_mac_wait_tx_complete(ar); - if (ret) { - ath11k_warn(ar->ab, "failed to wait tx complete: %d\n", ret); - goto cleanup; - } - ret = ath11k_wow_set_hw_filter(ar); if (ret) { ath11k_warn(ar->ab, "failed to set hw filter: %d\n", diff --git a/drivers/net/wireless/ath/ath6kl/cfg80211.c b/drivers/net/wireless/ath/ath6kl/cfg80211.c index e11c7e9accc0..a20e0aeae284 100644 --- a/drivers/net/wireless/ath/ath6kl/cfg80211.c +++ b/drivers/net/wireless/ath/ath6kl/cfg80211.c @@ -1124,7 +1124,7 @@ void ath6kl_cfg80211_ch_switch_notify(struct ath6kl_vif *vif, int freq, } static int ath6kl_cfg80211_add_key(struct wiphy *wiphy, struct net_device *ndev, - u8 key_index, bool pairwise, + int link_id, u8 key_index, bool pairwise, const u8 *mac_addr, struct key_params *params) { @@ -1249,7 +1249,7 @@ static int ath6kl_cfg80211_add_key(struct wiphy *wiphy, struct net_device *ndev, } static int ath6kl_cfg80211_del_key(struct wiphy *wiphy, struct net_device *ndev, - u8 key_index, bool pairwise, + int link_id, u8 key_index, bool pairwise, const u8 *mac_addr) { struct ath6kl *ar = ath6kl_priv(ndev); @@ -1279,7 +1279,7 @@ static int ath6kl_cfg80211_del_key(struct wiphy *wiphy, struct net_device *ndev, } static int ath6kl_cfg80211_get_key(struct wiphy *wiphy, struct net_device *ndev, - u8 key_index, bool pairwise, + int link_id, u8 key_index, bool pairwise, const u8 *mac_addr, void *cookie, void (*callback) (void *cookie, struct key_params *)) @@ -1314,7 +1314,7 @@ static int ath6kl_cfg80211_get_key(struct wiphy *wiphy, struct net_device *ndev, } static int ath6kl_cfg80211_set_default_key(struct wiphy *wiphy, - struct net_device *ndev, + struct net_device *ndev, int link_id, u8 key_index, bool unicast, bool multicast) { diff --git a/drivers/net/wireless/ath/ath6kl/init.c b/drivers/net/wireless/ath/ath6kl/init.c index 9b5c7d8f2b95..201e45554070 100644 --- a/drivers/net/wireless/ath/ath6kl/init.c +++ b/drivers/net/wireless/ath/ath6kl/init.c @@ -1014,7 +1014,7 @@ static int ath6kl_fetch_fw_apin(struct ath6kl *ar, const char *name) switch (ie_id) { case ATH6KL_FW_IE_FW_VERSION: - strlcpy(ar->wiphy->fw_version, data, + strscpy(ar->wiphy->fw_version, data, min(sizeof(ar->wiphy->fw_version), ie_len+1)); ath6kl_dbg(ATH6KL_DBG_BOOT, diff --git a/drivers/net/wireless/ath/ath9k/ar9003_phy.c b/drivers/net/wireless/ath/ath9k/ar9003_phy.c index dc0e5ea25673..090ff0600c81 100644 --- a/drivers/net/wireless/ath/ath9k/ar9003_phy.c +++ b/drivers/net/wireless/ath/ath9k/ar9003_phy.c @@ -1744,7 +1744,7 @@ static void ar9003_hw_spectral_scan_config(struct ath_hw *ah, REG_SET_BIT(ah, AR_PHY_RADAR_0, AR_PHY_RADAR_0_FFT_ENA); REG_SET_BIT(ah, AR_PHY_SPECTRAL_SCAN, AR_PHY_SPECTRAL_SCAN_ENABLE); - /* on AR93xx and newer, count = 0 will make the the chip send + /* on AR93xx and newer, count = 0 will make the chip send * spectral samples endlessly. Check if this really was intended, * and fix otherwise. */ diff --git a/drivers/net/wireless/ath/ath9k/channel.c b/drivers/net/wireless/ath/ath9k/channel.c index 6cf087522157..571062f2e82a 100644 --- a/drivers/net/wireless/ath/ath9k/channel.c +++ b/drivers/net/wireless/ath/ath9k/channel.c @@ -1113,7 +1113,7 @@ ath_chanctx_send_vif_ps_frame(struct ath_softc *sc, struct ath_vif *avp, if (!avp->assoc) return false; - skb = ieee80211_nullfunc_get(sc->hw, vif, false); + skb = ieee80211_nullfunc_get(sc->hw, vif, -1, false); if (!skb) return false; diff --git a/drivers/net/wireless/ath/ath9k/htc_hst.c b/drivers/net/wireless/ath/ath9k/htc_hst.c index 994ec48b2f66..ca05b07a45e6 100644 --- a/drivers/net/wireless/ath/ath9k/htc_hst.c +++ b/drivers/net/wireless/ath/ath9k/htc_hst.c @@ -364,33 +364,27 @@ ret: } static void ath9k_htc_fw_panic_report(struct htc_target *htc_handle, - struct sk_buff *skb) + struct sk_buff *skb, u32 len) { uint32_t *pattern = (uint32_t *)skb->data; - switch (*pattern) { - case 0x33221199: - { + if (*pattern == 0x33221199 && len >= sizeof(struct htc_panic_bad_vaddr)) { struct htc_panic_bad_vaddr *htc_panic; htc_panic = (struct htc_panic_bad_vaddr *) skb->data; dev_err(htc_handle->dev, "ath: firmware panic! " "exccause: 0x%08x; pc: 0x%08x; badvaddr: 0x%08x.\n", htc_panic->exccause, htc_panic->pc, htc_panic->badvaddr); - break; - } - case 0x33221299: - { + return; + } + if (*pattern == 0x33221299) { struct htc_panic_bad_epid *htc_panic; htc_panic = (struct htc_panic_bad_epid *) skb->data; dev_err(htc_handle->dev, "ath: firmware panic! " "bad epid: 0x%08x\n", htc_panic->epid); - break; - } - default: - dev_err(htc_handle->dev, "ath: unknown panic pattern!\n"); - break; + return; } + dev_err(htc_handle->dev, "ath: unknown panic pattern!\n"); } /* @@ -411,16 +405,26 @@ void ath9k_htc_rx_msg(struct htc_target *htc_handle, if (!htc_handle || !skb) return; + /* A valid message requires len >= 8. + * + * sizeof(struct htc_frame_hdr) == 8 + * sizeof(struct htc_ready_msg) == 8 + * sizeof(struct htc_panic_bad_vaddr) == 16 + * sizeof(struct htc_panic_bad_epid) == 8 + */ + if (unlikely(len < sizeof(struct htc_frame_hdr))) + goto invalid; htc_hdr = (struct htc_frame_hdr *) skb->data; epid = htc_hdr->endpoint_id; if (epid == 0x99) { - ath9k_htc_fw_panic_report(htc_handle, skb); + ath9k_htc_fw_panic_report(htc_handle, skb, len); kfree_skb(skb); return; } if (epid < 0 || epid >= ENDPOINT_MAX) { +invalid: if (pipe_id != USB_REG_IN_PIPE) dev_kfree_skb_any(skb); else @@ -432,21 +436,30 @@ void ath9k_htc_rx_msg(struct htc_target *htc_handle, /* Handle trailer */ if (htc_hdr->flags & HTC_FLAGS_RECV_TRAILER) { - if (be32_to_cpu(*(__be32 *) skb->data) == 0x00C60000) + if (be32_to_cpu(*(__be32 *) skb->data) == 0x00C60000) { /* Move past the Watchdog pattern */ htc_hdr = (struct htc_frame_hdr *)(skb->data + 4); + len -= 4; + } } /* Get the message ID */ + if (unlikely(len < sizeof(struct htc_frame_hdr) + sizeof(__be16))) + goto invalid; msg_id = (__be16 *) ((void *) htc_hdr + sizeof(struct htc_frame_hdr)); /* Now process HTC messages */ switch (be16_to_cpu(*msg_id)) { case HTC_MSG_READY_ID: + if (unlikely(len < sizeof(struct htc_ready_msg))) + goto invalid; htc_process_target_rdy(htc_handle, htc_hdr); break; case HTC_MSG_CONNECT_SERVICE_RESPONSE_ID: + if (unlikely(len < sizeof(struct htc_frame_hdr) + + sizeof(struct htc_conn_svc_rspmsg))) + goto invalid; htc_process_conn_rsp(htc_handle, htc_hdr); break; default: diff --git a/drivers/net/wireless/ath/ath9k/hw.h b/drivers/net/wireless/ath/ath9k/hw.h index 096a206f49ed..450ab19b1d4e 100644 --- a/drivers/net/wireless/ath/ath9k/hw.h +++ b/drivers/net/wireless/ath/ath9k/hw.h @@ -710,7 +710,7 @@ struct ath_spec_scan { /** * struct ath_hw_ops - callbacks used by hardware code and driver code * - * This structure contains callbacks designed to to be used internally by + * This structure contains callbacks designed to be used internally by * hardware code and also by the lower level driver. * * @config_pci_powersave: diff --git a/drivers/net/wireless/ath/ath9k/xmit.c b/drivers/net/wireless/ath/ath9k/xmit.c index ba16a7f3e23d..ba271a10d4ab 100644 --- a/drivers/net/wireless/ath/ath9k/xmit.c +++ b/drivers/net/wireless/ath/ath9k/xmit.c @@ -2160,7 +2160,7 @@ static void setup_frame_info(struct ieee80211_hw *hw, fi->keyix = an->ps_key; else fi->keyix = ATH9K_TXKEYIX_INVALID; - fi->dyn_smps = sta && sta->smps_mode == IEEE80211_SMPS_DYNAMIC; + fi->dyn_smps = sta && sta->deflink.smps_mode == IEEE80211_SMPS_DYNAMIC; fi->keytype = keytype; fi->framelen = framelen; fi->tx_power = txpower; diff --git a/drivers/net/wireless/ath/carl9170/fw.c b/drivers/net/wireless/ath/carl9170/fw.c index 1ab09e1c9ec5..4c1aecd1163c 100644 --- a/drivers/net/wireless/ath/carl9170/fw.c +++ b/drivers/net/wireless/ath/carl9170/fw.c @@ -105,7 +105,7 @@ static void carl9170_fw_info(struct ar9170 *ar) CARL9170FW_GET_MONTH(fw_date), CARL9170FW_GET_DAY(fw_date)); - strlcpy(ar->hw->wiphy->fw_version, motd_desc->release, + strscpy(ar->hw->wiphy->fw_version, motd_desc->release, sizeof(ar->hw->wiphy->fw_version)); } } diff --git a/drivers/net/wireless/ath/wcn36xx/hal.h b/drivers/net/wireless/ath/wcn36xx/hal.h index f1a43fd1d957..d3a9d00e65e1 100644 --- a/drivers/net/wireless/ath/wcn36xx/hal.h +++ b/drivers/net/wireless/ath/wcn36xx/hal.h @@ -2677,7 +2677,7 @@ struct ani_global_security_stats { * management information base (MIB) object is enabled */ u32 rx_wep_unencrypted_frm_cnt; - /* The number of received MSDU packets that that the 802.11 station + /* The number of received MSDU packets that the 802.11 station * discarded because of MIC failures */ u32 rx_mic_fail_cnt; diff --git a/drivers/net/wireless/ath/wcn36xx/txrx.c b/drivers/net/wireless/ath/wcn36xx/txrx.c index 8da3955995b6..0802ed728824 100644 --- a/drivers/net/wireless/ath/wcn36xx/txrx.c +++ b/drivers/net/wireless/ath/wcn36xx/txrx.c @@ -16,6 +16,7 @@ #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt +#include <linux/random.h> #include "txrx.h" static inline int get_rssi0(struct wcn36xx_rx_bd *bd) @@ -278,6 +279,7 @@ static void wcn36xx_update_survey(struct wcn36xx *wcn, int rssi, int snr, struct ieee80211_supported_band *sband; int idx; int i; + u8 snr_sample = snr & 0xff; idx = 0; if (band == NL80211_BAND_5GHZ) @@ -297,6 +299,8 @@ static void wcn36xx_update_survey(struct wcn36xx *wcn, int rssi, int snr, wcn->chan_survey[idx].rssi = rssi; wcn->chan_survey[idx].snr = snr; spin_unlock(&wcn->survey_lock); + + add_device_randomness(&snr_sample, sizeof(snr_sample)); } int wcn36xx_rx_skb(struct wcn36xx *wcn, struct sk_buff *skb) diff --git a/drivers/net/wireless/ath/wil6210/cfg80211.c b/drivers/net/wireless/ath/wil6210/cfg80211.c index f93bdffa4d1d..40f9a7ef8980 100644 --- a/drivers/net/wireless/ath/wil6210/cfg80211.c +++ b/drivers/net/wireless/ath/wil6210/cfg80211.c @@ -1620,7 +1620,7 @@ static void wil_del_rx_key(u8 key_index, enum wmi_key_usage key_usage, } static int wil_cfg80211_add_key(struct wiphy *wiphy, - struct net_device *ndev, + struct net_device *ndev, int link_id, u8 key_index, bool pairwise, const u8 *mac_addr, struct key_params *params) @@ -1696,7 +1696,7 @@ static int wil_cfg80211_add_key(struct wiphy *wiphy, } static int wil_cfg80211_del_key(struct wiphy *wiphy, - struct net_device *ndev, + struct net_device *ndev, int link_id, u8 key_index, bool pairwise, const u8 *mac_addr) { @@ -1723,7 +1723,7 @@ static int wil_cfg80211_del_key(struct wiphy *wiphy, /* Need to be present or wiphy_new() will WARN */ static int wil_cfg80211_set_default_key(struct wiphy *wiphy, - struct net_device *ndev, + struct net_device *ndev, int link_id, u8 key_index, bool unicast, bool multicast) { @@ -2072,8 +2072,8 @@ void wil_cfg80211_ap_recovery(struct wil6210_priv *wil) key_params.key = vif->gtk; key_params.key_len = vif->gtk_len; key_params.seq_len = IEEE80211_GCMP_PN_LEN; - rc = wil_cfg80211_add_key(wiphy, ndev, vif->gtk_index, false, - NULL, &key_params); + rc = wil_cfg80211_add_key(wiphy, ndev, -1, vif->gtk_index, + false, NULL, &key_params); if (rc) wil_err(wil, "vif %d recovery add key failed (%d)\n", i, rc); diff --git a/drivers/net/wireless/ath/wil6210/main.c b/drivers/net/wireless/ath/wil6210/main.c index 7da87c9f363f..94e61dbe94f8 100644 --- a/drivers/net/wireless/ath/wil6210/main.c +++ b/drivers/net/wireless/ath/wil6210/main.c @@ -1305,7 +1305,7 @@ void wil_get_board_file(struct wil6210_priv *wil, char *buf, size_t len) board_file = WIL_BOARD_FILE_NAME; } - strlcpy(buf, board_file, len); + strscpy(buf, board_file, len); } static int wil_get_bl_info(struct wil6210_priv *wil) diff --git a/drivers/net/wireless/ath/wil6210/netdev.c b/drivers/net/wireless/ath/wil6210/netdev.c index 87a88f26233e..ee7d7e9c2718 100644 --- a/drivers/net/wireless/ath/wil6210/netdev.c +++ b/drivers/net/wireless/ath/wil6210/netdev.c @@ -445,7 +445,7 @@ int wil_if_add(struct wil6210_priv *wil) wil_dbg_misc(wil, "entered"); - strlcpy(wiphy->fw_version, wil->fw_version, sizeof(wiphy->fw_version)); + strscpy(wiphy->fw_version, wil->fw_version, sizeof(wiphy->fw_version)); rc = wiphy_register(wiphy); if (rc < 0) { @@ -456,14 +456,12 @@ int wil_if_add(struct wil6210_priv *wil) init_dummy_netdev(&wil->napi_ndev); if (wil->use_enhanced_dma_hw) { netif_napi_add(&wil->napi_ndev, &wil->napi_rx, - wil6210_netdev_poll_rx_edma, - NAPI_POLL_WEIGHT); + wil6210_netdev_poll_rx_edma); netif_napi_add_tx(&wil->napi_ndev, &wil->napi_tx, wil6210_netdev_poll_tx_edma); } else { netif_napi_add(&wil->napi_ndev, &wil->napi_rx, - wil6210_netdev_poll_rx, - NAPI_POLL_WEIGHT); + wil6210_netdev_poll_rx); netif_napi_add_tx(&wil->napi_ndev, &wil->napi_tx, wil6210_netdev_poll_tx); } diff --git a/drivers/net/wireless/ath/wil6210/wmi.c b/drivers/net/wireless/ath/wil6210/wmi.c index ea7bd403e706..6a5976a2944c 100644 --- a/drivers/net/wireless/ath/wil6210/wmi.c +++ b/drivers/net/wireless/ath/wil6210/wmi.c @@ -780,7 +780,7 @@ static void wmi_evt_ready(struct wil6210_vif *vif, int id, void *d, int len) return; /* FW load will fail after timeout */ } /* ignore MAC address, we already have it from the boot loader */ - strlcpy(wiphy->fw_version, wil->fw_version, sizeof(wiphy->fw_version)); + strscpy(wiphy->fw_version, wil->fw_version, sizeof(wiphy->fw_version)); if (len > offsetof(struct wmi_ready_event, rfc_read_calib_result)) { wil_dbg_wmi(wil, "rfc calibration result %d\n", diff --git a/drivers/net/wireless/atmel/atmel.c b/drivers/net/wireless/atmel/atmel.c index 0361c8eb2008..45d079b93384 100644 --- a/drivers/net/wireless/atmel/atmel.c +++ b/drivers/net/wireless/atmel/atmel.c @@ -1518,7 +1518,7 @@ struct net_device *init_atmel_card(unsigned short irq, unsigned long port, priv->firmware = NULL; priv->firmware_type = fw_type; if (firmware) /* module parameter */ - strlcpy(priv->firmware_id, firmware, sizeof(priv->firmware_id)); + strscpy(priv->firmware_id, firmware, sizeof(priv->firmware_id)); priv->bus_type = card_present ? BUS_TYPE_PCCARD : BUS_TYPE_PCI; priv->station_state = STATION_STATE_DOWN; priv->do_rx_crc = 0; diff --git a/drivers/net/wireless/broadcom/b43/leds.c b/drivers/net/wireless/broadcom/b43/leds.c index 982a772a9d87..bfe1be345844 100644 --- a/drivers/net/wireless/broadcom/b43/leds.c +++ b/drivers/net/wireless/broadcom/b43/leds.c @@ -118,7 +118,7 @@ static int b43_register_led(struct b43_wldev *dev, struct b43_led *led, led->wl = dev->wl; led->index = led_index; led->activelow = activelow; - strlcpy(led->name, name, sizeof(led->name)); + strscpy(led->name, name, sizeof(led->name)); atomic_set(&led->state, 0); led->led_dev.name = led->name; diff --git a/drivers/net/wireless/broadcom/b43/phy_n.c b/drivers/net/wireless/broadcom/b43/phy_n.c index aa5c99465674..2c0c019a815d 100644 --- a/drivers/net/wireless/broadcom/b43/phy_n.c +++ b/drivers/net/wireless/broadcom/b43/phy_n.c @@ -2479,11 +2479,7 @@ static void b43_nphy_gain_ctl_workarounds_rev19(struct b43_wldev *dev) static void b43_nphy_gain_ctl_workarounds_rev7(struct b43_wldev *dev) { - struct b43_phy *phy = &dev->phy; - - switch (phy->rev) { - /* TODO */ - } + /* TODO - should depend on phy->rev */ } static void b43_nphy_gain_ctl_workarounds_rev3(struct b43_wldev *dev) diff --git a/drivers/net/wireless/broadcom/b43legacy/leds.c b/drivers/net/wireless/broadcom/b43legacy/leds.c index 38b5be3a84e2..79e6fd205bfb 100644 --- a/drivers/net/wireless/broadcom/b43legacy/leds.c +++ b/drivers/net/wireless/broadcom/b43legacy/leds.c @@ -88,7 +88,7 @@ static int b43legacy_register_led(struct b43legacy_wldev *dev, led->dev = dev; led->index = led_index; led->activelow = activelow; - strlcpy(led->name, name, sizeof(led->name)); + strscpy(led->name, name, sizeof(led->name)); led->led_dev.name = led->name; led->led_dev.default_trigger = default_trigger; diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/bcdc.c b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/bcdc.c index 2c95a08a5871..9ec0c60b6da1 100644 --- a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/bcdc.c +++ b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/bcdc.c @@ -87,6 +87,8 @@ struct brcmf_proto_bcdc_header { * plus any space that might be needed * for bus alignment padding. */ +#define ROUND_UP_MARGIN 2048 + struct brcmf_bcdc { u16 reqid; u8 bus_header[BUS_HEADER_LEN]; @@ -368,8 +370,7 @@ brcmf_proto_bcdc_txcomplete(struct device *dev, struct sk_buff *txp, /* await txstatus signal for firmware if active */ if (brcmf_fws_fc_active(bcdc->fws)) { - if (!success) - brcmf_fws_bustxfail(bcdc->fws, txp); + brcmf_fws_bustxcomplete(bcdc->fws, txp, success); } else { if (brcmf_proto_bcdc_hdrpull(bus_if->drvr, false, txp, &ifp)) brcmu_pkt_buf_free_skb(txp); @@ -471,7 +472,7 @@ int brcmf_proto_bcdc_attach(struct brcmf_pub *drvr) drvr->hdrlen += BCDC_HEADER_LEN + BRCMF_PROT_FW_SIGNAL_MAX_TXBYTES; drvr->bus_if->maxctl = BRCMF_DCMD_MAXLEN + - sizeof(struct brcmf_proto_bcdc_dcmd); + sizeof(struct brcmf_proto_bcdc_dcmd) + ROUND_UP_MARGIN; return 0; fail: diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/bcmsdh.c b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/bcmsdh.c index d639bb8b51ae..d0daef674e72 100644 --- a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/bcmsdh.c +++ b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/bcmsdh.c @@ -983,6 +983,7 @@ static const struct sdio_device_id brcmf_sdmmc_ids[] = { BRCMF_SDIO_DEVICE(SDIO_DEVICE_ID_BROADCOM_4359), BRCMF_SDIO_DEVICE(SDIO_DEVICE_ID_BROADCOM_CYPRESS_4373), BRCMF_SDIO_DEVICE(SDIO_DEVICE_ID_BROADCOM_CYPRESS_43012), + BRCMF_SDIO_DEVICE(SDIO_DEVICE_ID_BROADCOM_CYPRESS_43439), BRCMF_SDIO_DEVICE(SDIO_DEVICE_ID_BROADCOM_CYPRESS_43752), BRCMF_SDIO_DEVICE(SDIO_DEVICE_ID_BROADCOM_CYPRESS_89359), { /* end: all zeroes */ } diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/bus.h b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/bus.h index ae5af76e2568..2208ab3aa795 100644 --- a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/bus.h +++ b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/bus.h @@ -6,6 +6,8 @@ #ifndef BRCMFMAC_BUS_H #define BRCMFMAC_BUS_H +#include <linux/kernel.h> +#include <linux/firmware.h> #include "debug.h" /* IDs of the 6 default common rings of msgbuf protocol */ @@ -34,6 +36,11 @@ enum brcmf_bus_protocol_type { BRCMF_PROTO_MSGBUF }; +/* Firmware blobs that may be available */ +enum brcmf_blob_type { + BRCMF_BLOB_CLM, +}; + struct brcmf_mp_device; struct brcmf_bus_dcmd { @@ -60,7 +67,7 @@ struct brcmf_bus_dcmd { * @wowl_config: specify if dongle is configured for wowl when going to suspend * @get_ramsize: obtain size of device memory. * @get_memdump: obtain device memory dump in provided buffer. - * @get_fwname: obtain firmware name. + * @get_blob: obtain a firmware blob. * * This structure provides an abstract interface towards the * bus specific driver. For control messages to common driver @@ -77,8 +84,8 @@ struct brcmf_bus_ops { void (*wowl_config)(struct device *dev, bool enabled); size_t (*get_ramsize)(struct device *dev); int (*get_memdump)(struct device *dev, void *data, size_t len); - int (*get_fwname)(struct device *dev, const char *ext, - unsigned char *fw_name); + int (*get_blob)(struct device *dev, const struct firmware **fw, + enum brcmf_blob_type type); void (*debugfs_create)(struct device *dev); int (*reset)(struct device *dev); }; @@ -220,10 +227,10 @@ int brcmf_bus_get_memdump(struct brcmf_bus *bus, void *data, size_t len) } static inline -int brcmf_bus_get_fwname(struct brcmf_bus *bus, const char *ext, - unsigned char *fw_name) +int brcmf_bus_get_blob(struct brcmf_bus *bus, const struct firmware **fw, + enum brcmf_blob_type type) { - return bus->ops->get_fwname(bus->dev, ext, fw_name); + return bus->ops->get_blob(bus->dev, fw, type); } static inline diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/cfg80211.c b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/cfg80211.c index db45da33adfd..dfcfb3333369 100644 --- a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/cfg80211.c +++ b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/cfg80211.c @@ -2361,7 +2361,8 @@ done: static s32 brcmf_cfg80211_config_default_key(struct wiphy *wiphy, struct net_device *ndev, - u8 key_idx, bool unicast, bool multicast) + int link_id, u8 key_idx, bool unicast, + bool multicast) { struct brcmf_if *ifp = netdev_priv(ndev); struct brcmf_pub *drvr = ifp->drvr; @@ -2395,7 +2396,8 @@ done: static s32 brcmf_cfg80211_del_key(struct wiphy *wiphy, struct net_device *ndev, - u8 key_idx, bool pairwise, const u8 *mac_addr) + int link_id, u8 key_idx, bool pairwise, + const u8 *mac_addr) { struct brcmf_if *ifp = netdev_priv(ndev); struct brcmf_wsec_key *key; @@ -2432,8 +2434,8 @@ brcmf_cfg80211_del_key(struct wiphy *wiphy, struct net_device *ndev, static s32 brcmf_cfg80211_add_key(struct wiphy *wiphy, struct net_device *ndev, - u8 key_idx, bool pairwise, const u8 *mac_addr, - struct key_params *params) + int link_id, u8 key_idx, bool pairwise, + const u8 *mac_addr, struct key_params *params) { struct brcmf_cfg80211_info *cfg = wiphy_to_cfg(wiphy); struct brcmf_if *ifp = netdev_priv(ndev); @@ -2457,8 +2459,8 @@ brcmf_cfg80211_add_key(struct wiphy *wiphy, struct net_device *ndev, } if (params->key_len == 0) - return brcmf_cfg80211_del_key(wiphy, ndev, key_idx, pairwise, - mac_addr); + return brcmf_cfg80211_del_key(wiphy, ndev, -1, key_idx, + pairwise, mac_addr); if (params->key_len > sizeof(key->data)) { bphy_err(drvr, "Too long key length (%u)\n", params->key_len); @@ -2553,8 +2555,9 @@ done: } static s32 -brcmf_cfg80211_get_key(struct wiphy *wiphy, struct net_device *ndev, u8 key_idx, - bool pairwise, const u8 *mac_addr, void *cookie, +brcmf_cfg80211_get_key(struct wiphy *wiphy, struct net_device *ndev, + int link_id, u8 key_idx, bool pairwise, + const u8 *mac_addr, void *cookie, void (*callback)(void *cookie, struct key_params *params)) { @@ -2610,7 +2613,8 @@ done: static s32 brcmf_cfg80211_config_default_mgmt_key(struct wiphy *wiphy, - struct net_device *ndev, u8 key_idx) + struct net_device *ndev, int link_id, + u8 key_idx) { struct brcmf_if *ifp = netdev_priv(ndev); @@ -3160,10 +3164,7 @@ static s32 brcmf_update_bss_info(struct brcmf_cfg80211_info *cfg, struct brcmf_if *ifp) { struct brcmf_pub *drvr = cfg->pub; - struct brcmf_bss_info_le *bi; - const struct brcmf_tlv *tim; - size_t ie_len; - u8 *ie; + struct brcmf_bss_info_le *bi = NULL; s32 err = 0; brcmf_dbg(TRACE, "Enter\n"); @@ -3177,29 +3178,8 @@ static s32 brcmf_update_bss_info(struct brcmf_cfg80211_info *cfg, bphy_err(drvr, "Could not get bss info %d\n", err); goto update_bss_info_out; } - bi = (struct brcmf_bss_info_le *)(cfg->extra_buf + 4); err = brcmf_inform_single_bss(cfg, bi); - if (err) - goto update_bss_info_out; - - ie = ((u8 *)bi) + le16_to_cpu(bi->ie_offset); - ie_len = le32_to_cpu(bi->ie_length); - - tim = brcmf_parse_tlvs(ie, ie_len, WLAN_EID_TIM); - if (!tim) { - /* - * active scan was done so we could not get dtim - * information out of probe response. - * so we speficially query dtim information to dongle. - */ - u32 var; - err = brcmf_fil_iovar_int_get(ifp, "dtim_assoc", &var); - if (err) { - bphy_err(drvr, "wl dtim_assoc failed (%d)\n", err); - goto update_bss_info_out; - } - } update_bss_info_out: brcmf_dbg(TRACE, "Exit"); @@ -3984,7 +3964,6 @@ brcmf_update_pmklist(struct brcmf_cfg80211_info *cfg, struct brcmf_if *ifp) struct brcmf_pmk_list_le *pmk_list; int i; u32 npmk; - s32 err; pmk_list = &cfg->pmk_list; npmk = le32_to_cpu(pmk_list->npmk); @@ -3993,10 +3972,8 @@ brcmf_update_pmklist(struct brcmf_cfg80211_info *cfg, struct brcmf_if *ifp) for (i = 0; i < npmk; i++) brcmf_dbg(CONN, "PMK[%d]: %pM\n", i, &pmk_list->pmk[i].bssid); - err = brcmf_fil_iovar_data_set(ifp, "pmkid_info", pmk_list, - sizeof(*pmk_list)); - - return err; + return brcmf_fil_iovar_data_set(ifp, "pmkid_info", pmk_list, + sizeof(*pmk_list)); } static s32 @@ -5042,13 +5019,10 @@ brcmf_cfg80211_change_beacon(struct wiphy *wiphy, struct net_device *ndev, struct cfg80211_beacon_data *info) { struct brcmf_if *ifp = netdev_priv(ndev); - s32 err; brcmf_dbg(TRACE, "Enter\n"); - err = brcmf_config_ap_mgmt_ie(ifp->vif, info); - - return err; + return brcmf_config_ap_mgmt_ie(ifp->vif, info); } static int @@ -6431,6 +6405,7 @@ static void wl_deinit_priv(struct brcmf_cfg80211_info *cfg) cfg->dongle_up = false; /* dongle down */ brcmf_abort_scanning(cfg); brcmf_deinit_priv_mem(cfg); + brcmf_clear_assoc_ies(cfg); } static void init_vif_event(struct brcmf_cfg80211_vif_event *event) @@ -7485,6 +7460,7 @@ static bool brmcf_use_iso3166_ccode_fallback(struct brcmf_pub *drvr) return true; switch (drvr->bus_if->chip) { + case BRCM_CC_43430_CHIP_ID: case BRCM_CC_4345_CHIP_ID: case BRCM_CC_43602_CHIP_ID: return true; diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/chip.c b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/chip.c index 4ec7773b6906..121893bbaa1d 100644 --- a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/chip.c +++ b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/chip.c @@ -641,6 +641,7 @@ static void brcmf_chip_socram_ramsize(struct brcmf_core_priv *sr, u32 *ramsize, *srsize = (32 * 1024); break; case BRCM_CC_43430_CHIP_ID: + case CY_CC_43439_CHIP_ID: /* assume sr for now as we can not check * firmware sr capability at this point. */ @@ -732,6 +733,10 @@ static u32 brcmf_chip_tcm_rambase(struct brcmf_chip_priv *ci) return 0x160000; case CY_CC_43752_CHIP_ID: return 0x170000; + case BRCM_CC_4378_CHIP_ID: + return 0x352000; + case CY_CC_89459_CHIP_ID: + return ((ci->pub.chiprev < 9) ? 0x180000 : 0x160000); default: brcmf_err("unknown chip: %s\n", ci->pub.name); break; @@ -1258,7 +1263,8 @@ brcmf_chip_cm3_set_passive(struct brcmf_chip_priv *chip) brcmf_chip_resetcore(core, 0, 0, 0); /* disable bank #3 remap for this device */ - if (chip->pub.chip == BRCM_CC_43430_CHIP_ID) { + if (chip->pub.chip == BRCM_CC_43430_CHIP_ID || + chip->pub.chip == CY_CC_43439_CHIP_ID) { sr = container_of(core, struct brcmf_core_priv, pub); brcmf_chip_core_write32(sr, SOCRAMREGOFFS(bankidx), 3); brcmf_chip_core_write32(sr, SOCRAMREGOFFS(bankpda), 0); @@ -1416,10 +1422,12 @@ bool brcmf_chip_sr_capable(struct brcmf_chip *pub) reg = chip->ops->read32(chip->ctx, addr); return (reg & pmu_cc3_mask) != 0; case BRCM_CC_43430_CHIP_ID: + case CY_CC_43439_CHIP_ID: addr = CORE_CC_REG(base, sr_control1); reg = chip->ops->read32(chip->ctx, addr); return reg != 0; case CY_CC_4373_CHIP_ID: + case CY_CC_89459_CHIP_ID: /* explicitly check SR engine enable bit */ addr = CORE_CC_REG(base, sr_control0); reg = chip->ops->read32(chip->ctx, addr); diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/common.c b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/common.c index 7485e784be2a..74020fa10065 100644 --- a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/common.c +++ b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/common.c @@ -123,7 +123,6 @@ static int brcmf_c_process_clm_blob(struct brcmf_if *ifp) struct brcmf_bus *bus = drvr->bus_if; struct brcmf_dload_data_le *chunk_buf; const struct firmware *clm = NULL; - u8 clm_name[BRCMF_FW_NAME_LEN]; u32 chunk_len; u32 datalen; u32 cumulative_len; @@ -133,15 +132,8 @@ static int brcmf_c_process_clm_blob(struct brcmf_if *ifp) brcmf_dbg(TRACE, "Enter\n"); - memset(clm_name, 0, sizeof(clm_name)); - err = brcmf_bus_get_fwname(bus, ".clm_blob", clm_name); - if (err) { - bphy_err(drvr, "get CLM blob file name failed (%d)\n", err); - return err; - } - - err = firmware_request_nowarn(&clm, clm_name, bus->dev); - if (err) { + err = brcmf_bus_get_blob(bus, &clm, BRCMF_BLOB_CLM); + if (err || !clm) { brcmf_info("no clm_blob available (err=%d), device may have limited channels available\n", err); return 0; @@ -261,7 +253,7 @@ int brcmf_c_preinit_dcmds(struct brcmf_if *ifp) &revinfo, sizeof(revinfo)); if (err < 0) { bphy_err(drvr, "retrieving revision info failed, %d\n", err); - strlcpy(ri->chipname, "UNKNOWN", sizeof(ri->chipname)); + strscpy(ri->chipname, "UNKNOWN", sizeof(ri->chipname)); } else { ri->vendorid = le32_to_cpu(revinfo.vendorid); ri->deviceid = le32_to_cpu(revinfo.deviceid); @@ -314,7 +306,7 @@ int brcmf_c_preinit_dcmds(struct brcmf_if *ifp) /* locate firmware version number for ethtool */ ptr = strrchr(buf, ' ') + 1; - strlcpy(ifp->drvr->fwver, ptr, sizeof(ifp->drvr->fwver)); + strscpy(ifp->drvr->fwver, ptr, sizeof(ifp->drvr->fwver)); /* Query for 'clmver' to get CLM version info from firmware */ memset(buf, 0, sizeof(buf)); @@ -424,11 +416,11 @@ static void brcmf_mp_attach(void) * if not set then if available use the platform data version. To make * sure it gets initialized at all, always copy the module param version */ - strlcpy(brcmf_mp_global.firmware_path, brcmf_firmware_path, + strscpy(brcmf_mp_global.firmware_path, brcmf_firmware_path, BRCMF_FW_ALTPATH_LEN); if ((brcmfmac_pdata) && (brcmfmac_pdata->fw_alternative_path) && (brcmf_mp_global.firmware_path[0] == '\0')) { - strlcpy(brcmf_mp_global.firmware_path, + strscpy(brcmf_mp_global.firmware_path, brcmfmac_pdata->fw_alternative_path, BRCMF_FW_ALTPATH_LEN); } diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/common.h b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/common.h index 6c5a22a32a96..aa25abffcc7d 100644 --- a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/common.h +++ b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/common.h @@ -53,6 +53,7 @@ struct brcmf_mp_device { struct brcmfmac_pd_cc *country_codes; const char *board_type; unsigned char mac[ETH_ALEN]; + const char *antenna_sku; union { struct brcmfmac_sdio_pd sdio; } bus; diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/core.c b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/core.c index bd164a0821f9..595ae3ae561e 100644 --- a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/core.c +++ b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/core.c @@ -292,6 +292,7 @@ static netdev_tx_t brcmf_netdev_start_xmit(struct sk_buff *skb, struct brcmf_pub *drvr = ifp->drvr; struct ethhdr *eh; int head_delta; + unsigned int tx_bytes = skb->len; brcmf_dbg(DATA, "Enter, bsscfgidx=%d\n", ifp->bsscfgidx); @@ -366,7 +367,7 @@ done: ndev->stats.tx_dropped++; } else { ndev->stats.tx_packets++; - ndev->stats.tx_bytes += skb->len; + ndev->stats.tx_bytes += tx_bytes; } /* Return ok: we always eat the packet */ @@ -561,10 +562,10 @@ static void brcmf_ethtool_get_drvinfo(struct net_device *ndev, if (drvr->revinfo.result == 0) brcmu_dotrev_str(drvr->revinfo.driverrev, drev); - strlcpy(info->driver, KBUILD_MODNAME, sizeof(info->driver)); - strlcpy(info->version, drev, sizeof(info->version)); - strlcpy(info->fw_version, drvr->fwver, sizeof(info->fw_version)); - strlcpy(info->bus_info, dev_name(drvr->bus_if->dev), + strscpy(info->driver, KBUILD_MODNAME, sizeof(info->driver)); + strscpy(info->version, drev, sizeof(info->version)); + strscpy(info->fw_version, drvr->fwver, sizeof(info->fw_version)); + strscpy(info->bus_info, dev_name(drvr->bus_if->dev), sizeof(info->bus_info)); } @@ -1480,8 +1481,10 @@ int brcmf_netdev_wait_pend8021x(struct brcmf_if *ifp) !brcmf_get_pend_8021x_cnt(ifp), MAX_WAIT_FOR_8021X_TX); - if (!err) + if (!err) { bphy_err(drvr, "Timed out waiting for no pending 802.1x packets\n"); + atomic_set(&ifp->pend_8021x_cnt, 0); + } return !err; } diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/dmi.c b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/dmi.c index 0af452dca766..86ff174936a9 100644 --- a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/dmi.c +++ b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/dmi.c @@ -24,6 +24,13 @@ static const struct brcmf_dmi_data acepc_t8_data = { BRCM_CC_4345_CHIP_ID, 6, "acepc-t8" }; +/* The Chuwi Hi8 Pro uses the same Ampak AP6212 module as the Chuwi Vi8 Plus + * and the nvram for the Vi8 Plus is already in linux-firmware, so use that. + */ +static const struct brcmf_dmi_data chuwi_hi8_pro_data = { + BRCM_CC_43430_CHIP_ID, 0, "ilife-S806" +}; + static const struct brcmf_dmi_data gpd_win_pocket_data = { BRCM_CC_4356_CHIP_ID, 2, "gpd-win-pocket" }; @@ -76,6 +83,17 @@ static const struct dmi_system_id dmi_platform_data[] = { .driver_data = (void *)&acepc_t8_data, }, { + /* Chuwi Hi8 Pro with D2D3_Hi8Pro.233 BIOS */ + .matches = { + DMI_EXACT_MATCH(DMI_BOARD_VENDOR, "Hampoo"), + DMI_EXACT_MATCH(DMI_BOARD_NAME, "Cherry Trail CR"), + DMI_EXACT_MATCH(DMI_PRODUCT_SKU, "MRD"), + /* Above strings are too generic, also match on BIOS date */ + DMI_MATCH(DMI_BIOS_DATE, "05/10/2016"), + }, + .driver_data = (void *)&chuwi_hi8_pro_data, + }, + { /* Cyberbook T116 rugged tablet */ .matches = { DMI_EXACT_MATCH(DMI_BOARD_VENDOR, "Default string"), diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/feature.c b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/feature.c index d2ac844e1e9f..2c2f3e026c13 100644 --- a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/feature.c +++ b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/feature.c @@ -249,7 +249,8 @@ void brcmf_feat_attach(struct brcmf_pub *drvr) memset(&gscan_cfg, 0, sizeof(gscan_cfg)); if (drvr->bus_if->chip != BRCM_CC_43430_CHIP_ID && drvr->bus_if->chip != BRCM_CC_4345_CHIP_ID && - drvr->bus_if->chip != BRCM_CC_43454_CHIP_ID) + drvr->bus_if->chip != BRCM_CC_43454_CHIP_ID && + drvr->bus_if->chip != CY_CC_43439_CHIP_ID) brcmf_feat_iovar_data_set(ifp, BRCMF_FEAT_GSCAN, "pfn_gscan_cfg", &gscan_cfg, sizeof(gscan_cfg)); diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/firmware.c b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/firmware.c index b8379e4034a4..f2207793f6e2 100644 --- a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/firmware.c +++ b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/firmware.c @@ -21,6 +21,8 @@ #define BRCMF_FW_NVRAM_DEVPATH_LEN 19 /* devpath0=pcie/1/4/ */ #define BRCMF_FW_NVRAM_PCIEDEV_LEN 10 /* pcie/1/4/ + \0 */ #define BRCMF_FW_DEFAULT_BOARDREV "boardrev=0xff" +#define BRCMF_FW_MACADDR_FMT "macaddr=%pM" +#define BRCMF_FW_MACADDR_LEN (7 + ETH_ALEN * 3) enum nvram_parser_state { IDLE, @@ -44,6 +46,7 @@ enum nvram_parser_state { * @multi_dev_v1: detect pcie multi device v1 (compressed). * @multi_dev_v2: detect pcie multi device v2. * @boardrev_found: nvram contains boardrev information. + * @strip_mac: strip the MAC address. */ struct nvram_parser { enum nvram_parser_state state; @@ -57,6 +60,7 @@ struct nvram_parser { bool multi_dev_v1; bool multi_dev_v2; bool boardrev_found; + bool strip_mac; }; /* @@ -121,6 +125,10 @@ static enum nvram_parser_state brcmf_nvram_handle_key(struct nvram_parser *nvp) nvp->multi_dev_v2 = true; if (strncmp(&nvp->data[nvp->entry], "boardrev", 8) == 0) nvp->boardrev_found = true; + /* strip macaddr if platform MAC overrides */ + if (nvp->strip_mac && + strncmp(&nvp->data[nvp->entry], "macaddr", 7) == 0) + st = COMMENT; } else if (!is_nvram_char(c) || c == ' ') { brcmf_dbg(INFO, "warning: ln=%d:col=%d: '=' expected, skip invalid key entry\n", nvp->line, nvp->column); @@ -209,6 +217,7 @@ static int brcmf_init_nvram_parser(struct nvram_parser *nvp, size = data_len; /* Add space for properties we may add */ size += strlen(BRCMF_FW_DEFAULT_BOARDREV) + 1; + size += BRCMF_FW_MACADDR_LEN + 1; /* Alloc for extra 0 byte + roundup by 4 + length field */ size += 1 + 3 + sizeof(u32); nvp->nvram = kzalloc(size, GFP_KERNEL); @@ -368,22 +377,37 @@ static void brcmf_fw_add_defaults(struct nvram_parser *nvp) nvp->nvram_len++; } +static void brcmf_fw_add_macaddr(struct nvram_parser *nvp, u8 *mac) +{ + int len; + + len = scnprintf(&nvp->nvram[nvp->nvram_len], BRCMF_FW_MACADDR_LEN + 1, + BRCMF_FW_MACADDR_FMT, mac); + WARN_ON(len != BRCMF_FW_MACADDR_LEN); + nvp->nvram_len += len + 1; +} + /* brcmf_nvram_strip :Takes a buffer of "<var>=<value>\n" lines read from a fil * and ending in a NUL. Removes carriage returns, empty lines, comment lines, * and converts newlines to NULs. Shortens buffer as needed and pads with NULs. * End of buffer is completed with token identifying length of buffer. */ static void *brcmf_fw_nvram_strip(const u8 *data, size_t data_len, - u32 *new_length, u16 domain_nr, u16 bus_nr) + u32 *new_length, u16 domain_nr, u16 bus_nr, + struct device *dev) { struct nvram_parser nvp; u32 pad; u32 token; __le32 token_le; + u8 mac[ETH_ALEN]; if (brcmf_init_nvram_parser(&nvp, data, data_len) < 0) return NULL; + if (eth_platform_get_mac_address(dev, mac) == 0) + nvp.strip_mac = true; + while (nvp.pos < data_len) { nvp.state = nv_parser_states[nvp.state](&nvp); if (nvp.state == END) @@ -404,6 +428,9 @@ static void *brcmf_fw_nvram_strip(const u8 *data, size_t data_len, brcmf_fw_add_defaults(&nvp); + if (nvp.strip_mac) + brcmf_fw_add_macaddr(&nvp, mac); + pad = nvp.nvram_len; *new_length = roundup(nvp.nvram_len + 1, 4); while (pad != *new_length) { @@ -430,6 +457,7 @@ struct brcmf_fw { struct device *dev; struct brcmf_fw_request *req; u32 curpos; + unsigned int board_index; void (*done)(struct device *dev, int err, struct brcmf_fw_request *req); }; @@ -537,7 +565,8 @@ static int brcmf_fw_request_nvram_done(const struct firmware *fw, void *ctx) if (data) nvram = brcmf_fw_nvram_strip(data, data_len, &nvram_length, fwctx->req->domain_nr, - fwctx->req->bus_nr); + fwctx->req->bus_nr, + fwctx->dev); if (free_bcm47xx_nvram) bcm47xx_nvram_release_contents(data); @@ -587,39 +616,50 @@ static int brcmf_fw_complete_request(const struct firmware *fw, static char *brcm_alt_fw_path(const char *path, const char *board_type) { - char alt_path[BRCMF_FW_NAME_LEN]; - char suffix[5]; + char base[BRCMF_FW_NAME_LEN]; + const char *suffix; + char *ret; + + if (!board_type) + return NULL; - strscpy(alt_path, path, BRCMF_FW_NAME_LEN); - /* At least one character + suffix */ - if (strlen(alt_path) < 5) + suffix = strrchr(path, '.'); + if (!suffix || suffix == path) return NULL; - /* strip .txt or .bin at the end */ - strscpy(suffix, alt_path + strlen(alt_path) - 4, 5); - alt_path[strlen(alt_path) - 4] = 0; - strlcat(alt_path, ".", BRCMF_FW_NAME_LEN); - strlcat(alt_path, board_type, BRCMF_FW_NAME_LEN); - strlcat(alt_path, suffix, BRCMF_FW_NAME_LEN); + /* strip extension at the end */ + strscpy(base, path, BRCMF_FW_NAME_LEN); + base[suffix - path] = 0; - return kstrdup(alt_path, GFP_KERNEL); + ret = kasprintf(GFP_KERNEL, "%s.%s%s", base, board_type, suffix); + if (!ret) + brcmf_err("out of memory allocating firmware path for '%s'\n", + path); + + brcmf_dbg(TRACE, "FW alt path: %s\n", ret); + + return ret; } static int brcmf_fw_request_firmware(const struct firmware **fw, struct brcmf_fw *fwctx) { struct brcmf_fw_item *cur = &fwctx->req->items[fwctx->curpos]; + unsigned int i; int ret; - /* Files can be board-specific, first try a board-specific path */ - if (cur->type == BRCMF_FW_TYPE_NVRAM && fwctx->req->board_type) { + /* Files can be board-specific, first try board-specific paths */ + for (i = 0; i < ARRAY_SIZE(fwctx->req->board_types); i++) { char *alt_path; - alt_path = brcm_alt_fw_path(cur->path, fwctx->req->board_type); + if (!fwctx->req->board_types[i]) + goto fallback; + alt_path = brcm_alt_fw_path(cur->path, + fwctx->req->board_types[i]); if (!alt_path) goto fallback; - ret = request_firmware(fw, alt_path, fwctx->dev); + ret = firmware_request_nowarn(fw, alt_path, fwctx->dev); kfree(alt_path); if (ret == 0) return ret; @@ -653,15 +693,40 @@ static void brcmf_fw_request_done_alt_path(const struct firmware *fw, void *ctx) { struct brcmf_fw *fwctx = ctx; struct brcmf_fw_item *first = &fwctx->req->items[0]; + const char *board_type, *alt_path; int ret = 0; - /* Fall back to canonical path if board firmware not found */ - if (!fw) - ret = request_firmware_nowait(THIS_MODULE, true, first->path, + if (fw) { + brcmf_fw_request_done(fw, ctx); + return; + } + + /* Try next board firmware */ + if (fwctx->board_index < ARRAY_SIZE(fwctx->req->board_types)) { + board_type = fwctx->req->board_types[fwctx->board_index++]; + if (!board_type) + goto fallback; + alt_path = brcm_alt_fw_path(first->path, board_type); + if (!alt_path) + goto fallback; + + ret = request_firmware_nowait(THIS_MODULE, true, alt_path, fwctx->dev, GFP_KERNEL, fwctx, - brcmf_fw_request_done); + brcmf_fw_request_done_alt_path); + kfree(alt_path); + + if (ret < 0) + brcmf_fw_request_done(fw, ctx); + return; + } - if (fw || ret < 0) +fallback: + /* Fall back to canonical path if board firmware not found */ + ret = request_firmware_nowait(THIS_MODULE, true, first->path, + fwctx->dev, GFP_KERNEL, fwctx, + brcmf_fw_request_done); + + if (ret < 0) brcmf_fw_request_done(fw, ctx); } @@ -705,10 +770,11 @@ int brcmf_fw_get_firmwares(struct device *dev, struct brcmf_fw_request *req, fwctx->done = fw_cb; /* First try alternative board-specific path if any */ - if (fwctx->req->board_type) + if (fwctx->req->board_types[0]) alt_path = brcm_alt_fw_path(first->path, - fwctx->req->board_type); + fwctx->req->board_types[0]); if (alt_path) { + fwctx->board_index++; ret = request_firmware_nowait(THIS_MODULE, true, alt_path, fwctx->dev, GFP_KERNEL, fwctx, brcmf_fw_request_done_alt_path); @@ -769,7 +835,7 @@ brcmf_fw_alloc_request(u32 chip, u32 chiprev, fwnames[j].path[0] = '\0'; /* check if firmware path is provided by module parameter */ if (brcmf_mp_global.firmware_path[0] != '\0') { - strlcpy(fwnames[j].path, mp_path, + strscpy(fwnames[j].path, mp_path, BRCMF_FW_NAME_LEN); if (end != '/') { diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/firmware.h b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/firmware.h index e290dec9c53d..1266cbaee072 100644 --- a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/firmware.h +++ b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/firmware.h @@ -11,6 +11,8 @@ #define BRCMF_FW_DEFAULT_PATH "brcm/" +#define BRCMF_FW_MAX_BOARD_TYPES 8 + /** * struct brcmf_firmware_mapping - Used to map chipid/revmask to firmware * filename and nvram filename. Each bus type implementation should create @@ -66,7 +68,7 @@ struct brcmf_fw_request { u16 domain_nr; u16 bus_nr; u32 n_items; - const char *board_type; + const char *board_types[BRCMF_FW_MAX_BOARD_TYPES]; struct brcmf_fw_item items[]; }; diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/flowring.c b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/flowring.c index 096f6b969dd8..e1127d7e086d 100644 --- a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/flowring.c +++ b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/flowring.c @@ -419,7 +419,6 @@ void brcmf_flowring_configure_addr_mode(struct brcmf_flowring *flow, int ifidx, flowid = flow->hash[i].flowid; if (flow->rings[flowid]->status != RING_OPEN) continue; - flow->rings[flowid]->status = RING_CLOSING; brcmf_msgbuf_delete_flowring(drvr, flowid); } } @@ -458,10 +457,8 @@ void brcmf_flowring_delete_peer(struct brcmf_flowring *flow, int ifidx, if ((sta || (memcmp(hash[i].mac, peer, ETH_ALEN) == 0)) && (hash[i].ifidx == ifidx)) { flowid = flow->hash[i].flowid; - if (flow->rings[flowid]->status == RING_OPEN) { - flow->rings[flowid]->status = RING_CLOSING; + if (flow->rings[flowid]->status == RING_OPEN) brcmf_msgbuf_delete_flowring(drvr, flowid); - } } } diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/fwil_types.h b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/fwil_types.h index c87b829adb0d..f518e025d6e4 100644 --- a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/fwil_types.h +++ b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/fwil_types.h @@ -135,7 +135,7 @@ /* Link Down indication in WoWL mode: */ #define BRCMF_WOWL_LINKDOWN (1 << 31) -#define BRCMF_WOWL_MAXPATTERNS 8 +#define BRCMF_WOWL_MAXPATTERNS 16 #define BRCMF_WOWL_MAXPATTERNSIZE 128 #define BRCMF_COUNTRY_BUF_SZ 4 diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/fwsignal.c b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/fwsignal.c index d58525ebe618..36af81975855 100644 --- a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/fwsignal.c +++ b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/fwsignal.c @@ -688,7 +688,7 @@ static void brcmf_fws_macdesc_set_name(struct brcmf_fws_info *fws, struct brcmf_fws_mac_descriptor *desc) { if (desc == &fws->desc.other) - strlcpy(desc->name, "MAC-OTHER", sizeof(desc->name)); + strscpy(desc->name, "MAC-OTHER", sizeof(desc->name)); else if (desc->mac_handle) scnprintf(desc->name, sizeof(desc->name), "MAC-%d:%d", desc->mac_handle, desc->interface_id); @@ -2475,7 +2475,8 @@ bool brcmf_fws_fc_active(struct brcmf_fws_info *fws) return fws->fcmode != BRCMF_FWS_FCMODE_NONE; } -void brcmf_fws_bustxfail(struct brcmf_fws_info *fws, struct sk_buff *skb) +void brcmf_fws_bustxcomplete(struct brcmf_fws_info *fws, struct sk_buff *skb, + bool success) { u32 hslot; @@ -2483,11 +2484,14 @@ void brcmf_fws_bustxfail(struct brcmf_fws_info *fws, struct sk_buff *skb) brcmu_pkt_buf_free_skb(skb); return; } - brcmf_fws_lock(fws); - hslot = brcmf_skb_htod_tag_get_field(skb, HSLOT); - brcmf_fws_txs_process(fws, BRCMF_FWS_TXSTATUS_HOST_TOSSED, hslot, 0, 0, - 1); - brcmf_fws_unlock(fws); + + if (!success) { + brcmf_fws_lock(fws); + hslot = brcmf_skb_htod_tag_get_field(skb, HSLOT); + brcmf_fws_txs_process(fws, BRCMF_FWS_TXSTATUS_HOST_TOSSED, hslot, + 0, 0, 1); + brcmf_fws_unlock(fws); + } } void brcmf_fws_bus_blocked(struct brcmf_pub *drvr, bool flow_blocked) diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/fwsignal.h b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/fwsignal.h index b16a9d1c0508..f9c36cd8f1de 100644 --- a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/fwsignal.h +++ b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/fwsignal.h @@ -40,7 +40,8 @@ int brcmf_fws_process_skb(struct brcmf_if *ifp, struct sk_buff *skb); void brcmf_fws_reset_interface(struct brcmf_if *ifp); void brcmf_fws_add_interface(struct brcmf_if *ifp); void brcmf_fws_del_interface(struct brcmf_if *ifp); -void brcmf_fws_bustxfail(struct brcmf_fws_info *fws, struct sk_buff *skb); +void brcmf_fws_bustxcomplete(struct brcmf_fws_info *fws, struct sk_buff *skb, + bool success); void brcmf_fws_bus_blocked(struct brcmf_pub *drvr, bool flow_blocked); void brcmf_fws_rxreorder(struct brcmf_if *ifp, struct sk_buff *skb); diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/msgbuf.c b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/msgbuf.c index b2d0f7570aa9..cec53f934940 100644 --- a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/msgbuf.c +++ b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/msgbuf.c @@ -71,6 +71,7 @@ #define BRCMF_MSGBUF_TRICKLE_TXWORKER_THRS 32 #define BRCMF_MSGBUF_UPDATE_RX_PTR_THRS 48 +#define BRCMF_MAX_TXSTATUS_WAIT_RETRIES 10 struct msgbuf_common_hdr { u8 msgtype; @@ -806,8 +807,12 @@ static int brcmf_msgbuf_tx_queue_data(struct brcmf_pub *drvr, int ifidx, flowid = brcmf_flowring_lookup(flow, eh->h_dest, skb->priority, ifidx); if (flowid == BRCMF_FLOWRING_INVALID_ID) { flowid = brcmf_msgbuf_flowring_create(msgbuf, ifidx, skb); - if (flowid == BRCMF_FLOWRING_INVALID_ID) + if (flowid == BRCMF_FLOWRING_INVALID_ID) { return -ENOMEM; + } else { + brcmf_flowring_enqueue(flow, flowid, skb); + return 0; + } } queue_count = brcmf_flowring_enqueue(flow, flowid, skb); force = ((queue_count % BRCMF_MSGBUF_TRICKLE_TXWORKER_THRS) == 0); @@ -1395,9 +1400,27 @@ void brcmf_msgbuf_delete_flowring(struct brcmf_pub *drvr, u16 flowid) struct brcmf_msgbuf *msgbuf = (struct brcmf_msgbuf *)drvr->proto->pd; struct msgbuf_tx_flowring_delete_req *delete; struct brcmf_commonring *commonring; + struct brcmf_commonring *commonring_del = msgbuf->flowrings[flowid]; + struct brcmf_flowring *flow = msgbuf->flow; void *ret_ptr; u8 ifidx; int err; + int retry = BRCMF_MAX_TXSTATUS_WAIT_RETRIES; + + /* make sure it is not in txflow */ + brcmf_commonring_lock(commonring_del); + flow->rings[flowid]->status = RING_CLOSING; + brcmf_commonring_unlock(commonring_del); + + /* wait for commonring txflow finished */ + while (retry && atomic_read(&commonring_del->outstanding_tx)) { + usleep_range(5000, 10000); + retry--; + } + if (!retry) { + brcmf_err("timed out waiting for txstatus\n"); + atomic_set(&commonring_del->outstanding_tx, 0); + } /* no need to submit if firmware can not be reached */ if (drvr->bus_if->state != BRCMF_BUS_UP) { diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/msgbuf.h b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/msgbuf.h index 2e322edbb907..6a849f4a94dd 100644 --- a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/msgbuf.h +++ b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/msgbuf.h @@ -8,10 +8,10 @@ #ifdef CONFIG_BRCMFMAC_PROTO_MSGBUF #define BRCMF_H2D_MSGRING_CONTROL_SUBMIT_MAX_ITEM 64 -#define BRCMF_H2D_MSGRING_RXPOST_SUBMIT_MAX_ITEM 512 +#define BRCMF_H2D_MSGRING_RXPOST_SUBMIT_MAX_ITEM 1024 #define BRCMF_D2H_MSGRING_CONTROL_COMPLETE_MAX_ITEM 64 #define BRCMF_D2H_MSGRING_TX_COMPLETE_MAX_ITEM 1024 -#define BRCMF_D2H_MSGRING_RX_COMPLETE_MAX_ITEM 512 +#define BRCMF_D2H_MSGRING_RX_COMPLETE_MAX_ITEM 1024 #define BRCMF_H2D_TXFLOWRING_MAX_ITEM 512 #define BRCMF_H2D_MSGRING_CONTROL_SUBMIT_ITEMSIZE 40 diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/of.c b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/of.c index 79388d49c256..a83699de01ec 100644 --- a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/of.c +++ b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/of.c @@ -70,14 +70,24 @@ void brcmf_of_probe(struct device *dev, enum brcmf_bus_type bus_type, { struct brcmfmac_sdio_pd *sdio = &settings->bus.sdio; struct device_node *root, *np = dev->of_node; + const char *prop; int irq; int err; u32 irqf; u32 val; + /* Apple ARM64 platforms have their own idea of board type, passed in + * via the device tree. They also have an antenna SKU parameter + */ + if (!of_property_read_string(np, "brcm,board-type", &prop)) + settings->board_type = prop; + + if (!of_property_read_string(np, "apple,antenna-sku", &prop)) + settings->antenna_sku = prop; + /* Set board-type to the first string of the machine compatible prop */ root = of_find_node_by_path("/"); - if (root) { + if (root && !settings->board_type) { char *board_type; const char *tmp; diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/pcie.c b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/pcie.c index 97f0f13dfe50..80083f9ea311 100644 --- a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/pcie.c +++ b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/pcie.c @@ -59,6 +59,8 @@ BRCMF_FW_DEF(4365C, "brcmfmac4365c-pcie"); BRCMF_FW_DEF(4366B, "brcmfmac4366b-pcie"); BRCMF_FW_DEF(4366C, "brcmfmac4366c-pcie"); BRCMF_FW_DEF(4371, "brcmfmac4371-pcie"); +BRCMF_FW_CLM_DEF(4378B1, "brcmfmac4378b1-pcie"); +BRCMF_FW_DEF(4355, "brcmfmac89459-pcie"); /* firmware config files */ MODULE_FIRMWARE(BRCMF_FW_DEFAULT_PATH "brcmfmac*-pcie.txt"); @@ -66,6 +68,7 @@ MODULE_FIRMWARE(BRCMF_FW_DEFAULT_PATH "brcmfmac*-pcie.*.txt"); /* per-board firmware binaries */ MODULE_FIRMWARE(BRCMF_FW_DEFAULT_PATH "brcmfmac*-pcie.*.bin"); +MODULE_FIRMWARE(BRCMF_FW_DEFAULT_PATH "brcmfmac*-pcie.*.clm_blob"); static const struct brcmf_firmware_mapping brcmf_pcie_fwnames[] = { BRCMF_FW_ENTRY(BRCM_CC_43602_CHIP_ID, 0xFFFFFFFF, 43602), @@ -87,6 +90,8 @@ static const struct brcmf_firmware_mapping brcmf_pcie_fwnames[] = { BRCMF_FW_ENTRY(BRCM_CC_43664_CHIP_ID, 0xFFFFFFF0, 4366C), BRCMF_FW_ENTRY(BRCM_CC_43666_CHIP_ID, 0xFFFFFFF0, 4366C), BRCMF_FW_ENTRY(BRCM_CC_4371_CHIP_ID, 0xFFFFFFFF, 4371), + BRCMF_FW_ENTRY(BRCM_CC_4378_CHIP_ID, 0xFFFFFFFF, 4378B1), /* revision ID 3 */ + BRCMF_FW_ENTRY(CY_CC_89459_CHIP_ID, 0xFFFFFFFF, 4355), }; #define BRCMF_PCIE_FW_UP_TIMEOUT 5000 /* msec */ @@ -118,6 +123,12 @@ static const struct brcmf_firmware_mapping brcmf_pcie_fwnames[] = { #define BRCMF_PCIE_PCIE2REG_H2D_MAILBOX_0 0x140 #define BRCMF_PCIE_PCIE2REG_H2D_MAILBOX_1 0x144 +#define BRCMF_PCIE_64_PCIE2REG_INTMASK 0xC14 +#define BRCMF_PCIE_64_PCIE2REG_MAILBOXINT 0xC30 +#define BRCMF_PCIE_64_PCIE2REG_MAILBOXMASK 0xC34 +#define BRCMF_PCIE_64_PCIE2REG_H2D_MAILBOX_0 0xA20 +#define BRCMF_PCIE_64_PCIE2REG_H2D_MAILBOX_1 0xA24 + #define BRCMF_PCIE2_INTA 0x01 #define BRCMF_PCIE2_INTB 0x02 @@ -137,6 +148,8 @@ static const struct brcmf_firmware_mapping brcmf_pcie_fwnames[] = { #define BRCMF_PCIE_MB_INT_D2H3_DB0 0x400000 #define BRCMF_PCIE_MB_INT_D2H3_DB1 0x800000 +#define BRCMF_PCIE_MB_INT_FN0 (BRCMF_PCIE_MB_INT_FN0_0 | \ + BRCMF_PCIE_MB_INT_FN0_1) #define BRCMF_PCIE_MB_INT_D2H_DB (BRCMF_PCIE_MB_INT_D2H0_DB0 | \ BRCMF_PCIE_MB_INT_D2H0_DB1 | \ BRCMF_PCIE_MB_INT_D2H1_DB0 | \ @@ -146,6 +159,40 @@ static const struct brcmf_firmware_mapping brcmf_pcie_fwnames[] = { BRCMF_PCIE_MB_INT_D2H3_DB0 | \ BRCMF_PCIE_MB_INT_D2H3_DB1) +#define BRCMF_PCIE_64_MB_INT_D2H0_DB0 0x1 +#define BRCMF_PCIE_64_MB_INT_D2H0_DB1 0x2 +#define BRCMF_PCIE_64_MB_INT_D2H1_DB0 0x4 +#define BRCMF_PCIE_64_MB_INT_D2H1_DB1 0x8 +#define BRCMF_PCIE_64_MB_INT_D2H2_DB0 0x10 +#define BRCMF_PCIE_64_MB_INT_D2H2_DB1 0x20 +#define BRCMF_PCIE_64_MB_INT_D2H3_DB0 0x40 +#define BRCMF_PCIE_64_MB_INT_D2H3_DB1 0x80 +#define BRCMF_PCIE_64_MB_INT_D2H4_DB0 0x100 +#define BRCMF_PCIE_64_MB_INT_D2H4_DB1 0x200 +#define BRCMF_PCIE_64_MB_INT_D2H5_DB0 0x400 +#define BRCMF_PCIE_64_MB_INT_D2H5_DB1 0x800 +#define BRCMF_PCIE_64_MB_INT_D2H6_DB0 0x1000 +#define BRCMF_PCIE_64_MB_INT_D2H6_DB1 0x2000 +#define BRCMF_PCIE_64_MB_INT_D2H7_DB0 0x4000 +#define BRCMF_PCIE_64_MB_INT_D2H7_DB1 0x8000 + +#define BRCMF_PCIE_64_MB_INT_D2H_DB (BRCMF_PCIE_64_MB_INT_D2H0_DB0 | \ + BRCMF_PCIE_64_MB_INT_D2H0_DB1 | \ + BRCMF_PCIE_64_MB_INT_D2H1_DB0 | \ + BRCMF_PCIE_64_MB_INT_D2H1_DB1 | \ + BRCMF_PCIE_64_MB_INT_D2H2_DB0 | \ + BRCMF_PCIE_64_MB_INT_D2H2_DB1 | \ + BRCMF_PCIE_64_MB_INT_D2H3_DB0 | \ + BRCMF_PCIE_64_MB_INT_D2H3_DB1 | \ + BRCMF_PCIE_64_MB_INT_D2H4_DB0 | \ + BRCMF_PCIE_64_MB_INT_D2H4_DB1 | \ + BRCMF_PCIE_64_MB_INT_D2H5_DB0 | \ + BRCMF_PCIE_64_MB_INT_D2H5_DB1 | \ + BRCMF_PCIE_64_MB_INT_D2H6_DB0 | \ + BRCMF_PCIE_64_MB_INT_D2H6_DB1 | \ + BRCMF_PCIE_64_MB_INT_D2H7_DB0 | \ + BRCMF_PCIE_64_MB_INT_D2H7_DB1) + #define BRCMF_PCIE_SHARED_VERSION_7 7 #define BRCMF_PCIE_MIN_SHARED_VERSION 5 #define BRCMF_PCIE_MAX_SHARED_VERSION BRCMF_PCIE_SHARED_VERSION_7 @@ -255,12 +302,24 @@ struct brcmf_pcie_core_info { u32 wrapbase; }; +#define BRCMF_OTP_MAX_PARAM_LEN 16 + +struct brcmf_otp_params { + char module[BRCMF_OTP_MAX_PARAM_LEN]; + char vendor[BRCMF_OTP_MAX_PARAM_LEN]; + char version[BRCMF_OTP_MAX_PARAM_LEN]; + bool valid; +}; + struct brcmf_pciedev_info { enum brcmf_pcie_state state; bool in_irq; struct pci_dev *pdev; char fw_name[BRCMF_FW_NAME_LEN]; char nvram_name[BRCMF_FW_NAME_LEN]; + char clm_name[BRCMF_FW_NAME_LEN]; + const struct firmware *clm_fw; + const struct brcmf_pcie_reginfo *reginfo; void __iomem *regs; void __iomem *tcm; u32 ram_base; @@ -280,6 +339,7 @@ struct brcmf_pciedev_info { void (*write_ptr)(struct brcmf_pciedev_info *devinfo, u32 mem_offset, u16 value); struct brcmf_mp_device *settings; + struct brcmf_otp_params otp; }; struct brcmf_pcie_ringbuf { @@ -346,11 +406,49 @@ static const u32 brcmf_ring_itemsize[BRCMF_NROF_COMMON_MSGRINGS] = { BRCMF_D2H_MSGRING_RX_COMPLETE_ITEMSIZE }; +struct brcmf_pcie_reginfo { + u32 intmask; + u32 mailboxint; + u32 mailboxmask; + u32 h2d_mailbox_0; + u32 h2d_mailbox_1; + u32 int_d2h_db; + u32 int_fn0; +}; + +static const struct brcmf_pcie_reginfo brcmf_reginfo_default = { + .intmask = BRCMF_PCIE_PCIE2REG_INTMASK, + .mailboxint = BRCMF_PCIE_PCIE2REG_MAILBOXINT, + .mailboxmask = BRCMF_PCIE_PCIE2REG_MAILBOXMASK, + .h2d_mailbox_0 = BRCMF_PCIE_PCIE2REG_H2D_MAILBOX_0, + .h2d_mailbox_1 = BRCMF_PCIE_PCIE2REG_H2D_MAILBOX_1, + .int_d2h_db = BRCMF_PCIE_MB_INT_D2H_DB, + .int_fn0 = BRCMF_PCIE_MB_INT_FN0, +}; + +static const struct brcmf_pcie_reginfo brcmf_reginfo_64 = { + .intmask = BRCMF_PCIE_64_PCIE2REG_INTMASK, + .mailboxint = BRCMF_PCIE_64_PCIE2REG_MAILBOXINT, + .mailboxmask = BRCMF_PCIE_64_PCIE2REG_MAILBOXMASK, + .h2d_mailbox_0 = BRCMF_PCIE_64_PCIE2REG_H2D_MAILBOX_0, + .h2d_mailbox_1 = BRCMF_PCIE_64_PCIE2REG_H2D_MAILBOX_1, + .int_d2h_db = BRCMF_PCIE_64_MB_INT_D2H_DB, + .int_fn0 = 0, +}; + static void brcmf_pcie_setup(struct device *dev, int ret, struct brcmf_fw_request *fwreq); static struct brcmf_fw_request * brcmf_pcie_prepare_fw_request(struct brcmf_pciedev_info *devinfo); +static u16 +brcmf_pcie_read_reg16(struct brcmf_pciedev_info *devinfo, u32 reg_offset) +{ + void __iomem *address = devinfo->regs + reg_offset; + + return ioread16(address); +} + static u32 brcmf_pcie_read_reg32(struct brcmf_pciedev_info *devinfo, u32 reg_offset) { @@ -496,6 +594,8 @@ brcmf_pcie_copy_dev_tomem(struct brcmf_pciedev_info *devinfo, u32 mem_offset, } +#define READCC32(devinfo, reg) brcmf_pcie_read_reg32(devinfo, \ + CHIPCREGOFFS(reg)) #define WRITECC32(devinfo, reg, value) brcmf_pcie_write_reg32(devinfo, \ CHIPCREGOFFS(reg), value) @@ -779,30 +879,29 @@ static void brcmf_pcie_bus_console_read(struct brcmf_pciedev_info *devinfo, static void brcmf_pcie_intr_disable(struct brcmf_pciedev_info *devinfo) { - brcmf_pcie_write_reg32(devinfo, BRCMF_PCIE_PCIE2REG_MAILBOXMASK, 0); + brcmf_pcie_write_reg32(devinfo, devinfo->reginfo->mailboxmask, 0); } static void brcmf_pcie_intr_enable(struct brcmf_pciedev_info *devinfo) { - brcmf_pcie_write_reg32(devinfo, BRCMF_PCIE_PCIE2REG_MAILBOXMASK, - BRCMF_PCIE_MB_INT_D2H_DB | - BRCMF_PCIE_MB_INT_FN0_0 | - BRCMF_PCIE_MB_INT_FN0_1); + brcmf_pcie_write_reg32(devinfo, devinfo->reginfo->mailboxmask, + devinfo->reginfo->int_d2h_db | + devinfo->reginfo->int_fn0); } static void brcmf_pcie_hostready(struct brcmf_pciedev_info *devinfo) { if (devinfo->shared.flags & BRCMF_PCIE_SHARED_HOSTRDY_DB1) brcmf_pcie_write_reg32(devinfo, - BRCMF_PCIE_PCIE2REG_H2D_MAILBOX_1, 1); + devinfo->reginfo->h2d_mailbox_1, 1); } static irqreturn_t brcmf_pcie_quick_check_isr(int irq, void *arg) { struct brcmf_pciedev_info *devinfo = (struct brcmf_pciedev_info *)arg; - if (brcmf_pcie_read_reg32(devinfo, BRCMF_PCIE_PCIE2REG_MAILBOXINT)) { + if (brcmf_pcie_read_reg32(devinfo, devinfo->reginfo->mailboxint)) { brcmf_pcie_intr_disable(devinfo); brcmf_dbg(PCIE, "Enter\n"); return IRQ_WAKE_THREAD; @@ -817,15 +916,14 @@ static irqreturn_t brcmf_pcie_isr_thread(int irq, void *arg) u32 status; devinfo->in_irq = true; - status = brcmf_pcie_read_reg32(devinfo, BRCMF_PCIE_PCIE2REG_MAILBOXINT); + status = brcmf_pcie_read_reg32(devinfo, devinfo->reginfo->mailboxint); brcmf_dbg(PCIE, "Enter %x\n", status); if (status) { - brcmf_pcie_write_reg32(devinfo, BRCMF_PCIE_PCIE2REG_MAILBOXINT, + brcmf_pcie_write_reg32(devinfo, devinfo->reginfo->mailboxint, status); - if (status & (BRCMF_PCIE_MB_INT_FN0_0 | - BRCMF_PCIE_MB_INT_FN0_1)) + if (status & devinfo->reginfo->int_fn0) brcmf_pcie_handle_mb_data(devinfo); - if (status & BRCMF_PCIE_MB_INT_D2H_DB) { + if (status & devinfo->reginfo->int_d2h_db) { if (devinfo->state == BRCMFMAC_PCIE_STATE_UP) brcmf_proto_msgbuf_rx_trigger( &devinfo->pdev->dev); @@ -884,8 +982,8 @@ static void brcmf_pcie_release_irq(struct brcmf_pciedev_info *devinfo) if (devinfo->in_irq) brcmf_err(bus, "Still in IRQ (processing) !!!\n"); - status = brcmf_pcie_read_reg32(devinfo, BRCMF_PCIE_PCIE2REG_MAILBOXINT); - brcmf_pcie_write_reg32(devinfo, BRCMF_PCIE_PCIE2REG_MAILBOXINT, status); + status = brcmf_pcie_read_reg32(devinfo, devinfo->reginfo->mailboxint); + brcmf_pcie_write_reg32(devinfo, devinfo->reginfo->mailboxint, status); devinfo->irq_allocated = false; } @@ -937,7 +1035,7 @@ static int brcmf_pcie_ring_mb_ring_bell(void *ctx) brcmf_dbg(PCIE, "RING !\n"); /* Any arbitrary value will do, lets use 1 */ - brcmf_pcie_write_reg32(devinfo, BRCMF_PCIE_PCIE2REG_H2D_MAILBOX_0, 1); + brcmf_pcie_write_reg32(devinfo, devinfo->reginfo->h2d_mailbox_0, 1); return 0; } @@ -1382,23 +1480,25 @@ static int brcmf_pcie_get_memdump(struct device *dev, void *data, size_t len) return 0; } -static -int brcmf_pcie_get_fwname(struct device *dev, const char *ext, u8 *fw_name) +static int brcmf_pcie_get_blob(struct device *dev, const struct firmware **fw, + enum brcmf_blob_type type) { struct brcmf_bus *bus_if = dev_get_drvdata(dev); - struct brcmf_fw_request *fwreq; - struct brcmf_fw_name fwnames[] = { - { ext, fw_name }, - }; + struct brcmf_pciedev *buspub = bus_if->bus_priv.pcie; + struct brcmf_pciedev_info *devinfo = buspub->devinfo; - fwreq = brcmf_fw_alloc_request(bus_if->chip, bus_if->chiprev, - brcmf_pcie_fwnames, - ARRAY_SIZE(brcmf_pcie_fwnames), - fwnames, ARRAY_SIZE(fwnames)); - if (!fwreq) - return -ENOMEM; + switch (type) { + case BRCMF_BLOB_CLM: + *fw = devinfo->clm_fw; + devinfo->clm_fw = NULL; + break; + default: + return -ENOENT; + } + + if (!*fw) + return -ENOENT; - kfree(fwreq); return 0; } @@ -1445,7 +1545,7 @@ static const struct brcmf_bus_ops brcmf_pcie_bus_ops = { .wowl_config = brcmf_pcie_wowl_config, .get_ramsize = brcmf_pcie_get_ramsize, .get_memdump = brcmf_pcie_get_memdump, - .get_fwname = brcmf_pcie_get_fwname, + .get_blob = brcmf_pcie_get_blob, .reset = brcmf_pcie_reset, }; @@ -1698,15 +1798,22 @@ static int brcmf_pcie_buscoreprep(void *ctx) static int brcmf_pcie_buscore_reset(void *ctx, struct brcmf_chip *chip) { struct brcmf_pciedev_info *devinfo = (struct brcmf_pciedev_info *)ctx; - u32 val; + struct brcmf_core *core; + u32 val, reg; devinfo->ci = chip; brcmf_pcie_reset_device(devinfo); - val = brcmf_pcie_read_reg32(devinfo, BRCMF_PCIE_PCIE2REG_MAILBOXINT); + /* reginfo is not ready yet */ + core = brcmf_chip_get_core(chip, BCMA_CORE_PCIE2); + if (core->rev >= 64) + reg = BRCMF_PCIE_64_PCIE2REG_MAILBOXINT; + else + reg = BRCMF_PCIE_PCIE2REG_MAILBOXINT; + + val = brcmf_pcie_read_reg32(devinfo, reg); if (val != 0xffffffff) - brcmf_pcie_write_reg32(devinfo, BRCMF_PCIE_PCIE2REG_MAILBOXINT, - val); + brcmf_pcie_write_reg32(devinfo, reg, val); return 0; } @@ -1729,8 +1836,206 @@ static const struct brcmf_buscore_ops brcmf_pcie_buscore_ops = { .write32 = brcmf_pcie_buscore_write32, }; +#define BRCMF_OTP_SYS_VENDOR 0x15 +#define BRCMF_OTP_BRCM_CIS 0x80 + +#define BRCMF_OTP_VENDOR_HDR 0x00000008 + +static int +brcmf_pcie_parse_otp_sys_vendor(struct brcmf_pciedev_info *devinfo, + u8 *data, size_t size) +{ + int idx = 4; + const char *chip_params; + const char *board_params; + const char *p; + + /* 4-byte header and two empty strings */ + if (size < 6) + return -EINVAL; + + if (get_unaligned_le32(data) != BRCMF_OTP_VENDOR_HDR) + return -EINVAL; + + chip_params = &data[idx]; + + /* Skip first string, including terminator */ + idx += strnlen(chip_params, size - idx) + 1; + if (idx >= size) + return -EINVAL; + + board_params = &data[idx]; + + /* Skip to terminator of second string */ + idx += strnlen(board_params, size - idx); + if (idx >= size) + return -EINVAL; + + /* At this point both strings are guaranteed NUL-terminated */ + brcmf_dbg(PCIE, "OTP: chip_params='%s' board_params='%s'\n", + chip_params, board_params); + + p = skip_spaces(board_params); + while (*p) { + char tag = *p++; + const char *end; + size_t len; + + if (*p++ != '=') /* implicit NUL check */ + return -EINVAL; + + /* *p might be NUL here, if so end == p and len == 0 */ + end = strchrnul(p, ' '); + len = end - p; + + /* leave 1 byte for NUL in destination string */ + if (len > (BRCMF_OTP_MAX_PARAM_LEN - 1)) + return -EINVAL; + + /* Copy len characters plus a NUL terminator */ + switch (tag) { + case 'M': + strscpy(devinfo->otp.module, p, len + 1); + break; + case 'V': + strscpy(devinfo->otp.vendor, p, len + 1); + break; + case 'm': + strscpy(devinfo->otp.version, p, len + 1); + break; + } + + /* Skip to next arg, if any */ + p = skip_spaces(end); + } + + brcmf_dbg(PCIE, "OTP: module=%s vendor=%s version=%s\n", + devinfo->otp.module, devinfo->otp.vendor, + devinfo->otp.version); + + if (!devinfo->otp.module[0] || + !devinfo->otp.vendor[0] || + !devinfo->otp.version[0]) + return -EINVAL; + + devinfo->otp.valid = true; + return 0; +} + +static int +brcmf_pcie_parse_otp(struct brcmf_pciedev_info *devinfo, u8 *otp, size_t size) +{ + int p = 0; + int ret = -EINVAL; + + brcmf_dbg(PCIE, "parse_otp size=%zd\n", size); + + while (p < (size - 1)) { + u8 type = otp[p]; + u8 length = otp[p + 1]; + + if (type == 0) + break; + + if ((p + 2 + length) > size) + break; + + switch (type) { + case BRCMF_OTP_SYS_VENDOR: + brcmf_dbg(PCIE, "OTP @ 0x%x (%d): SYS_VENDOR\n", + p, length); + ret = brcmf_pcie_parse_otp_sys_vendor(devinfo, + &otp[p + 2], + length); + break; + case BRCMF_OTP_BRCM_CIS: + brcmf_dbg(PCIE, "OTP @ 0x%x (%d): BRCM_CIS\n", + p, length); + break; + default: + brcmf_dbg(PCIE, "OTP @ 0x%x (%d): Unknown type 0x%x\n", + p, length, type); + break; + } + + p += 2 + length; + } + + return ret; +} + +static int brcmf_pcie_read_otp(struct brcmf_pciedev_info *devinfo) +{ + const struct pci_dev *pdev = devinfo->pdev; + struct brcmf_bus *bus = dev_get_drvdata(&pdev->dev); + u32 coreid, base, words, idx, sromctl; + u16 *otp; + struct brcmf_core *core; + int ret; + + switch (devinfo->ci->chip) { + case BRCM_CC_4378_CHIP_ID: + coreid = BCMA_CORE_GCI; + base = 0x1120; + words = 0x170; + break; + default: + /* OTP not supported on this chip */ + return 0; + } + + core = brcmf_chip_get_core(devinfo->ci, coreid); + if (!core) { + brcmf_err(bus, "No OTP core\n"); + return -ENODEV; + } + + if (coreid == BCMA_CORE_CHIPCOMMON) { + /* Chips with OTP accessed via ChipCommon need additional + * handling to access the OTP + */ + brcmf_pcie_select_core(devinfo, coreid); + sromctl = READCC32(devinfo, sromcontrol); + + if (!(sromctl & BCMA_CC_SROM_CONTROL_OTP_PRESENT)) { + /* Chip lacks OTP, try without it... */ + brcmf_err(bus, + "OTP unavailable, using default firmware\n"); + return 0; + } + + /* Map OTP to shadow area */ + WRITECC32(devinfo, sromcontrol, + sromctl | BCMA_CC_SROM_CONTROL_OTPSEL); + } + + otp = kcalloc(words, sizeof(u16), GFP_KERNEL); + if (!otp) + return -ENOMEM; + + /* Map bus window to SROM/OTP shadow area in core */ + base = brcmf_pcie_buscore_prep_addr(devinfo->pdev, base + core->base); + + brcmf_dbg(PCIE, "OTP data:\n"); + for (idx = 0; idx < words; idx++) { + otp[idx] = brcmf_pcie_read_reg16(devinfo, base + 2 * idx); + brcmf_dbg(PCIE, "[%8x] 0x%04x\n", base + 2 * idx, otp[idx]); + } + + if (coreid == BCMA_CORE_CHIPCOMMON) { + brcmf_pcie_select_core(devinfo, coreid); + WRITECC32(devinfo, sromcontrol, sromctl); + } + + ret = brcmf_pcie_parse_otp(devinfo, (u8 *)otp, 2 * words); + kfree(otp); + + return ret; +} + #define BRCMF_PCIE_FW_CODE 0 #define BRCMF_PCIE_FW_NVRAM 1 +#define BRCMF_PCIE_FW_CLM 2 static void brcmf_pcie_setup(struct device *dev, int ret, struct brcmf_fw_request *fwreq) @@ -1755,6 +2060,7 @@ static void brcmf_pcie_setup(struct device *dev, int ret, fw = fwreq->items[BRCMF_PCIE_FW_CODE].binary; nvram = fwreq->items[BRCMF_PCIE_FW_NVRAM].nv_data.data; nvram_len = fwreq->items[BRCMF_PCIE_FW_NVRAM].nv_data.len; + devinfo->clm_fw = fwreq->items[BRCMF_PCIE_FW_CLM].binary; kfree(fwreq); ret = brcmf_chip_get_raminfo(devinfo->ci); @@ -1830,6 +2136,7 @@ brcmf_pcie_prepare_fw_request(struct brcmf_pciedev_info *devinfo) struct brcmf_fw_name fwnames[] = { { ".bin", devinfo->fw_name }, { ".txt", devinfo->nvram_name }, + { ".clm_blob", devinfo->clm_name }, }; fwreq = brcmf_fw_alloc_request(devinfo->ci->chip, devinfo->ci->chiprev, @@ -1842,11 +2149,51 @@ brcmf_pcie_prepare_fw_request(struct brcmf_pciedev_info *devinfo) fwreq->items[BRCMF_PCIE_FW_CODE].type = BRCMF_FW_TYPE_BINARY; fwreq->items[BRCMF_PCIE_FW_NVRAM].type = BRCMF_FW_TYPE_NVRAM; fwreq->items[BRCMF_PCIE_FW_NVRAM].flags = BRCMF_FW_REQF_OPTIONAL; - fwreq->board_type = devinfo->settings->board_type; + fwreq->items[BRCMF_PCIE_FW_CLM].type = BRCMF_FW_TYPE_BINARY; + fwreq->items[BRCMF_PCIE_FW_CLM].flags = BRCMF_FW_REQF_OPTIONAL; /* NVRAM reserves PCI domain 0 for Broadcom's SDK faked bus */ fwreq->domain_nr = pci_domain_nr(devinfo->pdev->bus) + 1; fwreq->bus_nr = devinfo->pdev->bus->number; + /* Apple platforms with fancy firmware/NVRAM selection */ + if (devinfo->settings->board_type && + devinfo->settings->antenna_sku && + devinfo->otp.valid) { + const struct brcmf_otp_params *otp = &devinfo->otp; + struct device *dev = &devinfo->pdev->dev; + const char **bt = fwreq->board_types; + + brcmf_dbg(PCIE, "Apple board: %s\n", + devinfo->settings->board_type); + + /* Example: apple,shikoku-RASP-m-6.11-X3 */ + bt[0] = devm_kasprintf(dev, GFP_KERNEL, "%s-%s-%s-%s-%s", + devinfo->settings->board_type, + otp->module, otp->vendor, otp->version, + devinfo->settings->antenna_sku); + bt[1] = devm_kasprintf(dev, GFP_KERNEL, "%s-%s-%s-%s", + devinfo->settings->board_type, + otp->module, otp->vendor, otp->version); + bt[2] = devm_kasprintf(dev, GFP_KERNEL, "%s-%s-%s", + devinfo->settings->board_type, + otp->module, otp->vendor); + bt[3] = devm_kasprintf(dev, GFP_KERNEL, "%s-%s", + devinfo->settings->board_type, + otp->module); + bt[4] = devm_kasprintf(dev, GFP_KERNEL, "%s-%s", + devinfo->settings->board_type, + devinfo->settings->antenna_sku); + bt[5] = devinfo->settings->board_type; + + if (!bt[0] || !bt[1] || !bt[2] || !bt[3] || !bt[4]) { + kfree(fwreq); + return NULL; + } + } else { + brcmf_dbg(PCIE, "Board: %s\n", devinfo->settings->board_type); + fwreq->board_types[0] = devinfo->settings->board_type; + } + return fwreq; } @@ -1857,6 +2204,7 @@ brcmf_pcie_probe(struct pci_dev *pdev, const struct pci_device_id *id) struct brcmf_fw_request *fwreq; struct brcmf_pciedev_info *devinfo; struct brcmf_pciedev *pcie_bus_dev; + struct brcmf_core *core; struct brcmf_bus *bus; brcmf_dbg(PCIE, "Enter %x:%x\n", pdev->vendor, pdev->device); @@ -1876,6 +2224,12 @@ brcmf_pcie_probe(struct pci_dev *pdev, const struct pci_device_id *id) goto fail; } + core = brcmf_chip_get_core(devinfo->ci, BCMA_CORE_PCIE2); + if (core->rev >= 64) + devinfo->reginfo = &brcmf_reginfo_64; + else + devinfo->reginfo = &brcmf_reginfo_default; + pcie_bus_dev = kzalloc(sizeof(*pcie_bus_dev), GFP_KERNEL); if (pcie_bus_dev == NULL) { ret = -ENOMEM; @@ -1918,6 +2272,12 @@ brcmf_pcie_probe(struct pci_dev *pdev, const struct pci_device_id *id) if (ret) goto fail_bus; + ret = brcmf_pcie_read_otp(devinfo); + if (ret) { + brcmf_err(bus, "failed to parse OTP\n"); + goto fail_brcmf; + } + fwreq = brcmf_pcie_prepare_fw_request(devinfo); if (!fwreq) { ret = -ENOMEM; @@ -1981,6 +2341,7 @@ brcmf_pcie_remove(struct pci_dev *pdev) brcmf_pcie_release_ringbuffers(devinfo); brcmf_pcie_reset_device(devinfo); brcmf_pcie_release_resource(devinfo); + release_firmware(devinfo->clm_fw); if (devinfo->ci) brcmf_chip_detach(devinfo->ci); @@ -2038,7 +2399,7 @@ static int brcmf_pcie_pm_leave_D3(struct device *dev) brcmf_dbg(PCIE, "Enter, dev=%p, bus=%p\n", dev, bus); /* Check if device is still up and running, if so we are ready */ - if (brcmf_pcie_read_reg32(devinfo, BRCMF_PCIE_PCIE2REG_INTMASK) != 0) { + if (brcmf_pcie_read_reg32(devinfo, devinfo->reginfo->intmask) != 0) { brcmf_dbg(PCIE, "Try to wakeup device....\n"); if (brcmf_pcie_send_mb_data(devinfo, BRCMF_H2D_HOST_D0_INFORM)) goto cleanup; @@ -2105,6 +2466,9 @@ static const struct pci_device_id brcmf_pcie_devid_table[] = { BRCMF_PCIE_DEVICE(BRCM_PCIE_4366_2G_DEVICE_ID), BRCMF_PCIE_DEVICE(BRCM_PCIE_4366_5G_DEVICE_ID), BRCMF_PCIE_DEVICE(BRCM_PCIE_4371_DEVICE_ID), + BRCMF_PCIE_DEVICE(BRCM_PCIE_4378_DEVICE_ID), + BRCMF_PCIE_DEVICE(CY_PCIE_89459_DEVICE_ID), + BRCMF_PCIE_DEVICE(CY_PCIE_89459_RAW_DEVICE_ID), { /* end: all zeroes */ } }; diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/pno.c b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/pno.c index fabfbb0b40b0..d0a7465be586 100644 --- a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/pno.c +++ b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/pno.c @@ -158,12 +158,12 @@ static int brcmf_pno_set_random(struct brcmf_if *ifp, struct brcmf_pno_info *pi) struct brcmf_pno_macaddr_le pfn_mac; u8 *mac_addr = NULL; u8 *mac_mask = NULL; - int err, i; + int err, i, ri; - for (i = 0; i < pi->n_reqs; i++) - if (pi->reqs[i]->flags & NL80211_SCAN_FLAG_RANDOM_ADDR) { - mac_addr = pi->reqs[i]->mac_addr; - mac_mask = pi->reqs[i]->mac_addr_mask; + for (ri = 0; ri < pi->n_reqs; ri++) + if (pi->reqs[ri]->flags & NL80211_SCAN_FLAG_RANDOM_ADDR) { + mac_addr = pi->reqs[ri]->mac_addr; + mac_mask = pi->reqs[ri]->mac_addr_mask; break; } @@ -185,7 +185,7 @@ static int brcmf_pno_set_random(struct brcmf_if *ifp, struct brcmf_pno_info *pi) pfn_mac.mac[0] |= 0x02; brcmf_dbg(SCAN, "enabling random mac: reqid=%llu mac=%pM\n", - pi->reqs[i]->reqid, pfn_mac.mac); + pi->reqs[ri]->reqid, pfn_mac.mac); err = brcmf_fil_iovar_data_set(ifp, "pfn_macaddr", &pfn_mac, sizeof(pfn_mac)); if (err) diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/sdio.c b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/sdio.c index 8968809399c7..465d95d83759 100644 --- a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/sdio.c +++ b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/sdio.c @@ -618,6 +618,7 @@ BRCMF_FW_DEF(43430A0, "brcmfmac43430a0-sdio"); /* Note the names are not postfixed with a1 for backward compatibility */ BRCMF_FW_CLM_DEF(43430A1, "brcmfmac43430-sdio"); BRCMF_FW_DEF(43430B0, "brcmfmac43430b0-sdio"); +BRCMF_FW_CLM_DEF(43439, "brcmfmac43439-sdio"); BRCMF_FW_CLM_DEF(43455, "brcmfmac43455-sdio"); BRCMF_FW_DEF(43456, "brcmfmac43456-sdio"); BRCMF_FW_CLM_DEF(4354, "brcmfmac4354-sdio"); @@ -657,6 +658,7 @@ static const struct brcmf_firmware_mapping brcmf_sdio_fwnames[] = { BRCMF_FW_ENTRY(BRCM_CC_4359_CHIP_ID, 0xFFFFFFFF, 4359), BRCMF_FW_ENTRY(CY_CC_4373_CHIP_ID, 0xFFFFFFFF, 4373), BRCMF_FW_ENTRY(CY_CC_43012_CHIP_ID, 0xFFFFFFFF, 43012), + BRCMF_FW_ENTRY(CY_CC_43439_CHIP_ID, 0xFFFFFFFF, 43439), BRCMF_FW_ENTRY(CY_CC_43752_CHIP_ID, 0xFFFFFFFF, 43752) }; @@ -4129,23 +4131,24 @@ brcmf_sdio_watchdog(struct timer_list *t) } } -static -int brcmf_sdio_get_fwname(struct device *dev, const char *ext, u8 *fw_name) +static int brcmf_sdio_get_blob(struct device *dev, const struct firmware **fw, + enum brcmf_blob_type type) { struct brcmf_bus *bus_if = dev_get_drvdata(dev); - struct brcmf_fw_request *fwreq; - struct brcmf_fw_name fwnames[] = { - { ext, fw_name }, - }; + struct brcmf_sdio_dev *sdiodev = bus_if->bus_priv.sdio; - fwreq = brcmf_fw_alloc_request(bus_if->chip, bus_if->chiprev, - brcmf_sdio_fwnames, - ARRAY_SIZE(brcmf_sdio_fwnames), - fwnames, ARRAY_SIZE(fwnames)); - if (!fwreq) - return -ENOMEM; + switch (type) { + case BRCMF_BLOB_CLM: + *fw = sdiodev->clm_fw; + sdiodev->clm_fw = NULL; + break; + default: + return -ENOENT; + } + + if (!*fw) + return -ENOENT; - kfree(fwreq); return 0; } @@ -4180,13 +4183,14 @@ static const struct brcmf_bus_ops brcmf_sdio_bus_ops = { .wowl_config = brcmf_sdio_wowl_config, .get_ramsize = brcmf_sdio_bus_get_ramsize, .get_memdump = brcmf_sdio_bus_get_memdump, - .get_fwname = brcmf_sdio_get_fwname, + .get_blob = brcmf_sdio_get_blob, .debugfs_create = brcmf_sdio_debugfs_create, .reset = brcmf_sdio_bus_reset }; #define BRCMF_SDIO_FW_CODE 0 #define BRCMF_SDIO_FW_NVRAM 1 +#define BRCMF_SDIO_FW_CLM 2 static void brcmf_sdio_firmware_callback(struct device *dev, int err, struct brcmf_fw_request *fwreq) @@ -4209,6 +4213,7 @@ static void brcmf_sdio_firmware_callback(struct device *dev, int err, code = fwreq->items[BRCMF_SDIO_FW_CODE].binary; nvram = fwreq->items[BRCMF_SDIO_FW_NVRAM].nv_data.data; nvram_len = fwreq->items[BRCMF_SDIO_FW_NVRAM].nv_data.len; + sdiod->clm_fw = fwreq->items[BRCMF_SDIO_FW_CLM].binary; kfree(fwreq); /* try to download image and nvram to the dongle */ @@ -4407,6 +4412,7 @@ brcmf_sdio_prepare_fw_request(struct brcmf_sdio *bus) struct brcmf_fw_name fwnames[] = { { ".bin", bus->sdiodev->fw_name }, { ".txt", bus->sdiodev->nvram_name }, + { ".clm_blob", bus->sdiodev->clm_name }, }; fwreq = brcmf_fw_alloc_request(bus->ci->chip, bus->ci->chiprev, @@ -4418,7 +4424,9 @@ brcmf_sdio_prepare_fw_request(struct brcmf_sdio *bus) fwreq->items[BRCMF_SDIO_FW_CODE].type = BRCMF_FW_TYPE_BINARY; fwreq->items[BRCMF_SDIO_FW_NVRAM].type = BRCMF_FW_TYPE_NVRAM; - fwreq->board_type = bus->sdiodev->settings->board_type; + fwreq->items[BRCMF_SDIO_FW_CLM].type = BRCMF_FW_TYPE_BINARY; + fwreq->items[BRCMF_SDIO_FW_CLM].flags = BRCMF_FW_REQF_OPTIONAL; + fwreq->board_types[0] = bus->sdiodev->settings->board_type; return fwreq; } @@ -4574,6 +4582,8 @@ void brcmf_sdio_remove(struct brcmf_sdio *bus) if (bus->sdiodev->settings) brcmf_release_module_param(bus->sdiodev->settings); + release_firmware(bus->sdiodev->clm_fw); + bus->sdiodev->clm_fw = NULL; kfree(bus->rxbuf); kfree(bus->hdrbuf); kfree(bus); diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/sdio.h b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/sdio.h index 47351ff458ca..b76d34d36bde 100644 --- a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/sdio.h +++ b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/sdio.h @@ -186,9 +186,11 @@ struct brcmf_sdio_dev { struct sg_table sgtable; char fw_name[BRCMF_FW_NAME_LEN]; char nvram_name[BRCMF_FW_NAME_LEN]; + char clm_name[BRCMF_FW_NAME_LEN]; bool wowl_enabled; enum brcmf_sdiod_state state; struct brcmf_sdiod_freezer *freezer; + const struct firmware *clm_fw; }; /* sdio core registers */ diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/usb.c b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/usb.c index 9fb68c2dc7e3..85e18fb9c497 100644 --- a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/usb.c +++ b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/usb.c @@ -1154,24 +1154,11 @@ error: return NULL; } -static -int brcmf_usb_get_fwname(struct device *dev, const char *ext, u8 *fw_name) +static int brcmf_usb_get_blob(struct device *dev, const struct firmware **fw, + enum brcmf_blob_type type) { - struct brcmf_bus *bus = dev_get_drvdata(dev); - struct brcmf_fw_request *fwreq; - struct brcmf_fw_name fwnames[] = { - { ext, fw_name }, - }; - - fwreq = brcmf_fw_alloc_request(bus->chip, bus->chiprev, - brcmf_usb_fwnames, - ARRAY_SIZE(brcmf_usb_fwnames), - fwnames, ARRAY_SIZE(fwnames)); - if (!fwreq) - return -ENOMEM; - - kfree(fwreq); - return 0; + /* No blobs for USB devices... */ + return -ENOENT; } static const struct brcmf_bus_ops brcmf_usb_bus_ops = { @@ -1180,7 +1167,7 @@ static const struct brcmf_bus_ops brcmf_usb_bus_ops = { .txdata = brcmf_usb_tx, .txctl = brcmf_usb_tx_ctlpkt, .rxctl = brcmf_usb_rx_ctlpkt, - .get_fwname = brcmf_usb_get_fwname, + .get_blob = brcmf_usb_get_blob, }; #define BRCMF_USB_FW_CODE 0 diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmsmac/types.h b/drivers/net/wireless/broadcom/brcm80211/brcmsmac/types.h index ae1f3ad40d45..2b0df07ced74 100644 --- a/drivers/net/wireless/broadcom/brcm80211/brcmsmac/types.h +++ b/drivers/net/wireless/broadcom/brcm80211/brcmsmac/types.h @@ -123,7 +123,7 @@ */ /******************************************************************** - * Phy/Core Configuration. Defines macros to to check core phy/rev * + * Phy/Core Configuration. Defines macros to check core phy/rev * * compile-time configuration. Defines default core support. * * ****************************************************************** */ diff --git a/drivers/net/wireless/broadcom/brcm80211/include/brcm_hw_ids.h b/drivers/net/wireless/broadcom/brcm80211/include/brcm_hw_ids.h index ed0b707f0cdf..f4939cf62767 100644 --- a/drivers/net/wireless/broadcom/brcm80211/include/brcm_hw_ids.h +++ b/drivers/net/wireless/broadcom/brcm80211/include/brcm_hw_ids.h @@ -51,9 +51,12 @@ #define BRCM_CC_43664_CHIP_ID 43664 #define BRCM_CC_43666_CHIP_ID 43666 #define BRCM_CC_4371_CHIP_ID 0x4371 +#define BRCM_CC_4378_CHIP_ID 0x4378 #define CY_CC_4373_CHIP_ID 0x4373 #define CY_CC_43012_CHIP_ID 43012 +#define CY_CC_43439_CHIP_ID 43439 #define CY_CC_43752_CHIP_ID 43752 +#define CY_CC_89459_CHIP_ID 0x4355 /* USB Device IDs */ #define BRCM_USB_43143_DEVICE_ID 0xbd1e @@ -87,7 +90,9 @@ #define BRCM_PCIE_4366_2G_DEVICE_ID 0x43c4 #define BRCM_PCIE_4366_5G_DEVICE_ID 0x43c5 #define BRCM_PCIE_4371_DEVICE_ID 0x440d - +#define BRCM_PCIE_4378_DEVICE_ID 0x4425 +#define CY_PCIE_89459_DEVICE_ID 0x4415 +#define CY_PCIE_89459_RAW_DEVICE_ID 0x4355 /* brcmsmac IDs */ #define BCM4313_D11N2G_ID 0x4727 /* 4313 802.11n 2.4G device */ diff --git a/drivers/net/wireless/intel/ipw2x00/ipw2100.c b/drivers/net/wireless/intel/ipw2x00/ipw2100.c index 5234511dac78..b0f23cf1a621 100644 --- a/drivers/net/wireless/intel/ipw2x00/ipw2100.c +++ b/drivers/net/wireless/intel/ipw2x00/ipw2100.c @@ -5907,8 +5907,8 @@ static void ipw_ethtool_get_drvinfo(struct net_device *dev, struct ipw2100_priv *priv = libipw_priv(dev); char fw_ver[64], ucode_ver[64]; - strlcpy(info->driver, DRV_NAME, sizeof(info->driver)); - strlcpy(info->version, DRV_VERSION, sizeof(info->version)); + strscpy(info->driver, DRV_NAME, sizeof(info->driver)); + strscpy(info->version, DRV_VERSION, sizeof(info->version)); ipw2100_get_fwversion(priv, fw_ver, sizeof(fw_ver)); ipw2100_get_ucodeversion(priv, ucode_ver, sizeof(ucode_ver)); @@ -5916,7 +5916,7 @@ static void ipw_ethtool_get_drvinfo(struct net_device *dev, snprintf(info->fw_version, sizeof(info->fw_version), "%s:%d:%s", fw_ver, priv->eeprom_version, ucode_ver); - strlcpy(info->bus_info, pci_name(priv->pci_dev), + strscpy(info->bus_info, pci_name(priv->pci_dev), sizeof(info->bus_info)); } @@ -6529,7 +6529,7 @@ static struct pci_driver ipw2100_pci_driver = { .shutdown = ipw2100_shutdown, }; -/** +/* * Initialize the ipw2100 driver/module * * @returns 0 if ok, < 0 errno node con error. @@ -6561,7 +6561,7 @@ out: return ret; } -/** +/* * Cleanup ipw2100 driver registration */ static void __exit ipw2100_exit(void) diff --git a/drivers/net/wireless/intel/ipw2x00/ipw2200.c b/drivers/net/wireless/intel/ipw2x00/ipw2200.c index 029dacebe751..5b483de18c81 100644 --- a/drivers/net/wireless/intel/ipw2x00/ipw2200.c +++ b/drivers/net/wireless/intel/ipw2x00/ipw2200.c @@ -10424,8 +10424,8 @@ static void ipw_ethtool_get_drvinfo(struct net_device *dev, char date[32]; u32 len; - strlcpy(info->driver, DRV_NAME, sizeof(info->driver)); - strlcpy(info->version, DRV_VERSION, sizeof(info->version)); + strscpy(info->driver, DRV_NAME, sizeof(info->driver)); + strscpy(info->version, DRV_VERSION, sizeof(info->version)); len = sizeof(vers); ipw_get_ordinal(p, IPW_ORD_STAT_FW_VERSION, vers, &len); @@ -10434,7 +10434,7 @@ static void ipw_ethtool_get_drvinfo(struct net_device *dev, snprintf(info->fw_version, sizeof(info->fw_version), "%s (%s)", vers, date); - strlcpy(info->bus_info, pci_name(p->pci_dev), + strscpy(info->bus_info, pci_name(p->pci_dev), sizeof(info->bus_info)); } diff --git a/drivers/net/wireless/intel/ipw2x00/ipw2200.h b/drivers/net/wireless/intel/ipw2x00/ipw2200.h index 55cac934f4ee..09ddd21608d4 100644 --- a/drivers/net/wireless/intel/ipw2x00/ipw2200.h +++ b/drivers/net/wireless/intel/ipw2x00/ipw2200.h @@ -651,7 +651,7 @@ struct ipw_rx_notification { struct notif_link_deterioration link_deterioration; struct notif_calibration calibration; struct notif_noise noise; - u8 raw[0]; + DECLARE_FLEX_ARRAY(u8, raw); } u; } __packed; diff --git a/drivers/net/wireless/intel/ipw2x00/libipw.h b/drivers/net/wireless/intel/ipw2x00/libipw.h index 7964ef7d15f0..bec7bc273748 100644 --- a/drivers/net/wireless/intel/ipw2x00/libipw.h +++ b/drivers/net/wireless/intel/ipw2x00/libipw.h @@ -405,7 +405,7 @@ struct libipw_auth { __le16 transaction; __le16 status; /* challenge */ - struct libipw_info_element info_element[]; + u8 variable[]; } __packed; struct libipw_channel_switch { @@ -423,7 +423,6 @@ struct libipw_action { union { struct libipw_action_exchange { u8 token; - struct libipw_info_element info_element[0]; } exchange; struct libipw_channel_switch channel_switch; @@ -441,7 +440,7 @@ struct libipw_disassoc { struct libipw_probe_request { struct libipw_hdr_3addr header; /* SSID, supported rates */ - struct libipw_info_element info_element[]; + u8 variable[]; } __packed; struct libipw_probe_response { @@ -451,7 +450,7 @@ struct libipw_probe_response { __le16 capability; /* SSID, supported rates, FH params, DS params, * CF params, IBSS params, TIM (if beacon), RSN */ - struct libipw_info_element info_element[]; + u8 variable[]; } __packed; /* Alias beacon for probe_response */ @@ -462,7 +461,7 @@ struct libipw_assoc_request { __le16 capability; __le16 listen_interval; /* SSID, supported rates, RSN */ - struct libipw_info_element info_element[]; + u8 variable[]; } __packed; struct libipw_reassoc_request { @@ -470,7 +469,7 @@ struct libipw_reassoc_request { __le16 capability; __le16 listen_interval; u8 current_ap[ETH_ALEN]; - struct libipw_info_element info_element[]; + u8 variable[]; } __packed; struct libipw_assoc_response { @@ -479,7 +478,7 @@ struct libipw_assoc_response { __le16 status; __le16 aid; /* supported rates */ - struct libipw_info_element info_element[]; + u8 variable[]; } __packed; struct libipw_txb { diff --git a/drivers/net/wireless/intel/ipw2x00/libipw_rx.c b/drivers/net/wireless/intel/ipw2x00/libipw_rx.c index 7a684b76f39b..48d6870bbf4e 100644 --- a/drivers/net/wireless/intel/ipw2x00/libipw_rx.c +++ b/drivers/net/wireless/intel/ipw2x00/libipw_rx.c @@ -1329,8 +1329,8 @@ static int libipw_handle_assoc_resp(struct libipw_device *ieee, struct libipw_as network->wpa_ie_len = 0; network->rsn_ie_len = 0; - if (libipw_parse_info_param - (frame->info_element, stats->len - sizeof(*frame), network)) + if (libipw_parse_info_param((void *)frame->variable, + stats->len - sizeof(*frame), network)) return 1; network->mode = 0; @@ -1389,8 +1389,8 @@ static int libipw_network_init(struct libipw_device *ieee, struct libipw_probe_r network->wpa_ie_len = 0; network->rsn_ie_len = 0; - if (libipw_parse_info_param - (beacon->info_element, stats->len - sizeof(*beacon), network)) + if (libipw_parse_info_param((void *)beacon->variable, + stats->len - sizeof(*beacon), network)) return 1; network->mode = 0; @@ -1510,7 +1510,7 @@ static void libipw_process_probe_response(struct libipw_device struct libipw_network *target; struct libipw_network *oldest = NULL; #ifdef CONFIG_LIBIPW_DEBUG - struct libipw_info_element *info_element = beacon->info_element; + struct libipw_info_element *info_element = (void *)beacon->variable; #endif unsigned long flags; diff --git a/drivers/net/wireless/intel/iwlegacy/3945-mac.c b/drivers/net/wireless/intel/iwlegacy/3945-mac.c index 846138d6e33d..7352d5b2095f 100644 --- a/drivers/net/wireless/intel/iwlegacy/3945-mac.c +++ b/drivers/net/wireless/intel/iwlegacy/3945-mac.c @@ -3254,7 +3254,7 @@ il3945_store_measurement(struct device *d, struct device_attribute *attr, if (count) { char *p = buffer; - strlcpy(buffer, buf, sizeof(buffer)); + strscpy(buffer, buf, sizeof(buffer)); channel = simple_strtoul(p, NULL, 0); if (channel) params.channel = channel; diff --git a/drivers/net/wireless/intel/iwlegacy/4965-rs.c b/drivers/net/wireless/intel/iwlegacy/4965-rs.c index d8a5dbf89a02..718efb1aa1b0 100644 --- a/drivers/net/wireless/intel/iwlegacy/4965-rs.c +++ b/drivers/net/wireless/intel/iwlegacy/4965-rs.c @@ -1167,7 +1167,7 @@ il4965_rs_switch_to_mimo2(struct il_priv *il, struct il_lq_sta *lq_sta, if (!conf_is_ht(conf) || !sta->deflink.ht_cap.ht_supported) return -1; - if (sta->smps_mode == IEEE80211_SMPS_STATIC) + if (sta->deflink.smps_mode == IEEE80211_SMPS_STATIC) return -1; /* Need both Tx chains/antennas to support MIMO */ diff --git a/drivers/net/wireless/intel/iwlegacy/commands.h b/drivers/net/wireless/intel/iwlegacy/commands.h index 4a97310f8fee..28cf4e832152 100644 --- a/drivers/net/wireless/intel/iwlegacy/commands.h +++ b/drivers/net/wireless/intel/iwlegacy/commands.h @@ -1710,7 +1710,7 @@ struct il4965_tx_resp { */ union { __le32 status; - struct agg_tx_status agg_status[0]; /* for each agg frame */ + DECLARE_FLEX_ARRAY(struct agg_tx_status, agg_status); /* for each agg frame */ } u; } __packed; @@ -3365,7 +3365,7 @@ struct il_rx_pkt { struct il_compressed_ba_resp compressed_ba; struct il_missed_beacon_notif missed_beacon; __le32 status; - u8 raw[0]; + DECLARE_FLEX_ARRAY(u8, raw); } u; } __packed; diff --git a/drivers/net/wireless/intel/iwlegacy/common.c b/drivers/net/wireless/intel/iwlegacy/common.c index 04d27a26260b..341c17fe2af4 100644 --- a/drivers/net/wireless/intel/iwlegacy/common.c +++ b/drivers/net/wireless/intel/iwlegacy/common.c @@ -1870,15 +1870,15 @@ il_set_ht_add_station(struct il_priv *il, u8 idx, struct ieee80211_sta *sta) goto done; D_ASSOC("spatial multiplexing power save mode: %s\n", - (sta->smps_mode == IEEE80211_SMPS_STATIC) ? "static" : - (sta->smps_mode == IEEE80211_SMPS_DYNAMIC) ? "dynamic" : + (sta->deflink.smps_mode == IEEE80211_SMPS_STATIC) ? "static" : + (sta->deflink.smps_mode == IEEE80211_SMPS_DYNAMIC) ? "dynamic" : "disabled"); sta_flags = il->stations[idx].sta.station_flags; sta_flags &= ~(STA_FLG_RTS_MIMO_PROT_MSK | STA_FLG_MIMO_DIS_MSK); - switch (sta->smps_mode) { + switch (sta->deflink.smps_mode) { case IEEE80211_SMPS_STATIC: sta_flags |= STA_FLG_MIMO_DIS_MSK; break; @@ -1888,7 +1888,7 @@ il_set_ht_add_station(struct il_priv *il, u8 idx, struct ieee80211_sta *sta) case IEEE80211_SMPS_OFF: break; default: - IL_WARN("Invalid MIMO PS mode %d\n", sta->smps_mode); + IL_WARN("Invalid MIMO PS mode %d\n", sta->deflink.smps_mode); break; } diff --git a/drivers/net/wireless/intel/iwlwifi/cfg/22000.c b/drivers/net/wireless/intel/iwlwifi/cfg/22000.c index 8ff967edc8f0..110fda65bd21 100644 --- a/drivers/net/wireless/intel/iwlwifi/cfg/22000.c +++ b/drivers/net/wireless/intel/iwlwifi/cfg/22000.c @@ -56,13 +56,16 @@ #define IWL_BZ_A_GF4_A_FW_PRE "iwlwifi-bz-a0-gf4-a0-" #define IWL_BZ_A_MR_A_FW_PRE "iwlwifi-bz-a0-mr-a0-" #define IWL_BZ_A_FM_A_FW_PRE "iwlwifi-bz-a0-fm-a0-" +#define IWL_BZ_A_FM4_A_FW_PRE "iwlwifi-bz-a0-fm4-a0-" #define IWL_GL_A_FM_A_FW_PRE "iwlwifi-gl-a0-fm-a0-" +#define IWL_GL_B_FM_B_FW_PRE "iwlwifi-gl-b0-fm-b0-" #define IWL_BZ_Z_GF_A_FW_PRE "iwlwifi-bz-z0-gf-a0-" #define IWL_BNJ_A_FM_A_FW_PRE "iwlwifi-BzBnj-a0-fm-a0-" #define IWL_BNJ_A_FM4_A_FW_PRE "iwlwifi-BzBnj-a0-fm4-a0-" #define IWL_BNJ_A_GF_A_FW_PRE "iwlwifi-BzBnj-a0-gf-a0-" #define IWL_BNJ_A_GF4_A_FW_PRE "iwlwifi-BzBnj-a0-gf4-a0-" #define IWL_BNJ_A_HR_B_FW_PRE "iwlwifi-BzBnj-a0-hr-b0-" +#define IWL_BNJ_B_FM_B_FW_PRE "iwlwifi-BzBnj-b0-fm-b0-" #define IWL_QU_B_HR_B_MODULE_FIRMWARE(api) \ @@ -119,8 +122,12 @@ IWL_BZ_A_MR_A_FW_PRE __stringify(api) ".ucode" #define IWL_BZ_A_FM_A_MODULE_FIRMWARE(api) \ IWL_BZ_A_FM_A_FW_PRE __stringify(api) ".ucode" +#define IWL_BZ_A_FM4_A_MODULE_FIRMWARE(api) \ + IWL_BZ_A_FM4_A_FW_PRE __stringify(api) ".ucode" #define IWL_GL_A_FM_A_MODULE_FIRMWARE(api) \ IWL_GL_A_FM_A_FW_PRE __stringify(api) ".ucode" +#define IWL_GL_B_FM_B_MODULE_FIRMWARE(api) \ + IWL_GL_B_FM_B_FW_PRE __stringify(api) ".ucode" #define IWL_BNJ_A_FM_A_MODULE_FIRMWARE(api) \ IWL_BNJ_A_FM_A_FW_PRE __stringify(api) ".ucode" #define IWL_BNJ_A_FM4_A_MODULE_FIRMWARE(api) \ @@ -131,6 +138,8 @@ IWL_BNJ_A_GF4_A_FW_PRE __stringify(api) ".ucode" #define IWL_BNJ_A_HR_B_MODULE_FIRMWARE(api) \ IWL_BNJ_A_HR_B_FW_PRE __stringify(api) ".ucode" +#define IWL_BNJ_B_FM_B_MODULE_FIRMWARE(api) \ + IWL_BNJ_B_FM_B_FW_PRE __stringify(api) ".ucode" static const struct iwl_base_params iwl_22000_base_params = { .eeprom_size = OTP_LOW_IMAGE_SIZE_32K, @@ -240,7 +249,7 @@ static const struct iwl_ht_params iwl_22000_ht_params = { }, \ } -#define IWL_DEVICE_BZ_COMMON \ +#define IWL_DEVICE_BZ \ .ucode_api_max = IWL_22000_UCODE_API_MAX, \ .ucode_api_min = IWL_22000_UCODE_API_MIN, \ .led_mode = IWL_LED_RF_STATE, \ @@ -276,16 +285,13 @@ static const struct iwl_ht_params iwl_22000_ht_params = { .addr = LDBG_M2S_BUF_WRAP_CNT, \ .mask = LDBG_M2S_BUF_WRAP_CNT_VAL_MSK, \ }, \ - } - -#define IWL_DEVICE_BZ \ - IWL_DEVICE_BZ_COMMON, \ + }, \ .trans.umac_prph_offset = 0x300000, \ .trans.device_family = IWL_DEVICE_FAMILY_BZ, \ .trans.base_params = &iwl_ax210_base_params, \ .min_txq_size = 128, \ .gp2_reg_addr = 0xd02c68, \ - .min_ba_txq_size = IWL_DEFAULT_QUEUE_SIZE_EHT, \ + .min_ba_txq_size = IWL_DEFAULT_QUEUE_SIZE_EHT, \ .mon_dram_regs = { \ .write_ptr = { \ .addr = DBGC_CUR_DBGBUF_STATUS, \ @@ -926,6 +932,13 @@ const struct iwl_cfg iwl_cfg_bz_a0_fm_a0 = { .num_rbds = IWL_NUM_RBDS_AX210_HE, }; +const struct iwl_cfg iwl_cfg_bz_a0_fm4_a0 = { + .fw_name_pre = IWL_BZ_A_FM4_A_FW_PRE, + .uhb_supported = true, + IWL_DEVICE_BZ, + .num_rbds = IWL_NUM_RBDS_AX210_HE, +}; + const struct iwl_cfg iwl_cfg_gl_a0_fm_a0 = { .fw_name_pre = IWL_GL_A_FM_A_FW_PRE, .uhb_supported = true, @@ -933,6 +946,13 @@ const struct iwl_cfg iwl_cfg_gl_a0_fm_a0 = { .num_rbds = IWL_NUM_RBDS_AX210_HE, }; +const struct iwl_cfg iwl_cfg_gl_b0_fm_b0 = { + .fw_name_pre = IWL_GL_B_FM_B_FW_PRE, + .uhb_supported = true, + IWL_DEVICE_BZ, + .num_rbds = IWL_NUM_RBDS_AX210_HE, +}; + const struct iwl_cfg iwl_cfg_bz_z0_gf_a0 = { .fw_name_pre = IWL_BZ_Z_GF_A_FW_PRE, .uhb_supported = true, @@ -974,6 +994,13 @@ const struct iwl_cfg iwl_cfg_bnj_a0_hr_b0 = { IWL_DEVICE_BZ, .num_rbds = IWL_NUM_RBDS_AX210_HE, }; + +const struct iwl_cfg iwl_cfg_bnj_b0_fm_b0 = { + .fw_name_pre = IWL_BNJ_B_FM_B_FW_PRE, + .uhb_supported = true, + IWL_DEVICE_BZ, + .num_rbds = IWL_NUM_RBDS_AX210_HE, +}; MODULE_FIRMWARE(IWL_QU_B_HR_B_MODULE_FIRMWARE(IWL_22000_UCODE_API_MAX)); MODULE_FIRMWARE(IWL_QNJ_B_HR_B_MODULE_FIRMWARE(IWL_22000_UCODE_API_MAX)); MODULE_FIRMWARE(IWL_QU_C_HR_B_MODULE_FIRMWARE(IWL_22000_UCODE_API_MAX)); @@ -1007,3 +1034,6 @@ MODULE_FIRMWARE(IWL_BNJ_A_FM4_A_MODULE_FIRMWARE(IWL_22000_UCODE_API_MAX)); MODULE_FIRMWARE(IWL_BNJ_A_GF_A_MODULE_FIRMWARE(IWL_22000_UCODE_API_MAX)); MODULE_FIRMWARE(IWL_BNJ_A_GF4_A_MODULE_FIRMWARE(IWL_22000_UCODE_API_MAX)); MODULE_FIRMWARE(IWL_BNJ_A_HR_B_MODULE_FIRMWARE(IWL_22000_UCODE_API_MAX)); +MODULE_FIRMWARE(IWL_BZ_A_FM4_A_MODULE_FIRMWARE(IWL_22000_UCODE_API_MAX)); +MODULE_FIRMWARE(IWL_GL_B_FM_B_MODULE_FIRMWARE(IWL_22000_UCODE_API_MAX)); +MODULE_FIRMWARE(IWL_BNJ_B_FM_B_MODULE_FIRMWARE(IWL_22000_UCODE_API_MAX)); diff --git a/drivers/net/wireless/intel/iwlwifi/dvm/agn.h b/drivers/net/wireless/intel/iwlwifi/dvm/agn.h index 411a6f6638b4..fefaa414272b 100644 --- a/drivers/net/wireless/intel/iwlwifi/dvm/agn.h +++ b/drivers/net/wireless/intel/iwlwifi/dvm/agn.h @@ -112,7 +112,7 @@ int iwl_load_ucode_wait_alive(struct iwl_priv *priv, enum iwl_ucode_type ucode_type); int iwl_send_calib_results(struct iwl_priv *priv); int iwl_calib_set(struct iwl_priv *priv, - const struct iwl_calib_hdr *cmd, int len); + const struct iwl_calib_cmd *cmd, size_t len); void iwl_calib_free_results(struct iwl_priv *priv); int iwl_dump_nic_event_log(struct iwl_priv *priv, bool full_log, char **buf); diff --git a/drivers/net/wireless/intel/iwlwifi/dvm/calib.c b/drivers/net/wireless/intel/iwlwifi/dvm/calib.c index a11884fa254b..f488620d2844 100644 --- a/drivers/net/wireless/intel/iwlwifi/dvm/calib.c +++ b/drivers/net/wireless/intel/iwlwifi/dvm/calib.c @@ -19,8 +19,7 @@ struct iwl_calib_result { struct list_head list; size_t cmd_len; - struct iwl_calib_hdr hdr; - /* data follows */ + struct iwl_calib_cmd cmd; }; struct statistics_general_data { @@ -43,12 +42,12 @@ int iwl_send_calib_results(struct iwl_priv *priv) int ret; hcmd.len[0] = res->cmd_len; - hcmd.data[0] = &res->hdr; + hcmd.data[0] = &res->cmd; hcmd.dataflags[0] = IWL_HCMD_DFL_NOCOPY; ret = iwl_dvm_send_cmd(priv, &hcmd); if (ret) { IWL_ERR(priv, "Error %d on calib cmd %d\n", - ret, res->hdr.op_code); + ret, res->cmd.hdr.op_code); return ret; } } @@ -57,19 +56,22 @@ int iwl_send_calib_results(struct iwl_priv *priv) } int iwl_calib_set(struct iwl_priv *priv, - const struct iwl_calib_hdr *cmd, int len) + const struct iwl_calib_cmd *cmd, size_t len) { struct iwl_calib_result *res, *tmp; - res = kmalloc(sizeof(*res) + len - sizeof(struct iwl_calib_hdr), - GFP_ATOMIC); + if (check_sub_overflow(len, sizeof(*cmd), &len)) + return -ENOMEM; + + res = kmalloc(struct_size(res, cmd.data, len), GFP_ATOMIC); if (!res) return -ENOMEM; - memcpy(&res->hdr, cmd, len); - res->cmd_len = len; + res->cmd = *cmd; + memcpy(res->cmd.data, cmd->data, len); + res->cmd_len = struct_size(cmd, data, len); list_for_each_entry(tmp, &priv->calib_results, list) { - if (tmp->hdr.op_code == res->hdr.op_code) { + if (tmp->cmd.hdr.op_code == res->cmd.hdr.op_code) { list_replace(&tmp->list, &res->list); kfree(tmp); return 0; diff --git a/drivers/net/wireless/intel/iwlwifi/dvm/dev.h b/drivers/net/wireless/intel/iwlwifi/dvm/dev.h index bbd574091201..1a9eadace188 100644 --- a/drivers/net/wireless/intel/iwlwifi/dvm/dev.h +++ b/drivers/net/wireless/intel/iwlwifi/dvm/dev.h @@ -696,6 +696,7 @@ struct iwl_priv { /* Scan related variables */ unsigned long scan_start; unsigned long scan_start_tsf; + size_t scan_cmd_size; void *scan_cmd; enum nl80211_band scan_band; struct cfg80211_scan_request *scan_request; diff --git a/drivers/net/wireless/intel/iwlwifi/dvm/rs.c b/drivers/net/wireless/intel/iwlwifi/dvm/rs.c index baffa1cbe8fc..687c906a9d72 100644 --- a/drivers/net/wireless/intel/iwlwifi/dvm/rs.c +++ b/drivers/net/wireless/intel/iwlwifi/dvm/rs.c @@ -2,7 +2,7 @@ /****************************************************************************** * * Copyright(c) 2005 - 2014 Intel Corporation. All rights reserved. - * Copyright (C) 2019 - 2020 Intel Corporation + * Copyright (C) 2019 - 2020, 2022 Intel Corporation *****************************************************************************/ #include <linux/kernel.h> #include <linux/skbuff.h> @@ -1242,7 +1242,7 @@ static int rs_switch_to_mimo2(struct iwl_priv *priv, if (!conf_is_ht(conf) || !sta->deflink.ht_cap.ht_supported) return -1; - if (sta->smps_mode == IEEE80211_SMPS_STATIC) + if (sta->deflink.smps_mode == IEEE80211_SMPS_STATIC) return -1; /* Need both Tx chains/antennas to support MIMO */ @@ -1297,7 +1297,7 @@ static int rs_switch_to_mimo3(struct iwl_priv *priv, if (!conf_is_ht(conf) || !sta->deflink.ht_cap.ht_supported) return -1; - if (sta->smps_mode == IEEE80211_SMPS_STATIC) + if (sta->deflink.smps_mode == IEEE80211_SMPS_STATIC) return -1; /* Need both Tx chains/antennas to support MIMO */ diff --git a/drivers/net/wireless/intel/iwlwifi/dvm/scan.c b/drivers/net/wireless/intel/iwlwifi/dvm/scan.c index 2d38227dfdd2..a7e85c5c8c72 100644 --- a/drivers/net/wireless/intel/iwlwifi/dvm/scan.c +++ b/drivers/net/wireless/intel/iwlwifi/dvm/scan.c @@ -626,7 +626,7 @@ static int iwlagn_request_scan(struct iwl_priv *priv, struct ieee80211_vif *vif) u8 active_chains; u8 scan_tx_antennas = priv->nvm_data->valid_tx_ant; int ret; - int scan_cmd_size = sizeof(struct iwl_scan_cmd) + + size_t scan_cmd_size = sizeof(struct iwl_scan_cmd) + MAX_SCAN_CHANNEL * sizeof(struct iwl_scan_channel) + priv->fw->ucode_capa.max_probe_length; const u8 *ssid = NULL; @@ -649,9 +649,15 @@ static int iwlagn_request_scan(struct iwl_priv *priv, struct ieee80211_vif *vif) "fail to allocate memory for scan\n"); return -ENOMEM; } + priv->scan_cmd_size = scan_cmd_size; + } + if (priv->scan_cmd_size < scan_cmd_size) { + IWL_DEBUG_SCAN(priv, + "memory needed for scan grew unexpectedly\n"); + return -ENOMEM; } scan = priv->scan_cmd; - memset(scan, 0, scan_cmd_size); + memset(scan, 0, priv->scan_cmd_size); scan->quiet_plcp_th = IWL_PLCP_QUIET_THRESH; scan->quiet_time = IWL_ACTIVE_QUIET_TIME; diff --git a/drivers/net/wireless/intel/iwlwifi/dvm/sta.c b/drivers/net/wireless/intel/iwlwifi/dvm/sta.c index 476068c0abb7..cef43cf80620 100644 --- a/drivers/net/wireless/intel/iwlwifi/dvm/sta.c +++ b/drivers/net/wireless/intel/iwlwifi/dvm/sta.c @@ -1,7 +1,7 @@ // SPDX-License-Identifier: GPL-2.0-only /****************************************************************************** * - * Copyright(c) 2003 - 2014 Intel Corporation. All rights reserved. + * Copyright(c) 2003 - 2014, 2022 Intel Corporation. All rights reserved. * * Portions of this file are derived from the ipw3945 project, as well * as portions of the ieee80211 subsystem header files. @@ -161,12 +161,12 @@ static void iwl_sta_calc_ht_flags(struct iwl_priv *priv, IWL_DEBUG_INFO(priv, "STA %pM SM PS mode: %s\n", sta->addr, - (sta->smps_mode == IEEE80211_SMPS_STATIC) ? + (sta->deflink.smps_mode == IEEE80211_SMPS_STATIC) ? "static" : - (sta->smps_mode == IEEE80211_SMPS_DYNAMIC) ? + (sta->deflink.smps_mode == IEEE80211_SMPS_DYNAMIC) ? "dynamic" : "disabled"); - switch (sta->smps_mode) { + switch (sta->deflink.smps_mode) { case IEEE80211_SMPS_STATIC: *flags |= STA_FLG_MIMO_DIS_MSK; break; @@ -176,7 +176,7 @@ static void iwl_sta_calc_ht_flags(struct iwl_priv *priv, case IEEE80211_SMPS_OFF: break; default: - IWL_WARN(priv, "Invalid MIMO PS mode %d\n", sta->smps_mode); + IWL_WARN(priv, "Invalid MIMO PS mode %d\n", sta->deflink.smps_mode); break; } diff --git a/drivers/net/wireless/intel/iwlwifi/dvm/ucode.c b/drivers/net/wireless/intel/iwlwifi/dvm/ucode.c index 4b27a53d0bb4..bb13ca5d666c 100644 --- a/drivers/net/wireless/intel/iwlwifi/dvm/ucode.c +++ b/drivers/net/wireless/intel/iwlwifi/dvm/ucode.c @@ -356,18 +356,18 @@ static bool iwlagn_wait_calib(struct iwl_notif_wait_data *notif_wait, struct iwl_rx_packet *pkt, void *data) { struct iwl_priv *priv = data; - struct iwl_calib_hdr *hdr; + struct iwl_calib_cmd *cmd; if (pkt->hdr.cmd != CALIBRATION_RES_NOTIFICATION) { WARN_ON(pkt->hdr.cmd != CALIBRATION_COMPLETE_NOTIFICATION); return true; } - hdr = (struct iwl_calib_hdr *)pkt->data; + cmd = (struct iwl_calib_cmd *)pkt->data; - if (iwl_calib_set(priv, hdr, iwl_rx_packet_payload_len(pkt))) + if (iwl_calib_set(priv, cmd, iwl_rx_packet_payload_len(pkt))) IWL_ERR(priv, "Failed to record calibration data %d\n", - hdr->op_code); + cmd->hdr.op_code); return false; } diff --git a/drivers/net/wireless/intel/iwlwifi/fw/api/commands.h b/drivers/net/wireless/intel/iwlwifi/fw/api/commands.h index c78d2f1c722c..0b052c2e563a 100644 --- a/drivers/net/wireless/intel/iwlwifi/fw/api/commands.h +++ b/drivers/net/wireless/intel/iwlwifi/fw/api/commands.h @@ -2,7 +2,7 @@ /* * Copyright (C) 2013-2015 Intel Mobile Communications GmbH * Copyright (C) 2016-2017 Intel Deutschland GmbH - * Copyright (C) 2018-2020 Intel Corporation + * Copyright (C) 2018-2022 Intel Corporation */ #ifndef __iwl_fw_api_commands_h__ #define __iwl_fw_api_commands_h__ @@ -20,6 +20,8 @@ * &enum iwl_phy_ops_subcmd_ids * @DATA_PATH_GROUP: data path group, uses command IDs from * &enum iwl_data_path_subcmd_ids + * @SCAN_GROUP: scan group, uses command IDs from + * &enum iwl_scan_subcmd_ids * @NAN_GROUP: NAN group, uses command IDs from &enum iwl_nan_subcmd_ids * @LOCATION_GROUP: location group, uses command IDs from * &enum iwl_location_subcmd_ids @@ -36,6 +38,7 @@ enum iwl_mvm_command_groups { MAC_CONF_GROUP = 0x3, PHY_OPS_GROUP = 0x4, DATA_PATH_GROUP = 0x5, + SCAN_GROUP = 0x6, NAN_GROUP = 0x7, LOCATION_GROUP = 0x8, PROT_OFFLOAD_GROUP = 0xb, diff --git a/drivers/net/wireless/intel/iwlwifi/fw/api/d3.h b/drivers/net/wireless/intel/iwlwifi/fw/api/d3.h index 4cd9ab23954e..df0833890e55 100644 --- a/drivers/net/wireless/intel/iwlwifi/fw/api/d3.h +++ b/drivers/net/wireless/intel/iwlwifi/fw/api/d3.h @@ -1,6 +1,6 @@ /* SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause */ /* - * Copyright (C) 2012-2014, 2018-2021 Intel Corporation + * Copyright (C) 2012-2014, 2018-2022 Intel Corporation * Copyright (C) 2013-2014 Intel Mobile Communications GmbH * Copyright (C) 2015-2017 Intel Deutschland GmbH */ @@ -766,6 +766,65 @@ struct iwl_wowlan_status_v12 { u8 wake_packet[]; /* can be truncated from _length to _bufsize */ } __packed; /* WOWLAN_STATUSES_RSP_API_S_VER_12 */ +/** + * struct iwl_wowlan_info_notif - WoWLAN information notification + * @gtk: GTK data + * @igtk: IGTK data + * @replay_ctr: GTK rekey replay counter + * @pattern_number: number of the matched patterns + * @reserved1: reserved + * @qos_seq_ctr: QoS sequence counters to use next + * @wakeup_reasons: wakeup reasons, see &enum iwl_wowlan_wakeup_reason + * @num_of_gtk_rekeys: number of GTK rekeys + * @transmitted_ndps: number of transmitted neighbor discovery packets + * @received_beacons: number of received beacons + * @wake_packet_length: wakeup packet length + * @wake_packet_bufsize: wakeup packet buffer size + * @tid_tear_down: bit mask of tids whose BA sessions were closed + * in suspend state + * @station_id: station id + * @reserved2: reserved + */ +struct iwl_wowlan_info_notif { + struct iwl_wowlan_gtk_status_v3 gtk[WOWLAN_GTK_KEYS_NUM]; + struct iwl_wowlan_igtk_status igtk[WOWLAN_IGTK_KEYS_NUM]; + __le64 replay_ctr; + __le16 pattern_number; + __le16 reserved1; + __le16 qos_seq_ctr[8]; + __le32 wakeup_reasons; + __le32 num_of_gtk_rekeys; + __le32 transmitted_ndps; + __le32 received_beacons; + __le32 wake_packet_length; + __le32 wake_packet_bufsize; + u8 tid_tear_down; + u8 station_id; + u8 reserved2[2]; +} __packed; /* WOWLAN_INFO_NTFY_API_S_VER_1 */ + +/** + * struct iwl_wowlan_wake_pkt_notif - WoWLAN wake packet notification + * @wake_packet_length: wakeup packet length + * @station_id: station id + * @reserved: unused + * @wake_packet: wakeup packet + */ +struct iwl_wowlan_wake_pkt_notif { + __le32 wake_packet_length; + u8 station_id; + u8 reserved[3]; + u8 wake_packet[1]; +} __packed; /* WOWLAN_WAKE_PKT_NTFY_API_S_VER_1 */ + +/** + * struct iwl_mvm_d3_end_notif - d3 end notification + * @flags: See &enum iwl_d0i3_flags + */ +struct iwl_mvm_d3_end_notif { + __le32 flags; +} __packed; + /* TODO: NetDetect API */ #endif /* __iwl_fw_api_d3_h__ */ diff --git a/drivers/net/wireless/intel/iwlwifi/fw/api/offload.h b/drivers/net/wireless/intel/iwlwifi/fw/api/offload.h index 5204aa94e72a..a0123f81f5d8 100644 --- a/drivers/net/wireless/intel/iwlwifi/fw/api/offload.h +++ b/drivers/net/wireless/intel/iwlwifi/fw/api/offload.h @@ -3,7 +3,7 @@ * Copyright (C) 2012-2014 Intel Corporation * Copyright (C) 2013-2015 Intel Mobile Communications GmbH * Copyright (C) 2016-2017 Intel Deutschland GmbH - * Copyright (C) 2021 Intel Corporation + * Copyright (C) 2021-2022 Intel Corporation */ #ifndef __iwl_fw_api_offload_h__ #define __iwl_fw_api_offload_h__ @@ -13,6 +13,21 @@ */ enum iwl_prot_offload_subcmd_ids { /** + * @WOWLAN_WAKE_PKT_NOTIFICATION: Notification in &struct iwl_wowlan_wake_pkt_notif + */ + WOWLAN_WAKE_PKT_NOTIFICATION = 0xFC, + + /** + * @WOWLAN_INFO_NOTIFICATION: Notification in &struct iwl_wowlan_info_notif + */ + WOWLAN_INFO_NOTIFICATION = 0xFD, + + /** + * @D3_END_NOTIFICATION: End D3 state notification + */ + D3_END_NOTIFICATION = 0xFE, + + /** * @STORED_BEACON_NTF: &struct iwl_stored_beacon_notif */ STORED_BEACON_NTF = 0xFF, diff --git a/drivers/net/wireless/intel/iwlwifi/fw/api/rx.h b/drivers/net/wireless/intel/iwlwifi/fw/api/rx.h index 1989b270862b..74a01888715b 100644 --- a/drivers/net/wireless/intel/iwlwifi/fw/api/rx.h +++ b/drivers/net/wireless/intel/iwlwifi/fw/api/rx.h @@ -1,6 +1,6 @@ /* SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause */ /* - * Copyright (C) 2012-2014, 2018-2021 Intel Corporation + * Copyright (C) 2012-2014, 2018-2022 Intel Corporation * Copyright (C) 2013-2015 Intel Mobile Communications GmbH * Copyright (C) 2015-2017 Intel Deutschland GmbH */ @@ -668,7 +668,7 @@ struct iwl_rx_no_data { __le32 phy_info[2]; __le32 rx_vec[2]; } __packed; /* RX_NO_DATA_NTFY_API_S_VER_1, - TX_NO_DATA_NTFY_API_S_VER_2 */ + RX_NO_DATA_NTFY_API_S_VER_2 */ struct iwl_frame_release { u8 baid; diff --git a/drivers/net/wireless/intel/iwlwifi/fw/api/scan.h b/drivers/net/wireless/intel/iwlwifi/fw/api/scan.h index 5543d9cb74c8..7ba0e3409199 100644 --- a/drivers/net/wireless/intel/iwlwifi/fw/api/scan.h +++ b/drivers/net/wireless/intel/iwlwifi/fw/api/scan.h @@ -1,6 +1,6 @@ /* SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause */ /* - * Copyright (C) 2012-2014, 2018-2021 Intel Corporation + * Copyright (C) 2012-2014, 2018-2022 Intel Corporation * Copyright (C) 2013-2015 Intel Mobile Communications GmbH * Copyright (C) 2016-2017 Intel Deutschland GmbH */ @@ -9,6 +9,16 @@ /* Scan Commands, Responses, Notifications */ +/** + * enum iwl_scan_subcmd_ids - scan commands + */ +enum iwl_scan_subcmd_ids { + /** + * @OFFLOAD_MATCH_INFO_NOTIF: &struct iwl_scan_offload_match_info + */ + OFFLOAD_MATCH_INFO_NOTIF = 0xFC, +}; + /* Max number of IEs for direct SSID scans in a command */ #define PROBE_OPTION_MAX 20 @@ -1188,7 +1198,7 @@ struct iwl_scan_offload_profile_match { } __packed; /* SCAN_OFFLOAD_PROFILE_MATCH_RESULTS_S_VER_2 */ /** - * struct iwl_scan_offload_profiles_query - match results query response + * struct iwl_scan_offload_match_info - match results information * @matched_profiles: bitmap of matched profiles, referencing the * matches passed in the scan offload request * @last_scan_age: age of the last offloaded scan @@ -1200,7 +1210,7 @@ struct iwl_scan_offload_profile_match { * @reserved: reserved * @matches: array of match information, one for each match */ -struct iwl_scan_offload_profiles_query { +struct iwl_scan_offload_match_info { __le32 matched_profiles; __le32 last_scan_age; __le32 n_scans_done; @@ -1210,7 +1220,9 @@ struct iwl_scan_offload_profiles_query { u8 self_recovery; __le16 reserved; struct iwl_scan_offload_profile_match matches[]; -} __packed; /* SCAN_OFFLOAD_PROFILES_QUERY_RSP_S_VER_3 */ +} __packed; /* SCAN_OFFLOAD_PROFILES_QUERY_RSP_S_VER_3 and + * SCAN_OFFLOAD_MATCH_INFO_NOTIFICATION_S_VER_1 + */ /** * struct iwl_umac_scan_iter_complete_notif - notifies end of scanning iteration diff --git a/drivers/net/wireless/intel/iwlwifi/iwl-config.h b/drivers/net/wireless/intel/iwlwifi/iwl-config.h index f5b556a103e8..cfa5e1b3c3f6 100644 --- a/drivers/net/wireless/intel/iwlwifi/iwl-config.h +++ b/drivers/net/wireless/intel/iwlwifi/iwl-config.h @@ -649,13 +649,16 @@ extern const struct iwl_cfg iwl_cfg_bz_a0_gf_a0; extern const struct iwl_cfg iwl_cfg_bz_a0_gf4_a0; extern const struct iwl_cfg iwl_cfg_bz_a0_mr_a0; extern const struct iwl_cfg iwl_cfg_bz_a0_fm_a0; +extern const struct iwl_cfg iwl_cfg_bz_a0_fm4_a0; extern const struct iwl_cfg iwl_cfg_gl_a0_fm_a0; +extern const struct iwl_cfg iwl_cfg_gl_b0_fm_b0; extern const struct iwl_cfg iwl_cfg_bz_z0_gf_a0; extern const struct iwl_cfg iwl_cfg_bnj_a0_fm_a0; extern const struct iwl_cfg iwl_cfg_bnj_a0_fm4_a0; extern const struct iwl_cfg iwl_cfg_bnj_a0_gf_a0; extern const struct iwl_cfg iwl_cfg_bnj_a0_gf4_a0; extern const struct iwl_cfg iwl_cfg_bnj_a0_hr_b0; +extern const struct iwl_cfg iwl_cfg_bnj_b0_fm_b0; #endif /* CONFIG_IWLMVM */ #endif /* __IWL_CONFIG_H__ */ diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/d3.c b/drivers/net/wireless/intel/iwlwifi/mvm/d3.c index aeb0015b73d2..919b1f478b4c 100644 --- a/drivers/net/wireless/intel/iwlwifi/mvm/d3.c +++ b/drivers/net/wireless/intel/iwlwifi/mvm/d3.c @@ -1427,7 +1427,7 @@ struct iwl_wowlan_status_data { u8 flags; } igtk; - u8 wake_packet[]; + u8 *wake_packet; }; static void iwl_mvm_report_wakeup_reasons(struct iwl_mvm *mvm, @@ -1480,7 +1480,7 @@ static void iwl_mvm_report_wakeup_reasons(struct iwl_mvm *mvm, if (reasons & IWL_WOWLAN_WAKEUP_BY_REM_WAKE_WAKEUP_PACKET) wakeup.tcp_match = true; - if (status->wake_packet_bufsize) { + if (status->wake_packet) { int pktsize = status->wake_packet_bufsize; int pktlen = status->wake_packet_length; const u8 *pktdata = status->wake_packet; @@ -1944,57 +1944,6 @@ out: return true; } -/* Occasionally, templates would be nice. This is one of those times ... */ -#define iwl_mvm_parse_wowlan_status_common(_ver) \ -static struct iwl_wowlan_status_data * \ -iwl_mvm_parse_wowlan_status_common_ ## _ver(struct iwl_mvm *mvm, \ - struct iwl_wowlan_status_ ##_ver *data,\ - int len) \ -{ \ - struct iwl_wowlan_status_data *status; \ - int data_size, i; \ - \ - if (len < sizeof(*data)) { \ - IWL_ERR(mvm, "Invalid WoWLAN status response!\n"); \ - return NULL; \ - } \ - \ - data_size = ALIGN(le32_to_cpu(data->wake_packet_bufsize), 4); \ - if (len != sizeof(*data) + data_size) { \ - IWL_ERR(mvm, "Invalid WoWLAN status response!\n"); \ - return NULL; \ - } \ - \ - status = kzalloc(sizeof(*status) + data_size, GFP_KERNEL); \ - if (!status) \ - return NULL; \ - \ - /* copy all the common fields */ \ - status->replay_ctr = le64_to_cpu(data->replay_ctr); \ - status->pattern_number = le16_to_cpu(data->pattern_number); \ - status->non_qos_seq_ctr = le16_to_cpu(data->non_qos_seq_ctr); \ - for (i = 0; i < 8; i++) \ - status->qos_seq_ctr[i] = \ - le16_to_cpu(data->qos_seq_ctr[i]); \ - status->wakeup_reasons = le32_to_cpu(data->wakeup_reasons); \ - status->num_of_gtk_rekeys = \ - le32_to_cpu(data->num_of_gtk_rekeys); \ - status->received_beacons = le32_to_cpu(data->received_beacons); \ - status->wake_packet_length = \ - le32_to_cpu(data->wake_packet_length); \ - status->wake_packet_bufsize = \ - le32_to_cpu(data->wake_packet_bufsize); \ - memcpy(status->wake_packet, data->wake_packet, \ - status->wake_packet_bufsize); \ - \ - return status; \ -} - -iwl_mvm_parse_wowlan_status_common(v6) -iwl_mvm_parse_wowlan_status_common(v7) -iwl_mvm_parse_wowlan_status_common(v9) -iwl_mvm_parse_wowlan_status_common(v12) - static void iwl_mvm_convert_gtk_v2(struct iwl_wowlan_status_data *status, struct iwl_wowlan_gtk_status_v2 *data) { @@ -2054,6 +2003,96 @@ static void iwl_mvm_convert_igtk(struct iwl_wowlan_status_data *status, ((u64)ipn[0] << 40); } +static void iwl_mvm_parse_wowlan_info_notif(struct iwl_mvm *mvm, + struct iwl_wowlan_info_notif *data, + struct iwl_wowlan_status_data *status, + u32 len) +{ + u32 i; + + if (len < sizeof(*data)) { + IWL_ERR(mvm, "Invalid WoWLAN info notification!\n"); + status = NULL; + return; + } + + iwl_mvm_convert_key_counters_v5(status, &data->gtk[0].sc); + iwl_mvm_convert_gtk_v3(status, &data->gtk[0]); + iwl_mvm_convert_igtk(status, &data->igtk[0]); + + status->replay_ctr = le64_to_cpu(data->replay_ctr); + status->pattern_number = le16_to_cpu(data->pattern_number); + for (i = 0; i < IWL_MAX_TID_COUNT; i++) + status->qos_seq_ctr[i] = + le16_to_cpu(data->qos_seq_ctr[i]); + status->wakeup_reasons = le32_to_cpu(data->wakeup_reasons); + status->num_of_gtk_rekeys = + le32_to_cpu(data->num_of_gtk_rekeys); + status->received_beacons = le32_to_cpu(data->received_beacons); + status->tid_tear_down = data->tid_tear_down; +} + +/* Occasionally, templates would be nice. This is one of those times ... */ +#define iwl_mvm_parse_wowlan_status_common(_ver) \ +static struct iwl_wowlan_status_data * \ +iwl_mvm_parse_wowlan_status_common_ ## _ver(struct iwl_mvm *mvm, \ + struct iwl_wowlan_status_ ##_ver *data,\ + int len) \ +{ \ + struct iwl_wowlan_status_data *status; \ + int data_size, i; \ + \ + if (len < sizeof(*data)) { \ + IWL_ERR(mvm, "Invalid WoWLAN status response!\n"); \ + return NULL; \ + } \ + \ + data_size = ALIGN(le32_to_cpu(data->wake_packet_bufsize), 4); \ + if (len != sizeof(*data) + data_size) { \ + IWL_ERR(mvm, "Invalid WoWLAN status response!\n"); \ + return NULL; \ + } \ + \ + status = kzalloc(sizeof(*status), GFP_KERNEL); \ + if (!status) \ + return NULL; \ + \ + /* copy all the common fields */ \ + status->replay_ctr = le64_to_cpu(data->replay_ctr); \ + status->pattern_number = le16_to_cpu(data->pattern_number); \ + status->non_qos_seq_ctr = le16_to_cpu(data->non_qos_seq_ctr); \ + for (i = 0; i < 8; i++) \ + status->qos_seq_ctr[i] = \ + le16_to_cpu(data->qos_seq_ctr[i]); \ + status->wakeup_reasons = le32_to_cpu(data->wakeup_reasons); \ + status->num_of_gtk_rekeys = \ + le32_to_cpu(data->num_of_gtk_rekeys); \ + status->received_beacons = le32_to_cpu(data->received_beacons); \ + status->wake_packet_length = \ + le32_to_cpu(data->wake_packet_length); \ + status->wake_packet_bufsize = \ + le32_to_cpu(data->wake_packet_bufsize); \ + if (status->wake_packet_bufsize) { \ + status->wake_packet = \ + kmemdup(data->wake_packet, \ + status->wake_packet_bufsize, \ + GFP_KERNEL); \ + if (!status->wake_packet) { \ + kfree(status); \ + return NULL; \ + } \ + } else { \ + status->wake_packet = NULL; \ + } \ + \ + return status; \ +} + +iwl_mvm_parse_wowlan_status_common(v6) +iwl_mvm_parse_wowlan_status_common(v7) +iwl_mvm_parse_wowlan_status_common(v9) +iwl_mvm_parse_wowlan_status_common(v12) + static struct iwl_wowlan_status_data * iwl_mvm_send_wowlan_get_status(struct iwl_mvm *mvm, u8 sta_id) { @@ -2173,36 +2212,15 @@ out_free_resp: return status; } -static struct iwl_wowlan_status_data * -iwl_mvm_get_wakeup_status(struct iwl_mvm *mvm, u8 sta_id) -{ - u8 cmd_ver = iwl_fw_lookup_cmd_ver(mvm->fw, OFFLOADS_QUERY_CMD, - IWL_FW_CMD_VER_UNKNOWN); - __le32 station_id = cpu_to_le32(sta_id); - u32 cmd_size = cmd_ver != IWL_FW_CMD_VER_UNKNOWN ? sizeof(station_id) : 0; - - if (!mvm->net_detect) { - /* only for tracing for now */ - int ret = iwl_mvm_send_cmd_pdu(mvm, OFFLOADS_QUERY_CMD, 0, - cmd_size, &station_id); - if (ret) - IWL_ERR(mvm, "failed to query offload statistics (%d)\n", ret); - } - - return iwl_mvm_send_wowlan_get_status(mvm, sta_id); -} - /* releases the MVM mutex */ static bool iwl_mvm_query_wakeup_reasons(struct iwl_mvm *mvm, - struct ieee80211_vif *vif) + struct ieee80211_vif *vif, + struct iwl_wowlan_status_data *status) { - struct iwl_mvm_vif *mvmvif = iwl_mvm_vif_from_mac80211(vif); - struct iwl_wowlan_status_data *status; int i; bool keep; struct iwl_mvm_sta *mvm_ap_sta; - status = iwl_mvm_get_wakeup_status(mvm, mvmvif->ap_sta_id); if (!status) goto out_unlock; @@ -2212,7 +2230,7 @@ static bool iwl_mvm_query_wakeup_reasons(struct iwl_mvm *mvm, /* still at hard-coded place 0 for D3 image */ mvm_ap_sta = iwl_mvm_sta_from_staid_protected(mvm, 0); if (!mvm_ap_sta) - goto out_free; + goto out_unlock; for (i = 0; i < IWL_MAX_TID_COUNT; i++) { u16 seq = status->qos_seq_ctr[i]; @@ -2235,11 +2253,8 @@ static bool iwl_mvm_query_wakeup_reasons(struct iwl_mvm *mvm, keep = iwl_mvm_setup_connection_keep(mvm, vif, status); - kfree(status); return keep; -out_free: - kfree(status); out_unlock: mutex_unlock(&mvm->mutex); return false; @@ -2248,16 +2263,16 @@ out_unlock: #define ND_QUERY_BUF_LEN (sizeof(struct iwl_scan_offload_profile_match) * \ IWL_SCAN_MAX_PROFILES) -struct iwl_mvm_nd_query_results { +struct iwl_mvm_nd_results { u32 matched_profiles; u8 matches[ND_QUERY_BUF_LEN]; }; static int iwl_mvm_netdetect_query_results(struct iwl_mvm *mvm, - struct iwl_mvm_nd_query_results *results) + struct iwl_mvm_nd_results *results) { - struct iwl_scan_offload_profiles_query *query; + struct iwl_scan_offload_match_info *query; struct iwl_host_cmd cmd = { .id = SCAN_OFFLOAD_PROFILES_QUERY_CMD, .flags = CMD_WANT_SKB, @@ -2274,7 +2289,7 @@ iwl_mvm_netdetect_query_results(struct iwl_mvm *mvm, if (fw_has_api(&mvm->fw->ucode_capa, IWL_UCODE_TLV_API_SCAN_OFFLOAD_CHANS)) { - query_len = sizeof(struct iwl_scan_offload_profiles_query); + query_len = sizeof(struct iwl_scan_offload_match_info); matches_len = sizeof(struct iwl_scan_offload_profile_match) * max_profiles; } else { @@ -2305,7 +2320,7 @@ out_free_resp: } static int iwl_mvm_query_num_match_chans(struct iwl_mvm *mvm, - struct iwl_mvm_nd_query_results *query, + struct iwl_mvm_nd_results *results, int idx) { int n_chans = 0, i; @@ -2313,13 +2328,13 @@ static int iwl_mvm_query_num_match_chans(struct iwl_mvm *mvm, if (fw_has_api(&mvm->fw->ucode_capa, IWL_UCODE_TLV_API_SCAN_OFFLOAD_CHANS)) { struct iwl_scan_offload_profile_match *matches = - (struct iwl_scan_offload_profile_match *)query->matches; + (void *)results->matches; for (i = 0; i < SCAN_OFFLOAD_MATCHING_CHANNELS_LEN; i++) n_chans += hweight8(matches[idx].matching_channels[i]); } else { struct iwl_scan_offload_profile_match_v1 *matches = - (struct iwl_scan_offload_profile_match_v1 *)query->matches; + (void *)results->matches; for (i = 0; i < SCAN_OFFLOAD_MATCHING_CHANNELS_LEN_V1; i++) n_chans += hweight8(matches[idx].matching_channels[i]); @@ -2329,7 +2344,7 @@ static int iwl_mvm_query_num_match_chans(struct iwl_mvm *mvm, } static void iwl_mvm_query_set_freqs(struct iwl_mvm *mvm, - struct iwl_mvm_nd_query_results *query, + struct iwl_mvm_nd_results *results, struct cfg80211_wowlan_nd_match *match, int idx) { @@ -2338,7 +2353,7 @@ static void iwl_mvm_query_set_freqs(struct iwl_mvm *mvm, if (fw_has_api(&mvm->fw->ucode_capa, IWL_UCODE_TLV_API_SCAN_OFFLOAD_CHANS)) { struct iwl_scan_offload_profile_match *matches = - (struct iwl_scan_offload_profile_match *)query->matches; + (void *)results->matches; for (i = 0; i < SCAN_OFFLOAD_MATCHING_CHANNELS_LEN * 8; i++) if (matches[idx].matching_channels[i / 8] & (BIT(i % 8))) @@ -2346,7 +2361,7 @@ static void iwl_mvm_query_set_freqs(struct iwl_mvm *mvm, mvm->nd_channels[i]->center_freq; } else { struct iwl_scan_offload_profile_match_v1 *matches = - (struct iwl_scan_offload_profile_match_v1 *)query->matches; + (void *)results->matches; for (i = 0; i < SCAN_OFFLOAD_MATCHING_CHANNELS_LEN_V1 * 8; i++) if (matches[idx].matching_channels[i / 8] & (BIT(i % 8))) @@ -2355,25 +2370,50 @@ static void iwl_mvm_query_set_freqs(struct iwl_mvm *mvm, } } +/** + * enum iwl_d3_notif - d3 notifications + * @IWL_D3_NOTIF_WOWLAN_INFO: WOWLAN_INFO_NOTIF was received + * @IWL_D3_NOTIF_WOWLAN_WAKE_PKT: WOWLAN_WAKE_PKT_NOTIF was received + * @IWL_D3_NOTIF_PROT_OFFLOAD: PROT_OFFLOAD_NOTIF was received + * @IWL_D3_ND_MATCH_INFO: OFFLOAD_MATCH_INFO_NOTIF was received + * @IWL_D3_NOTIF_D3_END_NOTIF: D3_END_NOTIF was received + */ +enum iwl_d3_notif { + IWL_D3_NOTIF_WOWLAN_INFO = BIT(0), + IWL_D3_NOTIF_WOWLAN_WAKE_PKT = BIT(1), + IWL_D3_NOTIF_PROT_OFFLOAD = BIT(2), + IWL_D3_ND_MATCH_INFO = BIT(3), + IWL_D3_NOTIF_D3_END_NOTIF = BIT(4) +}; + +/* manage d3 resume data */ +struct iwl_d3_data { + struct iwl_wowlan_status_data *status; + bool test; + u32 d3_end_flags; + u32 notif_expected; /* bitmap - see &enum iwl_d3_notif */ + u32 notif_received; /* bitmap - see &enum iwl_d3_notif */ + struct iwl_mvm_nd_results *nd_results; + bool nd_results_valid; +}; + static void iwl_mvm_query_netdetect_reasons(struct iwl_mvm *mvm, - struct ieee80211_vif *vif) + struct ieee80211_vif *vif, + struct iwl_d3_data *d3_data) { struct cfg80211_wowlan_nd_info *net_detect = NULL; struct cfg80211_wowlan_wakeup wakeup = { .pattern_idx = -1, }; struct cfg80211_wowlan_wakeup *wakeup_report = &wakeup; - struct iwl_wowlan_status_data *status; - struct iwl_mvm_nd_query_results query; unsigned long matched_profiles; u32 reasons = 0; int i, n_matches, ret; - status = iwl_mvm_get_wakeup_status(mvm, IWL_MVM_INVALID_STA); - if (status) { - reasons = status->wakeup_reasons; - kfree(status); - } + if (WARN_ON(!d3_data || !d3_data->status)) + goto out; + + reasons = d3_data->status->wakeup_reasons; if (reasons & IWL_WOWLAN_WAKEUP_BY_RFKILL_DEASSERTED) wakeup.rfkill_release = true; @@ -2381,13 +2421,22 @@ static void iwl_mvm_query_netdetect_reasons(struct iwl_mvm *mvm, if (reasons != IWL_WOWLAN_WAKEUP_BY_NON_WIRELESS) goto out; - ret = iwl_mvm_netdetect_query_results(mvm, &query); - if (ret || !query.matched_profiles) { + if (!iwl_fw_lookup_notif_ver(mvm->fw, PROT_OFFLOAD_GROUP, + WOWLAN_INFO_NOTIFICATION, 0)) { + IWL_INFO(mvm, "Query FW for ND results\n"); + ret = iwl_mvm_netdetect_query_results(mvm, d3_data->nd_results); + + } else { + IWL_INFO(mvm, "Notification based ND results\n"); + ret = d3_data->nd_results_valid ? 0 : -1; + } + + if (ret || !d3_data->nd_results->matched_profiles) { wakeup_report = NULL; goto out; } - matched_profiles = query.matched_profiles; + matched_profiles = d3_data->nd_results->matched_profiles; if (mvm->n_nd_match_sets) { n_matches = hweight_long(matched_profiles); } else { @@ -2404,7 +2453,9 @@ static void iwl_mvm_query_netdetect_reasons(struct iwl_mvm *mvm, struct cfg80211_wowlan_nd_match *match; int idx, n_channels = 0; - n_channels = iwl_mvm_query_num_match_chans(mvm, &query, i); + n_channels = iwl_mvm_query_num_match_chans(mvm, + d3_data->nd_results, + i); match = kzalloc(struct_size(match, channels, n_channels), GFP_KERNEL); @@ -2424,7 +2475,7 @@ static void iwl_mvm_query_netdetect_reasons(struct iwl_mvm *mvm, if (mvm->n_nd_channels < n_channels) continue; - iwl_mvm_query_set_freqs(mvm, &query, match, i); + iwl_mvm_query_set_freqs(mvm, d3_data->nd_results, match, i); } out_report_nd: @@ -2504,16 +2555,317 @@ static bool iwl_mvm_check_rt_status(struct iwl_mvm *mvm, return false; } +/* + * This function assumes: + * 1. The mutex is already held. + * 2. The callee functions unlock the mutex. + */ +static bool +iwl_mvm_choose_query_wakeup_reasons(struct iwl_mvm *mvm, + struct ieee80211_vif *vif, + struct iwl_d3_data *d3_data) +{ + lockdep_assert_held(&mvm->mutex); + + /* if FW uses status notification, status shouldn't be NULL here */ + if (!d3_data->status) { + struct iwl_mvm_vif *mvmvif = iwl_mvm_vif_from_mac80211(vif); + u8 sta_id = mvm->net_detect ? IWL_MVM_INVALID_STA : mvmvif->ap_sta_id; + + d3_data->status = iwl_mvm_send_wowlan_get_status(mvm, sta_id); + } + + if (mvm->net_detect) { + iwl_mvm_query_netdetect_reasons(mvm, vif, d3_data); + } else { + bool keep = iwl_mvm_query_wakeup_reasons(mvm, vif, + d3_data->status); + +#ifdef CONFIG_IWLWIFI_DEBUGFS + if (keep) + mvm->keep_vif = vif; +#endif + + return keep; + } + return false; +} + +#define IWL_WOWLAN_WAKEUP_REASON_HAS_WAKEUP_PKT (IWL_WOWLAN_WAKEUP_BY_MAGIC_PACKET | \ + IWL_WOWLAN_WAKEUP_BY_PATTERN | \ + IWL_WAKEUP_BY_PATTERN_IPV4_TCP_SYN |\ + IWL_WAKEUP_BY_PATTERN_IPV4_TCP_SYN_WILDCARD |\ + IWL_WAKEUP_BY_PATTERN_IPV6_TCP_SYN |\ + IWL_WAKEUP_BY_PATTERN_IPV6_TCP_SYN_WILDCARD) + +static int iwl_mvm_wowlan_store_wake_pkt(struct iwl_mvm *mvm, + struct iwl_wowlan_wake_pkt_notif *notif, + struct iwl_wowlan_status_data *status, + u32 len) +{ + u32 data_size, packet_len = le32_to_cpu(notif->wake_packet_length); + + if (len < sizeof(*notif)) { + IWL_ERR(mvm, "Invalid WoWLAN wake packet notification!\n"); + return -EIO; + } + + if (WARN_ON(!status)) { + IWL_ERR(mvm, "Got wake packet notification but wowlan status data is NULL\n"); + return -EIO; + } + + if (WARN_ON(!(status->wakeup_reasons & + IWL_WOWLAN_WAKEUP_REASON_HAS_WAKEUP_PKT))) { + IWL_ERR(mvm, "Got wakeup packet but wakeup reason is %x\n", + status->wakeup_reasons); + return -EIO; + } + + data_size = len - offsetof(struct iwl_wowlan_wake_pkt_notif, wake_packet); + + /* data_size got the padding from the notification, remove it. */ + if (packet_len < data_size) + data_size = packet_len; + + status->wake_packet = kmemdup(notif->wake_packet, data_size, + GFP_ATOMIC); + + if (!status->wake_packet) + return -ENOMEM; + + status->wake_packet_length = packet_len; + status->wake_packet_bufsize = data_size; + + return 0; +} + +static void iwl_mvm_nd_match_info_handler(struct iwl_mvm *mvm, + struct iwl_d3_data *d3_data, + struct iwl_scan_offload_match_info *notif, + u32 len) +{ + struct iwl_wowlan_status_data *status = d3_data->status; + struct ieee80211_vif *vif = iwl_mvm_get_bss_vif(mvm); + struct iwl_mvm_nd_results *results = d3_data->nd_results; + size_t i, matches_len = sizeof(struct iwl_scan_offload_profile_match) * + iwl_umac_scan_get_max_profiles(mvm->fw); + + if (IS_ERR_OR_NULL(vif)) + return; + + if (len < sizeof(struct iwl_scan_offload_match_info)) { + IWL_ERR(mvm, "Invalid scan match info notification\n"); + return; + } + + if (!mvm->net_detect) { + IWL_ERR(mvm, "Unexpected scan match info notification\n"); + return; + } + + if (!status || status->wakeup_reasons != IWL_WOWLAN_WAKEUP_BY_NON_WIRELESS) { + IWL_ERR(mvm, + "Ignore scan match info notification: no reason\n"); + return; + } + +#ifdef CONFIG_IWLWIFI_DEBUGFS + mvm->last_netdetect_scans = le32_to_cpu(notif->n_scans_done); +#endif + + results->matched_profiles = le32_to_cpu(notif->matched_profiles); + IWL_INFO(mvm, "number of matched profiles=%u\n", + results->matched_profiles); + + if (results->matched_profiles) { + memcpy(results->matches, notif->matches, matches_len); + d3_data->nd_results_valid = TRUE; + } + + /* no scan should be active at this point */ + mvm->scan_status = 0; + for (i = 0; i < mvm->max_scans; i++) + mvm->scan_uid_status[i] = 0; +} + +static bool iwl_mvm_wait_d3_notif(struct iwl_notif_wait_data *notif_wait, + struct iwl_rx_packet *pkt, void *data) +{ + struct iwl_mvm *mvm = + container_of(notif_wait, struct iwl_mvm, notif_wait); + struct iwl_d3_data *d3_data = data; + u32 len; + int ret; + + switch (WIDE_ID(pkt->hdr.group_id, pkt->hdr.cmd)) { + case WIDE_ID(PROT_OFFLOAD_GROUP, WOWLAN_INFO_NOTIFICATION): { + struct iwl_wowlan_info_notif *notif = (void *)pkt->data; + + if (d3_data->notif_received & IWL_D3_NOTIF_WOWLAN_INFO) { + /* We might get two notifications due to dual bss */ + IWL_DEBUG_WOWLAN(mvm, + "Got additional wowlan info notification\n"); + break; + } + + d3_data->notif_received |= IWL_D3_NOTIF_WOWLAN_INFO; + len = iwl_rx_packet_payload_len(pkt); + iwl_mvm_parse_wowlan_info_notif(mvm, notif, d3_data->status, + len); + if (d3_data->status && + d3_data->status->wakeup_reasons & IWL_WOWLAN_WAKEUP_REASON_HAS_WAKEUP_PKT) + /* We are supposed to get also wake packet notif */ + d3_data->notif_expected |= IWL_D3_NOTIF_WOWLAN_WAKE_PKT; + + break; + } + case WIDE_ID(PROT_OFFLOAD_GROUP, WOWLAN_WAKE_PKT_NOTIFICATION): { + struct iwl_wowlan_wake_pkt_notif *notif = (void *)pkt->data; + + if (d3_data->notif_received & IWL_D3_NOTIF_WOWLAN_WAKE_PKT) { + /* We shouldn't get two wake packet notifications */ + IWL_ERR(mvm, + "Got additional wowlan wake packet notification\n"); + } else { + d3_data->notif_received |= IWL_D3_NOTIF_WOWLAN_WAKE_PKT; + len = iwl_rx_packet_payload_len(pkt); + ret = iwl_mvm_wowlan_store_wake_pkt(mvm, notif, + d3_data->status, + len); + if (ret) + IWL_ERR(mvm, + "Can't parse WOWLAN_WAKE_PKT_NOTIFICATION\n"); + } + + break; + } + case WIDE_ID(SCAN_GROUP, OFFLOAD_MATCH_INFO_NOTIF): { + struct iwl_scan_offload_match_info *notif = (void *)pkt->data; + + if (d3_data->notif_received & IWL_D3_ND_MATCH_INFO) { + IWL_ERR(mvm, + "Got additional netdetect match info\n"); + break; + } + + d3_data->notif_received |= IWL_D3_ND_MATCH_INFO; + + /* explicitly set this in the 'expected' as well */ + d3_data->notif_expected |= IWL_D3_ND_MATCH_INFO; + + len = iwl_rx_packet_payload_len(pkt); + iwl_mvm_nd_match_info_handler(mvm, d3_data, notif, len); + break; + } + case WIDE_ID(PROT_OFFLOAD_GROUP, D3_END_NOTIFICATION): { + struct iwl_mvm_d3_end_notif *notif = (void *)pkt->data; + + d3_data->d3_end_flags = __le32_to_cpu(notif->flags); + d3_data->notif_received |= IWL_D3_NOTIF_D3_END_NOTIF; + + break; + } + default: + WARN_ON(1); + } + + return d3_data->notif_received == d3_data->notif_expected; +} + +static int iwl_mvm_resume_firmware(struct iwl_mvm *mvm, bool test) +{ + int ret; + enum iwl_d3_status d3_status; + struct iwl_host_cmd cmd = { + .id = D0I3_END_CMD, + .flags = CMD_WANT_SKB | CMD_SEND_IN_D3, + }; + bool reset = fw_has_capa(&mvm->fw->ucode_capa, + IWL_UCODE_TLV_CAPA_CNSLDTD_D3_D0_IMG); + + ret = iwl_trans_d3_resume(mvm->trans, &d3_status, test, !reset); + if (ret) + return ret; + + if (d3_status != IWL_D3_STATUS_ALIVE) { + IWL_INFO(mvm, "Device was reset during suspend\n"); + return -ENOENT; + } + + /* + * We should trigger resume flow using command only for 22000 family + * AX210 and above don't need the command since they have + * the doorbell interrupt. + */ + if (mvm->trans->trans_cfg->device_family <= IWL_DEVICE_FAMILY_22000 && + fw_has_capa(&mvm->fw->ucode_capa, IWL_UCODE_TLV_CAPA_D0I3_END_FIRST)) { + ret = iwl_mvm_send_cmd(mvm, &cmd); + if (ret < 0) + IWL_ERR(mvm, "Failed to send D0I3_END_CMD first (%d)\n", + ret); + } + + return ret; +} + +#define IWL_MVM_D3_NOTIF_TIMEOUT (HZ / 5) + +static int iwl_mvm_d3_notif_wait(struct iwl_mvm *mvm, + struct iwl_d3_data *d3_data) +{ + static const u16 d3_resume_notif[] = { + WIDE_ID(PROT_OFFLOAD_GROUP, WOWLAN_INFO_NOTIFICATION), + WIDE_ID(PROT_OFFLOAD_GROUP, WOWLAN_WAKE_PKT_NOTIFICATION), + WIDE_ID(SCAN_GROUP, OFFLOAD_MATCH_INFO_NOTIF), + WIDE_ID(PROT_OFFLOAD_GROUP, D3_END_NOTIFICATION) + }; + struct iwl_notification_wait wait_d3_notif; + int ret; + + iwl_init_notification_wait(&mvm->notif_wait, &wait_d3_notif, + d3_resume_notif, ARRAY_SIZE(d3_resume_notif), + iwl_mvm_wait_d3_notif, d3_data); + + ret = iwl_mvm_resume_firmware(mvm, d3_data->test); + if (ret) { + iwl_remove_notification(&mvm->notif_wait, &wait_d3_notif); + return ret; + } + + return iwl_wait_notification(&mvm->notif_wait, &wait_d3_notif, + IWL_MVM_D3_NOTIF_TIMEOUT); +} + +static inline bool iwl_mvm_d3_resume_notif_based(struct iwl_mvm *mvm) +{ + return iwl_fw_lookup_notif_ver(mvm->fw, PROT_OFFLOAD_GROUP, + WOWLAN_INFO_NOTIFICATION, 0) && + iwl_fw_lookup_notif_ver(mvm->fw, PROT_OFFLOAD_GROUP, + WOWLAN_WAKE_PKT_NOTIFICATION, 0) && + iwl_fw_lookup_notif_ver(mvm->fw, PROT_OFFLOAD_GROUP, + D3_END_NOTIFICATION, 0); +} + static int __iwl_mvm_resume(struct iwl_mvm *mvm, bool test) { struct ieee80211_vif *vif = NULL; int ret = 1; - enum iwl_d3_status d3_status; - bool keep = false; + struct iwl_mvm_nd_results results = {}; + struct iwl_d3_data d3_data = { + .test = test, + .notif_expected = + IWL_D3_NOTIF_WOWLAN_INFO | + IWL_D3_NOTIF_D3_END_NOTIF, + .nd_results_valid = false, + .nd_results = &results, + }; bool unified_image = fw_has_capa(&mvm->fw->ucode_capa, IWL_UCODE_TLV_CAPA_CNSLDTD_D3_D0_IMG); bool d0i3_first = fw_has_capa(&mvm->fw->ucode_capa, IWL_UCODE_TLV_CAPA_D0I3_END_FIRST); + bool resume_notif_based = iwl_mvm_d3_resume_notif_based(mvm); + bool keep = false; mutex_lock(&mvm->mutex); @@ -2537,54 +2889,30 @@ static int __iwl_mvm_resume(struct iwl_mvm *mvm, bool test) goto err; } - ret = iwl_trans_d3_resume(mvm->trans, &d3_status, test, !unified_image); - if (ret) - goto err; - - if (d3_status != IWL_D3_STATUS_ALIVE) { - IWL_INFO(mvm, "Device was reset during suspend\n"); - goto err; - } - - if (d0i3_first) { - struct iwl_host_cmd cmd = { - .id = D0I3_END_CMD, - .flags = CMD_WANT_SKB | CMD_SEND_IN_D3, - }; - int len; - - ret = iwl_mvm_send_cmd(mvm, &cmd); - if (ret < 0) { - IWL_ERR(mvm, "Failed to send D0I3_END_CMD first (%d)\n", - ret); + if (resume_notif_based) { + d3_data.status = kzalloc(sizeof(*d3_data.status), GFP_KERNEL); + if (!d3_data.status) { + IWL_ERR(mvm, "Failed to allocate wowlan status\n"); + ret = -ENOMEM; goto err; } - switch (mvm->cmd_ver.d0i3_resp) { - case 0: - break; - case 1: - len = iwl_rx_packet_payload_len(cmd.resp_pkt); - if (len != sizeof(u32)) { - IWL_ERR(mvm, - "Error with D0I3_END_CMD response size (%d)\n", - len); - goto err; - } - if (IWL_D0I3_RESET_REQUIRE & - le32_to_cpu(*(__le32 *)cmd.resp_pkt->data)) { - iwl_write32(mvm->trans, CSR_RESET, - CSR_RESET_REG_FLAG_FORCE_NMI); - iwl_free_resp(&cmd); - } - break; - default: - WARN_ON(1); - } + + ret = iwl_mvm_d3_notif_wait(mvm, &d3_data); + if (ret) + goto err; + } else { + ret = iwl_mvm_resume_firmware(mvm, test); + if (ret < 0) + goto err; } /* after the successful handshake, we're out of D3 */ mvm->trans->system_pm_mode = IWL_PLAT_PM_MODE_DISABLED; + /* when reset is required we can't send these following commands */ + if (d3_data.d3_end_flags & IWL_D0I3_RESET_REQUIRE) + goto query_wakeup_reasons; + /* * Query the current location and source from the D3 firmware so we * can play it back when we re-intiailize the D0 firmware @@ -2598,41 +2926,36 @@ static int __iwl_mvm_resume(struct iwl_mvm *mvm, bool test) /* Re-configure default SAR profile */ iwl_mvm_sar_select_profile(mvm, 1, 1); - if (mvm->net_detect) { + if (mvm->net_detect && unified_image) { /* If this is a non-unified image, we restart the FW, * so no need to stop the netdetect scan. If that * fails, continue and try to get the wake-up reasons, * but trigger a HW restart by keeping a failure code * in ret. */ - if (unified_image) - ret = iwl_mvm_scan_stop(mvm, IWL_MVM_SCAN_NETDETECT, - false); - - iwl_mvm_query_netdetect_reasons(mvm, vif); - /* has unlocked the mutex, so skip that */ - goto out; - } else { - keep = iwl_mvm_query_wakeup_reasons(mvm, vif); -#ifdef CONFIG_IWLWIFI_DEBUGFS - if (keep) - mvm->keep_vif = vif; -#endif - /* has unlocked the mutex, so skip that */ - goto out_iterate; + ret = iwl_mvm_scan_stop(mvm, IWL_MVM_SCAN_NETDETECT, + false); } +query_wakeup_reasons: + keep = iwl_mvm_choose_query_wakeup_reasons(mvm, vif, &d3_data); + /* has unlocked the mutex, so skip that */ + goto out; + err: - iwl_mvm_free_nd(mvm); mutex_unlock(&mvm->mutex); +out: + if (d3_data.status) + kfree(d3_data.status->wake_packet); + kfree(d3_data.status); + iwl_mvm_free_nd(mvm); -out_iterate: - if (!test) + if (!d3_data.test && !mvm->net_detect) ieee80211_iterate_active_interfaces_mtx(mvm->hw, - IEEE80211_IFACE_ITER_NORMAL, - iwl_mvm_d3_disconnect_iter, keep ? vif : NULL); + IEEE80211_IFACE_ITER_NORMAL, + iwl_mvm_d3_disconnect_iter, + keep ? vif : NULL); -out: clear_bit(IWL_MVM_STATUS_IN_D3, &mvm->status); /* no need to reset the device in unified images, if successful */ @@ -2641,9 +2964,14 @@ out: if (d0i3_first) return 0; - ret = iwl_mvm_send_cmd_pdu(mvm, D0I3_END_CMD, 0, 0, NULL); - if (!ret) + if (!iwl_fw_lookup_notif_ver(mvm->fw, PROT_OFFLOAD_GROUP, + D3_END_NOTIFICATION, 0)) { + ret = iwl_mvm_send_cmd_pdu(mvm, D0I3_END_CMD, 0, 0, NULL); + if (!ret) + return 0; + } else if (!(d3_data.d3_end_flags & IWL_D0I3_RESET_REQUIRE)) { return 0; + } } /* diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/debugfs.c b/drivers/net/wireless/intel/iwlwifi/mvm/debugfs.c index c0bd697b080a..1e8123140973 100644 --- a/drivers/net/wireless/intel/iwlwifi/mvm/debugfs.c +++ b/drivers/net/wireless/intel/iwlwifi/mvm/debugfs.c @@ -430,14 +430,16 @@ static ssize_t iwl_dbgfs_amsdu_len_write(struct ieee80211_sta *sta, return -EBUSY; if (amsdu_len) { - mvmsta->orig_amsdu_len = sta->max_amsdu_len; - sta->max_amsdu_len = amsdu_len; - for (i = 0; i < ARRAY_SIZE(sta->max_tid_amsdu_len); i++) - sta->max_tid_amsdu_len[i] = amsdu_len; + mvmsta->orig_amsdu_len = sta->cur->max_amsdu_len; + sta->deflink.agg.max_amsdu_len = amsdu_len; + sta->deflink.agg.max_amsdu_len = amsdu_len; + for (i = 0; i < ARRAY_SIZE(sta->deflink.agg.max_tid_amsdu_len); i++) + sta->deflink.agg.max_tid_amsdu_len[i] = amsdu_len; } else { - sta->max_amsdu_len = mvmsta->orig_amsdu_len; + sta->deflink.agg.max_amsdu_len = mvmsta->orig_amsdu_len; mvmsta->orig_amsdu_len = 0; } + return count; } @@ -451,7 +453,7 @@ static ssize_t iwl_dbgfs_amsdu_len_read(struct file *file, char buf[32]; int pos; - pos = scnprintf(buf, sizeof(buf), "current %d ", sta->max_amsdu_len); + pos = scnprintf(buf, sizeof(buf), "current %d ", sta->cur->max_amsdu_len); pos += scnprintf(buf + pos, sizeof(buf) - pos, "stored %d\n", mvmsta->orig_amsdu_len); diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/mac80211.c b/drivers/net/wireless/intel/iwlwifi/mvm/mac80211.c index 11536f115198..8464c9b7baf1 100644 --- a/drivers/net/wireless/intel/iwlwifi/mvm/mac80211.c +++ b/drivers/net/wireless/intel/iwlwifi/mvm/mac80211.c @@ -3193,7 +3193,7 @@ static int iwl_mvm_mac_sta_state(struct ieee80211_hw *hw, NL80211_TDLS_SETUP); } - sta->max_rc_amsdu_len = 1; + sta->deflink.agg.max_rc_amsdu_len = 1; } else if (old_state == IEEE80211_STA_NONE && new_state == IEEE80211_STA_AUTH) { /* @@ -4949,6 +4949,7 @@ static int iwl_mvm_mac_get_survey(struct ieee80211_hw *hw, int idx, static void iwl_mvm_set_sta_rate(u32 rate_n_flags, struct rate_info *rinfo) { u32 format = rate_n_flags & RATE_MCS_MOD_TYPE_MSK; + u32 gi_ltf; switch (rate_n_flags & RATE_MCS_CHAN_WIDTH_MSK) { case RATE_MCS_CHAN_WIDTH_20: @@ -5019,9 +5020,12 @@ static void iwl_mvm_set_sta_rate(u32 rate_n_flags, struct rate_info *rinfo) RATE_HT_MCS_INDEX(rate_n_flags) : u32_get_bits(rate_n_flags, RATE_MCS_CODE_MSK); - if (format == RATE_MCS_HE_MSK) { - u32 gi_ltf = u32_get_bits(rate_n_flags, - RATE_MCS_HE_GI_LTF_MSK); + if (rate_n_flags & RATE_MCS_SGI_MSK) + rinfo->flags |= RATE_INFO_FLAGS_SHORT_GI; + + switch (format) { + case RATE_MCS_HE_MSK: + gi_ltf = u32_get_bits(rate_n_flags, RATE_MCS_HE_GI_LTF_MSK); rinfo->flags |= RATE_INFO_FLAGS_HE_MCS; @@ -5060,19 +5064,14 @@ static void iwl_mvm_set_sta_rate(u32 rate_n_flags, struct rate_info *rinfo) if (rate_n_flags & RATE_HE_DUAL_CARRIER_MODE_MSK) rinfo->he_dcm = 1; - return; - } - - if (rate_n_flags & RATE_MCS_SGI_MSK) - rinfo->flags |= RATE_INFO_FLAGS_SHORT_GI; - - if (format == RATE_MCS_HT_MSK) { + break; + case RATE_MCS_HT_MSK: rinfo->flags |= RATE_INFO_FLAGS_MCS; - - } else if (format == RATE_MCS_VHT_MSK) { + break; + case RATE_MCS_VHT_MSK: rinfo->flags |= RATE_INFO_FLAGS_VHT_MCS; + break; } - } static void iwl_mvm_mac_sta_statistics(struct ieee80211_hw *hw, diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/mvm.h b/drivers/net/wireless/intel/iwlwifi/mvm/mvm.h index bf35e130c876..97cba526e465 100644 --- a/drivers/net/wireless/intel/iwlwifi/mvm/mvm.h +++ b/drivers/net/wireless/intel/iwlwifi/mvm/mvm.h @@ -860,6 +860,7 @@ struct iwl_mvm { /* Scan status, cmd (pre-allocated) and auxiliary station */ unsigned int scan_status; + size_t scan_cmd_size; void *scan_cmd; struct iwl_mcast_filter_cmd *mcast_filter_cmd; /* For CDB this is low band scan type, for non-CDB - type. */ @@ -1079,7 +1080,6 @@ struct iwl_mvm { struct list_head resp_pasn_list; struct { - u8 d0i3_resp; u8 range_resp; } cmd_ver; @@ -1705,7 +1705,7 @@ int iwl_mvm_update_quotas(struct iwl_mvm *mvm, bool force_upload, int iwl_mvm_reg_scan_start(struct iwl_mvm *mvm, struct ieee80211_vif *vif, struct cfg80211_scan_request *req, struct ieee80211_scan_ies *ies); -int iwl_mvm_scan_size(struct iwl_mvm *mvm); +size_t iwl_mvm_scan_size(struct iwl_mvm *mvm); int iwl_mvm_scan_stop(struct iwl_mvm *mvm, int type, bool notify); int iwl_mvm_max_scan_ie_len(struct iwl_mvm *mvm); void iwl_mvm_report_scan_aborted(struct iwl_mvm *mvm); diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/ops.c b/drivers/net/wireless/intel/iwlwifi/mvm/ops.c index db43c8a83a31..d2d42cd48af2 100644 --- a/drivers/net/wireless/intel/iwlwifi/mvm/ops.c +++ b/drivers/net/wireless/intel/iwlwifi/mvm/ops.c @@ -557,6 +557,13 @@ static const struct iwl_hcmd_names iwl_mvm_data_path_names[] = { /* Please keep this array *SORTED* by hex value. * Access is done through binary search */ +static const struct iwl_hcmd_names iwl_mvm_scan_names[] = { + HCMD_NAME(OFFLOAD_MATCH_INFO_NOTIF), +}; + +/* Please keep this array *SORTED* by hex value. + * Access is done through binary search + */ static const struct iwl_hcmd_names iwl_mvm_location_names[] = { HCMD_NAME(TOF_RANGE_REQ_CMD), HCMD_NAME(TOF_CONFIG_CMD), @@ -574,6 +581,9 @@ static const struct iwl_hcmd_names iwl_mvm_location_names[] = { * Access is done through binary search */ static const struct iwl_hcmd_names iwl_mvm_prot_offload_names[] = { + HCMD_NAME(WOWLAN_WAKE_PKT_NOTIFICATION), + HCMD_NAME(WOWLAN_INFO_NOTIFICATION), + HCMD_NAME(D3_END_NOTIFICATION), HCMD_NAME(STORED_BEACON_NTF), }; @@ -593,6 +603,7 @@ static const struct iwl_hcmd_arr iwl_mvm_groups[] = { [MAC_CONF_GROUP] = HCMD_ARR(iwl_mvm_mac_conf_names), [PHY_OPS_GROUP] = HCMD_ARR(iwl_mvm_phy_names), [DATA_PATH_GROUP] = HCMD_ARR(iwl_mvm_data_path_names), + [SCAN_GROUP] = HCMD_ARR(iwl_mvm_scan_names), [LOCATION_GROUP] = HCMD_ARR(iwl_mvm_location_names), [PROT_OFFLOAD_GROUP] = HCMD_ARR(iwl_mvm_prot_offload_names), [REGULATORY_AND_NVM_GROUP] = @@ -1065,7 +1076,7 @@ iwl_op_mode_mvm_start(struct iwl_trans *trans, const struct iwl_cfg *cfg, static const u8 no_reclaim_cmds[] = { TX_CMD, }; - int scan_size; + size_t scan_size; u32 min_backoff; struct iwl_mvm_csme_conn_info *csme_conn_info __maybe_unused; @@ -1188,13 +1199,6 @@ iwl_op_mode_mvm_start(struct iwl_trans *trans, const struct iwl_cfg *cfg, INIT_DELAYED_WORK(&mvm->cs_tx_unblock_dwork, iwl_mvm_tx_unblock_dwork); - mvm->cmd_ver.d0i3_resp = - iwl_fw_lookup_notif_ver(mvm->fw, LEGACY_GROUP, D0I3_END_CMD, - 0); - /* we only support version 1 */ - if (WARN_ON_ONCE(mvm->cmd_ver.d0i3_resp > 1)) - goto out_free; - mvm->cmd_ver.range_resp = iwl_fw_lookup_notif_ver(mvm->fw, LOCATION_GROUP, TOF_RANGE_RESPONSE_NOTIF, 5); @@ -1299,6 +1303,7 @@ iwl_op_mode_mvm_start(struct iwl_trans *trans, const struct iwl_cfg *cfg, mvm->scan_cmd = kmalloc(scan_size, GFP_KERNEL); if (!mvm->scan_cmd) goto out_free; + mvm->scan_cmd_size = scan_size; /* invalidate ids to prevent accidental removal of sta_id 0 */ mvm->aux_sta.sta_id = IWL_MVM_INVALID_STA; diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/rs-fw.c b/drivers/net/wireless/intel/iwlwifi/mvm/rs-fw.c index d8c3d7ff4f44..2e9081cb6627 100644 --- a/drivers/net/wireless/intel/iwlwifi/mvm/rs-fw.c +++ b/drivers/net/wireless/intel/iwlwifi/mvm/rs-fw.c @@ -143,7 +143,7 @@ rs_fw_vht_set_enabled_rates(const struct ieee80211_sta *sta, }; /* the station support only a single receive chain */ - if (sta->smps_mode == IEEE80211_SMPS_STATIC) + if (sta->deflink.smps_mode == IEEE80211_SMPS_STATIC) max_nss = 1; for (i = 0; i < max_nss && i < IWL_TLC_NSS_MAX; i++) { @@ -205,7 +205,7 @@ rs_fw_he_set_enabled_rates(const struct ieee80211_sta *sta, u8 nss = sta->deflink.rx_nss; /* the station support only a single receive chain */ - if (sta->smps_mode == IEEE80211_SMPS_STATIC) + if (sta->deflink.smps_mode == IEEE80211_SMPS_STATIC) nss = 1; for (i = 0; i < nss && i < IWL_TLC_NSS_MAX; i++) { @@ -270,7 +270,7 @@ static void rs_fw_set_supp_rates(struct ieee80211_sta *sta, cpu_to_le16(ht_cap->mcs.rx_mask[0]); /* the station support only a single receive chain */ - if (sta->smps_mode == IEEE80211_SMPS_STATIC) + if (sta->deflink.smps_mode == IEEE80211_SMPS_STATIC) cmd->ht_rates[IWL_TLC_NSS_2][IWL_TLC_MCS_PER_BW_80] = 0; else @@ -340,9 +340,9 @@ void iwl_mvm_tlc_update_notif(struct iwl_mvm *mvm, u16 size = le32_to_cpu(notif->amsdu_size); int i; - if (sta->max_amsdu_len < size) { + if (sta->deflink.agg.max_amsdu_len < size) { /* - * In debug sta->max_amsdu_len < size + * In debug sta->deflink.agg.max_amsdu_len < size * so also check with orig_amsdu_len which holds the * original data before debugfs changed the value */ @@ -352,18 +352,18 @@ void iwl_mvm_tlc_update_notif(struct iwl_mvm *mvm, mvmsta->amsdu_enabled = le32_to_cpu(notif->amsdu_enabled); mvmsta->max_amsdu_len = size; - sta->max_rc_amsdu_len = mvmsta->max_amsdu_len; + sta->deflink.agg.max_rc_amsdu_len = mvmsta->max_amsdu_len; for (i = 0; i < IWL_MAX_TID_COUNT; i++) { if (mvmsta->amsdu_enabled & BIT(i)) - sta->max_tid_amsdu_len[i] = + sta->deflink.agg.max_tid_amsdu_len[i] = iwl_mvm_max_amsdu_size(mvm, sta, i); else /* * Not so elegant, but this will effectively * prevent AMSDU on this TID */ - sta->max_tid_amsdu_len[i] = 1; + sta->deflink.agg.max_tid_amsdu_len[i] = 1; } IWL_DEBUG_RATE(mvm, @@ -450,7 +450,7 @@ void rs_fw_rate_init(struct iwl_mvm *mvm, struct ieee80211_sta *sta, * since TLC offload works with one mode we can assume * that only vht/ht is used and also set it as station max amsdu */ - sta->max_amsdu_len = max_amsdu_len; + sta->deflink.agg.max_amsdu_len = max_amsdu_len; cmd_ver = iwl_fw_lookup_cmd_ver(mvm->fw, WIDE_ID(DATA_PATH_GROUP, diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/rs.c b/drivers/net/wireless/intel/iwlwifi/mvm/rs.c index a79043f30775..0b50b816684a 100644 --- a/drivers/net/wireless/intel/iwlwifi/mvm/rs.c +++ b/drivers/net/wireless/intel/iwlwifi/mvm/rs.c @@ -138,7 +138,7 @@ static bool rs_mimo_allow(struct iwl_mvm *mvm, struct ieee80211_sta *sta, if (!sta->deflink.ht_cap.ht_supported) return false; - if (sta->smps_mode == IEEE80211_SMPS_STATIC) + if (sta->deflink.smps_mode == IEEE80211_SMPS_STATIC) return false; if (num_of_ant(iwl_mvm_get_valid_tx_ant(mvm)) < 2) @@ -1491,7 +1491,7 @@ static void rs_set_amsdu_len(struct iwl_mvm *mvm, struct ieee80211_sta *sta, struct iwl_mvm_sta *mvmsta = iwl_mvm_sta_from_mac80211(sta); int i; - sta->max_amsdu_len = rs_fw_get_max_amsdu_len(sta); + sta->deflink.agg.max_amsdu_len = rs_fw_get_max_amsdu_len(sta); /* * In case TLC offload is not active amsdu_enabled is either 0xFFFF @@ -1506,22 +1506,23 @@ static void rs_set_amsdu_len(struct iwl_mvm *mvm, struct ieee80211_sta *sta, if (mvmsta->vif->bss_conf.he_support && !iwlwifi_mod_params.disable_11ax) - mvmsta->max_amsdu_len = sta->max_amsdu_len; + mvmsta->max_amsdu_len = sta->deflink.agg.max_amsdu_len; else - mvmsta->max_amsdu_len = min_t(int, sta->max_amsdu_len, 8500); + mvmsta->max_amsdu_len = + min_t(int, sta->deflink.agg.max_amsdu_len, 8500); - sta->max_rc_amsdu_len = mvmsta->max_amsdu_len; + sta->deflink.agg.max_rc_amsdu_len = mvmsta->max_amsdu_len; for (i = 0; i < IWL_MAX_TID_COUNT; i++) { if (mvmsta->amsdu_enabled) - sta->max_tid_amsdu_len[i] = + sta->deflink.agg.max_tid_amsdu_len[i] = iwl_mvm_max_amsdu_size(mvm, sta, i); else /* * Not so elegant, but this will effectively * prevent AMSDU on this TID */ - sta->max_tid_amsdu_len[i] = 1; + sta->deflink.agg.max_tid_amsdu_len[i] = 1; } } @@ -2933,7 +2934,7 @@ static void rs_drv_rate_init(struct iwl_mvm *mvm, struct ieee80211_sta *sta, lq_sta->lq.sta_id = mvmsta->sta_id; mvmsta->amsdu_enabled = 0; - mvmsta->max_amsdu_len = sta->max_amsdu_len; + mvmsta->max_amsdu_len = sta->cur->max_amsdu_len; for (j = 0; j < LQ_SIZE; j++) rs_rate_scale_clear_tbl_windows(mvm, &lq_sta->lq_info[j]); diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/rxmq.c b/drivers/net/wireless/intel/iwlwifi/mvm/rxmq.c index 2c43a9989783..1aadccd8841f 100644 --- a/drivers/net/wireless/intel/iwlwifi/mvm/rxmq.c +++ b/drivers/net/wireless/intel/iwlwifi/mvm/rxmq.c @@ -1,6 +1,6 @@ // SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause /* - * Copyright (C) 2012-2014, 2018-2021 Intel Corporation + * Copyright (C) 2012-2014, 2018-2022 Intel Corporation * Copyright (C) 2013-2015 Intel Mobile Communications GmbH * Copyright (C) 2015-2017 Intel Deutschland GmbH */ @@ -1191,16 +1191,22 @@ struct iwl_mvm_rx_phy_data { enum iwl_rx_phy_info_type info_type; __le32 d0, d1, d2, d3; __le16 d4; + + u32 rate_n_flags; + u32 gp2_on_air_rise; + u16 phy_info; + u8 energy_a, energy_b; + u8 channel; }; static void iwl_mvm_decode_he_mu_ext(struct iwl_mvm *mvm, struct iwl_mvm_rx_phy_data *phy_data, - u32 rate_n_flags, struct ieee80211_radiotap_he_mu *he_mu) { u32 phy_data2 = le32_to_cpu(phy_data->d2); u32 phy_data3 = le32_to_cpu(phy_data->d3); u16 phy_data4 = le16_to_cpu(phy_data->d4); + u32 rate_n_flags = phy_data->rate_n_flags; if (FIELD_GET(IWL_RX_PHY_DATA4_HE_MU_EXT_CH1_CRC_OK, phy_data4)) { he_mu->flags1 |= @@ -1246,7 +1252,6 @@ static void iwl_mvm_decode_he_mu_ext(struct iwl_mvm *mvm, static void iwl_mvm_decode_he_phy_ru_alloc(struct iwl_mvm_rx_phy_data *phy_data, - u32 rate_n_flags, struct ieee80211_radiotap_he *he, struct ieee80211_radiotap_he_mu *he_mu, struct ieee80211_rx_status *rx_status) @@ -1260,6 +1265,7 @@ iwl_mvm_decode_he_phy_ru_alloc(struct iwl_mvm_rx_phy_data *phy_data, * the TSF/timers are not be transmitted in HE-MU. */ u8 ru = le32_get_bits(phy_data->d1, IWL_RX_PHY_DATA1_HE_RU_ALLOC_MASK); + u32 rate_n_flags = phy_data->rate_n_flags; u32 he_type = rate_n_flags & RATE_MCS_HE_TYPE_MSK_V1; u8 offs = 0; @@ -1331,7 +1337,7 @@ static void iwl_mvm_decode_he_phy_data(struct iwl_mvm *mvm, struct ieee80211_radiotap_he *he, struct ieee80211_radiotap_he_mu *he_mu, struct ieee80211_rx_status *rx_status, - u32 rate_n_flags, int queue) + int queue) { switch (phy_data->info_type) { case IWL_RX_PHY_INFO_TYPE_NONE: @@ -1430,7 +1436,7 @@ static void iwl_mvm_decode_he_phy_data(struct iwl_mvm *mvm, le16_encode_bits(le16_get_bits(phy_data->d4, IWL_RX_PHY_DATA4_HE_MU_EXT_PREAMBLE_PUNC_TYPE_MASK), IEEE80211_RADIOTAP_HE_MU_FLAGS2_PUNC_FROM_SIG_A_BW); - iwl_mvm_decode_he_mu_ext(mvm, phy_data, rate_n_flags, he_mu); + iwl_mvm_decode_he_mu_ext(mvm, phy_data, he_mu); fallthrough; case IWL_RX_PHY_INFO_TYPE_HE_MU: he_mu->flags2 |= @@ -1444,8 +1450,7 @@ static void iwl_mvm_decode_he_phy_data(struct iwl_mvm *mvm, fallthrough; case IWL_RX_PHY_INFO_TYPE_HE_TB: case IWL_RX_PHY_INFO_TYPE_HE_TB_EXT: - iwl_mvm_decode_he_phy_ru_alloc(phy_data, rate_n_flags, - he, he_mu, rx_status); + iwl_mvm_decode_he_phy_ru_alloc(phy_data, he, he_mu, rx_status); break; case IWL_RX_PHY_INFO_TYPE_HE_SU: he->data1 |= cpu_to_le16(IEEE80211_RADIOTAP_HE_DATA1_BEAM_CHANGE_KNOWN); @@ -1461,13 +1466,14 @@ static void iwl_mvm_decode_he_phy_data(struct iwl_mvm *mvm, static void iwl_mvm_rx_he(struct iwl_mvm *mvm, struct sk_buff *skb, struct iwl_mvm_rx_phy_data *phy_data, - u32 rate_n_flags, u16 phy_info, int queue) + int queue) { struct ieee80211_rx_status *rx_status = IEEE80211_SKB_RXCB(skb); struct ieee80211_radiotap_he *he = NULL; struct ieee80211_radiotap_he_mu *he_mu = NULL; + u32 rate_n_flags = phy_data->rate_n_flags; u32 he_type = rate_n_flags & RATE_MCS_HE_TYPE_MSK; - u8 stbc, ltf; + u8 ltf; static const struct ieee80211_radiotap_he known = { .data1 = cpu_to_le16(IEEE80211_RADIOTAP_HE_DATA1_DATA_MCS_KNOWN | IEEE80211_RADIOTAP_HE_DATA1_DATA_DCM_KNOWN | @@ -1484,6 +1490,7 @@ static void iwl_mvm_rx_he(struct iwl_mvm *mvm, struct sk_buff *skb, .flags2 = cpu_to_le16(IEEE80211_RADIOTAP_HE_MU_FLAGS2_PUNC_FROM_SIG_A_BW_KNOWN | IEEE80211_RADIOTAP_HE_MU_FLAGS2_BW_FROM_SIG_A_BW_KNOWN), }; + u16 phy_info = phy_data->phy_info; he = skb_put_data(skb, &known, sizeof(known)); rx_status->flag |= RX_FLAG_RADIOTAP_HE; @@ -1504,7 +1511,7 @@ static void iwl_mvm_rx_he(struct iwl_mvm *mvm, struct sk_buff *skb, if (phy_info & IWL_RX_MPDU_PHY_TSF_OVERLOAD) iwl_mvm_decode_he_phy_data(mvm, phy_data, he, he_mu, rx_status, - rate_n_flags, queue); + queue); /* update aggregation data for monitor sake on default queue */ if (!queue && (phy_info & IWL_RX_MPDU_PHY_TSF_OVERLOAD) && @@ -1531,19 +1538,6 @@ static void iwl_mvm_rx_he(struct iwl_mvm *mvm, struct sk_buff *skb, he->data1 |= cpu_to_le16(IEEE80211_RADIOTAP_HE_DATA1_BW_RU_ALLOC_KNOWN); - stbc = (rate_n_flags & RATE_MCS_STBC_MSK) >> RATE_MCS_STBC_POS; - rx_status->nss = - ((rate_n_flags & RATE_MCS_NSS_MSK) >> - RATE_MCS_NSS_POS) + 1; - rx_status->rate_idx = rate_n_flags & RATE_MCS_CODE_MSK; - rx_status->encoding = RX_ENC_HE; - rx_status->enc_flags |= stbc << RX_ENC_FLAG_STBC_SHIFT; - if (rate_n_flags & RATE_MCS_BF_MSK) - rx_status->enc_flags |= RX_ENC_FLAG_BF; - - rx_status->he_dcm = - !!(rate_n_flags & RATE_HE_DUAL_CARRIER_MODE_MSK); - #define CHECK_TYPE(F) \ BUILD_BUG_ON(IEEE80211_RADIOTAP_HE_DATA1_FORMAT_ ## F != \ (RATE_MCS_HE_TYPE_ ## F >> RATE_MCS_HE_TYPE_POS)) @@ -1661,6 +1655,107 @@ static void iwl_mvm_rx_get_sta_block_tx(void *data, struct ieee80211_sta *sta) rx_sta_csa->all_sta_unblocked = false; } +/* + * Note: requires also rx_status->band to be prefilled, as well + * as phy_data (apart from phy_data->info_type) + */ +static void iwl_mvm_rx_fill_status(struct iwl_mvm *mvm, + struct sk_buff *skb, + struct iwl_mvm_rx_phy_data *phy_data, + int queue) +{ + struct ieee80211_rx_status *rx_status = IEEE80211_SKB_RXCB(skb); + u32 rate_n_flags = phy_data->rate_n_flags; + u8 stbc = u32_get_bits(rate_n_flags, RATE_MCS_STBC_MSK); + u32 format = rate_n_flags & RATE_MCS_MOD_TYPE_MSK; + bool is_sgi; + + phy_data->info_type = IWL_RX_PHY_INFO_TYPE_NONE; + + if (phy_data->phy_info & IWL_RX_MPDU_PHY_TSF_OVERLOAD) + phy_data->info_type = + le32_get_bits(phy_data->d1, + IWL_RX_PHY_DATA1_INFO_TYPE_MASK); + + /* This may be overridden by iwl_mvm_rx_he() to HE_RU */ + switch (rate_n_flags & RATE_MCS_CHAN_WIDTH_MSK) { + case RATE_MCS_CHAN_WIDTH_20: + break; + case RATE_MCS_CHAN_WIDTH_40: + rx_status->bw = RATE_INFO_BW_40; + break; + case RATE_MCS_CHAN_WIDTH_80: + rx_status->bw = RATE_INFO_BW_80; + break; + case RATE_MCS_CHAN_WIDTH_160: + rx_status->bw = RATE_INFO_BW_160; + break; + } + + /* must be before L-SIG data */ + if (format == RATE_MCS_HE_MSK) + iwl_mvm_rx_he(mvm, skb, phy_data, queue); + + iwl_mvm_decode_lsig(skb, phy_data); + + rx_status->device_timestamp = phy_data->gp2_on_air_rise; + rx_status->freq = ieee80211_channel_to_frequency(phy_data->channel, + rx_status->band); + iwl_mvm_get_signal_strength(mvm, rx_status, rate_n_flags, + phy_data->energy_a, phy_data->energy_b); + + if (unlikely(mvm->monitor_on)) + iwl_mvm_add_rtap_sniffer_config(mvm, skb); + + is_sgi = format == RATE_MCS_HE_MSK ? + iwl_he_is_sgi(rate_n_flags) : + rate_n_flags & RATE_MCS_SGI_MSK; + + if (!(format == RATE_MCS_CCK_MSK) && is_sgi) + rx_status->enc_flags |= RX_ENC_FLAG_SHORT_GI; + + if (rate_n_flags & RATE_MCS_LDPC_MSK) + rx_status->enc_flags |= RX_ENC_FLAG_LDPC; + + switch (format) { + case RATE_MCS_VHT_MSK: + rx_status->encoding = RX_ENC_VHT; + break; + case RATE_MCS_HE_MSK: + rx_status->encoding = RX_ENC_HE; + rx_status->he_dcm = + !!(rate_n_flags & RATE_HE_DUAL_CARRIER_MODE_MSK); + break; + } + + switch (format) { + case RATE_MCS_HT_MSK: + rx_status->encoding = RX_ENC_HT; + rx_status->rate_idx = RATE_HT_MCS_INDEX(rate_n_flags); + rx_status->enc_flags |= stbc << RX_ENC_FLAG_STBC_SHIFT; + break; + case RATE_MCS_VHT_MSK: + case RATE_MCS_HE_MSK: + rx_status->nss = + u32_get_bits(rate_n_flags, RATE_MCS_NSS_MSK) + 1; + rx_status->rate_idx = rate_n_flags & RATE_MCS_CODE_MSK; + rx_status->enc_flags |= stbc << RX_ENC_FLAG_STBC_SHIFT; + break; + default: { + int rate = iwl_mvm_legacy_hw_idx_to_mac80211_idx(rate_n_flags, + rx_status->band); + + rx_status->rate_idx = rate; + + if (WARN_ONCE(rate < 0 || rate > 0xFF, + "Invalid rate flags 0x%x, band %d,\n", + rate_n_flags, rx_status->band)) + rx_status->rate_idx = 0; + break; + } + } +} + void iwl_mvm_rx_mpdu_mq(struct iwl_mvm *mvm, struct napi_struct *napi, struct iwl_rx_cmd_buffer *rxb, int queue) { @@ -1670,17 +1765,12 @@ void iwl_mvm_rx_mpdu_mq(struct iwl_mvm *mvm, struct napi_struct *napi, struct ieee80211_hdr *hdr; u32 len; u32 pkt_len = iwl_rx_packet_payload_len(pkt); - u32 rate_n_flags, gp2_on_air_rise; - u16 phy_info; struct ieee80211_sta *sta = NULL; struct sk_buff *skb; - u8 crypt_len = 0, channel, energy_a, energy_b; + u8 crypt_len = 0; size_t desc_size; - struct iwl_mvm_rx_phy_data phy_data = { - .info_type = IWL_RX_PHY_INFO_TYPE_NONE, - }; + struct iwl_mvm_rx_phy_data phy_data = {}; u32 format; - bool is_sgi; if (unlikely(test_bit(IWL_MVM_STATUS_IN_HW_RESTART, &mvm->status))) return; @@ -1696,35 +1786,37 @@ void iwl_mvm_rx_mpdu_mq(struct iwl_mvm *mvm, struct napi_struct *napi, } if (mvm->trans->trans_cfg->device_family >= IWL_DEVICE_FAMILY_AX210) { - rate_n_flags = le32_to_cpu(desc->v3.rate_n_flags); - channel = desc->v3.channel; - gp2_on_air_rise = le32_to_cpu(desc->v3.gp2_on_air_rise); - energy_a = desc->v3.energy_a; - energy_b = desc->v3.energy_b; + phy_data.rate_n_flags = le32_to_cpu(desc->v3.rate_n_flags); + phy_data.channel = desc->v3.channel; + phy_data.gp2_on_air_rise = le32_to_cpu(desc->v3.gp2_on_air_rise); + phy_data.energy_a = desc->v3.energy_a; + phy_data.energy_b = desc->v3.energy_b; phy_data.d0 = desc->v3.phy_data0; phy_data.d1 = desc->v3.phy_data1; phy_data.d2 = desc->v3.phy_data2; phy_data.d3 = desc->v3.phy_data3; } else { - rate_n_flags = le32_to_cpu(desc->v1.rate_n_flags); - channel = desc->v1.channel; - gp2_on_air_rise = le32_to_cpu(desc->v1.gp2_on_air_rise); - energy_a = desc->v1.energy_a; - energy_b = desc->v1.energy_b; + phy_data.rate_n_flags = le32_to_cpu(desc->v1.rate_n_flags); + phy_data.channel = desc->v1.channel; + phy_data.gp2_on_air_rise = le32_to_cpu(desc->v1.gp2_on_air_rise); + phy_data.energy_a = desc->v1.energy_a; + phy_data.energy_b = desc->v1.energy_b; phy_data.d0 = desc->v1.phy_data0; phy_data.d1 = desc->v1.phy_data1; phy_data.d2 = desc->v1.phy_data2; phy_data.d3 = desc->v1.phy_data3; } + if (iwl_fw_lookup_notif_ver(mvm->fw, LEGACY_GROUP, REPLY_RX_MPDU_CMD, 0) < 4) { - rate_n_flags = iwl_new_rate_from_v1(rate_n_flags); + phy_data.rate_n_flags = iwl_new_rate_from_v1(phy_data.rate_n_flags); IWL_DEBUG_DROP(mvm, "Got old format rate, converting. New rate: 0x%x\n", - rate_n_flags); + phy_data.rate_n_flags); } - format = rate_n_flags & RATE_MCS_MOD_TYPE_MSK; + + format = phy_data.rate_n_flags & RATE_MCS_MOD_TYPE_MSK; len = le16_to_cpu(desc->mpdu_len); @@ -1733,14 +1825,9 @@ void iwl_mvm_rx_mpdu_mq(struct iwl_mvm *mvm, struct napi_struct *napi, return; } - phy_info = le16_to_cpu(desc->phy_info); + phy_data.phy_info = le16_to_cpu(desc->phy_info); phy_data.d4 = desc->phy_data4; - if (phy_info & IWL_RX_MPDU_PHY_TSF_OVERLOAD) - phy_data.info_type = - le32_get_bits(phy_data.d1, - IWL_RX_PHY_DATA1_INFO_TYPE_MASK); - hdr = (void *)(pkt->data + desc_size); /* Dont use dev_alloc_skb(), we'll have enough headroom once * ieee80211_hdr pulled. @@ -1763,27 +1850,6 @@ void iwl_mvm_rx_mpdu_mq(struct iwl_mvm *mvm, struct napi_struct *napi, rx_status = IEEE80211_SKB_RXCB(skb); - /* This may be overridden by iwl_mvm_rx_he() to HE_RU */ - switch (rate_n_flags & RATE_MCS_CHAN_WIDTH_MSK) { - case RATE_MCS_CHAN_WIDTH_20: - break; - case RATE_MCS_CHAN_WIDTH_40: - rx_status->bw = RATE_INFO_BW_40; - break; - case RATE_MCS_CHAN_WIDTH_80: - rx_status->bw = RATE_INFO_BW_80; - break; - case RATE_MCS_CHAN_WIDTH_160: - rx_status->bw = RATE_INFO_BW_160; - break; - } - - if (format == RATE_MCS_HE_MSK) - iwl_mvm_rx_he(mvm, skb, &phy_data, rate_n_flags, - phy_info, queue); - - iwl_mvm_decode_lsig(skb, &phy_data); - /* * Keep packets with CRC errors (and with overrun) for monitor mode * (otherwise the firmware discards them) but mark them as bad. @@ -1794,12 +1860,13 @@ void iwl_mvm_rx_mpdu_mq(struct iwl_mvm *mvm, struct napi_struct *napi, le32_to_cpu(desc->status)); rx_status->flag |= RX_FLAG_FAILED_FCS_CRC; } + /* set the preamble flag if appropriate */ if (format == RATE_MCS_CCK_MSK && - phy_info & IWL_RX_MPDU_PHY_SHORT_PREAMBLE) + phy_data.phy_info & IWL_RX_MPDU_PHY_SHORT_PREAMBLE) rx_status->enc_flags |= RX_ENC_FLAG_SHORTPRE; - if (likely(!(phy_info & IWL_RX_MPDU_PHY_TSF_OVERLOAD))) { + if (likely(!(phy_data.phy_info & IWL_RX_MPDU_PHY_TSF_OVERLOAD))) { u64 tsf_on_air_rise; if (mvm->trans->trans_cfg->device_family >= @@ -1813,24 +1880,20 @@ void iwl_mvm_rx_mpdu_mq(struct iwl_mvm *mvm, struct napi_struct *napi, rx_status->flag |= RX_FLAG_MACTIME_PLCP_START; } - rx_status->device_timestamp = gp2_on_air_rise; if (iwl_mvm_is_band_in_rx_supported(mvm)) { u8 band = BAND_IN_RX_STATUS(desc->mac_phy_idx); rx_status->band = iwl_mvm_nl80211_band_from_rx_msdu(band); } else { - rx_status->band = channel > 14 ? NL80211_BAND_5GHZ : + rx_status->band = phy_data.channel > 14 ? NL80211_BAND_5GHZ : NL80211_BAND_2GHZ; } - rx_status->freq = ieee80211_channel_to_frequency(channel, - rx_status->band); - iwl_mvm_get_signal_strength(mvm, rx_status, rate_n_flags, energy_a, - energy_b); /* update aggregation data for monitor sake on default queue */ - if (!queue && (phy_info & IWL_RX_MPDU_PHY_AMPDU)) { - bool toggle_bit = phy_info & IWL_RX_MPDU_PHY_AMPDU_TOGGLE; + if (!queue && (phy_data.phy_info & IWL_RX_MPDU_PHY_AMPDU)) { + bool toggle_bit; + toggle_bit = phy_data.phy_info & IWL_RX_MPDU_PHY_AMPDU_TOGGLE; rx_status->flag |= RX_FLAG_AMPDU_DETAILS; /* * Toggle is switched whenever new aggregation starts. Make @@ -1846,9 +1909,6 @@ void iwl_mvm_rx_mpdu_mq(struct iwl_mvm *mvm, struct napi_struct *napi, rx_status->ampdu_reference = mvm->ampdu_ref; } - if (unlikely(mvm->monitor_on)) - iwl_mvm_add_rtap_sniffer_config(mvm, skb); - rcu_read_lock(); if (desc->status & cpu_to_le32(IWL_RX_MPDU_STATUS_SRC_STA_FOUND)) { @@ -1867,13 +1927,15 @@ void iwl_mvm_rx_mpdu_mq(struct iwl_mvm *mvm, struct napi_struct *napi, sta = ieee80211_find_sta_by_ifaddr(mvm->hw, hdr->addr2, NULL); } - if (iwl_mvm_rx_crypto(mvm, sta, hdr, rx_status, phy_info, desc, + if (iwl_mvm_rx_crypto(mvm, sta, hdr, rx_status, phy_data.phy_info, desc, le32_to_cpu(pkt->len_n_flags), queue, &crypt_len)) { kfree_skb(skb); goto out; } + iwl_mvm_rx_fill_status(mvm, skb, &phy_data, queue); + if (sta) { struct iwl_mvm_sta *mvmsta = iwl_mvm_sta_from_mac80211(sta); struct ieee80211_vif *tx_blocked_vif = @@ -1971,43 +2033,6 @@ void iwl_mvm_rx_mpdu_mq(struct iwl_mvm *mvm, struct napi_struct *napi, } } - is_sgi = format == RATE_MCS_HE_MSK ? - iwl_he_is_sgi(rate_n_flags) : - rate_n_flags & RATE_MCS_SGI_MSK; - - if (!(format == RATE_MCS_CCK_MSK) && is_sgi) - rx_status->enc_flags |= RX_ENC_FLAG_SHORT_GI; - if (rate_n_flags & RATE_MCS_LDPC_MSK) - rx_status->enc_flags |= RX_ENC_FLAG_LDPC; - if (format == RATE_MCS_HT_MSK) { - u8 stbc = (rate_n_flags & RATE_MCS_STBC_MSK) >> - RATE_MCS_STBC_POS; - rx_status->encoding = RX_ENC_HT; - rx_status->rate_idx = RATE_HT_MCS_INDEX(rate_n_flags); - rx_status->enc_flags |= stbc << RX_ENC_FLAG_STBC_SHIFT; - } else if (format == RATE_MCS_VHT_MSK) { - u8 stbc = (rate_n_flags & RATE_MCS_STBC_MSK) >> - RATE_MCS_STBC_POS; - rx_status->nss = ((rate_n_flags & RATE_MCS_NSS_MSK) >> - RATE_MCS_NSS_POS) + 1; - rx_status->rate_idx = rate_n_flags & RATE_MCS_CODE_MSK; - rx_status->encoding = RX_ENC_VHT; - rx_status->enc_flags |= stbc << RX_ENC_FLAG_STBC_SHIFT; - if (rate_n_flags & RATE_MCS_BF_MSK) - rx_status->enc_flags |= RX_ENC_FLAG_BF; - } else if (!(format == RATE_MCS_HE_MSK)) { - int rate = iwl_mvm_legacy_hw_idx_to_mac80211_idx(rate_n_flags, - rx_status->band); - - if (WARN(rate < 0 || rate > 0xFF, - "Invalid rate flags 0x%x, band %d,\n", - rate_n_flags, rx_status->band)) { - kfree_skb(skb); - goto out; - } - rx_status->rate_idx = rate; - } - /* management stuff on default queue */ if (!queue) { if (unlikely((ieee80211_is_beacon(hdr->frame_control) || @@ -2039,32 +2064,32 @@ void iwl_mvm_rx_monitor_no_data(struct iwl_mvm *mvm, struct napi_struct *napi, struct ieee80211_rx_status *rx_status; struct iwl_rx_packet *pkt = rxb_addr(rxb); struct iwl_rx_no_data *desc = (void *)pkt->data; - u32 rate_n_flags = le32_to_cpu(desc->rate); - u32 gp2_on_air_rise = le32_to_cpu(desc->on_air_rise_time); u32 rssi = le32_to_cpu(desc->rssi); u32 info_type = le32_to_cpu(desc->info) & RX_NO_DATA_INFO_TYPE_MSK; - u16 phy_info = IWL_RX_MPDU_PHY_TSF_OVERLOAD; struct ieee80211_sta *sta = NULL; struct sk_buff *skb; - u8 channel, energy_a, energy_b; - u32 format; struct iwl_mvm_rx_phy_data phy_data = { - .info_type = le32_get_bits(desc->phy_info[1], - IWL_RX_PHY_DATA1_INFO_TYPE_MASK), .d0 = desc->phy_info[0], .d1 = desc->phy_info[1], + .phy_info = IWL_RX_MPDU_PHY_TSF_OVERLOAD, + .gp2_on_air_rise = le32_to_cpu(desc->on_air_rise_time), + .rate_n_flags = le32_to_cpu(desc->rate), + .energy_a = u32_get_bits(rssi, RX_NO_DATA_CHAIN_A_MSK), + .energy_b = u32_get_bits(rssi, RX_NO_DATA_CHAIN_B_MSK), + .channel = u32_get_bits(rssi, RX_NO_DATA_CHANNEL_MSK), }; - bool is_sgi; + u32 format; if (iwl_fw_lookup_notif_ver(mvm->fw, DATA_PATH_GROUP, RX_NO_DATA_NOTIF, 0) < 2) { IWL_DEBUG_DROP(mvm, "Got an old rate format. Old rate: 0x%x\n", - rate_n_flags); - rate_n_flags = iwl_new_rate_from_v1(rate_n_flags); + phy_data.rate_n_flags); + phy_data.rate_n_flags = iwl_new_rate_from_v1(phy_data.rate_n_flags); IWL_DEBUG_DROP(mvm, " Rate after conversion to the new format: 0x%x\n", - rate_n_flags); + phy_data.rate_n_flags); } - format = rate_n_flags & RATE_MCS_MOD_TYPE_MSK; + + format = phy_data.rate_n_flags & RATE_MCS_MOD_TYPE_MSK; if (unlikely(iwl_rx_packet_payload_len(pkt) < sizeof(*desc))) return; @@ -2072,10 +2097,6 @@ void iwl_mvm_rx_monitor_no_data(struct iwl_mvm *mvm, struct napi_struct *napi, if (unlikely(test_bit(IWL_MVM_STATUS_IN_HW_RESTART, &mvm->status))) return; - energy_a = (rssi & RX_NO_DATA_CHAIN_A_MSK) >> RX_NO_DATA_CHAIN_A_POS; - energy_b = (rssi & RX_NO_DATA_CHAIN_B_MSK) >> RX_NO_DATA_CHAIN_B_POS; - channel = (rssi & RX_NO_DATA_CHANNEL_MSK) >> RX_NO_DATA_CHANNEL_POS; - /* Dont use dev_alloc_skb(), we'll have enough headroom once * ieee80211_hdr pulled. */ @@ -2106,86 +2127,31 @@ void iwl_mvm_rx_monitor_no_data(struct iwl_mvm *mvm, struct napi_struct *napi, break; } - /* This may be overridden by iwl_mvm_rx_he() to HE_RU */ - switch (rate_n_flags & RATE_MCS_CHAN_WIDTH_MSK) { - case RATE_MCS_CHAN_WIDTH_20: - break; - case RATE_MCS_CHAN_WIDTH_40: - rx_status->bw = RATE_INFO_BW_40; - break; - case RATE_MCS_CHAN_WIDTH_80: - rx_status->bw = RATE_INFO_BW_80; - break; - case RATE_MCS_CHAN_WIDTH_160: - rx_status->bw = RATE_INFO_BW_160; - break; - } - - if (format == RATE_MCS_HE_MSK) - iwl_mvm_rx_he(mvm, skb, &phy_data, rate_n_flags, - phy_info, queue); - - iwl_mvm_decode_lsig(skb, &phy_data); - - rx_status->device_timestamp = gp2_on_air_rise; - rx_status->band = channel > 14 ? NL80211_BAND_5GHZ : + rx_status->band = phy_data.channel > 14 ? NL80211_BAND_5GHZ : NL80211_BAND_2GHZ; - rx_status->freq = ieee80211_channel_to_frequency(channel, - rx_status->band); - iwl_mvm_get_signal_strength(mvm, rx_status, rate_n_flags, energy_a, - energy_b); - rcu_read_lock(); + iwl_mvm_rx_fill_status(mvm, skb, &phy_data, queue); - is_sgi = format == RATE_MCS_HE_MSK ? - iwl_he_is_sgi(rate_n_flags) : - rate_n_flags & RATE_MCS_SGI_MSK; - - if (!(format == RATE_MCS_CCK_MSK) && is_sgi) - rx_status->enc_flags |= RX_ENC_FLAG_SHORT_GI; - if (rate_n_flags & RATE_MCS_LDPC_MSK) - rx_status->enc_flags |= RX_ENC_FLAG_LDPC; - if (format == RATE_MCS_HT_MSK) { - u8 stbc = (rate_n_flags & RATE_MCS_STBC_MSK) >> - RATE_MCS_STBC_POS; - rx_status->encoding = RX_ENC_HT; - rx_status->rate_idx = RATE_HT_MCS_INDEX(rate_n_flags); - rx_status->enc_flags |= stbc << RX_ENC_FLAG_STBC_SHIFT; - } else if (format == RATE_MCS_VHT_MSK) { - u8 stbc = (rate_n_flags & RATE_MCS_STBC_MSK) >> - RATE_MCS_STBC_POS; - rx_status->rate_idx = rate_n_flags & RATE_MCS_CODE_MSK; - rx_status->encoding = RX_ENC_VHT; - rx_status->enc_flags |= stbc << RX_ENC_FLAG_STBC_SHIFT; - if (rate_n_flags & RATE_MCS_BF_MSK) - rx_status->enc_flags |= RX_ENC_FLAG_BF; - /* - * take the nss from the rx_vec since the rate_n_flags has - * only 2 bits for the nss which gives a max of 4 ss but - * there may be up to 8 spatial streams - */ + /* + * Override the nss from the rx_vec since the rate_n_flags has + * only 2 bits for the nss which gives a max of 4 ss but there + * may be up to 8 spatial streams. + */ + switch (format) { + case RATE_MCS_VHT_MSK: rx_status->nss = le32_get_bits(desc->rx_vec[0], RX_NO_DATA_RX_VEC0_VHT_NSTS_MSK) + 1; - } else if (format == RATE_MCS_HE_MSK) { + break; + case RATE_MCS_HE_MSK: rx_status->nss = le32_get_bits(desc->rx_vec[0], RX_NO_DATA_RX_VEC0_HE_NSTS_MSK) + 1; - } else { - int rate = iwl_mvm_legacy_hw_idx_to_mac80211_idx(rate_n_flags, - rx_status->band); - - if (WARN(rate < 0 || rate > 0xFF, - "Invalid rate flags 0x%x, band %d,\n", - rate_n_flags, rx_status->band)) { - kfree_skb(skb); - goto out; - } - rx_status->rate_idx = rate; + break; } + rcu_read_lock(); ieee80211_rx_napi(mvm->hw, sta, skb, napi); -out: rcu_read_unlock(); } diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/scan.c b/drivers/net/wireless/intel/iwlwifi/mvm/scan.c index 582a95ffc7ab..acd8803dbcdd 100644 --- a/drivers/net/wireless/intel/iwlwifi/mvm/scan.c +++ b/drivers/net/wireless/intel/iwlwifi/mvm/scan.c @@ -2626,7 +2626,7 @@ static int iwl_mvm_build_scan_cmd(struct iwl_mvm *mvm, u8 scan_ver; lockdep_assert_held(&mvm->mutex); - memset(mvm->scan_cmd, 0, ksize(mvm->scan_cmd)); + memset(mvm->scan_cmd, 0, mvm->scan_cmd_size); if (!fw_has_capa(&mvm->fw->ucode_capa, IWL_UCODE_TLV_CAPA_UMAC_SCAN)) { hcmd->id = SCAN_OFFLOAD_REQUEST_CMD; @@ -3091,7 +3091,7 @@ static int iwl_mvm_scan_stop_wait(struct iwl_mvm *mvm, int type) 1 * HZ); } -static int iwl_scan_req_umac_get_size(u8 scan_ver) +static size_t iwl_scan_req_umac_get_size(u8 scan_ver) { switch (scan_ver) { case 12: @@ -3104,7 +3104,7 @@ static int iwl_scan_req_umac_get_size(u8 scan_ver) return 0; } -int iwl_mvm_scan_size(struct iwl_mvm *mvm) +size_t iwl_mvm_scan_size(struct iwl_mvm *mvm) { int base_size, tail_size; u8 scan_ver = iwl_fw_lookup_cmd_ver(mvm->fw, SCAN_REQ_UMAC, diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/sta.c b/drivers/net/wireless/intel/iwlwifi/mvm/sta.c index ff0d3b3df140..cc92706b3d16 100644 --- a/drivers/net/wireless/intel/iwlwifi/mvm/sta.c +++ b/drivers/net/wireless/intel/iwlwifi/mvm/sta.c @@ -116,7 +116,7 @@ int iwl_mvm_sta_send_to_fw(struct iwl_mvm *mvm, struct ieee80211_sta *sta, break; } - switch (sta->smps_mode) { + switch (sta->deflink.smps_mode) { case IEEE80211_SMPS_AUTOMATIC: case IEEE80211_SMPS_NUM_MODES: WARN_ON(1); diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/tx.c b/drivers/net/wireless/intel/iwlwifi/mvm/tx.c index f9e08b339e0c..86d20e13bf47 100644 --- a/drivers/net/wireless/intel/iwlwifi/mvm/tx.c +++ b/drivers/net/wireless/intel/iwlwifi/mvm/tx.c @@ -926,7 +926,7 @@ static int iwl_mvm_tx_tso(struct iwl_mvm *mvm, struct sk_buff *skb, * Take the min of ieee80211 station and mvm station */ max_amsdu_len = - min_t(unsigned int, sta->max_amsdu_len, + min_t(unsigned int, sta->cur->max_amsdu_len, iwl_mvm_max_amsdu_size(mvm, sta, tid)); /* diff --git a/drivers/net/wireless/intel/iwlwifi/pcie/drv.c b/drivers/net/wireless/intel/iwlwifi/pcie/drv.c index b16d4ae182d1..4f699862e7f7 100644 --- a/drivers/net/wireless/intel/iwlwifi/pcie/drv.c +++ b/drivers/net/wireless/intel/iwlwifi/pcie/drv.c @@ -1155,10 +1155,20 @@ static const struct iwl_dev_info iwl_dev_info_table[] = { IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_ANY, iwl_cfg_bz_a0_fm_a0, iwl_bz_name), _IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY, - IWL_CFG_MAC_TYPE_GL, IWL_CFG_ANY, + IWL_CFG_MAC_TYPE_BZ, IWL_CFG_ANY, + IWL_CFG_RF_TYPE_FM, IWL_CFG_ANY, + IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_CDB, IWL_CFG_NO_JACKET, + iwl_cfg_bz_a0_fm4_a0, iwl_bz_name), + _IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY, + IWL_CFG_MAC_TYPE_GL, SILICON_A_STEP, IWL_CFG_RF_TYPE_FM, IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_NO_JACKET, iwl_cfg_gl_a0_fm_a0, iwl_bz_name), + _IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY, + IWL_CFG_MAC_TYPE_GL, SILICON_B_STEP, + IWL_CFG_RF_TYPE_FM, IWL_CFG_ANY, + IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_NO_JACKET, + iwl_cfg_gl_b0_fm_b0, iwl_bz_name), /* BZ Z step */ _IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY, @@ -1169,11 +1179,16 @@ static const struct iwl_dev_info iwl_dev_info_table[] = { /* BNJ */ _IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY, - IWL_CFG_MAC_TYPE_GL, IWL_CFG_ANY, + IWL_CFG_MAC_TYPE_GL, SILICON_A_STEP, IWL_CFG_RF_TYPE_FM, IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_IS_JACKET, iwl_cfg_bnj_a0_fm_a0, iwl_bz_name), _IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY, + IWL_CFG_MAC_TYPE_GL, SILICON_B_STEP, + IWL_CFG_RF_TYPE_FM, IWL_CFG_ANY, + IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_IS_JACKET, + iwl_cfg_bnj_b0_fm_b0, iwl_bz_name), + _IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_MAC_TYPE_GL, IWL_CFG_ANY, IWL_CFG_RF_TYPE_FM, IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_CDB, IWL_CFG_IS_JACKET, diff --git a/drivers/net/wireless/intel/iwlwifi/pcie/rx.c b/drivers/net/wireless/intel/iwlwifi/pcie/rx.c index 68a4572cee53..9c9f87fe8377 100644 --- a/drivers/net/wireless/intel/iwlwifi/pcie/rx.c +++ b/drivers/net/wireless/intel/iwlwifi/pcie/rx.c @@ -1110,7 +1110,7 @@ static int _iwl_pcie_rx_init(struct iwl_trans *trans) poll = iwl_pcie_napi_poll_msix; netif_napi_add(&trans_pcie->napi_dev, &rxq->napi, - poll, NAPI_POLL_WEIGHT); + poll); napi_enable(&rxq->napi); } diff --git a/drivers/net/wireless/intersil/hostap/hostap_ioctl.c b/drivers/net/wireless/intersil/hostap/hostap_ioctl.c index 0a376f112db9..4e0a0c881697 100644 --- a/drivers/net/wireless/intersil/hostap/hostap_ioctl.c +++ b/drivers/net/wireless/intersil/hostap/hostap_ioctl.c @@ -3848,7 +3848,7 @@ static void prism2_get_drvinfo(struct net_device *dev, iface = netdev_priv(dev); local = iface->local; - strlcpy(info->driver, "hostap", sizeof(info->driver)); + strscpy(info->driver, "hostap", sizeof(info->driver)); snprintf(info->fw_version, sizeof(info->fw_version), "%d.%d.%d", (local->sta_fw_ver >> 16) & 0xff, (local->sta_fw_ver >> 8) & 0xff, diff --git a/drivers/net/wireless/intersil/p54/main.c b/drivers/net/wireless/intersil/p54/main.c index b925e327e091..e127453ab51a 100644 --- a/drivers/net/wireless/intersil/p54/main.c +++ b/drivers/net/wireless/intersil/p54/main.c @@ -635,7 +635,7 @@ static int p54_get_survey(struct ieee80211_hw *dev, int idx, /* * hw/fw has not accumulated enough sample sets. * Wait for 100ms, this ought to be enough to - * to get at least one non-null set of channel + * get at least one non-null set of channel * usage statistics. */ msleep(100); diff --git a/drivers/net/wireless/mac80211_hwsim.c b/drivers/net/wireless/mac80211_hwsim.c index 1f301a5fb396..df51b5b1f171 100644 --- a/drivers/net/wireless/mac80211_hwsim.c +++ b/drivers/net/wireless/mac80211_hwsim.c @@ -229,6 +229,7 @@ static inline void hwsim_clear_magic(struct ieee80211_vif *vif) struct hwsim_sta_priv { u32 magic; unsigned int last_link; + u16 active_links_rx; }; #define HWSIM_STA_MAGIC 0x6d537749 @@ -652,7 +653,6 @@ struct mac80211_hwsim_data { u32 ciphers[ARRAY_SIZE(hwsim_ciphers)]; struct mac_address addresses[2]; - struct ieee80211_chanctx_conf *chanctx; int channels, idx; bool use_chanctx; bool destroy_on_close; @@ -1299,6 +1299,8 @@ static void mac80211_hwsim_config_mac_nl(struct ieee80211_hw *hw, struct sk_buff *skb; void *msg_head; + WARN_ON(!is_valid_ether_addr(addr)); + if (!_portid && !hwsim_virtio_enabled) return; @@ -1561,6 +1563,42 @@ static void mac80211_hwsim_add_vendor_rtap(struct sk_buff *skb) #endif } +static void mac80211_hwsim_rx(struct mac80211_hwsim_data *data, + struct ieee80211_rx_status *rx_status, + struct sk_buff *skb) +{ + struct ieee80211_hdr *hdr = (void *)skb->data; + + if (!ieee80211_has_morefrags(hdr->frame_control) && + !is_multicast_ether_addr(hdr->addr1) && + (ieee80211_is_mgmt(hdr->frame_control) || + ieee80211_is_data(hdr->frame_control))) { + struct ieee80211_sta *sta; + unsigned int link_id; + + rcu_read_lock(); + sta = ieee80211_find_sta_by_link_addrs(data->hw, hdr->addr2, + hdr->addr1, &link_id); + if (sta) { + struct hwsim_sta_priv *sp = (void *)sta->drv_priv; + + if (ieee80211_has_pm(hdr->frame_control)) + sp->active_links_rx &= ~BIT(link_id); + else + sp->active_links_rx |= BIT(link_id); + } + rcu_read_unlock(); + } + + memcpy(IEEE80211_SKB_RXCB(skb), rx_status, sizeof(*rx_status)); + + mac80211_hwsim_add_vendor_rtap(skb); + + data->rx_pkts++; + data->rx_bytes += skb->len; + ieee80211_rx_irqsafe(data->hw, skb); +} + static bool mac80211_hwsim_tx_frame_no_nl(struct ieee80211_hw *hw, struct sk_buff *skb, struct ieee80211_channel *chan) @@ -1688,13 +1726,7 @@ static bool mac80211_hwsim_tx_frame_no_nl(struct ieee80211_hw *hw, rx_status.mactime = now + data2->tsf_offset; - memcpy(IEEE80211_SKB_RXCB(nskb), &rx_status, sizeof(rx_status)); - - mac80211_hwsim_add_vendor_rtap(nskb); - - data2->rx_pkts++; - data2->rx_bytes += nskb->len; - ieee80211_rx_irqsafe(data2->hw, nskb); + mac80211_hwsim_rx(data2, &rx_status, nskb); } spin_unlock(&hwsim_radio_lock); @@ -1714,12 +1746,7 @@ mac80211_hwsim_select_tx_link(struct mac80211_hwsim_data *data, if (!vif->valid_links) return &vif->bss_conf; - /* FIXME: handle multicast TX properly */ - if (is_multicast_ether_addr(hdr->addr1) || WARN_ON_ONCE(!sta)) { - unsigned int first_link = ffs(vif->valid_links) - 1; - - return rcu_dereference(vif->link_conf[first_link]); - } + WARN_ON(is_multicast_ether_addr(hdr->addr1)); if (WARN_ON_ONCE(!sta->valid_links)) return &vif->bss_conf; @@ -1731,6 +1758,12 @@ mac80211_hwsim_select_tx_link(struct mac80211_hwsim_data *data, /* round-robin the available link IDs */ link_id = (sp->last_link + i + 1) % ARRAY_SIZE(vif->link_conf); + if (!(vif->active_links & BIT(link_id))) + continue; + + if (!(sp->active_links_rx & BIT(link_id))) + continue; + *link_sta = rcu_dereference(sta->link[link_id]); if (!*link_sta) continue; @@ -1739,6 +1772,10 @@ mac80211_hwsim_select_tx_link(struct mac80211_hwsim_data *data, if (WARN_ON_ONCE(!bss_conf)) continue; + /* can happen while switching links */ + if (!rcu_access_pointer(bss_conf->chanctx_conf)) + continue; + sp->last_link = link_id; return bss_conf; } @@ -2401,10 +2438,19 @@ static int mac80211_hwsim_sta_add(struct ieee80211_hw *hw, struct ieee80211_vif *vif, struct ieee80211_sta *sta) { + struct hwsim_sta_priv *sp = (void *)sta->drv_priv; + hwsim_check_magic(vif); hwsim_set_sta_magic(sta); mac80211_hwsim_sta_rc_update(hw, vif, sta, 0); + if (sta->valid_links) { + WARN(hweight16(sta->valid_links) > 1, + "expect to add STA with single link, have 0x%x\n", + sta->valid_links); + sp->active_links_rx = sta->valid_links; + } + return 0; } @@ -2430,6 +2476,14 @@ static int mac80211_hwsim_sta_state(struct ieee80211_hw *hw, if (old_state == IEEE80211_STA_NOTEXIST) return mac80211_hwsim_sta_add(hw, vif, sta); + /* + * when client is authorized (AP station marked as such), + * enable all links + */ + if (vif->type == NL80211_IFTYPE_STATION && + new_state == IEEE80211_STA_AUTHORIZED && !sta->tdls) + ieee80211_set_active_links_async(vif, vif->valid_links); + return 0; } @@ -2866,11 +2920,6 @@ static int mac80211_hwsim_croc(struct ieee80211_hw *hw, static int mac80211_hwsim_add_chanctx(struct ieee80211_hw *hw, struct ieee80211_chanctx_conf *ctx) { - struct mac80211_hwsim_data *hwsim = hw->priv; - - mutex_lock(&hwsim->mutex); - hwsim->chanctx = ctx; - mutex_unlock(&hwsim->mutex); hwsim_set_chanctx_magic(ctx); wiphy_dbg(hw->wiphy, "add channel context control: %d MHz/width: %d/cfreqs:%d/%d MHz\n", @@ -2882,11 +2931,6 @@ static int mac80211_hwsim_add_chanctx(struct ieee80211_hw *hw, static void mac80211_hwsim_remove_chanctx(struct ieee80211_hw *hw, struct ieee80211_chanctx_conf *ctx) { - struct mac80211_hwsim_data *hwsim = hw->priv; - - mutex_lock(&hwsim->mutex); - hwsim->chanctx = NULL; - mutex_unlock(&hwsim->mutex); wiphy_dbg(hw->wiphy, "remove channel context control: %d MHz/width: %d/cfreqs:%d/%d MHz\n", ctx->def.chan->center_freq, ctx->def.width, @@ -2899,11 +2943,6 @@ static void mac80211_hwsim_change_chanctx(struct ieee80211_hw *hw, struct ieee80211_chanctx_conf *ctx, u32 changed) { - struct mac80211_hwsim_data *hwsim = hw->priv; - - mutex_lock(&hwsim->mutex); - hwsim->chanctx = ctx; - mutex_unlock(&hwsim->mutex); hwsim_check_chanctx_magic(ctx); wiphy_dbg(hw->wiphy, "change channel context control: %d MHz/width: %d/cfreqs:%d/%d MHz\n", @@ -2919,6 +2958,18 @@ static int mac80211_hwsim_assign_vif_chanctx(struct ieee80211_hw *hw, hwsim_check_magic(vif); hwsim_check_chanctx_magic(ctx); + /* if we activate a link while already associated wake it up */ + if (vif->type == NL80211_IFTYPE_STATION && vif->cfg.assoc) { + struct sk_buff *skb; + + skb = ieee80211_nullfunc_get(hw, vif, link_conf->link_id, true); + if (skb) { + local_bh_disable(); + mac80211_hwsim_tx_frame(hw, skb, ctx->def.chan); + local_bh_enable(); + } + } + return 0; } @@ -2929,6 +2980,22 @@ static void mac80211_hwsim_unassign_vif_chanctx(struct ieee80211_hw *hw, { hwsim_check_magic(vif); hwsim_check_chanctx_magic(ctx); + + /* if we deactivate a link while associated suspend it first */ + if (vif->type == NL80211_IFTYPE_STATION && vif->cfg.assoc) { + struct sk_buff *skb; + + skb = ieee80211_nullfunc_get(hw, vif, link_conf->link_id, true); + if (skb) { + struct ieee80211_hdr *hdr = (void *)skb->data; + + hdr->frame_control |= cpu_to_le16(IEEE80211_FCTL_PM); + + local_bh_disable(); + mac80211_hwsim_tx_frame(hw, skb, ctx->def.chan); + local_bh_enable(); + } + } } static const char mac80211_hwsim_gstrings_stats[][ETH_GSTRING_LEN] = { @@ -2995,18 +3062,22 @@ static int mac80211_hwsim_change_vif_links(struct ieee80211_hw *hw, u16 old_links, u16 new_links, struct ieee80211_bss_conf *old[IEEE80211_MLD_MAX_NUM_LINKS]) { - unsigned long rem = old_links & ~new_links ?: BIT(0); + unsigned long rem = old_links & ~new_links; unsigned long add = new_links & ~old_links; int i; + if (!old_links) + rem |= BIT(0); + if (!new_links) + add |= BIT(0); + for_each_set_bit(i, &rem, IEEE80211_MLD_MAX_NUM_LINKS) mac80211_hwsim_config_mac_nl(hw, old[i]->addr, false); for_each_set_bit(i, &add, IEEE80211_MLD_MAX_NUM_LINKS) { struct ieee80211_bss_conf *link_conf; - /* FIXME: figure out how to get the locking here */ - link_conf = rcu_dereference_protected(vif->link_conf[i], 1); + link_conf = link_conf_dereference_protected(vif, i); if (WARN_ON(!link_conf)) continue; @@ -3021,6 +3092,13 @@ static int mac80211_hwsim_change_sta_links(struct ieee80211_hw *hw, struct ieee80211_sta *sta, u16 old_links, u16 new_links) { + struct hwsim_sta_priv *sp = (void *)sta->drv_priv; + + hwsim_check_sta_magic(sta); + + if (vif->type == NL80211_IFTYPE_STATION) + sp->active_links_rx = new_links; + return 0; } @@ -3208,8 +3286,112 @@ out_err: static const struct ieee80211_sband_iftype_data sband_capa_2ghz[] = { { - .types_mask = BIT(NL80211_IFTYPE_STATION) | - BIT(NL80211_IFTYPE_AP), + .types_mask = BIT(NL80211_IFTYPE_STATION), + .he_cap = { + .has_he = true, + .he_cap_elem = { + .mac_cap_info[0] = + IEEE80211_HE_MAC_CAP0_HTC_HE, + .mac_cap_info[1] = + IEEE80211_HE_MAC_CAP1_TF_MAC_PAD_DUR_16US | + IEEE80211_HE_MAC_CAP1_MULTI_TID_AGG_RX_QOS_8, + .mac_cap_info[2] = + IEEE80211_HE_MAC_CAP2_BSR | + IEEE80211_HE_MAC_CAP2_MU_CASCADING | + IEEE80211_HE_MAC_CAP2_ACK_EN, + .mac_cap_info[3] = + IEEE80211_HE_MAC_CAP3_OMI_CONTROL | + IEEE80211_HE_MAC_CAP3_MAX_AMPDU_LEN_EXP_EXT_3, + .mac_cap_info[4] = IEEE80211_HE_MAC_CAP4_AMSDU_IN_AMPDU, + .phy_cap_info[1] = + IEEE80211_HE_PHY_CAP1_PREAMBLE_PUNC_RX_MASK | + IEEE80211_HE_PHY_CAP1_DEVICE_CLASS_A | + IEEE80211_HE_PHY_CAP1_LDPC_CODING_IN_PAYLOAD | + IEEE80211_HE_PHY_CAP1_MIDAMBLE_RX_TX_MAX_NSTS, + .phy_cap_info[2] = + IEEE80211_HE_PHY_CAP2_NDP_4x_LTF_AND_3_2US | + IEEE80211_HE_PHY_CAP2_STBC_TX_UNDER_80MHZ | + IEEE80211_HE_PHY_CAP2_STBC_RX_UNDER_80MHZ | + IEEE80211_HE_PHY_CAP2_UL_MU_FULL_MU_MIMO | + IEEE80211_HE_PHY_CAP2_UL_MU_PARTIAL_MU_MIMO, + + /* Leave all the other PHY capability bytes + * unset, as DCM, beam forming, RU and PPE + * threshold information are not supported + */ + }, + .he_mcs_nss_supp = { + .rx_mcs_80 = cpu_to_le16(0xfffa), + .tx_mcs_80 = cpu_to_le16(0xfffa), + .rx_mcs_160 = cpu_to_le16(0xffff), + .tx_mcs_160 = cpu_to_le16(0xffff), + .rx_mcs_80p80 = cpu_to_le16(0xffff), + .tx_mcs_80p80 = cpu_to_le16(0xffff), + }, + }, + .eht_cap = { + .has_eht = true, + .eht_cap_elem = { + .mac_cap_info[0] = + IEEE80211_EHT_MAC_CAP0_EPCS_PRIO_ACCESS | + IEEE80211_EHT_MAC_CAP0_OM_CONTROL | + IEEE80211_EHT_MAC_CAP0_TRIG_TXOP_SHARING_MODE1, + .phy_cap_info[0] = + IEEE80211_EHT_PHY_CAP0_242_TONE_RU_GT20MHZ | + IEEE80211_EHT_PHY_CAP0_NDP_4_EHT_LFT_32_GI | + IEEE80211_EHT_PHY_CAP0_PARTIAL_BW_UL_MU_MIMO | + IEEE80211_EHT_PHY_CAP0_SU_BEAMFORMER | + IEEE80211_EHT_PHY_CAP0_SU_BEAMFORMEE, + .phy_cap_info[3] = + IEEE80211_EHT_PHY_CAP3_NG_16_SU_FEEDBACK | + IEEE80211_EHT_PHY_CAP3_NG_16_MU_FEEDBACK | + IEEE80211_EHT_PHY_CAP3_CODEBOOK_4_2_SU_FDBK | + IEEE80211_EHT_PHY_CAP3_CODEBOOK_7_5_MU_FDBK | + IEEE80211_EHT_PHY_CAP3_TRIG_SU_BF_FDBK | + IEEE80211_EHT_PHY_CAP3_TRIG_MU_BF_PART_BW_FDBK | + IEEE80211_EHT_PHY_CAP3_TRIG_CQI_FDBK, + .phy_cap_info[4] = + IEEE80211_EHT_PHY_CAP4_PART_BW_DL_MU_MIMO | + IEEE80211_EHT_PHY_CAP4_PSR_SR_SUPP | + IEEE80211_EHT_PHY_CAP4_POWER_BOOST_FACT_SUPP | + IEEE80211_EHT_PHY_CAP4_EHT_MU_PPDU_4_EHT_LTF_08_GI | + IEEE80211_EHT_PHY_CAP4_MAX_NC_MASK, + .phy_cap_info[5] = + IEEE80211_EHT_PHY_CAP5_NON_TRIG_CQI_FEEDBACK | + IEEE80211_EHT_PHY_CAP5_TX_LESS_242_TONE_RU_SUPP | + IEEE80211_EHT_PHY_CAP5_RX_LESS_242_TONE_RU_SUPP | + IEEE80211_EHT_PHY_CAP5_PPE_THRESHOLD_PRESENT | + IEEE80211_EHT_PHY_CAP5_COMMON_NOMINAL_PKT_PAD_MASK | + IEEE80211_EHT_PHY_CAP5_MAX_NUM_SUPP_EHT_LTF_MASK, + .phy_cap_info[6] = + IEEE80211_EHT_PHY_CAP6_MAX_NUM_SUPP_EHT_LTF_MASK | + IEEE80211_EHT_PHY_CAP6_MCS15_SUPP_MASK, + .phy_cap_info[7] = + IEEE80211_EHT_PHY_CAP7_20MHZ_STA_RX_NDP_WIDER_BW, + }, + + /* For all MCS and bandwidth, set 8 NSS for both Tx and + * Rx + */ + .eht_mcs_nss_supp = { + /* + * Since B0, B1, B2 and B3 are not set in + * the supported channel width set field in the + * HE PHY capabilities information field the + * device is a 20MHz only device on 2.4GHz band. + */ + .only_20mhz = { + .rx_tx_mcs7_max_nss = 0x88, + .rx_tx_mcs9_max_nss = 0x88, + .rx_tx_mcs11_max_nss = 0x88, + .rx_tx_mcs13_max_nss = 0x88, + }, + }, + /* PPE threshold information is not supported */ + }, + }, + { + .types_mask = BIT(NL80211_IFTYPE_AP), .he_cap = { .has_he = true, .he_cap_elem = { @@ -3356,9 +3538,132 @@ static const struct ieee80211_sband_iftype_data sband_capa_2ghz[] = { static const struct ieee80211_sband_iftype_data sband_capa_5ghz[] = { { - /* TODO: should we support other types, e.g., P2P?*/ - .types_mask = BIT(NL80211_IFTYPE_STATION) | - BIT(NL80211_IFTYPE_AP), + /* TODO: should we support other types, e.g., P2P? */ + .types_mask = BIT(NL80211_IFTYPE_STATION), + .he_cap = { + .has_he = true, + .he_cap_elem = { + .mac_cap_info[0] = + IEEE80211_HE_MAC_CAP0_HTC_HE, + .mac_cap_info[1] = + IEEE80211_HE_MAC_CAP1_TF_MAC_PAD_DUR_16US | + IEEE80211_HE_MAC_CAP1_MULTI_TID_AGG_RX_QOS_8, + .mac_cap_info[2] = + IEEE80211_HE_MAC_CAP2_BSR | + IEEE80211_HE_MAC_CAP2_MU_CASCADING | + IEEE80211_HE_MAC_CAP2_ACK_EN, + .mac_cap_info[3] = + IEEE80211_HE_MAC_CAP3_OMI_CONTROL | + IEEE80211_HE_MAC_CAP3_MAX_AMPDU_LEN_EXP_EXT_3, + .mac_cap_info[4] = IEEE80211_HE_MAC_CAP4_AMSDU_IN_AMPDU, + .phy_cap_info[0] = + IEEE80211_HE_PHY_CAP0_CHANNEL_WIDTH_SET_40MHZ_80MHZ_IN_5G | + IEEE80211_HE_PHY_CAP0_CHANNEL_WIDTH_SET_160MHZ_IN_5G | + IEEE80211_HE_PHY_CAP0_CHANNEL_WIDTH_SET_80PLUS80_MHZ_IN_5G, + .phy_cap_info[1] = + IEEE80211_HE_PHY_CAP1_PREAMBLE_PUNC_RX_MASK | + IEEE80211_HE_PHY_CAP1_DEVICE_CLASS_A | + IEEE80211_HE_PHY_CAP1_LDPC_CODING_IN_PAYLOAD | + IEEE80211_HE_PHY_CAP1_MIDAMBLE_RX_TX_MAX_NSTS, + .phy_cap_info[2] = + IEEE80211_HE_PHY_CAP2_NDP_4x_LTF_AND_3_2US | + IEEE80211_HE_PHY_CAP2_STBC_TX_UNDER_80MHZ | + IEEE80211_HE_PHY_CAP2_STBC_RX_UNDER_80MHZ | + IEEE80211_HE_PHY_CAP2_UL_MU_FULL_MU_MIMO | + IEEE80211_HE_PHY_CAP2_UL_MU_PARTIAL_MU_MIMO, + + /* Leave all the other PHY capability bytes + * unset, as DCM, beam forming, RU and PPE + * threshold information are not supported + */ + }, + .he_mcs_nss_supp = { + .rx_mcs_80 = cpu_to_le16(0xfffa), + .tx_mcs_80 = cpu_to_le16(0xfffa), + .rx_mcs_160 = cpu_to_le16(0xfffa), + .tx_mcs_160 = cpu_to_le16(0xfffa), + .rx_mcs_80p80 = cpu_to_le16(0xfffa), + .tx_mcs_80p80 = cpu_to_le16(0xfffa), + }, + }, + .eht_cap = { + .has_eht = true, + .eht_cap_elem = { + .mac_cap_info[0] = + IEEE80211_EHT_MAC_CAP0_EPCS_PRIO_ACCESS | + IEEE80211_EHT_MAC_CAP0_OM_CONTROL | + IEEE80211_EHT_MAC_CAP0_TRIG_TXOP_SHARING_MODE1, + .phy_cap_info[0] = + IEEE80211_EHT_PHY_CAP0_242_TONE_RU_GT20MHZ | + IEEE80211_EHT_PHY_CAP0_NDP_4_EHT_LFT_32_GI | + IEEE80211_EHT_PHY_CAP0_PARTIAL_BW_UL_MU_MIMO | + IEEE80211_EHT_PHY_CAP0_SU_BEAMFORMER | + IEEE80211_EHT_PHY_CAP0_SU_BEAMFORMEE | + IEEE80211_EHT_PHY_CAP0_BEAMFORMEE_SS_80MHZ_MASK, + .phy_cap_info[1] = + IEEE80211_EHT_PHY_CAP1_BEAMFORMEE_SS_80MHZ_MASK | + IEEE80211_EHT_PHY_CAP1_BEAMFORMEE_SS_160MHZ_MASK, + .phy_cap_info[2] = + IEEE80211_EHT_PHY_CAP2_SOUNDING_DIM_80MHZ_MASK | + IEEE80211_EHT_PHY_CAP2_SOUNDING_DIM_160MHZ_MASK, + .phy_cap_info[3] = + IEEE80211_EHT_PHY_CAP3_NG_16_SU_FEEDBACK | + IEEE80211_EHT_PHY_CAP3_NG_16_MU_FEEDBACK | + IEEE80211_EHT_PHY_CAP3_CODEBOOK_4_2_SU_FDBK | + IEEE80211_EHT_PHY_CAP3_CODEBOOK_7_5_MU_FDBK | + IEEE80211_EHT_PHY_CAP3_TRIG_SU_BF_FDBK | + IEEE80211_EHT_PHY_CAP3_TRIG_MU_BF_PART_BW_FDBK | + IEEE80211_EHT_PHY_CAP3_TRIG_CQI_FDBK, + .phy_cap_info[4] = + IEEE80211_EHT_PHY_CAP4_PART_BW_DL_MU_MIMO | + IEEE80211_EHT_PHY_CAP4_PSR_SR_SUPP | + IEEE80211_EHT_PHY_CAP4_POWER_BOOST_FACT_SUPP | + IEEE80211_EHT_PHY_CAP4_EHT_MU_PPDU_4_EHT_LTF_08_GI | + IEEE80211_EHT_PHY_CAP4_MAX_NC_MASK, + .phy_cap_info[5] = + IEEE80211_EHT_PHY_CAP5_NON_TRIG_CQI_FEEDBACK | + IEEE80211_EHT_PHY_CAP5_TX_LESS_242_TONE_RU_SUPP | + IEEE80211_EHT_PHY_CAP5_RX_LESS_242_TONE_RU_SUPP | + IEEE80211_EHT_PHY_CAP5_PPE_THRESHOLD_PRESENT | + IEEE80211_EHT_PHY_CAP5_COMMON_NOMINAL_PKT_PAD_MASK | + IEEE80211_EHT_PHY_CAP5_MAX_NUM_SUPP_EHT_LTF_MASK, + .phy_cap_info[6] = + IEEE80211_EHT_PHY_CAP6_MAX_NUM_SUPP_EHT_LTF_MASK | + IEEE80211_EHT_PHY_CAP6_MCS15_SUPP_MASK, + .phy_cap_info[7] = + IEEE80211_EHT_PHY_CAP7_20MHZ_STA_RX_NDP_WIDER_BW | + IEEE80211_EHT_PHY_CAP7_NON_OFDMA_UL_MU_MIMO_80MHZ | + IEEE80211_EHT_PHY_CAP7_NON_OFDMA_UL_MU_MIMO_160MHZ | + IEEE80211_EHT_PHY_CAP7_MU_BEAMFORMER_80MHZ | + IEEE80211_EHT_PHY_CAP7_MU_BEAMFORMER_160MHZ, + }, + + /* For all MCS and bandwidth, set 8 NSS for both Tx and + * Rx + */ + .eht_mcs_nss_supp = { + /* + * As B1 and B2 are set in the supported + * channel width set field in the HE PHY + * capabilities information field include all + * the following MCS/NSS. + */ + .bw._80 = { + .rx_tx_mcs9_max_nss = 0x88, + .rx_tx_mcs11_max_nss = 0x88, + .rx_tx_mcs13_max_nss = 0x88, + }, + .bw._160 = { + .rx_tx_mcs9_max_nss = 0x88, + .rx_tx_mcs11_max_nss = 0x88, + .rx_tx_mcs13_max_nss = 0x88, + }, + }, + /* PPE threshold information is not supported */ + }, + }, + { + .types_mask = BIT(NL80211_IFTYPE_AP), .he_cap = { .has_he = true, .he_cap_elem = { @@ -3529,9 +3834,153 @@ static const struct ieee80211_sband_iftype_data sband_capa_5ghz[] = { static const struct ieee80211_sband_iftype_data sband_capa_6ghz[] = { { - /* TODO: should we support other types, e.g., P2P?*/ - .types_mask = BIT(NL80211_IFTYPE_STATION) | - BIT(NL80211_IFTYPE_AP), + /* TODO: should we support other types, e.g., P2P? */ + .types_mask = BIT(NL80211_IFTYPE_STATION), + .he_6ghz_capa = { + .capa = cpu_to_le16(IEEE80211_HE_6GHZ_CAP_MIN_MPDU_START | + IEEE80211_HE_6GHZ_CAP_MAX_AMPDU_LEN_EXP | + IEEE80211_HE_6GHZ_CAP_MAX_MPDU_LEN | + IEEE80211_HE_6GHZ_CAP_SM_PS | + IEEE80211_HE_6GHZ_CAP_RD_RESPONDER | + IEEE80211_HE_6GHZ_CAP_TX_ANTPAT_CONS | + IEEE80211_HE_6GHZ_CAP_RX_ANTPAT_CONS), + }, + .he_cap = { + .has_he = true, + .he_cap_elem = { + .mac_cap_info[0] = + IEEE80211_HE_MAC_CAP0_HTC_HE, + .mac_cap_info[1] = + IEEE80211_HE_MAC_CAP1_TF_MAC_PAD_DUR_16US | + IEEE80211_HE_MAC_CAP1_MULTI_TID_AGG_RX_QOS_8, + .mac_cap_info[2] = + IEEE80211_HE_MAC_CAP2_BSR | + IEEE80211_HE_MAC_CAP2_MU_CASCADING | + IEEE80211_HE_MAC_CAP2_ACK_EN, + .mac_cap_info[3] = + IEEE80211_HE_MAC_CAP3_OMI_CONTROL | + IEEE80211_HE_MAC_CAP3_MAX_AMPDU_LEN_EXP_EXT_3, + .mac_cap_info[4] = IEEE80211_HE_MAC_CAP4_AMSDU_IN_AMPDU, + .phy_cap_info[0] = + IEEE80211_HE_PHY_CAP0_CHANNEL_WIDTH_SET_40MHZ_80MHZ_IN_5G | + IEEE80211_HE_PHY_CAP0_CHANNEL_WIDTH_SET_160MHZ_IN_5G | + IEEE80211_HE_PHY_CAP0_CHANNEL_WIDTH_SET_80PLUS80_MHZ_IN_5G, + .phy_cap_info[1] = + IEEE80211_HE_PHY_CAP1_PREAMBLE_PUNC_RX_MASK | + IEEE80211_HE_PHY_CAP1_DEVICE_CLASS_A | + IEEE80211_HE_PHY_CAP1_LDPC_CODING_IN_PAYLOAD | + IEEE80211_HE_PHY_CAP1_MIDAMBLE_RX_TX_MAX_NSTS, + .phy_cap_info[2] = + IEEE80211_HE_PHY_CAP2_NDP_4x_LTF_AND_3_2US | + IEEE80211_HE_PHY_CAP2_STBC_TX_UNDER_80MHZ | + IEEE80211_HE_PHY_CAP2_STBC_RX_UNDER_80MHZ | + IEEE80211_HE_PHY_CAP2_UL_MU_FULL_MU_MIMO | + IEEE80211_HE_PHY_CAP2_UL_MU_PARTIAL_MU_MIMO, + + /* Leave all the other PHY capability bytes + * unset, as DCM, beam forming, RU and PPE + * threshold information are not supported + */ + }, + .he_mcs_nss_supp = { + .rx_mcs_80 = cpu_to_le16(0xfffa), + .tx_mcs_80 = cpu_to_le16(0xfffa), + .rx_mcs_160 = cpu_to_le16(0xfffa), + .tx_mcs_160 = cpu_to_le16(0xfffa), + .rx_mcs_80p80 = cpu_to_le16(0xfffa), + .tx_mcs_80p80 = cpu_to_le16(0xfffa), + }, + }, + .eht_cap = { + .has_eht = true, + .eht_cap_elem = { + .mac_cap_info[0] = + IEEE80211_EHT_MAC_CAP0_EPCS_PRIO_ACCESS | + IEEE80211_EHT_MAC_CAP0_OM_CONTROL | + IEEE80211_EHT_MAC_CAP0_TRIG_TXOP_SHARING_MODE1, + .phy_cap_info[0] = + IEEE80211_EHT_PHY_CAP0_320MHZ_IN_6GHZ | + IEEE80211_EHT_PHY_CAP0_242_TONE_RU_GT20MHZ | + IEEE80211_EHT_PHY_CAP0_NDP_4_EHT_LFT_32_GI | + IEEE80211_EHT_PHY_CAP0_PARTIAL_BW_UL_MU_MIMO | + IEEE80211_EHT_PHY_CAP0_SU_BEAMFORMER | + IEEE80211_EHT_PHY_CAP0_SU_BEAMFORMEE | + IEEE80211_EHT_PHY_CAP0_BEAMFORMEE_SS_80MHZ_MASK, + .phy_cap_info[1] = + IEEE80211_EHT_PHY_CAP1_BEAMFORMEE_SS_80MHZ_MASK | + IEEE80211_EHT_PHY_CAP1_BEAMFORMEE_SS_160MHZ_MASK | + IEEE80211_EHT_PHY_CAP1_BEAMFORMEE_SS_320MHZ_MASK, + .phy_cap_info[2] = + IEEE80211_EHT_PHY_CAP2_SOUNDING_DIM_80MHZ_MASK | + IEEE80211_EHT_PHY_CAP2_SOUNDING_DIM_160MHZ_MASK | + IEEE80211_EHT_PHY_CAP2_SOUNDING_DIM_320MHZ_MASK, + .phy_cap_info[3] = + IEEE80211_EHT_PHY_CAP3_NG_16_SU_FEEDBACK | + IEEE80211_EHT_PHY_CAP3_NG_16_MU_FEEDBACK | + IEEE80211_EHT_PHY_CAP3_CODEBOOK_4_2_SU_FDBK | + IEEE80211_EHT_PHY_CAP3_CODEBOOK_7_5_MU_FDBK | + IEEE80211_EHT_PHY_CAP3_TRIG_SU_BF_FDBK | + IEEE80211_EHT_PHY_CAP3_TRIG_MU_BF_PART_BW_FDBK | + IEEE80211_EHT_PHY_CAP3_TRIG_CQI_FDBK, + .phy_cap_info[4] = + IEEE80211_EHT_PHY_CAP4_PART_BW_DL_MU_MIMO | + IEEE80211_EHT_PHY_CAP4_PSR_SR_SUPP | + IEEE80211_EHT_PHY_CAP4_POWER_BOOST_FACT_SUPP | + IEEE80211_EHT_PHY_CAP4_EHT_MU_PPDU_4_EHT_LTF_08_GI | + IEEE80211_EHT_PHY_CAP4_MAX_NC_MASK, + .phy_cap_info[5] = + IEEE80211_EHT_PHY_CAP5_NON_TRIG_CQI_FEEDBACK | + IEEE80211_EHT_PHY_CAP5_TX_LESS_242_TONE_RU_SUPP | + IEEE80211_EHT_PHY_CAP5_RX_LESS_242_TONE_RU_SUPP | + IEEE80211_EHT_PHY_CAP5_PPE_THRESHOLD_PRESENT | + IEEE80211_EHT_PHY_CAP5_COMMON_NOMINAL_PKT_PAD_MASK | + IEEE80211_EHT_PHY_CAP5_MAX_NUM_SUPP_EHT_LTF_MASK, + .phy_cap_info[6] = + IEEE80211_EHT_PHY_CAP6_MAX_NUM_SUPP_EHT_LTF_MASK | + IEEE80211_EHT_PHY_CAP6_MCS15_SUPP_MASK | + IEEE80211_EHT_PHY_CAP6_EHT_DUP_6GHZ_SUPP, + .phy_cap_info[7] = + IEEE80211_EHT_PHY_CAP7_20MHZ_STA_RX_NDP_WIDER_BW | + IEEE80211_EHT_PHY_CAP7_NON_OFDMA_UL_MU_MIMO_80MHZ | + IEEE80211_EHT_PHY_CAP7_NON_OFDMA_UL_MU_MIMO_160MHZ | + IEEE80211_EHT_PHY_CAP7_NON_OFDMA_UL_MU_MIMO_320MHZ | + IEEE80211_EHT_PHY_CAP7_MU_BEAMFORMER_80MHZ | + IEEE80211_EHT_PHY_CAP7_MU_BEAMFORMER_160MHZ | + IEEE80211_EHT_PHY_CAP7_MU_BEAMFORMER_320MHZ, + }, + + /* For all MCS and bandwidth, set 8 NSS for both Tx and + * Rx + */ + .eht_mcs_nss_supp = { + /* + * As B1 and B2 are set in the supported + * channel width set field in the HE PHY + * capabilities information field and 320MHz in + * 6GHz is supported include all the following + * MCS/NSS. + */ + .bw._80 = { + .rx_tx_mcs9_max_nss = 0x88, + .rx_tx_mcs11_max_nss = 0x88, + .rx_tx_mcs13_max_nss = 0x88, + }, + .bw._160 = { + .rx_tx_mcs9_max_nss = 0x88, + .rx_tx_mcs11_max_nss = 0x88, + .rx_tx_mcs13_max_nss = 0x88, + }, + .bw._320 = { + .rx_tx_mcs9_max_nss = 0x88, + .rx_tx_mcs11_max_nss = 0x88, + .rx_tx_mcs13_max_nss = 0x88, + }, + }, + /* PPE threshold information is not supported */ + }, + }, + { + .types_mask = BIT(NL80211_IFTYPE_AP), .he_6ghz_capa = { .capa = cpu_to_le16(IEEE80211_HE_6GHZ_CAP_MIN_MPDU_START | IEEE80211_HE_6GHZ_CAP_MAX_AMPDU_LEN_EXP | @@ -3896,7 +4345,6 @@ static int mac80211_hwsim_new_radio(struct genl_info *info, hw->wiphy->max_remain_on_channel_duration = 1000; data->if_combination.radar_detect_widths = 0; data->if_combination.num_different_channels = data->channels; - data->chanctx = NULL; } else { data->if_combination.num_different_channels = 1; data->if_combination.radar_detect_widths = @@ -4471,13 +4919,9 @@ static int hwsim_cloned_frame_received_nl(struct sk_buff *skb_2, if (data2->use_chanctx) { if (data2->tmp_chan) channel = data2->tmp_chan; - else if (data2->chanctx) - channel = data2->chanctx->def.chan; } else { channel = data2->channel; } - if (!channel) - goto out; if (!hwsim_virtio_enabled) { if (hwsim_net_get_netgroup(genl_info_net(info)) != @@ -4508,6 +4952,7 @@ static int hwsim_cloned_frame_received_nl(struct sk_buff *skb_2, rx_status.freq); if (!iter_data.channel) goto out; + rx_status.band = iter_data.channel->band; mutex_lock(&data2->mutex); if (!hwsim_chans_compat(iter_data.channel, channel)) { @@ -4520,11 +4965,13 @@ static int hwsim_cloned_frame_received_nl(struct sk_buff *skb_2, } } mutex_unlock(&data2->mutex); + } else if (!channel) { + goto out; } else { rx_status.freq = channel->center_freq; + rx_status.band = channel->band; } - rx_status.band = channel->band; rx_status.rate_idx = nla_get_u32(info->attrs[HWSIM_ATTR_RX_RATE]); rx_status.signal = nla_get_u32(info->attrs[HWSIM_ATTR_SIGNAL]); @@ -4534,10 +4981,7 @@ static int hwsim_cloned_frame_received_nl(struct sk_buff *skb_2, ieee80211_is_probe_resp(hdr->frame_control)) rx_status.boottime_ns = ktime_get_boottime_ns(); - memcpy(IEEE80211_SKB_RXCB(skb), &rx_status, sizeof(rx_status)); - data2->rx_pkts++; - data2->rx_bytes += skb->len; - ieee80211_rx_irqsafe(data2->hw, skb); + mac80211_hwsim_rx(data2, &rx_status, skb); return 0; err: @@ -4912,6 +5356,7 @@ static struct genl_family hwsim_genl_family __ro_after_init = { .module = THIS_MODULE, .small_ops = hwsim_ops, .n_small_ops = ARRAY_SIZE(hwsim_ops), + .resv_start_op = HWSIM_CMD_DEL_MAC_ADDR + 1, .mcgrps = hwsim_mcgrps, .n_mcgrps = ARRAY_SIZE(hwsim_mcgrps), }; diff --git a/drivers/net/wireless/marvell/libertas/cfg.c b/drivers/net/wireless/marvell/libertas/cfg.c index b0b3f59dabc6..3e065cbb0af9 100644 --- a/drivers/net/wireless/marvell/libertas/cfg.c +++ b/drivers/net/wireless/marvell/libertas/cfg.c @@ -546,7 +546,7 @@ static int lbs_ret_scan(struct lbs_private *priv, unsigned long dummy, pos = scanresp->bssdesc_and_tlvbuffer; lbs_deb_hex(LBS_DEB_SCAN, "SCAN_RSP", scanresp->bssdesc_and_tlvbuffer, - scanresp->bssdescriptsize); + bsssize); tsfdesc = pos + bsssize; tsfsize = 4 + 8 * scanresp->nr_sets; @@ -1435,7 +1435,7 @@ static int lbs_cfg_disconnect(struct wiphy *wiphy, struct net_device *dev, } static int lbs_cfg_set_default_key(struct wiphy *wiphy, - struct net_device *netdev, + struct net_device *netdev, int link_id, u8 key_index, bool unicast, bool multicast) { @@ -1455,8 +1455,8 @@ static int lbs_cfg_set_default_key(struct wiphy *wiphy, static int lbs_cfg_add_key(struct wiphy *wiphy, struct net_device *netdev, - u8 idx, bool pairwise, const u8 *mac_addr, - struct key_params *params) + int link_id, u8 idx, bool pairwise, + const u8 *mac_addr, struct key_params *params) { struct lbs_private *priv = wiphy_priv(wiphy); u16 key_info; @@ -1516,7 +1516,8 @@ static int lbs_cfg_add_key(struct wiphy *wiphy, struct net_device *netdev, static int lbs_cfg_del_key(struct wiphy *wiphy, struct net_device *netdev, - u8 key_index, bool pairwise, const u8 *mac_addr) + int link_id, u8 key_index, bool pairwise, + const u8 *mac_addr) { lbs_deb_assoc("del_key: key_idx %d, mac_addr %pM\n", diff --git a/drivers/net/wireless/marvell/libertas/ethtool.c b/drivers/net/wireless/marvell/libertas/ethtool.c index d8e4f29b690d..9f53308a9935 100644 --- a/drivers/net/wireless/marvell/libertas/ethtool.c +++ b/drivers/net/wireless/marvell/libertas/ethtool.c @@ -20,8 +20,8 @@ static void lbs_ethtool_get_drvinfo(struct net_device *dev, priv->fwrelease >> 16 & 0xff, priv->fwrelease >> 8 & 0xff, priv->fwrelease & 0xff); - strlcpy(info->driver, "libertas", sizeof(info->driver)); - strlcpy(info->version, lbs_driver_version, sizeof(info->version)); + strscpy(info->driver, "libertas", sizeof(info->driver)); + strscpy(info->version, lbs_driver_version, sizeof(info->version)); } /* diff --git a/drivers/net/wireless/marvell/libertas/main.c b/drivers/net/wireless/marvell/libertas/main.c index 5c9f295536ea..8f5220cee112 100644 --- a/drivers/net/wireless/marvell/libertas/main.c +++ b/drivers/net/wireless/marvell/libertas/main.c @@ -39,8 +39,7 @@ unsigned int lbs_debug; EXPORT_SYMBOL_GPL(lbs_debug); module_param_named(libertas_debug, lbs_debug, int, 0644); -unsigned int lbs_disablemesh; -EXPORT_SYMBOL_GPL(lbs_disablemesh); +static unsigned int lbs_disablemesh; module_param_named(libertas_disablemesh, lbs_disablemesh, int, 0644); diff --git a/drivers/net/wireless/marvell/mwifiex/11n_rxreorder.c b/drivers/net/wireless/marvell/mwifiex/11n_rxreorder.c index bd835288ce57..a04b66284af4 100644 --- a/drivers/net/wireless/marvell/mwifiex/11n_rxreorder.c +++ b/drivers/net/wireless/marvell/mwifiex/11n_rxreorder.c @@ -335,7 +335,7 @@ mwifiex_11n_create_rx_reorder_tbl(struct mwifiex_private *priv, u8 *ta, struct mwifiex_sta_node *node; /* - * If we get a TID, ta pair which is already present dispatch all the + * If we get a TID, ta pair which is already present dispatch all * the packets and move the window size until the ssn */ tbl = mwifiex_11n_get_rx_reorder_tbl(priv, tid, ta); diff --git a/drivers/net/wireless/marvell/mwifiex/cfg80211.c b/drivers/net/wireless/marvell/mwifiex/cfg80211.c index 134114ac1ac0..535995e8279f 100644 --- a/drivers/net/wireless/marvell/mwifiex/cfg80211.c +++ b/drivers/net/wireless/marvell/mwifiex/cfg80211.c @@ -142,7 +142,8 @@ static void *mwifiex_cfg80211_get_adapter(struct wiphy *wiphy) */ static int mwifiex_cfg80211_del_key(struct wiphy *wiphy, struct net_device *netdev, - u8 key_index, bool pairwise, const u8 *mac_addr) + int link_id, u8 key_index, bool pairwise, + const u8 *mac_addr) { struct mwifiex_private *priv = mwifiex_netdev_get_priv(netdev); static const u8 bc_mac[] = {0xff, 0xff, 0xff, 0xff, 0xff, 0xff}; @@ -431,7 +432,7 @@ mwifiex_cfg80211_set_power_mgmt(struct wiphy *wiphy, */ static int mwifiex_cfg80211_set_default_key(struct wiphy *wiphy, struct net_device *netdev, - u8 key_index, bool unicast, + int link_id, u8 key_index, bool unicast, bool multicast) { struct mwifiex_private *priv = mwifiex_netdev_get_priv(netdev); @@ -456,8 +457,8 @@ mwifiex_cfg80211_set_default_key(struct wiphy *wiphy, struct net_device *netdev, */ static int mwifiex_cfg80211_add_key(struct wiphy *wiphy, struct net_device *netdev, - u8 key_index, bool pairwise, const u8 *mac_addr, - struct key_params *params) + int link_id, u8 key_index, bool pairwise, + const u8 *mac_addr, struct key_params *params) { struct mwifiex_private *priv = mwifiex_netdev_get_priv(netdev); struct mwifiex_wep_key *wep_key; @@ -494,6 +495,7 @@ mwifiex_cfg80211_add_key(struct wiphy *wiphy, struct net_device *netdev, static int mwifiex_cfg80211_set_default_mgmt_key(struct wiphy *wiphy, struct net_device *netdev, + int link_id, u8 key_index) { struct mwifiex_private *priv = mwifiex_netdev_get_priv(netdev); diff --git a/drivers/net/wireless/marvell/mwifiex/fw.h b/drivers/net/wireless/marvell/mwifiex/fw.h index 26a48d8f49be..b4f945a549f7 100644 --- a/drivers/net/wireless/marvell/mwifiex/fw.h +++ b/drivers/net/wireless/marvell/mwifiex/fw.h @@ -2104,7 +2104,7 @@ struct mwifiex_fw_mef_entry { struct host_cmd_ds_mef_cfg { __le32 criteria; __le16 num_entries; - struct mwifiex_fw_mef_entry mef_entry[]; + u8 mef_entry_data[]; } __packed; #define CONNECTION_TYPE_INFRA 0 @@ -2254,7 +2254,7 @@ struct coalesce_receive_filt_rule { struct host_cmd_ds_coalesce_cfg { __le16 action; __le16 num_of_rules; - struct coalesce_receive_filt_rule rule[]; + u8 rule_data[]; } __packed; struct host_cmd_ds_multi_chan_policy { diff --git a/drivers/net/wireless/marvell/mwifiex/init.c b/drivers/net/wireless/marvell/mwifiex/init.c index fc77489cc511..7dddb4b5dea1 100644 --- a/drivers/net/wireless/marvell/mwifiex/init.c +++ b/drivers/net/wireless/marvell/mwifiex/init.c @@ -51,9 +51,10 @@ static void wakeup_timer_fn(struct timer_list *t) adapter->if_ops.card_reset(adapter); } -static void fw_dump_timer_fn(struct timer_list *t) +static void fw_dump_work(struct work_struct *work) { - struct mwifiex_adapter *adapter = from_timer(adapter, t, devdump_timer); + struct mwifiex_adapter *adapter = + container_of(work, struct mwifiex_adapter, devdump_work.work); mwifiex_upload_device_dump(adapter); } @@ -309,7 +310,7 @@ static void mwifiex_init_adapter(struct mwifiex_adapter *adapter) adapter->active_scan_triggered = false; timer_setup(&adapter->wakeup_timer, wakeup_timer_fn, 0); adapter->devdump_len = 0; - timer_setup(&adapter->devdump_timer, fw_dump_timer_fn, 0); + INIT_DELAYED_WORK(&adapter->devdump_work, fw_dump_work); } /* @@ -388,7 +389,7 @@ static void mwifiex_adapter_cleanup(struct mwifiex_adapter *adapter) { del_timer(&adapter->wakeup_timer); - del_timer_sync(&adapter->devdump_timer); + cancel_delayed_work_sync(&adapter->devdump_work); mwifiex_cancel_all_pending_cmd(adapter); wake_up_interruptible(&adapter->cmd_wait_q.wait); wake_up_interruptible(&adapter->hs_activate_wait_q); diff --git a/drivers/net/wireless/marvell/mwifiex/main.h b/drivers/net/wireless/marvell/mwifiex/main.h index 87729d251fed..63f861e6b28a 100644 --- a/drivers/net/wireless/marvell/mwifiex/main.h +++ b/drivers/net/wireless/marvell/mwifiex/main.h @@ -37,6 +37,7 @@ #include <linux/pm_runtime.h> #include <linux/slab.h> #include <linux/of_irq.h> +#include <linux/workqueue.h> #include "decl.h" #include "ioctl.h" @@ -1043,7 +1044,7 @@ struct mwifiex_adapter { /* Device dump data/length */ void *devdump_data; int devdump_len; - struct timer_list devdump_timer; + struct delayed_work devdump_work; bool ignore_btcoex_events; }; diff --git a/drivers/net/wireless/marvell/mwifiex/pcie.c b/drivers/net/wireless/marvell/mwifiex/pcie.c index f7f9277602a5..5dcf61761a16 100644 --- a/drivers/net/wireless/marvell/mwifiex/pcie.c +++ b/drivers/net/wireless/marvell/mwifiex/pcie.c @@ -644,7 +644,7 @@ static int mwifiex_pm_wakeup_card(struct mwifiex_adapter *adapter) { struct pcie_service_card *card = adapter->card; const struct mwifiex_pcie_card_reg *reg = card->pcie.reg; - int retval; + int retval __maybe_unused; mwifiex_dbg(adapter, EVENT, "event: Wakeup device...\n"); diff --git a/drivers/net/wireless/marvell/mwifiex/sta_cmd.c b/drivers/net/wireless/marvell/mwifiex/sta_cmd.c index 512b5bb9cf6f..e2800a831c8e 100644 --- a/drivers/net/wireless/marvell/mwifiex/sta_cmd.c +++ b/drivers/net/wireless/marvell/mwifiex/sta_cmd.c @@ -1435,7 +1435,7 @@ mwifiex_cmd_mef_cfg(struct mwifiex_private *priv, mef_entry = (struct mwifiex_fw_mef_entry *)pos; mef_entry->mode = mef->mef_entry[i].mode; mef_entry->action = mef->mef_entry[i].action; - pos += sizeof(*mef_cfg->mef_entry); + pos += sizeof(*mef_entry); if (mwifiex_cmd_append_rpn_expression(priv, &mef->mef_entry[i], &pos)) @@ -1631,7 +1631,7 @@ mwifiex_cmd_coalesce_cfg(struct mwifiex_private *priv, coalesce_cfg->action = cpu_to_le16(cmd_action); coalesce_cfg->num_of_rules = cpu_to_le16(cfg->num_of_rules); - rule = coalesce_cfg->rule; + rule = (void *)coalesce_cfg->rule_data; for (cnt = 0; cnt < cfg->num_of_rules; cnt++) { rule->header.type = cpu_to_le16(TLV_TYPE_COALESCE_RULE); diff --git a/drivers/net/wireless/marvell/mwifiex/sta_event.c b/drivers/net/wireless/marvell/mwifiex/sta_event.c index b95e90a7d124..df9cdd10a494 100644 --- a/drivers/net/wireless/marvell/mwifiex/sta_event.c +++ b/drivers/net/wireless/marvell/mwifiex/sta_event.c @@ -611,8 +611,8 @@ mwifiex_fw_dump_info_event(struct mwifiex_private *priv, * transmission event get lost, in this cornel case, * user would still get partial of the dump. */ - mod_timer(&adapter->devdump_timer, - jiffies + msecs_to_jiffies(MWIFIEX_TIMER_10S)); + schedule_delayed_work(&adapter->devdump_work, + msecs_to_jiffies(MWIFIEX_TIMER_10S)); } /* Overflow check */ @@ -623,7 +623,7 @@ mwifiex_fw_dump_info_event(struct mwifiex_private *priv, adapter->event_skb->data, event_skb->len); adapter->devdump_len += event_skb->len; - if (le16_to_cpu(fw_dump_hdr->type == FW_DUMP_INFO_ENDED)) { + if (le16_to_cpu(fw_dump_hdr->type) == FW_DUMP_INFO_ENDED) { mwifiex_dbg(adapter, MSG, "receive end of transmission flag event!\n"); goto upload_dump; @@ -631,7 +631,7 @@ mwifiex_fw_dump_info_event(struct mwifiex_private *priv, return; upload_dump: - del_timer_sync(&adapter->devdump_timer); + cancel_delayed_work_sync(&adapter->devdump_work); mwifiex_upload_device_dump(adapter); } diff --git a/drivers/net/wireless/marvell/mwifiex/usb.c b/drivers/net/wireless/marvell/mwifiex/usb.c index c2f2ce2a3f95..d3ab9572e711 100644 --- a/drivers/net/wireless/marvell/mwifiex/usb.c +++ b/drivers/net/wireless/marvell/mwifiex/usb.c @@ -911,14 +911,14 @@ static int mwifiex_usb_prepare_tx_aggr_skb(struct mwifiex_adapter *adapter, memcpy(payload, skb_tmp->data, skb_tmp->len); if (skb_queue_empty(&port->tx_aggr.aggr_list)) { /* do not padding for last packet*/ - *(u16 *)payload = cpu_to_le16(skb_tmp->len); - *(u16 *)&payload[2] = + *(__le16 *)payload = cpu_to_le16(skb_tmp->len); + *(__le16 *)&payload[2] = cpu_to_le16(MWIFIEX_TYPE_AGGR_DATA_V2 | 0x80); skb_trim(skb_aggr, skb_aggr->len - pad); } else { /* add aggregation interface header */ - *(u16 *)payload = cpu_to_le16(skb_tmp->len + pad); - *(u16 *)&payload[2] = + *(__le16 *)payload = cpu_to_le16(skb_tmp->len + pad); + *(__le16 *)&payload[2] = cpu_to_le16(MWIFIEX_TYPE_AGGR_DATA_V2); } @@ -1097,9 +1097,9 @@ send_aggr_buf: } payload = skb->data; - *(u16 *)&payload[2] = + *(__le16 *)&payload[2] = cpu_to_le16(MWIFIEX_TYPE_AGGR_DATA_V2 | 0x80); - *(u16 *)payload = cpu_to_le16(skb->len); + *(__le16 *)payload = cpu_to_le16(skb->len); skb_send = skb; context = &port->tx_data_list[port->tx_data_ix++]; return mwifiex_usb_construct_send_urb(adapter, port, ep, diff --git a/drivers/net/wireless/mediatek/mt76/dma.c b/drivers/net/wireless/mediatek/mt76/dma.c index 40cb91097b2e..4901aa02b4fb 100644 --- a/drivers/net/wireless/mediatek/mt76/dma.c +++ b/drivers/net/wireless/mediatek/mt76/dma.c @@ -758,7 +758,7 @@ mt76_dma_init(struct mt76_dev *dev, dev->napi_dev.threaded = 1; mt76_for_each_q_rx(dev, i) { - netif_napi_add(&dev->napi_dev, &dev->napi[i], poll, 64); + netif_napi_add(&dev->napi_dev, &dev->napi[i], poll); mt76_dma_rx_fill(dev, &dev->q_rx[i]); napi_enable(&dev->napi[i]); } diff --git a/drivers/net/wireless/mediatek/mt76/mt76.h b/drivers/net/wireless/mediatek/mt76/mt76.h index 4da77d47b0a6..87db9498dea4 100644 --- a/drivers/net/wireless/mediatek/mt76/mt76.h +++ b/drivers/net/wireless/mediatek/mt76/mt76.h @@ -252,6 +252,30 @@ struct mt76_queue_ops { void (*reset_q)(struct mt76_dev *dev, struct mt76_queue *q); }; +enum mt76_phy_type { + MT_PHY_TYPE_CCK, + MT_PHY_TYPE_OFDM, + MT_PHY_TYPE_HT, + MT_PHY_TYPE_HT_GF, + MT_PHY_TYPE_VHT, + MT_PHY_TYPE_HE_SU = 8, + MT_PHY_TYPE_HE_EXT_SU, + MT_PHY_TYPE_HE_TB, + MT_PHY_TYPE_HE_MU, + __MT_PHY_TYPE_HE_MAX, +}; + +struct mt76_sta_stats { + u64 tx_mode[__MT_PHY_TYPE_HE_MAX]; + u64 tx_bw[4]; /* 20, 40, 80, 160 */ + u64 tx_nss[4]; /* 1, 2, 3, 4 */ + u64 tx_mcs[16]; /* mcs idx */ + u64 tx_bytes; + u32 tx_packets; + u32 tx_retries; + u32 tx_failed; +}; + enum mt76_wcid_flags { MT_WCID_FLAG_CHECK_PS, MT_WCID_FLAG_PS, @@ -299,6 +323,8 @@ struct mt76_wcid { struct list_head list; struct idr pktid; + + struct mt76_sta_stats stats; }; struct mt76_txq { @@ -342,7 +368,8 @@ struct mt76_rx_tid { #define MT_PACKET_ID_MASK GENMASK(6, 0) #define MT_PACKET_ID_NO_ACK 0 #define MT_PACKET_ID_NO_SKB 1 -#define MT_PACKET_ID_FIRST 2 +#define MT_PACKET_ID_WED 2 +#define MT_PACKET_ID_FIRST 3 #define MT_PACKET_ID_HAS_RATE BIT(7) /* This is timer for when to give up when waiting for TXS callback, * with starting time being the time at which the DMA_DONE callback @@ -527,7 +554,6 @@ struct mt76_usb { struct mt76_reg_pair *rp; int rp_len; u32 base; - bool burst; } mcu; }; @@ -815,26 +841,6 @@ struct mt76_power_limits { s8 ru[7][12]; }; -enum mt76_phy_type { - MT_PHY_TYPE_CCK, - MT_PHY_TYPE_OFDM, - MT_PHY_TYPE_HT, - MT_PHY_TYPE_HT_GF, - MT_PHY_TYPE_VHT, - MT_PHY_TYPE_HE_SU = 8, - MT_PHY_TYPE_HE_EXT_SU, - MT_PHY_TYPE_HE_TB, - MT_PHY_TYPE_HE_MU, - __MT_PHY_TYPE_HE_MAX, -}; - -struct mt76_sta_stats { - u64 tx_mode[__MT_PHY_TYPE_HE_MAX]; - u64 tx_bw[4]; /* 20, 40, 80, 160 */ - u64 tx_nss[4]; /* 1, 2, 3, 4 */ - u64 tx_mcs[16]; /* mcs idx */ -}; - struct mt76_ethtool_worker_info { u64 *data; int idx; diff --git a/drivers/net/wireless/mediatek/mt76/mt7603/main.c b/drivers/net/wireless/mediatek/mt76/mt7603/main.c index 051715ed90dd..ca50feb0b3a9 100644 --- a/drivers/net/wireless/mediatek/mt76/mt7603/main.c +++ b/drivers/net/wireless/mediatek/mt76/mt7603/main.c @@ -658,7 +658,7 @@ mt7603_sta_rate_tbl_update(struct ieee80211_hw *hw, struct ieee80211_vif *vif, mt7603_wtbl_set_rates(dev, msta, NULL, msta->rates); msta->rate_probe = false; mt7603_wtbl_set_smps(dev, msta, - sta->smps_mode == IEEE80211_SMPS_DYNAMIC); + sta->deflink.smps_mode == IEEE80211_SMPS_DYNAMIC); spin_unlock_bh(&dev->mt76.lock); } diff --git a/drivers/net/wireless/mediatek/mt76/mt7615/main.c b/drivers/net/wireless/mediatek/mt76/mt7615/main.c index 9bf8545c8c17..8d4733f87cda 100644 --- a/drivers/net/wireless/mediatek/mt76/mt7615/main.c +++ b/drivers/net/wireless/mediatek/mt76/mt7615/main.c @@ -1195,12 +1195,16 @@ static void mt7615_sta_set_decap_offload(struct ieee80211_hw *hw, struct mt7615_dev *dev = mt7615_hw_dev(hw); struct mt7615_sta *msta = (struct mt7615_sta *)sta->drv_priv; + mt7615_mutex_acquire(dev); + if (enabled) set_bit(MT_WCID_FLAG_HDR_TRANS, &msta->wcid.flags); else clear_bit(MT_WCID_FLAG_HDR_TRANS, &msta->wcid.flags); mt7615_mcu_set_sta_decap_offload(dev, vif, sta); + + mt7615_mutex_release(dev); } #ifdef CONFIG_PM diff --git a/drivers/net/wireless/mediatek/mt76/mt7615/sdio.c b/drivers/net/wireless/mediatek/mt76/mt7615/sdio.c index 49ab3a1f3b9b..304212f5f8da 100644 --- a/drivers/net/wireless/mediatek/mt76/mt7615/sdio.c +++ b/drivers/net/wireless/mediatek/mt76/mt7615/sdio.c @@ -83,6 +83,7 @@ static int mt7663s_probe(struct sdio_func *func, .tx_complete_skb = mt7663_usb_sdio_tx_complete_skb, .tx_status_data = mt7663_usb_sdio_tx_status_data, .rx_skb = mt7615_queue_rx_skb, + .rx_check = mt7615_rx_check, .sta_ps = mt7615_sta_ps, .sta_add = mt7615_mac_sta_add, .sta_remove = mt7615_mac_sta_remove, @@ -180,7 +181,6 @@ static void mt7663s_remove(struct sdio_func *func) mt76_free_device(&dev->mt76); } -#ifdef CONFIG_PM static int mt7663s_suspend(struct device *dev) { struct sdio_func *func = dev_to_sdio_func(dev); @@ -235,28 +235,20 @@ static int mt7663s_resume(struct device *dev) return err; } -static const struct dev_pm_ops mt7663s_pm_ops = { - .suspend = mt7663s_suspend, - .resume = mt7663s_resume, -}; -#endif - MODULE_DEVICE_TABLE(sdio, mt7663s_table); MODULE_FIRMWARE(MT7663_OFFLOAD_FIRMWARE_N9); MODULE_FIRMWARE(MT7663_OFFLOAD_ROM_PATCH); MODULE_FIRMWARE(MT7663_FIRMWARE_N9); MODULE_FIRMWARE(MT7663_ROM_PATCH); +static DEFINE_SIMPLE_DEV_PM_OPS(mt7663s_pm_ops, mt7663s_suspend, mt7663s_resume); + static struct sdio_driver mt7663s_driver = { .name = KBUILD_MODNAME, .probe = mt7663s_probe, .remove = mt7663s_remove, .id_table = mt7663s_table, -#ifdef CONFIG_PM - .drv = { - .pm = &mt7663s_pm_ops, - } -#endif + .drv.pm = pm_sleep_ptr(&mt7663s_pm_ops), }; module_sdio_driver(mt7663s_driver); diff --git a/drivers/net/wireless/mediatek/mt76/mt7615/usb.c b/drivers/net/wireless/mediatek/mt76/mt7615/usb.c index 967641aebf5f..f2d651d7adff 100644 --- a/drivers/net/wireless/mediatek/mt76/mt7615/usb.c +++ b/drivers/net/wireless/mediatek/mt76/mt7615/usb.c @@ -119,6 +119,7 @@ static int mt7663u_probe(struct usb_interface *usb_intf, .tx_complete_skb = mt7663_usb_sdio_tx_complete_skb, .tx_status_data = mt7663_usb_sdio_tx_status_data, .rx_skb = mt7615_queue_rx_skb, + .rx_check = mt7615_rx_check, .sta_ps = mt7615_sta_ps, .sta_add = mt7615_mac_sta_add, .sta_remove = mt7615_mac_sta_remove, diff --git a/drivers/net/wireless/mediatek/mt76/mt76_connac.h b/drivers/net/wireless/mediatek/mt76/mt76_connac.h index 75afcb469d3c..635192c878cb 100644 --- a/drivers/net/wireless/mediatek/mt76/mt76_connac.h +++ b/drivers/net/wireless/mediatek/mt76/mt76_connac.h @@ -63,6 +63,12 @@ enum { REPEATER_BSSID_MAX = 0x3f, }; +struct mt76_connac_reg_map { + u32 phys; + u32 maps; + u32 size; +}; + struct mt76_connac_pm { bool enable:1; bool enable_user:1; @@ -348,9 +354,10 @@ void mt76_connac2_mac_write_txwi(struct mt76_dev *dev, __le32 *txwi, struct sk_buff *skb, struct mt76_wcid *wcid, struct ieee80211_key_conf *key, int pid, enum mt76_txq_id qid, u32 changed); +bool mt76_connac2_mac_fill_txs(struct mt76_dev *dev, struct mt76_wcid *wcid, + __le32 *txs_data); bool mt76_connac2_mac_add_txs_skb(struct mt76_dev *dev, struct mt76_wcid *wcid, - int pid, __le32 *txs_data, - struct mt76_sta_stats *stats); + int pid, __le32 *txs_data); void mt76_connac2_mac_decode_he_radiotap(struct mt76_dev *dev, struct sk_buff *skb, __le32 *rxv, u32 mode); diff --git a/drivers/net/wireless/mediatek/mt76/mt76_connac2_mac.h b/drivers/net/wireless/mediatek/mt76/mt76_connac2_mac.h index 67ce216fb564..f33171bcd343 100644 --- a/drivers/net/wireless/mediatek/mt76/mt76_connac2_mac.h +++ b/drivers/net/wireless/mediatek/mt76/mt76_connac2_mac.h @@ -158,6 +158,14 @@ enum { #define MT_TXS4_TIMESTAMP GENMASK(31, 0) +/* PPDU based TXS */ +#define MT_TXS5_MPDU_TX_BYTE GENMASK(22, 0) +#define MT_TXS5_MPDU_TX_CNT GENMASK(31, 23) + +#define MT_TXS6_MPDU_FAIL_CNT GENMASK(31, 23) + +#define MT_TXS7_MPDU_RETRY_CNT GENMASK(31, 23) + /* RXD DW1 */ #define MT_RXD1_NORMAL_WLAN_IDX GENMASK(9, 0) #define MT_RXD1_NORMAL_GROUP_1 BIT(11) diff --git a/drivers/net/wireless/mediatek/mt76/mt76_connac_mac.c b/drivers/net/wireless/mediatek/mt76/mt76_connac_mac.c index 18dea8e1fb20..34ac3d81a510 100644 --- a/drivers/net/wireless/mediatek/mt76/mt76_connac_mac.c +++ b/drivers/net/wireless/mediatek/mt76/mt76_connac_mac.c @@ -490,6 +490,10 @@ void mt76_connac2_mac_write_txwi(struct mt76_dev *dev, __le32 *txwi, p_fmt = mt76_is_mmio(dev) ? MT_TX_TYPE_CT : MT_TX_TYPE_SF; q_idx = wmm_idx * MT76_CONNAC_MAX_WMM_SETS + mt76_connac_lmac_mapping(skb_get_queue_mapping(skb)); + + /* counting non-offloading skbs */ + wcid->stats.tx_bytes += skb->len; + wcid->stats.tx_packets++; } val = FIELD_PREP(MT_TXD0_TX_BYTES, skb->len + sz_txd) | @@ -550,35 +554,29 @@ void mt76_connac2_mac_write_txwi(struct mt76_dev *dev, __le32 *txwi, } EXPORT_SYMBOL_GPL(mt76_connac2_mac_write_txwi); -bool mt76_connac2_mac_add_txs_skb(struct mt76_dev *dev, struct mt76_wcid *wcid, - int pid, __le32 *txs_data, - struct mt76_sta_stats *stats) +bool mt76_connac2_mac_fill_txs(struct mt76_dev *dev, struct mt76_wcid *wcid, + __le32 *txs_data) { + struct mt76_sta_stats *stats = &wcid->stats; struct ieee80211_supported_band *sband; struct mt76_phy *mphy; - struct ieee80211_tx_info *info; - struct sk_buff_head list; struct rate_info rate = {}; - struct sk_buff *skb; bool cck = false; u32 txrate, txs, mode; - mt76_tx_status_lock(dev, &list); - skb = mt76_tx_status_skb_get(dev, wcid, pid, &list); - if (!skb) - goto out; - txs = le32_to_cpu(txs_data[0]); - info = IEEE80211_SKB_CB(skb); - if (!(txs & MT_TXS0_ACK_ERROR_MASK)) - info->flags |= IEEE80211_TX_STAT_ACK; - - info->status.ampdu_len = 1; - info->status.ampdu_ack_len = !!(info->flags & - IEEE80211_TX_STAT_ACK); - - info->status.rates[0].idx = -1; + /* PPDU based reporting */ + if (FIELD_GET(MT_TXS0_TXS_FORMAT, txs) > 1) { + stats->tx_bytes += + le32_get_bits(txs_data[5], MT_TXS5_MPDU_TX_BYTE); + stats->tx_packets += + le32_get_bits(txs_data[5], MT_TXS5_MPDU_TX_CNT); + stats->tx_failed += + le32_get_bits(txs_data[6], MT_TXS6_MPDU_FAIL_CNT); + stats->tx_retries += + le32_get_bits(txs_data[7], MT_TXS7_MPDU_RETRY_CNT); + } txrate = FIELD_GET(MT_TXS0_TX_RATE, txs); @@ -613,7 +611,7 @@ bool mt76_connac2_mac_add_txs_skb(struct mt76_dev *dev, struct mt76_wcid *wcid, case MT_PHY_TYPE_HT: case MT_PHY_TYPE_HT_GF: if (rate.mcs > 31) - goto out; + return false; rate.flags = RATE_INFO_FLAGS_MCS; if (wcid->rate.flags & RATE_INFO_FLAGS_SHORT_GI) @@ -621,7 +619,7 @@ bool mt76_connac2_mac_add_txs_skb(struct mt76_dev *dev, struct mt76_wcid *wcid, break; case MT_PHY_TYPE_VHT: if (rate.mcs > 9) - goto out; + return false; rate.flags = RATE_INFO_FLAGS_VHT_MCS; break; @@ -630,14 +628,14 @@ bool mt76_connac2_mac_add_txs_skb(struct mt76_dev *dev, struct mt76_wcid *wcid, case MT_PHY_TYPE_HE_TB: case MT_PHY_TYPE_HE_MU: if (rate.mcs > 11) - goto out; + return false; rate.he_gi = wcid->rate.he_gi; rate.he_dcm = FIELD_GET(MT_TX_RATE_DCM, txrate); rate.flags = RATE_INFO_FLAGS_HE_MCS; break; default: - goto out; + return false; } stats->tx_mode[mode]++; @@ -662,10 +660,34 @@ bool mt76_connac2_mac_add_txs_skb(struct mt76_dev *dev, struct mt76_wcid *wcid, } wcid->rate = rate; -out: - if (skb) - mt76_tx_status_skb_done(dev, skb, &list); + return true; +} +EXPORT_SYMBOL_GPL(mt76_connac2_mac_fill_txs); + +bool mt76_connac2_mac_add_txs_skb(struct mt76_dev *dev, struct mt76_wcid *wcid, + int pid, __le32 *txs_data) +{ + struct sk_buff_head list; + struct sk_buff *skb; + + mt76_tx_status_lock(dev, &list); + skb = mt76_tx_status_skb_get(dev, wcid, pid, &list); + if (skb) { + struct ieee80211_tx_info *info = IEEE80211_SKB_CB(skb); + bool noacked = !(info->flags & IEEE80211_TX_STAT_ACK); + + if (!(le32_to_cpu(txs_data[0]) & MT_TXS0_ACK_ERROR_MASK)) + info->flags |= IEEE80211_TX_STAT_ACK; + + info->status.ampdu_len = 1; + info->status.ampdu_ack_len = !noacked; + info->status.rates[0].idx = -1; + wcid->stats.tx_failed += noacked; + + mt76_connac2_mac_fill_txs(dev, wcid, txs_data); + mt76_tx_status_skb_done(dev, skb, &list); + } mt76_tx_status_unlock(dev, &list); return !!skb; diff --git a/drivers/net/wireless/mediatek/mt76/mt76_connac_mcu.c b/drivers/net/wireless/mediatek/mt76/mt76_connac_mcu.c index 9b17bd97ec09..011fc9729b38 100644 --- a/drivers/net/wireless/mediatek/mt76/mt76_connac_mcu.c +++ b/drivers/net/wireless/mediatek/mt76/mt76_connac_mcu.c @@ -260,8 +260,10 @@ mt76_connac_mcu_add_nested_tlv(struct sk_buff *skb, int tag, int len, ntlv = le16_to_cpu(ntlv_hdr->tlv_num); ntlv_hdr->tlv_num = cpu_to_le16(ntlv + 1); - if (sta_hdr) - le16_add_cpu(&sta_hdr->len, len); + if (sta_hdr) { + len += le16_to_cpu(sta_hdr->len); + sta_hdr->len = cpu_to_le16(len); + } return ptlv; } @@ -594,14 +596,14 @@ mt76_connac_mcu_sta_amsdu_tlv(struct sk_buff *skb, struct ieee80211_sta *sta, vif->type != NL80211_IFTYPE_STATION) return; - if (!sta->max_amsdu_len) + if (!sta->deflink.agg.max_amsdu_len) return; tlv = mt76_connac_mcu_add_tlv(skb, STA_REC_HW_AMSDU, sizeof(*amsdu)); amsdu = (struct sta_rec_amsdu *)tlv; amsdu->max_amsdu_num = 8; amsdu->amsdu_en = true; - amsdu->max_mpdu_size = sta->max_amsdu_len >= + amsdu->max_mpdu_size = sta->deflink.agg.max_amsdu_len >= IEEE80211_MAX_MPDU_LEN_VHT_7991; wcid->amsdu = true; @@ -896,7 +898,7 @@ void mt76_connac_mcu_wtbl_smps_tlv(struct sk_buff *skb, tlv = mt76_connac_mcu_add_nested_tlv(skb, WTBL_SMPS, sizeof(*smps), wtbl_tlv, sta_wtbl); smps = (struct wtbl_smps *)tlv; - smps->smps = (sta->smps_mode == IEEE80211_SMPS_DYNAMIC); + smps->smps = (sta->deflink.smps_mode == IEEE80211_SMPS_DYNAMIC); } EXPORT_SYMBOL_GPL(mt76_connac_mcu_wtbl_smps_tlv); @@ -2648,7 +2650,7 @@ int mt76_connac_mcu_add_key(struct mt76_dev *dev, struct ieee80211_vif *vif, } EXPORT_SYMBOL_GPL(mt76_connac_mcu_add_key); -/* SIFS 20us + 512 byte beacon tranmitted by 1Mbps (3906us) */ +/* SIFS 20us + 512 byte beacon transmitted by 1Mbps (3906us) */ #define BCN_TX_ESTIMATE_TIME (4096 + 20) void mt76_connac_mcu_bss_ext_tlv(struct sk_buff *skb, struct mt76_vif *mvif) { @@ -2886,6 +2888,10 @@ int mt76_connac2_load_ram(struct mt76_dev *dev, const char *fw_wm, goto out; } + snprintf(dev->hw->wiphy->fw_version, + sizeof(dev->hw->wiphy->fw_version), + "%.10s-%.15s", hdr->fw_ver, hdr->build_date); + release_firmware(fw); if (!fw_wa) diff --git a/drivers/net/wireless/mediatek/mt76/mt76_connac_mcu.h b/drivers/net/wireless/mediatek/mt76/mt76_connac_mcu.h index f1d7c05bd794..718f427d8f6b 100644 --- a/drivers/net/wireless/mediatek/mt76/mt76_connac_mcu.h +++ b/drivers/net/wireless/mediatek/mt76/mt76_connac_mcu.h @@ -10,6 +10,7 @@ #define FW_FEATURE_SET_KEY_IDX GENMASK(2, 1) #define FW_FEATURE_ENCRY_MODE BIT(4) #define FW_FEATURE_OVERRIDE_ADDR BIT(5) +#define FW_FEATURE_NON_DL BIT(6) #define DL_MODE_ENCRYPT BIT(0) #define DL_MODE_KEY_IDX GENMASK(2, 1) @@ -33,6 +34,12 @@ #define PATCH_SEC_ENC_SCRAMBLE_INFO_MASK GENMASK(15, 0) #define PATCH_SEC_ENC_AES_KEY_MASK GENMASK(7, 0) +enum { + FW_TYPE_DEFAULT = 0, + FW_TYPE_CLC = 2, + FW_TYPE_MAX_NUM = 255 +}; + #define MCU_PQ_ID(p, q) (((p) << 15) | ((q) << 10)) #define MCU_PKT_ID 0xa0 @@ -174,7 +181,8 @@ struct mt76_connac2_fw_region { __le32 addr; __le32 len; u8 feature_set; - u8 rsv1[15]; + u8 type; + u8 rsv1[14]; } __packed; struct tlv { @@ -1172,6 +1180,7 @@ enum { MCU_CE_CMD_SET_ROC = 0x1c, MCU_CE_CMD_SET_EDCA_PARMS = 0x1d, MCU_CE_CMD_SET_P2P_OPPPS = 0x33, + MCU_CE_CMD_SET_CLC = 0x5c, MCU_CE_CMD_SET_RATE_TX_POWER = 0x5d, MCU_CE_CMD_SCHED_SCAN_ENABLE = 0x61, MCU_CE_CMD_SCHED_SCAN_REQ = 0x62, diff --git a/drivers/net/wireless/mediatek/mt76/mt76x02_mac.c b/drivers/net/wireless/mediatek/mt76/mt76x02_mac.c index de30cf5e2d2f..93d96739f802 100644 --- a/drivers/net/wireless/mediatek/mt76/mt76x02_mac.c +++ b/drivers/net/wireless/mediatek/mt76/mt76x02_mac.c @@ -404,7 +404,7 @@ void mt76x02_mac_write_txwi(struct mt76x02_dev *dev, struct mt76x02_txwi *txwi, txwi->rate |= cpu_to_le16(MT_RXWI_RATE_LDPC); if ((info->flags & IEEE80211_TX_CTL_STBC) && nss == 1) txwi->rate |= cpu_to_le16(MT_RXWI_RATE_STBC); - if (nss > 1 && sta && sta->smps_mode == IEEE80211_SMPS_DYNAMIC) + if (nss > 1 && sta && sta->deflink.smps_mode == IEEE80211_SMPS_DYNAMIC) txwi_flags |= MT_TXWI_FLAGS_MMPS; if (!(info->flags & IEEE80211_TX_CTL_NO_ACK)) txwi->ack_ctl |= MT_TXWI_ACK_CTL_REQ; diff --git a/drivers/net/wireless/mediatek/mt76/mt76x02_usb_mcu.c b/drivers/net/wireless/mediatek/mt76/mt76x02_usb_mcu.c index c6c16fe8ee85..02da543dfc5c 100644 --- a/drivers/net/wireless/mediatek/mt76/mt76x02_usb_mcu.c +++ b/drivers/net/wireless/mediatek/mt76/mt76x02_usb_mcu.c @@ -21,29 +21,16 @@ static void mt76x02u_multiple_mcu_reads(struct mt76_dev *dev, u8 *data, int len) { struct mt76_usb *usb = &dev->usb; - u32 reg, val; int i; - if (usb->mcu.burst) { - WARN_ON_ONCE(len / 4 != usb->mcu.rp_len); - - reg = usb->mcu.rp[0].reg - usb->mcu.base; - for (i = 0; i < usb->mcu.rp_len; i++) { - val = get_unaligned_le32(data + 4 * i); - usb->mcu.rp[i].reg = reg++; - usb->mcu.rp[i].value = val; - } - } else { - WARN_ON_ONCE(len / 8 != usb->mcu.rp_len); - - for (i = 0; i < usb->mcu.rp_len; i++) { - reg = get_unaligned_le32(data + 8 * i) - - usb->mcu.base; - val = get_unaligned_le32(data + 8 * i + 4); - - WARN_ON_ONCE(usb->mcu.rp[i].reg != reg); - usb->mcu.rp[i].value = val; - } + WARN_ON_ONCE(len / 8 != usb->mcu.rp_len); + + for (i = 0; i < usb->mcu.rp_len; i++) { + u32 reg = get_unaligned_le32(data + 8 * i) - usb->mcu.base; + u32 val = get_unaligned_le32(data + 8 * i + 4); + + WARN_ON_ONCE(usb->mcu.rp[i].reg != reg); + usb->mcu.rp[i].value = val; } } @@ -207,7 +194,6 @@ mt76x02u_mcu_rd_rp(struct mt76_dev *dev, u32 base, usb->mcu.rp = data; usb->mcu.rp_len = n; usb->mcu.base = base; - usb->mcu.burst = false; ret = __mt76x02u_mcu_send_msg(dev, skb, CMD_RANDOM_READ, true); diff --git a/drivers/net/wireless/mediatek/mt76/mt7915/debugfs.c b/drivers/net/wireless/mediatek/mt76/mt7915/debugfs.c index fd76db8f5269..6ef3431cad64 100644 --- a/drivers/net/wireless/mediatek/mt76/mt7915/debugfs.c +++ b/drivers/net/wireless/mediatek/mt76/mt7915/debugfs.c @@ -23,9 +23,9 @@ mt7915_implicit_txbf_set(void *data, u64 val) { struct mt7915_dev *dev = data; - if (test_bit(MT76_STATE_RUNNING, &dev->mphy.state)) - return -EBUSY; - + /* The existing connected stations shall reconnect to apply + * new implicit txbf configuration. + */ dev->ibf = !!val; return mt7915_mcu_set_txbf(dev, MT_BF_TYPE_UPDATE); diff --git a/drivers/net/wireless/mediatek/mt76/mt7915/mac.c b/drivers/net/wireless/mediatek/mt76/mt7915/mac.c index 60ae834d95a6..be97dede2634 100644 --- a/drivers/net/wireless/mediatek/mt76/mt7915/mac.c +++ b/drivers/net/wireless/mediatek/mt76/mt7915/mac.c @@ -176,7 +176,7 @@ static void mt7915_mac_sta_poll(struct mt7915_dev *dev) /* * We don't support reading GI info from txs packets. * For accurate tx status reporting and AQL improvement, - we need to make sure that flags match so polling GI + * we need to make sure that flags match so polling GI * from per-sta counters directly. */ rate = &msta->wcid.rate; @@ -232,7 +232,7 @@ mt7915_mac_fill_rx(struct mt7915_dev *dev, struct sk_buff *skb) bool unicast, insert_ccmp_hdr = false; u8 remove_pad, amsdu_info; u8 mode = 0, qos_ctl = 0; - struct mt7915_sta *msta; + struct mt7915_sta *msta = NULL; bool hdr_trans; u16 hdr_gap; u16 seq_ctrl = 0; @@ -1001,7 +1001,7 @@ static void mt7915_mac_add_txs(struct mt7915_dev *dev, void *data) wcidx = le32_get_bits(txs_data[2], MT_TXS2_WCID); pid = le32_get_bits(txs_data[3], MT_TXS3_PID); - if (pid < MT_PACKET_ID_FIRST) + if (pid < MT_PACKET_ID_WED) return; if (wcidx >= mt7915_wtbl_size(dev)) @@ -1015,8 +1015,11 @@ static void mt7915_mac_add_txs(struct mt7915_dev *dev, void *data) msta = container_of(wcid, struct mt7915_sta, wcid); - mt76_connac2_mac_add_txs_skb(&dev->mt76, wcid, pid, txs_data, - &msta->stats); + if (pid == MT_PACKET_ID_WED) + mt76_connac2_mac_fill_txs(&dev->mt76, wcid, txs_data); + else + mt76_connac2_mac_add_txs_skb(&dev->mt76, wcid, pid, txs_data); + if (!wcid->sta) goto out; @@ -1047,7 +1050,7 @@ bool mt7915_rx_check(struct mt76_dev *mdev, void *data, int len) return false; case PKT_TYPE_TXS: for (rxd += 2; rxd + 8 <= end; rxd += 8) - mt7915_mac_add_txs(dev, rxd); + mt7915_mac_add_txs(dev, rxd); return false; case PKT_TYPE_RX_FW_MONITOR: mt7915_debugfs_rx_fw_monitor(dev, data, len); @@ -1084,7 +1087,7 @@ void mt7915_queue_rx_skb(struct mt76_dev *mdev, enum mt76_rxq_id q, break; case PKT_TYPE_TXS: for (rxd += 2; rxd + 8 <= end; rxd += 8) - mt7915_mac_add_txs(dev, rxd); + mt7915_mac_add_txs(dev, rxd); dev_kfree_skb(skb); break; case PKT_TYPE_RX_FW_MONITOR: @@ -2071,8 +2074,9 @@ void mt7915_mac_add_twt_setup(struct ieee80211_hw *hw, } flowid = ffs(~msta->twt.flowid_mask) - 1; - le16p_replace_bits(&twt_agrt->req_type, flowid, - IEEE80211_TWT_REQTYPE_FLOWID); + twt_agrt->req_type &= ~cpu_to_le16(IEEE80211_TWT_REQTYPE_FLOWID); + twt_agrt->req_type |= le16_encode_bits(flowid, + IEEE80211_TWT_REQTYPE_FLOWID); table_id = ffs(~dev->twt.table_mask) - 1; exp = FIELD_GET(IEEE80211_TWT_REQTYPE_WAKE_INT_EXP, req_type); @@ -2122,8 +2126,9 @@ void mt7915_mac_add_twt_setup(struct ieee80211_hw *hw, unlock: mutex_unlock(&dev->mt76.mutex); out: - le16p_replace_bits(&twt_agrt->req_type, setup_cmd, - IEEE80211_TWT_REQTYPE_SETUP_CMD); + twt_agrt->req_type &= ~cpu_to_le16(IEEE80211_TWT_REQTYPE_SETUP_CMD); + twt_agrt->req_type |= + le16_encode_bits(setup_cmd, IEEE80211_TWT_REQTYPE_SETUP_CMD); twt->control = (twt->control & IEEE80211_TWT_CONTROL_WAKE_DUR_UNIT) | (twt->control & IEEE80211_TWT_CONTROL_RX_DISABLED); } diff --git a/drivers/net/wireless/mediatek/mt76/mt7915/main.c b/drivers/net/wireless/mediatek/mt76/mt7915/main.c index bd3386bf0f8a..89b519cfd14c 100644 --- a/drivers/net/wireless/mediatek/mt76/mt7915/main.c +++ b/drivers/net/wireless/mediatek/mt76/mt7915/main.c @@ -1010,6 +1010,23 @@ static void mt7915_sta_statistics(struct ieee80211_hw *hw, } sinfo->txrate.flags = txrate->flags; sinfo->filled |= BIT_ULL(NL80211_STA_INFO_TX_BITRATE); + + /* offloading flows bypass networking stack, so driver counts and + * reports sta statistics via NL80211_STA_INFO when WED is active. + */ + if (mtk_wed_device_active(&phy->dev->mt76.mmio.wed)) { + sinfo->tx_bytes = msta->wcid.stats.tx_bytes; + sinfo->filled |= BIT_ULL(NL80211_STA_INFO_TX_BYTES64); + + sinfo->tx_packets = msta->wcid.stats.tx_packets; + sinfo->filled |= BIT_ULL(NL80211_STA_INFO_TX_PACKETS); + + sinfo->tx_failed = msta->wcid.stats.tx_failed; + sinfo->filled |= BIT_ULL(NL80211_STA_INFO_TX_FAILED); + + sinfo->tx_retries = msta->wcid.stats.tx_retries; + sinfo->filled |= BIT_ULL(NL80211_STA_INFO_TX_RETRIES); + } } static void mt7915_sta_rc_work(void *data, struct ieee80211_sta *sta) @@ -1224,7 +1241,7 @@ static void mt7915_ethtool_worker(void *wi_data, struct ieee80211_sta *sta) if (msta->vif->mt76.idx != wi->idx) return; - mt76_ethtool_worker(wi, &msta->stats); + mt76_ethtool_worker(wi, &msta->wcid.stats); } static diff --git a/drivers/net/wireless/mediatek/mt76/mt7915/mcu.c b/drivers/net/wireless/mediatek/mt76/mt7915/mcu.c index f83067961945..8d297e4aa7d4 100644 --- a/drivers/net/wireless/mediatek/mt76/mt7915/mcu.c +++ b/drivers/net/wireless/mediatek/mt76/mt7915/mcu.c @@ -925,7 +925,7 @@ mt7915_mcu_sta_amsdu_tlv(struct mt7915_dev *dev, struct sk_buff *skb, vif->type != NL80211_IFTYPE_AP) return; - if (!sta->max_amsdu_len) + if (!sta->deflink.agg.max_amsdu_len) return; tlv = mt76_connac_mcu_add_tlv(skb, STA_REC_HW_AMSDU, sizeof(*amsdu)); @@ -934,7 +934,7 @@ mt7915_mcu_sta_amsdu_tlv(struct mt7915_dev *dev, struct sk_buff *skb, amsdu->amsdu_en = true; msta->wcid.amsdu = true; - switch (sta->max_amsdu_len) { + switch (sta->deflink.agg.max_amsdu_len) { case IEEE80211_MAX_MPDU_LEN_VHT_11454: if (!is_mt7915(&dev->mt76)) { amsdu->max_mpdu_size = @@ -1304,7 +1304,7 @@ int mt7915_mcu_set_fixed_rate_ctrl(struct mt7915_dev *dev, ra->phy = *phy; break; case RATE_PARAM_MMPS_UPDATE: - ra->mmps_mode = mt7915_mcu_get_mmps_mode(sta->smps_mode); + ra->mmps_mode = mt7915_mcu_get_mmps_mode(sta->deflink.smps_mode); break; default: break; @@ -1360,7 +1360,7 @@ mt7915_mcu_add_rate_ctrl_fixed(struct mt7915_dev *dev, struct sta_phy phy = {}; int ret, nrates = 0; -#define __sta_phy_bitrate_mask_check(_mcs, _gi, _he) \ +#define __sta_phy_bitrate_mask_check(_mcs, _gi, _ht, _he) \ do { \ u8 i, gi = mask->control[band]._gi; \ gi = (_he) ? gi : gi == NL80211_TXRATE_FORCE_SGI; \ @@ -1373,15 +1373,17 @@ mt7915_mcu_add_rate_ctrl_fixed(struct mt7915_dev *dev, continue; \ nrates += hweight16(mask->control[band]._mcs[i]); \ phy.mcs = ffs(mask->control[band]._mcs[i]) - 1; \ + if (_ht) \ + phy.mcs += 8 * i; \ } \ } while (0) if (sta->deflink.he_cap.has_he) { - __sta_phy_bitrate_mask_check(he_mcs, he_gi, 1); + __sta_phy_bitrate_mask_check(he_mcs, he_gi, 0, 1); } else if (sta->deflink.vht_cap.vht_supported) { - __sta_phy_bitrate_mask_check(vht_mcs, gi, 0); + __sta_phy_bitrate_mask_check(vht_mcs, gi, 0, 0); } else if (sta->deflink.ht_cap.ht_supported) { - __sta_phy_bitrate_mask_check(ht_mcs, gi, 0); + __sta_phy_bitrate_mask_check(ht_mcs, gi, 1, 0); } else { nrates = hweight32(mask->control[band].legacy); phy.mcs = ffs(mask->control[band].legacy) - 1; @@ -1459,7 +1461,7 @@ mt7915_mcu_sta_rate_ctrl_tlv(struct sk_buff *skb, struct mt7915_dev *dev, ra->channel = chandef->chan->hw_value; ra->bw = sta->deflink.bandwidth; ra->phy.bw = sta->deflink.bandwidth; - ra->mmps_mode = mt7915_mcu_get_mmps_mode(sta->smps_mode); + ra->mmps_mode = mt7915_mcu_get_mmps_mode(sta->deflink.smps_mode); if (supp_rate) { supp_rate &= mask->control[band].legacy; diff --git a/drivers/net/wireless/mediatek/mt76/mt7915/mmio.c b/drivers/net/wireless/mediatek/mt76/mt7915/mmio.c index 4499a630e8f1..7bd5f6725d7b 100644 --- a/drivers/net/wireless/mediatek/mt76/mt7915/mmio.c +++ b/drivers/net/wireless/mediatek/mt76/mt7915/mmio.c @@ -75,6 +75,7 @@ static const u32 mt7915_offs[] = { [AGG_AWSCR0] = 0x05c, [AGG_PCR0] = 0x06c, [AGG_ACR0] = 0x084, + [AGG_ACR4] = 0x08c, [AGG_MRCR] = 0x098, [AGG_ATCR1] = 0x0f0, [AGG_ATCR3] = 0x0f4, @@ -148,6 +149,7 @@ static const u32 mt7916_offs[] = { [AGG_AWSCR0] = 0x030, [AGG_PCR0] = 0x040, [AGG_ACR0] = 0x054, + [AGG_ACR4] = 0x05c, [AGG_MRCR] = 0x068, [AGG_ATCR1] = 0x1a8, [AGG_ATCR3] = 0x080, @@ -204,147 +206,147 @@ static const u32 mt7916_offs[] = { [ETBF_PAR_RPT0] = 0x100, }; -static const struct __map mt7915_reg_map[] = { +static const struct mt76_connac_reg_map mt7915_reg_map[] = { { 0x00400000, 0x80000, 0x10000 }, /* WF_MCU_SYSRAM */ { 0x00410000, 0x90000, 0x10000 }, /* WF_MCU_SYSRAM (configure regs) */ { 0x40000000, 0x70000, 0x10000 }, /* WF_UMAC_SYSRAM */ - { 0x54000000, 0x02000, 0x1000 }, /* WFDMA PCIE0 MCU DMA0 */ - { 0x55000000, 0x03000, 0x1000 }, /* WFDMA PCIE0 MCU DMA1 */ - { 0x58000000, 0x06000, 0x1000 }, /* WFDMA PCIE1 MCU DMA0 (MEM_DMA) */ - { 0x59000000, 0x07000, 0x1000 }, /* WFDMA PCIE1 MCU DMA1 */ + { 0x54000000, 0x02000, 0x01000 }, /* WFDMA PCIE0 MCU DMA0 */ + { 0x55000000, 0x03000, 0x01000 }, /* WFDMA PCIE0 MCU DMA1 */ + { 0x58000000, 0x06000, 0x01000 }, /* WFDMA PCIE1 MCU DMA0 (MEM_DMA) */ + { 0x59000000, 0x07000, 0x01000 }, /* WFDMA PCIE1 MCU DMA1 */ { 0x7c000000, 0xf0000, 0x10000 }, /* CONN_INFRA */ { 0x7c020000, 0xd0000, 0x10000 }, /* CONN_INFRA, WFDMA */ { 0x80020000, 0xb0000, 0x10000 }, /* WF_TOP_MISC_OFF */ { 0x81020000, 0xc0000, 0x10000 }, /* WF_TOP_MISC_ON */ - { 0x820c0000, 0x08000, 0x4000 }, /* WF_UMAC_TOP (PLE) */ - { 0x820c8000, 0x0c000, 0x2000 }, /* WF_UMAC_TOP (PSE) */ - { 0x820cc000, 0x0e000, 0x2000 }, /* WF_UMAC_TOP (PP) */ - { 0x820ce000, 0x21c00, 0x0200 }, /* WF_LMAC_TOP (WF_SEC) */ - { 0x820cf000, 0x22000, 0x1000 }, /* WF_LMAC_TOP (WF_PF) */ + { 0x820c0000, 0x08000, 0x04000 }, /* WF_UMAC_TOP (PLE) */ + { 0x820c8000, 0x0c000, 0x02000 }, /* WF_UMAC_TOP (PSE) */ + { 0x820cc000, 0x0e000, 0x02000 }, /* WF_UMAC_TOP (PP) */ + { 0x820ce000, 0x21c00, 0x00200 }, /* WF_LMAC_TOP (WF_SEC) */ + { 0x820cf000, 0x22000, 0x01000 }, /* WF_LMAC_TOP (WF_PF) */ { 0x820d0000, 0x30000, 0x10000 }, /* WF_LMAC_TOP (WF_WTBLON) */ - { 0x820e0000, 0x20000, 0x0400 }, /* WF_LMAC_TOP BN0 (WF_CFG) */ - { 0x820e1000, 0x20400, 0x0200 }, /* WF_LMAC_TOP BN0 (WF_TRB) */ - { 0x820e2000, 0x20800, 0x0400 }, /* WF_LMAC_TOP BN0 (WF_AGG) */ - { 0x820e3000, 0x20c00, 0x0400 }, /* WF_LMAC_TOP BN0 (WF_ARB) */ - { 0x820e4000, 0x21000, 0x0400 }, /* WF_LMAC_TOP BN0 (WF_TMAC) */ - { 0x820e5000, 0x21400, 0x0800 }, /* WF_LMAC_TOP BN0 (WF_RMAC) */ - { 0x820e7000, 0x21e00, 0x0200 }, /* WF_LMAC_TOP BN0 (WF_DMA) */ - { 0x820e9000, 0x23400, 0x0200 }, /* WF_LMAC_TOP BN0 (WF_WTBLOFF) */ - { 0x820ea000, 0x24000, 0x0200 }, /* WF_LMAC_TOP BN0 (WF_ETBF) */ - { 0x820eb000, 0x24200, 0x0400 }, /* WF_LMAC_TOP BN0 (WF_LPON) */ - { 0x820ec000, 0x24600, 0x0200 }, /* WF_LMAC_TOP BN0 (WF_INT) */ - { 0x820ed000, 0x24800, 0x0800 }, /* WF_LMAC_TOP BN0 (WF_MIB) */ - { 0x820f0000, 0xa0000, 0x0400 }, /* WF_LMAC_TOP BN1 (WF_CFG) */ - { 0x820f1000, 0xa0600, 0x0200 }, /* WF_LMAC_TOP BN1 (WF_TRB) */ - { 0x820f2000, 0xa0800, 0x0400 }, /* WF_LMAC_TOP BN1 (WF_AGG) */ - { 0x820f3000, 0xa0c00, 0x0400 }, /* WF_LMAC_TOP BN1 (WF_ARB) */ - { 0x820f4000, 0xa1000, 0x0400 }, /* WF_LMAC_TOP BN1 (WF_TMAC) */ - { 0x820f5000, 0xa1400, 0x0800 }, /* WF_LMAC_TOP BN1 (WF_RMAC) */ - { 0x820f7000, 0xa1e00, 0x0200 }, /* WF_LMAC_TOP BN1 (WF_DMA) */ - { 0x820f9000, 0xa3400, 0x0200 }, /* WF_LMAC_TOP BN1 (WF_WTBLOFF) */ - { 0x820fa000, 0xa4000, 0x0200 }, /* WF_LMAC_TOP BN1 (WF_ETBF) */ - { 0x820fb000, 0xa4200, 0x0400 }, /* WF_LMAC_TOP BN1 (WF_LPON) */ - { 0x820fc000, 0xa4600, 0x0200 }, /* WF_LMAC_TOP BN1 (WF_INT) */ - { 0x820fd000, 0xa4800, 0x0800 }, /* WF_LMAC_TOP BN1 (WF_MIB) */ + { 0x820e0000, 0x20000, 0x00400 }, /* WF_LMAC_TOP BN0 (WF_CFG) */ + { 0x820e1000, 0x20400, 0x00200 }, /* WF_LMAC_TOP BN0 (WF_TRB) */ + { 0x820e2000, 0x20800, 0x00400 }, /* WF_LMAC_TOP BN0 (WF_AGG) */ + { 0x820e3000, 0x20c00, 0x00400 }, /* WF_LMAC_TOP BN0 (WF_ARB) */ + { 0x820e4000, 0x21000, 0x00400 }, /* WF_LMAC_TOP BN0 (WF_TMAC) */ + { 0x820e5000, 0x21400, 0x00800 }, /* WF_LMAC_TOP BN0 (WF_RMAC) */ + { 0x820e7000, 0x21e00, 0x00200 }, /* WF_LMAC_TOP BN0 (WF_DMA) */ + { 0x820e9000, 0x23400, 0x00200 }, /* WF_LMAC_TOP BN0 (WF_WTBLOFF) */ + { 0x820ea000, 0x24000, 0x00200 }, /* WF_LMAC_TOP BN0 (WF_ETBF) */ + { 0x820eb000, 0x24200, 0x00400 }, /* WF_LMAC_TOP BN0 (WF_LPON) */ + { 0x820ec000, 0x24600, 0x00200 }, /* WF_LMAC_TOP BN0 (WF_INT) */ + { 0x820ed000, 0x24800, 0x00800 }, /* WF_LMAC_TOP BN0 (WF_MIB) */ + { 0x820f0000, 0xa0000, 0x00400 }, /* WF_LMAC_TOP BN1 (WF_CFG) */ + { 0x820f1000, 0xa0600, 0x00200 }, /* WF_LMAC_TOP BN1 (WF_TRB) */ + { 0x820f2000, 0xa0800, 0x00400 }, /* WF_LMAC_TOP BN1 (WF_AGG) */ + { 0x820f3000, 0xa0c00, 0x00400 }, /* WF_LMAC_TOP BN1 (WF_ARB) */ + { 0x820f4000, 0xa1000, 0x00400 }, /* WF_LMAC_TOP BN1 (WF_TMAC) */ + { 0x820f5000, 0xa1400, 0x00800 }, /* WF_LMAC_TOP BN1 (WF_RMAC) */ + { 0x820f7000, 0xa1e00, 0x00200 }, /* WF_LMAC_TOP BN1 (WF_DMA) */ + { 0x820f9000, 0xa3400, 0x00200 }, /* WF_LMAC_TOP BN1 (WF_WTBLOFF) */ + { 0x820fa000, 0xa4000, 0x00200 }, /* WF_LMAC_TOP BN1 (WF_ETBF) */ + { 0x820fb000, 0xa4200, 0x00400 }, /* WF_LMAC_TOP BN1 (WF_LPON) */ + { 0x820fc000, 0xa4600, 0x00200 }, /* WF_LMAC_TOP BN1 (WF_INT) */ + { 0x820fd000, 0xa4800, 0x00800 }, /* WF_LMAC_TOP BN1 (WF_MIB) */ { 0x0, 0x0, 0x0 }, /* imply end of search */ }; -static const struct __map mt7916_reg_map[] = { - { 0x54000000, 0x02000, 0x1000 }, /* WFDMA_0 (PCIE0 MCU DMA0) */ - { 0x55000000, 0x03000, 0x1000 }, /* WFDMA_1 (PCIE0 MCU DMA1) */ - { 0x56000000, 0x04000, 0x1000 }, /* WFDMA_2 (Reserved) */ - { 0x57000000, 0x05000, 0x1000 }, /* WFDMA_3 (MCU wrap CR) */ - { 0x58000000, 0x06000, 0x1000 }, /* WFDMA_4 (PCIE1 MCU DMA0) */ - { 0x59000000, 0x07000, 0x1000 }, /* WFDMA_5 (PCIE1 MCU DMA1) */ - { 0x820c0000, 0x08000, 0x4000 }, /* WF_UMAC_TOP (PLE) */ - { 0x820c8000, 0x0c000, 0x2000 }, /* WF_UMAC_TOP (PSE) */ - { 0x820cc000, 0x0e000, 0x2000 }, /* WF_UMAC_TOP (PP) */ - { 0x820e0000, 0x20000, 0x0400 }, /* WF_LMAC_TOP BN0 (WF_CFG) */ - { 0x820e1000, 0x20400, 0x0200 }, /* WF_LMAC_TOP BN0 (WF_TRB) */ - { 0x820e2000, 0x20800, 0x0400 }, /* WF_LMAC_TOP BN0 (WF_AGG) */ - { 0x820e3000, 0x20c00, 0x0400 }, /* WF_LMAC_TOP BN0 (WF_ARB) */ - { 0x820e4000, 0x21000, 0x0400 }, /* WF_LMAC_TOP BN0 (WF_TMAC) */ - { 0x820e5000, 0x21400, 0x0800 }, /* WF_LMAC_TOP BN0 (WF_RMAC) */ - { 0x820ce000, 0x21c00, 0x0200 }, /* WF_LMAC_TOP (WF_SEC) */ - { 0x820e7000, 0x21e00, 0x0200 }, /* WF_LMAC_TOP BN0 (WF_DMA) */ - { 0x820cf000, 0x22000, 0x1000 }, /* WF_LMAC_TOP (WF_PF) */ - { 0x820e9000, 0x23400, 0x0200 }, /* WF_LMAC_TOP BN0 (WF_WTBLOFF) */ - { 0x820ea000, 0x24000, 0x0200 }, /* WF_LMAC_TOP BN0 (WF_ETBF) */ - { 0x820eb000, 0x24200, 0x0400 }, /* WF_LMAC_TOP BN0 (WF_LPON) */ - { 0x820ec000, 0x24600, 0x0200 }, /* WF_LMAC_TOP BN0 (WF_INT) */ - { 0x820ed000, 0x24800, 0x0800 }, /* WF_LMAC_TOP BN0 (WF_MIB) */ - { 0x820ca000, 0x26000, 0x2000 }, /* WF_LMAC_TOP BN0 (WF_MUCOP) */ - { 0x820d0000, 0x30000, 0x10000}, /* WF_LMAC_TOP (WF_WTBLON) */ - { 0x00400000, 0x80000, 0x10000}, /* WF_MCU_SYSRAM */ - { 0x00410000, 0x90000, 0x10000}, /* WF_MCU_SYSRAM (configure cr) */ - { 0x820f0000, 0xa0000, 0x0400 }, /* WF_LMAC_TOP BN1 (WF_CFG) */ - { 0x820f1000, 0xa0600, 0x0200 }, /* WF_LMAC_TOP BN1 (WF_TRB) */ - { 0x820f2000, 0xa0800, 0x0400 }, /* WF_LMAC_TOP BN1 (WF_AGG) */ - { 0x820f3000, 0xa0c00, 0x0400 }, /* WF_LMAC_TOP BN1 (WF_ARB) */ - { 0x820f4000, 0xa1000, 0x0400 }, /* WF_LMAC_TOP BN1 (WF_TMAC) */ - { 0x820f5000, 0xa1400, 0x0800 }, /* WF_LMAC_TOP BN1 (WF_RMAC) */ - { 0x820f7000, 0xa1e00, 0x0200 }, /* WF_LMAC_TOP BN1 (WF_DMA) */ - { 0x820f9000, 0xa3400, 0x0200 }, /* WF_LMAC_TOP BN1 (WF_WTBLOFF) */ - { 0x820fa000, 0xa4000, 0x0200 }, /* WF_LMAC_TOP BN1 (WF_ETBF) */ - { 0x820fb000, 0xa4200, 0x0400 }, /* WF_LMAC_TOP BN1 (WF_LPON) */ - { 0x820fc000, 0xa4600, 0x0200 }, /* WF_LMAC_TOP BN1 (WF_INT) */ - { 0x820fd000, 0xa4800, 0x0800 }, /* WF_LMAC_TOP BN1 (WF_MIB) */ - { 0x820c4000, 0xa8000, 0x1000 }, /* WF_LMAC_TOP (WF_UWTBL ) */ - { 0x820b0000, 0xae000, 0x1000 }, /* [APB2] WFSYS_ON */ - { 0x80020000, 0xb0000, 0x10000}, /* WF_TOP_MISC_OFF */ - { 0x81020000, 0xc0000, 0x10000}, /* WF_TOP_MISC_ON */ +static const struct mt76_connac_reg_map mt7916_reg_map[] = { + { 0x54000000, 0x02000, 0x01000 }, /* WFDMA_0 (PCIE0 MCU DMA0) */ + { 0x55000000, 0x03000, 0x01000 }, /* WFDMA_1 (PCIE0 MCU DMA1) */ + { 0x56000000, 0x04000, 0x01000 }, /* WFDMA_2 (Reserved) */ + { 0x57000000, 0x05000, 0x01000 }, /* WFDMA_3 (MCU wrap CR) */ + { 0x58000000, 0x06000, 0x01000 }, /* WFDMA_4 (PCIE1 MCU DMA0) */ + { 0x59000000, 0x07000, 0x01000 }, /* WFDMA_5 (PCIE1 MCU DMA1) */ + { 0x820c0000, 0x08000, 0x04000 }, /* WF_UMAC_TOP (PLE) */ + { 0x820c8000, 0x0c000, 0x02000 }, /* WF_UMAC_TOP (PSE) */ + { 0x820cc000, 0x0e000, 0x02000 }, /* WF_UMAC_TOP (PP) */ + { 0x820e0000, 0x20000, 0x00400 }, /* WF_LMAC_TOP BN0 (WF_CFG) */ + { 0x820e1000, 0x20400, 0x00200 }, /* WF_LMAC_TOP BN0 (WF_TRB) */ + { 0x820e2000, 0x20800, 0x00400 }, /* WF_LMAC_TOP BN0 (WF_AGG) */ + { 0x820e3000, 0x20c00, 0x00400 }, /* WF_LMAC_TOP BN0 (WF_ARB) */ + { 0x820e4000, 0x21000, 0x00400 }, /* WF_LMAC_TOP BN0 (WF_TMAC) */ + { 0x820e5000, 0x21400, 0x00800 }, /* WF_LMAC_TOP BN0 (WF_RMAC) */ + { 0x820ce000, 0x21c00, 0x00200 }, /* WF_LMAC_TOP (WF_SEC) */ + { 0x820e7000, 0x21e00, 0x00200 }, /* WF_LMAC_TOP BN0 (WF_DMA) */ + { 0x820cf000, 0x22000, 0x01000 }, /* WF_LMAC_TOP (WF_PF) */ + { 0x820e9000, 0x23400, 0x00200 }, /* WF_LMAC_TOP BN0 (WF_WTBLOFF) */ + { 0x820ea000, 0x24000, 0x00200 }, /* WF_LMAC_TOP BN0 (WF_ETBF) */ + { 0x820eb000, 0x24200, 0x00400 }, /* WF_LMAC_TOP BN0 (WF_LPON) */ + { 0x820ec000, 0x24600, 0x00200 }, /* WF_LMAC_TOP BN0 (WF_INT) */ + { 0x820ed000, 0x24800, 0x00800 }, /* WF_LMAC_TOP BN0 (WF_MIB) */ + { 0x820ca000, 0x26000, 0x02000 }, /* WF_LMAC_TOP BN0 (WF_MUCOP) */ + { 0x820d0000, 0x30000, 0x10000 }, /* WF_LMAC_TOP (WF_WTBLON) */ + { 0x00400000, 0x80000, 0x10000 }, /* WF_MCU_SYSRAM */ + { 0x00410000, 0x90000, 0x10000 }, /* WF_MCU_SYSRAM (configure cr) */ + { 0x820f0000, 0xa0000, 0x00400 }, /* WF_LMAC_TOP BN1 (WF_CFG) */ + { 0x820f1000, 0xa0600, 0x00200 }, /* WF_LMAC_TOP BN1 (WF_TRB) */ + { 0x820f2000, 0xa0800, 0x00400 }, /* WF_LMAC_TOP BN1 (WF_AGG) */ + { 0x820f3000, 0xa0c00, 0x00400 }, /* WF_LMAC_TOP BN1 (WF_ARB) */ + { 0x820f4000, 0xa1000, 0x00400 }, /* WF_LMAC_TOP BN1 (WF_TMAC) */ + { 0x820f5000, 0xa1400, 0x00800 }, /* WF_LMAC_TOP BN1 (WF_RMAC) */ + { 0x820f7000, 0xa1e00, 0x00200 }, /* WF_LMAC_TOP BN1 (WF_DMA) */ + { 0x820f9000, 0xa3400, 0x00200 }, /* WF_LMAC_TOP BN1 (WF_WTBLOFF) */ + { 0x820fa000, 0xa4000, 0x00200 }, /* WF_LMAC_TOP BN1 (WF_ETBF) */ + { 0x820fb000, 0xa4200, 0x00400 }, /* WF_LMAC_TOP BN1 (WF_LPON) */ + { 0x820fc000, 0xa4600, 0x00200 }, /* WF_LMAC_TOP BN1 (WF_INT) */ + { 0x820fd000, 0xa4800, 0x00800 }, /* WF_LMAC_TOP BN1 (WF_MIB) */ + { 0x820c4000, 0xa8000, 0x01000 }, /* WF_LMAC_TOP (WF_UWTBL ) */ + { 0x820b0000, 0xae000, 0x01000 }, /* [APB2] WFSYS_ON */ + { 0x80020000, 0xb0000, 0x10000 }, /* WF_TOP_MISC_OFF */ + { 0x81020000, 0xc0000, 0x10000 }, /* WF_TOP_MISC_ON */ { 0x0, 0x0, 0x0 }, /* imply end of search */ }; -static const struct __map mt7986_reg_map[] = { - { 0x54000000, 0x402000, 0x1000 }, /* WFDMA_0 (PCIE0 MCU DMA0) */ - { 0x55000000, 0x403000, 0x1000 }, /* WFDMA_1 (PCIE0 MCU DMA1) */ - { 0x56000000, 0x404000, 0x1000 }, /* WFDMA_2 (Reserved) */ - { 0x57000000, 0x405000, 0x1000 }, /* WFDMA_3 (MCU wrap CR) */ - { 0x58000000, 0x406000, 0x1000 }, /* WFDMA_4 (PCIE1 MCU DMA0) */ - { 0x59000000, 0x407000, 0x1000 }, /* WFDMA_5 (PCIE1 MCU DMA1) */ - { 0x820c0000, 0x408000, 0x4000 }, /* WF_UMAC_TOP (PLE) */ - { 0x820c8000, 0x40c000, 0x2000 }, /* WF_UMAC_TOP (PSE) */ - { 0x820cc000, 0x40e000, 0x2000 }, /* WF_UMAC_TOP (PP) */ - { 0x820e0000, 0x420000, 0x0400 }, /* WF_LMAC_TOP BN0 (WF_CFG) */ - { 0x820e1000, 0x420400, 0x0200 }, /* WF_LMAC_TOP BN0 (WF_TRB) */ - { 0x820e2000, 0x420800, 0x0400 }, /* WF_LMAC_TOP BN0 (WF_AGG) */ - { 0x820e3000, 0x420c00, 0x0400 }, /* WF_LMAC_TOP BN0 (WF_ARB) */ - { 0x820e4000, 0x421000, 0x0400 }, /* WF_LMAC_TOP BN0 (WF_TMAC) */ - { 0x820e5000, 0x421400, 0x0800 }, /* WF_LMAC_TOP BN0 (WF_RMAC) */ - { 0x820ce000, 0x421c00, 0x0200 }, /* WF_LMAC_TOP (WF_SEC) */ - { 0x820e7000, 0x421e00, 0x0200 }, /* WF_LMAC_TOP BN0 (WF_DMA) */ - { 0x820cf000, 0x422000, 0x1000 }, /* WF_LMAC_TOP (WF_PF) */ - { 0x820e9000, 0x423400, 0x0200 }, /* WF_LMAC_TOP BN0 (WF_WTBLOFF) */ - { 0x820ea000, 0x424000, 0x0200 }, /* WF_LMAC_TOP BN0 (WF_ETBF) */ - { 0x820eb000, 0x424200, 0x0400 }, /* WF_LMAC_TOP BN0 (WF_LPON) */ - { 0x820ec000, 0x424600, 0x0200 }, /* WF_LMAC_TOP BN0 (WF_INT) */ - { 0x820ed000, 0x424800, 0x0800 }, /* WF_LMAC_TOP BN0 (WF_MIB) */ - { 0x820ca000, 0x426000, 0x2000 }, /* WF_LMAC_TOP BN0 (WF_MUCOP) */ - { 0x820d0000, 0x430000, 0x10000}, /* WF_LMAC_TOP (WF_WTBLON) */ - { 0x00400000, 0x480000, 0x10000}, /* WF_MCU_SYSRAM */ - { 0x00410000, 0x490000, 0x10000}, /* WF_MCU_SYSRAM */ - { 0x820f0000, 0x4a0000, 0x0400 }, /* WF_LMAC_TOP BN1 (WF_CFG) */ - { 0x820f1000, 0x4a0600, 0x0200 }, /* WF_LMAC_TOP BN1 (WF_TRB) */ - { 0x820f2000, 0x4a0800, 0x0400 }, /* WF_LMAC_TOP BN1 (WF_AGG) */ - { 0x820f3000, 0x4a0c00, 0x0400 }, /* WF_LMAC_TOP BN1 (WF_ARB) */ - { 0x820f4000, 0x4a1000, 0x0400 }, /* WF_LMAC_TOP BN1 (WF_TMAC) */ - { 0x820f5000, 0x4a1400, 0x0800 }, /* WF_LMAC_TOP BN1 (WF_RMAC) */ - { 0x820f7000, 0x4a1e00, 0x0200 }, /* WF_LMAC_TOP BN1 (WF_DMA) */ - { 0x820f9000, 0x4a3400, 0x0200 }, /* WF_LMAC_TOP BN1 (WF_WTBLOFF) */ - { 0x820fa000, 0x4a4000, 0x0200 }, /* WF_LMAC_TOP BN1 (WF_ETBF) */ - { 0x820fb000, 0x4a4200, 0x0400 }, /* WF_LMAC_TOP BN1 (WF_LPON) */ - { 0x820fc000, 0x4a4600, 0x0200 }, /* WF_LMAC_TOP BN1 (WF_INT) */ - { 0x820fd000, 0x4a4800, 0x0800 }, /* WF_LMAC_TOP BN1 (WF_MIB) */ - { 0x820c4000, 0x4a8000, 0x1000 }, /* WF_LMAC_TOP (WF_UWTBL ) */ - { 0x820b0000, 0x4ae000, 0x1000 }, /* [APB2] WFSYS_ON */ - { 0x80020000, 0x4b0000, 0x10000}, /* WF_TOP_MISC_OFF */ - { 0x81020000, 0x4c0000, 0x10000}, /* WF_TOP_MISC_ON */ - { 0x89000000, 0x4d0000, 0x1000 }, /* WF_MCU_CFG_ON */ - { 0x89010000, 0x4d1000, 0x1000 }, /* WF_MCU_CIRQ */ - { 0x89020000, 0x4d2000, 0x1000 }, /* WF_MCU_GPT */ - { 0x89030000, 0x4d3000, 0x1000 }, /* WF_MCU_WDT */ - { 0x80010000, 0x4d4000, 0x1000 }, /* WF_AXIDMA */ +static const struct mt76_connac_reg_map mt7986_reg_map[] = { + { 0x54000000, 0x402000, 0x01000 }, /* WFDMA_0 (PCIE0 MCU DMA0) */ + { 0x55000000, 0x403000, 0x01000 }, /* WFDMA_1 (PCIE0 MCU DMA1) */ + { 0x56000000, 0x404000, 0x01000 }, /* WFDMA_2 (Reserved) */ + { 0x57000000, 0x405000, 0x01000 }, /* WFDMA_3 (MCU wrap CR) */ + { 0x58000000, 0x406000, 0x01000 }, /* WFDMA_4 (PCIE1 MCU DMA0) */ + { 0x59000000, 0x407000, 0x01000 }, /* WFDMA_5 (PCIE1 MCU DMA1) */ + { 0x820c0000, 0x408000, 0x04000 }, /* WF_UMAC_TOP (PLE) */ + { 0x820c8000, 0x40c000, 0x02000 }, /* WF_UMAC_TOP (PSE) */ + { 0x820cc000, 0x40e000, 0x02000 }, /* WF_UMAC_TOP (PP) */ + { 0x820e0000, 0x420000, 0x00400 }, /* WF_LMAC_TOP BN0 (WF_CFG) */ + { 0x820e1000, 0x420400, 0x00200 }, /* WF_LMAC_TOP BN0 (WF_TRB) */ + { 0x820e2000, 0x420800, 0x00400 }, /* WF_LMAC_TOP BN0 (WF_AGG) */ + { 0x820e3000, 0x420c00, 0x00400 }, /* WF_LMAC_TOP BN0 (WF_ARB) */ + { 0x820e4000, 0x421000, 0x00400 }, /* WF_LMAC_TOP BN0 (WF_TMAC) */ + { 0x820e5000, 0x421400, 0x00800 }, /* WF_LMAC_TOP BN0 (WF_RMAC) */ + { 0x820ce000, 0x421c00, 0x00200 }, /* WF_LMAC_TOP (WF_SEC) */ + { 0x820e7000, 0x421e00, 0x00200 }, /* WF_LMAC_TOP BN0 (WF_DMA) */ + { 0x820cf000, 0x422000, 0x01000 }, /* WF_LMAC_TOP (WF_PF) */ + { 0x820e9000, 0x423400, 0x00200 }, /* WF_LMAC_TOP BN0 (WF_WTBLOFF) */ + { 0x820ea000, 0x424000, 0x00200 }, /* WF_LMAC_TOP BN0 (WF_ETBF) */ + { 0x820eb000, 0x424200, 0x00400 }, /* WF_LMAC_TOP BN0 (WF_LPON) */ + { 0x820ec000, 0x424600, 0x00200 }, /* WF_LMAC_TOP BN0 (WF_INT) */ + { 0x820ed000, 0x424800, 0x00800 }, /* WF_LMAC_TOP BN0 (WF_MIB) */ + { 0x820ca000, 0x426000, 0x02000 }, /* WF_LMAC_TOP BN0 (WF_MUCOP) */ + { 0x820d0000, 0x430000, 0x10000 }, /* WF_LMAC_TOP (WF_WTBLON) */ + { 0x00400000, 0x480000, 0x10000 }, /* WF_MCU_SYSRAM */ + { 0x00410000, 0x490000, 0x10000 }, /* WF_MCU_SYSRAM */ + { 0x820f0000, 0x4a0000, 0x00400 }, /* WF_LMAC_TOP BN1 (WF_CFG) */ + { 0x820f1000, 0x4a0600, 0x00200 }, /* WF_LMAC_TOP BN1 (WF_TRB) */ + { 0x820f2000, 0x4a0800, 0x00400 }, /* WF_LMAC_TOP BN1 (WF_AGG) */ + { 0x820f3000, 0x4a0c00, 0x00400 }, /* WF_LMAC_TOP BN1 (WF_ARB) */ + { 0x820f4000, 0x4a1000, 0x00400 }, /* WF_LMAC_TOP BN1 (WF_TMAC) */ + { 0x820f5000, 0x4a1400, 0x00800 }, /* WF_LMAC_TOP BN1 (WF_RMAC) */ + { 0x820f7000, 0x4a1e00, 0x00200 }, /* WF_LMAC_TOP BN1 (WF_DMA) */ + { 0x820f9000, 0x4a3400, 0x00200 }, /* WF_LMAC_TOP BN1 (WF_WTBLOFF) */ + { 0x820fa000, 0x4a4000, 0x00200 }, /* WF_LMAC_TOP BN1 (WF_ETBF) */ + { 0x820fb000, 0x4a4200, 0x00400 }, /* WF_LMAC_TOP BN1 (WF_LPON) */ + { 0x820fc000, 0x4a4600, 0x00200 }, /* WF_LMAC_TOP BN1 (WF_INT) */ + { 0x820fd000, 0x4a4800, 0x00800 }, /* WF_LMAC_TOP BN1 (WF_MIB) */ + { 0x820c4000, 0x4a8000, 0x01000 }, /* WF_LMAC_TOP (WF_UWTBL ) */ + { 0x820b0000, 0x4ae000, 0x01000 }, /* [APB2] WFSYS_ON */ + { 0x80020000, 0x4b0000, 0x10000 }, /* WF_TOP_MISC_OFF */ + { 0x81020000, 0x4c0000, 0x10000 }, /* WF_TOP_MISC_ON */ + { 0x89000000, 0x4d0000, 0x01000 }, /* WF_MCU_CFG_ON */ + { 0x89010000, 0x4d1000, 0x01000 }, /* WF_MCU_CIRQ */ + { 0x89020000, 0x4d2000, 0x01000 }, /* WF_MCU_GPT */ + { 0x89030000, 0x4d3000, 0x01000 }, /* WF_MCU_WDT */ + { 0x80010000, 0x4d4000, 0x01000 }, /* WF_AXIDMA */ { 0x0, 0x0, 0x0 }, /* imply end of search */ }; diff --git a/drivers/net/wireless/mediatek/mt76/mt7915/mt7915.h b/drivers/net/wireless/mediatek/mt76/mt7915/mt7915.h index 54ef2a12a443..1eb11617a625 100644 --- a/drivers/net/wireless/mediatek/mt76/mt7915/mt7915.h +++ b/drivers/net/wireless/mediatek/mt76/mt7915/mt7915.h @@ -127,8 +127,6 @@ struct mt7915_sta { unsigned long jiffies; unsigned long ampdu_state; - struct mt76_sta_stats stats; - struct mt76_connac_sta_key_conf bip; struct { diff --git a/drivers/net/wireless/mediatek/mt76/mt7915/pci.c b/drivers/net/wireless/mediatek/mt76/mt7915/pci.c index d74f609775d3..728a879c3b00 100644 --- a/drivers/net/wireless/mediatek/mt76/mt7915/pci.c +++ b/drivers/net/wireless/mediatek/mt76/mt7915/pci.c @@ -99,6 +99,7 @@ static int mt7915_pci_hif2_probe(struct pci_dev *pdev) static int mt7915_wed_offload_enable(struct mtk_wed_device *wed) { struct mt7915_dev *dev; + struct mt7915_phy *phy; int ret; dev = container_of(wed, struct mt7915_dev, mt76.mmio.wed); @@ -112,18 +113,38 @@ static int mt7915_wed_offload_enable(struct mtk_wed_device *wed) if (!ret) return -EAGAIN; + phy = &dev->phy; + mt76_set(dev, MT_AGG_ACR4(phy->band_idx), MT_AGG_ACR_PPDU_TXS2H); + + phy = dev->mt76.phys[MT_BAND1] ? dev->mt76.phys[MT_BAND1]->priv : NULL; + if (phy) + mt76_set(dev, MT_AGG_ACR4(phy->band_idx), + MT_AGG_ACR_PPDU_TXS2H); + return 0; } static void mt7915_wed_offload_disable(struct mtk_wed_device *wed) { struct mt7915_dev *dev; + struct mt7915_phy *phy; dev = container_of(wed, struct mt7915_dev, mt76.mmio.wed); spin_lock_bh(&dev->mt76.token_lock); dev->mt76.token_size = MT7915_TOKEN_SIZE; spin_unlock_bh(&dev->mt76.token_lock); + + /* MT_TXD5_TX_STATUS_HOST (MPDU format) has higher priority than + * MT_AGG_ACR_PPDU_TXS2H (PPDU format) even though ACR bit is set. + */ + phy = &dev->phy; + mt76_clear(dev, MT_AGG_ACR4(phy->band_idx), MT_AGG_ACR_PPDU_TXS2H); + + phy = dev->mt76.phys[MT_BAND1] ? dev->mt76.phys[MT_BAND1]->priv : NULL; + if (phy) + mt76_clear(dev, MT_AGG_ACR4(phy->band_idx), + MT_AGG_ACR_PPDU_TXS2H); } #endif diff --git a/drivers/net/wireless/mediatek/mt76/mt7915/regs.h b/drivers/net/wireless/mediatek/mt76/mt7915/regs.h index 2493c3ad3c56..5920e705835a 100644 --- a/drivers/net/wireless/mediatek/mt76/mt7915/regs.h +++ b/drivers/net/wireless/mediatek/mt76/mt7915/regs.h @@ -4,17 +4,11 @@ #ifndef __MT7915_REGS_H #define __MT7915_REGS_H -struct __map { - u32 phys; - u32 maps; - u32 size; -}; - /* used to differentiate between generations */ struct mt7915_reg_desc { const u32 *reg_rev; const u32 *offs_rev; - const struct __map *map; + const struct mt76_connac_reg_map *map; u32 map_size; }; @@ -52,6 +46,7 @@ enum offs_rev { AGG_AWSCR0, AGG_PCR0, AGG_ACR0, + AGG_ACR4, AGG_MRCR, AGG_ATCR1, AGG_ATCR3, @@ -471,6 +466,9 @@ enum offs_rev { #define MT_AGG_ACR_CFEND_RATE GENMASK(13, 0) #define MT_AGG_ACR_BAR_RATE GENMASK(29, 16) +#define MT_AGG_ACR4(_band) MT_WF_AGG(_band, __OFFS(AGG_ACR4)) +#define MT_AGG_ACR_PPDU_TXS2H BIT(1) + #define MT_AGG_MRCR(_band) MT_WF_AGG(_band, __OFFS(AGG_MRCR)) #define MT_AGG_MRCR_BAR_CNT_LIMIT GENMASK(15, 12) #define MT_AGG_MRCR_LAST_RTS_CTS_RN BIT(6) diff --git a/drivers/net/wireless/mediatek/mt76/mt7921/acpi_sar.c b/drivers/net/wireless/mediatek/mt76/mt7921/acpi_sar.c index be4f07ad3af9..47e034a9b003 100644 --- a/drivers/net/wireless/mediatek/mt76/mt7921/acpi_sar.c +++ b/drivers/net/wireless/mediatek/mt76/mt7921/acpi_sar.c @@ -13,6 +13,7 @@ mt7921_acpi_read(struct mt7921_dev *dev, u8 *method, u8 **tbl, u32 *len) acpi_handle root, handle; acpi_status status; u32 i = 0; + int ret; root = ACPI_HANDLE(mdev->dev); if (!root) @@ -52,9 +53,11 @@ mt7921_acpi_read(struct mt7921_dev *dev, u8 *method, u8 **tbl, u32 *len) *(*tbl + i) = (u8)sar_unit->integer.value; } free: + ret = (i == sar_root->package.count) ? 0 : -EINVAL; + kfree(sar_root); - return (i == sar_root->package.count) ? 0 : -EINVAL; + return ret; } /* MTCL : Country List Table for 6G band */ diff --git a/drivers/net/wireless/mediatek/mt76/mt7921/eeprom.h b/drivers/net/wireless/mediatek/mt76/mt7921/eeprom.h index 54f30401343c..4b647278eb30 100644 --- a/drivers/net/wireless/mediatek/mt76/mt7921/eeprom.h +++ b/drivers/net/wireless/mediatek/mt76/mt7921/eeprom.h @@ -11,12 +11,15 @@ enum mt7921_eeprom_field { MT_EE_VERSION = 0x002, MT_EE_MAC_ADDR = 0x004, MT_EE_WIFI_CONF = 0x07c, - __MT_EE_MAX = 0x3bf + MT_EE_HW_TYPE = 0x55b, + __MT_EE_MAX = 0x9ff }; #define MT_EE_WIFI_CONF_TX_MASK BIT(0) #define MT_EE_WIFI_CONF_BAND_SEL GENMASK(3, 2) +#define MT_EE_HW_TYPE_ENCAP BIT(0) + enum mt7921_eeprom_band { MT_EE_NA, MT_EE_5GHZ, diff --git a/drivers/net/wireless/mediatek/mt76/mt7921/init.c b/drivers/net/wireless/mediatek/mt76/mt7921/init.c index cd960e23770f..dcdb3cf04ac1 100644 --- a/drivers/net/wireless/mediatek/mt76/mt7921/init.c +++ b/drivers/net/wireless/mediatek/mt76/mt7921/init.c @@ -39,6 +39,7 @@ mt7921_regd_notifier(struct wiphy *wiphy, dev->mt76.region = request->dfs_region; mt7921_mutex_acquire(dev); + mt7921_mcu_set_clc(dev, request->alpha2, request->country_ie_env); mt76_connac_mcu_set_channel_domain(hw->priv); mt7921_set_tx_sar_pwr(hw, NULL); mt7921_mutex_release(dev); diff --git a/drivers/net/wireless/mediatek/mt76/mt7921/mac.c b/drivers/net/wireless/mediatek/mt76/mt7921/mac.c index 47f0aa81ab02..e4868c492bc0 100644 --- a/drivers/net/wireless/mediatek/mt76/mt7921/mac.c +++ b/drivers/net/wireless/mediatek/mt76/mt7921/mac.c @@ -235,7 +235,7 @@ mt7921_mac_fill_rx(struct mt7921_dev *dev, struct sk_buff *skb) u32 rxd2 = le32_to_cpu(rxd[2]); u32 rxd3 = le32_to_cpu(rxd[3]); u32 rxd4 = le32_to_cpu(rxd[4]); - struct mt7921_sta *msta; + struct mt7921_sta *msta = NULL; u16 seq_ctrl = 0; __le16 fc = 0; u8 mode = 0; @@ -486,7 +486,7 @@ mt7921_mac_fill_rx(struct mt7921_dev *dev, struct sk_buff *skb) return 0; } -void mt7921_tx_check_aggr(struct ieee80211_sta *sta, __le32 *txwi) +static void mt7921_tx_check_aggr(struct ieee80211_sta *sta, __le32 *txwi) { struct mt7921_sta *msta; u16 fc, tid; @@ -509,7 +509,6 @@ void mt7921_tx_check_aggr(struct ieee80211_sta *sta, __le32 *txwi) if (!test_and_set_bit(tid, &msta->ampdu_state)) ieee80211_start_tx_ba_session(sta, tid, 0); } -EXPORT_SYMBOL_GPL(mt7921_tx_check_aggr); void mt7921_mac_add_txs(struct mt7921_dev *dev, void *data) { @@ -539,8 +538,7 @@ void mt7921_mac_add_txs(struct mt7921_dev *dev, void *data) msta = container_of(wcid, struct mt7921_sta, wcid); - mt76_connac2_mac_add_txs_skb(&dev->mt76, wcid, pid, txs_data, - &msta->stats); + mt76_connac2_mac_add_txs_skb(&dev->mt76, wcid, pid, txs_data); if (!wcid->sta) goto out; @@ -552,7 +550,134 @@ void mt7921_mac_add_txs(struct mt7921_dev *dev, void *data) out: rcu_read_unlock(); } -EXPORT_SYMBOL_GPL(mt7921_mac_add_txs); + +void mt7921_txwi_free(struct mt7921_dev *dev, struct mt76_txwi_cache *t, + struct ieee80211_sta *sta, bool clear_status, + struct list_head *free_list) +{ + struct mt76_dev *mdev = &dev->mt76; + __le32 *txwi; + u16 wcid_idx; + + mt76_connac_txp_skb_unmap(mdev, t); + if (!t->skb) + goto out; + + txwi = (__le32 *)mt76_get_txwi_ptr(mdev, t); + if (sta) { + struct mt76_wcid *wcid = (struct mt76_wcid *)sta->drv_priv; + + if (likely(t->skb->protocol != cpu_to_be16(ETH_P_PAE))) + mt7921_tx_check_aggr(sta, txwi); + + wcid_idx = wcid->idx; + } else { + wcid_idx = le32_get_bits(txwi[1], MT_TXD1_WLAN_IDX); + } + + __mt76_tx_complete_skb(mdev, wcid_idx, t->skb, free_list); +out: + t->skb = NULL; + mt76_put_txwi(mdev, t); +} +EXPORT_SYMBOL_GPL(mt7921_txwi_free); + +static void mt7921_mac_tx_free(struct mt7921_dev *dev, void *data, int len) +{ + struct mt76_connac_tx_free *free = data; + __le32 *tx_info = (__le32 *)(data + sizeof(*free)); + struct mt76_dev *mdev = &dev->mt76; + struct mt76_txwi_cache *txwi; + struct ieee80211_sta *sta = NULL; + struct sk_buff *skb, *tmp; + void *end = data + len; + LIST_HEAD(free_list); + bool wake = false; + u8 i, count; + + /* clean DMA queues and unmap buffers first */ + mt76_queue_tx_cleanup(dev, dev->mphy.q_tx[MT_TXQ_PSD], false); + mt76_queue_tx_cleanup(dev, dev->mphy.q_tx[MT_TXQ_BE], false); + + count = le16_get_bits(free->ctrl, MT_TX_FREE_MSDU_CNT); + if (WARN_ON_ONCE((void *)&tx_info[count] > end)) + return; + + for (i = 0; i < count; i++) { + u32 msdu, info = le32_to_cpu(tx_info[i]); + u8 stat; + + /* 1'b1: new wcid pair. + * 1'b0: msdu_id with the same 'wcid pair' as above. + */ + if (info & MT_TX_FREE_PAIR) { + struct mt7921_sta *msta; + struct mt76_wcid *wcid; + u16 idx; + + count++; + idx = FIELD_GET(MT_TX_FREE_WLAN_ID, info); + wcid = rcu_dereference(dev->mt76.wcid[idx]); + sta = wcid_to_sta(wcid); + if (!sta) + continue; + + msta = container_of(wcid, struct mt7921_sta, wcid); + spin_lock_bh(&dev->sta_poll_lock); + if (list_empty(&msta->poll_list)) + list_add_tail(&msta->poll_list, &dev->sta_poll_list); + spin_unlock_bh(&dev->sta_poll_lock); + continue; + } + + msdu = FIELD_GET(MT_TX_FREE_MSDU_ID, info); + stat = FIELD_GET(MT_TX_FREE_STATUS, info); + + txwi = mt76_token_release(mdev, msdu, &wake); + if (!txwi) + continue; + + mt7921_txwi_free(dev, txwi, sta, stat, &free_list); + } + + if (wake) + mt76_set_tx_blocked(&dev->mt76, false); + + list_for_each_entry_safe(skb, tmp, &free_list, list) { + skb_list_del_init(skb); + napi_consume_skb(skb, 1); + } + + rcu_read_lock(); + mt7921_mac_sta_poll(dev); + rcu_read_unlock(); + + mt76_worker_schedule(&dev->mt76.tx_worker); +} + +bool mt7921_rx_check(struct mt76_dev *mdev, void *data, int len) +{ + struct mt7921_dev *dev = container_of(mdev, struct mt7921_dev, mt76); + __le32 *rxd = (__le32 *)data; + __le32 *end = (__le32 *)&rxd[len / 4]; + enum rx_pkt_type type; + + type = le32_get_bits(rxd[0], MT_RXD0_PKT_TYPE); + + switch (type) { + case PKT_TYPE_TXRX_NOTIFY: + /* PKT_TYPE_TXRX_NOTIFY can be received only by mmio devices */ + mt7921_mac_tx_free(dev, data, len); /* mmio */ + return false; + case PKT_TYPE_TXS: + for (rxd += 2; rxd + 8 <= end; rxd += 8) + mt7921_mac_add_txs(dev, rxd); + return false; + default: + return true; + } +} +EXPORT_SYMBOL_GPL(mt7921_rx_check); void mt7921_queue_rx_skb(struct mt76_dev *mdev, enum mt76_rxq_id q, struct sk_buff *skb) @@ -570,6 +695,11 @@ void mt7921_queue_rx_skb(struct mt76_dev *mdev, enum mt76_rxq_id q, type = PKT_TYPE_NORMAL_MCU; switch (type) { + case PKT_TYPE_TXRX_NOTIFY: + /* PKT_TYPE_TXRX_NOTIFY can be received only by mmio devices */ + mt7921_mac_tx_free(dev, skb->data, skb->len); + napi_consume_skb(skb, 1); + break; case PKT_TYPE_RX_EVENT: mt7921_mcu_rx_event(dev, skb); break; @@ -780,6 +910,7 @@ void mt7921_mac_reset_work(struct work_struct *work) void mt7921_reset(struct mt76_dev *mdev) { struct mt7921_dev *dev = container_of(mdev, struct mt7921_dev, mt76); + struct mt76_connac_pm *pm = &dev->pm; if (!dev->hw_init_done) return; @@ -787,8 +918,12 @@ void mt7921_reset(struct mt76_dev *mdev) if (dev->hw_full_reset) return; + if (pm->suspended) + return; + queue_work(dev->mt76.wq, &dev->reset_work); } +EXPORT_SYMBOL_GPL(mt7921_reset); void mt7921_mac_update_mib_stats(struct mt7921_phy *phy) { diff --git a/drivers/net/wireless/mediatek/mt76/mt7921/main.c b/drivers/net/wireless/mediatek/mt76/mt7921/main.c index 1438a9f8d1fd..7e409ac7d9a8 100644 --- a/drivers/net/wireless/mediatek/mt76/mt7921/main.c +++ b/drivers/net/wireless/mediatek/mt76/mt7921/main.c @@ -752,6 +752,7 @@ void mt7921_mac_sta_assoc(struct mt76_dev *mdev, struct ieee80211_vif *vif, mt7921_mac_wtbl_update(dev, msta->wcid.idx, MT_WTBL_UPDATE_ADM_COUNT_CLEAR); + memset(msta->airtime_ac, 0, sizeof(msta->airtime_ac)); mt7921_mcu_sta_update(dev, sta, vif, true, MT76_STA_INFO_STATE_ASSOC); @@ -1045,7 +1046,7 @@ mt7921_ethtool_worker(void *wi_data, struct ieee80211_sta *sta) if (msta->vif->mt76.idx != wi->idx) return; - mt76_ethtool_worker(wi, &msta->stats); + mt76_ethtool_worker(wi, &msta->wcid.stats); } static @@ -1404,6 +1405,8 @@ static void mt7921_sta_set_decap_offload(struct ieee80211_hw *hw, struct mt7921_sta *msta = (struct mt7921_sta *)sta->drv_priv; struct mt7921_dev *dev = mt7921_hw_dev(hw); + mt7921_mutex_acquire(dev); + if (enabled) set_bit(MT_WCID_FLAG_HDR_TRANS, &msta->wcid.flags); else @@ -1411,6 +1414,8 @@ static void mt7921_sta_set_decap_offload(struct ieee80211_hw *hw, mt76_connac_mcu_sta_update_hdr_trans(&dev->mt76, vif, &msta->wcid, MCU_UNI_CMD(STA_REC_UPDATE)); + + mt7921_mutex_release(dev); } #if IS_ENABLED(CONFIG_IPV6) @@ -1526,17 +1531,23 @@ mt7921_start_ap(struct ieee80211_hw *hw, struct ieee80211_vif *vif, struct mt7921_dev *dev = mt7921_hw_dev(hw); int err; + mt7921_mutex_acquire(dev); + err = mt76_connac_mcu_uni_add_bss(phy->mt76, vif, &mvif->sta.wcid, true); if (err) - return err; + goto out; err = mt7921_mcu_set_bss_pm(dev, vif, true); if (err) - return err; + goto out; + + err = mt7921_mcu_sta_update(dev, NULL, vif, true, + MT76_STA_INFO_STATE_NONE); +out: + mt7921_mutex_release(dev); - return mt7921_mcu_sta_update(dev, NULL, vif, true, - MT76_STA_INFO_STATE_NONE); + return err; } static void @@ -1548,11 +1559,16 @@ mt7921_stop_ap(struct ieee80211_hw *hw, struct ieee80211_vif *vif, struct mt7921_dev *dev = mt7921_hw_dev(hw); int err; + mt7921_mutex_acquire(dev); + err = mt7921_mcu_set_bss_pm(dev, vif, false); if (err) - return; + goto out; mt76_connac_mcu_uni_add_bss(phy->mt76, vif, &mvif->sta.wcid, false); + +out: + mt7921_mutex_release(dev); } const struct ieee80211_ops mt7921_ops = { diff --git a/drivers/net/wireless/mediatek/mt76/mt7921/mcu.c b/drivers/net/wireless/mediatek/mt76/mt7921/mcu.c index da12d0ae0835..67bf92969a7b 100644 --- a/drivers/net/wireless/mediatek/mt76/mt7921/mcu.c +++ b/drivers/net/wireless/mediatek/mt76/mt7921/mcu.c @@ -2,14 +2,20 @@ /* Copyright (C) 2020 MediaTek Inc. */ #include <linux/fs.h> +#include <linux/firmware.h> #include "mt7921.h" #include "mt7921_trace.h" +#include "eeprom.h" #include "mcu.h" #include "mac.h" #define MT_STA_BFER BIT(0) #define MT_STA_BFEE BIT(1) +static bool mt7921_disable_clc; +module_param_named(disable_clc, mt7921_disable_clc, bool, 0644); +MODULE_PARM_DESC(disable_clc, "disable CLC support"); + static int mt7921_mcu_parse_eeprom(struct mt76_dev *dev, struct sk_buff *skb) { @@ -84,6 +90,27 @@ int mt7921_mcu_parse_response(struct mt76_dev *mdev, int cmd, } EXPORT_SYMBOL_GPL(mt7921_mcu_parse_response); +static int mt7921_mcu_read_eeprom(struct mt7921_dev *dev, u32 offset, u8 *val) +{ + struct mt7921_mcu_eeprom_info *res, req = { + .addr = cpu_to_le32(round_down(offset, + MT7921_EEPROM_BLOCK_SIZE)), + }; + struct sk_buff *skb; + int ret; + + ret = mt76_mcu_send_and_get_msg(&dev->mt76, MCU_EXT_QUERY(EFUSE_ACCESS), + &req, sizeof(req), true, &skb); + if (ret) + return ret; + + res = (struct mt7921_mcu_eeprom_info *)skb->data; + *val = res->data[offset % MT7921_EEPROM_BLOCK_SIZE]; + dev_kfree_skb(skb); + + return 0; +} + #ifdef CONFIG_PM static int @@ -354,6 +381,90 @@ static char *mt7921_ram_name(struct mt7921_dev *dev) return ret; } +static int mt7921_load_clc(struct mt7921_dev *dev, const char *fw_name) +{ + const struct mt76_connac2_fw_trailer *hdr; + const struct mt76_connac2_fw_region *region; + const struct mt7921_clc *clc; + struct mt76_dev *mdev = &dev->mt76; + struct mt7921_phy *phy = &dev->phy; + const struct firmware *fw; + int ret, i, len, offset = 0; + u8 *clc_base = NULL, hw_encap = 0; + + if (mt7921_disable_clc || + mt76_is_usb(&dev->mt76)) + return 0; + + if (mt76_is_mmio(&dev->mt76)) { + ret = mt7921_mcu_read_eeprom(dev, MT_EE_HW_TYPE, &hw_encap); + if (ret) + return ret; + hw_encap = u8_get_bits(hw_encap, MT_EE_HW_TYPE_ENCAP); + } + + ret = request_firmware(&fw, fw_name, mdev->dev); + if (ret) + return ret; + + if (!fw || !fw->data || fw->size < sizeof(*hdr)) { + dev_err(mdev->dev, "Invalid firmware\n"); + ret = -EINVAL; + goto out; + } + + hdr = (const void *)(fw->data + fw->size - sizeof(*hdr)); + for (i = 0; i < hdr->n_region; i++) { + region = (const void *)((const u8 *)hdr - + (hdr->n_region - i) * sizeof(*region)); + len = le32_to_cpu(region->len); + + /* check if we have valid buffer size */ + if (offset + len > fw->size) { + dev_err(mdev->dev, "Invalid firmware region\n"); + ret = -EINVAL; + goto out; + } + + if ((region->feature_set & FW_FEATURE_NON_DL) && + region->type == FW_TYPE_CLC) { + clc_base = (u8 *)(fw->data + offset); + break; + } + offset += len; + } + + if (!clc_base) + goto out; + + for (offset = 0; offset < len; offset += le32_to_cpu(clc->len)) { + clc = (const struct mt7921_clc *)(clc_base + offset); + + /* do not init buf again if chip reset triggered */ + if (phy->clc[clc->idx]) + continue; + + /* header content sanity */ + if (clc->idx == MT7921_CLC_POWER && + u8_get_bits(clc->type, MT_EE_HW_TYPE_ENCAP) != hw_encap) + continue; + + phy->clc[clc->idx] = devm_kmemdup(mdev->dev, clc, + le32_to_cpu(clc->len), + GFP_KERNEL); + + if (!phy->clc[clc->idx]) { + ret = -ENOMEM; + goto out; + } + } + ret = mt7921_mcu_set_clc(dev, "00", ENVIRON_INDOOR); +out: + release_firmware(fw); + + return ret; +} + static int mt7921_load_firmware(struct mt7921_dev *dev) { int ret; @@ -423,6 +534,10 @@ int mt7921_run_firmware(struct mt7921_dev *dev) return err; set_bit(MT76_STATE_MCU_RUNNING, &dev->mphy.state); + err = mt7921_load_clc(dev, mt7921_ram_name(dev)); + if (err) + return err; + return mt7921_mcu_fw_log_2_host(dev, 1); } EXPORT_SYMBOL_GPL(mt7921_run_firmware); @@ -930,3 +1045,86 @@ mt7921_mcu_uni_add_beacon_offload(struct mt7921_dev *dev, return mt76_mcu_send_msg(&dev->mt76, MCU_UNI_CMD(BSS_INFO_UPDATE), &req, sizeof(req), true); } + +static +int __mt7921_mcu_set_clc(struct mt7921_dev *dev, u8 *alpha2, + enum environment_cap env_cap, + struct mt7921_clc *clc, + u8 idx) +{ + struct sk_buff *skb; + struct { + u8 ver; + u8 pad0; + __le16 len; + u8 idx; + u8 env; + u8 pad1[2]; + u8 alpha2[2]; + u8 type[2]; + u8 rsvd[64]; + } __packed req = { + .idx = idx, + .env = env_cap, + }; + int ret, valid_cnt = 0; + u8 i, *pos; + + if (!clc) + return 0; + + pos = clc->data; + for (i = 0; i < clc->nr_country; i++) { + struct mt7921_clc_rule *rule = (struct mt7921_clc_rule *)pos; + u16 len = le16_to_cpu(rule->len); + + pos += len + sizeof(*rule); + if (rule->alpha2[0] != alpha2[0] || + rule->alpha2[1] != alpha2[1]) + continue; + + memcpy(req.alpha2, rule->alpha2, 2); + memcpy(req.type, rule->type, 2); + + req.len = cpu_to_le16(sizeof(req) + len); + skb = __mt76_mcu_msg_alloc(&dev->mt76, &req, + le16_to_cpu(req.len), + sizeof(req), GFP_KERNEL); + if (!skb) + return -ENOMEM; + skb_put_data(skb, rule->data, len); + + ret = mt76_mcu_skb_send_msg(&dev->mt76, skb, + MCU_CE_CMD(SET_CLC), false); + if (ret < 0) + return ret; + valid_cnt++; + } + + if (!valid_cnt) + return -ENOENT; + + return 0; +} + +int mt7921_mcu_set_clc(struct mt7921_dev *dev, u8 *alpha2, + enum environment_cap env_cap) +{ + struct mt7921_phy *phy = (struct mt7921_phy *)&dev->phy; + int i, ret; + + /* submit all clc config */ + for (i = 0; i < ARRAY_SIZE(phy->clc); i++) { + ret = __mt7921_mcu_set_clc(dev, alpha2, env_cap, + phy->clc[i], i); + + /* If no country found, set "00" as default */ + if (ret == -ENOENT) + ret = __mt7921_mcu_set_clc(dev, "00", + ENVIRON_INDOOR, + phy->clc[i], i); + if (ret < 0) + return ret; + } + return 0; +} diff --git a/drivers/net/wireless/mediatek/mt76/mt7921/mcu.h b/drivers/net/wireless/mediatek/mt76/mt7921/mcu.h index 0d20f7d8d474..96dc870fd35e 100644 --- a/drivers/net/wireless/mediatek/mt76/mt7921/mcu.h +++ b/drivers/net/wireless/mediatek/mt76/mt7921/mcu.h @@ -41,7 +41,7 @@ enum { struct mt7921_mcu_eeprom_info { __le32 addr; __le32 valid; - u8 data[16]; + u8 data[MT7921_EEPROM_BLOCK_SIZE]; } __packed; #define MT_RA_RATE_NSS GENMASK(8, 6) diff --git a/drivers/net/wireless/mediatek/mt76/mt7921/mt7921.h b/drivers/net/wireless/mediatek/mt76/mt7921/mt7921.h index c161031ac62a..eaba114a9c7e 100644 --- a/drivers/net/wireless/mediatek/mt76/mt7921/mt7921.h +++ b/drivers/net/wireless/mediatek/mt76/mt7921/mt7921.h @@ -41,6 +41,8 @@ #define MT7921_EEPROM_SIZE 3584 #define MT7921_TOKEN_SIZE 8192 +#define MT7921_EEPROM_BLOCK_SIZE 16 + #define MT7921_CFEND_RATE_DEFAULT 0x49 /* OFDM 24M */ #define MT7921_CFEND_RATE_11B 0x03 /* 11B LP, 11M */ @@ -100,7 +102,6 @@ struct mt7921_sta { unsigned long last_txs; unsigned long ampdu_state; - struct mt76_sta_stats stats; struct mt76_connac_sta_key_conf bip; }; @@ -149,6 +150,29 @@ struct mib_stats { u32 tx_amsdu_cnt; }; +enum { + MT7921_CLC_POWER, + MT7921_CLC_CHAN, + MT7921_CLC_MAX_NUM, +}; + +struct mt7921_clc_rule { + u8 alpha2[2]; + u8 type[2]; + __le16 len; + u8 data[]; +} __packed; + +struct mt7921_clc { + __le32 len; + u8 idx; + u8 ver; + u8 nr_country; + u8 type; + u8 rsv[8]; + u8 data[]; +}; + struct mt7921_phy { struct mt76_phy *mt76; struct mt7921_dev *dev; @@ -174,6 +198,8 @@ struct mt7921_phy { #ifdef CONFIG_ACPI struct mt7921_acpi_sar *acpisar; #endif + + struct mt7921_clc *clc[MT7921_CLC_MAX_NUM]; }; #define mt7921_init_reset(dev) ((dev)->hif_ops->init_reset(dev)) @@ -380,6 +406,7 @@ int mt7921e_tx_prepare_skb(struct mt76_dev *mdev, void *txwi_ptr, void mt7921_tx_worker(struct mt76_worker *w); void mt7921_tx_token_put(struct mt7921_dev *dev); +bool mt7921_rx_check(struct mt76_dev *mdev, void *data, int len); void mt7921_queue_rx_skb(struct mt76_dev *mdev, enum mt76_rxq_id q, struct sk_buff *skb); void mt7921_sta_ps(struct mt76_dev *mdev, struct ieee80211_sta *sta, bool ps); @@ -410,14 +437,13 @@ int mt7921_testmode_cmd(struct ieee80211_hw *hw, struct ieee80211_vif *vif, void *data, int len); int mt7921_testmode_dump(struct ieee80211_hw *hw, struct sk_buff *msg, struct netlink_callback *cb, void *data, int len); -void mt7921_tx_check_aggr(struct ieee80211_sta *sta, __le32 *txwi); +void mt7921_txwi_free(struct mt7921_dev *dev, struct mt76_txwi_cache *t, + struct ieee80211_sta *sta, bool clear_status, + struct list_head *free_list); void mt7921_mac_sta_poll(struct mt7921_dev *dev); int mt7921_mcu_parse_response(struct mt76_dev *mdev, int cmd, struct sk_buff *skb, int seq); -bool mt7921e_rx_check(struct mt76_dev *mdev, void *data, int len); -void mt7921e_queue_rx_skb(struct mt76_dev *mdev, enum mt76_rxq_id q, - struct sk_buff *skb); int mt7921e_driver_own(struct mt7921_dev *dev); int mt7921e_mac_reset(struct mt7921_dev *dev); int mt7921e_mcu_init(struct mt7921_dev *dev); @@ -479,4 +505,7 @@ mt7921_init_acpi_sar_power(struct mt7921_phy *phy, bool set_default) #endif int mt7921_set_tx_sar_pwr(struct ieee80211_hw *hw, const struct cfg80211_sar_specs *sar); + +int mt7921_mcu_set_clc(struct mt7921_dev *dev, u8 *alpha2, + enum environment_cap env_cap); #endif diff --git a/drivers/net/wireless/mediatek/mt76/mt7921/pci.c b/drivers/net/wireless/mediatek/mt76/mt7921/pci.c index ea3069d18c35..8a53d8f286db 100644 --- a/drivers/net/wireless/mediatek/mt76/mt7921/pci.c +++ b/drivers/net/wireless/mediatek/mt76/mt7921/pci.c @@ -123,54 +123,51 @@ static void mt7921e_unregister_device(struct mt7921_dev *dev) static u32 __mt7921_reg_addr(struct mt7921_dev *dev, u32 addr) { - static const struct { - u32 phys; - u32 mapped; - u32 size; - } fixed_map[] = { + static const struct mt76_connac_reg_map fixed_map[] = { { 0x820d0000, 0x30000, 0x10000 }, /* WF_LMAC_TOP (WF_WTBLON) */ - { 0x820ed000, 0x24800, 0x0800 }, /* WF_LMAC_TOP BN0 (WF_MIB) */ - { 0x820e4000, 0x21000, 0x0400 }, /* WF_LMAC_TOP BN0 (WF_TMAC) */ - { 0x820e7000, 0x21e00, 0x0200 }, /* WF_LMAC_TOP BN0 (WF_DMA) */ - { 0x820eb000, 0x24200, 0x0400 }, /* WF_LMAC_TOP BN0 (WF_LPON) */ - { 0x820e2000, 0x20800, 0x0400 }, /* WF_LMAC_TOP BN0 (WF_AGG) */ - { 0x820e3000, 0x20c00, 0x0400 }, /* WF_LMAC_TOP BN0 (WF_ARB) */ - { 0x820e5000, 0x21400, 0x0800 }, /* WF_LMAC_TOP BN0 (WF_RMAC) */ + { 0x820ed000, 0x24800, 0x00800 }, /* WF_LMAC_TOP BN0 (WF_MIB) */ + { 0x820e4000, 0x21000, 0x00400 }, /* WF_LMAC_TOP BN0 (WF_TMAC) */ + { 0x820e7000, 0x21e00, 0x00200 }, /* WF_LMAC_TOP BN0 (WF_DMA) */ + { 0x820eb000, 0x24200, 0x00400 }, /* WF_LMAC_TOP BN0 (WF_LPON) */ + { 0x820e2000, 0x20800, 0x00400 }, /* WF_LMAC_TOP BN0 (WF_AGG) */ + { 0x820e3000, 0x20c00, 0x00400 }, /* WF_LMAC_TOP BN0 (WF_ARB) */ + { 0x820e5000, 0x21400, 0x00800 }, /* WF_LMAC_TOP BN0 (WF_RMAC) */ { 0x00400000, 0x80000, 0x10000 }, /* WF_MCU_SYSRAM */ { 0x00410000, 0x90000, 0x10000 }, /* WF_MCU_SYSRAM (configure register) */ { 0x40000000, 0x70000, 0x10000 }, /* WF_UMAC_SYSRAM */ - { 0x54000000, 0x02000, 0x1000 }, /* WFDMA PCIE0 MCU DMA0 */ - { 0x55000000, 0x03000, 0x1000 }, /* WFDMA PCIE0 MCU DMA1 */ - { 0x58000000, 0x06000, 0x1000 }, /* WFDMA PCIE1 MCU DMA0 (MEM_DMA) */ - { 0x59000000, 0x07000, 0x1000 }, /* WFDMA PCIE1 MCU DMA1 */ + { 0x54000000, 0x02000, 0x01000 }, /* WFDMA PCIE0 MCU DMA0 */ + { 0x55000000, 0x03000, 0x01000 }, /* WFDMA PCIE0 MCU DMA1 */ + { 0x58000000, 0x06000, 0x01000 }, /* WFDMA PCIE1 MCU DMA0 (MEM_DMA) */ + { 0x59000000, 0x07000, 0x01000 }, /* WFDMA PCIE1 MCU DMA1 */ { 0x7c000000, 0xf0000, 0x10000 }, /* CONN_INFRA */ { 0x7c020000, 0xd0000, 0x10000 }, /* CONN_INFRA, WFDMA */ { 0x7c060000, 0xe0000, 0x10000 }, /* CONN_INFRA, conn_host_csr_top */ { 0x80020000, 0xb0000, 0x10000 }, /* WF_TOP_MISC_OFF */ { 0x81020000, 0xc0000, 0x10000 }, /* WF_TOP_MISC_ON */ - { 0x820c0000, 0x08000, 0x4000 }, /* WF_UMAC_TOP (PLE) */ - { 0x820c8000, 0x0c000, 0x2000 }, /* WF_UMAC_TOP (PSE) */ - { 0x820cc000, 0x0e000, 0x1000 }, /* WF_UMAC_TOP (PP) */ - { 0x820cd000, 0x0f000, 0x1000 }, /* WF_MDP_TOP */ - { 0x820ce000, 0x21c00, 0x0200 }, /* WF_LMAC_TOP (WF_SEC) */ - { 0x820cf000, 0x22000, 0x1000 }, /* WF_LMAC_TOP (WF_PF) */ - { 0x820e0000, 0x20000, 0x0400 }, /* WF_LMAC_TOP BN0 (WF_CFG) */ - { 0x820e1000, 0x20400, 0x0200 }, /* WF_LMAC_TOP BN0 (WF_TRB) */ - { 0x820e9000, 0x23400, 0x0200 }, /* WF_LMAC_TOP BN0 (WF_WTBLOFF) */ - { 0x820ea000, 0x24000, 0x0200 }, /* WF_LMAC_TOP BN0 (WF_ETBF) */ - { 0x820ec000, 0x24600, 0x0200 }, /* WF_LMAC_TOP BN0 (WF_INT) */ - { 0x820f0000, 0xa0000, 0x0400 }, /* WF_LMAC_TOP BN1 (WF_CFG) */ - { 0x820f1000, 0xa0600, 0x0200 }, /* WF_LMAC_TOP BN1 (WF_TRB) */ - { 0x820f2000, 0xa0800, 0x0400 }, /* WF_LMAC_TOP BN1 (WF_AGG) */ - { 0x820f3000, 0xa0c00, 0x0400 }, /* WF_LMAC_TOP BN1 (WF_ARB) */ - { 0x820f4000, 0xa1000, 0x0400 }, /* WF_LMAC_TOP BN1 (WF_TMAC) */ - { 0x820f5000, 0xa1400, 0x0800 }, /* WF_LMAC_TOP BN1 (WF_RMAC) */ - { 0x820f7000, 0xa1e00, 0x0200 }, /* WF_LMAC_TOP BN1 (WF_DMA) */ - { 0x820f9000, 0xa3400, 0x0200 }, /* WF_LMAC_TOP BN1 (WF_WTBLOFF) */ - { 0x820fa000, 0xa4000, 0x0200 }, /* WF_LMAC_TOP BN1 (WF_ETBF) */ - { 0x820fb000, 0xa4200, 0x0400 }, /* WF_LMAC_TOP BN1 (WF_LPON) */ - { 0x820fc000, 0xa4600, 0x0200 }, /* WF_LMAC_TOP BN1 (WF_INT) */ - { 0x820fd000, 0xa4800, 0x0800 }, /* WF_LMAC_TOP BN1 (WF_MIB) */ + { 0x820c0000, 0x08000, 0x04000 }, /* WF_UMAC_TOP (PLE) */ + { 0x820c8000, 0x0c000, 0x02000 }, /* WF_UMAC_TOP (PSE) */ + { 0x820cc000, 0x0e000, 0x01000 }, /* WF_UMAC_TOP (PP) */ + { 0x820cd000, 0x0f000, 0x01000 }, /* WF_MDP_TOP */ + { 0x74030000, 0x10000, 0x10000 }, /* PCIE_MAC_IREG */ + { 0x820ce000, 0x21c00, 0x00200 }, /* WF_LMAC_TOP (WF_SEC) */ + { 0x820cf000, 0x22000, 0x01000 }, /* WF_LMAC_TOP (WF_PF) */ + { 0x820e0000, 0x20000, 0x00400 }, /* WF_LMAC_TOP BN0 (WF_CFG) */ + { 0x820e1000, 0x20400, 0x00200 }, /* WF_LMAC_TOP BN0 (WF_TRB) */ + { 0x820e9000, 0x23400, 0x00200 }, /* WF_LMAC_TOP BN0 (WF_WTBLOFF) */ + { 0x820ea000, 0x24000, 0x00200 }, /* WF_LMAC_TOP BN0 (WF_ETBF) */ + { 0x820ec000, 0x24600, 0x00200 }, /* WF_LMAC_TOP BN0 (WF_INT) */ + { 0x820f0000, 0xa0000, 0x00400 }, /* WF_LMAC_TOP BN1 (WF_CFG) */ + { 0x820f1000, 0xa0600, 0x00200 }, /* WF_LMAC_TOP BN1 (WF_TRB) */ + { 0x820f2000, 0xa0800, 0x00400 }, /* WF_LMAC_TOP BN1 (WF_AGG) */ + { 0x820f3000, 0xa0c00, 0x00400 }, /* WF_LMAC_TOP BN1 (WF_ARB) */ + { 0x820f4000, 0xa1000, 0x00400 }, /* WF_LMAC_TOP BN1 (WF_TMAC) */ + { 0x820f5000, 0xa1400, 0x00800 }, /* WF_LMAC_TOP BN1 (WF_RMAC) */ + { 0x820f7000, 0xa1e00, 0x00200 }, /* WF_LMAC_TOP BN1 (WF_DMA) */ + { 0x820f9000, 0xa3400, 0x00200 }, /* WF_LMAC_TOP BN1 (WF_WTBLOFF) */ + { 0x820fa000, 0xa4000, 0x00200 }, /* WF_LMAC_TOP BN1 (WF_ETBF) */ + { 0x820fb000, 0xa4200, 0x00400 }, /* WF_LMAC_TOP BN1 (WF_LPON) */ + { 0x820fc000, 0xa4600, 0x00200 }, /* WF_LMAC_TOP BN1 (WF_INT) */ + { 0x820fd000, 0xa4800, 0x00800 }, /* WF_LMAC_TOP BN1 (WF_MIB) */ }; int i; @@ -187,7 +184,7 @@ static u32 __mt7921_reg_addr(struct mt7921_dev *dev, u32 addr) if (ofs > fixed_map[i].size) continue; - return fixed_map[i].mapped + ofs; + return fixed_map[i].maps + ofs; } if ((addr >= 0x18000000 && addr < 0x18c00000) || @@ -238,8 +235,8 @@ static int mt7921_pci_probe(struct pci_dev *pdev, .token_size = MT7921_TOKEN_SIZE, .tx_prepare_skb = mt7921e_tx_prepare_skb, .tx_complete_skb = mt76_connac_tx_complete_skb, - .rx_check = mt7921e_rx_check, - .rx_skb = mt7921e_queue_rx_skb, + .rx_check = mt7921_rx_check, + .rx_skb = mt7921_queue_rx_skb, .rx_poll_complete = mt7921_rx_poll_complete, .sta_ps = mt7921_sta_ps, .sta_add = mt7921_mac_sta_add, @@ -288,6 +285,8 @@ static int mt7921_pci_probe(struct pci_dev *pdev, goto err_free_pci_vec; } + pci_set_drvdata(pdev, mdev); + dev = container_of(mdev, struct mt7921_dev, mt76); dev->hif_ops = &mt7921_pcie_ops; @@ -367,6 +366,7 @@ static int mt7921_pci_suspend(struct device *device) int i, err; pm->suspended = true; + flush_work(&dev->reset_work); cancel_delayed_work_sync(&pm->ps_work); cancel_work_sync(&pm->wake_work); @@ -409,9 +409,6 @@ static int mt7921_pci_suspend(struct device *device) if (err) goto restore_napi; - if (err) - goto restore_napi; - return 0; restore_napi: @@ -428,6 +425,9 @@ restore_napi: restore_suspend: pm->suspended = false; + if (err < 0) + mt7921_reset(&dev->mt76); + return err; } @@ -441,7 +441,7 @@ static int mt7921_pci_resume(struct device *device) err = mt7921_mcu_drv_pmctrl(dev); if (err < 0) - return err; + goto failed; mt7921_wpdma_reinit_cond(dev); @@ -471,11 +471,12 @@ static int mt7921_pci_resume(struct device *device) mt76_connac_mcu_set_deep_sleep(&dev->mt76, false); err = mt76_connac_mcu_set_hif_suspend(mdev, false); - if (err) - return err; - +failed: pm->suspended = false; + if (err < 0) + mt7921_reset(&dev->mt76); + return err; } diff --git a/drivers/net/wireless/mediatek/mt76/mt7921/pci_mac.c b/drivers/net/wireless/mediatek/mt76/mt7921/pci_mac.c index 576a0149251b..8dd60408b117 100644 --- a/drivers/net/wireless/mediatek/mt76/mt7921/pci_mac.c +++ b/drivers/net/wireless/mediatek/mt76/mt7921/pci_mac.c @@ -53,154 +53,6 @@ int mt7921e_tx_prepare_skb(struct mt76_dev *mdev, void *txwi_ptr, return 0; } -static void -mt7921_txwi_free(struct mt7921_dev *dev, struct mt76_txwi_cache *t, - struct ieee80211_sta *sta, bool clear_status, - struct list_head *free_list) -{ - struct mt76_dev *mdev = &dev->mt76; - __le32 *txwi; - u16 wcid_idx; - - mt76_connac_txp_skb_unmap(mdev, t); - if (!t->skb) - goto out; - - txwi = (__le32 *)mt76_get_txwi_ptr(mdev, t); - if (sta) { - struct mt76_wcid *wcid = (struct mt76_wcid *)sta->drv_priv; - - if (likely(t->skb->protocol != cpu_to_be16(ETH_P_PAE))) - mt7921_tx_check_aggr(sta, txwi); - - wcid_idx = wcid->idx; - } else { - wcid_idx = le32_get_bits(txwi[1], MT_TXD1_WLAN_IDX); - } - - __mt76_tx_complete_skb(mdev, wcid_idx, t->skb, free_list); - -out: - t->skb = NULL; - mt76_put_txwi(mdev, t); -} - -static void -mt7921e_mac_tx_free(struct mt7921_dev *dev, void *data, int len) -{ - struct mt76_connac_tx_free *free = data; - __le32 *tx_info = (__le32 *)(data + sizeof(*free)); - struct mt76_dev *mdev = &dev->mt76; - struct mt76_txwi_cache *txwi; - struct ieee80211_sta *sta = NULL; - struct sk_buff *skb, *tmp; - void *end = data + len; - LIST_HEAD(free_list); - bool wake = false; - u8 i, count; - - /* clean DMA queues and unmap buffers first */ - mt76_queue_tx_cleanup(dev, dev->mphy.q_tx[MT_TXQ_PSD], false); - mt76_queue_tx_cleanup(dev, dev->mphy.q_tx[MT_TXQ_BE], false); - - count = le16_get_bits(free->ctrl, MT_TX_FREE_MSDU_CNT); - if (WARN_ON_ONCE((void *)&tx_info[count] > end)) - return; - - for (i = 0; i < count; i++) { - u32 msdu, info = le32_to_cpu(tx_info[i]); - u8 stat; - - /* 1'b1: new wcid pair. - * 1'b0: msdu_id with the same 'wcid pair' as above. - */ - if (info & MT_TX_FREE_PAIR) { - struct mt7921_sta *msta; - struct mt76_wcid *wcid; - u16 idx; - - count++; - idx = FIELD_GET(MT_TX_FREE_WLAN_ID, info); - wcid = rcu_dereference(dev->mt76.wcid[idx]); - sta = wcid_to_sta(wcid); - if (!sta) - continue; - - msta = container_of(wcid, struct mt7921_sta, wcid); - spin_lock_bh(&dev->sta_poll_lock); - if (list_empty(&msta->poll_list)) - list_add_tail(&msta->poll_list, &dev->sta_poll_list); - spin_unlock_bh(&dev->sta_poll_lock); - continue; - } - - msdu = FIELD_GET(MT_TX_FREE_MSDU_ID, info); - stat = FIELD_GET(MT_TX_FREE_STATUS, info); - - txwi = mt76_token_release(mdev, msdu, &wake); - if (!txwi) - continue; - - mt7921_txwi_free(dev, txwi, sta, stat, &free_list); - } - - if (wake) - mt76_set_tx_blocked(&dev->mt76, false); - - list_for_each_entry_safe(skb, tmp, &free_list, list) { - skb_list_del_init(skb); - napi_consume_skb(skb, 1); - } - - rcu_read_lock(); - mt7921_mac_sta_poll(dev); - rcu_read_unlock(); - - mt76_worker_schedule(&dev->mt76.tx_worker); -} - -bool mt7921e_rx_check(struct mt76_dev *mdev, void *data, int len) -{ - struct mt7921_dev *dev = container_of(mdev, struct mt7921_dev, mt76); - __le32 *rxd = (__le32 *)data; - __le32 *end = (__le32 *)&rxd[len / 4]; - enum rx_pkt_type type; - - type = le32_get_bits(rxd[0], MT_RXD0_PKT_TYPE); - - switch (type) { - case PKT_TYPE_TXRX_NOTIFY: - mt7921e_mac_tx_free(dev, data, len); - return false; - case PKT_TYPE_TXS: - for (rxd += 2; rxd + 8 <= end; rxd += 8) - mt7921_mac_add_txs(dev, rxd); - return false; - default: - return true; - } -} - -void mt7921e_queue_rx_skb(struct mt76_dev *mdev, enum mt76_rxq_id q, - struct sk_buff *skb) -{ - struct mt7921_dev *dev = container_of(mdev, struct mt7921_dev, mt76); - __le32 *rxd = (__le32 *)skb->data; - enum rx_pkt_type type; - - type = le32_get_bits(rxd[0], MT_RXD0_PKT_TYPE); - - switch (type) { - case PKT_TYPE_TXRX_NOTIFY: - mt7921e_mac_tx_free(dev, skb->data, skb->len); - napi_consume_skb(skb, 1); - break; - default: - mt7921_queue_rx_skb(mdev, q, skb); - break; - } -} - void mt7921_tx_token_put(struct mt7921_dev *dev) { struct mt76_txwi_cache *txwi; diff --git a/drivers/net/wireless/mediatek/mt76/mt7921/pci_mcu.c b/drivers/net/wireless/mediatek/mt76/mt7921/pci_mcu.c index 5efda694fb9d..86340d3205c5 100644 --- a/drivers/net/wireless/mediatek/mt76/mt7921/pci_mcu.c +++ b/drivers/net/wireless/mediatek/mt76/mt7921/pci_mcu.c @@ -30,12 +30,7 @@ mt7921_mcu_send_message(struct mt76_dev *mdev, struct sk_buff *skb, if (ret) return ret; - if (cmd == MCU_UNI_CMD(HIF_CTRL) || - cmd == MCU_UNI_CMD(SUSPEND) || - cmd == MCU_UNI_CMD(OFFLOAD)) - mdev->mcu.timeout = HZ; - else - mdev->mcu.timeout = 3 * HZ; + mdev->mcu.timeout = 3 * HZ; if (cmd == MCU_CMD(FW_SCATTER)) txq = MT_MCUQ_FWDL; @@ -59,6 +54,8 @@ int mt7921e_mcu_init(struct mt7921_dev *dev) if (err) return err; + mt76_rmw_field(dev, MT_PCIE_MAC_PM, MT_PCIE_MAC_PM_L0S_DIS, 1); + err = mt7921_run_firmware(dev); mt76_queue_tx_cleanup(dev, dev->mt76.q_mcu[MT_MCUQ_FWDL], false); diff --git a/drivers/net/wireless/mediatek/mt76/mt7921/regs.h b/drivers/net/wireless/mediatek/mt76/mt7921/regs.h index ea643260ceb6..c65582acfa55 100644 --- a/drivers/net/wireless/mediatek/mt76/mt7921/regs.h +++ b/drivers/net/wireless/mediatek/mt76/mt7921/regs.h @@ -440,6 +440,8 @@ #define MT_PCIE_MAC_BASE 0x10000 #define MT_PCIE_MAC(ofs) (MT_PCIE_MAC_BASE + (ofs)) #define MT_PCIE_MAC_INT_ENABLE MT_PCIE_MAC(0x188) +#define MT_PCIE_MAC_PM MT_PCIE_MAC(0x194) +#define MT_PCIE_MAC_PM_L0S_DIS BIT(8) #define MT_DMA_SHDL(ofs) (0x7c026000 + (ofs)) #define MT_DMASHDL_SW_CONTROL MT_DMA_SHDL(0x004) diff --git a/drivers/net/wireless/mediatek/mt76/mt7921/sdio.c b/drivers/net/wireless/mediatek/mt76/mt7921/sdio.c index 487acd6e2be8..3b25a06fd946 100644 --- a/drivers/net/wireless/mediatek/mt76/mt7921/sdio.c +++ b/drivers/net/wireless/mediatek/mt76/mt7921/sdio.c @@ -96,6 +96,7 @@ static int mt7921s_probe(struct sdio_func *func, .tx_complete_skb = mt7921_usb_sdio_tx_complete_skb, .tx_status_data = mt7921_usb_sdio_tx_status_data, .rx_skb = mt7921_queue_rx_skb, + .rx_check = mt7921_rx_check, .sta_ps = mt7921_sta_ps, .sta_add = mt7921_mac_sta_add, .sta_assoc = mt7921_mac_sta_assoc, @@ -194,7 +195,6 @@ static void mt7921s_remove(struct sdio_func *func) mt7921s_unregister_device(dev); } -#ifdef CONFIG_PM static int mt7921s_suspend(struct device *__dev) { struct sdio_func *func = dev_to_sdio_func(__dev); @@ -206,6 +206,7 @@ static int mt7921s_suspend(struct device *__dev) pm->suspended = true; set_bit(MT76_STATE_SUSPEND, &mdev->phy.state); + flush_work(&dev->reset_work); cancel_delayed_work_sync(&pm->ps_work); cancel_work_sync(&pm->wake_work); @@ -261,6 +262,9 @@ restore_suspend: clear_bit(MT76_STATE_SUSPEND, &mdev->phy.state); pm->suspended = false; + if (err < 0) + mt7921_reset(&dev->mt76); + return err; } @@ -276,7 +280,7 @@ static int mt7921s_resume(struct device *__dev) err = mt7921_mcu_drv_pmctrl(dev); if (err < 0) - return err; + goto failed; mt76_worker_enable(&mdev->tx_worker); mt76_worker_enable(&mdev->sdio.txrx_worker); @@ -288,34 +292,27 @@ static int mt7921s_resume(struct device *__dev) mt76_connac_mcu_set_deep_sleep(mdev, false); err = mt76_connac_mcu_set_hif_suspend(mdev, false); - if (err) - return err; - +failed: pm->suspended = false; + if (err < 0) + mt7921_reset(&dev->mt76); + return err; } -static const struct dev_pm_ops mt7921s_pm_ops = { - .suspend = mt7921s_suspend, - .resume = mt7921s_resume, -}; -#endif - MODULE_DEVICE_TABLE(sdio, mt7921s_table); MODULE_FIRMWARE(MT7921_FIRMWARE_WM); MODULE_FIRMWARE(MT7921_ROM_PATCH); +static DEFINE_SIMPLE_DEV_PM_OPS(mt7921s_pm_ops, mt7921s_suspend, mt7921s_resume); + static struct sdio_driver mt7921s_driver = { .name = KBUILD_MODNAME, .probe = mt7921s_probe, .remove = mt7921s_remove, .id_table = mt7921s_table, -#ifdef CONFIG_PM - .drv = { - .pm = &mt7921s_pm_ops, - } -#endif + .drv.pm = pm_sleep_ptr(&mt7921s_pm_ops), }; module_sdio_driver(mt7921s_driver); MODULE_AUTHOR("Sean Wang <sean.wang@mediatek.com>"); diff --git a/drivers/net/wireless/mediatek/mt76/mt7921/sdio_mcu.c b/drivers/net/wireless/mediatek/mt76/mt7921/sdio_mcu.c index e038d7404323..5c1489766d9f 100644 --- a/drivers/net/wireless/mediatek/mt76/mt7921/sdio_mcu.c +++ b/drivers/net/wireless/mediatek/mt76/mt7921/sdio_mcu.c @@ -33,12 +33,7 @@ mt7921s_mcu_send_message(struct mt76_dev *mdev, struct sk_buff *skb, if (ret) return ret; - if (cmd == MCU_UNI_CMD(HIF_CTRL) || - cmd == MCU_UNI_CMD(SUSPEND) || - cmd == MCU_UNI_CMD(OFFLOAD)) - mdev->mcu.timeout = HZ; - else - mdev->mcu.timeout = 3 * HZ; + mdev->mcu.timeout = 3 * HZ; if (cmd == MCU_CMD(FW_SCATTER)) type = MT7921_SDIO_FWDL; diff --git a/drivers/net/wireless/mediatek/mt76/mt7921/usb.c b/drivers/net/wireless/mediatek/mt76/mt7921/usb.c index dd3b8884e162..29c0ee330dbe 100644 --- a/drivers/net/wireless/mediatek/mt76/mt7921/usb.c +++ b/drivers/net/wireless/mediatek/mt76/mt7921/usb.c @@ -106,12 +106,7 @@ mt7921u_mcu_send_message(struct mt76_dev *mdev, struct sk_buff *skb, if (ret) return ret; - if (cmd == MCU_UNI_CMD(HIF_CTRL) || - cmd == MCU_UNI_CMD(SUSPEND) || - cmd == MCU_UNI_CMD(OFFLOAD)) - mdev->mcu.timeout = HZ; - else - mdev->mcu.timeout = 3 * HZ; + mdev->mcu.timeout = 3 * HZ; if (cmd != MCU_CMD(FW_SCATTER)) ep = MT_EP_OUT_INBAND_CMD; @@ -183,6 +178,7 @@ static int mt7921u_probe(struct usb_interface *usb_intf, .tx_complete_skb = mt7921_usb_sdio_tx_complete_skb, .tx_status_data = mt7921_usb_sdio_tx_status_data, .rx_skb = mt7921_queue_rx_skb, + .rx_check = mt7921_rx_check, .sta_ps = mt7921_sta_ps, .sta_add = mt7921_mac_sta_add, .sta_assoc = mt7921_mac_sta_assoc, @@ -300,23 +296,34 @@ static void mt7921u_disconnect(struct usb_interface *usb_intf) static int mt7921u_suspend(struct usb_interface *intf, pm_message_t state) { struct mt7921_dev *dev = usb_get_intfdata(intf); + struct mt76_connac_pm *pm = &dev->pm; int err; + pm->suspended = true; + flush_work(&dev->reset_work); + err = mt76_connac_mcu_set_hif_suspend(&dev->mt76, true); if (err) - return err; + goto failed; mt76u_stop_rx(&dev->mt76); mt76u_stop_tx(&dev->mt76); - set_bit(MT76_STATE_SUSPEND, &dev->mphy.state); - return 0; + +failed: + pm->suspended = false; + + if (err < 0) + mt7921_reset(&dev->mt76); + + return err; } static int mt7921u_resume(struct usb_interface *intf) { struct mt7921_dev *dev = usb_get_intfdata(intf); + struct mt76_connac_pm *pm = &dev->pm; bool reinit = true; int err, i; @@ -338,16 +345,21 @@ static int mt7921u_resume(struct usb_interface *intf) if (reinit || mt7921_dma_need_reinit(dev)) { err = mt7921u_dma_init(dev, true); if (err) - return err; + goto failed; } - clear_bit(MT76_STATE_SUSPEND, &dev->mphy.state); - err = mt76u_resume_rx(&dev->mt76); if (err < 0) - return err; + goto failed; + + err = mt76_connac_mcu_set_hif_suspend(&dev->mt76, false); +failed: + pm->suspended = false; + + if (err < 0) + mt7921_reset(&dev->mt76); - return mt76_connac_mcu_set_hif_suspend(&dev->mt76, false); + return err; } #endif /* CONFIG_PM */ diff --git a/drivers/net/wireless/mediatek/mt76/sdio.c b/drivers/net/wireless/mediatek/mt76/sdio.c index aba2a9865821..0ec308f99af5 100644 --- a/drivers/net/wireless/mediatek/mt76/sdio.c +++ b/drivers/net/wireless/mediatek/mt76/sdio.c @@ -478,14 +478,14 @@ static void mt76s_status_worker(struct mt76_worker *w) if (ndata_frames > 0) resched = true; - if (dev->drv->tx_status_data && + if (dev->drv->tx_status_data && ndata_frames > 0 && !test_and_set_bit(MT76_READING_STATS, &dev->phy.state) && !test_bit(MT76_STATE_SUSPEND, &dev->phy.state)) - queue_work(dev->wq, &dev->sdio.stat_work); + ieee80211_queue_work(dev->hw, &dev->sdio.stat_work); } while (nframes > 0); if (resched) - mt76_worker_schedule(&dev->sdio.txrx_worker); + mt76_worker_schedule(&dev->tx_worker); } static void mt76s_tx_status_data(struct work_struct *work) @@ -508,7 +508,7 @@ static void mt76s_tx_status_data(struct work_struct *work) } if (count && test_bit(MT76_STATE_RUNNING, &dev->phy.state)) - queue_work(dev->wq, &sdio->stat_work); + ieee80211_queue_work(dev->hw, &sdio->stat_work); else clear_bit(MT76_READING_STATS, &dev->phy.state); } diff --git a/drivers/net/wireless/mediatek/mt76/sdio_txrx.c b/drivers/net/wireless/mediatek/mt76/sdio_txrx.c index a2601aa9e7b1..bfc4de50a4d2 100644 --- a/drivers/net/wireless/mediatek/mt76/sdio_txrx.c +++ b/drivers/net/wireless/mediatek/mt76/sdio_txrx.c @@ -85,7 +85,7 @@ mt76s_rx_run_queue(struct mt76_dev *dev, enum mt76_rxq_id qid, struct mt76_sdio *sdio = &dev->sdio; int len = 0, err, i; struct page *page; - u8 *buf; + u8 *buf, *end; for (i = 0; i < intr->rx.num[qid]; i++) len += round_up(intr->rx.len[qid][i] + 4, 4); @@ -112,20 +112,29 @@ mt76s_rx_run_queue(struct mt76_dev *dev, enum mt76_rxq_id qid, return err; } - for (i = 0; i < intr->rx.num[qid]; i++) { + end = buf + len; + i = 0; + + while (i < intr->rx.num[qid] && buf < end) { int index = (q->head + i) % q->ndesc; struct mt76_queue_entry *e = &q->entry[index]; __le32 *rxd = (__le32 *)buf; /* parse rxd to get the actual packet length */ len = le32_get_bits(rxd[0], GENMASK(15, 0)); - e->skb = mt76s_build_rx_skb(buf, len, round_up(len + 4, 4)); - if (!e->skb) - break; + /* Optimized path for TXS */ + if (!dev->drv->rx_check || dev->drv->rx_check(dev, buf, len)) { + e->skb = mt76s_build_rx_skb(buf, len, + round_up(len + 4, 4)); + if (!e->skb) + break; + + if (q->queued + i + 1 == q->ndesc) + break; + i++; + } buf += round_up(len + 4, 4); - if (q->queued + i + 1 == q->ndesc) - break; } put_page(page); diff --git a/drivers/net/wireless/mediatek/mt76/testmode.c b/drivers/net/wireless/mediatek/mt76/testmode.c index 71fd3fbfa7d2..0accc71a91c9 100644 --- a/drivers/net/wireless/mediatek/mt76/testmode.c +++ b/drivers/net/wireless/mediatek/mt76/testmode.c @@ -1,5 +1,7 @@ // SPDX-License-Identifier: ISC /* Copyright (C) 2020 Felix Fietkau <nbd@nbd.name> */ + +#include <linux/random.h> #include "mt76.h" const struct nla_policy mt76_tm_policy[NUM_MT76_TM_ATTRS] = { @@ -123,12 +125,14 @@ int mt76_testmode_alloc_skb(struct mt76_phy *phy, u32 len) if (!head) return -ENOMEM; - hdr = __skb_put_zero(head, head_len); + hdr = __skb_put_zero(head, sizeof(*hdr)); hdr->frame_control = cpu_to_le16(fc); memcpy(hdr->addr1, td->addr[0], ETH_ALEN); memcpy(hdr->addr2, td->addr[1], ETH_ALEN); memcpy(hdr->addr3, td->addr[2], ETH_ALEN); skb_set_queue_mapping(head, IEEE80211_AC_BE); + get_random_bytes(__skb_put(head, head_len - sizeof(*hdr)), + head_len - sizeof(*hdr)); info = IEEE80211_SKB_CB(head); info->flags = IEEE80211_TX_CTL_INJECTED | @@ -154,7 +158,7 @@ int mt76_testmode_alloc_skb(struct mt76_phy *phy, u32 len) return -ENOMEM; } - __skb_put_zero(frag, frag_len); + get_random_bytes(__skb_put(frag, frag_len), frag_len); head->len += frag->len; head->data_len += frag->len; diff --git a/drivers/net/wireless/mediatek/mt76/usb.c b/drivers/net/wireless/mediatek/mt76/usb.c index 6b8964c19f50..4c4033bb1bb3 100644 --- a/drivers/net/wireless/mediatek/mt76/usb.c +++ b/drivers/net/wireless/mediatek/mt76/usb.c @@ -528,6 +528,11 @@ mt76u_process_rx_entry(struct mt76_dev *dev, struct urb *urb, head_room = drv_flags & MT_DRV_RX_DMA_HDR ? 0 : MT_DMA_HDR_LEN; data_len = min_t(int, len, data_len - head_room); + + if (len == data_len && + dev->drv->rx_check && !dev->drv->rx_check(dev, data, data_len)) + return 0; + skb = mt76u_build_rx_skb(dev, data, data_len, buf_size); if (!skb) return 0; diff --git a/drivers/net/wireless/microchip/wilc1000/cfg80211.c b/drivers/net/wireless/microchip/wilc1000/cfg80211.c index 3ac373d29d93..b89047965e78 100644 --- a/drivers/net/wireless/microchip/wilc1000/cfg80211.c +++ b/drivers/net/wireless/microchip/wilc1000/cfg80211.c @@ -540,8 +540,9 @@ static int wilc_wfi_cfg_copy_wpa_info(struct wilc_wfi_key *key_info, return 0; } -static int add_key(struct wiphy *wiphy, struct net_device *netdev, u8 key_index, - bool pairwise, const u8 *mac_addr, struct key_params *params) +static int add_key(struct wiphy *wiphy, struct net_device *netdev, int link_id, + u8 key_index, bool pairwise, const u8 *mac_addr, + struct key_params *params) { int ret = 0, keylen = params->key_len; @@ -644,7 +645,7 @@ static int add_key(struct wiphy *wiphy, struct net_device *netdev, u8 key_index, return ret; } -static int del_key(struct wiphy *wiphy, struct net_device *netdev, +static int del_key(struct wiphy *wiphy, struct net_device *netdev, int link_id, u8 key_index, bool pairwise, const u8 *mac_addr) @@ -685,8 +686,9 @@ static int del_key(struct wiphy *wiphy, struct net_device *netdev, return 0; } -static int get_key(struct wiphy *wiphy, struct net_device *netdev, u8 key_index, - bool pairwise, const u8 *mac_addr, void *cookie, +static int get_key(struct wiphy *wiphy, struct net_device *netdev, int link_id, + u8 key_index, bool pairwise, const u8 *mac_addr, + void *cookie, void (*callback)(void *cookie, struct key_params *)) { struct wilc_vif *vif = netdev_priv(netdev); @@ -723,13 +725,14 @@ static int get_key(struct wiphy *wiphy, struct net_device *netdev, u8 key_index, /* wiphy_new_nm() will WARNON if not present */ static int set_default_key(struct wiphy *wiphy, struct net_device *netdev, - u8 key_index, bool unicast, bool multicast) + int link_id, u8 key_index, bool unicast, + bool multicast) { return 0; } static int set_default_mgmt_key(struct wiphy *wiphy, struct net_device *netdev, - u8 key_index) + int link_id, u8 key_index) { struct wilc_vif *vif = netdev_priv(netdev); @@ -994,12 +997,11 @@ bool wilc_wfi_mgmt_frame_rx(struct wilc_vif *vif, u8 *buff, u32 size) { struct wilc *wl = vif->wilc; struct wilc_priv *priv = &vif->priv; - int freq, ret; + int freq; freq = ieee80211_channel_to_frequency(wl->op_ch, NL80211_BAND_2GHZ); - ret = cfg80211_rx_mgmt(&priv->wdev, freq, 0, buff, size, 0); - return ret; + return cfg80211_rx_mgmt(&priv->wdev, freq, 0, buff, size, 0); } void wilc_wfi_p2p_rx(struct wilc_vif *vif, u8 *buff, u32 size) diff --git a/drivers/net/wireless/microchip/wilc1000/mon.c b/drivers/net/wireless/microchip/wilc1000/mon.c index b5a1b65c087c..03b7229a0ff5 100644 --- a/drivers/net/wireless/microchip/wilc1000/mon.c +++ b/drivers/net/wireless/microchip/wilc1000/mon.c @@ -229,7 +229,7 @@ struct net_device *wilc_wfi_init_mon_interface(struct wilc *wl, return NULL; wl->monitor_dev->type = ARPHRD_IEEE80211_RADIOTAP; - strlcpy(wl->monitor_dev->name, name, IFNAMSIZ); + strscpy(wl->monitor_dev->name, name, IFNAMSIZ); wl->monitor_dev->netdev_ops = &wilc_wfi_netdev_ops; wl->monitor_dev->needs_free_netdev = true; diff --git a/drivers/net/wireless/quantenna/qtnfmac/cfg80211.c b/drivers/net/wireless/quantenna/qtnfmac/cfg80211.c index 1593e810b3ca..bfdf03bfa6c5 100644 --- a/drivers/net/wireless/quantenna/qtnfmac/cfg80211.c +++ b/drivers/net/wireless/quantenna/qtnfmac/cfg80211.c @@ -532,8 +532,8 @@ qtnf_dump_station(struct wiphy *wiphy, struct net_device *dev, } static int qtnf_add_key(struct wiphy *wiphy, struct net_device *dev, - u8 key_index, bool pairwise, const u8 *mac_addr, - struct key_params *params) + int link_id, u8 key_index, bool pairwise, + const u8 *mac_addr, struct key_params *params) { struct qtnf_vif *vif = qtnf_netdev_get_priv(dev); int ret; @@ -548,7 +548,8 @@ static int qtnf_add_key(struct wiphy *wiphy, struct net_device *dev, } static int qtnf_del_key(struct wiphy *wiphy, struct net_device *dev, - u8 key_index, bool pairwise, const u8 *mac_addr) + int link_id, u8 key_index, bool pairwise, + const u8 *mac_addr) { struct qtnf_vif *vif = qtnf_netdev_get_priv(dev); int ret; @@ -569,7 +570,8 @@ static int qtnf_del_key(struct wiphy *wiphy, struct net_device *dev, } static int qtnf_set_default_key(struct wiphy *wiphy, struct net_device *dev, - u8 key_index, bool unicast, bool multicast) + int link_id, u8 key_index, bool unicast, + bool multicast) { struct qtnf_vif *vif = qtnf_netdev_get_priv(dev); int ret; @@ -585,7 +587,7 @@ static int qtnf_set_default_key(struct wiphy *wiphy, struct net_device *dev, static int qtnf_set_default_mgmt_key(struct wiphy *wiphy, struct net_device *dev, - u8 key_index) + int link_id, u8 key_index) { struct qtnf_vif *vif = qtnf_netdev_get_priv(dev); int ret; @@ -721,9 +723,8 @@ qtnf_disconnect(struct wiphy *wiphy, struct net_device *dev, return -EFAULT; } - if (vif->wdev.iftype != NL80211_IFTYPE_STATION) { + if (vif->wdev.iftype != NL80211_IFTYPE_STATION) return -EOPNOTSUPP; - } ret = qtnf_cmd_send_disconnect(vif, reason_code); if (ret) @@ -750,7 +751,6 @@ qtnf_dump_survey(struct wiphy *wiphy, struct net_device *dev, struct ieee80211_channel *chan; int ret; - sband = wiphy->bands[NL80211_BAND_2GHZ]; if (sband && idx >= sband->n_channels) { idx -= sband->n_channels; @@ -1223,7 +1223,7 @@ int qtnf_wiphy_register(struct qtnf_hw_info *hw_info, struct qtnf_wmac *mac) mac->macinfo.extended_capabilities_len; } - strlcpy(wiphy->fw_version, hw_info->fw_version, + strscpy(wiphy->fw_version, hw_info->fw_version, sizeof(wiphy->fw_version)); wiphy->hw_version = hw_info->hw_version; diff --git a/drivers/net/wireless/quantenna/qtnfmac/commands.c b/drivers/net/wireless/quantenna/qtnfmac/commands.c index 0fad53693292..b1b73478d89b 100644 --- a/drivers/net/wireless/quantenna/qtnfmac/commands.c +++ b/drivers/net/wireless/quantenna/qtnfmac/commands.c @@ -967,7 +967,7 @@ qtnf_cmd_resp_proc_hw_info(struct qtnf_bus *bus, hwinfo->total_rx_chain, hwinfo->total_tx_chain, hwinfo->fw_ver); - strlcpy(hwinfo->fw_version, bld_label, sizeof(hwinfo->fw_version)); + strscpy(hwinfo->fw_version, bld_label, sizeof(hwinfo->fw_version)); hwinfo->hw_version = hw_ver; return 0; diff --git a/drivers/net/wireless/ralink/rt2x00/rt2800.h b/drivers/net/wireless/ralink/rt2x00/rt2800.h index d758e8874457..de2ee5ffc34e 100644 --- a/drivers/net/wireless/ralink/rt2x00/rt2800.h +++ b/drivers/net/wireless/ralink/rt2x00/rt2800.h @@ -1016,6 +1016,8 @@ */ #define MAC_STATUS_CFG 0x1200 #define MAC_STATUS_CFG_BBP_RF_BUSY FIELD32(0x00000003) +#define MAC_STATUS_CFG_BBP_RF_BUSY_TX FIELD32(0x00000001) +#define MAC_STATUS_CFG_BBP_RF_BUSY_RX FIELD32(0x00000002) /* * PWR_PIN_CFG: @@ -2739,6 +2741,7 @@ enum rt2800_eeprom_word { #define EEPROM_NIC_CONF2_RX_STREAM FIELD16(0x000f) #define EEPROM_NIC_CONF2_TX_STREAM FIELD16(0x00f0) #define EEPROM_NIC_CONF2_CRYSTAL FIELD16(0x0600) +#define EEPROM_NIC_CONF2_EXTERNAL_PA FIELD16(0x8000) /* * EEPROM LNA diff --git a/drivers/net/wireless/ralink/rt2x00/rt2800lib.c b/drivers/net/wireless/ralink/rt2x00/rt2800lib.c index 18102fbe36d6..cbbb1a4849cf 100644 --- a/drivers/net/wireless/ralink/rt2x00/rt2800lib.c +++ b/drivers/net/wireless/ralink/rt2x00/rt2800lib.c @@ -198,6 +198,26 @@ static void rt2800_rfcsr_write_dccal(struct rt2x00_dev *rt2x00dev, rt2800_rfcsr_write_bank(rt2x00dev, 7, reg, value); } +static void rt2800_bbp_dcoc_write(struct rt2x00_dev *rt2x00dev, + const u8 reg, const u8 value) +{ + rt2800_bbp_write(rt2x00dev, 158, reg); + rt2800_bbp_write(rt2x00dev, 159, value); +} + +static u8 rt2800_bbp_dcoc_read(struct rt2x00_dev *rt2x00dev, const u8 reg) +{ + rt2800_bbp_write(rt2x00dev, 158, reg); + return rt2800_bbp_read(rt2x00dev, 159); +} + +static void rt2800_bbp_glrt_write(struct rt2x00_dev *rt2x00dev, + const u8 reg, const u8 value) +{ + rt2800_bbp_write(rt2x00dev, 195, reg); + rt2800_bbp_write(rt2x00dev, 196, value); +} + static u8 rt2800_rfcsr_read(struct rt2x00_dev *rt2x00dev, const unsigned int word) { @@ -2143,6 +2163,48 @@ void rt2800_config_erp(struct rt2x00_dev *rt2x00dev, struct rt2x00lib_erp *erp, } EXPORT_SYMBOL_GPL(rt2800_config_erp); +static int rt2800_wait_bbp_rf_ready(struct rt2x00_dev *rt2x00dev, + const struct rt2x00_field32 mask) +{ + unsigned int i; + u32 reg; + + for (i = 0; i < REGISTER_BUSY_COUNT; i++) { + reg = rt2800_register_read(rt2x00dev, MAC_STATUS_CFG); + if (!rt2x00_get_field32(reg, mask)) + return 0; + + udelay(REGISTER_BUSY_DELAY); + } + + rt2x00_err(rt2x00dev, "BBP/RF register access failed, aborting\n"); + return -EACCES; +} + +static int rt2800_wait_bbp_ready(struct rt2x00_dev *rt2x00dev) +{ + unsigned int i; + u8 value; + + /* + * BBP was enabled after firmware was loaded, + * but we need to reactivate it now. + */ + rt2800_register_write(rt2x00dev, H2M_BBP_AGENT, 0); + rt2800_register_write(rt2x00dev, H2M_MAILBOX_CSR, 0); + msleep(1); + + for (i = 0; i < REGISTER_BUSY_COUNT; i++) { + value = rt2800_bbp_read(rt2x00dev, 0); + if ((value != 0xff) && (value != 0x00)) + return 0; + udelay(REGISTER_BUSY_DELAY); + } + + rt2x00_err(rt2x00dev, "BBP register access failed, aborting\n"); + return -EACCES; +} + static void rt2800_config_3572bt_ant(struct rt2x00_dev *rt2x00dev) { u32 reg; @@ -3793,16 +3855,23 @@ static void rt2800_config_channel_rf7620(struct rt2x00_dev *rt2x00dev, rfcsr |= tx_agc_fc; rt2800_rfcsr_write_bank(rt2x00dev, 7, 59, rfcsr); } + + if (conf_is_ht40(conf)) { + rt2800_bbp_glrt_write(rt2x00dev, 141, 0x10); + rt2800_bbp_glrt_write(rt2x00dev, 157, 0x2f); + } else { + rt2800_bbp_glrt_write(rt2x00dev, 141, 0x1a); + rt2800_bbp_glrt_write(rt2x00dev, 157, 0x40); + } } static void rt2800_config_alc(struct rt2x00_dev *rt2x00dev, struct ieee80211_channel *chan, int power_level) { u16 eeprom, target_power, max_power; - u32 mac_sys_ctrl, mac_status; + u32 mac_sys_ctrl; u32 reg; u8 bbp; - int i; /* hardware unit is 0.5dBm, limited to 23.5dBm */ power_level *= 2; @@ -3838,16 +3907,8 @@ static void rt2800_config_alc(struct rt2x00_dev *rt2x00dev, /* Disable Tx/Rx */ rt2800_register_write(rt2x00dev, MAC_SYS_CTRL, 0); /* Check MAC Tx/Rx idle */ - for (i = 0; i < 10000; i++) { - mac_status = rt2800_register_read(rt2x00dev, MAC_STATUS_CFG); - if (mac_status & 0x3) - usleep_range(50, 200); - else - break; - } - - if (i == 10000) - rt2x00_warn(rt2x00dev, "Wait MAC Status to MAX !!!\n"); + if (unlikely(rt2800_wait_bbp_rf_ready(rt2x00dev, MAC_STATUS_CFG_BBP_RF_BUSY))) + rt2x00_warn(rt2x00dev, "RF busy while configuring ALC\n"); if (chan->center_freq > 2457) { bbp = rt2800_bbp_read(rt2x00dev, 30); @@ -4164,7 +4225,10 @@ static void rt2800_config_channel(struct rt2x00_dev *rt2x00dev, rt2800_bbp_write(rt2x00dev, 62, 0x37 - rt2x00dev->lna_gain); rt2800_bbp_write(rt2x00dev, 63, 0x37 - rt2x00dev->lna_gain); rt2800_bbp_write(rt2x00dev, 64, 0x37 - rt2x00dev->lna_gain); - rt2800_bbp_write(rt2x00dev, 86, 0); + if (rt2x00_rt(rt2x00dev, RT6352)) + rt2800_bbp_write(rt2x00dev, 86, 0x38); + else + rt2800_bbp_write(rt2x00dev, 86, 0); } if (rf->channel <= 14) { @@ -4365,7 +4429,45 @@ static void rt2800_config_channel(struct rt2x00_dev *rt2x00dev, reg = (rf->channel <= 14 ? 0x1c : 0x24) + 2*rt2x00dev->lna_gain; rt2800_bbp_write_with_rx_chain(rt2x00dev, 66, reg); - rt2800_iq_calibrate(rt2x00dev, rf->channel); + if (rt2x00_rt(rt2x00dev, RT5592)) + rt2800_iq_calibrate(rt2x00dev, rf->channel); + } + + if (rt2x00_rt(rt2x00dev, RT6352)) { + if (test_bit(CAPABILITY_EXTERNAL_PA_TX0, + &rt2x00dev->cap_flags)) { + reg = rt2800_register_read(rt2x00dev, RF_CONTROL3); + reg |= 0x00000101; + rt2800_register_write(rt2x00dev, RF_CONTROL3, reg); + + reg = rt2800_register_read(rt2x00dev, RF_BYPASS3); + reg |= 0x00000101; + rt2800_register_write(rt2x00dev, RF_BYPASS3, reg); + + rt2800_rfcsr_write_chanreg(rt2x00dev, 43, 0x73); + rt2800_rfcsr_write_chanreg(rt2x00dev, 44, 0x73); + rt2800_rfcsr_write_chanreg(rt2x00dev, 45, 0x73); + rt2800_rfcsr_write_chanreg(rt2x00dev, 46, 0x27); + rt2800_rfcsr_write_chanreg(rt2x00dev, 47, 0xC8); + rt2800_rfcsr_write_chanreg(rt2x00dev, 48, 0xA4); + rt2800_rfcsr_write_chanreg(rt2x00dev, 49, 0x05); + rt2800_rfcsr_write_chanreg(rt2x00dev, 54, 0x27); + rt2800_rfcsr_write_chanreg(rt2x00dev, 55, 0xC8); + rt2800_rfcsr_write_chanreg(rt2x00dev, 56, 0xA4); + rt2800_rfcsr_write_chanreg(rt2x00dev, 57, 0x05); + rt2800_rfcsr_write_chanreg(rt2x00dev, 58, 0x27); + rt2800_rfcsr_write_chanreg(rt2x00dev, 59, 0xC8); + rt2800_rfcsr_write_chanreg(rt2x00dev, 60, 0xA4); + rt2800_rfcsr_write_chanreg(rt2x00dev, 61, 0x05); + rt2800_rfcsr_write_dccal(rt2x00dev, 05, 0x00); + + rt2800_register_write(rt2x00dev, TX0_RF_GAIN_CORRECT, + 0x36303636); + rt2800_register_write(rt2x00dev, TX0_RF_GAIN_ATTEN, + 0x6C6C6B6C); + rt2800_register_write(rt2x00dev, TX1_RF_GAIN_ATTEN, + 0x6C6C6B6C); + } } bbp = rt2800_bbp_read(rt2x00dev, 4); @@ -5644,7 +5746,8 @@ static inline void rt2800_set_vgc(struct rt2x00_dev *rt2x00dev, if (qual->vgc_level != vgc_level) { if (rt2x00_rt(rt2x00dev, RT3572) || rt2x00_rt(rt2x00dev, RT3593) || - rt2x00_rt(rt2x00dev, RT3883)) { + rt2x00_rt(rt2x00dev, RT3883) || + rt2x00_rt(rt2x00dev, RT6352)) { rt2800_bbp_write_with_rx_chain(rt2x00dev, 66, vgc_level); } else if (rt2x00_rt(rt2x00dev, RT5592)) { @@ -5867,7 +5970,7 @@ static int rt2800_init_registers(struct rt2x00_dev *rt2x00dev) rt2800_register_write(rt2x00dev, TX_SW_CFG0, 0x00000404); } else if (rt2x00_rt(rt2x00dev, RT6352)) { rt2800_register_write(rt2x00dev, TX_SW_CFG0, 0x00000401); - rt2800_register_write(rt2x00dev, TX_SW_CFG1, 0x000C0000); + rt2800_register_write(rt2x00dev, TX_SW_CFG1, 0x000C0001); rt2800_register_write(rt2x00dev, TX_SW_CFG2, 0x00000000); rt2800_register_write(rt2x00dev, TX_ALC_VGA3, 0x00000000); rt2800_register_write(rt2x00dev, TX0_BB_GAIN_ATTEN, 0x0); @@ -6129,6 +6232,27 @@ static int rt2800_init_registers(struct rt2x00_dev *rt2x00dev) reg = rt2800_register_read(rt2x00dev, US_CYC_CNT); rt2x00_set_field32(®, US_CYC_CNT_CLOCK_CYCLE, 125); rt2800_register_write(rt2x00dev, US_CYC_CNT, reg); + } else if (rt2x00_is_soc(rt2x00dev)) { + struct clk *clk = clk_get_sys("bus", NULL); + int rate; + + if (IS_ERR(clk)) { + clk = clk_get_sys("cpu", NULL); + + if (IS_ERR(clk)) { + rate = 125; + } else { + rate = clk_get_rate(clk) / 3000000; + clk_put(clk); + } + } else { + rate = clk_get_rate(clk) / 1000000; + clk_put(clk); + } + + reg = rt2800_register_read(rt2x00dev, US_CYC_CNT); + rt2x00_set_field32(®, US_CYC_CNT_CLOCK_CYCLE, rate); + rt2800_register_write(rt2x00dev, US_CYC_CNT, reg); } reg = rt2800_register_read(rt2x00dev, HT_FBK_CFG0); @@ -6212,46 +6336,6 @@ static int rt2800_init_registers(struct rt2x00_dev *rt2x00dev) return 0; } -static int rt2800_wait_bbp_rf_ready(struct rt2x00_dev *rt2x00dev) -{ - unsigned int i; - u32 reg; - - for (i = 0; i < REGISTER_BUSY_COUNT; i++) { - reg = rt2800_register_read(rt2x00dev, MAC_STATUS_CFG); - if (!rt2x00_get_field32(reg, MAC_STATUS_CFG_BBP_RF_BUSY)) - return 0; - - udelay(REGISTER_BUSY_DELAY); - } - - rt2x00_err(rt2x00dev, "BBP/RF register access failed, aborting\n"); - return -EACCES; -} - -static int rt2800_wait_bbp_ready(struct rt2x00_dev *rt2x00dev) -{ - unsigned int i; - u8 value; - - /* - * BBP was enabled after firmware was loaded, - * but we need to reactivate it now. - */ - rt2800_register_write(rt2x00dev, H2M_BBP_AGENT, 0); - rt2800_register_write(rt2x00dev, H2M_MAILBOX_CSR, 0); - msleep(1); - - for (i = 0; i < REGISTER_BUSY_COUNT; i++) { - value = rt2800_bbp_read(rt2x00dev, 0); - if ((value != 0xff) && (value != 0x00)) - return 0; - udelay(REGISTER_BUSY_DELAY); - } - - rt2x00_err(rt2x00dev, "BBP register access failed, aborting\n"); - return -EACCES; -} static void rt2800_bbp4_mac_if_ctrl(struct rt2x00_dev *rt2x00dev) { @@ -6916,26 +7000,6 @@ static void rt2800_init_bbp_5592(struct rt2x00_dev *rt2x00dev) rt2800_bbp_write(rt2x00dev, 103, 0xc0); } -static void rt2800_bbp_glrt_write(struct rt2x00_dev *rt2x00dev, - const u8 reg, const u8 value) -{ - rt2800_bbp_write(rt2x00dev, 195, reg); - rt2800_bbp_write(rt2x00dev, 196, value); -} - -static void rt2800_bbp_dcoc_write(struct rt2x00_dev *rt2x00dev, - const u8 reg, const u8 value) -{ - rt2800_bbp_write(rt2x00dev, 158, reg); - rt2800_bbp_write(rt2x00dev, 159, value); -} - -static u8 rt2800_bbp_dcoc_read(struct rt2x00_dev *rt2x00dev, const u8 reg) -{ - rt2800_bbp_write(rt2x00dev, 158, reg); - return rt2800_bbp_read(rt2x00dev, 159); -} - static void rt2800_init_bbp_6352(struct rt2x00_dev *rt2x00dev) { u8 bbp; @@ -8398,6 +8462,1519 @@ static void rt2800_init_rfcsr_5592(struct rt2x00_dev *rt2x00dev) rt2800_led_open_drain_enable(rt2x00dev); } +static void rt2800_rf_self_txdc_cal(struct rt2x00_dev *rt2x00dev) +{ + u8 rfb5r1_org, rfb7r1_org, rfvalue; + u32 mac0518, mac051c, mac0528, mac052c; + u8 i; + + mac0518 = rt2800_register_read(rt2x00dev, RF_CONTROL0); + mac051c = rt2800_register_read(rt2x00dev, RF_BYPASS0); + mac0528 = rt2800_register_read(rt2x00dev, RF_CONTROL2); + mac052c = rt2800_register_read(rt2x00dev, RF_BYPASS2); + + rt2800_register_write(rt2x00dev, RF_BYPASS0, 0x0); + rt2800_register_write(rt2x00dev, RF_BYPASS2, 0x0); + + rt2800_register_write(rt2x00dev, RF_CONTROL0, 0xC); + rt2800_register_write(rt2x00dev, RF_BYPASS0, 0x3306); + rt2800_register_write(rt2x00dev, RF_CONTROL2, 0x3330); + rt2800_register_write(rt2x00dev, RF_BYPASS2, 0xfffff); + rfb5r1_org = rt2800_rfcsr_read_bank(rt2x00dev, 5, 1); + rfb7r1_org = rt2800_rfcsr_read_bank(rt2x00dev, 7, 1); + + rt2800_rfcsr_write_bank(rt2x00dev, 5, 1, 0x4); + for (i = 0; i < 100; ++i) { + usleep_range(50, 100); + rfvalue = rt2800_rfcsr_read_bank(rt2x00dev, 5, 1); + if ((rfvalue & 0x04) != 0x4) + break; + } + rt2800_rfcsr_write_bank(rt2x00dev, 5, 1, rfb5r1_org); + + rt2800_rfcsr_write_bank(rt2x00dev, 7, 1, 0x4); + for (i = 0; i < 100; ++i) { + usleep_range(50, 100); + rfvalue = rt2800_rfcsr_read_bank(rt2x00dev, 7, 1); + if ((rfvalue & 0x04) != 0x4) + break; + } + rt2800_rfcsr_write_bank(rt2x00dev, 7, 1, rfb7r1_org); + + rt2800_register_write(rt2x00dev, RF_BYPASS0, 0x0); + rt2800_register_write(rt2x00dev, RF_BYPASS2, 0x0); + rt2800_register_write(rt2x00dev, RF_CONTROL0, mac0518); + rt2800_register_write(rt2x00dev, RF_BYPASS0, mac051c); + rt2800_register_write(rt2x00dev, RF_CONTROL2, mac0528); + rt2800_register_write(rt2x00dev, RF_BYPASS2, mac052c); +} + +static int rt2800_calcrcalibrationcode(struct rt2x00_dev *rt2x00dev, int d1, int d2) +{ + int calcode = ((d2 - d1) * 1000) / 43; + + if ((calcode % 10) >= 5) + calcode += 10; + calcode = (calcode / 10); + + return calcode; +} + +static void rt2800_r_calibration(struct rt2x00_dev *rt2x00dev) +{ + u32 savemacsysctrl; + u8 saverfb0r1, saverfb0r34, saverfb0r35; + u8 saverfb5r4, saverfb5r17, saverfb5r18; + u8 saverfb5r19, saverfb5r20; + u8 savebbpr22, savebbpr47, savebbpr49; + u8 bytevalue = 0; + int rcalcode; + u8 r_cal_code = 0; + char d1 = 0, d2 = 0; + u8 rfvalue; + u32 MAC_RF_BYPASS0, MAC_RF_CONTROL0, MAC_PWR_PIN_CFG; + u32 maccfg; + + saverfb0r1 = rt2800_rfcsr_read_bank(rt2x00dev, 0, 1); + saverfb0r34 = rt2800_rfcsr_read_bank(rt2x00dev, 0, 34); + saverfb0r35 = rt2800_rfcsr_read_bank(rt2x00dev, 0, 35); + saverfb5r4 = rt2800_rfcsr_read_bank(rt2x00dev, 5, 4); + saverfb5r17 = rt2800_rfcsr_read_bank(rt2x00dev, 5, 17); + saverfb5r18 = rt2800_rfcsr_read_bank(rt2x00dev, 5, 18); + saverfb5r19 = rt2800_rfcsr_read_bank(rt2x00dev, 5, 19); + saverfb5r20 = rt2800_rfcsr_read_bank(rt2x00dev, 5, 20); + + savebbpr22 = rt2800_bbp_read(rt2x00dev, 22); + savebbpr47 = rt2800_bbp_read(rt2x00dev, 47); + savebbpr49 = rt2800_bbp_read(rt2x00dev, 49); + + savemacsysctrl = rt2800_register_read(rt2x00dev, MAC_SYS_CTRL); + MAC_RF_BYPASS0 = rt2800_register_read(rt2x00dev, RF_BYPASS0); + MAC_RF_CONTROL0 = rt2800_register_read(rt2x00dev, RF_CONTROL0); + MAC_PWR_PIN_CFG = rt2800_register_read(rt2x00dev, PWR_PIN_CFG); + + maccfg = rt2800_register_read(rt2x00dev, MAC_SYS_CTRL); + maccfg &= (~0x04); + rt2800_register_write(rt2x00dev, MAC_SYS_CTRL, maccfg); + + if (unlikely(rt2800_wait_bbp_rf_ready(rt2x00dev, MAC_STATUS_CFG_BBP_RF_BUSY_TX))) + rt2x00_warn(rt2x00dev, "Wait MAC Tx Status to MAX !!!\n"); + + maccfg = rt2800_register_read(rt2x00dev, MAC_SYS_CTRL); + maccfg &= (~0x04); + rt2800_register_write(rt2x00dev, MAC_SYS_CTRL, maccfg); + + if (unlikely(rt2800_wait_bbp_rf_ready(rt2x00dev, MAC_STATUS_CFG_BBP_RF_BUSY_RX))) + rt2x00_warn(rt2x00dev, "Wait MAC Rx Status to MAX !!!\n"); + + rfvalue = (MAC_RF_BYPASS0 | 0x3004); + rt2800_register_write(rt2x00dev, RF_BYPASS0, rfvalue); + rfvalue = (MAC_RF_CONTROL0 | (~0x3002)); + rt2800_register_write(rt2x00dev, RF_CONTROL0, rfvalue); + + rt2800_rfcsr_write_bank(rt2x00dev, 5, 4, 0x27); + rt2800_rfcsr_write_bank(rt2x00dev, 5, 17, 0x80); + rt2800_rfcsr_write_bank(rt2x00dev, 5, 18, 0x83); + rt2800_rfcsr_write_bank(rt2x00dev, 5, 19, 0x00); + rt2800_rfcsr_write_bank(rt2x00dev, 5, 20, 0x20); + + rt2800_rfcsr_write_bank(rt2x00dev, 0, 1, 0x00); + rt2800_rfcsr_write_bank(rt2x00dev, 0, 34, 0x13); + rt2800_rfcsr_write_bank(rt2x00dev, 0, 35, 0x00); + + rt2800_register_write(rt2x00dev, PWR_PIN_CFG, 0x1); + + rt2800_bbp_write(rt2x00dev, 47, 0x04); + rt2800_bbp_write(rt2x00dev, 22, 0x80); + usleep_range(100, 200); + bytevalue = rt2800_bbp_read(rt2x00dev, 49); + if (bytevalue > 128) + d1 = bytevalue - 256; + else + d1 = (char)bytevalue; + rt2800_bbp_write(rt2x00dev, 22, 0x0); + rt2800_rfcsr_write_bank(rt2x00dev, 0, 35, 0x01); + + rt2800_bbp_write(rt2x00dev, 22, 0x80); + usleep_range(100, 200); + bytevalue = rt2800_bbp_read(rt2x00dev, 49); + if (bytevalue > 128) + d2 = bytevalue - 256; + else + d2 = (char)bytevalue; + rt2800_bbp_write(rt2x00dev, 22, 0x0); + + rcalcode = rt2800_calcrcalibrationcode(rt2x00dev, d1, d2); + if (rcalcode < 0) + r_cal_code = 256 + rcalcode; + else + r_cal_code = (u8)rcalcode; + + rt2800_rfcsr_write_bank(rt2x00dev, 0, 7, r_cal_code); + + rt2800_bbp_write(rt2x00dev, 22, 0x0); + + bytevalue = rt2800_bbp_read(rt2x00dev, 21); + bytevalue |= 0x1; + rt2800_bbp_write(rt2x00dev, 21, bytevalue); + bytevalue = rt2800_bbp_read(rt2x00dev, 21); + bytevalue &= (~0x1); + rt2800_bbp_write(rt2x00dev, 21, bytevalue); + + rt2800_rfcsr_write_bank(rt2x00dev, 0, 1, saverfb0r1); + rt2800_rfcsr_write_bank(rt2x00dev, 0, 34, saverfb0r34); + rt2800_rfcsr_write_bank(rt2x00dev, 0, 35, saverfb0r35); + rt2800_rfcsr_write_bank(rt2x00dev, 5, 4, saverfb5r4); + rt2800_rfcsr_write_bank(rt2x00dev, 5, 17, saverfb5r17); + rt2800_rfcsr_write_bank(rt2x00dev, 5, 18, saverfb5r18); + rt2800_rfcsr_write_bank(rt2x00dev, 5, 19, saverfb5r19); + rt2800_rfcsr_write_bank(rt2x00dev, 5, 20, saverfb5r20); + + rt2800_bbp_write(rt2x00dev, 22, savebbpr22); + rt2800_bbp_write(rt2x00dev, 47, savebbpr47); + rt2800_bbp_write(rt2x00dev, 49, savebbpr49); + + rt2800_register_write(rt2x00dev, RF_BYPASS0, MAC_RF_BYPASS0); + rt2800_register_write(rt2x00dev, RF_CONTROL0, MAC_RF_CONTROL0); + + rt2800_register_write(rt2x00dev, MAC_SYS_CTRL, savemacsysctrl); + rt2800_register_write(rt2x00dev, PWR_PIN_CFG, MAC_PWR_PIN_CFG); +} + +static void rt2800_rxdcoc_calibration(struct rt2x00_dev *rt2x00dev) +{ + u8 bbpreg = 0; + u32 macvalue = 0; + u8 saverfb0r2, saverfb5r4, saverfb7r4, rfvalue; + int i; + + saverfb0r2 = rt2800_rfcsr_read_bank(rt2x00dev, 0, 2); + rfvalue = saverfb0r2; + rfvalue |= 0x03; + rt2800_rfcsr_write_bank(rt2x00dev, 0, 2, rfvalue); + + rt2800_bbp_write(rt2x00dev, 158, 141); + bbpreg = rt2800_bbp_read(rt2x00dev, 159); + bbpreg |= 0x10; + rt2800_bbp_write(rt2x00dev, 159, bbpreg); + + macvalue = rt2800_register_read(rt2x00dev, MAC_SYS_CTRL); + rt2800_register_write(rt2x00dev, MAC_SYS_CTRL, 0x8); + + if (unlikely(rt2800_wait_bbp_rf_ready(rt2x00dev, MAC_STATUS_CFG_BBP_RF_BUSY_TX))) + rt2x00_warn(rt2x00dev, "RF TX busy in RX RXDCOC calibration\n"); + + saverfb5r4 = rt2800_rfcsr_read_bank(rt2x00dev, 5, 4); + saverfb7r4 = rt2800_rfcsr_read_bank(rt2x00dev, 7, 4); + saverfb5r4 = saverfb5r4 & (~0x40); + saverfb7r4 = saverfb7r4 & (~0x40); + rt2800_rfcsr_write_dccal(rt2x00dev, 4, 0x64); + rt2800_rfcsr_write_bank(rt2x00dev, 5, 4, saverfb5r4); + rt2800_rfcsr_write_bank(rt2x00dev, 7, 4, saverfb7r4); + + rt2800_bbp_write(rt2x00dev, 158, 141); + bbpreg = rt2800_bbp_read(rt2x00dev, 159); + bbpreg = bbpreg & (~0x40); + rt2800_bbp_write(rt2x00dev, 159, bbpreg); + bbpreg |= 0x48; + rt2800_bbp_write(rt2x00dev, 159, bbpreg); + + for (i = 0; i < 10000; i++) { + bbpreg = rt2800_bbp_read(rt2x00dev, 159); + if ((bbpreg & 0x40) == 0) + break; + usleep_range(50, 100); + } + + bbpreg = rt2800_bbp_read(rt2x00dev, 159); + bbpreg = bbpreg & (~0x40); + rt2800_bbp_write(rt2x00dev, 159, bbpreg); + + rt2800_register_write(rt2x00dev, MAC_SYS_CTRL, macvalue); + + rt2800_bbp_write(rt2x00dev, 158, 141); + bbpreg = rt2800_bbp_read(rt2x00dev, 159); + bbpreg &= (~0x10); + rt2800_bbp_write(rt2x00dev, 159, bbpreg); + + rt2800_rfcsr_write_bank(rt2x00dev, 0, 2, saverfb0r2); +} + +static u32 rt2800_do_sqrt_accumulation(u32 si) +{ + u32 root, root_pre, bit; + char i; + + bit = 1 << 15; + root = 0; + for (i = 15; i >= 0; i = i - 1) { + root_pre = root + bit; + if ((root_pre * root_pre) <= si) + root = root_pre; + bit = bit >> 1; + } + + return root; +} + +static void rt2800_rxiq_calibration(struct rt2x00_dev *rt2x00dev) +{ + u8 rfb0r1, rfb0r2, rfb0r42; + u8 rfb4r0, rfb4r19; + u8 rfb5r3, rfb5r4, rfb5r17, rfb5r18, rfb5r19, rfb5r20; + u8 rfb6r0, rfb6r19; + u8 rfb7r3, rfb7r4, rfb7r17, rfb7r18, rfb7r19, rfb7r20; + + u8 bbp1, bbp4; + u8 bbpr241, bbpr242; + u32 i; + u8 ch_idx; + u8 bbpval; + u8 rfval, vga_idx = 0; + int mi = 0, mq = 0, si = 0, sq = 0, riq = 0; + int sigma_i, sigma_q, r_iq, g_rx; + int g_imb; + int ph_rx; + u32 savemacsysctrl = 0; + u32 orig_RF_CONTROL0 = 0; + u32 orig_RF_BYPASS0 = 0; + u32 orig_RF_CONTROL1 = 0; + u32 orig_RF_BYPASS1 = 0; + u32 orig_RF_CONTROL3 = 0; + u32 orig_RF_BYPASS3 = 0; + u32 bbpval1 = 0; + static const u8 rf_vga_table[] = {0x20, 0x21, 0x22, 0x38, 0x39, 0x3a, 0x3b, 0x3c, 0x3d, 0x3e, 0x3f}; + + savemacsysctrl = rt2800_register_read(rt2x00dev, MAC_SYS_CTRL); + orig_RF_CONTROL0 = rt2800_register_read(rt2x00dev, RF_CONTROL0); + orig_RF_BYPASS0 = rt2800_register_read(rt2x00dev, RF_BYPASS0); + orig_RF_CONTROL1 = rt2800_register_read(rt2x00dev, RF_CONTROL1); + orig_RF_BYPASS1 = rt2800_register_read(rt2x00dev, RF_BYPASS1); + orig_RF_CONTROL3 = rt2800_register_read(rt2x00dev, RF_CONTROL3); + orig_RF_BYPASS3 = rt2800_register_read(rt2x00dev, RF_BYPASS3); + + bbp1 = rt2800_bbp_read(rt2x00dev, 1); + bbp4 = rt2800_bbp_read(rt2x00dev, 4); + + rt2800_register_write(rt2x00dev, MAC_SYS_CTRL, 0x0); + + if (unlikely(rt2800_wait_bbp_rf_ready(rt2x00dev, MAC_STATUS_CFG_BBP_RF_BUSY))) + rt2x00_warn(rt2x00dev, "Timeout waiting for MAC status in RXIQ calibration\n"); + + bbpval = bbp4 & (~0x18); + bbpval = bbp4 | 0x00; + rt2800_bbp_write(rt2x00dev, 4, bbpval); + + bbpval = rt2800_bbp_read(rt2x00dev, 21); + bbpval = bbpval | 1; + rt2800_bbp_write(rt2x00dev, 21, bbpval); + bbpval = bbpval & 0xfe; + rt2800_bbp_write(rt2x00dev, 21, bbpval); + + rt2800_register_write(rt2x00dev, RF_CONTROL1, 0x00000202); + rt2800_register_write(rt2x00dev, RF_BYPASS1, 0x00000303); + if (test_bit(CAPABILITY_EXTERNAL_PA_TX0, &rt2x00dev->cap_flags)) + rt2800_register_write(rt2x00dev, RF_CONTROL3, 0x0101); + else + rt2800_register_write(rt2x00dev, RF_CONTROL3, 0x0000); + + rt2800_register_write(rt2x00dev, RF_BYPASS3, 0xf1f1); + + rfb0r1 = rt2800_rfcsr_read_bank(rt2x00dev, 0, 1); + rfb0r2 = rt2800_rfcsr_read_bank(rt2x00dev, 0, 2); + rfb0r42 = rt2800_rfcsr_read_bank(rt2x00dev, 0, 42); + rfb4r0 = rt2800_rfcsr_read_bank(rt2x00dev, 4, 0); + rfb4r19 = rt2800_rfcsr_read_bank(rt2x00dev, 4, 19); + rfb5r3 = rt2800_rfcsr_read_bank(rt2x00dev, 5, 3); + rfb5r4 = rt2800_rfcsr_read_bank(rt2x00dev, 5, 4); + rfb5r17 = rt2800_rfcsr_read_bank(rt2x00dev, 5, 17); + rfb5r18 = rt2800_rfcsr_read_bank(rt2x00dev, 5, 18); + rfb5r19 = rt2800_rfcsr_read_bank(rt2x00dev, 5, 19); + rfb5r20 = rt2800_rfcsr_read_bank(rt2x00dev, 5, 20); + + rfb6r0 = rt2800_rfcsr_read_bank(rt2x00dev, 6, 0); + rfb6r19 = rt2800_rfcsr_read_bank(rt2x00dev, 6, 19); + rfb7r3 = rt2800_rfcsr_read_bank(rt2x00dev, 7, 3); + rfb7r4 = rt2800_rfcsr_read_bank(rt2x00dev, 7, 4); + rfb7r17 = rt2800_rfcsr_read_bank(rt2x00dev, 7, 17); + rfb7r18 = rt2800_rfcsr_read_bank(rt2x00dev, 7, 18); + rfb7r19 = rt2800_rfcsr_read_bank(rt2x00dev, 7, 19); + rfb7r20 = rt2800_rfcsr_read_bank(rt2x00dev, 7, 20); + + rt2800_rfcsr_write_chanreg(rt2x00dev, 0, 0x87); + rt2800_rfcsr_write_chanreg(rt2x00dev, 19, 0x27); + rt2800_rfcsr_write_dccal(rt2x00dev, 3, 0x38); + rt2800_rfcsr_write_dccal(rt2x00dev, 4, 0x38); + rt2800_rfcsr_write_dccal(rt2x00dev, 17, 0x80); + rt2800_rfcsr_write_dccal(rt2x00dev, 18, 0xC1); + rt2800_rfcsr_write_dccal(rt2x00dev, 19, 0x60); + rt2800_rfcsr_write_dccal(rt2x00dev, 20, 0x00); + + rt2800_bbp_write(rt2x00dev, 23, 0x0); + rt2800_bbp_write(rt2x00dev, 24, 0x0); + + rt2800_bbp_dcoc_write(rt2x00dev, 5, 0x0); + + bbpr241 = rt2800_bbp_read(rt2x00dev, 241); + bbpr242 = rt2800_bbp_read(rt2x00dev, 242); + + rt2800_bbp_write(rt2x00dev, 241, 0x10); + rt2800_bbp_write(rt2x00dev, 242, 0x84); + rt2800_bbp_write(rt2x00dev, 244, 0x31); + + bbpval = rt2800_bbp_dcoc_read(rt2x00dev, 3); + bbpval = bbpval & (~0x7); + rt2800_bbp_dcoc_write(rt2x00dev, 3, bbpval); + + rt2800_register_write(rt2x00dev, RF_CONTROL0, 0x00000004); + udelay(1); + rt2800_register_write(rt2x00dev, RF_CONTROL0, 0x00000006); + usleep_range(1, 200); + rt2800_register_write(rt2x00dev, RF_BYPASS0, 0x00003376); + rt2800_register_write(rt2x00dev, RF_CONTROL0, 0x00001006); + udelay(1); + if (test_bit(CAPABILITY_EXTERNAL_PA_TX0, &rt2x00dev->cap_flags)) { + rt2800_bbp_write(rt2x00dev, 23, 0x06); + rt2800_bbp_write(rt2x00dev, 24, 0x06); + } else { + rt2800_bbp_write(rt2x00dev, 23, 0x02); + rt2800_bbp_write(rt2x00dev, 24, 0x02); + } + + for (ch_idx = 0; ch_idx < 2; ch_idx = ch_idx + 1) { + if (ch_idx == 0) { + rfval = rfb0r1 & (~0x3); + rfval = rfb0r1 | 0x1; + rt2800_rfcsr_write_bank(rt2x00dev, 0, 1, rfval); + rfval = rfb0r2 & (~0x33); + rfval = rfb0r2 | 0x11; + rt2800_rfcsr_write_bank(rt2x00dev, 0, 2, rfval); + rfval = rfb0r42 & (~0x50); + rfval = rfb0r42 | 0x10; + rt2800_rfcsr_write_bank(rt2x00dev, 0, 42, rfval); + + rt2800_register_write(rt2x00dev, RF_CONTROL0, 0x00001006); + udelay(1); + + bbpval = bbp1 & (~0x18); + bbpval = bbpval | 0x00; + rt2800_bbp_write(rt2x00dev, 1, bbpval); + + rt2800_bbp_dcoc_write(rt2x00dev, 1, 0x00); + } else { + rfval = rfb0r1 & (~0x3); + rfval = rfb0r1 | 0x2; + rt2800_rfcsr_write_bank(rt2x00dev, 0, 1, rfval); + rfval = rfb0r2 & (~0x33); + rfval = rfb0r2 | 0x22; + rt2800_rfcsr_write_bank(rt2x00dev, 0, 2, rfval); + rfval = rfb0r42 & (~0x50); + rfval = rfb0r42 | 0x40; + rt2800_rfcsr_write_bank(rt2x00dev, 0, 42, rfval); + + rt2800_register_write(rt2x00dev, RF_CONTROL0, 0x00002006); + udelay(1); + + bbpval = bbp1 & (~0x18); + bbpval = bbpval | 0x08; + rt2800_bbp_write(rt2x00dev, 1, bbpval); + + rt2800_bbp_dcoc_write(rt2x00dev, 1, 0x01); + } + usleep_range(500, 1500); + + vga_idx = 0; + while (vga_idx < 11) { + rt2800_rfcsr_write_dccal(rt2x00dev, 3, rf_vga_table[vga_idx]); + rt2800_rfcsr_write_dccal(rt2x00dev, 4, rf_vga_table[vga_idx]); + + rt2800_bbp_dcoc_write(rt2x00dev, 0, 0x93); + + for (i = 0; i < 10000; i++) { + bbpval = rt2800_bbp_read(rt2x00dev, 159); + if ((bbpval & 0xff) == 0x93) + usleep_range(50, 100); + else + break; + } + + if ((bbpval & 0xff) == 0x93) { + rt2x00_warn(rt2x00dev, "Fatal Error: Calibration doesn't finish"); + goto restore_value; + } + for (i = 0; i < 5; i++) { + u32 bbptemp = 0; + u8 value = 0; + int result = 0; + + rt2800_bbp_write(rt2x00dev, 158, 0x1e); + rt2800_bbp_write(rt2x00dev, 159, i); + rt2800_bbp_write(rt2x00dev, 158, 0x22); + value = rt2800_bbp_read(rt2x00dev, 159); + bbptemp = bbptemp + (value << 24); + rt2800_bbp_write(rt2x00dev, 158, 0x21); + value = rt2800_bbp_read(rt2x00dev, 159); + bbptemp = bbptemp + (value << 16); + rt2800_bbp_write(rt2x00dev, 158, 0x20); + value = rt2800_bbp_read(rt2x00dev, 159); + bbptemp = bbptemp + (value << 8); + rt2800_bbp_write(rt2x00dev, 158, 0x1f); + value = rt2800_bbp_read(rt2x00dev, 159); + bbptemp = bbptemp + value; + + if (i < 2 && (bbptemp & 0x800000)) + result = (bbptemp & 0xffffff) - 0x1000000; + else if (i == 4) + result = bbptemp; + else + result = bbptemp; + + if (i == 0) + mi = result / 4096; + else if (i == 1) + mq = result / 4096; + else if (i == 2) + si = bbptemp / 4096; + else if (i == 3) + sq = bbptemp / 4096; + else + riq = result / 4096; + } + + bbpval1 = si - mi * mi; + rt2x00_dbg(rt2x00dev, + "RXIQ si=%d, sq=%d, riq=%d, bbpval %d, vga_idx %d", + si, sq, riq, bbpval1, vga_idx); + + if (bbpval1 >= (100 * 100)) + break; + + if (bbpval1 <= 100) + vga_idx = vga_idx + 9; + else if (bbpval1 <= 158) + vga_idx = vga_idx + 8; + else if (bbpval1 <= 251) + vga_idx = vga_idx + 7; + else if (bbpval1 <= 398) + vga_idx = vga_idx + 6; + else if (bbpval1 <= 630) + vga_idx = vga_idx + 5; + else if (bbpval1 <= 1000) + vga_idx = vga_idx + 4; + else if (bbpval1 <= 1584) + vga_idx = vga_idx + 3; + else if (bbpval1 <= 2511) + vga_idx = vga_idx + 2; + else + vga_idx = vga_idx + 1; + } + + sigma_i = rt2800_do_sqrt_accumulation(100 * (si - mi * mi)); + sigma_q = rt2800_do_sqrt_accumulation(100 * (sq - mq * mq)); + r_iq = 10 * (riq - (mi * mq)); + + rt2x00_dbg(rt2x00dev, "Sigma_i=%d, Sigma_q=%d, R_iq=%d", sigma_i, sigma_q, r_iq); + + if (sigma_i <= 1400 && sigma_i >= 1000 && + (sigma_i - sigma_q) <= 112 && + (sigma_i - sigma_q) >= -112 && + mi <= 32 && mi >= -32 && + mq <= 32 && mq >= -32) { + r_iq = 10 * (riq - (mi * mq)); + rt2x00_dbg(rt2x00dev, "RXIQ Sigma_i=%d, Sigma_q=%d, R_iq=%d\n", + sigma_i, sigma_q, r_iq); + + g_rx = (1000 * sigma_q) / sigma_i; + g_imb = ((-2) * 128 * (1000 - g_rx)) / (1000 + g_rx); + ph_rx = (r_iq * 2292) / (sigma_i * sigma_q); + + if (ph_rx > 20 || ph_rx < -20) { + ph_rx = 0; + rt2x00_warn(rt2x00dev, "RXIQ calibration FAIL"); + } + + if (g_imb > 12 || g_imb < -12) { + g_imb = 0; + rt2x00_warn(rt2x00dev, "RXIQ calibration FAIL"); + } + } else { + g_imb = 0; + ph_rx = 0; + rt2x00_dbg(rt2x00dev, "RXIQ Sigma_i=%d, Sigma_q=%d, R_iq=%d\n", + sigma_i, sigma_q, r_iq); + rt2x00_warn(rt2x00dev, "RXIQ calibration FAIL"); + } + + if (ch_idx == 0) { + rt2800_bbp_write(rt2x00dev, 158, 0x37); + rt2800_bbp_write(rt2x00dev, 159, g_imb & 0x3f); + rt2800_bbp_write(rt2x00dev, 158, 0x35); + rt2800_bbp_write(rt2x00dev, 159, ph_rx & 0x3f); + } else { + rt2800_bbp_write(rt2x00dev, 158, 0x55); + rt2800_bbp_write(rt2x00dev, 159, g_imb & 0x3f); + rt2800_bbp_write(rt2x00dev, 158, 0x53); + rt2800_bbp_write(rt2x00dev, 159, ph_rx & 0x3f); + } + } + +restore_value: + rt2800_bbp_write(rt2x00dev, 158, 0x3); + bbpval = rt2800_bbp_read(rt2x00dev, 159); + rt2800_bbp_write(rt2x00dev, 159, (bbpval | 0x07)); + + rt2800_bbp_write(rt2x00dev, 158, 0x00); + rt2800_bbp_write(rt2x00dev, 159, 0x00); + rt2800_bbp_write(rt2x00dev, 1, bbp1); + rt2800_bbp_write(rt2x00dev, 4, bbp4); + rt2800_bbp_write(rt2x00dev, 241, bbpr241); + rt2800_bbp_write(rt2x00dev, 242, bbpr242); + + rt2800_bbp_write(rt2x00dev, 244, 0x00); + bbpval = rt2800_bbp_read(rt2x00dev, 21); + bbpval |= 0x1; + rt2800_bbp_write(rt2x00dev, 21, bbpval); + usleep_range(10, 200); + bbpval &= 0xfe; + rt2800_bbp_write(rt2x00dev, 21, bbpval); + + rt2800_rfcsr_write_bank(rt2x00dev, 0, 1, rfb0r1); + rt2800_rfcsr_write_bank(rt2x00dev, 0, 2, rfb0r2); + rt2800_rfcsr_write_bank(rt2x00dev, 0, 42, rfb0r42); + + rt2800_rfcsr_write_bank(rt2x00dev, 4, 0, rfb4r0); + rt2800_rfcsr_write_bank(rt2x00dev, 4, 19, rfb4r19); + rt2800_rfcsr_write_bank(rt2x00dev, 5, 3, rfb5r3); + rt2800_rfcsr_write_bank(rt2x00dev, 5, 4, rfb5r4); + rt2800_rfcsr_write_bank(rt2x00dev, 5, 17, rfb5r17); + rt2800_rfcsr_write_bank(rt2x00dev, 5, 18, rfb5r18); + rt2800_rfcsr_write_bank(rt2x00dev, 5, 19, rfb5r19); + rt2800_rfcsr_write_bank(rt2x00dev, 5, 20, rfb5r20); + + rt2800_rfcsr_write_bank(rt2x00dev, 6, 0, rfb6r0); + rt2800_rfcsr_write_bank(rt2x00dev, 6, 19, rfb6r19); + rt2800_rfcsr_write_bank(rt2x00dev, 7, 3, rfb7r3); + rt2800_rfcsr_write_bank(rt2x00dev, 7, 4, rfb7r4); + rt2800_rfcsr_write_bank(rt2x00dev, 7, 17, rfb7r17); + rt2800_rfcsr_write_bank(rt2x00dev, 7, 18, rfb7r18); + rt2800_rfcsr_write_bank(rt2x00dev, 7, 19, rfb7r19); + rt2800_rfcsr_write_bank(rt2x00dev, 7, 20, rfb7r20); + + rt2800_register_write(rt2x00dev, RF_CONTROL0, 0x00000006); + udelay(1); + rt2800_register_write(rt2x00dev, RF_CONTROL0, 0x00000004); + udelay(1); + rt2800_register_write(rt2x00dev, RF_CONTROL0, orig_RF_CONTROL0); + udelay(1); + rt2800_register_write(rt2x00dev, RF_BYPASS0, orig_RF_BYPASS0); + rt2800_register_write(rt2x00dev, RF_CONTROL1, orig_RF_CONTROL1); + rt2800_register_write(rt2x00dev, RF_BYPASS1, orig_RF_BYPASS1); + rt2800_register_write(rt2x00dev, RF_CONTROL3, orig_RF_CONTROL3); + rt2800_register_write(rt2x00dev, RF_BYPASS3, orig_RF_BYPASS3); + rt2800_register_write(rt2x00dev, MAC_SYS_CTRL, savemacsysctrl); +} + +static void rt2800_rf_configstore(struct rt2x00_dev *rt2x00dev, + struct rf_reg_pair rf_reg_record[][13], u8 chain) +{ + u8 rfvalue = 0; + + if (chain == CHAIN_0) { + rfvalue = rt2800_rfcsr_read_bank(rt2x00dev, 0, 1); + rf_reg_record[CHAIN_0][0].bank = 0; + rf_reg_record[CHAIN_0][0].reg = 1; + rf_reg_record[CHAIN_0][0].value = rfvalue; + rfvalue = rt2800_rfcsr_read_bank(rt2x00dev, 0, 2); + rf_reg_record[CHAIN_0][1].bank = 0; + rf_reg_record[CHAIN_0][1].reg = 2; + rf_reg_record[CHAIN_0][1].value = rfvalue; + rfvalue = rt2800_rfcsr_read_bank(rt2x00dev, 0, 35); + rf_reg_record[CHAIN_0][2].bank = 0; + rf_reg_record[CHAIN_0][2].reg = 35; + rf_reg_record[CHAIN_0][2].value = rfvalue; + rfvalue = rt2800_rfcsr_read_bank(rt2x00dev, 0, 42); + rf_reg_record[CHAIN_0][3].bank = 0; + rf_reg_record[CHAIN_0][3].reg = 42; + rf_reg_record[CHAIN_0][3].value = rfvalue; + rfvalue = rt2800_rfcsr_read_bank(rt2x00dev, 4, 0); + rf_reg_record[CHAIN_0][4].bank = 4; + rf_reg_record[CHAIN_0][4].reg = 0; + rf_reg_record[CHAIN_0][4].value = rfvalue; + rfvalue = rt2800_rfcsr_read_bank(rt2x00dev, 4, 2); + rf_reg_record[CHAIN_0][5].bank = 4; + rf_reg_record[CHAIN_0][5].reg = 2; + rf_reg_record[CHAIN_0][5].value = rfvalue; + rfvalue = rt2800_rfcsr_read_bank(rt2x00dev, 4, 34); + rf_reg_record[CHAIN_0][6].bank = 4; + rf_reg_record[CHAIN_0][6].reg = 34; + rf_reg_record[CHAIN_0][6].value = rfvalue; + rfvalue = rt2800_rfcsr_read_bank(rt2x00dev, 5, 3); + rf_reg_record[CHAIN_0][7].bank = 5; + rf_reg_record[CHAIN_0][7].reg = 3; + rf_reg_record[CHAIN_0][7].value = rfvalue; + rfvalue = rt2800_rfcsr_read_bank(rt2x00dev, 5, 4); + rf_reg_record[CHAIN_0][8].bank = 5; + rf_reg_record[CHAIN_0][8].reg = 4; + rf_reg_record[CHAIN_0][8].value = rfvalue; + rfvalue = rt2800_rfcsr_read_bank(rt2x00dev, 5, 17); + rf_reg_record[CHAIN_0][9].bank = 5; + rf_reg_record[CHAIN_0][9].reg = 17; + rf_reg_record[CHAIN_0][9].value = rfvalue; + rfvalue = rt2800_rfcsr_read_bank(rt2x00dev, 5, 18); + rf_reg_record[CHAIN_0][10].bank = 5; + rf_reg_record[CHAIN_0][10].reg = 18; + rf_reg_record[CHAIN_0][10].value = rfvalue; + rfvalue = rt2800_rfcsr_read_bank(rt2x00dev, 5, 19); + rf_reg_record[CHAIN_0][11].bank = 5; + rf_reg_record[CHAIN_0][11].reg = 19; + rf_reg_record[CHAIN_0][11].value = rfvalue; + rfvalue = rt2800_rfcsr_read_bank(rt2x00dev, 5, 20); + rf_reg_record[CHAIN_0][12].bank = 5; + rf_reg_record[CHAIN_0][12].reg = 20; + rf_reg_record[CHAIN_0][12].value = rfvalue; + } else if (chain == CHAIN_1) { + rfvalue = rt2800_rfcsr_read_bank(rt2x00dev, 0, 1); + rf_reg_record[CHAIN_1][0].bank = 0; + rf_reg_record[CHAIN_1][0].reg = 1; + rf_reg_record[CHAIN_1][0].value = rfvalue; + rfvalue = rt2800_rfcsr_read_bank(rt2x00dev, 0, 2); + rf_reg_record[CHAIN_1][1].bank = 0; + rf_reg_record[CHAIN_1][1].reg = 2; + rf_reg_record[CHAIN_1][1].value = rfvalue; + rfvalue = rt2800_rfcsr_read_bank(rt2x00dev, 0, 35); + rf_reg_record[CHAIN_1][2].bank = 0; + rf_reg_record[CHAIN_1][2].reg = 35; + rf_reg_record[CHAIN_1][2].value = rfvalue; + rfvalue = rt2800_rfcsr_read_bank(rt2x00dev, 0, 42); + rf_reg_record[CHAIN_1][3].bank = 0; + rf_reg_record[CHAIN_1][3].reg = 42; + rf_reg_record[CHAIN_1][3].value = rfvalue; + rfvalue = rt2800_rfcsr_read_bank(rt2x00dev, 6, 0); + rf_reg_record[CHAIN_1][4].bank = 6; + rf_reg_record[CHAIN_1][4].reg = 0; + rf_reg_record[CHAIN_1][4].value = rfvalue; + rfvalue = rt2800_rfcsr_read_bank(rt2x00dev, 6, 2); + rf_reg_record[CHAIN_1][5].bank = 6; + rf_reg_record[CHAIN_1][5].reg = 2; + rf_reg_record[CHAIN_1][5].value = rfvalue; + rfvalue = rt2800_rfcsr_read_bank(rt2x00dev, 6, 34); + rf_reg_record[CHAIN_1][6].bank = 6; + rf_reg_record[CHAIN_1][6].reg = 34; + rf_reg_record[CHAIN_1][6].value = rfvalue; + rfvalue = rt2800_rfcsr_read_bank(rt2x00dev, 7, 3); + rf_reg_record[CHAIN_1][7].bank = 7; + rf_reg_record[CHAIN_1][7].reg = 3; + rf_reg_record[CHAIN_1][7].value = rfvalue; + rfvalue = rt2800_rfcsr_read_bank(rt2x00dev, 7, 4); + rf_reg_record[CHAIN_1][8].bank = 7; + rf_reg_record[CHAIN_1][8].reg = 4; + rf_reg_record[CHAIN_1][8].value = rfvalue; + rfvalue = rt2800_rfcsr_read_bank(rt2x00dev, 7, 17); + rf_reg_record[CHAIN_1][9].bank = 7; + rf_reg_record[CHAIN_1][9].reg = 17; + rf_reg_record[CHAIN_1][9].value = rfvalue; + rfvalue = rt2800_rfcsr_read_bank(rt2x00dev, 7, 18); + rf_reg_record[CHAIN_1][10].bank = 7; + rf_reg_record[CHAIN_1][10].reg = 18; + rf_reg_record[CHAIN_1][10].value = rfvalue; + rfvalue = rt2800_rfcsr_read_bank(rt2x00dev, 7, 19); + rf_reg_record[CHAIN_1][11].bank = 7; + rf_reg_record[CHAIN_1][11].reg = 19; + rf_reg_record[CHAIN_1][11].value = rfvalue; + rfvalue = rt2800_rfcsr_read_bank(rt2x00dev, 7, 20); + rf_reg_record[CHAIN_1][12].bank = 7; + rf_reg_record[CHAIN_1][12].reg = 20; + rf_reg_record[CHAIN_1][12].value = rfvalue; + } else { + rt2x00_warn(rt2x00dev, "Unknown chain = %u\n", chain); + } +} + +static void rt2800_rf_configrecover(struct rt2x00_dev *rt2x00dev, + struct rf_reg_pair rf_record[][13]) +{ + u8 chain_index = 0, record_index = 0; + u8 bank = 0, rf_register = 0, value = 0; + + for (chain_index = 0; chain_index < 2; chain_index++) { + for (record_index = 0; record_index < 13; record_index++) { + bank = rf_record[chain_index][record_index].bank; + rf_register = rf_record[chain_index][record_index].reg; + value = rf_record[chain_index][record_index].value; + rt2800_rfcsr_write_bank(rt2x00dev, bank, rf_register, value); + rt2x00_dbg(rt2x00dev, "bank: %d, rf_register: %d, value: %x\n", + bank, rf_register, value); + } + } +} + +static void rt2800_setbbptonegenerator(struct rt2x00_dev *rt2x00dev) +{ + rt2800_bbp_write(rt2x00dev, 158, 0xAA); + rt2800_bbp_write(rt2x00dev, 159, 0x00); + + rt2800_bbp_write(rt2x00dev, 158, 0xAB); + rt2800_bbp_write(rt2x00dev, 159, 0x0A); + + rt2800_bbp_write(rt2x00dev, 158, 0xAC); + rt2800_bbp_write(rt2x00dev, 159, 0x3F); + + rt2800_bbp_write(rt2x00dev, 158, 0xAD); + rt2800_bbp_write(rt2x00dev, 159, 0x3F); + + rt2800_bbp_write(rt2x00dev, 244, 0x40); +} + +static u32 rt2800_do_fft_accumulation(struct rt2x00_dev *rt2x00dev, u8 tidx, u8 read_neg) +{ + u32 macvalue = 0; + int fftout_i = 0, fftout_q = 0; + u32 ptmp = 0, pint = 0; + u8 bbp = 0; + u8 tidxi; + + rt2800_bbp_write(rt2x00dev, 158, 0x00); + rt2800_bbp_write(rt2x00dev, 159, 0x9b); + + bbp = 0x9b; + + while (bbp == 0x9b) { + usleep_range(10, 50); + bbp = rt2800_bbp_read(rt2x00dev, 159); + bbp = bbp & 0xff; + } + + rt2800_bbp_write(rt2x00dev, 158, 0xba); + rt2800_bbp_write(rt2x00dev, 159, tidx); + rt2800_bbp_write(rt2x00dev, 159, tidx); + rt2800_bbp_write(rt2x00dev, 159, tidx); + + macvalue = rt2800_register_read(rt2x00dev, 0x057C); + + fftout_i = (macvalue >> 16); + fftout_i = (fftout_i & 0x8000) ? (fftout_i - 0x10000) : fftout_i; + fftout_q = (macvalue & 0xffff); + fftout_q = (fftout_q & 0x8000) ? (fftout_q - 0x10000) : fftout_q; + ptmp = (fftout_i * fftout_i); + ptmp = ptmp + (fftout_q * fftout_q); + pint = ptmp; + rt2x00_dbg(rt2x00dev, "I = %d, Q = %d, power = %x\n", fftout_i, fftout_q, pint); + if (read_neg) { + pint = pint >> 1; + tidxi = 0x40 - tidx; + tidxi = tidxi & 0x3f; + + rt2800_bbp_write(rt2x00dev, 158, 0xba); + rt2800_bbp_write(rt2x00dev, 159, tidxi); + rt2800_bbp_write(rt2x00dev, 159, tidxi); + rt2800_bbp_write(rt2x00dev, 159, tidxi); + + macvalue = rt2800_register_read(rt2x00dev, 0x057C); + + fftout_i = (macvalue >> 16); + fftout_i = (fftout_i & 0x8000) ? (fftout_i - 0x10000) : fftout_i; + fftout_q = (macvalue & 0xffff); + fftout_q = (fftout_q & 0x8000) ? (fftout_q - 0x10000) : fftout_q; + ptmp = (fftout_i * fftout_i); + ptmp = ptmp + (fftout_q * fftout_q); + ptmp = ptmp >> 1; + pint = pint + ptmp; + } + + return pint; +} + +static u32 rt2800_read_fft_accumulation(struct rt2x00_dev *rt2x00dev, u8 tidx) +{ + u32 macvalue = 0; + int fftout_i = 0, fftout_q = 0; + u32 ptmp = 0, pint = 0; + + rt2800_bbp_write(rt2x00dev, 158, 0xBA); + rt2800_bbp_write(rt2x00dev, 159, tidx); + rt2800_bbp_write(rt2x00dev, 159, tidx); + rt2800_bbp_write(rt2x00dev, 159, tidx); + + macvalue = rt2800_register_read(rt2x00dev, 0x057C); + + fftout_i = (macvalue >> 16); + fftout_i = (fftout_i & 0x8000) ? (fftout_i - 0x10000) : fftout_i; + fftout_q = (macvalue & 0xffff); + fftout_q = (fftout_q & 0x8000) ? (fftout_q - 0x10000) : fftout_q; + ptmp = (fftout_i * fftout_i); + ptmp = ptmp + (fftout_q * fftout_q); + pint = ptmp; + + return pint; +} + +static void rt2800_write_dc(struct rt2x00_dev *rt2x00dev, u8 ch_idx, u8 alc, u8 iorq, u8 dc) +{ + u8 bbp = 0; + + rt2800_bbp_write(rt2x00dev, 158, 0xb0); + bbp = alc | 0x80; + rt2800_bbp_write(rt2x00dev, 159, bbp); + + if (ch_idx == 0) + bbp = (iorq == 0) ? 0xb1 : 0xb2; + else + bbp = (iorq == 0) ? 0xb8 : 0xb9; + + rt2800_bbp_write(rt2x00dev, 158, bbp); + bbp = dc; + rt2800_bbp_write(rt2x00dev, 159, bbp); +} + +static void rt2800_loft_search(struct rt2x00_dev *rt2x00dev, u8 ch_idx, + u8 alc_idx, u8 dc_result[][RF_ALC_NUM][2]) +{ + u32 p0 = 0, p1 = 0, pf = 0; + char idx0 = 0, idx1 = 0; + u8 idxf[] = {0x00, 0x00}; + u8 ibit = 0x20; + u8 iorq; + char bidx; + + rt2800_bbp_write(rt2x00dev, 158, 0xb0); + rt2800_bbp_write(rt2x00dev, 159, 0x80); + + for (bidx = 5; bidx >= 0; bidx--) { + for (iorq = 0; iorq <= 1; iorq++) { + if (idxf[iorq] == 0x20) { + idx0 = 0x20; + p0 = pf; + } else { + idx0 = idxf[iorq] - ibit; + idx0 = idx0 & 0x3F; + rt2800_write_dc(rt2x00dev, ch_idx, 0, iorq, idx0); + p0 = rt2800_do_fft_accumulation(rt2x00dev, 0x0A, 0); + } + + idx1 = idxf[iorq] + (bidx == 5 ? 0 : ibit); + idx1 = idx1 & 0x3F; + rt2800_write_dc(rt2x00dev, ch_idx, 0, iorq, idx1); + p1 = rt2800_do_fft_accumulation(rt2x00dev, 0x0A, 0); + + rt2x00_dbg(rt2x00dev, "alc=%u, IorQ=%u, idx_final=%2x\n", + alc_idx, iorq, idxf[iorq]); + rt2x00_dbg(rt2x00dev, "p0=%x, p1=%x, pf=%x, idx_0=%x, idx_1=%x, ibit=%x\n", + p0, p1, pf, idx0, idx1, ibit); + + if (bidx != 5 && pf <= p0 && pf < p1) { + idxf[iorq] = idxf[iorq]; + } else if (p0 < p1) { + pf = p0; + idxf[iorq] = idx0 & 0x3F; + } else { + pf = p1; + idxf[iorq] = idx1 & 0x3F; + } + rt2x00_dbg(rt2x00dev, "IorQ=%u, idx_final[%u]:%x, pf:%8x\n", + iorq, iorq, idxf[iorq], pf); + + rt2800_write_dc(rt2x00dev, ch_idx, 0, iorq, idxf[iorq]); + } + ibit = ibit >> 1; + } + dc_result[ch_idx][alc_idx][0] = idxf[0]; + dc_result[ch_idx][alc_idx][1] = idxf[1]; +} + +static void rt2800_iq_search(struct rt2x00_dev *rt2x00dev, u8 ch_idx, u8 *ges, u8 *pes) +{ + u32 p0 = 0, p1 = 0, pf = 0; + char perr = 0, gerr = 0, iq_err = 0; + char pef = 0, gef = 0; + char psta, pend; + char gsta, gend; + + u8 ibit = 0x20; + u8 first_search = 0x00, touch_neg_max = 0x00; + char idx0 = 0, idx1 = 0; + u8 gop; + u8 bbp = 0; + char bidx; + + for (bidx = 5; bidx >= 1; bidx--) { + for (gop = 0; gop < 2; gop++) { + if (gop == 1 || bidx < 4) { + if (gop == 0) + iq_err = gerr; + else + iq_err = perr; + + first_search = (gop == 0) ? (bidx == 3) : (bidx == 5); + touch_neg_max = (gop) ? ((iq_err & 0x0F) == 0x08) : + ((iq_err & 0x3F) == 0x20); + + if (touch_neg_max) { + p0 = pf; + idx0 = iq_err; + } else { + idx0 = iq_err - ibit; + bbp = (ch_idx == 0) ? ((gop == 0) ? 0x28 : 0x29) : + ((gop == 0) ? 0x46 : 0x47); + + rt2800_bbp_write(rt2x00dev, 158, bbp); + rt2800_bbp_write(rt2x00dev, 159, idx0); + + p0 = rt2800_do_fft_accumulation(rt2x00dev, 0x14, 1); + } + + idx1 = iq_err + (first_search ? 0 : ibit); + idx1 = (gop == 0) ? (idx1 & 0x0F) : (idx1 & 0x3F); + + bbp = (ch_idx == 0) ? (gop == 0) ? 0x28 : 0x29 : + (gop == 0) ? 0x46 : 0x47; + + rt2800_bbp_write(rt2x00dev, 158, bbp); + rt2800_bbp_write(rt2x00dev, 159, idx1); + + p1 = rt2800_do_fft_accumulation(rt2x00dev, 0x14, 1); + + rt2x00_dbg(rt2x00dev, + "p0=%x, p1=%x, pwer_final=%x, idx0=%x, idx1=%x, iq_err=%x, gop=%d, ibit=%x\n", + p0, p1, pf, idx0, idx1, iq_err, gop, ibit); + + if (!(!first_search && pf <= p0 && pf < p1)) { + if (p0 < p1) { + pf = p0; + iq_err = idx0; + } else { + pf = p1; + iq_err = idx1; + } + } + + bbp = (ch_idx == 0) ? (gop == 0) ? 0x28 : 0x29 : + (gop == 0) ? 0x46 : 0x47; + + rt2800_bbp_write(rt2x00dev, 158, bbp); + rt2800_bbp_write(rt2x00dev, 159, iq_err); + + if (gop == 0) + gerr = iq_err; + else + perr = iq_err; + + rt2x00_dbg(rt2x00dev, "IQCalibration pf=%8x (%2x, %2x) !\n", + pf, gerr & 0x0F, perr & 0x3F); + } + } + + if (bidx > 0) + ibit = (ibit >> 1); + } + gerr = (gerr & 0x08) ? (gerr & 0x0F) - 0x10 : (gerr & 0x0F); + perr = (perr & 0x20) ? (perr & 0x3F) - 0x40 : (perr & 0x3F); + + gerr = (gerr < -0x07) ? -0x07 : (gerr > 0x05) ? 0x05 : gerr; + gsta = gerr - 1; + gend = gerr + 2; + + perr = (perr < -0x1f) ? -0x1f : (perr > 0x1d) ? 0x1d : perr; + psta = perr - 1; + pend = perr + 2; + + for (gef = gsta; gef <= gend; gef = gef + 1) + for (pef = psta; pef <= pend; pef = pef + 1) { + bbp = (ch_idx == 0) ? 0x28 : 0x46; + rt2800_bbp_write(rt2x00dev, 158, bbp); + rt2800_bbp_write(rt2x00dev, 159, gef & 0x0F); + + bbp = (ch_idx == 0) ? 0x29 : 0x47; + rt2800_bbp_write(rt2x00dev, 158, bbp); + rt2800_bbp_write(rt2x00dev, 159, pef & 0x3F); + + p1 = rt2800_do_fft_accumulation(rt2x00dev, 0x14, 1); + if (gef == gsta && pef == psta) { + pf = p1; + gerr = gef; + perr = pef; + } else if (pf > p1) { + pf = p1; + gerr = gef; + perr = pef; + } + rt2x00_dbg(rt2x00dev, "Fine IQCalibration p1=%8x pf=%8x (%2x, %2x) !\n", + p1, pf, gef & 0x0F, pef & 0x3F); + } + + ges[ch_idx] = gerr & 0x0F; + pes[ch_idx] = perr & 0x3F; +} + +static void rt2800_rf_aux_tx0_loopback(struct rt2x00_dev *rt2x00dev) +{ + rt2800_rfcsr_write_bank(rt2x00dev, 0, 1, 0x21); + rt2800_rfcsr_write_bank(rt2x00dev, 0, 2, 0x10); + rt2800_rfcsr_write_bank(rt2x00dev, 0, 35, 0x00); + rt2800_rfcsr_write_bank(rt2x00dev, 0, 42, 0x1b); + rt2800_rfcsr_write_bank(rt2x00dev, 4, 0, 0x81); + rt2800_rfcsr_write_bank(rt2x00dev, 4, 2, 0x81); + rt2800_rfcsr_write_bank(rt2x00dev, 4, 34, 0xee); + rt2800_rfcsr_write_bank(rt2x00dev, 5, 3, 0x2d); + rt2800_rfcsr_write_bank(rt2x00dev, 5, 4, 0x2d); + rt2800_rfcsr_write_bank(rt2x00dev, 5, 17, 0x80); + rt2800_rfcsr_write_bank(rt2x00dev, 5, 18, 0xd7); + rt2800_rfcsr_write_bank(rt2x00dev, 5, 19, 0xa2); + rt2800_rfcsr_write_bank(rt2x00dev, 5, 20, 0x20); +} + +static void rt2800_rf_aux_tx1_loopback(struct rt2x00_dev *rt2x00dev) +{ + rt2800_rfcsr_write_bank(rt2x00dev, 0, 1, 0x22); + rt2800_rfcsr_write_bank(rt2x00dev, 0, 2, 0x20); + rt2800_rfcsr_write_bank(rt2x00dev, 0, 35, 0x00); + rt2800_rfcsr_write_bank(rt2x00dev, 0, 42, 0x4b); + rt2800_rfcsr_write_bank(rt2x00dev, 6, 0, 0x81); + rt2800_rfcsr_write_bank(rt2x00dev, 6, 2, 0x81); + rt2800_rfcsr_write_bank(rt2x00dev, 6, 34, 0xee); + rt2800_rfcsr_write_bank(rt2x00dev, 7, 3, 0x2d); + rt2800_rfcsr_write_bank(rt2x00dev, 7, 4, 0x2d); + rt2800_rfcsr_write_bank(rt2x00dev, 7, 17, 0x80); + rt2800_rfcsr_write_bank(rt2x00dev, 7, 18, 0xd7); + rt2800_rfcsr_write_bank(rt2x00dev, 7, 19, 0xa2); + rt2800_rfcsr_write_bank(rt2x00dev, 7, 20, 0x20); +} + +static void rt2800_loft_iq_calibration(struct rt2x00_dev *rt2x00dev) +{ + struct rf_reg_pair rf_store[CHAIN_NUM][13]; + u32 macorg1 = 0; + u32 macorg2 = 0; + u32 macorg3 = 0; + u32 macorg4 = 0; + u32 macorg5 = 0; + u32 orig528 = 0; + u32 orig52c = 0; + + u32 savemacsysctrl = 0; + u32 macvalue = 0; + u32 mac13b8 = 0; + u32 p0 = 0, p1 = 0; + u32 p0_idx10 = 0, p1_idx10 = 0; + + u8 rfvalue; + u8 loft_dc_search_result[CHAIN_NUM][RF_ALC_NUM][2]; + u8 ger[CHAIN_NUM], per[CHAIN_NUM]; + + u8 vga_gain[] = {14, 14}; + u8 bbp = 0, ch_idx = 0, rf_alc_idx = 0, idx = 0; + u8 bbpr30, rfb0r39, rfb0r42; + u8 bbpr1; + u8 bbpr4; + u8 bbpr241, bbpr242; + u8 count_step; + + static const u8 rf_gain[] = {0x00, 0x01, 0x02, 0x04, 0x08, 0x0c}; + static const u8 rfvga_gain_table[] = {0x24, 0x25, 0x26, 0x27, 0x28, 0x2c, 0x2d, 0x2e, 0x2f, 0x30, + 0x31, 0x38, 0x39, 0x3a, 0x3b, 0x3c, 0x3d, 0x3e, 0x3F}; + static const u8 bbp_2324gain[] = {0x16, 0x14, 0x12, 0x10, 0x0c, 0x08}; + + savemacsysctrl = rt2800_register_read(rt2x00dev, MAC_SYS_CTRL); + macorg1 = rt2800_register_read(rt2x00dev, TX_PIN_CFG); + macorg2 = rt2800_register_read(rt2x00dev, RF_CONTROL0); + macorg3 = rt2800_register_read(rt2x00dev, RF_BYPASS0); + macorg4 = rt2800_register_read(rt2x00dev, RF_CONTROL3); + macorg5 = rt2800_register_read(rt2x00dev, RF_BYPASS3); + mac13b8 = rt2800_register_read(rt2x00dev, 0x13b8); + orig528 = rt2800_register_read(rt2x00dev, RF_CONTROL2); + orig52c = rt2800_register_read(rt2x00dev, RF_BYPASS2); + + macvalue = rt2800_register_read(rt2x00dev, MAC_SYS_CTRL); + macvalue &= (~0x04); + rt2800_register_write(rt2x00dev, MAC_SYS_CTRL, macvalue); + + if (unlikely(rt2800_wait_bbp_rf_ready(rt2x00dev, MAC_STATUS_CFG_BBP_RF_BUSY_TX))) + rt2x00_warn(rt2x00dev, "RF TX busy in LOFT IQ calibration\n"); + + macvalue = rt2800_register_read(rt2x00dev, MAC_SYS_CTRL); + macvalue &= (~0x08); + rt2800_register_write(rt2x00dev, MAC_SYS_CTRL, macvalue); + + if (unlikely(rt2800_wait_bbp_rf_ready(rt2x00dev, MAC_STATUS_CFG_BBP_RF_BUSY_RX))) + rt2x00_warn(rt2x00dev, "RF RX busy in LOFT IQ calibration\n"); + + for (ch_idx = 0; ch_idx < 2; ch_idx++) + rt2800_rf_configstore(rt2x00dev, rf_store, ch_idx); + + bbpr30 = rt2800_bbp_read(rt2x00dev, 30); + rfb0r39 = rt2800_rfcsr_read_bank(rt2x00dev, 0, 39); + rfb0r42 = rt2800_rfcsr_read_bank(rt2x00dev, 0, 42); + + rt2800_bbp_write(rt2x00dev, 30, 0x1F); + rt2800_rfcsr_write_bank(rt2x00dev, 0, 39, 0x80); + rt2800_rfcsr_write_bank(rt2x00dev, 0, 42, 0x5B); + + rt2800_bbp_write(rt2x00dev, 23, 0x00); + rt2800_bbp_write(rt2x00dev, 24, 0x00); + + rt2800_setbbptonegenerator(rt2x00dev); + + for (ch_idx = 0; ch_idx < 2; ch_idx++) { + rt2800_bbp_write(rt2x00dev, 23, 0x00); + rt2800_bbp_write(rt2x00dev, 24, 0x00); + rt2800_register_write(rt2x00dev, MAC_SYS_CTRL, 0x00); + rt2800_register_write(rt2x00dev, TX_PIN_CFG, 0x0000000F); + rt2800_register_write(rt2x00dev, RF_CONTROL0, 0x00000004); + rt2800_register_write(rt2x00dev, RF_BYPASS0, 0x00003306); + rt2800_register_write(rt2x00dev, 0x13b8, 0x10); + udelay(1); + + if (ch_idx == 0) + rt2800_rf_aux_tx0_loopback(rt2x00dev); + else + rt2800_rf_aux_tx1_loopback(rt2x00dev); + + udelay(1); + + if (ch_idx == 0) + rt2800_register_write(rt2x00dev, RF_CONTROL0, 0x00001004); + else + rt2800_register_write(rt2x00dev, RF_CONTROL0, 0x00002004); + + rt2800_bbp_write(rt2x00dev, 158, 0x05); + rt2800_bbp_write(rt2x00dev, 159, 0x00); + + rt2800_bbp_write(rt2x00dev, 158, 0x01); + if (ch_idx == 0) + rt2800_bbp_write(rt2x00dev, 159, 0x00); + else + rt2800_bbp_write(rt2x00dev, 159, 0x01); + + vga_gain[ch_idx] = 18; + for (rf_alc_idx = 0; rf_alc_idx < 3; rf_alc_idx++) { + rt2800_bbp_write(rt2x00dev, 23, bbp_2324gain[rf_alc_idx]); + rt2800_bbp_write(rt2x00dev, 24, bbp_2324gain[rf_alc_idx]); + + macvalue = rt2800_register_read(rt2x00dev, RF_CONTROL3); + macvalue &= (~0x0000F1F1); + macvalue |= (rf_gain[rf_alc_idx] << 4); + macvalue |= (rf_gain[rf_alc_idx] << 12); + rt2800_register_write(rt2x00dev, RF_CONTROL3, macvalue); + macvalue = (0x0000F1F1); + rt2800_register_write(rt2x00dev, RF_BYPASS3, macvalue); + + if (rf_alc_idx == 0) { + rt2800_write_dc(rt2x00dev, ch_idx, 0, 1, 0x21); + for (; vga_gain[ch_idx] > 0; + vga_gain[ch_idx] = vga_gain[ch_idx] - 2) { + rfvalue = rfvga_gain_table[vga_gain[ch_idx]]; + rt2800_rfcsr_write_dccal(rt2x00dev, 3, rfvalue); + rt2800_rfcsr_write_dccal(rt2x00dev, 4, rfvalue); + rt2800_write_dc(rt2x00dev, ch_idx, 0, 1, 0x00); + rt2800_write_dc(rt2x00dev, ch_idx, 0, 0, 0x00); + p0 = rt2800_do_fft_accumulation(rt2x00dev, 0x0A, 0); + rt2800_write_dc(rt2x00dev, ch_idx, 0, 0, 0x21); + p1 = rt2800_do_fft_accumulation(rt2x00dev, 0x0A, 0); + rt2x00_dbg(rt2x00dev, "LOFT AGC %d %d\n", p0, p1); + if ((p0 < 7000 * 7000) && (p1 < (7000 * 7000))) + break; + } + + rt2800_write_dc(rt2x00dev, ch_idx, 0, 0, 0x00); + rt2800_write_dc(rt2x00dev, ch_idx, 0, 1, 0x00); + + rt2x00_dbg(rt2x00dev, "Used VGA %d %x\n", vga_gain[ch_idx], + rfvga_gain_table[vga_gain[ch_idx]]); + + if (vga_gain[ch_idx] < 0) + vga_gain[ch_idx] = 0; + } + + rfvalue = rfvga_gain_table[vga_gain[ch_idx]]; + + rt2800_rfcsr_write_dccal(rt2x00dev, 3, rfvalue); + rt2800_rfcsr_write_dccal(rt2x00dev, 4, rfvalue); + + rt2800_loft_search(rt2x00dev, ch_idx, rf_alc_idx, loft_dc_search_result); + } + } + + for (rf_alc_idx = 0; rf_alc_idx < 3; rf_alc_idx++) { + for (idx = 0; idx < 4; idx++) { + rt2800_bbp_write(rt2x00dev, 158, 0xB0); + bbp = (idx << 2) + rf_alc_idx; + rt2800_bbp_write(rt2x00dev, 159, bbp); + rt2x00_dbg(rt2x00dev, " ALC %2x,", bbp); + + rt2800_bbp_write(rt2x00dev, 158, 0xb1); + bbp = loft_dc_search_result[CHAIN_0][rf_alc_idx][0x00]; + bbp = bbp & 0x3F; + rt2800_bbp_write(rt2x00dev, 159, bbp); + rt2x00_dbg(rt2x00dev, " I0 %2x,", bbp); + + rt2800_bbp_write(rt2x00dev, 158, 0xb2); + bbp = loft_dc_search_result[CHAIN_0][rf_alc_idx][0x01]; + bbp = bbp & 0x3F; + rt2800_bbp_write(rt2x00dev, 159, bbp); + rt2x00_dbg(rt2x00dev, " Q0 %2x,", bbp); + + rt2800_bbp_write(rt2x00dev, 158, 0xb8); + bbp = loft_dc_search_result[CHAIN_1][rf_alc_idx][0x00]; + bbp = bbp & 0x3F; + rt2800_bbp_write(rt2x00dev, 159, bbp); + rt2x00_dbg(rt2x00dev, " I1 %2x,", bbp); + + rt2800_bbp_write(rt2x00dev, 158, 0xb9); + bbp = loft_dc_search_result[CHAIN_1][rf_alc_idx][0x01]; + bbp = bbp & 0x3F; + rt2800_bbp_write(rt2x00dev, 159, bbp); + rt2x00_dbg(rt2x00dev, " Q1 %2x\n", bbp); + } + } + + rt2800_bbp_write(rt2x00dev, 23, 0x00); + rt2800_bbp_write(rt2x00dev, 24, 0x00); + + rt2800_register_write(rt2x00dev, RF_CONTROL0, 0x04); + + rt2800_bbp_write(rt2x00dev, 158, 0x00); + rt2800_bbp_write(rt2x00dev, 159, 0x00); + + bbp = 0x00; + rt2800_bbp_write(rt2x00dev, 244, 0x00); + + rt2800_bbp_write(rt2x00dev, 21, 0x01); + udelay(1); + rt2800_bbp_write(rt2x00dev, 21, 0x00); + + rt2800_rf_configrecover(rt2x00dev, rf_store); + + rt2800_register_write(rt2x00dev, TX_PIN_CFG, macorg1); + rt2800_register_write(rt2x00dev, RF_CONTROL0, 0x04); + rt2800_register_write(rt2x00dev, RF_CONTROL0, 0x00); + rt2800_register_write(rt2x00dev, RF_BYPASS0, 0x00); + rt2800_register_write(rt2x00dev, RF_CONTROL0, macorg2); + udelay(1); + rt2800_register_write(rt2x00dev, RF_BYPASS0, macorg3); + rt2800_register_write(rt2x00dev, RF_CONTROL3, macorg4); + rt2800_register_write(rt2x00dev, RF_BYPASS3, macorg5); + rt2800_register_write(rt2x00dev, MAC_SYS_CTRL, savemacsysctrl); + rt2800_register_write(rt2x00dev, RF_CONTROL2, orig528); + rt2800_register_write(rt2x00dev, RF_BYPASS2, orig52c); + rt2800_register_write(rt2x00dev, 0x13b8, mac13b8); + + savemacsysctrl = rt2800_register_read(rt2x00dev, MAC_SYS_CTRL); + macorg1 = rt2800_register_read(rt2x00dev, TX_PIN_CFG); + macorg2 = rt2800_register_read(rt2x00dev, RF_CONTROL0); + macorg3 = rt2800_register_read(rt2x00dev, RF_BYPASS0); + macorg4 = rt2800_register_read(rt2x00dev, RF_CONTROL3); + macorg5 = rt2800_register_read(rt2x00dev, RF_BYPASS3); + + bbpr1 = rt2800_bbp_read(rt2x00dev, 1); + bbpr4 = rt2800_bbp_read(rt2x00dev, 4); + bbpr241 = rt2800_bbp_read(rt2x00dev, 241); + bbpr242 = rt2800_bbp_read(rt2x00dev, 242); + mac13b8 = rt2800_register_read(rt2x00dev, 0x13b8); + + macvalue = rt2800_register_read(rt2x00dev, MAC_SYS_CTRL); + macvalue &= (~0x04); + rt2800_register_write(rt2x00dev, MAC_SYS_CTRL, macvalue); + + if (unlikely(rt2800_wait_bbp_rf_ready(rt2x00dev, MAC_STATUS_CFG_BBP_RF_BUSY_TX))) + rt2x00_warn(rt2x00dev, "RF TX busy in LOFT IQ calibration\n"); + + macvalue = rt2800_register_read(rt2x00dev, MAC_SYS_CTRL); + macvalue &= (~0x08); + rt2800_register_write(rt2x00dev, MAC_SYS_CTRL, macvalue); + + if (unlikely(rt2800_wait_bbp_rf_ready(rt2x00dev, MAC_STATUS_CFG_BBP_RF_BUSY_RX))) + rt2x00_warn(rt2x00dev, "RF RX busy in LOFT IQ calibration\n"); + + if (test_bit(CAPABILITY_EXTERNAL_PA_TX0, &rt2x00dev->cap_flags)) { + rt2800_register_write(rt2x00dev, RF_CONTROL3, 0x00000101); + rt2800_register_write(rt2x00dev, RF_BYPASS3, 0x0000F1F1); + } + + rt2800_bbp_write(rt2x00dev, 23, 0x00); + rt2800_bbp_write(rt2x00dev, 24, 0x00); + + if (test_bit(CAPABILITY_EXTERNAL_PA_TX0, &rt2x00dev->cap_flags)) { + rt2800_bbp_write(rt2x00dev, 4, bbpr4 & (~0x18)); + rt2800_bbp_write(rt2x00dev, 21, 0x01); + udelay(1); + rt2800_bbp_write(rt2x00dev, 21, 0x00); + + rt2800_bbp_write(rt2x00dev, 241, 0x14); + rt2800_bbp_write(rt2x00dev, 242, 0x80); + rt2800_bbp_write(rt2x00dev, 244, 0x31); + } else { + rt2800_setbbptonegenerator(rt2x00dev); + } + + rt2800_register_write(rt2x00dev, RF_CONTROL0, 0x00000004); + rt2800_register_write(rt2x00dev, RF_BYPASS0, 0x00003306); + udelay(1); + + rt2800_register_write(rt2x00dev, TX_PIN_CFG, 0x0000000F); + + if (!test_bit(CAPABILITY_EXTERNAL_PA_TX0, &rt2x00dev->cap_flags)) { + rt2800_register_write(rt2x00dev, RF_CONTROL3, 0x00000000); + rt2800_register_write(rt2x00dev, RF_BYPASS3, 0x0000F1F1); + } + + rt2800_register_write(rt2x00dev, 0x13b8, 0x00000010); + + for (ch_idx = 0; ch_idx < 2; ch_idx++) + rt2800_rf_configstore(rt2x00dev, rf_store, ch_idx); + + rt2800_rfcsr_write_dccal(rt2x00dev, 3, 0x3B); + rt2800_rfcsr_write_dccal(rt2x00dev, 4, 0x3B); + + rt2800_bbp_write(rt2x00dev, 158, 0x03); + rt2800_bbp_write(rt2x00dev, 159, 0x60); + rt2800_bbp_write(rt2x00dev, 158, 0xB0); + rt2800_bbp_write(rt2x00dev, 159, 0x80); + + for (ch_idx = 0; ch_idx < 2; ch_idx++) { + rt2800_bbp_write(rt2x00dev, 23, 0x00); + rt2800_bbp_write(rt2x00dev, 24, 0x00); + + if (ch_idx == 0) { + rt2800_bbp_write(rt2x00dev, 158, 0x01); + rt2800_bbp_write(rt2x00dev, 159, 0x00); + if (test_bit(CAPABILITY_EXTERNAL_PA_TX0, &rt2x00dev->cap_flags)) { + bbp = bbpr1 & (~0x18); + bbp = bbp | 0x00; + rt2800_bbp_write(rt2x00dev, 1, bbp); + } + rt2800_rf_aux_tx0_loopback(rt2x00dev); + rt2800_register_write(rt2x00dev, RF_CONTROL0, 0x00001004); + } else { + rt2800_bbp_write(rt2x00dev, 158, 0x01); + rt2800_bbp_write(rt2x00dev, 159, 0x01); + if (test_bit(CAPABILITY_EXTERNAL_PA_TX1, &rt2x00dev->cap_flags)) { + bbp = bbpr1 & (~0x18); + bbp = bbp | 0x08; + rt2800_bbp_write(rt2x00dev, 1, bbp); + } + rt2800_rf_aux_tx1_loopback(rt2x00dev); + rt2800_register_write(rt2x00dev, RF_CONTROL0, 0x00002004); + } + + rt2800_bbp_write(rt2x00dev, 158, 0x05); + rt2800_bbp_write(rt2x00dev, 159, 0x04); + + bbp = (ch_idx == 0) ? 0x28 : 0x46; + rt2800_bbp_write(rt2x00dev, 158, bbp); + rt2800_bbp_write(rt2x00dev, 159, 0x00); + + if (test_bit(CAPABILITY_EXTERNAL_PA_TX0, &rt2x00dev->cap_flags)) { + rt2800_bbp_write(rt2x00dev, 23, 0x06); + rt2800_bbp_write(rt2x00dev, 24, 0x06); + count_step = 1; + } else { + rt2800_bbp_write(rt2x00dev, 23, 0x1F); + rt2800_bbp_write(rt2x00dev, 24, 0x1F); + count_step = 2; + } + + for (; vga_gain[ch_idx] < 19; vga_gain[ch_idx] = (vga_gain[ch_idx] + count_step)) { + rfvalue = rfvga_gain_table[vga_gain[ch_idx]]; + rt2800_rfcsr_write_dccal(rt2x00dev, 3, rfvalue); + rt2800_rfcsr_write_dccal(rt2x00dev, 4, rfvalue); + + bbp = (ch_idx == 0) ? 0x29 : 0x47; + rt2800_bbp_write(rt2x00dev, 158, bbp); + rt2800_bbp_write(rt2x00dev, 159, 0x00); + p0 = rt2800_do_fft_accumulation(rt2x00dev, 0x14, 0); + if (test_bit(CAPABILITY_EXTERNAL_PA_TX0, &rt2x00dev->cap_flags)) + p0_idx10 = rt2800_read_fft_accumulation(rt2x00dev, 0x0A); + + bbp = (ch_idx == 0) ? 0x29 : 0x47; + rt2800_bbp_write(rt2x00dev, 158, bbp); + rt2800_bbp_write(rt2x00dev, 159, 0x21); + p1 = rt2800_do_fft_accumulation(rt2x00dev, 0x14, 0); + if (test_bit(CAPABILITY_EXTERNAL_PA_TX1, &rt2x00dev->cap_flags)) + p1_idx10 = rt2800_read_fft_accumulation(rt2x00dev, 0x0A); + + rt2x00_dbg(rt2x00dev, "IQ AGC %d %d\n", p0, p1); + + if (test_bit(CAPABILITY_EXTERNAL_PA_TX0, &rt2x00dev->cap_flags)) { + rt2x00_dbg(rt2x00dev, "IQ AGC IDX 10 %d %d\n", p0_idx10, p1_idx10); + if ((p0_idx10 > 7000 * 7000) || (p1_idx10 > 7000 * 7000)) { + if (vga_gain[ch_idx] != 0) + vga_gain[ch_idx] = vga_gain[ch_idx] - 1; + break; + } + } + + if ((p0 > 2500 * 2500) || (p1 > 2500 * 2500)) + break; + } + + if (vga_gain[ch_idx] > 18) + vga_gain[ch_idx] = 18; + rt2x00_dbg(rt2x00dev, "Used VGA %d %x\n", vga_gain[ch_idx], + rfvga_gain_table[vga_gain[ch_idx]]); + + bbp = (ch_idx == 0) ? 0x29 : 0x47; + rt2800_bbp_write(rt2x00dev, 158, bbp); + rt2800_bbp_write(rt2x00dev, 159, 0x00); + + rt2800_iq_search(rt2x00dev, ch_idx, ger, per); + } + + rt2800_bbp_write(rt2x00dev, 23, 0x00); + rt2800_bbp_write(rt2x00dev, 24, 0x00); + rt2800_register_write(rt2x00dev, RF_CONTROL0, 0x04); + + rt2800_bbp_write(rt2x00dev, 158, 0x28); + bbp = ger[CHAIN_0] & 0x0F; + rt2800_bbp_write(rt2x00dev, 159, bbp); + + rt2800_bbp_write(rt2x00dev, 158, 0x29); + bbp = per[CHAIN_0] & 0x3F; + rt2800_bbp_write(rt2x00dev, 159, bbp); + + rt2800_bbp_write(rt2x00dev, 158, 0x46); + bbp = ger[CHAIN_1] & 0x0F; + rt2800_bbp_write(rt2x00dev, 159, bbp); + + rt2800_bbp_write(rt2x00dev, 158, 0x47); + bbp = per[CHAIN_1] & 0x3F; + rt2800_bbp_write(rt2x00dev, 159, bbp); + + if (test_bit(CAPABILITY_EXTERNAL_PA_TX0, &rt2x00dev->cap_flags)) { + rt2800_bbp_write(rt2x00dev, 1, bbpr1); + rt2800_bbp_write(rt2x00dev, 241, bbpr241); + rt2800_bbp_write(rt2x00dev, 242, bbpr242); + } + rt2800_bbp_write(rt2x00dev, 244, 0x00); + + rt2800_bbp_write(rt2x00dev, 158, 0x00); + rt2800_bbp_write(rt2x00dev, 159, 0x00); + rt2800_bbp_write(rt2x00dev, 158, 0xB0); + rt2800_bbp_write(rt2x00dev, 159, 0x00); + + rt2800_bbp_write(rt2x00dev, 30, bbpr30); + rt2800_rfcsr_write_bank(rt2x00dev, 0, 39, rfb0r39); + rt2800_rfcsr_write_bank(rt2x00dev, 0, 42, rfb0r42); + + if (test_bit(CAPABILITY_EXTERNAL_PA_TX0, &rt2x00dev->cap_flags)) + rt2800_bbp_write(rt2x00dev, 4, bbpr4); + + rt2800_bbp_write(rt2x00dev, 21, 0x01); + udelay(1); + rt2800_bbp_write(rt2x00dev, 21, 0x00); + + rt2800_rf_configrecover(rt2x00dev, rf_store); + + rt2800_register_write(rt2x00dev, TX_PIN_CFG, macorg1); + rt2800_register_write(rt2x00dev, RF_CONTROL0, 0x00); + rt2800_register_write(rt2x00dev, RF_BYPASS0, 0x00); + rt2800_register_write(rt2x00dev, RF_CONTROL0, macorg2); + udelay(1); + rt2800_register_write(rt2x00dev, RF_BYPASS0, macorg3); + rt2800_register_write(rt2x00dev, RF_CONTROL3, macorg4); + rt2800_register_write(rt2x00dev, RF_BYPASS3, macorg5); + rt2800_register_write(rt2x00dev, MAC_SYS_CTRL, savemacsysctrl); + rt2800_register_write(rt2x00dev, 0x13b8, mac13b8); +} + static void rt2800_bbp_core_soft_reset(struct rt2x00_dev *rt2x00dev, bool set_bw, bool is_ht40) { @@ -9005,8 +10582,13 @@ static void rt2800_init_rfcsr_6352(struct rt2x00_dev *rt2x00dev) rt2800_rfcsr_write_dccal(rt2x00dev, 5, 0x00); rt2800_rfcsr_write_dccal(rt2x00dev, 17, 0x7C); + rt2800_r_calibration(rt2x00dev); + rt2800_rf_self_txdc_cal(rt2x00dev); + rt2800_rxdcoc_calibration(rt2x00dev); rt2800_bw_filter_calibration(rt2x00dev, true); rt2800_bw_filter_calibration(rt2x00dev, false); + rt2800_loft_iq_calibration(rt2x00dev); + rt2800_rxiq_calibration(rt2x00dev); } static void rt2800_init_rfcsr(struct rt2x00_dev *rt2x00dev) @@ -9073,7 +10655,7 @@ int rt2800_enable_radio(struct rt2x00_dev *rt2x00dev) /* * Wait BBP/RF to wake up. */ - if (unlikely(rt2800_wait_bbp_rf_ready(rt2x00dev))) + if (unlikely(rt2800_wait_bbp_rf_ready(rt2x00dev, MAC_STATUS_CFG_BBP_RF_BUSY))) return -EIO; /* @@ -9435,6 +11017,8 @@ static int rt2800_init_eeprom(struct rt2x00_dev *rt2x00dev) rf = RF3853; else if (rt2x00_rt(rt2x00dev, RT5350)) rf = RF5350; + else if (rt2x00_rt(rt2x00dev, RT5592)) + rf = RF5592; else rf = rt2x00_get_field16(eeprom, EEPROM_NIC_CONF0_RF_TYPE); @@ -9564,7 +11148,8 @@ static int rt2800_init_eeprom(struct rt2x00_dev *rt2x00dev) */ eeprom = rt2800_eeprom_read(rt2x00dev, EEPROM_NIC_CONF1); - if (rt2x00_rt(rt2x00dev, RT3352)) { + if (rt2x00_rt(rt2x00dev, RT3352) || + rt2x00_rt(rt2x00dev, RT6352)) { if (rt2x00_get_field16(eeprom, EEPROM_NIC_CONF1_EXTERNAL_TX0_PA_3352)) __set_bit(CAPABILITY_EXTERNAL_PA_TX0, @@ -9575,6 +11160,18 @@ static int rt2800_init_eeprom(struct rt2x00_dev *rt2x00dev) &rt2x00dev->cap_flags); } + eeprom = rt2800_eeprom_read(rt2x00dev, EEPROM_NIC_CONF2); + + if (rt2x00_rt(rt2x00dev, RT6352) && eeprom != 0 && eeprom != 0xffff) { + if (!rt2x00_get_field16(eeprom, + EEPROM_NIC_CONF2_EXTERNAL_PA)) { + __clear_bit(CAPABILITY_EXTERNAL_PA_TX0, + &rt2x00dev->cap_flags); + __clear_bit(CAPABILITY_EXTERNAL_PA_TX1, + &rt2x00dev->cap_flags); + } + } + return 0; } diff --git a/drivers/net/wireless/ralink/rt2x00/rt2800lib.h b/drivers/net/wireless/ralink/rt2x00/rt2800lib.h index e1761f467b94..3cbef77b4bd3 100644 --- a/drivers/net/wireless/ralink/rt2x00/rt2800lib.h +++ b/drivers/net/wireless/ralink/rt2x00/rt2800lib.h @@ -17,6 +17,16 @@ #define WCID_START 33 #define WCID_END 222 #define STA_IDS_SIZE (WCID_END - WCID_START + 2) +#define CHAIN_0 0x0 +#define CHAIN_1 0x1 +#define RF_ALC_NUM 6 +#define CHAIN_NUM 2 + +struct rf_reg_pair { + u8 bank; + u8 reg; + u8 value; +}; /* RT2800 driver data structure */ struct rt2800_drv_data { diff --git a/drivers/net/wireless/ralink/rt2x00/rt2x00.h b/drivers/net/wireless/ralink/rt2x00/rt2x00.h index 8f5772b98f58..07a6a5a9ce13 100644 --- a/drivers/net/wireless/ralink/rt2x00/rt2x00.h +++ b/drivers/net/wireless/ralink/rt2x00/rt2x00.h @@ -1309,8 +1309,11 @@ void rt2x00queue_unmap_skb(struct queue_entry *entry); */ static inline struct data_queue * rt2x00queue_get_tx_queue(struct rt2x00_dev *rt2x00dev, - const enum data_queue_qid queue) + enum data_queue_qid queue) { + if (queue >= rt2x00dev->ops->tx_queues && queue < IEEE80211_NUM_ACS) + queue = rt2x00dev->ops->tx_queues - 1; + if (queue < rt2x00dev->ops->tx_queues && rt2x00dev->tx) return &rt2x00dev->tx[queue]; diff --git a/drivers/net/wireless/ralink/rt2x00/rt2x00dev.c b/drivers/net/wireless/ralink/rt2x00/rt2x00dev.c index e95c101c2711..3a035afcf7f9 100644 --- a/drivers/net/wireless/ralink/rt2x00/rt2x00dev.c +++ b/drivers/net/wireless/ralink/rt2x00/rt2x00dev.c @@ -1093,6 +1093,19 @@ static void rt2x00lib_remove_hw(struct rt2x00_dev *rt2x00dev) kfree(rt2x00dev->spec.channels_info); } +static const struct ieee80211_tpt_blink rt2x00_tpt_blink[] = { + { .throughput = 0 * 1024, .blink_time = 334 }, + { .throughput = 1 * 1024, .blink_time = 260 }, + { .throughput = 2 * 1024, .blink_time = 220 }, + { .throughput = 5 * 1024, .blink_time = 190 }, + { .throughput = 10 * 1024, .blink_time = 170 }, + { .throughput = 25 * 1024, .blink_time = 150 }, + { .throughput = 54 * 1024, .blink_time = 130 }, + { .throughput = 120 * 1024, .blink_time = 110 }, + { .throughput = 265 * 1024, .blink_time = 80 }, + { .throughput = 586 * 1024, .blink_time = 50 }, +}; + static int rt2x00lib_probe_hw(struct rt2x00_dev *rt2x00dev) { struct hw_mode_spec *spec = &rt2x00dev->spec; @@ -1174,6 +1187,11 @@ static int rt2x00lib_probe_hw(struct rt2x00_dev *rt2x00dev) #undef RT2X00_TASKLET_INIT + ieee80211_create_tpt_led_trigger(rt2x00dev->hw, + IEEE80211_TPT_LEDTRIG_FL_RADIO, + rt2x00_tpt_blink, + ARRAY_SIZE(rt2x00_tpt_blink)); + /* * Register HW. */ diff --git a/drivers/net/wireless/ralink/rt2x00/rt2x00queue.c b/drivers/net/wireless/ralink/rt2x00/rt2x00queue.c index 4d06038afd83..98df0aef8168 100644 --- a/drivers/net/wireless/ralink/rt2x00/rt2x00queue.c +++ b/drivers/net/wireless/ralink/rt2x00/rt2x00queue.c @@ -318,7 +318,7 @@ static void rt2x00queue_create_tx_descriptor_ht(struct rt2x00_dev *rt2x00dev, * when using more then one tx stream (>MCS7). */ if (sta && txdesc->u.ht.mcs > 7 && - sta->smps_mode == IEEE80211_SMPS_DYNAMIC) + sta->deflink.smps_mode == IEEE80211_SMPS_DYNAMIC) __set_bit(ENTRY_TXD_HT_MIMO_PS, &txdesc->flags); } else { txdesc->u.ht.mcs = rt2x00_get_rate_mcs(hwrate->mcs); diff --git a/drivers/net/wireless/realtek/rtl818x/rtl8187/leds.c b/drivers/net/wireless/realtek/rtl818x/rtl8187/leds.c index 49421d10e22b..f7d95c9624a0 100644 --- a/drivers/net/wireless/realtek/rtl818x/rtl8187/leds.c +++ b/drivers/net/wireless/realtek/rtl818x/rtl8187/leds.c @@ -143,7 +143,7 @@ static int rtl8187_register_led(struct ieee80211_hw *dev, led->dev = dev; led->ledpin = ledpin; led->is_radio = is_radio; - strlcpy(led->name, name, sizeof(led->name)); + strscpy(led->name, name, sizeof(led->name)); led->led_dev.name = led->name; led->led_dev.default_trigger = default_trigger; diff --git a/drivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu.h b/drivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu.h index 7ddce3c3f0c4..782b089a2e1b 100644 --- a/drivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu.h +++ b/drivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu.h @@ -1425,7 +1425,7 @@ struct rtl8xxxu_fileops { void (*set_tx_power) (struct rtl8xxxu_priv *priv, int channel, bool ht40); void (*update_rate_mask) (struct rtl8xxxu_priv *priv, - u32 ramask, u8 rateid, int sgi); + u32 ramask, u8 rateid, int sgi, int txbw_40mhz); void (*report_connect) (struct rtl8xxxu_priv *priv, u8 macid, bool connect); void (*fill_txdesc) (struct ieee80211_hw *hw, struct ieee80211_hdr *hdr, @@ -1511,9 +1511,9 @@ void rtl8xxxu_gen2_config_channel(struct ieee80211_hw *hw); void rtl8xxxu_gen1_usb_quirks(struct rtl8xxxu_priv *priv); void rtl8xxxu_gen2_usb_quirks(struct rtl8xxxu_priv *priv); void rtl8xxxu_update_rate_mask(struct rtl8xxxu_priv *priv, - u32 ramask, u8 rateid, int sgi); + u32 ramask, u8 rateid, int sgi, int txbw_40mhz); void rtl8xxxu_gen2_update_rate_mask(struct rtl8xxxu_priv *priv, - u32 ramask, u8 rateid, int sgi); + u32 ramask, u8 rateid, int sgi, int txbw_40mhz); void rtl8xxxu_gen1_report_connect(struct rtl8xxxu_priv *priv, u8 macid, bool connect); void rtl8xxxu_gen2_report_connect(struct rtl8xxxu_priv *priv, diff --git a/drivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu_core.c b/drivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu_core.c index c66f0726b253..ac641a56efb0 100644 --- a/drivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu_core.c +++ b/drivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu_core.c @@ -1878,13 +1878,6 @@ static int rtl8xxxu_read_efuse(struct rtl8xxxu_priv *priv) /* We have 8 bits to indicate validity */ map_addr = offset * 8; - if (map_addr >= EFUSE_MAP_LEN) { - dev_warn(dev, "%s: Illegal map_addr (%04x), " - "efuse corrupt!\n", - __func__, map_addr); - ret = -EINVAL; - goto exit; - } for (i = 0; i < EFUSE_MAX_WORD_UNIT; i++) { /* Check word enable condition in the section */ if (word_mask & BIT(i)) { @@ -1895,6 +1888,13 @@ static int rtl8xxxu_read_efuse(struct rtl8xxxu_priv *priv) ret = rtl8xxxu_read_efuse8(priv, efuse_addr++, &val8); if (ret) goto exit; + if (map_addr >= EFUSE_MAP_LEN - 1) { + dev_warn(dev, "%s: Illegal map_addr (%04x), " + "efuse corrupt!\n", + __func__, map_addr); + ret = -EINVAL; + goto exit; + } priv->efuse_wifi.raw[map_addr++] = val8; ret = rtl8xxxu_read_efuse8(priv, efuse_addr++, &val8); @@ -2929,12 +2929,12 @@ bool rtl8xxxu_gen2_simularity_compare(struct rtl8xxxu_priv *priv, } if (!(simubitmap & 0x30) && priv->tx_paths > 1) { - /* path B RX OK */ + /* path B TX OK */ for (i = 4; i < 6; i++) result[3][i] = result[c1][i]; } - if (!(simubitmap & 0x30) && priv->tx_paths > 1) { + if (!(simubitmap & 0xc0) && priv->tx_paths > 1) { /* path B RX OK */ for (i = 6; i < 8; i++) result[3][i] = result[c1][i]; @@ -4320,7 +4320,7 @@ static void rtl8xxxu_sw_scan_complete(struct ieee80211_hw *hw, } void rtl8xxxu_update_rate_mask(struct rtl8xxxu_priv *priv, - u32 ramask, u8 rateid, int sgi) + u32 ramask, u8 rateid, int sgi, int txbw_40mhz) { struct h2c_cmd h2c; @@ -4340,10 +4340,15 @@ void rtl8xxxu_update_rate_mask(struct rtl8xxxu_priv *priv, } void rtl8xxxu_gen2_update_rate_mask(struct rtl8xxxu_priv *priv, - u32 ramask, u8 rateid, int sgi) + u32 ramask, u8 rateid, int sgi, int txbw_40mhz) { struct h2c_cmd h2c; - u8 bw = RTL8XXXU_CHANNEL_WIDTH_20; + u8 bw; + + if (txbw_40mhz) + bw = RTL8XXXU_CHANNEL_WIDTH_40; + else + bw = RTL8XXXU_CHANNEL_WIDTH_20; memset(&h2c, 0, sizeof(struct h2c_cmd)); @@ -4353,15 +4358,14 @@ void rtl8xxxu_gen2_update_rate_mask(struct rtl8xxxu_priv *priv, h2c.b_macid_cfg.ramask2 = (ramask >> 16) & 0xff; h2c.b_macid_cfg.ramask3 = (ramask >> 24) & 0xff; - h2c.ramask.arg = 0x80; h2c.b_macid_cfg.data1 = rateid; if (sgi) h2c.b_macid_cfg.data1 |= BIT(7); h2c.b_macid_cfg.data2 = bw; - dev_dbg(&priv->udev->dev, "%s: rate mask %08x, arg %02x, size %zi\n", - __func__, ramask, h2c.ramask.arg, sizeof(h2c.b_macid_cfg)); + dev_dbg(&priv->udev->dev, "%s: rate mask %08x, rateid %02x, sgi %d, size %zi\n", + __func__, ramask, rateid, sgi, sizeof(h2c.b_macid_cfg)); rtl8xxxu_gen2_h2c_cmd(priv, &h2c, sizeof(h2c.b_macid_cfg)); } @@ -4556,6 +4560,53 @@ rtl8xxxu_wireless_mode(struct ieee80211_hw *hw, struct ieee80211_sta *sta) return network_type; } +static void rtl8xxxu_set_aifs(struct rtl8xxxu_priv *priv, u8 slot_time) +{ + u32 reg_edca_param[IEEE80211_NUM_ACS] = { + [IEEE80211_AC_VO] = REG_EDCA_VO_PARAM, + [IEEE80211_AC_VI] = REG_EDCA_VI_PARAM, + [IEEE80211_AC_BE] = REG_EDCA_BE_PARAM, + [IEEE80211_AC_BK] = REG_EDCA_BK_PARAM, + }; + u32 val32; + u16 wireless_mode = 0; + u8 aifs, aifsn, sifs; + int i; + + if (priv->vif) { + struct ieee80211_sta *sta; + + rcu_read_lock(); + sta = ieee80211_find_sta(priv->vif, priv->vif->bss_conf.bssid); + if (sta) + wireless_mode = rtl8xxxu_wireless_mode(priv->hw, sta); + rcu_read_unlock(); + } + + if (priv->hw->conf.chandef.chan->band == NL80211_BAND_5GHZ || + (wireless_mode & WIRELESS_MODE_N_24G)) + sifs = 16; + else + sifs = 10; + + for (i = 0; i < IEEE80211_NUM_ACS; i++) { + val32 = rtl8xxxu_read32(priv, reg_edca_param[i]); + + /* It was set in conf_tx. */ + aifsn = val32 & 0xff; + + /* aifsn not set yet or already fixed */ + if (aifsn < 2 || aifsn > 15) + continue; + + aifs = aifsn * slot_time + sifs; + + val32 &= ~0xff; + val32 |= aifs; + rtl8xxxu_write32(priv, reg_edca_param[i], val32); + } +} + static void rtl8xxxu_bss_info_changed(struct ieee80211_hw *hw, struct ieee80211_vif *vif, struct ieee80211_bss_conf *bss_conf, u64 changed) @@ -4622,7 +4673,11 @@ rtl8xxxu_bss_info_changed(struct ieee80211_hw *hw, struct ieee80211_vif *vif, RATE_INFO_FLAGS_SHORT_GI; } - rarpt->txrate.bw |= RATE_INFO_BW_20; + if (rtl8xxxu_ht40_2g && + (sta->deflink.ht_cap.cap & IEEE80211_HT_CAP_SUP_WIDTH_20_40)) + rarpt->txrate.bw = RATE_INFO_BW_40; + else + rarpt->txrate.bw = RATE_INFO_BW_20; } bit_rate = cfg80211_calculate_bitrate(&rarpt->txrate); rarpt->bit_rate = bit_rate; @@ -4631,7 +4686,7 @@ rtl8xxxu_bss_info_changed(struct ieee80211_hw *hw, struct ieee80211_vif *vif, priv->vif = vif; priv->rssi_level = RTL8XXXU_RATR_STA_INIT; - priv->fops->update_rate_mask(priv, ramask, 0, sgi); + priv->fops->update_rate_mask(priv, ramask, 0, sgi, rarpt->txrate.bw == RATE_INFO_BW_40); rtl8xxxu_write8(priv, REG_BCN_MAX_ERR, 0xff); @@ -4671,6 +4726,8 @@ rtl8xxxu_bss_info_changed(struct ieee80211_hw *hw, struct ieee80211_vif *vif, else val8 = 20; rtl8xxxu_write8(priv, REG_SLOT, val8); + + rtl8xxxu_set_aifs(priv, val8); } if (changed & BSS_CHANGED_BSSID) { @@ -4710,9 +4767,8 @@ static u32 rtl8xxxu_80211_to_rtl_queue(u32 queue) return rtlqueue; } -static u32 rtl8xxxu_queue_select(struct ieee80211_hw *hw, struct sk_buff *skb) +static u32 rtl8xxxu_queue_select(struct ieee80211_hdr *hdr, struct sk_buff *skb) { - struct ieee80211_hdr *hdr = (struct ieee80211_hdr *)skb->data; u32 queue; if (ieee80211_is_mgmt(hdr->frame_control)) @@ -5062,6 +5118,8 @@ static void rtl8xxxu_tx(struct ieee80211_hw *hw, if (control && control->sta) sta = control->sta; + queue = rtl8xxxu_queue_select(hdr, skb); + tx_desc = skb_push(skb, tx_desc_size); memset(tx_desc, 0, tx_desc_size); @@ -5074,7 +5132,6 @@ static void rtl8xxxu_tx(struct ieee80211_hw *hw, is_broadcast_ether_addr(ieee80211_get_DA(hdr))) tx_desc->txdw0 |= TXDESC_BROADMULTICAST; - queue = rtl8xxxu_queue_select(hw, skb); tx_desc->txdw1 = cpu_to_le32(queue << TXDESC_QUEUE_SHIFT); if (tx_info->control.hw_key) { @@ -6344,7 +6401,7 @@ static void rtl8xxxu_refresh_rate_mask(struct rtl8xxxu_priv *priv, } priv->rssi_level = rssi_level; - priv->fops->update_rate_mask(priv, rate_bitmap, ratr_idx, sgi); + priv->fops->update_rate_mask(priv, rate_bitmap, ratr_idx, sgi, txbw_40mhz); } } @@ -6657,7 +6714,6 @@ static int rtl8xxxu_probe(struct usb_interface *interface, hw = ieee80211_alloc_hw(sizeof(struct rtl8xxxu_priv), &rtl8xxxu_ops); if (!hw) { ret = -ENOMEM; - priv = NULL; goto err_put_dev; } @@ -6768,11 +6824,9 @@ static int rtl8xxxu_probe(struct usb_interface *interface, err_set_intfdata: usb_set_intfdata(interface, NULL); - if (priv) { - kfree(priv->fw_data); - mutex_destroy(&priv->usb_buf_mutex); - mutex_destroy(&priv->h2c_mutex); - } + kfree(priv->fw_data); + mutex_destroy(&priv->usb_buf_mutex); + mutex_destroy(&priv->h2c_mutex); ieee80211_free_hw(hw); err_put_dev: diff --git a/drivers/net/wireless/realtek/rtlwifi/rtl8192de/phy.c b/drivers/net/wireless/realtek/rtlwifi/rtl8192de/phy.c index 15e6a6aded31..d18c092b6142 100644 --- a/drivers/net/wireless/realtek/rtlwifi/rtl8192de/phy.c +++ b/drivers/net/wireless/realtek/rtlwifi/rtl8192de/phy.c @@ -2386,11 +2386,10 @@ void rtl92d_phy_reload_iqk_setting(struct ieee80211_hw *hw, u8 channel) rtl_dbg(rtlpriv, COMP_SCAN, DBG_LOUD, "Just Read IQK Matrix reg for channel:%d....\n", channel); - _rtl92d_phy_patha_fill_iqk_matrix(hw, true, - rtlphy->iqk_matrix[ - indexforchannel].value, 0, - (rtlphy->iqk_matrix[ - indexforchannel].value[0][2] == 0)); + if (rtlphy->iqk_matrix[indexforchannel].value[0][0] != 0) + _rtl92d_phy_patha_fill_iqk_matrix(hw, true, + rtlphy->iqk_matrix[indexforchannel].value, 0, + rtlphy->iqk_matrix[indexforchannel].value[0][2] == 0); if (IS_92D_SINGLEPHY(rtlhal->version)) { if ((rtlphy->iqk_matrix[ indexforchannel].value[0][4] != 0) diff --git a/drivers/net/wireless/realtek/rtw88/bf.c b/drivers/net/wireless/realtek/rtw88/bf.c index 76c7f3257dd3..038a30b170ef 100644 --- a/drivers/net/wireless/realtek/rtw88/bf.c +++ b/drivers/net/wireless/realtek/rtw88/bf.c @@ -30,11 +30,11 @@ void rtw_bf_disassoc(struct rtw_dev *rtwdev, struct ieee80211_vif *vif, void rtw_bf_assoc(struct rtw_dev *rtwdev, struct ieee80211_vif *vif, struct ieee80211_bss_conf *bss_conf) { + const struct rtw_chip_info *chip = rtwdev->chip; struct ieee80211_hw *hw = rtwdev->hw; struct rtw_vif *rtwvif = (struct rtw_vif *)vif->drv_priv; struct rtw_bfee *bfee = &rtwvif->bfee; struct rtw_bf_info *bfinfo = &rtwdev->bf_info; - struct rtw_chip_info *chip = rtwdev->chip; struct ieee80211_sta *sta; struct ieee80211_sta_vht_cap *vht_cap; struct ieee80211_sta_vht_cap *ic_vht_cap; diff --git a/drivers/net/wireless/realtek/rtw88/coex.c b/drivers/net/wireless/realtek/rtw88/coex.c index cac053f485c3..6276ad624299 100644 --- a/drivers/net/wireless/realtek/rtw88/coex.c +++ b/drivers/net/wireless/realtek/rtw88/coex.c @@ -13,7 +13,7 @@ static u8 rtw_coex_next_rssi_state(struct rtw_dev *rtwdev, u8 pre_state, u8 rssi, u8 rssi_thresh) { - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; u8 tol = chip->rssi_tolerance; u8 next_state; @@ -36,7 +36,7 @@ static u8 rtw_coex_next_rssi_state(struct rtw_dev *rtwdev, u8 pre_state, static void rtw_coex_limited_tx(struct rtw_dev *rtwdev, bool tx_limit_en, bool ampdu_limit_en) { - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; struct rtw_coex *coex = &rtwdev->coex; struct rtw_coex_stat *coex_stat = &coex->stat; u8 num_of_active_port = 1; @@ -365,7 +365,7 @@ static void rtw_coex_set_wl_pri_mask(struct rtw_dev *rtwdev, u8 bitmap, void rtw_coex_write_scbd(struct rtw_dev *rtwdev, u16 bitpos, bool set) { - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; struct rtw_coex *coex = &rtwdev->coex; struct rtw_coex_stat *coex_stat = &coex->stat; u16 val = 0x2; @@ -400,7 +400,7 @@ EXPORT_SYMBOL(rtw_coex_write_scbd); static u16 rtw_coex_read_scbd(struct rtw_dev *rtwdev) { - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; if (!chip->scbd_support) return 0; @@ -410,7 +410,7 @@ static u16 rtw_coex_read_scbd(struct rtw_dev *rtwdev) static void rtw_coex_check_rfk(struct rtw_dev *rtwdev) { - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; struct rtw_coex *coex = &rtwdev->coex; struct rtw_coex_stat *coex_stat = &coex->stat; struct rtw_coex_rfe *coex_rfe = &coex->rfe; @@ -489,7 +489,7 @@ static void rtw_coex_monitor_bt_ctr(struct rtw_dev *rtwdev) static void rtw_coex_monitor_bt_enable(struct rtw_dev *rtwdev) { - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; struct rtw_coex *coex = &rtwdev->coex; struct rtw_coex_stat *coex_stat = &coex->stat; struct rtw_coex_dm *coex_dm = &coex->dm; @@ -524,10 +524,10 @@ static void rtw_coex_monitor_bt_enable(struct rtw_dev *rtwdev) static void rtw_coex_update_wl_link_info(struct rtw_dev *rtwdev, u8 reason) { + const struct rtw_chip_info *chip = rtwdev->chip; struct rtw_coex *coex = &rtwdev->coex; struct rtw_coex_stat *coex_stat = &coex->stat; struct rtw_coex_dm *coex_dm = &coex->dm; - struct rtw_chip_info *chip = rtwdev->chip; struct rtw_traffic_stats *stats = &rtwdev->stats; bool is_5G = false; bool wl_busy = false; @@ -706,10 +706,10 @@ static const char *rtw_coex_get_bt_status_string(u8 bt_status) static void rtw_coex_update_bt_link_info(struct rtw_dev *rtwdev) { + const struct rtw_chip_info *chip = rtwdev->chip; struct rtw_coex *coex = &rtwdev->coex; struct rtw_coex_stat *coex_stat = &coex->stat; struct rtw_coex_dm *coex_dm = &coex->dm; - struct rtw_chip_info *chip = rtwdev->chip; u8 i; u8 rssi_state; u8 rssi_step; @@ -806,7 +806,7 @@ static void rtw_coex_update_bt_link_info(struct rtw_dev *rtwdev) static void rtw_coex_update_wl_ch_info(struct rtw_dev *rtwdev, u8 type) { - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; struct rtw_efuse *efuse = &rtwdev->efuse; struct rtw_coex_dm *coex_dm = &rtwdev->coex.dm; struct rtw_coex_stat *coex_stat = &rtwdev->coex.stat; @@ -933,7 +933,7 @@ EXPORT_SYMBOL(rtw_coex_write_indirect_reg); static void rtw_coex_coex_ctrl_owner(struct rtw_dev *rtwdev, bool wifi_control) { - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; const struct rtw_hw_reg *btg_reg = chip->btg_reg; if (wifi_control) { @@ -981,7 +981,7 @@ static void rtw_coex_mimo_ps(struct rtw_dev *rtwdev, bool force, bool state) static void rtw_btc_wltoggle_table_a(struct rtw_dev *rtwdev, bool force, u8 table_case) { - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; struct rtw_efuse *efuse = &rtwdev->efuse; u8 h2c_para[6] = {0}; u32 table_wl = 0x5a5a5a5a; @@ -1065,9 +1065,9 @@ static void rtw_coex_set_table(struct rtw_dev *rtwdev, bool force, u32 table0, static void rtw_coex_table(struct rtw_dev *rtwdev, bool force, u8 type) { + const struct rtw_chip_info *chip = rtwdev->chip; struct rtw_coex *coex = &rtwdev->coex; struct rtw_coex_dm *coex_dm = &coex->dm; - struct rtw_chip_info *chip = rtwdev->chip; struct rtw_efuse *efuse = &rtwdev->efuse; struct rtw_coex_stat *coex_stat = &coex->stat; @@ -1135,9 +1135,9 @@ static void rtw_coex_power_save_state(struct rtw_dev *rtwdev, u8 ps_type, static void rtw_coex_set_tdma(struct rtw_dev *rtwdev, u8 byte1, u8 byte2, u8 byte3, u8 byte4, u8 byte5) { + const struct rtw_chip_info *chip = rtwdev->chip; struct rtw_coex *coex = &rtwdev->coex; struct rtw_coex_dm *coex_dm = &coex->dm; - struct rtw_chip_info *chip = rtwdev->chip; struct rtw_coex_stat *coex_stat = &coex->stat; u8 ps_type = COEX_PS_WIFI_NATIVE; bool ap_enable = false; @@ -1193,10 +1193,10 @@ static void rtw_coex_set_tdma(struct rtw_dev *rtwdev, u8 byte1, u8 byte2, static void rtw_coex_tdma(struct rtw_dev *rtwdev, bool force, u32 tcase) { + const struct rtw_chip_info *chip = rtwdev->chip; struct rtw_coex *coex = &rtwdev->coex; struct rtw_coex_dm *coex_dm = &coex->dm; struct rtw_coex_stat *coex_stat = &coex->stat; - struct rtw_chip_info *chip = rtwdev->chip; struct rtw_efuse *efuse = &rtwdev->efuse; u8 n, type; bool turn_on; @@ -1526,8 +1526,8 @@ static u8 rtw_coex_algorithm(struct rtw_dev *rtwdev) static void rtw_coex_action_coex_all_off(struct rtw_dev *rtwdev) { + const struct rtw_chip_info *chip = rtwdev->chip; struct rtw_efuse *efuse = &rtwdev->efuse; - struct rtw_chip_info *chip = rtwdev->chip; u8 table_case, tdma_case; rtw_dbg(rtwdev, RTW_DBG_COEX, "[BTCoex], %s()\n", __func__); @@ -1549,11 +1549,11 @@ static void rtw_coex_action_coex_all_off(struct rtw_dev *rtwdev) static void rtw_coex_action_freerun(struct rtw_dev *rtwdev) { + const struct rtw_chip_info *chip = rtwdev->chip; struct rtw_coex *coex = &rtwdev->coex; struct rtw_coex_stat *coex_stat = &coex->stat; struct rtw_coex_dm *coex_dm = &coex->dm; struct rtw_efuse *efuse = &rtwdev->efuse; - struct rtw_chip_info *chip = rtwdev->chip; u8 level = 0; bool bt_afh_loss = true; @@ -1594,8 +1594,8 @@ static void rtw_coex_action_freerun(struct rtw_dev *rtwdev) static void rtw_coex_action_rf4ce(struct rtw_dev *rtwdev) { + const struct rtw_chip_info *chip = rtwdev->chip; struct rtw_efuse *efuse = &rtwdev->efuse; - struct rtw_chip_info *chip = rtwdev->chip; u8 table_case, tdma_case; rtw_dbg(rtwdev, RTW_DBG_COEX, "[BTCoex], %s()\n", __func__); @@ -1619,8 +1619,8 @@ static void rtw_coex_action_rf4ce(struct rtw_dev *rtwdev) static void rtw_coex_action_bt_whql_test(struct rtw_dev *rtwdev) { + const struct rtw_chip_info *chip = rtwdev->chip; struct rtw_efuse *efuse = &rtwdev->efuse; - struct rtw_chip_info *chip = rtwdev->chip; u8 table_case, tdma_case; rtw_dbg(rtwdev, RTW_DBG_COEX, "[BTCoex], %s()\n", __func__); @@ -1644,10 +1644,10 @@ static void rtw_coex_action_bt_whql_test(struct rtw_dev *rtwdev) static void rtw_coex_action_bt_relink(struct rtw_dev *rtwdev) { + const struct rtw_chip_info *chip = rtwdev->chip; struct rtw_coex *coex = &rtwdev->coex; struct rtw_coex_stat *coex_stat = &coex->stat; struct rtw_efuse *efuse = &rtwdev->efuse; - struct rtw_chip_info *chip = rtwdev->chip; u8 table_case, tdma_case; u32 slot_type = 0; @@ -1684,11 +1684,11 @@ static void rtw_coex_action_bt_relink(struct rtw_dev *rtwdev) static void rtw_coex_action_bt_idle(struct rtw_dev *rtwdev) { + const struct rtw_chip_info *chip = rtwdev->chip; struct rtw_coex *coex = &rtwdev->coex; struct rtw_coex_stat *coex_stat = &coex->stat; struct rtw_coex_dm *coex_dm = &coex->dm; struct rtw_efuse *efuse = &rtwdev->efuse; - struct rtw_chip_info *chip = rtwdev->chip; struct rtw_coex_rfe *coex_rfe = &coex->rfe; u8 table_case = 0xff, tdma_case = 0xff; @@ -1753,10 +1753,10 @@ exit: static void rtw_coex_action_bt_inquiry(struct rtw_dev *rtwdev) { + const struct rtw_chip_info *chip = rtwdev->chip; struct rtw_coex *coex = &rtwdev->coex; struct rtw_coex_stat *coex_stat = &coex->stat; struct rtw_efuse *efuse = &rtwdev->efuse; - struct rtw_chip_info *chip = rtwdev->chip; bool wl_hi_pri = false; u8 table_case, tdma_case; u32 slot_type = 0; @@ -1853,11 +1853,11 @@ static void rtw_coex_action_bt_inquiry(struct rtw_dev *rtwdev) static void rtw_coex_action_bt_game_hid(struct rtw_dev *rtwdev) { + const struct rtw_chip_info *chip = rtwdev->chip; struct rtw_coex *coex = &rtwdev->coex; struct rtw_coex_stat *coex_stat = &coex->stat; struct rtw_efuse *efuse = &rtwdev->efuse; struct rtw_coex_dm *coex_dm = &coex->dm; - struct rtw_chip_info *chip = rtwdev->chip; u8 table_case, tdma_case; rtw_dbg(rtwdev, RTW_DBG_COEX, "[BTCoex], %s()\n", __func__); @@ -1901,10 +1901,10 @@ static void rtw_coex_action_bt_game_hid(struct rtw_dev *rtwdev) static void rtw_coex_action_bt_hfp(struct rtw_dev *rtwdev) { + const struct rtw_chip_info *chip = rtwdev->chip; struct rtw_coex *coex = &rtwdev->coex; struct rtw_coex_stat *coex_stat = &coex->stat; struct rtw_efuse *efuse = &rtwdev->efuse; - struct rtw_chip_info *chip = rtwdev->chip; u8 table_case, tdma_case; rtw_dbg(rtwdev, RTW_DBG_COEX, "[BTCoex], %s()\n", __func__); @@ -1932,10 +1932,10 @@ static void rtw_coex_action_bt_hfp(struct rtw_dev *rtwdev) static void rtw_coex_action_bt_hid(struct rtw_dev *rtwdev) { + const struct rtw_chip_info *chip = rtwdev->chip; struct rtw_coex *coex = &rtwdev->coex; struct rtw_coex_stat *coex_stat = &coex->stat; struct rtw_efuse *efuse = &rtwdev->efuse; - struct rtw_chip_info *chip = rtwdev->chip; u8 table_case, tdma_case; u32 slot_type = 0; bool bt_multi_link_remain = false, is_toggle_table = false; @@ -2015,11 +2015,11 @@ static void rtw_coex_action_bt_hid(struct rtw_dev *rtwdev) static void rtw_coex_action_bt_a2dp(struct rtw_dev *rtwdev) { + const struct rtw_chip_info *chip = rtwdev->chip; struct rtw_coex *coex = &rtwdev->coex; struct rtw_coex_stat *coex_stat = &coex->stat; struct rtw_coex_dm *coex_dm = &coex->dm; struct rtw_efuse *efuse = &rtwdev->efuse; - struct rtw_chip_info *chip = rtwdev->chip; u8 table_case, tdma_case; u32 slot_type = 0; @@ -2057,10 +2057,10 @@ static void rtw_coex_action_bt_a2dp(struct rtw_dev *rtwdev) static void rtw_coex_action_bt_a2dpsink(struct rtw_dev *rtwdev) { + const struct rtw_chip_info *chip = rtwdev->chip; struct rtw_coex *coex = &rtwdev->coex; struct rtw_coex_stat *coex_stat = &coex->stat; struct rtw_efuse *efuse = &rtwdev->efuse; - struct rtw_chip_info *chip = rtwdev->chip; u8 table_case, tdma_case; bool ap_enable = false; @@ -2096,10 +2096,10 @@ static void rtw_coex_action_bt_a2dpsink(struct rtw_dev *rtwdev) static void rtw_coex_action_bt_pan(struct rtw_dev *rtwdev) { + const struct rtw_chip_info *chip = rtwdev->chip; struct rtw_coex *coex = &rtwdev->coex; struct rtw_coex_stat *coex_stat = &coex->stat; struct rtw_efuse *efuse = &rtwdev->efuse; - struct rtw_chip_info *chip = rtwdev->chip; u8 table_case, tdma_case; rtw_dbg(rtwdev, RTW_DBG_COEX, "[BTCoex], %s()\n", __func__); @@ -2133,11 +2133,11 @@ static void rtw_coex_action_bt_pan(struct rtw_dev *rtwdev) static void rtw_coex_action_bt_a2dp_hid(struct rtw_dev *rtwdev) { + const struct rtw_chip_info *chip = rtwdev->chip; struct rtw_coex *coex = &rtwdev->coex; struct rtw_coex_stat *coex_stat = &coex->stat; struct rtw_coex_dm *coex_dm = &coex->dm; struct rtw_efuse *efuse = &rtwdev->efuse; - struct rtw_chip_info *chip = rtwdev->chip; u8 table_case, tdma_case, interval = 0; u32 slot_type = 0; bool is_toggle_table = false; @@ -2190,10 +2190,10 @@ static void rtw_coex_action_bt_a2dp_hid(struct rtw_dev *rtwdev) static void rtw_coex_action_bt_a2dp_pan(struct rtw_dev *rtwdev) { + const struct rtw_chip_info *chip = rtwdev->chip; struct rtw_coex *coex = &rtwdev->coex; struct rtw_coex_stat *coex_stat = &coex->stat; struct rtw_efuse *efuse = &rtwdev->efuse; - struct rtw_chip_info *chip = rtwdev->chip; u8 table_case, tdma_case; bool wl_cpt_test = false, bt_cpt_test = false; @@ -2247,10 +2247,10 @@ static void rtw_coex_action_bt_a2dp_pan(struct rtw_dev *rtwdev) static void rtw_coex_action_bt_pan_hid(struct rtw_dev *rtwdev) { + const struct rtw_chip_info *chip = rtwdev->chip; struct rtw_coex *coex = &rtwdev->coex; struct rtw_coex_stat *coex_stat = &coex->stat; struct rtw_efuse *efuse = &rtwdev->efuse; - struct rtw_chip_info *chip = rtwdev->chip; u8 table_case, tdma_case; rtw_dbg(rtwdev, RTW_DBG_COEX, "[BTCoex], %s()\n", __func__); @@ -2282,10 +2282,10 @@ static void rtw_coex_action_bt_pan_hid(struct rtw_dev *rtwdev) static void rtw_coex_action_bt_a2dp_pan_hid(struct rtw_dev *rtwdev) { + const struct rtw_chip_info *chip = rtwdev->chip; struct rtw_coex *coex = &rtwdev->coex; struct rtw_coex_stat *coex_stat = &coex->stat; struct rtw_efuse *efuse = &rtwdev->efuse; - struct rtw_chip_info *chip = rtwdev->chip; u8 table_case, tdma_case; rtw_dbg(rtwdev, RTW_DBG_COEX, "[BTCoex], %s()\n", __func__); @@ -2316,9 +2316,9 @@ static void rtw_coex_action_bt_a2dp_pan_hid(struct rtw_dev *rtwdev) static void rtw_coex_action_wl_under5g(struct rtw_dev *rtwdev) { + const struct rtw_chip_info *chip = rtwdev->chip; struct rtw_coex *coex = &rtwdev->coex; struct rtw_efuse *efuse = &rtwdev->efuse; - struct rtw_chip_info *chip = rtwdev->chip; struct rtw_coex_stat *coex_stat = &coex->stat; u8 table_case, tdma_case; @@ -2348,8 +2348,8 @@ static void rtw_coex_action_wl_under5g(struct rtw_dev *rtwdev) static void rtw_coex_action_wl_only(struct rtw_dev *rtwdev) { + const struct rtw_chip_info *chip = rtwdev->chip; struct rtw_efuse *efuse = &rtwdev->efuse; - struct rtw_chip_info *chip = rtwdev->chip; u8 table_case, tdma_case; rtw_dbg(rtwdev, RTW_DBG_COEX, "[BTCoex], %s()\n", __func__); @@ -2372,9 +2372,9 @@ static void rtw_coex_action_wl_only(struct rtw_dev *rtwdev) static void rtw_coex_action_wl_native_lps(struct rtw_dev *rtwdev) { + const struct rtw_chip_info *chip = rtwdev->chip; struct rtw_coex *coex = &rtwdev->coex; struct rtw_efuse *efuse = &rtwdev->efuse; - struct rtw_chip_info *chip = rtwdev->chip; struct rtw_coex_stat *coex_stat = &coex->stat; u8 table_case, tdma_case; @@ -2411,10 +2411,10 @@ static void rtw_coex_action_wl_native_lps(struct rtw_dev *rtwdev) static void rtw_coex_action_wl_linkscan(struct rtw_dev *rtwdev) { + const struct rtw_chip_info *chip = rtwdev->chip; struct rtw_coex *coex = &rtwdev->coex; struct rtw_coex_stat *coex_stat = &coex->stat; struct rtw_efuse *efuse = &rtwdev->efuse; - struct rtw_chip_info *chip = rtwdev->chip; u8 table_case, tdma_case; u32 slot_type = 0; @@ -2451,8 +2451,8 @@ static void rtw_coex_action_wl_linkscan(struct rtw_dev *rtwdev) static void rtw_coex_action_wl_not_connected(struct rtw_dev *rtwdev) { + const struct rtw_chip_info *chip = rtwdev->chip; struct rtw_efuse *efuse = &rtwdev->efuse; - struct rtw_chip_info *chip = rtwdev->chip; u8 table_case, tdma_case; rtw_dbg(rtwdev, RTW_DBG_COEX, "[BTCoex], %s()\n", __func__); @@ -2528,8 +2528,8 @@ static void rtw_coex_action_wl_connected(struct rtw_dev *rtwdev) static void rtw_coex_run_coex(struct rtw_dev *rtwdev, u8 reason) { + const struct rtw_chip_info *chip = rtwdev->chip; struct rtw_coex *coex = &rtwdev->coex; - struct rtw_chip_info *chip = rtwdev->chip; struct rtw_coex_dm *coex_dm = &coex->dm; struct rtw_coex_stat *coex_stat = &coex->stat; bool rf4ce_en = false; @@ -3002,9 +3002,9 @@ void rtw_coex_media_status_notify(struct rtw_dev *rtwdev, u8 type) void rtw_coex_bt_info_notify(struct rtw_dev *rtwdev, u8 *buf, u8 length) { + const struct rtw_chip_info *chip = rtwdev->chip; struct rtw_coex *coex = &rtwdev->coex; struct rtw_coex_stat *coex_stat = &coex->stat; - struct rtw_chip_info *chip = rtwdev->chip; struct rtw_coex_dm *coex_dm = &coex->dm; u32 bt_relink_time; u8 i, rsp_source = 0, type; @@ -3270,8 +3270,8 @@ static const u8 coex_bt_hidinfo_xb[] = {0x58, 0x62, 0x6f}; void rtw_coex_bt_hid_info_notify(struct rtw_dev *rtwdev, u8 *buf, u8 length) { + const struct rtw_chip_info *chip = rtwdev->chip; struct rtw_coex *coex = &rtwdev->coex; - struct rtw_chip_info *chip = rtwdev->chip; struct rtw_coex_stat *coex_stat = &coex->stat; struct rtw_coex_hid *hidinfo; struct rtw_coex_hid_info_a *hida; @@ -3360,8 +3360,8 @@ void rtw_coex_bt_hid_info_notify(struct rtw_dev *rtwdev, u8 *buf, u8 length) void rtw_coex_query_bt_hid_list(struct rtw_dev *rtwdev) { + const struct rtw_chip_info *chip = rtwdev->chip; struct rtw_coex *coex = &rtwdev->coex; - struct rtw_chip_info *chip = rtwdev->chip; struct rtw_coex_stat *coex_stat = &coex->stat; struct rtw_coex_hid *hidinfo; u8 i, handle; @@ -3582,7 +3582,7 @@ static const char *rtw_coex_get_reason_string(u8 reason) static u8 rtw_coex_get_table_index(struct rtw_dev *rtwdev, u32 wl_reg_6c0, u32 wl_reg_6c4) { - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; struct rtw_efuse *efuse = &rtwdev->efuse; u8 ans = 0xFF; u8 n, i; @@ -3618,8 +3618,8 @@ static u8 rtw_coex_get_table_index(struct rtw_dev *rtwdev, u32 wl_reg_6c0, static u8 rtw_coex_get_tdma_index(struct rtw_dev *rtwdev, u8 *tdma_para) { + const struct rtw_chip_info *chip = rtwdev->chip; struct rtw_efuse *efuse = &rtwdev->efuse; - struct rtw_chip_info *chip = rtwdev->chip; u8 ans = 0xFF; u8 n, i, j; u8 load_cur_tab_val; @@ -3736,7 +3736,7 @@ static int rtw_coex_val_info(struct rtw_dev *rtwdev, static void rtw_coex_set_coexinfo_hw(struct rtw_dev *rtwdev, struct seq_file *m) { - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; const struct rtw_reg_domain *reg; char addr_info[INFO_SIZE]; int n_addr = 0; @@ -3910,7 +3910,7 @@ static const char *rtw_coex_get_wl_coex_mode(u8 coex_wl_link_mode) void rtw_coex_display_coex_info(struct rtw_dev *rtwdev, struct seq_file *m) { - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; struct rtw_dm_info *dm_info = &rtwdev->dm_info; struct rtw_coex *coex = &rtwdev->coex; struct rtw_coex_stat *coex_stat = &coex->stat; diff --git a/drivers/net/wireless/realtek/rtw88/coex.h b/drivers/net/wireless/realtek/rtw88/coex.h index 07fa7aa34d4b..57cf29da9ea4 100644 --- a/drivers/net/wireless/realtek/rtw88/coex.h +++ b/drivers/net/wireless/realtek/rtw88/coex.h @@ -327,7 +327,7 @@ struct coex_rf_para { static inline void rtw_coex_set_init(struct rtw_dev *rtwdev) { - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; chip->ops->coex_set_init(rtwdev); } @@ -335,7 +335,7 @@ static inline void rtw_coex_set_init(struct rtw_dev *rtwdev) static inline void rtw_coex_set_ant_switch(struct rtw_dev *rtwdev, u8 ctrl_type, u8 pos_type) { - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; if (!chip->ops->coex_set_ant_switch) return; @@ -345,28 +345,28 @@ void rtw_coex_set_ant_switch(struct rtw_dev *rtwdev, u8 ctrl_type, u8 pos_type) static inline void rtw_coex_set_gnt_fix(struct rtw_dev *rtwdev) { - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; chip->ops->coex_set_gnt_fix(rtwdev); } static inline void rtw_coex_set_gnt_debug(struct rtw_dev *rtwdev) { - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; chip->ops->coex_set_gnt_debug(rtwdev); } static inline void rtw_coex_set_rfe_type(struct rtw_dev *rtwdev) { - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; chip->ops->coex_set_rfe_type(rtwdev); } static inline void rtw_coex_set_wl_tx_power(struct rtw_dev *rtwdev, u8 wl_pwr) { - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; chip->ops->coex_set_wl_tx_power(rtwdev, wl_pwr); } @@ -374,7 +374,7 @@ static inline void rtw_coex_set_wl_tx_power(struct rtw_dev *rtwdev, u8 wl_pwr) static inline void rtw_coex_set_wl_rx_gain(struct rtw_dev *rtwdev, bool low_gain) { - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; chip->ops->coex_set_wl_rx_gain(rtwdev, low_gain); } diff --git a/drivers/net/wireless/realtek/rtw88/debug.c b/drivers/net/wireless/realtek/rtw88/debug.c index 7cde6bcf253b..9ebe544e51d0 100644 --- a/drivers/net/wireless/realtek/rtw88/debug.c +++ b/drivers/net/wireless/realtek/rtw88/debug.c @@ -621,11 +621,13 @@ static int rtw_debugfs_get_tx_pwr_tbl(struct seq_file *m, void *v) struct rtw_debugfs_priv *debugfs_priv = m->private; struct rtw_dev *rtwdev = debugfs_priv->rtwdev; struct rtw_hal *hal = &rtwdev->hal; - u8 path, rate; + u8 path, rate, bw, ch, regd; struct rtw_power_params pwr_param = {0}; - u8 bw = hal->current_band_width; - u8 ch = hal->current_channel; - u8 regd = rtw_regd_get(rtwdev); + + mutex_lock(&rtwdev->mutex); + bw = hal->current_band_width; + ch = hal->current_channel; + regd = rtw_regd_get(rtwdev); seq_printf(m, "channel: %u\n", ch); seq_printf(m, "bandwidth: %u\n", bw); @@ -667,6 +669,7 @@ static int rtw_debugfs_get_tx_pwr_tbl(struct seq_file *m, void *v) } mutex_unlock(&hal->tx_power_mutex); + mutex_unlock(&rtwdev->mutex); return 0; } diff --git a/drivers/net/wireless/realtek/rtw88/efuse.c b/drivers/net/wireless/realtek/rtw88/efuse.c index c266c84ef233..b85075cd68d0 100644 --- a/drivers/net/wireless/realtek/rtw88/efuse.c +++ b/drivers/net/wireless/realtek/rtw88/efuse.c @@ -86,7 +86,7 @@ static int rtw_dump_logical_efuse_map(struct rtw_dev *rtwdev, u8 *phy_map, static int rtw_dump_physical_efuse_map(struct rtw_dev *rtwdev, u8 *map) { - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; u32 size = rtwdev->efuse.physical_size; u32 efuse_ctl; u32 addr; @@ -145,7 +145,7 @@ EXPORT_SYMBOL(rtw_read8_physical_efuse); int rtw_parse_efuse_map(struct rtw_dev *rtwdev) { - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; struct rtw_efuse *efuse = &rtwdev->efuse; u32 phy_size = efuse->physical_size; u32 log_size = efuse->logical_size; diff --git a/drivers/net/wireless/realtek/rtw88/fw.c b/drivers/net/wireless/realtek/rtw88/fw.c index 4fdab0329695..0b5f903c0f36 100644 --- a/drivers/net/wireless/realtek/rtw88/fw.c +++ b/drivers/net/wireless/realtek/rtw88/fw.c @@ -14,6 +14,8 @@ #include "util.h" #include "wow.h" #include "ps.h" +#include "phy.h" +#include "mac.h" static void rtw_fw_c2h_cmd_handle_ext(struct rtw_dev *rtwdev, struct sk_buff *skb) @@ -116,7 +118,7 @@ legacy: si->ra_report.desc_rate = rate; si->ra_report.bit_rate = bit_rate; - sta->max_rc_amsdu_len = get_max_amsdu_len(bit_rate); + sta->deflink.agg.max_rc_amsdu_len = get_max_amsdu_len(bit_rate); } static void rtw_fw_ra_report_handle(struct rtw_dev *rtwdev, u8 *payload, @@ -904,7 +906,7 @@ void rtw_send_rsvd_page_h2c(struct rtw_dev *rtwdev) static struct sk_buff *rtw_nlo_info_get(struct ieee80211_hw *hw) { struct rtw_dev *rtwdev = hw->priv; - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; struct rtw_pno_request *pno_req = &rtwdev->wow.pno_req; struct rtw_nlo_info_hdr *nlo_hdr; struct cfg80211_ssid *ssid; @@ -959,7 +961,7 @@ static struct sk_buff *rtw_nlo_info_get(struct ieee80211_hw *hw) static struct sk_buff *rtw_cs_channel_info_get(struct ieee80211_hw *hw) { struct rtw_dev *rtwdev = hw->priv; - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; struct rtw_pno_request *pno_req = &rtwdev->wow.pno_req; struct ieee80211_channel *channels = pno_req->channels; struct sk_buff *skb; @@ -993,7 +995,7 @@ static struct sk_buff *rtw_cs_channel_info_get(struct ieee80211_hw *hw) static struct sk_buff *rtw_lps_pg_dpk_get(struct ieee80211_hw *hw) { struct rtw_dev *rtwdev = hw->priv; - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; struct rtw_dpk_info *dpk_info = &rtwdev->dm_info.dpk_info; struct rtw_lps_pg_dpk_hdr *dpk_hdr; struct sk_buff *skb; @@ -1018,7 +1020,7 @@ static struct sk_buff *rtw_lps_pg_dpk_get(struct ieee80211_hw *hw) static struct sk_buff *rtw_lps_pg_info_get(struct ieee80211_hw *hw) { struct rtw_dev *rtwdev = hw->priv; - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; struct rtw_lps_conf *conf = &rtwdev->lps_conf; struct rtw_lps_pg_info_hdr *pg_info_hdr; struct rtw_wow_param *rtw_wow = &rtwdev->wow; @@ -1080,10 +1082,10 @@ static struct sk_buff *rtw_get_rsvd_page_skb(struct ieee80211_hw *hw, skb_new = ieee80211_proberesp_get(hw, vif); break; case RSVD_NULL: - skb_new = ieee80211_nullfunc_get(hw, vif, false); + skb_new = ieee80211_nullfunc_get(hw, vif, -1, false); break; case RSVD_QOS_NULL: - skb_new = ieee80211_nullfunc_get(hw, vif, true); + skb_new = ieee80211_nullfunc_get(hw, vif, -1, true); break; case RSVD_LPS_PG_DPK: skb_new = rtw_lps_pg_dpk_get(hw); @@ -1122,7 +1124,7 @@ static void rtw_fill_rsvd_page_desc(struct rtw_dev *rtwdev, struct sk_buff *skb, enum rtw_rsvd_packet_type type) { struct rtw_tx_pkt_info pkt_info = {0}; - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; u8 *pkt_desc; rtw_tx_rsvd_page_pkt_info_update(rtwdev, &pkt_info, skb, type); @@ -1433,7 +1435,7 @@ static int __rtw_build_rsvd_page_from_vifs(struct rtw_dev *rtwdev) static u8 *rtw_build_rsvd_page(struct rtw_dev *rtwdev, u32 *size) { struct ieee80211_hw *hw = rtwdev->hw; - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; struct sk_buff *iter; struct rtw_rsvd_page *rsvd_pkt; u32 page = 0; @@ -1647,7 +1649,7 @@ out: static void rtw_fw_read_fifo(struct rtw_dev *rtwdev, enum rtw_fw_fifo_sel sel, u32 offset, u32 size, u32 *buf) { - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; u32 start_pg, residue; if (sel >= RTW_FW_FIFO_MAX) { @@ -1706,7 +1708,7 @@ int rtw_fw_dump_fifo(struct rtw_dev *rtwdev, u8 fifo_sel, u32 addr, u32 size, static void __rtw_fw_update_pkt(struct rtw_dev *rtwdev, u8 pkt_id, u16 size, u8 location) { - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; u8 h2c_pkt[H2C_PKT_SIZE] = {0}; u16 total_size = H2C_PKT_HDR_SIZE + H2C_PKT_UPDATE_PKT_LEN; @@ -1818,8 +1820,8 @@ static int rtw_append_probe_req_ie(struct rtw_dev *rtwdev, struct sk_buff *skb, struct sk_buff_head *list, u8 *bands, struct rtw_vif *rtwvif) { + const struct rtw_chip_info *chip = rtwdev->chip; struct ieee80211_scan_ies *ies = rtwvif->scan_ies; - struct rtw_chip_info *chip = rtwdev->chip; struct sk_buff *new; u8 idx; @@ -1841,16 +1843,23 @@ static int rtw_append_probe_req_ie(struct rtw_dev *rtwdev, struct sk_buff *skb, static int _rtw_hw_scan_update_probe_req(struct rtw_dev *rtwdev, u8 num_probes, struct sk_buff_head *probe_req_list) { - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; struct sk_buff *skb, *tmp; u8 page_offset = 1, *buf, page_size = chip->page_size; - u8 pages = page_offset + num_probes * RTW_PROBE_PG_CNT; u16 pg_addr = rtwdev->fifo.rsvd_h2c_info_addr, loc; u16 buf_offset = page_size * page_offset; u8 tx_desc_sz = chip->tx_pkt_desc_sz; + u8 page_cnt, pages; unsigned int pkt_len; int ret; + if (rtw_fw_feature_ext_check(&rtwdev->fw, FW_FEATURE_EXT_OLD_PAGE_NUM)) + page_cnt = RTW_OLD_PROBE_PG_CNT; + else + page_cnt = RTW_PROBE_PG_CNT; + + pages = page_offset + num_probes * page_cnt; + buf = kzalloc(page_size * pages, GFP_KERNEL); if (!buf) return -ENOMEM; @@ -1859,7 +1868,7 @@ static int _rtw_hw_scan_update_probe_req(struct rtw_dev *rtwdev, u8 num_probes, skb_queue_walk_safe(probe_req_list, skb, tmp) { skb_unlink(skb, probe_req_list); rtw_fill_rsvd_page_desc(rtwdev, skb, RSVD_PROBE_REQ); - if (skb->len > page_size * RTW_PROBE_PG_CNT) { + if (skb->len > page_size * page_cnt) { ret = -EINVAL; goto out; } @@ -1869,8 +1878,8 @@ static int _rtw_hw_scan_update_probe_req(struct rtw_dev *rtwdev, u8 num_probes, loc = pg_addr - rtwdev->fifo.rsvd_boundary + page_offset; __rtw_fw_update_pkt(rtwdev, RTW_PACKET_PROBE_REQ, pkt_len, loc); - buf_offset += RTW_PROBE_PG_CNT * page_size; - page_offset += RTW_PROBE_PG_CNT; + buf_offset += page_cnt * page_size; + page_offset += page_cnt; kfree_skb(skb); } @@ -2048,6 +2057,9 @@ void rtw_hw_scan_start(struct rtw_dev *rtwdev, struct ieee80211_vif *vif, rtwvif->scan_req = req; ieee80211_stop_queues(rtwdev->hw); + rtw_leave_lps_deep(rtwdev); + rtw_hci_flush_all_queues(rtwdev, false); + rtw_mac_flush_all_queues(rtwdev, false); if (req->flags & NL80211_SCAN_FLAG_RANDOM_ADDR) get_random_mask_addr(mac_addr, req->mac_addr, req->mac_addr_mask); @@ -2080,10 +2092,9 @@ void rtw_hw_scan_complete(struct rtw_dev *rtwdev, struct ieee80211_vif *vif, rtw_core_scan_complete(rtwdev, vif, true); rtwvif = (struct rtw_vif *)vif->drv_priv; - if (rtwvif->net_type == RTW_NET_MGD_LINKED) { - hal->current_channel = chan; - hal->current_band_type = chan > 14 ? RTW_BAND_5G : RTW_BAND_2G; - } + if (chan) + rtw_store_op_chan(rtwdev, false); + rtw_phy_set_tx_power_level(rtwdev, hal->current_channel); ieee80211_wake_queues(rtwdev->hw); ieee80211_scan_completed(rtwdev->hw, &info); @@ -2124,6 +2135,7 @@ int rtw_hw_scan_offload(struct rtw_dev *rtwdev, struct ieee80211_vif *vif, bool enable) { struct rtw_vif *rtwvif = vif ? (struct rtw_vif *)vif->drv_priv : NULL; + struct rtw_hw_scan_info *scan_info = &rtwdev->scan_info; struct rtw_ch_switch_option cs_option = {0}; struct rtw_chan_list chan_list = {0}; int ret = 0; @@ -2132,7 +2144,7 @@ int rtw_hw_scan_offload(struct rtw_dev *rtwdev, struct ieee80211_vif *vif, return -EINVAL; cs_option.switch_en = enable; - cs_option.back_op_en = rtwvif->net_type == RTW_NET_MGD_LINKED; + cs_option.back_op_en = scan_info->op_chan != 0; if (enable) { ret = rtw_hw_scan_prehandle(rtwdev, rtwvif, &chan_list); if (ret) @@ -2171,14 +2183,33 @@ void rtw_hw_scan_status_report(struct rtw_dev *rtwdev, struct sk_buff *skb) rtw_dbg(rtwdev, RTW_DBG_HW_SCAN, "HW scan aborted with code: %d\n", rc); } -void rtw_store_op_chan(struct rtw_dev *rtwdev) +void rtw_store_op_chan(struct rtw_dev *rtwdev, bool backup) { struct rtw_hw_scan_info *scan_info = &rtwdev->scan_info; struct rtw_hal *hal = &rtwdev->hal; + u8 band; + + if (backup) { + scan_info->op_chan = hal->current_channel; + scan_info->op_bw = hal->current_band_width; + scan_info->op_pri_ch_idx = hal->current_primary_channel_index; + scan_info->op_pri_ch = hal->primary_channel; + } else { + band = scan_info->op_chan > 14 ? RTW_BAND_5G : RTW_BAND_2G; + rtw_update_channel(rtwdev, scan_info->op_chan, + scan_info->op_pri_ch, + band, scan_info->op_bw); + } +} - scan_info->op_chan = hal->current_channel; - scan_info->op_bw = hal->current_band_width; - scan_info->op_pri_ch_idx = hal->current_primary_channel_index; +void rtw_clear_op_chan(struct rtw_dev *rtwdev) +{ + struct rtw_hw_scan_info *scan_info = &rtwdev->scan_info; + + scan_info->op_chan = 0; + scan_info->op_bw = 0; + scan_info->op_pri_ch_idx = 0; + scan_info->op_pri_ch = 0; } static bool rtw_is_op_chan(struct rtw_dev *rtwdev, u8 channel) @@ -2193,7 +2224,7 @@ void rtw_hw_scan_chan_switch(struct rtw_dev *rtwdev, struct sk_buff *skb) struct rtw_hal *hal = &rtwdev->hal; struct rtw_c2h_cmd *c2h; enum rtw_scan_notify_id id; - u8 chan, status; + u8 chan, band, status; if (!test_bit(RTW_FLAG_SCANNING, rtwdev->flags)) return; @@ -2204,10 +2235,13 @@ void rtw_hw_scan_chan_switch(struct rtw_dev *rtwdev, struct sk_buff *skb) status = GET_CHAN_SWITCH_STATUS(c2h->payload); if (id == RTW_SCAN_NOTIFY_ID_POSTSWITCH) { - if (rtw_is_op_chan(rtwdev, chan)) + band = chan > 14 ? RTW_BAND_5G : RTW_BAND_2G; + rtw_update_channel(rtwdev, chan, chan, band, + RTW_CHANNEL_WIDTH_20); + if (rtw_is_op_chan(rtwdev, chan)) { + rtw_store_op_chan(rtwdev, false); ieee80211_wake_queues(rtwdev->hw); - hal->current_channel = chan; - hal->current_band_type = chan > 14 ? RTW_BAND_5G : RTW_BAND_2G; + } } else if (id == RTW_SCAN_NOTIFY_ID_PRESWITCH) { if (IS_CH_5G_BAND(chan)) { rtw_coex_switchband_notify(rtwdev, COEX_SWITCH_TO_5G); @@ -2220,7 +2254,12 @@ void rtw_hw_scan_chan_switch(struct rtw_dev *rtwdev, struct sk_buff *skb) chan_type = COEX_SWITCH_TO_24G_NOFORSCAN; rtw_coex_switchband_notify(rtwdev, chan_type); } - if (rtw_is_op_chan(rtwdev, chan)) + /* The channel of C2H RTW_SCAN_NOTIFY_ID_PRESWITCH is next + * channel that hardware will switch. We need to stop queue + * if next channel is non-op channel. + */ + if (!rtw_is_op_chan(rtwdev, chan) && + rtw_is_op_chan(rtwdev, hal->current_channel)) ieee80211_stop_queues(rtwdev->hw); } diff --git a/drivers/net/wireless/realtek/rtw88/fw.h b/drivers/net/wireless/realtek/rtw88/fw.h index 7a37675c61e8..a5a965803a3c 100644 --- a/drivers/net/wireless/realtek/rtw88/fw.h +++ b/drivers/net/wireless/realtek/rtw88/fw.h @@ -41,7 +41,8 @@ #define RTW_EX_CH_INFO_HDR_SIZE 2 #define RTW_SCAN_WIDTH 0 #define RTW_PRI_CH_IDX 1 -#define RTW_PROBE_PG_CNT 2 +#define RTW_OLD_PROBE_PG_CNT 2 +#define RTW_PROBE_PG_CNT 4 enum rtw_c2h_cmd_id { C2H_CCX_TX_RPT = 0x03, @@ -120,6 +121,10 @@ enum rtw_fw_feature { FW_FEATURE_MAX = BIT(31), }; +enum rtw_fw_feature_ext { + FW_FEATURE_EXT_OLD_PAGE_NUM = BIT(0), +}; + enum rtw_beacon_filter_offload_mode { BCN_FILTER_OFFLOAD_MODE_0 = 0, BCN_FILTER_OFFLOAD_MODE_1, @@ -323,6 +328,11 @@ struct rtw_fw_hdr_legacy { __le32 rsvd5; } __packed; +#define RTW_FW_VER_CODE(ver, sub_ver, idx) \ + (((ver) << 16) | ((sub_ver) << 8) | (idx)) +#define RTW_FW_SUIT_VER_CODE(s) \ + RTW_FW_VER_CODE((s).version, (s).sub_version, (s).sub_index) + /* C2H */ #define GET_CCX_REPORT_SEQNUM_V0(c2h_payload) (c2h_payload[6] & 0xfc) #define GET_CCX_REPORT_STATUS_V0(c2h_payload) (c2h_payload[0] & 0xc0) @@ -770,6 +780,12 @@ static inline bool rtw_fw_feature_check(struct rtw_fw_state *fw, return !!(fw->feature & feature); } +static inline bool rtw_fw_feature_ext_check(struct rtw_fw_state *fw, + enum rtw_fw_feature_ext feature) +{ + return !!(fw->feature_ext & feature); +} + void rtw_fw_c2h_cmd_rx_irqsafe(struct rtw_dev *rtwdev, u32 pkt_offset, struct sk_buff *skb); void rtw_fw_c2h_cmd_handle(struct rtw_dev *rtwdev, struct sk_buff *skb); @@ -831,7 +847,8 @@ int rtw_fw_dump_fifo(struct rtw_dev *rtwdev, u8 fifo_sel, u32 addr, u32 size, u32 *buffer); void rtw_fw_scan_notify(struct rtw_dev *rtwdev, bool start); void rtw_fw_adaptivity(struct rtw_dev *rtwdev); -void rtw_store_op_chan(struct rtw_dev *rtwdev); +void rtw_store_op_chan(struct rtw_dev *rtwdev, bool backup); +void rtw_clear_op_chan(struct rtw_dev *rtwdev); void rtw_hw_scan_start(struct rtw_dev *rtwdev, struct ieee80211_vif *vif, struct ieee80211_scan_request *req); void rtw_hw_scan_complete(struct rtw_dev *rtwdev, struct ieee80211_vif *vif, diff --git a/drivers/net/wireless/realtek/rtw88/mac.c b/drivers/net/wireless/realtek/rtw88/mac.c index caf2603da2d6..52076e89d59a 100644 --- a/drivers/net/wireless/realtek/rtw88/mac.c +++ b/drivers/net/wireless/realtek/rtw88/mac.c @@ -243,7 +243,7 @@ static int rtw_pwr_seq_parser(struct rtw_dev *rtwdev, static int rtw_mac_power_switch(struct rtw_dev *rtwdev, bool pwr_on) { - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; const struct rtw_pwr_seq_cmd **pwr_seq; u8 rpwm; bool cur_pwr; @@ -587,7 +587,7 @@ static int download_firmware_to_mem(struct rtw_dev *rtwdev, const u8 *data, u32 src, u32 dst, u32 size) { - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; u32 desc_size = chip->tx_pkt_desc_sz; u8 first_part; u32 mem_offset; @@ -934,7 +934,7 @@ static u32 get_priority_queues(struct rtw_dev *rtwdev, u32 queues) static void __rtw_mac_flush_prio_queue(struct rtw_dev *rtwdev, u32 prio_queue, bool drop) { - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; const struct rtw_prioq_addr *addr; bool wsize; u16 avail_page, rsvd_page; @@ -996,7 +996,7 @@ void rtw_mac_flush_queues(struct rtw_dev *rtwdev, u32 queues, bool drop) static int txdma_queue_mapping(struct rtw_dev *rtwdev) { - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; const struct rtw_rqpn *rqpn = NULL; u16 txdma_pq_map = 0; @@ -1037,8 +1037,8 @@ static int txdma_queue_mapping(struct rtw_dev *rtwdev) static int set_trx_fifo_info(struct rtw_dev *rtwdev) { + const struct rtw_chip_info *chip = rtwdev->chip; struct rtw_fifo_conf *fifo = &rtwdev->fifo; - struct rtw_chip_info *chip = rtwdev->chip; u16 cur_pg_addr; u8 csi_buf_pg_num = chip->csi_buf_pg_num; @@ -1092,8 +1092,8 @@ static int __priority_queue_cfg(struct rtw_dev *rtwdev, const struct rtw_page_table *pg_tbl, u16 pubq_num) { + const struct rtw_chip_info *chip = rtwdev->chip; struct rtw_fifo_conf *fifo = &rtwdev->fifo; - struct rtw_chip_info *chip = rtwdev->chip; rtw_write16(rtwdev, REG_FIFOPAGE_INFO_1, pg_tbl->hq_num); rtw_write16(rtwdev, REG_FIFOPAGE_INFO_2, pg_tbl->lq_num); @@ -1123,8 +1123,8 @@ static int __priority_queue_cfg_legacy(struct rtw_dev *rtwdev, const struct rtw_page_table *pg_tbl, u16 pubq_num) { + const struct rtw_chip_info *chip = rtwdev->chip; struct rtw_fifo_conf *fifo = &rtwdev->fifo; - struct rtw_chip_info *chip = rtwdev->chip; u32 val32; val32 = BIT_RQPN_NE(pg_tbl->nq_num, pg_tbl->exq_num); @@ -1149,8 +1149,8 @@ static int __priority_queue_cfg_legacy(struct rtw_dev *rtwdev, static int priority_queue_cfg(struct rtw_dev *rtwdev) { + const struct rtw_chip_info *chip = rtwdev->chip; struct rtw_fifo_conf *fifo = &rtwdev->fifo; - struct rtw_chip_info *chip = rtwdev->chip; const struct rtw_page_table *pg_tbl = NULL; u16 pubq_num; int ret; @@ -1277,7 +1277,7 @@ static int rtw_drv_info_cfg(struct rtw_dev *rtwdev) int rtw_mac_init(struct rtw_dev *rtwdev) { - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; int ret; ret = rtw_init_trx_cfg(rtwdev); diff --git a/drivers/net/wireless/realtek/rtw88/mac80211.c b/drivers/net/wireless/realtek/rtw88/mac80211.c index c7b98a0599d5..07578ccc4bab 100644 --- a/drivers/net/wireless/realtek/rtw88/mac80211.c +++ b/drivers/net/wireless/realtek/rtw88/mac80211.c @@ -101,7 +101,8 @@ static int rtw_ops_config(struct ieee80211_hw *hw, u32 changed) rtw_set_channel(rtwdev); if ((changed & IEEE80211_CONF_CHANGE_IDLE) && - (hw->conf.flags & IEEE80211_CONF_IDLE)) + (hw->conf.flags & IEEE80211_CONF_IDLE) && + !test_bit(RTW_FLAG_SCANNING, rtwdev->flags)) rtw_enter_ips(rtwdev); out: @@ -377,7 +378,6 @@ static void rtw_ops_bss_info_changed(struct ieee80211_hw *hw, rtw_coex_media_status_notify(rtwdev, vif->cfg.assoc); if (rtw_bf_support) rtw_bf_assoc(rtwdev, vif, conf); - rtw_store_op_chan(rtwdev); } else { rtw_leave_lps(rtwdev); rtw_bf_disassoc(rtwdev, vif, conf); @@ -395,6 +395,10 @@ static void rtw_ops_bss_info_changed(struct ieee80211_hw *hw, if (changed & BSS_CHANGED_BSSID) { ether_addr_copy(rtwvif->bssid, conf->bssid); config |= PORT_SET_BSSID; + if (is_zero_ether_addr(rtwvif->bssid)) + rtw_clear_op_chan(rtwdev); + else + rtw_store_op_chan(rtwdev, true); } if (changed & BSS_CHANGED_BEACON_INT) { @@ -434,7 +438,7 @@ static int rtw_ops_start_ap(struct ieee80211_hw *hw, struct ieee80211_bss_conf *link_conf) { struct rtw_dev *rtwdev = hw->priv; - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; mutex_lock(&rtwdev->mutex); chip->ops->phy_calibration(rtwdev); @@ -752,7 +756,7 @@ static int rtw_ops_set_antenna(struct ieee80211_hw *hw, u32 rx_antenna) { struct rtw_dev *rtwdev = hw->priv; - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; int ret; if (!chip->ops->set_antenna) @@ -872,7 +876,9 @@ static int rtw_ops_set_sar_specs(struct ieee80211_hw *hw, { struct rtw_dev *rtwdev = hw->priv; + mutex_lock(&rtwdev->mutex); rtw_set_sar_specs(rtwdev, sar); + mutex_unlock(&rtwdev->mutex); return 0; } diff --git a/drivers/net/wireless/realtek/rtw88/main.c b/drivers/net/wireless/realtek/rtw88/main.c index 76dc9da88f6c..67151dbf8384 100644 --- a/drivers/net/wireless/realtek/rtw88/main.c +++ b/drivers/net/wireless/realtek/rtw88/main.c @@ -353,7 +353,7 @@ struct rtw_fwcd_hdr { static int rtw_fwcd_prep(struct rtw_dev *rtwdev) { - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; struct rtw_fwcd_desc *desc = &rtwdev->fw.fwcd_desc; const struct rtw_fwcd_segs *segs = chip->fwcd_segs; u32 prep_size = chip->fw_rxff_size + sizeof(struct rtw_fwcd_hdr); @@ -675,67 +675,126 @@ void rtw_set_dtim_period(struct rtw_dev *rtwdev, int dtim_period) rtw_write8(rtwdev, REG_DTIM_COUNTER_ROOT, dtim_period - 1); } +void rtw_update_channel(struct rtw_dev *rtwdev, u8 center_channel, + u8 primary_channel, enum rtw_supported_band band, + enum rtw_bandwidth bandwidth) +{ + enum nl80211_band nl_band = rtw_hw_to_nl80211_band(band); + struct rtw_hal *hal = &rtwdev->hal; + u8 *cch_by_bw = hal->cch_by_bw; + u32 center_freq, primary_freq; + enum rtw_sar_bands sar_band; + u8 primary_channel_idx; + + center_freq = ieee80211_channel_to_frequency(center_channel, nl_band); + primary_freq = ieee80211_channel_to_frequency(primary_channel, nl_band); + + /* assign the center channel used while 20M bw is selected */ + cch_by_bw[RTW_CHANNEL_WIDTH_20] = primary_channel; + + /* assign the center channel used while current bw is selected */ + cch_by_bw[bandwidth] = center_channel; + + switch (bandwidth) { + case RTW_CHANNEL_WIDTH_20: + default: + primary_channel_idx = RTW_SC_DONT_CARE; + break; + case RTW_CHANNEL_WIDTH_40: + if (primary_freq > center_freq) + primary_channel_idx = RTW_SC_20_UPPER; + else + primary_channel_idx = RTW_SC_20_LOWER; + break; + case RTW_CHANNEL_WIDTH_80: + if (primary_freq > center_freq) { + if (primary_freq - center_freq == 10) + primary_channel_idx = RTW_SC_20_UPPER; + else + primary_channel_idx = RTW_SC_20_UPMOST; + + /* assign the center channel used + * while 40M bw is selected + */ + cch_by_bw[RTW_CHANNEL_WIDTH_40] = center_channel + 4; + } else { + if (center_freq - primary_freq == 10) + primary_channel_idx = RTW_SC_20_LOWER; + else + primary_channel_idx = RTW_SC_20_LOWEST; + + /* assign the center channel used + * while 40M bw is selected + */ + cch_by_bw[RTW_CHANNEL_WIDTH_40] = center_channel - 4; + } + break; + } + + switch (center_channel) { + case 1 ... 14: + sar_band = RTW_SAR_BAND_0; + break; + case 36 ... 64: + sar_band = RTW_SAR_BAND_1; + break; + case 100 ... 144: + sar_band = RTW_SAR_BAND_3; + break; + case 149 ... 177: + sar_band = RTW_SAR_BAND_4; + break; + default: + WARN(1, "unknown ch(%u) to SAR band\n", center_channel); + sar_band = RTW_SAR_BAND_0; + break; + } + + hal->current_primary_channel_index = primary_channel_idx; + hal->current_band_width = bandwidth; + hal->primary_channel = primary_channel; + hal->current_channel = center_channel; + hal->current_band_type = band; + hal->sar_band = sar_band; +} + void rtw_get_channel_params(struct cfg80211_chan_def *chandef, struct rtw_channel_params *chan_params) { struct ieee80211_channel *channel = chandef->chan; enum nl80211_chan_width width = chandef->width; - u8 *cch_by_bw = chan_params->cch_by_bw; u32 primary_freq, center_freq; u8 center_chan; u8 bandwidth = RTW_CHANNEL_WIDTH_20; - u8 primary_chan_idx = 0; - u8 i; center_chan = channel->hw_value; primary_freq = channel->center_freq; center_freq = chandef->center_freq1; - /* assign the center channel used while 20M bw is selected */ - cch_by_bw[RTW_CHANNEL_WIDTH_20] = channel->hw_value; - switch (width) { case NL80211_CHAN_WIDTH_20_NOHT: case NL80211_CHAN_WIDTH_20: bandwidth = RTW_CHANNEL_WIDTH_20; - primary_chan_idx = RTW_SC_DONT_CARE; break; case NL80211_CHAN_WIDTH_40: bandwidth = RTW_CHANNEL_WIDTH_40; - if (primary_freq > center_freq) { - primary_chan_idx = RTW_SC_20_UPPER; + if (primary_freq > center_freq) center_chan -= 2; - } else { - primary_chan_idx = RTW_SC_20_LOWER; + else center_chan += 2; - } break; case NL80211_CHAN_WIDTH_80: bandwidth = RTW_CHANNEL_WIDTH_80; if (primary_freq > center_freq) { - if (primary_freq - center_freq == 10) { - primary_chan_idx = RTW_SC_20_UPPER; + if (primary_freq - center_freq == 10) center_chan -= 2; - } else { - primary_chan_idx = RTW_SC_20_UPMOST; + else center_chan -= 6; - } - /* assign the center channel used - * while 40M bw is selected - */ - cch_by_bw[RTW_CHANNEL_WIDTH_40] = center_chan + 4; } else { - if (center_freq - primary_freq == 10) { - primary_chan_idx = RTW_SC_20_LOWER; + if (center_freq - primary_freq == 10) center_chan += 2; - } else { - primary_chan_idx = RTW_SC_20_LOWEST; + else center_chan += 6; - } - /* assign the center channel used - * while 40M bw is selected - */ - cch_by_bw[RTW_CHANNEL_WIDTH_40] = center_chan - 4; } break; default: @@ -745,60 +804,30 @@ void rtw_get_channel_params(struct cfg80211_chan_def *chandef, chan_params->center_chan = center_chan; chan_params->bandwidth = bandwidth; - chan_params->primary_chan_idx = primary_chan_idx; - - /* assign the center channel used while current bw is selected */ - cch_by_bw[bandwidth] = center_chan; - - for (i = bandwidth + 1; i <= RTW_MAX_CHANNEL_WIDTH; i++) - cch_by_bw[i] = 0; + chan_params->primary_chan = channel->hw_value; } void rtw_set_channel(struct rtw_dev *rtwdev) { + const struct rtw_chip_info *chip = rtwdev->chip; struct ieee80211_hw *hw = rtwdev->hw; struct rtw_hal *hal = &rtwdev->hal; - struct rtw_chip_info *chip = rtwdev->chip; struct rtw_channel_params ch_param; - u8 center_chan, bandwidth, primary_chan_idx; - u8 i; + u8 center_chan, primary_chan, bandwidth, band; rtw_get_channel_params(&hw->conf.chandef, &ch_param); if (WARN(ch_param.center_chan == 0, "Invalid channel\n")) return; center_chan = ch_param.center_chan; + primary_chan = ch_param.primary_chan; bandwidth = ch_param.bandwidth; - primary_chan_idx = ch_param.primary_chan_idx; - - hal->current_band_width = bandwidth; - hal->current_channel = center_chan; - hal->current_primary_channel_index = primary_chan_idx; - hal->current_band_type = center_chan > 14 ? RTW_BAND_5G : RTW_BAND_2G; - - switch (center_chan) { - case 1 ... 14: - hal->sar_band = RTW_SAR_BAND_0; - break; - case 36 ... 64: - hal->sar_band = RTW_SAR_BAND_1; - break; - case 100 ... 144: - hal->sar_band = RTW_SAR_BAND_3; - break; - case 149 ... 177: - hal->sar_band = RTW_SAR_BAND_4; - break; - default: - WARN(1, "unknown ch(%u) to SAR band\n", center_chan); - hal->sar_band = RTW_SAR_BAND_0; - break; - } + band = ch_param.center_chan > 14 ? RTW_BAND_5G : RTW_BAND_2G; - for (i = RTW_CHANNEL_WIDTH_20; i <= RTW_MAX_CHANNEL_WIDTH; i++) - hal->cch_by_bw[i] = ch_param.cch_by_bw[i]; + rtw_update_channel(rtwdev, center_chan, primary_chan, band, bandwidth); - chip->ops->set_channel(rtwdev, center_chan, bandwidth, primary_chan_idx); + chip->ops->set_channel(rtwdev, center_chan, bandwidth, + hal->current_primary_channel_index); if (hal->current_band_type == RTW_BAND_5G) { rtw_coex_switchband_notify(rtwdev, COEX_SWITCH_TO_5G); @@ -821,7 +850,7 @@ void rtw_set_channel(struct rtw_dev *rtwdev) void rtw_chip_prepare_tx(struct rtw_dev *rtwdev) { - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; if (rtwdev->need_rfk) { rtwdev->need_rfk = false; @@ -890,8 +919,8 @@ static u8 hw_bw_cap_to_bitamp(u8 bw_cap) static void rtw_hw_config_rf_ant_num(struct rtw_dev *rtwdev, u8 hw_ant_num) { + const struct rtw_chip_info *chip = rtwdev->chip; struct rtw_hal *hal = &rtwdev->hal; - struct rtw_chip_info *chip = rtwdev->chip; if (hw_ant_num == EFUSE_HW_CAP_IGNORE || hw_ant_num >= hal->rf_path_num) @@ -1240,7 +1269,7 @@ void rtw_update_sta_info(struct rtw_dev *rtwdev, struct rtw_sta_info *si, static int rtw_wait_firmware_completion(struct rtw_dev *rtwdev) { - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; struct rtw_fw_state *fw; fw = &rtwdev->fw; @@ -1261,7 +1290,7 @@ static int rtw_wait_firmware_completion(struct rtw_dev *rtwdev) static enum rtw_lps_deep_mode rtw_update_lps_deep_mode(struct rtw_dev *rtwdev, struct rtw_fw_state *fw) { - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; if (rtw_disable_lps_deep_mode || !chip->lps_deep_mode_supported || !fw->feature) @@ -1280,7 +1309,7 @@ static enum rtw_lps_deep_mode rtw_update_lps_deep_mode(struct rtw_dev *rtwdev, static int rtw_power_on(struct rtw_dev *rtwdev) { - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; struct rtw_fw_state *fw = &rtwdev->fw; bool wifi_only; int ret; @@ -1469,8 +1498,8 @@ void rtw_core_stop(struct rtw_dev *rtwdev) static void rtw_init_ht_cap(struct rtw_dev *rtwdev, struct ieee80211_sta_ht_cap *ht_cap) { + const struct rtw_chip_info *chip = rtwdev->chip; struct rtw_efuse *efuse = &rtwdev->efuse; - struct rtw_chip_info *chip = rtwdev->chip; ht_cap->ht_supported = true; ht_cap->cap = 0; @@ -1552,8 +1581,23 @@ static void rtw_init_vht_cap(struct rtw_dev *rtwdev, vht_cap->vht_mcs.tx_highest = highest; } +static u16 rtw_get_max_scan_ie_len(struct rtw_dev *rtwdev) +{ + u16 len; + + len = rtwdev->chip->max_scan_ie_len; + + if (!rtw_fw_feature_check(&rtwdev->fw, FW_FEATURE_SCAN_OFFLOAD) && + rtwdev->chip->id == RTW_CHIP_TYPE_8822C) + len = IEEE80211_MAX_DATA_LEN; + else if (rtw_fw_feature_ext_check(&rtwdev->fw, FW_FEATURE_EXT_OLD_PAGE_NUM)) + len -= RTW_OLD_PROBE_PG_CNT * TX_PAGE_SIZE; + + return len; +} + static void rtw_set_supported_band(struct ieee80211_hw *hw, - struct rtw_chip_info *chip) + const struct rtw_chip_info *chip) { struct rtw_dev *rtwdev = hw->priv; struct ieee80211_supported_band *sband; @@ -1585,7 +1629,7 @@ err_out: } static void rtw_unset_supported_band(struct ieee80211_hw *hw, - struct rtw_chip_info *chip) + const struct rtw_chip_info *chip) { kfree(hw->wiphy->bands[NL80211_BAND_2GHZ]); kfree(hw->wiphy->bands[NL80211_BAND_5GHZ]); @@ -1607,7 +1651,7 @@ static void rtw_vif_smps_iter(void *data, u8 *mac, void rtw_set_txrx_1ss(struct rtw_dev *rtwdev, bool txrx_1ss) { - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; struct rtw_hal *hal = &rtwdev->hal; if (!chip->ops->config_txrx_mode || rtwdev->hal.txrx_1ss == txrx_1ss) @@ -1631,6 +1675,10 @@ static void __update_firmware_feature(struct rtw_dev *rtwdev, feature = le32_to_cpu(fw_hdr->feature); fw->feature = feature & FW_FEATURE_SIG ? feature : 0; + + if (rtwdev->chip->id == RTW_CHIP_TYPE_8822C && + RTW_FW_SUIT_VER_CODE(rtwdev->fw) < RTW_FW_VER_CODE(9, 9, 13)) + fw->feature_ext |= FW_FEATURE_EXT_OLD_PAGE_NUM; } static void __update_firmware_info(struct rtw_dev *rtwdev, @@ -1724,7 +1772,7 @@ static int rtw_load_firmware(struct rtw_dev *rtwdev, enum rtw_fw_type type) static int rtw_chip_parameter_setup(struct rtw_dev *rtwdev) { - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; struct rtw_hal *hal = &rtwdev->hal; struct rtw_efuse *efuse = &rtwdev->efuse; @@ -1982,7 +2030,7 @@ static void rtw_stats_init(struct rtw_dev *rtwdev) int rtw_core_init(struct rtw_dev *rtwdev) { - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; struct rtw_coex *coex = &rtwdev->coex; int ret; @@ -2045,7 +2093,7 @@ int rtw_core_init(struct rtw_dev *rtwdev) ret = rtw_load_firmware(rtwdev, RTW_NORMAL_FW); if (ret) { rtw_warn(rtwdev, "no firmware loaded\n"); - return ret; + goto out; } if (chip->wow_fw_name) { @@ -2055,11 +2103,15 @@ int rtw_core_init(struct rtw_dev *rtwdev) wait_for_completion(&rtwdev->fw.completion); if (rtwdev->fw.firmware) release_firmware(rtwdev->fw.firmware); - return ret; + goto out; } } return 0; + +out: + destroy_workqueue(rtwdev->tx_wq); + return ret; } EXPORT_SYMBOL(rtw_core_init); @@ -2136,7 +2188,7 @@ int rtw_register_hw(struct rtw_dev *rtwdev, struct ieee80211_hw *hw) hw->wiphy->features |= NL80211_FEATURE_SCAN_RANDOM_MAC_ADDR; hw->wiphy->max_scan_ssids = RTW_SCAN_MAX_SSIDS; - hw->wiphy->max_scan_ie_len = RTW_SCAN_MAX_IE_LEN; + hw->wiphy->max_scan_ie_len = rtw_get_max_scan_ie_len(rtwdev); wiphy_ext_feature_set(hw->wiphy, NL80211_EXT_FEATURE_CAN_REPLACE_PTK0); wiphy_ext_feature_set(hw->wiphy, NL80211_EXT_FEATURE_SCAN_RANDOM_SN); @@ -2180,7 +2232,7 @@ EXPORT_SYMBOL(rtw_register_hw); void rtw_unregister_hw(struct rtw_dev *rtwdev, struct ieee80211_hw *hw) { - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; ieee80211_unregister_hw(hw); rtw_unset_supported_band(hw, chip); diff --git a/drivers/net/wireless/realtek/rtw88/main.h b/drivers/net/wireless/realtek/rtw88/main.h index 7db627fc26be..bccd7b28f60c 100644 --- a/drivers/net/wireless/realtek/rtw88/main.h +++ b/drivers/net/wireless/realtek/rtw88/main.h @@ -22,7 +22,6 @@ #define MAX_PG_CAM_BACKUP_NUM 8 #define RTW_SCAN_MAX_SSIDS 4 -#define RTW_SCAN_MAX_IE_LEN 128 #define RTW_MAX_PATTERN_NUM 12 #define RTW_MAX_PATTERN_MASK_SIZE 16 @@ -33,6 +32,7 @@ #define RFREG_MASK 0xfffff #define INV_RF_DATA 0xffffffff #define TX_PAGE_SIZE_SHIFT 7 +#define TX_PAGE_SIZE (1 << TX_PAGE_SIZE_SHIFT) #define RTW_CHANNEL_WIDTH_MAX 3 #define RTW_RF_PATH_MAX 4 @@ -510,12 +510,8 @@ struct rtw_timer_list { struct rtw_channel_params { u8 center_chan; + u8 primary_chan; u8 bandwidth; - u8 primary_chan_idx; - /* center channel by different available bandwidth, - * val of (bw > current bandwidth) is invalid - */ - u8 cch_by_bw[RTW_MAX_CHANNEL_WIDTH + 1]; }; struct rtw_hw_reg { @@ -1232,6 +1228,7 @@ struct rtw_chip_info { const char *wow_fw_name; const struct wiphy_wowlan_support *wowlan_stub; const u8 max_sched_scan_ssids; + const u16 max_scan_ie_len; /* coex paras */ u32 coex_para_ver; @@ -1853,6 +1850,7 @@ struct rtw_fw_state { u8 sub_index; u16 h2c_version; u32 feature; + u32 feature_ext; }; enum rtw_sar_sources { @@ -1896,6 +1894,7 @@ struct rtw_hal { u8 current_primary_channel_index; u8 current_band_width; u8 current_band_type; + u8 primary_channel; /* center channel for different available bandwidth, * val of (bw > current_band_width) is invalid @@ -1967,6 +1966,7 @@ struct rtw_hw_scan_info { struct ieee80211_vif *scanning_vif; u8 probe_pg_size; u8 op_pri_ch_idx; + u8 op_pri_ch; u8 op_chan; u8 op_bw; }; @@ -1978,7 +1978,7 @@ struct rtw_dev { struct rtw_hci hci; struct rtw_hw_scan_info scan_info; - struct rtw_chip_info *chip; + const struct rtw_chip_info *chip; struct rtw_hal hal; struct rtw_fifo_conf fifo; struct rtw_fw_state fw; @@ -2132,6 +2132,20 @@ static inline int rtw_chip_dump_fw_crash(struct rtw_dev *rtwdev) return 0; } +static inline +enum nl80211_band rtw_hw_to_nl80211_band(enum rtw_supported_band hw_band) +{ + switch (hw_band) { + default: + case RTW_BAND_2G: + return NL80211_BAND_2GHZ; + case RTW_BAND_5G: + return NL80211_BAND_5GHZ; + case RTW_BAND_60G: + return NL80211_BAND_60GHZ; + } +} + void rtw_set_rx_freq_band(struct rtw_rx_pkt_stat *pkt_stat, u8 channel); void rtw_set_dtim_period(struct rtw_dev *rtwdev, int dtim_period); void rtw_get_channel_params(struct cfg80211_chan_def *chandef, @@ -2173,4 +2187,7 @@ int rtw_dump_fw(struct rtw_dev *rtwdev, const u32 ocp_src, u32 size, u32 fwcd_item); int rtw_dump_reg(struct rtw_dev *rtwdev, const u32 addr, const u32 size); void rtw_set_txrx_1ss(struct rtw_dev *rtwdev, bool config_1ss); +void rtw_update_channel(struct rtw_dev *rtwdev, u8 center_channel, + u8 primary_channel, enum rtw_supported_band band, + enum rtw_bandwidth bandwidth); #endif diff --git a/drivers/net/wireless/realtek/rtw88/pci.c b/drivers/net/wireless/realtek/rtw88/pci.c index 24d5695363d3..0975d27240e4 100644 --- a/drivers/net/wireless/realtek/rtw88/pci.c +++ b/drivers/net/wireless/realtek/rtw88/pci.c @@ -322,7 +322,7 @@ static int rtw_pci_init_trx_ring(struct rtw_dev *rtwdev) struct rtw_pci *rtwpci = (struct rtw_pci *)rtwdev->priv; struct rtw_pci_tx_ring *tx_ring; struct rtw_pci_rx_ring *rx_ring; - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; int i = 0, j = 0, tx_alloced = 0, rx_alloced = 0; int tx_desc_size, rx_desc_size; u32 len; @@ -721,7 +721,7 @@ static void rtw_pci_dma_check(struct rtw_dev *rtwdev, u32 idx) { struct rtw_pci *rtwpci = (struct rtw_pci *)rtwdev->priv; - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; struct rtw_pci_rx_buffer_desc *buf_desc; u32 desc_sz = chip->rx_buf_desc_sz; u16 total_pkt_size; @@ -834,7 +834,7 @@ static int rtw_pci_tx_write_data(struct rtw_dev *rtwdev, struct sk_buff *skb, u8 queue) { struct rtw_pci *rtwpci = (struct rtw_pci *)rtwdev->priv; - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; struct rtw_pci_tx_ring *ring; struct rtw_pci_tx_data *tx_data; dma_addr_t dma; @@ -1073,7 +1073,7 @@ static int rtw_pci_get_hw_rx_ring_nr(struct rtw_dev *rtwdev, static u32 rtw_pci_rx_napi(struct rtw_dev *rtwdev, struct rtw_pci *rtwpci, u8 hw_queue, u32 limit) { - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; struct napi_struct *napi = &rtwpci->napi; struct rtw_pci_rx_ring *ring = &rtwpci->rx_rings[RTW_RX_QUEUE_MPDU]; struct rtw_rx_pkt_stat pkt_stat; @@ -1425,7 +1425,7 @@ static void rtw_pci_link_ps(struct rtw_dev *rtwdev, bool enter) static void rtw_pci_link_cfg(struct rtw_dev *rtwdev) { - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; struct rtw_pci *rtwpci = (struct rtw_pci *)rtwdev->priv; struct pci_dev *pdev = rtwpci->pdev; u16 link_ctrl; @@ -1467,7 +1467,7 @@ static void rtw_pci_link_cfg(struct rtw_dev *rtwdev) static void rtw_pci_interface_cfg(struct rtw_dev *rtwdev) { - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; switch (chip->id) { case RTW_CHIP_TYPE_8822C: @@ -1483,7 +1483,7 @@ static void rtw_pci_interface_cfg(struct rtw_dev *rtwdev) static void rtw_pci_phy_cfg(struct rtw_dev *rtwdev) { struct rtw_pci *rtwpci = (struct rtw_pci *)rtwdev->priv; - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; struct pci_dev *pdev = rtwpci->pdev; const struct rtw_intf_phy_para *para; u16 cut; @@ -1538,7 +1538,7 @@ static int __maybe_unused rtw_pci_suspend(struct device *dev) { struct ieee80211_hw *hw = dev_get_drvdata(dev); struct rtw_dev *rtwdev = hw->priv; - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; struct rtw_efuse *efuse = &rtwdev->efuse; if (chip->id == RTW_CHIP_TYPE_8822C && efuse->rfe_option == 6) @@ -1550,7 +1550,7 @@ static int __maybe_unused rtw_pci_resume(struct device *dev) { struct ieee80211_hw *hw = dev_get_drvdata(dev); struct rtw_dev *rtwdev = hw->priv; - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; struct rtw_efuse *efuse = &rtwdev->efuse; if (chip->id == RTW_CHIP_TYPE_8822C && efuse->rfe_option == 6) @@ -1717,8 +1717,7 @@ static void rtw_pci_napi_init(struct rtw_dev *rtwdev) struct rtw_pci *rtwpci = (struct rtw_pci *)rtwdev->priv; init_dummy_netdev(&rtwpci->netdev); - netif_napi_add(&rtwpci->netdev, &rtwpci->napi, rtw_pci_napi_poll, - NAPI_POLL_WEIGHT); + netif_napi_add(&rtwpci->netdev, &rtwpci->napi, rtw_pci_napi_poll); } static void rtw_pci_napi_deinit(struct rtw_dev *rtwdev) @@ -1848,7 +1847,7 @@ void rtw_pci_shutdown(struct pci_dev *pdev) { struct ieee80211_hw *hw = pci_get_drvdata(pdev); struct rtw_dev *rtwdev; - struct rtw_chip_info *chip; + const struct rtw_chip_info *chip; if (!hw) return; diff --git a/drivers/net/wireless/realtek/rtw88/phy.c b/drivers/net/wireless/realtek/rtw88/phy.c index 8982e0c98dac..bd7d05e08084 100644 --- a/drivers/net/wireless/realtek/rtw88/phy.c +++ b/drivers/net/wireless/realtek/rtw88/phy.c @@ -138,7 +138,7 @@ EXPORT_SYMBOL(rtw_phy_set_edcca_th); void rtw_phy_adaptivity_set_mode(struct rtw_dev *rtwdev) { - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; struct rtw_dm_info *dm_info = &rtwdev->dm_info; /* turn off in debugfs for debug usage */ @@ -165,7 +165,7 @@ void rtw_phy_adaptivity_set_mode(struct rtw_dev *rtwdev) static void rtw_phy_adaptivity_init(struct rtw_dev *rtwdev) { - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; rtw_phy_adaptivity_set_mode(rtwdev); if (chip->ops->adaptivity_init) @@ -180,7 +180,7 @@ static void rtw_phy_adaptivity(struct rtw_dev *rtwdev) static void rtw_phy_cfo_init(struct rtw_dev *rtwdev) { - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; if (chip->ops->cfo_init) chip->ops->cfo_init(rtwdev); @@ -199,7 +199,7 @@ static void rtw_phy_tx_path_div_init(struct rtw_dev *rtwdev) void rtw_phy_init(struct rtw_dev *rtwdev) { - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; struct rtw_dm_info *dm_info = &rtwdev->dm_info; u32 addr, mask; @@ -226,7 +226,7 @@ EXPORT_SYMBOL(rtw_phy_init); void rtw_phy_dig_write(struct rtw_dev *rtwdev, u8 igi) { - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; struct rtw_hal *hal = &rtwdev->hal; u32 addr, mask; u8 path; @@ -245,7 +245,7 @@ void rtw_phy_dig_write(struct rtw_dev *rtwdev, u8 igi) static void rtw_phy_stat_false_alarm(struct rtw_dev *rtwdev) { - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; chip->ops->false_alarm_statistics(rtwdev); } @@ -603,7 +603,7 @@ static void rtw_phy_rrsr_update(struct rtw_dev *rtwdev) static void rtw_phy_dpk_track(struct rtw_dev *rtwdev) { - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; if (chip->ops->dpk_track) chip->ops->dpk_track(rtwdev); @@ -659,7 +659,7 @@ EXPORT_SYMBOL(rtw_phy_parsing_cfo); static void rtw_phy_cfo_track(struct rtw_dev *rtwdev) { - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; if (chip->ops->cfo_track) chip->ops->cfo_track(rtwdev); @@ -720,8 +720,8 @@ static u8 rtw_phy_cck_pd_lv(struct rtw_dev *rtwdev) static void rtw_phy_cck_pd(struct rtw_dev *rtwdev) { + const struct rtw_chip_info *chip = rtwdev->chip; struct rtw_dm_info *dm_info = &rtwdev->dm_info; - struct rtw_chip_info *chip = rtwdev->chip; u32 cck_fa = dm_info->cck_fa_cnt; u8 level; @@ -816,23 +816,18 @@ static u8 rtw_phy_linear_2_db(u64 linear) u8 j; u32 dB; - if (linear >= db_invert_table[11][7]) - return 96; /* maximum 96 dB */ - for (i = 0; i < 12; i++) { - if (i <= 2 && (linear << FRAC_BITS) <= db_invert_table[i][7]) - break; - else if (i > 2 && linear <= db_invert_table[i][7]) - break; + for (j = 0; j < 8; j++) { + if (i <= 2 && (linear << FRAC_BITS) <= db_invert_table[i][j]) + goto cnt; + else if (i > 2 && linear <= db_invert_table[i][j]) + goto cnt; + } } - for (j = 0; j < 8; j++) { - if (i <= 2 && (linear << FRAC_BITS) <= db_invert_table[i][j]) - break; - else if (i > 2 && linear <= db_invert_table[i][j]) - break; - } + return 96; /* maximum 96 dB */ +cnt: if (j == 0 && i == 0) goto end; @@ -900,7 +895,7 @@ u32 rtw_phy_read_rf(struct rtw_dev *rtwdev, enum rtw_rf_path rf_path, u32 addr, u32 mask) { struct rtw_hal *hal = &rtwdev->hal; - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; const u32 *base_addr = chip->rf_base_addr; u32 val, direct_addr; @@ -923,7 +918,7 @@ u32 rtw_phy_read_rf_sipi(struct rtw_dev *rtwdev, enum rtw_rf_path rf_path, u32 addr, u32 mask) { struct rtw_hal *hal = &rtwdev->hal; - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; const struct rtw_rf_sipi_addr *rf_sipi_addr; const struct rtw_rf_sipi_addr *rf_sipi_addr_a; u32 val32; @@ -972,8 +967,8 @@ bool rtw_phy_write_rf_reg_sipi(struct rtw_dev *rtwdev, enum rtw_rf_path rf_path, u32 addr, u32 mask, u32 data) { struct rtw_hal *hal = &rtwdev->hal; - struct rtw_chip_info *chip = rtwdev->chip; - u32 *sipi_addr = chip->rf_sipi_addr; + const struct rtw_chip_info *chip = rtwdev->chip; + const u32 *sipi_addr = chip->rf_sipi_addr; u32 data_and_addr; u32 old_data = 0; u32 shift; @@ -1012,7 +1007,7 @@ bool rtw_phy_write_rf_reg(struct rtw_dev *rtwdev, enum rtw_rf_path rf_path, u32 addr, u32 mask, u32 data) { struct rtw_hal *hal = &rtwdev->hal; - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; const u32 *base_addr = chip->rf_base_addr; u32 direct_addr; @@ -1747,7 +1742,7 @@ EXPORT_SYMBOL(rtw_phy_cfg_rf); static void rtw_load_rfk_table(struct rtw_dev *rtwdev) { - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; struct rtw_dpk_info *dpk_info = &rtwdev->dm_info.dpk_info; if (!chip->rfk_init_tbl) @@ -1766,7 +1761,7 @@ static void rtw_load_rfk_table(struct rtw_dev *rtwdev) void rtw_phy_load_tables(struct rtw_dev *rtwdev) { - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; u8 rf_path; rtw_load_table(rtwdev, chip->mac_tbl); @@ -1875,7 +1870,7 @@ static u8 rtw_get_channel_group(u8 channel, u8 rate) static s8 rtw_phy_get_dis_dpd_by_rate_diff(struct rtw_dev *rtwdev, u16 rate) { - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; s8 dpd_diff = 0; if (!chip->en_dis_dpd) @@ -1909,7 +1904,7 @@ static u8 rtw_phy_get_2g_tx_power_index(struct rtw_dev *rtwdev, enum rtw_bandwidth bandwidth, u8 rate, u8 group) { - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; u8 tx_power; bool mcs_rate; bool above_2ss; @@ -1956,7 +1951,7 @@ static u8 rtw_phy_get_5g_tx_power_index(struct rtw_dev *rtwdev, enum rtw_bandwidth bandwidth, u8 rate, u8 group) { - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; u8 tx_power; u8 upper, lower; bool mcs_rate; @@ -2209,7 +2204,7 @@ static void rtw_phy_set_tx_power_level_by_path(struct rtw_dev *rtwdev, void rtw_phy_set_tx_power_level(struct rtw_dev *rtwdev, u8 channel) { - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; struct rtw_hal *hal = &rtwdev->hal; u8 path; @@ -2484,7 +2479,7 @@ static void rtw_phy_set_tx_path_by_reg(struct rtw_dev *rtwdev, { struct rtw_path_div *path_div = &rtwdev->dm_path_div; enum rtw_bb_path tx_path_sel_cck = tx_path_sel_1ss; - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; if (tx_path_sel_1ss == path_div->current_tx_path) return; @@ -2539,7 +2534,7 @@ static void rtw_phy_tx_path_diversity_2ss(struct rtw_dev *rtwdev) void rtw_phy_tx_path_diversity(struct rtw_dev *rtwdev) { - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; if (!chip->path_div_supported) return; diff --git a/drivers/net/wireless/realtek/rtw88/phy.h b/drivers/net/wireless/realtek/rtw88/phy.h index b6c5ae60a462..ccfcbd3ced03 100644 --- a/drivers/net/wireless/realtek/rtw88/phy.h +++ b/drivers/net/wireless/realtek/rtw88/phy.h @@ -114,7 +114,7 @@ const struct rtw_table name ## _tbl = { \ static inline const struct rtw_rfe_def *rtw_get_rfe_def(struct rtw_dev *rtwdev) { - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; struct rtw_efuse *efuse = &rtwdev->efuse; const struct rtw_rfe_def *rfe_def = NULL; diff --git a/drivers/net/wireless/realtek/rtw88/ps.c b/drivers/net/wireless/realtek/rtw88/ps.c index bfa64c038f5f..c93da743681f 100644 --- a/drivers/net/wireless/realtek/rtw88/ps.c +++ b/drivers/net/wireless/realtek/rtw88/ps.c @@ -19,14 +19,14 @@ static int rtw_ips_pwr_up(struct rtw_dev *rtwdev) rtw_err(rtwdev, "leave idle state failed\n"); rtw_set_channel(rtwdev); - clear_bit(RTW_FLAG_INACTIVE_PS, rtwdev->flags); return ret; } int rtw_enter_ips(struct rtw_dev *rtwdev) { - set_bit(RTW_FLAG_INACTIVE_PS, rtwdev->flags); + if (test_and_set_bit(RTW_FLAG_INACTIVE_PS, rtwdev->flags)) + return 0; rtw_coex_ips_notify(rtwdev, COEX_IPS_ENTER); @@ -50,6 +50,9 @@ int rtw_leave_ips(struct rtw_dev *rtwdev) { int ret; + if (!test_and_clear_bit(RTW_FLAG_INACTIVE_PS, rtwdev->flags)) + return 0; + rtw_hci_link_ps(rtwdev, false); ret = rtw_ips_pwr_up(rtwdev); diff --git a/drivers/net/wireless/realtek/rtw88/regd.c b/drivers/net/wireless/realtek/rtw88/regd.c index 315c2b193e92..2f547cbcf6da 100644 --- a/drivers/net/wireless/realtek/rtw88/regd.c +++ b/drivers/net/wireless/realtek/rtw88/regd.c @@ -479,6 +479,7 @@ void rtw_regd_notifier(struct wiphy *wiphy, struct regulatory_request *request) rtw_dbg(rtwdev, RTW_DBG_REGD, "regd state: %d -> %d\n", rtwdev->regd.state, next_regd.state); + mutex_lock(&rtwdev->mutex); rtwdev->regd = next_regd; rtw_dbg_regd_dump(rtwdev, "get alpha2 %c%c from initiator %d: ", request->alpha2[0], @@ -487,6 +488,7 @@ void rtw_regd_notifier(struct wiphy *wiphy, struct regulatory_request *request) rtw_phy_adaptivity_set_mode(rtwdev); rtw_phy_set_tx_power_level(rtwdev, hal->current_channel); + mutex_unlock(&rtwdev->mutex); } u8 rtw_regd_get(struct rtw_dev *rtwdev) diff --git a/drivers/net/wireless/realtek/rtw88/rtw8723d.c b/drivers/net/wireless/realtek/rtw88/rtw8723d.c index 993bd6b1d723..0a4f770fcbb7 100644 --- a/drivers/net/wireless/realtek/rtw88/rtw8723d.c +++ b/drivers/net/wireless/realtek/rtw88/rtw8723d.c @@ -2720,7 +2720,7 @@ const struct rtw_chip_info rtw8723d_hw_spec = { .max_power_index = 0x3f, .csi_buf_pg_num = 0, .band = RTW_BAND_2G, - .page_size = 128, + .page_size = TX_PAGE_SIZE, .dig_min = 0x20, .ht_supported = true, .vht_supported = false, @@ -2748,6 +2748,7 @@ const struct rtw_chip_info rtw8723d_hw_spec = { .pwr_track_tbl = &rtw8723d_rtw_pwr_track_tbl, .iqk_threshold = 8, .ampdu_density = IEEE80211_HT_MPDU_DENSITY_16, + .max_scan_ie_len = IEEE80211_MAX_DATA_LEN, .coex_para_ver = 0x2007022f, .bt_desired_ver = 0x2f, diff --git a/drivers/net/wireless/realtek/rtw88/rtw8821c.c b/drivers/net/wireless/realtek/rtw88/rtw8821c.c index 025262a8970e..9afdc5ce86b4 100644 --- a/drivers/net/wireless/realtek/rtw88/rtw8821c.c +++ b/drivers/net/wireless/realtek/rtw88/rtw8821c.c @@ -1898,7 +1898,7 @@ const struct rtw_chip_info rtw8821c_hw_spec = { .max_power_index = 0x3f, .csi_buf_pg_num = 0, .band = RTW_BAND_2G | RTW_BAND_5G, - .page_size = 128, + .page_size = TX_PAGE_SIZE, .dig_min = 0x1c, .ht_supported = true, .vht_supported = true, @@ -1926,6 +1926,7 @@ const struct rtw_chip_info rtw8821c_hw_spec = { .bfer_su_max_num = 2, .bfer_mu_max_num = 1, .ampdu_density = IEEE80211_HT_MPDU_DENSITY_2, + .max_scan_ie_len = IEEE80211_MAX_DATA_LEN, .coex_para_ver = 0x19092746, .bt_desired_ver = 0x46, diff --git a/drivers/net/wireless/realtek/rtw88/rtw8822b.c b/drivers/net/wireless/realtek/rtw88/rtw8822b.c index 321848870561..690e35c98f6e 100644 --- a/drivers/net/wireless/realtek/rtw88/rtw8822b.c +++ b/drivers/net/wireless/realtek/rtw88/rtw8822b.c @@ -2517,7 +2517,7 @@ const struct rtw_chip_info rtw8822b_hw_spec = { .max_power_index = 0x3f, .csi_buf_pg_num = 0, .band = RTW_BAND_2G | RTW_BAND_5G, - .page_size = 128, + .page_size = TX_PAGE_SIZE, .dig_min = 0x1c, .ht_supported = true, .vht_supported = true, @@ -2549,6 +2549,7 @@ const struct rtw_chip_info rtw8822b_hw_spec = { .l2h_th_ini_cs = 10 + EDCCA_IGI_BASE, .l2h_th_ini_ad = -14 + EDCCA_IGI_BASE, .ampdu_density = IEEE80211_HT_MPDU_DENSITY_2, + .max_scan_ie_len = IEEE80211_MAX_DATA_LEN, .coex_para_ver = 0x20070206, .bt_desired_ver = 0x6, diff --git a/drivers/net/wireless/realtek/rtw88/rtw8822c.c b/drivers/net/wireless/realtek/rtw88/rtw8822c.c index 09f9e4adcf34..fccb15dfb959 100644 --- a/drivers/net/wireless/realtek/rtw88/rtw8822c.c +++ b/drivers/net/wireless/realtek/rtw88/rtw8822c.c @@ -5330,7 +5330,7 @@ const struct rtw_chip_info rtw8822c_hw_spec = { .max_power_index = 0x7f, .csi_buf_pg_num = 50, .band = RTW_BAND_2G | RTW_BAND_5G, - .page_size = 128, + .page_size = TX_PAGE_SIZE, .dig_min = 0x20, .default_1ss_tx_path = BB_PATH_A, .path_div_supported = true, @@ -5375,6 +5375,7 @@ const struct rtw_chip_info rtw8822c_hw_spec = { .wowlan_stub = &rtw_wowlan_stub_8822c, .max_sched_scan_ssids = 4, #endif + .max_scan_ie_len = (RTW_PROBE_PG_CNT - 1) * TX_PAGE_SIZE, .coex_para_ver = 0x22020720, .bt_desired_ver = 0x20, .scbd_support = true, diff --git a/drivers/net/wireless/realtek/rtw88/tx.c b/drivers/net/wireless/realtek/rtw88/tx.c index 60d40a5c2c6a..ab39245e9c2f 100644 --- a/drivers/net/wireless/realtek/rtw88/tx.c +++ b/drivers/net/wireless/realtek/rtw88/tx.c @@ -384,7 +384,7 @@ void rtw_tx_pkt_info_update(struct rtw_dev *rtwdev, struct ieee80211_sta *sta, struct sk_buff *skb) { - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; struct ieee80211_tx_info *info = IEEE80211_SKB_CB(skb); struct ieee80211_hdr *hdr = (struct ieee80211_hdr *)skb->data; struct rtw_sta_info *si; @@ -424,7 +424,7 @@ void rtw_tx_rsvd_page_pkt_info_update(struct rtw_dev *rtwdev, struct sk_buff *skb, enum rtw_rsvd_packet_type type) { - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; struct ieee80211_hdr *hdr = (struct ieee80211_hdr *)skb->data; bool bmc; @@ -475,7 +475,7 @@ rtw_tx_write_data_rsvd_page_get(struct rtw_dev *rtwdev, struct rtw_tx_pkt_info *pkt_info, u8 *buf, u32 size) { - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; struct sk_buff *skb; u32 tx_pkt_desc_sz; u32 length; @@ -501,7 +501,7 @@ rtw_tx_write_data_h2c_get(struct rtw_dev *rtwdev, struct rtw_tx_pkt_info *pkt_info, u8 *buf, u32 size) { - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; struct sk_buff *skb; u32 tx_pkt_desc_sz; u32 length; diff --git a/drivers/net/wireless/realtek/rtw88/util.c b/drivers/net/wireless/realtek/rtw88/util.c index 2c515af214e7..cdfd66a85075 100644 --- a/drivers/net/wireless/realtek/rtw88/util.c +++ b/drivers/net/wireless/realtek/rtw88/util.c @@ -23,7 +23,7 @@ EXPORT_SYMBOL(check_hw_ready); bool ltecoex_read_reg(struct rtw_dev *rtwdev, u16 offset, u32 *val) { - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; const struct rtw_ltecoex_addr *ltecoex = chip->ltecoex_addr; if (!check_hw_ready(rtwdev, ltecoex->ctrl, LTECOEX_READY, 1)) @@ -37,7 +37,7 @@ bool ltecoex_read_reg(struct rtw_dev *rtwdev, u16 offset, u32 *val) bool ltecoex_reg_write(struct rtw_dev *rtwdev, u16 offset, u32 value) { - struct rtw_chip_info *chip = rtwdev->chip; + const struct rtw_chip_info *chip = rtwdev->chip; const struct rtw_ltecoex_addr *ltecoex = chip->ltecoex_addr; if (!check_hw_ready(rtwdev, ltecoex->ctrl, LTECOEX_READY, 1)) diff --git a/drivers/net/wireless/realtek/rtw89/Makefile b/drivers/net/wireless/realtek/rtw89/Makefile index 3006482d25c7..a87f2aff4def 100644 --- a/drivers/net/wireless/realtek/rtw89/Makefile +++ b/drivers/net/wireless/realtek/rtw89/Makefile @@ -12,6 +12,7 @@ rtw89_core-y += core.o \ sar.o \ coex.o \ ps.o \ + chan.o \ ser.o obj-$(CONFIG_RTW89_8852A) += rtw89_8852a.o diff --git a/drivers/net/wireless/realtek/rtw89/chan.c b/drivers/net/wireless/realtek/rtw89/chan.c new file mode 100644 index 000000000000..a4f61c2f6512 --- /dev/null +++ b/drivers/net/wireless/realtek/rtw89/chan.c @@ -0,0 +1,235 @@ +// SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause +/* Copyright(c) 2020-2022 Realtek Corporation + */ + +#include "chan.h" +#include "debug.h" + +static enum rtw89_subband rtw89_get_subband_type(enum rtw89_band band, + u8 center_chan) +{ + switch (band) { + default: + case RTW89_BAND_2G: + switch (center_chan) { + default: + case 1 ... 14: + return RTW89_CH_2G; + } + case RTW89_BAND_5G: + switch (center_chan) { + default: + case 36 ... 64: + return RTW89_CH_5G_BAND_1; + case 100 ... 144: + return RTW89_CH_5G_BAND_3; + case 149 ... 177: + return RTW89_CH_5G_BAND_4; + } + case RTW89_BAND_6G: + switch (center_chan) { + default: + case 1 ... 29: + return RTW89_CH_6G_BAND_IDX0; + case 33 ... 61: + return RTW89_CH_6G_BAND_IDX1; + case 65 ... 93: + return RTW89_CH_6G_BAND_IDX2; + case 97 ... 125: + return RTW89_CH_6G_BAND_IDX3; + case 129 ... 157: + return RTW89_CH_6G_BAND_IDX4; + case 161 ... 189: + return RTW89_CH_6G_BAND_IDX5; + case 193 ... 221: + return RTW89_CH_6G_BAND_IDX6; + case 225 ... 253: + return RTW89_CH_6G_BAND_IDX7; + } + } +} + +static enum rtw89_sc_offset rtw89_get_primary_chan_idx(enum rtw89_bandwidth bw, + u32 center_freq, + u32 primary_freq) +{ + u8 primary_chan_idx; + u32 offset; + + switch (bw) { + default: + case RTW89_CHANNEL_WIDTH_20: + primary_chan_idx = RTW89_SC_DONT_CARE; + break; + case RTW89_CHANNEL_WIDTH_40: + if (primary_freq > center_freq) + primary_chan_idx = RTW89_SC_20_UPPER; + else + primary_chan_idx = RTW89_SC_20_LOWER; + break; + case RTW89_CHANNEL_WIDTH_80: + case RTW89_CHANNEL_WIDTH_160: + if (primary_freq > center_freq) { + offset = (primary_freq - center_freq - 10) / 20; + primary_chan_idx = RTW89_SC_20_UPPER + offset * 2; + } else { + offset = (center_freq - primary_freq - 10) / 20; + primary_chan_idx = RTW89_SC_20_LOWER + offset * 2; + } + break; + } + + return primary_chan_idx; +} + +void rtw89_chan_create(struct rtw89_chan *chan, u8 center_chan, u8 primary_chan, + enum rtw89_band band, enum rtw89_bandwidth bandwidth) +{ + enum nl80211_band nl_band = rtw89_hw_to_nl80211_band(band); + u32 center_freq, primary_freq; + + memset(chan, 0, sizeof(*chan)); + chan->channel = center_chan; + chan->primary_channel = primary_chan; + chan->band_type = band; + chan->band_width = bandwidth; + + center_freq = ieee80211_channel_to_frequency(center_chan, nl_band); + primary_freq = ieee80211_channel_to_frequency(primary_chan, nl_band); + + chan->freq = center_freq; + chan->subband_type = rtw89_get_subband_type(band, center_chan); + chan->pri_ch_idx = rtw89_get_primary_chan_idx(bandwidth, center_freq, + primary_freq); +} + +bool rtw89_assign_entity_chan(struct rtw89_dev *rtwdev, + enum rtw89_sub_entity_idx idx, + const struct rtw89_chan *new) +{ + struct rtw89_hal *hal = &rtwdev->hal; + struct rtw89_chan *chan = &hal->chan[idx]; + struct rtw89_chan_rcd *rcd = &hal->chan_rcd[idx]; + bool band_changed; + + rcd->prev_primary_channel = chan->primary_channel; + rcd->prev_band_type = chan->band_type; + band_changed = new->band_type != chan->band_type; + + *chan = *new; + return band_changed; +} + +static void __rtw89_config_entity_chandef(struct rtw89_dev *rtwdev, + enum rtw89_sub_entity_idx idx, + const struct cfg80211_chan_def *chandef, + bool from_stack) +{ + struct rtw89_hal *hal = &rtwdev->hal; + + hal->chandef[idx] = *chandef; + + if (from_stack) + set_bit(idx, hal->entity_map); +} + +void rtw89_config_entity_chandef(struct rtw89_dev *rtwdev, + enum rtw89_sub_entity_idx idx, + const struct cfg80211_chan_def *chandef) +{ + __rtw89_config_entity_chandef(rtwdev, idx, chandef, true); +} + +static void rtw89_config_default_chandef(struct rtw89_dev *rtwdev) +{ + struct cfg80211_chan_def chandef = {0}; + + rtw89_get_default_chandef(&chandef); + __rtw89_config_entity_chandef(rtwdev, RTW89_SUB_ENTITY_0, &chandef, false); +} + +void rtw89_entity_init(struct rtw89_dev *rtwdev) +{ + struct rtw89_hal *hal = &rtwdev->hal; + + bitmap_zero(hal->entity_map, NUM_OF_RTW89_SUB_ENTITY); + rtw89_config_default_chandef(rtwdev); +} + +enum rtw89_entity_mode rtw89_entity_recalc(struct rtw89_dev *rtwdev) +{ + struct rtw89_hal *hal = &rtwdev->hal; + enum rtw89_entity_mode mode; + u8 weight; + + weight = bitmap_weight(hal->entity_map, NUM_OF_RTW89_SUB_ENTITY); + switch (weight) { + default: + rtw89_warn(rtwdev, "unknown ent chan weight: %d\n", weight); + bitmap_zero(hal->entity_map, NUM_OF_RTW89_SUB_ENTITY); + fallthrough; + case 0: + rtw89_config_default_chandef(rtwdev); + fallthrough; + case 1: + mode = RTW89_ENTITY_MODE_SCC; + break; + } + + rtw89_set_entity_mode(rtwdev, mode); + return mode; +} + +int rtw89_chanctx_ops_add(struct rtw89_dev *rtwdev, + struct ieee80211_chanctx_conf *ctx) +{ + struct rtw89_hal *hal = &rtwdev->hal; + struct rtw89_chanctx_cfg *cfg = (struct rtw89_chanctx_cfg *)ctx->drv_priv; + const struct rtw89_chip_info *chip = rtwdev->chip; + u8 idx; + + idx = find_first_zero_bit(hal->entity_map, NUM_OF_RTW89_SUB_ENTITY); + if (idx >= chip->support_chanctx_num) + return -ENOENT; + + rtw89_config_entity_chandef(rtwdev, idx, &ctx->def); + rtw89_set_channel(rtwdev); + cfg->idx = idx; + return 0; +} + +void rtw89_chanctx_ops_remove(struct rtw89_dev *rtwdev, + struct ieee80211_chanctx_conf *ctx) +{ + struct rtw89_hal *hal = &rtwdev->hal; + struct rtw89_chanctx_cfg *cfg = (struct rtw89_chanctx_cfg *)ctx->drv_priv; + + clear_bit(cfg->idx, hal->entity_map); + rtw89_set_channel(rtwdev); +} + +void rtw89_chanctx_ops_change(struct rtw89_dev *rtwdev, + struct ieee80211_chanctx_conf *ctx, + u32 changed) +{ + struct rtw89_chanctx_cfg *cfg = (struct rtw89_chanctx_cfg *)ctx->drv_priv; + u8 idx = cfg->idx; + + if (changed & IEEE80211_CHANCTX_CHANGE_WIDTH) { + rtw89_config_entity_chandef(rtwdev, idx, &ctx->def); + rtw89_set_channel(rtwdev); + } +} + +int rtw89_chanctx_ops_assign_vif(struct rtw89_dev *rtwdev, + struct rtw89_vif *rtwvif, + struct ieee80211_chanctx_conf *ctx) +{ + return 0; +} + +void rtw89_chanctx_ops_unassign_vif(struct rtw89_dev *rtwdev, + struct rtw89_vif *rtwvif, + struct ieee80211_chanctx_conf *ctx) +{ +} diff --git a/drivers/net/wireless/realtek/rtw89/chan.h b/drivers/net/wireless/realtek/rtw89/chan.h new file mode 100644 index 000000000000..ecbd4503bead --- /dev/null +++ b/drivers/net/wireless/realtek/rtw89/chan.h @@ -0,0 +1,64 @@ +/* SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause + * Copyright(c) 2020-2022 Realtek Corporation + */ + +#ifndef __RTW89_CHAN_H__ +#define __RTW89_CHAN_H__ + +#include "core.h" + +static inline bool rtw89_get_entity_state(struct rtw89_dev *rtwdev) +{ + struct rtw89_hal *hal = &rtwdev->hal; + + return READ_ONCE(hal->entity_active); +} + +static inline void rtw89_set_entity_state(struct rtw89_dev *rtwdev, bool active) +{ + struct rtw89_hal *hal = &rtwdev->hal; + + WRITE_ONCE(hal->entity_active, active); +} + +static inline +enum rtw89_entity_mode rtw89_get_entity_mode(struct rtw89_dev *rtwdev) +{ + struct rtw89_hal *hal = &rtwdev->hal; + + return READ_ONCE(hal->entity_mode); +} + +static inline void rtw89_set_entity_mode(struct rtw89_dev *rtwdev, + enum rtw89_entity_mode mode) +{ + struct rtw89_hal *hal = &rtwdev->hal; + + WRITE_ONCE(hal->entity_mode, mode); +} + +void rtw89_chan_create(struct rtw89_chan *chan, u8 center_chan, u8 primary_chan, + enum rtw89_band band, enum rtw89_bandwidth bandwidth); +bool rtw89_assign_entity_chan(struct rtw89_dev *rtwdev, + enum rtw89_sub_entity_idx idx, + const struct rtw89_chan *new); +void rtw89_config_entity_chandef(struct rtw89_dev *rtwdev, + enum rtw89_sub_entity_idx idx, + const struct cfg80211_chan_def *chandef); +void rtw89_entity_init(struct rtw89_dev *rtwdev); +enum rtw89_entity_mode rtw89_entity_recalc(struct rtw89_dev *rtwdev); +int rtw89_chanctx_ops_add(struct rtw89_dev *rtwdev, + struct ieee80211_chanctx_conf *ctx); +void rtw89_chanctx_ops_remove(struct rtw89_dev *rtwdev, + struct ieee80211_chanctx_conf *ctx); +void rtw89_chanctx_ops_change(struct rtw89_dev *rtwdev, + struct ieee80211_chanctx_conf *ctx, + u32 changed); +int rtw89_chanctx_ops_assign_vif(struct rtw89_dev *rtwdev, + struct rtw89_vif *rtwvif, + struct ieee80211_chanctx_conf *ctx); +void rtw89_chanctx_ops_unassign_vif(struct rtw89_dev *rtwdev, + struct rtw89_vif *rtwvif, + struct ieee80211_chanctx_conf *ctx); + +#endif diff --git a/drivers/net/wireless/realtek/rtw89/coex.c b/drivers/net/wireless/realtek/rtw89/coex.c index 683854bba217..bbdfa9ac203c 100644 --- a/drivers/net/wireless/realtek/rtw89/coex.c +++ b/drivers/net/wireless/realtek/rtw89/coex.c @@ -9,6 +9,7 @@ #include "ps.h" #include "reg.h" +#define RTW89_COEX_VERSION 0x06030013 #define FCXDEF_STEP 50 /* MUST <= FCXMAX_STEP and match with wl fw*/ enum btc_fbtc_tdma_template { @@ -77,21 +78,21 @@ static const struct rtw89_btc_fbtc_tdma t_def[] = { static const struct rtw89_btc_fbtc_slot s_def[] = { [CXST_OFF] = __DEF_FBTC_SLOT(100, 0x55555555, SLOT_MIX), - [CXST_B2W] = __DEF_FBTC_SLOT(5, 0x5a5a5a5a, SLOT_ISO), - [CXST_W1] = __DEF_FBTC_SLOT(70, 0x5a5a5a5a, SLOT_ISO), - [CXST_W2] = __DEF_FBTC_SLOT(70, 0x5a5a5aaa, SLOT_ISO), - [CXST_W2B] = __DEF_FBTC_SLOT(15, 0x5a5a5a5a, SLOT_ISO), - [CXST_B1] = __DEF_FBTC_SLOT(100, 0x55555555, SLOT_MIX), - [CXST_B2] = __DEF_FBTC_SLOT(7, 0x6a5a5a5a, SLOT_MIX), - [CXST_B3] = __DEF_FBTC_SLOT(5, 0x55555555, SLOT_MIX), - [CXST_B4] = __DEF_FBTC_SLOT(50, 0x55555555, SLOT_MIX), - [CXST_LK] = __DEF_FBTC_SLOT(20, 0x5a5a5a5a, SLOT_ISO), + [CXST_B2W] = __DEF_FBTC_SLOT(5, 0xea5a5a5a, SLOT_ISO), + [CXST_W1] = __DEF_FBTC_SLOT(70, 0xea5a5a5a, SLOT_ISO), + [CXST_W2] = __DEF_FBTC_SLOT(70, 0xea5a5aaa, SLOT_ISO), + [CXST_W2B] = __DEF_FBTC_SLOT(15, 0xea5a5a5a, SLOT_ISO), + [CXST_B1] = __DEF_FBTC_SLOT(100, 0xe5555555, SLOT_MIX), + [CXST_B2] = __DEF_FBTC_SLOT(7, 0xea5a5a5a, SLOT_MIX), + [CXST_B3] = __DEF_FBTC_SLOT(5, 0xe5555555, SLOT_MIX), + [CXST_B4] = __DEF_FBTC_SLOT(50, 0xe5555555, SLOT_MIX), + [CXST_LK] = __DEF_FBTC_SLOT(20, 0xea5a5a5a, SLOT_ISO), [CXST_BLK] = __DEF_FBTC_SLOT(250, 0x55555555, SLOT_MIX), - [CXST_E2G] = __DEF_FBTC_SLOT(20, 0x6a5a5a5a, SLOT_MIX), + [CXST_E2G] = __DEF_FBTC_SLOT(20, 0xea5a5a5a, SLOT_MIX), [CXST_E5G] = __DEF_FBTC_SLOT(20, 0xffffffff, SLOT_MIX), - [CXST_EBT] = __DEF_FBTC_SLOT(20, 0x55555555, SLOT_MIX), + [CXST_EBT] = __DEF_FBTC_SLOT(20, 0xe5555555, SLOT_MIX), [CXST_ENULL] = __DEF_FBTC_SLOT(7, 0xaaaaaaaa, SLOT_ISO), - [CXST_WLK] = __DEF_FBTC_SLOT(250, 0x6a5a6a5a, SLOT_MIX), + [CXST_WLK] = __DEF_FBTC_SLOT(250, 0xea5a5a5a, SLOT_MIX), [CXST_W1FDD] = __DEF_FBTC_SLOT(35, 0xfafafafa, SLOT_ISO), [CXST_B1FDD] = __DEF_FBTC_SLOT(100, 0xffffffff, SLOT_MIX), }; @@ -99,13 +100,13 @@ static const struct rtw89_btc_fbtc_slot s_def[] = { static const u32 cxtbl[] = { 0xffffffff, /* 0 */ 0xaaaaaaaa, /* 1 */ - 0x55555555, /* 2 */ - 0x66555555, /* 3 */ - 0x66556655, /* 4 */ + 0xe5555555, /* 2 */ + 0xee555555, /* 3 */ + 0xd5555555, /* 4 */ 0x5a5a5a5a, /* 5 */ - 0x5a5a5aaa, /* 6 */ - 0xaa5a5a5a, /* 7 */ - 0x6a5a5a5a, /* 8 */ + 0xfa5a5a5a, /* 6 */ + 0xda5a5a5a, /* 7 */ + 0xea5a5a5a, /* 8 */ 0x6a5a5aaa, /* 9 */ 0x6a5a6a5a, /* 10 */ 0x6a5a6aaa, /* 11 */ @@ -261,6 +262,12 @@ enum btc_cx_poicy_type { /* TDMA off + pri: WL_Hi-Tx > BT_Hi_Rx, BT_Hi > WL > BT_Lo */ BTC_CXP_OFF_BWB1 = (BTC_CXP_OFF << 8) | 7, + /* TDMA off + pri: WL_Hi-Tx > BT, BT_Hi > other-WL > BT_Lo */ + BTC_CXP_OFF_BWB2 = (BTC_CXP_OFF << 8) | 8, + + /* TDMA off + pri: WL_Hi-Tx = BT */ + BTC_CXP_OFF_BWB3 = (BTC_CXP_OFF << 8) | 9, + /* TDMA off+Bcn-Protect + pri: WL_Hi-Tx > BT_Hi_Rx, BT_Hi > WL > BT_Lo*/ BTC_CXP_OFFB_BWB0 = (BTC_CXP_OFFB << 8) | 0, @@ -270,6 +277,21 @@ enum btc_cx_poicy_type { /* TDMA off + Ext-Ctrl + pri: E2G-slot block all BT */ BTC_CXP_OFFE_DEF2 = (BTC_CXP_OFFE << 8) | 1, + /* TDMA off + Ext-Ctrl + pri: default */ + BTC_CXP_OFFE_2GBWISOB = (BTC_CXP_OFFE << 8) | 2, + + /* TDMA off + Ext-Ctrl + pri: E2G-slot block all BT */ + BTC_CXP_OFFE_2GISOB = (BTC_CXP_OFFE << 8) | 3, + + /* TDMA off + Ext-Ctrl + pri: E2G-slot WL > BT */ + BTC_CXP_OFFE_2GBWMIXB = (BTC_CXP_OFFE << 8) | 4, + + /* TDMA off + Ext-Ctrl + pri: E2G/EBT-slot WL > BT */ + BTC_CXP_OFFE_WL = (BTC_CXP_OFFE << 8) | 5, + + /* TDMA off + Ext-Ctrl + pri: default */ + BTC_CXP_OFFE_2GBWMIXB2 = (BTC_CXP_OFFE << 8) | 6, + /* TDMA Fix slot-0: W1:B1 = 30:30 */ BTC_CXP_FIX_TD3030 = (BTC_CXP_FIX << 8) | 0, @@ -300,6 +322,9 @@ enum btc_cx_poicy_type { /* TDMA Fix slot-9: W1:B1 = 40:20 */ BTC_CXP_FIX_TD4020 = (BTC_CXP_FIX << 8) | 9, + /* TDMA Fix slot-9: W1:B1 = 40:10 */ + BTC_CXP_FIX_TD4010ISO = (BTC_CXP_FIX << 8) | 10, + /* PS-TDMA Fix slot-0: W1:B1 = 30:30 */ BTC_CXP_PFIX_TD3030 = (BTC_CXP_PFIX << 8) | 0, @@ -322,25 +347,25 @@ enum btc_cx_poicy_type { BTC_CXP_PFIX_TDW1B1 = (BTC_CXP_PFIX << 8) | 6, /* TDMA Auto slot-0: W1:B1 = 50:200 */ - BTC_CXP_AUTO_TD50200 = (BTC_CXP_AUTO << 8) | 0, + BTC_CXP_AUTO_TD50B1 = (BTC_CXP_AUTO << 8) | 0, /* TDMA Auto slot-1: W1:B1 = 60:200 */ - BTC_CXP_AUTO_TD60200 = (BTC_CXP_AUTO << 8) | 1, + BTC_CXP_AUTO_TD60B1 = (BTC_CXP_AUTO << 8) | 1, /* TDMA Auto slot-2: W1:B1 = 20:200 */ - BTC_CXP_AUTO_TD20200 = (BTC_CXP_AUTO << 8) | 2, + BTC_CXP_AUTO_TD20B1 = (BTC_CXP_AUTO << 8) | 2, /* TDMA Auto slot-3: W1:B1 = user-define */ BTC_CXP_AUTO_TDW1B1 = (BTC_CXP_AUTO << 8) | 3, /* PS-TDMA Auto slot-0: W1:B1 = 50:200 */ - BTC_CXP_PAUTO_TD50200 = (BTC_CXP_PAUTO << 8) | 0, + BTC_CXP_PAUTO_TD50B1 = (BTC_CXP_PAUTO << 8) | 0, /* PS-TDMA Auto slot-1: W1:B1 = 60:200 */ - BTC_CXP_PAUTO_TD60200 = (BTC_CXP_PAUTO << 8) | 1, + BTC_CXP_PAUTO_TD60B1 = (BTC_CXP_PAUTO << 8) | 1, /* PS-TDMA Auto slot-2: W1:B1 = 20:200 */ - BTC_CXP_PAUTO_TD20200 = (BTC_CXP_PAUTO << 8) | 2, + BTC_CXP_PAUTO_TD20B1 = (BTC_CXP_PAUTO << 8) | 2, /* PS-TDMA Auto slot-3: W1:B1 = user-define */ BTC_CXP_PAUTO_TDW1B1 = (BTC_CXP_PAUTO << 8) | 3, @@ -412,7 +437,7 @@ enum btc_w2b_scoreboard { BTC_WSCB_TDMA = BIT(9), BTC_WSCB_FIX2M = BIT(10), BTC_WSCB_WLRFK = BIT(11), - BTC_WSCB_BTRFK_GNT = BIT(12), /* not used, use mailbox to inform BT */ + BTC_WSCB_RXSCAN_PRI = BIT(12), BTC_WSCB_BT_HILNA = BIT(13), BTC_WSCB_BTLOG = BIT(14), BTC_WSCB_ALL = GENMASK(23, 0), @@ -434,6 +459,16 @@ enum btc_wl_link_mode { BTC_WLINK_MAX }; +enum btc_wl_mrole_type { + BTC_WLMROLE_NONE = 0x0, + BTC_WLMROLE_STA_GC, + BTC_WLMROLE_STA_GC_NOA, + BTC_WLMROLE_STA_GO, + BTC_WLMROLE_STA_GO_NOA, + BTC_WLMROLE_STA_STA, + BTC_WLMROLE_MAX +}; + enum btc_bt_hid_type { BTC_HID_218 = BIT(0), BTC_HID_418 = BIT(1), @@ -460,6 +495,11 @@ enum btc_gnt_state { BTC_GNT_MAX }; +enum btc_ctr_path { + BTC_CTRL_BY_BT = 0, + BTC_CTRL_BY_WL +}; + enum btc_wl_max_tx_time { BTC_MAX_TX_TIME_L1 = 500, BTC_MAX_TX_TIME_L2 = 1000, @@ -531,6 +571,7 @@ enum btc_reason_and_action { #define BTC_FREERUN_ANTISO_MIN 30 #define BTC_TDMA_BTHID_MAX 2 #define BTC_BLINK_NOCONNECT 0 +#define BTC_B1_MAX 250 /* unit ms */ static void _run_coex(struct rtw89_dev *rtwdev, enum btc_reason_and_action reason); @@ -551,8 +592,10 @@ static void _send_fw_cmd(struct rtw89_dev *rtwdev, u8 h2c_class, u8 h2c_func, "[BTC], %s(): return by btc not init!!\n", __func__); pfwinfo->cnt_h2c_fail++; return; - } else if ((wl->status.map.rf_off_pre == 1 && wl->status.map.rf_off == 1) || - (wl->status.map.lps_pre == 1 && wl->status.map.lps == 1)) { + } else if ((wl->status.map.rf_off_pre == BTC_LPS_RF_OFF && + wl->status.map.rf_off == BTC_LPS_RF_OFF) || + (wl->status.map.lps_pre == BTC_LPS_RF_OFF && + wl->status.map.lps == BTC_LPS_RF_OFF)) { rtw89_debug(rtwdev, RTW89_DBG_BTC, "[BTC], %s(): return by wl off!!\n", __func__); pfwinfo->cnt_h2c_fail++; @@ -616,8 +659,6 @@ static void _reset_btc_var(struct rtw89_dev *rtwdev, u8 type) memset(&btc->mdinfo, 0, sizeof(btc->mdinfo)); } -#define BTC_FWINFO_BUF 1024 - #define BTC_RPT_HDR_SIZE 3 #define BTC_CHK_WLSLOT_DRIFT_MAX 15 #define BTC_CHK_HANG_MAX 3 @@ -869,18 +910,24 @@ static u32 _chk_btc_report(struct rtw89_dev *rtwdev, struct rtw89_btc_btf_fwinfo *pfwinfo, u8 *prptbuf, u32 index) { + const struct rtw89_chip_info *chip = rtwdev->chip; struct rtw89_btc *btc = &rtwdev->btc; struct rtw89_btc_dm *dm = &btc->dm; struct rtw89_btc_rpt_cmn_info *pcinfo = NULL; struct rtw89_btc_wl_info *wl = &btc->cx.wl; - struct rtw89_btc_fbtc_rpt_ctrl *prpt = NULL; + struct rtw89_btc_bt_info *bt = &btc->cx.bt; + struct rtw89_btc_fbtc_rpt_ctrl *prpt; + struct rtw89_btc_fbtc_rpt_ctrl_v1 *prpt_v1; struct rtw89_btc_fbtc_cysta *pcysta_le32 = NULL; + struct rtw89_btc_fbtc_cysta_v1 *pcysta_v1 = NULL; struct rtw89_btc_fbtc_cysta_cpu pcysta[1]; struct rtw89_btc_prpt *btc_prpt = NULL; struct rtw89_btc_fbtc_slot *rtp_slot = NULL; - u8 rpt_type = 0, *rpt_content = NULL, *pfinfo = NULL; - u16 wl_slot_set = 0; + void *rpt_content = NULL, *pfinfo = NULL; + u8 rpt_type = 0; + u16 wl_slot_set = 0, wl_slot_real = 0; u32 trace_step = btc->ctrl.trace_step, rpt_len = 0, diff_t; + u32 cnt_leak_slot = 0, bt_slot_real = 0, cnt_rx_imr = 0; u8 i; rtw89_debug(rtwdev, RTW89_DBG_BTC, @@ -904,100 +951,129 @@ static u32 _chk_btc_report(struct rtw89_dev *rtwdev, switch (rpt_type) { case BTC_RPT_TYPE_CTRL: pcinfo = &pfwinfo->rpt_ctrl.cinfo; - pfinfo = (u8 *)(&pfwinfo->rpt_ctrl.finfo); - pcinfo->req_len = sizeof(pfwinfo->rpt_ctrl.finfo); - pcinfo->req_fver = BTCRPT_VER; + if (chip->chip_id == RTL8852A) { + pfinfo = &pfwinfo->rpt_ctrl.finfo; + pcinfo->req_len = sizeof(pfwinfo->rpt_ctrl.finfo); + } else { + pfinfo = &pfwinfo->rpt_ctrl.finfo_v1; + pcinfo->req_len = sizeof(pfwinfo->rpt_ctrl.finfo_v1); + } + pcinfo->req_fver = chip->fcxbtcrpt_ver; pcinfo->rx_len = rpt_len; pcinfo->rx_cnt++; break; case BTC_RPT_TYPE_TDMA: pcinfo = &pfwinfo->rpt_fbtc_tdma.cinfo; - pfinfo = (u8 *)(&pfwinfo->rpt_fbtc_tdma.finfo); - pcinfo->req_len = sizeof(pfwinfo->rpt_fbtc_tdma.finfo); - pcinfo->req_fver = FCXTDMA_VER; + if (chip->chip_id == RTL8852A) { + pfinfo = &pfwinfo->rpt_fbtc_tdma.finfo; + pcinfo->req_len = sizeof(pfwinfo->rpt_fbtc_tdma.finfo); + } else { + pfinfo = &pfwinfo->rpt_fbtc_tdma.finfo_v1; + pcinfo->req_len = sizeof(pfwinfo->rpt_fbtc_tdma.finfo_v1); + } + pcinfo->req_fver = chip->fcxtdma_ver; pcinfo->rx_len = rpt_len; pcinfo->rx_cnt++; break; case BTC_RPT_TYPE_SLOT: pcinfo = &pfwinfo->rpt_fbtc_slots.cinfo; - pfinfo = (u8 *)(&pfwinfo->rpt_fbtc_slots.finfo); + pfinfo = &pfwinfo->rpt_fbtc_slots.finfo; pcinfo->req_len = sizeof(pfwinfo->rpt_fbtc_slots.finfo); - pcinfo->req_fver = FCXSLOTS_VER; + pcinfo->req_fver = chip->fcxslots_ver; pcinfo->rx_len = rpt_len; pcinfo->rx_cnt++; break; case BTC_RPT_TYPE_CYSTA: pcinfo = &pfwinfo->rpt_fbtc_cysta.cinfo; - pfinfo = (u8 *)(&pfwinfo->rpt_fbtc_cysta.finfo); - pcysta_le32 = &pfwinfo->rpt_fbtc_cysta.finfo; - rtw89_btc_fbtc_cysta_to_cpu(pcysta_le32, pcysta); - pcinfo->req_len = sizeof(pfwinfo->rpt_fbtc_cysta.finfo); - pcinfo->req_fver = FCXCYSTA_VER; + if (chip->chip_id == RTL8852A) { + pfinfo = &pfwinfo->rpt_fbtc_cysta.finfo; + pcysta_le32 = &pfwinfo->rpt_fbtc_cysta.finfo; + rtw89_btc_fbtc_cysta_to_cpu(pcysta_le32, pcysta); + pcinfo->req_len = sizeof(pfwinfo->rpt_fbtc_cysta.finfo); + } else { + pfinfo = &pfwinfo->rpt_fbtc_cysta.finfo_v1; + pcysta_v1 = &pfwinfo->rpt_fbtc_cysta.finfo_v1; + pcinfo->req_len = sizeof(pfwinfo->rpt_fbtc_cysta.finfo_v1); + } + pcinfo->req_fver = chip->fcxcysta_ver; pcinfo->rx_len = rpt_len; pcinfo->rx_cnt++; break; case BTC_RPT_TYPE_STEP: pcinfo = &pfwinfo->rpt_fbtc_step.cinfo; - pfinfo = (u8 *)(&pfwinfo->rpt_fbtc_step.finfo); - pcinfo->req_len = sizeof(pfwinfo->rpt_fbtc_step.finfo.step[0]) * - trace_step + 8; - pcinfo->req_fver = FCXSTEP_VER; + if (chip->chip_id == RTL8852A) { + pfinfo = &pfwinfo->rpt_fbtc_step.finfo; + pcinfo->req_len = sizeof(pfwinfo->rpt_fbtc_step.finfo.step[0]) * + trace_step + + offsetof(struct rtw89_btc_fbtc_steps, step); + } else { + pfinfo = &pfwinfo->rpt_fbtc_step.finfo_v1; + pcinfo->req_len = sizeof(pfwinfo->rpt_fbtc_step.finfo_v1.step[0]) * + trace_step + + offsetof(struct rtw89_btc_fbtc_steps_v1, step); + } + pcinfo->req_fver = chip->fcxstep_ver; pcinfo->rx_len = rpt_len; pcinfo->rx_cnt++; break; case BTC_RPT_TYPE_NULLSTA: pcinfo = &pfwinfo->rpt_fbtc_nullsta.cinfo; - pfinfo = (u8 *)(&pfwinfo->rpt_fbtc_nullsta.finfo); - pcinfo->req_len = sizeof(pfwinfo->rpt_fbtc_nullsta.finfo); - pcinfo->req_fver = FCXNULLSTA_VER; + if (chip->chip_id == RTL8852A) { + pfinfo = &pfwinfo->rpt_fbtc_nullsta.finfo; + pcinfo->req_len = sizeof(pfwinfo->rpt_fbtc_nullsta.finfo); + } else { + pfinfo = &pfwinfo->rpt_fbtc_nullsta.finfo_v1; + pcinfo->req_len = sizeof(pfwinfo->rpt_fbtc_nullsta.finfo_v1); + } + pcinfo->req_fver = chip->fcxnullsta_ver; pcinfo->rx_len = rpt_len; pcinfo->rx_cnt++; break; case BTC_RPT_TYPE_MREG: pcinfo = &pfwinfo->rpt_fbtc_mregval.cinfo; - pfinfo = (u8 *)(&pfwinfo->rpt_fbtc_mregval.finfo); + pfinfo = &pfwinfo->rpt_fbtc_mregval.finfo; pcinfo->req_len = sizeof(pfwinfo->rpt_fbtc_mregval.finfo); - pcinfo->req_fver = FCXMREG_VER; + pcinfo->req_fver = chip->fcxmreg_ver; pcinfo->rx_len = rpt_len; pcinfo->rx_cnt++; break; case BTC_RPT_TYPE_GPIO_DBG: pcinfo = &pfwinfo->rpt_fbtc_gpio_dbg.cinfo; - pfinfo = (u8 *)(&pfwinfo->rpt_fbtc_gpio_dbg.finfo); + pfinfo = &pfwinfo->rpt_fbtc_gpio_dbg.finfo; pcinfo->req_len = sizeof(pfwinfo->rpt_fbtc_gpio_dbg.finfo); - pcinfo->req_fver = FCXGPIODBG_VER; + pcinfo->req_fver = chip->fcxgpiodbg_ver; pcinfo->rx_len = rpt_len; pcinfo->rx_cnt++; break; case BTC_RPT_TYPE_BT_VER: pcinfo = &pfwinfo->rpt_fbtc_btver.cinfo; - pfinfo = (u8 *)(&pfwinfo->rpt_fbtc_btver.finfo); + pfinfo = &pfwinfo->rpt_fbtc_btver.finfo; pcinfo->req_len = sizeof(pfwinfo->rpt_fbtc_btver.finfo); - pcinfo->req_fver = FCX_BTVER_VER; + pcinfo->req_fver = chip->fcxbtver_ver; pcinfo->rx_len = rpt_len; pcinfo->rx_cnt++; break; case BTC_RPT_TYPE_BT_SCAN: pcinfo = &pfwinfo->rpt_fbtc_btscan.cinfo; - pfinfo = (u8 *)(&pfwinfo->rpt_fbtc_btscan.finfo); + pfinfo = &pfwinfo->rpt_fbtc_btscan.finfo; pcinfo->req_len = sizeof(pfwinfo->rpt_fbtc_btscan.finfo); - pcinfo->req_fver = FCX_BTSCAN_VER; + pcinfo->req_fver = chip->fcxbtscan_ver; pcinfo->rx_len = rpt_len; pcinfo->rx_cnt++; break; case BTC_RPT_TYPE_BT_AFH: pcinfo = &pfwinfo->rpt_fbtc_btafh.cinfo; - pfinfo = (u8 *)(&pfwinfo->rpt_fbtc_btafh.finfo); + pfinfo = &pfwinfo->rpt_fbtc_btafh.finfo; pcinfo->req_len = sizeof(pfwinfo->rpt_fbtc_btafh.finfo); - pcinfo->req_fver = FCX_BTAFH_VER; + pcinfo->req_fver = chip->fcxbtafh_ver; pcinfo->rx_len = rpt_len; pcinfo->rx_cnt++; break; case BTC_RPT_TYPE_BT_DEVICE: pcinfo = &pfwinfo->rpt_fbtc_btdev.cinfo; - pfinfo = (u8 *)(&pfwinfo->rpt_fbtc_btdev.finfo); + pfinfo = &pfwinfo->rpt_fbtc_btdev.finfo; pcinfo->req_len = sizeof(pfwinfo->rpt_fbtc_btdev.finfo); - pcinfo->req_fver = FCX_BTDEVINFO_VER; + pcinfo->req_fver = chip->fcxbtdevinfo_ver; pcinfo->rx_len = rpt_len; pcinfo->rx_cnt++; break; @@ -1026,7 +1102,7 @@ static u32 _chk_btc_report(struct rtw89_dev *rtwdev, memcpy(pfinfo, rpt_content, pcinfo->req_len); pcinfo->valid = 1; - if (rpt_type == BTC_RPT_TYPE_TDMA) { + if (rpt_type == BTC_RPT_TYPE_TDMA && chip->chip_id == RTL8852A) { rtw89_debug(rtwdev, RTW89_DBG_BTC, "[BTC], %s(): check %d %zu\n", __func__, BTC_DCNT_TDMA_NONSYNC, sizeof(dm->tdma_now)); @@ -1039,7 +1115,8 @@ static u32 _chk_btc_report(struct rtw89_dev *rtwdev, dm->tdma_now.type, dm->tdma_now.rxflctrl, dm->tdma_now.txpause, dm->tdma_now.wtgle_n, dm->tdma_now.leak_n, dm->tdma_now.ext_ctrl, - dm->tdma_now.rsvd0, dm->tdma_now.rsvd1); + dm->tdma_now.rxflctrl_role, + dm->tdma_now.option_ctrl); rtw89_debug(rtwdev, RTW89_DBG_BTC, "[BTC], %s(): %d rpt_fbtc_tdma %x %x %x %x %x %x %x %x\n", @@ -1050,14 +1127,46 @@ static u32 _chk_btc_report(struct rtw89_dev *rtwdev, pfwinfo->rpt_fbtc_tdma.finfo.wtgle_n, pfwinfo->rpt_fbtc_tdma.finfo.leak_n, pfwinfo->rpt_fbtc_tdma.finfo.ext_ctrl, - pfwinfo->rpt_fbtc_tdma.finfo.rsvd0, - pfwinfo->rpt_fbtc_tdma.finfo.rsvd1); + pfwinfo->rpt_fbtc_tdma.finfo.rxflctrl_role, + pfwinfo->rpt_fbtc_tdma.finfo.option_ctrl); } _chk_btc_err(rtwdev, BTC_DCNT_TDMA_NONSYNC, memcmp(&dm->tdma_now, &pfwinfo->rpt_fbtc_tdma.finfo, sizeof(dm->tdma_now))); + } else if (rpt_type == BTC_RPT_TYPE_TDMA) { + rtw89_debug(rtwdev, RTW89_DBG_BTC, + "[BTC], %s(): check %d %zu\n", __func__, + BTC_DCNT_TDMA_NONSYNC, sizeof(dm->tdma_now)); + + if (memcmp(&dm->tdma_now, &pfwinfo->rpt_fbtc_tdma.finfo_v1.tdma, + sizeof(dm->tdma_now)) != 0) { + rtw89_debug(rtwdev, RTW89_DBG_BTC, + "[BTC], %s(): %d tdma_now %x %x %x %x %x %x %x %x\n", + __func__, BTC_DCNT_TDMA_NONSYNC, + dm->tdma_now.type, dm->tdma_now.rxflctrl, + dm->tdma_now.txpause, dm->tdma_now.wtgle_n, + dm->tdma_now.leak_n, dm->tdma_now.ext_ctrl, + dm->tdma_now.rxflctrl_role, + dm->tdma_now.option_ctrl); + rtw89_debug(rtwdev, RTW89_DBG_BTC, + "[BTC], %s(): %d rpt_fbtc_tdma %x %x %x %x %x %x %x %x\n", + __func__, BTC_DCNT_TDMA_NONSYNC, + pfwinfo->rpt_fbtc_tdma.finfo_v1.tdma.type, + pfwinfo->rpt_fbtc_tdma.finfo_v1.tdma.rxflctrl, + pfwinfo->rpt_fbtc_tdma.finfo_v1.tdma.txpause, + pfwinfo->rpt_fbtc_tdma.finfo_v1.tdma.wtgle_n, + pfwinfo->rpt_fbtc_tdma.finfo_v1.tdma.leak_n, + pfwinfo->rpt_fbtc_tdma.finfo_v1.tdma.ext_ctrl, + pfwinfo->rpt_fbtc_tdma.finfo_v1.tdma.rxflctrl_role, + pfwinfo->rpt_fbtc_tdma.finfo_v1.tdma.option_ctrl); + } + + _chk_btc_err(rtwdev, BTC_DCNT_TDMA_NONSYNC, + memcmp(&dm->tdma_now, + &pfwinfo->rpt_fbtc_tdma.finfo_v1.tdma, + sizeof(dm->tdma_now))); } if (rpt_type == BTC_RPT_TYPE_SLOT) { @@ -1097,7 +1206,7 @@ static u32 _chk_btc_report(struct rtw89_dev *rtwdev, sizeof(dm->slot_now))); } - if (rpt_type == BTC_RPT_TYPE_CYSTA && + if (rpt_type == BTC_RPT_TYPE_CYSTA && chip->chip_id == RTL8852A && pcysta->cycles >= BTC_CYSTA_CHK_PERIOD) { /* Check Leak-AP */ if (pcysta->slot_cnt[CXST_LK] != 0 && @@ -1120,16 +1229,55 @@ static u32 _chk_btc_report(struct rtw89_dev *rtwdev, } _chk_btc_err(rtwdev, BTC_DCNT_W1_FREEZE, pcysta->slot_cnt[CXST_W1]); - _chk_btc_err(rtwdev, BTC_DCNT_W1_FREEZE, pcysta->slot_cnt[CXST_W1]); + _chk_btc_err(rtwdev, BTC_DCNT_W1_FREEZE, pcysta->slot_cnt[CXST_B1]); _chk_btc_err(rtwdev, BTC_DCNT_CYCLE_FREEZE, (u32)pcysta->cycles); + } else if (rpt_type == BTC_RPT_TYPE_CYSTA && pcysta_v1 && + le16_to_cpu(pcysta_v1->cycles) >= BTC_CYSTA_CHK_PERIOD) { + cnt_leak_slot = le32_to_cpu(pcysta_v1->slot_cnt[CXST_LK]); + cnt_rx_imr = le32_to_cpu(pcysta_v1->leak_slot.cnt_rximr); + /* Check Leak-AP */ + if (cnt_leak_slot != 0 && cnt_rx_imr != 0 && + dm->tdma_now.rxflctrl) { + if (cnt_leak_slot < BTC_LEAK_AP_TH * cnt_rx_imr) + dm->leak_ap = 1; + } + + /* Check diff time between real WL slot and W1 slot */ + if (dm->tdma_now.type == CXTDMA_OFF) { + wl_slot_set = le16_to_cpu(dm->slot_now[CXST_W1].dur); + wl_slot_real = le16_to_cpu(pcysta_v1->cycle_time.tavg[CXT_WL]); + if (wl_slot_real > wl_slot_set) { + diff_t = wl_slot_real - wl_slot_set; + _chk_btc_err(rtwdev, BTC_DCNT_WL_SLOT_DRIFT, diff_t); + } + } + + /* Check diff time between real BT slot and EBT/E5G slot */ + if (dm->tdma_now.type == CXTDMA_OFF && + dm->tdma_now.ext_ctrl == CXECTL_EXT && + btc->bt_req_len != 0) { + bt_slot_real = le16_to_cpu(pcysta_v1->cycle_time.tavg[CXT_BT]); + + if (btc->bt_req_len > bt_slot_real) { + diff_t = btc->bt_req_len - bt_slot_real; + _chk_btc_err(rtwdev, BTC_DCNT_BT_SLOT_DRIFT, diff_t); + } + } + + _chk_btc_err(rtwdev, BTC_DCNT_W1_FREEZE, + le32_to_cpu(pcysta_v1->slot_cnt[CXST_W1])); + _chk_btc_err(rtwdev, BTC_DCNT_B1_FREEZE, + le32_to_cpu(pcysta_v1->slot_cnt[CXST_B1])); + _chk_btc_err(rtwdev, BTC_DCNT_CYCLE_FREEZE, + (u32)le16_to_cpu(pcysta_v1->cycles)); } - if (rpt_type == BTC_RPT_TYPE_CTRL) { + if (rpt_type == BTC_RPT_TYPE_CTRL && chip->chip_id == RTL8852A) { prpt = &pfwinfo->rpt_ctrl.finfo; btc->fwinfo.rpt_en_map = prpt->rpt_enable; wl->ver_info.fw_coex = prpt->wl_fw_coex_ver; wl->ver_info.fw = prpt->wl_fw_ver; - dm->wl_fw_cx_offload = !!(prpt->wl_fw_cx_offload); + dm->wl_fw_cx_offload = !!prpt->wl_fw_cx_offload; _chk_btc_err(rtwdev, BTC_DCNT_RPT_FREEZE, pfwinfo->event[BTF_EVNT_RPT]); @@ -1142,6 +1290,33 @@ static u32 _chk_btc_report(struct rtw89_dev *rtwdev, btc->cx.cnt_bt[BTC_BCNT_POLUT] = rtw89_mac_get_plt_cnt(rtwdev, RTW89_MAC_0); } + } else if (rpt_type == BTC_RPT_TYPE_CTRL) { + prpt_v1 = &pfwinfo->rpt_ctrl.finfo_v1; + btc->fwinfo.rpt_en_map = le32_to_cpu(prpt_v1->rpt_info.en); + wl->ver_info.fw_coex = le32_to_cpu(prpt_v1->wl_fw_info.cx_ver); + wl->ver_info.fw = le32_to_cpu(prpt_v1->wl_fw_info.fw_ver); + dm->wl_fw_cx_offload = !!le32_to_cpu(prpt_v1->wl_fw_info.cx_offload); + + for (i = RTW89_PHY_0; i < RTW89_PHY_MAX; i++) + memcpy(&dm->gnt.band[i], &prpt_v1->gnt_val[i], + sizeof(dm->gnt.band[i])); + + btc->cx.cnt_bt[BTC_BCNT_HIPRI_TX] = le32_to_cpu(prpt_v1->bt_cnt[BTC_BCNT_HI_TX]); + btc->cx.cnt_bt[BTC_BCNT_HIPRI_RX] = le32_to_cpu(prpt_v1->bt_cnt[BTC_BCNT_HI_RX]); + btc->cx.cnt_bt[BTC_BCNT_LOPRI_TX] = le32_to_cpu(prpt_v1->bt_cnt[BTC_BCNT_LO_TX]); + btc->cx.cnt_bt[BTC_BCNT_LOPRI_RX] = le32_to_cpu(prpt_v1->bt_cnt[BTC_BCNT_LO_RX]); + btc->cx.cnt_bt[BTC_BCNT_POLUT] = le32_to_cpu(prpt_v1->bt_cnt[BTC_BCNT_POLLUTED]); + + _chk_btc_err(rtwdev, BTC_DCNT_BTCNT_FREEZE, 0); + _chk_btc_err(rtwdev, BTC_DCNT_RPT_FREEZE, + pfwinfo->event[BTF_EVNT_RPT]); + + if (le32_to_cpu(prpt_v1->bt_cnt[BTC_BCNT_RFK_TIMEOUT]) > 0) + bt->rfk_info.map.timeout = 1; + else + bt->rfk_info.map.timeout = 0; + + dm->error.map.bt_rfk_timeout = bt->rfk_info.map.timeout; } if (rpt_type >= BTC_RPT_TYPE_BT_VER && @@ -1155,6 +1330,7 @@ static void _parse_btc_report(struct rtw89_dev *rtwdev, struct rtw89_btc_btf_fwinfo *pfwinfo, u8 *pbuf, u32 buf_len) { + const struct rtw89_chip_info *chip = rtwdev->chip; struct rtw89_btc_prpt *btc_prpt = NULL; u32 index = 0, rpt_len = 0; @@ -1164,7 +1340,7 @@ static void _parse_btc_report(struct rtw89_dev *rtwdev, while (pbuf) { btc_prpt = (struct rtw89_btc_prpt *)&pbuf[index]; - if (index + 2 >= BTC_FWINFO_BUF) + if (index + 2 >= chip->btc_fwinfo_buf) break; /* At least 3 bytes: type(1) & len(2) */ rpt_len = le16_to_cpu(btc_prpt->len); @@ -1182,10 +1358,12 @@ static void _parse_btc_report(struct rtw89_dev *rtwdev, static void _append_tdma(struct rtw89_dev *rtwdev) { + const struct rtw89_chip_info *chip = rtwdev->chip; struct rtw89_btc *btc = &rtwdev->btc; struct rtw89_btc_dm *dm = &btc->dm; - struct rtw89_btc_btf_tlv *tlv = NULL; - struct rtw89_btc_fbtc_tdma *v = NULL; + struct rtw89_btc_btf_tlv *tlv; + struct rtw89_btc_fbtc_tdma *v; + struct rtw89_btc_fbtc_tdma_v1 *v1; u16 len = btc->policy_len; if (!btc->update_policy_force && @@ -1197,12 +1375,19 @@ static void _append_tdma(struct rtw89_dev *rtwdev) } tlv = (struct rtw89_btc_btf_tlv *)&btc->policy[len]; - v = (struct rtw89_btc_fbtc_tdma *)&tlv->val[0]; tlv->type = CXPOLICY_TDMA; - tlv->len = sizeof(*v); - - memcpy(v, &dm->tdma, sizeof(*v)); - btc->policy_len += BTC_TLV_HDR_LEN + sizeof(*v); + if (chip->chip_id == RTL8852A) { + v = (struct rtw89_btc_fbtc_tdma *)&tlv->val[0]; + tlv->len = sizeof(*v); + memcpy(v, &dm->tdma, sizeof(*v)); + btc->policy_len += BTC_TLV_HDR_LEN + sizeof(*v); + } else { + tlv->len = sizeof(*v1); + v1 = (struct rtw89_btc_fbtc_tdma_v1 *)&tlv->val[0]; + v1->fver = chip->fcxtdma_ver; + v1->tdma = dm->tdma; + btc->policy_len += BTC_TLV_HDR_LEN + sizeof(*v1); + } rtw89_debug(rtwdev, RTW89_DBG_BTC, "[BTC], %s(): type:%d, rxflctrl=%d, txpause=%d, wtgle_n=%d, leak_n=%d, ext_ctrl=%d\n", @@ -1408,12 +1593,17 @@ static void _fw_set_policy(struct rtw89_dev *rtwdev, u16 policy_type, static void _fw_set_drv_info(struct rtw89_dev *rtwdev, u8 type) { + const struct rtw89_chip_info *chip = rtwdev->chip; + switch (type) { case CXDRVINFO_INIT: rtw89_fw_h2c_cxdrv_init(rtwdev); break; case CXDRVINFO_ROLE: - rtw89_fw_h2c_cxdrv_role(rtwdev); + if (chip->chip_id == RTL8852A) + rtw89_fw_h2c_cxdrv_role(rtwdev); + else + rtw89_fw_h2c_cxdrv_role_v1(rtwdev); break; case CXDRVINFO_CTRL: rtw89_fw_h2c_cxdrv_ctrl(rtwdev); @@ -1448,7 +1638,7 @@ void btc_fw_event(struct rtw89_dev *rtwdev, u8 evt_id, void *data, u32 len) } } -static void _set_gnt_wl(struct rtw89_dev *rtwdev, u8 phy_map, u8 state) +static void _set_gnt(struct rtw89_dev *rtwdev, u8 phy_map, u8 wl_state, u8 bt_state) { struct rtw89_btc *btc = &rtwdev->btc; struct rtw89_btc_dm *dm = &btc->dm; @@ -1462,7 +1652,7 @@ static void _set_gnt_wl(struct rtw89_dev *rtwdev, u8 phy_map, u8 state) if (!(phy_map & BIT(i))) continue; - switch (state) { + switch (wl_state) { case BTC_GNT_HW: g[i].gnt_wl_sw_en = 0; g[i].gnt_wl = 0; @@ -1476,6 +1666,21 @@ static void _set_gnt_wl(struct rtw89_dev *rtwdev, u8 phy_map, u8 state) g[i].gnt_wl = 1; break; } + + switch (bt_state) { + case BTC_GNT_HW: + g[i].gnt_bt_sw_en = 0; + g[i].gnt_bt = 0; + break; + case BTC_GNT_SW_LO: + g[i].gnt_bt_sw_en = 1; + g[i].gnt_bt = 0; + break; + case BTC_GNT_SW_HI: + g[i].gnt_bt_sw_en = 1; + g[i].gnt_bt = 1; + break; + } } rtw89_chip_mac_cfg_gnt(rtwdev, &dm->gnt); @@ -1534,6 +1739,7 @@ static void _set_wl_tx_power(struct rtw89_dev *rtwdev, u32 level) static void _set_wl_rx_gain(struct rtw89_dev *rtwdev, u32 level) { + const struct rtw89_chip_info *chip = rtwdev->chip; struct rtw89_btc *btc = &rtwdev->btc; struct rtw89_btc_wl_info *wl = &btc->cx.wl; @@ -1546,6 +1752,8 @@ static void _set_wl_rx_gain(struct rtw89_dev *rtwdev, u32 level) rtw89_debug(rtwdev, RTW89_DBG_BTC, "[BTC], %s(): level = %d\n", __func__, level); + + chip->ops->btc_set_wl_rx_gain(rtwdev, level); } static void _set_bt_tx_power(struct rtw89_dev *rtwdev, u8 level) @@ -1683,28 +1891,45 @@ static void _set_bt_afh_info(struct rtw89_dev *rtwdev) struct rtw89_btc_bt_info *bt = &btc->cx.bt; struct rtw89_btc_bt_link_info *b = &bt->link_info; struct rtw89_btc_wl_role_info *wl_rinfo = &wl->role_info; + struct rtw89_btc_wl_role_info_v1 *wl_rinfo_v1 = &wl->role_info_v1; + struct rtw89_btc_wl_active_role *r; + struct rtw89_btc_wl_active_role_v1 *r1; u8 en = 0, i, ch = 0, bw = 0; + u8 mode, connect_cnt; if (btc->ctrl.manual || wl->status.map.scan) return; - /* TODO if include module->ant.type == BTC_ANT_SHARED */ + if (chip->chip_id == RTL8852A) { + mode = wl_rinfo->link_mode; + connect_cnt = wl_rinfo->connect_cnt; + } else { + mode = wl_rinfo_v1->link_mode; + connect_cnt = wl_rinfo_v1->connect_cnt; + } + if (wl->status.map.rf_off || bt->whql_test || - wl_rinfo->link_mode == BTC_WLINK_NOLINK || - wl_rinfo->link_mode == BTC_WLINK_5G || - wl_rinfo->connect_cnt > BTC_TDMA_WLROLE_MAX) { + mode == BTC_WLINK_NOLINK || mode == BTC_WLINK_5G || + connect_cnt > BTC_TDMA_WLROLE_MAX) { en = false; - } else if (wl_rinfo->link_mode == BTC_WLINK_2G_MCC || - wl_rinfo->link_mode == BTC_WLINK_2G_SCC) { + } else if (mode == BTC_WLINK_2G_MCC || mode == BTC_WLINK_2G_SCC) { en = true; /* get p2p channel */ for (i = 0; i < RTW89_PORT_NUM; i++) { - if (wl_rinfo->active_role[i].role == - RTW89_WIFI_ROLE_P2P_GO || - wl_rinfo->active_role[i].role == - RTW89_WIFI_ROLE_P2P_CLIENT) { - ch = wl_rinfo->active_role[i].ch; - bw = wl_rinfo->active_role[i].bw; + r = &wl_rinfo->active_role[i]; + r1 = &wl_rinfo_v1->active_role_v1[i]; + + if (chip->chip_id == RTL8852A && + (r->role == RTW89_WIFI_ROLE_P2P_GO || + r->role == RTW89_WIFI_ROLE_P2P_CLIENT)) { + ch = r->ch; + bw = r->bw; + break; + } else if (chip->chip_id != RTL8852A && + (r1->role == RTW89_WIFI_ROLE_P2P_GO || + r1->role == RTW89_WIFI_ROLE_P2P_CLIENT)) { + ch = r1->ch; + bw = r1->bw; break; } } @@ -1712,10 +1937,18 @@ static void _set_bt_afh_info(struct rtw89_dev *rtwdev) en = true; /* get 2g channel */ for (i = 0; i < RTW89_PORT_NUM; i++) { - if (wl_rinfo->active_role[i].connected && - wl_rinfo->active_role[i].band == RTW89_BAND_2G) { - ch = wl_rinfo->active_role[i].ch; - bw = wl_rinfo->active_role[i].bw; + r = &wl_rinfo->active_role[i]; + r1 = &wl_rinfo_v1->active_role_v1[i]; + + if (chip->chip_id == RTL8852A && + r->connected && r->band == RTW89_BAND_2G) { + ch = r->ch; + bw = r->bw; + break; + } else if (chip->chip_id != RTL8852A && + r1->connected && r1->band == RTW89_BAND_2G) { + ch = r1->ch; + bw = r1->bw; break; } } @@ -1768,6 +2001,7 @@ static bool _check_freerun(struct rtw89_dev *rtwdev) struct rtw89_btc_wl_info *wl = &btc->cx.wl; struct rtw89_btc_bt_info *bt = &btc->cx.bt; struct rtw89_btc_wl_role_info *wl_rinfo = &wl->role_info; + struct rtw89_btc_wl_role_info_v1 *wl_rinfo_v1 = &wl->role_info_v1; struct rtw89_btc_bt_link_info *bt_linfo = &bt->link_info; struct rtw89_btc_bt_hid_desc *hid = &bt_linfo->hid_desc; @@ -1777,7 +2011,8 @@ static bool _check_freerun(struct rtw89_dev *rtwdev) } /* The below is dedicated antenna case */ - if (wl_rinfo->connect_cnt > BTC_TDMA_WLROLE_MAX) { + if (wl_rinfo->connect_cnt > BTC_TDMA_WLROLE_MAX || + wl_rinfo_v1->connect_cnt > BTC_TDMA_WLROLE_MAX) { btc->dm.trx_para_level = 5; return true; } @@ -1826,6 +2061,7 @@ static bool _check_freerun(struct rtw89_dev *rtwdev) } #define _tdma_set_flctrl(btc, flc) ({(btc)->dm.tdma.rxflctrl = flc; }) +#define _tdma_set_flctrl_role(btc, role) ({(btc)->dm.tdma.rxflctrl_role = role; }) #define _tdma_set_tog(btc, wtg) ({(btc)->dm.tdma.wtgle_n = wtg; }) #define _tdma_set_lek(btc, lek) ({(btc)->dm.tdma.leak_n = lek; }) @@ -1904,6 +2140,15 @@ union btc_btinfo { static void _set_policy(struct rtw89_dev *rtwdev, u16 policy_type, enum btc_reason_and_action action) { + const struct rtw89_chip_info *chip = rtwdev->chip; + + chip->ops->btc_set_policy(rtwdev, policy_type); + _fw_set_policy(rtwdev, policy_type, action); +} + +#define BTC_B1_MAX 250 /* unit ms */ +void rtw89_btc_set_policy(struct rtw89_dev *rtwdev, u16 policy_type) +{ struct rtw89_btc *btc = &rtwdev->btc; struct rtw89_btc_dm *dm = &btc->dm; struct rtw89_btc_fbtc_tdma *t = &dm->tdma; @@ -1964,6 +2209,9 @@ static void _set_policy(struct rtw89_dev *rtwdev, u16 policy_type, case BTC_CXP_OFF_BWB1: _slot_set_tbl(btc, CXST_OFF, cxtbl[8]); break; + case BTC_CXP_OFF_BWB3: + _slot_set_tbl(btc, CXST_OFF, cxtbl[6]); + break; } break; case BTC_CXP_OFFB: /* TDMA off + beacon protect */ @@ -2080,17 +2328,361 @@ static void _set_policy(struct rtw89_dev *rtwdev, u16 policy_type, _write_scbd(rtwdev, BTC_WSCB_TDMA, true); *t = t_def[CXTD_AUTO]; switch (policy_type) { - case BTC_CXP_AUTO_TD50200: + case BTC_CXP_AUTO_TD50B1: + _slot_set(btc, CXST_W1, 50, tbl_w1, SLOT_ISO); + _slot_set(btc, CXST_B1, BTC_B1_MAX, tbl_b1, SLOT_MIX); + break; + case BTC_CXP_AUTO_TD60B1: + _slot_set(btc, CXST_W1, 60, tbl_w1, SLOT_ISO); + _slot_set(btc, CXST_B1, BTC_B1_MAX, tbl_b1, SLOT_MIX); + break; + case BTC_CXP_AUTO_TD20B1: + _slot_set(btc, CXST_W1, 20, tbl_w1, SLOT_ISO); + _slot_set(btc, CXST_B1, BTC_B1_MAX, tbl_b1, SLOT_MIX); + break; + case BTC_CXP_AUTO_TDW1B1: /* W1:B1 = user-define */ + _slot_set(btc, CXST_W1, dm->slot_dur[CXST_W1], + tbl_w1, SLOT_ISO); + _slot_set(btc, CXST_B1, dm->slot_dur[CXST_B1], + tbl_b1, SLOT_MIX); + break; + } + break; + case BTC_CXP_PAUTO: /* PS-TDMA Auto-Slot */ + _write_scbd(rtwdev, BTC_WSCB_TDMA, true); + *t = t_def[CXTD_PAUTO]; + switch (policy_type) { + case BTC_CXP_PAUTO_TD50B1: + _slot_set(btc, CXST_W1, 50, tbl_w1, SLOT_ISO); + _slot_set(btc, CXST_B1, BTC_B1_MAX, tbl_b1, SLOT_MIX); + break; + case BTC_CXP_PAUTO_TD60B1: + _slot_set(btc, CXST_W1, 60, tbl_w1, SLOT_ISO); + _slot_set(btc, CXST_B1, BTC_B1_MAX, tbl_b1, SLOT_MIX); + break; + case BTC_CXP_PAUTO_TD20B1: + _slot_set(btc, CXST_W1, 20, tbl_w1, SLOT_ISO); + _slot_set(btc, CXST_B1, BTC_B1_MAX, tbl_b1, SLOT_MIX); + break; + case BTC_CXP_PAUTO_TDW1B1: + _slot_set(btc, CXST_W1, dm->slot_dur[CXST_W1], + tbl_w1, SLOT_ISO); + _slot_set(btc, CXST_B1, dm->slot_dur[CXST_B1], + tbl_b1, SLOT_MIX); + break; + } + break; + case BTC_CXP_AUTO2: /* TDMA Auto-Slot2 */ + _write_scbd(rtwdev, BTC_WSCB_TDMA, true); + *t = t_def[CXTD_AUTO2]; + switch (policy_type) { + case BTC_CXP_AUTO2_TD3050: + _slot_set(btc, CXST_W1, 30, tbl_w1, SLOT_ISO); + _slot_set(btc, CXST_B4, 50, tbl_b4, SLOT_MIX); + _slot_set(btc, CXST_B1, BTC_B1_MAX, tbl_b1, SLOT_MIX); + break; + case BTC_CXP_AUTO2_TD3070: + _slot_set(btc, CXST_W1, 30, tbl_w1, SLOT_ISO); + _slot_set(btc, CXST_B4, 70, tbl_b4, SLOT_MIX); + _slot_set(btc, CXST_B1, BTC_B1_MAX, tbl_b1, SLOT_MIX); + break; + case BTC_CXP_AUTO2_TD5050: + _slot_set(btc, CXST_W1, 50, tbl_w1, SLOT_ISO); + _slot_set(btc, CXST_B4, 50, tbl_b4, SLOT_MIX); + _slot_set(btc, CXST_B1, BTC_B1_MAX, tbl_b1, SLOT_MIX); + break; + case BTC_CXP_AUTO2_TD6060: + _slot_set(btc, CXST_W1, 60, tbl_w1, SLOT_ISO); + _slot_set(btc, CXST_B4, 60, tbl_b4, SLOT_MIX); + _slot_set(btc, CXST_B1, BTC_B1_MAX, tbl_b1, SLOT_MIX); + break; + case BTC_CXP_AUTO2_TD2080: + _slot_set(btc, CXST_W1, 20, tbl_w1, SLOT_ISO); + _slot_set(btc, CXST_B4, 80, tbl_b4, SLOT_MIX); + _slot_set(btc, CXST_B1, BTC_B1_MAX, tbl_b1, SLOT_MIX); + break; + case BTC_CXP_AUTO2_TDW1B4: /* W1:B1 = user-define */ + _slot_set(btc, CXST_W1, dm->slot_dur[CXST_W1], + tbl_w1, SLOT_ISO); + _slot_set(btc, CXST_B4, dm->slot_dur[CXST_B4], + tbl_b4, SLOT_MIX); + break; + } + break; + case BTC_CXP_PAUTO2: /* PS-TDMA Auto-Slot2 */ + _write_scbd(rtwdev, BTC_WSCB_TDMA, true); + *t = t_def[CXTD_PAUTO2]; + switch (policy_type) { + case BTC_CXP_PAUTO2_TD3050: + _slot_set(btc, CXST_W1, 30, tbl_w1, SLOT_ISO); + _slot_set(btc, CXST_B4, 50, tbl_b4, SLOT_MIX); + _slot_set(btc, CXST_B1, BTC_B1_MAX, tbl_b1, SLOT_MIX); + break; + case BTC_CXP_PAUTO2_TD3070: + _slot_set(btc, CXST_W1, 30, tbl_w1, SLOT_ISO); + _slot_set(btc, CXST_B4, 70, tbl_b4, SLOT_MIX); + _slot_set(btc, CXST_B1, BTC_B1_MAX, tbl_b1, SLOT_MIX); + break; + case BTC_CXP_PAUTO2_TD5050: + _slot_set(btc, CXST_W1, 50, tbl_w1, SLOT_ISO); + _slot_set(btc, CXST_B4, 50, tbl_b4, SLOT_MIX); + _slot_set(btc, CXST_B1, BTC_B1_MAX, tbl_b1, SLOT_MIX); + break; + case BTC_CXP_PAUTO2_TD6060: + _slot_set(btc, CXST_W1, 60, tbl_w1, SLOT_ISO); + _slot_set(btc, CXST_B4, 60, tbl_b4, SLOT_MIX); + _slot_set(btc, CXST_B1, BTC_B1_MAX, tbl_b1, SLOT_MIX); + break; + case BTC_CXP_PAUTO2_TD2080: + _slot_set(btc, CXST_W1, 20, tbl_w1, SLOT_ISO); + _slot_set(btc, CXST_B4, 80, tbl_b4, SLOT_MIX); + _slot_set(btc, CXST_B1, BTC_B1_MAX, tbl_b1, SLOT_MIX); + break; + case BTC_CXP_PAUTO2_TDW1B4: /* W1:B1 = user-define */ + _slot_set(btc, CXST_W1, dm->slot_dur[CXST_W1], + tbl_w1, SLOT_ISO); + _slot_set(btc, CXST_B4, dm->slot_dur[CXST_B4], + tbl_b4, SLOT_MIX); + break; + } + break; + } +} +EXPORT_SYMBOL(rtw89_btc_set_policy); + +void rtw89_btc_set_policy_v1(struct rtw89_dev *rtwdev, u16 policy_type) +{ + struct rtw89_btc *btc = &rtwdev->btc; + struct rtw89_btc_dm *dm = &btc->dm; + struct rtw89_btc_fbtc_tdma *t = &dm->tdma; + struct rtw89_btc_fbtc_slot *s = dm->slot; + struct rtw89_btc_wl_role_info_v1 *wl_rinfo = &btc->cx.wl.role_info_v1; + struct rtw89_btc_bt_hid_desc *hid = &btc->cx.bt.link_info.hid_desc; + struct rtw89_btc_bt_hfp_desc *hfp = &btc->cx.bt.link_info.hfp_desc; + u8 type, null_role; + u32 tbl_w1, tbl_b1, tbl_b4; + + type = FIELD_GET(BTC_CXP_MASK, policy_type); + + if (btc->mdinfo.ant.type == BTC_ANT_SHARED) { + if (btc->cx.wl.status.map._4way) + tbl_w1 = cxtbl[1]; + else if (hid->exist && hid->type == BTC_HID_218) + tbl_w1 = cxtbl[7]; /* Ack/BA no break bt Hi-Pri-rx */ + else + tbl_w1 = cxtbl[8]; + + if (dm->leak_ap && + (type == BTC_CXP_PFIX || type == BTC_CXP_PAUTO2)) { + tbl_b1 = cxtbl[3]; + tbl_b4 = cxtbl[3]; + } else if (hid->exist && hid->type == BTC_HID_218) { + tbl_b1 = cxtbl[4]; /* Ack/BA no break bt Hi-Pri-rx */ + tbl_b4 = cxtbl[4]; + } else { + tbl_b1 = cxtbl[2]; + tbl_b4 = cxtbl[2]; + } + } else { + tbl_w1 = cxtbl[16]; + tbl_b1 = cxtbl[17]; + tbl_b4 = cxtbl[17]; + } + + btc->bt_req_en = false; + + switch (type) { + case BTC_CXP_USERDEF0: + btc->update_policy_force = true; + *t = t_def[CXTD_OFF]; + s[CXST_OFF] = s_def[CXST_OFF]; + _slot_set_tbl(btc, CXST_OFF, cxtbl[2]); + break; + case BTC_CXP_OFF: /* TDMA off */ + _write_scbd(rtwdev, BTC_WSCB_TDMA, false); + *t = t_def[CXTD_OFF]; + s[CXST_OFF] = s_def[CXST_OFF]; + + switch (policy_type) { + case BTC_CXP_OFF_BT: + _slot_set_tbl(btc, CXST_OFF, cxtbl[2]); + break; + case BTC_CXP_OFF_WL: + _slot_set_tbl(btc, CXST_OFF, cxtbl[1]); + break; + case BTC_CXP_OFF_EQ0: + _slot_set_tbl(btc, CXST_OFF, cxtbl[0]); + break; + case BTC_CXP_OFF_EQ1: + _slot_set_tbl(btc, CXST_OFF, cxtbl[16]); + break; + case BTC_CXP_OFF_EQ2: + _slot_set_tbl(btc, CXST_OFF, cxtbl[17]); + break; + case BTC_CXP_OFF_EQ3: + _slot_set_tbl(btc, CXST_OFF, cxtbl[18]); + break; + case BTC_CXP_OFF_BWB0: + _slot_set_tbl(btc, CXST_OFF, cxtbl[5]); + break; + case BTC_CXP_OFF_BWB1: + _slot_set_tbl(btc, CXST_OFF, cxtbl[8]); + break; + case BTC_CXP_OFF_BWB2: + _slot_set_tbl(btc, CXST_OFF, cxtbl[7]); + break; + case BTC_CXP_OFF_BWB3: + _slot_set_tbl(btc, CXST_OFF, cxtbl[6]); + break; + default: + break; + } + break; + case BTC_CXP_OFFB: /* TDMA off + beacon protect */ + _write_scbd(rtwdev, BTC_WSCB_TDMA, false); + *t = t_def[CXTD_OFF_B2]; + s[CXST_OFF] = s_def[CXST_OFF]; + + switch (policy_type) { + case BTC_CXP_OFFB_BWB0: + _slot_set_tbl(btc, CXST_OFF, cxtbl[8]); + break; + default: + break; + } + break; + case BTC_CXP_OFFE: /* TDMA off + beacon protect + Ext_control */ + btc->bt_req_en = true; + _write_scbd(rtwdev, BTC_WSCB_TDMA, true); + *t = t_def[CXTD_OFF_EXT]; + + /* To avoid wl-s0 tx break by hid/hfp tx */ + if (hid->exist || hfp->exist) + tbl_w1 = cxtbl[16]; + + switch (policy_type) { + case BTC_CXP_OFFE_DEF: + s[CXST_E2G] = s_def[CXST_E2G]; + s[CXST_E5G] = s_def[CXST_E5G]; + s[CXST_EBT] = s_def[CXST_EBT]; + s[CXST_ENULL] = s_def[CXST_ENULL]; + break; + case BTC_CXP_OFFE_DEF2: + _slot_set(btc, CXST_E2G, 20, cxtbl[1], SLOT_ISO); + s[CXST_E5G] = s_def[CXST_E5G]; + s[CXST_EBT] = s_def[CXST_EBT]; + s[CXST_ENULL] = s_def[CXST_ENULL]; + break; + default: + break; + } + break; + case BTC_CXP_FIX: /* TDMA Fix-Slot */ + _write_scbd(rtwdev, BTC_WSCB_TDMA, true); + *t = t_def[CXTD_FIX]; + + switch (policy_type) { + case BTC_CXP_FIX_TD3030: + _slot_set(btc, CXST_W1, 30, tbl_w1, SLOT_ISO); + _slot_set(btc, CXST_B1, 30, tbl_b1, SLOT_MIX); + break; + case BTC_CXP_FIX_TD5050: + _slot_set(btc, CXST_W1, 50, tbl_w1, SLOT_ISO); + _slot_set(btc, CXST_B1, 50, tbl_b1, SLOT_MIX); + break; + case BTC_CXP_FIX_TD2030: + _slot_set(btc, CXST_W1, 20, tbl_w1, SLOT_ISO); + _slot_set(btc, CXST_B1, 30, tbl_b1, SLOT_MIX); + break; + case BTC_CXP_FIX_TD4010: + _slot_set(btc, CXST_W1, 40, tbl_w1, SLOT_ISO); + _slot_set(btc, CXST_B1, 10, tbl_b1, SLOT_MIX); + break; + case BTC_CXP_FIX_TD4010ISO: + _slot_set(btc, CXST_W1, 40, cxtbl[1], SLOT_ISO); + _slot_set(btc, CXST_B1, 10, tbl_b1, SLOT_MIX); + break; + case BTC_CXP_FIX_TD7010: + _slot_set(btc, CXST_W1, 70, tbl_w1, SLOT_ISO); + _slot_set(btc, CXST_B1, 10, tbl_b1, SLOT_MIX); + break; + case BTC_CXP_FIX_TD2060: + _slot_set(btc, CXST_W1, 20, tbl_w1, SLOT_ISO); + _slot_set(btc, CXST_B1, 60, tbl_b1, SLOT_MIX); + break; + case BTC_CXP_FIX_TD3060: + _slot_set(btc, CXST_W1, 30, tbl_w1, SLOT_ISO); + _slot_set(btc, CXST_B1, 60, tbl_b1, SLOT_MIX); + break; + case BTC_CXP_FIX_TD2080: + _slot_set(btc, CXST_W1, 20, tbl_w1, SLOT_ISO); + _slot_set(btc, CXST_B1, 80, tbl_b1, SLOT_MIX); + break; + case BTC_CXP_FIX_TDW1B1: /* W1:B1 = user-define */ + _slot_set(btc, CXST_W1, dm->slot_dur[CXST_W1], + tbl_w1, SLOT_ISO); + _slot_set(btc, CXST_B1, dm->slot_dur[CXST_B1], + tbl_b1, SLOT_MIX); + break; + default: + break; + } + break; + case BTC_CXP_PFIX: /* PS-TDMA Fix-Slot */ + _write_scbd(rtwdev, BTC_WSCB_TDMA, true); + *t = t_def[CXTD_PFIX]; + + switch (policy_type) { + case BTC_CXP_PFIX_TD3030: + _slot_set(btc, CXST_W1, 30, tbl_w1, SLOT_ISO); + _slot_set(btc, CXST_B1, 30, tbl_b1, SLOT_MIX); + break; + case BTC_CXP_PFIX_TD5050: + _slot_set(btc, CXST_W1, 50, tbl_w1, SLOT_ISO); + _slot_set(btc, CXST_B1, 50, tbl_b1, SLOT_MIX); + break; + case BTC_CXP_PFIX_TD2030: + _slot_set(btc, CXST_W1, 20, tbl_w1, SLOT_ISO); + _slot_set(btc, CXST_B1, 30, tbl_b1, SLOT_MIX); + break; + case BTC_CXP_PFIX_TD2060: + _slot_set(btc, CXST_W1, 20, tbl_w1, SLOT_ISO); + _slot_set(btc, CXST_B1, 60, tbl_b1, SLOT_MIX); + break; + case BTC_CXP_PFIX_TD3070: + _slot_set(btc, CXST_W1, 30, tbl_w1, SLOT_ISO); + _slot_set(btc, CXST_B1, 60, tbl_b1, SLOT_MIX); + break; + case BTC_CXP_PFIX_TD2080: + _slot_set(btc, CXST_W1, 20, tbl_w1, SLOT_ISO); + _slot_set(btc, CXST_B1, 80, tbl_b1, SLOT_MIX); + break; + case BTC_CXP_PFIX_TDW1B1: /* W1:B1 = user-define */ + _slot_set(btc, CXST_W1, dm->slot_dur[CXST_W1], + tbl_w1, SLOT_ISO); + _slot_set(btc, CXST_B1, dm->slot_dur[CXST_B1], + tbl_b1, SLOT_MIX); + break; + default: + break; + } + break; + case BTC_CXP_AUTO: /* TDMA Auto-Slot */ + _write_scbd(rtwdev, BTC_WSCB_TDMA, true); + *t = t_def[CXTD_AUTO]; + + switch (policy_type) { + case BTC_CXP_AUTO_TD50B1: _slot_set(btc, CXST_W1, 50, tbl_w1, SLOT_ISO); - _slot_set(btc, CXST_B1, 200, tbl_b1, SLOT_MIX); + _slot_set(btc, CXST_B1, BTC_B1_MAX, tbl_b1, SLOT_MIX); break; - case BTC_CXP_AUTO_TD60200: + case BTC_CXP_AUTO_TD60B1: _slot_set(btc, CXST_W1, 60, tbl_w1, SLOT_ISO); - _slot_set(btc, CXST_B1, 200, tbl_b1, SLOT_MIX); + _slot_set(btc, CXST_B1, BTC_B1_MAX, tbl_b1, SLOT_MIX); break; - case BTC_CXP_AUTO_TD20200: + case BTC_CXP_AUTO_TD20B1: _slot_set(btc, CXST_W1, 20, tbl_w1, SLOT_ISO); - _slot_set(btc, CXST_B1, 200, tbl_b1, SLOT_MIX); + _slot_set(btc, CXST_B1, BTC_B1_MAX, tbl_b1, SLOT_MIX); break; case BTC_CXP_AUTO_TDW1B1: /* W1:B1 = user-define */ _slot_set(btc, CXST_W1, dm->slot_dur[CXST_W1], @@ -2098,23 +2690,26 @@ static void _set_policy(struct rtw89_dev *rtwdev, u16 policy_type, _slot_set(btc, CXST_B1, dm->slot_dur[CXST_B1], tbl_b1, SLOT_MIX); break; + default: + break; } break; case BTC_CXP_PAUTO: /* PS-TDMA Auto-Slot */ _write_scbd(rtwdev, BTC_WSCB_TDMA, true); *t = t_def[CXTD_PAUTO]; + switch (policy_type) { - case BTC_CXP_PAUTO_TD50200: + case BTC_CXP_PAUTO_TD50B1: _slot_set(btc, CXST_W1, 50, tbl_w1, SLOT_ISO); - _slot_set(btc, CXST_B1, 200, tbl_b1, SLOT_MIX); + _slot_set(btc, CXST_B1, BTC_B1_MAX, tbl_b1, SLOT_MIX); break; - case BTC_CXP_PAUTO_TD60200: + case BTC_CXP_PAUTO_TD60B1: _slot_set(btc, CXST_W1, 60, tbl_w1, SLOT_ISO); - _slot_set(btc, CXST_B1, 200, tbl_b1, SLOT_MIX); + _slot_set(btc, CXST_B1, BTC_B1_MAX, tbl_b1, SLOT_MIX); break; - case BTC_CXP_PAUTO_TD20200: + case BTC_CXP_PAUTO_TD20B1: _slot_set(btc, CXST_W1, 20, tbl_w1, SLOT_ISO); - _slot_set(btc, CXST_B1, 200, tbl_b1, SLOT_MIX); + _slot_set(btc, CXST_B1, BTC_B1_MAX, tbl_b1, SLOT_MIX); break; case BTC_CXP_PAUTO_TDW1B1: _slot_set(btc, CXST_W1, dm->slot_dur[CXST_W1], @@ -2122,119 +2717,112 @@ static void _set_policy(struct rtw89_dev *rtwdev, u16 policy_type, _slot_set(btc, CXST_B1, dm->slot_dur[CXST_B1], tbl_b1, SLOT_MIX); break; + default: + break; } break; case BTC_CXP_AUTO2: /* TDMA Auto-Slot2 */ _write_scbd(rtwdev, BTC_WSCB_TDMA, true); *t = t_def[CXTD_AUTO2]; + switch (policy_type) { case BTC_CXP_AUTO2_TD3050: _slot_set(btc, CXST_W1, 30, tbl_w1, SLOT_ISO); - _slot_set(btc, CXST_B1, 200, tbl_b1, SLOT_MIX); + _slot_set(btc, CXST_B1, BTC_B1_MAX, tbl_b1, SLOT_MIX); _slot_set(btc, CXST_B4, 50, tbl_b4, SLOT_MIX); break; case BTC_CXP_AUTO2_TD3070: _slot_set(btc, CXST_W1, 30, tbl_w1, SLOT_ISO); - _slot_set(btc, CXST_B1, 200, tbl_b1, SLOT_MIX); + _slot_set(btc, CXST_B1, BTC_B1_MAX, tbl_b1, SLOT_MIX); _slot_set(btc, CXST_B4, 70, tbl_b4, SLOT_MIX); break; case BTC_CXP_AUTO2_TD5050: _slot_set(btc, CXST_W1, 50, tbl_w1, SLOT_ISO); - _slot_set(btc, CXST_B1, 200, tbl_b1, SLOT_MIX); + _slot_set(btc, CXST_B1, BTC_B1_MAX, tbl_b1, SLOT_MIX); _slot_set(btc, CXST_B4, 50, tbl_b4, SLOT_MIX); break; case BTC_CXP_AUTO2_TD6060: _slot_set(btc, CXST_W1, 60, tbl_w1, SLOT_ISO); - _slot_set(btc, CXST_B1, 200, tbl_b1, SLOT_MIX); + _slot_set(btc, CXST_B1, BTC_B1_MAX, tbl_b1, SLOT_MIX); _slot_set(btc, CXST_B4, 60, tbl_b4, SLOT_MIX); break; case BTC_CXP_AUTO2_TD2080: _slot_set(btc, CXST_W1, 20, tbl_w1, SLOT_ISO); - _slot_set(btc, CXST_B1, 200, tbl_b1, SLOT_MIX); + _slot_set(btc, CXST_B1, BTC_B1_MAX, tbl_b1, SLOT_MIX); _slot_set(btc, CXST_B4, 80, tbl_b4, SLOT_MIX); break; case BTC_CXP_AUTO2_TDW1B4: /* W1:B1 = user-define */ _slot_set(btc, CXST_W1, dm->slot_dur[CXST_W1], tbl_w1, SLOT_ISO); + _slot_set(btc, CXST_B1, dm->slot_dur[CXST_B1], + tbl_b1, SLOT_MIX); _slot_set(btc, CXST_B4, dm->slot_dur[CXST_B4], tbl_b4, SLOT_MIX); break; + default: + break; } break; case BTC_CXP_PAUTO2: /* PS-TDMA Auto-Slot2 */ _write_scbd(rtwdev, BTC_WSCB_TDMA, true); *t = t_def[CXTD_PAUTO2]; + switch (policy_type) { case BTC_CXP_PAUTO2_TD3050: _slot_set(btc, CXST_W1, 30, tbl_w1, SLOT_ISO); - _slot_set(btc, CXST_B1, 200, tbl_b1, SLOT_MIX); + _slot_set(btc, CXST_B1, BTC_B1_MAX, tbl_b1, SLOT_MIX); _slot_set(btc, CXST_B4, 50, tbl_b4, SLOT_MIX); break; case BTC_CXP_PAUTO2_TD3070: _slot_set(btc, CXST_W1, 30, tbl_w1, SLOT_ISO); - _slot_set(btc, CXST_B1, 200, tbl_b1, SLOT_MIX); + _slot_set(btc, CXST_B1, BTC_B1_MAX, tbl_b1, SLOT_MIX); _slot_set(btc, CXST_B4, 70, tbl_b4, SLOT_MIX); break; case BTC_CXP_PAUTO2_TD5050: _slot_set(btc, CXST_W1, 50, tbl_w1, SLOT_ISO); - _slot_set(btc, CXST_B1, 200, tbl_b1, SLOT_MIX); + _slot_set(btc, CXST_B1, BTC_B1_MAX, tbl_b1, SLOT_MIX); _slot_set(btc, CXST_B4, 50, tbl_b4, SLOT_MIX); break; case BTC_CXP_PAUTO2_TD6060: _slot_set(btc, CXST_W1, 60, tbl_w1, SLOT_ISO); - _slot_set(btc, CXST_B1, 200, tbl_b1, SLOT_MIX); + _slot_set(btc, CXST_B1, BTC_B1_MAX, tbl_b1, SLOT_MIX); _slot_set(btc, CXST_B4, 60, tbl_b4, SLOT_MIX); break; case BTC_CXP_PAUTO2_TD2080: _slot_set(btc, CXST_W1, 20, tbl_w1, SLOT_ISO); - _slot_set(btc, CXST_B1, 200, tbl_b1, SLOT_MIX); + _slot_set(btc, CXST_B1, BTC_B1_MAX, tbl_b1, SLOT_MIX); _slot_set(btc, CXST_B4, 80, tbl_b4, SLOT_MIX); break; case BTC_CXP_PAUTO2_TDW1B4: /* W1:B1 = user-define */ _slot_set(btc, CXST_W1, dm->slot_dur[CXST_W1], tbl_w1, SLOT_ISO); + _slot_set(btc, CXST_B1, dm->slot_dur[CXST_B1], + tbl_b1, SLOT_MIX); _slot_set(btc, CXST_B4, dm->slot_dur[CXST_B4], tbl_b4, SLOT_MIX); break; + default: + break; } break; } - _fw_set_policy(rtwdev, policy_type, action); -} - -static void _set_gnt_bt(struct rtw89_dev *rtwdev, u8 phy_map, u8 state) -{ - struct rtw89_btc *btc = &rtwdev->btc; - struct rtw89_btc_dm *dm = &btc->dm; - struct rtw89_mac_ax_gnt *g = dm->gnt.band; - u8 i; - - if (phy_map > BTC_PHY_ALL) - return; + if (wl_rinfo->link_mode == BTC_WLINK_2G_SCC && dm->tdma.rxflctrl) { + null_role = FIELD_PREP(0x0f, dm->wl_scc.null_role1) | + FIELD_PREP(0xf0, dm->wl_scc.null_role2); + _tdma_set_flctrl_role(btc, null_role); + } - for (i = 0; i < RTW89_PHY_MAX; i++) { - if (!(phy_map & BIT(i))) - continue; + /* enter leak_slot after each null-1 */ + if (dm->leak_ap && dm->tdma.leak_n > 1) + _tdma_set_lek(btc, 1); - switch (state) { - case BTC_GNT_HW: - g[i].gnt_bt_sw_en = 0; - g[i].gnt_bt = 0; - break; - case BTC_GNT_SW_LO: - g[i].gnt_bt_sw_en = 1; - g[i].gnt_bt = 0; - break; - case BTC_GNT_SW_HI: - g[i].gnt_bt_sw_en = 1; - g[i].gnt_bt = 1; - break; - } + if (dm->tdma_instant_excute) { + btc->dm.tdma.option_ctrl |= BIT(0); + btc->update_policy_force = true; } - - rtw89_chip_mac_cfg_gnt(rtwdev, &dm->gnt); } +EXPORT_SYMBOL(rtw89_btc_set_policy_v1); static void _set_bt_plut(struct rtw89_dev *rtwdev, u8 phy_map, u8 tx_val, u8 rx_val) @@ -2300,86 +2888,74 @@ static void _set_ant(struct rtw89_dev *rtwdev, bool force_exec, switch (type) { case BTC_ANT_WPOWERON: - rtw89_chip_cfg_ctrl_path(rtwdev, false); + rtw89_chip_cfg_ctrl_path(rtwdev, BTC_CTRL_BY_BT); break; case BTC_ANT_WINIT: - if (bt->enable.now) { - _set_gnt_wl(rtwdev, phy_map, BTC_GNT_SW_LO); - _set_gnt_bt(rtwdev, phy_map, BTC_GNT_SW_HI); - } else { - _set_gnt_wl(rtwdev, phy_map, BTC_GNT_SW_HI); - _set_gnt_bt(rtwdev, phy_map, BTC_GNT_SW_LO); - } - rtw89_chip_cfg_ctrl_path(rtwdev, true); + if (bt->enable.now) + _set_gnt(rtwdev, phy_map, BTC_GNT_SW_LO, BTC_GNT_SW_HI); + else + _set_gnt(rtwdev, phy_map, BTC_GNT_SW_HI, BTC_GNT_SW_LO); + + rtw89_chip_cfg_ctrl_path(rtwdev, BTC_CTRL_BY_WL); _set_bt_plut(rtwdev, BTC_PHY_ALL, BTC_PLT_BT, BTC_PLT_BT); break; case BTC_ANT_WONLY: - _set_gnt_wl(rtwdev, phy_map, BTC_GNT_SW_HI); - _set_gnt_bt(rtwdev, phy_map, BTC_GNT_SW_LO); - rtw89_chip_cfg_ctrl_path(rtwdev, true); + _set_gnt(rtwdev, phy_map, BTC_GNT_SW_HI, BTC_GNT_SW_LO); + rtw89_chip_cfg_ctrl_path(rtwdev, BTC_CTRL_BY_WL); _set_bt_plut(rtwdev, BTC_PHY_ALL, BTC_PLT_NONE, BTC_PLT_NONE); break; case BTC_ANT_WOFF: - rtw89_chip_cfg_ctrl_path(rtwdev, false); + rtw89_chip_cfg_ctrl_path(rtwdev, BTC_CTRL_BY_BT); _set_bt_plut(rtwdev, BTC_PHY_ALL, BTC_PLT_NONE, BTC_PLT_NONE); break; case BTC_ANT_W2G: - rtw89_chip_cfg_ctrl_path(rtwdev, true); + rtw89_chip_cfg_ctrl_path(rtwdev, BTC_CTRL_BY_WL); if (rtwdev->dbcc_en) { for (i = 0; i < RTW89_PHY_MAX; i++) { b2g = (wl_dinfo->real_band[i] == RTW89_BAND_2G); gnt_wl_ctrl = b2g ? BTC_GNT_HW : BTC_GNT_SW_HI; - _set_gnt_wl(rtwdev, BIT(i), gnt_wl_ctrl); - gnt_bt_ctrl = b2g ? BTC_GNT_HW : BTC_GNT_SW_HI; /* BT should control by GNT_BT if WL_2G at S0 */ if (i == 1 && wl_dinfo->real_band[0] == RTW89_BAND_2G && wl_dinfo->real_band[1] == RTW89_BAND_5G) gnt_bt_ctrl = BTC_GNT_HW; - _set_gnt_bt(rtwdev, BIT(i), gnt_bt_ctrl); - + _set_gnt(rtwdev, BIT(i), gnt_wl_ctrl, gnt_bt_ctrl); plt_ctrl = b2g ? BTC_PLT_BT : BTC_PLT_NONE; _set_bt_plut(rtwdev, BIT(i), plt_ctrl, plt_ctrl); } } else { - _set_gnt_wl(rtwdev, phy_map, BTC_GNT_HW); - _set_gnt_bt(rtwdev, phy_map, BTC_GNT_HW); + _set_gnt(rtwdev, phy_map, BTC_GNT_HW, BTC_GNT_HW); _set_bt_plut(rtwdev, BTC_PHY_ALL, BTC_PLT_BT, BTC_PLT_BT); } break; case BTC_ANT_W5G: - rtw89_chip_cfg_ctrl_path(rtwdev, true); - _set_gnt_wl(rtwdev, phy_map, BTC_GNT_SW_HI); - _set_gnt_bt(rtwdev, phy_map, BTC_GNT_HW); + rtw89_chip_cfg_ctrl_path(rtwdev, BTC_CTRL_BY_WL); + _set_gnt(rtwdev, phy_map, BTC_GNT_SW_HI, BTC_GNT_HW); _set_bt_plut(rtwdev, BTC_PHY_ALL, BTC_PLT_NONE, BTC_PLT_NONE); break; case BTC_ANT_W25G: - rtw89_chip_cfg_ctrl_path(rtwdev, true); - _set_gnt_wl(rtwdev, phy_map, BTC_GNT_HW); - _set_gnt_bt(rtwdev, phy_map, BTC_GNT_HW); + rtw89_chip_cfg_ctrl_path(rtwdev, BTC_CTRL_BY_WL); + _set_gnt(rtwdev, phy_map, BTC_GNT_HW, BTC_GNT_HW); _set_bt_plut(rtwdev, BTC_PHY_ALL, BTC_PLT_GNT_WL, BTC_PLT_GNT_WL); break; case BTC_ANT_FREERUN: - rtw89_chip_cfg_ctrl_path(rtwdev, true); - _set_gnt_wl(rtwdev, phy_map, BTC_GNT_SW_HI); - _set_gnt_bt(rtwdev, phy_map, BTC_GNT_SW_HI); + rtw89_chip_cfg_ctrl_path(rtwdev, BTC_CTRL_BY_WL); + _set_gnt(rtwdev, phy_map, BTC_GNT_SW_HI, BTC_GNT_SW_HI); _set_bt_plut(rtwdev, BTC_PHY_ALL, BTC_PLT_NONE, BTC_PLT_NONE); break; case BTC_ANT_WRFK: - rtw89_chip_cfg_ctrl_path(rtwdev, true); - _set_gnt_wl(rtwdev, phy_map, BTC_GNT_SW_HI); - _set_gnt_bt(rtwdev, phy_map, BTC_GNT_SW_LO); + rtw89_chip_cfg_ctrl_path(rtwdev, BTC_CTRL_BY_WL); + _set_gnt(rtwdev, phy_map, BTC_GNT_SW_HI, BTC_GNT_SW_LO); _set_bt_plut(rtwdev, phy_map, BTC_PLT_NONE, BTC_PLT_NONE); break; case BTC_ANT_BRFK: - rtw89_chip_cfg_ctrl_path(rtwdev, false); - _set_gnt_wl(rtwdev, phy_map, BTC_GNT_SW_LO); - _set_gnt_bt(rtwdev, phy_map, BTC_GNT_SW_HI); + rtw89_chip_cfg_ctrl_path(rtwdev, BTC_CTRL_BY_BT); + _set_gnt(rtwdev, phy_map, BTC_GNT_SW_LO, BTC_GNT_SW_HI); _set_bt_plut(rtwdev, phy_map, BTC_PLT_NONE, BTC_PLT_NONE); break; default: @@ -2491,14 +3067,19 @@ static void _action_bt_idle(struct rtw89_dev *rtwdev) static void _action_bt_hfp(struct rtw89_dev *rtwdev) { struct rtw89_btc *btc = &rtwdev->btc; + struct rtw89_btc_wl_info *wl = &btc->cx.wl; _set_ant(rtwdev, NM_EXEC, BTC_PHY_ALL, BTC_ANT_W2G); if (btc->mdinfo.ant.type == BTC_ANT_SHARED) { - if (btc->cx.wl.status.map._4way) + if (btc->cx.wl.status.map._4way) { _set_policy(rtwdev, BTC_CXP_OFF_WL, BTC_ACT_BT_HFP); - else - _set_policy(rtwdev, BTC_CXP_OFF_BWB0, BTC_ACT_BT_HFP); + } else if (wl->status.map.traffic_dir & BIT(RTW89_TFC_UL)) { + btc->cx.bt.scan_rx_low_pri = true; + _set_policy(rtwdev, BTC_CXP_OFF_BWB2, BTC_ACT_BT_HFP); + } else { + _set_policy(rtwdev, BTC_CXP_OFF_BWB1, BTC_ACT_BT_HFP); + } } else { _set_policy(rtwdev, BTC_CXP_OFF_EQ2, BTC_ACT_BT_HFP); } @@ -2506,17 +3087,37 @@ static void _action_bt_hfp(struct rtw89_dev *rtwdev) static void _action_bt_hid(struct rtw89_dev *rtwdev) { + const struct rtw89_chip_info *chip = rtwdev->chip; struct rtw89_btc *btc = &rtwdev->btc; + struct rtw89_btc_wl_info *wl = &btc->cx.wl; + struct rtw89_btc_bt_info *bt = &btc->cx.bt; + struct rtw89_btc_bt_hid_desc *hid = &bt->link_info.hid_desc; + u16 policy_type = BTC_CXP_OFF_BT; _set_ant(rtwdev, NM_EXEC, BTC_PHY_ALL, BTC_ANT_W2G); - if (btc->mdinfo.ant.type == BTC_ANT_SHARED) /* shared-antenna */ - if (btc->cx.wl.status.map._4way) - _set_policy(rtwdev, BTC_CXP_OFF_WL, BTC_ACT_BT_HID); - else - _set_policy(rtwdev, BTC_CXP_OFF_BWB0, BTC_ACT_BT_HID); - else /* dedicated-antenna */ - _set_policy(rtwdev, BTC_CXP_OFF_EQ3, BTC_ACT_BT_HID); + if (btc->mdinfo.ant.type == BTC_ANT_SHARED) { /* shared-antenna */ + if (wl->status.map._4way) { + policy_type = BTC_CXP_OFF_WL; + } else if (wl->status.map.traffic_dir & BIT(RTW89_TFC_UL)) { + btc->cx.bt.scan_rx_low_pri = true; + if (hid->type & BTC_HID_BLE) + policy_type = BTC_CXP_OFF_BWB0; + else + policy_type = BTC_CXP_OFF_BWB2; + } else if (hid->type == BTC_HID_218) { + bt->scan_rx_low_pri = true; + policy_type = BTC_CXP_OFF_BWB2; + } else if (chip->para_ver == 0x1) { + policy_type = BTC_CXP_OFF_BWB3; + } else { + policy_type = BTC_CXP_OFF_BWB1; + } + } else { /* dedicated-antenna */ + policy_type = BTC_CXP_OFF_EQ3; + } + + _set_policy(rtwdev, policy_type, BTC_ACT_BT_HID); } static void _action_bt_a2dp(struct rtw89_dev *rtwdev) @@ -2537,7 +3138,7 @@ static void _action_bt_a2dp(struct rtw89_dev *rtwdev) BTC_CXP_PAUTO_TDW1B1, BTC_ACT_BT_A2DP); } else { _set_policy(rtwdev, - BTC_CXP_PAUTO_TD50200, BTC_ACT_BT_A2DP); + BTC_CXP_PAUTO_TD50B1, BTC_ACT_BT_A2DP); } break; case BTC_WBUSY_BSCAN: /* wl-busy + bt-inq + bt-A2DP */ @@ -2554,12 +3155,12 @@ static void _action_bt_a2dp(struct rtw89_dev *rtwdev) _set_policy(rtwdev, BTC_CXP_AUTO_TDW1B1, BTC_ACT_BT_A2DP); } else { - _set_policy(rtwdev, BTC_CXP_AUTO_TD50200, + _set_policy(rtwdev, BTC_CXP_AUTO_TD50B1, BTC_ACT_BT_A2DP); } break; case BTC_WIDLE: /* wl-idle + bt-A2DP */ - _set_policy(rtwdev, BTC_CXP_AUTO_TD20200, BTC_ACT_BT_A2DP); + _set_policy(rtwdev, BTC_CXP_AUTO_TD20B1, BTC_ACT_BT_A2DP); break; } } @@ -2639,7 +3240,7 @@ static void _action_bt_a2dp_hid(struct rtw89_dev *rtwdev) BTC_CXP_PAUTO_TDW1B1, BTC_ACT_BT_A2DP_HID); } else { _set_policy(rtwdev, - BTC_CXP_PAUTO_TD50200, BTC_ACT_BT_A2DP_HID); + BTC_CXP_PAUTO_TD50B1, BTC_ACT_BT_A2DP_HID); } break; case BTC_WBUSY_BSCAN: /* wl-busy + bt-inq + bt-A2DP+HID */ @@ -2657,7 +3258,7 @@ static void _action_bt_a2dp_hid(struct rtw89_dev *rtwdev) _set_policy(rtwdev, BTC_CXP_AUTO_TDW1B1, BTC_ACT_BT_A2DP_HID); } else { - _set_policy(rtwdev, BTC_CXP_AUTO_TD50200, + _set_policy(rtwdev, BTC_CXP_AUTO_TD50B1, BTC_ACT_BT_A2DP_HID); } break; @@ -2792,19 +3393,27 @@ static void _action_wl_rfk(struct rtw89_dev *rtwdev) static void _set_btg_ctrl(struct rtw89_dev *rtwdev) { + const struct rtw89_chip_info *chip = rtwdev->chip; struct rtw89_btc *btc = &rtwdev->btc; struct rtw89_btc_wl_info *wl = &btc->cx.wl; struct rtw89_btc_wl_role_info *wl_rinfo = &wl->role_info; + struct rtw89_btc_wl_role_info_v1 *wl_rinfo_v1 = &wl->role_info_v1; struct rtw89_btc_wl_dbcc_info *wl_dinfo = &wl->dbcc_info; - bool is_btg = false; + bool is_btg; + u8 mode; if (btc->ctrl.manual) return; + if (chip->chip_id == RTL8852A) + mode = wl_rinfo->link_mode; + else + mode = wl_rinfo_v1->link_mode; + /* notify halbb ignore GNT_BT or not for WL BB Rx-AGC control */ - if (wl_rinfo->link_mode == BTC_WLINK_5G) /* always 0 if 5G */ + if (mode == BTC_WLINK_5G) /* always 0 if 5G */ is_btg = false; - else if (wl_rinfo->link_mode == BTC_WLINK_25G_DBCC && + else if (mode == BTC_WLINK_25G_DBCC && wl_dinfo->real_band[RTW89_PHY_1] != RTW89_BAND_2G) is_btg = false; else @@ -2816,7 +3425,7 @@ static void _set_btg_ctrl(struct rtw89_dev *rtwdev) btc->dm.wl_btg_rx = is_btg; - if (wl_rinfo->link_mode == BTC_WLINK_25G_MCC) + if (mode == BTC_WLINK_25G_MCC) return; rtw89_ctrl_btg(rtwdev, is_btg); @@ -2889,6 +3498,7 @@ static void rtw89_tx_time_iter(void *data, struct ieee80211_sta *sta) static void _set_wl_tx_limit(struct rtw89_dev *rtwdev) { + const struct rtw89_chip_info *chip = rtwdev->chip; struct rtw89_btc *btc = &rtwdev->btc; struct rtw89_btc_cx *cx = &btc->cx; struct rtw89_btc_dm *dm = &btc->dm; @@ -2898,16 +3508,22 @@ static void _set_wl_tx_limit(struct rtw89_dev *rtwdev) struct rtw89_btc_bt_hfp_desc *hfp = &b->hfp_desc; struct rtw89_btc_bt_hid_desc *hid = &b->hid_desc; struct rtw89_btc_wl_role_info *wl_rinfo = &wl->role_info; + struct rtw89_btc_wl_role_info_v1 *wl_rinfo_v1 = &wl->role_info_v1; struct rtw89_txtime_data data = {.rtwdev = rtwdev}; - u8 mode = wl_rinfo->link_mode; - u8 tx_retry = 0; - u32 tx_time = 0; - u16 enable = 0; + u8 mode; + u8 tx_retry; + u32 tx_time; + u16 enable; bool reenable = false; if (btc->ctrl.manual) return; + if (chip->chip_id == RTL8852A) + mode = wl_rinfo->link_mode; + else + mode = wl_rinfo_v1->link_mode; + if (btc->dm.freerun || btc->ctrl.igno_bt || b->profile_cnt.now == 0 || mode == BTC_WLINK_5G || mode == BTC_WLINK_NOLINK) { enable = 0; @@ -2951,13 +3567,21 @@ static void _set_wl_tx_limit(struct rtw89_dev *rtwdev) static void _set_bt_rx_agc(struct rtw89_dev *rtwdev) { + const struct rtw89_chip_info *chip = rtwdev->chip; struct rtw89_btc *btc = &rtwdev->btc; struct rtw89_btc_wl_info *wl = &btc->cx.wl; struct rtw89_btc_wl_role_info *wl_rinfo = &wl->role_info; + struct rtw89_btc_wl_role_info_v1 *wl_rinfo_v1 = &wl->role_info_v1; struct rtw89_btc_bt_info *bt = &btc->cx.bt; bool bt_hi_lna_rx = false; + u8 mode; + + if (chip->chip_id == RTL8852A) + mode = wl_rinfo->link_mode; + else + mode = wl_rinfo_v1->link_mode; - if (wl_rinfo->link_mode != BTC_WLINK_NOLINK && btc->dm.wl_btg_rx) + if (mode != BTC_WLINK_NOLINK && btc->dm.wl_btg_rx) bt_hi_lna_rx = true; if (bt_hi_lna_rx == bt->hi_lna_rx) @@ -2966,14 +3590,34 @@ static void _set_bt_rx_agc(struct rtw89_dev *rtwdev) _write_scbd(rtwdev, BTC_WSCB_BT_HILNA, bt_hi_lna_rx); } +static void _set_bt_rx_scan_pri(struct rtw89_dev *rtwdev) +{ + struct rtw89_btc *btc = &rtwdev->btc; + struct rtw89_btc_bt_info *bt = &btc->cx.bt; + + _write_scbd(rtwdev, BTC_WSCB_RXSCAN_PRI, (bool)(!!bt->scan_rx_low_pri)); +} + /* TODO add these functions */ static void _action_common(struct rtw89_dev *rtwdev) { + struct rtw89_btc *btc = &rtwdev->btc; + struct rtw89_btc_wl_info *wl = &btc->cx.wl; + _set_btg_ctrl(rtwdev); _set_wl_tx_limit(rtwdev); _set_bt_afh_info(rtwdev); _set_bt_rx_agc(rtwdev); _set_rf_trx_para(rtwdev); + _set_bt_rx_scan_pri(rtwdev); + + if (wl->scbd_change) { + rtw89_mac_cfg_sb(rtwdev, wl->scbd); + rtw89_debug(rtwdev, RTW89_DBG_BTC, "[BTC], write scbd: 0x%08x\n", + wl->scbd); + wl->scbd_change = false; + btc->cx.cnt_wl[BTC_WCNT_SCBDUPDATE]++; + } } static void _action_by_bt(struct rtw89_dev *rtwdev) @@ -3145,6 +3789,68 @@ static void _action_wl_2g_scc(struct rtw89_dev *rtwdev) } } +static void _action_wl_2g_scc_v1(struct rtw89_dev *rtwdev) +{ + struct rtw89_btc *btc = &rtwdev->btc; + struct rtw89_btc_wl_info *wl = &btc->cx.wl; + struct rtw89_btc_bt_info *bt = &btc->cx.bt; + struct rtw89_btc_dm *dm = &btc->dm; + struct rtw89_btc_wl_role_info_v1 *wl_rinfo = &wl->role_info_v1; + u16 policy_type = BTC_CXP_OFF_BT; + u32 dur; + + if (btc->mdinfo.ant.type == BTC_ANT_DEDICATED) { + policy_type = BTC_CXP_OFF_EQ0; + } else { + /* shared-antenna */ + switch (wl_rinfo->mrole_type) { + case BTC_WLMROLE_STA_GC: + dm->wl_scc.null_role1 = RTW89_WIFI_ROLE_STATION; + dm->wl_scc.null_role2 = RTW89_WIFI_ROLE_P2P_CLIENT; + dm->wl_scc.ebt_null = 0; /* no ext-slot-control */ + _action_by_bt(rtwdev); + return; + case BTC_WLMROLE_STA_STA: + dm->wl_scc.null_role1 = RTW89_WIFI_ROLE_STATION; + dm->wl_scc.null_role2 = RTW89_WIFI_ROLE_STATION; + dm->wl_scc.ebt_null = 0; /* no ext-slot-control */ + _action_by_bt(rtwdev); + return; + case BTC_WLMROLE_STA_GC_NOA: + case BTC_WLMROLE_STA_GO: + case BTC_WLMROLE_STA_GO_NOA: + dm->wl_scc.null_role1 = RTW89_WIFI_ROLE_STATION; + dm->wl_scc.null_role2 = RTW89_WIFI_ROLE_NONE; + dur = wl_rinfo->mrole_noa_duration; + + if (wl->status.map._4way) { + dm->wl_scc.ebt_null = 0; + policy_type = BTC_CXP_OFFE_WL; + } else if (bt->link_info.status.map.connect == 0) { + dm->wl_scc.ebt_null = 0; + policy_type = BTC_CXP_OFFE_2GISOB; + } else if (bt->link_info.a2dp_desc.exist && + dur < btc->bt_req_len) { + dm->wl_scc.ebt_null = 1; /* tx null at EBT */ + policy_type = BTC_CXP_OFFE_2GBWMIXB2; + } else if (bt->link_info.a2dp_desc.exist || + bt->link_info.pan_desc.exist) { + dm->wl_scc.ebt_null = 1; /* tx null at EBT */ + policy_type = BTC_CXP_OFFE_2GBWISOB; + } else { + dm->wl_scc.ebt_null = 0; + policy_type = BTC_CXP_OFFE_2GBWISOB; + } + break; + default: + break; + } + } + + _set_ant(rtwdev, NM_EXEC, BTC_PHY_ALL, BTC_ANT_W2G); + _set_policy(rtwdev, policy_type, BTC_ACT_WL_2G_SCC); +} + static void _action_wl_2g_ap(struct rtw89_dev *rtwdev) { struct rtw89_btc *btc = &rtwdev->btc; @@ -3234,20 +3940,20 @@ static void _write_scbd(struct rtw89_dev *rtwdev, u32 val, bool state) struct rtw89_btc *btc = &rtwdev->btc; struct rtw89_btc_wl_info *wl = &btc->cx.wl; u32 scbd_val = 0; + u8 force_exec = false; if (!chip->scbd) return; scbd_val = state ? wl->scbd | val : wl->scbd & ~val; - if (scbd_val == wl->scbd) - return; - rtw89_mac_cfg_sb(rtwdev, scbd_val); - rtw89_debug(rtwdev, RTW89_DBG_BTC, "[BTC], write scbd: 0x%08x\n", - scbd_val); - wl->scbd = scbd_val; + if (val & BTC_WSCB_ACTIVE || val & BTC_WSCB_ON) + force_exec = true; - btc->cx.cnt_wl[BTC_WCNT_SCBDUPDATE]++; + if (scbd_val != wl->scbd || force_exec) { + wl->scbd = scbd_val; + wl->scbd_change = true; + } } static u8 @@ -3428,8 +4134,158 @@ static void _update_wl_info(struct rtw89_dev *rtwdev) } rtw89_debug(rtwdev, RTW89_DBG_BTC, - "[BTC], cnt_connect = %d, link_mode = %d\n", - cnt_connect, wl_rinfo->link_mode); + "[BTC], cnt_connect = %d, connecting = %d, link_mode = %d\n", + cnt_connect, cnt_connecting, wl_rinfo->link_mode); + + _fw_set_drv_info(rtwdev, CXDRVINFO_ROLE); +} + +static void _update_wl_info_v1(struct rtw89_dev *rtwdev) +{ + struct rtw89_btc *btc = &rtwdev->btc; + struct rtw89_btc_wl_info *wl = &btc->cx.wl; + struct rtw89_btc_wl_link_info *wl_linfo = wl->link_info; + struct rtw89_btc_wl_role_info_v1 *wl_rinfo = &wl->role_info_v1; + struct rtw89_btc_wl_dbcc_info *wl_dinfo = &wl->dbcc_info; + u8 cnt_connect = 0, cnt_connecting = 0, cnt_active = 0; + u8 cnt_2g = 0, cnt_5g = 0, phy; + u32 wl_2g_ch[2] = {}, wl_5g_ch[2] = {}; + bool b2g = false, b5g = false, client_joined = false; + u8 i; + + memset(wl_rinfo, 0, sizeof(*wl_rinfo)); + + for (i = 0; i < RTW89_PORT_NUM; i++) { + if (!wl_linfo[i].active) + continue; + + cnt_active++; + wl_rinfo->active_role_v1[cnt_active - 1].role = wl_linfo[i].role; + wl_rinfo->active_role_v1[cnt_active - 1].pid = wl_linfo[i].pid; + wl_rinfo->active_role_v1[cnt_active - 1].phy = wl_linfo[i].phy; + wl_rinfo->active_role_v1[cnt_active - 1].band = wl_linfo[i].band; + wl_rinfo->active_role_v1[cnt_active - 1].noa = (u8)wl_linfo[i].noa; + wl_rinfo->active_role_v1[cnt_active - 1].connected = 0; + + wl->port_id[wl_linfo[i].role] = wl_linfo[i].pid; + + phy = wl_linfo[i].phy; + + if (rtwdev->dbcc_en && phy < RTW89_PHY_MAX) { + wl_dinfo->role[phy] = wl_linfo[i].role; + wl_dinfo->op_band[phy] = wl_linfo[i].band; + _update_dbcc_band(rtwdev, phy); + _fw_set_drv_info(rtwdev, CXDRVINFO_DBCC); + } + + if (wl_linfo[i].connected == MLME_NO_LINK) { + continue; + } else if (wl_linfo[i].connected == MLME_LINKING) { + cnt_connecting++; + } else { + cnt_connect++; + if ((wl_linfo[i].role == RTW89_WIFI_ROLE_P2P_GO || + wl_linfo[i].role == RTW89_WIFI_ROLE_AP) && + wl_linfo[i].client_cnt > 1) + client_joined = true; + } + + wl_rinfo->role_map.val |= BIT(wl_linfo[i].role); + wl_rinfo->active_role_v1[cnt_active - 1].ch = wl_linfo[i].ch; + wl_rinfo->active_role_v1[cnt_active - 1].bw = wl_linfo[i].bw; + wl_rinfo->active_role_v1[cnt_active - 1].connected = 1; + + /* only care 2 roles + BT coex */ + if (wl_linfo[i].band != RTW89_BAND_2G) { + if (cnt_5g <= ARRAY_SIZE(wl_5g_ch) - 1) + wl_5g_ch[cnt_5g] = wl_linfo[i].ch; + cnt_5g++; + b5g = true; + } else { + if (cnt_2g <= ARRAY_SIZE(wl_2g_ch) - 1) + wl_2g_ch[cnt_2g] = wl_linfo[i].ch; + cnt_2g++; + b2g = true; + } + } + + wl_rinfo->connect_cnt = cnt_connect; + + /* Be careful to change the following sequence!! */ + if (cnt_connect == 0) { + wl_rinfo->link_mode = BTC_WLINK_NOLINK; + wl_rinfo->role_map.role.none = 1; + } else if (!b2g && b5g) { + wl_rinfo->link_mode = BTC_WLINK_5G; + } else if (wl_rinfo->role_map.role.nan) { + wl_rinfo->link_mode = BTC_WLINK_2G_NAN; + } else if (cnt_connect > BTC_TDMA_WLROLE_MAX) { + wl_rinfo->link_mode = BTC_WLINK_OTHER; + } else if (b2g && b5g && cnt_connect == 2) { + if (rtwdev->dbcc_en) { + switch (wl_dinfo->role[RTW89_PHY_0]) { + case RTW89_WIFI_ROLE_STATION: + wl_rinfo->link_mode = BTC_WLINK_2G_STA; + break; + case RTW89_WIFI_ROLE_P2P_GO: + wl_rinfo->link_mode = BTC_WLINK_2G_GO; + break; + case RTW89_WIFI_ROLE_P2P_CLIENT: + wl_rinfo->link_mode = BTC_WLINK_2G_GC; + break; + case RTW89_WIFI_ROLE_AP: + wl_rinfo->link_mode = BTC_WLINK_2G_AP; + break; + default: + wl_rinfo->link_mode = BTC_WLINK_OTHER; + break; + } + } else { + wl_rinfo->link_mode = BTC_WLINK_25G_MCC; + } + } else if (!b5g && cnt_connect == 2) { + if (wl_rinfo->role_map.role.station && + (wl_rinfo->role_map.role.p2p_go || + wl_rinfo->role_map.role.p2p_gc || + wl_rinfo->role_map.role.ap)) { + if (wl_2g_ch[0] == wl_2g_ch[1]) + wl_rinfo->link_mode = BTC_WLINK_2G_SCC; + else + wl_rinfo->link_mode = BTC_WLINK_2G_MCC; + } else { + wl_rinfo->link_mode = BTC_WLINK_2G_MCC; + } + } else if (!b5g && cnt_connect == 1) { + if (wl_rinfo->role_map.role.station) + wl_rinfo->link_mode = BTC_WLINK_2G_STA; + else if (wl_rinfo->role_map.role.ap) + wl_rinfo->link_mode = BTC_WLINK_2G_AP; + else if (wl_rinfo->role_map.role.p2p_go) + wl_rinfo->link_mode = BTC_WLINK_2G_GO; + else if (wl_rinfo->role_map.role.p2p_gc) + wl_rinfo->link_mode = BTC_WLINK_2G_GC; + else + wl_rinfo->link_mode = BTC_WLINK_OTHER; + } + + /* if no client_joined, don't care P2P-GO/AP role */ + if (wl_rinfo->role_map.role.p2p_go || wl_rinfo->role_map.role.ap) { + if (!client_joined) { + if (wl_rinfo->link_mode == BTC_WLINK_2G_SCC || + wl_rinfo->link_mode == BTC_WLINK_2G_MCC) { + wl_rinfo->link_mode = BTC_WLINK_2G_STA; + wl_rinfo->connect_cnt = 1; + } else if (wl_rinfo->link_mode == BTC_WLINK_2G_GO || + wl_rinfo->link_mode == BTC_WLINK_2G_AP) { + wl_rinfo->link_mode = BTC_WLINK_NOLINK; + wl_rinfo->connect_cnt = 0; + } + } + } + + rtw89_debug(rtwdev, RTW89_DBG_BTC, + "[BTC], cnt_connect = %d, connecting = %d, link_mode = %d\n", + cnt_connect, cnt_connecting, wl_rinfo->link_mode); _fw_set_drv_info(rtwdev, CXDRVINFO_ROLE); } @@ -3584,23 +4440,32 @@ static bool _chk_wl_rfk_request(struct rtw89_dev *rtwdev) static void _run_coex(struct rtw89_dev *rtwdev, enum btc_reason_and_action reason) { + const struct rtw89_chip_info *chip = rtwdev->chip; struct rtw89_btc *btc = &rtwdev->btc; struct rtw89_btc_dm *dm = &rtwdev->btc.dm; struct rtw89_btc_cx *cx = &btc->cx; struct rtw89_btc_wl_info *wl = &btc->cx.wl; + struct rtw89_btc_bt_info *bt = &btc->cx.bt; struct rtw89_btc_wl_role_info *wl_rinfo = &wl->role_info; - u8 mode = wl_rinfo->link_mode; + struct rtw89_btc_wl_role_info_v1 *wl_rinfo_v1 = &wl->role_info_v1; + u8 mode; lockdep_assert_held(&rtwdev->mutex); - rtw89_debug(rtwdev, RTW89_DBG_BTC, "[BTC], %s(): reason=%d, mode=%d\n", - __func__, reason, mode); - rtw89_debug(rtwdev, RTW89_DBG_BTC, "[BTC], %s(): wl_only=%d, bt_only=%d\n", - __func__, dm->wl_only, dm->bt_only); dm->run_reason = reason; _update_dm_step(rtwdev, reason); _update_btc_state_map(rtwdev); + if (chip->chip_id == RTL8852A) + mode = wl_rinfo->link_mode; + else + mode = wl_rinfo_v1->link_mode; + + rtw89_debug(rtwdev, RTW89_DBG_BTC, "[BTC], %s(): reason=%d, mode=%d\n", + __func__, reason, mode); + rtw89_debug(rtwdev, RTW89_DBG_BTC, "[BTC], %s(): wl_only=%d, bt_only=%d\n", + __func__, dm->wl_only, dm->bt_only); + /* Be careful to change the following function sequence!! */ if (btc->ctrl.manual) { rtw89_debug(rtwdev, RTW89_DBG_BTC, @@ -3657,6 +4522,7 @@ void _run_coex(struct rtw89_dev *rtwdev, enum btc_reason_and_action reason) btc->ctrl.igno_bt = false; dm->freerun = false; + bt->scan_rx_low_pri = false; if (reason == BTC_RSN_NTFY_INIT) { _action_wl_init(rtwdev); @@ -3699,21 +4565,30 @@ void _run_coex(struct rtw89_dev *rtwdev, enum btc_reason_and_action reason) _action_wl_2g_sta(rtwdev); break; case BTC_WLINK_2G_AP: + bt->scan_rx_low_pri = true; _action_wl_2g_ap(rtwdev); break; case BTC_WLINK_2G_GO: + bt->scan_rx_low_pri = true; _action_wl_2g_go(rtwdev); break; case BTC_WLINK_2G_GC: + bt->scan_rx_low_pri = true; _action_wl_2g_gc(rtwdev); break; case BTC_WLINK_2G_SCC: - _action_wl_2g_scc(rtwdev); + bt->scan_rx_low_pri = true; + if (chip->chip_id == RTL8852A) + _action_wl_2g_scc(rtwdev); + else if (chip->chip_id == RTL8852C) + _action_wl_2g_scc_v1(rtwdev); break; case BTC_WLINK_2G_MCC: + bt->scan_rx_low_pri = true; _action_wl_2g_mcc(rtwdev); break; case BTC_WLINK_25G_MCC: + bt->scan_rx_low_pri = true; _action_wl_25g_mcc(rtwdev); break; case BTC_WLINK_5G: @@ -3743,11 +4618,14 @@ void rtw89_btc_ntfy_poweron(struct rtw89_dev *rtwdev) void rtw89_btc_ntfy_poweroff(struct rtw89_dev *rtwdev) { struct rtw89_btc *btc = &rtwdev->btc; + struct rtw89_btc_wl_info *wl = &btc->cx.wl; rtw89_debug(rtwdev, RTW89_DBG_BTC, "[BTC], %s(): !!\n", __func__); btc->dm.cnt_notify[BTC_NCNT_POWER_OFF]++; btc->cx.wl.status.map.rf_off = 1; + btc->cx.wl.status.map.busy = 0; + wl->status.map.lps = BTC_LPS_OFF; _write_scbd(rtwdev, BTC_WSCB_ALL, false); _run_coex(rtwdev, BTC_RSN_NTFY_POWEROFF); @@ -3807,7 +4685,7 @@ void rtw89_btc_ntfy_init(struct rtw89_dev *rtwdev, u8 mode) _write_scbd(rtwdev, BTC_WSCB_ACTIVE | BTC_WSCB_ON | BTC_WSCB_BTLOG, true); _update_bt_scbd(rtwdev, true); - if (rtw89_mac_get_ctrl_path(rtwdev)) { + if (rtw89_mac_get_ctrl_path(rtwdev) && chip->chip_id == RTL8852A) { rtw89_debug(rtwdev, RTW89_DBG_BTC, "[BTC], %s(): PTA owner warning!!\n", __func__); @@ -4150,7 +5028,8 @@ enum btc_wl_mode { void rtw89_btc_ntfy_role_info(struct rtw89_dev *rtwdev, struct rtw89_vif *rtwvif, struct rtw89_sta *rtwsta, enum btc_role_state state) { - struct rtw89_hal *hal = &rtwdev->hal; + const struct rtw89_chan *chan = rtw89_chan_get(rtwdev, RTW89_SUB_ENTITY_0); + const struct rtw89_chip_info *chip = rtwdev->chip; struct ieee80211_vif *vif = rtwvif_to_vif(rtwvif); struct ieee80211_sta *sta = rtwsta_to_sta(rtwsta); struct rtw89_btc *btc = &rtwdev->btc; @@ -4165,8 +5044,7 @@ void rtw89_btc_ntfy_role_info(struct rtw89_dev *rtwdev, struct rtw89_vif *rtwvif vif->type == NL80211_IFTYPE_STATION); rtw89_debug(rtwdev, RTW89_DBG_BTC, "[BTC], port=%d\n", rtwvif->port); rtw89_debug(rtwdev, RTW89_DBG_BTC, "[BTC], band=%d ch=%d bw=%d\n", - hal->current_band_type, hal->current_channel, - hal->current_band_width); + chan->band_type, chan->channel, chan->band_width); rtw89_debug(rtwdev, RTW89_DBG_BTC, "[BTC], associated=%d\n", state == BTC_ROLE_MSTS_STA_CONN_END); rtw89_debug(rtwdev, RTW89_DBG_BTC, @@ -4205,9 +5083,9 @@ void rtw89_btc_ntfy_role_info(struct rtw89_dev *rtwdev, struct rtw89_vif *rtwvif r.connected = MLME_LINKED; r.bcn_period = vif->bss_conf.beacon_int; r.dtim_period = vif->bss_conf.dtim_period; - r.band = hal->current_band_type; - r.ch = hal->current_channel; - r.bw = hal->current_band_width; + r.band = chan->band_type; + r.ch = chan->channel; + r.bw = chan->band_width; ether_addr_copy(r.mac_addr, rtwvif->mac_addr); if (rtwsta && vif->type == NL80211_IFTYPE_STATION) @@ -4218,7 +5096,10 @@ void rtw89_btc_ntfy_role_info(struct rtw89_dev *rtwdev, struct rtw89_vif *rtwvif wlinfo = &wl->link_info[r.pid]; memcpy(wlinfo, &r, sizeof(*wlinfo)); - _update_wl_info(rtwdev); + if (chip->chip_id == RTL8852A) + _update_wl_info(rtwdev); + else + _update_wl_info_v1(rtwdev); if (wlinfo->role == RTW89_WIFI_ROLE_STATION && wlinfo->connected == MLME_NO_LINK) @@ -4240,6 +5121,7 @@ void rtw89_btc_ntfy_radio_state(struct rtw89_dev *rtwdev, enum btc_rfctrl rf_sta const struct rtw89_chip_info *chip = rtwdev->chip; struct rtw89_btc *btc = &rtwdev->btc; struct rtw89_btc_wl_info *wl = &btc->cx.wl; + u32 val; rtw89_debug(rtwdev, RTW89_DBG_BTC, "[BTC], %s(): rf_state = %d\n", __func__, rf_state); @@ -4249,10 +5131,12 @@ void rtw89_btc_ntfy_radio_state(struct rtw89_dev *rtwdev, enum btc_rfctrl rf_sta case BTC_RFCTRL_WL_OFF: wl->status.map.rf_off = 1; wl->status.map.lps = BTC_LPS_OFF; + wl->status.map.busy = 0; break; case BTC_RFCTRL_FW_CTRL: wl->status.map.rf_off = 0; wl->status.map.lps = BTC_LPS_RF_OFF; + wl->status.map.busy = 0; break; case BTC_RFCTRL_WL_ON: default: @@ -4262,14 +5146,17 @@ void rtw89_btc_ntfy_radio_state(struct rtw89_dev *rtwdev, enum btc_rfctrl rf_sta } if (rf_state == BTC_RFCTRL_WL_ON) { + btc->dm.cnt_dm[BTC_DCNT_BTCNT_FREEZE] = 0; rtw89_btc_fw_en_rpt(rtwdev, RPT_EN_MREG | RPT_EN_BT_VER_INFO, true); - _write_scbd(rtwdev, BTC_WSCB_ACTIVE, true); + val = BTC_WSCB_ACTIVE | BTC_WSCB_ON | BTC_WSCB_BTLOG; + _write_scbd(rtwdev, val, true); _update_bt_scbd(rtwdev, true); chip->ops->btc_init_cfg(rtwdev); } else { rtw89_btc_fw_en_rpt(rtwdev, RPT_EN_ALL, false); - _write_scbd(rtwdev, BTC_WSCB_ACTIVE | BTC_WSCB_WLBUSY, false); + if (rf_state == BTC_RFCTRL_WL_OFF) + _write_scbd(rtwdev, BTC_WSCB_ALL, false); } _run_coex(rtwdev, BTC_RSN_NTFY_RADIO_STATE); @@ -4609,10 +5496,10 @@ static void _show_cx_info(struct rtw89_dev *rtwdev, struct seq_file *m) seq_printf(m, "========== [BTC COEX INFO (%d)] ==========\n", chip->chip_id); - ver_main = FIELD_GET(GENMASK(31, 24), chip->para_ver); - ver_sub = FIELD_GET(GENMASK(23, 16), chip->para_ver); - ver_hotfix = FIELD_GET(GENMASK(15, 8), chip->para_ver); - id_branch = FIELD_GET(GENMASK(7, 0), chip->para_ver); + ver_main = FIELD_GET(GENMASK(31, 24), RTW89_COEX_VERSION); + ver_sub = FIELD_GET(GENMASK(23, 16), RTW89_COEX_VERSION); + ver_hotfix = FIELD_GET(GENMASK(15, 8), RTW89_COEX_VERSION); + id_branch = FIELD_GET(GENMASK(7, 0), RTW89_COEX_VERSION); seq_printf(m, " %-15s : Coex:%d.%d.%d(branch:%d), ", "[coex_version]", ver_main, ver_sub, ver_hotfix, id_branch); @@ -4726,23 +5613,29 @@ static void _show_wl_role_info(struct rtw89_dev *rtwdev, struct seq_file *m) static void _show_wl_info(struct rtw89_dev *rtwdev, struct seq_file *m) { + const struct rtw89_chip_info *chip = rtwdev->chip; struct rtw89_btc *btc = &rtwdev->btc; struct rtw89_btc_cx *cx = &btc->cx; struct rtw89_btc_wl_info *wl = &cx->wl; struct rtw89_btc_wl_role_info *wl_rinfo = &wl->role_info; + struct rtw89_btc_wl_role_info_v1 *wl_rinfo_v1 = &wl->role_info_v1; + u8 mode; if (!(btc->dm.coex_info_map & BTC_COEX_INFO_WL)) return; seq_puts(m, "========== [WL Status] ==========\n"); - seq_printf(m, " %-15s : link_mode:%d, ", - "[status]", (u32)wl_rinfo->link_mode); + if (chip->chip_id == RTL8852A) + mode = wl_rinfo->link_mode; + else + mode = wl_rinfo_v1->link_mode; + + seq_printf(m, " %-15s : link_mode:%d, ", "[status]", mode); seq_printf(m, - "rf_off:%s, power_save:%s, scan:%s(band:%d/phy_map:0x%x), ", - wl->status.map.rf_off ? "Y" : "N", - wl->status.map.lps ? "Y" : "N", + "rf_off:%d, power_save:%d, scan:%s(band:%d/phy_map:0x%x), ", + wl->status.map.rf_off, wl->status.map.lps, wl->status.map.scan ? "Y" : "N", wl->scan_info.band[RTW89_PHY_0], wl->scan_info.phy_map); @@ -4908,6 +5801,7 @@ static void _show_bt_info(struct rtw89_dev *rtwdev, struct seq_file *m) #define CASE_BTC_ACT_STR(e) case BTC_ACT_ ## e | BTC_ACT_EXT_BIT: return #e #define CASE_BTC_POLICY_STR(e) \ case BTC_CXP_ ## e | BTC_POLICY_EXT_BIT: return #e +#define CASE_BTC_SLOT_STR(e) case CXST_ ## e: return #e static const char *steps_to_str(u16 step) { @@ -4969,9 +5863,16 @@ static const char *steps_to_str(u16 step) CASE_BTC_POLICY_STR(OFF_EQ3); CASE_BTC_POLICY_STR(OFF_BWB0); CASE_BTC_POLICY_STR(OFF_BWB1); + CASE_BTC_POLICY_STR(OFF_BWB2); + CASE_BTC_POLICY_STR(OFF_BWB3); CASE_BTC_POLICY_STR(OFFB_BWB0); CASE_BTC_POLICY_STR(OFFE_DEF); CASE_BTC_POLICY_STR(OFFE_DEF2); + CASE_BTC_POLICY_STR(OFFE_2GBWISOB); + CASE_BTC_POLICY_STR(OFFE_2GISOB); + CASE_BTC_POLICY_STR(OFFE_2GBWMIXB); + CASE_BTC_POLICY_STR(OFFE_WL); + CASE_BTC_POLICY_STR(OFFE_2GBWMIXB2); CASE_BTC_POLICY_STR(FIX_TD3030); CASE_BTC_POLICY_STR(FIX_TD5050); CASE_BTC_POLICY_STR(FIX_TD2030); @@ -4982,6 +5883,7 @@ static const char *steps_to_str(u16 step) CASE_BTC_POLICY_STR(FIX_TD2080); CASE_BTC_POLICY_STR(FIX_TDW1B1); CASE_BTC_POLICY_STR(FIX_TD4020); + CASE_BTC_POLICY_STR(FIX_TD4010ISO); CASE_BTC_POLICY_STR(PFIX_TD3030); CASE_BTC_POLICY_STR(PFIX_TD5050); CASE_BTC_POLICY_STR(PFIX_TD2030); @@ -4989,13 +5891,13 @@ static const char *steps_to_str(u16 step) CASE_BTC_POLICY_STR(PFIX_TD3070); CASE_BTC_POLICY_STR(PFIX_TD2080); CASE_BTC_POLICY_STR(PFIX_TDW1B1); - CASE_BTC_POLICY_STR(AUTO_TD50200); - CASE_BTC_POLICY_STR(AUTO_TD60200); - CASE_BTC_POLICY_STR(AUTO_TD20200); + CASE_BTC_POLICY_STR(AUTO_TD50B1); + CASE_BTC_POLICY_STR(AUTO_TD60B1); + CASE_BTC_POLICY_STR(AUTO_TD20B1); CASE_BTC_POLICY_STR(AUTO_TDW1B1); - CASE_BTC_POLICY_STR(PAUTO_TD50200); - CASE_BTC_POLICY_STR(PAUTO_TD60200); - CASE_BTC_POLICY_STR(PAUTO_TD20200); + CASE_BTC_POLICY_STR(PAUTO_TD50B1); + CASE_BTC_POLICY_STR(PAUTO_TD60B1); + CASE_BTC_POLICY_STR(PAUTO_TD20B1); CASE_BTC_POLICY_STR(PAUTO_TDW1B1); CASE_BTC_POLICY_STR(AUTO2_TD3050); CASE_BTC_POLICY_STR(AUTO2_TD3070); @@ -5014,6 +5916,32 @@ static const char *steps_to_str(u16 step) } } +static const char *id_to_slot(u32 id) +{ + switch (id) { + CASE_BTC_SLOT_STR(OFF); + CASE_BTC_SLOT_STR(B2W); + CASE_BTC_SLOT_STR(W1); + CASE_BTC_SLOT_STR(W2); + CASE_BTC_SLOT_STR(W2B); + CASE_BTC_SLOT_STR(B1); + CASE_BTC_SLOT_STR(B2); + CASE_BTC_SLOT_STR(B3); + CASE_BTC_SLOT_STR(B4); + CASE_BTC_SLOT_STR(LK); + CASE_BTC_SLOT_STR(BLK); + CASE_BTC_SLOT_STR(E2G); + CASE_BTC_SLOT_STR(E5G); + CASE_BTC_SLOT_STR(EBT); + CASE_BTC_SLOT_STR(ENULL); + CASE_BTC_SLOT_STR(WLK); + CASE_BTC_SLOT_STR(W1FDD); + CASE_BTC_SLOT_STR(B1FDD); + default: + return "unknown"; + } +} + static void seq_print_segment(struct seq_file *m, const char *prefix, u16 *data, u8 len, u8 seg_len, u8 start_idx, u8 ring_len) @@ -5105,21 +6033,31 @@ static void _show_dm_info(struct rtw89_dev *rtwdev, struct seq_file *m) (bt->hi_lna_rx ? "Hi" : "Ori"), dm->wl_btg_rx); seq_printf(m, - " %-15s : wl_tx_limit[en:%d/max_t:%dus/max_retry:%d], bt_slot_reg:%d-TU\n", + " %-15s : wl_tx_limit[en:%d/max_t:%dus/max_retry:%d], bt_slot_reg:%d-TU, bt_scan_rx_low_pri:%d\n", "[dm_ctrl]", dm->wl_tx_limit.enable, dm->wl_tx_limit.tx_time, - dm->wl_tx_limit.tx_retry, btc->bt_req_len); + dm->wl_tx_limit.tx_retry, btc->bt_req_len, bt->scan_rx_low_pri); } static void _show_error(struct rtw89_dev *rtwdev, struct seq_file *m) { + const struct rtw89_chip_info *chip = rtwdev->chip; struct rtw89_btc *btc = &rtwdev->btc; struct rtw89_btc_btf_fwinfo *pfwinfo = &btc->fwinfo; - struct rtw89_btc_fbtc_cysta *pcysta = NULL; - - pcysta = &pfwinfo->rpt_fbtc_cysta.finfo; + struct rtw89_btc_fbtc_cysta *pcysta; + struct rtw89_btc_fbtc_cysta_v1 *pcysta_v1; + u32 except_cnt, exception_map; + + if (chip->chip_id == RTL8852A) { + pcysta = &pfwinfo->rpt_fbtc_cysta.finfo; + except_cnt = le32_to_cpu(pcysta->except_cnt); + exception_map = le32_to_cpu(pcysta->exception); + } else { + pcysta_v1 = &pfwinfo->rpt_fbtc_cysta.finfo_v1; + except_cnt = le32_to_cpu(pcysta_v1->except_cnt); + exception_map = le32_to_cpu(pcysta_v1->except_map); + } - if (pfwinfo->event[BTF_EVNT_BUF_OVERFLOW] == 0 && - pcysta->except_cnt == 0 && + if (pfwinfo->event[BTF_EVNT_BUF_OVERFLOW] == 0 && except_cnt == 0 && !pfwinfo->len_mismch && !pfwinfo->fver_mismch) return; @@ -5144,16 +6082,17 @@ static void _show_error(struct rtw89_dev *rtwdev, struct seq_file *m) } /* cycle statistics exceptions */ - if (pcysta->exception || pcysta->except_cnt) { + if (exception_map || except_cnt) { seq_printf(m, "exception-type: 0x%x, exception-cnt = %d", - pcysta->exception, pcysta->except_cnt); + exception_map, except_cnt); } seq_puts(m, "\n"); } static void _show_fbtc_tdma(struct rtw89_dev *rtwdev, struct seq_file *m) { + const struct rtw89_chip_info *chip = rtwdev->chip; struct rtw89_btc *btc = &rtwdev->btc; struct rtw89_btc_btf_fwinfo *pfwinfo = &btc->fwinfo; struct rtw89_btc_rpt_cmn_info *pcinfo = NULL; @@ -5166,7 +6105,10 @@ static void _show_fbtc_tdma(struct rtw89_dev *rtwdev, struct seq_file *m) if (!pcinfo->valid) return; - t = &pfwinfo->rpt_fbtc_tdma.finfo; + if (chip->chip_id == RTL8852A) + t = &pfwinfo->rpt_fbtc_tdma.finfo; + else + t = &pfwinfo->rpt_fbtc_tdma.finfo_v1.tdma; seq_printf(m, " %-15s : ", "[tdma_policy]"); @@ -5369,12 +6311,145 @@ static void _show_fbtc_cysta(struct rtw89_dev *rtwdev, struct seq_file *m) } } +static void _show_fbtc_cysta_v1(struct rtw89_dev *rtwdev, struct seq_file *m) +{ + struct rtw89_btc *btc = &rtwdev->btc; + struct rtw89_btc_bt_a2dp_desc *a2dp = &btc->cx.bt.link_info.a2dp_desc; + struct rtw89_btc_btf_fwinfo *pfwinfo = &btc->fwinfo; + struct rtw89_btc_dm *dm = &btc->dm; + struct rtw89_btc_fbtc_a2dp_trx_stat *a2dp_trx; + struct rtw89_btc_fbtc_cysta_v1 *pcysta; + struct rtw89_btc_rpt_cmn_info *pcinfo; + u8 i, cnt = 0, slot_pair, divide_cnt; + u16 cycle, c_begin, c_end, store_index; + + pcinfo = &pfwinfo->rpt_fbtc_cysta.cinfo; + if (!pcinfo->valid) + return; + + pcysta = &pfwinfo->rpt_fbtc_cysta.finfo_v1; + seq_printf(m, + " %-15s : cycle:%d, bcn[all:%d/all_ok:%d/bt:%d/bt_ok:%d]", + "[cycle_cnt]", + le16_to_cpu(pcysta->cycles), + le32_to_cpu(pcysta->bcn_cnt[CXBCN_ALL]), + le32_to_cpu(pcysta->bcn_cnt[CXBCN_ALL_OK]), + le32_to_cpu(pcysta->bcn_cnt[CXBCN_BT_SLOT]), + le32_to_cpu(pcysta->bcn_cnt[CXBCN_BT_OK])); + + for (i = 0; i < CXST_MAX; i++) { + if (!le32_to_cpu(pcysta->slot_cnt[i])) + continue; + + seq_printf(m, ", %s:%d", id_to_slot(i), + le32_to_cpu(pcysta->slot_cnt[i])); + } + + if (dm->tdma_now.rxflctrl) + seq_printf(m, ", leak_rx:%d", le32_to_cpu(pcysta->leak_slot.cnt_rximr)); + + if (le32_to_cpu(pcysta->collision_cnt)) + seq_printf(m, ", collision:%d", le32_to_cpu(pcysta->collision_cnt)); + + if (le32_to_cpu(pcysta->skip_cnt)) + seq_printf(m, ", skip:%d", le32_to_cpu(pcysta->skip_cnt)); + + seq_puts(m, "\n"); + + seq_printf(m, " %-15s : avg_t[wl:%d/bt:%d/lk:%d.%03d]", + "[cycle_time]", + le16_to_cpu(pcysta->cycle_time.tavg[CXT_WL]), + le16_to_cpu(pcysta->cycle_time.tavg[CXT_BT]), + le16_to_cpu(pcysta->leak_slot.tavg) / 1000, + le16_to_cpu(pcysta->leak_slot.tavg) % 1000); + seq_printf(m, + ", max_t[wl:%d/bt:%d/lk:%d.%03d]", + le16_to_cpu(pcysta->cycle_time.tmax[CXT_WL]), + le16_to_cpu(pcysta->cycle_time.tmax[CXT_BT]), + le16_to_cpu(pcysta->leak_slot.tmax) / 1000, + le16_to_cpu(pcysta->leak_slot.tmax) % 1000); + seq_printf(m, + ", maxdiff_t[wl:%d/bt:%d]\n", + le16_to_cpu(pcysta->cycle_time.tmaxdiff[CXT_WL]), + le16_to_cpu(pcysta->cycle_time.tmaxdiff[CXT_BT])); + + cycle = le16_to_cpu(pcysta->cycles); + if (cycle == 0) + return; + + /* 1 cycle record 1 wl-slot and 1 bt-slot */ + slot_pair = BTC_CYCLE_SLOT_MAX / 2; + + if (cycle <= slot_pair) + c_begin = 1; + else + c_begin = cycle - slot_pair + 1; + + c_end = cycle; + + if (a2dp->exist) + divide_cnt = 3; + else + divide_cnt = BTC_CYCLE_SLOT_MAX / 4; + + for (cycle = c_begin; cycle <= c_end; cycle++) { + cnt++; + store_index = ((cycle - 1) % slot_pair) * 2; + + if (cnt % divide_cnt == 1) { + seq_printf(m, "\n\r %-15s : ", "[cycle_step]"); + } else { + seq_printf(m, "->b%02d", + le16_to_cpu(pcysta->slot_step_time[store_index])); + if (a2dp->exist) { + a2dp_trx = &pcysta->a2dp_trx[store_index]; + seq_printf(m, "(%d/%d/%dM/%d/%d/%d)", + a2dp_trx->empty_cnt, + a2dp_trx->retry_cnt, + a2dp_trx->tx_rate ? 3 : 2, + a2dp_trx->tx_cnt, + a2dp_trx->ack_cnt, + a2dp_trx->nack_cnt); + } + seq_printf(m, "->w%02d", + le16_to_cpu(pcysta->slot_step_time[store_index + 1])); + if (a2dp->exist) { + a2dp_trx = &pcysta->a2dp_trx[store_index + 1]; + seq_printf(m, "(%d/%d/%dM/%d/%d/%d)", + a2dp_trx->empty_cnt, + a2dp_trx->retry_cnt, + a2dp_trx->tx_rate ? 3 : 2, + a2dp_trx->tx_cnt, + a2dp_trx->ack_cnt, + a2dp_trx->nack_cnt); + } + } + if (cnt % (BTC_CYCLE_SLOT_MAX / 4) == 0 || cnt == c_end) + seq_puts(m, "\n"); + } + + if (a2dp->exist) { + seq_printf(m, "%-15s : a2dp_ept:%d, a2dp_late:%d", + "[a2dp_t_sta]", + le16_to_cpu(pcysta->a2dp_ept.cnt), + le16_to_cpu(pcysta->a2dp_ept.cnt_timeout)); + + seq_printf(m, ", avg_t:%d, max_t:%d", + le16_to_cpu(pcysta->a2dp_ept.tavg), + le16_to_cpu(pcysta->a2dp_ept.tmax)); + + seq_puts(m, "\n"); + } +} + static void _show_fbtc_nullsta(struct rtw89_dev *rtwdev, struct seq_file *m) { + const struct rtw89_chip_info *chip = rtwdev->chip; struct rtw89_btc *btc = &rtwdev->btc; struct rtw89_btc_btf_fwinfo *pfwinfo = &btc->fwinfo; - struct rtw89_btc_rpt_cmn_info *pcinfo = NULL; - struct rtw89_btc_fbtc_cynullsta *ns = NULL; + struct rtw89_btc_rpt_cmn_info *pcinfo; + struct rtw89_btc_fbtc_cynullsta *ns; + struct rtw89_btc_fbtc_cynullsta_v1 *ns_v1; u8 i = 0; if (!btc->dm.tdma_now.rxflctrl) @@ -5384,25 +6459,58 @@ static void _show_fbtc_nullsta(struct rtw89_dev *rtwdev, struct seq_file *m) if (!pcinfo->valid) return; - ns = &pfwinfo->rpt_fbtc_nullsta.finfo; + if (chip->chip_id == RTL8852A) { + ns = &pfwinfo->rpt_fbtc_nullsta.finfo; - seq_printf(m, " %-15s : ", "[null_sta]"); + seq_printf(m, " %-15s : ", "[null_sta]"); - for (i = 0; i < 2; i++) { - if (i != 0) - seq_printf(m, ", null-%d", i); - else - seq_printf(m, "null-%d", i); - seq_printf(m, "[ok:%d/", le32_to_cpu(ns->result[i][1])); - seq_printf(m, "fail:%d/", le32_to_cpu(ns->result[i][0])); - seq_printf(m, "on_time:%d/", le32_to_cpu(ns->result[i][2])); - seq_printf(m, "retry:%d/", le32_to_cpu(ns->result[i][3])); - seq_printf(m, "avg_t:%d.%03d/", - le32_to_cpu(ns->avg_t[i]) / 1000, - le32_to_cpu(ns->avg_t[i]) % 1000); - seq_printf(m, "max_t:%d.%03d]", - le32_to_cpu(ns->max_t[i]) / 1000, - le32_to_cpu(ns->max_t[i]) % 1000); + for (i = 0; i < 2; i++) { + if (i != 0) + seq_printf(m, ", null-%d", i); + else + seq_printf(m, "null-%d", i); + seq_printf(m, "[ok:%d/", + le32_to_cpu(ns->result[i][1])); + seq_printf(m, "fail:%d/", + le32_to_cpu(ns->result[i][0])); + seq_printf(m, "on_time:%d/", + le32_to_cpu(ns->result[i][2])); + seq_printf(m, "retry:%d/", + le32_to_cpu(ns->result[i][3])); + seq_printf(m, "avg_t:%d.%03d/", + le32_to_cpu(ns->avg_t[i]) / 1000, + le32_to_cpu(ns->avg_t[i]) % 1000); + seq_printf(m, "max_t:%d.%03d]", + le32_to_cpu(ns->max_t[i]) / 1000, + le32_to_cpu(ns->max_t[i]) % 1000); + } + } else { + ns_v1 = &pfwinfo->rpt_fbtc_nullsta.finfo_v1; + + seq_printf(m, " %-15s : ", "[null_sta]"); + + for (i = 0; i < 2; i++) { + if (i != 0) + seq_printf(m, ", null-%d", i); + else + seq_printf(m, "null-%d", i); + seq_printf(m, "[Tx:%d/", + le32_to_cpu(ns_v1->result[i][4])); + seq_printf(m, "[ok:%d/", + le32_to_cpu(ns_v1->result[i][1])); + seq_printf(m, "fail:%d/", + le32_to_cpu(ns_v1->result[i][0])); + seq_printf(m, "on_time:%d/", + le32_to_cpu(ns_v1->result[i][2])); + seq_printf(m, "retry:%d/", + le32_to_cpu(ns_v1->result[i][3])); + seq_printf(m, "avg_t:%d.%03d/", + le32_to_cpu(ns_v1->avg_t[i]) / 1000, + le32_to_cpu(ns_v1->avg_t[i]) % 1000); + seq_printf(m, "max_t:%d.%03d]", + le32_to_cpu(ns_v1->max_t[i]) / 1000, + le32_to_cpu(ns_v1->max_t[i]) % 1000); + } } seq_puts(m, "\n"); } @@ -5478,6 +6586,7 @@ static void _show_fbtc_step(struct rtw89_dev *rtwdev, struct seq_file *m) static void _show_fw_dm_msg(struct rtw89_dev *rtwdev, struct seq_file *m) { + const struct rtw89_chip_info *chip = rtwdev->chip; struct rtw89_btc *btc = &rtwdev->btc; if (!(btc->dm.coex_info_map & BTC_COEX_INFO_DM)) @@ -5486,11 +6595,57 @@ static void _show_fw_dm_msg(struct rtw89_dev *rtwdev, struct seq_file *m) _show_error(rtwdev, m); _show_fbtc_tdma(rtwdev, m); _show_fbtc_slots(rtwdev, m); - _show_fbtc_cysta(rtwdev, m); + + if (chip->chip_id == RTL8852A) + _show_fbtc_cysta(rtwdev, m); + else + _show_fbtc_cysta_v1(rtwdev, m); + _show_fbtc_nullsta(rtwdev, m); _show_fbtc_step(rtwdev, m); } +static void _get_gnt(struct rtw89_dev *rtwdev, struct rtw89_mac_ax_coex_gnt *gnt_cfg) +{ + const struct rtw89_chip_info *chip = rtwdev->chip; + struct rtw89_mac_ax_gnt *gnt; + u32 val, status; + + if (chip->chip_id == RTL8852A || chip->chip_id == RTL8852B) { + rtw89_mac_read_lte(rtwdev, R_AX_LTE_SW_CFG_1, &val); + rtw89_mac_read_lte(rtwdev, R_AX_GNT_VAL, &status); + + gnt = &gnt_cfg->band[0]; + gnt->gnt_bt_sw_en = !!(val & B_AX_GNT_BT_RFC_S0_SW_CTRL); + gnt->gnt_bt = !!(status & B_AX_GNT_BT_RFC_S0_STA); + gnt->gnt_wl_sw_en = !!(val & B_AX_GNT_WL_RFC_S0_SW_CTRL); + gnt->gnt_wl = !!(status & B_AX_GNT_WL_RFC_S0_STA); + + gnt = &gnt_cfg->band[1]; + gnt->gnt_bt_sw_en = !!(val & B_AX_GNT_BT_RFC_S1_SW_CTRL); + gnt->gnt_bt = !!(status & B_AX_GNT_BT_RFC_S1_STA); + gnt->gnt_wl_sw_en = !!(val & B_AX_GNT_WL_RFC_S1_SW_CTRL); + gnt->gnt_wl = !!(status & B_AX_GNT_WL_RFC_S1_STA); + } else if (chip->chip_id == RTL8852C) { + val = rtw89_read32(rtwdev, R_AX_GNT_SW_CTRL); + status = rtw89_read32(rtwdev, R_AX_GNT_VAL_V1); + + gnt = &gnt_cfg->band[0]; + gnt->gnt_bt_sw_en = !!(val & B_AX_GNT_BT_RFC_S0_SWCTRL); + gnt->gnt_bt = !!(status & B_AX_GNT_BT_RFC_S0); + gnt->gnt_wl_sw_en = !!(val & B_AX_GNT_WL_RFC_S0_SWCTRL); + gnt->gnt_wl = !!(status & B_AX_GNT_WL_RFC_S0); + + gnt = &gnt_cfg->band[1]; + gnt->gnt_bt_sw_en = !!(val & B_AX_GNT_BT_RFC_S1_SWCTRL); + gnt->gnt_bt = !!(status & B_AX_GNT_BT_RFC_S1); + gnt->gnt_wl_sw_en = !!(val & B_AX_GNT_WL_RFC_S1_SWCTRL); + gnt->gnt_wl = !!(status & B_AX_GNT_WL_RFC_S1); + } else { + return; + } +} + static void _show_mreg(struct rtw89_dev *rtwdev, struct seq_file *m) { const struct rtw89_chip_info *chip = rtwdev->chip; @@ -5502,7 +6657,8 @@ static void _show_mreg(struct rtw89_dev *rtwdev, struct seq_file *m) struct rtw89_btc_cx *cx = &btc->cx; struct rtw89_btc_wl_info *wl = &btc->cx.wl; struct rtw89_btc_bt_info *bt = &btc->cx.bt; - struct rtw89_mac_ax_gnt gnt[2] = {0}; + struct rtw89_mac_ax_coex_gnt gnt_cfg = {}; + struct rtw89_mac_ax_gnt gnt; u8 i = 0, type = 0, cnt = 0; u32 val, offset; @@ -5519,45 +6675,28 @@ static void _show_mreg(struct rtw89_dev *rtwdev, struct seq_file *m) /* To avoid I/O if WL LPS or power-off */ if (!wl->status.map.lps && !wl->status.map.rf_off) { - rtw89_mac_read_lte(rtwdev, R_AX_LTE_SW_CFG_1, &val); - if (val & (B_AX_GNT_BT_RFC_S0_SW_VAL | - B_AX_GNT_BT_BB_S0_SW_VAL)) - gnt[0].gnt_bt = true; - if (val & (B_AX_GNT_BT_RFC_S0_SW_CTRL | - B_AX_GNT_BT_BB_S0_SW_CTRL)) - gnt[0].gnt_bt_sw_en = true; - if (val & (B_AX_GNT_WL_RFC_S0_SW_VAL | - B_AX_GNT_WL_BB_S0_SW_VAL)) - gnt[0].gnt_wl = true; - if (val & (B_AX_GNT_WL_RFC_S0_SW_CTRL | - B_AX_GNT_WL_BB_S0_SW_CTRL)) - gnt[0].gnt_wl_sw_en = true; - - if (val & (B_AX_GNT_BT_RFC_S1_SW_VAL | - B_AX_GNT_BT_BB_S1_SW_VAL)) - gnt[1].gnt_bt = true; - if (val & (B_AX_GNT_BT_RFC_S1_SW_CTRL | - B_AX_GNT_BT_BB_S1_SW_CTRL)) - gnt[1].gnt_bt_sw_en = true; - if (val & (B_AX_GNT_WL_RFC_S1_SW_VAL | - B_AX_GNT_WL_BB_S1_SW_VAL)) - gnt[1].gnt_wl = true; - if (val & (B_AX_GNT_WL_RFC_S1_SW_CTRL | - B_AX_GNT_WL_BB_S1_SW_CTRL)) - gnt[1].gnt_wl_sw_en = true; + if (chip->chip_id == RTL8852A) + btc->dm.pta_owner = rtw89_mac_get_ctrl_path(rtwdev); + else if (chip->chip_id == RTL8852C) + btc->dm.pta_owner = 0; + _get_gnt(rtwdev, &gnt_cfg); + gnt = gnt_cfg.band[0]; seq_printf(m, " %-15s : pta_owner:%s, phy-0[gnt_wl:%s-%d/gnt_bt:%s-%d], ", "[gnt_status]", - (rtw89_mac_get_ctrl_path(rtwdev) ? "WL" : "BT"), - (gnt[0].gnt_wl_sw_en ? "SW" : "HW"), gnt[0].gnt_wl, - (gnt[0].gnt_bt_sw_en ? "SW" : "HW"), gnt[0].gnt_bt); + chip->chip_id == RTL8852C ? "HW" : + btc->dm.pta_owner == BTC_CTRL_BY_WL ? "WL" : "BT", + gnt.gnt_wl_sw_en ? "SW" : "HW", gnt.gnt_wl, + gnt.gnt_bt_sw_en ? "SW" : "HW", gnt.gnt_bt); + gnt = gnt_cfg.band[1]; seq_printf(m, "phy-1[gnt_wl:%s-%d/gnt_bt:%s-%d]\n", - (gnt[1].gnt_wl_sw_en ? "SW" : "HW"), gnt[1].gnt_wl, - (gnt[1].gnt_bt_sw_en ? "SW" : "HW"), gnt[1].gnt_bt); + gnt.gnt_wl_sw_en ? "SW" : "HW", + gnt.gnt_wl, + gnt.gnt_bt_sw_en ? "SW" : "HW", + gnt.gnt_bt); } - pcinfo = &pfwinfo->rpt_fbtc_mregval.cinfo; if (!pcinfo->valid) { rtw89_debug(rtwdev, RTW89_DBG_BTC, @@ -5714,8 +6853,121 @@ static void _show_summary(struct rtw89_dev *rtwdev, struct seq_file *m) cnt[BTC_NCNT_CUSTOMERIZE]); } +static void _show_summary_v1(struct rtw89_dev *rtwdev, struct seq_file *m) +{ + struct rtw89_btc *btc = &rtwdev->btc; + struct rtw89_btc_btf_fwinfo *pfwinfo = &btc->fwinfo; + struct rtw89_btc_fbtc_rpt_ctrl_v1 *prptctrl; + struct rtw89_btc_rpt_cmn_info *pcinfo; + struct rtw89_btc_cx *cx = &btc->cx; + struct rtw89_btc_dm *dm = &btc->dm; + struct rtw89_btc_wl_info *wl = &cx->wl; + struct rtw89_btc_bt_info *bt = &cx->bt; + u32 cnt_sum = 0, *cnt = btc->dm.cnt_notify; + u8 i; + + if (!(dm->coex_info_map & BTC_COEX_INFO_SUMMARY)) + return; + + seq_puts(m, "========== [Statistics] ==========\n"); + + pcinfo = &pfwinfo->rpt_ctrl.cinfo; + if (pcinfo->valid && !wl->status.map.lps && !wl->status.map.rf_off) { + prptctrl = &pfwinfo->rpt_ctrl.finfo_v1; + + seq_printf(m, + " %-15s : h2c_cnt=%d(fail:%d, fw_recv:%d), c2h_cnt=%d(fw_send:%d), ", + "[summary]", pfwinfo->cnt_h2c, + pfwinfo->cnt_h2c_fail, + le32_to_cpu(prptctrl->rpt_info.cnt_h2c), + pfwinfo->cnt_c2h, + le32_to_cpu(prptctrl->rpt_info.cnt_c2h)); + + seq_printf(m, + "rpt_cnt=%d(fw_send:%d), rpt_map=0x%x, dm_error_map:0x%x", + pfwinfo->event[BTF_EVNT_RPT], + le32_to_cpu(prptctrl->rpt_info.cnt), + le32_to_cpu(prptctrl->rpt_info.en), + dm->error.val); + + if (dm->error.map.wl_fw_hang) + seq_puts(m, " (WL FW Hang!!)"); + seq_puts(m, "\n"); + seq_printf(m, + " %-15s : send_ok:%d, send_fail:%d, recv:%d, ", + "[mailbox]", + le32_to_cpu(prptctrl->bt_mbx_info.cnt_send_ok), + le32_to_cpu(prptctrl->bt_mbx_info.cnt_send_fail), + le32_to_cpu(prptctrl->bt_mbx_info.cnt_recv)); + + seq_printf(m, + "A2DP_empty:%d(stop:%d, tx:%d, ack:%d, nack:%d)\n", + le32_to_cpu(prptctrl->bt_mbx_info.a2dp.cnt_empty), + le32_to_cpu(prptctrl->bt_mbx_info.a2dp.cnt_flowctrl), + le32_to_cpu(prptctrl->bt_mbx_info.a2dp.cnt_tx), + le32_to_cpu(prptctrl->bt_mbx_info.a2dp.cnt_ack), + le32_to_cpu(prptctrl->bt_mbx_info.a2dp.cnt_nack)); + + seq_printf(m, + " %-15s : wl_rfk[req:%d/go:%d/reject:%d/timeout:%d]", + "[RFK]", cx->cnt_wl[BTC_WCNT_RFK_REQ], + cx->cnt_wl[BTC_WCNT_RFK_GO], + cx->cnt_wl[BTC_WCNT_RFK_REJECT], + cx->cnt_wl[BTC_WCNT_RFK_TIMEOUT]); + + seq_printf(m, + ", bt_rfk[req:%d/go:%d/reject:%d/timeout:%d/fail:%d]\n", + le32_to_cpu(prptctrl->bt_cnt[BTC_BCNT_RFK_REQ]), + le32_to_cpu(prptctrl->bt_cnt[BTC_BCNT_RFK_GO]), + le32_to_cpu(prptctrl->bt_cnt[BTC_BCNT_RFK_REJECT]), + le32_to_cpu(prptctrl->bt_cnt[BTC_BCNT_RFK_TIMEOUT]), + le32_to_cpu(prptctrl->bt_cnt[BTC_BCNT_RFK_FAIL])); + + if (le32_to_cpu(prptctrl->bt_cnt[BTC_BCNT_RFK_TIMEOUT]) > 0) + bt->rfk_info.map.timeout = 1; + else + bt->rfk_info.map.timeout = 0; + + dm->error.map.wl_rfk_timeout = bt->rfk_info.map.timeout; + } else { + seq_printf(m, + " %-15s : h2c_cnt=%d(fail:%d), c2h_cnt=%d, rpt_cnt=%d, rpt_map=0x%x", + "[summary]", pfwinfo->cnt_h2c, + pfwinfo->cnt_h2c_fail, pfwinfo->cnt_c2h, + pfwinfo->event[BTF_EVNT_RPT], + btc->fwinfo.rpt_en_map); + seq_puts(m, " (WL FW report invalid!!)\n"); + } + + for (i = 0; i < BTC_NCNT_NUM; i++) + cnt_sum += dm->cnt_notify[i]; + + seq_printf(m, + " %-15s : total=%d, show_coex_info=%d, power_on=%d, init_coex=%d, ", + "[notify_cnt]", cnt_sum, cnt[BTC_NCNT_SHOW_COEX_INFO], + cnt[BTC_NCNT_POWER_ON], cnt[BTC_NCNT_INIT_COEX]); + + seq_printf(m, + "power_off=%d, radio_state=%d, role_info=%d, wl_rfk=%d, wl_sta=%d\n", + cnt[BTC_NCNT_POWER_OFF], cnt[BTC_NCNT_RADIO_STATE], + cnt[BTC_NCNT_ROLE_INFO], cnt[BTC_NCNT_WL_RFK], + cnt[BTC_NCNT_WL_STA]); + + seq_printf(m, + " %-15s : scan_start=%d, scan_finish=%d, switch_band=%d, special_pkt=%d, ", + "[notify_cnt]", cnt[BTC_NCNT_SCAN_START], + cnt[BTC_NCNT_SCAN_FINISH], cnt[BTC_NCNT_SWITCH_BAND], + cnt[BTC_NCNT_SPECIAL_PACKET]); + + seq_printf(m, + "timer=%d, control=%d, customerize=%d\n", + cnt[BTC_NCNT_TIMER], cnt[BTC_NCNT_CONTROL], + cnt[BTC_NCNT_CUSTOMERIZE]); +} + void rtw89_btc_dump_info(struct rtw89_dev *rtwdev, struct seq_file *m) { + const struct rtw89_chip_info *chip = rtwdev->chip; struct rtw89_fw_suit *fw_suit = &rtwdev->fw.normal; struct rtw89_btc *btc = &rtwdev->btc; struct rtw89_btc_cx *cx = &btc->cx; @@ -5746,5 +6998,8 @@ void rtw89_btc_dump_info(struct rtw89_dev *rtwdev, struct seq_file *m) _show_dm_info(rtwdev, m); _show_fw_dm_msg(rtwdev, m); _show_mreg(rtwdev, m); - _show_summary(rtwdev, m); + if (chip->chip_id == RTL8852A) + _show_summary(rtwdev, m); + else + _show_summary_v1(rtwdev, m); } diff --git a/drivers/net/wireless/realtek/rtw89/coex.h b/drivers/net/wireless/realtek/rtw89/coex.h index c3a722d259d7..ca16afa97ec0 100644 --- a/drivers/net/wireless/realtek/rtw89/coex.h +++ b/drivers/net/wireless/realtek/rtw89/coex.h @@ -162,17 +162,19 @@ void rtw89_coex_act1_work(struct work_struct *work); void rtw89_coex_bt_devinfo_work(struct work_struct *work); void rtw89_coex_rfk_chk_work(struct work_struct *work); void rtw89_coex_power_on(struct rtw89_dev *rtwdev); +void rtw89_btc_set_policy(struct rtw89_dev *rtwdev, u16 policy_type); +void rtw89_btc_set_policy_v1(struct rtw89_dev *rtwdev, u16 policy_type); static inline u8 rtw89_btc_phymap(struct rtw89_dev *rtwdev, enum rtw89_phy_idx phy_idx, enum rtw89_rf_path_bit paths) { - struct rtw89_hal *hal = &rtwdev->hal; + const struct rtw89_chan *chan = rtw89_chan_get(rtwdev, RTW89_SUB_ENTITY_0); u8 phy_map; phy_map = FIELD_PREP(BTC_RFK_PATH_MAP, paths) | FIELD_PREP(BTC_RFK_PHY_MAP, BIT(phy_idx)) | - FIELD_PREP(BTC_RFK_BAND_MAP, hal->current_band_type); + FIELD_PREP(BTC_RFK_BAND_MAP, chan->band_type); return phy_map; } diff --git a/drivers/net/wireless/realtek/rtw89/core.c b/drivers/net/wireless/realtek/rtw89/core.c index a5880a54812e..bc2994865372 100644 --- a/drivers/net/wireless/realtek/rtw89/core.c +++ b/drivers/net/wireless/realtek/rtw89/core.c @@ -5,6 +5,7 @@ #include <linux/udp.h> #include "cam.h" +#include "chan.h" #include "coex.h" #include "core.h" #include "efuse.h" @@ -224,18 +225,22 @@ static void rtw89_traffic_stats_accu(struct rtw89_dev *rtwdev, } } -static void rtw89_get_channel_params(struct cfg80211_chan_def *chandef, - struct rtw89_channel_params *chan_param) +void rtw89_get_default_chandef(struct cfg80211_chan_def *chandef) +{ + cfg80211_chandef_create(chandef, &rtw89_channels_2ghz[0], + NL80211_CHAN_NO_HT); +} + +static void rtw89_get_channel_params(const struct cfg80211_chan_def *chandef, + struct rtw89_chan *chan) { struct ieee80211_channel *channel = chandef->chan; enum nl80211_chan_width width = chandef->width; u32 primary_freq, center_freq; u8 center_chan; u8 bandwidth = RTW89_CHANNEL_WIDTH_20; - u8 primary_chan_idx = 0; u32 offset; u8 band; - u8 subband; center_chan = channel->hw_value; primary_freq = channel->center_freq; @@ -245,15 +250,12 @@ static void rtw89_get_channel_params(struct cfg80211_chan_def *chandef, case NL80211_CHAN_WIDTH_20_NOHT: case NL80211_CHAN_WIDTH_20: bandwidth = RTW89_CHANNEL_WIDTH_20; - primary_chan_idx = RTW89_SC_DONT_CARE; break; case NL80211_CHAN_WIDTH_40: bandwidth = RTW89_CHANNEL_WIDTH_40; if (primary_freq > center_freq) { - primary_chan_idx = RTW89_SC_20_UPPER; center_chan -= 2; } else { - primary_chan_idx = RTW89_SC_20_LOWER; center_chan += 2; } break; @@ -262,11 +264,9 @@ static void rtw89_get_channel_params(struct cfg80211_chan_def *chandef, bandwidth = nl_to_rtw89_bandwidth(width); if (primary_freq > center_freq) { offset = (primary_freq - center_freq - 10) / 20; - primary_chan_idx = RTW89_SC_20_UPPER + offset * 2; center_chan -= 2 + offset * 4; } else { offset = (center_freq - primary_freq - 10) / 20; - primary_chan_idx = RTW89_SC_20_LOWER + offset * 2; center_chan += 2 + offset * 4; } break; @@ -288,110 +288,76 @@ static void rtw89_get_channel_params(struct cfg80211_chan_def *chandef, break; } - switch (band) { - default: - case RTW89_BAND_2G: - switch (center_chan) { - default: - case 1 ... 14: - subband = RTW89_CH_2G; - break; - } - break; - case RTW89_BAND_5G: - switch (center_chan) { - default: - case 36 ... 64: - subband = RTW89_CH_5G_BAND_1; - break; - case 100 ... 144: - subband = RTW89_CH_5G_BAND_3; - break; - case 149 ... 177: - subband = RTW89_CH_5G_BAND_4; - break; - } - break; - case RTW89_BAND_6G: - switch (center_chan) { - default: - case 1 ... 29: - subband = RTW89_CH_6G_BAND_IDX0; - break; - case 33 ... 61: - subband = RTW89_CH_6G_BAND_IDX1; - break; - case 65 ... 93: - subband = RTW89_CH_6G_BAND_IDX2; - break; - case 97 ... 125: - subband = RTW89_CH_6G_BAND_IDX3; - break; - case 129 ... 157: - subband = RTW89_CH_6G_BAND_IDX4; - break; - case 161 ... 189: - subband = RTW89_CH_6G_BAND_IDX5; - break; - case 193 ... 221: - subband = RTW89_CH_6G_BAND_IDX6; - break; - case 225 ... 253: - subband = RTW89_CH_6G_BAND_IDX7; - break; - } - break; - } + rtw89_chan_create(chan, center_chan, channel->hw_value, band, bandwidth); +} - chan_param->center_chan = center_chan; - chan_param->center_freq = center_freq; - chan_param->primary_chan = channel->hw_value; - chan_param->bandwidth = bandwidth; - chan_param->pri_ch_idx = primary_chan_idx; - chan_param->band_type = band; - chan_param->subband_type = subband; +void rtw89_core_set_chip_txpwr(struct rtw89_dev *rtwdev) +{ + const struct rtw89_chip_info *chip = rtwdev->chip; + const struct rtw89_chan *chan; + enum rtw89_sub_entity_idx sub_entity_idx; + enum rtw89_phy_idx phy_idx; + enum rtw89_entity_mode mode; + bool entity_active; + + entity_active = rtw89_get_entity_state(rtwdev); + if (!entity_active) + return; + + mode = rtw89_get_entity_mode(rtwdev); + if (WARN(mode != RTW89_ENTITY_MODE_SCC, "Invalid ent mode: %d\n", mode)) + return; + + sub_entity_idx = RTW89_SUB_ENTITY_0; + phy_idx = RTW89_PHY_0; + chan = rtw89_chan_get(rtwdev, sub_entity_idx); + if (chip->ops->set_txpwr) + chip->ops->set_txpwr(rtwdev, chan, phy_idx); } void rtw89_set_channel(struct rtw89_dev *rtwdev) { - struct ieee80211_hw *hw = rtwdev->hw; const struct rtw89_chip_info *chip = rtwdev->chip; - struct rtw89_hal *hal = &rtwdev->hal; - struct rtw89_channel_params ch_param; + const struct cfg80211_chan_def *chandef; + enum rtw89_sub_entity_idx sub_entity_idx; + enum rtw89_mac_idx mac_idx; + enum rtw89_phy_idx phy_idx; + struct rtw89_chan chan; struct rtw89_channel_help_params bak; - u8 center_chan, bandwidth; + enum rtw89_entity_mode mode; bool band_changed; + bool entity_active; - rtw89_get_channel_params(&hw->conf.chandef, &ch_param); - if (WARN(ch_param.center_chan == 0, "Invalid channel\n")) + entity_active = rtw89_get_entity_state(rtwdev); + + mode = rtw89_entity_recalc(rtwdev); + if (WARN(mode != RTW89_ENTITY_MODE_SCC, "Invalid ent mode: %d\n", mode)) return; - center_chan = ch_param.center_chan; - bandwidth = ch_param.bandwidth; - band_changed = hal->current_band_type != ch_param.band_type || - hal->current_channel == 0; + sub_entity_idx = RTW89_SUB_ENTITY_0; + mac_idx = RTW89_MAC_0; + phy_idx = RTW89_PHY_0; + chandef = rtw89_chandef_get(rtwdev, sub_entity_idx); + rtw89_get_channel_params(chandef, &chan); + if (WARN(chan.channel == 0, "Invalid channel\n")) + return; - hal->current_band_width = bandwidth; - hal->current_channel = center_chan; - hal->current_freq = ch_param.center_freq; - hal->prev_primary_channel = hal->current_primary_channel; - hal->prev_band_type = hal->current_band_type; - hal->current_primary_channel = ch_param.primary_chan; - hal->current_band_type = ch_param.band_type; - hal->current_subband = ch_param.subband_type; + band_changed = rtw89_assign_entity_chan(rtwdev, sub_entity_idx, &chan); - rtw89_chip_set_channel_prepare(rtwdev, &bak); + rtw89_chip_set_channel_prepare(rtwdev, &bak, &chan, mac_idx, phy_idx); - chip->ops->set_channel(rtwdev, &ch_param); + chip->ops->set_channel(rtwdev, &chan, mac_idx, phy_idx); - rtw89_chip_set_txpwr(rtwdev); + rtw89_core_set_chip_txpwr(rtwdev); - rtw89_chip_set_channel_done(rtwdev, &bak); + rtw89_chip_set_channel_done(rtwdev, &bak, &chan, mac_idx, phy_idx); - if (band_changed) { - rtw89_btc_ntfy_switch_band(rtwdev, RTW89_PHY_0, hal->current_band_type); - rtw89_chip_rfk_band_changed(rtwdev); + if (!entity_active || band_changed) { + rtw89_btc_ntfy_switch_band(rtwdev, phy_idx, chan.band_type); + rtw89_chip_rfk_band_changed(rtwdev, phy_idx); } + + rtw89_set_entity_state(rtwdev, true); } static enum rtw89_core_tx_type @@ -529,9 +495,15 @@ static u16 rtw89_core_get_mgmt_rate(struct rtw89_dev *rtwdev, struct sk_buff *skb = tx_req->skb; struct ieee80211_tx_info *tx_info = IEEE80211_SKB_CB(skb); struct ieee80211_vif *vif = tx_info->control.vif; - struct rtw89_hal *hal = &rtwdev->hal; - u16 lowest_rate = hal->current_band_type == RTW89_BAND_2G ? - RTW89_HW_RATE_CCK1 : RTW89_HW_RATE_OFDM6; + const struct rtw89_chan *chan = rtw89_chan_get(rtwdev, RTW89_SUB_ENTITY_0); + u16 lowest_rate; + + if (tx_info->flags & IEEE80211_TX_CTL_NO_CCK_RATE || vif->p2p) + lowest_rate = RTW89_HW_RATE_OFDM6; + else if (chan->band_type == RTW89_BAND_2G) + lowest_rate = RTW89_HW_RATE_CCK1; + else + lowest_rate = RTW89_HW_RATE_OFDM6; if (!vif || !vif->bss_conf.basic_rates || !tx_req->sta) return lowest_rate; @@ -546,6 +518,7 @@ rtw89_core_tx_update_mgmt_info(struct rtw89_dev *rtwdev, struct ieee80211_vif *vif = tx_req->vif; struct rtw89_vif *rtwvif = (struct rtw89_vif *)vif->drv_priv; struct rtw89_tx_desc_info *desc_info = &tx_req->desc_info; + const struct rtw89_chan *chan = rtw89_chan_get(rtwdev, RTW89_SUB_ENTITY_0); u8 qsel, ch_dma; qsel = desc_info->hiq ? RTW89_TX_QSEL_B0_HI : RTW89_TX_QSEL_B0_MGMT; @@ -564,9 +537,9 @@ rtw89_core_tx_update_mgmt_info(struct rtw89_dev *rtwdev, desc_info->data_rate = rtw89_core_get_mgmt_rate(rtwdev, tx_req); rtw89_debug(rtwdev, RTW89_DBG_TXRX, - "tx mgmt frame with rate 0x%x on channel %d (bw %d)\n", - desc_info->data_rate, rtwdev->hal.current_channel, - rtwdev->hal.current_band_width); + "tx mgmt frame with rate 0x%x on channel %d (band %d, bw %d)\n", + desc_info->data_rate, chan->channel, chan->band_type, + chan->band_width); } static void @@ -591,15 +564,16 @@ static void rtw89_core_get_no_ul_ofdma_htc(struct rtw89_dev *rtwdev, __le32 *htc }; const struct rtw89_chip_info *chip = rtwdev->chip; struct rtw89_hal *hal = &rtwdev->hal; + const struct rtw89_chan *chan = rtw89_chan_get(rtwdev, RTW89_SUB_ENTITY_0); u8 om_bandwidth; if (!chip->dis_2g_40m_ul_ofdma || - hal->current_band_type != RTW89_BAND_2G || - hal->current_band_width != RTW89_CHANNEL_WIDTH_40) + chan->band_type != RTW89_BAND_2G || + chan->band_width != RTW89_CHANNEL_WIDTH_40) return; - om_bandwidth = hal->current_band_width < ARRAY_SIZE(rtw89_bandwidth_to_om) ? - rtw89_bandwidth_to_om[hal->current_band_width] : 0; + om_bandwidth = chan->band_width < ARRAY_SIZE(rtw89_bandwidth_to_om) ? + rtw89_bandwidth_to_om[chan->band_width] : 0; *htc = le32_encode_bits(RTW89_HTC_VARIANT_HE, RTW89_HTC_MASK_VARIANT) | le32_encode_bits(RTW89_HTC_VARIANT_HE_CID_OM, RTW89_HTC_MASK_CTL_ID) | le32_encode_bits(hal->rx_nss - 1, RTW89_HTC_MASK_HTC_OM_RX_NSS) | @@ -617,6 +591,7 @@ __rtw89_core_tx_check_he_qos_htc(struct rtw89_dev *rtwdev, enum btc_pkt_type pkt_type) { struct ieee80211_sta *sta = tx_req->sta; + struct rtw89_sta *rtwsta = sta_to_rtwsta_safe(sta); struct sk_buff *skb = tx_req->skb; struct ieee80211_hdr *hdr = (void *)skb->data; __le16 fc = hdr->frame_control; @@ -634,6 +609,9 @@ __rtw89_core_tx_check_he_qos_htc(struct rtw89_dev *rtwdev, if (skb_headroom(skb) < IEEE80211_HT_CTL_LEN) return false; + if (rtwsta && rtwsta->ra_report.might_fallback_legacy) + return false; + return true; } @@ -713,7 +691,7 @@ rtw89_core_tx_update_data_info(struct rtw89_dev *rtwdev, struct ieee80211_vif *vif = tx_req->vif; struct rtw89_vif *rtwvif = (struct rtw89_vif *)vif->drv_priv; struct rtw89_phy_rate_pattern *rate_pattern = &rtwvif->rate_pattern; - struct rtw89_hal *hal = &rtwdev->hal; + const struct rtw89_chan *chan = rtw89_chan_get(rtwdev, RTW89_SUB_ENTITY_0); struct rtw89_tx_desc_info *desc_info = &tx_req->desc_info; struct sk_buff *skb = tx_req->skb; u8 tid, tid_indicate; @@ -736,9 +714,11 @@ rtw89_core_tx_update_data_info(struct rtw89_dev *rtwdev, if (IEEE80211_SKB_CB(skb)->control.hw_key) rtw89_core_tx_update_sec_key(rtwdev, tx_req); - if (rate_pattern->enable) + if (vif->p2p) + desc_info->data_retry_lowest_rate = RTW89_HW_RATE_OFDM6; + else if (rate_pattern->enable) desc_info->data_retry_lowest_rate = rate_pattern->rate; - else if (hal->current_band_type == RTW89_BAND_2G) + else if (chan->band_type == RTW89_BAND_2G) desc_info->data_retry_lowest_rate = RTW89_HW_RATE_CCK1; else desc_info->data_retry_lowest_rate = RTW89_HW_RATE_OFDM6; @@ -796,13 +776,16 @@ static void rtw89_core_tx_wake(struct rtw89_dev *rtwdev, struct rtw89_core_tx_request *tx_req) { + const struct rtw89_chip_info *chip = rtwdev->chip; + if (!RTW89_CHK_FW_FEATURE(TX_WAKE, &rtwdev->fw)) return; if (!test_bit(RTW89_FLAG_LOW_POWER_MODE, rtwdev->flags)) return; - if (tx_req->tx_type != RTW89_CORE_TX_TYPE_MGMT) + if (chip->chip_id != RTL8852C && + tx_req->tx_type != RTW89_CORE_TX_TYPE_MGMT) return; rtw89_mac_notify_wake(rtwdev); @@ -872,6 +855,7 @@ int rtw89_h2c_tx(struct rtw89_dev *rtwdev, rtw89_debug(rtwdev, RTW89_DBG_FW, "ignore h2c due to power is off with firmware state=%d\n", test_bit(RTW89_FLAG_FW_RDY, rtwdev->flags)); + dev_kfree_skb(skb); return 0; } @@ -1021,7 +1005,8 @@ static __le32 rtw89_build_txwd_info0(struct rtw89_tx_desc_info *desc_info) static __le32 rtw89_build_txwd_info0_v1(struct rtw89_tx_desc_info *desc_info) { - u32 dword = FIELD_PREP(RTW89_TXWD_INFO0_DISDATAFB, desc_info->dis_data_fb); + u32 dword = FIELD_PREP(RTW89_TXWD_INFO0_DISDATAFB, desc_info->dis_data_fb) | + FIELD_PREP(RTW89_TXWD_INFO0_MULTIPORT_ID, desc_info->port); return cpu_to_le32(dword); } @@ -1171,9 +1156,14 @@ static void rtw89_core_rx_process_phy_ppdu_iter(void *data, { struct rtw89_sta *rtwsta = (struct rtw89_sta *)sta->drv_priv; struct rtw89_rx_phy_ppdu *phy_ppdu = (struct rtw89_rx_phy_ppdu *)data; + struct rtw89_dev *rtwdev = rtwsta->rtwdev; + int i; - if (rtwsta->mac_id == phy_ppdu->mac_id && phy_ppdu->to_self) + if (rtwsta->mac_id == phy_ppdu->mac_id && phy_ppdu->to_self) { ewma_rssi_add(&rtwsta->avg_rssi, phy_ppdu->rssi_avg); + for (i = 0; i < rtwdev->chip->rf_path_num; i++) + ewma_rssi_add(&rtwsta->rssi[i], phy_ppdu->rssi[i]); + } } #define VAR_LEN 0xff @@ -1229,15 +1219,15 @@ static int rtw89_core_process_phy_status_ie(struct rtw89_dev *rtwdev, u8 *addr, static void rtw89_core_update_phy_ppdu(struct rtw89_rx_phy_ppdu *phy_ppdu) { - s8 *rssi = phy_ppdu->rssi; + u8 *rssi = phy_ppdu->rssi; u8 *buf = phy_ppdu->buf; phy_ppdu->ie = RTW89_GET_PHY_STS_IE_MAP(buf); phy_ppdu->rssi_avg = RTW89_GET_PHY_STS_RSSI_AVG(buf); - rssi[RF_PATH_A] = RTW89_RSSI_RAW_TO_DBM(RTW89_GET_PHY_STS_RSSI_A(buf)); - rssi[RF_PATH_B] = RTW89_RSSI_RAW_TO_DBM(RTW89_GET_PHY_STS_RSSI_B(buf)); - rssi[RF_PATH_C] = RTW89_RSSI_RAW_TO_DBM(RTW89_GET_PHY_STS_RSSI_C(buf)); - rssi[RF_PATH_D] = RTW89_RSSI_RAW_TO_DBM(RTW89_GET_PHY_STS_RSSI_D(buf)); + rssi[RF_PATH_A] = RTW89_GET_PHY_STS_RSSI_A(buf); + rssi[RF_PATH_B] = RTW89_GET_PHY_STS_RSSI_B(buf); + rssi[RF_PATH_C] = RTW89_GET_PHY_STS_RSSI_C(buf); + rssi[RF_PATH_D] = RTW89_GET_PHY_STS_RSSI_D(buf); } static int rtw89_core_rx_process_phy_ppdu(struct rtw89_dev *rtwdev, @@ -1448,8 +1438,11 @@ static void rtw89_core_rx_stats(struct rtw89_dev *rtwdev, static void rtw89_correct_cck_chan(struct rtw89_dev *rtwdev, struct ieee80211_rx_status *status) { - u16 chan = rtwdev->hal.prev_primary_channel; - u8 band = chan <= 14 ? NL80211_BAND_2GHZ : NL80211_BAND_5GHZ; + const struct rtw89_chan_rcd *rcd = + rtw89_chan_rcd_get(rtwdev, RTW89_SUB_ENTITY_0); + u16 chan = rcd->prev_primary_channel; + u8 band = rcd->prev_band_type == RTW89_BAND_2G ? + NL80211_BAND_2GHZ : NL80211_BAND_5GHZ; if (status->band != NL80211_BAND_2GHZ && status->encoding == RX_ENC_LEGACY && @@ -1661,19 +1654,20 @@ static void rtw89_core_update_rx_status(struct rtw89_dev *rtwdev, struct rtw89_rx_desc_info *desc_info, struct ieee80211_rx_status *rx_status) { - struct ieee80211_hw *hw = rtwdev->hw; - struct rtw89_hal *hal = &rtwdev->hal; + const struct cfg80211_chan_def *chandef = + rtw89_chandef_get(rtwdev, RTW89_SUB_ENTITY_0); + const struct rtw89_chan *cur = rtw89_chan_get(rtwdev, RTW89_SUB_ENTITY_0); u16 data_rate; u8 data_rate_mode; /* currently using single PHY */ - rx_status->freq = hw->conf.chandef.chan->center_freq; - rx_status->band = hw->conf.chandef.chan->band; + rx_status->freq = chandef->chan->center_freq; + rx_status->band = chandef->chan->band; if (rtwdev->scanning && RTW89_CHK_FW_FEATURE(SCAN_OFFLOAD, &rtwdev->fw)) { - u8 chan = hal->current_primary_channel; - u8 band = hal->current_band_type; + u8 chan = cur->primary_channel; + u8 band = cur->band_type; enum nl80211_band nl_band; nl_band = rtw89_hw_to_nl80211_band(band); @@ -1727,7 +1721,8 @@ static enum rtw89_ps_mode rtw89_update_ps_mode(struct rtw89_dev *rtwdev) { const struct rtw89_chip_info *chip = rtwdev->chip; - if (rtw89_disable_ps_mode || !chip->ps_mode_supported) + if (rtw89_disable_ps_mode || !chip->ps_mode_supported || + RTW89_CHK_FW_FEATURE(NO_DEEP_PS, &rtwdev->fw)) return RTW89_PS_MODE_NONE; if (chip->ps_mode_supported & BIT(RTW89_PS_MODE_PWR_GATED)) @@ -1810,7 +1805,7 @@ void rtw89_core_napi_init(struct rtw89_dev *rtwdev) { init_dummy_netdev(&rtwdev->netdev); netif_napi_add(&rtwdev->netdev, &rtwdev->napi, - rtwdev->hci.ops->napi_poll, NAPI_POLL_WEIGHT); + rtwdev->hci.ops->napi_poll); } EXPORT_SYMBOL(rtw89_core_napi_init); @@ -1907,21 +1902,14 @@ static void rtw89_core_stop_tx_ba_session(struct rtw89_dev *rtwdev, return; spin_lock_bh(&rtwdev->ba_lock); - if (!list_empty(&rtwtxq->list)) { - list_del_init(&rtwtxq->list); - goto out; - } - - set_bit(RTW89_TXQ_F_FORBID_BA, &rtwtxq->flags); + if (!test_and_set_bit(RTW89_TXQ_F_FORBID_BA, &rtwtxq->flags)) + list_add_tail(&rtwtxq->list, &rtwdev->forbid_ba_list); + spin_unlock_bh(&rtwdev->ba_lock); - list_add_tail(&rtwtxq->list, &rtwdev->forbid_ba_list); ieee80211_stop_tx_ba_session(sta, txq->tid); cancel_delayed_work(&rtwdev->forbid_ba_work); ieee80211_queue_delayed_work(rtwdev->hw, &rtwdev->forbid_ba_work, RTW89_FORBID_BA_TIMER); - -out: - spin_unlock_bh(&rtwdev->ba_lock); } static void rtw89_core_txq_check_agg(struct rtw89_dev *rtwdev, @@ -1933,6 +1921,9 @@ static void rtw89_core_txq_check_agg(struct rtw89_dev *rtwdev, struct ieee80211_sta *sta = txq->sta; struct rtw89_sta *rtwsta = sta ? (struct rtw89_sta *)sta->drv_priv : NULL; + if (test_bit(RTW89_TXQ_F_FORBID_BA, &rtwtxq->flags)) + return; + if (unlikely(skb->protocol == cpu_to_be16(ETH_P_PAE))) { rtw89_core_stop_tx_ba_session(rtwdev, rtwtxq); return; @@ -1941,9 +1932,6 @@ static void rtw89_core_txq_check_agg(struct rtw89_dev *rtwdev, if (unlikely(!sta)) return; - if (test_bit(RTW89_TXQ_F_FORBID_BA, &rtwtxq->flags)) - return; - if (unlikely(test_bit(RTW89_TXQ_F_BLOCK_BA, &rtwtxq->flags))) return; @@ -2179,12 +2167,13 @@ static bool rtw89_traffic_stats_track(struct rtw89_dev *rtwdev) static void rtw89_vif_enter_lps(struct rtw89_dev *rtwdev, struct rtw89_vif *rtwvif) { - if (rtwvif->wifi_role != RTW89_WIFI_ROLE_STATION) + if (rtwvif->wifi_role != RTW89_WIFI_ROLE_STATION && + rtwvif->wifi_role != RTW89_WIFI_ROLE_P2P_CLIENT) return; if (rtwvif->stats.tx_tfc_lv == RTW89_TFC_IDLE && rtwvif->stats.rx_tfc_lv == RTW89_TFC_IDLE) - rtw89_enter_lps(rtwdev, rtwvif->mac_id); + rtw89_enter_lps(rtwdev, rtwvif); } static void rtw89_enter_lps_track(struct rtw89_dev *rtwdev) @@ -2237,6 +2226,7 @@ static void rtw89_track_work(struct work_struct *work) rtw89_chip_rfk_track(rtwdev); rtw89_phy_ra_update(rtwdev); rtw89_phy_cfo_track(rtwdev); + rtw89_phy_tx_path_div_track(rtwdev); if (rtwdev->lps_enabled && !rtwdev->btc.lps) rtw89_enter_lps_track(rtwdev); @@ -2266,45 +2256,69 @@ void rtw89_core_release_all_bits_map(unsigned long *addr, unsigned int nbits) bitmap_zero(addr, nbits); } -int rtw89_core_acquire_sta_ba_entry(struct rtw89_sta *rtwsta, u8 tid, u8 *cam_idx) +int rtw89_core_acquire_sta_ba_entry(struct rtw89_dev *rtwdev, + struct rtw89_sta *rtwsta, u8 tid, u8 *cam_idx) { - struct rtw89_ba_cam_entry *entry; + const struct rtw89_chip_info *chip = rtwdev->chip; + struct rtw89_cam_info *cam_info = &rtwdev->cam_info; + struct rtw89_ba_cam_entry *entry = NULL, *tmp; u8 idx; + int i; - idx = rtw89_core_acquire_bit_map(rtwsta->ba_cam_map, RTW89_BA_CAM_NUM); - if (idx == RTW89_BA_CAM_NUM) { - /* allocate a static BA CAM to tid=0, so replace the existing + lockdep_assert_held(&rtwdev->mutex); + + idx = rtw89_core_acquire_bit_map(cam_info->ba_cam_map, chip->bacam_num); + if (idx == chip->bacam_num) { + /* allocate a static BA CAM to tid=0/5, so replace the existing * one if BA CAM is full. Hardware will process the original tid * automatically. */ - if (tid != 0) + if (tid != 0 && tid != 5) return -ENOSPC; - idx = 0; + for_each_set_bit(i, cam_info->ba_cam_map, chip->bacam_num) { + tmp = &cam_info->ba_cam_entry[i]; + if (tmp->tid == 0 || tmp->tid == 5) + continue; + + idx = i; + entry = tmp; + list_del(&entry->list); + break; + } + + if (!entry) + return -ENOSPC; + } else { + entry = &cam_info->ba_cam_entry[idx]; } - entry = &rtwsta->ba_cam_entry[idx]; entry->tid = tid; + list_add_tail(&entry->list, &rtwsta->ba_cam_list); + *cam_idx = idx; return 0; } -int rtw89_core_release_sta_ba_entry(struct rtw89_sta *rtwsta, u8 tid, u8 *cam_idx) +int rtw89_core_release_sta_ba_entry(struct rtw89_dev *rtwdev, + struct rtw89_sta *rtwsta, u8 tid, u8 *cam_idx) { - struct rtw89_ba_cam_entry *entry; - int i; + struct rtw89_cam_info *cam_info = &rtwdev->cam_info; + struct rtw89_ba_cam_entry *entry = NULL, *tmp; + u8 idx; - for (i = 0; i < RTW89_BA_CAM_NUM; i++) { - if (!test_bit(i, rtwsta->ba_cam_map)) - continue; + lockdep_assert_held(&rtwdev->mutex); - entry = &rtwsta->ba_cam_entry[i]; + list_for_each_entry_safe(entry, tmp, &rtwsta->ba_cam_list, list) { if (entry->tid != tid) continue; - rtw89_core_release_bit_map(rtwsta->ba_cam_map, i); - *cam_idx = i; + idx = entry - cam_info->ba_cam_entry; + list_del(&entry->list); + + rtw89_core_release_bit_map(cam_info->ba_cam_map, idx); + *cam_idx = idx; return 0; } @@ -2320,9 +2334,19 @@ void rtw89_vif_type_mapping(struct ieee80211_vif *vif, bool assoc) struct rtw89_vif *rtwvif = (struct rtw89_vif *)vif->drv_priv; switch (vif->type) { + case NL80211_IFTYPE_STATION: + if (vif->p2p) + rtwvif->wifi_role = RTW89_WIFI_ROLE_P2P_CLIENT; + else + rtwvif->wifi_role = RTW89_WIFI_ROLE_STATION; + break; + case NL80211_IFTYPE_AP: + if (vif->p2p) + rtwvif->wifi_role = RTW89_WIFI_ROLE_P2P_GO; + else + rtwvif->wifi_role = RTW89_WIFI_ROLE_AP; + break; RTW89_TYPE_MAPPING(ADHOC); - RTW89_TYPE_MAPPING(STATION); - RTW89_TYPE_MAPPING(AP); RTW89_TYPE_MAPPING(MONITOR); RTW89_TYPE_MAPPING(MESH_POINT); default: @@ -2365,13 +2389,17 @@ int rtw89_core_sta_add(struct rtw89_dev *rtwdev, struct rtw89_sta *rtwsta = (struct rtw89_sta *)sta->drv_priv; int i; + rtwsta->rtwdev = rtwdev; rtwsta->rtwvif = rtwvif; rtwsta->prev_rssi = 0; + INIT_LIST_HEAD(&rtwsta->ba_cam_list); for (i = 0; i < ARRAY_SIZE(sta->txq); i++) rtw89_core_txq_init(rtwdev, sta->txq[i]); ewma_rssi_init(&rtwsta->avg_rssi); + for (i = 0; i < rtwdev->chip->rf_path_num; i++) + ewma_rssi_init(&rtwsta->rssi[i]); if (vif->type == NL80211_IFTYPE_STATION && !sta->tdls) { /* for station mode, assign the mac_id from itself */ @@ -2541,6 +2569,60 @@ int rtw89_core_sta_remove(struct rtw89_dev *rtwdev, return 0; } +static void _rtw89_core_set_tid_config(struct rtw89_dev *rtwdev, + struct ieee80211_sta *sta, + struct cfg80211_tid_cfg *tid_conf) +{ + struct ieee80211_txq *txq; + struct rtw89_txq *rtwtxq; + u32 mask = tid_conf->mask; + u8 tids = tid_conf->tids; + int tids_nbit = BITS_PER_BYTE; + int i; + + for (i = 0; i < tids_nbit; i++, tids >>= 1) { + if (!tids) + break; + + if (!(tids & BIT(0))) + continue; + + txq = sta->txq[i]; + rtwtxq = (struct rtw89_txq *)txq->drv_priv; + + if (mask & BIT(NL80211_TID_CONFIG_ATTR_AMPDU_CTRL)) { + if (tid_conf->ampdu == NL80211_TID_CONFIG_ENABLE) { + clear_bit(RTW89_TXQ_F_FORBID_BA, &rtwtxq->flags); + } else { + if (test_bit(RTW89_TXQ_F_AMPDU, &rtwtxq->flags)) + ieee80211_stop_tx_ba_session(sta, txq->tid); + spin_lock_bh(&rtwdev->ba_lock); + list_del_init(&rtwtxq->list); + set_bit(RTW89_TXQ_F_FORBID_BA, &rtwtxq->flags); + spin_unlock_bh(&rtwdev->ba_lock); + } + } + + if (mask & BIT(NL80211_TID_CONFIG_ATTR_AMSDU_CTRL) && tids == 0xff) { + if (tid_conf->amsdu == NL80211_TID_CONFIG_ENABLE) + sta->max_amsdu_subframes = 0; + else + sta->max_amsdu_subframes = 1; + } + } +} + +void rtw89_core_set_tid_config(struct rtw89_dev *rtwdev, + struct ieee80211_sta *sta, + struct cfg80211_tid_config *tid_config) +{ + int i; + + for (i = 0; i < tid_config->n_tid_conf; i++) + _rtw89_core_set_tid_config(rtwdev, sta, + &tid_config->tid_conf[i]); +} + static void rtw89_init_ht_cap(struct rtw89_dev *rtwdev, struct ieee80211_sta_ht_cap *ht_cap) { @@ -2669,8 +2751,7 @@ static void rtw89_init_he_cap(struct rtw89_dev *rtwdev, phy_cap_info = he_cap->he_cap_elem.phy_cap_info; he_cap->has_he = true; - if (i == NL80211_IFTYPE_AP) - mac_cap_info[0] = IEEE80211_HE_MAC_CAP0_HTC_HE; + mac_cap_info[0] = IEEE80211_HE_MAC_CAP0_HTC_HE; if (i == NL80211_IFTYPE_STATION) mac_cap_info[1] = IEEE80211_HE_MAC_CAP1_TF_MAC_PAD_DUR_16US; mac_cap_info[2] = IEEE80211_HE_MAC_CAP2_ALL_ACK | @@ -2706,6 +2787,8 @@ static void rtw89_init_he_cap(struct rtw89_dev *rtwdev, phy_cap_info[3] |= IEEE80211_HE_PHY_CAP3_RX_PARTIAL_BW_SU_IN_20MHZ_MU; phy_cap_info[4] = IEEE80211_HE_PHY_CAP4_SU_BEAMFORMEE | IEEE80211_HE_PHY_CAP4_BEAMFORMEE_MAX_STS_UNDER_80MHZ_4; + if (chip->support_bw160) + phy_cap_info[4] |= IEEE80211_HE_PHY_CAP4_BEAMFORMEE_MAX_STS_ABOVE_80MHZ_4; phy_cap_info[5] = no_ng16 ? 0 : IEEE80211_HE_PHY_CAP5_NG16_SU_FEEDBACK | IEEE80211_HE_PHY_CAP5_NG16_MU_FEEDBACK; @@ -2866,7 +2949,9 @@ int rtw89_core_start(struct rtw89_dev *rtwdev) /* efuse process */ /* pre-config BB/RF, BB reset/RFC reset */ - rtw89_chip_disable_bb_rf(rtwdev); + ret = rtw89_chip_disable_bb_rf(rtwdev); + if (ret) + return ret; ret = rtw89_chip_enable_bb_rf(rtwdev); if (ret) return ret; @@ -2894,6 +2979,7 @@ int rtw89_core_start(struct rtw89_dev *rtwdev) rtw89_btc_ntfy_radio_state(rtwdev, BTC_RFCTRL_WL_ON); rtw89_fw_h2c_fw_log(rtwdev, rtwdev->fw.fw_log_enable); + rtw89_fw_h2c_init_ba_cam(rtwdev); return 0; } @@ -2987,6 +3073,7 @@ int rtw89_core_init(struct rtw89_dev *rtwdev) return ret; } rtw89_ser_init(rtwdev); + rtw89_entity_init(rtwdev); return 0; } @@ -3007,7 +3094,7 @@ EXPORT_SYMBOL(rtw89_core_deinit); void rtw89_core_scan_start(struct rtw89_dev *rtwdev, struct rtw89_vif *rtwvif, const u8 *mac_addr, bool hw_scan) { - struct rtw89_hal *hal = &rtwdev->hal; + const struct rtw89_chan *chan = rtw89_chan_get(rtwdev, RTW89_SUB_ENTITY_0); rtwdev->scanning = true; rtw89_leave_lps(rtwdev); @@ -3015,7 +3102,7 @@ void rtw89_core_scan_start(struct rtw89_dev *rtwdev, struct rtw89_vif *rtwvif, rtw89_leave_ips(rtwdev); ether_addr_copy(rtwvif->mac_addr, mac_addr); - rtw89_btc_ntfy_scan_start(rtwdev, RTW89_PHY_0, hal->current_band_type); + rtw89_btc_ntfy_scan_start(rtwdev, RTW89_PHY_0, chan->band_type); rtw89_chip_rfk_scan(rtwdev, true); rtw89_hci_recalc_int_mit(rtwdev); @@ -3141,6 +3228,7 @@ static int rtw89_core_register_hw(struct rtw89_dev *rtwdev) hw->vif_data_size = sizeof(struct rtw89_vif); hw->sta_data_size = sizeof(struct rtw89_sta); hw->txq_data_size = sizeof(struct rtw89_txq); + hw->chanctx_data_size = sizeof(struct rtw89_chanctx_cfg); SET_IEEE80211_PERM_ADDR(hw, efuse->addr); @@ -3148,6 +3236,7 @@ static int rtw89_core_register_hw(struct rtw89_dev *rtwdev) hw->queues = IEEE80211_NUM_ACS; hw->max_rx_aggregation_subframes = RTW89_MAX_RX_AGG_NUM; hw->max_tx_aggregation_subframes = RTW89_MAX_TX_AGG_NUM; + hw->uapsd_max_sp_len = IEEE80211_WMM_IE_STA_QOSINFO_SP_ALL; ieee80211_hw_set(hw, SIGNAL_DBM); ieee80211_hw_set(hw, HAS_RATE_CONTROL); @@ -3164,17 +3253,26 @@ static int rtw89_core_register_hw(struct rtw89_dev *rtwdev) ieee80211_hw_set(hw, SUPPORTS_MULTI_BSSID); hw->wiphy->interface_modes = BIT(NL80211_IFTYPE_STATION) | - BIT(NL80211_IFTYPE_AP); + BIT(NL80211_IFTYPE_AP) | + BIT(NL80211_IFTYPE_P2P_CLIENT) | + BIT(NL80211_IFTYPE_P2P_GO); + hw->wiphy->available_antennas_tx = BIT(rtwdev->chip->rf_path_num) - 1; hw->wiphy->available_antennas_rx = BIT(rtwdev->chip->rf_path_num) - 1; hw->wiphy->flags |= WIPHY_FLAG_SUPPORTS_TDLS | - WIPHY_FLAG_TDLS_EXTERNAL_SETUP; + WIPHY_FLAG_TDLS_EXTERNAL_SETUP | + WIPHY_FLAG_AP_UAPSD; hw->wiphy->features |= NL80211_FEATURE_SCAN_RANDOM_MAC_ADDR; hw->wiphy->max_scan_ssids = RTW89_SCANOFLD_MAX_SSID; hw->wiphy->max_scan_ie_len = RTW89_SCANOFLD_MAX_IE_LEN; + hw->wiphy->tid_config_support.vif |= BIT(NL80211_TID_CONFIG_ATTR_AMPDU_CTRL); + hw->wiphy->tid_config_support.peer |= BIT(NL80211_TID_CONFIG_ATTR_AMPDU_CTRL); + hw->wiphy->tid_config_support.vif |= BIT(NL80211_TID_CONFIG_ATTR_AMSDU_CTRL); + hw->wiphy->tid_config_support.peer |= BIT(NL80211_TID_CONFIG_ATTR_AMSDU_CTRL); + wiphy_ext_feature_set(hw->wiphy, NL80211_EXT_FEATURE_CAN_REPLACE_PTK0); ret = rtw89_core_set_supported_band(rtwdev); @@ -3234,6 +3332,63 @@ void rtw89_core_unregister(struct rtw89_dev *rtwdev) } EXPORT_SYMBOL(rtw89_core_unregister); +struct rtw89_dev *rtw89_alloc_ieee80211_hw(struct device *device, + u32 bus_data_size, + const struct rtw89_chip_info *chip) +{ + struct ieee80211_hw *hw; + struct rtw89_dev *rtwdev; + struct ieee80211_ops *ops; + u32 driver_data_size; + u32 early_feat_map = 0; + bool no_chanctx; + + rtw89_early_fw_feature_recognize(device, chip, &early_feat_map); + + ops = kmemdup(&rtw89_ops, sizeof(rtw89_ops), GFP_KERNEL); + if (!ops) + goto err; + + no_chanctx = chip->support_chanctx_num == 0 || + !(early_feat_map & BIT(RTW89_FW_FEATURE_SCAN_OFFLOAD)); + + if (no_chanctx) { + ops->add_chanctx = NULL; + ops->remove_chanctx = NULL; + ops->change_chanctx = NULL; + ops->assign_vif_chanctx = NULL; + ops->unassign_vif_chanctx = NULL; + } + + driver_data_size = sizeof(struct rtw89_dev) + bus_data_size; + hw = ieee80211_alloc_hw(driver_data_size, ops); + if (!hw) + goto err; + + rtwdev = hw->priv; + rtwdev->hw = hw; + rtwdev->dev = device; + rtwdev->ops = ops; + rtwdev->chip = chip; + + rtw89_debug(rtwdev, RTW89_DBG_FW, "probe driver %s chanctx\n", + no_chanctx ? "without" : "with"); + + return rtwdev; + +err: + kfree(ops); + return NULL; +} +EXPORT_SYMBOL(rtw89_alloc_ieee80211_hw); + +void rtw89_free_ieee80211_hw(struct rtw89_dev *rtwdev) +{ + kfree(rtwdev->ops); + ieee80211_free_hw(rtwdev->hw); +} +EXPORT_SYMBOL(rtw89_free_ieee80211_hw); + MODULE_AUTHOR("Realtek Corporation"); MODULE_DESCRIPTION("Realtek 802.11ax wireless core module"); MODULE_LICENSE("Dual BSD/GPL"); diff --git a/drivers/net/wireless/realtek/rtw89/core.h b/drivers/net/wireless/realtek/rtw89/core.h index 7a9d6f5d8a51..db041b32a8c2 100644 --- a/drivers/net/wireless/realtek/rtw89/core.h +++ b/drivers/net/wireless/realtek/rtw89/core.h @@ -34,6 +34,7 @@ extern const struct ieee80211_ops rtw89_ops; #define MAX_RSSI 110 #define RSSI_FACTOR 1 #define RTW89_RSSI_RAW_TO_DBM(rssi) ((s8)((rssi) >> RSSI_FACTOR) - MAX_RSSI) +#define RTW89_TX_DIV_RSSI_RAW_TH (2 << RSSI_FACTOR) #define RTW89_HTC_MASK_VARIANT GENMASK(1, 0) #define RTW89_HTC_VARIANT_HE 3 @@ -522,7 +523,7 @@ struct rtw89_rx_phy_ppdu { u8 *buf; u32 len; u8 rssi_avg; - s8 rssi[RF_PATH_MAX]; + u8 rssi[RF_PATH_MAX]; u8 mac_id; u8 chan_idx; u8 ie; @@ -542,6 +543,12 @@ enum rtw89_phy_idx { RTW89_PHY_MAX }; +enum rtw89_sub_entity_idx { + RTW89_SUB_ENTITY_0 = 0, + + NUM_OF_RTW89_SUB_ENTITY, +}; + enum rtw89_rf_path { RF_PATH_A = 0, RF_PATH_B = 1, @@ -624,14 +631,23 @@ enum rtw89_sc_offset { RTW89_SC_40_LOWER = 10, }; -struct rtw89_channel_params { - u8 center_chan; - u32 center_freq; - u8 primary_chan; - u8 bandwidth; - u8 pri_ch_idx; - u8 band_type; - u8 subband_type; +struct rtw89_chan { + u8 channel; + u8 primary_channel; + enum rtw89_band band_type; + enum rtw89_bandwidth band_width; + + /* The follow-up are derived from the above. We must ensure that it + * is assigned correctly in rtw89_chan_create() if new one is added. + */ + u32 freq; + enum rtw89_subband subband_type; + enum rtw89_sc_offset pri_ch_idx; +}; + +struct rtw89_chan_rcd { + u8 prev_primary_channel; + enum rtw89_band prev_band_type; }; struct rtw89_channel_help_params { @@ -793,7 +809,7 @@ struct rtw89_mac_ax_gnt { u8 gnt_bt; u8 gnt_wl_sw_en; u8 gnt_wl; -}; +} __packed; #define RTW89_MAC_AX_COEX_GNT_NR 2 struct rtw89_mac_ax_coex_gnt { @@ -848,6 +864,7 @@ enum rtw89_btc_dcnt { BTC_DCNT_SLOT_NONSYNC, BTC_DCNT_BTCNT_FREEZE, BTC_DCNT_WL_SLOT_DRIFT, + BTC_DCNT_BT_SLOT_DRIFT, BTC_DCNT_WL_STA_LAST, BTC_DCNT_NUM, }; @@ -920,12 +937,12 @@ struct rtw89_btc_wl_smap { u32 roaming: 1; u32 _4way: 1; u32 rf_off: 1; - u32 lps: 1; + u32 lps: 2; u32 ips: 1; u32 init_ok: 1; u32 traffic_dir : 2; u32 rf_off_pre: 1; - u32 lps_pre: 1; + u32 lps_pre: 2; }; enum rtw89_tfc_lv { @@ -1108,6 +1125,27 @@ struct rtw89_btc_wl_active_role { u16 rx_rate; }; +struct rtw89_btc_wl_active_role_v1 { + u8 connected: 1; + u8 pid: 3; + u8 phy: 1; + u8 noa: 1; + u8 band: 2; + + u8 client_ps: 1; + u8 bw: 7; + + u8 role; + u8 ch; + + u16 tx_lvl; + u16 rx_lvl; + u16 tx_rate; + u16 rx_rate; + + u32 noa_duration; /* ms */ +}; + struct rtw89_btc_wl_role_info_bpos { u16 none: 1; u16 station: 1; @@ -1123,6 +1161,12 @@ struct rtw89_btc_wl_role_info_bpos { u16 nan: 1; }; +struct rtw89_btc_wl_scc_ctrl { + u8 null_role1; + u8 null_role2; + u8 ebt_null; /* if tx null at EBT slot */ +}; + union rtw89_btc_wl_role_info_map { u16 val; struct rtw89_btc_wl_role_info_bpos role; @@ -1135,6 +1179,21 @@ struct rtw89_btc_wl_role_info { /* struct size must be n*4 bytes */ struct rtw89_btc_wl_active_role active_role[RTW89_PORT_NUM]; }; +struct rtw89_btc_wl_role_info_v1 { /* struct size must be n*4 bytes */ + u8 connect_cnt; + u8 link_mode; + union rtw89_btc_wl_role_info_map role_map; + struct rtw89_btc_wl_active_role_v1 active_role_v1[RTW89_PORT_NUM]; + u32 mrole_type; /* btc_wl_mrole_type */ + u32 mrole_noa_duration; /* ms */ + + u32 dbcc_en: 1; + u32 dbcc_chg: 1; + u32 dbcc_2g_phy: 2; /* which phy operate in 2G, HW_PHY_0 or HW_PHY_1 */ + u32 link_mode_chg: 1; + u32 rsvd: 27; +}; + struct rtw89_btc_wl_ver_info { u32 fw_coex; /* match with which coex_ver */ u32 fw; @@ -1240,6 +1299,7 @@ struct rtw89_btc_wl_info { struct rtw89_btc_wl_ver_info ver_info; struct rtw89_btc_wl_afh_info afh_info; struct rtw89_btc_wl_role_info role_info; + struct rtw89_btc_wl_role_info_v1 role_info_v1; struct rtw89_btc_wl_scan_info scan_info; struct rtw89_btc_wl_dbcc_info dbcc_info; struct rtw89_btc_rf_para rf_para; @@ -1248,6 +1308,7 @@ struct rtw89_btc_wl_info { u8 port_id[RTW89_WIFI_ROLE_MLME_MAX]; u8 rssi_level; + bool scbd_change; u32 scbd; }; @@ -1333,7 +1394,8 @@ struct rtw89_btc_bt_info { u32 pag: 1; u32 run_patch_code: 1; u32 hi_lna_rx: 1; - u32 rsvd: 22; + u32 scan_rx_low_pri: 1; + u32 rsvd: 21; }; struct rtw89_btc_cx { @@ -1346,32 +1408,43 @@ struct rtw89_btc_cx { }; struct rtw89_btc_fbtc_tdma { - u8 type; + u8 type; /* chip_info::fcxtdma_ver */ u8 rxflctrl; u8 txpause; u8 wtgle_n; u8 leak_n; u8 ext_ctrl; - u8 rsvd0; - u8 rsvd1; + u8 rxflctrl_role; + u8 option_ctrl; +} __packed; + +struct rtw89_btc_fbtc_tdma_v1 { + u8 fver; /* chip_info::fcxtdma_ver */ + u8 rsvd; + __le16 rsvd1; + struct rtw89_btc_fbtc_tdma tdma; } __packed; #define CXMREG_MAX 30 #define FCXMAX_STEP 255 /*STEP trace record cnt, Max:65535, default:255*/ -#define BTCRPT_VER 1 #define BTC_CYCLE_SLOT_MAX 48 /* must be even number, non-zero */ -enum rtw89_btc_bt_rfk_counter { +enum rtw89_btc_bt_sta_counter { BTC_BCNT_RFK_REQ = 0, BTC_BCNT_RFK_GO = 1, BTC_BCNT_RFK_REJECT = 2, BTC_BCNT_RFK_FAIL = 3, BTC_BCNT_RFK_TIMEOUT = 4, - BTC_BCNT_RFK_MAX + BTC_BCNT_HI_TX = 5, + BTC_BCNT_HI_RX = 6, + BTC_BCNT_LO_TX = 7, + BTC_BCNT_LO_RX = 8, + BTC_BCNT_POLLUTED = 9, + BTC_BCNT_STA_MAX }; struct rtw89_btc_fbtc_rpt_ctrl { - u16 fver; + u16 fver; /* chip_info::fcxbtcrpt_ver */ u16 rpt_cnt; /* tmr counters */ u32 wl_fw_coex_ver; /* match which driver's coex version */ u32 wl_fw_cx_offload; @@ -1384,11 +1457,56 @@ struct rtw89_btc_fbtc_rpt_ctrl { u32 mb_a2dp_empty_cnt; /* a2dp empty count */ u32 mb_a2dp_flct_cnt; /* a2dp empty flow control counter */ u32 mb_a2dp_full_cnt; /* a2dp empty full counter */ - u32 bt_rfk_cnt[BTC_BCNT_RFK_MAX]; + u32 bt_rfk_cnt[BTC_BCNT_HI_TX]; u32 c2h_cnt; /* fw send c2h counter */ u32 h2c_cnt; /* fw recv h2c counter */ } __packed; +struct rtw89_btc_fbtc_rpt_ctrl_info { + __le32 cnt; /* fw report counter */ + __le32 en; /* report map */ + __le32 para; /* not used */ + + __le32 cnt_c2h; /* fw send c2h counter */ + __le32 cnt_h2c; /* fw recv h2c counter */ + __le32 len_c2h; /* The total length of the last C2H */ + + __le32 cnt_aoac_rf_on; /* rf-on counter for aoac switch notify */ + __le32 cnt_aoac_rf_off; /* rf-off counter for aoac switch notify */ +} __packed; + +struct rtw89_btc_fbtc_rpt_ctrl_wl_fw_info { + __le32 cx_ver; /* match which driver's coex version */ + __le32 cx_offload; + __le32 fw_ver; +} __packed; + +struct rtw89_btc_fbtc_rpt_ctrl_a2dp_empty { + __le32 cnt_empty; /* a2dp empty count */ + __le32 cnt_flowctrl; /* a2dp empty flow control counter */ + __le32 cnt_tx; + __le32 cnt_ack; + __le32 cnt_nack; +} __packed; + +struct rtw89_btc_fbtc_rpt_ctrl_bt_mailbox { + __le32 cnt_send_ok; /* fw send mailbox ok counter */ + __le32 cnt_send_fail; /* fw send mailbox fail counter */ + __le32 cnt_recv; /* fw recv mailbox counter */ + struct rtw89_btc_fbtc_rpt_ctrl_a2dp_empty a2dp; +} __packed; + +struct rtw89_btc_fbtc_rpt_ctrl_v1 { + u8 fver; + u8 rsvd; + __le16 rsvd1; + struct rtw89_btc_fbtc_rpt_ctrl_info rpt_info; + struct rtw89_btc_fbtc_rpt_ctrl_wl_fw_info wl_fw_info; + struct rtw89_btc_fbtc_rpt_ctrl_bt_mailbox bt_mbx_info; + __le32 bt_cnt[BTC_BCNT_STA_MAX]; + struct rtw89_mac_ax_gnt gnt_val[RTW89_PHY_MAX]; +} __packed; + enum rtw89_fbtc_ext_ctrl_type { CXECTL_OFF = 0x0, /* tdma off */ CXECTL_B2 = 0x1, /* allow B2 (beacon-early) */ @@ -1457,10 +1575,9 @@ enum { /* STEP TYPE */ CXSTEP_MAX, }; -#define FCXGPIODBG_VER 1 #define BTC_DBG_MAX1 32 struct rtw89_btc_fbtc_gpio_dbg { - u8 fver; + u8 fver; /* chip_info::fcxgpiodbg_ver */ u8 rsvd; u16 rsvd2; u32 en_map; /* which debug signal (see btc_wl_gpio_debug) is enable */ @@ -1468,9 +1585,8 @@ struct rtw89_btc_fbtc_gpio_dbg { u8 gpio_map[BTC_DBG_MAX1]; /*the debug signals to GPIO-Position */ } __packed; -#define FCXMREG_VER 1 struct rtw89_btc_fbtc_mreg_val { - u8 fver; + u8 fver; /* chip_info::fcxmreg_ver */ u8 reg_num; __le16 rsvd; __le32 mreg_val[CXMREG_MAX]; @@ -1492,16 +1608,14 @@ struct rtw89_btc_fbtc_slot { __le16 cxtype; } __packed; -#define FCXSLOTS_VER 1 struct rtw89_btc_fbtc_slots { - u8 fver; + u8 fver; /* chip_info::fcxslots_ver */ u8 tbl_num; __le16 rsvd; __le32 update_map; struct rtw89_btc_fbtc_slot slot[CXST_MAX]; } __packed; -#define FCXSTEP_VER 2 struct rtw89_btc_fbtc_step { u8 type; u8 val; @@ -1509,7 +1623,7 @@ struct rtw89_btc_fbtc_step { } __packed; struct rtw89_btc_fbtc_steps { - u8 fver; + u8 fver; /* chip_info::fcxstep_ver */ u8 rsvd; __le16 cnt; __le16 pos_old; @@ -1517,9 +1631,16 @@ struct rtw89_btc_fbtc_steps { struct rtw89_btc_fbtc_step step[FCXMAX_STEP]; } __packed; -#define FCXCYSTA_VER 2 -struct rtw89_btc_fbtc_cysta { /* statistics for cycles */ +struct rtw89_btc_fbtc_steps_v1 { u8 fver; + u8 en; + __le16 rsvd; + __le32 cnt; + struct rtw89_btc_fbtc_step step[FCXMAX_STEP]; +} __packed; + +struct rtw89_btc_fbtc_cysta { /* statistics for cycles */ + u8 fver; /* chip_info::fcxcysta_ver */ u8 rsvd; __le16 cycles; /* total cycle number */ __le16 cycles_a2dp[CXT_FLCTRL_MAX]; @@ -1544,19 +1665,80 @@ struct rtw89_btc_fbtc_cysta { /* statistics for cycles */ __le16 tslot_cycle[BTC_CYCLE_SLOT_MAX]; } __packed; -#define FCXNULLSTA_VER 1 -struct rtw89_btc_fbtc_cynullsta { /* cycle null statistics */ +struct rtw89_btc_fbtc_fdd_try_info { + __le16 cycles[CXT_FLCTRL_MAX]; + __le16 tavg[CXT_FLCTRL_MAX]; /* avg try BT-Slot-TDD/BT-slot-FDD time */ + __le16 tmax[CXT_FLCTRL_MAX]; /* max try BT-Slot-TDD/BT-slot-FDD time */ +} __packed; + +struct rtw89_btc_fbtc_cycle_time_info { + __le16 tavg[CXT_MAX]; /* avg wl/bt cycle time */ + __le16 tmax[CXT_MAX]; /* max wl/bt cycle time */ + __le16 tmaxdiff[CXT_MAX]; /* max wl-wl bt-bt cycle diff time */ +} __packed; + +struct rtw89_btc_fbtc_a2dp_trx_stat { + u8 empty_cnt; + u8 retry_cnt; + u8 tx_rate; + u8 tx_cnt; + u8 ack_cnt; + u8 nack_cnt; + u8 rsvd1; + u8 rsvd2; +} __packed; + +struct rtw89_btc_fbtc_cycle_a2dp_empty_info { + __le16 cnt; /* a2dp empty cnt */ + __le16 cnt_timeout; /* a2dp empty timeout cnt*/ + __le16 tavg; /* avg a2dp empty time */ + __le16 tmax; /* max a2dp empty time */ +} __packed; + +struct rtw89_btc_fbtc_cycle_leak_info { + __le32 cnt_rximr; /* the rximr occur at leak slot */ + __le16 tavg; /* avg leak-slot time */ + __le16 tmax; /* max leak-slot time */ +} __packed; + +struct rtw89_btc_fbtc_cysta_v1 { /* statistics for cycles */ u8 fver; u8 rsvd; + __le16 cycles; /* total cycle number */ + __le16 slot_step_time[BTC_CYCLE_SLOT_MAX]; + struct rtw89_btc_fbtc_cycle_time_info cycle_time; + struct rtw89_btc_fbtc_fdd_try_info fdd_try; + struct rtw89_btc_fbtc_cycle_a2dp_empty_info a2dp_ept; + struct rtw89_btc_fbtc_a2dp_trx_stat a2dp_trx[BTC_CYCLE_SLOT_MAX]; + struct rtw89_btc_fbtc_cycle_leak_info leak_slot; + __le32 slot_cnt[CXST_MAX]; /* slot count */ + __le32 bcn_cnt[CXBCN_MAX]; + __le32 collision_cnt; /* counter for event/timer occur at the same time */ + __le32 skip_cnt; + __le32 except_cnt; + __le32 except_map; +} __packed; + +struct rtw89_btc_fbtc_cynullsta { /* cycle null statistics */ + u8 fver; /* chip_info::fcxnullsta_ver */ + u8 rsvd; __le16 rsvd2; __le32 max_t[2]; /* max_t for 0:null0/1:null1 */ __le32 avg_t[2]; /* avg_t for 0:null0/1:null1 */ __le32 result[2][4]; /* 0:fail, 1:ok, 2:on_time, 3:retry */ } __packed; -#define FCX_BTVER_VER 1 +struct rtw89_btc_fbtc_cynullsta_v1 { /* cycle null statistics */ + u8 fver; /* chip_info::fcxnullsta_ver */ + u8 rsvd; + __le16 rsvd2; + __le32 max_t[2]; /* max_t for 0:null0/1:null1 */ + __le32 avg_t[2]; /* avg_t for 0:null0/1:null1 */ + __le32 result[2][5]; /* 0:fail, 1:ok, 2:on_time, 3:retry, 4:tx */ +} __packed; + struct rtw89_btc_fbtc_btver { - u8 fver; + u8 fver; /* chip_info::fcxbtver_ver */ u8 rsvd; __le16 rsvd2; __le32 coex_ver; /*bit[15:8]->shared, bit[7:0]->non-shared */ @@ -1564,17 +1746,15 @@ struct rtw89_btc_fbtc_btver { __le32 feature; } __packed; -#define FCX_BTSCAN_VER 1 struct rtw89_btc_fbtc_btscan { - u8 fver; + u8 fver; /* chip_info::fcxbtscan_ver */ u8 rsvd; __le16 rsvd2; u8 scan[6]; } __packed; -#define FCX_BTAFH_VER 1 struct rtw89_btc_fbtc_btafh { - u8 fver; + u8 fver; /* chip_info::fcxbtafh_ver */ u8 rsvd; __le16 rsvd2; u8 afh_l[4]; /*bit0:2402, bit1: 2403.... bit31:2433 */ @@ -1582,9 +1762,8 @@ struct rtw89_btc_fbtc_btafh { u8 afh_h[4]; /*bit0:2466, bit1:2467......bit14:2480 */ } __packed; -#define FCX_BTDEVINFO_VER 1 struct rtw89_btc_fbtc_btdevinfo { - u8 fver; + u8 fver; /* chip_info::fcxbtdevinfo_ver */ u8 rsvd; __le16 vendor_id; __le32 dev_name; /* only 24 bits valid */ @@ -1609,6 +1788,7 @@ struct rtw89_btc_dm { struct rtw89_btc_rf_trx_para rf_trx_para; struct rtw89_btc_wl_tx_limit_para wl_tx_limit; struct rtw89_btc_dm_step dm_step; + struct rtw89_btc_wl_scc_ctrl wl_scc; union rtw89_btc_dm_error_map error; u32 cnt_dm[BTC_DCNT_NUM]; u32 cnt_notify[BTC_NCNT_NUM]; @@ -1628,7 +1808,9 @@ struct rtw89_btc_dm { u32 wl_btg_rx: 1; u32 trx_para_level: 8; u32 wl_stb_chg: 1; - u32 rsvd: 3; + u32 pta_owner: 1; + u32 tdma_instant_excute: 1; + u32 rsvd: 1; u16 slot_dur[CXST_MAX]; @@ -1650,8 +1832,6 @@ struct rtw89_btc_dbg { u32 rb_val; }; -#define FCXTDMA_VER 1 - enum rtw89_btc_btf_fw_event { BTF_EVNT_RPT = 0, BTF_EVNT_BT_INFO = 1, @@ -1704,12 +1884,18 @@ struct rtw89_btc_rpt_cmn_info { struct rtw89_btc_report_ctrl_state { struct rtw89_btc_rpt_cmn_info cinfo; /* common info, by driver */ - struct rtw89_btc_fbtc_rpt_ctrl finfo; /* info from fw */ + union { + struct rtw89_btc_fbtc_rpt_ctrl finfo; /* info from fw for 52A*/ + struct rtw89_btc_fbtc_rpt_ctrl_v1 finfo_v1; /* info from fw for 52C*/ + }; }; struct rtw89_btc_rpt_fbtc_tdma { struct rtw89_btc_rpt_cmn_info cinfo; /* common info, by driver */ - struct rtw89_btc_fbtc_tdma finfo; /* info from fw */ + union { + struct rtw89_btc_fbtc_tdma finfo; /* info from fw */ + struct rtw89_btc_fbtc_tdma_v1 finfo_v1; /* info from fw for 52C*/ + }; }; struct rtw89_btc_rpt_fbtc_slots { @@ -1719,17 +1905,26 @@ struct rtw89_btc_rpt_fbtc_slots { struct rtw89_btc_rpt_fbtc_cysta { struct rtw89_btc_rpt_cmn_info cinfo; /* common info, by driver */ - struct rtw89_btc_fbtc_cysta finfo; /* info from fw */ + union { + struct rtw89_btc_fbtc_cysta finfo; /* info from fw for 52A*/ + struct rtw89_btc_fbtc_cysta_v1 finfo_v1; /* info from fw for 52C*/ + }; }; struct rtw89_btc_rpt_fbtc_step { struct rtw89_btc_rpt_cmn_info cinfo; /* common info, by driver */ - struct rtw89_btc_fbtc_steps finfo; /* info from fw */ + union { + struct rtw89_btc_fbtc_steps finfo; /* info from fw */ + struct rtw89_btc_fbtc_steps_v1 finfo_v1; /* info from fw */ + }; }; struct rtw89_btc_rpt_fbtc_nullsta { struct rtw89_btc_rpt_cmn_info cinfo; /* common info, by driver */ - struct rtw89_btc_fbtc_cynullsta finfo; /* info from fw */ + union { + struct rtw89_btc_fbtc_cynullsta finfo; /* info from fw */ + struct rtw89_btc_fbtc_cynullsta_v1 finfo_v1; /* info from fw */ + }; }; struct rtw89_btc_rpt_fbtc_mreg { @@ -1887,7 +2082,9 @@ struct rtw89_ra_info { u8 ra_csi_rate_en:1; u8 fixed_csi_rate_en:1; u8 cr_tbl_sel:1; - u8 rsvd2:5; + u8 fix_giltf_en:1; + u8 fix_giltf:3; + u8 rsvd2:1; u8 csi_mcs_ss_idx; u8 csi_mode:2; u8 csi_gi_ltf:3; @@ -1911,19 +2108,20 @@ struct rtw89_ra_report { struct rate_info txrate; u32 bit_rate; u16 hw_rate; + bool might_fallback_legacy; }; DECLARE_EWMA(rssi, 10, 16); -#define RTW89_BA_CAM_NUM 2 - struct rtw89_ba_cam_entry { + struct list_head list; u8 tid; }; #define RTW89_MAX_ADDR_CAM_NUM 128 #define RTW89_MAX_BSSID_CAM_NUM 20 #define RTW89_MAX_SEC_CAM_NUM 128 +#define RTW89_MAX_BA_CAM_NUM 8 #define RTW89_SEC_CAM_IN_ADDR_CAM 7 struct rtw89_addr_cam_entry { @@ -1967,18 +2165,21 @@ struct rtw89_sec_cam_entry { struct rtw89_sta { u8 mac_id; bool disassoc; + struct rtw89_dev *rtwdev; struct rtw89_vif *rtwvif; struct rtw89_ra_info ra; struct rtw89_ra_report ra_report; int max_agg_wait; u8 prev_rssi; struct ewma_rssi avg_rssi; + struct ewma_rssi rssi[RF_PATH_MAX]; struct rtw89_ampdu_params ampdu_params[IEEE80211_NUM_TIDS]; struct ieee80211_rx_status rx_status; u16 rx_hw_rate; __le32 htc_template; struct rtw89_addr_cam_entry addr_cam; /* AP mode or TDLS peer only */ struct rtw89_bssid_cam_entry bssid_cam; /* TDLS peer only */ + struct list_head ba_cam_list; bool use_cfg_mask; struct cfg80211_bitrate_mask mask; @@ -1987,9 +2188,6 @@ struct rtw89_sta { u32 ampdu_max_time:4; bool cctl_tx_retry_limit; u32 data_tx_cnt_lmt:6; - - DECLARE_BITMAP(ba_cam_map, RTW89_BA_CAM_NUM); - struct rtw89_ba_cam_entry ba_cam_entry[RTW89_BA_CAM_NUM]; }; struct rtw89_efuse { @@ -2007,6 +2205,8 @@ struct rtw89_phy_rate_pattern { bool enable; }; +#define RTW89_P2P_MAX_NOA_NUM 2 + struct rtw89_vif { struct list_head list; struct rtw89_dev *rtwdev; @@ -2022,6 +2222,7 @@ struct rtw89_vif { u8 wmm; u8 bcn_hit_cond; u8 hit_rule; + u8 last_noa_nr; bool trigger; bool lsig_txop; u8 tgt_ind; @@ -2091,7 +2292,7 @@ struct rtw89_hci_info { struct rtw89_chip_ops { int (*enable_bb_rf)(struct rtw89_dev *rtwdev); - void (*disable_bb_rf)(struct rtw89_dev *rtwdev); + int (*disable_bb_rf)(struct rtw89_dev *rtwdev); void (*bb_reset)(struct rtw89_dev *rtwdev, enum rtw89_phy_idx phy_idx); void (*bb_sethw)(struct rtw89_dev *rtwdev); @@ -2100,20 +2301,29 @@ struct rtw89_chip_ops { bool (*write_rf)(struct rtw89_dev *rtwdev, enum rtw89_rf_path rf_path, u32 addr, u32 mask, u32 data); void (*set_channel)(struct rtw89_dev *rtwdev, - struct rtw89_channel_params *param); + const struct rtw89_chan *chan, + enum rtw89_mac_idx mac_idx, + enum rtw89_phy_idx phy_idx); void (*set_channel_help)(struct rtw89_dev *rtwdev, bool enter, - struct rtw89_channel_help_params *p); + struct rtw89_channel_help_params *p, + const struct rtw89_chan *chan, + enum rtw89_mac_idx mac_idx, + enum rtw89_phy_idx phy_idx); int (*read_efuse)(struct rtw89_dev *rtwdev, u8 *log_map); int (*read_phycap)(struct rtw89_dev *rtwdev, u8 *phycap_map); void (*fem_setup)(struct rtw89_dev *rtwdev); void (*rfk_init)(struct rtw89_dev *rtwdev); void (*rfk_channel)(struct rtw89_dev *rtwdev); - void (*rfk_band_changed)(struct rtw89_dev *rtwdev); + void (*rfk_band_changed)(struct rtw89_dev *rtwdev, + enum rtw89_phy_idx phy_idx); void (*rfk_scan)(struct rtw89_dev *rtwdev, bool start); void (*rfk_track)(struct rtw89_dev *rtwdev); void (*power_trim)(struct rtw89_dev *rtwdev); - void (*set_txpwr)(struct rtw89_dev *rtwdev); - void (*set_txpwr_ctrl)(struct rtw89_dev *rtwdev); + void (*set_txpwr)(struct rtw89_dev *rtwdev, + const struct rtw89_chan *chan, + enum rtw89_phy_idx phy_idx); + void (*set_txpwr_ctrl)(struct rtw89_dev *rtwdev, + enum rtw89_phy_idx phy_idx); int (*init_txpwr_unit)(struct rtw89_dev *rtwdev, enum rtw89_phy_idx phy_idx); u8 (*get_thermal)(struct rtw89_dev *rtwdev, enum rtw89_rf_path rf_path); void (*ctrl_btg)(struct rtw89_dev *rtwdev, bool btg); @@ -2150,6 +2360,8 @@ struct rtw89_chip_ops { void (*btc_bt_aci_imp)(struct rtw89_dev *rtwdev); void (*btc_update_bt_cnt)(struct rtw89_dev *rtwdev); void (*btc_wl_s1_standby)(struct rtw89_dev *rtwdev, bool state); + void (*btc_set_policy)(struct rtw89_dev *rtwdev, u16 policy_type); + void (*btc_set_wl_rx_gain)(struct rtw89_dev *rtwdev, u32 level); }; enum rtw89_dma_ch { @@ -2351,6 +2563,7 @@ struct rtw89_imr_info { u32 cpu_disp_imr_set; u32 other_disp_imr_clr; u32 other_disp_imr_set; + u32 bbrpt_com_err_imr_reg; u32 bbrpt_chinfo_err_imr_reg; u32 bbrpt_err_imr_set; u32 bbrpt_dfs_err_imr_reg; @@ -2373,17 +2586,40 @@ struct rtw89_imr_info { u32 tmac_imr_set; }; +struct rtw89_rrsr_cfgs { + struct rtw89_reg3_def ref_rate; + struct rtw89_reg3_def rsc; +}; + +struct rtw89_dig_regs { + u32 seg0_pd_reg; + u32 pd_lower_bound_mask; + u32 pd_spatial_reuse_en; + struct rtw89_reg_def p0_lna_init; + struct rtw89_reg_def p1_lna_init; + struct rtw89_reg_def p0_tia_init; + struct rtw89_reg_def p1_tia_init; + struct rtw89_reg_def p0_rxb_init; + struct rtw89_reg_def p1_rxb_init; + struct rtw89_reg_def p0_p20_pagcugc_en; + struct rtw89_reg_def p0_s20_pagcugc_en; + struct rtw89_reg_def p1_p20_pagcugc_en; + struct rtw89_reg_def p1_s20_pagcugc_en; +}; + struct rtw89_chip_info { enum rtw89_core_chip_id chip_id; const struct rtw89_chip_ops *ops; const char *fw_name; u32 fifo_size; + u32 dle_scc_rsvd_size; u16 max_amsdu_limit; bool dis_2g_40m_ul_ofdma; u32 rsvd_ple_ofst; const struct rtw89_hfc_param_ini *hfc_param_ini; const struct rtw89_dle_mem *dle_mem; u32 rf_base_addr[2]; + u8 support_chanctx_num; u8 support_bands; bool support_bw160; bool hw_sec_hdr; @@ -2393,6 +2629,9 @@ struct rtw89_chip_info { u8 acam_num; u8 bcam_num; u8 scam_num; + u8 bacam_num; + u8 bacam_dynamic_num; + bool bacam_v1; u8 sec_ctrl_efuse_size; u32 physical_efuse_size; @@ -2411,6 +2650,7 @@ struct rtw89_chip_info { const struct rtw89_phy_table *nctl_table; const struct rtw89_txpwr_table *byr_table; const struct rtw89_phy_dig_gain_table *dig_table; + const struct rtw89_dig_regs *dig_regs; const struct rtw89_phy_tssi_dbw_table *tssi_dbw_table; const s8 (*txpwr_lmt_2g)[RTW89_2G_BW_NUM][RTW89_NTX_NUM] [RTW89_RS_LMT_NUM][RTW89_BF_NUM] @@ -2436,6 +2676,20 @@ struct rtw89_chip_info { u8 btcx_desired; u8 scbd; u8 mailbox; + u16 btc_fwinfo_buf; + + u8 fcxbtcrpt_ver; + u8 fcxtdma_ver; + u8 fcxslots_ver; + u8 fcxcysta_ver; + u8 fcxstep_ver; + u8 fcxnullsta_ver; + u8 fcxmreg_ver; + u8 fcxgpiodbg_ver; + u8 fcxbtver_ver; + u8 fcxbtscan_ver; + u8 fcxbtafh_ver; + u8 fcxbtdevinfo_ver; u8 afh_guard_ch; const u8 *wl_rssi_thres; @@ -2463,6 +2717,8 @@ struct rtw89_chip_info { const struct rtw89_reg_def *dcfo_comp; u8 dcfo_comp_sft; const struct rtw89_imr_info *imr_info; + const struct rtw89_rrsr_cfgs *rrsr_cfgs; + u32 dma_ch_mask; }; union rtw89_bus_info { @@ -2514,6 +2770,8 @@ enum rtw89_fw_feature { RTW89_FW_FEATURE_SCAN_OFFLOAD, RTW89_FW_FEATURE_TX_WAKE, RTW89_FW_FEATURE_CRASH_TRIGGER, + RTW89_FW_FEATURE_PACKET_DROP, + RTW89_FW_FEATURE_NO_DEEP_PS, }; struct rtw89_fw_suit { @@ -2536,6 +2794,18 @@ struct rtw89_fw_suit { #define RTW89_FW_SUIT_VER_CODE(s) \ RTW89_FW_VER_CODE((s)->major_ver, (s)->minor_ver, (s)->sub_ver, (s)->sub_idex) +#define RTW89_MFW_HDR_VER_CODE(mfw_hdr) \ + RTW89_FW_VER_CODE((mfw_hdr)->ver.major, \ + (mfw_hdr)->ver.minor, \ + (mfw_hdr)->ver.sub, \ + (mfw_hdr)->ver.idx) + +#define RTW89_FW_HDR_VER_CODE(fw_hdr) \ + RTW89_FW_VER_CODE(GET_FW_HDR_MAJOR_VERSION(fw_hdr), \ + GET_FW_HDR_MINOR_VERSION(fw_hdr), \ + GET_FW_HDR_SUBVERSION(fw_hdr), \ + GET_FW_HDR_SUBINDEX(fw_hdr)) + struct rtw89_fw_info { const struct firmware *firmware; struct rtw89_dev *rtwdev; @@ -2558,6 +2828,8 @@ struct rtw89_cam_info { DECLARE_BITMAP(addr_cam_map, RTW89_MAX_ADDR_CAM_NUM); DECLARE_BITMAP(bssid_cam_map, RTW89_MAX_BSSID_CAM_NUM); DECLARE_BITMAP(sec_cam_map, RTW89_MAX_SEC_CAM_NUM); + DECLARE_BITMAP(ba_cam_map, RTW89_MAX_BA_CAM_NUM); + struct rtw89_ba_cam_entry ba_cam_entry[RTW89_MAX_BA_CAM_NUM]; }; enum rtw89_sar_sources { @@ -2599,24 +2871,34 @@ struct rtw89_sar_info { }; }; +struct rtw89_chanctx_cfg { + enum rtw89_sub_entity_idx idx; +}; + +enum rtw89_entity_mode { + RTW89_ENTITY_MODE_SCC, +}; + struct rtw89_hal { u32 rx_fltr; u8 cv; - u8 current_channel; - u32 current_freq; - u8 prev_primary_channel; - u8 current_primary_channel; - enum rtw89_subband current_subband; - u8 current_band_width; - u8 prev_band_type; - u8 current_band_type; u32 sw_amsdu_max_size; u32 antenna_tx; u32 antenna_rx; u8 tx_nss; u8 rx_nss; + bool tx_path_diversity; bool support_cckpd; bool support_igi; + + DECLARE_BITMAP(entity_map, NUM_OF_RTW89_SUB_ENTITY); + struct cfg80211_chan_def chandef[NUM_OF_RTW89_SUB_ENTITY]; + + bool entity_active; + enum rtw89_entity_mode entity_mode; + + struct rtw89_chan chan[NUM_OF_RTW89_SUB_ENTITY]; + struct rtw89_chan_rcd chan_rcd[NUM_OF_RTW89_SUB_ENTITY]; }; #define RTW89_MAX_MAC_ID_NUM 128 @@ -2632,11 +2914,37 @@ enum rtw89_flags { RTW89_FLAG_LEISURE_PS, RTW89_FLAG_LOW_POWER_MODE, RTW89_FLAG_INACTIVE_PS, - RTW89_FLAG_RESTART_TRIGGER, + RTW89_FLAG_CRASH_SIMULATING, NUM_OF_RTW89_FLAGS, }; +enum rtw89_pkt_drop_sel { + RTW89_PKT_DROP_SEL_MACID_BE_ONCE, + RTW89_PKT_DROP_SEL_MACID_BK_ONCE, + RTW89_PKT_DROP_SEL_MACID_VI_ONCE, + RTW89_PKT_DROP_SEL_MACID_VO_ONCE, + RTW89_PKT_DROP_SEL_MACID_ALL, + RTW89_PKT_DROP_SEL_MG0_ONCE, + RTW89_PKT_DROP_SEL_HIQ_ONCE, + RTW89_PKT_DROP_SEL_HIQ_PORT, + RTW89_PKT_DROP_SEL_HIQ_MBSSID, + RTW89_PKT_DROP_SEL_BAND, + RTW89_PKT_DROP_SEL_BAND_ONCE, + RTW89_PKT_DROP_SEL_REL_MACID, + RTW89_PKT_DROP_SEL_REL_HIQ_PORT, + RTW89_PKT_DROP_SEL_REL_HIQ_MBSSID, +}; + +struct rtw89_pkt_drop_params { + enum rtw89_pkt_drop_sel sel; + enum rtw89_mac_idx mac_band; + u8 macid; + u8 port; + u8 mbssid; + bool tf_trs; +}; + struct rtw89_pkt_stat { u16 beacon_nr; u32 rx_rate_cnt[RTW89_HW_RATE_NR]; @@ -3073,6 +3381,7 @@ struct rtw89_hw_scan_info { u8 op_chan; u8 op_bw; u8 op_band; + u32 last_chan_idx; }; enum rtw89_phy_bb_gain_band { @@ -3119,6 +3428,7 @@ struct rtw89_phy_efuse_gain { struct rtw89_dev { struct ieee80211_hw *hw; struct device *dev; + const struct ieee80211_ops *ops; bool dbcc_en; struct rtw89_hw_scan_info scan_info; @@ -3498,6 +3808,16 @@ static inline struct ieee80211_vif *rtwvif_to_vif(struct rtw89_vif *rtwvif) return container_of(p, struct ieee80211_vif, drv_priv); } +static inline struct ieee80211_vif *rtwvif_to_vif_safe(struct rtw89_vif *rtwvif) +{ + return rtwvif ? rtwvif_to_vif(rtwvif) : NULL; +} + +static inline struct rtw89_vif *vif_to_rtwvif_safe(struct ieee80211_vif *vif) +{ + return vif ? (struct rtw89_vif *)vif->drv_priv : NULL; +} + static inline struct ieee80211_sta *rtwsta_to_sta(struct rtw89_sta *rtwsta) { void *p = rtwsta; @@ -3542,6 +3862,20 @@ enum nl80211_band rtw89_hw_to_nl80211_band(enum rtw89_band hw_band) } static inline +enum rtw89_band rtw89_nl80211_to_hw_band(enum nl80211_band nl_band) +{ + switch (nl_band) { + default: + case NL80211_BAND_2GHZ: + return RTW89_BAND_2G; + case NL80211_BAND_5GHZ: + return RTW89_BAND_5G; + case NL80211_BAND_6GHZ: + return RTW89_BAND_6G; + } +} + +static inline enum rtw89_bandwidth nl_to_rtw89_bandwidth(enum nl80211_chan_width width) { switch (width) { @@ -3588,16 +3922,51 @@ struct rtw89_bssid_cam_entry *rtw89_get_bssid_cam_of(struct rtw89_vif *rtwvif, static inline void rtw89_chip_set_channel_prepare(struct rtw89_dev *rtwdev, - struct rtw89_channel_help_params *p) + struct rtw89_channel_help_params *p, + const struct rtw89_chan *chan, + enum rtw89_mac_idx mac_idx, + enum rtw89_phy_idx phy_idx) { - rtwdev->chip->ops->set_channel_help(rtwdev, true, p); + rtwdev->chip->ops->set_channel_help(rtwdev, true, p, chan, + mac_idx, phy_idx); } static inline void rtw89_chip_set_channel_done(struct rtw89_dev *rtwdev, - struct rtw89_channel_help_params *p) + struct rtw89_channel_help_params *p, + const struct rtw89_chan *chan, + enum rtw89_mac_idx mac_idx, + enum rtw89_phy_idx phy_idx) +{ + rtwdev->chip->ops->set_channel_help(rtwdev, false, p, chan, + mac_idx, phy_idx); +} + +static inline +const struct cfg80211_chan_def *rtw89_chandef_get(struct rtw89_dev *rtwdev, + enum rtw89_sub_entity_idx idx) { - rtwdev->chip->ops->set_channel_help(rtwdev, false, p); + struct rtw89_hal *hal = &rtwdev->hal; + + return &hal->chandef[idx]; +} + +static inline +const struct rtw89_chan *rtw89_chan_get(struct rtw89_dev *rtwdev, + enum rtw89_sub_entity_idx idx) +{ + struct rtw89_hal *hal = &rtwdev->hal; + + return &hal->chan[idx]; +} + +static inline +const struct rtw89_chan_rcd *rtw89_chan_rcd_get(struct rtw89_dev *rtwdev, + enum rtw89_sub_entity_idx idx) +{ + struct rtw89_hal *hal = &rtwdev->hal; + + return &hal->chan_rcd[idx]; } static inline void rtw89_chip_fem_setup(struct rtw89_dev *rtwdev) @@ -3632,12 +4001,13 @@ static inline void rtw89_chip_rfk_channel(struct rtw89_dev *rtwdev) chip->ops->rfk_channel(rtwdev); } -static inline void rtw89_chip_rfk_band_changed(struct rtw89_dev *rtwdev) +static inline void rtw89_chip_rfk_band_changed(struct rtw89_dev *rtwdev, + enum rtw89_phy_idx phy_idx) { const struct rtw89_chip_info *chip = rtwdev->chip; if (chip->ops->rfk_band_changed) - chip->ops->rfk_band_changed(rtwdev); + chip->ops->rfk_band_changed(rtwdev, phy_idx); } static inline void rtw89_chip_rfk_scan(struct rtw89_dev *rtwdev, bool start) @@ -3661,19 +4031,7 @@ static inline void rtw89_chip_set_txpwr_ctrl(struct rtw89_dev *rtwdev) const struct rtw89_chip_info *chip = rtwdev->chip; if (chip->ops->set_txpwr_ctrl) - chip->ops->set_txpwr_ctrl(rtwdev); -} - -static inline void rtw89_chip_set_txpwr(struct rtw89_dev *rtwdev) -{ - const struct rtw89_chip_info *chip = rtwdev->chip; - u8 ch = rtwdev->hal.current_channel; - - if (!ch) - return; - - if (chip->ops->set_txpwr) - chip->ops->set_txpwr(rtwdev); + chip->ops->set_txpwr_ctrl(rtwdev, RTW89_PHY_0); } static inline void rtw89_chip_power_trim(struct rtw89_dev *rtwdev) @@ -3902,16 +4260,27 @@ int rtw89_core_sta_disconnect(struct rtw89_dev *rtwdev, int rtw89_core_sta_remove(struct rtw89_dev *rtwdev, struct ieee80211_vif *vif, struct ieee80211_sta *sta); +void rtw89_core_set_tid_config(struct rtw89_dev *rtwdev, + struct ieee80211_sta *sta, + struct cfg80211_tid_config *tid_config); int rtw89_core_init(struct rtw89_dev *rtwdev); void rtw89_core_deinit(struct rtw89_dev *rtwdev); int rtw89_core_register(struct rtw89_dev *rtwdev); void rtw89_core_unregister(struct rtw89_dev *rtwdev); +struct rtw89_dev *rtw89_alloc_ieee80211_hw(struct device *device, + u32 bus_data_size, + const struct rtw89_chip_info *chip); +void rtw89_free_ieee80211_hw(struct rtw89_dev *rtwdev); +void rtw89_core_set_chip_txpwr(struct rtw89_dev *rtwdev); +void rtw89_get_default_chandef(struct cfg80211_chan_def *chandef); void rtw89_set_channel(struct rtw89_dev *rtwdev); u8 rtw89_core_acquire_bit_map(unsigned long *addr, unsigned long size); void rtw89_core_release_bit_map(unsigned long *addr, u8 bit); void rtw89_core_release_all_bits_map(unsigned long *addr, unsigned int nbits); -int rtw89_core_acquire_sta_ba_entry(struct rtw89_sta *rtwsta, u8 tid, u8 *cam_idx); -int rtw89_core_release_sta_ba_entry(struct rtw89_sta *rtwsta, u8 tid, u8 *cam_idx); +int rtw89_core_acquire_sta_ba_entry(struct rtw89_dev *rtwdev, + struct rtw89_sta *rtwsta, u8 tid, u8 *cam_idx); +int rtw89_core_release_sta_ba_entry(struct rtw89_dev *rtwdev, + struct rtw89_sta *rtwsta, u8 tid, u8 *cam_idx); void rtw89_vif_type_mapping(struct ieee80211_vif *vif, bool assoc); int rtw89_chip_info_setup(struct rtw89_dev *rtwdev); bool rtw89_ra_report_to_bitrate(struct rtw89_dev *rtwdev, u8 rpt_rate, u16 *bitrate); diff --git a/drivers/net/wireless/realtek/rtw89/debug.c b/drivers/net/wireless/realtek/rtw89/debug.c index 829c61da99bb..730e83d54257 100644 --- a/drivers/net/wireless/realtek/rtw89/debug.c +++ b/drivers/net/wireless/realtek/rtw89/debug.c @@ -525,7 +525,8 @@ static int __print_txpwr_map(struct seq_file *m, struct rtw89_dev *rtwdev, static void __print_regd(struct seq_file *m, struct rtw89_dev *rtwdev) { - u8 band = rtwdev->hal.current_band_type; + const struct rtw89_chan *chan = rtw89_chan_get(rtwdev, RTW89_SUB_ENTITY_0); + u8 band = chan->band_type; u8 regd = rtw89_regd_get(rtwdev, band); switch (regd) { @@ -2189,6 +2190,37 @@ out: return count; } +static int rtw89_dbg_trigger_ctrl_error(struct rtw89_dev *rtwdev) +{ + struct rtw89_cpuio_ctrl ctrl_para = {0}; + u16 pkt_id; + + rtw89_leave_ps_mode(rtwdev); + + pkt_id = rtw89_mac_dle_buf_req(rtwdev, 0x20, true); + switch (pkt_id) { + case 0xffff: + return -ETIMEDOUT; + case 0xfff: + return -ENOMEM; + default: + break; + } + + /* intentionally, enqueue two pkt, but has only one pkt id */ + ctrl_para.cmd_type = CPUIO_OP_CMD_ENQ_TO_HEAD; + ctrl_para.start_pktid = pkt_id; + ctrl_para.end_pktid = pkt_id; + ctrl_para.pkt_num = 1; /* start from 0 */ + ctrl_para.dst_pid = WDE_DLE_PORT_ID_WDRLS; + ctrl_para.dst_qid = WDE_DLE_QUEID_NO_REPORT; + + if (rtw89_mac_set_cpuio(rtwdev, &ctrl_para, true)) + return -EFAULT; + + return 0; +} + static int rtw89_debug_priv_fw_crash_get(struct seq_file *m, void *v) { @@ -2196,10 +2228,15 @@ rtw89_debug_priv_fw_crash_get(struct seq_file *m, void *v) struct rtw89_dev *rtwdev = debugfs_priv->rtwdev; seq_printf(m, "%d\n", - test_bit(RTW89_FLAG_RESTART_TRIGGER, rtwdev->flags)); + test_bit(RTW89_FLAG_CRASH_SIMULATING, rtwdev->flags)); return 0; } +enum rtw89_dbg_crash_simulation_type { + RTW89_DBG_SIM_CPU_EXCEPTION = 1, + RTW89_DBG_SIM_CTRL_ERROR = 2, +}; + static ssize_t rtw89_debug_priv_fw_crash_set(struct file *filp, const char __user *user_buf, size_t count, loff_t *loff) @@ -2207,22 +2244,30 @@ rtw89_debug_priv_fw_crash_set(struct file *filp, const char __user *user_buf, struct seq_file *m = (struct seq_file *)filp->private_data; struct rtw89_debugfs_priv *debugfs_priv = m->private; struct rtw89_dev *rtwdev = debugfs_priv->rtwdev; - bool fw_crash; + int (*sim)(struct rtw89_dev *rtwdev); + u8 crash_type; int ret; - if (!RTW89_CHK_FW_FEATURE(CRASH_TRIGGER, &rtwdev->fw)) - return -EOPNOTSUPP; - - ret = kstrtobool_from_user(user_buf, count, &fw_crash); + ret = kstrtou8_from_user(user_buf, count, 0, &crash_type); if (ret) return -EINVAL; - if (!fw_crash) + switch (crash_type) { + case RTW89_DBG_SIM_CPU_EXCEPTION: + if (!RTW89_CHK_FW_FEATURE(CRASH_TRIGGER, &rtwdev->fw)) + return -EOPNOTSUPP; + sim = rtw89_fw_h2c_trigger_cpu_exception; + break; + case RTW89_DBG_SIM_CTRL_ERROR: + sim = rtw89_dbg_trigger_ctrl_error; + break; + default: return -EINVAL; + } mutex_lock(&rtwdev->mutex); - set_bit(RTW89_FLAG_RESTART_TRIGGER, rtwdev->flags); - ret = rtw89_fw_h2c_trigger_cpu_exception(rtwdev); + set_bit(RTW89_FLAG_CRASH_SIMULATING, rtwdev->flags); + ret = sim(rtwdev); mutex_unlock(&rtwdev->mutex); if (ret) @@ -2289,7 +2334,10 @@ static void rtw89_sta_info_get_iter(void *data, struct ieee80211_sta *sta) struct rate_info *rate = &rtwsta->ra_report.txrate; struct ieee80211_rx_status *status = &rtwsta->rx_status; struct seq_file *m = (struct seq_file *)data; + struct rtw89_dev *rtwdev = rtwsta->rtwdev; + struct rtw89_hal *hal = &rtwdev->hal; u8 rssi; + int i; seq_printf(m, "TX rate [%d]: ", rtwsta->mac_id); @@ -2305,9 +2353,10 @@ static void rtw89_sta_info_get_iter(void *data, struct ieee80211_sta *sta) he_gi_str[rate->he_gi] : "N/A"); else seq_printf(m, "Legacy %d", rate->legacy); + seq_printf(m, "%s", rtwsta->ra_report.might_fallback_legacy ? " FB_G" : ""); seq_printf(m, "\t(hw_rate=0x%x)", rtwsta->ra_report.hw_rate); seq_printf(m, "\t==> agg_wait=%d (%d)\n", rtwsta->max_agg_wait, - sta->max_rc_amsdu_len); + sta->deflink.agg.max_rc_amsdu_len); seq_printf(m, "RX rate [%d]: ", rtwsta->mac_id); @@ -2333,8 +2382,15 @@ static void rtw89_sta_info_get_iter(void *data, struct ieee80211_sta *sta) seq_printf(m, "\t(hw_rate=0x%x)\n", rtwsta->rx_hw_rate); rssi = ewma_rssi_read(&rtwsta->avg_rssi); - seq_printf(m, "RSSI: %d dBm (raw=%d, prev=%d)\n", + seq_printf(m, "RSSI: %d dBm (raw=%d, prev=%d) [", RTW89_RSSI_RAW_TO_DBM(rssi), rssi, rtwsta->prev_rssi); + for (i = 0; i < rtwdev->chip->rf_path_num; i++) { + rssi = ewma_rssi_read(&rtwsta->rssi[i]); + seq_printf(m, "%d%s%s", RTW89_RSSI_RAW_TO_DBM(rssi), + hal->tx_path_diversity && (hal->antenna_tx & BIT(i)) ? "*" : "", + i + 1 == rtwdev->chip->rf_path_num ? "" : ", "); + } + seq_puts(m, "]\n"); } static void @@ -2433,6 +2489,26 @@ void rtw89_vif_ids_get_iter(void *data, u8 *mac, struct ieee80211_vif *vif) rtw89_dump_addr_cam(m, &rtwvif->addr_cam); } +static void rtw89_dump_ba_cam(struct seq_file *m, struct rtw89_sta *rtwsta) +{ + struct rtw89_vif *rtwvif = rtwsta->rtwvif; + struct rtw89_dev *rtwdev = rtwvif->rtwdev; + struct rtw89_ba_cam_entry *entry; + bool first = true; + + list_for_each_entry(entry, &rtwsta->ba_cam_list, list) { + if (first) { + seq_puts(m, "\tba_cam "); + first = false; + } else { + seq_puts(m, ", "); + } + seq_printf(m, "tid[%u]=%d", entry->tid, + (int)(entry - rtwdev->cam_info.ba_cam_entry)); + } + seq_puts(m, "\n"); +} + static void rtw89_sta_ids_get_iter(void *data, struct ieee80211_sta *sta) { struct rtw89_sta *rtwsta = (struct rtw89_sta *)sta->drv_priv; @@ -2441,6 +2517,7 @@ static void rtw89_sta_ids_get_iter(void *data, struct ieee80211_sta *sta) seq_printf(m, "STA [%d] %pM %s\n", rtwsta->mac_id, sta->addr, sta->tdls ? "(TDLS)" : ""); rtw89_dump_addr_cam(m, &rtwsta->addr_cam); + rtw89_dump_ba_cam(m, rtwsta); } static int rtw89_debug_priv_stations_get(struct seq_file *m, void *v) @@ -2449,6 +2526,8 @@ static int rtw89_debug_priv_stations_get(struct seq_file *m, void *v) struct rtw89_dev *rtwdev = debugfs_priv->rtwdev; struct rtw89_cam_info *cam_info = &rtwdev->cam_info; + mutex_lock(&rtwdev->mutex); + seq_puts(m, "map:\n"); seq_printf(m, "\tmac_id: %*ph\n", (int)sizeof(rtwdev->mac_id_map), rtwdev->mac_id_map); @@ -2458,12 +2537,16 @@ static int rtw89_debug_priv_stations_get(struct seq_file *m, void *v) cam_info->bssid_cam_map); seq_printf(m, "\tsec_cam: %*ph\n", (int)sizeof(cam_info->sec_cam_map), cam_info->sec_cam_map); + seq_printf(m, "\tba_cam: %*ph\n", (int)sizeof(cam_info->ba_cam_map), + cam_info->ba_cam_map); ieee80211_iterate_active_interfaces_atomic(rtwdev->hw, IEEE80211_IFACE_ITER_NORMAL, rtw89_vif_ids_get_iter, m); ieee80211_iterate_stations_atomic(rtwdev->hw, rtw89_sta_ids_get_iter, m); + mutex_unlock(&rtwdev->mutex); + return 0; } diff --git a/drivers/net/wireless/realtek/rtw89/debug.h b/drivers/net/wireless/realtek/rtw89/debug.h index 6176152dbf6b..ee243aadde87 100644 --- a/drivers/net/wireless/realtek/rtw89/debug.h +++ b/drivers/net/wireless/realtek/rtw89/debug.h @@ -25,6 +25,7 @@ enum rtw89_debug_mask { RTW89_DBG_BF = BIT(14), RTW89_DBG_HW_SCAN = BIT(15), RTW89_DBG_SAR = BIT(16), + RTW89_DBG_STATE = BIT(17), RTW89_DBG_UNEXP = BIT(31), }; diff --git a/drivers/net/wireless/realtek/rtw89/fw.c b/drivers/net/wireless/realtek/rtw89/fw.c index 6473015a6b2a..d57e3610fb88 100644 --- a/drivers/net/wireless/realtek/rtw89/fw.c +++ b/drivers/net/wireless/realtek/rtw89/fw.c @@ -3,6 +3,7 @@ */ #include "cam.h" +#include "chan.h" #include "coex.h" #include "debug.h" #include "fw.h" @@ -224,6 +225,12 @@ static const struct __fw_feat_cfg fw_feat_tbl[] = { __CFG_FW_FEAT(RTL8852A, ge, 0, 13, 35, 0, SCAN_OFFLOAD), __CFG_FW_FEAT(RTL8852A, ge, 0, 13, 35, 0, TX_WAKE), __CFG_FW_FEAT(RTL8852A, ge, 0, 13, 36, 0, CRASH_TRIGGER), + __CFG_FW_FEAT(RTL8852A, ge, 0, 13, 38, 0, PACKET_DROP), + __CFG_FW_FEAT(RTL8852C, ge, 0, 27, 20, 0, PACKET_DROP), + __CFG_FW_FEAT(RTL8852C, le, 0, 27, 33, 0, NO_DEEP_PS), + __CFG_FW_FEAT(RTL8852C, ge, 0, 27, 34, 0, TX_WAKE), + __CFG_FW_FEAT(RTL8852C, ge, 0, 27, 36, 0, SCAN_OFFLOAD), + __CFG_FW_FEAT(RTL8852C, ge, 0, 27, 40, 0, CRASH_TRIGGER), }; static void rtw89_fw_recognize_features(struct rtw89_dev *rtwdev) @@ -247,6 +254,46 @@ static void rtw89_fw_recognize_features(struct rtw89_dev *rtwdev) } } +void rtw89_early_fw_feature_recognize(struct device *device, + const struct rtw89_chip_info *chip, + u32 *early_feat_map) +{ + union { + struct rtw89_mfw_hdr mfw_hdr; + u8 fw_hdr[RTW89_FW_HDR_SIZE]; + } buf = {}; + const struct firmware *firmware; + u32 ver_code; + int ret; + int i; + + ret = request_partial_firmware_into_buf(&firmware, chip->fw_name, + device, &buf, sizeof(buf), 0); + if (ret) { + dev_err(device, "failed to early request firmware: %d\n", ret); + return; + } + + ver_code = buf.mfw_hdr.sig != RTW89_MFW_SIG ? + RTW89_FW_HDR_VER_CODE(&buf.fw_hdr) : + RTW89_MFW_HDR_VER_CODE(&buf.mfw_hdr); + if (!ver_code) + goto out; + + for (i = 0; i < ARRAY_SIZE(fw_feat_tbl); i++) { + const struct __fw_feat_cfg *ent = &fw_feat_tbl[i]; + + if (chip->chip_id != ent->chip_id) + continue; + + if (ent->cond(ver_code, ent->ver_code)) + *early_feat_map |= BIT(ent->feature); + } + +out: + release_firmware(firmware); +} + int rtw89_fw_recognize(struct rtw89_dev *rtwdev) { int ret; @@ -571,6 +618,7 @@ int rtw89_fw_h2c_cam(struct rtw89_dev *rtwdev, struct rtw89_vif *rtwvif, struct rtw89_sta *rtwsta, const u8 *scan_mac_addr) { struct sk_buff *skb; + int ret; skb = rtw89_fw_h2c_alloc_skb_with_hdr(rtwdev, H2C_CAM_LEN); if (!skb) { @@ -587,7 +635,8 @@ int rtw89_fw_h2c_cam(struct rtw89_dev *rtwdev, struct rtw89_vif *rtwvif, H2C_FUNC_MAC_ADDR_CAM_UPD, 0, 1, H2C_CAM_LEN); - if (rtw89_h2c_tx(rtwdev, skb, false)) { + ret = rtw89_h2c_tx(rtwdev, skb, false); + if (ret) { rtw89_err(rtwdev, "failed to send h2c\n"); goto fail; } @@ -596,7 +645,7 @@ int rtw89_fw_h2c_cam(struct rtw89_dev *rtwdev, struct rtw89_vif *rtwvif, fail: dev_kfree_skb_any(skb); - return -EBUSY; + return ret; } #define H2C_DCTL_SEC_CAM_LEN 68 @@ -605,6 +654,7 @@ int rtw89_fw_h2c_dctl_sec_cam_v1(struct rtw89_dev *rtwdev, struct rtw89_sta *rtwsta) { struct sk_buff *skb; + int ret; skb = rtw89_fw_h2c_alloc_skb_with_hdr(rtwdev, H2C_DCTL_SEC_CAM_LEN); if (!skb) { @@ -621,7 +671,8 @@ int rtw89_fw_h2c_dctl_sec_cam_v1(struct rtw89_dev *rtwdev, H2C_FUNC_MAC_DCTLINFO_UD_V1, 0, 0, H2C_DCTL_SEC_CAM_LEN); - if (rtw89_h2c_tx(rtwdev, skb, false)) { + ret = rtw89_h2c_tx(rtwdev, skb, false); + if (ret) { rtw89_err(rtwdev, "failed to send h2c\n"); goto fail; } @@ -630,7 +681,7 @@ int rtw89_fw_h2c_dctl_sec_cam_v1(struct rtw89_dev *rtwdev, fail: dev_kfree_skb_any(skb); - return -EBUSY; + return ret; } EXPORT_SYMBOL(rtw89_fw_h2c_dctl_sec_cam_v1); @@ -638,14 +689,16 @@ EXPORT_SYMBOL(rtw89_fw_h2c_dctl_sec_cam_v1); int rtw89_fw_h2c_ba_cam(struct rtw89_dev *rtwdev, struct rtw89_sta *rtwsta, bool valid, struct ieee80211_ampdu_params *params) { + const struct rtw89_chip_info *chip = rtwdev->chip; + struct rtw89_vif *rtwvif = rtwsta->rtwvif; u8 macid = rtwsta->mac_id; struct sk_buff *skb; u8 entry_idx; int ret; ret = valid ? - rtw89_core_acquire_sta_ba_entry(rtwsta, params->tid, &entry_idx) : - rtw89_core_release_sta_ba_entry(rtwsta, params->tid, &entry_idx); + rtw89_core_acquire_sta_ba_entry(rtwdev, rtwsta, params->tid, &entry_idx) : + rtw89_core_release_sta_ba_entry(rtwdev, rtwsta, params->tid, &entry_idx); if (ret) { /* it still works even if we don't have static BA CAM, because * hardware can create dynamic BA CAM automatically. @@ -663,7 +716,10 @@ int rtw89_fw_h2c_ba_cam(struct rtw89_dev *rtwdev, struct rtw89_sta *rtwsta, } skb_put(skb, H2C_BA_CAM_LEN); SET_BA_CAM_MACID(skb->data, macid); - SET_BA_CAM_ENTRY_IDX(skb->data, entry_idx); + if (chip->bacam_v1) + SET_BA_CAM_ENTRY_IDX_V1(skb->data, entry_idx); + else + SET_BA_CAM_ENTRY_IDX(skb->data, entry_idx); if (!valid) goto end; SET_BA_CAM_VALID(skb->data, valid); @@ -676,6 +732,11 @@ int rtw89_fw_h2c_ba_cam(struct rtw89_dev *rtwdev, struct rtw89_sta *rtwsta, SET_BA_CAM_INIT_REQ(skb->data, 1); SET_BA_CAM_SSN(skb->data, params->ssn); + if (chip->bacam_v1) { + SET_BA_CAM_STD_EN(skb->data, 1); + SET_BA_CAM_BAND(skb->data, rtwvif->mac_idx); + } + end: rtw89_h2c_pkt_set_hdr(rtwdev, skb, FWCMD_TYPE_H2C, H2C_CAT_MAC, @@ -683,7 +744,8 @@ end: H2C_FUNC_MAC_BA_CAM, 0, 1, H2C_BA_CAM_LEN); - if (rtw89_h2c_tx(rtwdev, skb, false)) { + ret = rtw89_h2c_tx(rtwdev, skb, false); + if (ret) { rtw89_err(rtwdev, "failed to send h2c\n"); goto fail; } @@ -692,7 +754,59 @@ end: fail: dev_kfree_skb_any(skb); - return -EBUSY; + return ret; +} + +static int rtw89_fw_h2c_init_dynamic_ba_cam_v1(struct rtw89_dev *rtwdev, + u8 entry_idx, u8 uid) +{ + struct sk_buff *skb; + int ret; + + skb = rtw89_fw_h2c_alloc_skb_with_hdr(rtwdev, H2C_BA_CAM_LEN); + if (!skb) { + rtw89_err(rtwdev, "failed to alloc skb for dynamic h2c ba cam\n"); + return -ENOMEM; + } + skb_put(skb, H2C_BA_CAM_LEN); + + SET_BA_CAM_VALID(skb->data, 1); + SET_BA_CAM_ENTRY_IDX_V1(skb->data, entry_idx); + SET_BA_CAM_UID(skb->data, uid); + SET_BA_CAM_BAND(skb->data, 0); + SET_BA_CAM_STD_EN(skb->data, 0); + + rtw89_h2c_pkt_set_hdr(rtwdev, skb, FWCMD_TYPE_H2C, + H2C_CAT_MAC, + H2C_CL_BA_CAM, + H2C_FUNC_MAC_BA_CAM, 0, 1, + H2C_BA_CAM_LEN); + + ret = rtw89_h2c_tx(rtwdev, skb, false); + if (ret) { + rtw89_err(rtwdev, "failed to send h2c\n"); + goto fail; + } + + return 0; +fail: + dev_kfree_skb_any(skb); + + return ret; +} + +void rtw89_fw_h2c_init_ba_cam_v1(struct rtw89_dev *rtwdev) +{ + const struct rtw89_chip_info *chip = rtwdev->chip; + u8 entry_idx = chip->bacam_num; + u8 uid = 0; + int i; + + for (i = 0; i < chip->bacam_dynamic_num; i++) { + rtw89_fw_h2c_init_dynamic_ba_cam_v1(rtwdev, entry_idx, uid); + entry_idx++; + uid++; + } } #define H2C_LOG_CFG_LEN 12 @@ -701,6 +815,7 @@ int rtw89_fw_h2c_fw_log(struct rtw89_dev *rtwdev, bool enable) struct sk_buff *skb; u32 comp = enable ? BIT(RTW89_FW_LOG_COMP_INIT) | BIT(RTW89_FW_LOG_COMP_TASK) | BIT(RTW89_FW_LOG_COMP_PS) | BIT(RTW89_FW_LOG_COMP_ERROR) : 0; + int ret; skb = rtw89_fw_h2c_alloc_skb_with_hdr(rtwdev, H2C_LOG_CFG_LEN); if (!skb) { @@ -720,7 +835,8 @@ int rtw89_fw_h2c_fw_log(struct rtw89_dev *rtwdev, bool enable) H2C_FUNC_LOG_CFG, 0, 0, H2C_LOG_CFG_LEN); - if (rtw89_h2c_tx(rtwdev, skb, false)) { + ret = rtw89_h2c_tx(rtwdev, skb, false); + if (ret) { rtw89_err(rtwdev, "failed to send h2c\n"); goto fail; } @@ -729,7 +845,7 @@ int rtw89_fw_h2c_fw_log(struct rtw89_dev *rtwdev, bool enable) fail: dev_kfree_skb_any(skb); - return -EBUSY; + return ret; } #define H2C_GENERAL_PKT_LEN 6 @@ -737,6 +853,7 @@ fail: int rtw89_fw_h2c_general_pkt(struct rtw89_dev *rtwdev, u8 macid) { struct sk_buff *skb; + int ret; skb = rtw89_fw_h2c_alloc_skb_with_hdr(rtwdev, H2C_GENERAL_PKT_LEN); if (!skb) { @@ -757,7 +874,8 @@ int rtw89_fw_h2c_general_pkt(struct rtw89_dev *rtwdev, u8 macid) H2C_FUNC_MAC_GENERAL_PKT, 0, 1, H2C_GENERAL_PKT_LEN); - if (rtw89_h2c_tx(rtwdev, skb, false)) { + ret = rtw89_h2c_tx(rtwdev, skb, false); + if (ret) { rtw89_err(rtwdev, "failed to send h2c\n"); goto fail; } @@ -766,7 +884,7 @@ int rtw89_fw_h2c_general_pkt(struct rtw89_dev *rtwdev, u8 macid) fail: dev_kfree_skb_any(skb); - return -EBUSY; + return ret; } #define H2C_LPS_PARM_LEN 8 @@ -774,6 +892,7 @@ int rtw89_fw_h2c_lps_parm(struct rtw89_dev *rtwdev, struct rtw89_lps_parm *lps_param) { struct sk_buff *skb; + int ret; skb = rtw89_fw_h2c_alloc_skb_with_hdr(rtwdev, H2C_LPS_PARM_LEN); if (!skb) { @@ -799,7 +918,8 @@ int rtw89_fw_h2c_lps_parm(struct rtw89_dev *rtwdev, H2C_FUNC_MAC_LPS_PARM, 0, 1, H2C_LPS_PARM_LEN); - if (rtw89_h2c_tx(rtwdev, skb, false)) { + ret = rtw89_h2c_tx(rtwdev, skb, false); + if (ret) { rtw89_err(rtwdev, "failed to send h2c\n"); goto fail; } @@ -808,7 +928,73 @@ int rtw89_fw_h2c_lps_parm(struct rtw89_dev *rtwdev, fail: dev_kfree_skb_any(skb); - return -EBUSY; + return ret; +} + +#define H2C_P2P_ACT_LEN 20 +int rtw89_fw_h2c_p2p_act(struct rtw89_dev *rtwdev, struct ieee80211_vif *vif, + struct ieee80211_p2p_noa_desc *desc, + u8 act, u8 noa_id) +{ + struct rtw89_vif *rtwvif = (struct rtw89_vif *)vif->drv_priv; + bool p2p_type_gc = rtwvif->wifi_role == RTW89_WIFI_ROLE_P2P_CLIENT; + u8 ctwindow_oppps = vif->bss_conf.p2p_noa_attr.oppps_ctwindow; + struct sk_buff *skb; + u8 *cmd; + int ret; + + skb = rtw89_fw_h2c_alloc_skb_with_hdr(rtwdev, H2C_P2P_ACT_LEN); + if (!skb) { + rtw89_err(rtwdev, "failed to alloc skb for h2c p2p act\n"); + return -ENOMEM; + } + skb_put(skb, H2C_P2P_ACT_LEN); + cmd = skb->data; + + RTW89_SET_FWCMD_P2P_MACID(cmd, rtwvif->mac_id); + RTW89_SET_FWCMD_P2P_P2PID(cmd, 0); + RTW89_SET_FWCMD_P2P_NOAID(cmd, noa_id); + RTW89_SET_FWCMD_P2P_ACT(cmd, act); + RTW89_SET_FWCMD_P2P_TYPE(cmd, p2p_type_gc); + RTW89_SET_FWCMD_P2P_ALL_SLEP(cmd, 0); + if (desc) { + RTW89_SET_FWCMD_NOA_START_TIME(cmd, desc->start_time); + RTW89_SET_FWCMD_NOA_INTERVAL(cmd, desc->interval); + RTW89_SET_FWCMD_NOA_DURATION(cmd, desc->duration); + RTW89_SET_FWCMD_NOA_COUNT(cmd, desc->count); + RTW89_SET_FWCMD_NOA_CTWINDOW(cmd, ctwindow_oppps); + } + + rtw89_h2c_pkt_set_hdr(rtwdev, skb, FWCMD_TYPE_H2C, + H2C_CAT_MAC, H2C_CL_MAC_PS, + H2C_FUNC_P2P_ACT, 0, 0, + H2C_P2P_ACT_LEN); + + ret = rtw89_h2c_tx(rtwdev, skb, false); + if (ret) { + rtw89_err(rtwdev, "failed to send h2c\n"); + goto fail; + } + + return 0; +fail: + dev_kfree_skb_any(skb); + + return ret; +} + +static void __rtw89_fw_h2c_set_tx_path(struct rtw89_dev *rtwdev, + struct sk_buff *skb) +{ + struct rtw89_hal *hal = &rtwdev->hal; + u8 ntx_path = hal->antenna_tx ? hal->antenna_tx : RF_B; + u8 map_b = hal->antenna_tx == RF_AB ? 1 : 0; + + SET_CMC_TBL_NTX_PATH_EN(skb->data, ntx_path); + SET_CMC_TBL_PATH_MAP_A(skb->data, 0); + SET_CMC_TBL_PATH_MAP_B(skb->data, map_b); + SET_CMC_TBL_PATH_MAP_C(skb->data, 0); + SET_CMC_TBL_PATH_MAP_D(skb->data, 0); } #define H2C_CMC_TBL_LEN 68 @@ -816,11 +1002,9 @@ int rtw89_fw_h2c_default_cmac_tbl(struct rtw89_dev *rtwdev, struct rtw89_vif *rtwvif) { const struct rtw89_chip_info *chip = rtwdev->chip; - struct rtw89_hal *hal = &rtwdev->hal; struct sk_buff *skb; - u8 ntx_path = hal->antenna_tx ? hal->antenna_tx : RF_B; - u8 map_b = hal->antenna_tx == RF_AB ? 1 : 0; u8 macid = rtwvif->mac_id; + int ret; skb = rtw89_fw_h2c_alloc_skb_with_hdr(rtwdev, H2C_CMC_TBL_LEN); if (!skb) { @@ -832,11 +1016,7 @@ int rtw89_fw_h2c_default_cmac_tbl(struct rtw89_dev *rtwdev, SET_CTRL_INFO_OPERATION(skb->data, 1); if (chip->h2c_cctl_func_id == H2C_FUNC_MAC_CCTLINFO_UD) { SET_CMC_TBL_TXPWR_MODE(skb->data, 0); - SET_CMC_TBL_NTX_PATH_EN(skb->data, ntx_path); - SET_CMC_TBL_PATH_MAP_A(skb->data, 0); - SET_CMC_TBL_PATH_MAP_B(skb->data, map_b); - SET_CMC_TBL_PATH_MAP_C(skb->data, 0); - SET_CMC_TBL_PATH_MAP_D(skb->data, 0); + __rtw89_fw_h2c_set_tx_path(rtwdev, skb); SET_CMC_TBL_ANTSEL_A(skb->data, 0); SET_CMC_TBL_ANTSEL_B(skb->data, 0); SET_CMC_TBL_ANTSEL_C(skb->data, 0); @@ -852,7 +1032,8 @@ int rtw89_fw_h2c_default_cmac_tbl(struct rtw89_dev *rtwdev, chip->h2c_cctl_func_id, 0, 1, H2C_CMC_TBL_LEN); - if (rtw89_h2c_tx(rtwdev, skb, false)) { + ret = rtw89_h2c_tx(rtwdev, skb, false); + if (ret) { rtw89_err(rtwdev, "failed to send h2c\n"); goto fail; } @@ -861,7 +1042,7 @@ int rtw89_fw_h2c_default_cmac_tbl(struct rtw89_dev *rtwdev, fail: dev_kfree_skb_any(skb); - return -EBUSY; + return ret; } static void __get_sta_he_pkt_padding(struct rtw89_dev *rtwdev, @@ -926,17 +1107,26 @@ int rtw89_fw_h2c_assoc_cmac_tbl(struct rtw89_dev *rtwdev, struct ieee80211_sta *sta) { const struct rtw89_chip_info *chip = rtwdev->chip; - struct rtw89_hal *hal = &rtwdev->hal; struct rtw89_sta *rtwsta = sta_to_rtwsta_safe(sta); struct rtw89_vif *rtwvif = (struct rtw89_vif *)vif->drv_priv; + const struct rtw89_chan *chan = rtw89_chan_get(rtwdev, RTW89_SUB_ENTITY_0); struct sk_buff *skb; u8 pads[RTW89_PPE_BW_NUM]; u8 mac_id = rtwsta ? rtwsta->mac_id : rtwvif->mac_id; + u16 lowest_rate; + int ret; memset(pads, 0, sizeof(pads)); if (sta) __get_sta_he_pkt_padding(rtwdev, sta, pads); + if (vif->p2p) + lowest_rate = RTW89_HW_RATE_OFDM6; + else if (chan->band_type == RTW89_BAND_2G) + lowest_rate = RTW89_HW_RATE_CCK1; + else + lowest_rate = RTW89_HW_RATE_OFDM6; + skb = rtw89_fw_h2c_alloc_skb_with_hdr(rtwdev, H2C_CMC_TBL_LEN); if (!skb) { rtw89_err(rtwdev, "failed to alloc skb for fw dl\n"); @@ -947,10 +1137,7 @@ int rtw89_fw_h2c_assoc_cmac_tbl(struct rtw89_dev *rtwdev, SET_CTRL_INFO_OPERATION(skb->data, 1); SET_CMC_TBL_DISRTSFB(skb->data, 1); SET_CMC_TBL_DISDATAFB(skb->data, 1); - if (hal->current_band_type == RTW89_BAND_2G) - SET_CMC_TBL_RTS_RTY_LOWEST_RATE(skb->data, RTW89_HW_RATE_CCK1); - else - SET_CMC_TBL_RTS_RTY_LOWEST_RATE(skb->data, RTW89_HW_RATE_OFDM6); + SET_CMC_TBL_RTS_RTY_LOWEST_RATE(skb->data, lowest_rate); SET_CMC_TBL_RTS_TXCNT_LMT_SEL(skb->data, 0); SET_CMC_TBL_DATA_TXCNT_LMT_SEL(skb->data, 0); if (vif->type == NL80211_IFTYPE_STATION) @@ -980,7 +1167,8 @@ int rtw89_fw_h2c_assoc_cmac_tbl(struct rtw89_dev *rtwdev, chip->h2c_cctl_func_id, 0, 1, H2C_CMC_TBL_LEN); - if (rtw89_h2c_tx(rtwdev, skb, false)) { + ret = rtw89_h2c_tx(rtwdev, skb, false); + if (ret) { rtw89_err(rtwdev, "failed to send h2c\n"); goto fail; } @@ -989,7 +1177,7 @@ int rtw89_fw_h2c_assoc_cmac_tbl(struct rtw89_dev *rtwdev, fail: dev_kfree_skb_any(skb); - return -EBUSY; + return ret; } int rtw89_fw_h2c_txtime_cmac_tbl(struct rtw89_dev *rtwdev, @@ -997,6 +1185,7 @@ int rtw89_fw_h2c_txtime_cmac_tbl(struct rtw89_dev *rtwdev, { const struct rtw89_chip_info *chip = rtwdev->chip; struct sk_buff *skb; + int ret; skb = rtw89_fw_h2c_alloc_skb_with_hdr(rtwdev, H2C_CMC_TBL_LEN); if (!skb) { @@ -1020,7 +1209,47 @@ int rtw89_fw_h2c_txtime_cmac_tbl(struct rtw89_dev *rtwdev, chip->h2c_cctl_func_id, 0, 1, H2C_CMC_TBL_LEN); - if (rtw89_h2c_tx(rtwdev, skb, false)) { + ret = rtw89_h2c_tx(rtwdev, skb, false); + if (ret) { + rtw89_err(rtwdev, "failed to send h2c\n"); + goto fail; + } + + return 0; +fail: + dev_kfree_skb_any(skb); + + return ret; +} + +int rtw89_fw_h2c_txpath_cmac_tbl(struct rtw89_dev *rtwdev, + struct rtw89_sta *rtwsta) +{ + const struct rtw89_chip_info *chip = rtwdev->chip; + struct sk_buff *skb; + int ret; + + if (chip->h2c_cctl_func_id != H2C_FUNC_MAC_CCTLINFO_UD) + return 0; + + skb = rtw89_fw_h2c_alloc_skb_with_hdr(rtwdev, H2C_CMC_TBL_LEN); + if (!skb) { + rtw89_err(rtwdev, "failed to alloc skb for fw dl\n"); + return -ENOMEM; + } + skb_put(skb, H2C_CMC_TBL_LEN); + SET_CTRL_INFO_MACID(skb->data, rtwsta->mac_id); + SET_CTRL_INFO_OPERATION(skb->data, 1); + + __rtw89_fw_h2c_set_tx_path(rtwdev, skb); + + rtw89_h2c_pkt_set_hdr(rtwdev, skb, FWCMD_TYPE_H2C, + H2C_CAT_MAC, H2C_CL_MAC_FR_EXCHG, + H2C_FUNC_MAC_CCTLINFO_UD, 0, 1, + H2C_CMC_TBL_LEN); + + ret = rtw89_h2c_tx(rtwdev, skb, false); + if (ret) { rtw89_err(rtwdev, "failed to send h2c\n"); goto fail; } @@ -1029,19 +1258,28 @@ int rtw89_fw_h2c_txtime_cmac_tbl(struct rtw89_dev *rtwdev, fail: dev_kfree_skb_any(skb); - return -EBUSY; + return ret; } #define H2C_BCN_BASE_LEN 12 int rtw89_fw_h2c_update_beacon(struct rtw89_dev *rtwdev, struct rtw89_vif *rtwvif) { - struct rtw89_hal *hal = &rtwdev->hal; struct ieee80211_vif *vif = rtwvif_to_vif(rtwvif); + const struct rtw89_chan *chan = rtw89_chan_get(rtwdev, RTW89_SUB_ENTITY_0); struct sk_buff *skb; struct sk_buff *skb_beacon; u16 tim_offset; int bcn_total_len; + u16 beacon_rate; + int ret; + + if (vif->p2p) + beacon_rate = RTW89_HW_RATE_OFDM6; + else if (chan->band_type == RTW89_BAND_2G) + beacon_rate = RTW89_HW_RATE_CCK1; + else + beacon_rate = RTW89_HW_RATE_OFDM6; skb_beacon = ieee80211_beacon_get_tim(rtwdev->hw, vif, &tim_offset, NULL, 0); @@ -1066,8 +1304,7 @@ int rtw89_fw_h2c_update_beacon(struct rtw89_dev *rtwdev, SET_BCN_UPD_MACID(skb->data, rtwvif->mac_id); SET_BCN_UPD_SSN_SEL(skb->data, RTW89_MGMT_HW_SSN_SEL); SET_BCN_UPD_SSN_MODE(skb->data, RTW89_MGMT_HW_SEQ_MODE); - SET_BCN_UPD_RATE(skb->data, hal->current_band_type == RTW89_BAND_2G ? - RTW89_HW_RATE_CCK1 : RTW89_HW_RATE_OFDM6); + SET_BCN_UPD_RATE(skb->data, beacon_rate); skb_put_data(skb, skb_beacon->data, skb_beacon->len); dev_kfree_skb_any(skb_beacon); @@ -1077,10 +1314,11 @@ int rtw89_fw_h2c_update_beacon(struct rtw89_dev *rtwdev, H2C_FUNC_MAC_BCN_UPD, 0, 1, bcn_total_len); - if (rtw89_h2c_tx(rtwdev, skb, false)) { + ret = rtw89_h2c_tx(rtwdev, skb, false); + if (ret) { rtw89_err(rtwdev, "failed to send h2c\n"); dev_kfree_skb_any(skb); - return -EBUSY; + return ret; } return 0; @@ -1095,6 +1333,7 @@ int rtw89_fw_h2c_role_maintain(struct rtw89_dev *rtwdev, struct sk_buff *skb; u8 mac_id = rtwsta ? rtwsta->mac_id : rtwvif->mac_id; u8 self_role; + int ret; if (rtwvif->net_type == RTW89_NET_TYPE_AP_MODE) { if (rtwsta) @@ -1121,7 +1360,8 @@ int rtw89_fw_h2c_role_maintain(struct rtw89_dev *rtwdev, H2C_FUNC_MAC_FWROLE_MAINTAIN, 0, 1, H2C_ROLE_MAINTAIN_LEN); - if (rtw89_h2c_tx(rtwdev, skb, false)) { + ret = rtw89_h2c_tx(rtwdev, skb, false); + if (ret) { rtw89_err(rtwdev, "failed to send h2c\n"); goto fail; } @@ -1130,7 +1370,7 @@ int rtw89_fw_h2c_role_maintain(struct rtw89_dev *rtwdev, fail: dev_kfree_skb_any(skb); - return -EBUSY; + return ret; } #define H2C_JOIN_INFO_LEN 4 @@ -1141,6 +1381,7 @@ int rtw89_fw_h2c_join_info(struct rtw89_dev *rtwdev, struct rtw89_vif *rtwvif, u8 mac_id = rtwsta ? rtwsta->mac_id : rtwvif->mac_id; u8 self_role = rtwvif->self_role; u8 net_type = rtwvif->net_type; + int ret; if (net_type == RTW89_NET_TYPE_AP_MODE && rtwsta) { self_role = RTW89_SELF_ROLE_AP_CLIENT; @@ -1172,7 +1413,8 @@ int rtw89_fw_h2c_join_info(struct rtw89_dev *rtwdev, struct rtw89_vif *rtwvif, H2C_FUNC_MAC_JOININFO, 0, 1, H2C_JOIN_INFO_LEN); - if (rtw89_h2c_tx(rtwdev, skb, false)) { + ret = rtw89_h2c_tx(rtwdev, skb, false); + if (ret) { rtw89_err(rtwdev, "failed to send h2c\n"); goto fail; } @@ -1181,7 +1423,7 @@ int rtw89_fw_h2c_join_info(struct rtw89_dev *rtwdev, struct rtw89_vif *rtwvif, fail: dev_kfree_skb_any(skb); - return -EBUSY; + return ret; } int rtw89_fw_h2c_macid_pause(struct rtw89_dev *rtwdev, u8 sh, u8 grp, @@ -1190,6 +1432,7 @@ int rtw89_fw_h2c_macid_pause(struct rtw89_dev *rtwdev, u8 sh, u8 grp, struct rtw89_fw_macid_pause_grp h2c = {{0}}; u8 len = sizeof(struct rtw89_fw_macid_pause_grp); struct sk_buff *skb; + int ret; skb = rtw89_fw_h2c_alloc_skb_with_hdr(rtwdev, H2C_JOIN_INFO_LEN); if (!skb) { @@ -1206,7 +1449,8 @@ int rtw89_fw_h2c_macid_pause(struct rtw89_dev *rtwdev, u8 sh, u8 grp, H2C_FUNC_MAC_MACID_PAUSE, 1, 0, len); - if (rtw89_h2c_tx(rtwdev, skb, false)) { + ret = rtw89_h2c_tx(rtwdev, skb, false); + if (ret) { rtw89_err(rtwdev, "failed to send h2c\n"); goto fail; } @@ -1215,7 +1459,7 @@ int rtw89_fw_h2c_macid_pause(struct rtw89_dev *rtwdev, u8 sh, u8 grp, fail: dev_kfree_skb_any(skb); - return -EBUSY; + return ret; } #define H2C_EDCA_LEN 12 @@ -1223,6 +1467,7 @@ int rtw89_fw_h2c_set_edca(struct rtw89_dev *rtwdev, struct rtw89_vif *rtwvif, u8 ac, u32 val) { struct sk_buff *skb; + int ret; skb = rtw89_fw_h2c_alloc_skb_with_hdr(rtwdev, H2C_EDCA_LEN); if (!skb) { @@ -1241,7 +1486,8 @@ int rtw89_fw_h2c_set_edca(struct rtw89_dev *rtwdev, struct rtw89_vif *rtwvif, H2C_FUNC_USR_EDCA, 0, 1, H2C_EDCA_LEN); - if (rtw89_h2c_tx(rtwdev, skb, false)) { + ret = rtw89_h2c_tx(rtwdev, skb, false); + if (ret) { rtw89_err(rtwdev, "failed to send h2c\n"); goto fail; } @@ -1250,7 +1496,47 @@ int rtw89_fw_h2c_set_edca(struct rtw89_dev *rtwdev, struct rtw89_vif *rtwvif, fail: dev_kfree_skb_any(skb); - return -EBUSY; + return ret; +} + +#define H2C_TSF32_TOGL_LEN 4 +int rtw89_fw_h2c_tsf32_toggle(struct rtw89_dev *rtwdev, struct rtw89_vif *rtwvif, + bool en) +{ + struct sk_buff *skb; + u16 early_us = en ? 2000 : 0; + u8 *cmd; + int ret; + + skb = rtw89_fw_h2c_alloc_skb_with_hdr(rtwdev, H2C_TSF32_TOGL_LEN); + if (!skb) { + rtw89_err(rtwdev, "failed to alloc skb for h2c p2p act\n"); + return -ENOMEM; + } + skb_put(skb, H2C_TSF32_TOGL_LEN); + cmd = skb->data; + + RTW89_SET_FWCMD_TSF32_TOGL_BAND(cmd, rtwvif->mac_idx); + RTW89_SET_FWCMD_TSF32_TOGL_EN(cmd, en); + RTW89_SET_FWCMD_TSF32_TOGL_PORT(cmd, rtwvif->port); + RTW89_SET_FWCMD_TSF32_TOGL_EARLY(cmd, early_us); + + rtw89_h2c_pkt_set_hdr(rtwdev, skb, FWCMD_TYPE_H2C, + H2C_CAT_MAC, H2C_CL_MAC_FW_OFLD, + H2C_FUNC_TSF32_TOGL, 0, 0, + H2C_TSF32_TOGL_LEN); + + ret = rtw89_h2c_tx(rtwdev, skb, false); + if (ret) { + rtw89_err(rtwdev, "failed to send h2c\n"); + goto fail; + } + + return 0; +fail: + dev_kfree_skb_any(skb); + + return ret; } #define H2C_OFLD_CFG_LEN 8 @@ -1258,6 +1544,7 @@ int rtw89_fw_h2c_set_ofld_cfg(struct rtw89_dev *rtwdev) { static const u8 cfg[] = {0x09, 0x00, 0x00, 0x00, 0x5e, 0x00, 0x00, 0x00}; struct sk_buff *skb; + int ret; skb = rtw89_fw_h2c_alloc_skb_with_hdr(rtwdev, H2C_OFLD_CFG_LEN); if (!skb) { @@ -1271,7 +1558,8 @@ int rtw89_fw_h2c_set_ofld_cfg(struct rtw89_dev *rtwdev) H2C_FUNC_OFLD_CFG, 0, 1, H2C_OFLD_CFG_LEN); - if (rtw89_h2c_tx(rtwdev, skb, false)) { + ret = rtw89_h2c_tx(rtwdev, skb, false); + if (ret) { rtw89_err(rtwdev, "failed to send h2c\n"); goto fail; } @@ -1280,7 +1568,7 @@ int rtw89_fw_h2c_set_ofld_cfg(struct rtw89_dev *rtwdev) fail: dev_kfree_skb_any(skb); - return -EBUSY; + return ret; } #define H2C_RA_LEN 16 @@ -1288,6 +1576,7 @@ int rtw89_fw_h2c_ra(struct rtw89_dev *rtwdev, struct rtw89_ra_info *ra, bool csi { struct sk_buff *skb; u8 *cmd; + int ret; skb = rtw89_fw_h2c_alloc_skb_with_hdr(rtwdev, H2C_RA_LEN); if (!skb) { @@ -1318,6 +1607,8 @@ int rtw89_fw_h2c_ra(struct rtw89_dev *rtwdev, struct rtw89_ra_info *ra, bool csi RTW89_SET_FWCMD_RA_MASK_2(cmd, FIELD_GET(MASKBYTE2, ra->ra_mask)); RTW89_SET_FWCMD_RA_MASK_3(cmd, FIELD_GET(MASKBYTE3, ra->ra_mask)); RTW89_SET_FWCMD_RA_MASK_4(cmd, FIELD_GET(MASKBYTE4, ra->ra_mask)); + RTW89_SET_FWCMD_RA_FIX_GILTF_EN(cmd, ra->fix_giltf_en); + RTW89_SET_FWCMD_RA_FIX_GILTF(cmd, ra->fix_giltf); if (csi) { RTW89_SET_FWCMD_RA_BFEE_CSI_CTL(cmd, 1); @@ -1336,7 +1627,8 @@ int rtw89_fw_h2c_ra(struct rtw89_dev *rtwdev, struct rtw89_ra_info *ra, bool csi H2C_FUNC_OUTSRC_RA_MACIDCFG, 0, 0, H2C_RA_LEN); - if (rtw89_h2c_tx(rtwdev, skb, false)) { + ret = rtw89_h2c_tx(rtwdev, skb, false); + if (ret) { rtw89_err(rtwdev, "failed to send h2c\n"); goto fail; } @@ -1345,7 +1637,7 @@ int rtw89_fw_h2c_ra(struct rtw89_dev *rtwdev, struct rtw89_ra_info *ra, bool csi fail: dev_kfree_skb_any(skb); - return -EBUSY; + return ret; } #define H2C_LEN_CXDRVHDR 2 @@ -1359,6 +1651,7 @@ int rtw89_fw_h2c_cxdrv_init(struct rtw89_dev *rtwdev) struct rtw89_btc_ant_info *ant = &module->ant; struct sk_buff *skb; u8 *cmd; + int ret; skb = rtw89_fw_h2c_alloc_skb_with_hdr(rtwdev, H2C_LEN_CXDRVINFO_INIT); if (!skb) { @@ -1395,7 +1688,8 @@ int rtw89_fw_h2c_cxdrv_init(struct rtw89_dev *rtwdev) SET_DRV_INFO, 0, 0, H2C_LEN_CXDRVINFO_INIT); - if (rtw89_h2c_tx(rtwdev, skb, false)) { + ret = rtw89_h2c_tx(rtwdev, skb, false); + if (ret) { rtw89_err(rtwdev, "failed to send h2c\n"); goto fail; } @@ -1404,10 +1698,15 @@ int rtw89_fw_h2c_cxdrv_init(struct rtw89_dev *rtwdev) fail: dev_kfree_skb_any(skb); - return -EBUSY; + return ret; } +#define PORT_DATA_OFFSET 4 +#define H2C_LEN_CXDRVINFO_ROLE_DBCC_LEN 12 #define H2C_LEN_CXDRVINFO_ROLE (4 + 12 * RTW89_PORT_NUM + H2C_LEN_CXDRVHDR) +#define H2C_LEN_CXDRVINFO_ROLE_V1 (4 + 16 * RTW89_PORT_NUM + \ + H2C_LEN_CXDRVINFO_ROLE_DBCC_LEN + \ + H2C_LEN_CXDRVHDR) int rtw89_fw_h2c_cxdrv_role(struct rtw89_dev *rtwdev) { struct rtw89_btc *btc = &rtwdev->btc; @@ -1416,7 +1715,9 @@ int rtw89_fw_h2c_cxdrv_role(struct rtw89_dev *rtwdev) struct rtw89_btc_wl_role_info_bpos *bpos = &role_info->role_map.role; struct rtw89_btc_wl_active_role *active = role_info->active_role; struct sk_buff *skb; + u8 offset = 0; u8 *cmd; + int ret; int i; skb = rtw89_fw_h2c_alloc_skb_with_hdr(rtwdev, H2C_LEN_CXDRVINFO_ROLE); @@ -1447,19 +1748,19 @@ int rtw89_fw_h2c_cxdrv_role(struct rtw89_dev *rtwdev) RTW89_SET_FWCMD_CXROLE_ROLE_NAN(cmd, bpos->nan); for (i = 0; i < RTW89_PORT_NUM; i++, active++) { - RTW89_SET_FWCMD_CXROLE_ACT_CONNECTED(cmd, active->connected, i); - RTW89_SET_FWCMD_CXROLE_ACT_PID(cmd, active->pid, i); - RTW89_SET_FWCMD_CXROLE_ACT_PHY(cmd, active->phy, i); - RTW89_SET_FWCMD_CXROLE_ACT_NOA(cmd, active->noa, i); - RTW89_SET_FWCMD_CXROLE_ACT_BAND(cmd, active->band, i); - RTW89_SET_FWCMD_CXROLE_ACT_CLIENT_PS(cmd, active->client_ps, i); - RTW89_SET_FWCMD_CXROLE_ACT_BW(cmd, active->bw, i); - RTW89_SET_FWCMD_CXROLE_ACT_ROLE(cmd, active->role, i); - RTW89_SET_FWCMD_CXROLE_ACT_CH(cmd, active->ch, i); - RTW89_SET_FWCMD_CXROLE_ACT_TX_LVL(cmd, active->tx_lvl, i); - RTW89_SET_FWCMD_CXROLE_ACT_RX_LVL(cmd, active->rx_lvl, i); - RTW89_SET_FWCMD_CXROLE_ACT_TX_RATE(cmd, active->tx_rate, i); - RTW89_SET_FWCMD_CXROLE_ACT_RX_RATE(cmd, active->rx_rate, i); + RTW89_SET_FWCMD_CXROLE_ACT_CONNECTED(cmd, active->connected, i, offset); + RTW89_SET_FWCMD_CXROLE_ACT_PID(cmd, active->pid, i, offset); + RTW89_SET_FWCMD_CXROLE_ACT_PHY(cmd, active->phy, i, offset); + RTW89_SET_FWCMD_CXROLE_ACT_NOA(cmd, active->noa, i, offset); + RTW89_SET_FWCMD_CXROLE_ACT_BAND(cmd, active->band, i, offset); + RTW89_SET_FWCMD_CXROLE_ACT_CLIENT_PS(cmd, active->client_ps, i, offset); + RTW89_SET_FWCMD_CXROLE_ACT_BW(cmd, active->bw, i, offset); + RTW89_SET_FWCMD_CXROLE_ACT_ROLE(cmd, active->role, i, offset); + RTW89_SET_FWCMD_CXROLE_ACT_CH(cmd, active->ch, i, offset); + RTW89_SET_FWCMD_CXROLE_ACT_TX_LVL(cmd, active->tx_lvl, i, offset); + RTW89_SET_FWCMD_CXROLE_ACT_RX_LVL(cmd, active->rx_lvl, i, offset); + RTW89_SET_FWCMD_CXROLE_ACT_TX_RATE(cmd, active->tx_rate, i, offset); + RTW89_SET_FWCMD_CXROLE_ACT_RX_RATE(cmd, active->rx_rate, i, offset); } rtw89_h2c_pkt_set_hdr(rtwdev, skb, FWCMD_TYPE_H2C, @@ -1467,7 +1768,8 @@ int rtw89_fw_h2c_cxdrv_role(struct rtw89_dev *rtwdev) SET_DRV_INFO, 0, 0, H2C_LEN_CXDRVINFO_ROLE); - if (rtw89_h2c_tx(rtwdev, skb, false)) { + ret = rtw89_h2c_tx(rtwdev, skb, false); + if (ret) { rtw89_err(rtwdev, "failed to send h2c\n"); goto fail; } @@ -1476,16 +1778,101 @@ int rtw89_fw_h2c_cxdrv_role(struct rtw89_dev *rtwdev) fail: dev_kfree_skb_any(skb); - return -EBUSY; + return ret; +} + +int rtw89_fw_h2c_cxdrv_role_v1(struct rtw89_dev *rtwdev) +{ + struct rtw89_btc *btc = &rtwdev->btc; + struct rtw89_btc_wl_info *wl = &btc->cx.wl; + struct rtw89_btc_wl_role_info_v1 *role_info = &wl->role_info_v1; + struct rtw89_btc_wl_role_info_bpos *bpos = &role_info->role_map.role; + struct rtw89_btc_wl_active_role_v1 *active = role_info->active_role_v1; + struct sk_buff *skb; + u8 *cmd, offset; + int ret; + int i; + + skb = rtw89_fw_h2c_alloc_skb_with_hdr(rtwdev, H2C_LEN_CXDRVINFO_ROLE_V1); + if (!skb) { + rtw89_err(rtwdev, "failed to alloc skb for h2c cxdrv_role\n"); + return -ENOMEM; + } + skb_put(skb, H2C_LEN_CXDRVINFO_ROLE_V1); + cmd = skb->data; + + RTW89_SET_FWCMD_CXHDR_TYPE(cmd, CXDRVINFO_ROLE); + RTW89_SET_FWCMD_CXHDR_LEN(cmd, H2C_LEN_CXDRVINFO_ROLE_V1 - H2C_LEN_CXDRVHDR); + + RTW89_SET_FWCMD_CXROLE_CONNECT_CNT(cmd, role_info->connect_cnt); + RTW89_SET_FWCMD_CXROLE_LINK_MODE(cmd, role_info->link_mode); + + RTW89_SET_FWCMD_CXROLE_ROLE_NONE(cmd, bpos->none); + RTW89_SET_FWCMD_CXROLE_ROLE_STA(cmd, bpos->station); + RTW89_SET_FWCMD_CXROLE_ROLE_AP(cmd, bpos->ap); + RTW89_SET_FWCMD_CXROLE_ROLE_VAP(cmd, bpos->vap); + RTW89_SET_FWCMD_CXROLE_ROLE_ADHOC(cmd, bpos->adhoc); + RTW89_SET_FWCMD_CXROLE_ROLE_ADHOC_MASTER(cmd, bpos->adhoc_master); + RTW89_SET_FWCMD_CXROLE_ROLE_MESH(cmd, bpos->mesh); + RTW89_SET_FWCMD_CXROLE_ROLE_MONITOR(cmd, bpos->moniter); + RTW89_SET_FWCMD_CXROLE_ROLE_P2P_DEV(cmd, bpos->p2p_device); + RTW89_SET_FWCMD_CXROLE_ROLE_P2P_GC(cmd, bpos->p2p_gc); + RTW89_SET_FWCMD_CXROLE_ROLE_P2P_GO(cmd, bpos->p2p_go); + RTW89_SET_FWCMD_CXROLE_ROLE_NAN(cmd, bpos->nan); + + offset = PORT_DATA_OFFSET; + for (i = 0; i < RTW89_PORT_NUM; i++, active++) { + RTW89_SET_FWCMD_CXROLE_ACT_CONNECTED(cmd, active->connected, i, offset); + RTW89_SET_FWCMD_CXROLE_ACT_PID(cmd, active->pid, i, offset); + RTW89_SET_FWCMD_CXROLE_ACT_PHY(cmd, active->phy, i, offset); + RTW89_SET_FWCMD_CXROLE_ACT_NOA(cmd, active->noa, i, offset); + RTW89_SET_FWCMD_CXROLE_ACT_BAND(cmd, active->band, i, offset); + RTW89_SET_FWCMD_CXROLE_ACT_CLIENT_PS(cmd, active->client_ps, i, offset); + RTW89_SET_FWCMD_CXROLE_ACT_BW(cmd, active->bw, i, offset); + RTW89_SET_FWCMD_CXROLE_ACT_ROLE(cmd, active->role, i, offset); + RTW89_SET_FWCMD_CXROLE_ACT_CH(cmd, active->ch, i, offset); + RTW89_SET_FWCMD_CXROLE_ACT_TX_LVL(cmd, active->tx_lvl, i, offset); + RTW89_SET_FWCMD_CXROLE_ACT_RX_LVL(cmd, active->rx_lvl, i, offset); + RTW89_SET_FWCMD_CXROLE_ACT_TX_RATE(cmd, active->tx_rate, i, offset); + RTW89_SET_FWCMD_CXROLE_ACT_RX_RATE(cmd, active->rx_rate, i, offset); + RTW89_SET_FWCMD_CXROLE_ACT_NOA_DUR(cmd, active->noa_duration, i, offset); + } + + offset = H2C_LEN_CXDRVINFO_ROLE_V1 - H2C_LEN_CXDRVINFO_ROLE_DBCC_LEN; + RTW89_SET_FWCMD_CXROLE_MROLE_TYPE(cmd, role_info->mrole_type, offset); + RTW89_SET_FWCMD_CXROLE_MROLE_NOA(cmd, role_info->mrole_noa_duration, offset); + RTW89_SET_FWCMD_CXROLE_DBCC_EN(cmd, role_info->dbcc_en, offset); + RTW89_SET_FWCMD_CXROLE_DBCC_CHG(cmd, role_info->dbcc_chg, offset); + RTW89_SET_FWCMD_CXROLE_DBCC_2G_PHY(cmd, role_info->dbcc_2g_phy, offset); + RTW89_SET_FWCMD_CXROLE_LINK_MODE_CHG(cmd, role_info->link_mode_chg, offset); + + rtw89_h2c_pkt_set_hdr(rtwdev, skb, FWCMD_TYPE_H2C, + H2C_CAT_OUTSRC, BTFC_SET, + SET_DRV_INFO, 0, 0, + H2C_LEN_CXDRVINFO_ROLE_V1); + + ret = rtw89_h2c_tx(rtwdev, skb, false); + if (ret) { + rtw89_err(rtwdev, "failed to send h2c\n"); + goto fail; + } + + return 0; +fail: + dev_kfree_skb_any(skb); + + return ret; } #define H2C_LEN_CXDRVINFO_CTRL (4 + H2C_LEN_CXDRVHDR) int rtw89_fw_h2c_cxdrv_ctrl(struct rtw89_dev *rtwdev) { + const struct rtw89_chip_info *chip = rtwdev->chip; struct rtw89_btc *btc = &rtwdev->btc; struct rtw89_btc_ctrl *ctrl = &btc->ctrl; struct sk_buff *skb; u8 *cmd; + int ret; skb = rtw89_fw_h2c_alloc_skb_with_hdr(rtwdev, H2C_LEN_CXDRVINFO_CTRL); if (!skb) { @@ -1501,14 +1888,16 @@ int rtw89_fw_h2c_cxdrv_ctrl(struct rtw89_dev *rtwdev) RTW89_SET_FWCMD_CXCTRL_MANUAL(cmd, ctrl->manual); RTW89_SET_FWCMD_CXCTRL_IGNORE_BT(cmd, ctrl->igno_bt); RTW89_SET_FWCMD_CXCTRL_ALWAYS_FREERUN(cmd, ctrl->always_freerun); - RTW89_SET_FWCMD_CXCTRL_TRACE_STEP(cmd, ctrl->trace_step); + if (chip->chip_id == RTL8852A) + RTW89_SET_FWCMD_CXCTRL_TRACE_STEP(cmd, ctrl->trace_step); rtw89_h2c_pkt_set_hdr(rtwdev, skb, FWCMD_TYPE_H2C, H2C_CAT_OUTSRC, BTFC_SET, SET_DRV_INFO, 0, 0, H2C_LEN_CXDRVINFO_CTRL); - if (rtw89_h2c_tx(rtwdev, skb, false)) { + ret = rtw89_h2c_tx(rtwdev, skb, false); + if (ret) { rtw89_err(rtwdev, "failed to send h2c\n"); goto fail; } @@ -1517,7 +1906,7 @@ int rtw89_fw_h2c_cxdrv_ctrl(struct rtw89_dev *rtwdev) fail: dev_kfree_skb_any(skb); - return -EBUSY; + return ret; } #define H2C_LEN_CXDRVINFO_RFK (4 + H2C_LEN_CXDRVHDR) @@ -1528,6 +1917,7 @@ int rtw89_fw_h2c_cxdrv_rfk(struct rtw89_dev *rtwdev) struct rtw89_btc_wl_rfk_info *rfk_info = &wl->rfk_info; struct sk_buff *skb; u8 *cmd; + int ret; skb = rtw89_fw_h2c_alloc_skb_with_hdr(rtwdev, H2C_LEN_CXDRVINFO_RFK); if (!skb) { @@ -1551,7 +1941,8 @@ int rtw89_fw_h2c_cxdrv_rfk(struct rtw89_dev *rtwdev) SET_DRV_INFO, 0, 0, H2C_LEN_CXDRVINFO_RFK); - if (rtw89_h2c_tx(rtwdev, skb, false)) { + ret = rtw89_h2c_tx(rtwdev, skb, false); + if (ret) { rtw89_err(rtwdev, "failed to send h2c\n"); goto fail; } @@ -1560,7 +1951,7 @@ int rtw89_fw_h2c_cxdrv_rfk(struct rtw89_dev *rtwdev) fail: dev_kfree_skb_any(skb); - return -EBUSY; + return ret; } #define H2C_LEN_PKT_OFLD 4 @@ -1568,6 +1959,7 @@ int rtw89_fw_h2c_del_pkt_offload(struct rtw89_dev *rtwdev, u8 id) { struct sk_buff *skb; u8 *cmd; + int ret; skb = rtw89_fw_h2c_alloc_skb_with_hdr(rtwdev, H2C_LEN_PKT_OFLD); if (!skb) { @@ -1585,7 +1977,8 @@ int rtw89_fw_h2c_del_pkt_offload(struct rtw89_dev *rtwdev, u8 id) H2C_FUNC_PACKET_OFLD, 1, 1, H2C_LEN_PKT_OFLD); - if (rtw89_h2c_tx(rtwdev, skb, false)) { + ret = rtw89_h2c_tx(rtwdev, skb, false); + if (ret) { rtw89_err(rtwdev, "failed to send h2c\n"); goto fail; } @@ -1594,7 +1987,7 @@ int rtw89_fw_h2c_del_pkt_offload(struct rtw89_dev *rtwdev, u8 id) fail: dev_kfree_skb_any(skb); - return -EBUSY; + return ret; } int rtw89_fw_h2c_add_pkt_offload(struct rtw89_dev *rtwdev, u8 *id, @@ -1603,6 +1996,7 @@ int rtw89_fw_h2c_add_pkt_offload(struct rtw89_dev *rtwdev, u8 *id, struct sk_buff *skb; u8 *cmd; u8 alloc_id; + int ret; alloc_id = rtw89_core_acquire_bit_map(rtwdev->pkt_offload, RTW89_MAX_PKT_OFLD_NUM); @@ -1629,7 +2023,8 @@ int rtw89_fw_h2c_add_pkt_offload(struct rtw89_dev *rtwdev, u8 *id, H2C_FUNC_PACKET_OFLD, 1, 1, H2C_LEN_PKT_OFLD + skb_ofld->len); - if (rtw89_h2c_tx(rtwdev, skb, false)) { + ret = rtw89_h2c_tx(rtwdev, skb, false); + if (ret) { rtw89_err(rtwdev, "failed to send h2c\n"); goto fail; } @@ -1638,7 +2033,7 @@ int rtw89_fw_h2c_add_pkt_offload(struct rtw89_dev *rtwdev, u8 *id, fail: dev_kfree_skb_any(skb); - return -EBUSY; + return ret; } #define H2C_LEN_SCAN_LIST_OFFLOAD 4 @@ -1649,6 +2044,7 @@ int rtw89_fw_h2c_scan_list_offload(struct rtw89_dev *rtwdev, int len, struct sk_buff *skb; int skb_len = H2C_LEN_SCAN_LIST_OFFLOAD + len * RTW89_MAC_CHINFO_SIZE; u8 *cmd; + int ret; skb = rtw89_fw_h2c_alloc_skb_with_hdr(rtwdev, skb_len); if (!skb) { @@ -1693,7 +2089,8 @@ int rtw89_fw_h2c_scan_list_offload(struct rtw89_dev *rtwdev, int len, H2C_CAT_MAC, H2C_CL_MAC_FW_OFLD, H2C_FUNC_ADD_SCANOFLD_CH, 1, 1, skb_len); - if (rtw89_h2c_tx(rtwdev, skb, false)) { + ret = rtw89_h2c_tx(rtwdev, skb, false); + if (ret) { rtw89_err(rtwdev, "failed to send h2c\n"); goto fail; } @@ -1702,10 +2099,10 @@ int rtw89_fw_h2c_scan_list_offload(struct rtw89_dev *rtwdev, int len, fail: dev_kfree_skb_any(skb); - return -EBUSY; + return ret; } -#define H2C_LEN_SCAN_OFFLOAD 20 +#define H2C_LEN_SCAN_OFFLOAD 28 int rtw89_fw_h2c_scan_offload(struct rtw89_dev *rtwdev, struct rtw89_scan_option *option, struct rtw89_vif *rtwvif) @@ -1713,6 +2110,7 @@ int rtw89_fw_h2c_scan_offload(struct rtw89_dev *rtwdev, struct rtw89_hw_scan_info *scan_info = &rtwdev->scan_info; struct sk_buff *skb; u8 *cmd; + int ret; skb = rtw89_fw_h2c_alloc_skb_with_hdr(rtwdev, H2C_LEN_SCAN_OFFLOAD); if (!skb) { @@ -1736,6 +2134,8 @@ int rtw89_fw_h2c_scan_offload(struct rtw89_dev *rtwdev, scan_info->op_pri_ch); RTW89_SET_FWCMD_SCANOFLD_TARGET_CENTRAL_CH(cmd, scan_info->op_chan); + RTW89_SET_FWCMD_SCANOFLD_TARGET_CH_BAND(cmd, + scan_info->op_band); } rtw89_h2c_pkt_set_hdr(rtwdev, skb, FWCMD_TYPE_H2C, @@ -1743,7 +2143,8 @@ int rtw89_fw_h2c_scan_offload(struct rtw89_dev *rtwdev, H2C_FUNC_SCANOFLD, 1, 1, H2C_LEN_SCAN_OFFLOAD); - if (rtw89_h2c_tx(rtwdev, skb, false)) { + ret = rtw89_h2c_tx(rtwdev, skb, false); + if (ret) { rtw89_err(rtwdev, "failed to send h2c\n"); goto fail; } @@ -1752,7 +2153,7 @@ int rtw89_fw_h2c_scan_offload(struct rtw89_dev *rtwdev, fail: dev_kfree_skb_any(skb); - return -EBUSY; + return ret; } int rtw89_fw_h2c_rf_reg(struct rtw89_dev *rtwdev, @@ -1762,6 +2163,7 @@ int rtw89_fw_h2c_rf_reg(struct rtw89_dev *rtwdev, struct sk_buff *skb; u8 class = info->rf_path == RF_PATH_A ? H2C_CL_OUTSRC_RF_REG_A : H2C_CL_OUTSRC_RF_REG_B; + int ret; skb = rtw89_fw_h2c_alloc_skb_with_hdr(rtwdev, len); if (!skb) { @@ -1774,7 +2176,8 @@ int rtw89_fw_h2c_rf_reg(struct rtw89_dev *rtwdev, H2C_CAT_OUTSRC, class, page, 0, 0, len); - if (rtw89_h2c_tx(rtwdev, skb, false)) { + ret = rtw89_h2c_tx(rtwdev, skb, false); + if (ret) { rtw89_err(rtwdev, "failed to send h2c\n"); goto fail; } @@ -1783,14 +2186,16 @@ int rtw89_fw_h2c_rf_reg(struct rtw89_dev *rtwdev, fail: dev_kfree_skb_any(skb); - return -EBUSY; + return ret; } int rtw89_fw_h2c_rf_ntfy_mcc(struct rtw89_dev *rtwdev) { + const struct rtw89_chan *chan = rtw89_chan_get(rtwdev, RTW89_SUB_ENTITY_0); struct rtw89_mcc_info *mcc_info = &rtwdev->mcc; struct rtw89_fw_h2c_rf_get_mccch *mccch; struct sk_buff *skb; + int ret; skb = rtw89_fw_h2c_alloc_skb_with_hdr(rtwdev, sizeof(*mccch)); if (!skb) { @@ -1804,15 +2209,16 @@ int rtw89_fw_h2c_rf_ntfy_mcc(struct rtw89_dev *rtwdev) mccch->ch_1 = cpu_to_le32(mcc_info->ch[1]); mccch->band_0 = cpu_to_le32(mcc_info->band[0]); mccch->band_1 = cpu_to_le32(mcc_info->band[1]); - mccch->current_channel = cpu_to_le32(rtwdev->hal.current_channel); - mccch->current_band_type = cpu_to_le32(rtwdev->hal.current_band_type); + mccch->current_channel = cpu_to_le32(chan->channel); + mccch->current_band_type = cpu_to_le32(chan->band_type); rtw89_h2c_pkt_set_hdr(rtwdev, skb, FWCMD_TYPE_H2C, H2C_CAT_OUTSRC, H2C_CL_OUTSRC_RF_FW_NOTIFY, H2C_FUNC_OUTSRC_RF_GET_MCCCH, 0, 0, sizeof(*mccch)); - if (rtw89_h2c_tx(rtwdev, skb, false)) { + ret = rtw89_h2c_tx(rtwdev, skb, false); + if (ret) { rtw89_err(rtwdev, "failed to send h2c\n"); goto fail; } @@ -1821,7 +2227,7 @@ int rtw89_fw_h2c_rf_ntfy_mcc(struct rtw89_dev *rtwdev) fail: dev_kfree_skb_any(skb); - return -EBUSY; + return ret; } EXPORT_SYMBOL(rtw89_fw_h2c_rf_ntfy_mcc); @@ -1830,6 +2236,7 @@ int rtw89_fw_h2c_raw_with_hdr(struct rtw89_dev *rtwdev, bool rack, bool dack) { struct sk_buff *skb; + int ret; skb = rtw89_fw_h2c_alloc_skb_with_hdr(rtwdev, len); if (!skb) { @@ -1842,7 +2249,8 @@ int rtw89_fw_h2c_raw_with_hdr(struct rtw89_dev *rtwdev, H2C_CAT_OUTSRC, h2c_class, h2c_func, rack, dack, len); - if (rtw89_h2c_tx(rtwdev, skb, false)) { + ret = rtw89_h2c_tx(rtwdev, skb, false); + if (ret) { rtw89_err(rtwdev, "failed to send h2c\n"); goto fail; } @@ -1851,12 +2259,13 @@ int rtw89_fw_h2c_raw_with_hdr(struct rtw89_dev *rtwdev, fail: dev_kfree_skb_any(skb); - return -EBUSY; + return ret; } int rtw89_fw_h2c_raw(struct rtw89_dev *rtwdev, const u8 *buf, u16 len) { struct sk_buff *skb; + int ret; skb = rtw89_fw_h2c_alloc_skb_no_hdr(rtwdev, len); if (!skb) { @@ -1865,7 +2274,8 @@ int rtw89_fw_h2c_raw(struct rtw89_dev *rtwdev, const u8 *buf, u16 len) } skb_put_data(skb, buf, len); - if (rtw89_h2c_tx(rtwdev, skb, false)) { + ret = rtw89_h2c_tx(rtwdev, skb, false); + if (ret) { rtw89_err(rtwdev, "failed to send h2c\n"); goto fail; } @@ -1874,7 +2284,7 @@ int rtw89_fw_h2c_raw(struct rtw89_dev *rtwdev, const u8 *buf, u16 len) fail: dev_kfree_skb_any(skb); - return -EBUSY; + return ret; } void rtw89_fw_send_all_early_h2c(struct rtw89_dev *rtwdev) @@ -2169,7 +2579,7 @@ static void rtw89_hw_scan_add_chan(struct rtw89_dev *rtwdev, int chan_type, if (ssid_num) { ch_info->num_pkt = ssid_num; - band = ch_info->ch_band; + band = rtw89_hw_to_nl80211_band(ch_info->ch_band); list_for_each_entry(info, &scan_info->pkt_list[band], list) { ch_info->probe_id = info->id; @@ -2211,13 +2621,16 @@ static int rtw89_hw_scan_add_chan_list(struct rtw89_dev *rtwdev, struct ieee80211_channel *channel; struct list_head chan_list; bool random_seq = req->flags & NL80211_SCAN_FLAG_RANDOM_SN; - int list_len = req->n_channels, off_chan_time = 0; + int list_len, off_chan_time = 0; enum rtw89_chan_type type; - int ret = 0, i; + int ret = 0; + u32 idx; INIT_LIST_HEAD(&chan_list); - for (i = 0; i < req->n_channels; i++) { - channel = req->channels[i]; + for (idx = rtwdev->scan_info.last_chan_idx, list_len = 0; + idx < req->n_channels && list_len < RTW89_SCAN_LIST_LIMIT; + idx++, list_len++) { + channel = req->channels[idx]; ch_info = kzalloc(sizeof(*ch_info), GFP_KERNEL); if (!ch_info) { ret = -ENOMEM; @@ -2226,7 +2639,7 @@ static int rtw89_hw_scan_add_chan_list(struct rtw89_dev *rtwdev, ch_info->period = req->duration_mandatory ? req->duration : RTW89_CHANNEL_TIME; - ch_info->ch_band = channel->band; + ch_info->ch_band = rtw89_nl80211_to_hw_band(channel->band); ch_info->central_ch = channel->hw_value; ch_info->pri_ch = channel->hw_value; ch_info->rand_seq_num = random_seq; @@ -2258,6 +2671,7 @@ static int rtw89_hw_scan_add_chan_list(struct rtw89_dev *rtwdev, list_add_tail(&ch_info->list, &chan_list); off_chan_time += ch_info->period; } + rtwdev->scan_info.last_chan_idx = idx; ret = rtw89_fw_h2c_scan_list_offload(rtwdev, list_len, &chan_list); out: @@ -2289,9 +2703,11 @@ void rtw89_hw_scan_start(struct rtw89_dev *rtwdev, struct ieee80211_vif *vif, { struct rtw89_vif *rtwvif = (struct rtw89_vif *)vif->drv_priv; struct cfg80211_scan_request *req = &scan_req->req; + u32 rx_fltr = rtwdev->hal.rx_fltr; u8 mac_addr[ETH_ALEN]; rtwdev->scan_info.scanning_vif = vif; + rtwdev->scan_info.last_chan_idx = 0; rtwvif->scan_ies = &scan_req->ies; rtwvif->scan_req = req; ieee80211_stop_queues(rtwdev->hw); @@ -2303,13 +2719,13 @@ void rtw89_hw_scan_start(struct rtw89_dev *rtwdev, struct ieee80211_vif *vif, ether_addr_copy(mac_addr, vif->addr); rtw89_core_scan_start(rtwdev, rtwvif, mac_addr, true); - rtwdev->hal.rx_fltr &= ~B_AX_A_BCN_CHK_EN; - rtwdev->hal.rx_fltr &= ~B_AX_A_BC; - rtwdev->hal.rx_fltr &= ~B_AX_A_A1_MATCH; + rx_fltr &= ~B_AX_A_BCN_CHK_EN; + rx_fltr &= ~B_AX_A_BC; + rx_fltr &= ~B_AX_A_A1_MATCH; rtw89_write32_mask(rtwdev, rtw89_mac_reg_by_idx(R_AX_RX_FLTR_OPT, RTW89_MAC_0), B_AX_RX_FLTR_CFG_MASK, - rtwdev->hal.rx_fltr); + rx_fltr); } void rtw89_hw_scan_complete(struct rtw89_dev *rtwdev, struct ieee80211_vif *vif, @@ -2323,9 +2739,6 @@ void rtw89_hw_scan_complete(struct rtw89_dev *rtwdev, struct ieee80211_vif *vif, if (!vif) return; - rtwdev->hal.rx_fltr |= B_AX_A_BCN_CHK_EN; - rtwdev->hal.rx_fltr |= B_AX_A_BC; - rtwdev->hal.rx_fltr |= B_AX_A_A1_MATCH; rtw89_write32_mask(rtwdev, rtw89_mac_reg_by_idx(R_AX_RX_FLTR_OPT, RTW89_MAC_0), B_AX_RX_FLTR_CFG_MASK, @@ -2339,6 +2752,7 @@ void rtw89_hw_scan_complete(struct rtw89_dev *rtwdev, struct ieee80211_vif *vif, rtwvif = (struct rtw89_vif *)vif->drv_priv; rtwvif->scan_req = NULL; rtwvif->scan_ies = NULL; + rtwdev->scan_info.last_chan_idx = 0; rtwdev->scan_info.scanning_vif = NULL; if (rtwvif->net_type != RTW89_NET_TYPE_NO_LINK) @@ -2377,18 +2791,18 @@ out: void rtw89_store_op_chan(struct rtw89_dev *rtwdev, bool backup) { struct rtw89_hw_scan_info *scan_info = &rtwdev->scan_info; - struct rtw89_hal *hal = &rtwdev->hal; + const struct rtw89_chan *cur = rtw89_chan_get(rtwdev, RTW89_SUB_ENTITY_0); + struct rtw89_chan new; if (backup) { - scan_info->op_pri_ch = hal->current_primary_channel; - scan_info->op_chan = hal->current_channel; - scan_info->op_bw = hal->current_band_width; - scan_info->op_band = hal->current_band_type; + scan_info->op_pri_ch = cur->primary_channel; + scan_info->op_chan = cur->channel; + scan_info->op_bw = cur->band_width; + scan_info->op_band = cur->band_type; } else { - hal->current_primary_channel = scan_info->op_pri_ch; - hal->current_channel = scan_info->op_chan; - hal->current_band_width = scan_info->op_bw; - hal->current_band_type = scan_info->op_band; + rtw89_chan_create(&new, scan_info->op_chan, scan_info->op_pri_ch, + scan_info->op_band, scan_info->op_bw); + rtw89_assign_entity_chan(rtwdev, RTW89_SUB_ENTITY_0, &new); } } @@ -2397,6 +2811,7 @@ void rtw89_store_op_chan(struct rtw89_dev *rtwdev, bool backup) int rtw89_fw_h2c_trigger_cpu_exception(struct rtw89_dev *rtwdev) { struct sk_buff *skb; + int ret; skb = rtw89_fw_h2c_alloc_skb_with_hdr(rtwdev, H2C_FW_CPU_EXCEPTION_LEN); if (!skb) { @@ -2415,7 +2830,62 @@ int rtw89_fw_h2c_trigger_cpu_exception(struct rtw89_dev *rtwdev) H2C_FUNC_CPU_EXCEPTION, 0, 0, H2C_FW_CPU_EXCEPTION_LEN); - if (rtw89_h2c_tx(rtwdev, skb, false)) { + ret = rtw89_h2c_tx(rtwdev, skb, false); + if (ret) { + rtw89_err(rtwdev, "failed to send h2c\n"); + goto fail; + } + + return 0; + +fail: + dev_kfree_skb_any(skb); + return ret; +} + +#define H2C_PKT_DROP_LEN 24 +int rtw89_fw_h2c_pkt_drop(struct rtw89_dev *rtwdev, + const struct rtw89_pkt_drop_params *params) +{ + struct sk_buff *skb; + int ret; + + skb = rtw89_fw_h2c_alloc_skb_with_hdr(rtwdev, H2C_PKT_DROP_LEN); + if (!skb) { + rtw89_err(rtwdev, + "failed to alloc skb for packet drop\n"); + return -ENOMEM; + } + + switch (params->sel) { + case RTW89_PKT_DROP_SEL_MACID_BE_ONCE: + case RTW89_PKT_DROP_SEL_MACID_BK_ONCE: + case RTW89_PKT_DROP_SEL_MACID_VI_ONCE: + case RTW89_PKT_DROP_SEL_MACID_VO_ONCE: + break; + default: + rtw89_debug(rtwdev, RTW89_DBG_FW, + "H2C of pkt drop might not fully support sel: %d yet\n", + params->sel); + break; + } + + skb_put(skb, H2C_PKT_DROP_LEN); + RTW89_SET_FWCMD_PKT_DROP_SEL(skb->data, params->sel); + RTW89_SET_FWCMD_PKT_DROP_MACID(skb->data, params->macid); + RTW89_SET_FWCMD_PKT_DROP_BAND(skb->data, params->mac_band); + RTW89_SET_FWCMD_PKT_DROP_PORT(skb->data, params->port); + RTW89_SET_FWCMD_PKT_DROP_MBSSID(skb->data, params->mbssid); + RTW89_SET_FWCMD_PKT_DROP_ROLE_A_INFO_TF_TRS(skb->data, params->tf_trs); + + rtw89_h2c_pkt_set_hdr(rtwdev, skb, FWCMD_TYPE_H2C, + H2C_CAT_MAC, + H2C_CL_MAC_FW_OFLD, + H2C_FUNC_PKT_DROP, 0, 0, + H2C_PKT_DROP_LEN); + + ret = rtw89_h2c_tx(rtwdev, skb, false); + if (ret) { rtw89_err(rtwdev, "failed to send h2c\n"); goto fail; } @@ -2424,5 +2894,5 @@ int rtw89_fw_h2c_trigger_cpu_exception(struct rtw89_dev *rtwdev) fail: dev_kfree_skb_any(skb); - return -EBUSY; + return ret; } diff --git a/drivers/net/wireless/realtek/rtw89/fw.h b/drivers/net/wireless/realtek/rtw89/fw.h index e75ad22aa85d..0047d5d0e9b1 100644 --- a/drivers/net/wireless/realtek/rtw89/fw.h +++ b/drivers/net/wireless/realtek/rtw89/fw.h @@ -63,21 +63,32 @@ enum rtw89_mac_c2h_type { RTW89_FWCMD_C2HREG_FUNC_NULL = 0xFF }; -struct rtw89_c2h_phy_cap { - u32 func:7; - u32 ack:1; - u32 len:4; - u32 seq:4; - u32 rx_nss:8; - u32 bw:8; - - u32 tx_nss:8; - u32 prot:8; - u32 nic:8; - u32 wl_func:8; - - u32 hw_type:8; -} __packed; +#define RTW89_GET_C2H_PHYCAP_FUNC(info) \ + u32_get_bits(*((const u32 *)(info)), GENMASK(6, 0)) +#define RTW89_GET_C2H_PHYCAP_ACK(info) \ + u32_get_bits(*((const u32 *)(info)), BIT(7)) +#define RTW89_GET_C2H_PHYCAP_LEN(info) \ + u32_get_bits(*((const u32 *)(info)), GENMASK(11, 8)) +#define RTW89_GET_C2H_PHYCAP_SEQ(info) \ + u32_get_bits(*((const u32 *)(info)), GENMASK(15, 12)) +#define RTW89_GET_C2H_PHYCAP_RX_NSS(info) \ + u32_get_bits(*((const u32 *)(info)), GENMASK(23, 16)) +#define RTW89_GET_C2H_PHYCAP_BW(info) \ + u32_get_bits(*((const u32 *)(info)), GENMASK(31, 24)) +#define RTW89_GET_C2H_PHYCAP_TX_NSS(info) \ + u32_get_bits(*((const u32 *)(info) + 1), GENMASK(7, 0)) +#define RTW89_GET_C2H_PHYCAP_PROT(info) \ + u32_get_bits(*((const u32 *)(info) + 1), GENMASK(15, 8)) +#define RTW89_GET_C2H_PHYCAP_NIC(info) \ + u32_get_bits(*((const u32 *)(info) + 1), GENMASK(23, 16)) +#define RTW89_GET_C2H_PHYCAP_WL_FUNC(info) \ + u32_get_bits(*((const u32 *)(info) + 1), GENMASK(31, 24)) +#define RTW89_GET_C2H_PHYCAP_HW_TYPE(info) \ + u32_get_bits(*((const u32 *)(info) + 2), GENMASK(7, 0)) +#define RTW89_GET_C2H_PHYCAP_ANT_TX_NUM(info) \ + u32_get_bits(*((const u32 *)(info) + 3), GENMASK(15, 8)) +#define RTW89_GET_C2H_PHYCAP_ANT_RX_NUM(info) \ + u32_get_bits(*((const u32 *)(info) + 3), GENMASK(23, 16)) enum rtw89_fw_c2h_category { RTW89_C2H_CAT_TEST, @@ -144,6 +155,13 @@ enum rtw89_chan_type { RTW89_CHAN_DFS, }; +enum rtw89_p2pps_action { + RTW89_P2P_ACT_INIT = 0, + RTW89_P2P_ACT_UPDATE = 1, + RTW89_P2P_ACT_REMOVE = 2, + RTW89_P2P_ACT_TERMINATE = 3, +}; + #define FWDL_SECTION_MAX_NUM 10 #define FWDL_SECTION_CHKSUM_LEN 8 #define FWDL_SECTION_PER_PKT_LEN 2020 @@ -177,6 +195,7 @@ struct rtw89_h2creg_sch_tx_en { u16 rsvd:15; } __packed; +#define RTW89_H2C_MAX_SIZE 2048 #define RTW89_CHANNEL_TIME 45 #define RTW89_DFS_CHAN_TIME 105 #define RTW89_OFF_CHAN_TIME 100 @@ -186,7 +205,10 @@ struct rtw89_h2creg_sch_tx_en { #define RTW89_SCANOFLD_MAX_IE_LEN 512 #define RTW89_SCANOFLD_PKT_NONE 0xFF #define RTW89_SCANOFLD_DEBUG_MASK 0x1F -#define RTW89_MAC_CHINFO_SIZE 20 +#define RTW89_MAC_CHINFO_SIZE 24 +#define RTW89_SCAN_LIST_GUARD 4 +#define RTW89_SCAN_LIST_LIMIT \ + ((RTW89_H2C_MAX_SIZE / RTW89_MAC_CHINFO_SIZE) - RTW89_SCAN_LIST_GUARD) struct rtw89_mac_chinfo { u8 period; @@ -346,6 +368,16 @@ static inline void RTW89_SET_FWCMD_RA_CR_TBL_SEL(void *cmd, u32 val) le32p_replace_bits((__le32 *)(cmd) + 0x03, val, BIT(10)); } +static inline void RTW89_SET_FWCMD_RA_FIX_GILTF_EN(void *cmd, u32 val) +{ + le32p_replace_bits((__le32 *)(cmd) + 0x03, val, BIT(11)); +} + +static inline void RTW89_SET_FWCMD_RA_FIX_GILTF(void *cmd, u32 val) +{ + le32p_replace_bits((__le32 *)(cmd) + 0x03, val, GENMASK(14, 12)); +} + static inline void RTW89_SET_FWCMD_RA_FIXED_CSI_MCS_SS_IDX(void *cmd, u32 val) { le32p_replace_bits((__le32 *)(cmd) + 0x03, val, GENMASK(23, 16)); @@ -1798,6 +1830,36 @@ static inline void RTW89_SET_FWCMD_CPU_EXCEPTION_TYPE(void *cmd, u32 val) le32p_replace_bits((__le32 *)cmd, val, GENMASK(31, 0)); } +static inline void RTW89_SET_FWCMD_PKT_DROP_SEL(void *cmd, u32 val) +{ + le32p_replace_bits((__le32 *)cmd, val, GENMASK(7, 0)); +} + +static inline void RTW89_SET_FWCMD_PKT_DROP_MACID(void *cmd, u32 val) +{ + le32p_replace_bits((__le32 *)cmd, val, GENMASK(15, 8)); +} + +static inline void RTW89_SET_FWCMD_PKT_DROP_BAND(void *cmd, u32 val) +{ + le32p_replace_bits((__le32 *)cmd, val, GENMASK(23, 16)); +} + +static inline void RTW89_SET_FWCMD_PKT_DROP_PORT(void *cmd, u32 val) +{ + le32p_replace_bits((__le32 *)cmd, val, GENMASK(31, 24)); +} + +static inline void RTW89_SET_FWCMD_PKT_DROP_MBSSID(void *cmd, u32 val) +{ + le32p_replace_bits((__le32 *)cmd + 1, val, GENMASK(7, 0)); +} + +static inline void RTW89_SET_FWCMD_PKT_DROP_ROLE_A_INFO_TF_TRS(void *cmd, u32 val) +{ + le32p_replace_bits((__le32 *)cmd + 1, val, GENMASK(15, 8)); +} + enum rtw89_btc_btf_h2c_class { BTFC_SET = 0x10, BTFC_GET = 0x11, @@ -2006,69 +2068,104 @@ static inline void RTW89_SET_FWCMD_CXROLE_ROLE_NAN(void *cmd, u16 val) le16p_replace_bits((__le16 *)((u8 *)(cmd) + 4), val, BIT(11)); } -static inline void RTW89_SET_FWCMD_CXROLE_ACT_CONNECTED(void *cmd, u8 val, int n) +static inline void RTW89_SET_FWCMD_CXROLE_ACT_CONNECTED(void *cmd, u8 val, int n, u8 offset) { - u8p_replace_bits((u8 *)(cmd) + (6 + 12 * (n)), val, BIT(0)); + u8p_replace_bits((u8 *)cmd + (6 + (12 + offset) * n), val, BIT(0)); } -static inline void RTW89_SET_FWCMD_CXROLE_ACT_PID(void *cmd, u8 val, int n) +static inline void RTW89_SET_FWCMD_CXROLE_ACT_PID(void *cmd, u8 val, int n, u8 offset) { - u8p_replace_bits((u8 *)(cmd) + (6 + 12 * (n)), val, GENMASK(3, 1)); + u8p_replace_bits((u8 *)cmd + (6 + (12 + offset) * n), val, GENMASK(3, 1)); } -static inline void RTW89_SET_FWCMD_CXROLE_ACT_PHY(void *cmd, u8 val, int n) +static inline void RTW89_SET_FWCMD_CXROLE_ACT_PHY(void *cmd, u8 val, int n, u8 offset) { - u8p_replace_bits((u8 *)(cmd) + (6 + 12 * (n)), val, BIT(4)); + u8p_replace_bits((u8 *)cmd + (6 + (12 + offset) * n), val, BIT(4)); } -static inline void RTW89_SET_FWCMD_CXROLE_ACT_NOA(void *cmd, u8 val, int n) +static inline void RTW89_SET_FWCMD_CXROLE_ACT_NOA(void *cmd, u8 val, int n, u8 offset) { - u8p_replace_bits((u8 *)(cmd) + (6 + 12 * (n)), val, BIT(5)); + u8p_replace_bits((u8 *)cmd + (6 + (12 + offset) * n), val, BIT(5)); } -static inline void RTW89_SET_FWCMD_CXROLE_ACT_BAND(void *cmd, u8 val, int n) +static inline void RTW89_SET_FWCMD_CXROLE_ACT_BAND(void *cmd, u8 val, int n, u8 offset) { - u8p_replace_bits((u8 *)(cmd) + (6 + 12 * (n)), val, GENMASK(7, 6)); + u8p_replace_bits((u8 *)cmd + (6 + (12 + offset) * n), val, GENMASK(7, 6)); } -static inline void RTW89_SET_FWCMD_CXROLE_ACT_CLIENT_PS(void *cmd, u8 val, int n) +static inline void RTW89_SET_FWCMD_CXROLE_ACT_CLIENT_PS(void *cmd, u8 val, int n, u8 offset) { - u8p_replace_bits((u8 *)(cmd) + (7 + 12 * (n)), val, BIT(0)); + u8p_replace_bits((u8 *)cmd + (7 + (12 + offset) * n), val, BIT(0)); } -static inline void RTW89_SET_FWCMD_CXROLE_ACT_BW(void *cmd, u8 val, int n) +static inline void RTW89_SET_FWCMD_CXROLE_ACT_BW(void *cmd, u8 val, int n, u8 offset) { - u8p_replace_bits((u8 *)(cmd) + (7 + 12 * (n)), val, GENMASK(7, 1)); + u8p_replace_bits((u8 *)cmd + (7 + (12 + offset) * n), val, GENMASK(7, 1)); } -static inline void RTW89_SET_FWCMD_CXROLE_ACT_ROLE(void *cmd, u8 val, int n) +static inline void RTW89_SET_FWCMD_CXROLE_ACT_ROLE(void *cmd, u8 val, int n, u8 offset) { - u8p_replace_bits((u8 *)(cmd) + (8 + 12 * (n)), val, GENMASK(7, 0)); + u8p_replace_bits((u8 *)cmd + (8 + (12 + offset) * n), val, GENMASK(7, 0)); } -static inline void RTW89_SET_FWCMD_CXROLE_ACT_CH(void *cmd, u8 val, int n) +static inline void RTW89_SET_FWCMD_CXROLE_ACT_CH(void *cmd, u8 val, int n, u8 offset) { - u8p_replace_bits((u8 *)(cmd) + (9 + 12 * (n)), val, GENMASK(7, 0)); + u8p_replace_bits((u8 *)cmd + (9 + (12 + offset) * n), val, GENMASK(7, 0)); } -static inline void RTW89_SET_FWCMD_CXROLE_ACT_TX_LVL(void *cmd, u16 val, int n) +static inline void RTW89_SET_FWCMD_CXROLE_ACT_TX_LVL(void *cmd, u16 val, int n, u8 offset) { - le16p_replace_bits((__le16 *)((u8 *)(cmd) + (10 + 12 * (n))), val, GENMASK(15, 0)); + le16p_replace_bits((__le16 *)((u8 *)cmd + (10 + (12 + offset) * n)), val, GENMASK(15, 0)); } -static inline void RTW89_SET_FWCMD_CXROLE_ACT_RX_LVL(void *cmd, u16 val, int n) +static inline void RTW89_SET_FWCMD_CXROLE_ACT_RX_LVL(void *cmd, u16 val, int n, u8 offset) { - le16p_replace_bits((__le16 *)((u8 *)(cmd) + (12 + 12 * (n))), val, GENMASK(15, 0)); + le16p_replace_bits((__le16 *)((u8 *)cmd + (12 + (12 + offset) * n)), val, GENMASK(15, 0)); } -static inline void RTW89_SET_FWCMD_CXROLE_ACT_TX_RATE(void *cmd, u16 val, int n) +static inline void RTW89_SET_FWCMD_CXROLE_ACT_TX_RATE(void *cmd, u16 val, int n, u8 offset) { - le16p_replace_bits((__le16 *)((u8 *)(cmd) + (14 + 12 * (n))), val, GENMASK(15, 0)); + le16p_replace_bits((__le16 *)((u8 *)cmd + (14 + (12 + offset) * n)), val, GENMASK(15, 0)); } -static inline void RTW89_SET_FWCMD_CXROLE_ACT_RX_RATE(void *cmd, u16 val, int n) +static inline void RTW89_SET_FWCMD_CXROLE_ACT_RX_RATE(void *cmd, u16 val, int n, u8 offset) { - le16p_replace_bits((__le16 *)((u8 *)(cmd) + (16 + 12 * (n))), val, GENMASK(15, 0)); + le16p_replace_bits((__le16 *)((u8 *)cmd + (16 + (12 + offset) * n)), val, GENMASK(15, 0)); +} + +static inline void RTW89_SET_FWCMD_CXROLE_ACT_NOA_DUR(void *cmd, u32 val, int n, u8 offset) +{ + le32p_replace_bits((__le32 *)((u8 *)cmd + (20 + (12 + offset) * n)), val, GENMASK(31, 0)); +} + +static inline void RTW89_SET_FWCMD_CXROLE_MROLE_TYPE(void *cmd, u32 val, u8 offset) +{ + le32p_replace_bits((__le32 *)((u8 *)cmd + offset), val, GENMASK(31, 0)); +} + +static inline void RTW89_SET_FWCMD_CXROLE_MROLE_NOA(void *cmd, u32 val, u8 offset) +{ + le32p_replace_bits((__le32 *)((u8 *)cmd + offset + 4), val, GENMASK(31, 0)); +} + +static inline void RTW89_SET_FWCMD_CXROLE_DBCC_EN(void *cmd, u32 val, u8 offset) +{ + le32p_replace_bits((__le32 *)((u8 *)cmd + offset + 8), val, BIT(0)); +} + +static inline void RTW89_SET_FWCMD_CXROLE_DBCC_CHG(void *cmd, u32 val, u8 offset) +{ + le32p_replace_bits((__le32 *)((u8 *)cmd + offset + 8), val, BIT(1)); +} + +static inline void RTW89_SET_FWCMD_CXROLE_DBCC_2G_PHY(void *cmd, u32 val, u8 offset) +{ + le32p_replace_bits((__le32 *)((u8 *)cmd + offset + 8), val, GENMASK(3, 2)); +} + +static inline void RTW89_SET_FWCMD_CXROLE_LINK_MODE_CHG(void *cmd, u32 val, u8 offset) +{ + le32p_replace_bits((__le32 *)((u8 *)cmd + offset + 8), val, BIT(4)); } static inline void RTW89_SET_FWCMD_CXCTRL_MANUAL(void *cmd, u32 val) @@ -2352,6 +2449,86 @@ static inline void RTW89_SET_FWCMD_SCANOFLD_TSF_SLOW(void *cmd, u32 val) le32p_replace_bits((__le32 *)((u8 *)(cmd) + 16), val, GENMASK(31, 0)); } +static inline void RTW89_SET_FWCMD_P2P_MACID(void *cmd, u32 val) +{ + le32p_replace_bits((__le32 *)cmd, val, GENMASK(7, 0)); +} + +static inline void RTW89_SET_FWCMD_P2P_P2PID(void *cmd, u32 val) +{ + le32p_replace_bits((__le32 *)cmd, val, GENMASK(11, 8)); +} + +static inline void RTW89_SET_FWCMD_P2P_NOAID(void *cmd, u32 val) +{ + le32p_replace_bits((__le32 *)cmd, val, GENMASK(15, 12)); +} + +static inline void RTW89_SET_FWCMD_P2P_ACT(void *cmd, u32 val) +{ + le32p_replace_bits((__le32 *)cmd, val, GENMASK(19, 16)); +} + +static inline void RTW89_SET_FWCMD_P2P_TYPE(void *cmd, u32 val) +{ + le32p_replace_bits((__le32 *)cmd, val, BIT(20)); +} + +static inline void RTW89_SET_FWCMD_P2P_ALL_SLEP(void *cmd, u32 val) +{ + le32p_replace_bits((__le32 *)cmd, val, BIT(21)); +} + +static inline void RTW89_SET_FWCMD_NOA_START_TIME(void *cmd, __le32 val) +{ + *((__le32 *)cmd + 1) = val; +} + +static inline void RTW89_SET_FWCMD_NOA_INTERVAL(void *cmd, __le32 val) +{ + *((__le32 *)cmd + 2) = val; +} + +static inline void RTW89_SET_FWCMD_NOA_DURATION(void *cmd, __le32 val) +{ + *((__le32 *)cmd + 3) = val; +} + +static inline void RTW89_SET_FWCMD_NOA_COUNT(void *cmd, u32 val) +{ + le32p_replace_bits((__le32 *)(cmd) + 4, val, GENMASK(7, 0)); +} + +static inline void RTW89_SET_FWCMD_NOA_CTWINDOW(void *cmd, u32 val) +{ + u8 ctwnd; + + if (!(val & IEEE80211_P2P_OPPPS_ENABLE_BIT)) + return; + ctwnd = FIELD_GET(IEEE80211_P2P_OPPPS_CTWINDOW_MASK, val); + le32p_replace_bits((__le32 *)(cmd) + 4, ctwnd, GENMASK(23, 8)); +} + +static inline void RTW89_SET_FWCMD_TSF32_TOGL_BAND(void *cmd, u32 val) +{ + le32p_replace_bits((__le32 *)cmd, val, BIT(0)); +} + +static inline void RTW89_SET_FWCMD_TSF32_TOGL_EN(void *cmd, u32 val) +{ + le32p_replace_bits((__le32 *)cmd, val, BIT(1)); +} + +static inline void RTW89_SET_FWCMD_TSF32_TOGL_PORT(void *cmd, u32 val) +{ + le32p_replace_bits((__le32 *)cmd, val, GENMASK(4, 2)); +} + +static inline void RTW89_SET_FWCMD_TSF32_TOGL_EARLY(void *cmd, u32 val) +{ + le32p_replace_bits((__le32 *)cmd, val, GENMASK(31, 16)); +} + #define RTW89_C2H_HEADER_LEN 8 #define RTW89_GET_C2H_CATEGORY(c2h) \ @@ -2421,6 +2598,8 @@ static inline void RTW89_SET_FWCMD_SCANOFLD_TSF_SLOW(void *cmd, u32 val) le32_get_bits(*((const __le32 *)(c2h) + 2), GENMASK(19, 16)) #define RTW89_GET_MAC_C2H_SCANOFLD_STATUS(c2h) \ le32_get_bits(*((const __le32 *)(c2h) + 2), GENMASK(23, 20)) +#define RTW89_GET_MAC_C2H_ACTUAL_PERIOD(c2h) \ + le32_get_bits(*((const __le32 *)(c2h) + 2), GENMASK(31, 24)) #define RTW89_GET_MAC_C2H_SCANOFLD_TX_FAIL(c2h) \ le32_get_bits(*((const __le32 *)(c2h) + 5), GENMASK(3, 0)) #define RTW89_GET_MAC_C2H_SCANOFLD_AIR_DENSITY(c2h) \ @@ -2446,7 +2625,14 @@ struct rtw89_mfw_info { struct rtw89_mfw_hdr { u8 sig; /* RTW89_MFW_SIG */ u8 fw_nr; - u8 rsvd[14]; + u8 rsvd0[2]; + struct { + u8 major; + u8 minor; + u8 sub; + u8 idx; + } ver; + u8 rsvd1[8]; struct rtw89_mfw_info info[]; } __packed; @@ -2493,6 +2679,7 @@ struct rtw89_fw_h2c_rf_reg_info { /* CLASS 2 - PS */ #define H2C_CL_MAC_PS 0x2 #define H2C_FUNC_MAC_LPS_PARM 0x0 +#define H2C_FUNC_P2P_ACT 0x1 /* CLASS 3 - FW download */ #define H2C_CL_MAC_FWDL 0x3 @@ -2519,9 +2706,11 @@ struct rtw89_fw_h2c_rf_reg_info { #define H2C_FUNC_PACKET_OFLD 0x1 #define H2C_FUNC_MAC_MACID_PAUSE 0x8 #define H2C_FUNC_USR_EDCA 0xF +#define H2C_FUNC_TSF32_TOGL 0x10 #define H2C_FUNC_OFLD_CFG 0x14 #define H2C_FUNC_ADD_SCANOFLD_CH 0x16 #define H2C_FUNC_SCANOFLD 0x17 +#define H2C_FUNC_PKT_DROP 0x1b /* CLASS 10 - Security CAM */ #define H2C_CL_MAC_SEC_CAM 0xa @@ -2552,7 +2741,7 @@ struct rtw89_fw_h2c_rf_get_mccch { #define RTW89_FW_RSVD_PLE_SIZE 0x800 -#define RTW89_WCPU_BASE_ADDR 0xA0000000 +#define RTW89_WCPU_BASE_MASK GENMASK(27, 0) #define RTW89_FW_BACKTRACE_INFO_SIZE 8 #define RTW89_VALID_FW_BACKTRACE_SIZE(_size) \ @@ -2563,6 +2752,9 @@ struct rtw89_fw_h2c_rf_get_mccch { int rtw89_fw_check_rdy(struct rtw89_dev *rtwdev); int rtw89_fw_recognize(struct rtw89_dev *rtwdev); +void rtw89_early_fw_feature_recognize(struct device *device, + const struct rtw89_chip_info *chip, + u32 *early_feat_map); int rtw89_fw_download(struct rtw89_dev *rtwdev, enum rtw89_fw_type type); int rtw89_load_firmware(struct rtw89_dev *rtwdev); void rtw89_unload_firmware(struct rtw89_dev *rtwdev); @@ -2577,6 +2769,8 @@ int rtw89_fw_h2c_assoc_cmac_tbl(struct rtw89_dev *rtwdev, struct ieee80211_sta *sta); int rtw89_fw_h2c_txtime_cmac_tbl(struct rtw89_dev *rtwdev, struct rtw89_sta *rtwsta); +int rtw89_fw_h2c_txpath_cmac_tbl(struct rtw89_dev *rtwdev, + struct rtw89_sta *rtwsta); int rtw89_fw_h2c_update_beacon(struct rtw89_dev *rtwdev, struct rtw89_vif *rtwvif); int rtw89_fw_h2c_cam(struct rtw89_dev *rtwdev, struct rtw89_vif *vif, @@ -2600,6 +2794,7 @@ int rtw89_fw_h2c_set_ofld_cfg(struct rtw89_dev *rtwdev); int rtw89_fw_h2c_ra(struct rtw89_dev *rtwdev, struct rtw89_ra_info *ra, bool csi); int rtw89_fw_h2c_cxdrv_init(struct rtw89_dev *rtwdev); int rtw89_fw_h2c_cxdrv_role(struct rtw89_dev *rtwdev); +int rtw89_fw_h2c_cxdrv_role_v1(struct rtw89_dev *rtwdev); int rtw89_fw_h2c_cxdrv_ctrl(struct rtw89_dev *rtwdev); int rtw89_fw_h2c_cxdrv_rfk(struct rtw89_dev *rtwdev); int rtw89_fw_h2c_del_pkt_offload(struct rtw89_dev *rtwdev, u8 id); @@ -2623,6 +2818,7 @@ void rtw89_fw_free_all_early_h2c(struct rtw89_dev *rtwdev); int rtw89_fw_h2c_general_pkt(struct rtw89_dev *rtwdev, u8 macid); int rtw89_fw_h2c_ba_cam(struct rtw89_dev *rtwdev, struct rtw89_sta *rtwsta, bool valid, struct ieee80211_ampdu_params *params); +void rtw89_fw_h2c_init_ba_cam_v1(struct rtw89_dev *rtwdev); int rtw89_fw_h2c_lps_parm(struct rtw89_dev *rtwdev, struct rtw89_lps_parm *lps_param); @@ -2642,5 +2838,20 @@ int rtw89_hw_scan_offload(struct rtw89_dev *rtwdev, struct ieee80211_vif *vif, bool enable); void rtw89_hw_scan_abort(struct rtw89_dev *rtwdev, struct ieee80211_vif *vif); int rtw89_fw_h2c_trigger_cpu_exception(struct rtw89_dev *rtwdev); +int rtw89_fw_h2c_pkt_drop(struct rtw89_dev *rtwdev, + const struct rtw89_pkt_drop_params *params); +int rtw89_fw_h2c_p2p_act(struct rtw89_dev *rtwdev, struct ieee80211_vif *vif, + struct ieee80211_p2p_noa_desc *desc, + u8 act, u8 noa_id); +int rtw89_fw_h2c_tsf32_toggle(struct rtw89_dev *rtwdev, struct rtw89_vif *rtwvif, + bool en); + +static inline void rtw89_fw_h2c_init_ba_cam(struct rtw89_dev *rtwdev) +{ + const struct rtw89_chip_info *chip = rtwdev->chip; + + if (chip->bacam_v1) + rtw89_fw_h2c_init_ba_cam_v1(rtwdev); +} #endif diff --git a/drivers/net/wireless/realtek/rtw89/mac.c b/drivers/net/wireless/realtek/rtw89/mac.c index 93124b815825..0508dfca8edf 100644 --- a/drivers/net/wireless/realtek/rtw89/mac.c +++ b/drivers/net/wireless/realtek/rtw89/mac.c @@ -3,6 +3,7 @@ */ #include "cam.h" +#include "chan.h" #include "debug.h" #include "fw.h" #include "mac.h" @@ -826,6 +827,8 @@ static void hfc_func_en(struct rtw89_dev *rtwdev, bool en, bool h2c_en) static int hfc_init(struct rtw89_dev *rtwdev, bool reset, bool en, bool h2c_en) { + const struct rtw89_chip_info *chip = rtwdev->chip; + u32 dma_ch_mask = chip->dma_ch_mask; u8 ch; u32 ret = 0; @@ -847,6 +850,8 @@ static int hfc_init(struct rtw89_dev *rtwdev, bool reset, bool en, bool h2c_en) } for (ch = RTW89_DMA_ACH0; ch < RTW89_DMA_H2C; ch++) { + if (dma_ch_mask & BIT(ch)) + continue; ret = hfc_ch_ctrl(rtwdev, ch); if (ret) return ret; @@ -862,6 +867,8 @@ static int hfc_init(struct rtw89_dev *rtwdev, bool reset, bool en, bool h2c_en) udelay(10); } for (ch = RTW89_DMA_ACH0; ch < RTW89_DMA_H2C; ch++) { + if (dma_ch_mask & BIT(ch)) + continue; ret = hfc_upd_ch_info(rtwdev, ch); if (ret) return ret; @@ -1053,18 +1060,29 @@ void rtw89_mac_power_mode_change(struct rtw89_dev *rtwdev, bool enter) enum rtw89_rpwm_req_pwr_state state; unsigned long delay = enter ? 10 : 150; int ret; + int i; if (enter) state = rtw89_mac_get_req_pwr_state(rtwdev); else state = RTW89_MAC_RPWM_REQ_PWR_STATE_ACTIVE; - rtw89_mac_send_rpwm(rtwdev, state, false); - ret = read_poll_timeout_atomic(rtw89_mac_check_cpwm_state, ret, !ret, - delay, 15000, false, rtwdev, state); - if (ret) - rtw89_err(rtwdev, "firmware failed to ack for %s ps mode\n", - enter ? "entering" : "leaving"); + for (i = 0; i < RPWM_TRY_CNT; i++) { + rtw89_mac_send_rpwm(rtwdev, state, false); + ret = read_poll_timeout_atomic(rtw89_mac_check_cpwm_state, ret, + !ret, delay, 15000, false, + rtwdev, state); + if (!ret) + break; + + if (i == RPWM_TRY_CNT - 1) + rtw89_err(rtwdev, "firmware failed to ack for %s ps mode\n", + enter ? "entering" : "leaving"); + else + rtw89_debug(rtwdev, RTW89_DBG_UNEXP, + "%d time firmware failed to ack for %s ps mode\n", + i + 1, enter ? "entering" : "leaving"); + } } void rtw89_mac_notify_wake(struct rtw89_dev *rtwdev) @@ -1081,7 +1099,6 @@ static int rtw89_mac_power_switch(struct rtw89_dev *rtwdev, bool on) const struct rtw89_chip_info *chip = rtwdev->chip; const struct rtw89_pwr_cfg * const *cfg_seq; int (*cfg_func)(struct rtw89_dev *rtwdev); - struct rtw89_hal *hal = &rtwdev->hal; int ret; u8 val; @@ -1113,7 +1130,7 @@ static int rtw89_mac_power_switch(struct rtw89_dev *rtwdev, bool on) clear_bit(RTW89_FLAG_POWERON, rtwdev->flags); clear_bit(RTW89_FLAG_FW_RDY, rtwdev->flags); rtw89_write8(rtwdev, R_AX_SCOREBOARD + 3, MAC_AX_NOTIFY_PWR_MAJOR); - hal->current_channel = 0; + rtw89_set_entity_state(rtwdev, false); } return 0; @@ -1207,8 +1224,8 @@ static int chip_func_en(struct rtw89_dev *rtwdev) { enum rtw89_core_chip_id chip_id = rtwdev->chip->chip_id; - if (chip_id == RTL8852A) - rtw89_write32_set(rtwdev, R_AX_SPSLDO_ON_CTRL0, + if (chip_id == RTL8852A || chip_id == RTL8852B) + rtw89_write32_set(rtwdev, R_AX_SPS_DIG_ON_CTRL0, B_AX_OCP_L1_MASK); return 0; @@ -1239,6 +1256,10 @@ const struct rtw89_mac_size_set rtw89_mac_size = { .wde_size0 = {RTW89_WDE_PG_64, 4095, 1,}, /* DLFW */ .wde_size4 = {RTW89_WDE_PG_64, 0, 4096,}, + /* PCIE 64 */ + .wde_size6 = {RTW89_WDE_PG_64, 512, 0,}, + /* DLFW */ + .wde_size9 = {RTW89_WDE_PG_64, 0, 1024,}, /* 8852C DLFW */ .wde_size18 = {RTW89_WDE_PG_64, 0, 2048,}, /* 8852C PCIE SCC */ @@ -1247,6 +1268,10 @@ const struct rtw89_mac_size_set rtw89_mac_size = { .ple_size0 = {RTW89_PLE_PG_128, 1520, 16,}, /* DLFW */ .ple_size4 = {RTW89_PLE_PG_128, 64, 1472,}, + /* PCIE 64 */ + .ple_size6 = {RTW89_PLE_PG_128, 496, 16,}, + /* DLFW */ + .ple_size8 = {RTW89_PLE_PG_128, 64, 960,}, /* 8852C DLFW */ .ple_size18 = {RTW89_PLE_PG_128, 2544, 16,}, /* 8852C PCIE SCC */ @@ -1255,6 +1280,8 @@ const struct rtw89_mac_size_set rtw89_mac_size = { .wde_qt0 = {3792, 196, 0, 107,}, /* DLFW */ .wde_qt4 = {0, 0, 0, 0,}, + /* PCIE 64 */ + .wde_qt6 = {448, 48, 0, 16,}, /* 8852C DLFW */ .wde_qt17 = {0, 0, 0, 0,}, /* 8852C PCIE SCC */ @@ -1265,6 +1292,8 @@ const struct rtw89_mac_size_set rtw89_mac_size = { .ple_qt5 = {264, 0, 32, 20, 64, 13, 1101, 0, 64, 128, 120,}, /* DLFW */ .ple_qt13 = {0, 0, 16, 48, 0, 0, 0, 0, 0, 0, 0,}, + /* PCIE 64 */ + .ple_qt18 = {147, 0, 16, 20, 17, 13, 89, 0, 32, 14, 8, 0,}, /* DLFW 52C */ .ple_qt44 = {0, 0, 16, 256, 0, 0, 0, 0, 0, 0, 0, 0,}, /* DLFW 52C */ @@ -1273,6 +1302,8 @@ const struct rtw89_mac_size_set rtw89_mac_size = { .ple_qt46 = {525, 0, 16, 20, 13, 13, 178, 0, 32, 62, 8, 16,}, /* 8852C PCIE SCC */ .ple_qt47 = {525, 0, 32, 20, 1034, 13, 1199, 0, 1053, 62, 160, 1037,}, + /* PCIE 64 */ + .ple_qt58 = {147, 0, 16, 20, 157, 13, 229, 0, 172, 14, 24, 0,}, }; EXPORT_SYMBOL(rtw89_mac_size); @@ -1307,6 +1338,17 @@ static inline u32 dle_used_size(const struct rtw89_dle_size *wde, ple->pge_size * (ple->lnk_pge_num + ple->unlnk_pge_num); } +static u32 dle_expected_used_size(struct rtw89_dev *rtwdev, + enum rtw89_qta_mode mode) +{ + u32 size = rtwdev->chip->fifo_size; + + if (mode == RTW89_QTA_SCC) + size -= rtwdev->chip->dle_scc_rsvd_size; + + return size; +} + static void dle_func_en(struct rtw89_dev *rtwdev, bool enable) { if (enable) @@ -1474,7 +1516,8 @@ static int dle_init(struct rtw89_dev *rtwdev, enum rtw89_qta_mode mode, ext_wde_min_qt_wcpu = ext_cfg->wde_min_qt->wcpu; } - if (dle_used_size(cfg->wde_size, cfg->ple_size) != rtwdev->chip->fifo_size) { + if (dle_used_size(cfg->wde_size, cfg->ple_size) != + dle_expected_used_size(rtwdev, mode)) { rtw89_err(rtwdev, "[ERR]wd/dle mem cfg\n"); ret = -EINVAL; goto error; @@ -1734,7 +1777,7 @@ static int addr_cam_init(struct rtw89_dev *rtwdev, u8 mac_idx) rtw89_write32(rtwdev, reg, val); ret = read_poll_timeout(rtw89_read16, p_val, !(p_val & B_AX_ADDR_CAM_CLR), - 1, TRXCFG_WAIT_CNT, false, rtwdev, B_AX_ADDR_CAM_CLR); + 1, TRXCFG_WAIT_CNT, false, rtwdev, reg); if (ret) { rtw89_err(rtwdev, "[ERR]ADDR_CAM reset\n"); return ret; @@ -1747,13 +1790,19 @@ static int scheduler_init(struct rtw89_dev *rtwdev, u8 mac_idx) { u32 ret; u32 reg; + u32 val; ret = rtw89_mac_check_mac_en(rtwdev, mac_idx, RTW89_CMAC_SEL); if (ret) return ret; reg = rtw89_mac_reg_by_idx(R_AX_PREBKF_CFG_1, mac_idx); - rtw89_write32_mask(rtwdev, reg, B_AX_SIFS_MACTXEN_T1_MASK, SIFS_MACTXEN_T1); + if (rtwdev->chip->chip_id == RTL8852C) + rtw89_write32_mask(rtwdev, reg, B_AX_SIFS_MACTXEN_T1_MASK, + SIFS_MACTXEN_T1_V1); + else + rtw89_write32_mask(rtwdev, reg, B_AX_SIFS_MACTXEN_T1_MASK, + SIFS_MACTXEN_T1); if (rtwdev->chip->chip_id == RTL8852B) { reg = rtw89_mac_reg_by_idx(R_AX_SCH_EXT_CTRL, mac_idx); @@ -1764,7 +1813,16 @@ static int scheduler_init(struct rtw89_dev *rtwdev, u8 mac_idx) rtw89_write32_clr(rtwdev, reg, B_AX_BTCCA_EN); reg = rtw89_mac_reg_by_idx(R_AX_PREBKF_CFG_0, mac_idx); - rtw89_write32_mask(rtwdev, reg, B_AX_PREBKF_TIME_MASK, SCH_PREBKF_24US); + if (rtwdev->chip->chip_id == RTL8852C) { + val = rtw89_read32_mask(rtwdev, R_AX_SEC_ENG_CTRL, + B_AX_TX_PARTIAL_MODE); + if (!val) + rtw89_write32_mask(rtwdev, reg, B_AX_PREBKF_TIME_MASK, + SCH_PREBKF_24US); + } else { + rtw89_write32_mask(rtwdev, reg, B_AX_PREBKF_TIME_MASK, + SCH_PREBKF_24US); + } return 0; } @@ -1910,7 +1968,7 @@ static int nav_ctrl_init(struct rtw89_dev *rtwdev) rtw89_write32_set(rtwdev, R_AX_WMAC_NAV_CTL, B_AX_WMAC_PLCP_UP_NAV_EN | B_AX_WMAC_TF_UP_NAV_EN | B_AX_WMAC_NAV_UPPER_EN); - rtw89_write32_mask(rtwdev, R_AX_WMAC_NAV_CTL, B_AX_WMAC_NAV_UPPER_MASK, NAV_12MS); + rtw89_write32_mask(rtwdev, R_AX_WMAC_NAV_CTL, B_AX_WMAC_NAV_UPPER_MASK, NAV_25MS); return 0; } @@ -1953,6 +2011,8 @@ static int tmac_init(struct rtw89_dev *rtwdev, u8 mac_idx) static int trxptcl_init(struct rtw89_dev *rtwdev, u8 mac_idx) { + const struct rtw89_chip_info *chip = rtwdev->chip; + const struct rtw89_rrsr_cfgs *rrsr = chip->rrsr_cfgs; u32 reg, val, sifs; int ret; @@ -1983,6 +2043,11 @@ static int trxptcl_init(struct rtw89_dev *rtwdev, u8 mac_idx) reg = rtw89_mac_reg_by_idx(R_AX_RXTRIG_TEST_USER_2, mac_idx); rtw89_write32_set(rtwdev, reg, B_AX_RXTRIG_FCSCHK_EN); + reg = rtw89_mac_reg_by_idx(rrsr->ref_rate.addr, mac_idx); + rtw89_write32_mask(rtwdev, reg, rrsr->ref_rate.mask, rrsr->ref_rate.data); + reg = rtw89_mac_reg_by_idx(rrsr->rsc.addr, mac_idx); + rtw89_write32_mask(rtwdev, reg, rrsr->rsc.mask, rrsr->rsc.data); + return 0; } @@ -2061,6 +2126,7 @@ static int rmac_init(struct rtw89_dev *rtwdev, u8 mac_idx) static int cmac_com_init(struct rtw89_dev *rtwdev, u8 mac_idx) { + enum rtw89_core_chip_id chip_id = rtwdev->chip->chip_id; u32 val, reg; int ret; @@ -2075,6 +2141,11 @@ static int cmac_com_init(struct rtw89_dev *rtwdev, u8 mac_idx) val = u32_replace_bits(val, 0, B_AX_TXSC_80M_MASK); rtw89_write32(rtwdev, reg, val); + if (chip_id == RTL8852A || chip_id == RTL8852B) { + reg = rtw89_mac_reg_by_idx(R_AX_PTCL_RRSR1, mac_idx); + rtw89_write32_mask(rtwdev, reg, B_AX_RRSR_RATE_EN_MASK, RRSR_OFDM_CCK_EN); + } + return 0; } @@ -2134,6 +2205,25 @@ static int ptcl_init(struct rtw89_dev *rtwdev, u8 mac_idx) return 0; } +static int cmac_dma_init(struct rtw89_dev *rtwdev, u8 mac_idx) +{ + enum rtw89_core_chip_id chip_id = rtwdev->chip->chip_id; + u32 reg; + int ret; + + if (chip_id != RTL8852A && chip_id != RTL8852B) + return 0; + + ret = rtw89_mac_check_mac_en(rtwdev, mac_idx, RTW89_CMAC_SEL); + if (ret) + return ret; + + reg = rtw89_mac_reg_by_idx(R_AX_RXDMA_CTRL_0, mac_idx); + rtw89_write8_clr(rtwdev, reg, RX_FULL_MODE); + + return 0; +} + static int cmac_init(struct rtw89_dev *rtwdev, u8 mac_idx) { int ret; @@ -2209,6 +2299,12 @@ static int cmac_init(struct rtw89_dev *rtwdev, u8 mac_idx) return ret; } + ret = cmac_dma_init(rtwdev, mac_idx); + if (ret) { + rtw89_err(rtwdev, "[ERR]CMAC%d DMA init %d\n", mac_idx, ret); + return ret; + } + return ret; } @@ -2236,23 +2332,42 @@ int rtw89_mac_setup_phycap(struct rtw89_dev *rtwdev) struct rtw89_hal *hal = &rtwdev->hal; const struct rtw89_chip_info *chip = rtwdev->chip; struct rtw89_mac_c2h_info c2h_info = {0}; - struct rtw89_c2h_phy_cap *cap = - (struct rtw89_c2h_phy_cap *)&c2h_info.c2hreg[0]; + u8 tx_nss; + u8 rx_nss; + u8 tx_ant; + u8 rx_ant; u32 ret; ret = rtw89_mac_read_phycap(rtwdev, &c2h_info); if (ret) return ret; - hal->tx_nss = cap->tx_nss ? - min_t(u8, cap->tx_nss, chip->tx_nss) : chip->tx_nss; - hal->rx_nss = cap->rx_nss ? - min_t(u8, cap->rx_nss, chip->rx_nss) : chip->rx_nss; + tx_nss = RTW89_GET_C2H_PHYCAP_TX_NSS(c2h_info.c2hreg); + rx_nss = RTW89_GET_C2H_PHYCAP_RX_NSS(c2h_info.c2hreg); + tx_ant = RTW89_GET_C2H_PHYCAP_ANT_TX_NUM(c2h_info.c2hreg); + rx_ant = RTW89_GET_C2H_PHYCAP_ANT_RX_NUM(c2h_info.c2hreg); + + hal->tx_nss = tx_nss ? min_t(u8, tx_nss, chip->tx_nss) : chip->tx_nss; + hal->rx_nss = rx_nss ? min_t(u8, rx_nss, chip->rx_nss) : chip->rx_nss; + + if (tx_ant == 1) + hal->antenna_tx = RF_B; + if (rx_ant == 1) + hal->antenna_rx = RF_B; + + if (tx_nss == 1 && tx_ant == 2 && rx_ant == 2) { + hal->antenna_tx = RF_B; + hal->tx_path_diversity = true; + } rtw89_debug(rtwdev, RTW89_DBG_FW, "phycap hal/phy/chip: tx_nss=0x%x/0x%x/0x%x rx_nss=0x%x/0x%x/0x%x\n", - hal->tx_nss, cap->tx_nss, chip->tx_nss, - hal->rx_nss, cap->rx_nss, chip->rx_nss); + hal->tx_nss, tx_nss, chip->tx_nss, + hal->rx_nss, rx_nss, chip->rx_nss); + rtw89_debug(rtwdev, RTW89_DBG_FW, + "ant num/bitmap: tx=%d/0x%x rx=%d/0x%x\n", + tx_ant, hal->antenna_tx, rx_ant, hal->antenna_rx); + rtw89_debug(rtwdev, RTW89_DBG_FW, "TX path diversity=%d\n", hal->tx_path_diversity); return 0; } @@ -2429,8 +2544,7 @@ int rtw89_mac_resume_sch_tx_v1(struct rtw89_dev *rtwdev, u8 mac_idx, u32 tx_en) } EXPORT_SYMBOL(rtw89_mac_resume_sch_tx_v1); -static u16 rtw89_mac_dle_buf_req(struct rtw89_dev *rtwdev, u16 buf_len, - bool wd) +u16 rtw89_mac_dle_buf_req(struct rtw89_dev *rtwdev, u16 buf_len, bool wd) { u32 val, reg; int ret; @@ -2450,9 +2564,8 @@ static u16 rtw89_mac_dle_buf_req(struct rtw89_dev *rtwdev, u16 buf_len, return FIELD_GET(B_AX_WD_BUF_STAT_PKTID_MASK, val); } -static int rtw89_mac_set_cpuio(struct rtw89_dev *rtwdev, - struct rtw89_cpuio_ctrl *ctrl_para, - bool wd) +int rtw89_mac_set_cpuio(struct rtw89_dev *rtwdev, + struct rtw89_cpuio_ctrl *ctrl_para, bool wd) { u32 val, cmd_type, reg; int ret; @@ -2517,7 +2630,8 @@ static int dle_quota_change(struct rtw89_dev *rtwdev, enum rtw89_qta_mode mode) return -EINVAL; } - if (dle_used_size(cfg->wde_size, cfg->ple_size) != rtwdev->chip->fifo_size) { + if (dle_used_size(cfg->wde_size, cfg->ple_size) != + dle_expected_used_size(rtwdev, mode)) { rtw89_err(rtwdev, "[ERR]wd/dle mem cfg\n"); return -EINVAL; } @@ -2766,7 +2880,7 @@ static void rtw89_bbrpt_imr_enable(struct rtw89_dev *rtwdev) { const struct rtw89_imr_info *imr = rtwdev->chip->imr_info; - rtw89_write32_set(rtwdev, R_AX_BBRPT_COM_ERR_IMR, + rtw89_write32_set(rtwdev, imr->bbrpt_com_err_imr_reg, B_AX_BBRPT_COM_NULL_PLPKTID_ERR_INT_EN); rtw89_write32_clr(rtwdev, imr->bbrpt_chinfo_err_imr_reg, B_AX_BBRPT_CHINFO_IMR_CLR); @@ -3026,6 +3140,8 @@ static int rtw89_mac_enable_cpu(struct rtw89_dev *rtwdev, u8 boot_reason, rtw89_write32(rtwdev, R_AX_HALT_H2C_CTRL, 0); rtw89_write32(rtwdev, R_AX_HALT_C2H_CTRL, 0); + rtw89_write32(rtwdev, R_AX_HALT_H2C, 0); + rtw89_write32(rtwdev, R_AX_HALT_C2H, 0); rtw89_write32_set(rtwdev, R_AX_SYS_CLK_CTRL, B_AX_CPU_CLK_EN); @@ -3103,14 +3219,6 @@ dle: return ret; } -static void rtw89_mac_hci_func_en(struct rtw89_dev *rtwdev) -{ - const struct rtw89_chip_info *chip = rtwdev->chip; - - rtw89_write32_set(rtwdev, chip->hci_func_en_addr, - B_AX_HCI_TXDMA_EN | B_AX_HCI_RXDMA_EN); -} - int rtw89_mac_enable_bb_rf(struct rtw89_dev *rtwdev) { rtw89_write8_set(rtwdev, R_AX_SYS_FUNC_EN, @@ -3124,7 +3232,7 @@ int rtw89_mac_enable_bb_rf(struct rtw89_dev *rtwdev) } EXPORT_SYMBOL(rtw89_mac_enable_bb_rf); -void rtw89_mac_disable_bb_rf(struct rtw89_dev *rtwdev) +int rtw89_mac_disable_bb_rf(struct rtw89_dev *rtwdev) { rtw89_write8_clr(rtwdev, R_AX_SYS_FUNC_EN, B_AX_FEN_BBRSTB | B_AX_FEN_BB_GLB_RSTN); @@ -3132,6 +3240,8 @@ void rtw89_mac_disable_bb_rf(struct rtw89_dev *rtwdev) B_AX_WLRF1_CTRL_7 | B_AX_WLRF1_CTRL_1 | B_AX_WLRF_CTRL_7 | B_AX_WLRF_CTRL_1); rtw89_write8_clr(rtwdev, R_AX_PHYREG_SET, PHYREG_SET_ALL_CYCLE); + + return 0; } EXPORT_SYMBOL(rtw89_mac_disable_bb_rf); @@ -3147,7 +3257,7 @@ int rtw89_mac_partial_init(struct rtw89_dev *rtwdev) return ret; } - rtw89_mac_hci_func_en(rtwdev); + rtw89_mac_ctrl_hci_dma_trx(rtwdev, true); ret = rtw89_mac_dmac_pre_init(rtwdev); if (ret) @@ -3524,6 +3634,26 @@ static void rtw89_mac_port_cfg_bcn_early(struct rtw89_dev *rtwdev, BCN_ERLY_DEF); } +static void rtw89_mac_port_cfg_tbtt_shift(struct rtw89_dev *rtwdev, + struct rtw89_vif *rtwvif) +{ + const struct rtw89_port_reg *p = &rtw_port_base; + u16 val; + + if (rtwdev->chip->chip_id != RTL8852C) + return; + + if (rtwvif->wifi_role != RTW89_WIFI_ROLE_P2P_CLIENT && + rtwvif->wifi_role != RTW89_WIFI_ROLE_STATION) + return; + + val = FIELD_PREP(B_AX_TBTT_SHIFT_OFST_MAG, 1) | + B_AX_TBTT_SHIFT_OFST_SIGN; + + rtw89_write16_port_mask(rtwdev, rtwvif, p->tbtt_shift, + B_AX_TBTT_SHIFT_OFST_MASK, val); +} + int rtw89_mac_vif_init(struct rtw89_dev *rtwdev, struct rtw89_vif *rtwvif) { int ret; @@ -3598,6 +3728,7 @@ int rtw89_mac_port_update(struct rtw89_dev *rtwdev, struct rtw89_vif *rtwvif) rtw89_mac_port_cfg_bcn_hold_time(rtwdev, rtwvif); rtw89_mac_port_cfg_bcn_mask_area(rtwdev, rtwvif); rtw89_mac_port_cfg_tbtt_early(rtwdev, rtwvif); + rtw89_mac_port_cfg_tbtt_shift(rtwdev, rtwvif); rtw89_mac_port_cfg_bss_color(rtwdev, rtwvif); rtw89_mac_port_cfg_mbssid(rtwdev, rtwvif); rtw89_mac_port_cfg_func_en(rtwdev, rtwvif); @@ -3607,6 +3738,50 @@ int rtw89_mac_port_update(struct rtw89_dev *rtwdev, struct rtw89_vif *rtwvif) return 0; } +static void rtw89_mac_check_he_obss_narrow_bw_ru_iter(struct wiphy *wiphy, + struct cfg80211_bss *bss, + void *data) +{ + const struct cfg80211_bss_ies *ies; + const struct element *elem; + bool *tolerated = data; + + rcu_read_lock(); + ies = rcu_dereference(bss->ies); + elem = cfg80211_find_elem(WLAN_EID_EXT_CAPABILITY, ies->data, + ies->len); + + if (!elem || elem->datalen < 10 || + !(elem->data[10] & WLAN_EXT_CAPA10_OBSS_NARROW_BW_RU_TOLERANCE_SUPPORT)) + *tolerated = false; + rcu_read_unlock(); +} + +void rtw89_mac_set_he_obss_narrow_bw_ru(struct rtw89_dev *rtwdev, + struct ieee80211_vif *vif) +{ + struct rtw89_vif *rtwvif = (struct rtw89_vif *)vif->drv_priv; + struct ieee80211_hw *hw = rtwdev->hw; + bool tolerated = true; + u32 reg; + + if (!vif->bss_conf.he_support || vif->type != NL80211_IFTYPE_STATION) + return; + + if (!(vif->bss_conf.chandef.chan->flags & IEEE80211_CHAN_RADAR)) + return; + + cfg80211_bss_iter(hw->wiphy, &vif->bss_conf.chandef, + rtw89_mac_check_he_obss_narrow_bw_ru_iter, + &tolerated); + + reg = rtw89_mac_reg_by_idx(R_AX_RXTRIG_TEST_USER_2, rtwvif->mac_idx); + if (tolerated) + rtw89_write32_clr(rtwdev, reg, B_AX_RXTRIG_RU26_DIS); + else + rtw89_write32_set(rtwdev, reg, B_AX_RXTRIG_RU26_DIS); +} + int rtw89_mac_add_vif(struct rtw89_dev *rtwdev, struct rtw89_vif *rtwvif) { int ret; @@ -3655,22 +3830,26 @@ rtw89_mac_c2h_scanofld_rsp(struct rtw89_dev *rtwdev, struct sk_buff *c2h, u32 len) { struct ieee80211_vif *vif = rtwdev->scan_info.scanning_vif; - struct rtw89_hal *hal = &rtwdev->hal; - u8 reason, status, tx_fail, band; + struct rtw89_vif *rtwvif = vif_to_rtwvif_safe(vif); + struct rtw89_chan new; + u8 reason, status, tx_fail, band, actual_period; + u32 last_chan = rtwdev->scan_info.last_chan_idx; u16 chan; + int ret; tx_fail = RTW89_GET_MAC_C2H_SCANOFLD_TX_FAIL(c2h->data); status = RTW89_GET_MAC_C2H_SCANOFLD_STATUS(c2h->data); chan = RTW89_GET_MAC_C2H_SCANOFLD_PRI_CH(c2h->data); reason = RTW89_GET_MAC_C2H_SCANOFLD_RSP(c2h->data); band = RTW89_GET_MAC_C2H_SCANOFLD_BAND(c2h->data); + actual_period = RTW89_GET_MAC_C2H_ACTUAL_PERIOD(c2h->data); if (!(rtwdev->chip->support_bands & BIT(NL80211_BAND_6GHZ))) band = chan > 14 ? RTW89_BAND_5G : RTW89_BAND_2G; rtw89_debug(rtwdev, RTW89_DBG_HW_SCAN, - "band: %d, chan: %d, reason: %d, status: %d, tx_fail: %d\n", - band, chan, reason, status, tx_fail); + "band: %d, chan: %d, reason: %d, status: %d, tx_fail: %d, actual: %d\n", + band, chan, reason, status, tx_fail, actual_period); switch (reason) { case RTW89_SCAN_LEAVE_CH_NOTIFY: @@ -3678,15 +3857,20 @@ rtw89_mac_c2h_scanofld_rsp(struct rtw89_dev *rtwdev, struct sk_buff *c2h, ieee80211_stop_queues(rtwdev->hw); return; case RTW89_SCAN_END_SCAN_NOTIFY: - rtw89_hw_scan_complete(rtwdev, vif, false); + if (rtwvif && rtwvif->scan_req && + last_chan < rtwvif->scan_req->n_channels) { + ret = rtw89_hw_scan_offload(rtwdev, vif, true); + if (ret) { + rtw89_hw_scan_abort(rtwdev, vif); + rtw89_warn(rtwdev, "HW scan failed: %d\n", ret); + } + } else { + rtw89_hw_scan_complete(rtwdev, vif, false); + } break; case RTW89_SCAN_ENTER_CH_NOTIFY: - hal->prev_band_type = hal->current_band_type; - hal->current_band_type = band; - hal->prev_primary_channel = hal->current_primary_channel; - hal->current_primary_channel = chan; - hal->current_channel = chan; - hal->current_band_width = RTW89_CHANNEL_WIDTH_20; + rtw89_chan_create(&new, chan, chan, band, RTW89_CHANNEL_WIDTH_20); + rtw89_assign_entity_chan(rtwdev, RTW89_SUB_ENTITY_0, &new); if (rtw89_is_op_chan(rtwdev, band, chan)) { rtw89_store_op_chan(rtwdev, false); ieee80211_wake_queues(rtwdev->hw); @@ -3738,6 +3922,12 @@ rtw89_mac_c2h_pkt_ofld_rsp(struct rtw89_dev *rtwdev, struct sk_buff *c2h, { } +static void +rtw89_mac_c2h_tsf32_toggle_rpt(struct rtw89_dev *rtwdev, struct sk_buff *c2h, + u32 len) +{ +} + static void (* const rtw89_mac_c2h_ofld_handler[])(struct rtw89_dev *rtwdev, struct sk_buff *c2h, u32 len) = { @@ -3747,6 +3937,7 @@ void (* const rtw89_mac_c2h_ofld_handler[])(struct rtw89_dev *rtwdev, [RTW89_MAC_C2H_FUNC_BCN_RESEND] = NULL, [RTW89_MAC_C2H_FUNC_MACID_PAUSE] = rtw89_mac_c2h_macid_pause, [RTW89_MAC_C2H_FUNC_SCANOFLD_RSP] = rtw89_mac_c2h_scanofld_rsp, + [RTW89_MAC_C2H_FUNC_TSF32_TOGL_RPT] = rtw89_mac_c2h_tsf32_toggle_rpt, }; static @@ -4628,3 +4819,48 @@ int rtw89_mac_read_xtal_si(struct rtw89_dev *rtwdev, u8 offset, u8 *val) return 0; } + +static +void rtw89_mac_pkt_drop_sta(struct rtw89_dev *rtwdev, struct rtw89_sta *rtwsta) +{ + static const enum rtw89_pkt_drop_sel sels[] = { + RTW89_PKT_DROP_SEL_MACID_BE_ONCE, + RTW89_PKT_DROP_SEL_MACID_BK_ONCE, + RTW89_PKT_DROP_SEL_MACID_VI_ONCE, + RTW89_PKT_DROP_SEL_MACID_VO_ONCE, + }; + struct rtw89_vif *rtwvif = rtwsta->rtwvif; + struct rtw89_pkt_drop_params params = {0}; + int i; + + params.mac_band = RTW89_MAC_0; + params.macid = rtwsta->mac_id; + params.port = rtwvif->port; + params.mbssid = 0; + params.tf_trs = rtwvif->trigger; + + for (i = 0; i < ARRAY_SIZE(sels); i++) { + params.sel = sels[i]; + rtw89_fw_h2c_pkt_drop(rtwdev, ¶ms); + } +} + +static void rtw89_mac_pkt_drop_vif_iter(void *data, struct ieee80211_sta *sta) +{ + struct rtw89_sta *rtwsta = (struct rtw89_sta *)sta->drv_priv; + struct rtw89_vif *rtwvif = rtwsta->rtwvif; + struct rtw89_dev *rtwdev = rtwvif->rtwdev; + struct rtw89_vif *target = data; + + if (rtwvif != target) + return; + + rtw89_mac_pkt_drop_sta(rtwdev, rtwsta); +} + +void rtw89_mac_pkt_drop_vif(struct rtw89_dev *rtwdev, struct rtw89_vif *rtwvif) +{ + ieee80211_iterate_stations_atomic(rtwdev->hw, + rtw89_mac_pkt_drop_vif_iter, + rtwvif); +} diff --git a/drivers/net/wireless/realtek/rtw89/mac.h b/drivers/net/wireless/realtek/rtw89/mac.h index f66619354734..6f4ada1869a1 100644 --- a/drivers/net/wireless/realtek/rtw89/mac.h +++ b/drivers/net/wireless/realtek/rtw89/mac.h @@ -6,11 +6,13 @@ #define __RTW89_MAC_H__ #include "core.h" +#include "reg.h" #define MAC_MEM_DUMP_PAGE_SIZE 0x40000 #define ADDR_CAM_ENT_SIZE 0x40 #define BSSID_CAM_ENT_SIZE 0x08 #define HFC_PAGE_UNIT 64 +#define RPWM_TRY_CNT 3 enum rtw89_mac_hwmod_sel { RTW89_DMAC_SEL = 0, @@ -304,6 +306,7 @@ enum rtw89_mac_c2h_ofld_func { RTW89_MAC_C2H_FUNC_PKT_OFLD_RSP, RTW89_MAC_C2H_FUNC_BCN_RESEND, RTW89_MAC_C2H_FUNC_MACID_PAUSE, + RTW89_MAC_C2H_FUNC_TSF32_TOGL_RPT = 0x6, RTW89_MAC_C2H_FUNC_SCANOFLD_RSP = 0x9, RTW89_MAC_C2H_FUNC_OFLD_MAX, }; @@ -688,23 +691,30 @@ struct rtw89_mac_size_set { const struct rtw89_hfc_prec_cfg hfc_preccfg_pcie; const struct rtw89_dle_size wde_size0; const struct rtw89_dle_size wde_size4; + const struct rtw89_dle_size wde_size6; + const struct rtw89_dle_size wde_size9; const struct rtw89_dle_size wde_size18; const struct rtw89_dle_size wde_size19; const struct rtw89_dle_size ple_size0; const struct rtw89_dle_size ple_size4; + const struct rtw89_dle_size ple_size6; + const struct rtw89_dle_size ple_size8; const struct rtw89_dle_size ple_size18; const struct rtw89_dle_size ple_size19; const struct rtw89_wde_quota wde_qt0; const struct rtw89_wde_quota wde_qt4; + const struct rtw89_wde_quota wde_qt6; const struct rtw89_wde_quota wde_qt17; const struct rtw89_wde_quota wde_qt18; const struct rtw89_ple_quota ple_qt4; const struct rtw89_ple_quota ple_qt5; const struct rtw89_ple_quota ple_qt13; + const struct rtw89_ple_quota ple_qt18; const struct rtw89_ple_quota ple_qt44; const struct rtw89_ple_quota ple_qt45; const struct rtw89_ple_quota ple_qt46; const struct rtw89_ple_quota ple_qt47; + const struct rtw89_ple_quota ple_qt58; }; extern const struct rtw89_mac_size_set rtw89_mac_size; @@ -798,9 +808,11 @@ int rtw89_mac_write_lte(struct rtw89_dev *rtwdev, const u32 offset, u32 val); int rtw89_mac_read_lte(struct rtw89_dev *rtwdev, const u32 offset, u32 *val); int rtw89_mac_add_vif(struct rtw89_dev *rtwdev, struct rtw89_vif *vif); int rtw89_mac_port_update(struct rtw89_dev *rtwdev, struct rtw89_vif *rtwvif); +void rtw89_mac_set_he_obss_narrow_bw_ru(struct rtw89_dev *rtwdev, + struct ieee80211_vif *vif); int rtw89_mac_remove_vif(struct rtw89_dev *rtwdev, struct rtw89_vif *vif); int rtw89_mac_enable_bb_rf(struct rtw89_dev *rtwdev); -void rtw89_mac_disable_bb_rf(struct rtw89_dev *rtwdev); +int rtw89_mac_disable_bb_rf(struct rtw89_dev *rtwdev); static inline int rtw89_chip_enable_bb_rf(struct rtw89_dev *rtwdev) { @@ -809,11 +821,11 @@ static inline int rtw89_chip_enable_bb_rf(struct rtw89_dev *rtwdev) return chip->ops->enable_bb_rf(rtwdev); } -static inline void rtw89_chip_disable_bb_rf(struct rtw89_dev *rtwdev) +static inline int rtw89_chip_disable_bb_rf(struct rtw89_dev *rtwdev) { const struct rtw89_chip_info *chip = rtwdev->chip; - chip->ops->disable_bb_rf(rtwdev); + return chip->ops->disable_bb_rf(rtwdev); } u32 rtw89_mac_get_err_status(struct rtw89_dev *rtwdev); @@ -911,6 +923,45 @@ static inline int rtw89_mac_txpwr_write32_mask(struct rtw89_dev *rtwdev, return 0; } +static inline void rtw89_mac_ctrl_hci_dma_tx(struct rtw89_dev *rtwdev, + bool enable) +{ + const struct rtw89_chip_info *chip = rtwdev->chip; + + if (enable) + rtw89_write32_set(rtwdev, chip->hci_func_en_addr, + B_AX_HCI_TXDMA_EN); + else + rtw89_write32_clr(rtwdev, chip->hci_func_en_addr, + B_AX_HCI_TXDMA_EN); +} + +static inline void rtw89_mac_ctrl_hci_dma_rx(struct rtw89_dev *rtwdev, + bool enable) +{ + const struct rtw89_chip_info *chip = rtwdev->chip; + + if (enable) + rtw89_write32_set(rtwdev, chip->hci_func_en_addr, + B_AX_HCI_RXDMA_EN); + else + rtw89_write32_clr(rtwdev, chip->hci_func_en_addr, + B_AX_HCI_RXDMA_EN); +} + +static inline void rtw89_mac_ctrl_hci_dma_trx(struct rtw89_dev *rtwdev, + bool enable) +{ + const struct rtw89_chip_info *chip = rtwdev->chip; + + if (enable) + rtw89_write32_set(rtwdev, chip->hci_func_en_addr, + B_AX_HCI_TXDMA_EN | B_AX_HCI_RXDMA_EN); + else + rtw89_write32_clr(rtwdev, chip->hci_func_en_addr, + B_AX_HCI_TXDMA_EN | B_AX_HCI_RXDMA_EN); +} + int rtw89_mac_set_tx_time(struct rtw89_dev *rtwdev, struct rtw89_sta *rtwsta, bool resume, u32 tx_time); int rtw89_mac_get_tx_time(struct rtw89_dev *rtwdev, struct rtw89_sta *rtwsta, @@ -944,8 +995,10 @@ enum rtw89_mac_xtal_si_offset { #define XTAL_SI_HIGH_ADDR_MASK GENMASK(2, 0) XTAL_SI_READ_VAL = 0x7A, XTAL_SI_WL_RFC_S0 = 0x80, +#define XTAL_SI_RF00S_EN GENMASK(2, 0) #define XTAL_SI_RF00 BIT(0) XTAL_SI_WL_RFC_S1 = 0x81, +#define XTAL_SI_RF10S_EN GENMASK(2, 0) #define XTAL_SI_RF10 BIT(0) XTAL_SI_ANAPAR_WL = 0x90, #define XTAL_SI_SRAM2RFC BIT(7) @@ -962,5 +1015,9 @@ enum rtw89_mac_xtal_si_offset { int rtw89_mac_write_xtal_si(struct rtw89_dev *rtwdev, u8 offset, u8 val, u8 mask); int rtw89_mac_read_xtal_si(struct rtw89_dev *rtwdev, u8 offset, u8 *val); +void rtw89_mac_pkt_drop_vif(struct rtw89_dev *rtwdev, struct rtw89_vif *rtwvif); +u16 rtw89_mac_dle_buf_req(struct rtw89_dev *rtwdev, u16 buf_len, bool wd); +int rtw89_mac_set_cpuio(struct rtw89_dev *rtwdev, + struct rtw89_cpuio_ctrl *ctrl_para, bool wd); #endif diff --git a/drivers/net/wireless/realtek/rtw89/mac80211.c b/drivers/net/wireless/realtek/rtw89/mac80211.c index cef27e781ae2..a296bfa8188f 100644 --- a/drivers/net/wireless/realtek/rtw89/mac80211.c +++ b/drivers/net/wireless/realtek/rtw89/mac80211.c @@ -3,6 +3,7 @@ */ #include "cam.h" +#include "chan.h" #include "coex.h" #include "debug.h" #include "fw.h" @@ -12,6 +13,7 @@ #include "reg.h" #include "sar.h" #include "ser.h" +#include "util.h" static void rtw89_ops_tx(struct ieee80211_hw *hw, struct ieee80211_tx_control *control, @@ -85,8 +87,11 @@ static int rtw89_ops_config(struct ieee80211_hw *hw, u32 changed) } } - if (changed & IEEE80211_CONF_CHANGE_CHANNEL) + if (changed & IEEE80211_CONF_CHANGE_CHANNEL) { + rtw89_config_entity_chandef(rtwdev, RTW89_SUB_ENTITY_0, + &hw->conf.chandef); rtw89_set_channel(rtwdev); + } if ((changed & IEEE80211_CONF_CHANGE_IDLE) && (hw->conf.flags & IEEE80211_CONF_IDLE)) @@ -104,6 +109,9 @@ static int rtw89_ops_add_interface(struct ieee80211_hw *hw, struct rtw89_vif *rtwvif = (struct rtw89_vif *)vif->drv_priv; int ret = 0; + rtw89_debug(rtwdev, RTW89_DBG_STATE, "add vif %pM type %d, p2p %d\n", + vif->addr, vif->type, vif->p2p); + mutex_lock(&rtwdev->mutex); rtwvif->rtwdev = rtwdev; list_add_tail(&rtwvif->list, &rtwdev->rtwvifs_list); @@ -146,6 +154,9 @@ static void rtw89_ops_remove_interface(struct ieee80211_hw *hw, struct rtw89_dev *rtwdev = hw->priv; struct rtw89_vif *rtwvif = (struct rtw89_vif *)vif->drv_priv; + rtw89_debug(rtwdev, RTW89_DBG_STATE, "remove vif %pM type %d p2p %d\n", + vif->addr, vif->type, vif->p2p); + cancel_work_sync(&rtwvif->update_beacon_work); mutex_lock(&rtwdev->mutex); @@ -157,6 +168,23 @@ static void rtw89_ops_remove_interface(struct ieee80211_hw *hw, mutex_unlock(&rtwdev->mutex); } +static int rtw89_ops_change_interface(struct ieee80211_hw *hw, + struct ieee80211_vif *vif, + enum nl80211_iftype type, bool p2p) +{ + struct rtw89_dev *rtwdev = hw->priv; + + rtw89_debug(rtwdev, RTW89_DBG_STATE, "change vif %pM (%d)->(%d), p2p (%d)->(%d)\n", + vif->addr, vif->type, type, vif->p2p, p2p); + + rtw89_ops_remove_interface(hw, vif); + + vif->type = type; + vif->p2p = p2p; + + return rtw89_ops_add_interface(hw, vif); +} + static void rtw89_ops_configure_filter(struct ieee80211_hw *hw, unsigned int changed_flags, unsigned int *new_flags, @@ -235,11 +263,12 @@ static u8 rtw89_aifsn_to_aifs(struct rtw89_dev *rtwdev, struct rtw89_vif *rtwvif, u8 aifsn) { struct ieee80211_vif *vif = rtwvif_to_vif(rtwvif); + const struct rtw89_chan *chan = rtw89_chan_get(rtwdev, RTW89_SUB_ENTITY_0); u8 slot_time; u8 sifs; slot_time = vif->bss_conf.use_short_slot ? 9 : 20; - sifs = rtwdev->hal.current_band_type == RTW89_BAND_5G ? 16 : 10; + sifs = chan->band_type == RTW89_BAND_5G ? 16 : 10; return aifsn * slot_time + sifs; } @@ -350,6 +379,7 @@ static void rtw89_ops_bss_info_changed(struct ieee80211_hw *hw, rtw89_phy_set_bss_color(rtwdev, vif); rtw89_chip_cfg_txpwr_ul_tb_offset(rtwdev, vif); rtw89_mac_port_update(rtwdev, rtwvif); + rtw89_mac_set_he_obss_narrow_bw_ru(rtwdev, vif); rtw89_store_op_chan(rtwdev, true); } else { /* Abort ongoing scan if cancel_scan isn't issued @@ -378,6 +408,9 @@ static void rtw89_ops_bss_info_changed(struct ieee80211_hw *hw, if (changed & BSS_CHANGED_MU_GROUPS) rtw89_mac_bf_set_gid_table(rtwdev, vif, conf); + if (changed & BSS_CHANGED_P2P_PS) + rtw89_process_p2p_ps(rtwdev, vif); + mutex_unlock(&rtwdev->mutex); } @@ -605,6 +638,20 @@ static void rtw89_ops_sta_statistics(struct ieee80211_hw *hw, sinfo->filled |= BIT_ULL(NL80211_STA_INFO_TX_BITRATE); } +static +void __rtw89_drop_packets(struct rtw89_dev *rtwdev, struct ieee80211_vif *vif) +{ + struct rtw89_vif *rtwvif; + + if (vif) { + rtwvif = (struct rtw89_vif *)vif->drv_priv; + rtw89_mac_pkt_drop_vif(rtwdev, rtwvif); + } else { + rtw89_for_each_rtwvif(rtwdev, rtwvif) + rtw89_mac_pkt_drop_vif(rtwdev, rtwvif); + } +} + static void rtw89_ops_flush(struct ieee80211_hw *hw, struct ieee80211_vif *vif, u32 queues, bool drop) { @@ -613,7 +660,12 @@ static void rtw89_ops_flush(struct ieee80211_hw *hw, struct ieee80211_vif *vif, mutex_lock(&rtwdev->mutex); rtw89_leave_lps(rtwdev); rtw89_hci_flush_queues(rtwdev, queues, drop); - rtw89_mac_flush_txq(rtwdev, queues, drop); + + if (drop && RTW89_CHK_FW_FEATURE(PACKET_DROP, &rtwdev->fw)) + __rtw89_drop_packets(rtwdev, vif); + else + rtw89_mac_flush_txq(rtwdev, queues, drop); + mutex_unlock(&rtwdev->mutex); } @@ -629,7 +681,7 @@ static void rtw89_ra_mask_info_update_iter(void *data, struct ieee80211_sta *sta struct rtw89_sta *rtwsta = (struct rtw89_sta *)sta->drv_priv; struct ieee80211_vif *vif = rtwvif_to_vif(rtwsta->rtwvif); - if (vif != br_data->vif) + if (vif != br_data->vif || vif->p2p) return; rtwsta->use_cfg_mask = true; @@ -669,12 +721,13 @@ int rtw89_ops_set_antenna(struct ieee80211_hw *hw, u32 tx_ant, u32 rx_ant) struct rtw89_dev *rtwdev = hw->priv; struct rtw89_hal *hal = &rtwdev->hal; - if (rx_ant != hw->wiphy->available_antennas_rx) + if (rx_ant != hw->wiphy->available_antennas_rx && rx_ant != hal->antenna_rx) return -EINVAL; mutex_lock(&rtwdev->mutex); hal->antenna_tx = tx_ant; hal->antenna_rx = rx_ant; + hal->tx_path_diversity = false; mutex_unlock(&rtwdev->mutex); return 0; @@ -772,6 +825,97 @@ static void rtw89_ops_sta_rc_update(struct ieee80211_hw *hw, rtw89_phy_ra_updata_sta(rtwdev, sta, changed); } +static int rtw89_ops_add_chanctx(struct ieee80211_hw *hw, + struct ieee80211_chanctx_conf *ctx) +{ + struct rtw89_dev *rtwdev = hw->priv; + int ret; + + mutex_lock(&rtwdev->mutex); + ret = rtw89_chanctx_ops_add(rtwdev, ctx); + mutex_unlock(&rtwdev->mutex); + + return ret; +} + +static void rtw89_ops_remove_chanctx(struct ieee80211_hw *hw, + struct ieee80211_chanctx_conf *ctx) +{ + struct rtw89_dev *rtwdev = hw->priv; + + mutex_lock(&rtwdev->mutex); + rtw89_chanctx_ops_remove(rtwdev, ctx); + mutex_unlock(&rtwdev->mutex); +} + +static void rtw89_ops_change_chanctx(struct ieee80211_hw *hw, + struct ieee80211_chanctx_conf *ctx, + u32 changed) +{ + struct rtw89_dev *rtwdev = hw->priv; + + mutex_lock(&rtwdev->mutex); + rtw89_chanctx_ops_change(rtwdev, ctx, changed); + mutex_unlock(&rtwdev->mutex); +} + +static int rtw89_ops_assign_vif_chanctx(struct ieee80211_hw *hw, + struct ieee80211_vif *vif, + struct ieee80211_bss_conf *link_conf, + struct ieee80211_chanctx_conf *ctx) +{ + struct rtw89_dev *rtwdev = hw->priv; + struct rtw89_vif *rtwvif = (struct rtw89_vif *)vif->drv_priv; + int ret; + + mutex_lock(&rtwdev->mutex); + ret = rtw89_chanctx_ops_assign_vif(rtwdev, rtwvif, ctx); + mutex_unlock(&rtwdev->mutex); + + return ret; +} + +static void rtw89_ops_unassign_vif_chanctx(struct ieee80211_hw *hw, + struct ieee80211_vif *vif, + struct ieee80211_bss_conf *link_conf, + struct ieee80211_chanctx_conf *ctx) +{ + struct rtw89_dev *rtwdev = hw->priv; + struct rtw89_vif *rtwvif = (struct rtw89_vif *)vif->drv_priv; + + mutex_lock(&rtwdev->mutex); + rtw89_chanctx_ops_unassign_vif(rtwdev, rtwvif, ctx); + mutex_unlock(&rtwdev->mutex); +} + +static void rtw89_set_tid_config_iter(void *data, struct ieee80211_sta *sta) +{ + struct cfg80211_tid_config *tid_config = data; + struct rtw89_sta *rtwsta = (struct rtw89_sta *)sta->drv_priv; + struct rtw89_dev *rtwdev = rtwsta->rtwvif->rtwdev; + + rtw89_core_set_tid_config(rtwdev, sta, tid_config); +} + +static int rtw89_ops_set_tid_config(struct ieee80211_hw *hw, + struct ieee80211_vif *vif, + struct ieee80211_sta *sta, + struct cfg80211_tid_config *tid_config) +{ + struct rtw89_dev *rtwdev = hw->priv; + + mutex_lock(&rtwdev->mutex); + if (sta) + rtw89_core_set_tid_config(rtwdev, sta, tid_config); + else + ieee80211_iterate_stations_atomic(rtwdev->hw, + rtw89_set_tid_config_iter, + tid_config); + mutex_unlock(&rtwdev->mutex); + + return 0; +} + const struct ieee80211_ops rtw89_ops = { .tx = rtw89_ops_tx, .wake_tx_queue = rtw89_ops_wake_tx_queue, @@ -779,6 +923,7 @@ const struct ieee80211_ops rtw89_ops = { .stop = rtw89_ops_stop, .config = rtw89_ops_config, .add_interface = rtw89_ops_add_interface, + .change_interface = rtw89_ops_change_interface, .remove_interface = rtw89_ops_remove_interface, .configure_filter = rtw89_ops_configure_filter, .bss_info_changed = rtw89_ops_bss_info_changed, @@ -800,7 +945,13 @@ const struct ieee80211_ops rtw89_ops = { .reconfig_complete = rtw89_ops_reconfig_complete, .hw_scan = rtw89_ops_hw_scan, .cancel_hw_scan = rtw89_ops_cancel_hw_scan, + .add_chanctx = rtw89_ops_add_chanctx, + .remove_chanctx = rtw89_ops_remove_chanctx, + .change_chanctx = rtw89_ops_change_chanctx, + .assign_vif_chanctx = rtw89_ops_assign_vif_chanctx, + .unassign_vif_chanctx = rtw89_ops_unassign_vif_chanctx, .set_sar_specs = rtw89_ops_set_sar_specs, .sta_rc_update = rtw89_ops_sta_rc_update, + .set_tid_config = rtw89_ops_set_tid_config, }; EXPORT_SYMBOL(rtw89_ops); diff --git a/drivers/net/wireless/realtek/rtw89/pci.c b/drivers/net/wireless/realtek/rtw89/pci.c index c68fec9eb5a6..5f8e19639362 100644 --- a/drivers/net/wireless/realtek/rtw89/pci.c +++ b/drivers/net/wireless/realtek/rtw89/pci.c @@ -169,6 +169,23 @@ static int rtw89_pci_rxbd_info_update(struct rtw89_dev *rtwdev, return 0; } +static void rtw89_pci_ctrl_txdma_ch_pcie(struct rtw89_dev *rtwdev, bool enable) +{ + const struct rtw89_pci_info *info = rtwdev->pci_info; + const struct rtw89_reg_def *dma_stop1 = &info->dma_stop1; + const struct rtw89_reg_def *dma_stop2 = &info->dma_stop2; + + if (enable) { + rtw89_write32_clr(rtwdev, dma_stop1->addr, dma_stop1->mask); + if (dma_stop2->addr) + rtw89_write32_clr(rtwdev, dma_stop2->addr, dma_stop2->mask); + } else { + rtw89_write32_set(rtwdev, dma_stop1->addr, dma_stop1->mask); + if (dma_stop2->addr) + rtw89_write32_set(rtwdev, dma_stop2->addr, dma_stop2->mask); + } +} + static bool rtw89_skb_put_rx_data(struct rtw89_dev *rtwdev, bool fs, bool ls, struct sk_buff *new, @@ -760,7 +777,8 @@ static irqreturn_t rtw89_pci_interrupt_threadfn(int irq, void *dev) enable_intr: spin_lock_irqsave(&rtwpci->irq_lock, flags); - rtw89_chip_enable_intr(rtwdev, rtwpci); + if (likely(rtwpci->running)) + rtw89_chip_enable_intr(rtwdev, rtwpci); spin_unlock_irqrestore(&rtwpci->irq_lock, flags); return IRQ_HANDLED; } @@ -925,10 +943,12 @@ u32 __rtw89_pci_check_and_reclaim_tx_resource_noio(struct rtw89_dev *rtwdev, { struct rtw89_pci *rtwpci = (struct rtw89_pci *)rtwdev->priv; struct rtw89_pci_tx_ring *tx_ring = &rtwpci->tx_rings[txch]; + struct rtw89_pci_tx_wd_ring *wd_ring = &tx_ring->wd_ring; u32 cnt; spin_lock_bh(&rtwpci->trx_lock); cnt = rtw89_pci_get_avail_txbd_num(tx_ring); + cnt = min(cnt, wd_ring->curr_num); spin_unlock_bh(&rtwpci->trx_lock); return cnt; @@ -1073,12 +1093,15 @@ static void __pci_flush_txch(struct rtw89_dev *rtwdev, u8 txch, bool drop) static void __rtw89_pci_ops_flush_txchs(struct rtw89_dev *rtwdev, u32 txchs, bool drop) { + const struct rtw89_pci_info *info = rtwdev->pci_info; u8 i; for (i = 0; i < RTW89_TXCH_NUM; i++) { /* It may be unnecessary to flush FWCMD queue. */ if (i == RTW89_TXCH_CH12) continue; + if (info->tx_dma_ch_mask & BIT(i)) + continue; if (txchs & BIT(i)) __pci_flush_txch(rtwdev, i, drop); @@ -1357,6 +1380,7 @@ static const struct rtw89_pci_bd_ram bd_ram_table[RTW89_TXCH_NUM] = { static void rtw89_pci_reset_trx_rings(struct rtw89_dev *rtwdev) { struct rtw89_pci *rtwpci = (struct rtw89_pci *)rtwdev->priv; + const struct rtw89_pci_info *info = rtwdev->pci_info; struct rtw89_pci_tx_ring *tx_ring; struct rtw89_pci_rx_ring *rx_ring; struct rtw89_pci_dma_ring *bd_ring; @@ -1368,6 +1392,9 @@ static void rtw89_pci_reset_trx_rings(struct rtw89_dev *rtwdev) int i; for (i = 0; i < RTW89_TXCH_NUM; i++) { + if (info->tx_dma_ch_mask & BIT(i)) + continue; + tx_ring = &rtwpci->tx_rings[i]; bd_ring = &tx_ring->bd_ring; bd_ram = &bd_ram_table[i]; @@ -1411,12 +1438,15 @@ static void rtw89_pci_release_tx_ring(struct rtw89_dev *rtwdev, static void rtw89_pci_ops_reset(struct rtw89_dev *rtwdev) { struct rtw89_pci *rtwpci = (struct rtw89_pci *)rtwdev->priv; + const struct rtw89_pci_info *info = rtwdev->pci_info; int txch; rtw89_pci_reset_trx_rings(rtwdev); spin_lock_bh(&rtwpci->trx_lock); for (txch = 0; txch < RTW89_TXCH_NUM; txch++) { + if (info->tx_dma_ch_mask & BIT(txch)) + continue; if (txch == RTW89_TXCH_CH12) { rtw89_pci_release_fwcmd(rtwdev, rtwpci, skb_queue_len(&rtwpci->h2c_queue), true); @@ -1604,33 +1634,41 @@ static void rtw89_pci_ops_write32(struct rtw89_dev *rtwdev, u32 addr, u32 data) writel(data, rtwpci->mmap + addr); } -static void rtw89_pci_ctrl_dma_all(struct rtw89_dev *rtwdev, bool enable) +static void rtw89_pci_ctrl_dma_trx(struct rtw89_dev *rtwdev, bool enable) { - enum rtw89_core_chip_id chip_id = rtwdev->chip->chip_id; const struct rtw89_pci_info *info = rtwdev->pci_info; - u32 txhci_en = info->txhci_en_bit; - u32 rxhci_en = info->rxhci_en_bit; - if (enable) { - if (chip_id != RTL8852C) - rtw89_write32_clr(rtwdev, info->dma_stop1_reg, - B_AX_STOP_PCIEIO); - rtw89_write32_set(rtwdev, R_AX_PCIE_INIT_CFG1, - txhci_en | rxhci_en); - if (chip_id == RTL8852C) - rtw89_write32_clr(rtwdev, R_AX_PCIE_INIT_CFG1, - B_AX_STOP_AXI_MST); + if (enable) + rtw89_write32_set(rtwdev, info->init_cfg_reg, + info->rxhci_en_bit | info->txhci_en_bit); + else + rtw89_write32_clr(rtwdev, info->init_cfg_reg, + info->rxhci_en_bit | info->txhci_en_bit); +} + +static void rtw89_pci_ctrl_dma_io(struct rtw89_dev *rtwdev, bool enable) +{ + enum rtw89_core_chip_id chip_id = rtwdev->chip->chip_id; + u32 reg, mask; + + if (chip_id == RTL8852C) { + reg = R_AX_HAXI_INIT_CFG1; + mask = B_AX_STOP_AXI_MST; } else { - if (chip_id != RTL8852C) - rtw89_write32_set(rtwdev, info->dma_stop1_reg, - B_AX_STOP_PCIEIO); - else - rtw89_write32_clr(rtwdev, R_AX_PCIE_INIT_CFG1, - B_AX_STOP_AXI_MST); - if (chip_id == RTL8852C) - rtw89_write32_set(rtwdev, R_AX_PCIE_INIT_CFG1, - B_AX_STOP_AXI_MST); + reg = R_AX_PCIE_DMA_STOP1; + mask = B_AX_STOP_PCIEIO; } + + if (enable) + rtw89_write32_clr(rtwdev, reg, mask); + else + rtw89_write32_set(rtwdev, reg, mask); +} + +static void rtw89_pci_ctrl_dma_all(struct rtw89_dev *rtwdev, bool enable) +{ + rtw89_pci_ctrl_dma_io(rtwdev, enable); + rtw89_pci_ctrl_dma_trx(rtwdev, enable); } static int rtw89_pci_check_mdio(struct rtw89_dev *rtwdev, u8 addr, u8 speed, u16 rw_bit) @@ -1836,6 +1874,18 @@ __get_target(struct rtw89_dev *rtwdev, u16 *target, enum rtw89_pcie_phy phy_rate return 0; } +static int rtw89_pci_autok_x(struct rtw89_dev *rtwdev) +{ + int ret; + + if (rtwdev->chip->chip_id != RTL8852B) + return 0; + + ret = rtw89_write16_mdio_mask(rtwdev, RAC_REG_FLD_0, BAC_AUTOK_N_MASK, + PCIE_AUTOK_4, PCIE_PHY_GEN1); + return ret; +} + static int rtw89_pci_auto_refclk_cal(struct rtw89_dev *rtwdev, bool autook_en) { enum rtw89_pcie_phy phy_rate; @@ -2049,7 +2099,7 @@ static u32 rtw89_pci_l2_rxen_lat(struct rtw89_dev *rtwdev) static void rtw89_pci_aphy_pwrcut(struct rtw89_dev *rtwdev) { - if (rtwdev->chip->chip_id != RTL8852A) + if (rtwdev->chip->chip_id != RTL8852A && rtwdev->chip->chip_id != RTL8852B) return; rtw89_write32_clr(rtwdev, R_AX_SYS_PW_CTRL, B_AX_PSUS_OFF_CAPC_EN); @@ -2234,19 +2284,19 @@ static int rtw89_poll_txdma_ch_idle_pcie(struct rtw89_dev *rtwdev) { const struct rtw89_pci_info *info = rtwdev->pci_info; u32 ret, check, dma_busy; - u32 dma_busy1 = info->dma_busy1_reg; + u32 dma_busy1 = info->dma_busy1.addr; u32 dma_busy2 = info->dma_busy2_reg; - check = B_AX_ACH0_BUSY | B_AX_ACH1_BUSY | B_AX_ACH2_BUSY | - B_AX_ACH3_BUSY | B_AX_ACH4_BUSY | B_AX_ACH5_BUSY | - B_AX_ACH6_BUSY | B_AX_ACH7_BUSY | B_AX_CH8_BUSY | - B_AX_CH9_BUSY | B_AX_CH12_BUSY; + check = info->dma_busy1.mask; ret = read_poll_timeout(rtw89_read32, dma_busy, (dma_busy & check) == 0, 10, 100, false, rtwdev, dma_busy1); if (ret) return ret; + if (!dma_busy2) + return 0; + check = B_AX_CH10_BUSY | B_AX_CH11_BUSY; ret = read_poll_timeout(rtw89_read32, dma_busy, (dma_busy & check) == 0, @@ -2414,6 +2464,12 @@ static int rtw89_pci_ops_mac_pre_init(struct rtw89_dev *rtwdev) rtw89_pci_hci_ldo(rtwdev); rtw89_pci_dphy_delay(rtwdev); + ret = rtw89_pci_autok_x(rtwdev); + if (ret) { + rtw89_err(rtwdev, "[ERR] pcie autok_x fail %d\n", ret); + return ret; + } + ret = rtw89_pci_auto_refclk_cal(rtwdev, false); if (ret) { rtw89_err(rtwdev, "[ERR] pcie autok fail %d\n", ret); @@ -2432,7 +2488,7 @@ static int rtw89_pci_ops_mac_pre_init(struct rtw89_dev *rtwdev) rtw89_pci_set_dbg(rtwdev); rtw89_pci_set_keep_reg(rtwdev); - rtw89_write32_set(rtwdev, info->dma_stop1_reg, B_AX_STOP_WPDMA); + rtw89_write32_set(rtwdev, info->dma_stop1.addr, B_AX_STOP_WPDMA); /* stop DMA activities */ rtw89_pci_ctrl_dma_all(rtwdev, false); @@ -2455,10 +2511,9 @@ static int rtw89_pci_ops_mac_pre_init(struct rtw89_dev *rtwdev) return ret; } - /* enable FW CMD queue to download firmware */ - rtw89_write32_set(rtwdev, info->dma_stop1_reg, B_AX_TX_STOP1_ALL); - rtw89_write32_clr(rtwdev, info->dma_stop1_reg, B_AX_STOP_CH12); - rtw89_write32_set(rtwdev, info->dma_stop2_reg, B_AX_TX_STOP2_ALL); + /* disable all channels except to FW CMD channel to download firmware */ + rtw89_pci_ctrl_txdma_ch_pcie(rtwdev, false); + rtw89_write32_clr(rtwdev, info->dma_stop1.addr, B_AX_STOP_CH12); /* start DMA activities */ rtw89_pci_ctrl_dma_all(rtwdev, true); @@ -2486,15 +2541,15 @@ int rtw89_pci_ltr_set(struct rtw89_dev *rtwdev, bool en) if (rtw89_pci_ltr_is_err_reg_val(val)) return -EINVAL; - rtw89_write32_clr(rtwdev, R_AX_LTR_CTRL_0, B_AX_LTR_HW_EN); - rtw89_write32_set(rtwdev, R_AX_LTR_CTRL_0, B_AX_LTR_EN); + rtw89_write32_set(rtwdev, R_AX_LTR_CTRL_0, B_AX_LTR_HW_EN | B_AX_LTR_EN | + B_AX_LTR_WD_NOEMP_CHK); rtw89_write32_mask(rtwdev, R_AX_LTR_CTRL_0, B_AX_LTR_SPACE_IDX_MASK, PCI_LTR_SPC_500US); rtw89_write32_mask(rtwdev, R_AX_LTR_CTRL_0, B_AX_LTR_IDLE_TIMER_IDX_MASK, - PCI_LTR_IDLE_TIMER_800US); + PCI_LTR_IDLE_TIMER_3_2MS); rtw89_write32_mask(rtwdev, R_AX_LTR_CTRL_1, B_AX_LTR_RX0_TH_MASK, 0x28); rtw89_write32_mask(rtwdev, R_AX_LTR_CTRL_1, B_AX_LTR_RX1_TH_MASK, 0x28); - rtw89_write32(rtwdev, R_AX_LTR_IDLE_LATENCY, 0x88e088e0); + rtw89_write32(rtwdev, R_AX_LTR_IDLE_LATENCY, 0x90039003); rtw89_write32(rtwdev, R_AX_LTR_ACTIVE_LATENCY, 0x880b880b); return 0; @@ -2571,11 +2626,10 @@ static int rtw89_pci_ops_mac_post_init(struct rtw89_dev *rtwdev) } /* enable DMA for all queues */ - rtw89_write32_clr(rtwdev, info->dma_stop1_reg, B_AX_TX_STOP1_ALL); - rtw89_write32_clr(rtwdev, info->dma_stop2_reg, B_AX_TX_STOP2_ALL); + rtw89_pci_ctrl_txdma_ch_pcie(rtwdev, true); /* Release PCI IO */ - rtw89_write32_clr(rtwdev, info->dma_stop1_reg, + rtw89_write32_clr(rtwdev, info->dma_stop1.addr, B_AX_STOP_WPDMA | B_AX_STOP_PCIEIO); return 0; @@ -2696,10 +2750,13 @@ static void rtw89_pci_free_tx_rings(struct rtw89_dev *rtwdev, struct pci_dev *pdev) { struct rtw89_pci *rtwpci = (struct rtw89_pci *)rtwdev->priv; + const struct rtw89_pci_info *info = rtwdev->pci_info; struct rtw89_pci_tx_ring *tx_ring; int i; for (i = 0; i < RTW89_TXCH_NUM; i++) { + if (info->tx_dma_ch_mask & BIT(i)) + continue; tx_ring = &rtwpci->tx_rings[i]; rtw89_pci_free_tx_wd_ring(rtwdev, pdev, tx_ring); rtw89_pci_free_tx_ring(rtwdev, pdev, tx_ring); @@ -2887,6 +2944,7 @@ static int rtw89_pci_alloc_tx_rings(struct rtw89_dev *rtwdev, struct pci_dev *pdev) { struct rtw89_pci *rtwpci = (struct rtw89_pci *)rtwdev->priv; + const struct rtw89_pci_info *info = rtwdev->pci_info; struct rtw89_pci_tx_ring *tx_ring; u32 desc_size; u32 len; @@ -2894,6 +2952,8 @@ static int rtw89_pci_alloc_tx_rings(struct rtw89_dev *rtwdev, int ret; for (i = 0; i < RTW89_TXCH_NUM; i++) { + if (info->tx_dma_ch_mask & BIT(i)) + continue; tx_ring = &rtwpci->tx_rings[i]; desc_size = sizeof(struct rtw89_pci_tx_bd_32); len = RTW89_PCI_TXBD_NUM_MAX; @@ -3219,8 +3279,79 @@ static void rtw89_pci_free_irq(struct rtw89_dev *rtwdev, pci_free_irq_vectors(pdev); } +static u16 gray_code_to_bin(u16 gray_code, u32 bit_num) +{ + u16 bin = 0, gray_bit; + u32 bit_idx; + + for (bit_idx = 0; bit_idx < bit_num; bit_idx++) { + gray_bit = (gray_code >> bit_idx) & 0x1; + if (bit_num - bit_idx > 1) + gray_bit ^= (gray_code >> (bit_idx + 1)) & 0x1; + bin |= (gray_bit << bit_idx); + } + + return bin; +} + +static int rtw89_pci_filter_out(struct rtw89_dev *rtwdev) +{ + struct rtw89_pci *rtwpci = (struct rtw89_pci *)rtwdev->priv; + struct pci_dev *pdev = rtwpci->pdev; + u16 val16, filter_out_val; + u32 val, phy_offset; + int ret; + + if (rtwdev->chip->chip_id != RTL8852C) + return 0; + + val = rtw89_read32_mask(rtwdev, R_AX_PCIE_MIX_CFG_V1, B_AX_ASPM_CTRL_MASK); + if (val == B_AX_ASPM_CTRL_L1) + return 0; + + ret = pci_read_config_dword(pdev, RTW89_PCIE_L1_STS_V1, &val); + if (ret) + return ret; + + val = FIELD_GET(RTW89_BCFG_LINK_SPEED_MASK, val); + if (val == RTW89_PCIE_GEN1_SPEED) { + phy_offset = R_RAC_DIRECT_OFFSET_G1; + } else if (val == RTW89_PCIE_GEN2_SPEED) { + phy_offset = R_RAC_DIRECT_OFFSET_G2; + val16 = rtw89_read16(rtwdev, phy_offset + RAC_ANA10 * RAC_MULT); + rtw89_write16_set(rtwdev, phy_offset + RAC_ANA10 * RAC_MULT, + val16 | B_PCIE_BIT_PINOUT_DIS); + rtw89_write16_set(rtwdev, phy_offset + RAC_ANA19 * RAC_MULT, + val16 & ~B_PCIE_BIT_RD_SEL); + + val16 = rtw89_read16_mask(rtwdev, + phy_offset + RAC_ANA1F * RAC_MULT, + FILTER_OUT_EQ_MASK); + val16 = gray_code_to_bin(val16, hweight16(val16)); + filter_out_val = rtw89_read16(rtwdev, phy_offset + RAC_ANA24 * + RAC_MULT); + filter_out_val &= ~REG_FILTER_OUT_MASK; + filter_out_val |= FIELD_PREP(REG_FILTER_OUT_MASK, val16); + + rtw89_write16(rtwdev, phy_offset + RAC_ANA24 * RAC_MULT, + filter_out_val); + rtw89_write16_set(rtwdev, phy_offset + RAC_ANA0A * RAC_MULT, + B_BAC_EQ_SEL); + rtw89_write16_set(rtwdev, + R_RAC_DIRECT_OFFSET_G1 + RAC_ANA0C * RAC_MULT, + B_PCIE_BIT_PSAVE); + } else { + return -EOPNOTSUPP; + } + rtw89_write16_set(rtwdev, phy_offset + RAC_ANA0C * RAC_MULT, + B_PCIE_BIT_PSAVE); + + return 0; +} + static void rtw89_pci_clkreq_set(struct rtw89_dev *rtwdev, bool enable) { + enum rtw89_core_chip_id chip_id = rtwdev->chip->chip_id; int ret; if (rtw89_pci_disable_clkreq) @@ -3231,19 +3362,33 @@ static void rtw89_pci_clkreq_set(struct rtw89_dev *rtwdev, bool enable) if (ret) rtw89_err(rtwdev, "failed to set CLKREQ Delay\n"); - if (enable) - ret = rtw89_pci_config_byte_set(rtwdev, RTW89_PCIE_L1_CTRL, - RTW89_PCIE_BIT_CLK); - else - ret = rtw89_pci_config_byte_clr(rtwdev, RTW89_PCIE_L1_CTRL, - RTW89_PCIE_BIT_CLK); - if (ret) - rtw89_err(rtwdev, "failed to %s CLKREQ_L1, ret=%d", - enable ? "set" : "unset", ret); + if (chip_id == RTL8852A) { + if (enable) + ret = rtw89_pci_config_byte_set(rtwdev, + RTW89_PCIE_L1_CTRL, + RTW89_PCIE_BIT_CLK); + else + ret = rtw89_pci_config_byte_clr(rtwdev, + RTW89_PCIE_L1_CTRL, + RTW89_PCIE_BIT_CLK); + if (ret) + rtw89_err(rtwdev, "failed to %s CLKREQ_L1, ret=%d", + enable ? "set" : "unset", ret); + } else if (chip_id == RTL8852C) { + rtw89_write32_set(rtwdev, R_AX_PCIE_LAT_CTRL, + B_AX_CLK_REQ_SEL_OPT | B_AX_CLK_REQ_SEL); + if (enable) + rtw89_write32_set(rtwdev, R_AX_L1_CLK_CTRL, + B_AX_CLK_REQ_N); + else + rtw89_write32_clr(rtwdev, R_AX_L1_CLK_CTRL, + B_AX_CLK_REQ_N); + } } static void rtw89_pci_aspm_set(struct rtw89_dev *rtwdev, bool enable) { + enum rtw89_core_chip_id chip_id = rtwdev->chip->chip_id; u8 value = 0; int ret; @@ -3262,12 +3407,23 @@ static void rtw89_pci_aspm_set(struct rtw89_dev *rtwdev, bool enable) if (ret) rtw89_err(rtwdev, "failed to read ASPM Delay\n"); - if (enable) - ret = rtw89_pci_config_byte_set(rtwdev, RTW89_PCIE_L1_CTRL, - RTW89_PCIE_BIT_L1); - else - ret = rtw89_pci_config_byte_clr(rtwdev, RTW89_PCIE_L1_CTRL, - RTW89_PCIE_BIT_L1); + if (chip_id == RTL8852A || chip_id == RTL8852B) { + if (enable) + ret = rtw89_pci_config_byte_set(rtwdev, + RTW89_PCIE_L1_CTRL, + RTW89_PCIE_BIT_L1); + else + ret = rtw89_pci_config_byte_clr(rtwdev, + RTW89_PCIE_L1_CTRL, + RTW89_PCIE_BIT_L1); + } else if (chip_id == RTL8852C) { + if (enable) + rtw89_write32_set(rtwdev, R_AX_PCIE_MIX_CFG_V1, + B_AX_ASPM_CTRL_L1); + else + rtw89_write32_clr(rtwdev, R_AX_PCIE_MIX_CFG_V1, + B_AX_ASPM_CTRL_L1); + } if (ret) rtw89_err(rtwdev, "failed to %s ASPM L1, ret=%d", enable ? "set" : "unset", ret); @@ -3328,17 +3484,34 @@ static void rtw89_pci_link_cfg(struct rtw89_dev *rtwdev) static void rtw89_pci_l1ss_set(struct rtw89_dev *rtwdev, bool enable) { + enum rtw89_core_chip_id chip_id = rtwdev->chip->chip_id; int ret; - if (enable) - ret = rtw89_pci_config_byte_set(rtwdev, RTW89_PCIE_TIMER_CTRL, - RTW89_PCIE_BIT_L1SUB); - else - ret = rtw89_pci_config_byte_clr(rtwdev, RTW89_PCIE_TIMER_CTRL, - RTW89_PCIE_BIT_L1SUB); - if (ret) - rtw89_err(rtwdev, "failed to %s L1SS, ret=%d", - enable ? "set" : "unset", ret); + if (chip_id == RTL8852A || chip_id == RTL8852B) { + if (enable) + ret = rtw89_pci_config_byte_set(rtwdev, + RTW89_PCIE_TIMER_CTRL, + RTW89_PCIE_BIT_L1SUB); + else + ret = rtw89_pci_config_byte_clr(rtwdev, + RTW89_PCIE_TIMER_CTRL, + RTW89_PCIE_BIT_L1SUB); + if (ret) + rtw89_err(rtwdev, "failed to %s L1SS, ret=%d", + enable ? "set" : "unset", ret); + } else if (chip_id == RTL8852C) { + ret = rtw89_pci_config_byte_clr(rtwdev, RTW89_PCIE_L1SS_STS_V1, + RTW89_PCIE_BIT_ASPM_L11 | + RTW89_PCIE_BIT_PCI_L11); + if (ret) + rtw89_warn(rtwdev, "failed to unset ASPM L1.1, ret=%d", ret); + if (enable) + rtw89_write32_clr(rtwdev, R_AX_PCIE_MIX_CFG_V1, + B_AX_L1SUB_DISABLE); + else + rtw89_write32_set(rtwdev, R_AX_PCIE_MIX_CFG_V1, + B_AX_L1SUB_DISABLE); + } } static void rtw89_pci_l1ss_cfg(struct rtw89_dev *rtwdev) @@ -3360,26 +3533,6 @@ static void rtw89_pci_l1ss_cfg(struct rtw89_dev *rtwdev) rtw89_pci_l1ss_set(rtwdev, true); } -static void rtw89_pci_ctrl_dma_all_pcie(struct rtw89_dev *rtwdev, u8 en) -{ - const struct rtw89_pci_info *info = rtwdev->pci_info; - u32 val32; - - if (en == MAC_AX_FUNC_EN) { - val32 = B_AX_STOP_PCIEIO; - rtw89_write32_clr(rtwdev, info->dma_stop1_reg, val32); - - val32 = B_AX_TXHCI_EN | B_AX_RXHCI_EN; - rtw89_write32_set(rtwdev, R_AX_PCIE_INIT_CFG1, val32); - } else { - val32 = B_AX_STOP_PCIEIO; - rtw89_write32_set(rtwdev, info->dma_stop1_reg, val32); - - val32 = B_AX_TXHCI_EN | B_AX_RXHCI_EN; - rtw89_write32_clr(rtwdev, R_AX_PCIE_INIT_CFG1, val32); - } -} - static int rtw89_pci_poll_io_idle(struct rtw89_dev *rtwdev) { int ret = 0; @@ -3399,10 +3552,13 @@ static int rtw89_pci_poll_io_idle(struct rtw89_dev *rtwdev) static int rtw89_pci_lv1rst_stop_dma(struct rtw89_dev *rtwdev) { - u32 val, dma_rst = 0; + u32 val; int ret; - rtw89_pci_ctrl_dma_all_pcie(rtwdev, MAC_AX_FUNC_DIS); + if (rtwdev->chip->chip_id == RTL8852C) + return 0; + + rtw89_pci_ctrl_dma_all(rtwdev, false); ret = rtw89_pci_poll_io_idle(rtwdev); if (ret) { val = rtw89_read32(rtwdev, R_AX_DBG_ERR_FLAG); @@ -3410,12 +3566,10 @@ static int rtw89_pci_lv1rst_stop_dma(struct rtw89_dev *rtwdev) "[PCIe] poll_io_idle fail, before 0x%08x: 0x%08x\n", R_AX_DBG_ERR_FLAG, val); if (val & B_AX_TX_STUCK || val & B_AX_PCIE_TXBD_LEN0) - dma_rst |= B_AX_HCI_TXDMA_EN; + rtw89_mac_ctrl_hci_dma_tx(rtwdev, false); if (val & B_AX_RX_STUCK) - dma_rst |= B_AX_HCI_RXDMA_EN; - val = rtw89_read32(rtwdev, R_AX_HCI_FUNC_EN); - rtw89_write32(rtwdev, R_AX_HCI_FUNC_EN, val & ~dma_rst); - rtw89_write32(rtwdev, R_AX_HCI_FUNC_EN, val | dma_rst); + rtw89_mac_ctrl_hci_dma_rx(rtwdev, false); + rtw89_mac_ctrl_hci_dma_trx(rtwdev, true); ret = rtw89_pci_poll_io_idle(rtwdev); val = rtw89_read32(rtwdev, R_AX_DBG_ERR_FLAG); rtw89_debug(rtwdev, RTW89_DBG_HCI, @@ -3426,18 +3580,7 @@ static int rtw89_pci_lv1rst_stop_dma(struct rtw89_dev *rtwdev) return ret; } -static void rtw89_pci_ctrl_hci_dma_en(struct rtw89_dev *rtwdev, u8 en) -{ - u32 val32; - if (en == MAC_AX_FUNC_EN) { - val32 = B_AX_HCI_TXDMA_EN | B_AX_HCI_RXDMA_EN; - rtw89_write32_set(rtwdev, R_AX_HCI_FUNC_EN, val32); - } else { - val32 = B_AX_HCI_TXDMA_EN | B_AX_HCI_RXDMA_EN; - rtw89_write32_clr(rtwdev, R_AX_HCI_FUNC_EN, val32); - } -} static int rtw89_pci_rst_bdram(struct rtw89_dev *rtwdev) { @@ -3457,15 +3600,18 @@ static int rtw89_pci_lv1rst_start_dma(struct rtw89_dev *rtwdev) { u32 ret; - rtw89_pci_ctrl_hci_dma_en(rtwdev, MAC_AX_FUNC_DIS); - rtw89_pci_ctrl_hci_dma_en(rtwdev, MAC_AX_FUNC_EN); + if (rtwdev->chip->chip_id == RTL8852C) + return 0; + + rtw89_mac_ctrl_hci_dma_trx(rtwdev, false); + rtw89_mac_ctrl_hci_dma_trx(rtwdev, true); rtw89_pci_clr_idx_all(rtwdev); ret = rtw89_pci_rst_bdram(rtwdev); if (ret) return ret; - rtw89_pci_ctrl_dma_all_pcie(rtwdev, MAC_AX_FUNC_EN); + rtw89_pci_ctrl_dma_all(rtwdev, true); return ret; } @@ -3535,14 +3681,20 @@ static int __maybe_unused rtw89_pci_suspend(struct device *dev) { struct ieee80211_hw *hw = dev_get_drvdata(dev); struct rtw89_dev *rtwdev = hw->priv; + enum rtw89_core_chip_id chip_id = rtwdev->chip->chip_id; - rtw89_write32_clr(rtwdev, R_AX_SYS_SDIO_CTRL, - B_AX_PCIE_DIS_L2_CTRL_LDO_HCI); rtw89_write32_set(rtwdev, R_AX_RSV_CTRL, B_AX_WLOCK_1C_BIT6); rtw89_write32_set(rtwdev, R_AX_RSV_CTRL, B_AX_R_DIS_PRST); rtw89_write32_clr(rtwdev, R_AX_RSV_CTRL, B_AX_WLOCK_1C_BIT6); - rtw89_write32_set(rtwdev, R_AX_PCIE_INIT_CFG1, - B_AX_PCIE_PERST_KEEP_REG | B_AX_PCIE_TRAIN_KEEP_REG); + if (chip_id == RTL8852A || chip_id == RTL8852B) { + rtw89_write32_clr(rtwdev, R_AX_SYS_SDIO_CTRL, + B_AX_PCIE_DIS_L2_CTRL_LDO_HCI); + rtw89_write32_set(rtwdev, R_AX_PCIE_INIT_CFG1, + B_AX_PCIE_PERST_KEEP_REG | B_AX_PCIE_TRAIN_KEEP_REG); + } else { + rtw89_write32_clr(rtwdev, R_AX_PCIE_PS_CTRL_V1, + B_AX_CMAC_EXIT_L1_EN | B_AX_DMAC0_EXIT_L1_EN); + } return 0; } @@ -3563,15 +3715,24 @@ static int __maybe_unused rtw89_pci_resume(struct device *dev) { struct ieee80211_hw *hw = dev_get_drvdata(dev); struct rtw89_dev *rtwdev = hw->priv; + enum rtw89_core_chip_id chip_id = rtwdev->chip->chip_id; - rtw89_write32_set(rtwdev, R_AX_SYS_SDIO_CTRL, - B_AX_PCIE_DIS_L2_CTRL_LDO_HCI); rtw89_write32_set(rtwdev, R_AX_RSV_CTRL, B_AX_WLOCK_1C_BIT6); rtw89_write32_clr(rtwdev, R_AX_RSV_CTRL, B_AX_R_DIS_PRST); rtw89_write32_clr(rtwdev, R_AX_RSV_CTRL, B_AX_WLOCK_1C_BIT6); - rtw89_write32_clr(rtwdev, R_AX_PCIE_INIT_CFG1, - B_AX_PCIE_PERST_KEEP_REG | B_AX_PCIE_TRAIN_KEEP_REG); + if (chip_id == RTL8852A || chip_id == RTL8852B) { + rtw89_write32_set(rtwdev, R_AX_SYS_SDIO_CTRL, + B_AX_PCIE_DIS_L2_CTRL_LDO_HCI); + rtw89_write32_clr(rtwdev, R_AX_PCIE_INIT_CFG1, + B_AX_PCIE_PERST_KEEP_REG | B_AX_PCIE_TRAIN_KEEP_REG); + } else { + rtw89_write32_set(rtwdev, R_AX_PCIE_PS_CTRL_V1, + B_AX_CMAC_EXIT_L1_EN | B_AX_DMAC0_EXIT_L1_EN); + rtw89_write32_clr(rtwdev, R_AX_PCIE_PS_CTRL_V1, + B_AX_SEL_REQ_ENTR_L1); + } rtw89_pci_l2_hci_ldo(rtwdev); + rtw89_pci_filter_out(rtwdev); rtw89_pci_link_cfg(rtwdev); rtw89_pci_l1ss_cfg(rtwdev); @@ -3614,27 +3775,23 @@ static const struct rtw89_hci_ops rtw89_pci_ops = { int rtw89_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id) { - struct ieee80211_hw *hw; struct rtw89_dev *rtwdev; const struct rtw89_driver_info *info; const struct rtw89_pci_info *pci_info; - int driver_data_size; int ret; - driver_data_size = sizeof(struct rtw89_dev) + sizeof(struct rtw89_pci); - hw = ieee80211_alloc_hw(driver_data_size, &rtw89_ops); - if (!hw) { + info = (const struct rtw89_driver_info *)id->driver_data; + + rtwdev = rtw89_alloc_ieee80211_hw(&pdev->dev, + sizeof(struct rtw89_pci), + info->chip); + if (!rtwdev) { dev_err(&pdev->dev, "failed to allocate hw\n"); return -ENOMEM; } - info = (const struct rtw89_driver_info *)id->driver_data; pci_info = info->bus.pci; - rtwdev = hw->priv; - rtwdev->hw = hw; - rtwdev->dev = &pdev->dev; - rtwdev->chip = info->chip; rtwdev->pci_info = info->bus.pci; rtwdev->hci.ops = &rtw89_pci_ops; rtwdev->hci.type = RTW89_HCI_TYPE_PCIE; @@ -3667,6 +3824,7 @@ int rtw89_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id) goto err_clear_resource; } + rtw89_pci_filter_out(rtwdev); rtw89_pci_link_cfg(rtwdev); rtw89_pci_l1ss_cfg(rtwdev); @@ -3696,7 +3854,7 @@ err_declaim_pci: err_core_deinit: rtw89_core_deinit(rtwdev); err_release_hw: - ieee80211_free_hw(hw); + rtw89_free_ieee80211_hw(rtwdev); return ret; } @@ -3715,7 +3873,7 @@ void rtw89_pci_remove(struct pci_dev *pdev) rtw89_pci_clear_resource(rtwdev, pdev); rtw89_pci_declaim_device(rtwdev, pdev); rtw89_core_deinit(rtwdev); - ieee80211_free_hw(hw); + rtw89_free_ieee80211_hw(rtwdev); } EXPORT_SYMBOL(rtw89_pci_remove); diff --git a/drivers/net/wireless/realtek/rtw89/pci.h b/drivers/net/wireless/realtek/rtw89/pci.h index a118647213e3..179740607778 100644 --- a/drivers/net/wireless/realtek/rtw89/pci.h +++ b/drivers/net/wireless/realtek/rtw89/pci.h @@ -11,11 +11,21 @@ #define MDIO_PG1_G1 1 #define MDIO_PG0_G2 2 #define MDIO_PG1_G2 3 +#define RAC_CTRL_PPR 0x00 +#define RAC_ANA0A 0x0A +#define B_BAC_EQ_SEL BIT(5) +#define RAC_ANA0C 0x0C +#define B_PCIE_BIT_PSAVE BIT(15) #define RAC_ANA10 0x10 +#define B_PCIE_BIT_PINOUT_DIS BIT(3) #define RAC_REG_REV2 0x1B #define BAC_CMU_EN_DLY_MASK GENMASK(15, 12) #define PCIE_DPHY_DLY_25US 0x1 #define RAC_ANA19 0x19 +#define B_PCIE_BIT_RD_SEL BIT(2) +#define RAC_REG_FLD_0 0x1D +#define BAC_AUTOK_N_MASK GENMASK(3, 2) +#define PCIE_AUTOK_4 0x3 #define RAC_ANA1F 0x1F #define RAC_ANA24 0x24 #define B_AX_DEGLITCH GENMASK(11, 8) @@ -45,9 +55,26 @@ #define B_AX_SEL_REQ_ENTR_L1 BIT(2) #define B_AX_SEL_REQ_EXIT_L1 BIT(0) +#define R_AX_PCIE_MIX_CFG_V1 0x300C +#define B_AX_ASPM_CTRL_L1 BIT(17) +#define B_AX_ASPM_CTRL_L0 BIT(16) +#define B_AX_ASPM_CTRL_MASK GENMASK(17, 16) +#define B_AX_XFER_PENDING_FW BIT(11) +#define B_AX_XFER_PENDING BIT(10) +#define B_AX_REQ_EXIT_L1 BIT(9) +#define B_AX_REQ_ENTR_L1 BIT(8) +#define B_AX_L1SUB_DISABLE BIT(0) + +#define R_AX_L1_CLK_CTRL 0x3010 +#define B_AX_CLK_REQ_N BIT(1) + #define R_AX_PCIE_BG_CLR 0x303C #define B_AX_BG_CLR_ASYNC_M3 BIT(4) +#define R_AX_PCIE_LAT_CTRL 0x3044 +#define B_AX_CLK_REQ_SEL_OPT BIT(1) +#define B_AX_CLK_REQ_SEL BIT(0) + #define R_AX_PCIE_IO_RCY_M1 0x3100 #define B_AX_PCIE_IO_RCY_P_M1 BIT(5) #define B_AX_PCIE_IO_RCY_WDT_P_M1 BIT(4) @@ -88,7 +115,10 @@ #define B_AX_PCIE_WDT_TIMER_S1_MASK GENMASK(31, 0) #define R_RAC_DIRECT_OFFSET_G1 0x3800 +#define FILTER_OUT_EQ_MASK GENMASK(14, 10) #define R_RAC_DIRECT_OFFSET_G2 0x3880 +#define REG_FILTER_OUT_MASK GENMASK(6, 2) +#define RAC_MULT 2 #define RTW89_PCI_WR_RETRY_CNT 20 @@ -383,6 +413,16 @@ #define B_AX_STOP_RPQ BIT(1) #define B_AX_STOP_RXQ BIT(0) #define B_AX_TX_STOP1_ALL GENMASK(18, 8) +#define B_AX_TX_STOP1_MASK (B_AX_STOP_ACH0 | B_AX_STOP_ACH1 | \ + B_AX_STOP_ACH2 | B_AX_STOP_ACH3 | \ + B_AX_STOP_ACH4 | B_AX_STOP_ACH5 | \ + B_AX_STOP_ACH6 | B_AX_STOP_ACH7 | \ + B_AX_STOP_CH8 | B_AX_STOP_CH9 | \ + B_AX_STOP_CH12) +#define B_AX_TX_STOP1_MASK_V1 (B_AX_STOP_ACH0 | B_AX_STOP_ACH1 | \ + B_AX_STOP_ACH2 | B_AX_STOP_ACH3 | \ + B_AX_STOP_CH8 | B_AX_STOP_CH9 | \ + B_AX_STOP_CH12) #define R_AX_PCIE_DMA_STOP2 0x1310 #define B_AX_STOP_CH11 BIT(1) @@ -431,6 +471,13 @@ #define B_AX_ACH0_BUSY BIT(8) #define B_AX_RPQ_BUSY BIT(1) #define B_AX_RXQ_BUSY BIT(0) +#define DMA_BUSY1_CHECK (B_AX_ACH0_BUSY | B_AX_ACH1_BUSY | B_AX_ACH2_BUSY | \ + B_AX_ACH3_BUSY | B_AX_ACH4_BUSY | B_AX_ACH5_BUSY | \ + B_AX_ACH6_BUSY | B_AX_ACH7_BUSY | B_AX_CH8_BUSY | \ + B_AX_CH9_BUSY | B_AX_CH12_BUSY) +#define DMA_BUSY1_CHECK_V1 (B_AX_ACH0_BUSY | B_AX_ACH1_BUSY | B_AX_ACH2_BUSY | \ + B_AX_ACH3_BUSY | B_AX_CH8_BUSY | B_AX_CH9_BUSY | \ + B_AX_CH12_BUSY) #define R_AX_PCIE_DMA_BUSY2 0x131C #define B_AX_CH11_BUSY BIT(1) @@ -505,6 +552,17 @@ #define RTW89_PCI_MULTITAG 8 /* PCIE CFG register */ +#define RTW89_PCIE_L1_STS_V1 0x80 +#define RTW89_BCFG_LINK_SPEED_MASK GENMASK(19, 16) +#define RTW89_PCIE_GEN1_SPEED 0x01 +#define RTW89_PCIE_GEN2_SPEED 0x02 +#define RTW89_PCIE_PHY_RATE 0x82 +#define RTW89_PCIE_PHY_RATE_MASK GENMASK(1, 0) +#define RTW89_PCIE_L1SS_STS_V1 0x0168 +#define RTW89_PCIE_BIT_ASPM_L11 BIT(3) +#define RTW89_PCIE_BIT_ASPM_L12 BIT(2) +#define RTW89_PCIE_BIT_PCI_L11 BIT(1) +#define RTW89_PCIE_BIT_PCI_L12 BIT(0) #define RTW89_PCIE_ASPM_CTRL 0x070F #define RTW89_L1DLY_MASK GENMASK(5, 3) #define RTW89_L0DLY_MASK GENMASK(2, 0) @@ -516,8 +574,7 @@ #define RTW89_PCIE_CLK_CTRL 0x0725 #define RTW89_PCIE_RST_MSTATE 0x0B48 #define RTW89_PCIE_BIT_CFG_RST_MSTATE BIT(0) -#define RTW89_PCIE_PHY_RATE 0x82 -#define RTW89_PCIE_PHY_RATE_MASK GENMASK(1, 0) + #define INTF_INTGRA_MINREF_V1 90 #define INTF_INTGRA_HOSTREF_V1 100 @@ -527,11 +584,6 @@ enum rtw89_pcie_phy { PCIE_PHY_GEN1_UNDEFINE = 0x7F, }; -enum mac_ax_func_sw { - MAC_AX_FUNC_DIS, - MAC_AX_FUNC_EN, -}; - enum rtw89_pcie_l0sdly { PCIE_L0SDLY_1US = 0, PCIE_L0SDLY_2US = 1, @@ -710,14 +762,15 @@ struct rtw89_pci_info { u32 max_tag_num_mask; u32 rxbd_rwptr_clr_reg; u32 txbd_rwptr_clr2_reg; - u32 dma_stop1_reg; - u32 dma_stop2_reg; - u32 dma_busy1_reg; + struct rtw89_reg_def dma_stop1; + struct rtw89_reg_def dma_stop2; + struct rtw89_reg_def dma_busy1; u32 dma_busy2_reg; u32 dma_busy3_reg; u32 rpwm_addr; u32 cpwm_addr; + u32 tx_dma_ch_mask; const struct rtw89_pci_bd_idx_addr *bd_idx_addr_low_power; const struct rtw89_pci_ch_dma_addr_set *dma_addr_set; diff --git a/drivers/net/wireless/realtek/rtw89/phy.c b/drivers/net/wireless/realtek/rtw89/phy.c index 1532c0a6bbc4..6a6bdc652e09 100644 --- a/drivers/net/wireless/realtek/rtw89/phy.c +++ b/drivers/net/wireless/realtek/rtw89/phy.c @@ -14,23 +14,14 @@ static u16 get_max_amsdu_len(struct rtw89_dev *rtwdev, const struct rtw89_ra_report *report) { - const struct rate_info *txrate = &report->txrate; u32 bit_rate = report->bit_rate; - u8 mcs; /* lower than ofdm, do not aggregate */ if (bit_rate < 550) return 1; - /* prevent hardware rate fallback to G mode rate */ - if (txrate->flags & RATE_INFO_FLAGS_MCS) - mcs = txrate->mcs & 0x07; - else if (txrate->flags & (RATE_INFO_FLAGS_VHT_MCS | RATE_INFO_FLAGS_HE_MCS)) - mcs = txrate->mcs; - else - mcs = 0; - - if (mcs <= 2) + /* avoid AMSDU for legacy rate */ + if (report->might_fallback_legacy) return 1; /* lower than 20M vht 2ss mcs8, make it small */ @@ -142,8 +133,8 @@ static u64 rtw89_phy_ra_mask_recover(u64 ra_mask, u64 ra_mask_bak) static u64 rtw89_phy_ra_mask_cfg(struct rtw89_dev *rtwdev, struct rtw89_sta *rtwsta) { - struct rtw89_hal *hal = &rtwdev->hal; struct ieee80211_sta *sta = rtwsta_to_sta(rtwsta); + const struct rtw89_chan *chan = rtw89_chan_get(rtwdev, RTW89_SUB_ENTITY_0); struct cfg80211_bitrate_mask *mask = &rtwsta->mask; enum nl80211_band band; u64 cfg_mask; @@ -151,7 +142,7 @@ static u64 rtw89_phy_ra_mask_cfg(struct rtw89_dev *rtwdev, struct rtw89_sta *rtw if (!rtwsta->use_cfg_mask) return -1; - switch (hal->current_band_type) { + switch (chan->band_type) { case RTW89_BAND_2G: band = NL80211_BAND_2GHZ; cfg_mask = u64_encode_bits(mask->control[NL80211_BAND_2GHZ].legacy, @@ -168,7 +159,7 @@ static u64 rtw89_phy_ra_mask_cfg(struct rtw89_dev *rtwdev, struct rtw89_sta *rtw RA_MASK_OFDM_RATES); break; default: - rtw89_warn(rtwdev, "unhandled band type %d\n", hal->current_band_type); + rtw89_warn(rtwdev, "unhandled band type %d\n", chan->band_type); return -1; } @@ -202,6 +193,40 @@ static const u64 rtw89_ra_mask_he_rates[4] = {RA_MASK_HE_1SS_RATES, RA_MASK_HE_2SS_RATES, RA_MASK_HE_3SS_RATES, RA_MASK_HE_4SS_RATES}; +static void rtw89_phy_ra_gi_ltf(struct rtw89_dev *rtwdev, + struct rtw89_sta *rtwsta, + bool *fix_giltf_en, u8 *fix_giltf) +{ + const struct rtw89_chan *chan = rtw89_chan_get(rtwdev, RTW89_SUB_ENTITY_0); + struct cfg80211_bitrate_mask *mask = &rtwsta->mask; + u8 band = chan->band_type; + enum nl80211_band nl_band = rtw89_hw_to_nl80211_band(band); + u8 he_gi = mask->control[nl_band].he_gi; + u8 he_ltf = mask->control[nl_band].he_ltf; + + if (!rtwsta->use_cfg_mask) + return; + + if (he_ltf == 2 && he_gi == 2) { + *fix_giltf = RTW89_GILTF_LGI_4XHE32; + } else if (he_ltf == 2 && he_gi == 0) { + *fix_giltf = RTW89_GILTF_SGI_4XHE08; + } else if (he_ltf == 1 && he_gi == 1) { + *fix_giltf = RTW89_GILTF_2XHE16; + } else if (he_ltf == 1 && he_gi == 0) { + *fix_giltf = RTW89_GILTF_2XHE08; + } else if (he_ltf == 0 && he_gi == 1) { + *fix_giltf = RTW89_GILTF_1XHE16; + } else if (he_ltf == 0 && he_gi == 0) { + *fix_giltf = RTW89_GILTF_1XHE08; + } else { + *fix_giltf_en = false; + return; + } + + *fix_giltf_en = true; +} + static void rtw89_phy_ra_sta_update(struct rtw89_dev *rtwdev, struct ieee80211_sta *sta, bool csi) { @@ -209,6 +234,8 @@ static void rtw89_phy_ra_sta_update(struct rtw89_dev *rtwdev, struct rtw89_vif *rtwvif = rtwsta->rtwvif; struct rtw89_phy_rate_pattern *rate_pattern = &rtwvif->rate_pattern; struct rtw89_ra_info *ra = &rtwsta->ra; + const struct rtw89_chan *chan = rtw89_chan_get(rtwdev, RTW89_SUB_ENTITY_0); + struct ieee80211_vif *vif = rtwvif_to_vif(rtwsta->rtwvif); const u64 *high_rate_masks = rtw89_ra_mask_ht_rates; u8 rssi = ewma_rssi_read(&rtwsta->avg_rssi); u64 ra_mask = 0; @@ -218,8 +245,10 @@ static void rtw89_phy_ra_sta_update(struct rtw89_dev *rtwdev, u8 bw_mode = 0; u8 stbc_en = 0; u8 ldpc_en = 0; + u8 fix_giltf = 0; u8 i; bool sgi = false; + bool fix_giltf_en = false; memset(ra, 0, sizeof(*ra)); /* Set the ra mask from sta's capability */ @@ -234,6 +263,7 @@ static void rtw89_phy_ra_sta_update(struct rtw89_dev *rtwdev, if (sta->deflink.he_cap.he_cap_elem.phy_cap_info[1] & IEEE80211_HE_PHY_CAP1_LDPC_CODING_IN_PAYLOAD) ldpc_en = 1; + rtw89_phy_ra_gi_ltf(rtwdev, rtwsta, &fix_giltf_en, &fix_giltf); } else if (sta->deflink.vht_cap.vht_supported) { u16 mcs_map = le16_to_cpu(sta->deflink.vht_cap.vht_mcs.rx_mcs_map); @@ -260,13 +290,13 @@ static void rtw89_phy_ra_sta_update(struct rtw89_dev *rtwdev, ldpc_en = 1; } - switch (rtwdev->hal.current_band_type) { + switch (chan->band_type) { case RTW89_BAND_2G: ra_mask |= sta->deflink.supp_rates[NL80211_BAND_2GHZ]; - if (sta->deflink.supp_rates[NL80211_BAND_2GHZ] <= 0xf) + if (sta->deflink.supp_rates[NL80211_BAND_2GHZ] & 0xf) mode |= RTW89_RA_MODE_CCK; - else - mode |= RTW89_RA_MODE_CCK | RTW89_RA_MODE_OFDM; + if (sta->deflink.supp_rates[NL80211_BAND_2GHZ] & 0xff0) + mode |= RTW89_RA_MODE_OFDM; break; case RTW89_BAND_5G: ra_mask |= (u64)sta->deflink.supp_rates[NL80211_BAND_5GHZ] << 4; @@ -329,7 +359,7 @@ static void rtw89_phy_ra_sta_update(struct rtw89_dev *rtwdev, IEEE80211_HE_PHY_CAP3_DCM_MAX_CONST_RX_16_QAM) ra->dcm_cap = 1; - if (rate_pattern->enable) { + if (rate_pattern->enable && !vif->p2p) { ra_mask = rtw89_phy_ra_mask_cfg(rtwdev, rtwsta); ra_mask &= rate_pattern->ra_mask; mode = rate_pattern->ra_mode; @@ -343,6 +373,8 @@ static void rtw89_phy_ra_sta_update(struct rtw89_dev *rtwdev, ra->ss_num = min(sta->deflink.rx_nss, rtwdev->hal.tx_nss) - 1; ra->en_sgi = sgi; ra->ra_mask = ra_mask; + ra->fix_giltf_en = fix_giltf_en; + ra->fix_giltf = fix_giltf; if (!csi) return; @@ -416,6 +448,7 @@ void rtw89_phy_rate_pattern_vif(struct rtw89_dev *rtwdev, struct ieee80211_supported_band *sband; struct rtw89_vif *rtwvif = (struct rtw89_vif *)vif->drv_priv; struct rtw89_phy_rate_pattern next_pattern = {0}; + const struct rtw89_chan *chan = rtw89_chan_get(rtwdev, RTW89_SUB_ENTITY_0); static const u16 hw_rate_he[] = {RTW89_HW_RATE_HE_NSS1_MCS0, RTW89_HW_RATE_HE_NSS2_MCS0, RTW89_HW_RATE_HE_NSS3_MCS0, @@ -428,7 +461,7 @@ void rtw89_phy_rate_pattern_vif(struct rtw89_dev *rtwdev, RTW89_HW_RATE_MCS8, RTW89_HW_RATE_MCS16, RTW89_HW_RATE_MCS24}; - u8 band = rtwdev->hal.current_band_type; + u8 band = chan->band_type; enum nl80211_band nl_band = rtw89_hw_to_nl80211_band(band); u8 tx_nss = rtwdev->hal.tx_nss; u8 i; @@ -542,12 +575,12 @@ void rtw89_phy_ra_assoc(struct rtw89_dev *rtwdev, struct ieee80211_sta *sta) } u8 rtw89_phy_get_txsc(struct rtw89_dev *rtwdev, - struct rtw89_channel_params *param, + const struct rtw89_chan *chan, enum rtw89_bandwidth dbw) { - enum rtw89_bandwidth cbw = param->bandwidth; - u8 pri_ch = param->primary_chan; - u8 central_ch = param->center_chan; + enum rtw89_bandwidth cbw = chan->band_width; + u8 pri_ch = chan->primary_channel; + u8 central_ch = chan->channel; u8 txsc_idx = 0; u8 tmp = 0; @@ -1468,10 +1501,9 @@ EXPORT_SYMBOL(rtw89_phy_load_txpwr_byrate); (txpwr_rf) >> (__c->txpwr_factor_rf - __c->txpwr_factor_mac); \ }) -s8 rtw89_phy_read_txpwr_byrate(struct rtw89_dev *rtwdev, +s8 rtw89_phy_read_txpwr_byrate(struct rtw89_dev *rtwdev, u8 band, const struct rtw89_rate_desc *rate_desc) { - enum rtw89_band band = rtwdev->hal.current_band_type; s8 *byr; u8 idx; @@ -1538,11 +1570,10 @@ static u8 rtw89_channel_to_idx(struct rtw89_dev *rtwdev, u8 band, u8 channel) } } -s8 rtw89_phy_read_txpwr_limit(struct rtw89_dev *rtwdev, +s8 rtw89_phy_read_txpwr_limit(struct rtw89_dev *rtwdev, u8 band, u8 bw, u8 ntx, u8 rs, u8 bf, u8 ch) { const struct rtw89_chip_info *chip = rtwdev->chip; - u8 band = rtwdev->hal.current_band_type; u8 ch_idx = rtw89_channel_to_idx(rtwdev, band, ch); u8 regd = rtw89_regd_get(rtwdev, band); s8 lmt = 0, sar; @@ -1578,11 +1609,12 @@ s8 rtw89_phy_read_txpwr_limit(struct rtw89_dev *rtwdev, } EXPORT_SYMBOL(rtw89_phy_read_txpwr_limit); -#define __fill_txpwr_limit_nonbf_bf(ptr, bw, ntx, rs, ch) \ +#define __fill_txpwr_limit_nonbf_bf(ptr, band, bw, ntx, rs, ch) \ do { \ u8 __i; \ for (__i = 0; __i < RTW89_BF_NUM; __i++) \ ptr[__i] = rtw89_phy_read_txpwr_limit(rtwdev, \ + band, \ bw, ntx, \ rs, __i, \ (ch)); \ @@ -1590,64 +1622,75 @@ EXPORT_SYMBOL(rtw89_phy_read_txpwr_limit); static void rtw89_phy_fill_txpwr_limit_20m(struct rtw89_dev *rtwdev, struct rtw89_txpwr_limit *lmt, - u8 ntx, u8 ch) + u8 band, u8 ntx, u8 ch) { - __fill_txpwr_limit_nonbf_bf(lmt->cck_20m, RTW89_CHANNEL_WIDTH_20, + __fill_txpwr_limit_nonbf_bf(lmt->cck_20m, band, RTW89_CHANNEL_WIDTH_20, ntx, RTW89_RS_CCK, ch); - __fill_txpwr_limit_nonbf_bf(lmt->cck_40m, RTW89_CHANNEL_WIDTH_40, + __fill_txpwr_limit_nonbf_bf(lmt->cck_40m, band, RTW89_CHANNEL_WIDTH_40, ntx, RTW89_RS_CCK, ch); - __fill_txpwr_limit_nonbf_bf(lmt->ofdm, RTW89_CHANNEL_WIDTH_20, + __fill_txpwr_limit_nonbf_bf(lmt->ofdm, band, RTW89_CHANNEL_WIDTH_20, ntx, RTW89_RS_OFDM, ch); - __fill_txpwr_limit_nonbf_bf(lmt->mcs_20m[0], RTW89_CHANNEL_WIDTH_20, + __fill_txpwr_limit_nonbf_bf(lmt->mcs_20m[0], band, + RTW89_CHANNEL_WIDTH_20, ntx, RTW89_RS_MCS, ch); } static void rtw89_phy_fill_txpwr_limit_40m(struct rtw89_dev *rtwdev, struct rtw89_txpwr_limit *lmt, - u8 ntx, u8 ch, u8 pri_ch) + u8 band, u8 ntx, u8 ch, u8 pri_ch) { - __fill_txpwr_limit_nonbf_bf(lmt->cck_20m, RTW89_CHANNEL_WIDTH_20, + __fill_txpwr_limit_nonbf_bf(lmt->cck_20m, band, RTW89_CHANNEL_WIDTH_20, ntx, RTW89_RS_CCK, ch - 2); - __fill_txpwr_limit_nonbf_bf(lmt->cck_40m, RTW89_CHANNEL_WIDTH_40, + __fill_txpwr_limit_nonbf_bf(lmt->cck_40m, band, RTW89_CHANNEL_WIDTH_40, ntx, RTW89_RS_CCK, ch); - __fill_txpwr_limit_nonbf_bf(lmt->ofdm, RTW89_CHANNEL_WIDTH_20, + __fill_txpwr_limit_nonbf_bf(lmt->ofdm, band, RTW89_CHANNEL_WIDTH_20, ntx, RTW89_RS_OFDM, pri_ch); - __fill_txpwr_limit_nonbf_bf(lmt->mcs_20m[0], RTW89_CHANNEL_WIDTH_20, + __fill_txpwr_limit_nonbf_bf(lmt->mcs_20m[0], band, + RTW89_CHANNEL_WIDTH_20, ntx, RTW89_RS_MCS, ch - 2); - __fill_txpwr_limit_nonbf_bf(lmt->mcs_20m[1], RTW89_CHANNEL_WIDTH_20, + __fill_txpwr_limit_nonbf_bf(lmt->mcs_20m[1], band, + RTW89_CHANNEL_WIDTH_20, ntx, RTW89_RS_MCS, ch + 2); - __fill_txpwr_limit_nonbf_bf(lmt->mcs_40m[0], RTW89_CHANNEL_WIDTH_40, + __fill_txpwr_limit_nonbf_bf(lmt->mcs_40m[0], band, + RTW89_CHANNEL_WIDTH_40, ntx, RTW89_RS_MCS, ch); } static void rtw89_phy_fill_txpwr_limit_80m(struct rtw89_dev *rtwdev, struct rtw89_txpwr_limit *lmt, - u8 ntx, u8 ch, u8 pri_ch) + u8 band, u8 ntx, u8 ch, u8 pri_ch) { s8 val_0p5_n[RTW89_BF_NUM]; s8 val_0p5_p[RTW89_BF_NUM]; u8 i; - __fill_txpwr_limit_nonbf_bf(lmt->ofdm, RTW89_CHANNEL_WIDTH_20, + __fill_txpwr_limit_nonbf_bf(lmt->ofdm, band, RTW89_CHANNEL_WIDTH_20, ntx, RTW89_RS_OFDM, pri_ch); - __fill_txpwr_limit_nonbf_bf(lmt->mcs_20m[0], RTW89_CHANNEL_WIDTH_20, + __fill_txpwr_limit_nonbf_bf(lmt->mcs_20m[0], band, + RTW89_CHANNEL_WIDTH_20, ntx, RTW89_RS_MCS, ch - 6); - __fill_txpwr_limit_nonbf_bf(lmt->mcs_20m[1], RTW89_CHANNEL_WIDTH_20, + __fill_txpwr_limit_nonbf_bf(lmt->mcs_20m[1], band, + RTW89_CHANNEL_WIDTH_20, ntx, RTW89_RS_MCS, ch - 2); - __fill_txpwr_limit_nonbf_bf(lmt->mcs_20m[2], RTW89_CHANNEL_WIDTH_20, + __fill_txpwr_limit_nonbf_bf(lmt->mcs_20m[2], band, + RTW89_CHANNEL_WIDTH_20, ntx, RTW89_RS_MCS, ch + 2); - __fill_txpwr_limit_nonbf_bf(lmt->mcs_20m[3], RTW89_CHANNEL_WIDTH_20, + __fill_txpwr_limit_nonbf_bf(lmt->mcs_20m[3], band, + RTW89_CHANNEL_WIDTH_20, ntx, RTW89_RS_MCS, ch + 6); - __fill_txpwr_limit_nonbf_bf(lmt->mcs_40m[0], RTW89_CHANNEL_WIDTH_40, + __fill_txpwr_limit_nonbf_bf(lmt->mcs_40m[0], band, + RTW89_CHANNEL_WIDTH_40, ntx, RTW89_RS_MCS, ch - 4); - __fill_txpwr_limit_nonbf_bf(lmt->mcs_40m[1], RTW89_CHANNEL_WIDTH_40, + __fill_txpwr_limit_nonbf_bf(lmt->mcs_40m[1], band, + RTW89_CHANNEL_WIDTH_40, ntx, RTW89_RS_MCS, ch + 4); - __fill_txpwr_limit_nonbf_bf(lmt->mcs_80m[0], RTW89_CHANNEL_WIDTH_80, + __fill_txpwr_limit_nonbf_bf(lmt->mcs_80m[0], band, + RTW89_CHANNEL_WIDTH_80, ntx, RTW89_RS_MCS, ch); - __fill_txpwr_limit_nonbf_bf(val_0p5_n, RTW89_CHANNEL_WIDTH_40, + __fill_txpwr_limit_nonbf_bf(val_0p5_n, band, RTW89_CHANNEL_WIDTH_40, ntx, RTW89_RS_MCS, ch - 4); - __fill_txpwr_limit_nonbf_bf(val_0p5_p, RTW89_CHANNEL_WIDTH_40, + __fill_txpwr_limit_nonbf_bf(val_0p5_p, band, RTW89_CHANNEL_WIDTH_40, ntx, RTW89_RS_MCS, ch + 4); for (i = 0; i < RTW89_BF_NUM; i++) @@ -1656,7 +1699,7 @@ static void rtw89_phy_fill_txpwr_limit_80m(struct rtw89_dev *rtwdev, static void rtw89_phy_fill_txpwr_limit_160m(struct rtw89_dev *rtwdev, struct rtw89_txpwr_limit *lmt, - u8 ntx, u8 ch, u8 pri_ch) + u8 band, u8 ntx, u8 ch, u8 pri_ch) { s8 val_0p5_n[RTW89_BF_NUM]; s8 val_0p5_p[RTW89_BF_NUM]; @@ -1665,60 +1708,75 @@ static void rtw89_phy_fill_txpwr_limit_160m(struct rtw89_dev *rtwdev, u8 i; /* fill ofdm section */ - __fill_txpwr_limit_nonbf_bf(lmt->ofdm, RTW89_CHANNEL_WIDTH_20, + __fill_txpwr_limit_nonbf_bf(lmt->ofdm, band, RTW89_CHANNEL_WIDTH_20, ntx, RTW89_RS_OFDM, pri_ch); /* fill mcs 20m section */ - __fill_txpwr_limit_nonbf_bf(lmt->mcs_20m[0], RTW89_CHANNEL_WIDTH_20, + __fill_txpwr_limit_nonbf_bf(lmt->mcs_20m[0], band, + RTW89_CHANNEL_WIDTH_20, ntx, RTW89_RS_MCS, ch - 14); - __fill_txpwr_limit_nonbf_bf(lmt->mcs_20m[1], RTW89_CHANNEL_WIDTH_20, + __fill_txpwr_limit_nonbf_bf(lmt->mcs_20m[1], band, + RTW89_CHANNEL_WIDTH_20, ntx, RTW89_RS_MCS, ch - 10); - __fill_txpwr_limit_nonbf_bf(lmt->mcs_20m[2], RTW89_CHANNEL_WIDTH_20, + __fill_txpwr_limit_nonbf_bf(lmt->mcs_20m[2], band, + RTW89_CHANNEL_WIDTH_20, ntx, RTW89_RS_MCS, ch - 6); - __fill_txpwr_limit_nonbf_bf(lmt->mcs_20m[3], RTW89_CHANNEL_WIDTH_20, + __fill_txpwr_limit_nonbf_bf(lmt->mcs_20m[3], band, + RTW89_CHANNEL_WIDTH_20, ntx, RTW89_RS_MCS, ch - 2); - __fill_txpwr_limit_nonbf_bf(lmt->mcs_20m[4], RTW89_CHANNEL_WIDTH_20, + __fill_txpwr_limit_nonbf_bf(lmt->mcs_20m[4], band, + RTW89_CHANNEL_WIDTH_20, ntx, RTW89_RS_MCS, ch + 2); - __fill_txpwr_limit_nonbf_bf(lmt->mcs_20m[5], RTW89_CHANNEL_WIDTH_20, + __fill_txpwr_limit_nonbf_bf(lmt->mcs_20m[5], band, + RTW89_CHANNEL_WIDTH_20, ntx, RTW89_RS_MCS, ch + 6); - __fill_txpwr_limit_nonbf_bf(lmt->mcs_20m[6], RTW89_CHANNEL_WIDTH_20, + __fill_txpwr_limit_nonbf_bf(lmt->mcs_20m[6], band, + RTW89_CHANNEL_WIDTH_20, ntx, RTW89_RS_MCS, ch + 10); - __fill_txpwr_limit_nonbf_bf(lmt->mcs_20m[7], RTW89_CHANNEL_WIDTH_20, + __fill_txpwr_limit_nonbf_bf(lmt->mcs_20m[7], band, + RTW89_CHANNEL_WIDTH_20, ntx, RTW89_RS_MCS, ch + 14); /* fill mcs 40m section */ - __fill_txpwr_limit_nonbf_bf(lmt->mcs_40m[0], RTW89_CHANNEL_WIDTH_40, + __fill_txpwr_limit_nonbf_bf(lmt->mcs_40m[0], band, + RTW89_CHANNEL_WIDTH_40, ntx, RTW89_RS_MCS, ch - 12); - __fill_txpwr_limit_nonbf_bf(lmt->mcs_40m[1], RTW89_CHANNEL_WIDTH_40, + __fill_txpwr_limit_nonbf_bf(lmt->mcs_40m[1], band, + RTW89_CHANNEL_WIDTH_40, ntx, RTW89_RS_MCS, ch - 4); - __fill_txpwr_limit_nonbf_bf(lmt->mcs_40m[2], RTW89_CHANNEL_WIDTH_40, + __fill_txpwr_limit_nonbf_bf(lmt->mcs_40m[2], band, + RTW89_CHANNEL_WIDTH_40, ntx, RTW89_RS_MCS, ch + 4); - __fill_txpwr_limit_nonbf_bf(lmt->mcs_40m[3], RTW89_CHANNEL_WIDTH_40, + __fill_txpwr_limit_nonbf_bf(lmt->mcs_40m[3], band, + RTW89_CHANNEL_WIDTH_40, ntx, RTW89_RS_MCS, ch + 12); /* fill mcs 80m section */ - __fill_txpwr_limit_nonbf_bf(lmt->mcs_80m[0], RTW89_CHANNEL_WIDTH_80, + __fill_txpwr_limit_nonbf_bf(lmt->mcs_80m[0], band, + RTW89_CHANNEL_WIDTH_80, ntx, RTW89_RS_MCS, ch - 8); - __fill_txpwr_limit_nonbf_bf(lmt->mcs_80m[1], RTW89_CHANNEL_WIDTH_80, + __fill_txpwr_limit_nonbf_bf(lmt->mcs_80m[1], band, + RTW89_CHANNEL_WIDTH_80, ntx, RTW89_RS_MCS, ch + 8); /* fill mcs 160m section */ - __fill_txpwr_limit_nonbf_bf(lmt->mcs_160m, RTW89_CHANNEL_WIDTH_160, + __fill_txpwr_limit_nonbf_bf(lmt->mcs_160m, band, + RTW89_CHANNEL_WIDTH_160, ntx, RTW89_RS_MCS, ch); /* fill mcs 40m 0p5 section */ - __fill_txpwr_limit_nonbf_bf(val_0p5_n, RTW89_CHANNEL_WIDTH_40, + __fill_txpwr_limit_nonbf_bf(val_0p5_n, band, RTW89_CHANNEL_WIDTH_40, ntx, RTW89_RS_MCS, ch - 4); - __fill_txpwr_limit_nonbf_bf(val_0p5_p, RTW89_CHANNEL_WIDTH_40, + __fill_txpwr_limit_nonbf_bf(val_0p5_p, band, RTW89_CHANNEL_WIDTH_40, ntx, RTW89_RS_MCS, ch + 4); for (i = 0; i < RTW89_BF_NUM; i++) lmt->mcs_40m_0p5[i] = min_t(s8, val_0p5_n[i], val_0p5_p[i]); /* fill mcs 40m 2p5 section */ - __fill_txpwr_limit_nonbf_bf(val_2p5_n, RTW89_CHANNEL_WIDTH_40, + __fill_txpwr_limit_nonbf_bf(val_2p5_n, band, RTW89_CHANNEL_WIDTH_40, ntx, RTW89_RS_MCS, ch - 8); - __fill_txpwr_limit_nonbf_bf(val_2p5_p, RTW89_CHANNEL_WIDTH_40, + __fill_txpwr_limit_nonbf_bf(val_2p5_p, band, RTW89_CHANNEL_WIDTH_40, ntx, RTW89_RS_MCS, ch + 8); for (i = 0; i < RTW89_BF_NUM; i++) @@ -1726,37 +1784,41 @@ static void rtw89_phy_fill_txpwr_limit_160m(struct rtw89_dev *rtwdev, } void rtw89_phy_fill_txpwr_limit(struct rtw89_dev *rtwdev, + const struct rtw89_chan *chan, struct rtw89_txpwr_limit *lmt, u8 ntx) { - u8 pri_ch = rtwdev->hal.current_primary_channel; - u8 ch = rtwdev->hal.current_channel; - u8 bw = rtwdev->hal.current_band_width; + u8 band = chan->band_type; + u8 pri_ch = chan->primary_channel; + u8 ch = chan->channel; + u8 bw = chan->band_width; memset(lmt, 0, sizeof(*lmt)); switch (bw) { case RTW89_CHANNEL_WIDTH_20: - rtw89_phy_fill_txpwr_limit_20m(rtwdev, lmt, ntx, ch); + rtw89_phy_fill_txpwr_limit_20m(rtwdev, lmt, band, ntx, ch); break; case RTW89_CHANNEL_WIDTH_40: - rtw89_phy_fill_txpwr_limit_40m(rtwdev, lmt, ntx, ch, pri_ch); + rtw89_phy_fill_txpwr_limit_40m(rtwdev, lmt, band, ntx, ch, + pri_ch); break; case RTW89_CHANNEL_WIDTH_80: - rtw89_phy_fill_txpwr_limit_80m(rtwdev, lmt, ntx, ch, pri_ch); + rtw89_phy_fill_txpwr_limit_80m(rtwdev, lmt, band, ntx, ch, + pri_ch); break; case RTW89_CHANNEL_WIDTH_160: - rtw89_phy_fill_txpwr_limit_160m(rtwdev, lmt, ntx, ch, pri_ch); + rtw89_phy_fill_txpwr_limit_160m(rtwdev, lmt, band, ntx, ch, + pri_ch); break; } } EXPORT_SYMBOL(rtw89_phy_fill_txpwr_limit); -static s8 rtw89_phy_read_txpwr_limit_ru(struct rtw89_dev *rtwdev, +static s8 rtw89_phy_read_txpwr_limit_ru(struct rtw89_dev *rtwdev, u8 band, u8 ru, u8 ntx, u8 ch) { const struct rtw89_chip_info *chip = rtwdev->chip; - u8 band = rtwdev->hal.current_band_type; u8 ch_idx = rtw89_channel_to_idx(rtwdev, band, ch); u8 regd = rtw89_regd_get(rtwdev, band); s8 lmt_ru = 0, sar; @@ -1794,85 +1856,106 @@ static s8 rtw89_phy_read_txpwr_limit_ru(struct rtw89_dev *rtwdev, static void rtw89_phy_fill_txpwr_limit_ru_20m(struct rtw89_dev *rtwdev, struct rtw89_txpwr_limit_ru *lmt_ru, - u8 ntx, u8 ch) + u8 band, u8 ntx, u8 ch) { - lmt_ru->ru26[0] = rtw89_phy_read_txpwr_limit_ru(rtwdev, RTW89_RU26, + lmt_ru->ru26[0] = rtw89_phy_read_txpwr_limit_ru(rtwdev, band, + RTW89_RU26, ntx, ch); - lmt_ru->ru52[0] = rtw89_phy_read_txpwr_limit_ru(rtwdev, RTW89_RU52, + lmt_ru->ru52[0] = rtw89_phy_read_txpwr_limit_ru(rtwdev, band, + RTW89_RU52, ntx, ch); - lmt_ru->ru106[0] = rtw89_phy_read_txpwr_limit_ru(rtwdev, RTW89_RU106, + lmt_ru->ru106[0] = rtw89_phy_read_txpwr_limit_ru(rtwdev, band, + RTW89_RU106, ntx, ch); } static void rtw89_phy_fill_txpwr_limit_ru_40m(struct rtw89_dev *rtwdev, struct rtw89_txpwr_limit_ru *lmt_ru, - u8 ntx, u8 ch) + u8 band, u8 ntx, u8 ch) { - lmt_ru->ru26[0] = rtw89_phy_read_txpwr_limit_ru(rtwdev, RTW89_RU26, + lmt_ru->ru26[0] = rtw89_phy_read_txpwr_limit_ru(rtwdev, band, + RTW89_RU26, ntx, ch - 2); - lmt_ru->ru26[1] = rtw89_phy_read_txpwr_limit_ru(rtwdev, RTW89_RU26, + lmt_ru->ru26[1] = rtw89_phy_read_txpwr_limit_ru(rtwdev, band, + RTW89_RU26, ntx, ch + 2); - lmt_ru->ru52[0] = rtw89_phy_read_txpwr_limit_ru(rtwdev, RTW89_RU52, + lmt_ru->ru52[0] = rtw89_phy_read_txpwr_limit_ru(rtwdev, band, + RTW89_RU52, ntx, ch - 2); - lmt_ru->ru52[1] = rtw89_phy_read_txpwr_limit_ru(rtwdev, RTW89_RU52, + lmt_ru->ru52[1] = rtw89_phy_read_txpwr_limit_ru(rtwdev, band, + RTW89_RU52, ntx, ch + 2); - lmt_ru->ru106[0] = rtw89_phy_read_txpwr_limit_ru(rtwdev, RTW89_RU106, + lmt_ru->ru106[0] = rtw89_phy_read_txpwr_limit_ru(rtwdev, band, + RTW89_RU106, ntx, ch - 2); - lmt_ru->ru106[1] = rtw89_phy_read_txpwr_limit_ru(rtwdev, RTW89_RU106, + lmt_ru->ru106[1] = rtw89_phy_read_txpwr_limit_ru(rtwdev, band, + RTW89_RU106, ntx, ch + 2); } static void rtw89_phy_fill_txpwr_limit_ru_80m(struct rtw89_dev *rtwdev, struct rtw89_txpwr_limit_ru *lmt_ru, - u8 ntx, u8 ch) + u8 band, u8 ntx, u8 ch) { - lmt_ru->ru26[0] = rtw89_phy_read_txpwr_limit_ru(rtwdev, RTW89_RU26, + lmt_ru->ru26[0] = rtw89_phy_read_txpwr_limit_ru(rtwdev, band, + RTW89_RU26, ntx, ch - 6); - lmt_ru->ru26[1] = rtw89_phy_read_txpwr_limit_ru(rtwdev, RTW89_RU26, + lmt_ru->ru26[1] = rtw89_phy_read_txpwr_limit_ru(rtwdev, band, + RTW89_RU26, ntx, ch - 2); - lmt_ru->ru26[2] = rtw89_phy_read_txpwr_limit_ru(rtwdev, RTW89_RU26, + lmt_ru->ru26[2] = rtw89_phy_read_txpwr_limit_ru(rtwdev, band, + RTW89_RU26, ntx, ch + 2); - lmt_ru->ru26[3] = rtw89_phy_read_txpwr_limit_ru(rtwdev, RTW89_RU26, + lmt_ru->ru26[3] = rtw89_phy_read_txpwr_limit_ru(rtwdev, band, + RTW89_RU26, ntx, ch + 6); - lmt_ru->ru52[0] = rtw89_phy_read_txpwr_limit_ru(rtwdev, RTW89_RU52, + lmt_ru->ru52[0] = rtw89_phy_read_txpwr_limit_ru(rtwdev, band, + RTW89_RU52, ntx, ch - 6); - lmt_ru->ru52[1] = rtw89_phy_read_txpwr_limit_ru(rtwdev, RTW89_RU52, + lmt_ru->ru52[1] = rtw89_phy_read_txpwr_limit_ru(rtwdev, band, + RTW89_RU52, ntx, ch - 2); - lmt_ru->ru52[2] = rtw89_phy_read_txpwr_limit_ru(rtwdev, RTW89_RU52, + lmt_ru->ru52[2] = rtw89_phy_read_txpwr_limit_ru(rtwdev, band, + RTW89_RU52, ntx, ch + 2); - lmt_ru->ru52[3] = rtw89_phy_read_txpwr_limit_ru(rtwdev, RTW89_RU52, + lmt_ru->ru52[3] = rtw89_phy_read_txpwr_limit_ru(rtwdev, band, + RTW89_RU52, ntx, ch + 6); - lmt_ru->ru106[0] = rtw89_phy_read_txpwr_limit_ru(rtwdev, RTW89_RU106, + lmt_ru->ru106[0] = rtw89_phy_read_txpwr_limit_ru(rtwdev, band, + RTW89_RU106, ntx, ch - 6); - lmt_ru->ru106[1] = rtw89_phy_read_txpwr_limit_ru(rtwdev, RTW89_RU106, + lmt_ru->ru106[1] = rtw89_phy_read_txpwr_limit_ru(rtwdev, band, + RTW89_RU106, ntx, ch - 2); - lmt_ru->ru106[2] = rtw89_phy_read_txpwr_limit_ru(rtwdev, RTW89_RU106, + lmt_ru->ru106[2] = rtw89_phy_read_txpwr_limit_ru(rtwdev, band, + RTW89_RU106, ntx, ch + 2); - lmt_ru->ru106[3] = rtw89_phy_read_txpwr_limit_ru(rtwdev, RTW89_RU106, + lmt_ru->ru106[3] = rtw89_phy_read_txpwr_limit_ru(rtwdev, band, + RTW89_RU106, ntx, ch + 6); } static void rtw89_phy_fill_txpwr_limit_ru_160m(struct rtw89_dev *rtwdev, struct rtw89_txpwr_limit_ru *lmt_ru, - u8 ntx, u8 ch) + u8 band, u8 ntx, u8 ch) { static const int ofst[] = { -14, -10, -6, -2, 2, 6, 10, 14 }; int i; static_assert(ARRAY_SIZE(ofst) == RTW89_RU_SEC_NUM); for (i = 0; i < RTW89_RU_SEC_NUM; i++) { - lmt_ru->ru26[i] = rtw89_phy_read_txpwr_limit_ru(rtwdev, + lmt_ru->ru26[i] = rtw89_phy_read_txpwr_limit_ru(rtwdev, band, RTW89_RU26, ntx, ch + ofst[i]); - lmt_ru->ru52[i] = rtw89_phy_read_txpwr_limit_ru(rtwdev, + lmt_ru->ru52[i] = rtw89_phy_read_txpwr_limit_ru(rtwdev, band, RTW89_RU52, ntx, ch + ofst[i]); - lmt_ru->ru106[i] = rtw89_phy_read_txpwr_limit_ru(rtwdev, + lmt_ru->ru106[i] = rtw89_phy_read_txpwr_limit_ru(rtwdev, band, RTW89_RU106, ntx, ch + ofst[i]); @@ -1880,26 +1963,32 @@ rtw89_phy_fill_txpwr_limit_ru_160m(struct rtw89_dev *rtwdev, } void rtw89_phy_fill_txpwr_limit_ru(struct rtw89_dev *rtwdev, + const struct rtw89_chan *chan, struct rtw89_txpwr_limit_ru *lmt_ru, u8 ntx) { - u8 ch = rtwdev->hal.current_channel; - u8 bw = rtwdev->hal.current_band_width; + u8 band = chan->band_type; + u8 ch = chan->channel; + u8 bw = chan->band_width; memset(lmt_ru, 0, sizeof(*lmt_ru)); switch (bw) { case RTW89_CHANNEL_WIDTH_20: - rtw89_phy_fill_txpwr_limit_ru_20m(rtwdev, lmt_ru, ntx, ch); + rtw89_phy_fill_txpwr_limit_ru_20m(rtwdev, lmt_ru, band, ntx, + ch); break; case RTW89_CHANNEL_WIDTH_40: - rtw89_phy_fill_txpwr_limit_ru_40m(rtwdev, lmt_ru, ntx, ch); + rtw89_phy_fill_txpwr_limit_ru_40m(rtwdev, lmt_ru, band, ntx, + ch); break; case RTW89_CHANNEL_WIDTH_80: - rtw89_phy_fill_txpwr_limit_ru_80m(rtwdev, lmt_ru, ntx, ch); + rtw89_phy_fill_txpwr_limit_ru_80m(rtwdev, lmt_ru, band, ntx, + ch); break; case RTW89_CHANNEL_WIDTH_160: - rtw89_phy_fill_txpwr_limit_ru_160m(rtwdev, lmt_ru, ntx, ch); + rtw89_phy_fill_txpwr_limit_ru_160m(rtwdev, lmt_ru, band, ntx, + ch); break; } } @@ -1920,6 +2009,7 @@ static void rtw89_phy_c2h_ra_rpt_iter(void *data, struct ieee80211_sta *sta) u8 mode, rate, bw, giltf, mac_id; u16 legacy_bitrate; bool valid; + u8 mcs = 0; mac_id = RTW89_GET_PHY_C2H_RA_RPT_MACID(c2h->data); if (mac_id != rtwsta->mac_id) @@ -1936,7 +2026,7 @@ static void rtw89_phy_c2h_ra_rpt_iter(void *data, struct ieee80211_sta *sta) return; } - memset(ra_report, 0, sizeof(*ra_report)); + memset(&ra_report->txrate, 0, sizeof(ra_report->txrate)); switch (mode) { case RTW89_RA_RPT_MODE_LEGACY: @@ -1952,6 +2042,7 @@ static void rtw89_phy_c2h_ra_rpt_iter(void *data, struct ieee80211_sta *sta) ra_report->txrate.mcs = rate; if (giltf) ra_report->txrate.flags |= RATE_INFO_FLAGS_SHORT_GI; + mcs = ra_report->txrate.mcs & 0x07; break; case RTW89_RA_RPT_MODE_VHT: ra_report->txrate.flags |= RATE_INFO_FLAGS_VHT_MCS; @@ -1959,6 +2050,7 @@ static void rtw89_phy_c2h_ra_rpt_iter(void *data, struct ieee80211_sta *sta) ra_report->txrate.nss = FIELD_GET(RTW89_RA_RATE_MASK_NSS, rate) + 1; if (giltf) ra_report->txrate.flags |= RATE_INFO_FLAGS_SHORT_GI; + mcs = ra_report->txrate.mcs; break; case RTW89_RA_RPT_MODE_HE: ra_report->txrate.flags |= RATE_INFO_FLAGS_HE_MCS; @@ -1970,6 +2062,7 @@ static void rtw89_phy_c2h_ra_rpt_iter(void *data, struct ieee80211_sta *sta) ra_report->txrate.he_gi = NL80211_RATE_INFO_HE_GI_1_6; else ra_report->txrate.he_gi = NL80211_RATE_INFO_HE_GI_3_2; + mcs = ra_report->txrate.mcs; break; } @@ -1977,8 +2070,9 @@ static void rtw89_phy_c2h_ra_rpt_iter(void *data, struct ieee80211_sta *sta) ra_report->bit_rate = cfg80211_calculate_bitrate(&ra_report->txrate); ra_report->hw_rate = FIELD_PREP(RTW89_HW_RATE_MASK_MOD, mode) | FIELD_PREP(RTW89_HW_RATE_MASK_VAL, rate); - sta->max_rc_amsdu_len = get_max_amsdu_len(rtwdev, ra_report); - rtwsta->max_agg_wait = sta->max_rc_amsdu_len / 1500 - 1; + ra_report->might_fallback_legacy = mcs <= 2; + sta->deflink.agg.max_rc_amsdu_len = get_max_amsdu_len(rtwdev, ra_report); + rtwsta->max_agg_wait = sta->deflink.agg.max_rc_amsdu_len / 1500 - 1; } static void @@ -3247,10 +3341,11 @@ static void rtw89_phy_dig_update_rssi_info(struct rtw89_dev *rtwdev) static void rtw89_phy_dig_update_para(struct rtw89_dev *rtwdev) { struct rtw89_dig_info *dig = &rtwdev->dig; + const struct rtw89_chan *chan = rtw89_chan_get(rtwdev, RTW89_SUB_ENTITY_0); bool is_linked = rtwdev->total_sta_assoc > 0; const u16 *fa_th_src = NULL; - switch (rtwdev->hal.current_band_type) { + switch (chan->band_type) { case RTW89_BAND_2G: dig->lna_gain = dig->lna_gain_g; dig->tia_gain = dig->tia_gain_g; @@ -3410,26 +3505,32 @@ static void rtw89_phy_dig_igi_offset_by_env(struct rtw89_dev *rtwdev) static void rtw89_phy_dig_set_lna_idx(struct rtw89_dev *rtwdev, u8 lna_idx) { - rtw89_phy_write32_mask(rtwdev, R_PATH0_LNA_INIT, - B_PATH0_LNA_INIT_IDX_MSK, lna_idx); - rtw89_phy_write32_mask(rtwdev, R_PATH1_LNA_INIT, - B_PATH1_LNA_INIT_IDX_MSK, lna_idx); + const struct rtw89_dig_regs *dig_regs = rtwdev->chip->dig_regs; + + rtw89_phy_write32_mask(rtwdev, dig_regs->p0_lna_init.addr, + dig_regs->p0_lna_init.mask, lna_idx); + rtw89_phy_write32_mask(rtwdev, dig_regs->p1_lna_init.addr, + dig_regs->p1_lna_init.mask, lna_idx); } static void rtw89_phy_dig_set_tia_idx(struct rtw89_dev *rtwdev, u8 tia_idx) { - rtw89_phy_write32_mask(rtwdev, R_PATH0_TIA_INIT, - B_PATH0_TIA_INIT_IDX_MSK, tia_idx); - rtw89_phy_write32_mask(rtwdev, R_PATH1_TIA_INIT, - B_PATH1_TIA_INIT_IDX_MSK, tia_idx); + const struct rtw89_dig_regs *dig_regs = rtwdev->chip->dig_regs; + + rtw89_phy_write32_mask(rtwdev, dig_regs->p0_tia_init.addr, + dig_regs->p0_tia_init.mask, tia_idx); + rtw89_phy_write32_mask(rtwdev, dig_regs->p1_tia_init.addr, + dig_regs->p1_tia_init.mask, tia_idx); } static void rtw89_phy_dig_set_rxb_idx(struct rtw89_dev *rtwdev, u8 rxb_idx) { - rtw89_phy_write32_mask(rtwdev, R_PATH0_RXB_INIT, - B_PATH0_RXB_INIT_IDX_MSK, rxb_idx); - rtw89_phy_write32_mask(rtwdev, R_PATH1_RXB_INIT, - B_PATH1_RXB_INIT_IDX_MSK, rxb_idx); + const struct rtw89_dig_regs *dig_regs = rtwdev->chip->dig_regs; + + rtw89_phy_write32_mask(rtwdev, dig_regs->p0_rxb_init.addr, + dig_regs->p0_rxb_init.mask, rxb_idx); + rtw89_phy_write32_mask(rtwdev, dig_regs->p1_rxb_init.addr, + dig_regs->p1_rxb_init.mask, rxb_idx); } static void rtw89_phy_dig_set_igi_cr(struct rtw89_dev *rtwdev, @@ -3443,21 +3544,19 @@ static void rtw89_phy_dig_set_igi_cr(struct rtw89_dev *rtwdev, set.lna_idx, set.tia_idx, set.rxb_idx); } -static const struct rtw89_reg_def sdagc_config[4] = { - {R_PATH0_P20_FOLLOW_BY_PAGCUGC, B_PATH0_P20_FOLLOW_BY_PAGCUGC_EN_MSK}, - {R_PATH0_S20_FOLLOW_BY_PAGCUGC, B_PATH0_S20_FOLLOW_BY_PAGCUGC_EN_MSK}, - {R_PATH1_P20_FOLLOW_BY_PAGCUGC, B_PATH1_P20_FOLLOW_BY_PAGCUGC_EN_MSK}, - {R_PATH1_S20_FOLLOW_BY_PAGCUGC, B_PATH1_S20_FOLLOW_BY_PAGCUGC_EN_MSK}, -}; - static void rtw89_phy_dig_sdagc_follow_pagc_config(struct rtw89_dev *rtwdev, bool enable) { - u8 i = 0; + const struct rtw89_dig_regs *dig_regs = rtwdev->chip->dig_regs; - for (i = 0; i < ARRAY_SIZE(sdagc_config); i++) - rtw89_phy_write32_mask(rtwdev, sdagc_config[i].addr, - sdagc_config[i].mask, enable); + rtw89_phy_write32_mask(rtwdev, dig_regs->p0_p20_pagcugc_en.addr, + dig_regs->p0_p20_pagcugc_en.mask, enable); + rtw89_phy_write32_mask(rtwdev, dig_regs->p0_s20_pagcugc_en.addr, + dig_regs->p0_s20_pagcugc_en.mask, enable); + rtw89_phy_write32_mask(rtwdev, dig_regs->p1_p20_pagcugc_en.addr, + dig_regs->p1_p20_pagcugc_en.mask, enable); + rtw89_phy_write32_mask(rtwdev, dig_regs->p1_s20_pagcugc_en.addr, + dig_regs->p1_s20_pagcugc_en.mask, enable); rtw89_debug(rtwdev, RTW89_DBG_DIG, "sdagc_follow_pagc=%d\n", enable); } @@ -3483,7 +3582,9 @@ static void rtw89_phy_dig_config_igi(struct rtw89_dev *rtwdev) static void rtw89_phy_dig_dyn_pd_th(struct rtw89_dev *rtwdev, u8 rssi, bool enable) { - enum rtw89_bandwidth cbw = rtwdev->hal.current_band_width; + const struct rtw89_chan *chan = rtw89_chan_get(rtwdev, RTW89_SUB_ENTITY_0); + const struct rtw89_dig_regs *dig_regs = rtwdev->chip->dig_regs; + enum rtw89_bandwidth cbw = chan->band_width; struct rtw89_dig_info *dig = &rtwdev->dig; u8 final_rssi = 0, under_region = dig->pd_low_th_ofst; u8 ofdm_cca_th; @@ -3525,10 +3626,10 @@ static void rtw89_phy_dig_dyn_pd_th(struct rtw89_dev *rtwdev, u8 rssi, "Dynamic PD th disabled, Set PD_low_bd=0\n"); } - rtw89_phy_write32_mask(rtwdev, R_SEG0R_PD, B_SEG0R_PD_LOWER_BOUND_MSK, - pd_val); - rtw89_phy_write32_mask(rtwdev, R_SEG0R_PD, - B_SEG0R_PD_SPATIAL_REUSE_EN_MSK, enable); + rtw89_phy_write32_mask(rtwdev, dig_regs->seg0_pd_reg, + dig_regs->pd_lower_bound_mask, pd_val); + rtw89_phy_write32_mask(rtwdev, dig_regs->seg0_pd_reg, + dig_regs->pd_spatial_reuse_en, enable); if (!rtwdev->hal.support_cckpd) return; @@ -3604,6 +3705,62 @@ void rtw89_phy_dig(struct rtw89_dev *rtwdev) rtw89_phy_dig_sdagc_follow_pagc_config(rtwdev, false); } +static void rtw89_phy_tx_path_div_sta_iter(void *data, struct ieee80211_sta *sta) +{ + struct rtw89_sta *rtwsta = (struct rtw89_sta *)sta->drv_priv; + struct rtw89_dev *rtwdev = rtwsta->rtwdev; + struct rtw89_vif *rtwvif = rtwsta->rtwvif; + struct rtw89_hal *hal = &rtwdev->hal; + bool *done = data; + u8 rssi_a, rssi_b; + u32 candidate; + + if (rtwvif->wifi_role != RTW89_WIFI_ROLE_STATION || sta->tdls) + return; + + if (*done) + return; + + *done = true; + + rssi_a = ewma_rssi_read(&rtwsta->rssi[RF_PATH_A]); + rssi_b = ewma_rssi_read(&rtwsta->rssi[RF_PATH_B]); + + if (rssi_a > rssi_b + RTW89_TX_DIV_RSSI_RAW_TH) + candidate = RF_A; + else if (rssi_b > rssi_a + RTW89_TX_DIV_RSSI_RAW_TH) + candidate = RF_B; + else + return; + + if (hal->antenna_tx == candidate) + return; + + hal->antenna_tx = candidate; + rtw89_fw_h2c_txpath_cmac_tbl(rtwdev, rtwsta); + + if (hal->antenna_tx == RF_A) { + rtw89_phy_write32_mask(rtwdev, R_P0_RFMODE, B_P0_RFMODE_MUX, 0x12); + rtw89_phy_write32_mask(rtwdev, R_P1_RFMODE, B_P1_RFMODE_MUX, 0x11); + } else if (hal->antenna_tx == RF_B) { + rtw89_phy_write32_mask(rtwdev, R_P0_RFMODE, B_P0_RFMODE_MUX, 0x11); + rtw89_phy_write32_mask(rtwdev, R_P1_RFMODE, B_P1_RFMODE_MUX, 0x12); + } +} + +void rtw89_phy_tx_path_div_track(struct rtw89_dev *rtwdev) +{ + struct rtw89_hal *hal = &rtwdev->hal; + bool done = false; + + if (!hal->tx_path_diversity) + return; + + ieee80211_iterate_stations_atomic(rtwdev->hw, + rtw89_phy_tx_path_div_sta_iter, + &done); +} + static void rtw89_phy_env_monitor_init(struct rtw89_dev *rtwdev) { rtw89_phy_ccx_top_setting_init(rtwdev); diff --git a/drivers/net/wireless/realtek/rtw89/phy.h b/drivers/net/wireless/realtek/rtw89/phy.h index e20636f54b55..ee3bc5e111e1 100644 --- a/drivers/net/wireless/realtek/rtw89/phy.h +++ b/drivers/net/wireless/realtek/rtw89/phy.h @@ -56,7 +56,7 @@ #define CFO_TRK_STOP_TH (2 << 2) #define CFO_SW_COMP_FINE_TUNE (2 << 2) #define CFO_PERIOD_CNT 15 -#define CFO_BOUND 32 +#define CFO_BOUND 64 #define CFO_TP_UPPER 100 #define CFO_TP_LOWER 50 #define CFO_COMP_PERIOD 250 @@ -439,7 +439,7 @@ rtw89_rfk_parser(struct rtw89_dev *rtwdev, const struct rtw89_rfk_tbl *tbl); void rtw89_phy_write_reg3_tbl(struct rtw89_dev *rtwdev, const struct rtw89_phy_reg3_tbl *tbl); u8 rtw89_phy_get_txsc(struct rtw89_dev *rtwdev, - struct rtw89_channel_params *param, + const struct rtw89_chan *chan, enum rtw89_bandwidth dbw); u32 rtw89_phy_read_rf(struct rtw89_dev *rtwdev, enum rtw89_rf_path rf_path, u32 addr, u32 mask); @@ -460,15 +460,17 @@ void rtw89_phy_write32_idx(struct rtw89_dev *rtwdev, u32 addr, u32 mask, u32 data, enum rtw89_phy_idx phy_idx); void rtw89_phy_load_txpwr_byrate(struct rtw89_dev *rtwdev, const struct rtw89_txpwr_table *tbl); -s8 rtw89_phy_read_txpwr_byrate(struct rtw89_dev *rtwdev, +s8 rtw89_phy_read_txpwr_byrate(struct rtw89_dev *rtwdev, u8 band, const struct rtw89_rate_desc *rate_desc); void rtw89_phy_fill_txpwr_limit(struct rtw89_dev *rtwdev, + const struct rtw89_chan *chan, struct rtw89_txpwr_limit *lmt, u8 ntx); void rtw89_phy_fill_txpwr_limit_ru(struct rtw89_dev *rtwdev, + const struct rtw89_chan *chan, struct rtw89_txpwr_limit_ru *lmt_ru, u8 ntx); -s8 rtw89_phy_read_txpwr_limit(struct rtw89_dev *rtwdev, +s8 rtw89_phy_read_txpwr_limit(struct rtw89_dev *rtwdev, u8 band, u8 bw, u8 ntx, u8 rs, u8 bf, u8 ch); void rtw89_phy_ra_assoc(struct rtw89_dev *rtwdev, struct ieee80211_sta *sta); void rtw89_phy_ra_update(struct rtw89_dev *rtwdev); @@ -489,6 +491,7 @@ void rtw89_phy_set_phy_regs(struct rtw89_dev *rtwdev, u32 addr, u32 mask, u32 val); void rtw89_phy_dig_reset(struct rtw89_dev *rtwdev); void rtw89_phy_dig(struct rtw89_dev *rtwdev); +void rtw89_phy_tx_path_div_track(struct rtw89_dev *rtwdev); void rtw89_phy_set_bss_color(struct rtw89_dev *rtwdev, struct ieee80211_vif *vif); void rtw89_phy_tssi_ctrl_set_bandedge_cfg(struct rtw89_dev *rtwdev, enum rtw89_mac_idx mac_idx, diff --git a/drivers/net/wireless/realtek/rtw89/ps.c b/drivers/net/wireless/realtek/rtw89/ps.c index a90b33720588..bf41a1141679 100644 --- a/drivers/net/wireless/realtek/rtw89/ps.c +++ b/drivers/net/wireless/realtek/rtw89/ps.c @@ -59,8 +59,11 @@ static void rtw89_ps_power_mode_change(struct rtw89_dev *rtwdev, bool enter) rtw89_mac_power_mode_change(rtwdev, enter); } -static void __rtw89_enter_ps_mode(struct rtw89_dev *rtwdev) +static void __rtw89_enter_ps_mode(struct rtw89_dev *rtwdev, struct rtw89_vif *rtwvif) { + if (rtwvif->wifi_role == RTW89_WIFI_ROLE_P2P_CLIENT) + return; + if (!rtwdev->ps_mode) return; @@ -111,23 +114,23 @@ void rtw89_leave_ps_mode(struct rtw89_dev *rtwdev) __rtw89_leave_ps_mode(rtwdev); } -void rtw89_enter_lps(struct rtw89_dev *rtwdev, u8 mac_id) +void rtw89_enter_lps(struct rtw89_dev *rtwdev, struct rtw89_vif *rtwvif) { lockdep_assert_held(&rtwdev->mutex); if (test_and_set_bit(RTW89_FLAG_LEISURE_PS, rtwdev->flags)) return; - __rtw89_enter_lps(rtwdev, mac_id); - __rtw89_enter_ps_mode(rtwdev); + __rtw89_enter_lps(rtwdev, rtwvif->mac_id); + __rtw89_enter_ps_mode(rtwdev, rtwvif); } static void rtw89_leave_lps_vif(struct rtw89_dev *rtwdev, struct rtw89_vif *rtwvif) { - if (rtwvif->wifi_role != RTW89_WIFI_ROLE_STATION) + if (rtwvif->wifi_role != RTW89_WIFI_ROLE_STATION && + rtwvif->wifi_role != RTW89_WIFI_ROLE_P2P_CLIENT) return; - __rtw89_leave_ps_mode(rtwdev); __rtw89_leave_lps(rtwdev, rtwvif->mac_id); } @@ -140,6 +143,8 @@ void rtw89_leave_lps(struct rtw89_dev *rtwdev) if (!test_and_clear_bit(RTW89_FLAG_LEISURE_PS, rtwdev->flags)) return; + __rtw89_leave_ps_mode(rtwdev); + rtw89_for_each_rtwvif(rtwdev, rtwvif) rtw89_leave_lps_vif(rtwdev, rtwvif); } @@ -178,3 +183,64 @@ void rtw89_set_coex_ctrl_lps(struct rtw89_dev *rtwdev, bool btc_ctrl) if (btc_ctrl) rtw89_leave_lps(rtwdev); } + +static void rtw89_tsf32_toggle(struct rtw89_dev *rtwdev, struct rtw89_vif *rtwvif, + enum rtw89_p2pps_action act) +{ + if (act == RTW89_P2P_ACT_UPDATE || act == RTW89_P2P_ACT_REMOVE) + return; + + if (act == RTW89_P2P_ACT_INIT) + rtw89_fw_h2c_tsf32_toggle(rtwdev, rtwvif, true); + else if (act == RTW89_P2P_ACT_TERMINATE) + rtw89_fw_h2c_tsf32_toggle(rtwdev, rtwvif, false); +} + +static void rtw89_p2p_disable_all_noa(struct rtw89_dev *rtwdev, + struct ieee80211_vif *vif) +{ + struct rtw89_vif *rtwvif = (struct rtw89_vif *)vif->drv_priv; + enum rtw89_p2pps_action act; + u8 noa_id; + + if (rtwvif->last_noa_nr == 0) + return; + + for (noa_id = 0; noa_id < rtwvif->last_noa_nr; noa_id++) { + if (noa_id == rtwvif->last_noa_nr - 1) + act = RTW89_P2P_ACT_TERMINATE; + else + act = RTW89_P2P_ACT_REMOVE; + rtw89_tsf32_toggle(rtwdev, rtwvif, act); + rtw89_fw_h2c_p2p_act(rtwdev, vif, NULL, act, noa_id); + } +} + +static void rtw89_p2p_update_noa(struct rtw89_dev *rtwdev, + struct ieee80211_vif *vif) +{ + struct rtw89_vif *rtwvif = (struct rtw89_vif *)vif->drv_priv; + struct ieee80211_p2p_noa_desc *desc; + enum rtw89_p2pps_action act; + u8 noa_id; + + for (noa_id = 0; noa_id < RTW89_P2P_MAX_NOA_NUM; noa_id++) { + desc = &vif->bss_conf.p2p_noa_attr.desc[noa_id]; + if (!desc->count || !desc->duration) + break; + + if (noa_id == 0) + act = RTW89_P2P_ACT_INIT; + else + act = RTW89_P2P_ACT_UPDATE; + rtw89_tsf32_toggle(rtwdev, rtwvif, act); + rtw89_fw_h2c_p2p_act(rtwdev, vif, desc, act, noa_id); + } + rtwvif->last_noa_nr = noa_id; +} + +void rtw89_process_p2p_ps(struct rtw89_dev *rtwdev, struct ieee80211_vif *vif) +{ + rtw89_p2p_disable_all_noa(rtwdev, vif); + rtw89_p2p_update_noa(rtwdev, vif); +} diff --git a/drivers/net/wireless/realtek/rtw89/ps.h b/drivers/net/wireless/realtek/rtw89/ps.h index a184b68994aa..0feae3991623 100644 --- a/drivers/net/wireless/realtek/rtw89/ps.h +++ b/drivers/net/wireless/realtek/rtw89/ps.h @@ -5,12 +5,13 @@ #ifndef __RTW89_PS_H_ #define __RTW89_PS_H_ -void rtw89_enter_lps(struct rtw89_dev *rtwdev, u8 mac_id); +void rtw89_enter_lps(struct rtw89_dev *rtwdev, struct rtw89_vif *rtwvif); void rtw89_leave_lps(struct rtw89_dev *rtwdev); void __rtw89_leave_ps_mode(struct rtw89_dev *rtwdev); void rtw89_leave_ps_mode(struct rtw89_dev *rtwdev); void rtw89_enter_ips(struct rtw89_dev *rtwdev); void rtw89_leave_ips(struct rtw89_dev *rtwdev); void rtw89_set_coex_ctrl_lps(struct rtw89_dev *rtwdev, bool btc_ctrl); +void rtw89_process_p2p_ps(struct rtw89_dev *rtwdev, struct ieee80211_vif *vif); #endif diff --git a/drivers/net/wireless/realtek/rtw89/reg.h b/drivers/net/wireless/realtek/rtw89/reg.h index ebf28719d935..ca20bb024b40 100644 --- a/drivers/net/wireless/realtek/rtw89/reg.h +++ b/drivers/net/wireless/realtek/rtw89/reg.h @@ -51,9 +51,6 @@ #define B_AX_EF_POR BIT(10) #define B_AX_EF_CELL_SEL_MASK GENMASK(9, 8) -#define R_AX_SPSLDO_ON_CTRL0 0x0200 -#define B_AX_OCP_L1_MASK GENMASK(15, 13) - #define R_AX_EFUSE_CTRL 0x0030 #define B_AX_EF_MODE_SEL_MASK GENMASK(31, 30) #define B_AX_EF_RDY BIT(29) @@ -143,6 +140,18 @@ #define R_AX_PMC_DBG_CTRL2 0x00CC #define B_AX_SYSON_DIS_PMCR_AX_WRMSK BIT(2) +#define R_AX_PCIE_MIO_INTF 0x00E4 +#define B_AX_PCIE_MIO_ADDR_PAGE_V1_MASK GENMASK(20, 16) +#define B_AX_PCIE_MIO_BYIOREG BIT(13) +#define B_AX_PCIE_MIO_RE BIT(12) +#define B_AX_PCIE_MIO_WE_MASK GENMASK(11, 8) +#define MIO_WRITE_BYTE_ALL 0xF +#define B_AX_PCIE_MIO_ADDR_MASK GENMASK(7, 0) +#define MIO_ADDR_PAGE_MASK GENMASK(12, 8) + +#define R_AX_PCIE_MIO_INTD 0x00E8 +#define B_AX_PCIE_MIO_DATA_MASK GENMASK(31, 0) + #define R_AX_SYS_CFG1 0x00F0 #define B_AX_CHIP_VER_MASK GENMASK(15, 12) @@ -191,6 +200,12 @@ #define R_AX_UDM2 0x01F8 #define R_AX_UDM3 0x01FC +#define R_AX_SPS_DIG_ON_CTRL0 0x0200 +#define B_AX_VREFPFM_L_MASK GENMASK(25, 22) +#define B_AX_REG_ZCDC_H_MASK GENMASK(18, 17) +#define B_AX_OCP_L1_MASK GENMASK(15, 13) +#define B_AX_VOL_L1_MASK GENMASK(3, 0) + #define R_AX_LDO_AON_CTRL0 0x0218 #define B_AX_PD_REGU_L BIT(16) @@ -383,6 +398,7 @@ #define R_AX_PHYREG_SET 0x8040 #define PHYREG_SET_ALL_CYCLE 0x8 +#define PHYREG_SET_XYN_CYCLE 0xE #define R_AX_HD0IMR 0x8110 #define B_AX_WDT_PTFM_INT_EN BIT(5) @@ -467,6 +483,7 @@ #define R_AX_LTR_CTRL_0 0x8410 #define B_AX_LTR_SPACE_IDX_MASK GENMASK(13, 12) #define B_AX_LTR_IDLE_TIMER_IDX_MASK GENMASK(10, 8) +#define B_AX_LTR_WD_NOEMP_CHK BIT(6) #define B_AX_APP_LTR_ACT BIT(5) #define B_AX_APP_LTR_IDLE BIT(4) #define B_AX_LTR_EN BIT(1) @@ -1024,15 +1041,13 @@ B_AX_WDE_DATCHN_NULLPG_ERR_INT_EN | \ B_AX_WDE_DATCHN_FRZTO_ERR_INT_EN) #define B_AX_WDE_IMR_SET (B_AX_WDE_BUFREQ_QTAID_ERR_INT_EN | \ - B_AX_WDE_BUFREQ_SIZE0_INT_EN | \ - B_AX_WDE_BUFREQ_SIZELMT_INT_EN | \ - B_AX_WDE_BUFREQ_UNAVAL_ERR_INT_EN_V1 | \ - B_AX_WDE_BUFRTN_INVLD_PKTID_ERR_INT_EN_V1 | \ - B_AX_WDE_BUFRTN_SIZE_ERR_INT_EN_V1 | \ - B_AX_WDE_BUFREQ_SRCHTAILPG_ERR_INT_EN_V1 | \ - B_AX_WDE_GETNPG_STRPG_ERR_INT_EN_V1 | \ - B_AX_WDE_GETNPG_PGOFST_ERR_INT_EN_V1 | \ - B_AX_WDE_BUFMGN_FRZTO_ERR_INT_EN_V1 | \ + B_AX_WDE_BUFREQ_UNAVAL_ERR_INT_EN | \ + B_AX_WDE_BUFRTN_INVLD_PKTID_ERR_INT_EN | \ + B_AX_WDE_BUFRTN_SIZE_ERR_INT_EN | \ + B_AX_WDE_BUFREQ_SRCHTAILPG_ERR_INT_EN | \ + B_AX_WDE_GETNPG_STRPG_ERR_INT_EN | \ + B_AX_WDE_GETNPG_PGOFST_ERR_INT_EN | \ + B_AX_WDE_BUFMGN_FRZTO_ERR_INT_EN | \ B_AX_WDE_QUE_CMDTYPE_ERR_INT_EN | \ B_AX_WDE_QUE_DSTQUEID_ERR_INT_EN | \ B_AX_WDE_QUE_SRCQUEID_ERR_INT_EN | \ @@ -1043,10 +1058,7 @@ B_AX_WDE_QUEMGN_FRZTO_ERR_INT_EN | \ B_AX_WDE_DATCHN_ARBT_ERR_INT_EN | \ B_AX_WDE_DATCHN_NULLPG_ERR_INT_EN | \ - B_AX_WDE_DATCHN_FRZTO_ERR_INT_EN | \ - B_AX_WDE_DATCHN_RRDY_ERR_INT_EN | \ - B_AX_WDE_DATCHN_ADRERR_ERR_INT_EN | \ - B_AX_WDE_DATCHN_CAMREQ_ERR_INT_EN) + B_AX_WDE_DATCHN_FRZTO_ERR_INT_EN) #define B_AX_WDE_DATCHN_CAMREQ_ERR_INT_EN BIT(29) #define B_AX_WDE_DATCHN_ADRERR_ERR_INT_EN BIT(28) @@ -1826,6 +1838,13 @@ #define B_AX_TXSC_40M_MASK GENMASK(7, 4) #define B_AX_TXSC_20M_MASK GENMASK(3, 0) +#define R_AX_PTCL_RRSR1 0xC090 +#define R_AX_PTCL_RRSR1_C1 0xE090 +#define B_AX_RRSR_RATE_EN_MASK GENMASK(11, 8) +#define RRSR_OFDM_CCK_EN 3 +#define B_AX_RSC_MASK GENMASK(7, 6) +#define B_AX_RRSR_CCK_MASK GENMASK(3, 0) + #define R_AX_CMAC_ERR_IMR 0xC160 #define R_AX_CMAC_ERR_IMR_C1 0xE160 #define B_AX_WMAC_TX_ERR_IND_EN BIT(7) @@ -1882,6 +1901,7 @@ #define B_AX_SIFS_TIMEOUT_T2_MASK GENMASK(14, 8) #define B_AX_SIFS_MACTXEN_T1_MASK GENMASK(6, 0) #define SIFS_MACTXEN_T1 0x47 +#define SIFS_MACTXEN_T1_V1 0x41 #define R_AX_CCA_CFG_0 0xC340 #define R_AX_CCA_CFG_0_C1 0xE340 @@ -2098,6 +2118,8 @@ #define R_AX_TBTT_SHIFT_P3 0xC4E8 #define R_AX_TBTT_SHIFT_P4 0xC528 #define B_AX_TBTT_SHIFT_OFST_MASK GENMASK(11, 0) +#define B_AX_TBTT_SHIFT_OFST_SIGN BIT(11) +#define B_AX_TBTT_SHIFT_OFST_MAG GENMASK(10, 0) #define R_AX_BCN_CNT_TMR_P0 0xC434 #define R_AX_BCN_CNT_TMR_P1 0xC474 @@ -2258,6 +2280,7 @@ #define B_AX_F2PCMDRPT_FULL_DROP_ERR_INT_EN BIT(8) #define B_AX_FSM1_TIMEOUT_ERR_INT_EN BIT(1) #define B_AX_FSM_TIMEOUT_ERR_INT_EN BIT(0) +#define B_AX_PTCL_IMR_CLR_ALL GENMASK(31, 0) #define B_AX_PTCL_IMR_CLR (B_AX_FSM_TIMEOUT_ERR_INT_EN | \ B_AX_F2PCMDRPT_FULL_DROP_ERR_INT_EN | \ B_AX_TXPRT_FULL_DROP_ERR_INT_EN | \ @@ -2315,6 +2338,28 @@ #define B_AX_DLE_IMR_SET (B_AX_RXSTS_FSM_HANG_ERROR_IMR | \ B_AX_RXDATA_FSM_HANG_ERROR_IMR) +#define R_AX_RXDMA_CTRL_0 0xC804 +#define R_AX_RXDMA_CTRL_0_C1 0xE804 +#define B_AX_RXDMA_DBGOUT_EN BIT(31) +#define B_AX_RXDMA_DBG_SEL_MASK GENMASK(30, 29) +#define B_AX_RXDMA_FIFO_DBG_SEL_MASK GENMASK(28, 25) +#define B_AX_RXDMA_DEFAULT_PAGE_MASK GENMASK(22, 21) +#define B_AX_RXDMA_BUFF_REQ_PRI_MASK GENMASK(20, 19) +#define B_AX_RXDMA_TGT_QUEID_MASK GENMASK(18, 13) +#define B_AX_RXDMA_TGT_PRID_MASK GENMASK(12, 10) +#define B_AX_RXDMA_DIS_CSI_RELEASE BIT(9) +#define B_AX_RXDMA_DIS_RXSTS_WAIT_PTR_CLR BIT(7) +#define B_AX_RXDMA_DIS_CSI_WAIT_PTR_CLR BIT(6) +#define B_AX_RXSTS_PTR_FULL_MODE BIT(5) +#define B_AX_CSI_PTR_FULL_MODE BIT(4) +#define B_AX_RU3_PTR_FULL_MODE BIT(3) +#define B_AX_RU2_PTR_FULL_MODE BIT(2) +#define B_AX_RU1_PTR_FULL_MODE BIT(1) +#define B_AX_RU0_PTR_FULL_MODE BIT(0) +#define RX_FULL_MODE (B_AX_RU0_PTR_FULL_MODE | B_AX_RU1_PTR_FULL_MODE | \ + B_AX_RU2_PTR_FULL_MODE | B_AX_RU3_PTR_FULL_MODE | \ + B_AX_CSI_PTR_FULL_MODE | B_AX_RXSTS_PTR_FULL_MODE) + #define R_AX_RXDMA_PKT_INFO_0 0xC814 #define R_AX_RXDMA_PKT_INFO_1 0xC818 #define R_AX_RXDMA_PKT_INFO_2 0xC81C @@ -2553,6 +2598,20 @@ #define WMAC_SPEC_SIFS_OFDM_52C 0x11 #define WMAC_SPEC_SIFS_CCK 0xA +#define R_AX_TRXPTCL_RRSR_CTL_0 0xCC08 +#define R_AX_TRXPTCL_RRSR_CTL_0_C1 0xEC08 +#define B_AX_RESP_TX_MACID_CCA_TH_EN BIT(31) +#define B_AX_RESP_TX_PWRMODE_MASK GENMASK(30, 28) +#define B_AX_FTM_RRSR_RATE_EN_MASK GENMASK(27, 24) +#define B_AX_NESS_MASK GENMASK(23, 22) +#define B_AX_WMAC_RESP_DOPPLEB_AX_EN BIT(21) +#define B_AX_WMAC_RESP_DCM_EN BIT(20) +#define B_AX_WMAC_RRSB_AX_CCK_MASK GENMASK(19, 16) +#define B_AX_WMAC_RESP_RATE_EN_MASK GENMASK(15, 12) +#define B_AX_WMAC_RESP_RSC_MASK GENMASK(11, 10) +#define B_AX_WMAC_RESP_REF_RATE_SEL BIT(9) +#define B_AX_WMAC_RESP_REF_RATE_MASK GENMASK(8, 0) + #define R_AX_MAC_LOOPBACK 0xCC20 #define R_AX_MAC_LOOPBACK_C1 0xEC20 #define B_AX_MACLBK_EN BIT(0) @@ -2565,6 +2624,7 @@ #define B_AX_WMAC_TF_UP_NAV_EN BIT(16) #define B_AX_WMAC_NAV_UPPER_MASK GENMASK(15, 8) #define NAV_12MS 0xBC +#define NAV_25MS 0xC4 #define B_AX_WMAC_RTS_RST_DUR_MASK GENMASK(7, 0) #define R_AX_RXTRIG_TEST_USER_2 0xCCB0 @@ -2968,18 +3028,18 @@ #define R_AX_PATH_COM0 0xD800 #define AX_PATH_COM0_DFVAL 0x00000000 -#define AX_PATH_COM0_PATHA 0x08888880 -#define AX_PATH_COM0_PATHB 0x11111100 +#define AX_PATH_COM0_PATHA 0x08889880 +#define AX_PATH_COM0_PATHB 0x11111900 #define AX_PATH_COM0_PATHAB 0x19999980 #define R_AX_PATH_COM1 0xD804 #define AX_PATH_COM1_DFVAL 0x00000000 -#define AX_PATH_COM1_PATHA 0x11111111 -#define AX_PATH_COM1_PATHB 0x22222222 +#define AX_PATH_COM1_PATHA 0x13111111 +#define AX_PATH_COM1_PATHB 0x23222222 #define AX_PATH_COM1_PATHAB 0x33333333 #define R_AX_PATH_COM2 0xD808 #define AX_PATH_COM2_DFVAL 0x00000000 -#define AX_PATH_COM2_PATHA 0x01209111 -#define AX_PATH_COM2_PATHB 0x01209222 +#define AX_PATH_COM2_PATHA 0x01209313 +#define AX_PATH_COM2_PATHB 0x01209323 #define AX_PATH_COM2_PATHAB 0x01209333 #define R_AX_PATH_COM3 0xD80C #define AX_PATH_COM3_DFVAL 0x49249249 @@ -3125,6 +3185,18 @@ #define B_AX_GNT_WL_BB_VAL BIT(1) #define B_AX_GNT_WL_BB_SWCTRL BIT(0) +#define R_AX_GNT_VAL 0x0054 +#define B_AX_GNT_BT_RFC_S1_STA BIT(5) +#define B_AX_GNT_WL_RFC_S1_STA BIT(4) +#define B_AX_GNT_BT_RFC_S0_STA BIT(3) +#define B_AX_GNT_WL_RFC_S0_STA BIT(2) + +#define R_AX_GNT_VAL_V1 0xDA4C +#define B_AX_GNT_BT_RFC_S1 BIT(4) +#define B_AX_GNT_BT_RFC_S0 BIT(3) +#define B_AX_GNT_WL_RFC_S1 BIT(2) +#define B_AX_GNT_WL_RFC_S0 BIT(1) + #define R_AX_TDMA_MODE 0xDA4C #define R_AX_TDMA_MODE_C1 0xFA4C #define B_AX_R_BT_CMD_RPT_MASK GENMASK(31, 16) @@ -3356,6 +3428,7 @@ #define RR_DCK_FINE BIT(1) #define RR_DCK_LV BIT(0) #define RR_DCK1 0x93 +#define RR_DCK1_DONE BIT(5) #define RR_DCK1_CLR GENMASK(3, 0) #define RR_DCK1_SEL BIT(3) #define RR_DCK2 0x94 @@ -3431,8 +3504,9 @@ #define R_MAC_PIN_SEL 0x0734 #define B_CH_IDX_SEG0 GENMASK(23, 16) #define R_PLCP_HISTOGRAM 0x0738 -#define B_STS_DIS_TRIG_BY_BRK BIT(2) +#define B_STS_PARSING_TIME GENMASK(19, 16) #define B_STS_DIS_TRIG_BY_FAIL BIT(3) +#define B_STS_DIS_TRIG_BY_BRK BIT(2) #define R_PHY_STS_BITMAP_ADDR_START R_PHY_STS_BITMAP_SEARCH_FAIL #define B_PHY_STS_BITMAP_ADDR_MASK GENMASK(6, 2) #define R_PHY_STS_BITMAP_SEARCH_FAIL 0x073C @@ -3542,6 +3616,9 @@ #define B_P0_RXCK_VAL GENMASK(18, 16) #define B_P0_TXCK_ON BIT(15) #define B_P0_TXCK_VAL GENMASK(14, 12) +#define R_P0_RFMODE 0x12AC +#define B_P0_RFMODE_ORI_TXRX_FTM_TX GENMASK(31, 4) +#define B_P0_RFMODE_MUX GENMASK(11, 4) #define R_P0_NRBW 0x12B8 #define B_P0_NRBW_DBG BIT(30) #define R_S0_RXDC 0x12D4 @@ -3648,6 +3725,9 @@ #define B_P1_EN_SOUND_WO_NDP BIT(1) #define R_S1_HW_SI_DIS 0x3200 #define B_S1_HW_SI_DIS_W_R_TRIG GENMASK(30, 28) +#define R_P1_RFMODE 0x32AC +#define B_P1_RFMODE_ORI_TXRX_FTM_TX GENMASK(31, 4) +#define B_P1_RFMODE_MUX GENMASK(11, 4) #define R_P1_DBGMOD 0x32B8 #define B_P1_DBGMOD_ON BIT(30) #define R_S1_RXDC 0x32D4 @@ -3663,6 +3743,8 @@ #define R_S1_ADDCK 0x3E00 #define B_S1_ADDCK_I GENMASK(9, 0) #define B_S1_ADDCK_Q GENMASK(19, 10) +#define R_MUIC 0x40F8 +#define B_MUIC_EN BIT(0) #define R_DCFO 0x4264 #define B_DCFO GENMASK(1, 0) #define R_SEG0CSI 0x42AC @@ -3745,15 +3827,22 @@ #define R_PATH0_RXB_INIT 0x4658 #define B_PATH0_RXB_INIT_IDX_MSK GENMASK(9, 5) #define R_PATH0_LNA_INIT 0x4668 +#define R_PATH0_LNA_INIT_V1 0x472C #define B_PATH0_LNA_INIT_IDX_MSK GENMASK(26, 24) #define R_PATH0_BTG 0x466C #define B_PATH0_BTG_SHEN GENMASK(18, 17) #define R_PATH0_TIA_INIT 0x4674 #define B_PATH0_TIA_INIT_IDX_MSK BIT(17) #define R_PATH0_P20_FOLLOW_BY_PAGCUGC 0x46A0 +#define R_PATH0_P20_FOLLOW_BY_PAGCUGC_V1 0x4C24 +#define R_PATH0_P20_FOLLOW_BY_PAGCUGC_V2 0x46E8 #define B_PATH0_P20_FOLLOW_BY_PAGCUGC_EN_MSK BIT(5) #define R_PATH0_S20_FOLLOW_BY_PAGCUGC 0x46A4 +#define R_PATH0_S20_FOLLOW_BY_PAGCUGC_V1 0x4C28 +#define R_PATH0_S20_FOLLOW_BY_PAGCUGC_V2 0x46EC #define B_PATH0_S20_FOLLOW_BY_PAGCUGC_EN_MSK BIT(5) +#define R_PATH0_RXB_INIT_V1 0x46A8 +#define B_PATH0_RXB_INIT_IDX_MSK_V1 GENMASK(14, 10) #define R_PATH0_G_LNA6_OP1DB_V1 0x4688 #define B_PATH0_G_LNA6_OP1DB_V1 GENMASK(31, 24) #define R_PATH0_G_TIA0_LNA6_OP1DB_V1 0x4694 @@ -3780,7 +3869,10 @@ #define R_P0_AGC_CTL 0x4730 #define B_P0_AGC_EN BIT(31) #define R_PATH1_LNA_INIT 0x473C +#define R_PATH1_LNA_INIT_V1 0x4A80 #define B_PATH1_LNA_INIT_IDX_MSK GENMASK(26, 24) +#define R_PATH0_TIA_INIT_V1 0x473C +#define B_PATH0_TIA_INIT_IDX_MSK_V1 BIT(9) #define R_PATH1_TIA_INIT 0x4748 #define B_PATH1_TIA_INIT_IDX_MSK BIT(17) #define R_PATH1_BTG 0x4740 @@ -3790,8 +3882,12 @@ #define R_PATH1_G_LNA6_OP1DB_V1 0x476C #define B_PATH1_G_LNA6_OP1DB_V1 GENMASK(31, 24) #define R_PATH1_P20_FOLLOW_BY_PAGCUGC 0x4774 +#define R_PATH1_P20_FOLLOW_BY_PAGCUGC_V1 0x4CE8 +#define R_PATH1_P20_FOLLOW_BY_PAGCUGC_V2 0x47A8 #define B_PATH1_P20_FOLLOW_BY_PAGCUGC_EN_MSK BIT(5) #define R_PATH1_S20_FOLLOW_BY_PAGCUGC 0x4778 +#define R_PATH1_S20_FOLLOW_BY_PAGCUGC_V1 0x4CEC +#define R_PATH1_S20_FOLLOW_BY_PAGCUGC_V2 0x47AC #define B_PATH1_S20_FOLLOW_BY_PAGCUGC_EN_MSK BIT(5) #define R_PATH1_G_TIA0_LNA6_OP1DB_V1 0x4778 #define B_PATH1_G_TIA0_LNA6_OP1DB_V1 GENMASK(7, 0) @@ -3807,6 +3903,8 @@ #define B_P1_NBIIDX_VAL GENMASK(11, 0) #define B_P1_NBIIDX_NOTCH_EN BIT(12) #define R_SEG0R_PD 0x481C +#define R_SEG0R_PD_V1 0x4860 +#define B_SEG0R_PD_SPATIAL_REUSE_EN_MSK_V1 BIT(30) #define B_SEG0R_PD_SPATIAL_REUSE_EN_MSK BIT(29) #define B_SEG0R_PD_LOWER_BOUND_MSK GENMASK(10, 6) #define R_2P4G_BAND 0x4970 @@ -3830,8 +3928,12 @@ #define B_BK_FC0_INV_MSK_V1 GENMASK(18, 0) #define R_CCK_FC0_INV_V1 0x4A20 #define B_CCK_FC0_INV_MSK_V1 GENMASK(18, 0) +#define R_PATH1_RXB_INIT_V1 0x4A5C +#define B_PATH1_RXB_INIT_IDX_MSK_V1 GENMASK(14, 10) #define R_P1_AGC_CTL 0x4A9C #define B_P1_AGC_EN BIT(31) +#define R_PATH1_TIA_INIT_V1 0x4AA8 +#define B_PATH1_TIA_INIT_IDX_MSK_V1 BIT(9) #define R_PATH0_RXBB_V1 0x4AD4 #define B_PATH0_RXBB_MSK_V1 GENMASK(31, 0) #define R_PATH1_RXBB_V1 0x4AE0 diff --git a/drivers/net/wireless/realtek/rtw89/regd.c b/drivers/net/wireless/realtek/rtw89/regd.c index 20c7afd3e70f..6e5a740b128f 100644 --- a/drivers/net/wireless/realtek/rtw89/regd.c +++ b/drivers/net/wireless/realtek/rtw89/regd.c @@ -346,7 +346,7 @@ void rtw89_regd_notifier(struct wiphy *wiphy, struct regulatory_request *request rtw89_debug_regd(rtwdev, rtwdev->regd, "get from initiator %d, alpha2", request->initiator); - rtw89_chip_set_txpwr(rtwdev); + rtw89_core_set_chip_txpwr(rtwdev); exit: mutex_unlock(&rtwdev->mutex); diff --git a/drivers/net/wireless/realtek/rtw89/rtw8852a.c b/drivers/net/wireless/realtek/rtw89/rtw8852a.c index 81bd0c4fe21b..784147680353 100644 --- a/drivers/net/wireless/realtek/rtw89/rtw8852a.c +++ b/drivers/net/wireless/realtek/rtw89/rtw8852a.c @@ -431,6 +431,7 @@ static const struct rtw89_imr_info rtw8852a_imr_info = { .cpu_disp_imr_set = B_AX_CPU_DISP_IMR_SET, .other_disp_imr_clr = B_AX_OTHER_DISP_IMR_CLR, .other_disp_imr_set = 0, + .bbrpt_com_err_imr_reg = R_AX_BBRPT_COM_ERR_IMR_ISR, .bbrpt_chinfo_err_imr_reg = R_AX_BBRPT_CHINFO_ERR_IMR_ISR, .bbrpt_err_imr_set = 0, .bbrpt_dfs_err_imr_reg = R_AX_BBRPT_DFS_ERR_IMR_ISR, @@ -453,6 +454,31 @@ static const struct rtw89_imr_info rtw8852a_imr_info = { .tmac_imr_set = B_AX_TMAC_IMR_SET, }; +static const struct rtw89_rrsr_cfgs rtw8852a_rrsr_cfgs = { + .ref_rate = {R_AX_TRXPTCL_RRSR_CTL_0, B_AX_WMAC_RESP_REF_RATE_SEL, 0}, + .rsc = {R_AX_TRXPTCL_RRSR_CTL_0, B_AX_WMAC_RESP_RSC_MASK, 2}, +}; + +static const struct rtw89_dig_regs rtw8852a_dig_regs = { + .seg0_pd_reg = R_SEG0R_PD, + .pd_lower_bound_mask = B_SEG0R_PD_LOWER_BOUND_MSK, + .pd_spatial_reuse_en = B_SEG0R_PD_SPATIAL_REUSE_EN_MSK, + .p0_lna_init = {R_PATH0_LNA_INIT, B_PATH0_LNA_INIT_IDX_MSK}, + .p1_lna_init = {R_PATH1_LNA_INIT, B_PATH1_LNA_INIT_IDX_MSK}, + .p0_tia_init = {R_PATH0_TIA_INIT, B_PATH0_TIA_INIT_IDX_MSK}, + .p1_tia_init = {R_PATH1_TIA_INIT, B_PATH1_TIA_INIT_IDX_MSK}, + .p0_rxb_init = {R_PATH0_RXB_INIT, B_PATH0_RXB_INIT_IDX_MSK}, + .p1_rxb_init = {R_PATH1_RXB_INIT, B_PATH1_RXB_INIT_IDX_MSK}, + .p0_p20_pagcugc_en = {R_PATH0_P20_FOLLOW_BY_PAGCUGC, + B_PATH0_P20_FOLLOW_BY_PAGCUGC_EN_MSK}, + .p0_s20_pagcugc_en = {R_PATH0_S20_FOLLOW_BY_PAGCUGC, + B_PATH0_S20_FOLLOW_BY_PAGCUGC_EN_MSK}, + .p1_p20_pagcugc_en = {R_PATH1_P20_FOLLOW_BY_PAGCUGC, + B_PATH1_P20_FOLLOW_BY_PAGCUGC_EN_MSK}, + .p1_s20_pagcugc_en = {R_PATH1_S20_FOLLOW_BY_PAGCUGC, + B_PATH1_S20_FOLLOW_BY_PAGCUGC_EN_MSK}, +}; + static void rtw8852ae_efuse_parsing(struct rtw89_efuse *efuse, struct rtw8852a_efuse *map) { @@ -660,7 +686,7 @@ static void rtw8852a_power_trim(struct rtw89_dev *rtwdev) } static void rtw8852a_set_channel_mac(struct rtw89_dev *rtwdev, - struct rtw89_channel_params *param, + const struct rtw89_chan *chan, u8 mac_idx) { u32 rf_mod = rtw89_mac_reg_by_idx(R_AX_WMAC_RFMOD, mac_idx); @@ -669,20 +695,20 @@ static void rtw8852a_set_channel_mac(struct rtw89_dev *rtwdev, u32 chk_rate = rtw89_mac_reg_by_idx(R_AX_TXRATE_CHK, mac_idx); u8 txsc20 = 0, txsc40 = 0; - switch (param->bandwidth) { + switch (chan->band_width) { case RTW89_CHANNEL_WIDTH_80: - txsc40 = rtw89_phy_get_txsc(rtwdev, param, + txsc40 = rtw89_phy_get_txsc(rtwdev, chan, RTW89_CHANNEL_WIDTH_40); fallthrough; case RTW89_CHANNEL_WIDTH_40: - txsc20 = rtw89_phy_get_txsc(rtwdev, param, + txsc20 = rtw89_phy_get_txsc(rtwdev, chan, RTW89_CHANNEL_WIDTH_20); break; default: break; } - switch (param->bandwidth) { + switch (chan->band_width) { case RTW89_CHANNEL_WIDTH_80: rtw89_write8_mask(rtwdev, rf_mod, B_AX_WMAC_RFMOD_MASK, BIT(1)); rtw89_write32(rtwdev, sub_carr, txsc20 | (txsc40 << 4)); @@ -699,7 +725,7 @@ static void rtw8852a_set_channel_mac(struct rtw89_dev *rtwdev, break; } - if (param->center_chan > 14) + if (chan->channel > 14) rtw89_write8_set(rtwdev, chk_rate, B_AX_CHECK_CCK_EN | B_AX_RTS_LIMIT_IN_OFDM6); else @@ -1102,11 +1128,12 @@ static void rtw8852a_bb_sethw(struct rtw89_dev *rtwdev) if (rtwdev->hal.cv <= CHIP_CCV) { rtw89_phy_write32_set(rtwdev, R_RSTB_WATCH_DOG, B_P0_RSTB_WATCH_DOG); rtw89_phy_write32(rtwdev, R_BRK_ASYNC_RST_EN_1, 0x864FA000); - rtw89_phy_write32(rtwdev, R_BRK_ASYNC_RST_EN_2, 0x3F); + rtw89_phy_write32(rtwdev, R_BRK_ASYNC_RST_EN_2, 0x43F); rtw89_phy_write32(rtwdev, R_BRK_ASYNC_RST_EN_3, 0x7FFF); rtw89_phy_write32_set(rtwdev, R_SPOOF_ASYNC_RST, B_SPOOF_ASYNC_RST); rtw89_phy_write32_set(rtwdev, R_P0_TXPW_RSTB, B_P0_TXPW_RSTB_MANON); rtw89_phy_write32_set(rtwdev, R_P1_TXPW_RSTB, B_P1_TXPW_RSTB_MANON); + rtw89_phy_write32_set(rtwdev, R_PLCP_HISTOGRAM, B_STS_PARSING_TIME); } rtw89_phy_write32_mask(rtwdev, R_CFO_TRK0, B_CFO_TRK_MSK, 0x1f); rtw89_phy_write32_mask(rtwdev, R_CFO_TRK1, B_CFO_TRK_MSK, 0x0c); @@ -1130,35 +1157,38 @@ static void rtw8852a_bbrst_for_rfk(struct rtw89_dev *rtwdev, } static void rtw8852a_set_channel_bb(struct rtw89_dev *rtwdev, - struct rtw89_channel_params *param, + const struct rtw89_chan *chan, enum rtw89_phy_idx phy_idx) { - bool cck_en = param->center_chan <= 14; - u8 pri_ch_idx = param->pri_ch_idx; + bool cck_en = chan->channel <= 14; + u8 pri_ch_idx = chan->pri_ch_idx; if (cck_en) - rtw8852a_ctrl_sco_cck(rtwdev, param->center_chan, - param->primary_chan, param->bandwidth); + rtw8852a_ctrl_sco_cck(rtwdev, chan->channel, + chan->primary_channel, + chan->band_width); - rtw8852a_ctrl_ch(rtwdev, param->center_chan, phy_idx); - rtw8852a_ctrl_bw(rtwdev, pri_ch_idx, param->bandwidth, phy_idx); + rtw8852a_ctrl_ch(rtwdev, chan->channel, phy_idx); + rtw8852a_ctrl_bw(rtwdev, pri_ch_idx, chan->band_width, phy_idx); if (cck_en) { rtw89_phy_write32_mask(rtwdev, R_RXCCA, B_RXCCA_DIS, 0); } else { rtw89_phy_write32_mask(rtwdev, R_RXCCA, B_RXCCA_DIS, 1); rtw8852a_bbrst_for_rfk(rtwdev, phy_idx); } - rtw8852a_spur_elimination(rtwdev, param->center_chan); + rtw8852a_spur_elimination(rtwdev, chan->channel); rtw89_phy_write32_mask(rtwdev, R_MAC_PIN_SEL, B_CH_IDX_SEG0, - param->primary_chan); + chan->primary_channel); rtw8852a_bb_reset_all(rtwdev, phy_idx); } static void rtw8852a_set_channel(struct rtw89_dev *rtwdev, - struct rtw89_channel_params *params) + const struct rtw89_chan *chan, + enum rtw89_mac_idx mac_idx, + enum rtw89_phy_idx phy_idx) { - rtw8852a_set_channel_mac(rtwdev, params, RTW89_MAC_0); - rtw8852a_set_channel_bb(rtwdev, params, RTW89_PHY_0); + rtw8852a_set_channel_mac(rtwdev, chan, mac_idx); + rtw8852a_set_channel_bb(rtwdev, chan, phy_idx); } static void rtw8852a_dfs_en(struct rtw89_dev *rtwdev, bool en) @@ -1209,25 +1239,27 @@ static void rtw8852a_adc_en(struct rtw89_dev *rtwdev, bool en) } static void rtw8852a_set_channel_help(struct rtw89_dev *rtwdev, bool enter, - struct rtw89_channel_help_params *p) + struct rtw89_channel_help_params *p, + const struct rtw89_chan *chan, + enum rtw89_mac_idx mac_idx, + enum rtw89_phy_idx phy_idx) { - u8 phy_idx = RTW89_PHY_0; - if (enter) { - rtw89_chip_stop_sch_tx(rtwdev, RTW89_MAC_0, &p->tx_en, RTW89_SCH_TX_SEL_ALL); - rtw89_mac_cfg_ppdu_status(rtwdev, RTW89_MAC_0, false); + rtw89_chip_stop_sch_tx(rtwdev, mac_idx, &p->tx_en, + RTW89_SCH_TX_SEL_ALL); + rtw89_mac_cfg_ppdu_status(rtwdev, mac_idx, false); rtw8852a_dfs_en(rtwdev, false); - rtw8852a_tssi_cont_en_phyidx(rtwdev, false, RTW89_PHY_0); + rtw8852a_tssi_cont_en_phyidx(rtwdev, false, phy_idx); rtw8852a_adc_en(rtwdev, false); fsleep(40); rtw8852a_bb_reset_en(rtwdev, phy_idx, false); } else { - rtw89_mac_cfg_ppdu_status(rtwdev, RTW89_MAC_0, true); + rtw89_mac_cfg_ppdu_status(rtwdev, mac_idx, true); rtw8852a_adc_en(rtwdev, true); rtw8852a_dfs_en(rtwdev, true); - rtw8852a_tssi_cont_en_phyidx(rtwdev, true, RTW89_PHY_0); + rtw8852a_tssi_cont_en_phyidx(rtwdev, true, phy_idx); rtw8852a_bb_reset_en(rtwdev, phy_idx, true); - rtw89_chip_resume_sch_tx(rtwdev, RTW89_MAC_0, p->tx_en); + rtw89_chip_resume_sch_tx(rtwdev, mac_idx, p->tx_en); } } @@ -1277,9 +1309,10 @@ static void rtw8852a_rfk_channel(struct rtw89_dev *rtwdev) rtw8852a_dpk(rtwdev, phy_idx); } -static void rtw8852a_rfk_band_changed(struct rtw89_dev *rtwdev) +static void rtw8852a_rfk_band_changed(struct rtw89_dev *rtwdev, + enum rtw89_phy_idx phy_idx) { - rtw8852a_tssi_scan(rtwdev, RTW89_PHY_0); + rtw8852a_tssi_scan(rtwdev, phy_idx); } static void rtw8852a_rfk_scan(struct rtw89_dev *rtwdev, bool start) @@ -1378,9 +1411,11 @@ static void rtw8852a_set_txpwr_ref(struct rtw89_dev *rtwdev, } static void rtw8852a_set_txpwr_byrate(struct rtw89_dev *rtwdev, + const struct rtw89_chan *chan, enum rtw89_phy_idx phy_idx) { - u8 ch = rtwdev->hal.current_channel; + u8 band = chan->band_type; + u8 ch = chan->channel; static const u8 rs[] = { RTW89_RS_CCK, RTW89_RS_OFDM, @@ -1406,7 +1441,8 @@ static void rtw8852a_set_txpwr_byrate(struct rtw89_dev *rtwdev, for (j = 0; j < rtw89_rs_idx_max[rs[i]]; j++) { cur.idx = j; shf = (j % 4) * 8; - tmp = rtw89_phy_read_txpwr_byrate(rtwdev, &cur); + tmp = rtw89_phy_read_txpwr_byrate(rtwdev, band, + &cur); val |= (tmp << shf); if ((j + 1) % 4) @@ -1421,8 +1457,10 @@ static void rtw8852a_set_txpwr_byrate(struct rtw89_dev *rtwdev, } static void rtw8852a_set_txpwr_offset(struct rtw89_dev *rtwdev, + const struct rtw89_chan *chan, enum rtw89_phy_idx phy_idx) { + u8 band = chan->band_type; struct rtw89_rate_desc desc = { .nss = RTW89_NSS_1, .rs = RTW89_RS_OFFSET, @@ -1433,7 +1471,7 @@ static void rtw8852a_set_txpwr_offset(struct rtw89_dev *rtwdev, rtw89_debug(rtwdev, RTW89_DBG_TXPWR, "[TXPWR] set txpwr offset\n"); for (desc.idx = 0; desc.idx < RTW89_RATE_OFFSET_MAX; desc.idx++) { - v = rtw89_phy_read_txpwr_byrate(rtwdev, &desc); + v = rtw89_phy_read_txpwr_byrate(rtwdev, band, &desc); val |= ((v & 0xf) << (4 * desc.idx)); } @@ -1442,29 +1480,31 @@ static void rtw8852a_set_txpwr_offset(struct rtw89_dev *rtwdev, } static void rtw8852a_set_txpwr_limit(struct rtw89_dev *rtwdev, + const struct rtw89_chan *chan, enum rtw89_phy_idx phy_idx) { #define __MAC_TXPWR_LMT_PAGE_SIZE 40 - u8 ch = rtwdev->hal.current_channel; - u8 bw = rtwdev->hal.current_band_width; + u8 ch = chan->channel; + u8 bw = chan->band_width; struct rtw89_txpwr_limit lmt[NTX_NUM_8852A]; u32 addr, val; const s8 *ptr; - u8 i, j, k; + u8 i, j; rtw89_debug(rtwdev, RTW89_DBG_TXPWR, "[TXPWR] set txpwr limit with ch=%d bw=%d\n", ch, bw); for (i = 0; i < NTX_NUM_8852A; i++) { - rtw89_phy_fill_txpwr_limit(rtwdev, &lmt[i], i); + rtw89_phy_fill_txpwr_limit(rtwdev, chan, &lmt[i], i); for (j = 0; j < __MAC_TXPWR_LMT_PAGE_SIZE; j += 4) { addr = R_AX_PWR_LMT + j + __MAC_TXPWR_LMT_PAGE_SIZE * i; ptr = (s8 *)&lmt[i] + j; - val = 0; - for (k = 0; k < 4; k++) - val |= (ptr[k] << (8 * k)); + val = FIELD_PREP(GENMASK(7, 0), ptr[0]) | + FIELD_PREP(GENMASK(15, 8), ptr[1]) | + FIELD_PREP(GENMASK(23, 16), ptr[2]) | + FIELD_PREP(GENMASK(31, 24), ptr[3]); rtw89_mac_txpwr_write32(rtwdev, phy_idx, addr, val); } @@ -1473,30 +1513,32 @@ static void rtw8852a_set_txpwr_limit(struct rtw89_dev *rtwdev, } static void rtw8852a_set_txpwr_limit_ru(struct rtw89_dev *rtwdev, + const struct rtw89_chan *chan, enum rtw89_phy_idx phy_idx) { #define __MAC_TXPWR_LMT_RU_PAGE_SIZE 24 - u8 ch = rtwdev->hal.current_channel; - u8 bw = rtwdev->hal.current_band_width; + u8 ch = chan->channel; + u8 bw = chan->band_width; struct rtw89_txpwr_limit_ru lmt_ru[NTX_NUM_8852A]; u32 addr, val; const s8 *ptr; - u8 i, j, k; + u8 i, j; rtw89_debug(rtwdev, RTW89_DBG_TXPWR, "[TXPWR] set txpwr limit ru with ch=%d bw=%d\n", ch, bw); for (i = 0; i < NTX_NUM_8852A; i++) { - rtw89_phy_fill_txpwr_limit_ru(rtwdev, &lmt_ru[i], i); + rtw89_phy_fill_txpwr_limit_ru(rtwdev, chan, &lmt_ru[i], i); for (j = 0; j < __MAC_TXPWR_LMT_RU_PAGE_SIZE; j += 4) { addr = R_AX_PWR_RU_LMT + j + __MAC_TXPWR_LMT_RU_PAGE_SIZE * i; ptr = (s8 *)&lmt_ru[i] + j; - val = 0; - for (k = 0; k < 4; k++) - val |= (ptr[k] << (8 * k)); + val = FIELD_PREP(GENMASK(7, 0), ptr[0]) | + FIELD_PREP(GENMASK(15, 8), ptr[1]) | + FIELD_PREP(GENMASK(23, 16), ptr[2]) | + FIELD_PREP(GENMASK(31, 24), ptr[3]); rtw89_mac_txpwr_write32(rtwdev, phy_idx, addr, val); } @@ -1505,17 +1547,20 @@ static void rtw8852a_set_txpwr_limit_ru(struct rtw89_dev *rtwdev, #undef __MAC_TXPWR_LMT_RU_PAGE_SIZE } -static void rtw8852a_set_txpwr(struct rtw89_dev *rtwdev) +static void rtw8852a_set_txpwr(struct rtw89_dev *rtwdev, + const struct rtw89_chan *chan, + enum rtw89_phy_idx phy_idx) { - rtw8852a_set_txpwr_byrate(rtwdev, RTW89_PHY_0); - rtw8852a_set_txpwr_limit(rtwdev, RTW89_PHY_0); - rtw8852a_set_txpwr_limit_ru(rtwdev, RTW89_PHY_0); + rtw8852a_set_txpwr_byrate(rtwdev, chan, phy_idx); + rtw8852a_set_txpwr_offset(rtwdev, chan, phy_idx); + rtw8852a_set_txpwr_limit(rtwdev, chan, phy_idx); + rtw8852a_set_txpwr_limit_ru(rtwdev, chan, phy_idx); } -static void rtw8852a_set_txpwr_ctrl(struct rtw89_dev *rtwdev) +static void rtw8852a_set_txpwr_ctrl(struct rtw89_dev *rtwdev, + enum rtw89_phy_idx phy_idx) { - rtw8852a_set_txpwr_ref(rtwdev, RTW89_PHY_0); - rtw8852a_set_txpwr_offset(rtwdev, RTW89_PHY_0); + rtw8852a_set_txpwr_ref(rtwdev, phy_idx); } static int @@ -1592,10 +1637,12 @@ void rtw8852a_bb_set_pmac_tx(struct rtw89_dev *rtwdev, struct rtw8852a_bb_pmac_info *tx_info, enum rtw89_phy_idx idx) { + const struct rtw89_chan *chan = rtw89_chan_get(rtwdev, RTW89_SUB_ENTITY_0); + if (!tx_info->en_pmac_tx) { rtw8852a_stop_pmac_tx(rtwdev, tx_info, idx); rtw89_phy_write32_idx(rtwdev, R_PD_CTRL, B_PD_HIT_DIS, 0, idx); - if (rtwdev->hal.current_band_type == RTW89_BAND_2G) + if (chan->band_type == RTW89_BAND_2G) rtw89_phy_write32_clr(rtwdev, R_RXCCA, B_RXCCA_DIS); return; } @@ -1797,6 +1844,9 @@ static void rtw8852a_btc_init_cfg(struct rtw89_dev *rtwdev) RF_PATH_A, BTC_BT_SS_GROUP, 0x5ff); rtw8852a_set_trx_mask(rtwdev, RF_PATH_B, BTC_BT_SS_GROUP, 0x5ff); + /* set path-A(S0) Tx/Rx no-mask if GNT_WL=0 && BT_S1=tx group */ + rtw8852a_set_trx_mask(rtwdev, + RF_PATH_A, BTC_BT_TX_GROUP, 0x5ff); } else { /* set WL Tx stb if GNT_WL = 0 && BT_S1 = ss group for 3-ant */ rtw8852a_set_trx_mask(rtwdev, RF_PATH_A, BTC_BT_SS_GROUP, 0x5df); @@ -2010,6 +2060,51 @@ void rtw8852a_btc_wl_s1_standby(struct rtw89_dev *rtwdev, bool state) rtw89_write_rf(rtwdev, RF_PATH_B, RR_LUTWE, RFREG_MASK, 0x0); } +static void rtw8852a_set_wl_lna2(struct rtw89_dev *rtwdev, u8 level) +{ + /* level=0 Default: TIA 1/0= (LNA2,TIAN6) = (7,1)/(5,1) = 21dB/12dB + * level=1 Fix LNA2=5: TIA 1/0= (LNA2,TIAN6) = (5,0)/(5,1) = 18dB/12dB + * To improve BT ACI in co-rx + */ + + switch (level) { + case 0: /* default */ + rtw89_write_rf(rtwdev, RF_PATH_B, RR_LUTWE, RFREG_MASK, 0x1000); + rtw89_write_rf(rtwdev, RF_PATH_B, RR_LUTWA, RFREG_MASK, 0x3); + rtw89_write_rf(rtwdev, RF_PATH_B, RR_LUTWD0, RFREG_MASK, 0x17); + rtw89_write_rf(rtwdev, RF_PATH_B, RR_LUTWA, RFREG_MASK, 0x2); + rtw89_write_rf(rtwdev, RF_PATH_B, RR_LUTWD0, RFREG_MASK, 0x15); + rtw89_write_rf(rtwdev, RF_PATH_B, RR_LUTWE, RFREG_MASK, 0x0); + break; + case 1: /* Fix LNA2=5 */ + rtw89_write_rf(rtwdev, RF_PATH_B, RR_LUTWE, RFREG_MASK, 0x1000); + rtw89_write_rf(rtwdev, RF_PATH_B, RR_LUTWA, RFREG_MASK, 0x3); + rtw89_write_rf(rtwdev, RF_PATH_B, RR_LUTWD0, RFREG_MASK, 0x5); + rtw89_write_rf(rtwdev, RF_PATH_B, RR_LUTWA, RFREG_MASK, 0x2); + rtw89_write_rf(rtwdev, RF_PATH_B, RR_LUTWD0, RFREG_MASK, 0x15); + rtw89_write_rf(rtwdev, RF_PATH_B, RR_LUTWE, RFREG_MASK, 0x0); + break; + } +} + +static void rtw8852a_btc_set_wl_rx_gain(struct rtw89_dev *rtwdev, u32 level) +{ + switch (level) { + case 0: /* original */ + rtw8852a_bb_ctrl_btc_preagc(rtwdev, false); + rtw8852a_set_wl_lna2(rtwdev, 0); + break; + case 1: /* for FDD free-run */ + rtw8852a_bb_ctrl_btc_preagc(rtwdev, true); + rtw8852a_set_wl_lna2(rtwdev, 0); + break; + case 2: /* for BTG Co-Rx*/ + rtw8852a_bb_ctrl_btc_preagc(rtwdev, false); + rtw8852a_set_wl_lna2(rtwdev, 1); + break; + } +} + static void rtw8852a_fill_freq_with_ppdu(struct rtw89_dev *rtwdev, struct rtw89_rx_phy_ppdu *phy_ppdu, struct ieee80211_rx_status *status) @@ -2030,12 +2125,12 @@ static void rtw8852a_query_ppdu(struct rtw89_dev *rtwdev, struct ieee80211_rx_status *status) { u8 path; - s8 *rx_power = phy_ppdu->rssi; + u8 *rx_power = phy_ppdu->rssi; - status->signal = max_t(s8, rx_power[RF_PATH_A], rx_power[RF_PATH_B]); + status->signal = RTW89_RSSI_RAW_TO_DBM(max(rx_power[RF_PATH_A], rx_power[RF_PATH_B])); for (path = 0; path < rtwdev->chip->rf_path_num; path++) { status->chains |= BIT(path); - status->chain_signal[path] = rx_power[path]; + status->chain_signal[path] = RTW89_RSSI_RAW_TO_DBM(rx_power[path]); } if (phy_ppdu->valid) rtw8852a_fill_freq_with_ppdu(rtwdev, phy_ppdu, status); @@ -2086,6 +2181,8 @@ static const struct rtw89_chip_ops rtw8852a_chip_ops = { .btc_bt_aci_imp = rtw8852a_btc_bt_aci_imp, .btc_update_bt_cnt = rtw8852a_btc_update_bt_cnt, .btc_wl_s1_standby = rtw8852a_btc_wl_s1_standby, + .btc_set_wl_rx_gain = rtw8852a_btc_set_wl_rx_gain, + .btc_set_policy = rtw89_btc_set_policy, }; const struct rtw89_chip_info rtw8852a_chip_info = { @@ -2093,6 +2190,7 @@ const struct rtw89_chip_info rtw8852a_chip_info = { .ops = &rtw8852a_chip_ops, .fw_name = "rtw89/rtw8852a_fw.bin", .fifo_size = 458752, + .dle_scc_rsvd_size = 0, .max_amsdu_limit = 3500, .dis_2g_40m_ul_ofdma = true, .rsvd_ple_ofst = 0x6f800, @@ -2114,7 +2212,9 @@ const struct rtw89_chip_info rtw8852a_chip_info = { .txpwr_factor_rf = 2, .txpwr_factor_mac = 1, .dig_table = &rtw89_8852a_phy_dig_table, + .dig_regs = &rtw8852a_dig_regs, .tssi_dbw_table = NULL, + .support_chanctx_num = 1, .support_bands = BIT(NL80211_BAND_2GHZ) | BIT(NL80211_BAND_5GHZ), .support_bw160 = false, @@ -2125,6 +2225,9 @@ const struct rtw89_chip_info rtw8852a_chip_info = { .acam_num = 128, .bcam_num = 10, .scam_num = 128, + .bacam_num = 2, + .bacam_dynamic_num = 4, + .bacam_v1 = false, .sec_ctrl_efuse_size = 4, .physical_efuse_size = 1216, .logical_efuse_size = 1536, @@ -2133,11 +2236,26 @@ const struct rtw89_chip_info rtw8852a_chip_info = { .dav_log_efuse_size = 0, .phycap_addr = 0x580, .phycap_size = 128, - .para_ver = 0x05050864, - .wlcx_desired = 0x05050000, - .btcx_desired = 0x5, + .para_ver = 0x0, + .wlcx_desired = 0x06000000, + .btcx_desired = 0x7, .scbd = 0x1, .mailbox = 0x1, + .btc_fwinfo_buf = 1024, + + .fcxbtcrpt_ver = 1, + .fcxtdma_ver = 1, + .fcxslots_ver = 1, + .fcxcysta_ver = 2, + .fcxstep_ver = 2, + .fcxnullsta_ver = 1, + .fcxmreg_ver = 1, + .fcxgpiodbg_ver = 1, + .fcxbtver_ver = 1, + .fcxbtscan_ver = 1, + .fcxbtafh_ver = 1, + .fcxbtdevinfo_ver = 1, + .afh_guard_ch = 6, .wl_rssi_thres = rtw89_btc_8852a_wl_rssi_thres, .bt_rssi_thres = rtw89_btc_8852a_bt_rssi_thres, @@ -2163,7 +2281,9 @@ const struct rtw89_chip_info rtw8852a_chip_info = { .page_regs = &rtw8852a_page_regs, .dcfo_comp = &rtw8852a_dcfo_comp, .dcfo_comp_sft = 3, - .imr_info = &rtw8852a_imr_info + .imr_info = &rtw8852a_imr_info, + .rrsr_cfgs = &rtw8852a_rrsr_cfgs, + .dma_ch_mask = 0, }; EXPORT_SYMBOL(rtw8852a_chip_info); diff --git a/drivers/net/wireless/realtek/rtw89/rtw8852a_rfk.c b/drivers/net/wireless/realtek/rtw89/rtw8852a_rfk.c index 3d60feb78312..582ff0d3a9ea 100644 --- a/drivers/net/wireless/realtek/rtw89/rtw8852a_rfk.c +++ b/drivers/net/wireless/realtek/rtw89/rtw8852a_rfk.c @@ -1359,7 +1359,7 @@ static void _iqk_get_ch_info(struct rtw89_dev *rtwdev, enum rtw89_phy_idx phy, u8 path) { struct rtw89_iqk_info *iqk_info = &rtwdev->iqk; - struct rtw89_hal *hal = &rtwdev->hal; + const struct rtw89_chan *chan = rtw89_chan_get(rtwdev, RTW89_SUB_ENTITY_0); u32 reg_rf18 = 0x0, reg_35c = 0x0; u8 idx = 0; u8 get_empty_table = false; @@ -1380,9 +1380,9 @@ static void _iqk_get_ch_info(struct rtw89_dev *rtwdev, rtw89_debug(rtwdev, RTW89_DBG_RFK, "[IQK]cfg ch = %d\n", reg_rf18); reg_35c = rtw89_phy_read32_mask(rtwdev, 0x35c, 0x00000c00); - iqk_info->iqk_band[path] = hal->current_band_type; - iqk_info->iqk_bw[path] = hal->current_band_width; - iqk_info->iqk_ch[path] = hal->current_channel; + iqk_info->iqk_band[path] = chan->band_type; + iqk_info->iqk_bw[path] = chan->band_width; + iqk_info->iqk_ch[path] = chan->channel; rtw89_debug(rtwdev, RTW89_DBG_RFK, "[IQK]iqk_info->iqk_band[%x] = 0x%x\n", path, @@ -1879,13 +1879,12 @@ static void _dpk_information(struct rtw89_dev *rtwdev, enum rtw89_rf_path path) { struct rtw89_dpk_info *dpk = &rtwdev->dpk; - struct rtw89_hal *hal = &rtwdev->hal; - + const struct rtw89_chan *chan = rtw89_chan_get(rtwdev, RTW89_SUB_ENTITY_0); u8 kidx = dpk->cur_idx[path]; - dpk->bp[path][kidx].band = hal->current_band_type; - dpk->bp[path][kidx].ch = hal->current_channel; - dpk->bp[path][kidx].bw = hal->current_band_width; + dpk->bp[path][kidx].band = chan->band_type; + dpk->bp[path][kidx].ch = chan->channel; + dpk->bp[path][kidx].bw = chan->band_width; rtw89_debug(rtwdev, RTW89_DBG_RFK, "[DPK] S%d[%d] (PHY%d): TSSI %s/ DBCC %s/ %s/ CH%d/ %s\n", @@ -2358,6 +2357,7 @@ static u8 _dpk_agc(struct rtw89_dev *rtwdev, enum rtw89_phy_idx phy, #define DPK_RXBB_UPPER 0x1f #define DPK_RXBB_LOWER 0 #define DPK_GL_CRIT 7 + const struct rtw89_chan *chan = rtw89_chan_get(rtwdev, RTW89_SUB_ENTITY_0); u8 tmp_txagc, tmp_rxbb = 0, tmp_gl_idx = 0; u8 agc_cnt = 0; bool limited_rxbb = false; @@ -2404,7 +2404,7 @@ static u8 _dpk_agc(struct rtw89_dev *rtwdev, enum rtw89_phy_idx phy, "[DPK] Adjust RXBB (%d) = 0x%x\n", offset, tmp_rxbb); if (offset != 0 || agc_cnt == 0) { - if (rtwdev->hal.current_band_width < RTW89_CHANNEL_WIDTH_80) + if (chan->band_width < RTW89_CHANNEL_WIDTH_80) _dpk_bypass_rxcfir(rtwdev, path, true); else _dpk_lbk_rxiqk(rtwdev, phy, path); @@ -2548,11 +2548,12 @@ static bool _dpk_reload_check(struct rtw89_dev *rtwdev, enum rtw89_phy_idx phy, enum rtw89_rf_path path) { struct rtw89_dpk_info *dpk = &rtwdev->dpk; + const struct rtw89_chan *chan = rtw89_chan_get(rtwdev, RTW89_SUB_ENTITY_0); bool is_reload = false; u8 idx, cur_band, cur_ch; - cur_band = rtwdev->hal.current_band_type; - cur_ch = rtwdev->hal.current_channel; + cur_band = chan->band_type; + cur_ch = chan->channel; for (idx = 0; idx < RTW89_DPK_BKUP_NUM; idx++) { if (cur_band != dpk->bp[path][idx].band || @@ -2681,12 +2682,13 @@ static void _dpk_cal_select(struct rtw89_dev *rtwdev, bool force, static bool _dpk_bypass_check(struct rtw89_dev *rtwdev, enum rtw89_phy_idx phy) { struct rtw89_fem_info *fem = &rtwdev->fem; + const struct rtw89_chan *chan = rtw89_chan_get(rtwdev, RTW89_SUB_ENTITY_0); - if (fem->epa_2g && rtwdev->hal.current_band_type == RTW89_BAND_2G) { + if (fem->epa_2g && chan->band_type == RTW89_BAND_2G) { rtw89_debug(rtwdev, RTW89_DBG_RFK, "[DPK] Skip DPK due to 2G_ext_PA exist!!\n"); return true; - } else if (fem->epa_5g && rtwdev->hal.current_band_type == RTW89_BAND_5G) { + } else if (fem->epa_5g && chan->band_type == RTW89_BAND_5G) { rtw89_debug(rtwdev, RTW89_DBG_RFK, "[DPK] Skip DPK due to 5G_ext_PA exist!!\n"); return true; @@ -2842,7 +2844,8 @@ static void _dpk_track(struct rtw89_dev *rtwdev) static void _tssi_rf_setting(struct rtw89_dev *rtwdev, enum rtw89_phy_idx phy, enum rtw89_rf_path path) { - enum rtw89_band band = rtwdev->hal.current_band_type; + const struct rtw89_chan *chan = rtw89_chan_get(rtwdev, RTW89_SUB_ENTITY_0); + enum rtw89_band band = chan->band_type; if (band == RTW89_BAND_2G) rtw89_write_rf(rtwdev, path, RR_TXPOW, RR_TXPOW_TXG, 0x1); @@ -2852,7 +2855,8 @@ static void _tssi_rf_setting(struct rtw89_dev *rtwdev, enum rtw89_phy_idx phy, static void _tssi_set_sys(struct rtw89_dev *rtwdev, enum rtw89_phy_idx phy) { - enum rtw89_band band = rtwdev->hal.current_band_type; + const struct rtw89_chan *chan = rtw89_chan_get(rtwdev, RTW89_SUB_ENTITY_0); + enum rtw89_band band = chan->band_type; rtw89_rfk_parser(rtwdev, &rtw8852a_tssi_sys_defs_tbl); rtw89_rfk_parser_by_cond(rtwdev, band == RTW89_BAND_2G, @@ -2863,7 +2867,8 @@ static void _tssi_set_sys(struct rtw89_dev *rtwdev, enum rtw89_phy_idx phy) static void _tssi_ini_txpwr_ctrl_bb(struct rtw89_dev *rtwdev, enum rtw89_phy_idx phy, enum rtw89_rf_path path) { - enum rtw89_band band = rtwdev->hal.current_band_type; + const struct rtw89_chan *chan = rtw89_chan_get(rtwdev, RTW89_SUB_ENTITY_0); + enum rtw89_band band = chan->band_type; rtw89_rfk_parser_by_cond(rtwdev, path == RF_PATH_A, &rtw8852a_tssi_txpwr_ctrl_bb_defs_a_tbl, @@ -2905,8 +2910,9 @@ static void _tssi_set_tmeter_tbl(struct rtw89_dev *rtwdev, enum rtw89_phy_idx ph __val; \ }) struct rtw89_tssi_info *tssi_info = &rtwdev->tssi; - u8 ch = rtwdev->hal.current_channel; - u8 subband = rtwdev->hal.current_subband; + const struct rtw89_chan *chan = rtw89_chan_get(rtwdev, RTW89_SUB_ENTITY_0); + u8 ch = chan->channel; + u8 subband = chan->subband_type; const s8 *thm_up_a = NULL; const s8 *thm_down_a = NULL; const s8 *thm_up_b = NULL; @@ -3099,7 +3105,8 @@ static void _tssi_set_txagc_offset_mv_avg(struct rtw89_dev *rtwdev, static void _tssi_pak(struct rtw89_dev *rtwdev, enum rtw89_phy_idx phy, enum rtw89_rf_path path) { - u8 subband = rtwdev->hal.current_subband; + const struct rtw89_chan *chan = rtw89_chan_get(rtwdev, RTW89_SUB_ENTITY_0); + u8 subband = chan->subband_type; switch (subband) { default: @@ -3275,7 +3282,8 @@ static s8 _tssi_get_ofdm_de(struct rtw89_dev *rtwdev, enum rtw89_phy_idx phy, enum rtw89_rf_path path) { struct rtw89_tssi_info *tssi_info = &rtwdev->tssi; - u8 ch = rtwdev->hal.current_channel; + const struct rtw89_chan *chan = rtw89_chan_get(rtwdev, RTW89_SUB_ENTITY_0); + u8 ch = chan->channel; u32 gidx, gidx_1st, gidx_2nd; s8 de_1st = 0; s8 de_2nd = 0; @@ -3312,7 +3320,8 @@ static s8 _tssi_get_ofdm_trim_de(struct rtw89_dev *rtwdev, enum rtw89_rf_path path) { struct rtw89_tssi_info *tssi_info = &rtwdev->tssi; - u8 ch = rtwdev->hal.current_channel; + const struct rtw89_chan *chan = rtw89_chan_get(rtwdev, RTW89_SUB_ENTITY_0); + u8 ch = chan->channel; u32 tgidx, tgidx_1st, tgidx_2nd; s8 tde_1st = 0; s8 tde_2nd = 0; @@ -3350,6 +3359,7 @@ static void _tssi_set_efuse_to_de(struct rtw89_dev *rtwdev, { #define __DE_MASK 0x003ff000 struct rtw89_tssi_info *tssi_info = &rtwdev->tssi; + const struct rtw89_chan *chan = rtw89_chan_get(rtwdev, RTW89_SUB_ENTITY_0); static const u32 r_cck_long[RF_PATH_NUM_8852A] = {0x5858, 0x7858}; static const u32 r_cck_short[RF_PATH_NUM_8852A] = {0x5860, 0x7860}; static const u32 r_mcs_20m[RF_PATH_NUM_8852A] = {0x5838, 0x7838}; @@ -3358,7 +3368,7 @@ static void _tssi_set_efuse_to_de(struct rtw89_dev *rtwdev, static const u32 r_mcs_80m_80m[RF_PATH_NUM_8852A] = {0x5850, 0x7850}; static const u32 r_mcs_5m[RF_PATH_NUM_8852A] = {0x5828, 0x7828}; static const u32 r_mcs_10m[RF_PATH_NUM_8852A] = {0x5830, 0x7830}; - u8 ch = rtwdev->hal.current_channel; + u8 ch = chan->channel; u8 i, gidx; s8 ofdm_de; s8 trim_de; @@ -3478,9 +3488,11 @@ static void _tssi_track(struct rtw89_dev *rtwdev) static void _tssi_high_power(struct rtw89_dev *rtwdev, enum rtw89_phy_idx phy) { struct rtw89_tssi_info *tssi_info = &rtwdev->tssi; - u8 ch = rtwdev->hal.current_channel, ch_tmp; - u8 bw = rtwdev->hal.current_band_width; - u8 subband = rtwdev->hal.current_subband; + const struct rtw89_chan *chan = rtw89_chan_get(rtwdev, RTW89_SUB_ENTITY_0); + u8 ch = chan->channel, ch_tmp; + u8 bw = chan->band_width; + u8 band = chan->band_type; + u8 subband = chan->subband_type; s8 power; s32 xdbm; @@ -3491,7 +3503,7 @@ static void _tssi_high_power(struct rtw89_dev *rtwdev, enum rtw89_phy_idx phy) else ch_tmp = ch; - power = rtw89_phy_read_txpwr_limit(rtwdev, bw, RTW89_1TX, + power = rtw89_phy_read_txpwr_limit(rtwdev, band, bw, RTW89_1TX, RTW89_RS_MCS, RTW89_NONBF, ch_tmp); xdbm = power * 100 / 4; @@ -3523,9 +3535,11 @@ static void _tssi_hw_tx(struct rtw89_dev *rtwdev, enum rtw89_phy_idx phy, static void _tssi_pre_tx(struct rtw89_dev *rtwdev, enum rtw89_phy_idx phy) { struct rtw89_tssi_info *tssi_info = &rtwdev->tssi; + const struct rtw89_chan *chan = rtw89_chan_get(rtwdev, RTW89_SUB_ENTITY_0); const struct rtw89_chip_info *mac_reg = rtwdev->chip; - u8 ch = rtwdev->hal.current_channel, ch_tmp; - u8 bw = rtwdev->hal.current_band_width; + u8 ch = chan->channel, ch_tmp; + u8 bw = chan->band_width; + u8 band = chan->band_type; u32 tx_en; u8 phy_map = rtw89_btc_phymap(rtwdev, phy, 0); s8 power; @@ -3539,8 +3553,9 @@ static void _tssi_pre_tx(struct rtw89_dev *rtwdev, enum rtw89_phy_idx phy) else ch_tmp = ch; - power = rtw89_phy_read_txpwr_limit(rtwdev, RTW89_CHANNEL_WIDTH_20, RTW89_1TX, - RTW89_RS_OFDM, RTW89_NONBF, ch_tmp); + power = rtw89_phy_read_txpwr_limit(rtwdev, band, RTW89_CHANNEL_WIDTH_20, + RTW89_1TX, RTW89_RS_OFDM, + RTW89_NONBF, ch_tmp); xdbm = (power * 100) >> mac_reg->txpwr_factor_mac; diff --git a/drivers/net/wireless/realtek/rtw89/rtw8852ae.c b/drivers/net/wireless/realtek/rtw89/rtw8852ae.c index 190c4aefb02e..0cd8c0c44d19 100644 --- a/drivers/net/wireless/realtek/rtw89/rtw8852ae.c +++ b/drivers/net/wireless/realtek/rtw89/rtw8852ae.c @@ -33,14 +33,15 @@ static const struct rtw89_pci_info rtw8852a_pci_info = { .max_tag_num_mask = B_AX_MAX_TAG_NUM, .rxbd_rwptr_clr_reg = R_AX_RXBD_RWPTR_CLR, .txbd_rwptr_clr2_reg = R_AX_TXBD_RWPTR_CLR2, - .dma_stop1_reg = R_AX_PCIE_DMA_STOP1, - .dma_stop2_reg = R_AX_PCIE_DMA_STOP2, - .dma_busy1_reg = R_AX_PCIE_DMA_BUSY1, + .dma_stop1 = {R_AX_PCIE_DMA_STOP1, B_AX_TX_STOP1_MASK}, + .dma_stop2 = {R_AX_PCIE_DMA_STOP2, B_AX_TX_STOP2_ALL}, + .dma_busy1 = {R_AX_PCIE_DMA_BUSY1, DMA_BUSY1_CHECK}, .dma_busy2_reg = R_AX_PCIE_DMA_BUSY2, .dma_busy3_reg = R_AX_PCIE_DMA_BUSY1, .rpwm_addr = R_AX_PCIE_HRPWM, .cpwm_addr = R_AX_CPWM, + .tx_dma_ch_mask = 0, .bd_idx_addr_low_power = NULL, .dma_addr_set = &rtw89_pci_ch_dma_addr_set, diff --git a/drivers/net/wireless/realtek/rtw89/rtw8852b.c b/drivers/net/wireless/realtek/rtw89/rtw8852b.c new file mode 100644 index 000000000000..9f9908418ee4 --- /dev/null +++ b/drivers/net/wireless/realtek/rtw89/rtw8852b.c @@ -0,0 +1,94 @@ +// SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause +/* Copyright(c) 2019-2022 Realtek Corporation + */ + +#include "core.h" +#include "mac.h" +#include "reg.h" + +static const struct rtw89_dle_mem rtw8852b_dle_mem_pcie[] = { + [RTW89_QTA_SCC] = {RTW89_QTA_SCC, &rtw89_mac_size.wde_size6, + &rtw89_mac_size.ple_size6, &rtw89_mac_size.wde_qt6, + &rtw89_mac_size.wde_qt6, &rtw89_mac_size.ple_qt18, + &rtw89_mac_size.ple_qt58}, + [RTW89_QTA_DLFW] = {RTW89_QTA_DLFW, &rtw89_mac_size.wde_size9, + &rtw89_mac_size.ple_size8, &rtw89_mac_size.wde_qt4, + &rtw89_mac_size.wde_qt4, &rtw89_mac_size.ple_qt13, + &rtw89_mac_size.ple_qt13}, + [RTW89_QTA_INVALID] = {RTW89_QTA_INVALID, NULL, NULL, NULL, NULL, NULL, + NULL}, +}; + +static int rtw8852b_mac_enable_bb_rf(struct rtw89_dev *rtwdev) +{ + int ret; + + rtw89_write8_set(rtwdev, R_AX_SYS_FUNC_EN, + B_AX_FEN_BBRSTB | B_AX_FEN_BB_GLB_RSTN); + rtw89_write32_mask(rtwdev, R_AX_SPS_DIG_ON_CTRL0, B_AX_REG_ZCDC_H_MASK, 0x1); + rtw89_write32_set(rtwdev, R_AX_WLRF_CTRL, B_AX_AFC_AFEDIG); + rtw89_write32_clr(rtwdev, R_AX_WLRF_CTRL, B_AX_AFC_AFEDIG); + rtw89_write32_set(rtwdev, R_AX_WLRF_CTRL, B_AX_AFC_AFEDIG); + + ret = rtw89_mac_write_xtal_si(rtwdev, XTAL_SI_WL_RFC_S0, 0xC7, + FULL_BIT_MASK); + if (ret) + return ret; + + ret = rtw89_mac_write_xtal_si(rtwdev, XTAL_SI_WL_RFC_S1, 0xC7, + FULL_BIT_MASK); + if (ret) + return ret; + + rtw89_write8(rtwdev, R_AX_PHYREG_SET, PHYREG_SET_XYN_CYCLE); + + return 0; +} + +static int rtw8852b_mac_disable_bb_rf(struct rtw89_dev *rtwdev) +{ + u8 wl_rfc_s0; + u8 wl_rfc_s1; + int ret; + + rtw89_write8_clr(rtwdev, R_AX_SYS_FUNC_EN, + B_AX_FEN_BBRSTB | B_AX_FEN_BB_GLB_RSTN); + + ret = rtw89_mac_read_xtal_si(rtwdev, XTAL_SI_WL_RFC_S0, &wl_rfc_s0); + if (ret) + return ret; + wl_rfc_s0 &= ~XTAL_SI_RF00S_EN; + ret = rtw89_mac_write_xtal_si(rtwdev, XTAL_SI_WL_RFC_S0, wl_rfc_s0, + FULL_BIT_MASK); + if (ret) + return ret; + + ret = rtw89_mac_read_xtal_si(rtwdev, XTAL_SI_WL_RFC_S1, &wl_rfc_s1); + if (ret) + return ret; + wl_rfc_s1 &= ~XTAL_SI_RF10S_EN; + ret = rtw89_mac_write_xtal_si(rtwdev, XTAL_SI_WL_RFC_S1, wl_rfc_s1, + FULL_BIT_MASK); + return ret; +} + +static const struct rtw89_chip_ops rtw8852b_chip_ops = { + .enable_bb_rf = rtw8852b_mac_enable_bb_rf, + .disable_bb_rf = rtw8852b_mac_disable_bb_rf, +}; + +const struct rtw89_chip_info rtw8852b_chip_info = { + .chip_id = RTL8852B, + .fifo_size = 196608, + .dle_scc_rsvd_size = 98304, + .dle_mem = rtw8852b_dle_mem_pcie, + .dma_ch_mask = BIT(RTW89_DMA_ACH4) | BIT(RTW89_DMA_ACH5) | + BIT(RTW89_DMA_ACH6) | BIT(RTW89_DMA_ACH7) | + BIT(RTW89_DMA_B1MG) | BIT(RTW89_DMA_B1HI), +}; +EXPORT_SYMBOL(rtw8852b_chip_info); + +MODULE_FIRMWARE("rtw89/rtw8852b_fw.bin"); +MODULE_AUTHOR("Realtek Corporation"); +MODULE_DESCRIPTION("Realtek 802.11ax wireless 8852B driver"); +MODULE_LICENSE("Dual BSD/GPL"); diff --git a/drivers/net/wireless/realtek/rtw89/rtw8852be.c b/drivers/net/wireless/realtek/rtw89/rtw8852be.c new file mode 100644 index 000000000000..7bf95c38d3eb --- /dev/null +++ b/drivers/net/wireless/realtek/rtw89/rtw8852be.c @@ -0,0 +1,25 @@ +// SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause +/* Copyright(c) 2020-2022 Realtek Corporation + */ + +#include <linux/module.h> +#include <linux/pci.h> + +#include "pci.h" +#include "reg.h" + +static const struct rtw89_pci_info rtw8852b_pci_info = { + .dma_stop1 = {R_AX_PCIE_DMA_STOP1, B_AX_TX_STOP1_MASK_V1}, + .dma_stop2 = {0}, + .dma_busy1 = {R_AX_PCIE_DMA_BUSY1, DMA_BUSY1_CHECK_V1}, + .dma_busy2_reg = 0, + .dma_busy3_reg = R_AX_PCIE_DMA_BUSY1, + + .tx_dma_ch_mask = BIT(RTW89_TXCH_ACH4) | BIT(RTW89_TXCH_ACH5) | + BIT(RTW89_TXCH_ACH6) | BIT(RTW89_TXCH_ACH7) | + BIT(RTW89_TXCH_CH10) | BIT(RTW89_TXCH_CH11), +}; + +MODULE_AUTHOR("Realtek Corporation"); +MODULE_DESCRIPTION("Realtek 802.11ax wireless 8852BE driver"); +MODULE_LICENSE("Dual BSD/GPL"); diff --git a/drivers/net/wireless/realtek/rtw89/rtw8852c.c b/drivers/net/wireless/realtek/rtw89/rtw8852c.c index b697aef2faf2..67653b3e1a35 100644 --- a/drivers/net/wireless/realtek/rtw89/rtw8852c.c +++ b/drivers/net/wireless/realtek/rtw89/rtw8852c.c @@ -109,6 +109,7 @@ static const struct rtw89_imr_info rtw8852c_imr_info = { .cpu_disp_imr_set = B_AX_CPU_DISP_IMR_SET_V1, .other_disp_imr_clr = B_AX_OTHER_DISP_IMR_CLR_V1, .other_disp_imr_set = B_AX_OTHER_DISP_IMR_SET_V1, + .bbrpt_com_err_imr_reg = R_AX_BBRPT_COM_ERR_IMR, .bbrpt_chinfo_err_imr_reg = R_AX_BBRPT_CHINFO_ERR_IMR, .bbrpt_err_imr_set = R_AX_BBRPT_CHINFO_IMR_SET_V1, .bbrpt_dfs_err_imr_reg = R_AX_BBRPT_DFS_ERR_IMR, @@ -131,7 +132,34 @@ static const struct rtw89_imr_info rtw8852c_imr_info = { .tmac_imr_set = B_AX_TMAC_IMR_SET_V1, }; +static const struct rtw89_rrsr_cfgs rtw8852c_rrsr_cfgs = { + .ref_rate = {R_AX_TRXPTCL_RRSR_CTL_0, B_AX_WMAC_RESP_REF_RATE_SEL, 0}, + .rsc = {R_AX_PTCL_RRSR1, B_AX_RSC_MASK, 2}, +}; + +static const struct rtw89_dig_regs rtw8852c_dig_regs = { + .seg0_pd_reg = R_SEG0R_PD, + .pd_lower_bound_mask = B_SEG0R_PD_LOWER_BOUND_MSK, + .pd_spatial_reuse_en = B_SEG0R_PD_SPATIAL_REUSE_EN_MSK, + .p0_lna_init = {R_PATH0_LNA_INIT_V1, B_PATH0_LNA_INIT_IDX_MSK}, + .p1_lna_init = {R_PATH1_LNA_INIT_V1, B_PATH1_LNA_INIT_IDX_MSK}, + .p0_tia_init = {R_PATH0_TIA_INIT_V1, B_PATH0_TIA_INIT_IDX_MSK_V1}, + .p1_tia_init = {R_PATH1_TIA_INIT_V1, B_PATH1_TIA_INIT_IDX_MSK_V1}, + .p0_rxb_init = {R_PATH0_RXB_INIT_V1, B_PATH0_RXB_INIT_IDX_MSK_V1}, + .p1_rxb_init = {R_PATH1_RXB_INIT_V1, B_PATH1_RXB_INIT_IDX_MSK_V1}, + .p0_p20_pagcugc_en = {R_PATH0_P20_FOLLOW_BY_PAGCUGC_V1, + B_PATH0_P20_FOLLOW_BY_PAGCUGC_EN_MSK}, + .p0_s20_pagcugc_en = {R_PATH0_S20_FOLLOW_BY_PAGCUGC_V1, + B_PATH0_S20_FOLLOW_BY_PAGCUGC_EN_MSK}, + .p1_p20_pagcugc_en = {R_PATH1_P20_FOLLOW_BY_PAGCUGC_V1, + B_PATH1_P20_FOLLOW_BY_PAGCUGC_EN_MSK}, + .p1_s20_pagcugc_en = {R_PATH1_S20_FOLLOW_BY_PAGCUGC_V1, + B_PATH1_S20_FOLLOW_BY_PAGCUGC_EN_MSK}, +}; + static void rtw8852c_ctrl_btg(struct rtw89_dev *rtwdev, bool btg); +static void rtw8852c_ctrl_tx_path_tmac(struct rtw89_dev *rtwdev, u8 tx_path, + enum rtw89_mac_idx mac_idx); static int rtw8852c_pwr_on_func(struct rtw89_dev *rtwdev) { @@ -567,7 +595,7 @@ static void rtw8852c_power_trim(struct rtw89_dev *rtwdev) } static void rtw8852c_set_channel_mac(struct rtw89_dev *rtwdev, - struct rtw89_channel_params *param, + const struct rtw89_chan *chan, u8 mac_idx) { u32 rf_mod = rtw89_mac_reg_by_idx(R_AX_WMAC_RFMOD, mac_idx); @@ -578,24 +606,24 @@ static void rtw8852c_set_channel_mac(struct rtw89_dev *rtwdev, u8 rf_mod_val = 0, chk_rate_mask = 0; u32 txsc; - switch (param->bandwidth) { + switch (chan->band_width) { case RTW89_CHANNEL_WIDTH_160: - txsc80 = rtw89_phy_get_txsc(rtwdev, param, + txsc80 = rtw89_phy_get_txsc(rtwdev, chan, RTW89_CHANNEL_WIDTH_80); fallthrough; case RTW89_CHANNEL_WIDTH_80: - txsc40 = rtw89_phy_get_txsc(rtwdev, param, + txsc40 = rtw89_phy_get_txsc(rtwdev, chan, RTW89_CHANNEL_WIDTH_40); fallthrough; case RTW89_CHANNEL_WIDTH_40: - txsc20 = rtw89_phy_get_txsc(rtwdev, param, + txsc20 = rtw89_phy_get_txsc(rtwdev, chan, RTW89_CHANNEL_WIDTH_20); break; default: break; } - switch (param->bandwidth) { + switch (chan->band_width) { case RTW89_CHANNEL_WIDTH_160: rf_mod_val = AX_WMAC_RFMOD_160M; txsc = FIELD_PREP(B_AX_TXSC_20M_MASK, txsc20) | @@ -620,7 +648,7 @@ static void rtw8852c_set_channel_mac(struct rtw89_dev *rtwdev, rtw89_write8_mask(rtwdev, rf_mod, B_AX_WMAC_RFMOD_MASK, rf_mod_val); rtw89_write32(rtwdev, sub_carr, txsc); - switch (param->band_type) { + switch (chan->band_type) { case RTW89_BAND_2G: chk_rate_mask = B_AX_BAND_MODE; break; @@ -629,7 +657,7 @@ static void rtw8852c_set_channel_mac(struct rtw89_dev *rtwdev, chk_rate_mask = B_AX_CHECK_CCK_EN | B_AX_RTS_LIMIT_IN_OFDM6; break; default: - rtw89_warn(rtwdev, "Invalid band_type:%d\n", param->band_type); + rtw89_warn(rtwdev, "Invalid band_type:%d\n", chan->band_type); return; } rtw89_write8_clr(rtwdev, chk_rate, B_AX_BAND_MODE | B_AX_CHECK_CCK_EN | @@ -920,7 +948,7 @@ static void rtw8852c_decode_chan_idx(struct rtw89_dev *rtwdev, u8 chan_idx, } static void rtw8852c_set_gain_offset(struct rtw89_dev *rtwdev, - const struct rtw89_channel_params *param, + const struct rtw89_chan *chan, enum rtw89_phy_idx phy_idx, enum rtw89_rf_path path) { @@ -939,7 +967,7 @@ static void rtw8852c_set_gain_offset(struct rtw89_dev *rtwdev, if (rtwdev->dbcc_en && path == RF_PATH_B) phy_idx = RTW89_PHY_1; - if (param->band_type == RTW89_BAND_2G) { + if (chan->band_type == RTW89_BAND_2G) { offset_q0 = efuse_gain->offset[path][RTW89_GAIN_OFFSET_2G_CCK]; offset_base_q4 = efuse_gain->offset_base[phy_idx]; @@ -948,7 +976,7 @@ static void rtw8852c_set_gain_offset(struct rtw89_dev *rtwdev, rtw89_phy_write32_mask(rtwdev, R_RPL_OFST, B_RPL_OFST_MASK, tmp & 0x7f); } - switch (param->subband_type) { + switch (chan->subband_type) { default: case RTW89_CH_2G: gain_band = RTW89_GAIN_OFFSET_2G_OFDM; @@ -977,14 +1005,14 @@ static void rtw8852c_set_gain_offset(struct rtw89_dev *rtwdev, } static void rtw8852c_ctrl_ch(struct rtw89_dev *rtwdev, - const struct rtw89_channel_params *param, + const struct rtw89_chan *chan, enum rtw89_phy_idx phy_idx) { u8 sco; - u16 central_freq = param->center_freq; - u8 central_ch = param->center_chan; - u8 band = param->band_type; - u8 subband = param->subband_type; + u16 central_freq = chan->freq; + u8 central_ch = chan->channel; + u8 band = chan->band_type; + u8 subband = chan->subband_type; bool is_2g = band == RTW89_BAND_2G; u8 chan_idx; @@ -996,7 +1024,7 @@ static void rtw8852c_ctrl_ch(struct rtw89_dev *rtwdev, if (phy_idx == RTW89_PHY_0) { /* Path A */ rtw8852c_set_gain_error(rtwdev, subband, RF_PATH_A); - rtw8852c_set_gain_offset(rtwdev, param, phy_idx, RF_PATH_A); + rtw8852c_set_gain_offset(rtwdev, chan, phy_idx, RF_PATH_A); if (is_2g) rtw89_phy_write32_idx(rtwdev, R_PATH0_BAND_SEL_V1, @@ -1009,7 +1037,7 @@ static void rtw8852c_ctrl_ch(struct rtw89_dev *rtwdev, /* Path B */ if (!rtwdev->dbcc_en) { rtw8852c_set_gain_error(rtwdev, subband, RF_PATH_B); - rtw8852c_set_gain_offset(rtwdev, param, phy_idx, RF_PATH_B); + rtw8852c_set_gain_offset(rtwdev, chan, phy_idx, RF_PATH_B); if (is_2g) rtw89_phy_write32_idx(rtwdev, @@ -1038,7 +1066,7 @@ static void rtw8852c_ctrl_ch(struct rtw89_dev *rtwdev, } else { /* Path B */ rtw8852c_set_gain_error(rtwdev, subband, RF_PATH_B); - rtw8852c_set_gain_offset(rtwdev, param, phy_idx, RF_PATH_B); + rtw8852c_set_gain_offset(rtwdev, chan, phy_idx, RF_PATH_B); if (is_2g) rtw89_phy_write32_idx(rtwdev, R_PATH1_BAND_SEL_V1, @@ -1095,7 +1123,7 @@ static void rtw8852c_ctrl_ch(struct rtw89_dev *rtwdev, } } - chan_idx = rtw8852c_encode_chan_idx(rtwdev, param->primary_chan, band); + chan_idx = rtw8852c_encode_chan_idx(rtwdev, chan->primary_channel, band); rtw89_phy_write32_idx(rtwdev, R_MAC_PIN_SEL, B_CH_IDX_SEG0, chan_idx, phy_idx); } @@ -1246,12 +1274,12 @@ rtw8852c_ctrl_bw(struct rtw89_dev *rtwdev, u8 pri_ch, u8 bw, } static u32 rtw8852c_spur_freq(struct rtw89_dev *rtwdev, - struct rtw89_channel_params *param) + const struct rtw89_chan *chan) { - u8 center_chan = param->center_chan; - u8 bw = param->bandwidth; + u8 center_chan = chan->channel; + u8 bw = chan->band_width; - switch (param->band_type) { + switch (chan->band_type) { case RTW89_BAND_2G: if (bw == RTW89_CHANNEL_WIDTH_20) { if (center_chan >= 5 && center_chan <= 8) @@ -1285,19 +1313,19 @@ static u32 rtw8852c_spur_freq(struct rtw89_dev *rtwdev, #define MAX_TONE_NUM 2048 static void rtw8852c_set_csi_tone_idx(struct rtw89_dev *rtwdev, - struct rtw89_channel_params *param, + const struct rtw89_chan *chan, enum rtw89_phy_idx phy_idx) { u32 spur_freq; s32 freq_diff, csi_idx, csi_tone_idx; - spur_freq = rtw8852c_spur_freq(rtwdev, param); + spur_freq = rtw8852c_spur_freq(rtwdev, chan); if (spur_freq == 0) { rtw89_phy_write32_idx(rtwdev, R_SEG0CSI_EN, B_SEG0CSI_EN, 0, phy_idx); return; } - freq_diff = (spur_freq - param->center_freq) * 1000000; + freq_diff = (spur_freq - chan->freq) * 1000000; csi_idx = s32_div_u32_round_closest(freq_diff, CARRIER_SPACING_78_125); s32_div_u32_round_down(csi_idx, MAX_TONE_NUM, &csi_tone_idx); @@ -1325,7 +1353,7 @@ static const struct rtw89_nbi_reg_def rtw8852c_nbi_reg_def[] = { }; static void rtw8852c_set_nbi_tone_idx(struct rtw89_dev *rtwdev, - struct rtw89_channel_params *param, + const struct rtw89_chan *chan, enum rtw89_rf_path path) { const struct rtw89_nbi_reg_def *nbi = &rtw8852c_nbi_reg_def[path]; @@ -1335,34 +1363,37 @@ static void rtw8852c_set_nbi_tone_idx(struct rtw89_dev *rtwdev, s32 nbi_frac_idx, nbi_frac_tone_idx; bool notch2_chk = false; - spur_freq = rtw8852c_spur_freq(rtwdev, param); + spur_freq = rtw8852c_spur_freq(rtwdev, chan); if (spur_freq == 0) { rtw89_phy_write32_mask(rtwdev, nbi->notch1_en.addr, nbi->notch1_en.mask, 0); rtw89_phy_write32_mask(rtwdev, nbi->notch1_en.addr, nbi->notch1_en.mask, 0); return; } - fc = param->center_freq; - if (param->bandwidth == RTW89_CHANNEL_WIDTH_160) { + fc = chan->freq; + if (chan->band_width == RTW89_CHANNEL_WIDTH_160) { fc = (spur_freq > fc) ? fc + 40 : fc - 40; - if ((fc > spur_freq && param->center_chan < param->primary_chan) || - (fc < spur_freq && param->center_chan > param->primary_chan)) + if ((fc > spur_freq && + chan->channel < chan->primary_channel) || + (fc < spur_freq && + chan->channel > chan->primary_channel)) notch2_chk = true; } freq_diff = (spur_freq - fc) * 1000000; nbi_idx = s32_div_u32_round_down(freq_diff, CARRIER_SPACING_312_5, &nbi_frac_idx); - if (param->bandwidth == RTW89_CHANNEL_WIDTH_20) { + if (chan->band_width == RTW89_CHANNEL_WIDTH_20) { s32_div_u32_round_down(nbi_idx + 32, 64, &nbi_tone_idx); } else { - u16 tone_para = (param->bandwidth == RTW89_CHANNEL_WIDTH_40) ? 128 : 256; + u16 tone_para = (chan->band_width == RTW89_CHANNEL_WIDTH_40) ? + 128 : 256; s32_div_u32_round_down(nbi_idx, tone_para, &nbi_tone_idx); } nbi_frac_tone_idx = s32_div_u32_round_closest(nbi_frac_idx, CARRIER_SPACING_78_125); - if (param->bandwidth == RTW89_CHANNEL_WIDTH_160 && notch2_chk) { + if (chan->band_width == RTW89_CHANNEL_WIDTH_160 && notch2_chk) { rtw89_phy_write32_mask(rtwdev, nbi->notch2_idx.addr, nbi->notch2_idx.mask, nbi_tone_idx); rtw89_phy_write32_mask(rtwdev, nbi->notch2_frac_idx.addr, @@ -1404,42 +1435,42 @@ static void rtw8852c_spur_notch(struct rtw89_dev *rtwdev, u32 val, } static void rtw8852c_spur_elimination(struct rtw89_dev *rtwdev, - struct rtw89_channel_params *param, + const struct rtw89_chan *chan, u8 pri_ch_idx, enum rtw89_phy_idx phy_idx) { - rtw8852c_set_csi_tone_idx(rtwdev, param, phy_idx); + rtw8852c_set_csi_tone_idx(rtwdev, chan, phy_idx); if (phy_idx == RTW89_PHY_0) { - if (param->bandwidth == RTW89_CHANNEL_WIDTH_160 && + if (chan->band_width == RTW89_CHANNEL_WIDTH_160 && (pri_ch_idx == RTW89_SC_20_LOWER || pri_ch_idx == RTW89_SC_20_UP3X)) { rtw8852c_spur_notch(rtwdev, 0xe7f, RTW89_PHY_0); if (!rtwdev->dbcc_en) rtw8852c_spur_notch(rtwdev, 0xe7f, RTW89_PHY_1); - } else if (param->bandwidth == RTW89_CHANNEL_WIDTH_160 && + } else if (chan->band_width == RTW89_CHANNEL_WIDTH_160 && (pri_ch_idx == RTW89_SC_20_UPPER || pri_ch_idx == RTW89_SC_20_LOW3X)) { rtw8852c_spur_notch(rtwdev, 0x280, RTW89_PHY_0); if (!rtwdev->dbcc_en) rtw8852c_spur_notch(rtwdev, 0x280, RTW89_PHY_1); } else { - rtw8852c_set_nbi_tone_idx(rtwdev, param, RF_PATH_A); + rtw8852c_set_nbi_tone_idx(rtwdev, chan, RF_PATH_A); if (!rtwdev->dbcc_en) - rtw8852c_set_nbi_tone_idx(rtwdev, param, + rtw8852c_set_nbi_tone_idx(rtwdev, chan, RF_PATH_B); } } else { - if (param->bandwidth == RTW89_CHANNEL_WIDTH_160 && + if (chan->band_width == RTW89_CHANNEL_WIDTH_160 && (pri_ch_idx == RTW89_SC_20_LOWER || pri_ch_idx == RTW89_SC_20_UP3X)) { rtw8852c_spur_notch(rtwdev, 0xe7f, RTW89_PHY_1); - } else if (param->bandwidth == RTW89_CHANNEL_WIDTH_160 && + } else if (chan->band_width == RTW89_CHANNEL_WIDTH_160 && (pri_ch_idx == RTW89_SC_20_UPPER || pri_ch_idx == RTW89_SC_20_LOW3X)) { rtw8852c_spur_notch(rtwdev, 0x280, RTW89_PHY_1); } else { - rtw8852c_set_nbi_tone_idx(rtwdev, param, RF_PATH_B); + rtw8852c_set_nbi_tone_idx(rtwdev, chan, RF_PATH_B); } } @@ -1450,14 +1481,14 @@ static void rtw8852c_spur_elimination(struct rtw89_dev *rtwdev, } static void rtw8852c_5m_mask(struct rtw89_dev *rtwdev, - struct rtw89_channel_params *param, + const struct rtw89_chan *chan, enum rtw89_phy_idx phy_idx) { - u8 pri_ch = param->primary_chan; + u8 pri_ch = chan->primary_channel; bool mask_5m_low; bool mask_5m_en; - switch (param->bandwidth) { + switch (chan->band_width) { case RTW89_CHANNEL_WIDTH_40: mask_5m_en = true; mask_5m_low = pri_ch == 2; @@ -1526,11 +1557,9 @@ static void rtw8852c_bb_reset_all(struct rtw89_dev *rtwdev, phy_idx); } -static void rtw8852c_bb_reset_en(struct rtw89_dev *rtwdev, +static void rtw8852c_bb_reset_en(struct rtw89_dev *rtwdev, enum rtw89_band band, enum rtw89_phy_idx phy_idx, bool en) { - struct rtw89_hal *hal = &rtwdev->hal; - if (en) { rtw89_phy_write32_idx(rtwdev, R_S0_HW_SI_DIS, B_S0_HW_SI_DIS_W_R_TRIG, 0x0, phy_idx); @@ -1538,7 +1567,7 @@ static void rtw8852c_bb_reset_en(struct rtw89_dev *rtwdev, B_S1_HW_SI_DIS_W_R_TRIG, 0x0, phy_idx); rtw89_phy_write32_idx(rtwdev, R_RSTB_ASYNC, B_RSTB_ASYNC_ALL, 1, phy_idx); - if (hal->current_band_type == RTW89_BAND_2G) + if (band == RTW89_BAND_2G) rtw89_phy_write32_mask(rtwdev, R_RXCCA_V1, B_RXCCA_DIS_V1, 0x0); rtw89_phy_write32_mask(rtwdev, R_PD_CTRL, B_PD_HIT_DIS, 0x0); } else { @@ -1690,21 +1719,24 @@ static void rtw8852c_bb_sethw(struct rtw89_dev *rtwdev) } static void rtw8852c_set_channel_bb(struct rtw89_dev *rtwdev, - struct rtw89_channel_params *param, + const struct rtw89_chan *chan, enum rtw89_phy_idx phy_idx) { - bool cck_en = param->band_type == RTW89_BAND_2G; - u8 pri_ch_idx = param->pri_ch_idx; + struct rtw89_hal *hal = &rtwdev->hal; + bool cck_en = chan->band_type == RTW89_BAND_2G; + u8 pri_ch_idx = chan->pri_ch_idx; u32 mask, reg; u32 ru_alloc_msk[2] = {B_P80_AT_HIGH_FREQ_RU_ALLOC_PHY0, B_P80_AT_HIGH_FREQ_RU_ALLOC_PHY1}; + u8 ntx_path; - if (param->band_type == RTW89_BAND_2G) - rtw8852c_ctrl_sco_cck(rtwdev, param->center_chan, - param->primary_chan, param->bandwidth); + if (chan->band_type == RTW89_BAND_2G) + rtw8852c_ctrl_sco_cck(rtwdev, chan->channel, + chan->primary_channel, + chan->band_width); - rtw8852c_ctrl_ch(rtwdev, param, phy_idx); - rtw8852c_ctrl_bw(rtwdev, pri_ch_idx, param->bandwidth, phy_idx); + rtw8852c_ctrl_ch(rtwdev, chan, phy_idx); + rtw8852c_ctrl_bw(rtwdev, pri_ch_idx, chan->band_width, phy_idx); if (cck_en) { rtw89_phy_write32_mask(rtwdev, R_UPD_CLK_ADC, B_ENABLE_CCK, 1); rtw89_phy_write32_mask(rtwdev, R_RXCCA_V1, B_RXCCA_DIS_V1, 0); @@ -1717,17 +1749,17 @@ static void rtw8852c_set_channel_bb(struct rtw89_dev *rtwdev, B_PD_ARBITER_OFF, 0x1, phy_idx); } - rtw8852c_spur_elimination(rtwdev, param, pri_ch_idx, phy_idx); - rtw8852c_ctrl_btg(rtwdev, param->band_type == RTW89_BAND_2G); - rtw8852c_5m_mask(rtwdev, param, phy_idx); + rtw8852c_spur_elimination(rtwdev, chan, pri_ch_idx, phy_idx); + rtw8852c_ctrl_btg(rtwdev, chan->band_type == RTW89_BAND_2G); + rtw8852c_5m_mask(rtwdev, chan, phy_idx); - if (param->bandwidth == RTW89_CHANNEL_WIDTH_160 && + if (chan->band_width == RTW89_CHANNEL_WIDTH_160 && rtwdev->hal.cv != CHIP_CAV) { rtw89_phy_write32_idx(rtwdev, R_P80_AT_HIGH_FREQ, B_P80_AT_HIGH_FREQ, 0x0, phy_idx); reg = rtw89_mac_reg_by_idx(R_P80_AT_HIGH_FREQ_BB_WRP, phy_idx); - if (param->primary_chan > param->center_chan) { + if (chan->primary_channel > chan->channel) { rtw89_phy_write32_mask(rtwdev, R_P80_AT_HIGH_FREQ_RU_ALLOC, ru_alloc_msk[phy_idx], 1); @@ -1742,8 +1774,8 @@ static void rtw8852c_set_channel_bb(struct rtw89_dev *rtwdev, } } - if (param->band_type == RTW89_BAND_6G && - param->bandwidth == RTW89_CHANNEL_WIDTH_160) + if (chan->band_type == RTW89_BAND_6G && + chan->band_width == RTW89_CHANNEL_WIDTH_160) rtw89_phy_write32_idx(rtwdev, R_CDD_EVM_CHK_EN, B_CDD_EVM_CHK_EN, 0, phy_idx); else @@ -1769,15 +1801,29 @@ static void rtw8852c_set_channel_bb(struct rtw89_dev *rtwdev, } } + if (chan->band_type == RTW89_BAND_6G) + rtw89_phy_write32_set(rtwdev, R_MUIC, B_MUIC_EN); + else + rtw89_phy_write32_clr(rtwdev, R_MUIC, B_MUIC_EN); + + if (hal->antenna_tx) + ntx_path = hal->antenna_tx; + else + ntx_path = chan->band_type == RTW89_BAND_6G ? RF_B : RF_AB; + + rtw8852c_ctrl_tx_path_tmac(rtwdev, ntx_path, (enum rtw89_mac_idx)phy_idx); + rtw8852c_bb_reset_all(rtwdev, phy_idx); } static void rtw8852c_set_channel(struct rtw89_dev *rtwdev, - struct rtw89_channel_params *params) + const struct rtw89_chan *chan, + enum rtw89_mac_idx mac_idx, + enum rtw89_phy_idx phy_idx) { - rtw8852c_set_channel_mac(rtwdev, params, RTW89_MAC_0); - rtw8852c_set_channel_bb(rtwdev, params, RTW89_PHY_0); - rtw8852c_set_channel_rf(rtwdev, params, RTW89_PHY_0); + rtw8852c_set_channel_mac(rtwdev, chan, mac_idx); + rtw8852c_set_channel_bb(rtwdev, chan, phy_idx); + rtw8852c_set_channel_rf(rtwdev, chan, phy_idx); } static void rtw8852c_dfs_en(struct rtw89_dev *rtwdev, bool en) @@ -1799,25 +1845,27 @@ static void rtw8852c_adc_en(struct rtw89_dev *rtwdev, bool en) } static void rtw8852c_set_channel_help(struct rtw89_dev *rtwdev, bool enter, - struct rtw89_channel_help_params *p) + struct rtw89_channel_help_params *p, + const struct rtw89_chan *chan, + enum rtw89_mac_idx mac_idx, + enum rtw89_phy_idx phy_idx) { - u8 phy_idx = RTW89_PHY_0; - if (enter) { - rtw89_chip_stop_sch_tx(rtwdev, RTW89_MAC_0, &p->tx_en, RTW89_SCH_TX_SEL_ALL); - rtw89_mac_cfg_ppdu_status(rtwdev, RTW89_MAC_0, false); + rtw89_chip_stop_sch_tx(rtwdev, mac_idx, &p->tx_en, + RTW89_SCH_TX_SEL_ALL); + rtw89_mac_cfg_ppdu_status(rtwdev, mac_idx, false); rtw8852c_dfs_en(rtwdev, false); - rtw8852c_tssi_cont_en_phyidx(rtwdev, false, RTW89_PHY_0); + rtw8852c_tssi_cont_en_phyidx(rtwdev, false, phy_idx); rtw8852c_adc_en(rtwdev, false); fsleep(40); - rtw8852c_bb_reset_en(rtwdev, phy_idx, false); + rtw8852c_bb_reset_en(rtwdev, chan->band_type, phy_idx, false); } else { - rtw89_mac_cfg_ppdu_status(rtwdev, RTW89_MAC_0, true); + rtw89_mac_cfg_ppdu_status(rtwdev, mac_idx, true); rtw8852c_adc_en(rtwdev, true); rtw8852c_dfs_en(rtwdev, true); - rtw8852c_tssi_cont_en_phyidx(rtwdev, true, RTW89_PHY_0); - rtw8852c_bb_reset_en(rtwdev, phy_idx, true); - rtw89_chip_resume_sch_tx(rtwdev, RTW89_MAC_0, p->tx_en); + rtw8852c_tssi_cont_en_phyidx(rtwdev, true, phy_idx); + rtw8852c_bb_reset_en(rtwdev, chan->band_type, phy_idx, true); + rtw89_chip_resume_sch_tx(rtwdev, mac_idx, p->tx_en); } } @@ -1847,9 +1895,10 @@ static void rtw8852c_rfk_channel(struct rtw89_dev *rtwdev) rtw89_fw_h2c_rf_ntfy_mcc(rtwdev); } -static void rtw8852c_rfk_band_changed(struct rtw89_dev *rtwdev) +static void rtw8852c_rfk_band_changed(struct rtw89_dev *rtwdev, + enum rtw89_phy_idx phy_idx) { - rtw8852c_tssi_scan(rtwdev, RTW89_PHY_0); + rtw8852c_tssi_scan(rtwdev, phy_idx); } static void rtw8852c_rfk_scan(struct rtw89_dev *rtwdev, bool start) @@ -1958,9 +2007,11 @@ static void rtw8852c_set_txpwr_ref(struct rtw89_dev *rtwdev, } static void rtw8852c_set_txpwr_byrate(struct rtw89_dev *rtwdev, + const struct rtw89_chan *chan, enum rtw89_phy_idx phy_idx) { - u8 ch = rtwdev->hal.current_channel; + u8 band = chan->band_type; + u8 ch = chan->channel; static const u8 rs[] = { RTW89_RS_CCK, RTW89_RS_OFDM, @@ -1986,7 +2037,8 @@ static void rtw8852c_set_txpwr_byrate(struct rtw89_dev *rtwdev, for (j = 0; j < rtw89_rs_idx_max[rs[i]]; j++) { cur.idx = j; shf = (j % 4) * 8; - tmp = rtw89_phy_read_txpwr_byrate(rtwdev, &cur); + tmp = rtw89_phy_read_txpwr_byrate(rtwdev, band, + &cur); val |= (tmp << shf); if ((j + 1) % 4) @@ -2001,8 +2053,10 @@ static void rtw8852c_set_txpwr_byrate(struct rtw89_dev *rtwdev, } static void rtw8852c_set_txpwr_offset(struct rtw89_dev *rtwdev, + const struct rtw89_chan *chan, enum rtw89_phy_idx phy_idx) { + u8 band = chan->band_type; struct rtw89_rate_desc desc = { .nss = RTW89_NSS_1, .rs = RTW89_RS_OFFSET, @@ -2013,7 +2067,7 @@ static void rtw8852c_set_txpwr_offset(struct rtw89_dev *rtwdev, rtw89_debug(rtwdev, RTW89_DBG_TXPWR, "[TXPWR] set txpwr offset\n"); for (desc.idx = 0; desc.idx < RTW89_RATE_OFFSET_MAX; desc.idx++) { - v = rtw89_phy_read_txpwr_byrate(rtwdev, &desc); + v = rtw89_phy_read_txpwr_byrate(rtwdev, band, &desc); val |= ((v & 0xf) << (4 * desc.idx)); } @@ -2045,7 +2099,8 @@ static void rtw8852c_bb_set_tx_shape_dfir(struct rtw89_dev *rtwdev, __DECL_DFIR_ADDR(filter, 0x45BC, 0x45CC, 0x45D0, 0x45D4, 0x45D8, 0x45C0, 0x45C4, 0x45C8); - u8 ch = rtwdev->hal.current_channel; + const struct rtw89_chan *chan = rtw89_chan_get(rtwdev, RTW89_SUB_ENTITY_0); + u8 ch = chan->channel; const u32 *param; int i; @@ -2076,9 +2131,10 @@ static void rtw8852c_bb_set_tx_shape_dfir(struct rtw89_dev *rtwdev, } static void rtw8852c_set_tx_shape(struct rtw89_dev *rtwdev, + const struct rtw89_chan *chan, enum rtw89_phy_idx phy_idx) { - u8 band = rtwdev->hal.current_band_type; + u8 band = chan->band_type; u8 regd = rtw89_regd_get(rtwdev, band); u8 tx_shape_cck = rtw89_8852c_tx_shape[band][RTW89_RS_CCK][regd]; u8 tx_shape_ofdm = rtw89_8852c_tx_shape[band][RTW89_RS_OFDM][regd]; @@ -2092,29 +2148,31 @@ static void rtw8852c_set_tx_shape(struct rtw89_dev *rtwdev, } static void rtw8852c_set_txpwr_limit(struct rtw89_dev *rtwdev, + const struct rtw89_chan *chan, enum rtw89_phy_idx phy_idx) { #define __MAC_TXPWR_LMT_PAGE_SIZE 40 - u8 ch = rtwdev->hal.current_channel; - u8 bw = rtwdev->hal.current_band_width; + u8 ch = chan->channel; + u8 bw = chan->band_width; struct rtw89_txpwr_limit lmt[NTX_NUM_8852C]; u32 addr, val; const s8 *ptr; - u8 i, j, k; + u8 i, j; rtw89_debug(rtwdev, RTW89_DBG_TXPWR, "[TXPWR] set txpwr limit with ch=%d bw=%d\n", ch, bw); for (i = 0; i < NTX_NUM_8852C; i++) { - rtw89_phy_fill_txpwr_limit(rtwdev, &lmt[i], i); + rtw89_phy_fill_txpwr_limit(rtwdev, chan, &lmt[i], i); for (j = 0; j < __MAC_TXPWR_LMT_PAGE_SIZE; j += 4) { addr = R_AX_PWR_LMT + j + __MAC_TXPWR_LMT_PAGE_SIZE * i; ptr = (s8 *)&lmt[i] + j; - val = 0; - for (k = 0; k < 4; k++) - val |= (ptr[k] << (8 * k)); + val = FIELD_PREP(GENMASK(7, 0), ptr[0]) | + FIELD_PREP(GENMASK(15, 8), ptr[1]) | + FIELD_PREP(GENMASK(23, 16), ptr[2]) | + FIELD_PREP(GENMASK(31, 24), ptr[3]); rtw89_mac_txpwr_write32(rtwdev, phy_idx, addr, val); } @@ -2123,30 +2181,32 @@ static void rtw8852c_set_txpwr_limit(struct rtw89_dev *rtwdev, } static void rtw8852c_set_txpwr_limit_ru(struct rtw89_dev *rtwdev, + const struct rtw89_chan *chan, enum rtw89_phy_idx phy_idx) { #define __MAC_TXPWR_LMT_RU_PAGE_SIZE 24 - u8 ch = rtwdev->hal.current_channel; - u8 bw = rtwdev->hal.current_band_width; + u8 ch = chan->channel; + u8 bw = chan->band_width; struct rtw89_txpwr_limit_ru lmt_ru[NTX_NUM_8852C]; u32 addr, val; const s8 *ptr; - u8 i, j, k; + u8 i, j; rtw89_debug(rtwdev, RTW89_DBG_TXPWR, "[TXPWR] set txpwr limit ru with ch=%d bw=%d\n", ch, bw); for (i = 0; i < NTX_NUM_8852C; i++) { - rtw89_phy_fill_txpwr_limit_ru(rtwdev, &lmt_ru[i], i); + rtw89_phy_fill_txpwr_limit_ru(rtwdev, chan, &lmt_ru[i], i); for (j = 0; j < __MAC_TXPWR_LMT_RU_PAGE_SIZE; j += 4) { addr = R_AX_PWR_RU_LMT + j + __MAC_TXPWR_LMT_RU_PAGE_SIZE * i; ptr = (s8 *)&lmt_ru[i] + j; - val = 0; - for (k = 0; k < 4; k++) - val |= (ptr[k] << (8 * k)); + val = FIELD_PREP(GENMASK(7, 0), ptr[0]) | + FIELD_PREP(GENMASK(15, 8), ptr[1]) | + FIELD_PREP(GENMASK(23, 16), ptr[2]) | + FIELD_PREP(GENMASK(31, 24), ptr[3]); rtw89_mac_txpwr_write32(rtwdev, phy_idx, addr, val); } @@ -2155,18 +2215,21 @@ static void rtw8852c_set_txpwr_limit_ru(struct rtw89_dev *rtwdev, #undef __MAC_TXPWR_LMT_RU_PAGE_SIZE } -static void rtw8852c_set_txpwr(struct rtw89_dev *rtwdev) +static void rtw8852c_set_txpwr(struct rtw89_dev *rtwdev, + const struct rtw89_chan *chan, + enum rtw89_phy_idx phy_idx) { - rtw8852c_set_txpwr_byrate(rtwdev, RTW89_PHY_0); - rtw8852c_set_txpwr_offset(rtwdev, RTW89_PHY_0); - rtw8852c_set_tx_shape(rtwdev, RTW89_PHY_0); - rtw8852c_set_txpwr_limit(rtwdev, RTW89_PHY_0); - rtw8852c_set_txpwr_limit_ru(rtwdev, RTW89_PHY_0); + rtw8852c_set_txpwr_byrate(rtwdev, chan, phy_idx); + rtw8852c_set_txpwr_offset(rtwdev, chan, phy_idx); + rtw8852c_set_tx_shape(rtwdev, chan, phy_idx); + rtw8852c_set_txpwr_limit(rtwdev, chan, phy_idx); + rtw8852c_set_txpwr_limit_ru(rtwdev, chan, phy_idx); } -static void rtw8852c_set_txpwr_ctrl(struct rtw89_dev *rtwdev) +static void rtw8852c_set_txpwr_ctrl(struct rtw89_dev *rtwdev, + enum rtw89_phy_idx phy_idx) { - rtw8852c_set_txpwr_ref(rtwdev, RTW89_PHY_0); + rtw8852c_set_txpwr_ref(rtwdev, phy_idx); } static void @@ -2222,7 +2285,8 @@ rtw8852c_init_txpwr_unit(struct rtw89_dev *rtwdev, enum rtw89_phy_idx phy_idx) static void rtw8852c_bb_cfg_rx_path(struct rtw89_dev *rtwdev, u8 rx_path) { - struct rtw89_hal *hal = &rtwdev->hal; + const struct rtw89_chan *chan = rtw89_chan_get(rtwdev, RTW89_SUB_ENTITY_0); + u8 band = chan->band_type; u32 rst_mask0 = B_P0_TXPW_RSTB_MANON | B_P0_TXPW_RSTB_TSSI; u32 rst_mask1 = B_P1_TXPW_RSTB_MANON | B_P1_TXPW_RSTB_TSSI; @@ -2316,7 +2380,7 @@ static void rtw8852c_bb_cfg_rx_path(struct rtw89_dev *rtwdev, u8 rx_path) 1); rtw89_phy_write32_mask(rtwdev, R_RXHE, B_RXHETB_MAX_NSS, 1); - rtw8852c_ctrl_btg(rtwdev, hal->current_band_type == RTW89_BAND_2G); + rtw8852c_ctrl_btg(rtwdev, band == RTW89_BAND_2G); rtw89_phy_write32_mask(rtwdev, R_P0_TXPW_RSTB, rst_mask0, 1); rtw89_phy_write32_mask(rtwdev, R_P0_TXPW_RSTB, @@ -2458,7 +2522,6 @@ static void rtw8852c_bb_ctrl_btc_preagc(struct rtw89_dev *rtwdev, bool bt_en) static void rtw8852c_bb_cfg_txrx_path(struct rtw89_dev *rtwdev) { struct rtw89_hal *hal = &rtwdev->hal; - u8 ntx_path = hal->antenna_tx ? hal->antenna_tx : RF_AB; rtw8852c_bb_cfg_rx_path(rtwdev, RF_PATH_AB); @@ -2473,8 +2536,6 @@ static void rtw8852c_bb_cfg_txrx_path(struct rtw89_dev *rtwdev) rtw89_phy_write32_mask(rtwdev, R_RXHE, B_RXHE_MAX_NSS, 1); rtw89_phy_write32_mask(rtwdev, R_RXHE, B_RXHETB_MAX_NSS, 1); } - - rtw8852c_ctrl_tx_path_tmac(rtwdev, ntx_path, RTW89_MAC_0); } static u8 rtw8852c_get_thermal(struct rtw89_dev *rtwdev, enum rtw89_rf_path rf_path) @@ -2773,23 +2834,7 @@ void rtw8852c_btc_bt_aci_imp(struct rtw89_dev *rtwdev) static void rtw8852c_btc_update_bt_cnt(struct rtw89_dev *rtwdev) { - struct rtw89_btc *btc = &rtwdev->btc; - struct rtw89_btc_cx *cx = &btc->cx; - u32 val; - - val = rtw89_read32(rtwdev, R_BTC_BT_CNT_HIGH); - cx->cnt_bt[BTC_BCNT_HIPRI_TX] = FIELD_GET(B_AX_STATIS_BT_HI_TX_MASK, val); - cx->cnt_bt[BTC_BCNT_HIPRI_RX] = FIELD_GET(B_AX_STATIS_BT_HI_RX_MASK, val); - - val = rtw89_read32(rtwdev, R_BTC_BT_CNT_LOW); - cx->cnt_bt[BTC_BCNT_LOPRI_TX] = FIELD_GET(B_AX_STATIS_BT_LO_TX_1_MASK, val); - cx->cnt_bt[BTC_BCNT_LOPRI_RX] = FIELD_GET(B_AX_STATIS_BT_LO_RX_1_MASK, val); - - /* clock-gate off before reset counter*/ - rtw89_write32_set(rtwdev, R_AX_BTC_CFG, B_AX_DIS_BTC_CLK_G); - rtw89_write32_clr(rtwdev, R_AX_BT_CNT_CFG, B_AX_BT_CNT_RST); - rtw89_write32_set(rtwdev, R_AX_BT_CNT_CFG, B_AX_BT_CNT_RST); - rtw89_write32_clr(rtwdev, R_AX_BTC_CFG, B_AX_DIS_BTC_CLK_G); + /* Feature move to firmware */ } static @@ -2810,6 +2855,59 @@ void rtw8852c_btc_wl_s1_standby(struct rtw89_dev *rtwdev, bool state) rtw89_write_rf(rtwdev, RF_PATH_B, RR_LUTWE, RFREG_MASK, 0x0); } +static void rtw8852c_set_wl_lna2(struct rtw89_dev *rtwdev, u8 level) +{ + /* level=0 Default: TIA 1/0= (LNA2,TIAN6) = (7,1)/(5,1) = 21dB/12dB + * level=1 Fix LNA2=5: TIA 1/0= (LNA2,TIAN6) = (5,0)/(5,1) = 18dB/12dB + * To improve BT ACI in co-rx + */ + + switch (level) { + case 0: /* default */ + rtw89_write_rf(rtwdev, RF_PATH_B, RR_LUTWE, RFREG_MASK, 0x1000); + rtw89_write_rf(rtwdev, RF_PATH_B, RR_LUTWA, RFREG_MASK, 0x0); + rtw89_write_rf(rtwdev, RF_PATH_B, RR_LUTWD0, RFREG_MASK, 0x15); + rtw89_write_rf(rtwdev, RF_PATH_B, RR_LUTWA, RFREG_MASK, 0x1); + rtw89_write_rf(rtwdev, RF_PATH_B, RR_LUTWD0, RFREG_MASK, 0x17); + rtw89_write_rf(rtwdev, RF_PATH_B, RR_LUTWA, RFREG_MASK, 0x2); + rtw89_write_rf(rtwdev, RF_PATH_B, RR_LUTWD0, RFREG_MASK, 0x15); + rtw89_write_rf(rtwdev, RF_PATH_B, RR_LUTWA, RFREG_MASK, 0x3); + rtw89_write_rf(rtwdev, RF_PATH_B, RR_LUTWD0, RFREG_MASK, 0x17); + rtw89_write_rf(rtwdev, RF_PATH_B, RR_LUTWE, RFREG_MASK, 0x0); + break; + case 1: /* Fix LNA2=5 */ + rtw89_write_rf(rtwdev, RF_PATH_B, RR_LUTWE, RFREG_MASK, 0x1000); + rtw89_write_rf(rtwdev, RF_PATH_B, RR_LUTWA, RFREG_MASK, 0x0); + rtw89_write_rf(rtwdev, RF_PATH_B, RR_LUTWD0, RFREG_MASK, 0x15); + rtw89_write_rf(rtwdev, RF_PATH_B, RR_LUTWA, RFREG_MASK, 0x1); + rtw89_write_rf(rtwdev, RF_PATH_B, RR_LUTWD0, RFREG_MASK, 0x5); + rtw89_write_rf(rtwdev, RF_PATH_B, RR_LUTWA, RFREG_MASK, 0x2); + rtw89_write_rf(rtwdev, RF_PATH_B, RR_LUTWD0, RFREG_MASK, 0x15); + rtw89_write_rf(rtwdev, RF_PATH_B, RR_LUTWA, RFREG_MASK, 0x3); + rtw89_write_rf(rtwdev, RF_PATH_B, RR_LUTWD0, RFREG_MASK, 0x5); + rtw89_write_rf(rtwdev, RF_PATH_B, RR_LUTWE, RFREG_MASK, 0x0); + break; + } +} + +static void rtw8852c_btc_set_wl_rx_gain(struct rtw89_dev *rtwdev, u32 level) +{ + switch (level) { + case 0: /* original */ + rtw8852c_bb_ctrl_btc_preagc(rtwdev, false); + rtw8852c_set_wl_lna2(rtwdev, 0); + break; + case 1: /* for FDD free-run */ + rtw8852c_bb_ctrl_btc_preagc(rtwdev, true); + rtw8852c_set_wl_lna2(rtwdev, 0); + break; + case 2: /* for BTG Co-Rx*/ + rtw8852c_bb_ctrl_btc_preagc(rtwdev, false); + rtw8852c_set_wl_lna2(rtwdev, 1); + break; + } +} + static void rtw8852c_fill_freq_with_ppdu(struct rtw89_dev *rtwdev, struct rtw89_rx_phy_ppdu *phy_ppdu, struct ieee80211_rx_status *status) @@ -2831,12 +2929,12 @@ static void rtw8852c_query_ppdu(struct rtw89_dev *rtwdev, struct ieee80211_rx_status *status) { u8 path; - s8 *rx_power = phy_ppdu->rssi; + u8 *rx_power = phy_ppdu->rssi; - status->signal = max_t(s8, rx_power[RF_PATH_A], rx_power[RF_PATH_B]); + status->signal = RTW89_RSSI_RAW_TO_DBM(max(rx_power[RF_PATH_A], rx_power[RF_PATH_B])); for (path = 0; path < rtwdev->chip->rf_path_num; path++) { status->chains |= BIT(path); - status->chain_signal[path] = rx_power[path]; + status->chain_signal[path] = RTW89_RSSI_RAW_TO_DBM(rx_power[path]); } if (phy_ppdu->valid) rtw8852c_fill_freq_with_ppdu(rtwdev, phy_ppdu, status); @@ -2879,10 +2977,12 @@ static int rtw8852c_mac_enable_bb_rf(struct rtw89_dev *rtwdev) return 0; } -static void rtw8852c_mac_disable_bb_rf(struct rtw89_dev *rtwdev) +static int rtw8852c_mac_disable_bb_rf(struct rtw89_dev *rtwdev) { rtw89_write8_clr(rtwdev, R_AX_SYS_FUNC_EN, B_AX_FEN_BBRSTB | B_AX_FEN_BB_GLB_RSTN); + + return 0; } static const struct rtw89_chip_ops rtw8852c_chip_ops = { @@ -2930,6 +3030,8 @@ static const struct rtw89_chip_ops rtw8852c_chip_ops = { .btc_bt_aci_imp = rtw8852c_btc_bt_aci_imp, .btc_update_bt_cnt = rtw8852c_btc_update_bt_cnt, .btc_wl_s1_standby = rtw8852c_btc_wl_s1_standby, + .btc_set_wl_rx_gain = rtw8852c_btc_set_wl_rx_gain, + .btc_set_policy = rtw89_btc_set_policy_v1, }; const struct rtw89_chip_info rtw8852c_chip_info = { @@ -2937,6 +3039,7 @@ const struct rtw89_chip_info rtw8852c_chip_info = { .ops = &rtw8852c_chip_ops, .fw_name = "rtw89/rtw8852c_fw.bin", .fifo_size = 458752, + .dle_scc_rsvd_size = 0, .max_amsdu_limit = 8000, .dis_2g_40m_ul_ofdma = false, .rsvd_ple_ofst = 0x6f800, @@ -2960,7 +3063,9 @@ const struct rtw89_chip_info rtw8852c_chip_info = { .txpwr_factor_rf = 2, .txpwr_factor_mac = 1, .dig_table = NULL, + .dig_regs = &rtw8852c_dig_regs, .tssi_dbw_table = &rtw89_8852c_tssi_dbw_table, + .support_chanctx_num = 1, .support_bands = BIT(NL80211_BAND_2GHZ) | BIT(NL80211_BAND_5GHZ) | BIT(NL80211_BAND_6GHZ), @@ -2972,6 +3077,9 @@ const struct rtw89_chip_info rtw8852c_chip_info = { .acam_num = 128, .bcam_num = 20, .scam_num = 128, + .bacam_num = 8, + .bacam_dynamic_num = 8, + .bacam_v1 = true, .sec_ctrl_efuse_size = 4, .physical_efuse_size = 1216, .logical_efuse_size = 2048, @@ -2980,11 +3088,26 @@ const struct rtw89_chip_info rtw8852c_chip_info = { .dav_log_efuse_size = 16, .phycap_addr = 0x590, .phycap_size = 0x60, - .para_ver = 0x05050764, - .wlcx_desired = 0x05050000, - .btcx_desired = 0x5, + .para_ver = 0x1, + .wlcx_desired = 0x06000000, + .btcx_desired = 0x7, .scbd = 0x1, .mailbox = 0x1, + .btc_fwinfo_buf = 1280, + + .fcxbtcrpt_ver = 4, + .fcxtdma_ver = 3, + .fcxslots_ver = 1, + .fcxcysta_ver = 3, + .fcxstep_ver = 3, + .fcxnullsta_ver = 2, + .fcxmreg_ver = 1, + .fcxgpiodbg_ver = 1, + .fcxbtver_ver = 1, + .fcxbtscan_ver = 1, + .fcxbtafh_ver = 1, + .fcxbtdevinfo_ver = 1, + .afh_guard_ch = 6, .wl_rssi_thres = rtw89_btc_8852c_wl_rssi_thres, .bt_rssi_thres = rtw89_btc_8852c_bt_rssi_thres, @@ -2995,7 +3118,9 @@ const struct rtw89_chip_info rtw8852c_chip_info = { .rf_para_ulink = rtw89_btc_8852c_rf_ul, .rf_para_dlink_num = ARRAY_SIZE(rtw89_btc_8852c_rf_dl), .rf_para_dlink = rtw89_btc_8852c_rf_dl, - .ps_mode_supported = 0, + .ps_mode_supported = BIT(RTW89_PS_MODE_RFOFF) | + BIT(RTW89_PS_MODE_CLK_GATED) | + BIT(RTW89_PS_MODE_PWR_GATED), .low_power_hci_modes = BIT(RTW89_PS_MODE_CLK_GATED) | BIT(RTW89_PS_MODE_PWR_GATED), .h2c_cctl_func_id = H2C_FUNC_MAC_CCTLINFO_UD_V1, @@ -3009,7 +3134,9 @@ const struct rtw89_chip_info rtw8852c_chip_info = { .page_regs = &rtw8852c_page_regs, .dcfo_comp = &rtw8852c_dcfo_comp, .dcfo_comp_sft = 5, - .imr_info = &rtw8852c_imr_info + .imr_info = &rtw8852c_imr_info, + .rrsr_cfgs = &rtw8852c_rrsr_cfgs, + .dma_ch_mask = 0, }; EXPORT_SYMBOL(rtw8852c_chip_info); diff --git a/drivers/net/wireless/realtek/rtw89/rtw8852c_rfk.c b/drivers/net/wireless/realtek/rtw89/rtw8852c_rfk.c index 4186d825d19b..006c2cf93111 100644 --- a/drivers/net/wireless/realtek/rtw89/rtw8852c_rfk.c +++ b/drivers/net/wireless/realtek/rtw89/rtw8852c_rfk.c @@ -1294,14 +1294,14 @@ static void _iqk_by_path(struct rtw89_dev *rtwdev, enum rtw89_phy_idx phy_idx, u static void _iqk_get_ch_info(struct rtw89_dev *rtwdev, enum rtw89_phy_idx phy, u8 path) { + const struct rtw89_chan *chan = rtw89_chan_get(rtwdev, RTW89_SUB_ENTITY_0); struct rtw89_iqk_info *iqk_info = &rtwdev->iqk; - struct rtw89_hal *hal = &rtwdev->hal; rtw89_debug(rtwdev, RTW89_DBG_RFK, "[IQK]===>%s\n", __func__); - iqk_info->iqk_band[path] = hal->current_band_type; - iqk_info->iqk_bw[path] = hal->current_band_width; - iqk_info->iqk_ch[path] = hal->current_channel; + iqk_info->iqk_band[path] = chan->band_type; + iqk_info->iqk_bw[path] = chan->band_width; + iqk_info->iqk_ch[path] = chan->channel; rtw89_debug(rtwdev, RTW89_DBG_RFK, "[IQK]iqk_info->iqk_band[%x] = 0x%x\n", path, @@ -1546,7 +1546,8 @@ static void _rx_dck_toggle(struct rtw89_dev *rtwdev, u8 path) rtw89_write_rf(rtwdev, path, RR_DCK, RR_DCK_LV, 0x1); ret = read_poll_timeout_atomic(rtw89_read_rf, val, val, - 2, 1000, false, rtwdev, path, 0x93, BIT(5)); + 2, 2000, false, rtwdev, path, + RR_DCK1, RR_DCK1_DONE); if (ret) rtw89_warn(rtwdev, "[RX_DCK] S%d RXDCK timeout\n", path); else @@ -1691,14 +1692,14 @@ static void _dpk_information(struct rtw89_dev *rtwdev, enum rtw89_phy_idx phy, enum rtw89_rf_path path) { + const struct rtw89_chan *chan = rtw89_chan_get(rtwdev, RTW89_SUB_ENTITY_0); struct rtw89_dpk_info *dpk = &rtwdev->dpk; - struct rtw89_hal *hal = &rtwdev->hal; u8 kidx = dpk->cur_idx[path]; - dpk->bp[path][kidx].band = hal->current_band_type; - dpk->bp[path][kidx].ch = hal->current_channel; - dpk->bp[path][kidx].bw = hal->current_band_width; + dpk->bp[path][kidx].band = chan->band_type; + dpk->bp[path][kidx].ch = chan->channel; + dpk->bp[path][kidx].bw = chan->band_width; rtw89_debug(rtwdev, RTW89_DBG_RFK, "[DPK] S%d[%d] (PHY%d): TSSI %s/ DBCC %s/ %s/ CH%d/ %s\n", @@ -2272,12 +2273,13 @@ static void _dpk_idl_mpa(struct rtw89_dev *rtwdev, enum rtw89_phy_idx phy, static bool _dpk_reload_check(struct rtw89_dev *rtwdev, enum rtw89_phy_idx phy, enum rtw89_rf_path path) { + const struct rtw89_chan *chan = rtw89_chan_get(rtwdev, RTW89_SUB_ENTITY_0); struct rtw89_dpk_info *dpk = &rtwdev->dpk; bool is_reload = false; u8 idx, cur_band, cur_ch; - cur_band = rtwdev->hal.current_band_type; - cur_ch = rtwdev->hal.current_channel; + cur_band = chan->band_type; + cur_ch = chan->channel; for (idx = 0; idx < RTW89_DPK_BKUP_NUM; idx++) { if (cur_band != dpk->bp[path][idx].band || @@ -2530,17 +2532,19 @@ static void _dpk_cal_select(struct rtw89_dev *rtwdev, bool force, static bool _dpk_bypass_check(struct rtw89_dev *rtwdev, enum rtw89_phy_idx phy) { struct rtw89_fem_info *fem = &rtwdev->fem; + const struct rtw89_chan *chan = rtw89_chan_get(rtwdev, RTW89_SUB_ENTITY_0); + u8 band = chan->band_type; - if (rtwdev->hal.cv == CHIP_CAV && rtwdev->hal.current_band_type != RTW89_BAND_2G) { + if (rtwdev->hal.cv == CHIP_CAV && band != RTW89_BAND_2G) { rtw89_debug(rtwdev, RTW89_DBG_RFK, "[DPK] Skip DPK due to CAV & not 2G!!\n"); return true; - } else if (fem->epa_2g && rtwdev->hal.current_band_type == RTW89_BAND_2G) { + } else if (fem->epa_2g && band == RTW89_BAND_2G) { rtw89_debug(rtwdev, RTW89_DBG_RFK, "[DPK] Skip DPK due to 2G_ext_PA exist!!\n"); return true; - } else if (fem->epa_5g && rtwdev->hal.current_band_type == RTW89_BAND_5G) { + } else if (fem->epa_5g && band == RTW89_BAND_5G) { rtw89_debug(rtwdev, RTW89_DBG_RFK, "[DPK] Skip DPK due to 5G_ext_PA exist!!\n"); return true; - } else if (fem->epa_6g && rtwdev->hal.current_band_type == RTW89_BAND_6G) { + } else if (fem->epa_6g && band == RTW89_BAND_6G) { rtw89_debug(rtwdev, RTW89_DBG_RFK, "[DPK] Skip DPK due to 6G_ext_PA exist!!\n"); return true; } @@ -2663,7 +2667,8 @@ static void _dpk_track(struct rtw89_dev *rtwdev) static void _tssi_set_sys(struct rtw89_dev *rtwdev, enum rtw89_phy_idx phy, enum rtw89_rf_path path) { - enum rtw89_band band = rtwdev->hal.current_band_type; + const struct rtw89_chan *chan = rtw89_chan_get(rtwdev, RTW89_SUB_ENTITY_0); + enum rtw89_band band = chan->band_type; rtw89_rfk_parser(rtwdev, &rtw8852c_tssi_sys_defs_tbl); @@ -2697,7 +2702,8 @@ static void _tssi_ini_txpwr_ctrl_bb_he_tb(struct rtw89_dev *rtwdev, static void _tssi_set_dck(struct rtw89_dev *rtwdev, enum rtw89_phy_idx phy, enum rtw89_rf_path path) { - enum rtw89_band band = rtwdev->hal.current_band_type; + const struct rtw89_chan *chan = rtw89_chan_get(rtwdev, RTW89_SUB_ENTITY_0); + enum rtw89_band band = chan->band_type; if (path == RF_PATH_A) { rtw89_rfk_parser(rtwdev, &rtw8852c_tssi_dck_defs_a_tbl); @@ -2735,8 +2741,9 @@ static void _tssi_set_tmeter_tbl(struct rtw89_dev *rtwdev, enum rtw89_phy_idx ph __val; \ }) struct rtw89_tssi_info *tssi_info = &rtwdev->tssi; - u8 ch = rtwdev->hal.current_channel; - u8 subband = rtwdev->hal.current_subband; + const struct rtw89_chan *chan = rtw89_chan_get(rtwdev, RTW89_SUB_ENTITY_0); + u8 ch = chan->channel; + u8 subband = chan->subband_type; const s8 *thm_up_a = NULL; const s8 *thm_down_a = NULL; const s8 *thm_up_b = NULL; @@ -2908,7 +2915,8 @@ static void _tssi_set_tmeter_tbl(struct rtw89_dev *rtwdev, enum rtw89_phy_idx ph static void _tssi_slope_cal_org(struct rtw89_dev *rtwdev, enum rtw89_phy_idx phy, enum rtw89_rf_path path) { - enum rtw89_band band = rtwdev->hal.current_band_type; + const struct rtw89_chan *chan = rtw89_chan_get(rtwdev, RTW89_SUB_ENTITY_0); + enum rtw89_band band = chan->band_type; if (path == RF_PATH_A) { rtw89_rfk_parser_by_cond(rtwdev, band == RTW89_BAND_2G, @@ -2924,7 +2932,8 @@ static void _tssi_slope_cal_org(struct rtw89_dev *rtwdev, enum rtw89_phy_idx phy static void _tssi_set_aligk_default(struct rtw89_dev *rtwdev, enum rtw89_phy_idx phy, enum rtw89_rf_path path) { - enum rtw89_band band = rtwdev->hal.current_band_type; + const struct rtw89_chan *chan = rtw89_chan_get(rtwdev, RTW89_SUB_ENTITY_0); + enum rtw89_band band = chan->band_type; const struct rtw89_rfk_tbl *tbl; if (path == RF_PATH_A) { @@ -3335,8 +3344,9 @@ static s8 _tssi_get_ofdm_de(struct rtw89_dev *rtwdev, enum rtw89_phy_idx phy, enum rtw89_rf_path path) { struct rtw89_tssi_info *tssi_info = &rtwdev->tssi; - enum rtw89_band band = rtwdev->hal.current_band_type; - u8 ch = rtwdev->hal.current_channel; + const struct rtw89_chan *chan = rtw89_chan_get(rtwdev, RTW89_SUB_ENTITY_0); + enum rtw89_band band = chan->band_type; + u8 ch = chan->channel; u32 gidx, gidx_1st, gidx_2nd; s8 de_1st; s8 de_2nd; @@ -3398,8 +3408,9 @@ static s8 _tssi_get_ofdm_trim_de(struct rtw89_dev *rtwdev, enum rtw89_rf_path path) { struct rtw89_tssi_info *tssi_info = &rtwdev->tssi; - enum rtw89_band band = rtwdev->hal.current_band_type; - u8 ch = rtwdev->hal.current_channel; + const struct rtw89_chan *chan = rtw89_chan_get(rtwdev, RTW89_SUB_ENTITY_0); + enum rtw89_band band = chan->band_type; + u8 ch = chan->channel; u32 tgidx, tgidx_1st, tgidx_2nd; s8 tde_1st = 0; s8 tde_2nd = 0; @@ -3462,7 +3473,8 @@ static void _tssi_set_efuse_to_de(struct rtw89_dev *rtwdev, enum rtw89_phy_idx phy) { struct rtw89_tssi_info *tssi_info = &rtwdev->tssi; - u8 ch = rtwdev->hal.current_channel; + const struct rtw89_chan *chan = rtw89_chan_get(rtwdev, RTW89_SUB_ENTITY_0); + u8 ch = chan->channel; u8 gidx; s8 ofdm_de; s8 trim_de; @@ -3802,15 +3814,17 @@ void rtw8852c_ctrl_bw_ch(struct rtw89_dev *rtwdev, enum rtw89_phy_idx phy, } void rtw8852c_set_channel_rf(struct rtw89_dev *rtwdev, - struct rtw89_channel_params *param, + const struct rtw89_chan *chan, enum rtw89_phy_idx phy_idx) { - rtw8852c_ctrl_bw_ch(rtwdev, phy_idx, param->center_chan, param->band_type, - param->bandwidth); + rtw8852c_ctrl_bw_ch(rtwdev, phy_idx, chan->channel, + chan->band_type, + chan->band_width); } void rtw8852c_mcc_get_ch_info(struct rtw89_dev *rtwdev, enum rtw89_phy_idx phy_idx) { + const struct rtw89_chan *chan = rtw89_chan_get(rtwdev, RTW89_SUB_ENTITY_0); struct rtw89_mcc_info *mcc_info = &rtwdev->mcc; u8 idx = mcc_info->table_idx; int i; @@ -3823,8 +3837,8 @@ void rtw8852c_mcc_get_ch_info(struct rtw89_dev *rtwdev, enum rtw89_phy_idx phy_i } mcc_info->table_idx = idx; - mcc_info->ch[idx] = rtwdev->hal.current_channel; - mcc_info->band[idx] = rtwdev->hal.current_band_type; + mcc_info->ch[idx] = chan->channel; + mcc_info->band[idx] = chan->band_type; } void rtw8852c_rck(struct rtw89_dev *rtwdev) diff --git a/drivers/net/wireless/realtek/rtw89/rtw8852c_rfk.h b/drivers/net/wireless/realtek/rtw89/rtw8852c_rfk.h index 5118a49da8d3..928a587cdd05 100644 --- a/drivers/net/wireless/realtek/rtw89/rtw8852c_rfk.h +++ b/drivers/net/wireless/realtek/rtw89/rtw8852c_rfk.h @@ -21,7 +21,7 @@ void rtw8852c_tssi_cont_en_phyidx(struct rtw89_dev *rtwdev, bool en, u8 phy_idx) void rtw8852c_wifi_scan_notify(struct rtw89_dev *rtwdev, bool scan_start, enum rtw89_phy_idx phy_idx); void rtw8852c_set_channel_rf(struct rtw89_dev *rtwdev, - struct rtw89_channel_params *param, + const struct rtw89_chan *chan, enum rtw89_phy_idx phy_idx); void rtw8852c_lck_init(struct rtw89_dev *rtwdev); void rtw8852c_lck_track(struct rtw89_dev *rtwdev); diff --git a/drivers/net/wireless/realtek/rtw89/rtw8852c_table.c b/drivers/net/wireless/realtek/rtw89/rtw8852c_table.c index feaa83b16171..11f35e7a7f0e 100644 --- a/drivers/net/wireless/realtek/rtw89/rtw8852c_table.c +++ b/drivers/net/wireless/realtek/rtw89/rtw8852c_table.c @@ -1767,7 +1767,7 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_bb_reg_gain[] = { {0x3070103, 0x34343C3C}, }; -static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { +static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0xF0010000, 0x00000000}, {0xF0020000, 0x00000001}, {0xF0320000, 0x00000002}, @@ -1777,13 +1777,17 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0xF0360000, 0x00000006}, {0xF0010001, 0x00000007}, {0xF0020001, 0x00000008}, - {0xF0320001, 0x00000009}, - {0xF0330001, 0x0000000A}, - {0xF0340001, 0x0000000B}, - {0xF0350001, 0x0000000C}, - {0xF0360001, 0x0000000D}, - {0xF03F0001, 0x0000000E}, - {0xF0400001, 0x0000000F}, + {0xF0030001, 0x00000009}, + {0xF0040001, 0x0000000A}, + {0xF0050001, 0x0000000B}, + {0xF0070001, 0x0000000C}, + {0xF0320001, 0x0000000D}, + {0xF0330001, 0x0000000E}, + {0xF0340001, 0x0000000F}, + {0xF0350001, 0x00000010}, + {0xF0360001, 0x00000011}, + {0xF03F0001, 0x00000012}, + {0xF0400001, 0x00000013}, {0x005, 0x00000000}, {0x10005, 0x00000000}, {0x000, 0x00030001}, @@ -1795,7 +1799,7 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x03E, 0x00000620}, {0x03F, 0x0000020C}, {0x0EF, 0x00000000}, - {0x05F, 0x00000032}, + {0x05F, 0x00000038}, {0x097, 0x00043200}, {0x0A6, 0x00066DB7}, {0x0EF, 0x00004000}, @@ -1821,8 +1825,8 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x000, 0x00033C01}, {0x10000, 0x00033C00}, {0x01A, 0x00040004}, - {0x0FE, 0x00000000}, {0x096, 0x00015200}, + {0x10055, 0x00080080}, {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, {0x067, 0x0004D000}, {0x0DA, 0x000D4009}, @@ -1850,6 +1854,18 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, {0x067, 0x0000D300}, {0x0DA, 0x000D4000}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x067, 0x0000D300}, + {0x0DA, 0x000D4000}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x067, 0x0000D300}, + {0x0DA, 0x000D4000}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x067, 0x0000D300}, + {0x0DA, 0x000D4000}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x067, 0x0000D300}, + {0x0DA, 0x000D4000}, {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, {0x067, 0x0000D300}, {0x0DA, 0x000D4000}, @@ -1922,6 +1938,14 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x03F, 0x000000CC}, {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x000000CC}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000000CC}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000000CC}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000000CC}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000000CC}, {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x000000CC}, {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, @@ -1958,6 +1982,14 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x03F, 0x000000C4}, {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x000000C4}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000000C4}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000000C4}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000000C4}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000000C4}, {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x000000C4}, {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, @@ -1994,6 +2026,14 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x03F, 0x000000BC}, {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x000000BC}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000000BC}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000000BC}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000000BC}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000000BC}, {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x000000BC}, {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, @@ -2030,6 +2070,14 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x03F, 0x000000B4}, {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x000000B4}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000000B4}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000000B4}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000000B4}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000000B4}, {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x000000B4}, {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, @@ -2066,6 +2114,14 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x03F, 0x000000AC}, {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x000000AC}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000000AC}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000000AC}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000000AC}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000000AC}, {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x000000AC}, {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, @@ -2102,6 +2158,14 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x03F, 0x000000A4}, {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x000000A4}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000000A4}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000000A4}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000000A4}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000000A4}, {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x000000A4}, {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, @@ -2138,6 +2202,14 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x03F, 0x0000009C}, {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x0000009C}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x0000009C}, {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, @@ -2174,6 +2246,14 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x03F, 0x00000094}, {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000094}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000094}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000094}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000094}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000094}, {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000094}, {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, @@ -2210,6 +2290,14 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x03F, 0x0000008C}, {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x0000008C}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000008C}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000008C}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000008C}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000008C}, {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x0000008C}, {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, @@ -2246,6 +2334,14 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x03F, 0x00000084}, {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000084}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000084}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000084}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000084}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000084}, {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000084}, {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, @@ -2282,6 +2378,14 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x03F, 0x000000BC}, {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x000000BC}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000000BC}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000000BC}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000000BC}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000000BC}, {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x000000BC}, {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, @@ -2318,6 +2422,14 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x03F, 0x000000B4}, {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x000000B4}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000000B4}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000000B4}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000000B4}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000000B4}, {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x000000B4}, {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, @@ -2354,6 +2466,14 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x03F, 0x000000AC}, {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x000000AC}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000000AC}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000000AC}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000000AC}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000000AC}, {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x000000AC}, {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, @@ -2390,6 +2510,14 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x03F, 0x000000A4}, {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x000000A4}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000000A4}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000000A4}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000000A4}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000000A4}, {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x000000A4}, {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, @@ -2426,6 +2554,14 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x03F, 0x0000009C}, {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x0000009C}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x0000009C}, {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, @@ -2462,6 +2598,14 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x03F, 0x00000094}, {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000094}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000094}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000094}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000094}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000094}, {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000094}, {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, @@ -2498,6 +2642,14 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x03F, 0x0000008C}, {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x0000008C}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000008C}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000008C}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000008C}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000008C}, {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x0000008C}, {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, @@ -2534,6 +2686,14 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x03F, 0x00000084}, {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000084}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000084}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000084}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000084}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000084}, {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000084}, {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, @@ -2570,6 +2730,14 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x03F, 0x0000003C}, {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x0000003C}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000003C}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000003C}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000003C}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000003C}, {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x0000003C}, {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, @@ -2606,6 +2774,14 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x03F, 0x00000034}, {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000034}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000034}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000034}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000034}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000034}, {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000034}, {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, @@ -2642,6 +2818,14 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x03F, 0x0000002C}, {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x0000002C}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000002C}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000002C}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000002C}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000002C}, {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x0000002C}, {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, @@ -2678,6 +2862,14 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x03F, 0x00000024}, {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000024}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000024}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000024}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000024}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000024}, {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000024}, {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, @@ -2714,6 +2906,14 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x03F, 0x0000001C}, {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x0000001C}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000001C}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000001C}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000001C}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000001C}, {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x0000001C}, {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, @@ -2750,6 +2950,14 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x03F, 0x00000014}, {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000014}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000014}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000014}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000014}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000014}, {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000014}, {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, @@ -2786,6 +2994,14 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x03F, 0x0000000C}, {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x0000000C}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000000C}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000000C}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000000C}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000000C}, {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x0000000C}, {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, @@ -2822,6 +3038,14 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x03F, 0x00000004}, {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000004}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000004}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000004}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000004}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000004}, {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000004}, {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, @@ -2871,6 +3095,14 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x08F, 0x000D1352}, {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, {0x08F, 0x000D1352}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x08F, 0x000D1352}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x08F, 0x000D1352}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x08F, 0x000D1352}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x08F, 0x000D1352}, {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, {0x08F, 0x000D1352}, {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, @@ -2905,6 +3137,52 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x03F, 0x00000015}, {0x033, 0x00000001}, {0x03F, 0x00000017}, + {0x033, 0x00000002}, + {0x03F, 0x00000017}, + {0x033, 0x00000003}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000007}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000007}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000007}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000017}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000017}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000017}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000017}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000007}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000007}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000007}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000007}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000007}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000007}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000007}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000017}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000017}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000017}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000017}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000017}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000017}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000007}, + {0xB0000000, 0x00000000}, {0x0EF, 0x00000000}, {0x0EF, 0x00008000}, {0x033, 0x00000020}, @@ -3416,6 +3694,14 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x03F, 0x0000EFFF}, {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x0000EFFF}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000EFFF}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000EFFF}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000EFFF}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000EFFF}, {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x0000EFFF}, {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, @@ -3522,7 +3808,7 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x033, 0x00000005}, {0x03F, 0x00004344}, {0x033, 0x00000006}, - {0x03F, 0x00004324}, + {0x03F, 0x00004344}, {0x033, 0x00000007}, {0x03F, 0x00004344}, {0x033, 0x00000008}, @@ -3585,6 +3871,33 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x03F, 0x00000200}, {0x0EF, 0x00000000}, {0x0EF, 0x00000010}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x030, 0x000084DC}, + {0x030, 0x000103C9}, + {0x030, 0x00018399}, + {0x030, 0x00020287}, + {0x030, 0x00028277}, + {0x030, 0x00030165}, + {0x030, 0x00038144}, + {0x030, 0x00040044}, + {0x030, 0x00048022}, + {0x030, 0x00050011}, + {0x030, 0x00058000}, + {0x030, 0x00060000}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x030, 0x000084DC}, + {0x030, 0x000103C9}, + {0x030, 0x00018399}, + {0x030, 0x00020287}, + {0x030, 0x00028277}, + {0x030, 0x00030165}, + {0x030, 0x00038144}, + {0x030, 0x00040044}, + {0x030, 0x00048022}, + {0x030, 0x00050011}, + {0x030, 0x00058000}, + {0x030, 0x00060000}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, {0x030, 0x000084DC}, {0x030, 0x000103C9}, {0x030, 0x00018399}, @@ -3597,6 +3910,241 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x030, 0x00050011}, {0x030, 0x00058000}, {0x030, 0x00060000}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x030, 0x000084DC}, + {0x030, 0x000103C9}, + {0x030, 0x00018399}, + {0x030, 0x00020287}, + {0x030, 0x00028277}, + {0x030, 0x00030165}, + {0x030, 0x00038144}, + {0x030, 0x00040044}, + {0x030, 0x00048022}, + {0x030, 0x00050011}, + {0x030, 0x00058000}, + {0x030, 0x00060000}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x030, 0x000084DC}, + {0x030, 0x000103C9}, + {0x030, 0x00018399}, + {0x030, 0x00020287}, + {0x030, 0x00028277}, + {0x030, 0x00030165}, + {0x030, 0x00038144}, + {0x030, 0x00040044}, + {0x030, 0x00048022}, + {0x030, 0x00050011}, + {0x030, 0x00058000}, + {0x030, 0x00060000}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x030, 0x000084DC}, + {0x030, 0x000103C9}, + {0x030, 0x00018399}, + {0x030, 0x00020287}, + {0x030, 0x00028277}, + {0x030, 0x00030165}, + {0x030, 0x00038144}, + {0x030, 0x00040044}, + {0x030, 0x00048022}, + {0x030, 0x00050011}, + {0x030, 0x00058000}, + {0x030, 0x00060000}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x030, 0x000084DC}, + {0x030, 0x000103C9}, + {0x030, 0x00018399}, + {0x030, 0x00020287}, + {0x030, 0x00028277}, + {0x030, 0x00030165}, + {0x030, 0x00038144}, + {0x030, 0x00040044}, + {0x030, 0x00048022}, + {0x030, 0x00050011}, + {0x030, 0x00058000}, + {0x030, 0x00060000}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x030, 0x000085ED}, + {0x030, 0x000105CC}, + {0x030, 0x000184AA}, + {0x030, 0x00020388}, + {0x030, 0x00028377}, + {0x030, 0x00030377}, + {0x030, 0x00038255}, + {0x030, 0x00040244}, + {0x030, 0x00048133}, + {0x030, 0x00050112}, + {0x030, 0x00058101}, + {0x030, 0x00060001}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x030, 0x000085ED}, + {0x030, 0x000105CC}, + {0x030, 0x000184AA}, + {0x030, 0x00020388}, + {0x030, 0x00028377}, + {0x030, 0x00030377}, + {0x030, 0x00038255}, + {0x030, 0x00040244}, + {0x030, 0x00048133}, + {0x030, 0x00050112}, + {0x030, 0x00058101}, + {0x030, 0x00060001}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x030, 0x000085ED}, + {0x030, 0x000105CC}, + {0x030, 0x000184AA}, + {0x030, 0x00020388}, + {0x030, 0x00028377}, + {0x030, 0x00030377}, + {0x030, 0x00038255}, + {0x030, 0x00040244}, + {0x030, 0x00048133}, + {0x030, 0x00050112}, + {0x030, 0x00058101}, + {0x030, 0x00060001}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x030, 0x000085ED}, + {0x030, 0x000105CC}, + {0x030, 0x000184AA}, + {0x030, 0x00020388}, + {0x030, 0x00028377}, + {0x030, 0x00030377}, + {0x030, 0x00038255}, + {0x030, 0x00040244}, + {0x030, 0x00048133}, + {0x030, 0x00050112}, + {0x030, 0x00058101}, + {0x030, 0x00060001}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x030, 0x000085ED}, + {0x030, 0x000105CC}, + {0x030, 0x000184AA}, + {0x030, 0x00020388}, + {0x030, 0x00028377}, + {0x030, 0x00030377}, + {0x030, 0x00038255}, + {0x030, 0x00040244}, + {0x030, 0x00048133}, + {0x030, 0x00050112}, + {0x030, 0x00058101}, + {0x030, 0x00060001}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x030, 0x000085ED}, + {0x030, 0x000105CC}, + {0x030, 0x000184AA}, + {0x030, 0x00020388}, + {0x030, 0x00028377}, + {0x030, 0x00030377}, + {0x030, 0x00038255}, + {0x030, 0x00040244}, + {0x030, 0x00048133}, + {0x030, 0x00050112}, + {0x030, 0x00058101}, + {0x030, 0x00060001}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x030, 0x000085ED}, + {0x030, 0x000105CC}, + {0x030, 0x000184AA}, + {0x030, 0x00020388}, + {0x030, 0x00028377}, + {0x030, 0x00030377}, + {0x030, 0x00038255}, + {0x030, 0x00040244}, + {0x030, 0x00048133}, + {0x030, 0x00050112}, + {0x030, 0x00058101}, + {0x030, 0x00060001}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x030, 0x000085ED}, + {0x030, 0x000105CC}, + {0x030, 0x000184AA}, + {0x030, 0x00020388}, + {0x030, 0x00028377}, + {0x030, 0x00030377}, + {0x030, 0x00038255}, + {0x030, 0x00040244}, + {0x030, 0x00048133}, + {0x030, 0x00050112}, + {0x030, 0x00058101}, + {0x030, 0x00060001}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x030, 0x000085ED}, + {0x030, 0x000105CC}, + {0x030, 0x000184AA}, + {0x030, 0x00020388}, + {0x030, 0x00028377}, + {0x030, 0x00030377}, + {0x030, 0x00038255}, + {0x030, 0x00040244}, + {0x030, 0x00048133}, + {0x030, 0x00050112}, + {0x030, 0x00058101}, + {0x030, 0x00060001}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x030, 0x000085ED}, + {0x030, 0x000105CC}, + {0x030, 0x000184AA}, + {0x030, 0x00020388}, + {0x030, 0x00028377}, + {0x030, 0x00030377}, + {0x030, 0x00038255}, + {0x030, 0x00040244}, + {0x030, 0x00048133}, + {0x030, 0x00050112}, + {0x030, 0x00058101}, + {0x030, 0x00060001}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x030, 0x000085ED}, + {0x030, 0x000105CC}, + {0x030, 0x000184AA}, + {0x030, 0x00020388}, + {0x030, 0x00028377}, + {0x030, 0x00030377}, + {0x030, 0x00038255}, + {0x030, 0x00040244}, + {0x030, 0x00048133}, + {0x030, 0x00050112}, + {0x030, 0x00058101}, + {0x030, 0x00060001}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x030, 0x000085ED}, + {0x030, 0x000105CC}, + {0x030, 0x000184AA}, + {0x030, 0x00020388}, + {0x030, 0x00028377}, + {0x030, 0x00030377}, + {0x030, 0x00038255}, + {0x030, 0x00040244}, + {0x030, 0x00048133}, + {0x030, 0x00050112}, + {0x030, 0x00058101}, + {0x030, 0x00060001}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x030, 0x000085ED}, + {0x030, 0x000105CC}, + {0x030, 0x000184AA}, + {0x030, 0x00020388}, + {0x030, 0x00028377}, + {0x030, 0x00030377}, + {0x030, 0x00038255}, + {0x030, 0x00040244}, + {0x030, 0x00048133}, + {0x030, 0x00050112}, + {0x030, 0x00058101}, + {0x030, 0x00060001}, + {0xA0000000, 0x00000000}, + {0x030, 0x000084DC}, + {0x030, 0x000103C9}, + {0x030, 0x00018399}, + {0x030, 0x00020287}, + {0x030, 0x00028277}, + {0x030, 0x00030165}, + {0x030, 0x00038144}, + {0x030, 0x00040044}, + {0x030, 0x00048022}, + {0x030, 0x00050011}, + {0x030, 0x00058000}, + {0x030, 0x00060000}, + {0xB0000000, 0x00000000}, {0x030, 0x00068000}, {0x030, 0x00070000}, {0x0EF, 0x00000000}, @@ -3831,6 +4379,14 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x030, 0x000300FF}, {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, {0x030, 0x000300FF}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x030, 0x000300FF}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x030, 0x000300FF}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x030, 0x000300FF}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x030, 0x000300FF}, {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, {0x030, 0x000300FF}, {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, @@ -3901,6 +4457,14 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x095, 0x00000008}, {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, {0x095, 0x00000008}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x095, 0x00000008}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x095, 0x00000008}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x095, 0x00000008}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x095, 0x00000008}, {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, {0x095, 0x00000008}, {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, @@ -3920,101 +4484,2033 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0xB0000000, 0x00000000}, {0x0EE, 0x00001000}, {0x033, 0x00000020}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000052}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000052}, + {0xB0000000, 0x00000000}, {0x033, 0x00000024}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x0000005A}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0xA0000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0xB0000000, 0x00000000}, {0x033, 0x00000028}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0xA0000000, 0x00000000}, {0x03F, 0x0000009C}, + {0xB0000000, 0x00000000}, {0x033, 0x0000002C}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0xA0000000, 0x00000000}, {0x03F, 0x0000019C}, + {0xB0000000, 0x00000000}, {0x033, 0x00000030}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x000001A4}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0xA0000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0xB0000000, 0x00000000}, {0x033, 0x00000034}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E7}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E7}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E7}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E7}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E7}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x000001E7}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E7}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E7}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E7}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E7}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E7}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E7}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E7}, + {0xA0000000, 0x00000000}, + {0x03F, 0x000001E7}, + {0xB0000000, 0x00000000}, {0x033, 0x00000038}, {0x03F, 0x000002E7}, {0x033, 0x0000003C}, {0x03F, 0x000003E7}, {0x033, 0x00000021}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000052}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000052}, + {0xB0000000, 0x00000000}, {0x033, 0x00000025}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x0000005A}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0xA0000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0xB0000000, 0x00000000}, {0x033, 0x00000029}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0xA0000000, 0x00000000}, {0x03F, 0x0000009C}, + {0xB0000000, 0x00000000}, {0x033, 0x0000002D}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x0000019C}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0xA0000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0xB0000000, 0x00000000}, {0x033, 0x00000031}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x000001A4}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0xA0000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0xB0000000, 0x00000000}, {0x033, 0x00000035}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x000001E6}, + {0xA0000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0xB0000000, 0x00000000}, {0x033, 0x00000039}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x000002E6}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0xA0000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0xB0000000, 0x00000000}, {0x033, 0x0000003D}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x000003E6}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0xA0000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0xB0000000, 0x00000000}, {0x033, 0x00000022}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000052}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000052}, + {0xB0000000, 0x00000000}, {0x033, 0x00000026}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0xA0000000, 0x00000000}, {0x03F, 0x0000005A}, + {0xB0000000, 0x00000000}, {0x033, 0x0000002A}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x0000009C}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0xA0000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0xB0000000, 0x00000000}, {0x033, 0x0000002E}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x0000019C}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0xA0000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0xB0000000, 0x00000000}, {0x033, 0x00000032}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x000001A4}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0xA0000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0xB0000000, 0x00000000}, {0x033, 0x00000036}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x000001E6}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0xA0000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0xB0000000, 0x00000000}, {0x033, 0x0000003A}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x000002E6}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0xA0000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0xB0000000, 0x00000000}, {0x033, 0x0000003E}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x000003E6}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0xA0000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0xB0000000, 0x00000000}, {0x033, 0x00000060}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000052}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000052}, + {0xB0000000, 0x00000000}, {0x033, 0x00000064}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x0000005A}, + {0xA0000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0xB0000000, 0x00000000}, {0x033, 0x00000068}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x0000009C}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0xA0000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0xB0000000, 0x00000000}, {0x033, 0x0000006C}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x0000019C}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0xA0000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0xB0000000, 0x00000000}, {0x033, 0x00000070}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0xA0000000, 0x00000000}, {0x03F, 0x000001A4}, + {0xB0000000, 0x00000000}, {0x033, 0x00000074}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x000001E6}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0xA0000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0xB0000000, 0x00000000}, {0x033, 0x00000078}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0xA0000000, 0x00000000}, {0x03F, 0x000002E6}, + {0xB0000000, 0x00000000}, {0x033, 0x0000007C}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x000003E6}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0xA0000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0xB0000000, 0x00000000}, {0x033, 0x00000061}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000052}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000052}, + {0xB0000000, 0x00000000}, {0x033, 0x00000065}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x0000005A}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0xA0000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0xB0000000, 0x00000000}, {0x033, 0x00000069}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x0000009C}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0xA0000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0xB0000000, 0x00000000}, {0x033, 0x0000006D}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x0000019C}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0xA0000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0xB0000000, 0x00000000}, {0x033, 0x00000071}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x000001A4}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0xA0000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0xB0000000, 0x00000000}, {0x033, 0x00000075}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x000001E6}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0xA0000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0xB0000000, 0x00000000}, {0x033, 0x00000079}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x000002E6}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0xA0000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0xB0000000, 0x00000000}, {0x033, 0x0000007D}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x000003E6}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0xA0000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0xB0000000, 0x00000000}, {0x033, 0x00000062}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000052}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000052}, + {0xB0000000, 0x00000000}, {0x033, 0x00000066}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x0000005A}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0xA0000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0xB0000000, 0x00000000}, {0x033, 0x0000006A}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x0000009C}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0xA0000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0xB0000000, 0x00000000}, {0x033, 0x0000006E}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0xA0000000, 0x00000000}, {0x03F, 0x0000019C}, + {0xB0000000, 0x00000000}, {0x033, 0x00000072}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x000001A4}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0xA0000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0xB0000000, 0x00000000}, {0x033, 0x00000076}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x000001E6}, + {0xA0000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0xB0000000, 0x00000000}, {0x033, 0x0000007A}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x000002E6}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0xA0000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0xB0000000, 0x00000000}, {0x033, 0x0000007E}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x000003E6}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0xA0000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0xB0000000, 0x00000000}, {0x033, 0x00000063}, {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000052}, @@ -4034,20 +6530,28 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x03F, 0x00000152}, {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000152}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000152}, {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, - {0x03F, 0x00000152}, + {0x03F, 0x00000052}, {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, - {0x03F, 0x00000152}, + {0x03F, 0x00000052}, {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, - {0x03F, 0x00000152}, + {0x03F, 0x00000052}, {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, - {0x03F, 0x00000152}, + {0x03F, 0x00000052}, {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, - {0x03F, 0x00000152}, + {0x03F, 0x00000052}, {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, - {0x03F, 0x00000152}, + {0x03F, 0x00000052}, {0xA0000000, 0x00000000}, {0x03F, 0x00000052}, {0xB0000000, 0x00000000}, @@ -4070,20 +6574,28 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x03F, 0x0000015A}, {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x0000015A}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x0000015A}, {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, - {0x03F, 0x0000015A}, + {0x03F, 0x0000005A}, {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, - {0x03F, 0x0000015A}, + {0x03F, 0x0000005A}, {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, - {0x03F, 0x0000015A}, + {0x03F, 0x0000005A}, {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, - {0x03F, 0x0000015A}, + {0x03F, 0x0000005A}, {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, - {0x03F, 0x0000015A}, + {0x03F, 0x0000005A}, {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, - {0x03F, 0x0000015A}, + {0x03F, 0x0000005A}, {0xA0000000, 0x00000000}, {0x03F, 0x0000005A}, {0xB0000000, 0x00000000}, @@ -4106,20 +6618,28 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x03F, 0x0000019C}, {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x0000019C}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x0000019C}, {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, - {0x03F, 0x0000019C}, + {0x03F, 0x0000009C}, {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, - {0x03F, 0x0000019C}, + {0x03F, 0x0000009C}, {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, - {0x03F, 0x0000019C}, + {0x03F, 0x0000009C}, {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, - {0x03F, 0x0000019C}, + {0x03F, 0x0000009C}, {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, - {0x03F, 0x0000019C}, + {0x03F, 0x0000009C}, {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, - {0x03F, 0x0000019C}, + {0x03F, 0x0000009C}, {0xA0000000, 0x00000000}, {0x03F, 0x0000009C}, {0xB0000000, 0x00000000}, @@ -4142,20 +6662,28 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x03F, 0x000001A4}, {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x000001A4}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x000001A4}, {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, - {0x03F, 0x000001A4}, + {0x03F, 0x0000019C}, {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, - {0x03F, 0x000001A4}, + {0x03F, 0x0000019C}, {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, - {0x03F, 0x000001A4}, + {0x03F, 0x0000019C}, {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, - {0x03F, 0x000001A4}, + {0x03F, 0x0000019C}, {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, - {0x03F, 0x000001A4}, + {0x03F, 0x0000019C}, {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, - {0x03F, 0x000001A4}, + {0x03F, 0x0000019C}, {0xA0000000, 0x00000000}, {0x03F, 0x0000019C}, {0xB0000000, 0x00000000}, @@ -4178,20 +6706,28 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x03F, 0x000001E6}, {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x000001E6}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x000001E6}, {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, - {0x03F, 0x000001E6}, + {0x03F, 0x000001A4}, {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, - {0x03F, 0x000001E6}, + {0x03F, 0x000001A4}, {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, - {0x03F, 0x000001E6}, + {0x03F, 0x000001A4}, {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, - {0x03F, 0x000001E6}, + {0x03F, 0x000001A4}, {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, - {0x03F, 0x000001E6}, + {0x03F, 0x000001A4}, {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, - {0x03F, 0x000001E6}, + {0x03F, 0x000001A4}, {0xA0000000, 0x00000000}, {0x03F, 0x000001A4}, {0xB0000000, 0x00000000}, @@ -4214,20 +6750,28 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x03F, 0x000002E6}, {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x000002E6}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x000002E6}, {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, - {0x03F, 0x000002E6}, + {0x03F, 0x000001E6}, {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, - {0x03F, 0x000002E6}, + {0x03F, 0x000001E6}, {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, - {0x03F, 0x000002E6}, + {0x03F, 0x000001E6}, {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, - {0x03F, 0x000002E6}, + {0x03F, 0x000001E6}, {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, - {0x03F, 0x000002E6}, + {0x03F, 0x000001E6}, {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, - {0x03F, 0x000002E6}, + {0x03F, 0x000001E6}, {0xA0000000, 0x00000000}, {0x03F, 0x000001E6}, {0xB0000000, 0x00000000}, @@ -5271,131 +7815,131 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x10030, 0x00003C5F}, {0x10030, 0x00004059}, {0x10030, 0x00004453}, - {0x10030, 0x000201ED}, - {0x10030, 0x000205AD}, - {0x10030, 0x000209A7}, - {0x10030, 0x00020DA1}, - {0x10030, 0x0002119B}, - {0x10030, 0x00021561}, - {0x10030, 0x0002195B}, - {0x10030, 0x00021D27}, - {0x10030, 0x00022121}, - {0x10030, 0x000224E9}, - {0x10030, 0x000228E3}, - {0x10030, 0x00022CA9}, - {0x10030, 0x000230A3}, - {0x10030, 0x00023469}, - {0x10030, 0x00023863}, - {0x10030, 0x00023C29}, - {0x10030, 0x00024023}, - {0x10030, 0x0002441D}, + {0x10030, 0x000201EF}, + {0x10030, 0x000205E9}, + {0x10030, 0x000209E3}, + {0x10030, 0x00020DA3}, + {0x10030, 0x00021161}, + {0x10030, 0x0002155B}, + {0x10030, 0x0002191F}, + {0x10030, 0x00021D19}, + {0x10030, 0x000220E1}, + {0x10030, 0x000224DB}, + {0x10030, 0x000228A3}, + {0x10030, 0x00022C9D}, + {0x10030, 0x00023063}, + {0x10030, 0x0002345D}, + {0x10030, 0x00023823}, + {0x10030, 0x00023C1B}, + {0x10030, 0x00024015}, + {0x10030, 0x0002440F}, {0x10030, 0x000281EF}, - {0x10030, 0x000285AF}, - {0x10030, 0x000289A9}, - {0x10030, 0x00028DA3}, - {0x10030, 0x0002919D}, - {0x10030, 0x00029563}, - {0x10030, 0x0002995D}, - {0x10030, 0x00029D25}, - {0x10030, 0x0002A11F}, - {0x10030, 0x0002A4E7}, - {0x10030, 0x0002A8E1}, - {0x10030, 0x0002ACA7}, - {0x10030, 0x0002B0A1}, - {0x10030, 0x0002B467}, - {0x10030, 0x0002B861}, - {0x10030, 0x0002BC27}, - {0x10030, 0x0002C021}, - {0x10030, 0x0002C41B}, + {0x10030, 0x000285E7}, + {0x10030, 0x000289A7}, + {0x10030, 0x00028D65}, + {0x10030, 0x0002915F}, + {0x10030, 0x00029523}, + {0x10030, 0x0002991D}, + {0x10030, 0x00029CE5}, + {0x10030, 0x0002A0DF}, + {0x10030, 0x0002A4A7}, + {0x10030, 0x0002A8A1}, + {0x10030, 0x0002AC67}, + {0x10030, 0x0002B061}, + {0x10030, 0x0002B427}, + {0x10030, 0x0002B821}, + {0x10030, 0x0002BC19}, + {0x10030, 0x0002C013}, + {0x10030, 0x0002C40D}, {0x10030, 0x000301EF}, - {0x10030, 0x000305AF}, - {0x10030, 0x000309A9}, - {0x10030, 0x00030DA3}, - {0x10030, 0x0003119D}, - {0x10030, 0x00031563}, - {0x10030, 0x0003195D}, - {0x10030, 0x00031D25}, - {0x10030, 0x0003211F}, - {0x10030, 0x000324E7}, - {0x10030, 0x000328E1}, - {0x10030, 0x00032CA7}, - {0x10030, 0x000330A1}, - {0x10030, 0x00033467}, - {0x10030, 0x00033861}, - {0x10030, 0x00033C27}, - {0x10030, 0x00034021}, - {0x10030, 0x0003441B}, - {0x10030, 0x000601EB}, - {0x10030, 0x000605AB}, - {0x10030, 0x000609A5}, - {0x10030, 0x00060D9F}, - {0x10030, 0x00061199}, - {0x10030, 0x00061593}, - {0x10030, 0x00061959}, - {0x10030, 0x00061D53}, - {0x10030, 0x0006211B}, - {0x10030, 0x00062515}, - {0x10030, 0x000628DD}, - {0x10030, 0x00062CD7}, - {0x10030, 0x0006309D}, - {0x10030, 0x00063497}, - {0x10030, 0x0006385D}, - {0x10030, 0x00063C57}, - {0x10030, 0x0006401D}, - {0x10030, 0x00064417}, - {0x10030, 0x000681E7}, - {0x10030, 0x000685A7}, - {0x10030, 0x000689A1}, - {0x10030, 0x00068D9B}, - {0x10030, 0x00069195}, - {0x10030, 0x0006955F}, - {0x10030, 0x00069959}, - {0x10030, 0x00069D21}, - {0x10030, 0x0006A11B}, - {0x10030, 0x0006A4E3}, - {0x10030, 0x0006A8DD}, - {0x10030, 0x0006ACA5}, - {0x10030, 0x0006B09F}, - {0x10030, 0x0006B465}, - {0x10030, 0x0006B85F}, - {0x10030, 0x0006BC25}, - {0x10030, 0x0006C01F}, - {0x10030, 0x0006C419}, - {0x10030, 0x000701E7}, - {0x10030, 0x000705A7}, - {0x10030, 0x000709A1}, - {0x10030, 0x00070D9B}, - {0x10030, 0x00071195}, - {0x10030, 0x0007155B}, - {0x10030, 0x00071955}, - {0x10030, 0x00071D1D}, - {0x10030, 0x00072117}, - {0x10030, 0x000724DF}, - {0x10030, 0x000728D9}, - {0x10030, 0x00072CA1}, - {0x10030, 0x0007309B}, - {0x10030, 0x00073461}, - {0x10030, 0x0007385B}, - {0x10030, 0x00073C21}, - {0x10030, 0x0007401B}, - {0x10030, 0x0007441B}, + {0x10030, 0x000305E7}, + {0x10030, 0x000309A7}, + {0x10030, 0x00030D65}, + {0x10030, 0x0003115F}, + {0x10030, 0x00031525}, + {0x10030, 0x0003191F}, + {0x10030, 0x00031CE7}, + {0x10030, 0x000320E1}, + {0x10030, 0x000324A9}, + {0x10030, 0x000328A3}, + {0x10030, 0x00032C69}, + {0x10030, 0x00033063}, + {0x10030, 0x00033429}, + {0x10030, 0x00033823}, + {0x10030, 0x00033C1D}, + {0x10030, 0x00034013}, + {0x10030, 0x0003440D}, + {0x10030, 0x000601F1}, + {0x10030, 0x000605E9}, + {0x10030, 0x000609A9}, + {0x10030, 0x00060D65}, + {0x10030, 0x0006115F}, + {0x10030, 0x00061525}, + {0x10030, 0x0006191F}, + {0x10030, 0x00061CE7}, + {0x10030, 0x000620E1}, + {0x10030, 0x000624DB}, + {0x10030, 0x000628A3}, + {0x10030, 0x00062C69}, + {0x10030, 0x00063063}, + {0x10030, 0x00063429}, + {0x10030, 0x00063823}, + {0x10030, 0x00063C1D}, + {0x10030, 0x00064013}, + {0x10030, 0x0006440D}, + {0x10030, 0x000681EF}, + {0x10030, 0x000685E7}, + {0x10030, 0x000689A7}, + {0x10030, 0x00068D61}, + {0x10030, 0x0006915B}, + {0x10030, 0x00069523}, + {0x10030, 0x0006991D}, + {0x10030, 0x00069CE5}, + {0x10030, 0x0006A0DF}, + {0x10030, 0x0006A4A7}, + {0x10030, 0x0006A8A1}, + {0x10030, 0x0006AC67}, + {0x10030, 0x0006B061}, + {0x10030, 0x0006B429}, + {0x10030, 0x0006B823}, + {0x10030, 0x0006BC1D}, + {0x10030, 0x0006C017}, + {0x10030, 0x0006C40D}, + {0x10030, 0x000701EF}, + {0x10030, 0x000705E7}, + {0x10030, 0x000709A7}, + {0x10030, 0x00070D61}, + {0x10030, 0x0007115B}, + {0x10030, 0x00071523}, + {0x10030, 0x0007191D}, + {0x10030, 0x00071D17}, + {0x10030, 0x000720DF}, + {0x10030, 0x000724D9}, + {0x10030, 0x000728A1}, + {0x10030, 0x00072C67}, + {0x10030, 0x00073061}, + {0x10030, 0x00073427}, + {0x10030, 0x00073821}, + {0x10030, 0x00073C1B}, + {0x10030, 0x00074015}, + {0x10030, 0x0007440D}, {0x10030, 0x000781EF}, {0x10030, 0x000785E9}, {0x10030, 0x000789E3}, - {0x10030, 0x00078DA3}, - {0x10030, 0x00079161}, - {0x10030, 0x0007955B}, + {0x10030, 0x00078DA1}, + {0x10030, 0x0007915F}, + {0x10030, 0x00079559}, {0x10030, 0x00079921}, {0x10030, 0x00079D1B}, - {0x10030, 0x0007A0E1}, - {0x10030, 0x0007A4DB}, - {0x10030, 0x0007A8A1}, - {0x10030, 0x0007AC9B}, - {0x10030, 0x0007B061}, - {0x10030, 0x0007B45B}, - {0x10030, 0x0007B821}, - {0x10030, 0x0007BC1B}, - {0x10030, 0x0007C015}, + {0x10030, 0x0007A0E3}, + {0x10030, 0x0007A4DD}, + {0x10030, 0x0007A8D7}, + {0x10030, 0x0007AC9D}, + {0x10030, 0x0007B063}, + {0x10030, 0x0007B45D}, + {0x10030, 0x0007B823}, + {0x10030, 0x0007BC1D}, + {0x10030, 0x0007C017}, {0x10030, 0x0007C40F}, {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, {0x10030, 0x000001EF}, @@ -5416,131 +7960,711 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x10030, 0x00003C5F}, {0x10030, 0x00004059}, {0x10030, 0x00004453}, - {0x10030, 0x000201ED}, - {0x10030, 0x000205AD}, - {0x10030, 0x000209A7}, - {0x10030, 0x00020DA1}, - {0x10030, 0x0002119B}, - {0x10030, 0x00021561}, - {0x10030, 0x0002195B}, - {0x10030, 0x00021D27}, - {0x10030, 0x00022121}, - {0x10030, 0x000224E9}, - {0x10030, 0x000228E3}, - {0x10030, 0x00022CA9}, - {0x10030, 0x000230A3}, - {0x10030, 0x00023469}, - {0x10030, 0x00023863}, - {0x10030, 0x00023C29}, - {0x10030, 0x00024023}, - {0x10030, 0x0002441D}, + {0x10030, 0x000201EF}, + {0x10030, 0x000205E9}, + {0x10030, 0x000209E3}, + {0x10030, 0x00020DA3}, + {0x10030, 0x00021161}, + {0x10030, 0x0002155B}, + {0x10030, 0x0002191F}, + {0x10030, 0x00021D19}, + {0x10030, 0x000220E1}, + {0x10030, 0x000224DB}, + {0x10030, 0x000228A3}, + {0x10030, 0x00022C9D}, + {0x10030, 0x00023063}, + {0x10030, 0x0002345D}, + {0x10030, 0x00023823}, + {0x10030, 0x00023C1B}, + {0x10030, 0x00024015}, + {0x10030, 0x0002440F}, {0x10030, 0x000281EF}, - {0x10030, 0x000285AF}, - {0x10030, 0x000289A9}, - {0x10030, 0x00028DA3}, - {0x10030, 0x0002919D}, - {0x10030, 0x00029563}, - {0x10030, 0x0002995D}, - {0x10030, 0x00029D25}, - {0x10030, 0x0002A11F}, - {0x10030, 0x0002A4E7}, - {0x10030, 0x0002A8E1}, - {0x10030, 0x0002ACA7}, - {0x10030, 0x0002B0A1}, - {0x10030, 0x0002B467}, - {0x10030, 0x0002B861}, - {0x10030, 0x0002BC27}, - {0x10030, 0x0002C021}, - {0x10030, 0x0002C41B}, + {0x10030, 0x000285E7}, + {0x10030, 0x000289A7}, + {0x10030, 0x00028D65}, + {0x10030, 0x0002915F}, + {0x10030, 0x00029523}, + {0x10030, 0x0002991D}, + {0x10030, 0x00029CE5}, + {0x10030, 0x0002A0DF}, + {0x10030, 0x0002A4A7}, + {0x10030, 0x0002A8A1}, + {0x10030, 0x0002AC67}, + {0x10030, 0x0002B061}, + {0x10030, 0x0002B427}, + {0x10030, 0x0002B821}, + {0x10030, 0x0002BC19}, + {0x10030, 0x0002C013}, + {0x10030, 0x0002C40D}, {0x10030, 0x000301EF}, - {0x10030, 0x000305AF}, - {0x10030, 0x000309A9}, - {0x10030, 0x00030DA3}, - {0x10030, 0x0003119D}, - {0x10030, 0x00031563}, - {0x10030, 0x0003195D}, - {0x10030, 0x00031D25}, - {0x10030, 0x0003211F}, - {0x10030, 0x000324E7}, - {0x10030, 0x000328E1}, - {0x10030, 0x00032CA7}, - {0x10030, 0x000330A1}, - {0x10030, 0x00033467}, - {0x10030, 0x00033861}, - {0x10030, 0x00033C27}, - {0x10030, 0x00034021}, - {0x10030, 0x0003441B}, - {0x10030, 0x000601EB}, - {0x10030, 0x000605AB}, - {0x10030, 0x000609A5}, - {0x10030, 0x00060D9F}, - {0x10030, 0x00061199}, - {0x10030, 0x00061593}, - {0x10030, 0x00061959}, - {0x10030, 0x00061D53}, - {0x10030, 0x0006211B}, - {0x10030, 0x00062515}, - {0x10030, 0x000628DD}, - {0x10030, 0x00062CD7}, - {0x10030, 0x0006309D}, - {0x10030, 0x00063497}, - {0x10030, 0x0006385D}, - {0x10030, 0x00063C57}, - {0x10030, 0x0006401D}, - {0x10030, 0x00064417}, - {0x10030, 0x000681E7}, - {0x10030, 0x000685A7}, - {0x10030, 0x000689A1}, - {0x10030, 0x00068D9B}, - {0x10030, 0x00069195}, - {0x10030, 0x0006955F}, - {0x10030, 0x00069959}, - {0x10030, 0x00069D21}, - {0x10030, 0x0006A11B}, - {0x10030, 0x0006A4E3}, - {0x10030, 0x0006A8DD}, - {0x10030, 0x0006ACA5}, - {0x10030, 0x0006B09F}, - {0x10030, 0x0006B465}, - {0x10030, 0x0006B85F}, - {0x10030, 0x0006BC25}, - {0x10030, 0x0006C01F}, - {0x10030, 0x0006C419}, - {0x10030, 0x000701E7}, - {0x10030, 0x000705A7}, - {0x10030, 0x000709A1}, - {0x10030, 0x00070D9B}, - {0x10030, 0x00071195}, - {0x10030, 0x0007155B}, - {0x10030, 0x00071955}, - {0x10030, 0x00071D1D}, - {0x10030, 0x00072117}, - {0x10030, 0x000724DF}, - {0x10030, 0x000728D9}, - {0x10030, 0x00072CA1}, - {0x10030, 0x0007309B}, - {0x10030, 0x00073461}, - {0x10030, 0x0007385B}, - {0x10030, 0x00073C21}, - {0x10030, 0x0007401B}, - {0x10030, 0x0007441B}, + {0x10030, 0x000305E7}, + {0x10030, 0x000309A7}, + {0x10030, 0x00030D65}, + {0x10030, 0x0003115F}, + {0x10030, 0x00031525}, + {0x10030, 0x0003191F}, + {0x10030, 0x00031CE7}, + {0x10030, 0x000320E1}, + {0x10030, 0x000324A9}, + {0x10030, 0x000328A3}, + {0x10030, 0x00032C69}, + {0x10030, 0x00033063}, + {0x10030, 0x00033429}, + {0x10030, 0x00033823}, + {0x10030, 0x00033C1D}, + {0x10030, 0x00034013}, + {0x10030, 0x0003440D}, + {0x10030, 0x000601F1}, + {0x10030, 0x000605E9}, + {0x10030, 0x000609A9}, + {0x10030, 0x00060D65}, + {0x10030, 0x0006115F}, + {0x10030, 0x00061525}, + {0x10030, 0x0006191F}, + {0x10030, 0x00061CE7}, + {0x10030, 0x000620E1}, + {0x10030, 0x000624DB}, + {0x10030, 0x000628A3}, + {0x10030, 0x00062C69}, + {0x10030, 0x00063063}, + {0x10030, 0x00063429}, + {0x10030, 0x00063823}, + {0x10030, 0x00063C1D}, + {0x10030, 0x00064013}, + {0x10030, 0x0006440D}, + {0x10030, 0x000681EF}, + {0x10030, 0x000685E7}, + {0x10030, 0x000689A7}, + {0x10030, 0x00068D61}, + {0x10030, 0x0006915B}, + {0x10030, 0x00069523}, + {0x10030, 0x0006991D}, + {0x10030, 0x00069CE5}, + {0x10030, 0x0006A0DF}, + {0x10030, 0x0006A4A7}, + {0x10030, 0x0006A8A1}, + {0x10030, 0x0006AC67}, + {0x10030, 0x0006B061}, + {0x10030, 0x0006B429}, + {0x10030, 0x0006B823}, + {0x10030, 0x0006BC1D}, + {0x10030, 0x0006C017}, + {0x10030, 0x0006C40D}, + {0x10030, 0x000701EF}, + {0x10030, 0x000705E7}, + {0x10030, 0x000709A7}, + {0x10030, 0x00070D61}, + {0x10030, 0x0007115B}, + {0x10030, 0x00071523}, + {0x10030, 0x0007191D}, + {0x10030, 0x00071D17}, + {0x10030, 0x000720DF}, + {0x10030, 0x000724D9}, + {0x10030, 0x000728A1}, + {0x10030, 0x00072C67}, + {0x10030, 0x00073061}, + {0x10030, 0x00073427}, + {0x10030, 0x00073821}, + {0x10030, 0x00073C1B}, + {0x10030, 0x00074015}, + {0x10030, 0x0007440D}, {0x10030, 0x000781EF}, {0x10030, 0x000785E9}, {0x10030, 0x000789E3}, - {0x10030, 0x00078DA3}, - {0x10030, 0x00079161}, - {0x10030, 0x0007955B}, + {0x10030, 0x00078DA1}, + {0x10030, 0x0007915F}, + {0x10030, 0x00079559}, + {0x10030, 0x00079921}, + {0x10030, 0x00079D1B}, + {0x10030, 0x0007A0E3}, + {0x10030, 0x0007A4DD}, + {0x10030, 0x0007A8D7}, + {0x10030, 0x0007AC9D}, + {0x10030, 0x0007B063}, + {0x10030, 0x0007B45D}, + {0x10030, 0x0007B823}, + {0x10030, 0x0007BC1D}, + {0x10030, 0x0007C017}, + {0x10030, 0x0007C40F}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x000001EF}, + {0x10030, 0x000005E9}, + {0x10030, 0x000009E3}, + {0x10030, 0x00000DDD}, + {0x10030, 0x000011D7}, + {0x10030, 0x0000159F}, + {0x10030, 0x00001999}, + {0x10030, 0x00001D5F}, + {0x10030, 0x00002159}, + {0x10030, 0x0000251F}, + {0x10030, 0x00002919}, + {0x10030, 0x00002CDF}, + {0x10030, 0x000030D9}, + {0x10030, 0x0000349F}, + {0x10030, 0x00003899}, + {0x10030, 0x00003C5F}, + {0x10030, 0x00004059}, + {0x10030, 0x00004453}, + {0x10030, 0x000201EF}, + {0x10030, 0x000205E9}, + {0x10030, 0x000209E3}, + {0x10030, 0x00020DA3}, + {0x10030, 0x00021161}, + {0x10030, 0x0002155B}, + {0x10030, 0x0002191F}, + {0x10030, 0x00021D19}, + {0x10030, 0x000220E1}, + {0x10030, 0x000224DB}, + {0x10030, 0x000228A3}, + {0x10030, 0x00022C9D}, + {0x10030, 0x00023063}, + {0x10030, 0x0002345D}, + {0x10030, 0x00023823}, + {0x10030, 0x00023C1B}, + {0x10030, 0x00024015}, + {0x10030, 0x0002440F}, + {0x10030, 0x000281EF}, + {0x10030, 0x000285E7}, + {0x10030, 0x000289A7}, + {0x10030, 0x00028D65}, + {0x10030, 0x0002915F}, + {0x10030, 0x00029523}, + {0x10030, 0x0002991D}, + {0x10030, 0x00029CE5}, + {0x10030, 0x0002A0DF}, + {0x10030, 0x0002A4A7}, + {0x10030, 0x0002A8A1}, + {0x10030, 0x0002AC67}, + {0x10030, 0x0002B061}, + {0x10030, 0x0002B427}, + {0x10030, 0x0002B821}, + {0x10030, 0x0002BC19}, + {0x10030, 0x0002C013}, + {0x10030, 0x0002C40D}, + {0x10030, 0x000301EF}, + {0x10030, 0x000305E7}, + {0x10030, 0x000309A7}, + {0x10030, 0x00030D65}, + {0x10030, 0x0003115F}, + {0x10030, 0x00031525}, + {0x10030, 0x0003191F}, + {0x10030, 0x00031CE7}, + {0x10030, 0x000320E1}, + {0x10030, 0x000324A9}, + {0x10030, 0x000328A3}, + {0x10030, 0x00032C69}, + {0x10030, 0x00033063}, + {0x10030, 0x00033429}, + {0x10030, 0x00033823}, + {0x10030, 0x00033C1D}, + {0x10030, 0x00034013}, + {0x10030, 0x0003440D}, + {0x10030, 0x000601F1}, + {0x10030, 0x000605E9}, + {0x10030, 0x000609A9}, + {0x10030, 0x00060D65}, + {0x10030, 0x0006115F}, + {0x10030, 0x00061525}, + {0x10030, 0x0006191F}, + {0x10030, 0x00061CE7}, + {0x10030, 0x000620E1}, + {0x10030, 0x000624DB}, + {0x10030, 0x000628A3}, + {0x10030, 0x00062C69}, + {0x10030, 0x00063063}, + {0x10030, 0x00063429}, + {0x10030, 0x00063823}, + {0x10030, 0x00063C1D}, + {0x10030, 0x00064013}, + {0x10030, 0x0006440D}, + {0x10030, 0x000681EF}, + {0x10030, 0x000685E7}, + {0x10030, 0x000689A7}, + {0x10030, 0x00068D61}, + {0x10030, 0x0006915B}, + {0x10030, 0x00069523}, + {0x10030, 0x0006991D}, + {0x10030, 0x00069CE5}, + {0x10030, 0x0006A0DF}, + {0x10030, 0x0006A4A7}, + {0x10030, 0x0006A8A1}, + {0x10030, 0x0006AC67}, + {0x10030, 0x0006B061}, + {0x10030, 0x0006B429}, + {0x10030, 0x0006B823}, + {0x10030, 0x0006BC1D}, + {0x10030, 0x0006C017}, + {0x10030, 0x0006C40D}, + {0x10030, 0x000701EF}, + {0x10030, 0x000705E7}, + {0x10030, 0x000709A7}, + {0x10030, 0x00070D61}, + {0x10030, 0x0007115B}, + {0x10030, 0x00071523}, + {0x10030, 0x0007191D}, + {0x10030, 0x00071D17}, + {0x10030, 0x000720DF}, + {0x10030, 0x000724D9}, + {0x10030, 0x000728A1}, + {0x10030, 0x00072C67}, + {0x10030, 0x00073061}, + {0x10030, 0x00073427}, + {0x10030, 0x00073821}, + {0x10030, 0x00073C1B}, + {0x10030, 0x00074015}, + {0x10030, 0x0007440D}, + {0x10030, 0x000781EF}, + {0x10030, 0x000785E9}, + {0x10030, 0x000789E3}, + {0x10030, 0x00078DA1}, + {0x10030, 0x0007915F}, + {0x10030, 0x00079559}, + {0x10030, 0x00079921}, + {0x10030, 0x00079D1B}, + {0x10030, 0x0007A0E3}, + {0x10030, 0x0007A4DD}, + {0x10030, 0x0007A8D7}, + {0x10030, 0x0007AC9D}, + {0x10030, 0x0007B063}, + {0x10030, 0x0007B45D}, + {0x10030, 0x0007B823}, + {0x10030, 0x0007BC1D}, + {0x10030, 0x0007C017}, + {0x10030, 0x0007C40F}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x000001EF}, + {0x10030, 0x000005E9}, + {0x10030, 0x000009E3}, + {0x10030, 0x00000DDD}, + {0x10030, 0x000011D7}, + {0x10030, 0x0000159F}, + {0x10030, 0x00001999}, + {0x10030, 0x00001D5F}, + {0x10030, 0x00002159}, + {0x10030, 0x0000251F}, + {0x10030, 0x00002919}, + {0x10030, 0x00002CDF}, + {0x10030, 0x000030D9}, + {0x10030, 0x0000349F}, + {0x10030, 0x00003899}, + {0x10030, 0x00003C5F}, + {0x10030, 0x00004059}, + {0x10030, 0x00004453}, + {0x10030, 0x000201EF}, + {0x10030, 0x000205E9}, + {0x10030, 0x000209E3}, + {0x10030, 0x00020DA3}, + {0x10030, 0x00021161}, + {0x10030, 0x0002155B}, + {0x10030, 0x0002191F}, + {0x10030, 0x00021D19}, + {0x10030, 0x000220E1}, + {0x10030, 0x000224DB}, + {0x10030, 0x000228A3}, + {0x10030, 0x00022C9D}, + {0x10030, 0x00023063}, + {0x10030, 0x0002345D}, + {0x10030, 0x00023823}, + {0x10030, 0x00023C1B}, + {0x10030, 0x00024015}, + {0x10030, 0x0002440F}, + {0x10030, 0x000281EF}, + {0x10030, 0x000285E7}, + {0x10030, 0x000289A7}, + {0x10030, 0x00028D65}, + {0x10030, 0x0002915F}, + {0x10030, 0x00029523}, + {0x10030, 0x0002991D}, + {0x10030, 0x00029CE5}, + {0x10030, 0x0002A0DF}, + {0x10030, 0x0002A4A7}, + {0x10030, 0x0002A8A1}, + {0x10030, 0x0002AC67}, + {0x10030, 0x0002B061}, + {0x10030, 0x0002B427}, + {0x10030, 0x0002B821}, + {0x10030, 0x0002BC19}, + {0x10030, 0x0002C013}, + {0x10030, 0x0002C40D}, + {0x10030, 0x000301EF}, + {0x10030, 0x000305E7}, + {0x10030, 0x000309A7}, + {0x10030, 0x00030D65}, + {0x10030, 0x0003115F}, + {0x10030, 0x00031525}, + {0x10030, 0x0003191F}, + {0x10030, 0x00031CE7}, + {0x10030, 0x000320E1}, + {0x10030, 0x000324A9}, + {0x10030, 0x000328A3}, + {0x10030, 0x00032C69}, + {0x10030, 0x00033063}, + {0x10030, 0x00033429}, + {0x10030, 0x00033823}, + {0x10030, 0x00033C1D}, + {0x10030, 0x00034013}, + {0x10030, 0x0003440D}, + {0x10030, 0x000601F1}, + {0x10030, 0x000605E9}, + {0x10030, 0x000609A9}, + {0x10030, 0x00060D65}, + {0x10030, 0x0006115F}, + {0x10030, 0x00061525}, + {0x10030, 0x0006191F}, + {0x10030, 0x00061CE7}, + {0x10030, 0x000620E1}, + {0x10030, 0x000624DB}, + {0x10030, 0x000628A3}, + {0x10030, 0x00062C69}, + {0x10030, 0x00063063}, + {0x10030, 0x00063429}, + {0x10030, 0x00063823}, + {0x10030, 0x00063C1D}, + {0x10030, 0x00064013}, + {0x10030, 0x0006440D}, + {0x10030, 0x000681EF}, + {0x10030, 0x000685E7}, + {0x10030, 0x000689A7}, + {0x10030, 0x00068D61}, + {0x10030, 0x0006915B}, + {0x10030, 0x00069523}, + {0x10030, 0x0006991D}, + {0x10030, 0x00069CE5}, + {0x10030, 0x0006A0DF}, + {0x10030, 0x0006A4A7}, + {0x10030, 0x0006A8A1}, + {0x10030, 0x0006AC67}, + {0x10030, 0x0006B061}, + {0x10030, 0x0006B429}, + {0x10030, 0x0006B823}, + {0x10030, 0x0006BC1D}, + {0x10030, 0x0006C017}, + {0x10030, 0x0006C40D}, + {0x10030, 0x000701EF}, + {0x10030, 0x000705E7}, + {0x10030, 0x000709A7}, + {0x10030, 0x00070D61}, + {0x10030, 0x0007115B}, + {0x10030, 0x00071523}, + {0x10030, 0x0007191D}, + {0x10030, 0x00071D17}, + {0x10030, 0x000720DF}, + {0x10030, 0x000724D9}, + {0x10030, 0x000728A1}, + {0x10030, 0x00072C67}, + {0x10030, 0x00073061}, + {0x10030, 0x00073427}, + {0x10030, 0x00073821}, + {0x10030, 0x00073C1B}, + {0x10030, 0x00074015}, + {0x10030, 0x0007440D}, + {0x10030, 0x000781EF}, + {0x10030, 0x000785E9}, + {0x10030, 0x000789E3}, + {0x10030, 0x00078DA1}, + {0x10030, 0x0007915F}, + {0x10030, 0x00079559}, + {0x10030, 0x00079921}, + {0x10030, 0x00079D1B}, + {0x10030, 0x0007A0E3}, + {0x10030, 0x0007A4DD}, + {0x10030, 0x0007A8D7}, + {0x10030, 0x0007AC9D}, + {0x10030, 0x0007B063}, + {0x10030, 0x0007B45D}, + {0x10030, 0x0007B823}, + {0x10030, 0x0007BC1D}, + {0x10030, 0x0007C017}, + {0x10030, 0x0007C40F}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x000001EF}, + {0x10030, 0x000005E9}, + {0x10030, 0x000009E3}, + {0x10030, 0x00000DDD}, + {0x10030, 0x000011D7}, + {0x10030, 0x0000159F}, + {0x10030, 0x00001999}, + {0x10030, 0x00001D5F}, + {0x10030, 0x00002159}, + {0x10030, 0x0000251F}, + {0x10030, 0x00002919}, + {0x10030, 0x00002CDF}, + {0x10030, 0x000030D9}, + {0x10030, 0x0000349F}, + {0x10030, 0x00003899}, + {0x10030, 0x00003C5F}, + {0x10030, 0x00004059}, + {0x10030, 0x00004453}, + {0x10030, 0x000201EF}, + {0x10030, 0x000205E9}, + {0x10030, 0x000209E3}, + {0x10030, 0x00020DA3}, + {0x10030, 0x00021161}, + {0x10030, 0x0002155B}, + {0x10030, 0x0002191F}, + {0x10030, 0x00021D19}, + {0x10030, 0x000220E1}, + {0x10030, 0x000224DB}, + {0x10030, 0x000228A3}, + {0x10030, 0x00022C9D}, + {0x10030, 0x00023063}, + {0x10030, 0x0002345D}, + {0x10030, 0x00023823}, + {0x10030, 0x00023C1B}, + {0x10030, 0x00024015}, + {0x10030, 0x0002440F}, + {0x10030, 0x000281EF}, + {0x10030, 0x000285E7}, + {0x10030, 0x000289A7}, + {0x10030, 0x00028D65}, + {0x10030, 0x0002915F}, + {0x10030, 0x00029523}, + {0x10030, 0x0002991D}, + {0x10030, 0x00029CE5}, + {0x10030, 0x0002A0DF}, + {0x10030, 0x0002A4A7}, + {0x10030, 0x0002A8A1}, + {0x10030, 0x0002AC67}, + {0x10030, 0x0002B061}, + {0x10030, 0x0002B427}, + {0x10030, 0x0002B821}, + {0x10030, 0x0002BC19}, + {0x10030, 0x0002C013}, + {0x10030, 0x0002C40D}, + {0x10030, 0x000301EF}, + {0x10030, 0x000305E7}, + {0x10030, 0x000309A7}, + {0x10030, 0x00030D65}, + {0x10030, 0x0003115F}, + {0x10030, 0x00031525}, + {0x10030, 0x0003191F}, + {0x10030, 0x00031CE7}, + {0x10030, 0x000320E1}, + {0x10030, 0x000324A9}, + {0x10030, 0x000328A3}, + {0x10030, 0x00032C69}, + {0x10030, 0x00033063}, + {0x10030, 0x00033429}, + {0x10030, 0x00033823}, + {0x10030, 0x00033C1D}, + {0x10030, 0x00034013}, + {0x10030, 0x0003440D}, + {0x10030, 0x000601F1}, + {0x10030, 0x000605E9}, + {0x10030, 0x000609A9}, + {0x10030, 0x00060D65}, + {0x10030, 0x0006115F}, + {0x10030, 0x00061525}, + {0x10030, 0x0006191F}, + {0x10030, 0x00061CE7}, + {0x10030, 0x000620E1}, + {0x10030, 0x000624DB}, + {0x10030, 0x000628A3}, + {0x10030, 0x00062C69}, + {0x10030, 0x00063063}, + {0x10030, 0x00063429}, + {0x10030, 0x00063823}, + {0x10030, 0x00063C1D}, + {0x10030, 0x00064013}, + {0x10030, 0x0006440D}, + {0x10030, 0x000681EF}, + {0x10030, 0x000685E7}, + {0x10030, 0x000689A7}, + {0x10030, 0x00068D61}, + {0x10030, 0x0006915B}, + {0x10030, 0x00069523}, + {0x10030, 0x0006991D}, + {0x10030, 0x00069CE5}, + {0x10030, 0x0006A0DF}, + {0x10030, 0x0006A4A7}, + {0x10030, 0x0006A8A1}, + {0x10030, 0x0006AC67}, + {0x10030, 0x0006B061}, + {0x10030, 0x0006B429}, + {0x10030, 0x0006B823}, + {0x10030, 0x0006BC1D}, + {0x10030, 0x0006C017}, + {0x10030, 0x0006C40D}, + {0x10030, 0x000701EF}, + {0x10030, 0x000705E7}, + {0x10030, 0x000709A7}, + {0x10030, 0x00070D61}, + {0x10030, 0x0007115B}, + {0x10030, 0x00071523}, + {0x10030, 0x0007191D}, + {0x10030, 0x00071D17}, + {0x10030, 0x000720DF}, + {0x10030, 0x000724D9}, + {0x10030, 0x000728A1}, + {0x10030, 0x00072C67}, + {0x10030, 0x00073061}, + {0x10030, 0x00073427}, + {0x10030, 0x00073821}, + {0x10030, 0x00073C1B}, + {0x10030, 0x00074015}, + {0x10030, 0x0007440D}, + {0x10030, 0x000781EF}, + {0x10030, 0x000785E9}, + {0x10030, 0x000789E3}, + {0x10030, 0x00078DA1}, + {0x10030, 0x0007915F}, + {0x10030, 0x00079559}, {0x10030, 0x00079921}, {0x10030, 0x00079D1B}, - {0x10030, 0x0007A0E1}, - {0x10030, 0x0007A4DB}, - {0x10030, 0x0007A8A1}, - {0x10030, 0x0007AC9B}, - {0x10030, 0x0007B061}, - {0x10030, 0x0007B45B}, - {0x10030, 0x0007B821}, - {0x10030, 0x0007BC1B}, - {0x10030, 0x0007C015}, + {0x10030, 0x0007A0E3}, + {0x10030, 0x0007A4DD}, + {0x10030, 0x0007A8D7}, + {0x10030, 0x0007AC9D}, + {0x10030, 0x0007B063}, + {0x10030, 0x0007B45D}, + {0x10030, 0x0007B823}, + {0x10030, 0x0007BC1D}, + {0x10030, 0x0007C017}, + {0x10030, 0x0007C40F}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x000001EF}, + {0x10030, 0x000005E9}, + {0x10030, 0x000009E3}, + {0x10030, 0x00000DDD}, + {0x10030, 0x000011D7}, + {0x10030, 0x0000159F}, + {0x10030, 0x00001999}, + {0x10030, 0x00001D5F}, + {0x10030, 0x00002159}, + {0x10030, 0x0000251F}, + {0x10030, 0x00002919}, + {0x10030, 0x00002CDF}, + {0x10030, 0x000030D9}, + {0x10030, 0x0000349F}, + {0x10030, 0x00003899}, + {0x10030, 0x00003C5F}, + {0x10030, 0x00004059}, + {0x10030, 0x00004453}, + {0x10030, 0x000201EF}, + {0x10030, 0x000205E9}, + {0x10030, 0x000209E3}, + {0x10030, 0x00020DA3}, + {0x10030, 0x00021161}, + {0x10030, 0x0002155B}, + {0x10030, 0x0002191F}, + {0x10030, 0x00021D19}, + {0x10030, 0x000220E1}, + {0x10030, 0x000224DB}, + {0x10030, 0x000228A3}, + {0x10030, 0x00022C9D}, + {0x10030, 0x00023063}, + {0x10030, 0x0002345D}, + {0x10030, 0x00023823}, + {0x10030, 0x00023C1B}, + {0x10030, 0x00024015}, + {0x10030, 0x0002440F}, + {0x10030, 0x000281EF}, + {0x10030, 0x000285E7}, + {0x10030, 0x000289A7}, + {0x10030, 0x00028D65}, + {0x10030, 0x0002915F}, + {0x10030, 0x00029523}, + {0x10030, 0x0002991D}, + {0x10030, 0x00029CE5}, + {0x10030, 0x0002A0DF}, + {0x10030, 0x0002A4A7}, + {0x10030, 0x0002A8A1}, + {0x10030, 0x0002AC67}, + {0x10030, 0x0002B061}, + {0x10030, 0x0002B427}, + {0x10030, 0x0002B821}, + {0x10030, 0x0002BC19}, + {0x10030, 0x0002C013}, + {0x10030, 0x0002C40D}, + {0x10030, 0x000301EF}, + {0x10030, 0x000305E7}, + {0x10030, 0x000309A7}, + {0x10030, 0x00030D65}, + {0x10030, 0x0003115F}, + {0x10030, 0x00031525}, + {0x10030, 0x0003191F}, + {0x10030, 0x00031CE7}, + {0x10030, 0x000320E1}, + {0x10030, 0x000324A9}, + {0x10030, 0x000328A3}, + {0x10030, 0x00032C69}, + {0x10030, 0x00033063}, + {0x10030, 0x00033429}, + {0x10030, 0x00033823}, + {0x10030, 0x00033C1D}, + {0x10030, 0x00034013}, + {0x10030, 0x0003440D}, + {0x10030, 0x000601F1}, + {0x10030, 0x000605E9}, + {0x10030, 0x000609A9}, + {0x10030, 0x00060D65}, + {0x10030, 0x0006115F}, + {0x10030, 0x00061525}, + {0x10030, 0x0006191F}, + {0x10030, 0x00061CE7}, + {0x10030, 0x000620E1}, + {0x10030, 0x000624DB}, + {0x10030, 0x000628A3}, + {0x10030, 0x00062C69}, + {0x10030, 0x00063063}, + {0x10030, 0x00063429}, + {0x10030, 0x00063823}, + {0x10030, 0x00063C1D}, + {0x10030, 0x00064013}, + {0x10030, 0x0006440D}, + {0x10030, 0x000681EF}, + {0x10030, 0x000685E7}, + {0x10030, 0x000689A7}, + {0x10030, 0x00068D61}, + {0x10030, 0x0006915B}, + {0x10030, 0x00069523}, + {0x10030, 0x0006991D}, + {0x10030, 0x00069CE5}, + {0x10030, 0x0006A0DF}, + {0x10030, 0x0006A4A7}, + {0x10030, 0x0006A8A1}, + {0x10030, 0x0006AC67}, + {0x10030, 0x0006B061}, + {0x10030, 0x0006B429}, + {0x10030, 0x0006B823}, + {0x10030, 0x0006BC1D}, + {0x10030, 0x0006C017}, + {0x10030, 0x0006C40D}, + {0x10030, 0x000701EF}, + {0x10030, 0x000705E7}, + {0x10030, 0x000709A7}, + {0x10030, 0x00070D61}, + {0x10030, 0x0007115B}, + {0x10030, 0x00071523}, + {0x10030, 0x0007191D}, + {0x10030, 0x00071D17}, + {0x10030, 0x000720DF}, + {0x10030, 0x000724D9}, + {0x10030, 0x000728A1}, + {0x10030, 0x00072C67}, + {0x10030, 0x00073061}, + {0x10030, 0x00073427}, + {0x10030, 0x00073821}, + {0x10030, 0x00073C1B}, + {0x10030, 0x00074015}, + {0x10030, 0x0007440D}, + {0x10030, 0x000781EF}, + {0x10030, 0x000785E9}, + {0x10030, 0x000789E3}, + {0x10030, 0x00078DA1}, + {0x10030, 0x0007915F}, + {0x10030, 0x00079559}, + {0x10030, 0x00079921}, + {0x10030, 0x00079D1B}, + {0x10030, 0x0007A0E3}, + {0x10030, 0x0007A4DD}, + {0x10030, 0x0007A8D7}, + {0x10030, 0x0007AC9D}, + {0x10030, 0x0007B063}, + {0x10030, 0x0007B45D}, + {0x10030, 0x0007B823}, + {0x10030, 0x0007BC1D}, + {0x10030, 0x0007C017}, {0x10030, 0x0007C40F}, {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, {0x10030, 0x000001EF}, @@ -5561,1002 +8685,1002 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x10030, 0x00003C5F}, {0x10030, 0x00004059}, {0x10030, 0x00004453}, - {0x10030, 0x000201ED}, - {0x10030, 0x000205AD}, - {0x10030, 0x000209A7}, - {0x10030, 0x00020DA1}, - {0x10030, 0x0002119B}, - {0x10030, 0x00021561}, - {0x10030, 0x0002195B}, - {0x10030, 0x00021D27}, - {0x10030, 0x00022121}, - {0x10030, 0x000224E9}, - {0x10030, 0x000228E3}, - {0x10030, 0x00022CA9}, - {0x10030, 0x000230A3}, - {0x10030, 0x00023469}, - {0x10030, 0x00023863}, - {0x10030, 0x00023C29}, - {0x10030, 0x00024023}, - {0x10030, 0x0002441D}, + {0x10030, 0x000201EF}, + {0x10030, 0x000205E9}, + {0x10030, 0x000209E3}, + {0x10030, 0x00020DA3}, + {0x10030, 0x00021161}, + {0x10030, 0x0002155B}, + {0x10030, 0x0002191F}, + {0x10030, 0x00021D19}, + {0x10030, 0x000220E1}, + {0x10030, 0x000224DB}, + {0x10030, 0x000228A3}, + {0x10030, 0x00022C9D}, + {0x10030, 0x00023063}, + {0x10030, 0x0002345D}, + {0x10030, 0x00023823}, + {0x10030, 0x00023C1B}, + {0x10030, 0x00024015}, + {0x10030, 0x0002440F}, {0x10030, 0x000281EF}, - {0x10030, 0x000285AF}, - {0x10030, 0x000289A9}, - {0x10030, 0x00028DA3}, - {0x10030, 0x0002919D}, - {0x10030, 0x00029563}, - {0x10030, 0x0002995D}, - {0x10030, 0x00029D25}, - {0x10030, 0x0002A11F}, - {0x10030, 0x0002A4E7}, - {0x10030, 0x0002A8E1}, - {0x10030, 0x0002ACA7}, - {0x10030, 0x0002B0A1}, - {0x10030, 0x0002B467}, - {0x10030, 0x0002B861}, - {0x10030, 0x0002BC27}, - {0x10030, 0x0002C021}, - {0x10030, 0x0002C41B}, + {0x10030, 0x000285E7}, + {0x10030, 0x000289A7}, + {0x10030, 0x00028D65}, + {0x10030, 0x0002915F}, + {0x10030, 0x00029523}, + {0x10030, 0x0002991D}, + {0x10030, 0x00029CE5}, + {0x10030, 0x0002A0DF}, + {0x10030, 0x0002A4A7}, + {0x10030, 0x0002A8A1}, + {0x10030, 0x0002AC67}, + {0x10030, 0x0002B061}, + {0x10030, 0x0002B427}, + {0x10030, 0x0002B821}, + {0x10030, 0x0002BC19}, + {0x10030, 0x0002C013}, + {0x10030, 0x0002C40D}, {0x10030, 0x000301EF}, - {0x10030, 0x000305AF}, - {0x10030, 0x000309A9}, - {0x10030, 0x00030DA3}, - {0x10030, 0x0003119D}, - {0x10030, 0x00031563}, - {0x10030, 0x0003195D}, - {0x10030, 0x00031D25}, - {0x10030, 0x0003211F}, - {0x10030, 0x000324E7}, - {0x10030, 0x000328E1}, - {0x10030, 0x00032CA7}, - {0x10030, 0x000330A1}, - {0x10030, 0x00033467}, - {0x10030, 0x00033861}, - {0x10030, 0x00033C27}, - {0x10030, 0x00034021}, - {0x10030, 0x0003441B}, - {0x10030, 0x000601EB}, - {0x10030, 0x000605AB}, - {0x10030, 0x000609A5}, - {0x10030, 0x00060D9F}, - {0x10030, 0x00061199}, - {0x10030, 0x00061593}, - {0x10030, 0x00061959}, - {0x10030, 0x00061D53}, - {0x10030, 0x0006211B}, - {0x10030, 0x00062515}, - {0x10030, 0x000628DD}, - {0x10030, 0x00062CD7}, - {0x10030, 0x0006309D}, - {0x10030, 0x00063497}, - {0x10030, 0x0006385D}, - {0x10030, 0x00063C57}, - {0x10030, 0x0006401D}, - {0x10030, 0x00064417}, - {0x10030, 0x000681E7}, - {0x10030, 0x000685A7}, - {0x10030, 0x000689A1}, - {0x10030, 0x00068D9B}, - {0x10030, 0x00069195}, - {0x10030, 0x0006955F}, - {0x10030, 0x00069959}, - {0x10030, 0x00069D21}, - {0x10030, 0x0006A11B}, - {0x10030, 0x0006A4E3}, - {0x10030, 0x0006A8DD}, - {0x10030, 0x0006ACA5}, - {0x10030, 0x0006B09F}, - {0x10030, 0x0006B465}, - {0x10030, 0x0006B85F}, - {0x10030, 0x0006BC25}, - {0x10030, 0x0006C01F}, - {0x10030, 0x0006C419}, - {0x10030, 0x000701E7}, - {0x10030, 0x000705A7}, - {0x10030, 0x000709A1}, - {0x10030, 0x00070D9B}, - {0x10030, 0x00071195}, - {0x10030, 0x0007155B}, - {0x10030, 0x00071955}, - {0x10030, 0x00071D1D}, - {0x10030, 0x00072117}, - {0x10030, 0x000724DF}, - {0x10030, 0x000728D9}, - {0x10030, 0x00072CA1}, - {0x10030, 0x0007309B}, - {0x10030, 0x00073461}, - {0x10030, 0x0007385B}, - {0x10030, 0x00073C21}, - {0x10030, 0x0007401B}, - {0x10030, 0x0007441B}, + {0x10030, 0x000305E7}, + {0x10030, 0x000309A7}, + {0x10030, 0x00030D65}, + {0x10030, 0x0003115F}, + {0x10030, 0x00031525}, + {0x10030, 0x0003191F}, + {0x10030, 0x00031CE7}, + {0x10030, 0x000320E1}, + {0x10030, 0x000324A9}, + {0x10030, 0x000328A3}, + {0x10030, 0x00032C69}, + {0x10030, 0x00033063}, + {0x10030, 0x00033429}, + {0x10030, 0x00033823}, + {0x10030, 0x00033C1D}, + {0x10030, 0x00034013}, + {0x10030, 0x0003440D}, + {0x10030, 0x000601F1}, + {0x10030, 0x000605E9}, + {0x10030, 0x000609A9}, + {0x10030, 0x00060D65}, + {0x10030, 0x0006115F}, + {0x10030, 0x00061525}, + {0x10030, 0x0006191F}, + {0x10030, 0x00061CE7}, + {0x10030, 0x000620E1}, + {0x10030, 0x000624DB}, + {0x10030, 0x000628A3}, + {0x10030, 0x00062C69}, + {0x10030, 0x00063063}, + {0x10030, 0x00063429}, + {0x10030, 0x00063823}, + {0x10030, 0x00063C1D}, + {0x10030, 0x00064013}, + {0x10030, 0x0006440D}, + {0x10030, 0x000681EF}, + {0x10030, 0x000685E7}, + {0x10030, 0x000689A7}, + {0x10030, 0x00068D61}, + {0x10030, 0x0006915B}, + {0x10030, 0x00069523}, + {0x10030, 0x0006991D}, + {0x10030, 0x00069CE5}, + {0x10030, 0x0006A0DF}, + {0x10030, 0x0006A4A7}, + {0x10030, 0x0006A8A1}, + {0x10030, 0x0006AC67}, + {0x10030, 0x0006B061}, + {0x10030, 0x0006B429}, + {0x10030, 0x0006B823}, + {0x10030, 0x0006BC1D}, + {0x10030, 0x0006C017}, + {0x10030, 0x0006C40D}, + {0x10030, 0x000701EF}, + {0x10030, 0x000705E7}, + {0x10030, 0x000709A7}, + {0x10030, 0x00070D61}, + {0x10030, 0x0007115B}, + {0x10030, 0x00071523}, + {0x10030, 0x0007191D}, + {0x10030, 0x00071D17}, + {0x10030, 0x000720DF}, + {0x10030, 0x000724D9}, + {0x10030, 0x000728A1}, + {0x10030, 0x00072C67}, + {0x10030, 0x00073061}, + {0x10030, 0x00073427}, + {0x10030, 0x00073821}, + {0x10030, 0x00073C1B}, + {0x10030, 0x00074015}, + {0x10030, 0x0007440D}, {0x10030, 0x000781EF}, {0x10030, 0x000785E9}, {0x10030, 0x000789E3}, - {0x10030, 0x00078DA3}, - {0x10030, 0x00079161}, - {0x10030, 0x0007955B}, + {0x10030, 0x00078DA1}, + {0x10030, 0x0007915F}, + {0x10030, 0x00079559}, {0x10030, 0x00079921}, {0x10030, 0x00079D1B}, - {0x10030, 0x0007A0E1}, - {0x10030, 0x0007A4DB}, - {0x10030, 0x0007A8A1}, - {0x10030, 0x0007AC9B}, - {0x10030, 0x0007B061}, - {0x10030, 0x0007B45B}, - {0x10030, 0x0007B821}, - {0x10030, 0x0007BC1B}, - {0x10030, 0x0007C015}, + {0x10030, 0x0007A0E3}, + {0x10030, 0x0007A4DD}, + {0x10030, 0x0007A8D7}, + {0x10030, 0x0007AC9D}, + {0x10030, 0x0007B063}, + {0x10030, 0x0007B45D}, + {0x10030, 0x0007B823}, + {0x10030, 0x0007BC1D}, + {0x10030, 0x0007C017}, {0x10030, 0x0007C40F}, {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, - {0x10030, 0x000201DF}, - {0x10030, 0x000205D9}, - {0x10030, 0x000209D3}, - {0x10030, 0x00020D99}, - {0x10030, 0x00021193}, - {0x10030, 0x0002155F}, - {0x10030, 0x00021959}, - {0x10030, 0x00021D21}, - {0x10030, 0x00022119}, - {0x10030, 0x000224DF}, - {0x10030, 0x000228D9}, - {0x10030, 0x00022C9F}, - {0x10030, 0x00023099}, - {0x10030, 0x0002345F}, - {0x10030, 0x00023859}, - {0x10030, 0x00023C1F}, - {0x10030, 0x00024019}, - {0x10030, 0x00024413}, - {0x10030, 0x000281CD}, - {0x10030, 0x000285DB}, - {0x10030, 0x000289D5}, - {0x10030, 0x00028D9B}, - {0x10030, 0x0002918D}, - {0x10030, 0x00029555}, - {0x10030, 0x00029957}, - {0x10030, 0x00029D1F}, - {0x10030, 0x0002A119}, - {0x10030, 0x0002A4DF}, - {0x10030, 0x0002A8D9}, - {0x10030, 0x0002AC9F}, - {0x10030, 0x0002B099}, - {0x10030, 0x0002B45F}, - {0x10030, 0x0002B859}, - {0x10030, 0x0002BC1F}, - {0x10030, 0x0002C019}, - {0x10030, 0x0002C413}, - {0x10030, 0x000301D9}, - {0x10030, 0x000305DB}, - {0x10030, 0x000309D5}, - {0x10030, 0x00030D9B}, - {0x10030, 0x00031195}, - {0x10030, 0x0003155D}, - {0x10030, 0x00031955}, - {0x10030, 0x00031D1D}, - {0x10030, 0x00032119}, - {0x10030, 0x000324DF}, - {0x10030, 0x000328D9}, - {0x10030, 0x00032C9F}, - {0x10030, 0x00033099}, - {0x10030, 0x0003345F}, - {0x10030, 0x00033859}, - {0x10030, 0x00033C1F}, - {0x10030, 0x00034019}, - {0x10030, 0x00034413}, - {0x10030, 0x000601E1}, - {0x10030, 0x000605DB}, - {0x10030, 0x000609D5}, - {0x10030, 0x00060D9B}, - {0x10030, 0x00061195}, - {0x10030, 0x0006155B}, - {0x10030, 0x00061957}, - {0x10030, 0x00061D1F}, - {0x10030, 0x00062119}, - {0x10030, 0x000624DF}, - {0x10030, 0x000628D9}, - {0x10030, 0x00062C9F}, - {0x10030, 0x00063099}, - {0x10030, 0x0006345F}, - {0x10030, 0x00063859}, - {0x10030, 0x00063C1F}, - {0x10030, 0x00064019}, - {0x10030, 0x00064413}, - {0x10030, 0x000681E1}, - {0x10030, 0x000685DB}, - {0x10030, 0x000689D5}, - {0x10030, 0x00068D9B}, - {0x10030, 0x00069195}, - {0x10030, 0x0006955B}, - {0x10030, 0x00069957}, - {0x10030, 0x00069D1F}, - {0x10030, 0x0006A119}, - {0x10030, 0x0006A4DF}, - {0x10030, 0x0006A8D9}, - {0x10030, 0x0006AC9F}, - {0x10030, 0x0006B099}, - {0x10030, 0x0006B45F}, - {0x10030, 0x0006B859}, - {0x10030, 0x0006BC1F}, - {0x10030, 0x0006C019}, - {0x10030, 0x0006C413}, - {0x10030, 0x000701E1}, - {0x10030, 0x000705DB}, - {0x10030, 0x000709D5}, - {0x10030, 0x00070D9B}, - {0x10030, 0x00071195}, - {0x10030, 0x0007155B}, - {0x10030, 0x00071957}, - {0x10030, 0x00071D1F}, - {0x10030, 0x00072119}, - {0x10030, 0x000724DF}, - {0x10030, 0x000728D9}, - {0x10030, 0x00072C9F}, - {0x10030, 0x00073099}, - {0x10030, 0x0007345F}, - {0x10030, 0x00073859}, - {0x10030, 0x00073C1F}, - {0x10030, 0x00074019}, - {0x10030, 0x00074413}, - {0x10030, 0x000781DF}, - {0x10030, 0x000785D9}, - {0x10030, 0x000789D3}, - {0x10030, 0x00078D99}, - {0x10030, 0x00079193}, - {0x10030, 0x0007955F}, - {0x10030, 0x00079959}, - {0x10030, 0x00079D21}, - {0x10030, 0x0007A115}, - {0x10030, 0x0007A4DF}, - {0x10030, 0x0007A8D9}, - {0x10030, 0x0007AC9F}, - {0x10030, 0x0007B099}, - {0x10030, 0x0007B45F}, - {0x10030, 0x0007B859}, - {0x10030, 0x0007BC1F}, - {0x10030, 0x0007C019}, - {0x10030, 0x0007C413}, - {0x10030, 0x00000000}, - {0x10030, 0x000785A9}, - {0x10030, 0x000789A3}, - {0x10030, 0x00078D9D}, - {0x10030, 0x00079197}, - {0x10030, 0x00079591}, - {0x10030, 0x00079957}, - {0x10030, 0x00079D51}, - {0x10030, 0x0007A119}, - {0x10030, 0x0007A513}, - {0x10030, 0x0007A8D9}, - {0x10030, 0x0007ACD3}, - {0x10030, 0x0007B099}, - {0x10030, 0x0007B493}, - {0x10030, 0x0007B859}, - {0x10030, 0x0007BC53}, - {0x10030, 0x0007C019}, - {0x10030, 0x0007C413}, + {0x10030, 0x000001EF}, + {0x10030, 0x000005E9}, + {0x10030, 0x000009E3}, + {0x10030, 0x00000DDD}, + {0x10030, 0x000011D7}, + {0x10030, 0x0000159F}, + {0x10030, 0x00001999}, + {0x10030, 0x00001D5F}, + {0x10030, 0x00002159}, + {0x10030, 0x0000251F}, + {0x10030, 0x00002919}, + {0x10030, 0x00002CDF}, + {0x10030, 0x000030D9}, + {0x10030, 0x0000349F}, + {0x10030, 0x00003899}, + {0x10030, 0x00003C5F}, + {0x10030, 0x00004059}, + {0x10030, 0x00004453}, + {0x10030, 0x000201A7}, + {0x10030, 0x000205A1}, + {0x10030, 0x0002099B}, + {0x10030, 0x00020D95}, + {0x10030, 0x0002115B}, + {0x10030, 0x00021555}, + {0x10030, 0x00021921}, + {0x10030, 0x00021D1B}, + {0x10030, 0x000220E3}, + {0x10030, 0x000224DD}, + {0x10030, 0x000228A3}, + {0x10030, 0x00022C9D}, + {0x10030, 0x00023063}, + {0x10030, 0x0002345D}, + {0x10030, 0x00023823}, + {0x10030, 0x00023C1D}, + {0x10030, 0x00024017}, + {0x10030, 0x00024411}, + {0x10030, 0x000281A9}, + {0x10030, 0x000285A3}, + {0x10030, 0x0002899D}, + {0x10030, 0x00028D97}, + {0x10030, 0x0002915D}, + {0x10030, 0x00029557}, + {0x10030, 0x0002991F}, + {0x10030, 0x00029D19}, + {0x10030, 0x0002A0E1}, + {0x10030, 0x0002A4DB}, + {0x10030, 0x0002A8A1}, + {0x10030, 0x0002AC9B}, + {0x10030, 0x0002B061}, + {0x10030, 0x0002B45B}, + {0x10030, 0x0002B821}, + {0x10030, 0x0002BC1B}, + {0x10030, 0x0002C015}, + {0x10030, 0x0002C40F}, + {0x10030, 0x000301A9}, + {0x10030, 0x000305A3}, + {0x10030, 0x0003099D}, + {0x10030, 0x00030D97}, + {0x10030, 0x0003115D}, + {0x10030, 0x00031557}, + {0x10030, 0x0003191F}, + {0x10030, 0x00031D19}, + {0x10030, 0x000320E1}, + {0x10030, 0x000324DB}, + {0x10030, 0x000328A1}, + {0x10030, 0x00032C9B}, + {0x10030, 0x00033061}, + {0x10030, 0x0003345B}, + {0x10030, 0x00033821}, + {0x10030, 0x00033C1B}, + {0x10030, 0x00034015}, + {0x10030, 0x0003440F}, + {0x10030, 0x000601F1}, + {0x10030, 0x000605E9}, + {0x10030, 0x000609A9}, + {0x10030, 0x00060D65}, + {0x10030, 0x0006115F}, + {0x10030, 0x00061525}, + {0x10030, 0x0006191F}, + {0x10030, 0x00061CE7}, + {0x10030, 0x000620E1}, + {0x10030, 0x000624DB}, + {0x10030, 0x000628A3}, + {0x10030, 0x00062C69}, + {0x10030, 0x00063063}, + {0x10030, 0x00063429}, + {0x10030, 0x00063823}, + {0x10030, 0x00063C1D}, + {0x10030, 0x00064013}, + {0x10030, 0x0006440D}, + {0x10030, 0x000681EF}, + {0x10030, 0x000685E7}, + {0x10030, 0x000689A7}, + {0x10030, 0x00068D61}, + {0x10030, 0x0006915B}, + {0x10030, 0x00069523}, + {0x10030, 0x0006991D}, + {0x10030, 0x00069CE5}, + {0x10030, 0x0006A0DF}, + {0x10030, 0x0006A4A7}, + {0x10030, 0x0006A8A1}, + {0x10030, 0x0006AC67}, + {0x10030, 0x0006B061}, + {0x10030, 0x0006B429}, + {0x10030, 0x0006B823}, + {0x10030, 0x0006BC1D}, + {0x10030, 0x0006C017}, + {0x10030, 0x0006C40D}, + {0x10030, 0x000701EF}, + {0x10030, 0x000705E7}, + {0x10030, 0x000709A7}, + {0x10030, 0x00070D61}, + {0x10030, 0x0007115B}, + {0x10030, 0x00071523}, + {0x10030, 0x0007191D}, + {0x10030, 0x00071CE5}, + {0x10030, 0x000720DF}, + {0x10030, 0x000724D9}, + {0x10030, 0x000728A1}, + {0x10030, 0x00072C67}, + {0x10030, 0x00073061}, + {0x10030, 0x00073427}, + {0x10030, 0x00073821}, + {0x10030, 0x00073C1B}, + {0x10030, 0x00074015}, + {0x10030, 0x0007440D}, + {0x10030, 0x000781EF}, + {0x10030, 0x000785E9}, + {0x10030, 0x000789E3}, + {0x10030, 0x00078DA1}, + {0x10030, 0x0007915F}, + {0x10030, 0x00079559}, + {0x10030, 0x00079921}, + {0x10030, 0x00079D1B}, + {0x10030, 0x0007A0E3}, + {0x10030, 0x0007A4DD}, + {0x10030, 0x0007A8D7}, + {0x10030, 0x0007AC9D}, + {0x10030, 0x0007B063}, + {0x10030, 0x0007B45D}, + {0x10030, 0x0007B823}, + {0x10030, 0x0007BC1D}, + {0x10030, 0x0007C017}, + {0x10030, 0x0007C40F}, {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, - {0x10030, 0x000201DF}, - {0x10030, 0x000205D9}, - {0x10030, 0x000209D3}, - {0x10030, 0x00020D99}, - {0x10030, 0x00021193}, - {0x10030, 0x0002155F}, - {0x10030, 0x00021959}, - {0x10030, 0x00021D21}, - {0x10030, 0x00022119}, - {0x10030, 0x000224DF}, - {0x10030, 0x000228D9}, - {0x10030, 0x00022C9F}, - {0x10030, 0x00023099}, - {0x10030, 0x0002345F}, - {0x10030, 0x00023859}, - {0x10030, 0x00023C1F}, - {0x10030, 0x00024019}, - {0x10030, 0x00024413}, - {0x10030, 0x000281CD}, - {0x10030, 0x000285DB}, - {0x10030, 0x000289D5}, - {0x10030, 0x00028D9B}, - {0x10030, 0x0002918D}, - {0x10030, 0x00029555}, - {0x10030, 0x00029957}, - {0x10030, 0x00029D1F}, - {0x10030, 0x0002A119}, - {0x10030, 0x0002A4DF}, - {0x10030, 0x0002A8D9}, - {0x10030, 0x0002AC9F}, - {0x10030, 0x0002B099}, - {0x10030, 0x0002B45F}, - {0x10030, 0x0002B859}, - {0x10030, 0x0002BC1F}, - {0x10030, 0x0002C019}, - {0x10030, 0x0002C413}, - {0x10030, 0x000301D9}, - {0x10030, 0x000305DB}, - {0x10030, 0x000309D5}, - {0x10030, 0x00030D9B}, - {0x10030, 0x00031195}, - {0x10030, 0x0003155D}, - {0x10030, 0x00031955}, - {0x10030, 0x00031D1D}, - {0x10030, 0x00032119}, - {0x10030, 0x000324DF}, - {0x10030, 0x000328D9}, - {0x10030, 0x00032C9F}, - {0x10030, 0x00033099}, - {0x10030, 0x0003345F}, - {0x10030, 0x00033859}, - {0x10030, 0x00033C1F}, - {0x10030, 0x00034019}, - {0x10030, 0x00034413}, - {0x10030, 0x000601E1}, - {0x10030, 0x000605DB}, - {0x10030, 0x000609D5}, - {0x10030, 0x00060D9B}, - {0x10030, 0x00061195}, - {0x10030, 0x0006155B}, - {0x10030, 0x00061957}, - {0x10030, 0x00061D1F}, - {0x10030, 0x00062119}, - {0x10030, 0x000624DF}, - {0x10030, 0x000628D9}, - {0x10030, 0x00062C9F}, - {0x10030, 0x00063099}, - {0x10030, 0x0006345F}, - {0x10030, 0x00063859}, - {0x10030, 0x00063C1F}, - {0x10030, 0x00064019}, - {0x10030, 0x00064413}, - {0x10030, 0x000681E1}, - {0x10030, 0x000685DB}, - {0x10030, 0x000689D5}, - {0x10030, 0x00068D9B}, - {0x10030, 0x00069195}, - {0x10030, 0x0006955B}, - {0x10030, 0x00069957}, - {0x10030, 0x00069D1F}, - {0x10030, 0x0006A119}, - {0x10030, 0x0006A4DF}, - {0x10030, 0x0006A8D9}, - {0x10030, 0x0006AC9F}, - {0x10030, 0x0006B099}, - {0x10030, 0x0006B45F}, - {0x10030, 0x0006B859}, - {0x10030, 0x0006BC1F}, - {0x10030, 0x0006C019}, - {0x10030, 0x0006C413}, - {0x10030, 0x000701E1}, - {0x10030, 0x000705DB}, - {0x10030, 0x000709D5}, - {0x10030, 0x00070D9B}, - {0x10030, 0x00071195}, - {0x10030, 0x0007155B}, - {0x10030, 0x00071957}, - {0x10030, 0x00071D1F}, - {0x10030, 0x00072119}, - {0x10030, 0x000724DF}, - {0x10030, 0x000728D9}, - {0x10030, 0x00072C9F}, - {0x10030, 0x00073099}, - {0x10030, 0x0007345F}, - {0x10030, 0x00073859}, - {0x10030, 0x00073C1F}, - {0x10030, 0x00074019}, - {0x10030, 0x00074413}, - {0x10030, 0x000781DF}, - {0x10030, 0x000785D9}, - {0x10030, 0x000789D3}, - {0x10030, 0x00078D99}, - {0x10030, 0x00079193}, - {0x10030, 0x0007955F}, - {0x10030, 0x00079959}, - {0x10030, 0x00079D21}, - {0x10030, 0x0007A115}, - {0x10030, 0x0007A4DF}, - {0x10030, 0x0007A8D9}, - {0x10030, 0x0007AC9F}, - {0x10030, 0x0007B099}, - {0x10030, 0x0007B45F}, - {0x10030, 0x0007B859}, - {0x10030, 0x0007BC1F}, - {0x10030, 0x0007C019}, - {0x10030, 0x0007C413}, - {0x10030, 0x00000000}, - {0x10030, 0x000785A9}, - {0x10030, 0x000789A3}, - {0x10030, 0x00078D9D}, - {0x10030, 0x00079197}, - {0x10030, 0x00079591}, - {0x10030, 0x00079957}, - {0x10030, 0x00079D51}, - {0x10030, 0x0007A119}, - {0x10030, 0x0007A513}, - {0x10030, 0x0007A8D9}, - {0x10030, 0x0007ACD3}, - {0x10030, 0x0007B099}, - {0x10030, 0x0007B493}, - {0x10030, 0x0007B859}, - {0x10030, 0x0007BC53}, - {0x10030, 0x0007C019}, - {0x10030, 0x0007C413}, + {0x10030, 0x000001EF}, + {0x10030, 0x000005E9}, + {0x10030, 0x000009E3}, + {0x10030, 0x00000DDD}, + {0x10030, 0x000011D7}, + {0x10030, 0x0000159F}, + {0x10030, 0x00001999}, + {0x10030, 0x00001D5F}, + {0x10030, 0x00002159}, + {0x10030, 0x0000251F}, + {0x10030, 0x00002919}, + {0x10030, 0x00002CDF}, + {0x10030, 0x000030D9}, + {0x10030, 0x0000349F}, + {0x10030, 0x00003899}, + {0x10030, 0x00003C5F}, + {0x10030, 0x00004059}, + {0x10030, 0x00004453}, + {0x10030, 0x000201A7}, + {0x10030, 0x000205A1}, + {0x10030, 0x0002099B}, + {0x10030, 0x00020D95}, + {0x10030, 0x0002115B}, + {0x10030, 0x00021555}, + {0x10030, 0x00021921}, + {0x10030, 0x00021D1B}, + {0x10030, 0x000220E3}, + {0x10030, 0x000224DD}, + {0x10030, 0x000228A3}, + {0x10030, 0x00022C9D}, + {0x10030, 0x00023063}, + {0x10030, 0x0002345D}, + {0x10030, 0x00023823}, + {0x10030, 0x00023C1D}, + {0x10030, 0x00024017}, + {0x10030, 0x00024411}, + {0x10030, 0x000281A9}, + {0x10030, 0x000285A3}, + {0x10030, 0x0002899D}, + {0x10030, 0x00028D97}, + {0x10030, 0x0002915D}, + {0x10030, 0x00029557}, + {0x10030, 0x0002991F}, + {0x10030, 0x00029D19}, + {0x10030, 0x0002A0E1}, + {0x10030, 0x0002A4DB}, + {0x10030, 0x0002A8A1}, + {0x10030, 0x0002AC9B}, + {0x10030, 0x0002B061}, + {0x10030, 0x0002B45B}, + {0x10030, 0x0002B821}, + {0x10030, 0x0002BC1B}, + {0x10030, 0x0002C015}, + {0x10030, 0x0002C40F}, + {0x10030, 0x000301A9}, + {0x10030, 0x000305A3}, + {0x10030, 0x0003099D}, + {0x10030, 0x00030D97}, + {0x10030, 0x0003115D}, + {0x10030, 0x00031557}, + {0x10030, 0x0003191F}, + {0x10030, 0x00031D19}, + {0x10030, 0x000320E1}, + {0x10030, 0x000324DB}, + {0x10030, 0x000328A1}, + {0x10030, 0x00032C9B}, + {0x10030, 0x00033061}, + {0x10030, 0x0003345B}, + {0x10030, 0x00033821}, + {0x10030, 0x00033C1B}, + {0x10030, 0x00034015}, + {0x10030, 0x0003440F}, + {0x10030, 0x000601F1}, + {0x10030, 0x000605E9}, + {0x10030, 0x000609A9}, + {0x10030, 0x00060D65}, + {0x10030, 0x0006115F}, + {0x10030, 0x00061525}, + {0x10030, 0x0006191F}, + {0x10030, 0x00061CE7}, + {0x10030, 0x000620E1}, + {0x10030, 0x000624DB}, + {0x10030, 0x000628A3}, + {0x10030, 0x00062C69}, + {0x10030, 0x00063063}, + {0x10030, 0x00063429}, + {0x10030, 0x00063823}, + {0x10030, 0x00063C1D}, + {0x10030, 0x00064013}, + {0x10030, 0x0006440D}, + {0x10030, 0x000681EF}, + {0x10030, 0x000685E7}, + {0x10030, 0x000689A7}, + {0x10030, 0x00068D61}, + {0x10030, 0x0006915B}, + {0x10030, 0x00069523}, + {0x10030, 0x0006991D}, + {0x10030, 0x00069CE5}, + {0x10030, 0x0006A0DF}, + {0x10030, 0x0006A4A7}, + {0x10030, 0x0006A8A1}, + {0x10030, 0x0006AC67}, + {0x10030, 0x0006B061}, + {0x10030, 0x0006B429}, + {0x10030, 0x0006B823}, + {0x10030, 0x0006BC1D}, + {0x10030, 0x0006C017}, + {0x10030, 0x0006C40D}, + {0x10030, 0x000701EF}, + {0x10030, 0x000705E7}, + {0x10030, 0x000709A7}, + {0x10030, 0x00070D61}, + {0x10030, 0x0007115B}, + {0x10030, 0x00071523}, + {0x10030, 0x0007191D}, + {0x10030, 0x00071CE5}, + {0x10030, 0x000720DF}, + {0x10030, 0x000724D9}, + {0x10030, 0x000728A1}, + {0x10030, 0x00072C67}, + {0x10030, 0x00073061}, + {0x10030, 0x00073427}, + {0x10030, 0x00073821}, + {0x10030, 0x00073C1B}, + {0x10030, 0x00074015}, + {0x10030, 0x0007440D}, + {0x10030, 0x000781EF}, + {0x10030, 0x000785E9}, + {0x10030, 0x000789E3}, + {0x10030, 0x00078DA1}, + {0x10030, 0x0007915F}, + {0x10030, 0x00079559}, + {0x10030, 0x00079921}, + {0x10030, 0x00079D1B}, + {0x10030, 0x0007A0E3}, + {0x10030, 0x0007A4DD}, + {0x10030, 0x0007A8D7}, + {0x10030, 0x0007AC9D}, + {0x10030, 0x0007B063}, + {0x10030, 0x0007B45D}, + {0x10030, 0x0007B823}, + {0x10030, 0x0007BC1D}, + {0x10030, 0x0007C017}, + {0x10030, 0x0007C40F}, {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, - {0x10030, 0x000201DF}, - {0x10030, 0x000205D9}, - {0x10030, 0x000209D3}, - {0x10030, 0x00020D99}, - {0x10030, 0x00021193}, - {0x10030, 0x0002155F}, - {0x10030, 0x00021959}, - {0x10030, 0x00021D21}, - {0x10030, 0x00022119}, - {0x10030, 0x000224DF}, - {0x10030, 0x000228D9}, - {0x10030, 0x00022C9F}, - {0x10030, 0x00023099}, - {0x10030, 0x0002345F}, - {0x10030, 0x00023859}, - {0x10030, 0x00023C1F}, - {0x10030, 0x00024019}, - {0x10030, 0x00024413}, - {0x10030, 0x000281CD}, - {0x10030, 0x000285DB}, - {0x10030, 0x000289D5}, - {0x10030, 0x00028D9B}, - {0x10030, 0x0002918D}, - {0x10030, 0x00029555}, - {0x10030, 0x00029957}, - {0x10030, 0x00029D1F}, - {0x10030, 0x0002A119}, - {0x10030, 0x0002A4DF}, - {0x10030, 0x0002A8D9}, - {0x10030, 0x0002AC9F}, - {0x10030, 0x0002B099}, - {0x10030, 0x0002B45F}, - {0x10030, 0x0002B859}, - {0x10030, 0x0002BC1F}, - {0x10030, 0x0002C019}, - {0x10030, 0x0002C413}, - {0x10030, 0x000301D9}, - {0x10030, 0x000305DB}, - {0x10030, 0x000309D5}, - {0x10030, 0x00030D9B}, - {0x10030, 0x00031195}, - {0x10030, 0x0003155D}, - {0x10030, 0x00031955}, - {0x10030, 0x00031D1D}, - {0x10030, 0x00032119}, - {0x10030, 0x000324DF}, - {0x10030, 0x000328D9}, - {0x10030, 0x00032C9F}, - {0x10030, 0x00033099}, - {0x10030, 0x0003345F}, - {0x10030, 0x00033859}, - {0x10030, 0x00033C1F}, - {0x10030, 0x00034019}, - {0x10030, 0x00034413}, - {0x10030, 0x000601E1}, - {0x10030, 0x000605DB}, - {0x10030, 0x000609D5}, - {0x10030, 0x00060D9B}, - {0x10030, 0x00061195}, - {0x10030, 0x0006155B}, - {0x10030, 0x00061957}, - {0x10030, 0x00061D1F}, - {0x10030, 0x00062119}, - {0x10030, 0x000624DF}, - {0x10030, 0x000628D9}, - {0x10030, 0x00062C9F}, - {0x10030, 0x00063099}, - {0x10030, 0x0006345F}, - {0x10030, 0x00063859}, - {0x10030, 0x00063C1F}, - {0x10030, 0x00064019}, - {0x10030, 0x00064413}, - {0x10030, 0x000681E1}, - {0x10030, 0x000685DB}, - {0x10030, 0x000689D5}, - {0x10030, 0x00068D9B}, - {0x10030, 0x00069195}, - {0x10030, 0x0006955B}, - {0x10030, 0x00069957}, - {0x10030, 0x00069D1F}, - {0x10030, 0x0006A119}, - {0x10030, 0x0006A4DF}, - {0x10030, 0x0006A8D9}, - {0x10030, 0x0006AC9F}, - {0x10030, 0x0006B099}, - {0x10030, 0x0006B45F}, - {0x10030, 0x0006B859}, - {0x10030, 0x0006BC1F}, - {0x10030, 0x0006C019}, - {0x10030, 0x0006C413}, - {0x10030, 0x000701E1}, - {0x10030, 0x000705DB}, - {0x10030, 0x000709D5}, - {0x10030, 0x00070D9B}, - {0x10030, 0x00071195}, - {0x10030, 0x0007155B}, - {0x10030, 0x00071957}, - {0x10030, 0x00071D1F}, - {0x10030, 0x00072119}, - {0x10030, 0x000724DF}, - {0x10030, 0x000728D9}, - {0x10030, 0x00072C9F}, - {0x10030, 0x00073099}, - {0x10030, 0x0007345F}, - {0x10030, 0x00073859}, - {0x10030, 0x00073C1F}, - {0x10030, 0x00074019}, - {0x10030, 0x00074413}, - {0x10030, 0x000781DF}, - {0x10030, 0x000785D9}, - {0x10030, 0x000789D3}, - {0x10030, 0x00078D99}, - {0x10030, 0x00079193}, - {0x10030, 0x0007955F}, - {0x10030, 0x00079959}, - {0x10030, 0x00079D21}, - {0x10030, 0x0007A115}, - {0x10030, 0x0007A4DF}, - {0x10030, 0x0007A8D9}, - {0x10030, 0x0007AC9F}, - {0x10030, 0x0007B099}, - {0x10030, 0x0007B45F}, - {0x10030, 0x0007B859}, - {0x10030, 0x0007BC1F}, - {0x10030, 0x0007C019}, - {0x10030, 0x0007C413}, - {0x10030, 0x00000000}, - {0x10030, 0x000785A9}, - {0x10030, 0x000789A3}, - {0x10030, 0x00078D9D}, - {0x10030, 0x00079197}, - {0x10030, 0x00079591}, - {0x10030, 0x00079957}, - {0x10030, 0x00079D51}, - {0x10030, 0x0007A119}, - {0x10030, 0x0007A513}, - {0x10030, 0x0007A8D9}, - {0x10030, 0x0007ACD3}, - {0x10030, 0x0007B099}, - {0x10030, 0x0007B493}, - {0x10030, 0x0007B859}, - {0x10030, 0x0007BC53}, - {0x10030, 0x0007C019}, - {0x10030, 0x0007C413}, + {0x10030, 0x000001EF}, + {0x10030, 0x000005E9}, + {0x10030, 0x000009E3}, + {0x10030, 0x00000DDD}, + {0x10030, 0x000011D7}, + {0x10030, 0x0000159F}, + {0x10030, 0x00001999}, + {0x10030, 0x00001D5F}, + {0x10030, 0x00002159}, + {0x10030, 0x0000251F}, + {0x10030, 0x00002919}, + {0x10030, 0x00002CDF}, + {0x10030, 0x000030D9}, + {0x10030, 0x0000349F}, + {0x10030, 0x00003899}, + {0x10030, 0x00003C5F}, + {0x10030, 0x00004059}, + {0x10030, 0x00004453}, + {0x10030, 0x000201A7}, + {0x10030, 0x000205A1}, + {0x10030, 0x0002099B}, + {0x10030, 0x00020D95}, + {0x10030, 0x0002115B}, + {0x10030, 0x00021555}, + {0x10030, 0x00021921}, + {0x10030, 0x00021D1B}, + {0x10030, 0x000220E3}, + {0x10030, 0x000224DD}, + {0x10030, 0x000228A3}, + {0x10030, 0x00022C9D}, + {0x10030, 0x00023063}, + {0x10030, 0x0002345D}, + {0x10030, 0x00023823}, + {0x10030, 0x00023C1D}, + {0x10030, 0x00024017}, + {0x10030, 0x00024411}, + {0x10030, 0x000281A9}, + {0x10030, 0x000285A3}, + {0x10030, 0x0002899D}, + {0x10030, 0x00028D97}, + {0x10030, 0x0002915D}, + {0x10030, 0x00029557}, + {0x10030, 0x0002991F}, + {0x10030, 0x00029D19}, + {0x10030, 0x0002A0E1}, + {0x10030, 0x0002A4DB}, + {0x10030, 0x0002A8A1}, + {0x10030, 0x0002AC9B}, + {0x10030, 0x0002B061}, + {0x10030, 0x0002B45B}, + {0x10030, 0x0002B821}, + {0x10030, 0x0002BC1B}, + {0x10030, 0x0002C015}, + {0x10030, 0x0002C40F}, + {0x10030, 0x000301A9}, + {0x10030, 0x000305A3}, + {0x10030, 0x0003099D}, + {0x10030, 0x00030D97}, + {0x10030, 0x0003115D}, + {0x10030, 0x00031557}, + {0x10030, 0x0003191F}, + {0x10030, 0x00031D19}, + {0x10030, 0x000320E1}, + {0x10030, 0x000324DB}, + {0x10030, 0x000328A1}, + {0x10030, 0x00032C9B}, + {0x10030, 0x00033061}, + {0x10030, 0x0003345B}, + {0x10030, 0x00033821}, + {0x10030, 0x00033C1B}, + {0x10030, 0x00034015}, + {0x10030, 0x0003440F}, + {0x10030, 0x000601F1}, + {0x10030, 0x000605E9}, + {0x10030, 0x000609A9}, + {0x10030, 0x00060D65}, + {0x10030, 0x0006115F}, + {0x10030, 0x00061525}, + {0x10030, 0x0006191F}, + {0x10030, 0x00061CE7}, + {0x10030, 0x000620E1}, + {0x10030, 0x000624DB}, + {0x10030, 0x000628A3}, + {0x10030, 0x00062C69}, + {0x10030, 0x00063063}, + {0x10030, 0x00063429}, + {0x10030, 0x00063823}, + {0x10030, 0x00063C1D}, + {0x10030, 0x00064013}, + {0x10030, 0x0006440D}, + {0x10030, 0x000681EF}, + {0x10030, 0x000685E7}, + {0x10030, 0x000689A7}, + {0x10030, 0x00068D61}, + {0x10030, 0x0006915B}, + {0x10030, 0x00069523}, + {0x10030, 0x0006991D}, + {0x10030, 0x00069CE5}, + {0x10030, 0x0006A0DF}, + {0x10030, 0x0006A4A7}, + {0x10030, 0x0006A8A1}, + {0x10030, 0x0006AC67}, + {0x10030, 0x0006B061}, + {0x10030, 0x0006B429}, + {0x10030, 0x0006B823}, + {0x10030, 0x0006BC1D}, + {0x10030, 0x0006C017}, + {0x10030, 0x0006C40D}, + {0x10030, 0x000701EF}, + {0x10030, 0x000705E7}, + {0x10030, 0x000709A7}, + {0x10030, 0x00070D61}, + {0x10030, 0x0007115B}, + {0x10030, 0x00071523}, + {0x10030, 0x0007191D}, + {0x10030, 0x00071CE5}, + {0x10030, 0x000720DF}, + {0x10030, 0x000724D9}, + {0x10030, 0x000728A1}, + {0x10030, 0x00072C67}, + {0x10030, 0x00073061}, + {0x10030, 0x00073427}, + {0x10030, 0x00073821}, + {0x10030, 0x00073C1B}, + {0x10030, 0x00074015}, + {0x10030, 0x0007440D}, + {0x10030, 0x000781EF}, + {0x10030, 0x000785E9}, + {0x10030, 0x000789E3}, + {0x10030, 0x00078DA1}, + {0x10030, 0x0007915F}, + {0x10030, 0x00079559}, + {0x10030, 0x00079921}, + {0x10030, 0x00079D1B}, + {0x10030, 0x0007A0E3}, + {0x10030, 0x0007A4DD}, + {0x10030, 0x0007A8D7}, + {0x10030, 0x0007AC9D}, + {0x10030, 0x0007B063}, + {0x10030, 0x0007B45D}, + {0x10030, 0x0007B823}, + {0x10030, 0x0007BC1D}, + {0x10030, 0x0007C017}, + {0x10030, 0x0007C40F}, {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, - {0x10030, 0x000201DF}, - {0x10030, 0x000205D9}, - {0x10030, 0x000209D3}, - {0x10030, 0x00020D99}, - {0x10030, 0x00021193}, - {0x10030, 0x0002155F}, - {0x10030, 0x00021959}, - {0x10030, 0x00021D21}, - {0x10030, 0x00022119}, - {0x10030, 0x000224DF}, - {0x10030, 0x000228D9}, - {0x10030, 0x00022C9F}, - {0x10030, 0x00023099}, - {0x10030, 0x0002345F}, - {0x10030, 0x00023859}, - {0x10030, 0x00023C1F}, - {0x10030, 0x00024019}, - {0x10030, 0x00024413}, - {0x10030, 0x000281CD}, - {0x10030, 0x000285DB}, - {0x10030, 0x000289D5}, - {0x10030, 0x00028D9B}, - {0x10030, 0x0002918D}, - {0x10030, 0x00029555}, - {0x10030, 0x00029957}, - {0x10030, 0x00029D1F}, - {0x10030, 0x0002A119}, - {0x10030, 0x0002A4DF}, - {0x10030, 0x0002A8D9}, - {0x10030, 0x0002AC9F}, - {0x10030, 0x0002B099}, - {0x10030, 0x0002B45F}, - {0x10030, 0x0002B859}, - {0x10030, 0x0002BC1F}, - {0x10030, 0x0002C019}, - {0x10030, 0x0002C413}, - {0x10030, 0x000301D9}, - {0x10030, 0x000305DB}, - {0x10030, 0x000309D5}, - {0x10030, 0x00030D9B}, - {0x10030, 0x00031195}, - {0x10030, 0x0003155D}, - {0x10030, 0x00031955}, - {0x10030, 0x00031D1D}, - {0x10030, 0x00032119}, - {0x10030, 0x000324DF}, - {0x10030, 0x000328D9}, - {0x10030, 0x00032C9F}, - {0x10030, 0x00033099}, - {0x10030, 0x0003345F}, - {0x10030, 0x00033859}, - {0x10030, 0x00033C1F}, - {0x10030, 0x00034019}, - {0x10030, 0x00034413}, - {0x10030, 0x000601E1}, - {0x10030, 0x000605DB}, - {0x10030, 0x000609D5}, - {0x10030, 0x00060D9B}, - {0x10030, 0x00061195}, - {0x10030, 0x0006155B}, - {0x10030, 0x00061957}, - {0x10030, 0x00061D1F}, - {0x10030, 0x00062119}, - {0x10030, 0x000624DF}, - {0x10030, 0x000628D9}, - {0x10030, 0x00062C9F}, - {0x10030, 0x00063099}, - {0x10030, 0x0006345F}, - {0x10030, 0x00063859}, - {0x10030, 0x00063C1F}, - {0x10030, 0x00064019}, - {0x10030, 0x00064413}, - {0x10030, 0x000681E1}, - {0x10030, 0x000685DB}, - {0x10030, 0x000689D5}, - {0x10030, 0x00068D9B}, - {0x10030, 0x00069195}, - {0x10030, 0x0006955B}, - {0x10030, 0x00069957}, - {0x10030, 0x00069D1F}, - {0x10030, 0x0006A119}, - {0x10030, 0x0006A4DF}, - {0x10030, 0x0006A8D9}, - {0x10030, 0x0006AC9F}, - {0x10030, 0x0006B099}, - {0x10030, 0x0006B45F}, - {0x10030, 0x0006B859}, - {0x10030, 0x0006BC1F}, - {0x10030, 0x0006C019}, - {0x10030, 0x0006C413}, - {0x10030, 0x000701E1}, - {0x10030, 0x000705DB}, - {0x10030, 0x000709D5}, - {0x10030, 0x00070D9B}, - {0x10030, 0x00071195}, - {0x10030, 0x0007155B}, - {0x10030, 0x00071957}, - {0x10030, 0x00071D1F}, - {0x10030, 0x00072119}, - {0x10030, 0x000724DF}, - {0x10030, 0x000728D9}, - {0x10030, 0x00072C9F}, - {0x10030, 0x00073099}, - {0x10030, 0x0007345F}, - {0x10030, 0x00073859}, - {0x10030, 0x00073C1F}, - {0x10030, 0x00074019}, - {0x10030, 0x00074413}, - {0x10030, 0x000781DF}, - {0x10030, 0x000785D9}, - {0x10030, 0x000789D3}, - {0x10030, 0x00078D99}, - {0x10030, 0x00079193}, - {0x10030, 0x0007955F}, - {0x10030, 0x00079959}, - {0x10030, 0x00079D21}, - {0x10030, 0x0007A115}, - {0x10030, 0x0007A4DF}, - {0x10030, 0x0007A8D9}, - {0x10030, 0x0007AC9F}, - {0x10030, 0x0007B099}, - {0x10030, 0x0007B45F}, - {0x10030, 0x0007B859}, - {0x10030, 0x0007BC1F}, - {0x10030, 0x0007C019}, - {0x10030, 0x0007C413}, - {0x10030, 0x00000000}, - {0x10030, 0x000785A9}, - {0x10030, 0x000789A3}, - {0x10030, 0x00078D9D}, - {0x10030, 0x00079197}, - {0x10030, 0x00079591}, - {0x10030, 0x00079957}, - {0x10030, 0x00079D51}, - {0x10030, 0x0007A119}, - {0x10030, 0x0007A513}, - {0x10030, 0x0007A8D9}, - {0x10030, 0x0007ACD3}, - {0x10030, 0x0007B099}, - {0x10030, 0x0007B493}, - {0x10030, 0x0007B859}, - {0x10030, 0x0007BC53}, - {0x10030, 0x0007C019}, - {0x10030, 0x0007C413}, + {0x10030, 0x000001EF}, + {0x10030, 0x000005E9}, + {0x10030, 0x000009E3}, + {0x10030, 0x00000DDD}, + {0x10030, 0x000011D7}, + {0x10030, 0x0000159F}, + {0x10030, 0x00001999}, + {0x10030, 0x00001D5F}, + {0x10030, 0x00002159}, + {0x10030, 0x0000251F}, + {0x10030, 0x00002919}, + {0x10030, 0x00002CDF}, + {0x10030, 0x000030D9}, + {0x10030, 0x0000349F}, + {0x10030, 0x00003899}, + {0x10030, 0x00003C5F}, + {0x10030, 0x00004059}, + {0x10030, 0x00004453}, + {0x10030, 0x000201A7}, + {0x10030, 0x000205A1}, + {0x10030, 0x0002099B}, + {0x10030, 0x00020D95}, + {0x10030, 0x0002115B}, + {0x10030, 0x00021555}, + {0x10030, 0x00021921}, + {0x10030, 0x00021D1B}, + {0x10030, 0x000220E3}, + {0x10030, 0x000224DD}, + {0x10030, 0x000228A3}, + {0x10030, 0x00022C9D}, + {0x10030, 0x00023063}, + {0x10030, 0x0002345D}, + {0x10030, 0x00023823}, + {0x10030, 0x00023C1D}, + {0x10030, 0x00024017}, + {0x10030, 0x00024411}, + {0x10030, 0x000281A9}, + {0x10030, 0x000285A3}, + {0x10030, 0x0002899D}, + {0x10030, 0x00028D97}, + {0x10030, 0x0002915D}, + {0x10030, 0x00029557}, + {0x10030, 0x0002991F}, + {0x10030, 0x00029D19}, + {0x10030, 0x0002A0E1}, + {0x10030, 0x0002A4DB}, + {0x10030, 0x0002A8A1}, + {0x10030, 0x0002AC9B}, + {0x10030, 0x0002B061}, + {0x10030, 0x0002B45B}, + {0x10030, 0x0002B821}, + {0x10030, 0x0002BC1B}, + {0x10030, 0x0002C015}, + {0x10030, 0x0002C40F}, + {0x10030, 0x000301A9}, + {0x10030, 0x000305A3}, + {0x10030, 0x0003099D}, + {0x10030, 0x00030D97}, + {0x10030, 0x0003115D}, + {0x10030, 0x00031557}, + {0x10030, 0x0003191F}, + {0x10030, 0x00031D19}, + {0x10030, 0x000320E1}, + {0x10030, 0x000324DB}, + {0x10030, 0x000328A1}, + {0x10030, 0x00032C9B}, + {0x10030, 0x00033061}, + {0x10030, 0x0003345B}, + {0x10030, 0x00033821}, + {0x10030, 0x00033C1B}, + {0x10030, 0x00034015}, + {0x10030, 0x0003440F}, + {0x10030, 0x000601F1}, + {0x10030, 0x000605E9}, + {0x10030, 0x000609A9}, + {0x10030, 0x00060D65}, + {0x10030, 0x0006115F}, + {0x10030, 0x00061525}, + {0x10030, 0x0006191F}, + {0x10030, 0x00061CE7}, + {0x10030, 0x000620E1}, + {0x10030, 0x000624DB}, + {0x10030, 0x000628A3}, + {0x10030, 0x00062C69}, + {0x10030, 0x00063063}, + {0x10030, 0x00063429}, + {0x10030, 0x00063823}, + {0x10030, 0x00063C1D}, + {0x10030, 0x00064013}, + {0x10030, 0x0006440D}, + {0x10030, 0x000681EF}, + {0x10030, 0x000685E7}, + {0x10030, 0x000689A7}, + {0x10030, 0x00068D61}, + {0x10030, 0x0006915B}, + {0x10030, 0x00069523}, + {0x10030, 0x0006991D}, + {0x10030, 0x00069CE5}, + {0x10030, 0x0006A0DF}, + {0x10030, 0x0006A4A7}, + {0x10030, 0x0006A8A1}, + {0x10030, 0x0006AC67}, + {0x10030, 0x0006B061}, + {0x10030, 0x0006B429}, + {0x10030, 0x0006B823}, + {0x10030, 0x0006BC1D}, + {0x10030, 0x0006C017}, + {0x10030, 0x0006C40D}, + {0x10030, 0x000701EF}, + {0x10030, 0x000705E7}, + {0x10030, 0x000709A7}, + {0x10030, 0x00070D61}, + {0x10030, 0x0007115B}, + {0x10030, 0x00071523}, + {0x10030, 0x0007191D}, + {0x10030, 0x00071CE5}, + {0x10030, 0x000720DF}, + {0x10030, 0x000724D9}, + {0x10030, 0x000728A1}, + {0x10030, 0x00072C67}, + {0x10030, 0x00073061}, + {0x10030, 0x00073427}, + {0x10030, 0x00073821}, + {0x10030, 0x00073C1B}, + {0x10030, 0x00074015}, + {0x10030, 0x0007440D}, + {0x10030, 0x000781EF}, + {0x10030, 0x000785E9}, + {0x10030, 0x000789E3}, + {0x10030, 0x00078DA1}, + {0x10030, 0x0007915F}, + {0x10030, 0x00079559}, + {0x10030, 0x00079921}, + {0x10030, 0x00079D1B}, + {0x10030, 0x0007A0E3}, + {0x10030, 0x0007A4DD}, + {0x10030, 0x0007A8D7}, + {0x10030, 0x0007AC9D}, + {0x10030, 0x0007B063}, + {0x10030, 0x0007B45D}, + {0x10030, 0x0007B823}, + {0x10030, 0x0007BC1D}, + {0x10030, 0x0007C017}, + {0x10030, 0x0007C40F}, {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, - {0x10030, 0x000201DF}, - {0x10030, 0x000205D9}, - {0x10030, 0x000209D3}, - {0x10030, 0x00020D99}, - {0x10030, 0x00021193}, - {0x10030, 0x0002155F}, - {0x10030, 0x00021959}, - {0x10030, 0x00021D21}, - {0x10030, 0x00022119}, - {0x10030, 0x000224DF}, - {0x10030, 0x000228D9}, - {0x10030, 0x00022C9F}, - {0x10030, 0x00023099}, - {0x10030, 0x0002345F}, - {0x10030, 0x00023859}, - {0x10030, 0x00023C1F}, - {0x10030, 0x00024019}, - {0x10030, 0x00024413}, - {0x10030, 0x000281CD}, - {0x10030, 0x000285DB}, - {0x10030, 0x000289D5}, - {0x10030, 0x00028D9B}, - {0x10030, 0x0002918D}, - {0x10030, 0x00029555}, - {0x10030, 0x00029957}, - {0x10030, 0x00029D1F}, - {0x10030, 0x0002A119}, - {0x10030, 0x0002A4DF}, - {0x10030, 0x0002A8D9}, - {0x10030, 0x0002AC9F}, - {0x10030, 0x0002B099}, - {0x10030, 0x0002B45F}, - {0x10030, 0x0002B859}, - {0x10030, 0x0002BC1F}, - {0x10030, 0x0002C019}, - {0x10030, 0x0002C413}, - {0x10030, 0x000301D9}, - {0x10030, 0x000305DB}, - {0x10030, 0x000309D5}, - {0x10030, 0x00030D9B}, - {0x10030, 0x00031195}, - {0x10030, 0x0003155D}, - {0x10030, 0x00031955}, - {0x10030, 0x00031D1D}, - {0x10030, 0x00032119}, - {0x10030, 0x000324DF}, - {0x10030, 0x000328D9}, - {0x10030, 0x00032C9F}, - {0x10030, 0x00033099}, - {0x10030, 0x0003345F}, - {0x10030, 0x00033859}, - {0x10030, 0x00033C1F}, - {0x10030, 0x00034019}, - {0x10030, 0x00034413}, - {0x10030, 0x000601E1}, - {0x10030, 0x000605DB}, - {0x10030, 0x000609D5}, - {0x10030, 0x00060D9B}, - {0x10030, 0x00061195}, - {0x10030, 0x0006155B}, - {0x10030, 0x00061957}, - {0x10030, 0x00061D1F}, - {0x10030, 0x00062119}, - {0x10030, 0x000624DF}, - {0x10030, 0x000628D9}, - {0x10030, 0x00062C9F}, - {0x10030, 0x00063099}, - {0x10030, 0x0006345F}, - {0x10030, 0x00063859}, - {0x10030, 0x00063C1F}, - {0x10030, 0x00064019}, - {0x10030, 0x00064413}, - {0x10030, 0x000681E1}, - {0x10030, 0x000685DB}, - {0x10030, 0x000689D5}, - {0x10030, 0x00068D9B}, - {0x10030, 0x00069195}, - {0x10030, 0x0006955B}, - {0x10030, 0x00069957}, - {0x10030, 0x00069D1F}, - {0x10030, 0x0006A119}, - {0x10030, 0x0006A4DF}, - {0x10030, 0x0006A8D9}, - {0x10030, 0x0006AC9F}, - {0x10030, 0x0006B099}, - {0x10030, 0x0006B45F}, - {0x10030, 0x0006B859}, - {0x10030, 0x0006BC1F}, - {0x10030, 0x0006C019}, - {0x10030, 0x0006C413}, - {0x10030, 0x000701E1}, - {0x10030, 0x000705DB}, - {0x10030, 0x000709D5}, - {0x10030, 0x00070D9B}, - {0x10030, 0x00071195}, - {0x10030, 0x0007155B}, - {0x10030, 0x00071957}, - {0x10030, 0x00071D1F}, - {0x10030, 0x00072119}, - {0x10030, 0x000724DF}, - {0x10030, 0x000728D9}, - {0x10030, 0x00072C9F}, - {0x10030, 0x00073099}, - {0x10030, 0x0007345F}, - {0x10030, 0x00073859}, - {0x10030, 0x00073C1F}, - {0x10030, 0x00074019}, - {0x10030, 0x00074413}, - {0x10030, 0x000781DF}, - {0x10030, 0x000785D9}, - {0x10030, 0x000789D3}, - {0x10030, 0x00078D99}, - {0x10030, 0x00079193}, - {0x10030, 0x0007955F}, - {0x10030, 0x00079959}, - {0x10030, 0x00079D21}, - {0x10030, 0x0007A115}, - {0x10030, 0x0007A4DF}, - {0x10030, 0x0007A8D9}, - {0x10030, 0x0007AC9F}, - {0x10030, 0x0007B099}, - {0x10030, 0x0007B45F}, - {0x10030, 0x0007B859}, - {0x10030, 0x0007BC1F}, - {0x10030, 0x0007C019}, - {0x10030, 0x0007C413}, - {0x10030, 0x00000000}, - {0x10030, 0x000785A9}, - {0x10030, 0x000789A3}, - {0x10030, 0x00078D9D}, - {0x10030, 0x00079197}, - {0x10030, 0x00079591}, - {0x10030, 0x00079957}, - {0x10030, 0x00079D51}, - {0x10030, 0x0007A119}, - {0x10030, 0x0007A513}, - {0x10030, 0x0007A8D9}, - {0x10030, 0x0007ACD3}, - {0x10030, 0x0007B099}, - {0x10030, 0x0007B493}, - {0x10030, 0x0007B859}, - {0x10030, 0x0007BC53}, - {0x10030, 0x0007C019}, - {0x10030, 0x0007C413}, + {0x10030, 0x000001EF}, + {0x10030, 0x000005E9}, + {0x10030, 0x000009E3}, + {0x10030, 0x00000DDD}, + {0x10030, 0x000011D7}, + {0x10030, 0x0000159F}, + {0x10030, 0x00001999}, + {0x10030, 0x00001D5F}, + {0x10030, 0x00002159}, + {0x10030, 0x0000251F}, + {0x10030, 0x00002919}, + {0x10030, 0x00002CDF}, + {0x10030, 0x000030D9}, + {0x10030, 0x0000349F}, + {0x10030, 0x00003899}, + {0x10030, 0x00003C5F}, + {0x10030, 0x00004059}, + {0x10030, 0x00004453}, + {0x10030, 0x000201A7}, + {0x10030, 0x000205A1}, + {0x10030, 0x0002099B}, + {0x10030, 0x00020D95}, + {0x10030, 0x0002115B}, + {0x10030, 0x00021555}, + {0x10030, 0x00021921}, + {0x10030, 0x00021D1B}, + {0x10030, 0x000220E3}, + {0x10030, 0x000224DD}, + {0x10030, 0x000228A3}, + {0x10030, 0x00022C9D}, + {0x10030, 0x00023063}, + {0x10030, 0x0002345D}, + {0x10030, 0x00023823}, + {0x10030, 0x00023C1D}, + {0x10030, 0x00024017}, + {0x10030, 0x00024411}, + {0x10030, 0x000281A9}, + {0x10030, 0x000285A3}, + {0x10030, 0x0002899D}, + {0x10030, 0x00028D97}, + {0x10030, 0x0002915D}, + {0x10030, 0x00029557}, + {0x10030, 0x0002991F}, + {0x10030, 0x00029D19}, + {0x10030, 0x0002A0E1}, + {0x10030, 0x0002A4DB}, + {0x10030, 0x0002A8A1}, + {0x10030, 0x0002AC9B}, + {0x10030, 0x0002B061}, + {0x10030, 0x0002B45B}, + {0x10030, 0x0002B821}, + {0x10030, 0x0002BC1B}, + {0x10030, 0x0002C015}, + {0x10030, 0x0002C40F}, + {0x10030, 0x000301A9}, + {0x10030, 0x000305A3}, + {0x10030, 0x0003099D}, + {0x10030, 0x00030D97}, + {0x10030, 0x0003115D}, + {0x10030, 0x00031557}, + {0x10030, 0x0003191F}, + {0x10030, 0x00031D19}, + {0x10030, 0x000320E1}, + {0x10030, 0x000324DB}, + {0x10030, 0x000328A1}, + {0x10030, 0x00032C9B}, + {0x10030, 0x00033061}, + {0x10030, 0x0003345B}, + {0x10030, 0x00033821}, + {0x10030, 0x00033C1B}, + {0x10030, 0x00034015}, + {0x10030, 0x0003440F}, + {0x10030, 0x000601F1}, + {0x10030, 0x000605E9}, + {0x10030, 0x000609A9}, + {0x10030, 0x00060D65}, + {0x10030, 0x0006115F}, + {0x10030, 0x00061525}, + {0x10030, 0x0006191F}, + {0x10030, 0x00061CE7}, + {0x10030, 0x000620E1}, + {0x10030, 0x000624DB}, + {0x10030, 0x000628A3}, + {0x10030, 0x00062C69}, + {0x10030, 0x00063063}, + {0x10030, 0x00063429}, + {0x10030, 0x00063823}, + {0x10030, 0x00063C1D}, + {0x10030, 0x00064013}, + {0x10030, 0x0006440D}, + {0x10030, 0x000681EF}, + {0x10030, 0x000685E7}, + {0x10030, 0x000689A7}, + {0x10030, 0x00068D61}, + {0x10030, 0x0006915B}, + {0x10030, 0x00069523}, + {0x10030, 0x0006991D}, + {0x10030, 0x00069CE5}, + {0x10030, 0x0006A0DF}, + {0x10030, 0x0006A4A7}, + {0x10030, 0x0006A8A1}, + {0x10030, 0x0006AC67}, + {0x10030, 0x0006B061}, + {0x10030, 0x0006B429}, + {0x10030, 0x0006B823}, + {0x10030, 0x0006BC1D}, + {0x10030, 0x0006C017}, + {0x10030, 0x0006C40D}, + {0x10030, 0x000701EF}, + {0x10030, 0x000705E7}, + {0x10030, 0x000709A7}, + {0x10030, 0x00070D61}, + {0x10030, 0x0007115B}, + {0x10030, 0x00071523}, + {0x10030, 0x0007191D}, + {0x10030, 0x00071CE5}, + {0x10030, 0x000720DF}, + {0x10030, 0x000724D9}, + {0x10030, 0x000728A1}, + {0x10030, 0x00072C67}, + {0x10030, 0x00073061}, + {0x10030, 0x00073427}, + {0x10030, 0x00073821}, + {0x10030, 0x00073C1B}, + {0x10030, 0x00074015}, + {0x10030, 0x0007440D}, + {0x10030, 0x000781EF}, + {0x10030, 0x000785E9}, + {0x10030, 0x000789E3}, + {0x10030, 0x00078DA1}, + {0x10030, 0x0007915F}, + {0x10030, 0x00079559}, + {0x10030, 0x00079921}, + {0x10030, 0x00079D1B}, + {0x10030, 0x0007A0E3}, + {0x10030, 0x0007A4DD}, + {0x10030, 0x0007A8D7}, + {0x10030, 0x0007AC9D}, + {0x10030, 0x0007B063}, + {0x10030, 0x0007B45D}, + {0x10030, 0x0007B823}, + {0x10030, 0x0007BC1D}, + {0x10030, 0x0007C017}, + {0x10030, 0x0007C40F}, {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, - {0x10030, 0x000201DF}, - {0x10030, 0x000205D9}, - {0x10030, 0x000209D3}, - {0x10030, 0x00020D99}, - {0x10030, 0x00021193}, - {0x10030, 0x0002155F}, - {0x10030, 0x00021959}, - {0x10030, 0x00021D21}, - {0x10030, 0x00022119}, - {0x10030, 0x000224DF}, - {0x10030, 0x000228D9}, - {0x10030, 0x00022C9F}, - {0x10030, 0x00023099}, - {0x10030, 0x0002345F}, - {0x10030, 0x00023859}, - {0x10030, 0x00023C1F}, - {0x10030, 0x00024019}, - {0x10030, 0x00024413}, - {0x10030, 0x000281CD}, - {0x10030, 0x000285DB}, - {0x10030, 0x000289D5}, - {0x10030, 0x00028D9B}, - {0x10030, 0x0002918D}, - {0x10030, 0x00029555}, - {0x10030, 0x00029957}, - {0x10030, 0x00029D1F}, - {0x10030, 0x0002A119}, - {0x10030, 0x0002A4DF}, - {0x10030, 0x0002A8D9}, - {0x10030, 0x0002AC9F}, - {0x10030, 0x0002B099}, - {0x10030, 0x0002B45F}, - {0x10030, 0x0002B859}, - {0x10030, 0x0002BC1F}, - {0x10030, 0x0002C019}, - {0x10030, 0x0002C413}, - {0x10030, 0x000301D9}, - {0x10030, 0x000305DB}, - {0x10030, 0x000309D5}, - {0x10030, 0x00030D9B}, - {0x10030, 0x00031195}, - {0x10030, 0x0003155D}, - {0x10030, 0x00031955}, - {0x10030, 0x00031D1D}, - {0x10030, 0x00032119}, - {0x10030, 0x000324DF}, - {0x10030, 0x000328D9}, - {0x10030, 0x00032C9F}, - {0x10030, 0x00033099}, - {0x10030, 0x0003345F}, - {0x10030, 0x00033859}, - {0x10030, 0x00033C1F}, - {0x10030, 0x00034019}, - {0x10030, 0x00034413}, - {0x10030, 0x000601E1}, - {0x10030, 0x000605DB}, - {0x10030, 0x000609D5}, - {0x10030, 0x00060D9B}, - {0x10030, 0x00061195}, - {0x10030, 0x0006155B}, - {0x10030, 0x00061957}, - {0x10030, 0x00061D1F}, - {0x10030, 0x00062119}, - {0x10030, 0x000624DF}, - {0x10030, 0x000628D9}, - {0x10030, 0x00062C9F}, - {0x10030, 0x00063099}, - {0x10030, 0x0006345F}, - {0x10030, 0x00063859}, - {0x10030, 0x00063C1F}, - {0x10030, 0x00064019}, - {0x10030, 0x00064413}, - {0x10030, 0x000681E1}, - {0x10030, 0x000685DB}, - {0x10030, 0x000689D5}, - {0x10030, 0x00068D9B}, - {0x10030, 0x00069195}, - {0x10030, 0x0006955B}, - {0x10030, 0x00069957}, - {0x10030, 0x00069D1F}, - {0x10030, 0x0006A119}, - {0x10030, 0x0006A4DF}, - {0x10030, 0x0006A8D9}, - {0x10030, 0x0006AC9F}, - {0x10030, 0x0006B099}, - {0x10030, 0x0006B45F}, - {0x10030, 0x0006B859}, - {0x10030, 0x0006BC1F}, - {0x10030, 0x0006C019}, - {0x10030, 0x0006C413}, - {0x10030, 0x000701E1}, - {0x10030, 0x000705DB}, - {0x10030, 0x000709D5}, - {0x10030, 0x00070D9B}, - {0x10030, 0x00071195}, - {0x10030, 0x0007155B}, - {0x10030, 0x00071957}, - {0x10030, 0x00071D1F}, - {0x10030, 0x00072119}, - {0x10030, 0x000724DF}, - {0x10030, 0x000728D9}, - {0x10030, 0x00072C9F}, - {0x10030, 0x00073099}, - {0x10030, 0x0007345F}, - {0x10030, 0x00073859}, - {0x10030, 0x00073C1F}, - {0x10030, 0x00074019}, - {0x10030, 0x00074413}, - {0x10030, 0x000781DF}, - {0x10030, 0x000785D9}, - {0x10030, 0x000789D3}, - {0x10030, 0x00078D99}, - {0x10030, 0x00079193}, - {0x10030, 0x0007955F}, - {0x10030, 0x00079959}, - {0x10030, 0x00079D21}, - {0x10030, 0x0007A115}, - {0x10030, 0x0007A4DF}, - {0x10030, 0x0007A8D9}, - {0x10030, 0x0007AC9F}, - {0x10030, 0x0007B099}, - {0x10030, 0x0007B45F}, - {0x10030, 0x0007B859}, - {0x10030, 0x0007BC1F}, - {0x10030, 0x0007C019}, - {0x10030, 0x0007C413}, - {0x10030, 0x00000000}, - {0x10030, 0x000785A9}, - {0x10030, 0x000789A3}, - {0x10030, 0x00078D9D}, - {0x10030, 0x00079197}, - {0x10030, 0x00079591}, - {0x10030, 0x00079957}, - {0x10030, 0x00079D51}, - {0x10030, 0x0007A119}, - {0x10030, 0x0007A513}, - {0x10030, 0x0007A8D9}, - {0x10030, 0x0007ACD3}, - {0x10030, 0x0007B099}, - {0x10030, 0x0007B493}, - {0x10030, 0x0007B859}, - {0x10030, 0x0007BC53}, - {0x10030, 0x0007C019}, - {0x10030, 0x0007C413}, + {0x10030, 0x000001EF}, + {0x10030, 0x000005E9}, + {0x10030, 0x000009E3}, + {0x10030, 0x00000DDD}, + {0x10030, 0x000011D7}, + {0x10030, 0x0000159F}, + {0x10030, 0x00001999}, + {0x10030, 0x00001D5F}, + {0x10030, 0x00002159}, + {0x10030, 0x0000251F}, + {0x10030, 0x00002919}, + {0x10030, 0x00002CDF}, + {0x10030, 0x000030D9}, + {0x10030, 0x0000349F}, + {0x10030, 0x00003899}, + {0x10030, 0x00003C5F}, + {0x10030, 0x00004059}, + {0x10030, 0x00004453}, + {0x10030, 0x000201A7}, + {0x10030, 0x000205A1}, + {0x10030, 0x0002099B}, + {0x10030, 0x00020D95}, + {0x10030, 0x0002115B}, + {0x10030, 0x00021555}, + {0x10030, 0x00021921}, + {0x10030, 0x00021D1B}, + {0x10030, 0x000220E3}, + {0x10030, 0x000224DD}, + {0x10030, 0x000228A3}, + {0x10030, 0x00022C9D}, + {0x10030, 0x00023063}, + {0x10030, 0x0002345D}, + {0x10030, 0x00023823}, + {0x10030, 0x00023C1D}, + {0x10030, 0x00024017}, + {0x10030, 0x00024411}, + {0x10030, 0x000281A9}, + {0x10030, 0x000285A3}, + {0x10030, 0x0002899D}, + {0x10030, 0x00028D97}, + {0x10030, 0x0002915D}, + {0x10030, 0x00029557}, + {0x10030, 0x0002991F}, + {0x10030, 0x00029D19}, + {0x10030, 0x0002A0E1}, + {0x10030, 0x0002A4DB}, + {0x10030, 0x0002A8A1}, + {0x10030, 0x0002AC9B}, + {0x10030, 0x0002B061}, + {0x10030, 0x0002B45B}, + {0x10030, 0x0002B821}, + {0x10030, 0x0002BC1B}, + {0x10030, 0x0002C015}, + {0x10030, 0x0002C40F}, + {0x10030, 0x000301A9}, + {0x10030, 0x000305A3}, + {0x10030, 0x0003099D}, + {0x10030, 0x00030D97}, + {0x10030, 0x0003115D}, + {0x10030, 0x00031557}, + {0x10030, 0x0003191F}, + {0x10030, 0x00031D19}, + {0x10030, 0x000320E1}, + {0x10030, 0x000324DB}, + {0x10030, 0x000328A1}, + {0x10030, 0x00032C9B}, + {0x10030, 0x00033061}, + {0x10030, 0x0003345B}, + {0x10030, 0x00033821}, + {0x10030, 0x00033C1B}, + {0x10030, 0x00034015}, + {0x10030, 0x0003440F}, + {0x10030, 0x000601F1}, + {0x10030, 0x000605E9}, + {0x10030, 0x000609A9}, + {0x10030, 0x00060D65}, + {0x10030, 0x0006115F}, + {0x10030, 0x00061525}, + {0x10030, 0x0006191F}, + {0x10030, 0x00061CE7}, + {0x10030, 0x000620E1}, + {0x10030, 0x000624DB}, + {0x10030, 0x000628A3}, + {0x10030, 0x00062C69}, + {0x10030, 0x00063063}, + {0x10030, 0x00063429}, + {0x10030, 0x00063823}, + {0x10030, 0x00063C1D}, + {0x10030, 0x00064013}, + {0x10030, 0x0006440D}, + {0x10030, 0x000681EF}, + {0x10030, 0x000685E7}, + {0x10030, 0x000689A7}, + {0x10030, 0x00068D61}, + {0x10030, 0x0006915B}, + {0x10030, 0x00069523}, + {0x10030, 0x0006991D}, + {0x10030, 0x00069CE5}, + {0x10030, 0x0006A0DF}, + {0x10030, 0x0006A4A7}, + {0x10030, 0x0006A8A1}, + {0x10030, 0x0006AC67}, + {0x10030, 0x0006B061}, + {0x10030, 0x0006B429}, + {0x10030, 0x0006B823}, + {0x10030, 0x0006BC1D}, + {0x10030, 0x0006C017}, + {0x10030, 0x0006C40D}, + {0x10030, 0x000701EF}, + {0x10030, 0x000705E7}, + {0x10030, 0x000709A7}, + {0x10030, 0x00070D61}, + {0x10030, 0x0007115B}, + {0x10030, 0x00071523}, + {0x10030, 0x0007191D}, + {0x10030, 0x00071CE5}, + {0x10030, 0x000720DF}, + {0x10030, 0x000724D9}, + {0x10030, 0x000728A1}, + {0x10030, 0x00072C67}, + {0x10030, 0x00073061}, + {0x10030, 0x00073427}, + {0x10030, 0x00073821}, + {0x10030, 0x00073C1B}, + {0x10030, 0x00074015}, + {0x10030, 0x0007440D}, + {0x10030, 0x000781EF}, + {0x10030, 0x000785E9}, + {0x10030, 0x000789E3}, + {0x10030, 0x00078DA1}, + {0x10030, 0x0007915F}, + {0x10030, 0x00079559}, + {0x10030, 0x00079921}, + {0x10030, 0x00079D1B}, + {0x10030, 0x0007A0E3}, + {0x10030, 0x0007A4DD}, + {0x10030, 0x0007A8D7}, + {0x10030, 0x0007AC9D}, + {0x10030, 0x0007B063}, + {0x10030, 0x0007B45D}, + {0x10030, 0x0007B823}, + {0x10030, 0x0007BC1D}, + {0x10030, 0x0007C017}, + {0x10030, 0x0007C40F}, {0xA0000000, 0x00000000}, {0x10030, 0x000001EF}, {0x10030, 0x000005E9}, @@ -6724,6 +9848,1150 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x10030, 0x00004017}, {0x100EE, 0x00000000}, {0x100EE, 0x00002000}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x000200FA}, + {0x10030, 0x000204F7}, + {0x10030, 0x000208F4}, + {0x10030, 0x00020CF1}, + {0x10030, 0x000210EE}, + {0x10030, 0x000214EB}, + {0x10030, 0x000218E8}, + {0x10030, 0x00021CE5}, + {0x10030, 0x000220E2}, + {0x10030, 0x000224DF}, + {0x10030, 0x000228DC}, + {0x10030, 0x00022CD9}, + {0x10030, 0x000230D6}, + {0x10030, 0x000234D3}, + {0x10030, 0x000238D0}, + {0x10030, 0x00023C0D}, + {0x10030, 0x0002400A}, + {0x10030, 0x000280F9}, + {0x10030, 0x000284F6}, + {0x10030, 0x000288F3}, + {0x10030, 0x00028CF0}, + {0x10030, 0x000290ED}, + {0x10030, 0x000294EA}, + {0x10030, 0x000298E7}, + {0x10030, 0x00029CE4}, + {0x10030, 0x0002A0E1}, + {0x10030, 0x0002A4DE}, + {0x10030, 0x0002A8DB}, + {0x10030, 0x0002ACD8}, + {0x10030, 0x0002B0D5}, + {0x10030, 0x0002B4D2}, + {0x10030, 0x0002B8CF}, + {0x10030, 0x0002BC0C}, + {0x10030, 0x0002C009}, + {0x10030, 0x000300F6}, + {0x10030, 0x000304F3}, + {0x10030, 0x000308F0}, + {0x10030, 0x00030CED}, + {0x10030, 0x000310EA}, + {0x10030, 0x000314E7}, + {0x10030, 0x000318E4}, + {0x10030, 0x00031CE1}, + {0x10030, 0x000320DE}, + {0x10030, 0x000324DB}, + {0x10030, 0x000328D8}, + {0x10030, 0x00032CD5}, + {0x10030, 0x000330D2}, + {0x10030, 0x000334CF}, + {0x10030, 0x000338CC}, + {0x10030, 0x00033C09}, + {0x10030, 0x00034006}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x000200FA}, + {0x10030, 0x000204F7}, + {0x10030, 0x000208F4}, + {0x10030, 0x00020CF1}, + {0x10030, 0x000210EE}, + {0x10030, 0x000214EB}, + {0x10030, 0x000218E8}, + {0x10030, 0x00021CE5}, + {0x10030, 0x000220E2}, + {0x10030, 0x000224DF}, + {0x10030, 0x000228DC}, + {0x10030, 0x00022CD9}, + {0x10030, 0x000230D6}, + {0x10030, 0x000234D3}, + {0x10030, 0x000238D0}, + {0x10030, 0x00023C0D}, + {0x10030, 0x0002400A}, + {0x10030, 0x000280F9}, + {0x10030, 0x000284F6}, + {0x10030, 0x000288F3}, + {0x10030, 0x00028CF0}, + {0x10030, 0x000290ED}, + {0x10030, 0x000294EA}, + {0x10030, 0x000298E7}, + {0x10030, 0x00029CE4}, + {0x10030, 0x0002A0E1}, + {0x10030, 0x0002A4DE}, + {0x10030, 0x0002A8DB}, + {0x10030, 0x0002ACD8}, + {0x10030, 0x0002B0D5}, + {0x10030, 0x0002B4D2}, + {0x10030, 0x0002B8CF}, + {0x10030, 0x0002BC0C}, + {0x10030, 0x0002C009}, + {0x10030, 0x000300F6}, + {0x10030, 0x000304F3}, + {0x10030, 0x000308F0}, + {0x10030, 0x00030CED}, + {0x10030, 0x000310EA}, + {0x10030, 0x000314E7}, + {0x10030, 0x000318E4}, + {0x10030, 0x00031CE1}, + {0x10030, 0x000320DE}, + {0x10030, 0x000324DB}, + {0x10030, 0x000328D8}, + {0x10030, 0x00032CD5}, + {0x10030, 0x000330D2}, + {0x10030, 0x000334CF}, + {0x10030, 0x000338CC}, + {0x10030, 0x00033C09}, + {0x10030, 0x00034006}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x000200FA}, + {0x10030, 0x000204F7}, + {0x10030, 0x000208F4}, + {0x10030, 0x00020CF1}, + {0x10030, 0x000210EE}, + {0x10030, 0x000214EB}, + {0x10030, 0x000218E8}, + {0x10030, 0x00021CE5}, + {0x10030, 0x000220E2}, + {0x10030, 0x000224DF}, + {0x10030, 0x000228DC}, + {0x10030, 0x00022CD9}, + {0x10030, 0x000230D6}, + {0x10030, 0x000234D3}, + {0x10030, 0x000238D0}, + {0x10030, 0x00023C0D}, + {0x10030, 0x0002400A}, + {0x10030, 0x000280F9}, + {0x10030, 0x000284F6}, + {0x10030, 0x000288F3}, + {0x10030, 0x00028CF0}, + {0x10030, 0x000290ED}, + {0x10030, 0x000294EA}, + {0x10030, 0x000298E7}, + {0x10030, 0x00029CE4}, + {0x10030, 0x0002A0E1}, + {0x10030, 0x0002A4DE}, + {0x10030, 0x0002A8DB}, + {0x10030, 0x0002ACD8}, + {0x10030, 0x0002B0D5}, + {0x10030, 0x0002B4D2}, + {0x10030, 0x0002B8CF}, + {0x10030, 0x0002BC0C}, + {0x10030, 0x0002C009}, + {0x10030, 0x000300F6}, + {0x10030, 0x000304F3}, + {0x10030, 0x000308F0}, + {0x10030, 0x00030CED}, + {0x10030, 0x000310EA}, + {0x10030, 0x000314E7}, + {0x10030, 0x000318E4}, + {0x10030, 0x00031CE1}, + {0x10030, 0x000320DE}, + {0x10030, 0x000324DB}, + {0x10030, 0x000328D8}, + {0x10030, 0x00032CD5}, + {0x10030, 0x000330D2}, + {0x10030, 0x000334CF}, + {0x10030, 0x000338CC}, + {0x10030, 0x00033C09}, + {0x10030, 0x00034006}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x000200FA}, + {0x10030, 0x000204F7}, + {0x10030, 0x000208F4}, + {0x10030, 0x00020CF1}, + {0x10030, 0x000210EE}, + {0x10030, 0x000214EB}, + {0x10030, 0x000218E8}, + {0x10030, 0x00021CE5}, + {0x10030, 0x000220E2}, + {0x10030, 0x000224DF}, + {0x10030, 0x000228DC}, + {0x10030, 0x00022CD9}, + {0x10030, 0x000230D6}, + {0x10030, 0x000234D3}, + {0x10030, 0x000238D0}, + {0x10030, 0x00023C0D}, + {0x10030, 0x0002400A}, + {0x10030, 0x000280F9}, + {0x10030, 0x000284F6}, + {0x10030, 0x000288F3}, + {0x10030, 0x00028CF0}, + {0x10030, 0x000290ED}, + {0x10030, 0x000294EA}, + {0x10030, 0x000298E7}, + {0x10030, 0x00029CE4}, + {0x10030, 0x0002A0E1}, + {0x10030, 0x0002A4DE}, + {0x10030, 0x0002A8DB}, + {0x10030, 0x0002ACD8}, + {0x10030, 0x0002B0D5}, + {0x10030, 0x0002B4D2}, + {0x10030, 0x0002B8CF}, + {0x10030, 0x0002BC0C}, + {0x10030, 0x0002C009}, + {0x10030, 0x000300F6}, + {0x10030, 0x000304F3}, + {0x10030, 0x000308F0}, + {0x10030, 0x00030CED}, + {0x10030, 0x000310EA}, + {0x10030, 0x000314E7}, + {0x10030, 0x000318E4}, + {0x10030, 0x00031CE1}, + {0x10030, 0x000320DE}, + {0x10030, 0x000324DB}, + {0x10030, 0x000328D8}, + {0x10030, 0x00032CD5}, + {0x10030, 0x000330D2}, + {0x10030, 0x000334CF}, + {0x10030, 0x000338CC}, + {0x10030, 0x00033C09}, + {0x10030, 0x00034006}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x000200FA}, + {0x10030, 0x000204F7}, + {0x10030, 0x000208F4}, + {0x10030, 0x00020CF1}, + {0x10030, 0x000210EE}, + {0x10030, 0x000214EB}, + {0x10030, 0x000218E8}, + {0x10030, 0x00021CE5}, + {0x10030, 0x000220E2}, + {0x10030, 0x000224DF}, + {0x10030, 0x000228DC}, + {0x10030, 0x00022CD9}, + {0x10030, 0x000230D6}, + {0x10030, 0x000234D3}, + {0x10030, 0x000238D0}, + {0x10030, 0x00023C0D}, + {0x10030, 0x0002400A}, + {0x10030, 0x000280F9}, + {0x10030, 0x000284F6}, + {0x10030, 0x000288F3}, + {0x10030, 0x00028CF0}, + {0x10030, 0x000290ED}, + {0x10030, 0x000294EA}, + {0x10030, 0x000298E7}, + {0x10030, 0x00029CE4}, + {0x10030, 0x0002A0E1}, + {0x10030, 0x0002A4DE}, + {0x10030, 0x0002A8DB}, + {0x10030, 0x0002ACD8}, + {0x10030, 0x0002B0D5}, + {0x10030, 0x0002B4D2}, + {0x10030, 0x0002B8CF}, + {0x10030, 0x0002BC0C}, + {0x10030, 0x0002C009}, + {0x10030, 0x000300F6}, + {0x10030, 0x000304F3}, + {0x10030, 0x000308F0}, + {0x10030, 0x00030CED}, + {0x10030, 0x000310EA}, + {0x10030, 0x000314E7}, + {0x10030, 0x000318E4}, + {0x10030, 0x00031CE1}, + {0x10030, 0x000320DE}, + {0x10030, 0x000324DB}, + {0x10030, 0x000328D8}, + {0x10030, 0x00032CD5}, + {0x10030, 0x000330D2}, + {0x10030, 0x000334CF}, + {0x10030, 0x000338CC}, + {0x10030, 0x00033C09}, + {0x10030, 0x00034006}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x000200FA}, + {0x10030, 0x000204F7}, + {0x10030, 0x000208F4}, + {0x10030, 0x00020CF1}, + {0x10030, 0x000210EE}, + {0x10030, 0x000214EB}, + {0x10030, 0x000218E8}, + {0x10030, 0x00021CE5}, + {0x10030, 0x000220E2}, + {0x10030, 0x000224DF}, + {0x10030, 0x000228DC}, + {0x10030, 0x00022CD9}, + {0x10030, 0x000230D6}, + {0x10030, 0x000234D3}, + {0x10030, 0x000238D0}, + {0x10030, 0x00023C0D}, + {0x10030, 0x0002400A}, + {0x10030, 0x000280F9}, + {0x10030, 0x000284F6}, + {0x10030, 0x000288F3}, + {0x10030, 0x00028CF0}, + {0x10030, 0x000290ED}, + {0x10030, 0x000294EA}, + {0x10030, 0x000298E7}, + {0x10030, 0x00029CE4}, + {0x10030, 0x0002A0E1}, + {0x10030, 0x0002A4DE}, + {0x10030, 0x0002A8DB}, + {0x10030, 0x0002ACD8}, + {0x10030, 0x0002B0D5}, + {0x10030, 0x0002B4D2}, + {0x10030, 0x0002B8CF}, + {0x10030, 0x0002BC0C}, + {0x10030, 0x0002C009}, + {0x10030, 0x000300F6}, + {0x10030, 0x000304F3}, + {0x10030, 0x000308F0}, + {0x10030, 0x00030CED}, + {0x10030, 0x000310EA}, + {0x10030, 0x000314E7}, + {0x10030, 0x000318E4}, + {0x10030, 0x00031CE1}, + {0x10030, 0x000320DE}, + {0x10030, 0x000324DB}, + {0x10030, 0x000328D8}, + {0x10030, 0x00032CD5}, + {0x10030, 0x000330D2}, + {0x10030, 0x000334CF}, + {0x10030, 0x000338CC}, + {0x10030, 0x00033C09}, + {0x10030, 0x00034006}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x000200FA}, + {0x10030, 0x000204F7}, + {0x10030, 0x000208F4}, + {0x10030, 0x00020CF1}, + {0x10030, 0x000210EE}, + {0x10030, 0x000214EB}, + {0x10030, 0x000218E8}, + {0x10030, 0x00021CE5}, + {0x10030, 0x000220E2}, + {0x10030, 0x000224DF}, + {0x10030, 0x000228DC}, + {0x10030, 0x00022CD9}, + {0x10030, 0x000230D6}, + {0x10030, 0x000234D3}, + {0x10030, 0x000238D0}, + {0x10030, 0x00023C0D}, + {0x10030, 0x0002400A}, + {0x10030, 0x000280F9}, + {0x10030, 0x000284F6}, + {0x10030, 0x000288F3}, + {0x10030, 0x00028CF0}, + {0x10030, 0x000290ED}, + {0x10030, 0x000294EA}, + {0x10030, 0x000298E7}, + {0x10030, 0x00029CE4}, + {0x10030, 0x0002A0E1}, + {0x10030, 0x0002A4DE}, + {0x10030, 0x0002A8DB}, + {0x10030, 0x0002ACD8}, + {0x10030, 0x0002B0D5}, + {0x10030, 0x0002B4D2}, + {0x10030, 0x0002B8CF}, + {0x10030, 0x0002BC0C}, + {0x10030, 0x0002C009}, + {0x10030, 0x000300F6}, + {0x10030, 0x000304F3}, + {0x10030, 0x000308F0}, + {0x10030, 0x00030CED}, + {0x10030, 0x000310EA}, + {0x10030, 0x000314E7}, + {0x10030, 0x000318E4}, + {0x10030, 0x00031CE1}, + {0x10030, 0x000320DE}, + {0x10030, 0x000324DB}, + {0x10030, 0x000328D8}, + {0x10030, 0x00032CD5}, + {0x10030, 0x000330D2}, + {0x10030, 0x000334CF}, + {0x10030, 0x000338CC}, + {0x10030, 0x00033C09}, + {0x10030, 0x00034006}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x000200FA}, + {0x10030, 0x000204F7}, + {0x10030, 0x000208F4}, + {0x10030, 0x00020CF1}, + {0x10030, 0x000210EE}, + {0x10030, 0x000214EB}, + {0x10030, 0x000218E8}, + {0x10030, 0x00021CE5}, + {0x10030, 0x000220E2}, + {0x10030, 0x000224DF}, + {0x10030, 0x000228DC}, + {0x10030, 0x00022CD9}, + {0x10030, 0x000230D6}, + {0x10030, 0x000234D3}, + {0x10030, 0x000238D0}, + {0x10030, 0x00023C0D}, + {0x10030, 0x0002400A}, + {0x10030, 0x000280F9}, + {0x10030, 0x000284F6}, + {0x10030, 0x000288F3}, + {0x10030, 0x00028CF0}, + {0x10030, 0x000290ED}, + {0x10030, 0x000294EA}, + {0x10030, 0x000298E7}, + {0x10030, 0x00029CE4}, + {0x10030, 0x0002A0E1}, + {0x10030, 0x0002A4DE}, + {0x10030, 0x0002A8DB}, + {0x10030, 0x0002ACD8}, + {0x10030, 0x0002B0D5}, + {0x10030, 0x0002B4D2}, + {0x10030, 0x0002B8CF}, + {0x10030, 0x0002BC0C}, + {0x10030, 0x0002C009}, + {0x10030, 0x000300F6}, + {0x10030, 0x000304F3}, + {0x10030, 0x000308F0}, + {0x10030, 0x00030CED}, + {0x10030, 0x000310EA}, + {0x10030, 0x000314E7}, + {0x10030, 0x000318E4}, + {0x10030, 0x00031CE1}, + {0x10030, 0x000320DE}, + {0x10030, 0x000324DB}, + {0x10030, 0x000328D8}, + {0x10030, 0x00032CD5}, + {0x10030, 0x000330D2}, + {0x10030, 0x000334CF}, + {0x10030, 0x000338CC}, + {0x10030, 0x00033C09}, + {0x10030, 0x00034006}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x000200FA}, + {0x10030, 0x000204F7}, + {0x10030, 0x000208F4}, + {0x10030, 0x00020CF1}, + {0x10030, 0x000210EE}, + {0x10030, 0x000214EB}, + {0x10030, 0x000218E8}, + {0x10030, 0x00021CE5}, + {0x10030, 0x000220E2}, + {0x10030, 0x000224DF}, + {0x10030, 0x000228DC}, + {0x10030, 0x00022CD9}, + {0x10030, 0x000230D6}, + {0x10030, 0x000234D3}, + {0x10030, 0x000238D0}, + {0x10030, 0x00023C0D}, + {0x10030, 0x0002400A}, + {0x10030, 0x000280F9}, + {0x10030, 0x000284F6}, + {0x10030, 0x000288F3}, + {0x10030, 0x00028CF0}, + {0x10030, 0x000290ED}, + {0x10030, 0x000294EA}, + {0x10030, 0x000298E7}, + {0x10030, 0x00029CE4}, + {0x10030, 0x0002A0E1}, + {0x10030, 0x0002A4DE}, + {0x10030, 0x0002A8DB}, + {0x10030, 0x0002ACD8}, + {0x10030, 0x0002B0D5}, + {0x10030, 0x0002B4D2}, + {0x10030, 0x0002B8CF}, + {0x10030, 0x0002BC0C}, + {0x10030, 0x0002C009}, + {0x10030, 0x000300F6}, + {0x10030, 0x000304F3}, + {0x10030, 0x000308F0}, + {0x10030, 0x00030CED}, + {0x10030, 0x000310EA}, + {0x10030, 0x000314E7}, + {0x10030, 0x000318E4}, + {0x10030, 0x00031CE1}, + {0x10030, 0x000320DE}, + {0x10030, 0x000324DB}, + {0x10030, 0x000328D8}, + {0x10030, 0x00032CD5}, + {0x10030, 0x000330D2}, + {0x10030, 0x000334CF}, + {0x10030, 0x000338CC}, + {0x10030, 0x00033C09}, + {0x10030, 0x00034006}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x000200FA}, + {0x10030, 0x000204F7}, + {0x10030, 0x000208F4}, + {0x10030, 0x00020CF1}, + {0x10030, 0x000210EE}, + {0x10030, 0x000214EB}, + {0x10030, 0x000218E8}, + {0x10030, 0x00021CE5}, + {0x10030, 0x000220E2}, + {0x10030, 0x000224DF}, + {0x10030, 0x000228DC}, + {0x10030, 0x00022CD9}, + {0x10030, 0x000230D6}, + {0x10030, 0x000234D3}, + {0x10030, 0x000238D0}, + {0x10030, 0x00023C0D}, + {0x10030, 0x0002400A}, + {0x10030, 0x000280F9}, + {0x10030, 0x000284F6}, + {0x10030, 0x000288F3}, + {0x10030, 0x00028CF0}, + {0x10030, 0x000290ED}, + {0x10030, 0x000294EA}, + {0x10030, 0x000298E7}, + {0x10030, 0x00029CE4}, + {0x10030, 0x0002A0E1}, + {0x10030, 0x0002A4DE}, + {0x10030, 0x0002A8DB}, + {0x10030, 0x0002ACD8}, + {0x10030, 0x0002B0D5}, + {0x10030, 0x0002B4D2}, + {0x10030, 0x0002B8CF}, + {0x10030, 0x0002BC0C}, + {0x10030, 0x0002C009}, + {0x10030, 0x000300F6}, + {0x10030, 0x000304F3}, + {0x10030, 0x000308F0}, + {0x10030, 0x00030CED}, + {0x10030, 0x000310EA}, + {0x10030, 0x000314E7}, + {0x10030, 0x000318E4}, + {0x10030, 0x00031CE1}, + {0x10030, 0x000320DE}, + {0x10030, 0x000324DB}, + {0x10030, 0x000328D8}, + {0x10030, 0x00032CD5}, + {0x10030, 0x000330D2}, + {0x10030, 0x000334CF}, + {0x10030, 0x000338CC}, + {0x10030, 0x00033C09}, + {0x10030, 0x00034006}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x000200FA}, + {0x10030, 0x000204F7}, + {0x10030, 0x000208F4}, + {0x10030, 0x00020CF1}, + {0x10030, 0x000210EE}, + {0x10030, 0x000214EB}, + {0x10030, 0x000218E8}, + {0x10030, 0x00021CE5}, + {0x10030, 0x000220E2}, + {0x10030, 0x000224DF}, + {0x10030, 0x000228DC}, + {0x10030, 0x00022CD9}, + {0x10030, 0x000230D6}, + {0x10030, 0x000234D3}, + {0x10030, 0x000238D0}, + {0x10030, 0x00023C0D}, + {0x10030, 0x0002400A}, + {0x10030, 0x000280F9}, + {0x10030, 0x000284F6}, + {0x10030, 0x000288F3}, + {0x10030, 0x00028CF0}, + {0x10030, 0x000290ED}, + {0x10030, 0x000294EA}, + {0x10030, 0x000298E7}, + {0x10030, 0x00029CE4}, + {0x10030, 0x0002A0E1}, + {0x10030, 0x0002A4DE}, + {0x10030, 0x0002A8DB}, + {0x10030, 0x0002ACD8}, + {0x10030, 0x0002B0D5}, + {0x10030, 0x0002B4D2}, + {0x10030, 0x0002B8CF}, + {0x10030, 0x0002BC0C}, + {0x10030, 0x0002C009}, + {0x10030, 0x000300F6}, + {0x10030, 0x000304F3}, + {0x10030, 0x000308F0}, + {0x10030, 0x00030CED}, + {0x10030, 0x000310EA}, + {0x10030, 0x000314E7}, + {0x10030, 0x000318E4}, + {0x10030, 0x00031CE1}, + {0x10030, 0x000320DE}, + {0x10030, 0x000324DB}, + {0x10030, 0x000328D8}, + {0x10030, 0x00032CD5}, + {0x10030, 0x000330D2}, + {0x10030, 0x000334CF}, + {0x10030, 0x000338CC}, + {0x10030, 0x00033C09}, + {0x10030, 0x00034006}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x000200FA}, + {0x10030, 0x000204F7}, + {0x10030, 0x000208F4}, + {0x10030, 0x00020CF1}, + {0x10030, 0x000210EE}, + {0x10030, 0x000214EB}, + {0x10030, 0x000218E8}, + {0x10030, 0x00021CE5}, + {0x10030, 0x000220E2}, + {0x10030, 0x000224DF}, + {0x10030, 0x000228DC}, + {0x10030, 0x00022CD9}, + {0x10030, 0x000230D6}, + {0x10030, 0x000234D3}, + {0x10030, 0x000238D0}, + {0x10030, 0x00023C0D}, + {0x10030, 0x0002400A}, + {0x10030, 0x000280F9}, + {0x10030, 0x000284F6}, + {0x10030, 0x000288F3}, + {0x10030, 0x00028CF0}, + {0x10030, 0x000290ED}, + {0x10030, 0x000294EA}, + {0x10030, 0x000298E7}, + {0x10030, 0x00029CE4}, + {0x10030, 0x0002A0E1}, + {0x10030, 0x0002A4DE}, + {0x10030, 0x0002A8DB}, + {0x10030, 0x0002ACD8}, + {0x10030, 0x0002B0D5}, + {0x10030, 0x0002B4D2}, + {0x10030, 0x0002B8CF}, + {0x10030, 0x0002BC0C}, + {0x10030, 0x0002C009}, + {0x10030, 0x000300F6}, + {0x10030, 0x000304F3}, + {0x10030, 0x000308F0}, + {0x10030, 0x00030CED}, + {0x10030, 0x000310EA}, + {0x10030, 0x000314E7}, + {0x10030, 0x000318E4}, + {0x10030, 0x00031CE1}, + {0x10030, 0x000320DE}, + {0x10030, 0x000324DB}, + {0x10030, 0x000328D8}, + {0x10030, 0x00032CD5}, + {0x10030, 0x000330D2}, + {0x10030, 0x000334CF}, + {0x10030, 0x000338CC}, + {0x10030, 0x00033C09}, + {0x10030, 0x00034006}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x000200FA}, + {0x10030, 0x000204F7}, + {0x10030, 0x000208F4}, + {0x10030, 0x00020CF1}, + {0x10030, 0x000210EE}, + {0x10030, 0x000214EB}, + {0x10030, 0x000218E8}, + {0x10030, 0x00021CE5}, + {0x10030, 0x000220E2}, + {0x10030, 0x000224DF}, + {0x10030, 0x000228DC}, + {0x10030, 0x00022CD9}, + {0x10030, 0x000230D6}, + {0x10030, 0x000234D3}, + {0x10030, 0x000238D0}, + {0x10030, 0x00023C0D}, + {0x10030, 0x0002400A}, + {0x10030, 0x000280F9}, + {0x10030, 0x000284F6}, + {0x10030, 0x000288F3}, + {0x10030, 0x00028CF0}, + {0x10030, 0x000290ED}, + {0x10030, 0x000294EA}, + {0x10030, 0x000298E7}, + {0x10030, 0x00029CE4}, + {0x10030, 0x0002A0E1}, + {0x10030, 0x0002A4DE}, + {0x10030, 0x0002A8DB}, + {0x10030, 0x0002ACD8}, + {0x10030, 0x0002B0D5}, + {0x10030, 0x0002B4D2}, + {0x10030, 0x0002B8CF}, + {0x10030, 0x0002BC0C}, + {0x10030, 0x0002C009}, + {0x10030, 0x000300F6}, + {0x10030, 0x000304F3}, + {0x10030, 0x000308F0}, + {0x10030, 0x00030CED}, + {0x10030, 0x000310EA}, + {0x10030, 0x000314E7}, + {0x10030, 0x000318E4}, + {0x10030, 0x00031CE1}, + {0x10030, 0x000320DE}, + {0x10030, 0x000324DB}, + {0x10030, 0x000328D8}, + {0x10030, 0x00032CD5}, + {0x10030, 0x000330D2}, + {0x10030, 0x000334CF}, + {0x10030, 0x000338CC}, + {0x10030, 0x00033C09}, + {0x10030, 0x00034006}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x000200FA}, + {0x10030, 0x000204F7}, + {0x10030, 0x000208F4}, + {0x10030, 0x00020CF1}, + {0x10030, 0x000210EE}, + {0x10030, 0x000214EB}, + {0x10030, 0x000218E8}, + {0x10030, 0x00021CE5}, + {0x10030, 0x000220E2}, + {0x10030, 0x000224DF}, + {0x10030, 0x000228DC}, + {0x10030, 0x00022CD9}, + {0x10030, 0x000230D6}, + {0x10030, 0x000234D3}, + {0x10030, 0x000238D0}, + {0x10030, 0x00023C0D}, + {0x10030, 0x0002400A}, + {0x10030, 0x000280F9}, + {0x10030, 0x000284F6}, + {0x10030, 0x000288F3}, + {0x10030, 0x00028CF0}, + {0x10030, 0x000290ED}, + {0x10030, 0x000294EA}, + {0x10030, 0x000298E7}, + {0x10030, 0x00029CE4}, + {0x10030, 0x0002A0E1}, + {0x10030, 0x0002A4DE}, + {0x10030, 0x0002A8DB}, + {0x10030, 0x0002ACD8}, + {0x10030, 0x0002B0D5}, + {0x10030, 0x0002B4D2}, + {0x10030, 0x0002B8CF}, + {0x10030, 0x0002BC0C}, + {0x10030, 0x0002C009}, + {0x10030, 0x000300F6}, + {0x10030, 0x000304F3}, + {0x10030, 0x000308F0}, + {0x10030, 0x00030CED}, + {0x10030, 0x000310EA}, + {0x10030, 0x000314E7}, + {0x10030, 0x000318E4}, + {0x10030, 0x00031CE1}, + {0x10030, 0x000320DE}, + {0x10030, 0x000324DB}, + {0x10030, 0x000328D8}, + {0x10030, 0x00032CD5}, + {0x10030, 0x000330D2}, + {0x10030, 0x000334CF}, + {0x10030, 0x000338CC}, + {0x10030, 0x00033C09}, + {0x10030, 0x00034006}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x000200E8}, + {0x10030, 0x000204E5}, + {0x10030, 0x000208E2}, + {0x10030, 0x00020CDF}, + {0x10030, 0x000210DC}, + {0x10030, 0x000214D9}, + {0x10030, 0x000218D6}, + {0x10030, 0x00021CD3}, + {0x10030, 0x000220D0}, + {0x10030, 0x000224CD}, + {0x10030, 0x000228CD}, + {0x10030, 0x00022CCD}, + {0x10030, 0x000230CD}, + {0x10030, 0x000234CD}, + {0x10030, 0x000238CD}, + {0x10030, 0x00023CCD}, + {0x10030, 0x000240CD}, + {0x10030, 0x000280E8}, + {0x10030, 0x000284E5}, + {0x10030, 0x000288E2}, + {0x10030, 0x00028CDF}, + {0x10030, 0x000290DC}, + {0x10030, 0x000294D9}, + {0x10030, 0x000298D6}, + {0x10030, 0x00029CD3}, + {0x10030, 0x0002A0D0}, + {0x10030, 0x0002A4CD}, + {0x10030, 0x0002A8CD}, + {0x10030, 0x0002ACCD}, + {0x10030, 0x0002B0CD}, + {0x10030, 0x0002B4CD}, + {0x10030, 0x0002B8CD}, + {0x10030, 0x0002BCCD}, + {0x10030, 0x0002C0CD}, + {0x10030, 0x000300E8}, + {0x10030, 0x000304E5}, + {0x10030, 0x000308E2}, + {0x10030, 0x00030CDF}, + {0x10030, 0x000310DC}, + {0x10030, 0x000314D9}, + {0x10030, 0x000318D6}, + {0x10030, 0x00031CD3}, + {0x10030, 0x000320D0}, + {0x10030, 0x000324CD}, + {0x10030, 0x000328CD}, + {0x10030, 0x00032CCD}, + {0x10030, 0x000330CD}, + {0x10030, 0x000334CD}, + {0x10030, 0x000338CD}, + {0x10030, 0x00033CCD}, + {0x10030, 0x000340CD}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x000200E8}, + {0x10030, 0x000204E5}, + {0x10030, 0x000208E2}, + {0x10030, 0x00020CDF}, + {0x10030, 0x000210DC}, + {0x10030, 0x000214D9}, + {0x10030, 0x000218D6}, + {0x10030, 0x00021CD3}, + {0x10030, 0x000220D0}, + {0x10030, 0x000224CD}, + {0x10030, 0x000228CD}, + {0x10030, 0x00022CCD}, + {0x10030, 0x000230CD}, + {0x10030, 0x000234CD}, + {0x10030, 0x000238CD}, + {0x10030, 0x00023CCD}, + {0x10030, 0x000240CD}, + {0x10030, 0x000280E8}, + {0x10030, 0x000284E5}, + {0x10030, 0x000288E2}, + {0x10030, 0x00028CDF}, + {0x10030, 0x000290DC}, + {0x10030, 0x000294D9}, + {0x10030, 0x000298D6}, + {0x10030, 0x00029CD3}, + {0x10030, 0x0002A0D0}, + {0x10030, 0x0002A4CD}, + {0x10030, 0x0002A8CD}, + {0x10030, 0x0002ACCD}, + {0x10030, 0x0002B0CD}, + {0x10030, 0x0002B4CD}, + {0x10030, 0x0002B8CD}, + {0x10030, 0x0002BCCD}, + {0x10030, 0x0002C0CD}, + {0x10030, 0x000300E8}, + {0x10030, 0x000304E5}, + {0x10030, 0x000308E2}, + {0x10030, 0x00030CDF}, + {0x10030, 0x000310DC}, + {0x10030, 0x000314D9}, + {0x10030, 0x000318D6}, + {0x10030, 0x00031CD3}, + {0x10030, 0x000320D0}, + {0x10030, 0x000324CD}, + {0x10030, 0x000328CD}, + {0x10030, 0x00032CCD}, + {0x10030, 0x000330CD}, + {0x10030, 0x000334CD}, + {0x10030, 0x000338CD}, + {0x10030, 0x00033CCD}, + {0x10030, 0x000340CD}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x000200E8}, + {0x10030, 0x000204E5}, + {0x10030, 0x000208E2}, + {0x10030, 0x00020CDF}, + {0x10030, 0x000210DC}, + {0x10030, 0x000214D9}, + {0x10030, 0x000218D6}, + {0x10030, 0x00021CD3}, + {0x10030, 0x000220D0}, + {0x10030, 0x000224CD}, + {0x10030, 0x000228CD}, + {0x10030, 0x00022CCD}, + {0x10030, 0x000230CD}, + {0x10030, 0x000234CD}, + {0x10030, 0x000238CD}, + {0x10030, 0x00023CCD}, + {0x10030, 0x000240CD}, + {0x10030, 0x000280E8}, + {0x10030, 0x000284E5}, + {0x10030, 0x000288E2}, + {0x10030, 0x00028CDF}, + {0x10030, 0x000290DC}, + {0x10030, 0x000294D9}, + {0x10030, 0x000298D6}, + {0x10030, 0x00029CD3}, + {0x10030, 0x0002A0D0}, + {0x10030, 0x0002A4CD}, + {0x10030, 0x0002A8CD}, + {0x10030, 0x0002ACCD}, + {0x10030, 0x0002B0CD}, + {0x10030, 0x0002B4CD}, + {0x10030, 0x0002B8CD}, + {0x10030, 0x0002BCCD}, + {0x10030, 0x0002C0CD}, + {0x10030, 0x000300E8}, + {0x10030, 0x000304E5}, + {0x10030, 0x000308E2}, + {0x10030, 0x00030CDF}, + {0x10030, 0x000310DC}, + {0x10030, 0x000314D9}, + {0x10030, 0x000318D6}, + {0x10030, 0x00031CD3}, + {0x10030, 0x000320D0}, + {0x10030, 0x000324CD}, + {0x10030, 0x000328CD}, + {0x10030, 0x00032CCD}, + {0x10030, 0x000330CD}, + {0x10030, 0x000334CD}, + {0x10030, 0x000338CD}, + {0x10030, 0x00033CCD}, + {0x10030, 0x000340CD}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x000200E8}, + {0x10030, 0x000204E5}, + {0x10030, 0x000208E2}, + {0x10030, 0x00020CDF}, + {0x10030, 0x000210DC}, + {0x10030, 0x000214D9}, + {0x10030, 0x000218D6}, + {0x10030, 0x00021CD3}, + {0x10030, 0x000220D0}, + {0x10030, 0x000224CD}, + {0x10030, 0x000228CD}, + {0x10030, 0x00022CCD}, + {0x10030, 0x000230CD}, + {0x10030, 0x000234CD}, + {0x10030, 0x000238CD}, + {0x10030, 0x00023CCD}, + {0x10030, 0x000240CD}, + {0x10030, 0x000280E8}, + {0x10030, 0x000284E5}, + {0x10030, 0x000288E2}, + {0x10030, 0x00028CDF}, + {0x10030, 0x000290DC}, + {0x10030, 0x000294D9}, + {0x10030, 0x000298D6}, + {0x10030, 0x00029CD3}, + {0x10030, 0x0002A0D0}, + {0x10030, 0x0002A4CD}, + {0x10030, 0x0002A8CD}, + {0x10030, 0x0002ACCD}, + {0x10030, 0x0002B0CD}, + {0x10030, 0x0002B4CD}, + {0x10030, 0x0002B8CD}, + {0x10030, 0x0002BCCD}, + {0x10030, 0x0002C0CD}, + {0x10030, 0x000300E8}, + {0x10030, 0x000304E5}, + {0x10030, 0x000308E2}, + {0x10030, 0x00030CDF}, + {0x10030, 0x000310DC}, + {0x10030, 0x000314D9}, + {0x10030, 0x000318D6}, + {0x10030, 0x00031CD3}, + {0x10030, 0x000320D0}, + {0x10030, 0x000324CD}, + {0x10030, 0x000328CD}, + {0x10030, 0x00032CCD}, + {0x10030, 0x000330CD}, + {0x10030, 0x000334CD}, + {0x10030, 0x000338CD}, + {0x10030, 0x00033CCD}, + {0x10030, 0x000340CD}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x000200E8}, + {0x10030, 0x000204E5}, + {0x10030, 0x000208E2}, + {0x10030, 0x00020CDF}, + {0x10030, 0x000210DC}, + {0x10030, 0x000214D9}, + {0x10030, 0x000218D6}, + {0x10030, 0x00021CD3}, + {0x10030, 0x000220D0}, + {0x10030, 0x000224CD}, + {0x10030, 0x000228CD}, + {0x10030, 0x00022CCD}, + {0x10030, 0x000230CD}, + {0x10030, 0x000234CD}, + {0x10030, 0x000238CD}, + {0x10030, 0x00023CCD}, + {0x10030, 0x000240CD}, + {0x10030, 0x000280E8}, + {0x10030, 0x000284E5}, + {0x10030, 0x000288E2}, + {0x10030, 0x00028CDF}, + {0x10030, 0x000290DC}, + {0x10030, 0x000294D9}, + {0x10030, 0x000298D6}, + {0x10030, 0x00029CD3}, + {0x10030, 0x0002A0D0}, + {0x10030, 0x0002A4CD}, + {0x10030, 0x0002A8CD}, + {0x10030, 0x0002ACCD}, + {0x10030, 0x0002B0CD}, + {0x10030, 0x0002B4CD}, + {0x10030, 0x0002B8CD}, + {0x10030, 0x0002BCCD}, + {0x10030, 0x0002C0CD}, + {0x10030, 0x000300E8}, + {0x10030, 0x000304E5}, + {0x10030, 0x000308E2}, + {0x10030, 0x00030CDF}, + {0x10030, 0x000310DC}, + {0x10030, 0x000314D9}, + {0x10030, 0x000318D6}, + {0x10030, 0x00031CD3}, + {0x10030, 0x000320D0}, + {0x10030, 0x000324CD}, + {0x10030, 0x000328CD}, + {0x10030, 0x00032CCD}, + {0x10030, 0x000330CD}, + {0x10030, 0x000334CD}, + {0x10030, 0x000338CD}, + {0x10030, 0x00033CCD}, + {0x10030, 0x000340CD}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x000200E8}, + {0x10030, 0x000204E5}, + {0x10030, 0x000208E2}, + {0x10030, 0x00020CDF}, + {0x10030, 0x000210DC}, + {0x10030, 0x000214D9}, + {0x10030, 0x000218D6}, + {0x10030, 0x00021CD3}, + {0x10030, 0x000220D0}, + {0x10030, 0x000224CD}, + {0x10030, 0x000228CD}, + {0x10030, 0x00022CCD}, + {0x10030, 0x000230CD}, + {0x10030, 0x000234CD}, + {0x10030, 0x000238CD}, + {0x10030, 0x00023CCD}, + {0x10030, 0x000240CD}, + {0x10030, 0x000280E8}, + {0x10030, 0x000284E5}, + {0x10030, 0x000288E2}, + {0x10030, 0x00028CDF}, + {0x10030, 0x000290DC}, + {0x10030, 0x000294D9}, + {0x10030, 0x000298D6}, + {0x10030, 0x00029CD3}, + {0x10030, 0x0002A0D0}, + {0x10030, 0x0002A4CD}, + {0x10030, 0x0002A8CD}, + {0x10030, 0x0002ACCD}, + {0x10030, 0x0002B0CD}, + {0x10030, 0x0002B4CD}, + {0x10030, 0x0002B8CD}, + {0x10030, 0x0002BCCD}, + {0x10030, 0x0002C0CD}, + {0x10030, 0x000300E8}, + {0x10030, 0x000304E5}, + {0x10030, 0x000308E2}, + {0x10030, 0x00030CDF}, + {0x10030, 0x000310DC}, + {0x10030, 0x000314D9}, + {0x10030, 0x000318D6}, + {0x10030, 0x00031CD3}, + {0x10030, 0x000320D0}, + {0x10030, 0x000324CD}, + {0x10030, 0x000328CD}, + {0x10030, 0x00032CCD}, + {0x10030, 0x000330CD}, + {0x10030, 0x000334CD}, + {0x10030, 0x000338CD}, + {0x10030, 0x00033CCD}, + {0x10030, 0x000340CD}, + {0xA0000000, 0x00000000}, + {0x10030, 0x000200FA}, + {0x10030, 0x000204F7}, + {0x10030, 0x000208F4}, + {0x10030, 0x00020CF1}, + {0x10030, 0x000210EE}, + {0x10030, 0x000214EB}, + {0x10030, 0x000218E8}, + {0x10030, 0x00021CE5}, + {0x10030, 0x000220E2}, + {0x10030, 0x000224DF}, + {0x10030, 0x000228DC}, + {0x10030, 0x00022CD9}, + {0x10030, 0x000230D6}, + {0x10030, 0x000234D3}, + {0x10030, 0x000238D0}, + {0x10030, 0x00023C0D}, + {0x10030, 0x0002400A}, + {0x10030, 0x000280F9}, + {0x10030, 0x000284F6}, + {0x10030, 0x000288F3}, + {0x10030, 0x00028CF0}, + {0x10030, 0x000290ED}, + {0x10030, 0x000294EA}, + {0x10030, 0x000298E7}, + {0x10030, 0x00029CE4}, + {0x10030, 0x0002A0E1}, + {0x10030, 0x0002A4DE}, + {0x10030, 0x0002A8DB}, + {0x10030, 0x0002ACD8}, + {0x10030, 0x0002B0D5}, + {0x10030, 0x0002B4D2}, + {0x10030, 0x0002B8CF}, + {0x10030, 0x0002BC0C}, + {0x10030, 0x0002C009}, + {0x10030, 0x000300F6}, + {0x10030, 0x000304F3}, + {0x10030, 0x000308F0}, + {0x10030, 0x00030CED}, + {0x10030, 0x000310EA}, + {0x10030, 0x000314E7}, + {0x10030, 0x000318E4}, + {0x10030, 0x00031CE1}, + {0x10030, 0x000320DE}, + {0x10030, 0x000324DB}, + {0x10030, 0x000328D8}, + {0x10030, 0x00032CD5}, + {0x10030, 0x000330D2}, + {0x10030, 0x000334CF}, + {0x10030, 0x000338CC}, + {0x10030, 0x00033C09}, + {0x10030, 0x00034006}, + {0xB0000000, 0x00000000}, + {0x10030, 0x000600F6}, + {0x10030, 0x000604F3}, + {0x10030, 0x000608F0}, + {0x10030, 0x00060CED}, + {0x10030, 0x000610EA}, + {0x10030, 0x000614E7}, + {0x10030, 0x000618E4}, + {0x10030, 0x00061CE1}, + {0x10030, 0x000620DE}, + {0x10030, 0x000624DB}, + {0x10030, 0x000628D8}, + {0x10030, 0x00062CD5}, + {0x10030, 0x000630D2}, + {0x10030, 0x000634CF}, + {0x10030, 0x000638CC}, + {0x10030, 0x00063C09}, + {0x10030, 0x00064006}, + {0x10030, 0x000680F5}, + {0x10030, 0x000684F2}, + {0x10030, 0x000688EF}, + {0x10030, 0x00068CEC}, + {0x10030, 0x000690E9}, + {0x10030, 0x000694E6}, + {0x10030, 0x000698E3}, + {0x10030, 0x00069CE0}, + {0x10030, 0x0006A0DD}, + {0x10030, 0x0006A4DA}, + {0x10030, 0x0006A8D7}, + {0x10030, 0x0006ACD4}, + {0x10030, 0x0006B0D1}, + {0x10030, 0x0006B4CE}, + {0x10030, 0x0006B8CB}, + {0x10030, 0x0006BC08}, + {0x10030, 0x0006C005}, + {0x10030, 0x000700F5}, + {0x10030, 0x000704F2}, + {0x10030, 0x000708EF}, + {0x10030, 0x00070CEC}, + {0x10030, 0x000710E9}, + {0x10030, 0x000714E6}, + {0x10030, 0x000718E3}, + {0x10030, 0x00071CE0}, + {0x10030, 0x000720DD}, + {0x10030, 0x000724DA}, + {0x10030, 0x000728D7}, + {0x10030, 0x00072CD4}, + {0x10030, 0x000730D1}, + {0x10030, 0x000734CE}, + {0x10030, 0x000738CB}, + {0x10030, 0x00073C08}, + {0x10030, 0x00074005}, {0x10030, 0x000780F4}, {0x10030, 0x000784F1}, {0x10030, 0x000788EE}, @@ -6777,9 +11045,93 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x033, 0x00000025}, {0x03F, 0x00008002}, {0x033, 0x00000026}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x00000027}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x00000028}, {0x03F, 0x00050002}, {0x033, 0x00000029}, @@ -6793,9 +11145,93 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x033, 0x0000002D}, {0x03F, 0x00008002}, {0x033, 0x0000002E}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x0000002F}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x00000030}, {0x03F, 0x00050002}, {0x033, 0x00000031}, @@ -6809,9 +11245,93 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x033, 0x00000035}, {0x03F, 0x00008002}, {0x033, 0x00000036}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x00000037}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x00000060}, {0x03F, 0x00050002}, {0x033, 0x00000061}, @@ -6825,9 +11345,93 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x033, 0x00000065}, {0x03F, 0x00008002}, {0x033, 0x00000066}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x00000067}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x00000068}, {0x03F, 0x00050002}, {0x033, 0x00000069}, @@ -6841,9 +11445,93 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x033, 0x0000006D}, {0x03F, 0x00008002}, {0x033, 0x0000006E}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x0000006F}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x00000070}, {0x03F, 0x00050002}, {0x033, 0x00000071}, @@ -6857,9 +11545,93 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x033, 0x00000075}, {0x03F, 0x00008002}, {0x033, 0x00000076}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x00000077}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x00000078}, {0x03F, 0x00050002}, {0x033, 0x00000079}, @@ -6873,9 +11645,93 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x033, 0x0000007D}, {0x03F, 0x00008002}, {0x033, 0x0000007E}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x0000007F}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x000000A0}, {0x03F, 0x00050002}, {0x033, 0x000000A1}, @@ -6889,9 +11745,93 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x033, 0x000000A5}, {0x03F, 0x00008002}, {0x033, 0x000000A6}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x000000A7}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x000000A8}, {0x03F, 0x00050002}, {0x033, 0x000000A9}, @@ -6905,9 +11845,93 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x033, 0x000000AD}, {0x03F, 0x00008002}, {0x033, 0x000000AE}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x000000AF}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x000000B0}, {0x03F, 0x00050002}, {0x033, 0x000000B1}, @@ -6921,9 +11945,93 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x033, 0x000000B5}, {0x03F, 0x00008002}, {0x033, 0x000000B6}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x000000B7}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x000000E0}, {0x03F, 0x00050002}, {0x033, 0x000000E1}, @@ -6937,9 +12045,93 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x033, 0x000000E5}, {0x03F, 0x00008002}, {0x033, 0x000000E6}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x000000E7}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x000000E8}, {0x03F, 0x00050002}, {0x033, 0x000000E9}, @@ -6953,9 +12145,93 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x033, 0x000000ED}, {0x03F, 0x00008002}, {0x033, 0x000000EE}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x000000EF}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x000000F0}, {0x03F, 0x00050002}, {0x033, 0x000000F1}, @@ -6969,9 +12245,93 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x033, 0x000000F5}, {0x03F, 0x00008002}, {0x033, 0x000000F6}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x000000F7}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x000000F8}, {0x03F, 0x00050002}, {0x033, 0x000000F9}, @@ -6985,9 +12345,93 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x033, 0x000000FD}, {0x03F, 0x00008002}, {0x033, 0x000000FE}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x000000FF}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x00000120}, {0x03F, 0x00050002}, {0x033, 0x00000121}, @@ -7001,9 +12445,93 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x033, 0x00000125}, {0x03F, 0x00008002}, {0x033, 0x00000126}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x00000127}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x00000128}, {0x03F, 0x00050002}, {0x033, 0x00000129}, @@ -7017,9 +12545,93 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x033, 0x0000012D}, {0x03F, 0x00008002}, {0x033, 0x0000012E}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x0000012F}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x00000130}, {0x03F, 0x00050002}, {0x033, 0x00000131}, @@ -7033,9 +12645,93 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x033, 0x00000135}, {0x03F, 0x00008002}, {0x033, 0x00000136}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x00000137}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x00000160}, {0x03F, 0x00050002}, {0x033, 0x00000161}, @@ -7049,9 +12745,93 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x033, 0x00000165}, {0x03F, 0x00008002}, {0x033, 0x00000166}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x00000167}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x00000168}, {0x03F, 0x00050002}, {0x033, 0x00000169}, @@ -7065,9 +12845,93 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x033, 0x0000016D}, {0x03F, 0x00008002}, {0x033, 0x0000016E}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x0000016F}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x00000170}, {0x03F, 0x00050002}, {0x033, 0x00000171}, @@ -7081,9 +12945,93 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x033, 0x00000175}, {0x03F, 0x00008002}, {0x033, 0x00000176}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x00000177}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x00000178}, {0x03F, 0x00050002}, {0x033, 0x00000179}, @@ -7097,9 +13045,93 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x033, 0x0000017D}, {0x03F, 0x00008002}, {0x033, 0x0000017E}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x0000017F}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x000001A0}, {0x03F, 0x00050002}, {0x033, 0x000001A1}, @@ -7113,9 +13145,93 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x033, 0x000001A5}, {0x03F, 0x00008002}, {0x033, 0x000001A6}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x000001A7}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x000001A8}, {0x03F, 0x00050002}, {0x033, 0x000001A9}, @@ -7129,9 +13245,93 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x033, 0x000001AD}, {0x03F, 0x00008002}, {0x033, 0x000001AE}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x000001AF}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x000001B0}, {0x03F, 0x00050002}, {0x033, 0x000001B1}, @@ -7145,9 +13345,93 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x033, 0x000001B5}, {0x03F, 0x00008002}, {0x033, 0x000001B6}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x000001B7}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x000001E0}, {0x03F, 0x00050002}, {0x033, 0x000001E1}, @@ -7161,9 +13445,93 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x033, 0x000001E5}, {0x03F, 0x00008002}, {0x033, 0x000001E6}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x000001E7}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x000001E8}, {0x03F, 0x00050002}, {0x033, 0x000001E9}, @@ -7177,9 +13545,93 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x033, 0x000001ED}, {0x03F, 0x00008002}, {0x033, 0x000001EE}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x000001EF}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x000001F0}, {0x03F, 0x00050002}, {0x033, 0x000001F1}, @@ -7193,9 +13645,93 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x033, 0x000001F5}, {0x03F, 0x00008002}, {0x033, 0x000001F6}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x000001F7}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x000001F8}, {0x03F, 0x00050002}, {0x033, 0x000001F9}, @@ -7209,9 +13745,93 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x033, 0x000001FD}, {0x03F, 0x00008002}, {0x033, 0x000001FE}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x000001FF}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x0EF, 0x00000000}, {0x005, 0x00000001}, {0x10005, 0x00000001}, @@ -7253,7 +13873,49 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x10030, 0x00022000}, {0x10030, 0x00023000}, {0x10030, 0x00024000}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00025000}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00025000}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00025000}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, {0x10030, 0x00025000}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00025000}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00025000}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00025000}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00025003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00025003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00025003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00025003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00025003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00025003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00025003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00025003}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00025003}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00025003}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00025003}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00025003}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00025003}, + {0xA0000000, 0x00000000}, + {0x10030, 0x00025000}, + {0xB0000000, 0x00000000}, {0x10030, 0x00026003}, {0x10030, 0x00027003}, {0x10030, 0x00028000}, @@ -7261,7 +13923,49 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x10030, 0x0002A000}, {0x10030, 0x0002B000}, {0x10030, 0x0002C000}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0002D000}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0002D000}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0002D000}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0002D000}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, {0x10030, 0x0002D000}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0002D000}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0002D000}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0002D003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0002D003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0002D003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0002D003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0002D003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0002D003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0002D003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0002D003}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0002D003}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0002D003}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0002D003}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0002D003}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0002D003}, + {0xA0000000, 0x00000000}, + {0x10030, 0x0002D000}, + {0xB0000000, 0x00000000}, {0x10030, 0x0002E003}, {0x10030, 0x0002F003}, {0x10030, 0x00030000}, @@ -7269,7 +13973,49 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x10030, 0x00032000}, {0x10030, 0x00033000}, {0x10030, 0x00034000}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00035000}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00035000}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00035000}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, {0x10030, 0x00035000}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00035000}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00035000}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00035000}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00035003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00035003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00035003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00035003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00035003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00035003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00035003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00035003}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00035003}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00035003}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00035003}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00035003}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00035003}, + {0xA0000000, 0x00000000}, + {0x10030, 0x00035000}, + {0xB0000000, 0x00000000}, {0x10030, 0x00036003}, {0x10030, 0x00037003}, {0x10030, 0x00038000}, @@ -7277,7 +14023,49 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x10030, 0x0003A000}, {0x10030, 0x0003B000}, {0x10030, 0x0003C000}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0003D000}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, {0x10030, 0x0003D000}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0003D000}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0003D000}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0003D000}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0003D000}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0003D000}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0003D003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0003D003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0003D003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0003D003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0003D003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0003D003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0003D003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0003D003}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0003D003}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0003D003}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0003D003}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0003D003}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0003D003}, + {0xA0000000, 0x00000000}, + {0x10030, 0x0003D000}, + {0xB0000000, 0x00000000}, {0x10030, 0x0003E003}, {0x10030, 0x0003F003}, {0x10030, 0x00060000}, @@ -7285,35 +14073,283 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radioa_regs[] = { {0x10030, 0x00062000}, {0x10030, 0x00063000}, {0x10030, 0x00064000}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00065000}, + {0x10030, 0x00066000}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, {0x10030, 0x00065000}, {0x10030, 0x00066000}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00065000}, + {0x10030, 0x00066000}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00065000}, + {0x10030, 0x00066000}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00065000}, + {0x10030, 0x00066000}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00065000}, + {0x10030, 0x00066000}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00065000}, + {0x10030, 0x00066000}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00065003}, + {0x10030, 0x00066003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00065003}, + {0x10030, 0x00066003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00065003}, + {0x10030, 0x00066003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00065003}, + {0x10030, 0x00066003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00065003}, + {0x10030, 0x00066003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00065003}, + {0x10030, 0x00066003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00065003}, + {0x10030, 0x00066003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00065003}, + {0x10030, 0x00066003}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00065003}, + {0x10030, 0x00066003}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00065003}, + {0x10030, 0x00066003}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00065003}, + {0x10030, 0x00066003}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00065003}, + {0x10030, 0x00066003}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00065003}, + {0x10030, 0x00066003}, + {0xA0000000, 0x00000000}, + {0x10030, 0x00065000}, + {0x10030, 0x00066000}, + {0xB0000000, 0x00000000}, {0x10030, 0x00067003}, {0x10030, 0x00068000}, {0x10030, 0x00069000}, {0x10030, 0x0006A000}, {0x10030, 0x0006B000}, {0x10030, 0x0006C000}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0006D000}, + {0x10030, 0x0006E000}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0006D000}, + {0x10030, 0x0006E000}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0006D000}, + {0x10030, 0x0006E000}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0006D000}, + {0x10030, 0x0006E000}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0006D000}, + {0x10030, 0x0006E000}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0006D000}, + {0x10030, 0x0006E000}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, {0x10030, 0x0006D000}, {0x10030, 0x0006E000}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0006D003}, + {0x10030, 0x0006E003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0006D003}, + {0x10030, 0x0006E003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0006D003}, + {0x10030, 0x0006E003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0006D003}, + {0x10030, 0x0006E003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0006D003}, + {0x10030, 0x0006E003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0006D003}, + {0x10030, 0x0006E003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0006D003}, + {0x10030, 0x0006E003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0006D003}, + {0x10030, 0x0006E003}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0006D003}, + {0x10030, 0x0006E003}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0006D003}, + {0x10030, 0x0006E003}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0006D003}, + {0x10030, 0x0006E003}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0006D003}, + {0x10030, 0x0006E003}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0006D003}, + {0x10030, 0x0006E003}, + {0xA0000000, 0x00000000}, + {0x10030, 0x0006D000}, + {0x10030, 0x0006E000}, + {0xB0000000, 0x00000000}, {0x10030, 0x0006F003}, {0x10030, 0x00070000}, {0x10030, 0x00071000}, {0x10030, 0x00072000}, {0x10030, 0x00073000}, {0x10030, 0x00074000}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00075000}, + {0x10030, 0x00076000}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00075000}, + {0x10030, 0x00076000}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00075000}, + {0x10030, 0x00076000}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00075000}, + {0x10030, 0x00076000}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00075000}, + {0x10030, 0x00076000}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00075000}, + {0x10030, 0x00076000}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00075000}, + {0x10030, 0x00076000}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00075003}, + {0x10030, 0x00076003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00075003}, + {0x10030, 0x00076003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00075003}, + {0x10030, 0x00076003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00075003}, + {0x10030, 0x00076003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00075003}, + {0x10030, 0x00076003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00075003}, + {0x10030, 0x00076003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00075003}, + {0x10030, 0x00076003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00075003}, + {0x10030, 0x00076003}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00075003}, + {0x10030, 0x00076003}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00075003}, + {0x10030, 0x00076003}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00075003}, + {0x10030, 0x00076003}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00075003}, + {0x10030, 0x00076003}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00075003}, + {0x10030, 0x00076003}, + {0xA0000000, 0x00000000}, {0x10030, 0x00075000}, {0x10030, 0x00076000}, + {0xB0000000, 0x00000000}, {0x10030, 0x00077003}, {0x10030, 0x00078000}, {0x10030, 0x00079000}, {0x10030, 0x0007A000}, {0x10030, 0x0007B000}, {0x10030, 0x0007C000}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, {0x10030, 0x0007D000}, {0x10030, 0x0007E000}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0007D000}, + {0x10030, 0x0007E000}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0007D000}, + {0x10030, 0x0007E000}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0007D000}, + {0x10030, 0x0007E000}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0007D000}, + {0x10030, 0x0007E000}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0007D000}, + {0x10030, 0x0007E000}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0007D000}, + {0x10030, 0x0007E000}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0007D003}, + {0x10030, 0x0007E003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0007D003}, + {0x10030, 0x0007E003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0007D003}, + {0x10030, 0x0007E003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0007D003}, + {0x10030, 0x0007E003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0007D003}, + {0x10030, 0x0007E003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0007D003}, + {0x10030, 0x0007E003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0007D003}, + {0x10030, 0x0007E003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0007D003}, + {0x10030, 0x0007E003}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0007D003}, + {0x10030, 0x0007E003}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0007D003}, + {0x10030, 0x0007E003}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0007D003}, + {0x10030, 0x0007E003}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0007D003}, + {0x10030, 0x0007E003}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0007D003}, + {0x10030, 0x0007E003}, + {0xA0000000, 0x00000000}, + {0x10030, 0x0007D000}, + {0x10030, 0x0007E000}, + {0xB0000000, 0x00000000}, {0x10030, 0x0007F003}, {0x100EE, 0x00000000}, - {0x0FE, 0x00000031}, + {0x0FE, 0x00000048}, }; static const struct rtw89_reg2_def rtw89_8852c_phy_radiob_regs[] = { @@ -7326,13 +14362,17 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radiob_regs[] = { {0xF0360000, 0x00000006}, {0xF0010001, 0x00000007}, {0xF0020001, 0x00000008}, - {0xF0320001, 0x00000009}, - {0xF0330001, 0x0000000A}, - {0xF0340001, 0x0000000B}, - {0xF0350001, 0x0000000C}, - {0xF0360001, 0x0000000D}, - {0xF03F0001, 0x0000000E}, - {0xF0400001, 0x0000000F}, + {0xF0030001, 0x00000009}, + {0xF0040001, 0x0000000A}, + {0xF0050001, 0x0000000B}, + {0xF0070001, 0x0000000C}, + {0xF0320001, 0x0000000D}, + {0xF0330001, 0x0000000E}, + {0xF0340001, 0x0000000F}, + {0xF0350001, 0x00000010}, + {0xF0360001, 0x00000011}, + {0xF03F0001, 0x00000012}, + {0xF0400001, 0x00000013}, {0x005, 0x00000000}, {0x10005, 0x00000000}, {0x0B9, 0x00020440}, @@ -7340,42 +14380,69 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radiob_regs[] = { {0x10000, 0x00030000}, {0x018, 0x00011124}, {0x10018, 0x00011124}, - {0x05F, 0x00000032}, + {0x05F, 0x00000038}, {0x097, 0x00043200}, {0x0A6, 0x00066DB7}, {0x0EF, 0x00004000}, {0x033, 0x00000005}, {0x03E, 0x00000000}, {0x03F, 0x00010500}, + {0x033, 0x00000004}, + {0x03E, 0x00000000}, + {0x03F, 0x00000400}, {0x033, 0x00000003}, {0x03E, 0x00000000}, {0x03F, 0x00028B00}, {0x033, 0x00000002}, {0x03E, 0x00000000}, {0x03F, 0x0009AB00}, + {0x033, 0x00000001}, + {0x03E, 0x00000000}, + {0x03F, 0x00001A00}, + {0x033, 0x00000000}, + {0x03E, 0x00000000}, + {0x03F, 0x00002900}, {0x033, 0x0000000D}, {0x03E, 0x00000000}, {0x03F, 0x00010500}, + {0x033, 0x0000000C}, + {0x03E, 0x00000000}, + {0x03F, 0x00000400}, {0x033, 0x0000000B}, {0x03E, 0x00000000}, {0x03F, 0x00028B00}, {0x033, 0x0000000A}, {0x03E, 0x00000000}, {0x03F, 0x0009AB00}, + {0x033, 0x00000009}, + {0x03E, 0x00000000}, + {0x03F, 0x00001A00}, + {0x033, 0x00000008}, + {0x03E, 0x00000000}, + {0x03F, 0x00002900}, {0x033, 0x00000015}, {0x03E, 0x00000000}, {0x03F, 0x00010500}, + {0x033, 0x00000014}, + {0x03E, 0x00000000}, + {0x03F, 0x00000400}, {0x033, 0x00000013}, {0x03E, 0x00000000}, {0x03F, 0x00028B00}, {0x033, 0x00000012}, {0x03E, 0x00000000}, {0x03F, 0x0009AB00}, + {0x033, 0x00000011}, + {0x03E, 0x00000000}, + {0x03F, 0x00001A00}, + {0x033, 0x00000010}, + {0x03E, 0x00000000}, + {0x03F, 0x00002900}, {0x0EF, 0x00000000}, + {0x10055, 0x00080080}, {0x000, 0x00033C01}, {0x10000, 0x00033C00}, {0x01A, 0x00040004}, - {0x0FE, 0x00000000}, {0x096, 0x00015200}, {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, {0x067, 0x0004D000}, @@ -7404,6 +14471,18 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radiob_regs[] = { {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, {0x067, 0x0000D300}, {0x0DA, 0x000D4000}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x067, 0x0000D300}, + {0x0DA, 0x000D4000}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x067, 0x0000D300}, + {0x0DA, 0x000D4000}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x067, 0x0000D300}, + {0x0DA, 0x000D4000}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x067, 0x0000D300}, + {0x0DA, 0x000D4000}, {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, {0x067, 0x0000D300}, {0x0DA, 0x000D4000}, @@ -7430,7 +14509,7 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radiob_regs[] = { {0x0DA, 0x000D4009}, {0xB0000000, 0x00000000}, {0x057, 0x0000D589}, - {0x05A, 0x0007FFFF}, + {0x05A, 0x0007F0F8}, {0x043, 0x00005000}, {0x018, 0x00001001}, {0x10018, 0x00001001}, @@ -7462,6 +14541,14 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radiob_regs[] = { {0x08F, 0x000D1352}, {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, {0x08F, 0x000D1352}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x08F, 0x000D1352}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x08F, 0x000D1352}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x08F, 0x000D1352}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x08F, 0x000D1352}, {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, {0x08F, 0x000D1352}, {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, @@ -7496,6 +14583,52 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radiob_regs[] = { {0x03F, 0x00000015}, {0x033, 0x00000001}, {0x03F, 0x00000017}, + {0x033, 0x00000004}, + {0x03F, 0x00000017}, + {0x033, 0x00000005}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000007}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000007}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000007}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000017}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000017}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000017}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000017}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000007}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000007}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000007}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000007}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000007}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000007}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000007}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000017}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000017}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000017}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000017}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000017}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000017}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000007}, + {0xB0000000, 0x00000000}, {0x0EF, 0x00000000}, {0x0EF, 0x00008000}, {0x033, 0x00000020}, @@ -8007,6 +15140,14 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radiob_regs[] = { {0x03F, 0x0000EFFF}, {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x0000EFFF}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000EFFF}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000EFFF}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000EFFF}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000EFFF}, {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x0000EFFF}, {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, @@ -8113,7 +15254,7 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radiob_regs[] = { {0x033, 0x00000005}, {0x03F, 0x00004344}, {0x033, 0x00000006}, - {0x03F, 0x00004324}, + {0x03F, 0x00004344}, {0x033, 0x00000007}, {0x03F, 0x00004344}, {0x033, 0x00000008}, @@ -8176,6 +15317,85 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radiob_regs[] = { {0x03F, 0x00000200}, {0x0EF, 0x00000000}, {0x0EF, 0x00000010}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x030, 0x000084DC}, + {0x030, 0x000103C9}, + {0x030, 0x00018399}, + {0x030, 0x00020287}, + {0x030, 0x00028277}, + {0x030, 0x00030165}, + {0x030, 0x00038144}, + {0x030, 0x00040044}, + {0x030, 0x00048022}, + {0x030, 0x00050011}, + {0x030, 0x00058000}, + {0x030, 0x00060000}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x030, 0x000084DC}, + {0x030, 0x000103C9}, + {0x030, 0x00018399}, + {0x030, 0x00020287}, + {0x030, 0x00028277}, + {0x030, 0x00030165}, + {0x030, 0x00038144}, + {0x030, 0x00040044}, + {0x030, 0x00048022}, + {0x030, 0x00050011}, + {0x030, 0x00058000}, + {0x030, 0x00060000}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x030, 0x000084DC}, + {0x030, 0x000103C9}, + {0x030, 0x00018399}, + {0x030, 0x00020287}, + {0x030, 0x00028277}, + {0x030, 0x00030165}, + {0x030, 0x00038144}, + {0x030, 0x00040044}, + {0x030, 0x00048022}, + {0x030, 0x00050011}, + {0x030, 0x00058000}, + {0x030, 0x00060000}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x030, 0x000084DC}, + {0x030, 0x000103C9}, + {0x030, 0x00018399}, + {0x030, 0x00020287}, + {0x030, 0x00028277}, + {0x030, 0x00030165}, + {0x030, 0x00038144}, + {0x030, 0x00040044}, + {0x030, 0x00048022}, + {0x030, 0x00050011}, + {0x030, 0x00058000}, + {0x030, 0x00060000}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x030, 0x000084DC}, + {0x030, 0x000103C9}, + {0x030, 0x00018399}, + {0x030, 0x00020287}, + {0x030, 0x00028277}, + {0x030, 0x00030165}, + {0x030, 0x00038144}, + {0x030, 0x00040044}, + {0x030, 0x00048022}, + {0x030, 0x00050011}, + {0x030, 0x00058000}, + {0x030, 0x00060000}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x030, 0x000084DC}, + {0x030, 0x000103C9}, + {0x030, 0x00018399}, + {0x030, 0x00020287}, + {0x030, 0x00028277}, + {0x030, 0x00030165}, + {0x030, 0x00038144}, + {0x030, 0x00040044}, + {0x030, 0x00048022}, + {0x030, 0x00050011}, + {0x030, 0x00058000}, + {0x030, 0x00060000}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, {0x030, 0x000084DC}, {0x030, 0x000103C9}, {0x030, 0x00018399}, @@ -8188,6 +15408,189 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radiob_regs[] = { {0x030, 0x00050011}, {0x030, 0x00058000}, {0x030, 0x00060000}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x030, 0x000085ED}, + {0x030, 0x000105CC}, + {0x030, 0x000184AA}, + {0x030, 0x00020388}, + {0x030, 0x00028377}, + {0x030, 0x00030377}, + {0x030, 0x00038255}, + {0x030, 0x00040244}, + {0x030, 0x00048133}, + {0x030, 0x00050112}, + {0x030, 0x00058101}, + {0x030, 0x00060001}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x030, 0x000085ED}, + {0x030, 0x000105CC}, + {0x030, 0x000184AA}, + {0x030, 0x00020388}, + {0x030, 0x00028377}, + {0x030, 0x00030377}, + {0x030, 0x00038255}, + {0x030, 0x00040244}, + {0x030, 0x00048133}, + {0x030, 0x00050112}, + {0x030, 0x00058101}, + {0x030, 0x00060001}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x030, 0x000085ED}, + {0x030, 0x000105CC}, + {0x030, 0x000184AA}, + {0x030, 0x00020388}, + {0x030, 0x00028377}, + {0x030, 0x00030377}, + {0x030, 0x00038255}, + {0x030, 0x00040244}, + {0x030, 0x00048133}, + {0x030, 0x00050112}, + {0x030, 0x00058101}, + {0x030, 0x00060001}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x030, 0x000085ED}, + {0x030, 0x000105CC}, + {0x030, 0x000184AA}, + {0x030, 0x00020388}, + {0x030, 0x00028377}, + {0x030, 0x00030377}, + {0x030, 0x00038255}, + {0x030, 0x00040244}, + {0x030, 0x00048133}, + {0x030, 0x00050112}, + {0x030, 0x00058101}, + {0x030, 0x00060001}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x030, 0x000085ED}, + {0x030, 0x000105CC}, + {0x030, 0x000184AA}, + {0x030, 0x00020388}, + {0x030, 0x00028377}, + {0x030, 0x00030377}, + {0x030, 0x00038255}, + {0x030, 0x00040244}, + {0x030, 0x00048133}, + {0x030, 0x00050112}, + {0x030, 0x00058101}, + {0x030, 0x00060001}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x030, 0x000085ED}, + {0x030, 0x000105CC}, + {0x030, 0x000184AA}, + {0x030, 0x00020388}, + {0x030, 0x00028377}, + {0x030, 0x00030377}, + {0x030, 0x00038255}, + {0x030, 0x00040244}, + {0x030, 0x00048133}, + {0x030, 0x00050112}, + {0x030, 0x00058101}, + {0x030, 0x00060001}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x030, 0x000085ED}, + {0x030, 0x000105CC}, + {0x030, 0x000184AA}, + {0x030, 0x00020388}, + {0x030, 0x00028377}, + {0x030, 0x00030377}, + {0x030, 0x00038255}, + {0x030, 0x00040244}, + {0x030, 0x00048133}, + {0x030, 0x00050112}, + {0x030, 0x00058101}, + {0x030, 0x00060001}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x030, 0x000085ED}, + {0x030, 0x000105CC}, + {0x030, 0x000184AA}, + {0x030, 0x00020388}, + {0x030, 0x00028377}, + {0x030, 0x00030377}, + {0x030, 0x00038255}, + {0x030, 0x00040244}, + {0x030, 0x00048133}, + {0x030, 0x00050112}, + {0x030, 0x00058101}, + {0x030, 0x00060001}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x030, 0x000085ED}, + {0x030, 0x000105CC}, + {0x030, 0x000184AA}, + {0x030, 0x00020388}, + {0x030, 0x00028377}, + {0x030, 0x00030377}, + {0x030, 0x00038255}, + {0x030, 0x00040244}, + {0x030, 0x00048133}, + {0x030, 0x00050112}, + {0x030, 0x00058101}, + {0x030, 0x00060001}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x030, 0x000085ED}, + {0x030, 0x000105CC}, + {0x030, 0x000184AA}, + {0x030, 0x00020388}, + {0x030, 0x00028377}, + {0x030, 0x00030377}, + {0x030, 0x00038255}, + {0x030, 0x00040244}, + {0x030, 0x00048133}, + {0x030, 0x00050112}, + {0x030, 0x00058101}, + {0x030, 0x00060001}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x030, 0x000085ED}, + {0x030, 0x000105CC}, + {0x030, 0x000184AA}, + {0x030, 0x00020388}, + {0x030, 0x00028377}, + {0x030, 0x00030377}, + {0x030, 0x00038255}, + {0x030, 0x00040244}, + {0x030, 0x00048133}, + {0x030, 0x00050112}, + {0x030, 0x00058101}, + {0x030, 0x00060001}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x030, 0x000085ED}, + {0x030, 0x000105CC}, + {0x030, 0x000184AA}, + {0x030, 0x00020388}, + {0x030, 0x00028377}, + {0x030, 0x00030377}, + {0x030, 0x00038255}, + {0x030, 0x00040244}, + {0x030, 0x00048133}, + {0x030, 0x00050112}, + {0x030, 0x00058101}, + {0x030, 0x00060001}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x030, 0x000085ED}, + {0x030, 0x000105CC}, + {0x030, 0x000184AA}, + {0x030, 0x00020388}, + {0x030, 0x00028377}, + {0x030, 0x00030377}, + {0x030, 0x00038255}, + {0x030, 0x00040244}, + {0x030, 0x00048133}, + {0x030, 0x00050112}, + {0x030, 0x00058101}, + {0x030, 0x00060001}, + {0xA0000000, 0x00000000}, + {0x030, 0x000084DC}, + {0x030, 0x000103C9}, + {0x030, 0x00018399}, + {0x030, 0x00020287}, + {0x030, 0x00028277}, + {0x030, 0x00030165}, + {0x030, 0x00038144}, + {0x030, 0x00040044}, + {0x030, 0x00048022}, + {0x030, 0x00050011}, + {0x030, 0x00058000}, + {0x030, 0x00060000}, + {0xB0000000, 0x00000000}, {0x030, 0x00068000}, {0x030, 0x00070000}, {0x0EF, 0x00000000}, @@ -8458,6 +15861,14 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radiob_regs[] = { {0x095, 0x00000008}, {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, {0x095, 0x00000008}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x095, 0x00000008}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x095, 0x00000008}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x095, 0x00000008}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x095, 0x00000008}, {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, {0x095, 0x00000008}, {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, @@ -8477,101 +15888,2117 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radiob_regs[] = { {0xB0000000, 0x00000000}, {0x0EE, 0x00001000}, {0x033, 0x00000020}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000052}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000052}, + {0xB0000000, 0x00000000}, {0x033, 0x00000024}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x0000005A}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0xA0000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0xB0000000, 0x00000000}, {0x033, 0x00000028}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x0000009C}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0xA0000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0xB0000000, 0x00000000}, {0x033, 0x0000002C}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x0000019C}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0xA0000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0xB0000000, 0x00000000}, {0x033, 0x00000030}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x000001A4}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0xA0000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0xB0000000, 0x00000000}, {0x033, 0x00000034}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E7}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E7}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E7}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E7}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x000001E7}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E7}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E7}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E7}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E7}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E7}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E7}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E7}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E7}, + {0xA0000000, 0x00000000}, + {0x03F, 0x000001E7}, + {0xB0000000, 0x00000000}, {0x033, 0x00000038}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x000002E7}, + {0xA0000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0xB0000000, 0x00000000}, {0x033, 0x0000003C}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x000003E7}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0xA0000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0xB0000000, 0x00000000}, {0x033, 0x00000021}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000052}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000052}, + {0xB0000000, 0x00000000}, {0x033, 0x00000025}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x0000005A}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0xA0000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0xB0000000, 0x00000000}, {0x033, 0x00000029}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0xA0000000, 0x00000000}, {0x03F, 0x0000009C}, + {0xB0000000, 0x00000000}, {0x033, 0x0000002D}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x0000019C}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0xA0000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0xB0000000, 0x00000000}, {0x033, 0x00000031}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x000001A4}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0xA0000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0xB0000000, 0x00000000}, {0x033, 0x00000035}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x000001E6}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0xA0000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0xB0000000, 0x00000000}, {0x033, 0x00000039}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x000002E6}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0xA0000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0xB0000000, 0x00000000}, {0x033, 0x0000003D}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x000003E6}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0xA0000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0xB0000000, 0x00000000}, {0x033, 0x00000022}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000052}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000052}, + {0xB0000000, 0x00000000}, {0x033, 0x00000026}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x0000005A}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0xA0000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0xB0000000, 0x00000000}, {0x033, 0x0000002A}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x0000009C}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0xA0000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0xB0000000, 0x00000000}, {0x033, 0x0000002E}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x0000019C}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0xA0000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0xB0000000, 0x00000000}, {0x033, 0x00000032}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x000001A4}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0xA0000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0xB0000000, 0x00000000}, {0x033, 0x00000036}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x000001E6}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0xA0000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0xB0000000, 0x00000000}, {0x033, 0x0000003A}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x000002E6}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0xA0000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0xB0000000, 0x00000000}, {0x033, 0x0000003E}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x000003E6}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0xA0000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0xB0000000, 0x00000000}, {0x033, 0x00000060}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000052}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000052}, + {0xB0000000, 0x00000000}, {0x033, 0x00000064}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x0000005A}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0xA0000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0xB0000000, 0x00000000}, {0x033, 0x00000068}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0xA0000000, 0x00000000}, {0x03F, 0x0000009C}, + {0xB0000000, 0x00000000}, {0x033, 0x0000006C}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x0000019C}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0xA0000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0xB0000000, 0x00000000}, {0x033, 0x00000070}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x000001A4}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0xA0000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0xB0000000, 0x00000000}, {0x033, 0x00000074}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x000001E6}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0xA0000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0xB0000000, 0x00000000}, {0x033, 0x00000078}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x000002E6}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0xA0000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0xB0000000, 0x00000000}, {0x033, 0x0000007C}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x000003E6}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0xA0000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0xB0000000, 0x00000000}, {0x033, 0x00000061}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000052}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000052}, + {0xB0000000, 0x00000000}, {0x033, 0x00000065}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0xA0000000, 0x00000000}, {0x03F, 0x0000005A}, + {0xB0000000, 0x00000000}, {0x033, 0x00000069}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x0000009C}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0xA0000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0xB0000000, 0x00000000}, {0x033, 0x0000006D}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x0000019C}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0xA0000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0xB0000000, 0x00000000}, {0x033, 0x00000071}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0xA0000000, 0x00000000}, {0x03F, 0x000001A4}, + {0xB0000000, 0x00000000}, {0x033, 0x00000075}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0xA0000000, 0x00000000}, {0x03F, 0x000001E6}, + {0xB0000000, 0x00000000}, {0x033, 0x00000079}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x000002E6}, + {0xA0000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0xB0000000, 0x00000000}, {0x033, 0x0000007D}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x000003E6}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0xA0000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0xB0000000, 0x00000000}, {0x033, 0x00000062}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000052}, + {0xA0000000, 0x00000000}, {0x03F, 0x00000052}, + {0xB0000000, 0x00000000}, {0x033, 0x00000066}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x0000005A}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0xA0000000, 0x00000000}, + {0x03F, 0x0000005A}, + {0xB0000000, 0x00000000}, {0x033, 0x0000006A}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000009C}, + {0xA0000000, 0x00000000}, {0x03F, 0x0000009C}, + {0xB0000000, 0x00000000}, {0x033, 0x0000006E}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x0000019C}, + {0xA0000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0xB0000000, 0x00000000}, {0x033, 0x00000072}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x000001A4}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0xA0000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0xB0000000, 0x00000000}, {0x033, 0x00000076}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x000001E6}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0xA0000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0xB0000000, 0x00000000}, {0x033, 0x0000007A}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E7}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x000002E6}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0xA0000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0xB0000000, 0x00000000}, {0x033, 0x0000007E}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x000003E6}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E7}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0xA0000000, 0x00000000}, + {0x03F, 0x000003E6}, + {0xB0000000, 0x00000000}, {0x033, 0x00000063}, {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000052}, @@ -8591,20 +18018,28 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radiob_regs[] = { {0x03F, 0x00000152}, {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000152}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000152}, {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000152}, {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, - {0x03F, 0x00000152}, + {0x03F, 0x00000052}, {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, - {0x03F, 0x00000152}, + {0x03F, 0x00000052}, {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, - {0x03F, 0x00000152}, + {0x03F, 0x00000052}, {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, - {0x03F, 0x00000152}, + {0x03F, 0x00000052}, {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, - {0x03F, 0x00000152}, + {0x03F, 0x00000052}, {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, - {0x03F, 0x00000152}, + {0x03F, 0x00000052}, {0xA0000000, 0x00000000}, {0x03F, 0x00000052}, {0xB0000000, 0x00000000}, @@ -8627,20 +18062,28 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radiob_regs[] = { {0x03F, 0x0000015A}, {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x0000015A}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000015A}, {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x0000015A}, {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, - {0x03F, 0x0000015A}, + {0x03F, 0x0000005A}, {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, - {0x03F, 0x0000015A}, + {0x03F, 0x0000005A}, {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, - {0x03F, 0x0000015A}, + {0x03F, 0x0000005A}, {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, - {0x03F, 0x0000015A}, + {0x03F, 0x0000005A}, {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, - {0x03F, 0x0000015A}, + {0x03F, 0x0000005A}, {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, - {0x03F, 0x0000015A}, + {0x03F, 0x0000005A}, {0xA0000000, 0x00000000}, {0x03F, 0x0000005A}, {0xB0000000, 0x00000000}, @@ -8663,20 +18106,28 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radiob_regs[] = { {0x03F, 0x0000019C}, {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x0000019C}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x0000019C}, {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x0000019C}, {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, - {0x03F, 0x0000019C}, + {0x03F, 0x0000009C}, {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, - {0x03F, 0x0000019C}, + {0x03F, 0x0000009C}, {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, - {0x03F, 0x0000019C}, + {0x03F, 0x0000009C}, {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, - {0x03F, 0x0000019C}, + {0x03F, 0x0000009C}, {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, - {0x03F, 0x0000019C}, + {0x03F, 0x0000009C}, {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, - {0x03F, 0x0000019C}, + {0x03F, 0x0000009C}, {0xA0000000, 0x00000000}, {0x03F, 0x0000009C}, {0xB0000000, 0x00000000}, @@ -8699,20 +18150,28 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radiob_regs[] = { {0x03F, 0x000001A4}, {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x000001A4}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001A4}, {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x000001A4}, {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, - {0x03F, 0x000001A4}, + {0x03F, 0x0000019C}, {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, - {0x03F, 0x000001A4}, + {0x03F, 0x0000019C}, {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, - {0x03F, 0x000001A4}, + {0x03F, 0x0000019C}, {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, - {0x03F, 0x000001A4}, + {0x03F, 0x0000019C}, {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, - {0x03F, 0x000001A4}, + {0x03F, 0x0000019C}, {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, - {0x03F, 0x000001A4}, + {0x03F, 0x0000019C}, {0xA0000000, 0x00000000}, {0x03F, 0x0000019C}, {0xB0000000, 0x00000000}, @@ -8735,20 +18194,28 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radiob_regs[] = { {0x03F, 0x000001E6}, {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x000001E6}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000001E6}, {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x000001E6}, {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, - {0x03F, 0x000001E6}, + {0x03F, 0x000001A4}, {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, - {0x03F, 0x000001E6}, + {0x03F, 0x000001A4}, {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, - {0x03F, 0x000001E6}, + {0x03F, 0x000001A4}, {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, - {0x03F, 0x000001E6}, + {0x03F, 0x000001A4}, {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, - {0x03F, 0x000001E6}, + {0x03F, 0x000001A4}, {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, - {0x03F, 0x000001E6}, + {0x03F, 0x000001A4}, {0xA0000000, 0x00000000}, {0x03F, 0x000001A4}, {0xB0000000, 0x00000000}, @@ -8771,20 +18238,28 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radiob_regs[] = { {0x03F, 0x000002E6}, {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x000002E6}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x000002E6}, {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x000002E6}, {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, - {0x03F, 0x000002E6}, + {0x03F, 0x000001E6}, {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, - {0x03F, 0x000002E6}, + {0x03F, 0x000001E6}, {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, - {0x03F, 0x000002E6}, + {0x03F, 0x000001E6}, {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, - {0x03F, 0x000002E6}, + {0x03F, 0x000001E6}, {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, - {0x03F, 0x000002E6}, + {0x03F, 0x000001E6}, {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, - {0x03F, 0x000002E6}, + {0x03F, 0x000001E6}, {0xA0000000, 0x00000000}, {0x03F, 0x000001E6}, {0xB0000000, 0x00000000}, @@ -9828,131 +19303,131 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radiob_regs[] = { {0x10030, 0x00003C5F}, {0x10030, 0x00004059}, {0x10030, 0x00004453}, - {0x10030, 0x000201ED}, - {0x10030, 0x000205AD}, - {0x10030, 0x000209A7}, - {0x10030, 0x00020DA1}, - {0x10030, 0x0002119B}, - {0x10030, 0x00021561}, - {0x10030, 0x0002195B}, - {0x10030, 0x00021D27}, - {0x10030, 0x00022121}, - {0x10030, 0x000224E9}, - {0x10030, 0x000228E3}, - {0x10030, 0x00022CA9}, - {0x10030, 0x000230A3}, - {0x10030, 0x00023469}, - {0x10030, 0x00023863}, - {0x10030, 0x00023C29}, - {0x10030, 0x00024023}, - {0x10030, 0x0002441D}, + {0x10030, 0x000201EF}, + {0x10030, 0x000205E9}, + {0x10030, 0x000209E3}, + {0x10030, 0x00020DA3}, + {0x10030, 0x00021161}, + {0x10030, 0x0002155B}, + {0x10030, 0x0002191F}, + {0x10030, 0x00021D19}, + {0x10030, 0x000220E1}, + {0x10030, 0x000224DB}, + {0x10030, 0x000228A3}, + {0x10030, 0x00022C9D}, + {0x10030, 0x00023063}, + {0x10030, 0x0002345D}, + {0x10030, 0x00023823}, + {0x10030, 0x00023C1B}, + {0x10030, 0x00024015}, + {0x10030, 0x0002440F}, {0x10030, 0x000281EF}, - {0x10030, 0x000285AF}, - {0x10030, 0x000289A9}, - {0x10030, 0x00028DA3}, - {0x10030, 0x0002919D}, - {0x10030, 0x00029563}, - {0x10030, 0x0002995D}, - {0x10030, 0x00029D25}, - {0x10030, 0x0002A11F}, - {0x10030, 0x0002A4E7}, - {0x10030, 0x0002A8E1}, - {0x10030, 0x0002ACA7}, - {0x10030, 0x0002B0A1}, - {0x10030, 0x0002B467}, - {0x10030, 0x0002B861}, - {0x10030, 0x0002BC27}, - {0x10030, 0x0002C021}, - {0x10030, 0x0002C41B}, + {0x10030, 0x000285E7}, + {0x10030, 0x000289A7}, + {0x10030, 0x00028D65}, + {0x10030, 0x0002915F}, + {0x10030, 0x00029523}, + {0x10030, 0x0002991D}, + {0x10030, 0x00029CE5}, + {0x10030, 0x0002A0DF}, + {0x10030, 0x0002A4A7}, + {0x10030, 0x0002A8A1}, + {0x10030, 0x0002AC67}, + {0x10030, 0x0002B061}, + {0x10030, 0x0002B427}, + {0x10030, 0x0002B821}, + {0x10030, 0x0002BC19}, + {0x10030, 0x0002C013}, + {0x10030, 0x0002C40D}, {0x10030, 0x000301EF}, - {0x10030, 0x000305AF}, - {0x10030, 0x000309A9}, - {0x10030, 0x00030DA3}, - {0x10030, 0x0003119D}, - {0x10030, 0x00031563}, - {0x10030, 0x0003195D}, - {0x10030, 0x00031D25}, - {0x10030, 0x0003211F}, - {0x10030, 0x000324E7}, - {0x10030, 0x000328E1}, - {0x10030, 0x00032CA7}, - {0x10030, 0x000330A1}, - {0x10030, 0x00033467}, - {0x10030, 0x00033861}, - {0x10030, 0x00033C27}, - {0x10030, 0x00034021}, - {0x10030, 0x0003441B}, - {0x10030, 0x000601EB}, - {0x10030, 0x000605AB}, - {0x10030, 0x000609A5}, - {0x10030, 0x00060D9F}, - {0x10030, 0x00061199}, - {0x10030, 0x00061593}, - {0x10030, 0x00061959}, - {0x10030, 0x00061D53}, - {0x10030, 0x0006211B}, - {0x10030, 0x00062515}, - {0x10030, 0x000628DD}, - {0x10030, 0x00062CD7}, - {0x10030, 0x0006309D}, - {0x10030, 0x00063497}, - {0x10030, 0x0006385D}, - {0x10030, 0x00063C57}, - {0x10030, 0x0006401D}, - {0x10030, 0x00064417}, - {0x10030, 0x000681E7}, - {0x10030, 0x000685A7}, - {0x10030, 0x000689A1}, - {0x10030, 0x00068D9B}, - {0x10030, 0x00069195}, - {0x10030, 0x0006955F}, - {0x10030, 0x00069959}, - {0x10030, 0x00069D21}, - {0x10030, 0x0006A11B}, - {0x10030, 0x0006A4E3}, - {0x10030, 0x0006A8DD}, - {0x10030, 0x0006ACA5}, - {0x10030, 0x0006B09F}, - {0x10030, 0x0006B465}, - {0x10030, 0x0006B85F}, - {0x10030, 0x0006BC25}, - {0x10030, 0x0006C01F}, - {0x10030, 0x0006C419}, - {0x10030, 0x000701E7}, - {0x10030, 0x000705A7}, - {0x10030, 0x000709A1}, - {0x10030, 0x00070D9B}, - {0x10030, 0x00071195}, - {0x10030, 0x0007155B}, - {0x10030, 0x00071955}, - {0x10030, 0x00071D1D}, - {0x10030, 0x00072117}, - {0x10030, 0x000724DF}, - {0x10030, 0x000728D9}, - {0x10030, 0x00072CA1}, - {0x10030, 0x0007309B}, - {0x10030, 0x00073461}, - {0x10030, 0x0007385B}, - {0x10030, 0x00073C21}, - {0x10030, 0x0007401B}, - {0x10030, 0x0007441B}, - {0x10030, 0x000781EF}, - {0x10030, 0x000785E9}, - {0x10030, 0x000789E3}, + {0x10030, 0x000305E7}, + {0x10030, 0x000309A7}, + {0x10030, 0x00030D65}, + {0x10030, 0x0003115F}, + {0x10030, 0x00031525}, + {0x10030, 0x0003191F}, + {0x10030, 0x00031CE7}, + {0x10030, 0x000320E1}, + {0x10030, 0x000324A9}, + {0x10030, 0x000328A3}, + {0x10030, 0x00032C69}, + {0x10030, 0x00033063}, + {0x10030, 0x00033429}, + {0x10030, 0x00033823}, + {0x10030, 0x00033C1D}, + {0x10030, 0x00034013}, + {0x10030, 0x0003440D}, + {0x10030, 0x000601F1}, + {0x10030, 0x000605E9}, + {0x10030, 0x000609A9}, + {0x10030, 0x00060D65}, + {0x10030, 0x0006115F}, + {0x10030, 0x00061525}, + {0x10030, 0x0006191F}, + {0x10030, 0x00061CE7}, + {0x10030, 0x000620E1}, + {0x10030, 0x000624DB}, + {0x10030, 0x000628A3}, + {0x10030, 0x00062C69}, + {0x10030, 0x00063063}, + {0x10030, 0x00063429}, + {0x10030, 0x00063823}, + {0x10030, 0x00063C1D}, + {0x10030, 0x00064013}, + {0x10030, 0x0006440D}, + {0x10030, 0x000681EF}, + {0x10030, 0x000685E7}, + {0x10030, 0x000689A7}, + {0x10030, 0x00068D61}, + {0x10030, 0x0006915B}, + {0x10030, 0x00069523}, + {0x10030, 0x0006991D}, + {0x10030, 0x00069CE5}, + {0x10030, 0x0006A0DF}, + {0x10030, 0x0006A4A7}, + {0x10030, 0x0006A8A1}, + {0x10030, 0x0006AC67}, + {0x10030, 0x0006B061}, + {0x10030, 0x0006B429}, + {0x10030, 0x0006B823}, + {0x10030, 0x0006BC1D}, + {0x10030, 0x0006C017}, + {0x10030, 0x0006C40D}, + {0x10030, 0x000701F1}, + {0x10030, 0x000705E9}, + {0x10030, 0x000709A9}, + {0x10030, 0x00070D63}, + {0x10030, 0x0007115D}, + {0x10030, 0x00071523}, + {0x10030, 0x0007191D}, + {0x10030, 0x00071D17}, + {0x10030, 0x00072111}, + {0x10030, 0x000724D9}, + {0x10030, 0x000728D3}, + {0x10030, 0x00072C67}, + {0x10030, 0x00073061}, + {0x10030, 0x00073427}, + {0x10030, 0x00073821}, + {0x10030, 0x00073C1B}, + {0x10030, 0x00074015}, + {0x10030, 0x0007440D}, + {0x10030, 0x000781F1}, + {0x10030, 0x000785EB}, + {0x10030, 0x000789E5}, {0x10030, 0x00078DA3}, {0x10030, 0x00079161}, {0x10030, 0x0007955B}, - {0x10030, 0x00079921}, - {0x10030, 0x00079D1B}, - {0x10030, 0x0007A0E1}, - {0x10030, 0x0007A4DB}, - {0x10030, 0x0007A8A1}, - {0x10030, 0x0007AC9B}, - {0x10030, 0x0007B061}, - {0x10030, 0x0007B45B}, - {0x10030, 0x0007B821}, - {0x10030, 0x0007BC1B}, - {0x10030, 0x0007C015}, + {0x10030, 0x00079923}, + {0x10030, 0x00079D1D}, + {0x10030, 0x0007A117}, + {0x10030, 0x0007A4DD}, + {0x10030, 0x0007A8D7}, + {0x10030, 0x0007AC9D}, + {0x10030, 0x0007B063}, + {0x10030, 0x0007B45D}, + {0x10030, 0x0007B857}, + {0x10030, 0x0007BC1D}, + {0x10030, 0x0007C017}, {0x10030, 0x0007C40F}, {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, {0x10030, 0x000001EF}, @@ -9973,131 +19448,711 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radiob_regs[] = { {0x10030, 0x00003C5F}, {0x10030, 0x00004059}, {0x10030, 0x00004453}, - {0x10030, 0x000201ED}, - {0x10030, 0x000205AD}, - {0x10030, 0x000209A7}, - {0x10030, 0x00020DA1}, - {0x10030, 0x0002119B}, - {0x10030, 0x00021561}, - {0x10030, 0x0002195B}, - {0x10030, 0x00021D27}, - {0x10030, 0x00022121}, - {0x10030, 0x000224E9}, - {0x10030, 0x000228E3}, - {0x10030, 0x00022CA9}, - {0x10030, 0x000230A3}, - {0x10030, 0x00023469}, - {0x10030, 0x00023863}, - {0x10030, 0x00023C29}, - {0x10030, 0x00024023}, - {0x10030, 0x0002441D}, + {0x10030, 0x000201EF}, + {0x10030, 0x000205E9}, + {0x10030, 0x000209E3}, + {0x10030, 0x00020DA3}, + {0x10030, 0x00021161}, + {0x10030, 0x0002155B}, + {0x10030, 0x0002191F}, + {0x10030, 0x00021D19}, + {0x10030, 0x000220E1}, + {0x10030, 0x000224DB}, + {0x10030, 0x000228A3}, + {0x10030, 0x00022C9D}, + {0x10030, 0x00023063}, + {0x10030, 0x0002345D}, + {0x10030, 0x00023823}, + {0x10030, 0x00023C1B}, + {0x10030, 0x00024015}, + {0x10030, 0x0002440F}, {0x10030, 0x000281EF}, - {0x10030, 0x000285AF}, - {0x10030, 0x000289A9}, - {0x10030, 0x00028DA3}, - {0x10030, 0x0002919D}, - {0x10030, 0x00029563}, - {0x10030, 0x0002995D}, - {0x10030, 0x00029D25}, - {0x10030, 0x0002A11F}, - {0x10030, 0x0002A4E7}, - {0x10030, 0x0002A8E1}, - {0x10030, 0x0002ACA7}, - {0x10030, 0x0002B0A1}, - {0x10030, 0x0002B467}, - {0x10030, 0x0002B861}, - {0x10030, 0x0002BC27}, - {0x10030, 0x0002C021}, - {0x10030, 0x0002C41B}, + {0x10030, 0x000285E7}, + {0x10030, 0x000289A7}, + {0x10030, 0x00028D65}, + {0x10030, 0x0002915F}, + {0x10030, 0x00029523}, + {0x10030, 0x0002991D}, + {0x10030, 0x00029CE5}, + {0x10030, 0x0002A0DF}, + {0x10030, 0x0002A4A7}, + {0x10030, 0x0002A8A1}, + {0x10030, 0x0002AC67}, + {0x10030, 0x0002B061}, + {0x10030, 0x0002B427}, + {0x10030, 0x0002B821}, + {0x10030, 0x0002BC19}, + {0x10030, 0x0002C013}, + {0x10030, 0x0002C40D}, {0x10030, 0x000301EF}, - {0x10030, 0x000305AF}, - {0x10030, 0x000309A9}, - {0x10030, 0x00030DA3}, - {0x10030, 0x0003119D}, - {0x10030, 0x00031563}, - {0x10030, 0x0003195D}, - {0x10030, 0x00031D25}, - {0x10030, 0x0003211F}, - {0x10030, 0x000324E7}, - {0x10030, 0x000328E1}, - {0x10030, 0x00032CA7}, - {0x10030, 0x000330A1}, - {0x10030, 0x00033467}, - {0x10030, 0x00033861}, - {0x10030, 0x00033C27}, - {0x10030, 0x00034021}, - {0x10030, 0x0003441B}, - {0x10030, 0x000601EB}, - {0x10030, 0x000605AB}, - {0x10030, 0x000609A5}, - {0x10030, 0x00060D9F}, - {0x10030, 0x00061199}, - {0x10030, 0x00061593}, - {0x10030, 0x00061959}, - {0x10030, 0x00061D53}, - {0x10030, 0x0006211B}, - {0x10030, 0x00062515}, - {0x10030, 0x000628DD}, - {0x10030, 0x00062CD7}, - {0x10030, 0x0006309D}, - {0x10030, 0x00063497}, - {0x10030, 0x0006385D}, - {0x10030, 0x00063C57}, - {0x10030, 0x0006401D}, - {0x10030, 0x00064417}, - {0x10030, 0x000681E7}, - {0x10030, 0x000685A7}, - {0x10030, 0x000689A1}, - {0x10030, 0x00068D9B}, - {0x10030, 0x00069195}, - {0x10030, 0x0006955F}, - {0x10030, 0x00069959}, - {0x10030, 0x00069D21}, - {0x10030, 0x0006A11B}, - {0x10030, 0x0006A4E3}, - {0x10030, 0x0006A8DD}, - {0x10030, 0x0006ACA5}, - {0x10030, 0x0006B09F}, - {0x10030, 0x0006B465}, - {0x10030, 0x0006B85F}, - {0x10030, 0x0006BC25}, - {0x10030, 0x0006C01F}, - {0x10030, 0x0006C419}, - {0x10030, 0x000701E7}, - {0x10030, 0x000705A7}, - {0x10030, 0x000709A1}, - {0x10030, 0x00070D9B}, - {0x10030, 0x00071195}, - {0x10030, 0x0007155B}, - {0x10030, 0x00071955}, - {0x10030, 0x00071D1D}, - {0x10030, 0x00072117}, - {0x10030, 0x000724DF}, - {0x10030, 0x000728D9}, - {0x10030, 0x00072CA1}, - {0x10030, 0x0007309B}, - {0x10030, 0x00073461}, - {0x10030, 0x0007385B}, - {0x10030, 0x00073C21}, - {0x10030, 0x0007401B}, - {0x10030, 0x0007441B}, - {0x10030, 0x000781EF}, - {0x10030, 0x000785E9}, - {0x10030, 0x000789E3}, + {0x10030, 0x000305E7}, + {0x10030, 0x000309A7}, + {0x10030, 0x00030D65}, + {0x10030, 0x0003115F}, + {0x10030, 0x00031525}, + {0x10030, 0x0003191F}, + {0x10030, 0x00031CE7}, + {0x10030, 0x000320E1}, + {0x10030, 0x000324A9}, + {0x10030, 0x000328A3}, + {0x10030, 0x00032C69}, + {0x10030, 0x00033063}, + {0x10030, 0x00033429}, + {0x10030, 0x00033823}, + {0x10030, 0x00033C1D}, + {0x10030, 0x00034013}, + {0x10030, 0x0003440D}, + {0x10030, 0x000601F1}, + {0x10030, 0x000605E9}, + {0x10030, 0x000609A9}, + {0x10030, 0x00060D65}, + {0x10030, 0x0006115F}, + {0x10030, 0x00061525}, + {0x10030, 0x0006191F}, + {0x10030, 0x00061CE7}, + {0x10030, 0x000620E1}, + {0x10030, 0x000624DB}, + {0x10030, 0x000628A3}, + {0x10030, 0x00062C69}, + {0x10030, 0x00063063}, + {0x10030, 0x00063429}, + {0x10030, 0x00063823}, + {0x10030, 0x00063C1D}, + {0x10030, 0x00064013}, + {0x10030, 0x0006440D}, + {0x10030, 0x000681EF}, + {0x10030, 0x000685E7}, + {0x10030, 0x000689A7}, + {0x10030, 0x00068D61}, + {0x10030, 0x0006915B}, + {0x10030, 0x00069523}, + {0x10030, 0x0006991D}, + {0x10030, 0x00069CE5}, + {0x10030, 0x0006A0DF}, + {0x10030, 0x0006A4A7}, + {0x10030, 0x0006A8A1}, + {0x10030, 0x0006AC67}, + {0x10030, 0x0006B061}, + {0x10030, 0x0006B429}, + {0x10030, 0x0006B823}, + {0x10030, 0x0006BC1D}, + {0x10030, 0x0006C017}, + {0x10030, 0x0006C40D}, + {0x10030, 0x000701F1}, + {0x10030, 0x000705E9}, + {0x10030, 0x000709A9}, + {0x10030, 0x00070D63}, + {0x10030, 0x0007115D}, + {0x10030, 0x00071523}, + {0x10030, 0x0007191D}, + {0x10030, 0x00071D17}, + {0x10030, 0x00072111}, + {0x10030, 0x000724D9}, + {0x10030, 0x000728D3}, + {0x10030, 0x00072C67}, + {0x10030, 0x00073061}, + {0x10030, 0x00073427}, + {0x10030, 0x00073821}, + {0x10030, 0x00073C1B}, + {0x10030, 0x00074015}, + {0x10030, 0x0007440D}, + {0x10030, 0x000781F1}, + {0x10030, 0x000785EB}, + {0x10030, 0x000789E5}, {0x10030, 0x00078DA3}, {0x10030, 0x00079161}, {0x10030, 0x0007955B}, - {0x10030, 0x00079921}, - {0x10030, 0x00079D1B}, - {0x10030, 0x0007A0E1}, - {0x10030, 0x0007A4DB}, - {0x10030, 0x0007A8A1}, - {0x10030, 0x0007AC9B}, - {0x10030, 0x0007B061}, - {0x10030, 0x0007B45B}, - {0x10030, 0x0007B821}, - {0x10030, 0x0007BC1B}, - {0x10030, 0x0007C015}, + {0x10030, 0x00079923}, + {0x10030, 0x00079D1D}, + {0x10030, 0x0007A117}, + {0x10030, 0x0007A4DD}, + {0x10030, 0x0007A8D7}, + {0x10030, 0x0007AC9D}, + {0x10030, 0x0007B063}, + {0x10030, 0x0007B45D}, + {0x10030, 0x0007B857}, + {0x10030, 0x0007BC1D}, + {0x10030, 0x0007C017}, + {0x10030, 0x0007C40F}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x000001EF}, + {0x10030, 0x000005E9}, + {0x10030, 0x000009E3}, + {0x10030, 0x00000DDD}, + {0x10030, 0x000011D7}, + {0x10030, 0x0000159F}, + {0x10030, 0x00001999}, + {0x10030, 0x00001D5F}, + {0x10030, 0x00002159}, + {0x10030, 0x0000251F}, + {0x10030, 0x00002919}, + {0x10030, 0x00002CDF}, + {0x10030, 0x000030D9}, + {0x10030, 0x0000349F}, + {0x10030, 0x00003899}, + {0x10030, 0x00003C5F}, + {0x10030, 0x00004059}, + {0x10030, 0x00004453}, + {0x10030, 0x000201EF}, + {0x10030, 0x000205E9}, + {0x10030, 0x000209E3}, + {0x10030, 0x00020DA3}, + {0x10030, 0x00021161}, + {0x10030, 0x0002155B}, + {0x10030, 0x0002191F}, + {0x10030, 0x00021D19}, + {0x10030, 0x000220E1}, + {0x10030, 0x000224DB}, + {0x10030, 0x000228A3}, + {0x10030, 0x00022C9D}, + {0x10030, 0x00023063}, + {0x10030, 0x0002345D}, + {0x10030, 0x00023823}, + {0x10030, 0x00023C1B}, + {0x10030, 0x00024015}, + {0x10030, 0x0002440F}, + {0x10030, 0x000281EF}, + {0x10030, 0x000285E7}, + {0x10030, 0x000289A7}, + {0x10030, 0x00028D65}, + {0x10030, 0x0002915F}, + {0x10030, 0x00029523}, + {0x10030, 0x0002991D}, + {0x10030, 0x00029CE5}, + {0x10030, 0x0002A0DF}, + {0x10030, 0x0002A4A7}, + {0x10030, 0x0002A8A1}, + {0x10030, 0x0002AC67}, + {0x10030, 0x0002B061}, + {0x10030, 0x0002B427}, + {0x10030, 0x0002B821}, + {0x10030, 0x0002BC19}, + {0x10030, 0x0002C013}, + {0x10030, 0x0002C40D}, + {0x10030, 0x000301EF}, + {0x10030, 0x000305E7}, + {0x10030, 0x000309A7}, + {0x10030, 0x00030D65}, + {0x10030, 0x0003115F}, + {0x10030, 0x00031525}, + {0x10030, 0x0003191F}, + {0x10030, 0x00031CE7}, + {0x10030, 0x000320E1}, + {0x10030, 0x000324A9}, + {0x10030, 0x000328A3}, + {0x10030, 0x00032C69}, + {0x10030, 0x00033063}, + {0x10030, 0x00033429}, + {0x10030, 0x00033823}, + {0x10030, 0x00033C1D}, + {0x10030, 0x00034013}, + {0x10030, 0x0003440D}, + {0x10030, 0x000601F1}, + {0x10030, 0x000605E9}, + {0x10030, 0x000609A9}, + {0x10030, 0x00060D65}, + {0x10030, 0x0006115F}, + {0x10030, 0x00061525}, + {0x10030, 0x0006191F}, + {0x10030, 0x00061CE7}, + {0x10030, 0x000620E1}, + {0x10030, 0x000624DB}, + {0x10030, 0x000628A3}, + {0x10030, 0x00062C69}, + {0x10030, 0x00063063}, + {0x10030, 0x00063429}, + {0x10030, 0x00063823}, + {0x10030, 0x00063C1D}, + {0x10030, 0x00064013}, + {0x10030, 0x0006440D}, + {0x10030, 0x000681EF}, + {0x10030, 0x000685E7}, + {0x10030, 0x000689A7}, + {0x10030, 0x00068D61}, + {0x10030, 0x0006915B}, + {0x10030, 0x00069523}, + {0x10030, 0x0006991D}, + {0x10030, 0x00069CE5}, + {0x10030, 0x0006A0DF}, + {0x10030, 0x0006A4A7}, + {0x10030, 0x0006A8A1}, + {0x10030, 0x0006AC67}, + {0x10030, 0x0006B061}, + {0x10030, 0x0006B429}, + {0x10030, 0x0006B823}, + {0x10030, 0x0006BC1D}, + {0x10030, 0x0006C017}, + {0x10030, 0x0006C40D}, + {0x10030, 0x000701F1}, + {0x10030, 0x000705E9}, + {0x10030, 0x000709A9}, + {0x10030, 0x00070D63}, + {0x10030, 0x0007115D}, + {0x10030, 0x00071523}, + {0x10030, 0x0007191D}, + {0x10030, 0x00071D17}, + {0x10030, 0x00072111}, + {0x10030, 0x000724D9}, + {0x10030, 0x000728D3}, + {0x10030, 0x00072C67}, + {0x10030, 0x00073061}, + {0x10030, 0x00073427}, + {0x10030, 0x00073821}, + {0x10030, 0x00073C1B}, + {0x10030, 0x00074015}, + {0x10030, 0x0007440D}, + {0x10030, 0x000781F1}, + {0x10030, 0x000785EB}, + {0x10030, 0x000789E5}, + {0x10030, 0x00078DA3}, + {0x10030, 0x00079161}, + {0x10030, 0x0007955B}, + {0x10030, 0x00079923}, + {0x10030, 0x00079D1D}, + {0x10030, 0x0007A117}, + {0x10030, 0x0007A4DD}, + {0x10030, 0x0007A8D7}, + {0x10030, 0x0007AC9D}, + {0x10030, 0x0007B063}, + {0x10030, 0x0007B45D}, + {0x10030, 0x0007B857}, + {0x10030, 0x0007BC1D}, + {0x10030, 0x0007C017}, + {0x10030, 0x0007C40F}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x000001EF}, + {0x10030, 0x000005E9}, + {0x10030, 0x000009E3}, + {0x10030, 0x00000DDD}, + {0x10030, 0x000011D7}, + {0x10030, 0x0000159F}, + {0x10030, 0x00001999}, + {0x10030, 0x00001D5F}, + {0x10030, 0x00002159}, + {0x10030, 0x0000251F}, + {0x10030, 0x00002919}, + {0x10030, 0x00002CDF}, + {0x10030, 0x000030D9}, + {0x10030, 0x0000349F}, + {0x10030, 0x00003899}, + {0x10030, 0x00003C5F}, + {0x10030, 0x00004059}, + {0x10030, 0x00004453}, + {0x10030, 0x000201EF}, + {0x10030, 0x000205E9}, + {0x10030, 0x000209E3}, + {0x10030, 0x00020DA3}, + {0x10030, 0x00021161}, + {0x10030, 0x0002155B}, + {0x10030, 0x0002191F}, + {0x10030, 0x00021D19}, + {0x10030, 0x000220E1}, + {0x10030, 0x000224DB}, + {0x10030, 0x000228A3}, + {0x10030, 0x00022C9D}, + {0x10030, 0x00023063}, + {0x10030, 0x0002345D}, + {0x10030, 0x00023823}, + {0x10030, 0x00023C1B}, + {0x10030, 0x00024015}, + {0x10030, 0x0002440F}, + {0x10030, 0x000281EF}, + {0x10030, 0x000285E7}, + {0x10030, 0x000289A7}, + {0x10030, 0x00028D65}, + {0x10030, 0x0002915F}, + {0x10030, 0x00029523}, + {0x10030, 0x0002991D}, + {0x10030, 0x00029CE5}, + {0x10030, 0x0002A0DF}, + {0x10030, 0x0002A4A7}, + {0x10030, 0x0002A8A1}, + {0x10030, 0x0002AC67}, + {0x10030, 0x0002B061}, + {0x10030, 0x0002B427}, + {0x10030, 0x0002B821}, + {0x10030, 0x0002BC19}, + {0x10030, 0x0002C013}, + {0x10030, 0x0002C40D}, + {0x10030, 0x000301EF}, + {0x10030, 0x000305E7}, + {0x10030, 0x000309A7}, + {0x10030, 0x00030D65}, + {0x10030, 0x0003115F}, + {0x10030, 0x00031525}, + {0x10030, 0x0003191F}, + {0x10030, 0x00031CE7}, + {0x10030, 0x000320E1}, + {0x10030, 0x000324A9}, + {0x10030, 0x000328A3}, + {0x10030, 0x00032C69}, + {0x10030, 0x00033063}, + {0x10030, 0x00033429}, + {0x10030, 0x00033823}, + {0x10030, 0x00033C1D}, + {0x10030, 0x00034013}, + {0x10030, 0x0003440D}, + {0x10030, 0x000601F1}, + {0x10030, 0x000605E9}, + {0x10030, 0x000609A9}, + {0x10030, 0x00060D65}, + {0x10030, 0x0006115F}, + {0x10030, 0x00061525}, + {0x10030, 0x0006191F}, + {0x10030, 0x00061CE7}, + {0x10030, 0x000620E1}, + {0x10030, 0x000624DB}, + {0x10030, 0x000628A3}, + {0x10030, 0x00062C69}, + {0x10030, 0x00063063}, + {0x10030, 0x00063429}, + {0x10030, 0x00063823}, + {0x10030, 0x00063C1D}, + {0x10030, 0x00064013}, + {0x10030, 0x0006440D}, + {0x10030, 0x000681EF}, + {0x10030, 0x000685E7}, + {0x10030, 0x000689A7}, + {0x10030, 0x00068D61}, + {0x10030, 0x0006915B}, + {0x10030, 0x00069523}, + {0x10030, 0x0006991D}, + {0x10030, 0x00069CE5}, + {0x10030, 0x0006A0DF}, + {0x10030, 0x0006A4A7}, + {0x10030, 0x0006A8A1}, + {0x10030, 0x0006AC67}, + {0x10030, 0x0006B061}, + {0x10030, 0x0006B429}, + {0x10030, 0x0006B823}, + {0x10030, 0x0006BC1D}, + {0x10030, 0x0006C017}, + {0x10030, 0x0006C40D}, + {0x10030, 0x000701F1}, + {0x10030, 0x000705E9}, + {0x10030, 0x000709A9}, + {0x10030, 0x00070D63}, + {0x10030, 0x0007115D}, + {0x10030, 0x00071523}, + {0x10030, 0x0007191D}, + {0x10030, 0x00071D17}, + {0x10030, 0x00072111}, + {0x10030, 0x000724D9}, + {0x10030, 0x000728D3}, + {0x10030, 0x00072C67}, + {0x10030, 0x00073061}, + {0x10030, 0x00073427}, + {0x10030, 0x00073821}, + {0x10030, 0x00073C1B}, + {0x10030, 0x00074015}, + {0x10030, 0x0007440D}, + {0x10030, 0x000781F1}, + {0x10030, 0x000785EB}, + {0x10030, 0x000789E5}, + {0x10030, 0x00078DA3}, + {0x10030, 0x00079161}, + {0x10030, 0x0007955B}, + {0x10030, 0x00079923}, + {0x10030, 0x00079D1D}, + {0x10030, 0x0007A117}, + {0x10030, 0x0007A4DD}, + {0x10030, 0x0007A8D7}, + {0x10030, 0x0007AC9D}, + {0x10030, 0x0007B063}, + {0x10030, 0x0007B45D}, + {0x10030, 0x0007B857}, + {0x10030, 0x0007BC1D}, + {0x10030, 0x0007C017}, + {0x10030, 0x0007C40F}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x000001EF}, + {0x10030, 0x000005E9}, + {0x10030, 0x000009E3}, + {0x10030, 0x00000DDD}, + {0x10030, 0x000011D7}, + {0x10030, 0x0000159F}, + {0x10030, 0x00001999}, + {0x10030, 0x00001D5F}, + {0x10030, 0x00002159}, + {0x10030, 0x0000251F}, + {0x10030, 0x00002919}, + {0x10030, 0x00002CDF}, + {0x10030, 0x000030D9}, + {0x10030, 0x0000349F}, + {0x10030, 0x00003899}, + {0x10030, 0x00003C5F}, + {0x10030, 0x00004059}, + {0x10030, 0x00004453}, + {0x10030, 0x000201EF}, + {0x10030, 0x000205E9}, + {0x10030, 0x000209E3}, + {0x10030, 0x00020DA3}, + {0x10030, 0x00021161}, + {0x10030, 0x0002155B}, + {0x10030, 0x0002191F}, + {0x10030, 0x00021D19}, + {0x10030, 0x000220E1}, + {0x10030, 0x000224DB}, + {0x10030, 0x000228A3}, + {0x10030, 0x00022C9D}, + {0x10030, 0x00023063}, + {0x10030, 0x0002345D}, + {0x10030, 0x00023823}, + {0x10030, 0x00023C1B}, + {0x10030, 0x00024015}, + {0x10030, 0x0002440F}, + {0x10030, 0x000281EF}, + {0x10030, 0x000285E7}, + {0x10030, 0x000289A7}, + {0x10030, 0x00028D65}, + {0x10030, 0x0002915F}, + {0x10030, 0x00029523}, + {0x10030, 0x0002991D}, + {0x10030, 0x00029CE5}, + {0x10030, 0x0002A0DF}, + {0x10030, 0x0002A4A7}, + {0x10030, 0x0002A8A1}, + {0x10030, 0x0002AC67}, + {0x10030, 0x0002B061}, + {0x10030, 0x0002B427}, + {0x10030, 0x0002B821}, + {0x10030, 0x0002BC19}, + {0x10030, 0x0002C013}, + {0x10030, 0x0002C40D}, + {0x10030, 0x000301EF}, + {0x10030, 0x000305E7}, + {0x10030, 0x000309A7}, + {0x10030, 0x00030D65}, + {0x10030, 0x0003115F}, + {0x10030, 0x00031525}, + {0x10030, 0x0003191F}, + {0x10030, 0x00031CE7}, + {0x10030, 0x000320E1}, + {0x10030, 0x000324A9}, + {0x10030, 0x000328A3}, + {0x10030, 0x00032C69}, + {0x10030, 0x00033063}, + {0x10030, 0x00033429}, + {0x10030, 0x00033823}, + {0x10030, 0x00033C1D}, + {0x10030, 0x00034013}, + {0x10030, 0x0003440D}, + {0x10030, 0x000601F1}, + {0x10030, 0x000605E9}, + {0x10030, 0x000609A9}, + {0x10030, 0x00060D65}, + {0x10030, 0x0006115F}, + {0x10030, 0x00061525}, + {0x10030, 0x0006191F}, + {0x10030, 0x00061CE7}, + {0x10030, 0x000620E1}, + {0x10030, 0x000624DB}, + {0x10030, 0x000628A3}, + {0x10030, 0x00062C69}, + {0x10030, 0x00063063}, + {0x10030, 0x00063429}, + {0x10030, 0x00063823}, + {0x10030, 0x00063C1D}, + {0x10030, 0x00064013}, + {0x10030, 0x0006440D}, + {0x10030, 0x000681EF}, + {0x10030, 0x000685E7}, + {0x10030, 0x000689A7}, + {0x10030, 0x00068D61}, + {0x10030, 0x0006915B}, + {0x10030, 0x00069523}, + {0x10030, 0x0006991D}, + {0x10030, 0x00069CE5}, + {0x10030, 0x0006A0DF}, + {0x10030, 0x0006A4A7}, + {0x10030, 0x0006A8A1}, + {0x10030, 0x0006AC67}, + {0x10030, 0x0006B061}, + {0x10030, 0x0006B429}, + {0x10030, 0x0006B823}, + {0x10030, 0x0006BC1D}, + {0x10030, 0x0006C017}, + {0x10030, 0x0006C40D}, + {0x10030, 0x000701F1}, + {0x10030, 0x000705E9}, + {0x10030, 0x000709A9}, + {0x10030, 0x00070D63}, + {0x10030, 0x0007115D}, + {0x10030, 0x00071523}, + {0x10030, 0x0007191D}, + {0x10030, 0x00071D17}, + {0x10030, 0x00072111}, + {0x10030, 0x000724D9}, + {0x10030, 0x000728D3}, + {0x10030, 0x00072C67}, + {0x10030, 0x00073061}, + {0x10030, 0x00073427}, + {0x10030, 0x00073821}, + {0x10030, 0x00073C1B}, + {0x10030, 0x00074015}, + {0x10030, 0x0007440D}, + {0x10030, 0x000781F1}, + {0x10030, 0x000785EB}, + {0x10030, 0x000789E5}, + {0x10030, 0x00078DA3}, + {0x10030, 0x00079161}, + {0x10030, 0x0007955B}, + {0x10030, 0x00079923}, + {0x10030, 0x00079D1D}, + {0x10030, 0x0007A117}, + {0x10030, 0x0007A4DD}, + {0x10030, 0x0007A8D7}, + {0x10030, 0x0007AC9D}, + {0x10030, 0x0007B063}, + {0x10030, 0x0007B45D}, + {0x10030, 0x0007B857}, + {0x10030, 0x0007BC1D}, + {0x10030, 0x0007C017}, + {0x10030, 0x0007C40F}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x000001EF}, + {0x10030, 0x000005E9}, + {0x10030, 0x000009E3}, + {0x10030, 0x00000DDD}, + {0x10030, 0x000011D7}, + {0x10030, 0x0000159F}, + {0x10030, 0x00001999}, + {0x10030, 0x00001D5F}, + {0x10030, 0x00002159}, + {0x10030, 0x0000251F}, + {0x10030, 0x00002919}, + {0x10030, 0x00002CDF}, + {0x10030, 0x000030D9}, + {0x10030, 0x0000349F}, + {0x10030, 0x00003899}, + {0x10030, 0x00003C5F}, + {0x10030, 0x00004059}, + {0x10030, 0x00004453}, + {0x10030, 0x000201EF}, + {0x10030, 0x000205E9}, + {0x10030, 0x000209E3}, + {0x10030, 0x00020DA3}, + {0x10030, 0x00021161}, + {0x10030, 0x0002155B}, + {0x10030, 0x0002191F}, + {0x10030, 0x00021D19}, + {0x10030, 0x000220E1}, + {0x10030, 0x000224DB}, + {0x10030, 0x000228A3}, + {0x10030, 0x00022C9D}, + {0x10030, 0x00023063}, + {0x10030, 0x0002345D}, + {0x10030, 0x00023823}, + {0x10030, 0x00023C1B}, + {0x10030, 0x00024015}, + {0x10030, 0x0002440F}, + {0x10030, 0x000281EF}, + {0x10030, 0x000285E7}, + {0x10030, 0x000289A7}, + {0x10030, 0x00028D65}, + {0x10030, 0x0002915F}, + {0x10030, 0x00029523}, + {0x10030, 0x0002991D}, + {0x10030, 0x00029CE5}, + {0x10030, 0x0002A0DF}, + {0x10030, 0x0002A4A7}, + {0x10030, 0x0002A8A1}, + {0x10030, 0x0002AC67}, + {0x10030, 0x0002B061}, + {0x10030, 0x0002B427}, + {0x10030, 0x0002B821}, + {0x10030, 0x0002BC19}, + {0x10030, 0x0002C013}, + {0x10030, 0x0002C40D}, + {0x10030, 0x000301EF}, + {0x10030, 0x000305E7}, + {0x10030, 0x000309A7}, + {0x10030, 0x00030D65}, + {0x10030, 0x0003115F}, + {0x10030, 0x00031525}, + {0x10030, 0x0003191F}, + {0x10030, 0x00031CE7}, + {0x10030, 0x000320E1}, + {0x10030, 0x000324A9}, + {0x10030, 0x000328A3}, + {0x10030, 0x00032C69}, + {0x10030, 0x00033063}, + {0x10030, 0x00033429}, + {0x10030, 0x00033823}, + {0x10030, 0x00033C1D}, + {0x10030, 0x00034013}, + {0x10030, 0x0003440D}, + {0x10030, 0x000601F1}, + {0x10030, 0x000605E9}, + {0x10030, 0x000609A9}, + {0x10030, 0x00060D65}, + {0x10030, 0x0006115F}, + {0x10030, 0x00061525}, + {0x10030, 0x0006191F}, + {0x10030, 0x00061CE7}, + {0x10030, 0x000620E1}, + {0x10030, 0x000624DB}, + {0x10030, 0x000628A3}, + {0x10030, 0x00062C69}, + {0x10030, 0x00063063}, + {0x10030, 0x00063429}, + {0x10030, 0x00063823}, + {0x10030, 0x00063C1D}, + {0x10030, 0x00064013}, + {0x10030, 0x0006440D}, + {0x10030, 0x000681EF}, + {0x10030, 0x000685E7}, + {0x10030, 0x000689A7}, + {0x10030, 0x00068D61}, + {0x10030, 0x0006915B}, + {0x10030, 0x00069523}, + {0x10030, 0x0006991D}, + {0x10030, 0x00069CE5}, + {0x10030, 0x0006A0DF}, + {0x10030, 0x0006A4A7}, + {0x10030, 0x0006A8A1}, + {0x10030, 0x0006AC67}, + {0x10030, 0x0006B061}, + {0x10030, 0x0006B429}, + {0x10030, 0x0006B823}, + {0x10030, 0x0006BC1D}, + {0x10030, 0x0006C017}, + {0x10030, 0x0006C40D}, + {0x10030, 0x000701F1}, + {0x10030, 0x000705E9}, + {0x10030, 0x000709A9}, + {0x10030, 0x00070D63}, + {0x10030, 0x0007115D}, + {0x10030, 0x00071523}, + {0x10030, 0x0007191D}, + {0x10030, 0x00071D17}, + {0x10030, 0x00072111}, + {0x10030, 0x000724D9}, + {0x10030, 0x000728D3}, + {0x10030, 0x00072C67}, + {0x10030, 0x00073061}, + {0x10030, 0x00073427}, + {0x10030, 0x00073821}, + {0x10030, 0x00073C1B}, + {0x10030, 0x00074015}, + {0x10030, 0x0007440D}, + {0x10030, 0x000781F1}, + {0x10030, 0x000785EB}, + {0x10030, 0x000789E5}, + {0x10030, 0x00078DA3}, + {0x10030, 0x00079161}, + {0x10030, 0x0007955B}, + {0x10030, 0x00079923}, + {0x10030, 0x00079D1D}, + {0x10030, 0x0007A117}, + {0x10030, 0x0007A4DD}, + {0x10030, 0x0007A8D7}, + {0x10030, 0x0007AC9D}, + {0x10030, 0x0007B063}, + {0x10030, 0x0007B45D}, + {0x10030, 0x0007B857}, + {0x10030, 0x0007BC1D}, + {0x10030, 0x0007C017}, {0x10030, 0x0007C40F}, {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, {0x10030, 0x000001EF}, @@ -10118,1002 +20173,1002 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radiob_regs[] = { {0x10030, 0x00003C5F}, {0x10030, 0x00004059}, {0x10030, 0x00004453}, - {0x10030, 0x000201ED}, - {0x10030, 0x000205AD}, - {0x10030, 0x000209A7}, - {0x10030, 0x00020DA1}, - {0x10030, 0x0002119B}, - {0x10030, 0x00021561}, - {0x10030, 0x0002195B}, - {0x10030, 0x00021D27}, - {0x10030, 0x00022121}, - {0x10030, 0x000224E9}, - {0x10030, 0x000228E3}, - {0x10030, 0x00022CA9}, - {0x10030, 0x000230A3}, - {0x10030, 0x00023469}, - {0x10030, 0x00023863}, - {0x10030, 0x00023C29}, - {0x10030, 0x00024023}, - {0x10030, 0x0002441D}, + {0x10030, 0x000201EF}, + {0x10030, 0x000205E9}, + {0x10030, 0x000209E3}, + {0x10030, 0x00020DA3}, + {0x10030, 0x00021161}, + {0x10030, 0x0002155B}, + {0x10030, 0x0002191F}, + {0x10030, 0x00021D19}, + {0x10030, 0x000220E1}, + {0x10030, 0x000224DB}, + {0x10030, 0x000228A3}, + {0x10030, 0x00022C9D}, + {0x10030, 0x00023063}, + {0x10030, 0x0002345D}, + {0x10030, 0x00023823}, + {0x10030, 0x00023C1B}, + {0x10030, 0x00024015}, + {0x10030, 0x0002440F}, {0x10030, 0x000281EF}, - {0x10030, 0x000285AF}, - {0x10030, 0x000289A9}, - {0x10030, 0x00028DA3}, - {0x10030, 0x0002919D}, - {0x10030, 0x00029563}, - {0x10030, 0x0002995D}, - {0x10030, 0x00029D25}, - {0x10030, 0x0002A11F}, - {0x10030, 0x0002A4E7}, - {0x10030, 0x0002A8E1}, - {0x10030, 0x0002ACA7}, - {0x10030, 0x0002B0A1}, - {0x10030, 0x0002B467}, - {0x10030, 0x0002B861}, - {0x10030, 0x0002BC27}, - {0x10030, 0x0002C021}, - {0x10030, 0x0002C41B}, + {0x10030, 0x000285E7}, + {0x10030, 0x000289A7}, + {0x10030, 0x00028D65}, + {0x10030, 0x0002915F}, + {0x10030, 0x00029523}, + {0x10030, 0x0002991D}, + {0x10030, 0x00029CE5}, + {0x10030, 0x0002A0DF}, + {0x10030, 0x0002A4A7}, + {0x10030, 0x0002A8A1}, + {0x10030, 0x0002AC67}, + {0x10030, 0x0002B061}, + {0x10030, 0x0002B427}, + {0x10030, 0x0002B821}, + {0x10030, 0x0002BC19}, + {0x10030, 0x0002C013}, + {0x10030, 0x0002C40D}, {0x10030, 0x000301EF}, - {0x10030, 0x000305AF}, - {0x10030, 0x000309A9}, - {0x10030, 0x00030DA3}, - {0x10030, 0x0003119D}, - {0x10030, 0x00031563}, - {0x10030, 0x0003195D}, - {0x10030, 0x00031D25}, - {0x10030, 0x0003211F}, - {0x10030, 0x000324E7}, - {0x10030, 0x000328E1}, - {0x10030, 0x00032CA7}, - {0x10030, 0x000330A1}, - {0x10030, 0x00033467}, - {0x10030, 0x00033861}, - {0x10030, 0x00033C27}, - {0x10030, 0x00034021}, - {0x10030, 0x0003441B}, - {0x10030, 0x000601EB}, - {0x10030, 0x000605AB}, - {0x10030, 0x000609A5}, - {0x10030, 0x00060D9F}, - {0x10030, 0x00061199}, - {0x10030, 0x00061593}, - {0x10030, 0x00061959}, - {0x10030, 0x00061D53}, - {0x10030, 0x0006211B}, - {0x10030, 0x00062515}, - {0x10030, 0x000628DD}, - {0x10030, 0x00062CD7}, - {0x10030, 0x0006309D}, - {0x10030, 0x00063497}, - {0x10030, 0x0006385D}, - {0x10030, 0x00063C57}, - {0x10030, 0x0006401D}, - {0x10030, 0x00064417}, - {0x10030, 0x000681E7}, - {0x10030, 0x000685A7}, - {0x10030, 0x000689A1}, - {0x10030, 0x00068D9B}, - {0x10030, 0x00069195}, - {0x10030, 0x0006955F}, - {0x10030, 0x00069959}, - {0x10030, 0x00069D21}, - {0x10030, 0x0006A11B}, - {0x10030, 0x0006A4E3}, - {0x10030, 0x0006A8DD}, - {0x10030, 0x0006ACA5}, - {0x10030, 0x0006B09F}, - {0x10030, 0x0006B465}, - {0x10030, 0x0006B85F}, - {0x10030, 0x0006BC25}, - {0x10030, 0x0006C01F}, - {0x10030, 0x0006C419}, - {0x10030, 0x000701E7}, - {0x10030, 0x000705A7}, - {0x10030, 0x000709A1}, - {0x10030, 0x00070D9B}, - {0x10030, 0x00071195}, - {0x10030, 0x0007155B}, - {0x10030, 0x00071955}, - {0x10030, 0x00071D1D}, - {0x10030, 0x00072117}, - {0x10030, 0x000724DF}, - {0x10030, 0x000728D9}, - {0x10030, 0x00072CA1}, - {0x10030, 0x0007309B}, - {0x10030, 0x00073461}, - {0x10030, 0x0007385B}, - {0x10030, 0x00073C21}, - {0x10030, 0x0007401B}, - {0x10030, 0x0007441B}, - {0x10030, 0x000781EF}, - {0x10030, 0x000785E9}, - {0x10030, 0x000789E3}, + {0x10030, 0x000305E7}, + {0x10030, 0x000309A7}, + {0x10030, 0x00030D65}, + {0x10030, 0x0003115F}, + {0x10030, 0x00031525}, + {0x10030, 0x0003191F}, + {0x10030, 0x00031CE7}, + {0x10030, 0x000320E1}, + {0x10030, 0x000324A9}, + {0x10030, 0x000328A3}, + {0x10030, 0x00032C69}, + {0x10030, 0x00033063}, + {0x10030, 0x00033429}, + {0x10030, 0x00033823}, + {0x10030, 0x00033C1D}, + {0x10030, 0x00034013}, + {0x10030, 0x0003440D}, + {0x10030, 0x000601F1}, + {0x10030, 0x000605E9}, + {0x10030, 0x000609A9}, + {0x10030, 0x00060D65}, + {0x10030, 0x0006115F}, + {0x10030, 0x00061525}, + {0x10030, 0x0006191F}, + {0x10030, 0x00061CE7}, + {0x10030, 0x000620E1}, + {0x10030, 0x000624DB}, + {0x10030, 0x000628A3}, + {0x10030, 0x00062C69}, + {0x10030, 0x00063063}, + {0x10030, 0x00063429}, + {0x10030, 0x00063823}, + {0x10030, 0x00063C1D}, + {0x10030, 0x00064013}, + {0x10030, 0x0006440D}, + {0x10030, 0x000681EF}, + {0x10030, 0x000685E7}, + {0x10030, 0x000689A7}, + {0x10030, 0x00068D61}, + {0x10030, 0x0006915B}, + {0x10030, 0x00069523}, + {0x10030, 0x0006991D}, + {0x10030, 0x00069CE5}, + {0x10030, 0x0006A0DF}, + {0x10030, 0x0006A4A7}, + {0x10030, 0x0006A8A1}, + {0x10030, 0x0006AC67}, + {0x10030, 0x0006B061}, + {0x10030, 0x0006B429}, + {0x10030, 0x0006B823}, + {0x10030, 0x0006BC1D}, + {0x10030, 0x0006C017}, + {0x10030, 0x0006C40D}, + {0x10030, 0x000701F1}, + {0x10030, 0x000705E9}, + {0x10030, 0x000709A9}, + {0x10030, 0x00070D63}, + {0x10030, 0x0007115D}, + {0x10030, 0x00071523}, + {0x10030, 0x0007191D}, + {0x10030, 0x00071D17}, + {0x10030, 0x00072111}, + {0x10030, 0x000724D9}, + {0x10030, 0x000728D3}, + {0x10030, 0x00072C67}, + {0x10030, 0x00073061}, + {0x10030, 0x00073427}, + {0x10030, 0x00073821}, + {0x10030, 0x00073C1B}, + {0x10030, 0x00074015}, + {0x10030, 0x0007440D}, + {0x10030, 0x000781F1}, + {0x10030, 0x000785EB}, + {0x10030, 0x000789E5}, {0x10030, 0x00078DA3}, {0x10030, 0x00079161}, {0x10030, 0x0007955B}, - {0x10030, 0x00079921}, - {0x10030, 0x00079D1B}, - {0x10030, 0x0007A0E1}, - {0x10030, 0x0007A4DB}, - {0x10030, 0x0007A8A1}, - {0x10030, 0x0007AC9B}, - {0x10030, 0x0007B061}, - {0x10030, 0x0007B45B}, - {0x10030, 0x0007B821}, - {0x10030, 0x0007BC1B}, - {0x10030, 0x0007C015}, + {0x10030, 0x00079923}, + {0x10030, 0x00079D1D}, + {0x10030, 0x0007A117}, + {0x10030, 0x0007A4DD}, + {0x10030, 0x0007A8D7}, + {0x10030, 0x0007AC9D}, + {0x10030, 0x0007B063}, + {0x10030, 0x0007B45D}, + {0x10030, 0x0007B857}, + {0x10030, 0x0007BC1D}, + {0x10030, 0x0007C017}, {0x10030, 0x0007C40F}, {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, - {0x10030, 0x000201DF}, - {0x10030, 0x000205D9}, - {0x10030, 0x000209D3}, - {0x10030, 0x00020D99}, - {0x10030, 0x00021193}, - {0x10030, 0x0002155F}, - {0x10030, 0x00021959}, - {0x10030, 0x00021D21}, - {0x10030, 0x00022119}, - {0x10030, 0x000224DF}, - {0x10030, 0x000228D9}, - {0x10030, 0x00022C9F}, - {0x10030, 0x00023099}, - {0x10030, 0x0002345F}, - {0x10030, 0x00023859}, - {0x10030, 0x00023C1F}, - {0x10030, 0x00024019}, - {0x10030, 0x00024413}, - {0x10030, 0x000281CD}, - {0x10030, 0x000285DB}, - {0x10030, 0x000289D5}, - {0x10030, 0x00028D9B}, - {0x10030, 0x0002918D}, - {0x10030, 0x00029555}, - {0x10030, 0x00029957}, - {0x10030, 0x00029D1F}, - {0x10030, 0x0002A119}, - {0x10030, 0x0002A4DF}, - {0x10030, 0x0002A8D9}, - {0x10030, 0x0002AC9F}, - {0x10030, 0x0002B099}, - {0x10030, 0x0002B45F}, - {0x10030, 0x0002B859}, - {0x10030, 0x0002BC1F}, - {0x10030, 0x0002C019}, - {0x10030, 0x0002C413}, - {0x10030, 0x000301D9}, - {0x10030, 0x000305DB}, - {0x10030, 0x000309D5}, - {0x10030, 0x00030D9B}, - {0x10030, 0x00031195}, - {0x10030, 0x0003155D}, - {0x10030, 0x00031955}, - {0x10030, 0x00031D1D}, - {0x10030, 0x00032119}, - {0x10030, 0x000324DF}, - {0x10030, 0x000328D9}, - {0x10030, 0x00032C9F}, - {0x10030, 0x00033099}, - {0x10030, 0x0003345F}, - {0x10030, 0x00033859}, - {0x10030, 0x00033C1F}, - {0x10030, 0x00034019}, - {0x10030, 0x00034413}, - {0x10030, 0x000601E1}, - {0x10030, 0x000605DB}, - {0x10030, 0x000609D5}, - {0x10030, 0x00060D9B}, - {0x10030, 0x00061195}, - {0x10030, 0x0006155B}, - {0x10030, 0x00061957}, - {0x10030, 0x00061D1F}, - {0x10030, 0x00062119}, - {0x10030, 0x000624DF}, - {0x10030, 0x000628D9}, - {0x10030, 0x00062C9F}, - {0x10030, 0x00063099}, - {0x10030, 0x0006345F}, - {0x10030, 0x00063859}, - {0x10030, 0x00063C1F}, - {0x10030, 0x00064019}, - {0x10030, 0x00064413}, - {0x10030, 0x000681E1}, - {0x10030, 0x000685DB}, - {0x10030, 0x000689D5}, - {0x10030, 0x00068D9B}, - {0x10030, 0x00069195}, - {0x10030, 0x0006955B}, - {0x10030, 0x00069957}, - {0x10030, 0x00069D1F}, - {0x10030, 0x0006A119}, - {0x10030, 0x0006A4DF}, - {0x10030, 0x0006A8D9}, - {0x10030, 0x0006AC9F}, - {0x10030, 0x0006B099}, - {0x10030, 0x0006B45F}, - {0x10030, 0x0006B859}, - {0x10030, 0x0006BC1F}, - {0x10030, 0x0006C019}, - {0x10030, 0x0006C413}, - {0x10030, 0x000701E1}, - {0x10030, 0x000705DB}, - {0x10030, 0x000709D5}, - {0x10030, 0x00070D9B}, - {0x10030, 0x00071195}, - {0x10030, 0x0007155B}, - {0x10030, 0x00071957}, - {0x10030, 0x00071D1F}, - {0x10030, 0x00072119}, - {0x10030, 0x000724DF}, - {0x10030, 0x000728D9}, - {0x10030, 0x00072C9F}, - {0x10030, 0x00073099}, - {0x10030, 0x0007345F}, - {0x10030, 0x00073859}, - {0x10030, 0x00073C1F}, - {0x10030, 0x00074019}, - {0x10030, 0x00074413}, - {0x10030, 0x000781DF}, - {0x10030, 0x000785D9}, - {0x10030, 0x000789D3}, - {0x10030, 0x00078D99}, - {0x10030, 0x00079193}, - {0x10030, 0x0007955F}, - {0x10030, 0x00079959}, - {0x10030, 0x00079D21}, - {0x10030, 0x0007A115}, - {0x10030, 0x0007A4DF}, - {0x10030, 0x0007A8D9}, - {0x10030, 0x0007AC9F}, - {0x10030, 0x0007B099}, - {0x10030, 0x0007B45F}, - {0x10030, 0x0007B859}, - {0x10030, 0x0007BC1F}, - {0x10030, 0x0007C019}, - {0x10030, 0x0007C413}, - {0x10030, 0x00000000}, - {0x10030, 0x000785A9}, - {0x10030, 0x000789A3}, - {0x10030, 0x00078D9D}, - {0x10030, 0x00079197}, - {0x10030, 0x00079591}, - {0x10030, 0x00079957}, - {0x10030, 0x00079D51}, - {0x10030, 0x0007A119}, - {0x10030, 0x0007A513}, - {0x10030, 0x0007A8D9}, - {0x10030, 0x0007ACD3}, - {0x10030, 0x0007B099}, - {0x10030, 0x0007B493}, - {0x10030, 0x0007B859}, - {0x10030, 0x0007BC53}, - {0x10030, 0x0007C019}, - {0x10030, 0x0007C413}, + {0x10030, 0x000001EF}, + {0x10030, 0x000005E9}, + {0x10030, 0x000009E3}, + {0x10030, 0x00000DDD}, + {0x10030, 0x000011D7}, + {0x10030, 0x0000159F}, + {0x10030, 0x00001999}, + {0x10030, 0x00001D5F}, + {0x10030, 0x00002159}, + {0x10030, 0x0000251F}, + {0x10030, 0x00002919}, + {0x10030, 0x00002CDF}, + {0x10030, 0x000030D9}, + {0x10030, 0x0000349F}, + {0x10030, 0x00003899}, + {0x10030, 0x00003C5F}, + {0x10030, 0x00004059}, + {0x10030, 0x00004453}, + {0x10030, 0x000201A7}, + {0x10030, 0x000205A1}, + {0x10030, 0x0002099B}, + {0x10030, 0x00020D95}, + {0x10030, 0x0002115B}, + {0x10030, 0x00021555}, + {0x10030, 0x00021921}, + {0x10030, 0x00021D1B}, + {0x10030, 0x000220E3}, + {0x10030, 0x000224DD}, + {0x10030, 0x000228A3}, + {0x10030, 0x00022C9D}, + {0x10030, 0x00023063}, + {0x10030, 0x0002345D}, + {0x10030, 0x00023823}, + {0x10030, 0x00023C1D}, + {0x10030, 0x00024017}, + {0x10030, 0x00024411}, + {0x10030, 0x000281A9}, + {0x10030, 0x000285A3}, + {0x10030, 0x0002899D}, + {0x10030, 0x00028D97}, + {0x10030, 0x0002915D}, + {0x10030, 0x00029557}, + {0x10030, 0x0002991F}, + {0x10030, 0x00029D19}, + {0x10030, 0x0002A0E1}, + {0x10030, 0x0002A4DB}, + {0x10030, 0x0002A8A1}, + {0x10030, 0x0002AC9B}, + {0x10030, 0x0002B061}, + {0x10030, 0x0002B45B}, + {0x10030, 0x0002B821}, + {0x10030, 0x0002BC1B}, + {0x10030, 0x0002C015}, + {0x10030, 0x0002C40F}, + {0x10030, 0x000301A9}, + {0x10030, 0x000305A3}, + {0x10030, 0x0003099D}, + {0x10030, 0x00030D97}, + {0x10030, 0x0003115D}, + {0x10030, 0x00031557}, + {0x10030, 0x0003191F}, + {0x10030, 0x00031D19}, + {0x10030, 0x000320E1}, + {0x10030, 0x000324DB}, + {0x10030, 0x000328A1}, + {0x10030, 0x00032C9B}, + {0x10030, 0x00033061}, + {0x10030, 0x0003345B}, + {0x10030, 0x00033821}, + {0x10030, 0x00033C1B}, + {0x10030, 0x00034015}, + {0x10030, 0x0003440F}, + {0x10030, 0x000601F1}, + {0x10030, 0x000605E9}, + {0x10030, 0x000609A9}, + {0x10030, 0x00060D65}, + {0x10030, 0x0006115F}, + {0x10030, 0x00061525}, + {0x10030, 0x0006191F}, + {0x10030, 0x00061CE7}, + {0x10030, 0x000620E1}, + {0x10030, 0x000624DB}, + {0x10030, 0x000628A3}, + {0x10030, 0x00062C69}, + {0x10030, 0x00063063}, + {0x10030, 0x00063429}, + {0x10030, 0x00063823}, + {0x10030, 0x00063C1D}, + {0x10030, 0x00064013}, + {0x10030, 0x0006440D}, + {0x10030, 0x000681EF}, + {0x10030, 0x000685E7}, + {0x10030, 0x000689A7}, + {0x10030, 0x00068D61}, + {0x10030, 0x0006915B}, + {0x10030, 0x00069523}, + {0x10030, 0x0006991D}, + {0x10030, 0x00069CE5}, + {0x10030, 0x0006A0DF}, + {0x10030, 0x0006A4A7}, + {0x10030, 0x0006A8A1}, + {0x10030, 0x0006AC67}, + {0x10030, 0x0006B061}, + {0x10030, 0x0006B429}, + {0x10030, 0x0006B823}, + {0x10030, 0x0006BC1D}, + {0x10030, 0x0006C017}, + {0x10030, 0x0006C40D}, + {0x10030, 0x000701F1}, + {0x10030, 0x000705E9}, + {0x10030, 0x000709A9}, + {0x10030, 0x00070D63}, + {0x10030, 0x0007115D}, + {0x10030, 0x00071523}, + {0x10030, 0x0007191D}, + {0x10030, 0x00071D17}, + {0x10030, 0x000720DF}, + {0x10030, 0x000724D9}, + {0x10030, 0x000728D3}, + {0x10030, 0x00072C67}, + {0x10030, 0x00073061}, + {0x10030, 0x00073427}, + {0x10030, 0x00073821}, + {0x10030, 0x00073C1B}, + {0x10030, 0x00074015}, + {0x10030, 0x0007440D}, + {0x10030, 0x000781F1}, + {0x10030, 0x000785EB}, + {0x10030, 0x000789E5}, + {0x10030, 0x00078DA3}, + {0x10030, 0x00079161}, + {0x10030, 0x0007955B}, + {0x10030, 0x00079923}, + {0x10030, 0x00079D1D}, + {0x10030, 0x0007A117}, + {0x10030, 0x0007A4DD}, + {0x10030, 0x0007A8D7}, + {0x10030, 0x0007AC9D}, + {0x10030, 0x0007B063}, + {0x10030, 0x0007B45D}, + {0x10030, 0x0007B857}, + {0x10030, 0x0007BC1D}, + {0x10030, 0x0007C017}, + {0x10030, 0x0007C40F}, {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, - {0x10030, 0x000201DF}, - {0x10030, 0x000205D9}, - {0x10030, 0x000209D3}, - {0x10030, 0x00020D99}, - {0x10030, 0x00021193}, - {0x10030, 0x0002155F}, - {0x10030, 0x00021959}, - {0x10030, 0x00021D21}, - {0x10030, 0x00022119}, - {0x10030, 0x000224DF}, - {0x10030, 0x000228D9}, - {0x10030, 0x00022C9F}, - {0x10030, 0x00023099}, - {0x10030, 0x0002345F}, - {0x10030, 0x00023859}, - {0x10030, 0x00023C1F}, - {0x10030, 0x00024019}, - {0x10030, 0x00024413}, - {0x10030, 0x000281CD}, - {0x10030, 0x000285DB}, - {0x10030, 0x000289D5}, - {0x10030, 0x00028D9B}, - {0x10030, 0x0002918D}, - {0x10030, 0x00029555}, - {0x10030, 0x00029957}, - {0x10030, 0x00029D1F}, - {0x10030, 0x0002A119}, - {0x10030, 0x0002A4DF}, - {0x10030, 0x0002A8D9}, - {0x10030, 0x0002AC9F}, - {0x10030, 0x0002B099}, - {0x10030, 0x0002B45F}, - {0x10030, 0x0002B859}, - {0x10030, 0x0002BC1F}, - {0x10030, 0x0002C019}, - {0x10030, 0x0002C413}, - {0x10030, 0x000301D9}, - {0x10030, 0x000305DB}, - {0x10030, 0x000309D5}, - {0x10030, 0x00030D9B}, - {0x10030, 0x00031195}, - {0x10030, 0x0003155D}, - {0x10030, 0x00031955}, - {0x10030, 0x00031D1D}, - {0x10030, 0x00032119}, - {0x10030, 0x000324DF}, - {0x10030, 0x000328D9}, - {0x10030, 0x00032C9F}, - {0x10030, 0x00033099}, - {0x10030, 0x0003345F}, - {0x10030, 0x00033859}, - {0x10030, 0x00033C1F}, - {0x10030, 0x00034019}, - {0x10030, 0x00034413}, - {0x10030, 0x000601E1}, - {0x10030, 0x000605DB}, - {0x10030, 0x000609D5}, - {0x10030, 0x00060D9B}, - {0x10030, 0x00061195}, - {0x10030, 0x0006155B}, - {0x10030, 0x00061957}, - {0x10030, 0x00061D1F}, - {0x10030, 0x00062119}, - {0x10030, 0x000624DF}, - {0x10030, 0x000628D9}, - {0x10030, 0x00062C9F}, - {0x10030, 0x00063099}, - {0x10030, 0x0006345F}, - {0x10030, 0x00063859}, - {0x10030, 0x00063C1F}, - {0x10030, 0x00064019}, - {0x10030, 0x00064413}, - {0x10030, 0x000681E1}, - {0x10030, 0x000685DB}, - {0x10030, 0x000689D5}, - {0x10030, 0x00068D9B}, - {0x10030, 0x00069195}, - {0x10030, 0x0006955B}, - {0x10030, 0x00069957}, - {0x10030, 0x00069D1F}, - {0x10030, 0x0006A119}, - {0x10030, 0x0006A4DF}, - {0x10030, 0x0006A8D9}, - {0x10030, 0x0006AC9F}, - {0x10030, 0x0006B099}, - {0x10030, 0x0006B45F}, - {0x10030, 0x0006B859}, - {0x10030, 0x0006BC1F}, - {0x10030, 0x0006C019}, - {0x10030, 0x0006C413}, - {0x10030, 0x000701E1}, - {0x10030, 0x000705DB}, - {0x10030, 0x000709D5}, - {0x10030, 0x00070D9B}, - {0x10030, 0x00071195}, - {0x10030, 0x0007155B}, - {0x10030, 0x00071957}, - {0x10030, 0x00071D1F}, - {0x10030, 0x00072119}, - {0x10030, 0x000724DF}, - {0x10030, 0x000728D9}, - {0x10030, 0x00072C9F}, - {0x10030, 0x00073099}, - {0x10030, 0x0007345F}, - {0x10030, 0x00073859}, - {0x10030, 0x00073C1F}, - {0x10030, 0x00074019}, - {0x10030, 0x00074413}, - {0x10030, 0x000781DF}, - {0x10030, 0x000785D9}, - {0x10030, 0x000789D3}, - {0x10030, 0x00078D99}, - {0x10030, 0x00079193}, - {0x10030, 0x0007955F}, - {0x10030, 0x00079959}, - {0x10030, 0x00079D21}, - {0x10030, 0x0007A115}, - {0x10030, 0x0007A4DF}, - {0x10030, 0x0007A8D9}, - {0x10030, 0x0007AC9F}, - {0x10030, 0x0007B099}, - {0x10030, 0x0007B45F}, - {0x10030, 0x0007B859}, - {0x10030, 0x0007BC1F}, - {0x10030, 0x0007C019}, - {0x10030, 0x0007C413}, - {0x10030, 0x00000000}, - {0x10030, 0x000785A9}, - {0x10030, 0x000789A3}, - {0x10030, 0x00078D9D}, - {0x10030, 0x00079197}, - {0x10030, 0x00079591}, - {0x10030, 0x00079957}, - {0x10030, 0x00079D51}, - {0x10030, 0x0007A119}, - {0x10030, 0x0007A513}, - {0x10030, 0x0007A8D9}, - {0x10030, 0x0007ACD3}, - {0x10030, 0x0007B099}, - {0x10030, 0x0007B493}, - {0x10030, 0x0007B859}, - {0x10030, 0x0007BC53}, - {0x10030, 0x0007C019}, - {0x10030, 0x0007C413}, + {0x10030, 0x000001EF}, + {0x10030, 0x000005E9}, + {0x10030, 0x000009E3}, + {0x10030, 0x00000DDD}, + {0x10030, 0x000011D7}, + {0x10030, 0x0000159F}, + {0x10030, 0x00001999}, + {0x10030, 0x00001D5F}, + {0x10030, 0x00002159}, + {0x10030, 0x0000251F}, + {0x10030, 0x00002919}, + {0x10030, 0x00002CDF}, + {0x10030, 0x000030D9}, + {0x10030, 0x0000349F}, + {0x10030, 0x00003899}, + {0x10030, 0x00003C5F}, + {0x10030, 0x00004059}, + {0x10030, 0x00004453}, + {0x10030, 0x000201A7}, + {0x10030, 0x000205A1}, + {0x10030, 0x0002099B}, + {0x10030, 0x00020D95}, + {0x10030, 0x0002115B}, + {0x10030, 0x00021555}, + {0x10030, 0x00021921}, + {0x10030, 0x00021D1B}, + {0x10030, 0x000220E3}, + {0x10030, 0x000224DD}, + {0x10030, 0x000228A3}, + {0x10030, 0x00022C9D}, + {0x10030, 0x00023063}, + {0x10030, 0x0002345D}, + {0x10030, 0x00023823}, + {0x10030, 0x00023C1D}, + {0x10030, 0x00024017}, + {0x10030, 0x00024411}, + {0x10030, 0x000281A9}, + {0x10030, 0x000285A3}, + {0x10030, 0x0002899D}, + {0x10030, 0x00028D97}, + {0x10030, 0x0002915D}, + {0x10030, 0x00029557}, + {0x10030, 0x0002991F}, + {0x10030, 0x00029D19}, + {0x10030, 0x0002A0E1}, + {0x10030, 0x0002A4DB}, + {0x10030, 0x0002A8A1}, + {0x10030, 0x0002AC9B}, + {0x10030, 0x0002B061}, + {0x10030, 0x0002B45B}, + {0x10030, 0x0002B821}, + {0x10030, 0x0002BC1B}, + {0x10030, 0x0002C015}, + {0x10030, 0x0002C40F}, + {0x10030, 0x000301A9}, + {0x10030, 0x000305A3}, + {0x10030, 0x0003099D}, + {0x10030, 0x00030D97}, + {0x10030, 0x0003115D}, + {0x10030, 0x00031557}, + {0x10030, 0x0003191F}, + {0x10030, 0x00031D19}, + {0x10030, 0x000320E1}, + {0x10030, 0x000324DB}, + {0x10030, 0x000328A1}, + {0x10030, 0x00032C9B}, + {0x10030, 0x00033061}, + {0x10030, 0x0003345B}, + {0x10030, 0x00033821}, + {0x10030, 0x00033C1B}, + {0x10030, 0x00034015}, + {0x10030, 0x0003440F}, + {0x10030, 0x000601F1}, + {0x10030, 0x000605E9}, + {0x10030, 0x000609A9}, + {0x10030, 0x00060D65}, + {0x10030, 0x0006115F}, + {0x10030, 0x00061525}, + {0x10030, 0x0006191F}, + {0x10030, 0x00061CE7}, + {0x10030, 0x000620E1}, + {0x10030, 0x000624DB}, + {0x10030, 0x000628A3}, + {0x10030, 0x00062C69}, + {0x10030, 0x00063063}, + {0x10030, 0x00063429}, + {0x10030, 0x00063823}, + {0x10030, 0x00063C1D}, + {0x10030, 0x00064013}, + {0x10030, 0x0006440D}, + {0x10030, 0x000681EF}, + {0x10030, 0x000685E7}, + {0x10030, 0x000689A7}, + {0x10030, 0x00068D61}, + {0x10030, 0x0006915B}, + {0x10030, 0x00069523}, + {0x10030, 0x0006991D}, + {0x10030, 0x00069CE5}, + {0x10030, 0x0006A0DF}, + {0x10030, 0x0006A4A7}, + {0x10030, 0x0006A8A1}, + {0x10030, 0x0006AC67}, + {0x10030, 0x0006B061}, + {0x10030, 0x0006B429}, + {0x10030, 0x0006B823}, + {0x10030, 0x0006BC1D}, + {0x10030, 0x0006C017}, + {0x10030, 0x0006C40D}, + {0x10030, 0x000701F1}, + {0x10030, 0x000705E9}, + {0x10030, 0x000709A9}, + {0x10030, 0x00070D63}, + {0x10030, 0x0007115D}, + {0x10030, 0x00071523}, + {0x10030, 0x0007191D}, + {0x10030, 0x00071D17}, + {0x10030, 0x000720DF}, + {0x10030, 0x000724D9}, + {0x10030, 0x000728D3}, + {0x10030, 0x00072C67}, + {0x10030, 0x00073061}, + {0x10030, 0x00073427}, + {0x10030, 0x00073821}, + {0x10030, 0x00073C1B}, + {0x10030, 0x00074015}, + {0x10030, 0x0007440D}, + {0x10030, 0x000781F1}, + {0x10030, 0x000785EB}, + {0x10030, 0x000789E5}, + {0x10030, 0x00078DA3}, + {0x10030, 0x00079161}, + {0x10030, 0x0007955B}, + {0x10030, 0x00079923}, + {0x10030, 0x00079D1D}, + {0x10030, 0x0007A117}, + {0x10030, 0x0007A4DD}, + {0x10030, 0x0007A8D7}, + {0x10030, 0x0007AC9D}, + {0x10030, 0x0007B063}, + {0x10030, 0x0007B45D}, + {0x10030, 0x0007B857}, + {0x10030, 0x0007BC1D}, + {0x10030, 0x0007C017}, + {0x10030, 0x0007C40F}, {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, - {0x10030, 0x000201DF}, - {0x10030, 0x000205D9}, - {0x10030, 0x000209D3}, - {0x10030, 0x00020D99}, - {0x10030, 0x00021193}, - {0x10030, 0x0002155F}, - {0x10030, 0x00021959}, - {0x10030, 0x00021D21}, - {0x10030, 0x00022119}, - {0x10030, 0x000224DF}, - {0x10030, 0x000228D9}, - {0x10030, 0x00022C9F}, - {0x10030, 0x00023099}, - {0x10030, 0x0002345F}, - {0x10030, 0x00023859}, - {0x10030, 0x00023C1F}, - {0x10030, 0x00024019}, - {0x10030, 0x00024413}, - {0x10030, 0x000281CD}, - {0x10030, 0x000285DB}, - {0x10030, 0x000289D5}, - {0x10030, 0x00028D9B}, - {0x10030, 0x0002918D}, - {0x10030, 0x00029555}, - {0x10030, 0x00029957}, - {0x10030, 0x00029D1F}, - {0x10030, 0x0002A119}, - {0x10030, 0x0002A4DF}, - {0x10030, 0x0002A8D9}, - {0x10030, 0x0002AC9F}, - {0x10030, 0x0002B099}, - {0x10030, 0x0002B45F}, - {0x10030, 0x0002B859}, - {0x10030, 0x0002BC1F}, - {0x10030, 0x0002C019}, - {0x10030, 0x0002C413}, - {0x10030, 0x000301D9}, - {0x10030, 0x000305DB}, - {0x10030, 0x000309D5}, - {0x10030, 0x00030D9B}, - {0x10030, 0x00031195}, - {0x10030, 0x0003155D}, - {0x10030, 0x00031955}, - {0x10030, 0x00031D1D}, - {0x10030, 0x00032119}, - {0x10030, 0x000324DF}, - {0x10030, 0x000328D9}, - {0x10030, 0x00032C9F}, - {0x10030, 0x00033099}, - {0x10030, 0x0003345F}, - {0x10030, 0x00033859}, - {0x10030, 0x00033C1F}, - {0x10030, 0x00034019}, - {0x10030, 0x00034413}, - {0x10030, 0x000601E1}, - {0x10030, 0x000605DB}, - {0x10030, 0x000609D5}, - {0x10030, 0x00060D9B}, - {0x10030, 0x00061195}, - {0x10030, 0x0006155B}, - {0x10030, 0x00061957}, - {0x10030, 0x00061D1F}, - {0x10030, 0x00062119}, - {0x10030, 0x000624DF}, - {0x10030, 0x000628D9}, - {0x10030, 0x00062C9F}, - {0x10030, 0x00063099}, - {0x10030, 0x0006345F}, - {0x10030, 0x00063859}, - {0x10030, 0x00063C1F}, - {0x10030, 0x00064019}, - {0x10030, 0x00064413}, - {0x10030, 0x000681E1}, - {0x10030, 0x000685DB}, - {0x10030, 0x000689D5}, - {0x10030, 0x00068D9B}, - {0x10030, 0x00069195}, - {0x10030, 0x0006955B}, - {0x10030, 0x00069957}, - {0x10030, 0x00069D1F}, - {0x10030, 0x0006A119}, - {0x10030, 0x0006A4DF}, - {0x10030, 0x0006A8D9}, - {0x10030, 0x0006AC9F}, - {0x10030, 0x0006B099}, - {0x10030, 0x0006B45F}, - {0x10030, 0x0006B859}, - {0x10030, 0x0006BC1F}, - {0x10030, 0x0006C019}, - {0x10030, 0x0006C413}, - {0x10030, 0x000701E1}, - {0x10030, 0x000705DB}, - {0x10030, 0x000709D5}, - {0x10030, 0x00070D9B}, - {0x10030, 0x00071195}, - {0x10030, 0x0007155B}, - {0x10030, 0x00071957}, - {0x10030, 0x00071D1F}, - {0x10030, 0x00072119}, - {0x10030, 0x000724DF}, - {0x10030, 0x000728D9}, - {0x10030, 0x00072C9F}, - {0x10030, 0x00073099}, - {0x10030, 0x0007345F}, - {0x10030, 0x00073859}, - {0x10030, 0x00073C1F}, - {0x10030, 0x00074019}, - {0x10030, 0x00074413}, - {0x10030, 0x000781DF}, - {0x10030, 0x000785D9}, - {0x10030, 0x000789D3}, - {0x10030, 0x00078D99}, - {0x10030, 0x00079193}, - {0x10030, 0x0007955F}, - {0x10030, 0x00079959}, - {0x10030, 0x00079D21}, - {0x10030, 0x0007A115}, - {0x10030, 0x0007A4DF}, - {0x10030, 0x0007A8D9}, - {0x10030, 0x0007AC9F}, - {0x10030, 0x0007B099}, - {0x10030, 0x0007B45F}, - {0x10030, 0x0007B859}, - {0x10030, 0x0007BC1F}, - {0x10030, 0x0007C019}, - {0x10030, 0x0007C413}, - {0x10030, 0x00000000}, - {0x10030, 0x000785A9}, - {0x10030, 0x000789A3}, - {0x10030, 0x00078D9D}, - {0x10030, 0x00079197}, - {0x10030, 0x00079591}, - {0x10030, 0x00079957}, - {0x10030, 0x00079D51}, - {0x10030, 0x0007A119}, - {0x10030, 0x0007A513}, - {0x10030, 0x0007A8D9}, - {0x10030, 0x0007ACD3}, - {0x10030, 0x0007B099}, - {0x10030, 0x0007B493}, - {0x10030, 0x0007B859}, - {0x10030, 0x0007BC53}, - {0x10030, 0x0007C019}, - {0x10030, 0x0007C413}, + {0x10030, 0x000001EF}, + {0x10030, 0x000005E9}, + {0x10030, 0x000009E3}, + {0x10030, 0x00000DDD}, + {0x10030, 0x000011D7}, + {0x10030, 0x0000159F}, + {0x10030, 0x00001999}, + {0x10030, 0x00001D5F}, + {0x10030, 0x00002159}, + {0x10030, 0x0000251F}, + {0x10030, 0x00002919}, + {0x10030, 0x00002CDF}, + {0x10030, 0x000030D9}, + {0x10030, 0x0000349F}, + {0x10030, 0x00003899}, + {0x10030, 0x00003C5F}, + {0x10030, 0x00004059}, + {0x10030, 0x00004453}, + {0x10030, 0x000201A7}, + {0x10030, 0x000205A1}, + {0x10030, 0x0002099B}, + {0x10030, 0x00020D95}, + {0x10030, 0x0002115B}, + {0x10030, 0x00021555}, + {0x10030, 0x00021921}, + {0x10030, 0x00021D1B}, + {0x10030, 0x000220E3}, + {0x10030, 0x000224DD}, + {0x10030, 0x000228A3}, + {0x10030, 0x00022C9D}, + {0x10030, 0x00023063}, + {0x10030, 0x0002345D}, + {0x10030, 0x00023823}, + {0x10030, 0x00023C1D}, + {0x10030, 0x00024017}, + {0x10030, 0x00024411}, + {0x10030, 0x000281A9}, + {0x10030, 0x000285A3}, + {0x10030, 0x0002899D}, + {0x10030, 0x00028D97}, + {0x10030, 0x0002915D}, + {0x10030, 0x00029557}, + {0x10030, 0x0002991F}, + {0x10030, 0x00029D19}, + {0x10030, 0x0002A0E1}, + {0x10030, 0x0002A4DB}, + {0x10030, 0x0002A8A1}, + {0x10030, 0x0002AC9B}, + {0x10030, 0x0002B061}, + {0x10030, 0x0002B45B}, + {0x10030, 0x0002B821}, + {0x10030, 0x0002BC1B}, + {0x10030, 0x0002C015}, + {0x10030, 0x0002C40F}, + {0x10030, 0x000301A9}, + {0x10030, 0x000305A3}, + {0x10030, 0x0003099D}, + {0x10030, 0x00030D97}, + {0x10030, 0x0003115D}, + {0x10030, 0x00031557}, + {0x10030, 0x0003191F}, + {0x10030, 0x00031D19}, + {0x10030, 0x000320E1}, + {0x10030, 0x000324DB}, + {0x10030, 0x000328A1}, + {0x10030, 0x00032C9B}, + {0x10030, 0x00033061}, + {0x10030, 0x0003345B}, + {0x10030, 0x00033821}, + {0x10030, 0x00033C1B}, + {0x10030, 0x00034015}, + {0x10030, 0x0003440F}, + {0x10030, 0x000601F1}, + {0x10030, 0x000605E9}, + {0x10030, 0x000609A9}, + {0x10030, 0x00060D65}, + {0x10030, 0x0006115F}, + {0x10030, 0x00061525}, + {0x10030, 0x0006191F}, + {0x10030, 0x00061CE7}, + {0x10030, 0x000620E1}, + {0x10030, 0x000624DB}, + {0x10030, 0x000628A3}, + {0x10030, 0x00062C69}, + {0x10030, 0x00063063}, + {0x10030, 0x00063429}, + {0x10030, 0x00063823}, + {0x10030, 0x00063C1D}, + {0x10030, 0x00064013}, + {0x10030, 0x0006440D}, + {0x10030, 0x000681EF}, + {0x10030, 0x000685E7}, + {0x10030, 0x000689A7}, + {0x10030, 0x00068D61}, + {0x10030, 0x0006915B}, + {0x10030, 0x00069523}, + {0x10030, 0x0006991D}, + {0x10030, 0x00069CE5}, + {0x10030, 0x0006A0DF}, + {0x10030, 0x0006A4A7}, + {0x10030, 0x0006A8A1}, + {0x10030, 0x0006AC67}, + {0x10030, 0x0006B061}, + {0x10030, 0x0006B429}, + {0x10030, 0x0006B823}, + {0x10030, 0x0006BC1D}, + {0x10030, 0x0006C017}, + {0x10030, 0x0006C40D}, + {0x10030, 0x000701F1}, + {0x10030, 0x000705E9}, + {0x10030, 0x000709A9}, + {0x10030, 0x00070D63}, + {0x10030, 0x0007115D}, + {0x10030, 0x00071523}, + {0x10030, 0x0007191D}, + {0x10030, 0x00071D17}, + {0x10030, 0x000720DF}, + {0x10030, 0x000724D9}, + {0x10030, 0x000728D3}, + {0x10030, 0x00072C67}, + {0x10030, 0x00073061}, + {0x10030, 0x00073427}, + {0x10030, 0x00073821}, + {0x10030, 0x00073C1B}, + {0x10030, 0x00074015}, + {0x10030, 0x0007440D}, + {0x10030, 0x000781F1}, + {0x10030, 0x000785EB}, + {0x10030, 0x000789E5}, + {0x10030, 0x00078DA3}, + {0x10030, 0x00079161}, + {0x10030, 0x0007955B}, + {0x10030, 0x00079923}, + {0x10030, 0x00079D1D}, + {0x10030, 0x0007A117}, + {0x10030, 0x0007A4DD}, + {0x10030, 0x0007A8D7}, + {0x10030, 0x0007AC9D}, + {0x10030, 0x0007B063}, + {0x10030, 0x0007B45D}, + {0x10030, 0x0007B857}, + {0x10030, 0x0007BC1D}, + {0x10030, 0x0007C017}, + {0x10030, 0x0007C40F}, {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, - {0x10030, 0x000201DF}, - {0x10030, 0x000205D9}, - {0x10030, 0x000209D3}, - {0x10030, 0x00020D99}, - {0x10030, 0x00021193}, - {0x10030, 0x0002155F}, - {0x10030, 0x00021959}, - {0x10030, 0x00021D21}, - {0x10030, 0x00022119}, - {0x10030, 0x000224DF}, - {0x10030, 0x000228D9}, - {0x10030, 0x00022C9F}, - {0x10030, 0x00023099}, - {0x10030, 0x0002345F}, - {0x10030, 0x00023859}, - {0x10030, 0x00023C1F}, - {0x10030, 0x00024019}, - {0x10030, 0x00024413}, - {0x10030, 0x000281CD}, - {0x10030, 0x000285DB}, - {0x10030, 0x000289D5}, - {0x10030, 0x00028D9B}, - {0x10030, 0x0002918D}, - {0x10030, 0x00029555}, - {0x10030, 0x00029957}, - {0x10030, 0x00029D1F}, - {0x10030, 0x0002A119}, - {0x10030, 0x0002A4DF}, - {0x10030, 0x0002A8D9}, - {0x10030, 0x0002AC9F}, - {0x10030, 0x0002B099}, - {0x10030, 0x0002B45F}, - {0x10030, 0x0002B859}, - {0x10030, 0x0002BC1F}, - {0x10030, 0x0002C019}, - {0x10030, 0x0002C413}, - {0x10030, 0x000301D9}, - {0x10030, 0x000305DB}, - {0x10030, 0x000309D5}, - {0x10030, 0x00030D9B}, - {0x10030, 0x00031195}, - {0x10030, 0x0003155D}, - {0x10030, 0x00031955}, - {0x10030, 0x00031D1D}, - {0x10030, 0x00032119}, - {0x10030, 0x000324DF}, - {0x10030, 0x000328D9}, - {0x10030, 0x00032C9F}, - {0x10030, 0x00033099}, - {0x10030, 0x0003345F}, - {0x10030, 0x00033859}, - {0x10030, 0x00033C1F}, - {0x10030, 0x00034019}, - {0x10030, 0x00034413}, - {0x10030, 0x000601E1}, - {0x10030, 0x000605DB}, - {0x10030, 0x000609D5}, - {0x10030, 0x00060D9B}, - {0x10030, 0x00061195}, - {0x10030, 0x0006155B}, - {0x10030, 0x00061957}, - {0x10030, 0x00061D1F}, - {0x10030, 0x00062119}, - {0x10030, 0x000624DF}, - {0x10030, 0x000628D9}, - {0x10030, 0x00062C9F}, - {0x10030, 0x00063099}, - {0x10030, 0x0006345F}, - {0x10030, 0x00063859}, - {0x10030, 0x00063C1F}, - {0x10030, 0x00064019}, - {0x10030, 0x00064413}, - {0x10030, 0x000681E1}, - {0x10030, 0x000685DB}, - {0x10030, 0x000689D5}, - {0x10030, 0x00068D9B}, - {0x10030, 0x00069195}, - {0x10030, 0x0006955B}, - {0x10030, 0x00069957}, - {0x10030, 0x00069D1F}, - {0x10030, 0x0006A119}, - {0x10030, 0x0006A4DF}, - {0x10030, 0x0006A8D9}, - {0x10030, 0x0006AC9F}, - {0x10030, 0x0006B099}, - {0x10030, 0x0006B45F}, - {0x10030, 0x0006B859}, - {0x10030, 0x0006BC1F}, - {0x10030, 0x0006C019}, - {0x10030, 0x0006C413}, - {0x10030, 0x000701E1}, - {0x10030, 0x000705DB}, - {0x10030, 0x000709D5}, - {0x10030, 0x00070D9B}, - {0x10030, 0x00071195}, - {0x10030, 0x0007155B}, - {0x10030, 0x00071957}, - {0x10030, 0x00071D1F}, - {0x10030, 0x00072119}, - {0x10030, 0x000724DF}, - {0x10030, 0x000728D9}, - {0x10030, 0x00072C9F}, - {0x10030, 0x00073099}, - {0x10030, 0x0007345F}, - {0x10030, 0x00073859}, - {0x10030, 0x00073C1F}, - {0x10030, 0x00074019}, - {0x10030, 0x00074413}, - {0x10030, 0x000781DF}, - {0x10030, 0x000785D9}, - {0x10030, 0x000789D3}, - {0x10030, 0x00078D99}, - {0x10030, 0x00079193}, - {0x10030, 0x0007955F}, - {0x10030, 0x00079959}, - {0x10030, 0x00079D21}, - {0x10030, 0x0007A115}, - {0x10030, 0x0007A4DF}, - {0x10030, 0x0007A8D9}, - {0x10030, 0x0007AC9F}, - {0x10030, 0x0007B099}, - {0x10030, 0x0007B45F}, - {0x10030, 0x0007B859}, - {0x10030, 0x0007BC1F}, - {0x10030, 0x0007C019}, - {0x10030, 0x0007C413}, - {0x10030, 0x00000000}, - {0x10030, 0x000785A9}, - {0x10030, 0x000789A3}, - {0x10030, 0x00078D9D}, - {0x10030, 0x00079197}, - {0x10030, 0x00079591}, - {0x10030, 0x00079957}, - {0x10030, 0x00079D51}, - {0x10030, 0x0007A119}, - {0x10030, 0x0007A513}, - {0x10030, 0x0007A8D9}, - {0x10030, 0x0007ACD3}, - {0x10030, 0x0007B099}, - {0x10030, 0x0007B493}, - {0x10030, 0x0007B859}, - {0x10030, 0x0007BC53}, - {0x10030, 0x0007C019}, - {0x10030, 0x0007C413}, + {0x10030, 0x000001EF}, + {0x10030, 0x000005E9}, + {0x10030, 0x000009E3}, + {0x10030, 0x00000DDD}, + {0x10030, 0x000011D7}, + {0x10030, 0x0000159F}, + {0x10030, 0x00001999}, + {0x10030, 0x00001D5F}, + {0x10030, 0x00002159}, + {0x10030, 0x0000251F}, + {0x10030, 0x00002919}, + {0x10030, 0x00002CDF}, + {0x10030, 0x000030D9}, + {0x10030, 0x0000349F}, + {0x10030, 0x00003899}, + {0x10030, 0x00003C5F}, + {0x10030, 0x00004059}, + {0x10030, 0x00004453}, + {0x10030, 0x000201A7}, + {0x10030, 0x000205A1}, + {0x10030, 0x0002099B}, + {0x10030, 0x00020D95}, + {0x10030, 0x0002115B}, + {0x10030, 0x00021555}, + {0x10030, 0x00021921}, + {0x10030, 0x00021D1B}, + {0x10030, 0x000220E3}, + {0x10030, 0x000224DD}, + {0x10030, 0x000228A3}, + {0x10030, 0x00022C9D}, + {0x10030, 0x00023063}, + {0x10030, 0x0002345D}, + {0x10030, 0x00023823}, + {0x10030, 0x00023C1D}, + {0x10030, 0x00024017}, + {0x10030, 0x00024411}, + {0x10030, 0x000281A9}, + {0x10030, 0x000285A3}, + {0x10030, 0x0002899D}, + {0x10030, 0x00028D97}, + {0x10030, 0x0002915D}, + {0x10030, 0x00029557}, + {0x10030, 0x0002991F}, + {0x10030, 0x00029D19}, + {0x10030, 0x0002A0E1}, + {0x10030, 0x0002A4DB}, + {0x10030, 0x0002A8A1}, + {0x10030, 0x0002AC9B}, + {0x10030, 0x0002B061}, + {0x10030, 0x0002B45B}, + {0x10030, 0x0002B821}, + {0x10030, 0x0002BC1B}, + {0x10030, 0x0002C015}, + {0x10030, 0x0002C40F}, + {0x10030, 0x000301A9}, + {0x10030, 0x000305A3}, + {0x10030, 0x0003099D}, + {0x10030, 0x00030D97}, + {0x10030, 0x0003115D}, + {0x10030, 0x00031557}, + {0x10030, 0x0003191F}, + {0x10030, 0x00031D19}, + {0x10030, 0x000320E1}, + {0x10030, 0x000324DB}, + {0x10030, 0x000328A1}, + {0x10030, 0x00032C9B}, + {0x10030, 0x00033061}, + {0x10030, 0x0003345B}, + {0x10030, 0x00033821}, + {0x10030, 0x00033C1B}, + {0x10030, 0x00034015}, + {0x10030, 0x0003440F}, + {0x10030, 0x000601F1}, + {0x10030, 0x000605E9}, + {0x10030, 0x000609A9}, + {0x10030, 0x00060D65}, + {0x10030, 0x0006115F}, + {0x10030, 0x00061525}, + {0x10030, 0x0006191F}, + {0x10030, 0x00061CE7}, + {0x10030, 0x000620E1}, + {0x10030, 0x000624DB}, + {0x10030, 0x000628A3}, + {0x10030, 0x00062C69}, + {0x10030, 0x00063063}, + {0x10030, 0x00063429}, + {0x10030, 0x00063823}, + {0x10030, 0x00063C1D}, + {0x10030, 0x00064013}, + {0x10030, 0x0006440D}, + {0x10030, 0x000681EF}, + {0x10030, 0x000685E7}, + {0x10030, 0x000689A7}, + {0x10030, 0x00068D61}, + {0x10030, 0x0006915B}, + {0x10030, 0x00069523}, + {0x10030, 0x0006991D}, + {0x10030, 0x00069CE5}, + {0x10030, 0x0006A0DF}, + {0x10030, 0x0006A4A7}, + {0x10030, 0x0006A8A1}, + {0x10030, 0x0006AC67}, + {0x10030, 0x0006B061}, + {0x10030, 0x0006B429}, + {0x10030, 0x0006B823}, + {0x10030, 0x0006BC1D}, + {0x10030, 0x0006C017}, + {0x10030, 0x0006C40D}, + {0x10030, 0x000701F1}, + {0x10030, 0x000705E9}, + {0x10030, 0x000709A9}, + {0x10030, 0x00070D63}, + {0x10030, 0x0007115D}, + {0x10030, 0x00071523}, + {0x10030, 0x0007191D}, + {0x10030, 0x00071D17}, + {0x10030, 0x000720DF}, + {0x10030, 0x000724D9}, + {0x10030, 0x000728D3}, + {0x10030, 0x00072C67}, + {0x10030, 0x00073061}, + {0x10030, 0x00073427}, + {0x10030, 0x00073821}, + {0x10030, 0x00073C1B}, + {0x10030, 0x00074015}, + {0x10030, 0x0007440D}, + {0x10030, 0x000781F1}, + {0x10030, 0x000785EB}, + {0x10030, 0x000789E5}, + {0x10030, 0x00078DA3}, + {0x10030, 0x00079161}, + {0x10030, 0x0007955B}, + {0x10030, 0x00079923}, + {0x10030, 0x00079D1D}, + {0x10030, 0x0007A117}, + {0x10030, 0x0007A4DD}, + {0x10030, 0x0007A8D7}, + {0x10030, 0x0007AC9D}, + {0x10030, 0x0007B063}, + {0x10030, 0x0007B45D}, + {0x10030, 0x0007B857}, + {0x10030, 0x0007BC1D}, + {0x10030, 0x0007C017}, + {0x10030, 0x0007C40F}, {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, - {0x10030, 0x000201DF}, - {0x10030, 0x000205D9}, - {0x10030, 0x000209D3}, - {0x10030, 0x00020D99}, - {0x10030, 0x00021193}, - {0x10030, 0x0002155F}, - {0x10030, 0x00021959}, - {0x10030, 0x00021D21}, - {0x10030, 0x00022119}, - {0x10030, 0x000224DF}, - {0x10030, 0x000228D9}, - {0x10030, 0x00022C9F}, - {0x10030, 0x00023099}, - {0x10030, 0x0002345F}, - {0x10030, 0x00023859}, - {0x10030, 0x00023C1F}, - {0x10030, 0x00024019}, - {0x10030, 0x00024413}, - {0x10030, 0x000281CD}, - {0x10030, 0x000285DB}, - {0x10030, 0x000289D5}, - {0x10030, 0x00028D9B}, - {0x10030, 0x0002918D}, - {0x10030, 0x00029555}, - {0x10030, 0x00029957}, - {0x10030, 0x00029D1F}, - {0x10030, 0x0002A119}, - {0x10030, 0x0002A4DF}, - {0x10030, 0x0002A8D9}, - {0x10030, 0x0002AC9F}, - {0x10030, 0x0002B099}, - {0x10030, 0x0002B45F}, - {0x10030, 0x0002B859}, - {0x10030, 0x0002BC1F}, - {0x10030, 0x0002C019}, - {0x10030, 0x0002C413}, - {0x10030, 0x000301D9}, - {0x10030, 0x000305DB}, - {0x10030, 0x000309D5}, - {0x10030, 0x00030D9B}, - {0x10030, 0x00031195}, - {0x10030, 0x0003155D}, - {0x10030, 0x00031955}, - {0x10030, 0x00031D1D}, - {0x10030, 0x00032119}, - {0x10030, 0x000324DF}, - {0x10030, 0x000328D9}, - {0x10030, 0x00032C9F}, - {0x10030, 0x00033099}, - {0x10030, 0x0003345F}, - {0x10030, 0x00033859}, - {0x10030, 0x00033C1F}, - {0x10030, 0x00034019}, - {0x10030, 0x00034413}, - {0x10030, 0x000601E1}, - {0x10030, 0x000605DB}, - {0x10030, 0x000609D5}, - {0x10030, 0x00060D9B}, - {0x10030, 0x00061195}, - {0x10030, 0x0006155B}, - {0x10030, 0x00061957}, - {0x10030, 0x00061D1F}, - {0x10030, 0x00062119}, - {0x10030, 0x000624DF}, - {0x10030, 0x000628D9}, - {0x10030, 0x00062C9F}, - {0x10030, 0x00063099}, - {0x10030, 0x0006345F}, - {0x10030, 0x00063859}, - {0x10030, 0x00063C1F}, - {0x10030, 0x00064019}, - {0x10030, 0x00064413}, - {0x10030, 0x000681E1}, - {0x10030, 0x000685DB}, - {0x10030, 0x000689D5}, - {0x10030, 0x00068D9B}, - {0x10030, 0x00069195}, - {0x10030, 0x0006955B}, - {0x10030, 0x00069957}, - {0x10030, 0x00069D1F}, - {0x10030, 0x0006A119}, - {0x10030, 0x0006A4DF}, - {0x10030, 0x0006A8D9}, - {0x10030, 0x0006AC9F}, - {0x10030, 0x0006B099}, - {0x10030, 0x0006B45F}, - {0x10030, 0x0006B859}, - {0x10030, 0x0006BC1F}, - {0x10030, 0x0006C019}, - {0x10030, 0x0006C413}, - {0x10030, 0x000701E1}, - {0x10030, 0x000705DB}, - {0x10030, 0x000709D5}, - {0x10030, 0x00070D9B}, - {0x10030, 0x00071195}, - {0x10030, 0x0007155B}, - {0x10030, 0x00071957}, - {0x10030, 0x00071D1F}, - {0x10030, 0x00072119}, - {0x10030, 0x000724DF}, - {0x10030, 0x000728D9}, - {0x10030, 0x00072C9F}, - {0x10030, 0x00073099}, - {0x10030, 0x0007345F}, - {0x10030, 0x00073859}, - {0x10030, 0x00073C1F}, - {0x10030, 0x00074019}, - {0x10030, 0x00074413}, - {0x10030, 0x000781DF}, - {0x10030, 0x000785D9}, - {0x10030, 0x000789D3}, - {0x10030, 0x00078D99}, - {0x10030, 0x00079193}, - {0x10030, 0x0007955F}, - {0x10030, 0x00079959}, - {0x10030, 0x00079D21}, - {0x10030, 0x0007A115}, - {0x10030, 0x0007A4DF}, - {0x10030, 0x0007A8D9}, - {0x10030, 0x0007AC9F}, - {0x10030, 0x0007B099}, - {0x10030, 0x0007B45F}, - {0x10030, 0x0007B859}, - {0x10030, 0x0007BC1F}, - {0x10030, 0x0007C019}, - {0x10030, 0x0007C413}, - {0x10030, 0x00000000}, - {0x10030, 0x000785A9}, - {0x10030, 0x000789A3}, - {0x10030, 0x00078D9D}, - {0x10030, 0x00079197}, - {0x10030, 0x00079591}, - {0x10030, 0x00079957}, - {0x10030, 0x00079D51}, - {0x10030, 0x0007A119}, - {0x10030, 0x0007A513}, - {0x10030, 0x0007A8D9}, - {0x10030, 0x0007ACD3}, - {0x10030, 0x0007B099}, - {0x10030, 0x0007B493}, - {0x10030, 0x0007B859}, - {0x10030, 0x0007BC53}, - {0x10030, 0x0007C019}, - {0x10030, 0x0007C413}, + {0x10030, 0x000001EF}, + {0x10030, 0x000005E9}, + {0x10030, 0x000009E3}, + {0x10030, 0x00000DDD}, + {0x10030, 0x000011D7}, + {0x10030, 0x0000159F}, + {0x10030, 0x00001999}, + {0x10030, 0x00001D5F}, + {0x10030, 0x00002159}, + {0x10030, 0x0000251F}, + {0x10030, 0x00002919}, + {0x10030, 0x00002CDF}, + {0x10030, 0x000030D9}, + {0x10030, 0x0000349F}, + {0x10030, 0x00003899}, + {0x10030, 0x00003C5F}, + {0x10030, 0x00004059}, + {0x10030, 0x00004453}, + {0x10030, 0x000201A7}, + {0x10030, 0x000205A1}, + {0x10030, 0x0002099B}, + {0x10030, 0x00020D95}, + {0x10030, 0x0002115B}, + {0x10030, 0x00021555}, + {0x10030, 0x00021921}, + {0x10030, 0x00021D1B}, + {0x10030, 0x000220E3}, + {0x10030, 0x000224DD}, + {0x10030, 0x000228A3}, + {0x10030, 0x00022C9D}, + {0x10030, 0x00023063}, + {0x10030, 0x0002345D}, + {0x10030, 0x00023823}, + {0x10030, 0x00023C1D}, + {0x10030, 0x00024017}, + {0x10030, 0x00024411}, + {0x10030, 0x000281A9}, + {0x10030, 0x000285A3}, + {0x10030, 0x0002899D}, + {0x10030, 0x00028D97}, + {0x10030, 0x0002915D}, + {0x10030, 0x00029557}, + {0x10030, 0x0002991F}, + {0x10030, 0x00029D19}, + {0x10030, 0x0002A0E1}, + {0x10030, 0x0002A4DB}, + {0x10030, 0x0002A8A1}, + {0x10030, 0x0002AC9B}, + {0x10030, 0x0002B061}, + {0x10030, 0x0002B45B}, + {0x10030, 0x0002B821}, + {0x10030, 0x0002BC1B}, + {0x10030, 0x0002C015}, + {0x10030, 0x0002C40F}, + {0x10030, 0x000301A9}, + {0x10030, 0x000305A3}, + {0x10030, 0x0003099D}, + {0x10030, 0x00030D97}, + {0x10030, 0x0003115D}, + {0x10030, 0x00031557}, + {0x10030, 0x0003191F}, + {0x10030, 0x00031D19}, + {0x10030, 0x000320E1}, + {0x10030, 0x000324DB}, + {0x10030, 0x000328A1}, + {0x10030, 0x00032C9B}, + {0x10030, 0x00033061}, + {0x10030, 0x0003345B}, + {0x10030, 0x00033821}, + {0x10030, 0x00033C1B}, + {0x10030, 0x00034015}, + {0x10030, 0x0003440F}, + {0x10030, 0x000601F1}, + {0x10030, 0x000605E9}, + {0x10030, 0x000609A9}, + {0x10030, 0x00060D65}, + {0x10030, 0x0006115F}, + {0x10030, 0x00061525}, + {0x10030, 0x0006191F}, + {0x10030, 0x00061CE7}, + {0x10030, 0x000620E1}, + {0x10030, 0x000624DB}, + {0x10030, 0x000628A3}, + {0x10030, 0x00062C69}, + {0x10030, 0x00063063}, + {0x10030, 0x00063429}, + {0x10030, 0x00063823}, + {0x10030, 0x00063C1D}, + {0x10030, 0x00064013}, + {0x10030, 0x0006440D}, + {0x10030, 0x000681EF}, + {0x10030, 0x000685E7}, + {0x10030, 0x000689A7}, + {0x10030, 0x00068D61}, + {0x10030, 0x0006915B}, + {0x10030, 0x00069523}, + {0x10030, 0x0006991D}, + {0x10030, 0x00069CE5}, + {0x10030, 0x0006A0DF}, + {0x10030, 0x0006A4A7}, + {0x10030, 0x0006A8A1}, + {0x10030, 0x0006AC67}, + {0x10030, 0x0006B061}, + {0x10030, 0x0006B429}, + {0x10030, 0x0006B823}, + {0x10030, 0x0006BC1D}, + {0x10030, 0x0006C017}, + {0x10030, 0x0006C40D}, + {0x10030, 0x000701F1}, + {0x10030, 0x000705E9}, + {0x10030, 0x000709A9}, + {0x10030, 0x00070D63}, + {0x10030, 0x0007115D}, + {0x10030, 0x00071523}, + {0x10030, 0x0007191D}, + {0x10030, 0x00071D17}, + {0x10030, 0x000720DF}, + {0x10030, 0x000724D9}, + {0x10030, 0x000728D3}, + {0x10030, 0x00072C67}, + {0x10030, 0x00073061}, + {0x10030, 0x00073427}, + {0x10030, 0x00073821}, + {0x10030, 0x00073C1B}, + {0x10030, 0x00074015}, + {0x10030, 0x0007440D}, + {0x10030, 0x000781F1}, + {0x10030, 0x000785EB}, + {0x10030, 0x000789E5}, + {0x10030, 0x00078DA3}, + {0x10030, 0x00079161}, + {0x10030, 0x0007955B}, + {0x10030, 0x00079923}, + {0x10030, 0x00079D1D}, + {0x10030, 0x0007A117}, + {0x10030, 0x0007A4DD}, + {0x10030, 0x0007A8D7}, + {0x10030, 0x0007AC9D}, + {0x10030, 0x0007B063}, + {0x10030, 0x0007B45D}, + {0x10030, 0x0007B857}, + {0x10030, 0x0007BC1D}, + {0x10030, 0x0007C017}, + {0x10030, 0x0007C40F}, {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, - {0x10030, 0x000201DF}, - {0x10030, 0x000205D9}, - {0x10030, 0x000209D3}, - {0x10030, 0x00020D99}, - {0x10030, 0x00021193}, - {0x10030, 0x0002155F}, - {0x10030, 0x00021959}, - {0x10030, 0x00021D21}, - {0x10030, 0x00022119}, - {0x10030, 0x000224DF}, - {0x10030, 0x000228D9}, - {0x10030, 0x00022C9F}, - {0x10030, 0x00023099}, - {0x10030, 0x0002345F}, - {0x10030, 0x00023859}, - {0x10030, 0x00023C1F}, - {0x10030, 0x00024019}, - {0x10030, 0x00024413}, - {0x10030, 0x000281CD}, - {0x10030, 0x000285DB}, - {0x10030, 0x000289D5}, - {0x10030, 0x00028D9B}, - {0x10030, 0x0002918D}, - {0x10030, 0x00029555}, - {0x10030, 0x00029957}, - {0x10030, 0x00029D1F}, - {0x10030, 0x0002A119}, - {0x10030, 0x0002A4DF}, - {0x10030, 0x0002A8D9}, - {0x10030, 0x0002AC9F}, - {0x10030, 0x0002B099}, - {0x10030, 0x0002B45F}, - {0x10030, 0x0002B859}, - {0x10030, 0x0002BC1F}, - {0x10030, 0x0002C019}, - {0x10030, 0x0002C413}, - {0x10030, 0x000301D9}, - {0x10030, 0x000305DB}, - {0x10030, 0x000309D5}, - {0x10030, 0x00030D9B}, - {0x10030, 0x00031195}, - {0x10030, 0x0003155D}, - {0x10030, 0x00031955}, - {0x10030, 0x00031D1D}, - {0x10030, 0x00032119}, - {0x10030, 0x000324DF}, - {0x10030, 0x000328D9}, - {0x10030, 0x00032C9F}, - {0x10030, 0x00033099}, - {0x10030, 0x0003345F}, - {0x10030, 0x00033859}, - {0x10030, 0x00033C1F}, - {0x10030, 0x00034019}, - {0x10030, 0x00034413}, - {0x10030, 0x000601E1}, - {0x10030, 0x000605DB}, - {0x10030, 0x000609D5}, - {0x10030, 0x00060D9B}, - {0x10030, 0x00061195}, - {0x10030, 0x0006155B}, - {0x10030, 0x00061957}, - {0x10030, 0x00061D1F}, - {0x10030, 0x00062119}, - {0x10030, 0x000624DF}, - {0x10030, 0x000628D9}, - {0x10030, 0x00062C9F}, - {0x10030, 0x00063099}, - {0x10030, 0x0006345F}, - {0x10030, 0x00063859}, - {0x10030, 0x00063C1F}, - {0x10030, 0x00064019}, - {0x10030, 0x00064413}, - {0x10030, 0x000681E1}, - {0x10030, 0x000685DB}, - {0x10030, 0x000689D5}, - {0x10030, 0x00068D9B}, - {0x10030, 0x00069195}, - {0x10030, 0x0006955B}, - {0x10030, 0x00069957}, - {0x10030, 0x00069D1F}, - {0x10030, 0x0006A119}, - {0x10030, 0x0006A4DF}, - {0x10030, 0x0006A8D9}, - {0x10030, 0x0006AC9F}, - {0x10030, 0x0006B099}, - {0x10030, 0x0006B45F}, - {0x10030, 0x0006B859}, - {0x10030, 0x0006BC1F}, - {0x10030, 0x0006C019}, - {0x10030, 0x0006C413}, - {0x10030, 0x000701E1}, - {0x10030, 0x000705DB}, - {0x10030, 0x000709D5}, - {0x10030, 0x00070D9B}, - {0x10030, 0x00071195}, - {0x10030, 0x0007155B}, - {0x10030, 0x00071957}, - {0x10030, 0x00071D1F}, - {0x10030, 0x00072119}, - {0x10030, 0x000724DF}, - {0x10030, 0x000728D9}, - {0x10030, 0x00072C9F}, - {0x10030, 0x00073099}, - {0x10030, 0x0007345F}, - {0x10030, 0x00073859}, - {0x10030, 0x00073C1F}, - {0x10030, 0x00074019}, - {0x10030, 0x00074413}, - {0x10030, 0x000781DF}, - {0x10030, 0x000785D9}, - {0x10030, 0x000789D3}, - {0x10030, 0x00078D99}, - {0x10030, 0x00079193}, - {0x10030, 0x0007955F}, - {0x10030, 0x00079959}, - {0x10030, 0x00079D21}, - {0x10030, 0x0007A115}, - {0x10030, 0x0007A4DF}, - {0x10030, 0x0007A8D9}, - {0x10030, 0x0007AC9F}, - {0x10030, 0x0007B099}, - {0x10030, 0x0007B45F}, - {0x10030, 0x0007B859}, - {0x10030, 0x0007BC1F}, - {0x10030, 0x0007C019}, - {0x10030, 0x0007C413}, - {0x10030, 0x00000000}, - {0x10030, 0x000785A9}, - {0x10030, 0x000789A3}, - {0x10030, 0x00078D9D}, - {0x10030, 0x00079197}, - {0x10030, 0x00079591}, - {0x10030, 0x00079957}, - {0x10030, 0x00079D51}, - {0x10030, 0x0007A119}, - {0x10030, 0x0007A513}, - {0x10030, 0x0007A8D9}, - {0x10030, 0x0007ACD3}, - {0x10030, 0x0007B099}, - {0x10030, 0x0007B493}, - {0x10030, 0x0007B859}, - {0x10030, 0x0007BC53}, - {0x10030, 0x0007C019}, - {0x10030, 0x0007C413}, + {0x10030, 0x000001EF}, + {0x10030, 0x000005E9}, + {0x10030, 0x000009E3}, + {0x10030, 0x00000DDD}, + {0x10030, 0x000011D7}, + {0x10030, 0x0000159F}, + {0x10030, 0x00001999}, + {0x10030, 0x00001D5F}, + {0x10030, 0x00002159}, + {0x10030, 0x0000251F}, + {0x10030, 0x00002919}, + {0x10030, 0x00002CDF}, + {0x10030, 0x000030D9}, + {0x10030, 0x0000349F}, + {0x10030, 0x00003899}, + {0x10030, 0x00003C5F}, + {0x10030, 0x00004059}, + {0x10030, 0x00004453}, + {0x10030, 0x000201A7}, + {0x10030, 0x000205A1}, + {0x10030, 0x0002099B}, + {0x10030, 0x00020D95}, + {0x10030, 0x0002115B}, + {0x10030, 0x00021555}, + {0x10030, 0x00021921}, + {0x10030, 0x00021D1B}, + {0x10030, 0x000220E3}, + {0x10030, 0x000224DD}, + {0x10030, 0x000228A3}, + {0x10030, 0x00022C9D}, + {0x10030, 0x00023063}, + {0x10030, 0x0002345D}, + {0x10030, 0x00023823}, + {0x10030, 0x00023C1D}, + {0x10030, 0x00024017}, + {0x10030, 0x00024411}, + {0x10030, 0x000281A9}, + {0x10030, 0x000285A3}, + {0x10030, 0x0002899D}, + {0x10030, 0x00028D97}, + {0x10030, 0x0002915D}, + {0x10030, 0x00029557}, + {0x10030, 0x0002991F}, + {0x10030, 0x00029D19}, + {0x10030, 0x0002A0E1}, + {0x10030, 0x0002A4DB}, + {0x10030, 0x0002A8A1}, + {0x10030, 0x0002AC9B}, + {0x10030, 0x0002B061}, + {0x10030, 0x0002B45B}, + {0x10030, 0x0002B821}, + {0x10030, 0x0002BC1B}, + {0x10030, 0x0002C015}, + {0x10030, 0x0002C40F}, + {0x10030, 0x000301A9}, + {0x10030, 0x000305A3}, + {0x10030, 0x0003099D}, + {0x10030, 0x00030D97}, + {0x10030, 0x0003115D}, + {0x10030, 0x00031557}, + {0x10030, 0x0003191F}, + {0x10030, 0x00031D19}, + {0x10030, 0x000320E1}, + {0x10030, 0x000324DB}, + {0x10030, 0x000328A1}, + {0x10030, 0x00032C9B}, + {0x10030, 0x00033061}, + {0x10030, 0x0003345B}, + {0x10030, 0x00033821}, + {0x10030, 0x00033C1B}, + {0x10030, 0x00034015}, + {0x10030, 0x0003440F}, + {0x10030, 0x000601F1}, + {0x10030, 0x000605E9}, + {0x10030, 0x000609A9}, + {0x10030, 0x00060D65}, + {0x10030, 0x0006115F}, + {0x10030, 0x00061525}, + {0x10030, 0x0006191F}, + {0x10030, 0x00061CE7}, + {0x10030, 0x000620E1}, + {0x10030, 0x000624DB}, + {0x10030, 0x000628A3}, + {0x10030, 0x00062C69}, + {0x10030, 0x00063063}, + {0x10030, 0x00063429}, + {0x10030, 0x00063823}, + {0x10030, 0x00063C1D}, + {0x10030, 0x00064013}, + {0x10030, 0x0006440D}, + {0x10030, 0x000681EF}, + {0x10030, 0x000685E7}, + {0x10030, 0x000689A7}, + {0x10030, 0x00068D61}, + {0x10030, 0x0006915B}, + {0x10030, 0x00069523}, + {0x10030, 0x0006991D}, + {0x10030, 0x00069CE5}, + {0x10030, 0x0006A0DF}, + {0x10030, 0x0006A4A7}, + {0x10030, 0x0006A8A1}, + {0x10030, 0x0006AC67}, + {0x10030, 0x0006B061}, + {0x10030, 0x0006B429}, + {0x10030, 0x0006B823}, + {0x10030, 0x0006BC1D}, + {0x10030, 0x0006C017}, + {0x10030, 0x0006C40D}, + {0x10030, 0x000701F1}, + {0x10030, 0x000705E9}, + {0x10030, 0x000709A9}, + {0x10030, 0x00070D63}, + {0x10030, 0x0007115D}, + {0x10030, 0x00071523}, + {0x10030, 0x0007191D}, + {0x10030, 0x00071D17}, + {0x10030, 0x000720DF}, + {0x10030, 0x000724D9}, + {0x10030, 0x000728D3}, + {0x10030, 0x00072C67}, + {0x10030, 0x00073061}, + {0x10030, 0x00073427}, + {0x10030, 0x00073821}, + {0x10030, 0x00073C1B}, + {0x10030, 0x00074015}, + {0x10030, 0x0007440D}, + {0x10030, 0x000781F1}, + {0x10030, 0x000785EB}, + {0x10030, 0x000789E5}, + {0x10030, 0x00078DA3}, + {0x10030, 0x00079161}, + {0x10030, 0x0007955B}, + {0x10030, 0x00079923}, + {0x10030, 0x00079D1D}, + {0x10030, 0x0007A117}, + {0x10030, 0x0007A4DD}, + {0x10030, 0x0007A8D7}, + {0x10030, 0x0007AC9D}, + {0x10030, 0x0007B063}, + {0x10030, 0x0007B45D}, + {0x10030, 0x0007B857}, + {0x10030, 0x0007BC1D}, + {0x10030, 0x0007C017}, + {0x10030, 0x0007C40F}, {0xA0000000, 0x00000000}, {0x10030, 0x000001EF}, {0x10030, 0x000005E9}, @@ -11281,6 +21336,1150 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radiob_regs[] = { {0x10030, 0x00004017}, {0x100EE, 0x00000000}, {0x100EE, 0x00002000}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x000200FA}, + {0x10030, 0x000204F7}, + {0x10030, 0x000208F4}, + {0x10030, 0x00020CF1}, + {0x10030, 0x000210EE}, + {0x10030, 0x000214EB}, + {0x10030, 0x000218E8}, + {0x10030, 0x00021CE5}, + {0x10030, 0x000220E2}, + {0x10030, 0x000224DF}, + {0x10030, 0x000228DC}, + {0x10030, 0x00022CD9}, + {0x10030, 0x000230D6}, + {0x10030, 0x000234D3}, + {0x10030, 0x000238D0}, + {0x10030, 0x00023C0D}, + {0x10030, 0x0002400A}, + {0x10030, 0x000280F9}, + {0x10030, 0x000284F6}, + {0x10030, 0x000288F3}, + {0x10030, 0x00028CF0}, + {0x10030, 0x000290ED}, + {0x10030, 0x000294EA}, + {0x10030, 0x000298E7}, + {0x10030, 0x00029CE4}, + {0x10030, 0x0002A0E1}, + {0x10030, 0x0002A4DE}, + {0x10030, 0x0002A8DB}, + {0x10030, 0x0002ACD8}, + {0x10030, 0x0002B0D5}, + {0x10030, 0x0002B4D2}, + {0x10030, 0x0002B8CF}, + {0x10030, 0x0002BC0C}, + {0x10030, 0x0002C009}, + {0x10030, 0x000300F6}, + {0x10030, 0x000304F3}, + {0x10030, 0x000308F0}, + {0x10030, 0x00030CED}, + {0x10030, 0x000310EA}, + {0x10030, 0x000314E7}, + {0x10030, 0x000318E4}, + {0x10030, 0x00031CE1}, + {0x10030, 0x000320DE}, + {0x10030, 0x000324DB}, + {0x10030, 0x000328D8}, + {0x10030, 0x00032CD5}, + {0x10030, 0x000330D2}, + {0x10030, 0x000334CF}, + {0x10030, 0x000338CC}, + {0x10030, 0x00033C09}, + {0x10030, 0x00034006}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x000200FA}, + {0x10030, 0x000204F7}, + {0x10030, 0x000208F4}, + {0x10030, 0x00020CF1}, + {0x10030, 0x000210EE}, + {0x10030, 0x000214EB}, + {0x10030, 0x000218E8}, + {0x10030, 0x00021CE5}, + {0x10030, 0x000220E2}, + {0x10030, 0x000224DF}, + {0x10030, 0x000228DC}, + {0x10030, 0x00022CD9}, + {0x10030, 0x000230D6}, + {0x10030, 0x000234D3}, + {0x10030, 0x000238D0}, + {0x10030, 0x00023C0D}, + {0x10030, 0x0002400A}, + {0x10030, 0x000280F9}, + {0x10030, 0x000284F6}, + {0x10030, 0x000288F3}, + {0x10030, 0x00028CF0}, + {0x10030, 0x000290ED}, + {0x10030, 0x000294EA}, + {0x10030, 0x000298E7}, + {0x10030, 0x00029CE4}, + {0x10030, 0x0002A0E1}, + {0x10030, 0x0002A4DE}, + {0x10030, 0x0002A8DB}, + {0x10030, 0x0002ACD8}, + {0x10030, 0x0002B0D5}, + {0x10030, 0x0002B4D2}, + {0x10030, 0x0002B8CF}, + {0x10030, 0x0002BC0C}, + {0x10030, 0x0002C009}, + {0x10030, 0x000300F6}, + {0x10030, 0x000304F3}, + {0x10030, 0x000308F0}, + {0x10030, 0x00030CED}, + {0x10030, 0x000310EA}, + {0x10030, 0x000314E7}, + {0x10030, 0x000318E4}, + {0x10030, 0x00031CE1}, + {0x10030, 0x000320DE}, + {0x10030, 0x000324DB}, + {0x10030, 0x000328D8}, + {0x10030, 0x00032CD5}, + {0x10030, 0x000330D2}, + {0x10030, 0x000334CF}, + {0x10030, 0x000338CC}, + {0x10030, 0x00033C09}, + {0x10030, 0x00034006}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x000200FA}, + {0x10030, 0x000204F7}, + {0x10030, 0x000208F4}, + {0x10030, 0x00020CF1}, + {0x10030, 0x000210EE}, + {0x10030, 0x000214EB}, + {0x10030, 0x000218E8}, + {0x10030, 0x00021CE5}, + {0x10030, 0x000220E2}, + {0x10030, 0x000224DF}, + {0x10030, 0x000228DC}, + {0x10030, 0x00022CD9}, + {0x10030, 0x000230D6}, + {0x10030, 0x000234D3}, + {0x10030, 0x000238D0}, + {0x10030, 0x00023C0D}, + {0x10030, 0x0002400A}, + {0x10030, 0x000280F9}, + {0x10030, 0x000284F6}, + {0x10030, 0x000288F3}, + {0x10030, 0x00028CF0}, + {0x10030, 0x000290ED}, + {0x10030, 0x000294EA}, + {0x10030, 0x000298E7}, + {0x10030, 0x00029CE4}, + {0x10030, 0x0002A0E1}, + {0x10030, 0x0002A4DE}, + {0x10030, 0x0002A8DB}, + {0x10030, 0x0002ACD8}, + {0x10030, 0x0002B0D5}, + {0x10030, 0x0002B4D2}, + {0x10030, 0x0002B8CF}, + {0x10030, 0x0002BC0C}, + {0x10030, 0x0002C009}, + {0x10030, 0x000300F6}, + {0x10030, 0x000304F3}, + {0x10030, 0x000308F0}, + {0x10030, 0x00030CED}, + {0x10030, 0x000310EA}, + {0x10030, 0x000314E7}, + {0x10030, 0x000318E4}, + {0x10030, 0x00031CE1}, + {0x10030, 0x000320DE}, + {0x10030, 0x000324DB}, + {0x10030, 0x000328D8}, + {0x10030, 0x00032CD5}, + {0x10030, 0x000330D2}, + {0x10030, 0x000334CF}, + {0x10030, 0x000338CC}, + {0x10030, 0x00033C09}, + {0x10030, 0x00034006}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x000200FA}, + {0x10030, 0x000204F7}, + {0x10030, 0x000208F4}, + {0x10030, 0x00020CF1}, + {0x10030, 0x000210EE}, + {0x10030, 0x000214EB}, + {0x10030, 0x000218E8}, + {0x10030, 0x00021CE5}, + {0x10030, 0x000220E2}, + {0x10030, 0x000224DF}, + {0x10030, 0x000228DC}, + {0x10030, 0x00022CD9}, + {0x10030, 0x000230D6}, + {0x10030, 0x000234D3}, + {0x10030, 0x000238D0}, + {0x10030, 0x00023C0D}, + {0x10030, 0x0002400A}, + {0x10030, 0x000280F9}, + {0x10030, 0x000284F6}, + {0x10030, 0x000288F3}, + {0x10030, 0x00028CF0}, + {0x10030, 0x000290ED}, + {0x10030, 0x000294EA}, + {0x10030, 0x000298E7}, + {0x10030, 0x00029CE4}, + {0x10030, 0x0002A0E1}, + {0x10030, 0x0002A4DE}, + {0x10030, 0x0002A8DB}, + {0x10030, 0x0002ACD8}, + {0x10030, 0x0002B0D5}, + {0x10030, 0x0002B4D2}, + {0x10030, 0x0002B8CF}, + {0x10030, 0x0002BC0C}, + {0x10030, 0x0002C009}, + {0x10030, 0x000300F6}, + {0x10030, 0x000304F3}, + {0x10030, 0x000308F0}, + {0x10030, 0x00030CED}, + {0x10030, 0x000310EA}, + {0x10030, 0x000314E7}, + {0x10030, 0x000318E4}, + {0x10030, 0x00031CE1}, + {0x10030, 0x000320DE}, + {0x10030, 0x000324DB}, + {0x10030, 0x000328D8}, + {0x10030, 0x00032CD5}, + {0x10030, 0x000330D2}, + {0x10030, 0x000334CF}, + {0x10030, 0x000338CC}, + {0x10030, 0x00033C09}, + {0x10030, 0x00034006}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x000200FA}, + {0x10030, 0x000204F7}, + {0x10030, 0x000208F4}, + {0x10030, 0x00020CF1}, + {0x10030, 0x000210EE}, + {0x10030, 0x000214EB}, + {0x10030, 0x000218E8}, + {0x10030, 0x00021CE5}, + {0x10030, 0x000220E2}, + {0x10030, 0x000224DF}, + {0x10030, 0x000228DC}, + {0x10030, 0x00022CD9}, + {0x10030, 0x000230D6}, + {0x10030, 0x000234D3}, + {0x10030, 0x000238D0}, + {0x10030, 0x00023C0D}, + {0x10030, 0x0002400A}, + {0x10030, 0x000280F9}, + {0x10030, 0x000284F6}, + {0x10030, 0x000288F3}, + {0x10030, 0x00028CF0}, + {0x10030, 0x000290ED}, + {0x10030, 0x000294EA}, + {0x10030, 0x000298E7}, + {0x10030, 0x00029CE4}, + {0x10030, 0x0002A0E1}, + {0x10030, 0x0002A4DE}, + {0x10030, 0x0002A8DB}, + {0x10030, 0x0002ACD8}, + {0x10030, 0x0002B0D5}, + {0x10030, 0x0002B4D2}, + {0x10030, 0x0002B8CF}, + {0x10030, 0x0002BC0C}, + {0x10030, 0x0002C009}, + {0x10030, 0x000300F6}, + {0x10030, 0x000304F3}, + {0x10030, 0x000308F0}, + {0x10030, 0x00030CED}, + {0x10030, 0x000310EA}, + {0x10030, 0x000314E7}, + {0x10030, 0x000318E4}, + {0x10030, 0x00031CE1}, + {0x10030, 0x000320DE}, + {0x10030, 0x000324DB}, + {0x10030, 0x000328D8}, + {0x10030, 0x00032CD5}, + {0x10030, 0x000330D2}, + {0x10030, 0x000334CF}, + {0x10030, 0x000338CC}, + {0x10030, 0x00033C09}, + {0x10030, 0x00034006}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x000200FA}, + {0x10030, 0x000204F7}, + {0x10030, 0x000208F4}, + {0x10030, 0x00020CF1}, + {0x10030, 0x000210EE}, + {0x10030, 0x000214EB}, + {0x10030, 0x000218E8}, + {0x10030, 0x00021CE5}, + {0x10030, 0x000220E2}, + {0x10030, 0x000224DF}, + {0x10030, 0x000228DC}, + {0x10030, 0x00022CD9}, + {0x10030, 0x000230D6}, + {0x10030, 0x000234D3}, + {0x10030, 0x000238D0}, + {0x10030, 0x00023C0D}, + {0x10030, 0x0002400A}, + {0x10030, 0x000280F9}, + {0x10030, 0x000284F6}, + {0x10030, 0x000288F3}, + {0x10030, 0x00028CF0}, + {0x10030, 0x000290ED}, + {0x10030, 0x000294EA}, + {0x10030, 0x000298E7}, + {0x10030, 0x00029CE4}, + {0x10030, 0x0002A0E1}, + {0x10030, 0x0002A4DE}, + {0x10030, 0x0002A8DB}, + {0x10030, 0x0002ACD8}, + {0x10030, 0x0002B0D5}, + {0x10030, 0x0002B4D2}, + {0x10030, 0x0002B8CF}, + {0x10030, 0x0002BC0C}, + {0x10030, 0x0002C009}, + {0x10030, 0x000300F6}, + {0x10030, 0x000304F3}, + {0x10030, 0x000308F0}, + {0x10030, 0x00030CED}, + {0x10030, 0x000310EA}, + {0x10030, 0x000314E7}, + {0x10030, 0x000318E4}, + {0x10030, 0x00031CE1}, + {0x10030, 0x000320DE}, + {0x10030, 0x000324DB}, + {0x10030, 0x000328D8}, + {0x10030, 0x00032CD5}, + {0x10030, 0x000330D2}, + {0x10030, 0x000334CF}, + {0x10030, 0x000338CC}, + {0x10030, 0x00033C09}, + {0x10030, 0x00034006}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x000200FA}, + {0x10030, 0x000204F7}, + {0x10030, 0x000208F4}, + {0x10030, 0x00020CF1}, + {0x10030, 0x000210EE}, + {0x10030, 0x000214EB}, + {0x10030, 0x000218E8}, + {0x10030, 0x00021CE5}, + {0x10030, 0x000220E2}, + {0x10030, 0x000224DF}, + {0x10030, 0x000228DC}, + {0x10030, 0x00022CD9}, + {0x10030, 0x000230D6}, + {0x10030, 0x000234D3}, + {0x10030, 0x000238D0}, + {0x10030, 0x00023C0D}, + {0x10030, 0x0002400A}, + {0x10030, 0x000280F9}, + {0x10030, 0x000284F6}, + {0x10030, 0x000288F3}, + {0x10030, 0x00028CF0}, + {0x10030, 0x000290ED}, + {0x10030, 0x000294EA}, + {0x10030, 0x000298E7}, + {0x10030, 0x00029CE4}, + {0x10030, 0x0002A0E1}, + {0x10030, 0x0002A4DE}, + {0x10030, 0x0002A8DB}, + {0x10030, 0x0002ACD8}, + {0x10030, 0x0002B0D5}, + {0x10030, 0x0002B4D2}, + {0x10030, 0x0002B8CF}, + {0x10030, 0x0002BC0C}, + {0x10030, 0x0002C009}, + {0x10030, 0x000300F6}, + {0x10030, 0x000304F3}, + {0x10030, 0x000308F0}, + {0x10030, 0x00030CED}, + {0x10030, 0x000310EA}, + {0x10030, 0x000314E7}, + {0x10030, 0x000318E4}, + {0x10030, 0x00031CE1}, + {0x10030, 0x000320DE}, + {0x10030, 0x000324DB}, + {0x10030, 0x000328D8}, + {0x10030, 0x00032CD5}, + {0x10030, 0x000330D2}, + {0x10030, 0x000334CF}, + {0x10030, 0x000338CC}, + {0x10030, 0x00033C09}, + {0x10030, 0x00034006}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x000200FA}, + {0x10030, 0x000204F7}, + {0x10030, 0x000208F4}, + {0x10030, 0x00020CF1}, + {0x10030, 0x000210EE}, + {0x10030, 0x000214EB}, + {0x10030, 0x000218E8}, + {0x10030, 0x00021CE5}, + {0x10030, 0x000220E2}, + {0x10030, 0x000224DF}, + {0x10030, 0x000228DC}, + {0x10030, 0x00022CD9}, + {0x10030, 0x000230D6}, + {0x10030, 0x000234D3}, + {0x10030, 0x000238D0}, + {0x10030, 0x00023C0D}, + {0x10030, 0x0002400A}, + {0x10030, 0x000280F9}, + {0x10030, 0x000284F6}, + {0x10030, 0x000288F3}, + {0x10030, 0x00028CF0}, + {0x10030, 0x000290ED}, + {0x10030, 0x000294EA}, + {0x10030, 0x000298E7}, + {0x10030, 0x00029CE4}, + {0x10030, 0x0002A0E1}, + {0x10030, 0x0002A4DE}, + {0x10030, 0x0002A8DB}, + {0x10030, 0x0002ACD8}, + {0x10030, 0x0002B0D5}, + {0x10030, 0x0002B4D2}, + {0x10030, 0x0002B8CF}, + {0x10030, 0x0002BC0C}, + {0x10030, 0x0002C009}, + {0x10030, 0x000300F6}, + {0x10030, 0x000304F3}, + {0x10030, 0x000308F0}, + {0x10030, 0x00030CED}, + {0x10030, 0x000310EA}, + {0x10030, 0x000314E7}, + {0x10030, 0x000318E4}, + {0x10030, 0x00031CE1}, + {0x10030, 0x000320DE}, + {0x10030, 0x000324DB}, + {0x10030, 0x000328D8}, + {0x10030, 0x00032CD5}, + {0x10030, 0x000330D2}, + {0x10030, 0x000334CF}, + {0x10030, 0x000338CC}, + {0x10030, 0x00033C09}, + {0x10030, 0x00034006}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x000200FA}, + {0x10030, 0x000204F7}, + {0x10030, 0x000208F4}, + {0x10030, 0x00020CF1}, + {0x10030, 0x000210EE}, + {0x10030, 0x000214EB}, + {0x10030, 0x000218E8}, + {0x10030, 0x00021CE5}, + {0x10030, 0x000220E2}, + {0x10030, 0x000224DF}, + {0x10030, 0x000228DC}, + {0x10030, 0x00022CD9}, + {0x10030, 0x000230D6}, + {0x10030, 0x000234D3}, + {0x10030, 0x000238D0}, + {0x10030, 0x00023C0D}, + {0x10030, 0x0002400A}, + {0x10030, 0x000280F9}, + {0x10030, 0x000284F6}, + {0x10030, 0x000288F3}, + {0x10030, 0x00028CF0}, + {0x10030, 0x000290ED}, + {0x10030, 0x000294EA}, + {0x10030, 0x000298E7}, + {0x10030, 0x00029CE4}, + {0x10030, 0x0002A0E1}, + {0x10030, 0x0002A4DE}, + {0x10030, 0x0002A8DB}, + {0x10030, 0x0002ACD8}, + {0x10030, 0x0002B0D5}, + {0x10030, 0x0002B4D2}, + {0x10030, 0x0002B8CF}, + {0x10030, 0x0002BC0C}, + {0x10030, 0x0002C009}, + {0x10030, 0x000300F6}, + {0x10030, 0x000304F3}, + {0x10030, 0x000308F0}, + {0x10030, 0x00030CED}, + {0x10030, 0x000310EA}, + {0x10030, 0x000314E7}, + {0x10030, 0x000318E4}, + {0x10030, 0x00031CE1}, + {0x10030, 0x000320DE}, + {0x10030, 0x000324DB}, + {0x10030, 0x000328D8}, + {0x10030, 0x00032CD5}, + {0x10030, 0x000330D2}, + {0x10030, 0x000334CF}, + {0x10030, 0x000338CC}, + {0x10030, 0x00033C09}, + {0x10030, 0x00034006}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x000200FA}, + {0x10030, 0x000204F7}, + {0x10030, 0x000208F4}, + {0x10030, 0x00020CF1}, + {0x10030, 0x000210EE}, + {0x10030, 0x000214EB}, + {0x10030, 0x000218E8}, + {0x10030, 0x00021CE5}, + {0x10030, 0x000220E2}, + {0x10030, 0x000224DF}, + {0x10030, 0x000228DC}, + {0x10030, 0x00022CD9}, + {0x10030, 0x000230D6}, + {0x10030, 0x000234D3}, + {0x10030, 0x000238D0}, + {0x10030, 0x00023C0D}, + {0x10030, 0x0002400A}, + {0x10030, 0x000280F9}, + {0x10030, 0x000284F6}, + {0x10030, 0x000288F3}, + {0x10030, 0x00028CF0}, + {0x10030, 0x000290ED}, + {0x10030, 0x000294EA}, + {0x10030, 0x000298E7}, + {0x10030, 0x00029CE4}, + {0x10030, 0x0002A0E1}, + {0x10030, 0x0002A4DE}, + {0x10030, 0x0002A8DB}, + {0x10030, 0x0002ACD8}, + {0x10030, 0x0002B0D5}, + {0x10030, 0x0002B4D2}, + {0x10030, 0x0002B8CF}, + {0x10030, 0x0002BC0C}, + {0x10030, 0x0002C009}, + {0x10030, 0x000300F6}, + {0x10030, 0x000304F3}, + {0x10030, 0x000308F0}, + {0x10030, 0x00030CED}, + {0x10030, 0x000310EA}, + {0x10030, 0x000314E7}, + {0x10030, 0x000318E4}, + {0x10030, 0x00031CE1}, + {0x10030, 0x000320DE}, + {0x10030, 0x000324DB}, + {0x10030, 0x000328D8}, + {0x10030, 0x00032CD5}, + {0x10030, 0x000330D2}, + {0x10030, 0x000334CF}, + {0x10030, 0x000338CC}, + {0x10030, 0x00033C09}, + {0x10030, 0x00034006}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x000200FA}, + {0x10030, 0x000204F7}, + {0x10030, 0x000208F4}, + {0x10030, 0x00020CF1}, + {0x10030, 0x000210EE}, + {0x10030, 0x000214EB}, + {0x10030, 0x000218E8}, + {0x10030, 0x00021CE5}, + {0x10030, 0x000220E2}, + {0x10030, 0x000224DF}, + {0x10030, 0x000228DC}, + {0x10030, 0x00022CD9}, + {0x10030, 0x000230D6}, + {0x10030, 0x000234D3}, + {0x10030, 0x000238D0}, + {0x10030, 0x00023C0D}, + {0x10030, 0x0002400A}, + {0x10030, 0x000280F9}, + {0x10030, 0x000284F6}, + {0x10030, 0x000288F3}, + {0x10030, 0x00028CF0}, + {0x10030, 0x000290ED}, + {0x10030, 0x000294EA}, + {0x10030, 0x000298E7}, + {0x10030, 0x00029CE4}, + {0x10030, 0x0002A0E1}, + {0x10030, 0x0002A4DE}, + {0x10030, 0x0002A8DB}, + {0x10030, 0x0002ACD8}, + {0x10030, 0x0002B0D5}, + {0x10030, 0x0002B4D2}, + {0x10030, 0x0002B8CF}, + {0x10030, 0x0002BC0C}, + {0x10030, 0x0002C009}, + {0x10030, 0x000300F6}, + {0x10030, 0x000304F3}, + {0x10030, 0x000308F0}, + {0x10030, 0x00030CED}, + {0x10030, 0x000310EA}, + {0x10030, 0x000314E7}, + {0x10030, 0x000318E4}, + {0x10030, 0x00031CE1}, + {0x10030, 0x000320DE}, + {0x10030, 0x000324DB}, + {0x10030, 0x000328D8}, + {0x10030, 0x00032CD5}, + {0x10030, 0x000330D2}, + {0x10030, 0x000334CF}, + {0x10030, 0x000338CC}, + {0x10030, 0x00033C09}, + {0x10030, 0x00034006}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x000200FA}, + {0x10030, 0x000204F7}, + {0x10030, 0x000208F4}, + {0x10030, 0x00020CF1}, + {0x10030, 0x000210EE}, + {0x10030, 0x000214EB}, + {0x10030, 0x000218E8}, + {0x10030, 0x00021CE5}, + {0x10030, 0x000220E2}, + {0x10030, 0x000224DF}, + {0x10030, 0x000228DC}, + {0x10030, 0x00022CD9}, + {0x10030, 0x000230D6}, + {0x10030, 0x000234D3}, + {0x10030, 0x000238D0}, + {0x10030, 0x00023C0D}, + {0x10030, 0x0002400A}, + {0x10030, 0x000280F9}, + {0x10030, 0x000284F6}, + {0x10030, 0x000288F3}, + {0x10030, 0x00028CF0}, + {0x10030, 0x000290ED}, + {0x10030, 0x000294EA}, + {0x10030, 0x000298E7}, + {0x10030, 0x00029CE4}, + {0x10030, 0x0002A0E1}, + {0x10030, 0x0002A4DE}, + {0x10030, 0x0002A8DB}, + {0x10030, 0x0002ACD8}, + {0x10030, 0x0002B0D5}, + {0x10030, 0x0002B4D2}, + {0x10030, 0x0002B8CF}, + {0x10030, 0x0002BC0C}, + {0x10030, 0x0002C009}, + {0x10030, 0x000300F6}, + {0x10030, 0x000304F3}, + {0x10030, 0x000308F0}, + {0x10030, 0x00030CED}, + {0x10030, 0x000310EA}, + {0x10030, 0x000314E7}, + {0x10030, 0x000318E4}, + {0x10030, 0x00031CE1}, + {0x10030, 0x000320DE}, + {0x10030, 0x000324DB}, + {0x10030, 0x000328D8}, + {0x10030, 0x00032CD5}, + {0x10030, 0x000330D2}, + {0x10030, 0x000334CF}, + {0x10030, 0x000338CC}, + {0x10030, 0x00033C09}, + {0x10030, 0x00034006}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x000200FA}, + {0x10030, 0x000204F7}, + {0x10030, 0x000208F4}, + {0x10030, 0x00020CF1}, + {0x10030, 0x000210EE}, + {0x10030, 0x000214EB}, + {0x10030, 0x000218E8}, + {0x10030, 0x00021CE5}, + {0x10030, 0x000220E2}, + {0x10030, 0x000224DF}, + {0x10030, 0x000228DC}, + {0x10030, 0x00022CD9}, + {0x10030, 0x000230D6}, + {0x10030, 0x000234D3}, + {0x10030, 0x000238D0}, + {0x10030, 0x00023C0D}, + {0x10030, 0x0002400A}, + {0x10030, 0x000280F9}, + {0x10030, 0x000284F6}, + {0x10030, 0x000288F3}, + {0x10030, 0x00028CF0}, + {0x10030, 0x000290ED}, + {0x10030, 0x000294EA}, + {0x10030, 0x000298E7}, + {0x10030, 0x00029CE4}, + {0x10030, 0x0002A0E1}, + {0x10030, 0x0002A4DE}, + {0x10030, 0x0002A8DB}, + {0x10030, 0x0002ACD8}, + {0x10030, 0x0002B0D5}, + {0x10030, 0x0002B4D2}, + {0x10030, 0x0002B8CF}, + {0x10030, 0x0002BC0C}, + {0x10030, 0x0002C009}, + {0x10030, 0x000300F6}, + {0x10030, 0x000304F3}, + {0x10030, 0x000308F0}, + {0x10030, 0x00030CED}, + {0x10030, 0x000310EA}, + {0x10030, 0x000314E7}, + {0x10030, 0x000318E4}, + {0x10030, 0x00031CE1}, + {0x10030, 0x000320DE}, + {0x10030, 0x000324DB}, + {0x10030, 0x000328D8}, + {0x10030, 0x00032CD5}, + {0x10030, 0x000330D2}, + {0x10030, 0x000334CF}, + {0x10030, 0x000338CC}, + {0x10030, 0x00033C09}, + {0x10030, 0x00034006}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x000200FA}, + {0x10030, 0x000204F7}, + {0x10030, 0x000208F4}, + {0x10030, 0x00020CF1}, + {0x10030, 0x000210EE}, + {0x10030, 0x000214EB}, + {0x10030, 0x000218E8}, + {0x10030, 0x00021CE5}, + {0x10030, 0x000220E2}, + {0x10030, 0x000224DF}, + {0x10030, 0x000228DC}, + {0x10030, 0x00022CD9}, + {0x10030, 0x000230D6}, + {0x10030, 0x000234D3}, + {0x10030, 0x000238D0}, + {0x10030, 0x00023C0D}, + {0x10030, 0x0002400A}, + {0x10030, 0x000280F9}, + {0x10030, 0x000284F6}, + {0x10030, 0x000288F3}, + {0x10030, 0x00028CF0}, + {0x10030, 0x000290ED}, + {0x10030, 0x000294EA}, + {0x10030, 0x000298E7}, + {0x10030, 0x00029CE4}, + {0x10030, 0x0002A0E1}, + {0x10030, 0x0002A4DE}, + {0x10030, 0x0002A8DB}, + {0x10030, 0x0002ACD8}, + {0x10030, 0x0002B0D5}, + {0x10030, 0x0002B4D2}, + {0x10030, 0x0002B8CF}, + {0x10030, 0x0002BC0C}, + {0x10030, 0x0002C009}, + {0x10030, 0x000300F6}, + {0x10030, 0x000304F3}, + {0x10030, 0x000308F0}, + {0x10030, 0x00030CED}, + {0x10030, 0x000310EA}, + {0x10030, 0x000314E7}, + {0x10030, 0x000318E4}, + {0x10030, 0x00031CE1}, + {0x10030, 0x000320DE}, + {0x10030, 0x000324DB}, + {0x10030, 0x000328D8}, + {0x10030, 0x00032CD5}, + {0x10030, 0x000330D2}, + {0x10030, 0x000334CF}, + {0x10030, 0x000338CC}, + {0x10030, 0x00033C09}, + {0x10030, 0x00034006}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x000200E8}, + {0x10030, 0x000204E5}, + {0x10030, 0x000208E2}, + {0x10030, 0x00020CDF}, + {0x10030, 0x000210DC}, + {0x10030, 0x000214D9}, + {0x10030, 0x000218D6}, + {0x10030, 0x00021CD3}, + {0x10030, 0x000220D0}, + {0x10030, 0x000224CD}, + {0x10030, 0x000228CD}, + {0x10030, 0x00022CCD}, + {0x10030, 0x000230CD}, + {0x10030, 0x000234CD}, + {0x10030, 0x000238CD}, + {0x10030, 0x00023CCD}, + {0x10030, 0x000240CD}, + {0x10030, 0x000280E8}, + {0x10030, 0x000284E5}, + {0x10030, 0x000288E2}, + {0x10030, 0x00028CDF}, + {0x10030, 0x000290DC}, + {0x10030, 0x000294D9}, + {0x10030, 0x000298D6}, + {0x10030, 0x00029CD3}, + {0x10030, 0x0002A0D0}, + {0x10030, 0x0002A4CD}, + {0x10030, 0x0002A8CD}, + {0x10030, 0x0002ACCD}, + {0x10030, 0x0002B0CD}, + {0x10030, 0x0002B4CD}, + {0x10030, 0x0002B8CD}, + {0x10030, 0x0002BCCD}, + {0x10030, 0x0002C0CD}, + {0x10030, 0x000300E8}, + {0x10030, 0x000304E5}, + {0x10030, 0x000308E2}, + {0x10030, 0x00030CDF}, + {0x10030, 0x000310DC}, + {0x10030, 0x000314D9}, + {0x10030, 0x000318D6}, + {0x10030, 0x00031CD3}, + {0x10030, 0x000320D0}, + {0x10030, 0x000324CD}, + {0x10030, 0x000328CD}, + {0x10030, 0x00032CCD}, + {0x10030, 0x000330CD}, + {0x10030, 0x000334CD}, + {0x10030, 0x000338CD}, + {0x10030, 0x00033CCD}, + {0x10030, 0x000340CD}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x000200E8}, + {0x10030, 0x000204E5}, + {0x10030, 0x000208E2}, + {0x10030, 0x00020CDF}, + {0x10030, 0x000210DC}, + {0x10030, 0x000214D9}, + {0x10030, 0x000218D6}, + {0x10030, 0x00021CD3}, + {0x10030, 0x000220D0}, + {0x10030, 0x000224CD}, + {0x10030, 0x000228CD}, + {0x10030, 0x00022CCD}, + {0x10030, 0x000230CD}, + {0x10030, 0x000234CD}, + {0x10030, 0x000238CD}, + {0x10030, 0x00023CCD}, + {0x10030, 0x000240CD}, + {0x10030, 0x000280E8}, + {0x10030, 0x000284E5}, + {0x10030, 0x000288E2}, + {0x10030, 0x00028CDF}, + {0x10030, 0x000290DC}, + {0x10030, 0x000294D9}, + {0x10030, 0x000298D6}, + {0x10030, 0x00029CD3}, + {0x10030, 0x0002A0D0}, + {0x10030, 0x0002A4CD}, + {0x10030, 0x0002A8CD}, + {0x10030, 0x0002ACCD}, + {0x10030, 0x0002B0CD}, + {0x10030, 0x0002B4CD}, + {0x10030, 0x0002B8CD}, + {0x10030, 0x0002BCCD}, + {0x10030, 0x0002C0CD}, + {0x10030, 0x000300E8}, + {0x10030, 0x000304E5}, + {0x10030, 0x000308E2}, + {0x10030, 0x00030CDF}, + {0x10030, 0x000310DC}, + {0x10030, 0x000314D9}, + {0x10030, 0x000318D6}, + {0x10030, 0x00031CD3}, + {0x10030, 0x000320D0}, + {0x10030, 0x000324CD}, + {0x10030, 0x000328CD}, + {0x10030, 0x00032CCD}, + {0x10030, 0x000330CD}, + {0x10030, 0x000334CD}, + {0x10030, 0x000338CD}, + {0x10030, 0x00033CCD}, + {0x10030, 0x000340CD}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x000200E8}, + {0x10030, 0x000204E5}, + {0x10030, 0x000208E2}, + {0x10030, 0x00020CDF}, + {0x10030, 0x000210DC}, + {0x10030, 0x000214D9}, + {0x10030, 0x000218D6}, + {0x10030, 0x00021CD3}, + {0x10030, 0x000220D0}, + {0x10030, 0x000224CD}, + {0x10030, 0x000228CD}, + {0x10030, 0x00022CCD}, + {0x10030, 0x000230CD}, + {0x10030, 0x000234CD}, + {0x10030, 0x000238CD}, + {0x10030, 0x00023CCD}, + {0x10030, 0x000240CD}, + {0x10030, 0x000280E8}, + {0x10030, 0x000284E5}, + {0x10030, 0x000288E2}, + {0x10030, 0x00028CDF}, + {0x10030, 0x000290DC}, + {0x10030, 0x000294D9}, + {0x10030, 0x000298D6}, + {0x10030, 0x00029CD3}, + {0x10030, 0x0002A0D0}, + {0x10030, 0x0002A4CD}, + {0x10030, 0x0002A8CD}, + {0x10030, 0x0002ACCD}, + {0x10030, 0x0002B0CD}, + {0x10030, 0x0002B4CD}, + {0x10030, 0x0002B8CD}, + {0x10030, 0x0002BCCD}, + {0x10030, 0x0002C0CD}, + {0x10030, 0x000300E8}, + {0x10030, 0x000304E5}, + {0x10030, 0x000308E2}, + {0x10030, 0x00030CDF}, + {0x10030, 0x000310DC}, + {0x10030, 0x000314D9}, + {0x10030, 0x000318D6}, + {0x10030, 0x00031CD3}, + {0x10030, 0x000320D0}, + {0x10030, 0x000324CD}, + {0x10030, 0x000328CD}, + {0x10030, 0x00032CCD}, + {0x10030, 0x000330CD}, + {0x10030, 0x000334CD}, + {0x10030, 0x000338CD}, + {0x10030, 0x00033CCD}, + {0x10030, 0x000340CD}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x000200E8}, + {0x10030, 0x000204E5}, + {0x10030, 0x000208E2}, + {0x10030, 0x00020CDF}, + {0x10030, 0x000210DC}, + {0x10030, 0x000214D9}, + {0x10030, 0x000218D6}, + {0x10030, 0x00021CD3}, + {0x10030, 0x000220D0}, + {0x10030, 0x000224CD}, + {0x10030, 0x000228CD}, + {0x10030, 0x00022CCD}, + {0x10030, 0x000230CD}, + {0x10030, 0x000234CD}, + {0x10030, 0x000238CD}, + {0x10030, 0x00023CCD}, + {0x10030, 0x000240CD}, + {0x10030, 0x000280E8}, + {0x10030, 0x000284E5}, + {0x10030, 0x000288E2}, + {0x10030, 0x00028CDF}, + {0x10030, 0x000290DC}, + {0x10030, 0x000294D9}, + {0x10030, 0x000298D6}, + {0x10030, 0x00029CD3}, + {0x10030, 0x0002A0D0}, + {0x10030, 0x0002A4CD}, + {0x10030, 0x0002A8CD}, + {0x10030, 0x0002ACCD}, + {0x10030, 0x0002B0CD}, + {0x10030, 0x0002B4CD}, + {0x10030, 0x0002B8CD}, + {0x10030, 0x0002BCCD}, + {0x10030, 0x0002C0CD}, + {0x10030, 0x000300E8}, + {0x10030, 0x000304E5}, + {0x10030, 0x000308E2}, + {0x10030, 0x00030CDF}, + {0x10030, 0x000310DC}, + {0x10030, 0x000314D9}, + {0x10030, 0x000318D6}, + {0x10030, 0x00031CD3}, + {0x10030, 0x000320D0}, + {0x10030, 0x000324CD}, + {0x10030, 0x000328CD}, + {0x10030, 0x00032CCD}, + {0x10030, 0x000330CD}, + {0x10030, 0x000334CD}, + {0x10030, 0x000338CD}, + {0x10030, 0x00033CCD}, + {0x10030, 0x000340CD}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x000200E8}, + {0x10030, 0x000204E5}, + {0x10030, 0x000208E2}, + {0x10030, 0x00020CDF}, + {0x10030, 0x000210DC}, + {0x10030, 0x000214D9}, + {0x10030, 0x000218D6}, + {0x10030, 0x00021CD3}, + {0x10030, 0x000220D0}, + {0x10030, 0x000224CD}, + {0x10030, 0x000228CD}, + {0x10030, 0x00022CCD}, + {0x10030, 0x000230CD}, + {0x10030, 0x000234CD}, + {0x10030, 0x000238CD}, + {0x10030, 0x00023CCD}, + {0x10030, 0x000240CD}, + {0x10030, 0x000280E8}, + {0x10030, 0x000284E5}, + {0x10030, 0x000288E2}, + {0x10030, 0x00028CDF}, + {0x10030, 0x000290DC}, + {0x10030, 0x000294D9}, + {0x10030, 0x000298D6}, + {0x10030, 0x00029CD3}, + {0x10030, 0x0002A0D0}, + {0x10030, 0x0002A4CD}, + {0x10030, 0x0002A8CD}, + {0x10030, 0x0002ACCD}, + {0x10030, 0x0002B0CD}, + {0x10030, 0x0002B4CD}, + {0x10030, 0x0002B8CD}, + {0x10030, 0x0002BCCD}, + {0x10030, 0x0002C0CD}, + {0x10030, 0x000300E8}, + {0x10030, 0x000304E5}, + {0x10030, 0x000308E2}, + {0x10030, 0x00030CDF}, + {0x10030, 0x000310DC}, + {0x10030, 0x000314D9}, + {0x10030, 0x000318D6}, + {0x10030, 0x00031CD3}, + {0x10030, 0x000320D0}, + {0x10030, 0x000324CD}, + {0x10030, 0x000328CD}, + {0x10030, 0x00032CCD}, + {0x10030, 0x000330CD}, + {0x10030, 0x000334CD}, + {0x10030, 0x000338CD}, + {0x10030, 0x00033CCD}, + {0x10030, 0x000340CD}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x000200E8}, + {0x10030, 0x000204E5}, + {0x10030, 0x000208E2}, + {0x10030, 0x00020CDF}, + {0x10030, 0x000210DC}, + {0x10030, 0x000214D9}, + {0x10030, 0x000218D6}, + {0x10030, 0x00021CD3}, + {0x10030, 0x000220D0}, + {0x10030, 0x000224CD}, + {0x10030, 0x000228CD}, + {0x10030, 0x00022CCD}, + {0x10030, 0x000230CD}, + {0x10030, 0x000234CD}, + {0x10030, 0x000238CD}, + {0x10030, 0x00023CCD}, + {0x10030, 0x000240CD}, + {0x10030, 0x000280E8}, + {0x10030, 0x000284E5}, + {0x10030, 0x000288E2}, + {0x10030, 0x00028CDF}, + {0x10030, 0x000290DC}, + {0x10030, 0x000294D9}, + {0x10030, 0x000298D6}, + {0x10030, 0x00029CD3}, + {0x10030, 0x0002A0D0}, + {0x10030, 0x0002A4CD}, + {0x10030, 0x0002A8CD}, + {0x10030, 0x0002ACCD}, + {0x10030, 0x0002B0CD}, + {0x10030, 0x0002B4CD}, + {0x10030, 0x0002B8CD}, + {0x10030, 0x0002BCCD}, + {0x10030, 0x0002C0CD}, + {0x10030, 0x000300E8}, + {0x10030, 0x000304E5}, + {0x10030, 0x000308E2}, + {0x10030, 0x00030CDF}, + {0x10030, 0x000310DC}, + {0x10030, 0x000314D9}, + {0x10030, 0x000318D6}, + {0x10030, 0x00031CD3}, + {0x10030, 0x000320D0}, + {0x10030, 0x000324CD}, + {0x10030, 0x000328CD}, + {0x10030, 0x00032CCD}, + {0x10030, 0x000330CD}, + {0x10030, 0x000334CD}, + {0x10030, 0x000338CD}, + {0x10030, 0x00033CCD}, + {0x10030, 0x000340CD}, + {0xA0000000, 0x00000000}, + {0x10030, 0x000200FA}, + {0x10030, 0x000204F7}, + {0x10030, 0x000208F4}, + {0x10030, 0x00020CF1}, + {0x10030, 0x000210EE}, + {0x10030, 0x000214EB}, + {0x10030, 0x000218E8}, + {0x10030, 0x00021CE5}, + {0x10030, 0x000220E2}, + {0x10030, 0x000224DF}, + {0x10030, 0x000228DC}, + {0x10030, 0x00022CD9}, + {0x10030, 0x000230D6}, + {0x10030, 0x000234D3}, + {0x10030, 0x000238D0}, + {0x10030, 0x00023C0D}, + {0x10030, 0x0002400A}, + {0x10030, 0x000280F9}, + {0x10030, 0x000284F6}, + {0x10030, 0x000288F3}, + {0x10030, 0x00028CF0}, + {0x10030, 0x000290ED}, + {0x10030, 0x000294EA}, + {0x10030, 0x000298E7}, + {0x10030, 0x00029CE4}, + {0x10030, 0x0002A0E1}, + {0x10030, 0x0002A4DE}, + {0x10030, 0x0002A8DB}, + {0x10030, 0x0002ACD8}, + {0x10030, 0x0002B0D5}, + {0x10030, 0x0002B4D2}, + {0x10030, 0x0002B8CF}, + {0x10030, 0x0002BC0C}, + {0x10030, 0x0002C009}, + {0x10030, 0x000300F6}, + {0x10030, 0x000304F3}, + {0x10030, 0x000308F0}, + {0x10030, 0x00030CED}, + {0x10030, 0x000310EA}, + {0x10030, 0x000314E7}, + {0x10030, 0x000318E4}, + {0x10030, 0x00031CE1}, + {0x10030, 0x000320DE}, + {0x10030, 0x000324DB}, + {0x10030, 0x000328D8}, + {0x10030, 0x00032CD5}, + {0x10030, 0x000330D2}, + {0x10030, 0x000334CF}, + {0x10030, 0x000338CC}, + {0x10030, 0x00033C09}, + {0x10030, 0x00034006}, + {0xB0000000, 0x00000000}, + {0x10030, 0x000600F6}, + {0x10030, 0x000604F3}, + {0x10030, 0x000608F0}, + {0x10030, 0x00060CED}, + {0x10030, 0x000610EA}, + {0x10030, 0x000614E7}, + {0x10030, 0x000618E4}, + {0x10030, 0x00061CE1}, + {0x10030, 0x000620DE}, + {0x10030, 0x000624DB}, + {0x10030, 0x000628D8}, + {0x10030, 0x00062CD5}, + {0x10030, 0x000630D2}, + {0x10030, 0x000634CF}, + {0x10030, 0x000638CC}, + {0x10030, 0x00063C09}, + {0x10030, 0x00064006}, + {0x10030, 0x000680F5}, + {0x10030, 0x000684F2}, + {0x10030, 0x000688EF}, + {0x10030, 0x00068CEC}, + {0x10030, 0x000690E9}, + {0x10030, 0x000694E6}, + {0x10030, 0x000698E3}, + {0x10030, 0x00069CE0}, + {0x10030, 0x0006A0DD}, + {0x10030, 0x0006A4DA}, + {0x10030, 0x0006A8D7}, + {0x10030, 0x0006ACD4}, + {0x10030, 0x0006B0D1}, + {0x10030, 0x0006B4CE}, + {0x10030, 0x0006B8CB}, + {0x10030, 0x0006BC08}, + {0x10030, 0x0006C005}, + {0x10030, 0x000700F5}, + {0x10030, 0x000704F2}, + {0x10030, 0x000708EF}, + {0x10030, 0x00070CEC}, + {0x10030, 0x000710E9}, + {0x10030, 0x000714E6}, + {0x10030, 0x000718E3}, + {0x10030, 0x00071CE0}, + {0x10030, 0x000720DD}, + {0x10030, 0x000724DA}, + {0x10030, 0x000728D7}, + {0x10030, 0x00072CD4}, + {0x10030, 0x000730D1}, + {0x10030, 0x000734CE}, + {0x10030, 0x000738CB}, + {0x10030, 0x00073C08}, + {0x10030, 0x00074005}, {0x10030, 0x000780F4}, {0x10030, 0x000784F1}, {0x10030, 0x000788EE}, @@ -11334,9 +22533,93 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radiob_regs[] = { {0x033, 0x00000025}, {0x03F, 0x00008002}, {0x033, 0x00000026}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x00000027}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x00000028}, {0x03F, 0x00050002}, {0x033, 0x00000029}, @@ -11350,9 +22633,93 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radiob_regs[] = { {0x033, 0x0000002D}, {0x03F, 0x00008002}, {0x033, 0x0000002E}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x0000002F}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x00000030}, {0x03F, 0x00050002}, {0x033, 0x00000031}, @@ -11366,9 +22733,93 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radiob_regs[] = { {0x033, 0x00000035}, {0x03F, 0x00008002}, {0x033, 0x00000036}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x00000037}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x00000060}, {0x03F, 0x00050002}, {0x033, 0x00000061}, @@ -11382,9 +22833,93 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radiob_regs[] = { {0x033, 0x00000065}, {0x03F, 0x00008002}, {0x033, 0x00000066}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x00000067}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x00000068}, {0x03F, 0x00050002}, {0x033, 0x00000069}, @@ -11398,9 +22933,93 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radiob_regs[] = { {0x033, 0x0000006D}, {0x03F, 0x00008002}, {0x033, 0x0000006E}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x0000006F}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x00000070}, {0x03F, 0x00050002}, {0x033, 0x00000071}, @@ -11414,9 +23033,93 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radiob_regs[] = { {0x033, 0x00000075}, {0x03F, 0x00008002}, {0x033, 0x00000076}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x00000077}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x00000078}, {0x03F, 0x00050002}, {0x033, 0x00000079}, @@ -11430,9 +23133,93 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radiob_regs[] = { {0x033, 0x0000007D}, {0x03F, 0x00008002}, {0x033, 0x0000007E}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x0000007F}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x000000A0}, {0x03F, 0x00050002}, {0x033, 0x000000A1}, @@ -11446,9 +23233,93 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radiob_regs[] = { {0x033, 0x000000A5}, {0x03F, 0x00008002}, {0x033, 0x000000A6}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x000000A7}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x000000A8}, {0x03F, 0x00050002}, {0x033, 0x000000A9}, @@ -11462,9 +23333,93 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radiob_regs[] = { {0x033, 0x000000AD}, {0x03F, 0x00008002}, {0x033, 0x000000AE}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x000000AF}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x000000B0}, {0x03F, 0x00050002}, {0x033, 0x000000B1}, @@ -11478,9 +23433,93 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radiob_regs[] = { {0x033, 0x000000B5}, {0x03F, 0x00008002}, {0x033, 0x000000B6}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x000000B7}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x000000E0}, {0x03F, 0x00050002}, {0x033, 0x000000E1}, @@ -11494,9 +23533,93 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radiob_regs[] = { {0x033, 0x000000E5}, {0x03F, 0x00008002}, {0x033, 0x000000E6}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x000000E7}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x000000E8}, {0x03F, 0x00050002}, {0x033, 0x000000E9}, @@ -11510,9 +23633,93 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radiob_regs[] = { {0x033, 0x000000ED}, {0x03F, 0x00008002}, {0x033, 0x000000EE}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x000000EF}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x000000F0}, {0x03F, 0x00050002}, {0x033, 0x000000F1}, @@ -11526,9 +23733,93 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radiob_regs[] = { {0x033, 0x000000F5}, {0x03F, 0x00008002}, {0x033, 0x000000F6}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x000000F7}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x000000F8}, {0x03F, 0x00050002}, {0x033, 0x000000F9}, @@ -11542,9 +23833,93 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radiob_regs[] = { {0x033, 0x000000FD}, {0x03F, 0x00008002}, {0x033, 0x000000FE}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x000000FF}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x00000120}, {0x03F, 0x00050002}, {0x033, 0x00000121}, @@ -11558,9 +23933,93 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radiob_regs[] = { {0x033, 0x00000125}, {0x03F, 0x00008002}, {0x033, 0x00000126}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x00000127}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x00000128}, {0x03F, 0x00050002}, {0x033, 0x00000129}, @@ -11574,9 +24033,93 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radiob_regs[] = { {0x033, 0x0000012D}, {0x03F, 0x00008002}, {0x033, 0x0000012E}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x0000012F}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x00000130}, {0x03F, 0x00050002}, {0x033, 0x00000131}, @@ -11590,9 +24133,93 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radiob_regs[] = { {0x033, 0x00000135}, {0x03F, 0x00008002}, {0x033, 0x00000136}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x00000137}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x00000160}, {0x03F, 0x00050002}, {0x033, 0x00000161}, @@ -11606,9 +24233,93 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radiob_regs[] = { {0x033, 0x00000165}, {0x03F, 0x00008002}, {0x033, 0x00000166}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x00000167}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x00000168}, {0x03F, 0x00050002}, {0x033, 0x00000169}, @@ -11622,9 +24333,93 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radiob_regs[] = { {0x033, 0x0000016D}, {0x03F, 0x00008002}, {0x033, 0x0000016E}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x0000016F}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x00000170}, {0x03F, 0x00050002}, {0x033, 0x00000171}, @@ -11638,9 +24433,93 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radiob_regs[] = { {0x033, 0x00000175}, {0x03F, 0x00008002}, {0x033, 0x00000176}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x00000177}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x00000178}, {0x03F, 0x00050002}, {0x033, 0x00000179}, @@ -11654,9 +24533,93 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radiob_regs[] = { {0x033, 0x0000017D}, {0x03F, 0x00008002}, {0x033, 0x0000017E}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x0000017F}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x000001A0}, {0x03F, 0x00050002}, {0x033, 0x000001A1}, @@ -11670,9 +24633,93 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radiob_regs[] = { {0x033, 0x000001A5}, {0x03F, 0x00008002}, {0x033, 0x000001A6}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x000001A7}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x000001A8}, {0x03F, 0x00050002}, {0x033, 0x000001A9}, @@ -11686,9 +24733,93 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radiob_regs[] = { {0x033, 0x000001AD}, {0x03F, 0x00008002}, {0x033, 0x000001AE}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x000001AF}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x000001B0}, {0x03F, 0x00050002}, {0x033, 0x000001B1}, @@ -11702,9 +24833,93 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radiob_regs[] = { {0x033, 0x000001B5}, {0x03F, 0x00008002}, {0x033, 0x000001B6}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x000001B7}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x000001E0}, {0x03F, 0x00050002}, {0x033, 0x000001E1}, @@ -11718,9 +24933,93 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radiob_regs[] = { {0x033, 0x000001E5}, {0x03F, 0x00008002}, {0x033, 0x000001E6}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x000001E7}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x000001E8}, {0x03F, 0x00050002}, {0x033, 0x000001E9}, @@ -11734,9 +25033,93 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radiob_regs[] = { {0x033, 0x000001ED}, {0x03F, 0x00008002}, {0x033, 0x000001EE}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x000001EF}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x000001F0}, {0x03F, 0x00050002}, {0x033, 0x000001F1}, @@ -11750,9 +25133,93 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radiob_regs[] = { {0x033, 0x000001F5}, {0x03F, 0x00008002}, {0x033, 0x000001F6}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x000001F7}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x000001F8}, {0x03F, 0x00050002}, {0x033, 0x000001F9}, @@ -11766,9 +25233,93 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radiob_regs[] = { {0x033, 0x000001FD}, {0x03F, 0x00008002}, {0x033, 0x000001FE}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x033, 0x000001FF}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, {0x03F, 0x00000003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00000003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x03F, 0x00008002}, + {0xA0000000, 0x00000000}, + {0x03F, 0x00000003}, + {0xB0000000, 0x00000000}, {0x0EF, 0x00000000}, {0x005, 0x00000001}, {0x10005, 0x00000001}, @@ -11810,7 +25361,49 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radiob_regs[] = { {0x10030, 0x00022000}, {0x10030, 0x00023000}, {0x10030, 0x00024000}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00025000}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00025000}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00025000}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, {0x10030, 0x00025000}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00025000}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00025000}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00025000}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00025003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00025003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00025003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00025003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00025003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00025003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00025003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00025003}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00025003}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00025003}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00025003}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00025003}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00025003}, + {0xA0000000, 0x00000000}, + {0x10030, 0x00025000}, + {0xB0000000, 0x00000000}, {0x10030, 0x00026003}, {0x10030, 0x00027003}, {0x10030, 0x00028000}, @@ -11818,7 +25411,49 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radiob_regs[] = { {0x10030, 0x0002A000}, {0x10030, 0x0002B000}, {0x10030, 0x0002C000}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0002D000}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, {0x10030, 0x0002D000}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0002D000}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0002D000}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0002D000}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0002D000}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0002D000}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0002D003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0002D003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0002D003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0002D003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0002D003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0002D003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0002D003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0002D003}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0002D003}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0002D003}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0002D003}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0002D003}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0002D003}, + {0xA0000000, 0x00000000}, + {0x10030, 0x0002D000}, + {0xB0000000, 0x00000000}, {0x10030, 0x0002E003}, {0x10030, 0x0002F003}, {0x10030, 0x00030000}, @@ -11826,7 +25461,49 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radiob_regs[] = { {0x10030, 0x00032000}, {0x10030, 0x00033000}, {0x10030, 0x00034000}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00035000}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00035000}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00035000}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00035000}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00035000}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00035000}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00035000}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00035003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00035003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00035003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00035003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00035003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00035003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00035003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00035003}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00035003}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00035003}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00035003}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00035003}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00035003}, + {0xA0000000, 0x00000000}, {0x10030, 0x00035000}, + {0xB0000000, 0x00000000}, {0x10030, 0x00036003}, {0x10030, 0x00037003}, {0x10030, 0x00038000}, @@ -11834,7 +25511,49 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radiob_regs[] = { {0x10030, 0x0003A000}, {0x10030, 0x0003B000}, {0x10030, 0x0003C000}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0003D000}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, {0x10030, 0x0003D000}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0003D000}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0003D000}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0003D000}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0003D000}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0003D000}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0003D003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0003D003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0003D003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0003D003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0003D003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0003D003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0003D003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0003D003}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0003D003}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0003D003}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0003D003}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0003D003}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0003D003}, + {0xA0000000, 0x00000000}, + {0x10030, 0x0003D000}, + {0xB0000000, 0x00000000}, {0x10030, 0x0003E003}, {0x10030, 0x0003F003}, {0x10030, 0x00060000}, @@ -11842,32 +25561,280 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radiob_regs[] = { {0x10030, 0x00062000}, {0x10030, 0x00063000}, {0x10030, 0x00064000}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, {0x10030, 0x00065000}, {0x10030, 0x00066000}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00065000}, + {0x10030, 0x00066000}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00065000}, + {0x10030, 0x00066000}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00065000}, + {0x10030, 0x00066000}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00065000}, + {0x10030, 0x00066000}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00065000}, + {0x10030, 0x00066000}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00065000}, + {0x10030, 0x00066000}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00065003}, + {0x10030, 0x00066003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00065003}, + {0x10030, 0x00066003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00065003}, + {0x10030, 0x00066003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00065003}, + {0x10030, 0x00066003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00065003}, + {0x10030, 0x00066003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00065003}, + {0x10030, 0x00066003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00065003}, + {0x10030, 0x00066003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00065003}, + {0x10030, 0x00066003}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00065003}, + {0x10030, 0x00066003}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00065003}, + {0x10030, 0x00066003}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00065003}, + {0x10030, 0x00066003}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00065003}, + {0x10030, 0x00066003}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00065003}, + {0x10030, 0x00066003}, + {0xA0000000, 0x00000000}, + {0x10030, 0x00065000}, + {0x10030, 0x00066000}, + {0xB0000000, 0x00000000}, {0x10030, 0x00067003}, {0x10030, 0x00068000}, {0x10030, 0x00069000}, {0x10030, 0x0006A000}, {0x10030, 0x0006B000}, {0x10030, 0x0006C000}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0006D000}, + {0x10030, 0x0006E000}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0006D000}, + {0x10030, 0x0006E000}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0006D000}, + {0x10030, 0x0006E000}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0006D000}, + {0x10030, 0x0006E000}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0006D000}, + {0x10030, 0x0006E000}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, {0x10030, 0x0006D000}, {0x10030, 0x0006E000}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0006D000}, + {0x10030, 0x0006E000}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0006D003}, + {0x10030, 0x0006E003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0006D003}, + {0x10030, 0x0006E003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0006D003}, + {0x10030, 0x0006E003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0006D003}, + {0x10030, 0x0006E003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0006D003}, + {0x10030, 0x0006E003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0006D003}, + {0x10030, 0x0006E003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0006D003}, + {0x10030, 0x0006E003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0006D003}, + {0x10030, 0x0006E003}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0006D003}, + {0x10030, 0x0006E003}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0006D003}, + {0x10030, 0x0006E003}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0006D003}, + {0x10030, 0x0006E003}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0006D003}, + {0x10030, 0x0006E003}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0006D003}, + {0x10030, 0x0006E003}, + {0xA0000000, 0x00000000}, + {0x10030, 0x0006D000}, + {0x10030, 0x0006E000}, + {0xB0000000, 0x00000000}, {0x10030, 0x0006F003}, {0x10030, 0x00070000}, {0x10030, 0x00071000}, {0x10030, 0x00072000}, {0x10030, 0x00073000}, {0x10030, 0x00074000}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00075000}, + {0x10030, 0x00076000}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00075000}, + {0x10030, 0x00076000}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, {0x10030, 0x00075000}, {0x10030, 0x00076000}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00075000}, + {0x10030, 0x00076000}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00075000}, + {0x10030, 0x00076000}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00075000}, + {0x10030, 0x00076000}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00075000}, + {0x10030, 0x00076000}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00075003}, + {0x10030, 0x00076003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00075003}, + {0x10030, 0x00076003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00075003}, + {0x10030, 0x00076003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00075003}, + {0x10030, 0x00076003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00075003}, + {0x10030, 0x00076003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00075003}, + {0x10030, 0x00076003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00075003}, + {0x10030, 0x00076003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00075003}, + {0x10030, 0x00076003}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00075003}, + {0x10030, 0x00076003}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00075003}, + {0x10030, 0x00076003}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00075003}, + {0x10030, 0x00076003}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00075003}, + {0x10030, 0x00076003}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x00075003}, + {0x10030, 0x00076003}, + {0xA0000000, 0x00000000}, + {0x10030, 0x00075000}, + {0x10030, 0x00076000}, + {0xB0000000, 0x00000000}, {0x10030, 0x00077003}, {0x10030, 0x00078000}, {0x10030, 0x00079000}, {0x10030, 0x0007A000}, {0x10030, 0x0007B000}, {0x10030, 0x0007C000}, + {0x80010000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0007D000}, + {0x10030, 0x0007E000}, + {0x90020000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0007D000}, + {0x10030, 0x0007E000}, + {0x90320000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0007D000}, + {0x10030, 0x0007E000}, + {0x90330000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0007D000}, + {0x10030, 0x0007E000}, + {0x90340000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0007D000}, + {0x10030, 0x0007E000}, + {0x90350000, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0007D000}, + {0x10030, 0x0007E000}, + {0x90360000, 0x00000000}, {0x40000000, 0x00000000}, {0x10030, 0x0007D000}, {0x10030, 0x0007E000}, + {0x90010001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0007D003}, + {0x10030, 0x0007E003}, + {0x90020001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0007D003}, + {0x10030, 0x0007E003}, + {0x90030001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0007D003}, + {0x10030, 0x0007E003}, + {0x90040001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0007D003}, + {0x10030, 0x0007E003}, + {0x90050001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0007D003}, + {0x10030, 0x0007E003}, + {0x90070001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0007D003}, + {0x10030, 0x0007E003}, + {0x90320001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0007D003}, + {0x10030, 0x0007E003}, + {0x90330001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0007D003}, + {0x10030, 0x0007E003}, + {0x90340001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0007D003}, + {0x10030, 0x0007E003}, + {0x90350001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0007D003}, + {0x10030, 0x0007E003}, + {0x90360001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0007D003}, + {0x10030, 0x0007E003}, + {0x903f0001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0007D003}, + {0x10030, 0x0007E003}, + {0x90400001, 0x00000000}, {0x40000000, 0x00000000}, + {0x10030, 0x0007D003}, + {0x10030, 0x0007E003}, + {0xA0000000, 0x00000000}, + {0x10030, 0x0007D000}, + {0x10030, 0x0007E000}, + {0xB0000000, 0x00000000}, {0x10030, 0x0007F003}, {0x0ED, 0x00000010}, {0x033, 0x00000001}, @@ -11884,7 +25851,7 @@ static const struct rtw89_reg2_def rtw89_8852c_phy_radiob_regs[] = { {0x03F, 0x0000000A}, {0x0ED, 0x00000000}, {0x100EE, 0x00000000}, - {0x0FE, 0x00000031}, + {0x0FE, 0x00000048}, }; static const struct rtw89_reg2_def rtw89_8852c_phy_nctl_regs[] = { @@ -13825,1207 +27792,1722 @@ static const s8 _txpwr_track_delta_swingidx_2g_cck_a_p[] = { const u8 rtw89_8852c_tx_shape[RTW89_BAND_MAX][RTW89_RS_TX_SHAPE_NUM] [RTW89_REGD_NUM] = { [0][0][RTW89_ACMA] = 0, + [0][0][RTW89_CN] = 0, [0][0][RTW89_ETSI] = 0, [0][0][RTW89_FCC] = 1, [0][0][RTW89_IC] = 1, + [0][0][RTW89_KCC] = 0, [0][0][RTW89_MKK] = 0, + [0][0][RTW89_UK] = 0, [0][1][RTW89_ACMA] = 0, + [0][1][RTW89_CN] = 0, [0][1][RTW89_ETSI] = 0, [0][1][RTW89_FCC] = 3, [0][1][RTW89_IC] = 3, + [0][1][RTW89_KCC] = 0, [0][1][RTW89_MKK] = 0, + [0][1][RTW89_UK] = 0, [1][1][RTW89_ACMA] = 0, + [1][1][RTW89_CN] = 0, [1][1][RTW89_ETSI] = 0, [1][1][RTW89_FCC] = 3, [1][1][RTW89_IC] = 3, + [1][1][RTW89_KCC] = 0, [1][1][RTW89_MKK] = 0, - [2][1][RTW89_FCC] = 1, + [1][1][RTW89_UK] = 0, + [2][1][RTW89_ETSI] = 0, + [2][1][RTW89_FCC] = 0, + [2][1][RTW89_KCC] = 0, }; const s8 rtw89_8852c_txpwr_lmt_2g[RTW89_2G_BW_NUM][RTW89_NTX_NUM] [RTW89_RS_LMT_NUM][RTW89_BF_NUM] [RTW89_REGD_NUM][RTW89_2G_CH_NUM] = { - [0][0][0][0][RTW89_WW][0] = 60, - [0][0][0][0][RTW89_WW][1] = 60, - [0][0][0][0][RTW89_WW][2] = 60, - [0][0][0][0][RTW89_WW][3] = 60, - [0][0][0][0][RTW89_WW][4] = 60, - [0][0][0][0][RTW89_WW][5] = 60, - [0][0][0][0][RTW89_WW][6] = 60, - [0][0][0][0][RTW89_WW][7] = 60, - [0][0][0][0][RTW89_WW][8] = 60, - [0][0][0][0][RTW89_WW][9] = 60, - [0][0][0][0][RTW89_WW][10] = 60, - [0][0][0][0][RTW89_WW][11] = 60, - [0][0][0][0][RTW89_WW][12] = 48, + [0][0][0][0][RTW89_WW][0] = 58, + [0][0][0][0][RTW89_WW][1] = 58, + [0][0][0][0][RTW89_WW][2] = 58, + [0][0][0][0][RTW89_WW][3] = 58, + [0][0][0][0][RTW89_WW][4] = 58, + [0][0][0][0][RTW89_WW][5] = 58, + [0][0][0][0][RTW89_WW][6] = 58, + [0][0][0][0][RTW89_WW][7] = 58, + [0][0][0][0][RTW89_WW][8] = 58, + [0][0][0][0][RTW89_WW][9] = 58, + [0][0][0][0][RTW89_WW][10] = 58, + [0][0][0][0][RTW89_WW][11] = 58, + [0][0][0][0][RTW89_WW][12] = 46, [0][0][0][0][RTW89_WW][13] = 72, - [0][1][0][0][RTW89_WW][0] = 48, - [0][1][0][0][RTW89_WW][1] = 48, - [0][1][0][0][RTW89_WW][2] = 48, - [0][1][0][0][RTW89_WW][3] = 48, - [0][1][0][0][RTW89_WW][4] = 48, - [0][1][0][0][RTW89_WW][5] = 48, - [0][1][0][0][RTW89_WW][6] = 48, - [0][1][0][0][RTW89_WW][7] = 48, - [0][1][0][0][RTW89_WW][8] = 48, - [0][1][0][0][RTW89_WW][9] = 48, - [0][1][0][0][RTW89_WW][10] = 48, - [0][1][0][0][RTW89_WW][11] = 46, - [0][1][0][0][RTW89_WW][12] = 34, + [0][1][0][0][RTW89_WW][0] = 42, + [0][1][0][0][RTW89_WW][1] = 42, + [0][1][0][0][RTW89_WW][2] = 42, + [0][1][0][0][RTW89_WW][3] = 42, + [0][1][0][0][RTW89_WW][4] = 42, + [0][1][0][0][RTW89_WW][5] = 42, + [0][1][0][0][RTW89_WW][6] = 42, + [0][1][0][0][RTW89_WW][7] = 42, + [0][1][0][0][RTW89_WW][8] = 42, + [0][1][0][0][RTW89_WW][9] = 42, + [0][1][0][0][RTW89_WW][10] = 42, + [0][1][0][0][RTW89_WW][11] = 42, + [0][1][0][0][RTW89_WW][12] = 18, [0][1][0][0][RTW89_WW][13] = 60, [1][0][0][0][RTW89_WW][0] = 0, [1][0][0][0][RTW89_WW][1] = 0, - [1][0][0][0][RTW89_WW][2] = 42, - [1][0][0][0][RTW89_WW][3] = 42, - [1][0][0][0][RTW89_WW][4] = 42, + [1][0][0][0][RTW89_WW][2] = 44, + [1][0][0][0][RTW89_WW][3] = 58, + [1][0][0][0][RTW89_WW][4] = 58, [1][0][0][0][RTW89_WW][5] = 58, - [1][0][0][0][RTW89_WW][6] = 42, - [1][0][0][0][RTW89_WW][7] = 42, - [1][0][0][0][RTW89_WW][8] = 42, - [1][0][0][0][RTW89_WW][9] = 34, - [1][0][0][0][RTW89_WW][10] = 22, + [1][0][0][0][RTW89_WW][6] = 46, + [1][0][0][0][RTW89_WW][7] = 46, + [1][0][0][0][RTW89_WW][8] = 28, + [1][0][0][0][RTW89_WW][9] = 26, + [1][0][0][0][RTW89_WW][10] = 26, [1][0][0][0][RTW89_WW][11] = 0, [1][0][0][0][RTW89_WW][12] = 0, [1][0][0][0][RTW89_WW][13] = 0, [1][1][0][0][RTW89_WW][0] = 0, [1][1][0][0][RTW89_WW][1] = 0, - [1][1][0][0][RTW89_WW][2] = 38, - [1][1][0][0][RTW89_WW][3] = 38, - [1][1][0][0][RTW89_WW][4] = 38, - [1][1][0][0][RTW89_WW][5] = 48, - [1][1][0][0][RTW89_WW][6] = 26, - [1][1][0][0][RTW89_WW][7] = 26, - [1][1][0][0][RTW89_WW][8] = 26, - [1][1][0][0][RTW89_WW][9] = 22, - [1][1][0][0][RTW89_WW][10] = 22, + [1][1][0][0][RTW89_WW][2] = 46, + [1][1][0][0][RTW89_WW][3] = 46, + [1][1][0][0][RTW89_WW][4] = 46, + [1][1][0][0][RTW89_WW][5] = 46, + [1][1][0][0][RTW89_WW][6] = 40, + [1][1][0][0][RTW89_WW][7] = 40, + [1][1][0][0][RTW89_WW][8] = 14, + [1][1][0][0][RTW89_WW][9] = 14, + [1][1][0][0][RTW89_WW][10] = 12, [1][1][0][0][RTW89_WW][11] = 0, [1][1][0][0][RTW89_WW][12] = 0, [1][1][0][0][RTW89_WW][13] = 0, - [0][0][1][0][RTW89_WW][0] = 60, - [0][0][1][0][RTW89_WW][1] = 60, - [0][0][1][0][RTW89_WW][2] = 60, - [0][0][1][0][RTW89_WW][3] = 60, - [0][0][1][0][RTW89_WW][4] = 60, - [0][0][1][0][RTW89_WW][5] = 60, - [0][0][1][0][RTW89_WW][6] = 60, - [0][0][1][0][RTW89_WW][7] = 60, - [0][0][1][0][RTW89_WW][8] = 60, - [0][0][1][0][RTW89_WW][9] = 60, - [0][0][1][0][RTW89_WW][10] = 60, - [0][0][1][0][RTW89_WW][11] = 46, - [0][0][1][0][RTW89_WW][12] = 42, + [0][0][1][0][RTW89_WW][0] = 58, + [0][0][1][0][RTW89_WW][1] = 58, + [0][0][1][0][RTW89_WW][2] = 58, + [0][0][1][0][RTW89_WW][3] = 58, + [0][0][1][0][RTW89_WW][4] = 58, + [0][0][1][0][RTW89_WW][5] = 58, + [0][0][1][0][RTW89_WW][6] = 58, + [0][0][1][0][RTW89_WW][7] = 58, + [0][0][1][0][RTW89_WW][8] = 58, + [0][0][1][0][RTW89_WW][9] = 58, + [0][0][1][0][RTW89_WW][10] = 58, + [0][0][1][0][RTW89_WW][11] = 58, + [0][0][1][0][RTW89_WW][12] = 58, [0][0][1][0][RTW89_WW][13] = 0, - [0][1][1][0][RTW89_WW][0] = 48, - [0][1][1][0][RTW89_WW][1] = 48, - [0][1][1][0][RTW89_WW][2] = 48, - [0][1][1][0][RTW89_WW][3] = 48, - [0][1][1][0][RTW89_WW][4] = 48, - [0][1][1][0][RTW89_WW][5] = 48, - [0][1][1][0][RTW89_WW][6] = 48, - [0][1][1][0][RTW89_WW][7] = 48, - [0][1][1][0][RTW89_WW][8] = 48, - [0][1][1][0][RTW89_WW][9] = 48, - [0][1][1][0][RTW89_WW][10] = 48, - [0][1][1][0][RTW89_WW][11] = 38, - [0][1][1][0][RTW89_WW][12] = 34, + [0][1][1][0][RTW89_WW][0] = 46, + [0][1][1][0][RTW89_WW][1] = 46, + [0][1][1][0][RTW89_WW][2] = 46, + [0][1][1][0][RTW89_WW][3] = 46, + [0][1][1][0][RTW89_WW][4] = 46, + [0][1][1][0][RTW89_WW][5] = 46, + [0][1][1][0][RTW89_WW][6] = 46, + [0][1][1][0][RTW89_WW][7] = 46, + [0][1][1][0][RTW89_WW][8] = 46, + [0][1][1][0][RTW89_WW][9] = 46, + [0][1][1][0][RTW89_WW][10] = 46, + [0][1][1][0][RTW89_WW][11] = 46, + [0][1][1][0][RTW89_WW][12] = 36, [0][1][1][0][RTW89_WW][13] = 0, - [0][0][2][0][RTW89_WW][0] = 60, - [0][0][2][0][RTW89_WW][1] = 60, - [0][0][2][0][RTW89_WW][2] = 60, - [0][0][2][0][RTW89_WW][3] = 60, - [0][0][2][0][RTW89_WW][4] = 60, - [0][0][2][0][RTW89_WW][5] = 60, - [0][0][2][0][RTW89_WW][6] = 60, - [0][0][2][0][RTW89_WW][7] = 60, - [0][0][2][0][RTW89_WW][8] = 60, - [0][0][2][0][RTW89_WW][9] = 60, - [0][0][2][0][RTW89_WW][10] = 60, - [0][0][2][0][RTW89_WW][11] = 46, - [0][0][2][0][RTW89_WW][12] = 42, + [0][0][2][0][RTW89_WW][0] = 58, + [0][0][2][0][RTW89_WW][1] = 58, + [0][0][2][0][RTW89_WW][2] = 58, + [0][0][2][0][RTW89_WW][3] = 58, + [0][0][2][0][RTW89_WW][4] = 58, + [0][0][2][0][RTW89_WW][5] = 58, + [0][0][2][0][RTW89_WW][6] = 58, + [0][0][2][0][RTW89_WW][7] = 58, + [0][0][2][0][RTW89_WW][8] = 58, + [0][0][2][0][RTW89_WW][9] = 58, + [0][0][2][0][RTW89_WW][10] = 58, + [0][0][2][0][RTW89_WW][11] = 58, + [0][0][2][0][RTW89_WW][12] = 38, [0][0][2][0][RTW89_WW][13] = 0, - [0][1][2][0][RTW89_WW][0] = 48, - [0][1][2][0][RTW89_WW][1] = 48, - [0][1][2][0][RTW89_WW][2] = 48, - [0][1][2][0][RTW89_WW][3] = 48, - [0][1][2][0][RTW89_WW][4] = 48, - [0][1][2][0][RTW89_WW][5] = 48, - [0][1][2][0][RTW89_WW][6] = 48, - [0][1][2][0][RTW89_WW][7] = 48, - [0][1][2][0][RTW89_WW][8] = 48, - [0][1][2][0][RTW89_WW][9] = 48, - [0][1][2][0][RTW89_WW][10] = 48, - [0][1][2][0][RTW89_WW][11] = 38, - [0][1][2][0][RTW89_WW][12] = 34, + [0][1][2][0][RTW89_WW][0] = 46, + [0][1][2][0][RTW89_WW][1] = 46, + [0][1][2][0][RTW89_WW][2] = 46, + [0][1][2][0][RTW89_WW][3] = 46, + [0][1][2][0][RTW89_WW][4] = 46, + [0][1][2][0][RTW89_WW][5] = 46, + [0][1][2][0][RTW89_WW][6] = 46, + [0][1][2][0][RTW89_WW][7] = 46, + [0][1][2][0][RTW89_WW][8] = 46, + [0][1][2][0][RTW89_WW][9] = 46, + [0][1][2][0][RTW89_WW][10] = 46, + [0][1][2][0][RTW89_WW][11] = 46, + [0][1][2][0][RTW89_WW][12] = 16, [0][1][2][0][RTW89_WW][13] = 0, [0][1][2][1][RTW89_WW][0] = 36, - [0][1][2][1][RTW89_WW][1] = 36, - [0][1][2][1][RTW89_WW][2] = 36, - [0][1][2][1][RTW89_WW][3] = 36, - [0][1][2][1][RTW89_WW][4] = 36, - [0][1][2][1][RTW89_WW][5] = 36, - [0][1][2][1][RTW89_WW][6] = 36, - [0][1][2][1][RTW89_WW][7] = 36, - [0][1][2][1][RTW89_WW][8] = 36, - [0][1][2][1][RTW89_WW][9] = 36, - [0][1][2][1][RTW89_WW][10] = 36, - [0][1][2][1][RTW89_WW][11] = 36, - [0][1][2][1][RTW89_WW][12] = 34, + [0][1][2][1][RTW89_WW][1] = 34, + [0][1][2][1][RTW89_WW][2] = 34, + [0][1][2][1][RTW89_WW][3] = 34, + [0][1][2][1][RTW89_WW][4] = 34, + [0][1][2][1][RTW89_WW][5] = 34, + [0][1][2][1][RTW89_WW][6] = 34, + [0][1][2][1][RTW89_WW][7] = 34, + [0][1][2][1][RTW89_WW][8] = 34, + [0][1][2][1][RTW89_WW][9] = 34, + [0][1][2][1][RTW89_WW][10] = 34, + [0][1][2][1][RTW89_WW][11] = 34, + [0][1][2][1][RTW89_WW][12] = 16, [0][1][2][1][RTW89_WW][13] = 0, [1][0][2][0][RTW89_WW][0] = 0, [1][0][2][0][RTW89_WW][1] = 0, - [1][0][2][0][RTW89_WW][2] = 60, - [1][0][2][0][RTW89_WW][3] = 60, - [1][0][2][0][RTW89_WW][4] = 60, - [1][0][2][0][RTW89_WW][5] = 60, - [1][0][2][0][RTW89_WW][6] = 60, - [1][0][2][0][RTW89_WW][7] = 60, - [1][0][2][0][RTW89_WW][8] = 60, - [1][0][2][0][RTW89_WW][9] = 60, - [1][0][2][0][RTW89_WW][10] = 58, + [1][0][2][0][RTW89_WW][2] = 58, + [1][0][2][0][RTW89_WW][3] = 58, + [1][0][2][0][RTW89_WW][4] = 58, + [1][0][2][0][RTW89_WW][5] = 58, + [1][0][2][0][RTW89_WW][6] = 58, + [1][0][2][0][RTW89_WW][7] = 58, + [1][0][2][0][RTW89_WW][8] = 58, + [1][0][2][0][RTW89_WW][9] = 58, + [1][0][2][0][RTW89_WW][10] = 56, [1][0][2][0][RTW89_WW][11] = 0, [1][0][2][0][RTW89_WW][12] = 0, [1][0][2][0][RTW89_WW][13] = 0, [1][1][2][0][RTW89_WW][0] = 0, [1][1][2][0][RTW89_WW][1] = 0, - [1][1][2][0][RTW89_WW][2] = 46, - [1][1][2][0][RTW89_WW][3] = 46, - [1][1][2][0][RTW89_WW][4] = 48, - [1][1][2][0][RTW89_WW][5] = 48, - [1][1][2][0][RTW89_WW][6] = 48, - [1][1][2][0][RTW89_WW][7] = 46, - [1][1][2][0][RTW89_WW][8] = 46, + [1][1][2][0][RTW89_WW][2] = 34, + [1][1][2][0][RTW89_WW][3] = 34, + [1][1][2][0][RTW89_WW][4] = 34, + [1][1][2][0][RTW89_WW][5] = 34, + [1][1][2][0][RTW89_WW][6] = 34, + [1][1][2][0][RTW89_WW][7] = 34, + [1][1][2][0][RTW89_WW][8] = 34, [1][1][2][0][RTW89_WW][9] = 34, - [1][1][2][0][RTW89_WW][10] = 30, + [1][1][2][0][RTW89_WW][10] = 34, [1][1][2][0][RTW89_WW][11] = 0, [1][1][2][0][RTW89_WW][12] = 0, [1][1][2][0][RTW89_WW][13] = 0, [1][1][2][1][RTW89_WW][0] = 0, [1][1][2][1][RTW89_WW][1] = 0, - [1][1][2][1][RTW89_WW][2] = 36, - [1][1][2][1][RTW89_WW][3] = 36, - [1][1][2][1][RTW89_WW][4] = 36, - [1][1][2][1][RTW89_WW][5] = 36, - [1][1][2][1][RTW89_WW][6] = 36, - [1][1][2][1][RTW89_WW][7] = 36, - [1][1][2][1][RTW89_WW][8] = 36, + [1][1][2][1][RTW89_WW][2] = 34, + [1][1][2][1][RTW89_WW][3] = 34, + [1][1][2][1][RTW89_WW][4] = 34, + [1][1][2][1][RTW89_WW][5] = 34, + [1][1][2][1][RTW89_WW][6] = 34, + [1][1][2][1][RTW89_WW][7] = 34, + [1][1][2][1][RTW89_WW][8] = 34, [1][1][2][1][RTW89_WW][9] = 34, - [1][1][2][1][RTW89_WW][10] = 30, + [1][1][2][1][RTW89_WW][10] = 36, [1][1][2][1][RTW89_WW][11] = 0, [1][1][2][1][RTW89_WW][12] = 0, [1][1][2][1][RTW89_WW][13] = 0, - [0][0][0][0][RTW89_FCC][0] = 70, + [0][0][0][0][RTW89_FCC][0] = 76, [0][0][0][0][RTW89_ETSI][0] = 60, [0][0][0][0][RTW89_MKK][0] = 68, - [0][0][0][0][RTW89_IC][0] = 74, + [0][0][0][0][RTW89_IC][0] = 76, + [0][0][0][0][RTW89_KCC][0] = 68, [0][0][0][0][RTW89_ACMA][0] = 60, - [0][0][0][0][RTW89_FCC][1] = 70, + [0][0][0][0][RTW89_CN][0] = 58, + [0][0][0][0][RTW89_UK][0] = 60, + [0][0][0][0][RTW89_FCC][1] = 76, [0][0][0][0][RTW89_ETSI][1] = 60, [0][0][0][0][RTW89_MKK][1] = 68, - [0][0][0][0][RTW89_IC][1] = 74, + [0][0][0][0][RTW89_IC][1] = 76, + [0][0][0][0][RTW89_KCC][1] = 68, [0][0][0][0][RTW89_ACMA][1] = 60, - [0][0][0][0][RTW89_FCC][2] = 70, + [0][0][0][0][RTW89_CN][1] = 58, + [0][0][0][0][RTW89_UK][1] = 60, + [0][0][0][0][RTW89_FCC][2] = 76, [0][0][0][0][RTW89_ETSI][2] = 60, [0][0][0][0][RTW89_MKK][2] = 68, - [0][0][0][0][RTW89_IC][2] = 74, + [0][0][0][0][RTW89_IC][2] = 76, + [0][0][0][0][RTW89_KCC][2] = 68, [0][0][0][0][RTW89_ACMA][2] = 60, - [0][0][0][0][RTW89_FCC][3] = 70, + [0][0][0][0][RTW89_CN][2] = 58, + [0][0][0][0][RTW89_UK][2] = 60, + [0][0][0][0][RTW89_FCC][3] = 76, [0][0][0][0][RTW89_ETSI][3] = 60, [0][0][0][0][RTW89_MKK][3] = 68, - [0][0][0][0][RTW89_IC][3] = 74, + [0][0][0][0][RTW89_IC][3] = 76, + [0][0][0][0][RTW89_KCC][3] = 68, [0][0][0][0][RTW89_ACMA][3] = 60, - [0][0][0][0][RTW89_FCC][4] = 70, + [0][0][0][0][RTW89_CN][3] = 58, + [0][0][0][0][RTW89_UK][3] = 60, + [0][0][0][0][RTW89_FCC][4] = 76, [0][0][0][0][RTW89_ETSI][4] = 60, [0][0][0][0][RTW89_MKK][4] = 68, - [0][0][0][0][RTW89_IC][4] = 74, + [0][0][0][0][RTW89_IC][4] = 76, + [0][0][0][0][RTW89_KCC][4] = 68, [0][0][0][0][RTW89_ACMA][4] = 60, - [0][0][0][0][RTW89_FCC][5] = 70, + [0][0][0][0][RTW89_CN][4] = 58, + [0][0][0][0][RTW89_UK][4] = 60, + [0][0][0][0][RTW89_FCC][5] = 76, [0][0][0][0][RTW89_ETSI][5] = 60, [0][0][0][0][RTW89_MKK][5] = 68, - [0][0][0][0][RTW89_IC][5] = 74, + [0][0][0][0][RTW89_IC][5] = 76, + [0][0][0][0][RTW89_KCC][5] = 68, [0][0][0][0][RTW89_ACMA][5] = 60, - [0][0][0][0][RTW89_FCC][6] = 70, + [0][0][0][0][RTW89_CN][5] = 58, + [0][0][0][0][RTW89_UK][5] = 60, + [0][0][0][0][RTW89_FCC][6] = 76, [0][0][0][0][RTW89_ETSI][6] = 60, [0][0][0][0][RTW89_MKK][6] = 68, - [0][0][0][0][RTW89_IC][6] = 74, + [0][0][0][0][RTW89_IC][6] = 76, + [0][0][0][0][RTW89_KCC][6] = 68, [0][0][0][0][RTW89_ACMA][6] = 60, - [0][0][0][0][RTW89_FCC][7] = 70, + [0][0][0][0][RTW89_CN][6] = 58, + [0][0][0][0][RTW89_UK][6] = 60, + [0][0][0][0][RTW89_FCC][7] = 76, [0][0][0][0][RTW89_ETSI][7] = 60, [0][0][0][0][RTW89_MKK][7] = 68, - [0][0][0][0][RTW89_IC][7] = 74, + [0][0][0][0][RTW89_IC][7] = 76, + [0][0][0][0][RTW89_KCC][7] = 68, [0][0][0][0][RTW89_ACMA][7] = 60, - [0][0][0][0][RTW89_FCC][8] = 70, + [0][0][0][0][RTW89_CN][7] = 58, + [0][0][0][0][RTW89_UK][7] = 60, + [0][0][0][0][RTW89_FCC][8] = 76, [0][0][0][0][RTW89_ETSI][8] = 60, [0][0][0][0][RTW89_MKK][8] = 68, - [0][0][0][0][RTW89_IC][8] = 74, + [0][0][0][0][RTW89_IC][8] = 76, + [0][0][0][0][RTW89_KCC][8] = 68, [0][0][0][0][RTW89_ACMA][8] = 60, - [0][0][0][0][RTW89_FCC][9] = 70, + [0][0][0][0][RTW89_CN][8] = 58, + [0][0][0][0][RTW89_UK][8] = 60, + [0][0][0][0][RTW89_FCC][9] = 76, [0][0][0][0][RTW89_ETSI][9] = 60, [0][0][0][0][RTW89_MKK][9] = 68, - [0][0][0][0][RTW89_IC][9] = 74, + [0][0][0][0][RTW89_IC][9] = 76, + [0][0][0][0][RTW89_KCC][9] = 70, [0][0][0][0][RTW89_ACMA][9] = 60, - [0][0][0][0][RTW89_FCC][10] = 70, + [0][0][0][0][RTW89_CN][9] = 58, + [0][0][0][0][RTW89_UK][9] = 60, + [0][0][0][0][RTW89_FCC][10] = 76, [0][0][0][0][RTW89_ETSI][10] = 60, [0][0][0][0][RTW89_MKK][10] = 68, - [0][0][0][0][RTW89_IC][10] = 74, + [0][0][0][0][RTW89_IC][10] = 76, + [0][0][0][0][RTW89_KCC][10] = 70, [0][0][0][0][RTW89_ACMA][10] = 60, - [0][0][0][0][RTW89_FCC][11] = 62, + [0][0][0][0][RTW89_CN][10] = 58, + [0][0][0][0][RTW89_UK][10] = 60, + [0][0][0][0][RTW89_FCC][11] = 58, [0][0][0][0][RTW89_ETSI][11] = 60, [0][0][0][0][RTW89_MKK][11] = 68, - [0][0][0][0][RTW89_IC][11] = 72, + [0][0][0][0][RTW89_IC][11] = 58, + [0][0][0][0][RTW89_KCC][11] = 70, [0][0][0][0][RTW89_ACMA][11] = 60, - [0][0][0][0][RTW89_FCC][12] = 48, + [0][0][0][0][RTW89_CN][11] = 58, + [0][0][0][0][RTW89_UK][11] = 60, + [0][0][0][0][RTW89_FCC][12] = 46, [0][0][0][0][RTW89_ETSI][12] = 60, [0][0][0][0][RTW89_MKK][12] = 68, - [0][0][0][0][RTW89_IC][12] = 58, + [0][0][0][0][RTW89_IC][12] = 46, + [0][0][0][0][RTW89_KCC][12] = 70, [0][0][0][0][RTW89_ACMA][12] = 60, + [0][0][0][0][RTW89_CN][12] = 58, + [0][0][0][0][RTW89_UK][12] = 60, [0][0][0][0][RTW89_FCC][13] = 127, [0][0][0][0][RTW89_ETSI][13] = 127, [0][0][0][0][RTW89_MKK][13] = 72, [0][0][0][0][RTW89_IC][13] = 127, + [0][0][0][0][RTW89_KCC][13] = 127, [0][0][0][0][RTW89_ACMA][13] = 127, - [0][1][0][0][RTW89_FCC][0] = 66, + [0][0][0][0][RTW89_CN][13] = 127, + [0][0][0][0][RTW89_UK][13] = 127, + [0][1][0][0][RTW89_FCC][0] = 76, [0][1][0][0][RTW89_ETSI][0] = 48, [0][1][0][0][RTW89_MKK][0] = 58, - [0][1][0][0][RTW89_IC][0] = 74, + [0][1][0][0][RTW89_IC][0] = 76, + [0][1][0][0][RTW89_KCC][0] = 56, [0][1][0][0][RTW89_ACMA][0] = 48, - [0][1][0][0][RTW89_FCC][1] = 66, + [0][1][0][0][RTW89_CN][0] = 42, + [0][1][0][0][RTW89_UK][0] = 48, + [0][1][0][0][RTW89_FCC][1] = 76, [0][1][0][0][RTW89_ETSI][1] = 48, [0][1][0][0][RTW89_MKK][1] = 58, - [0][1][0][0][RTW89_IC][1] = 74, + [0][1][0][0][RTW89_IC][1] = 76, + [0][1][0][0][RTW89_KCC][1] = 56, [0][1][0][0][RTW89_ACMA][1] = 48, - [0][1][0][0][RTW89_FCC][2] = 66, + [0][1][0][0][RTW89_CN][1] = 42, + [0][1][0][0][RTW89_UK][1] = 48, + [0][1][0][0][RTW89_FCC][2] = 76, [0][1][0][0][RTW89_ETSI][2] = 48, [0][1][0][0][RTW89_MKK][2] = 58, - [0][1][0][0][RTW89_IC][2] = 74, + [0][1][0][0][RTW89_IC][2] = 76, + [0][1][0][0][RTW89_KCC][2] = 56, [0][1][0][0][RTW89_ACMA][2] = 48, - [0][1][0][0][RTW89_FCC][3] = 66, + [0][1][0][0][RTW89_CN][2] = 42, + [0][1][0][0][RTW89_UK][2] = 48, + [0][1][0][0][RTW89_FCC][3] = 76, [0][1][0][0][RTW89_ETSI][3] = 48, [0][1][0][0][RTW89_MKK][3] = 58, - [0][1][0][0][RTW89_IC][3] = 74, + [0][1][0][0][RTW89_IC][3] = 76, + [0][1][0][0][RTW89_KCC][3] = 56, [0][1][0][0][RTW89_ACMA][3] = 48, - [0][1][0][0][RTW89_FCC][4] = 66, + [0][1][0][0][RTW89_CN][3] = 42, + [0][1][0][0][RTW89_UK][3] = 48, + [0][1][0][0][RTW89_FCC][4] = 76, [0][1][0][0][RTW89_ETSI][4] = 48, [0][1][0][0][RTW89_MKK][4] = 58, - [0][1][0][0][RTW89_IC][4] = 74, + [0][1][0][0][RTW89_IC][4] = 76, + [0][1][0][0][RTW89_KCC][4] = 56, [0][1][0][0][RTW89_ACMA][4] = 48, - [0][1][0][0][RTW89_FCC][5] = 66, + [0][1][0][0][RTW89_CN][4] = 42, + [0][1][0][0][RTW89_UK][4] = 48, + [0][1][0][0][RTW89_FCC][5] = 76, [0][1][0][0][RTW89_ETSI][5] = 48, [0][1][0][0][RTW89_MKK][5] = 58, - [0][1][0][0][RTW89_IC][5] = 74, + [0][1][0][0][RTW89_IC][5] = 76, + [0][1][0][0][RTW89_KCC][5] = 56, [0][1][0][0][RTW89_ACMA][5] = 48, - [0][1][0][0][RTW89_FCC][6] = 66, + [0][1][0][0][RTW89_CN][5] = 42, + [0][1][0][0][RTW89_UK][5] = 48, + [0][1][0][0][RTW89_FCC][6] = 76, [0][1][0][0][RTW89_ETSI][6] = 48, [0][1][0][0][RTW89_MKK][6] = 58, - [0][1][0][0][RTW89_IC][6] = 74, + [0][1][0][0][RTW89_IC][6] = 76, + [0][1][0][0][RTW89_KCC][6] = 56, [0][1][0][0][RTW89_ACMA][6] = 48, - [0][1][0][0][RTW89_FCC][7] = 66, + [0][1][0][0][RTW89_CN][6] = 42, + [0][1][0][0][RTW89_UK][6] = 48, + [0][1][0][0][RTW89_FCC][7] = 76, [0][1][0][0][RTW89_ETSI][7] = 48, [0][1][0][0][RTW89_MKK][7] = 58, - [0][1][0][0][RTW89_IC][7] = 74, + [0][1][0][0][RTW89_IC][7] = 76, + [0][1][0][0][RTW89_KCC][7] = 56, [0][1][0][0][RTW89_ACMA][7] = 48, - [0][1][0][0][RTW89_FCC][8] = 66, + [0][1][0][0][RTW89_CN][7] = 42, + [0][1][0][0][RTW89_UK][7] = 48, + [0][1][0][0][RTW89_FCC][8] = 76, [0][1][0][0][RTW89_ETSI][8] = 48, [0][1][0][0][RTW89_MKK][8] = 58, - [0][1][0][0][RTW89_IC][8] = 74, + [0][1][0][0][RTW89_IC][8] = 76, + [0][1][0][0][RTW89_KCC][8] = 56, [0][1][0][0][RTW89_ACMA][8] = 48, - [0][1][0][0][RTW89_FCC][9] = 66, + [0][1][0][0][RTW89_CN][8] = 42, + [0][1][0][0][RTW89_UK][8] = 48, + [0][1][0][0][RTW89_FCC][9] = 70, [0][1][0][0][RTW89_ETSI][9] = 48, [0][1][0][0][RTW89_MKK][9] = 58, - [0][1][0][0][RTW89_IC][9] = 74, + [0][1][0][0][RTW89_IC][9] = 70, + [0][1][0][0][RTW89_KCC][9] = 56, [0][1][0][0][RTW89_ACMA][9] = 48, - [0][1][0][0][RTW89_FCC][10] = 66, + [0][1][0][0][RTW89_CN][9] = 42, + [0][1][0][0][RTW89_UK][9] = 48, + [0][1][0][0][RTW89_FCC][10] = 72, [0][1][0][0][RTW89_ETSI][10] = 48, [0][1][0][0][RTW89_MKK][10] = 58, - [0][1][0][0][RTW89_IC][10] = 74, + [0][1][0][0][RTW89_IC][10] = 72, + [0][1][0][0][RTW89_KCC][10] = 56, [0][1][0][0][RTW89_ACMA][10] = 48, - [0][1][0][0][RTW89_FCC][11] = 46, + [0][1][0][0][RTW89_CN][10] = 42, + [0][1][0][0][RTW89_UK][10] = 48, + [0][1][0][0][RTW89_FCC][11] = 44, [0][1][0][0][RTW89_ETSI][11] = 48, [0][1][0][0][RTW89_MKK][11] = 58, - [0][1][0][0][RTW89_IC][11] = 56, + [0][1][0][0][RTW89_IC][11] = 44, + [0][1][0][0][RTW89_KCC][11] = 56, [0][1][0][0][RTW89_ACMA][11] = 48, - [0][1][0][0][RTW89_FCC][12] = 34, + [0][1][0][0][RTW89_CN][11] = 42, + [0][1][0][0][RTW89_UK][11] = 48, + [0][1][0][0][RTW89_FCC][12] = 18, [0][1][0][0][RTW89_ETSI][12] = 48, [0][1][0][0][RTW89_MKK][12] = 58, - [0][1][0][0][RTW89_IC][12] = 44, + [0][1][0][0][RTW89_IC][12] = 18, + [0][1][0][0][RTW89_KCC][12] = 56, [0][1][0][0][RTW89_ACMA][12] = 48, + [0][1][0][0][RTW89_CN][12] = 42, + [0][1][0][0][RTW89_UK][12] = 48, [0][1][0][0][RTW89_FCC][13] = 127, [0][1][0][0][RTW89_ETSI][13] = 127, [0][1][0][0][RTW89_MKK][13] = 60, [0][1][0][0][RTW89_IC][13] = 127, + [0][1][0][0][RTW89_KCC][13] = 127, [0][1][0][0][RTW89_ACMA][13] = 127, + [0][1][0][0][RTW89_CN][13] = 127, + [0][1][0][0][RTW89_UK][13] = 127, [1][0][0][0][RTW89_FCC][0] = 127, [1][0][0][0][RTW89_ETSI][0] = 127, [1][0][0][0][RTW89_MKK][0] = 127, [1][0][0][0][RTW89_IC][0] = 127, + [1][0][0][0][RTW89_KCC][0] = 127, [1][0][0][0][RTW89_ACMA][0] = 127, + [1][0][0][0][RTW89_CN][0] = 127, + [1][0][0][0][RTW89_UK][0] = 127, [1][0][0][0][RTW89_FCC][1] = 127, [1][0][0][0][RTW89_ETSI][1] = 127, [1][0][0][0][RTW89_MKK][1] = 127, [1][0][0][0][RTW89_IC][1] = 127, + [1][0][0][0][RTW89_KCC][1] = 127, [1][0][0][0][RTW89_ACMA][1] = 127, - [1][0][0][0][RTW89_FCC][2] = 42, + [1][0][0][0][RTW89_CN][1] = 127, + [1][0][0][0][RTW89_UK][1] = 127, + [1][0][0][0][RTW89_FCC][2] = 44, [1][0][0][0][RTW89_ETSI][2] = 60, [1][0][0][0][RTW89_MKK][2] = 66, - [1][0][0][0][RTW89_IC][2] = 52, + [1][0][0][0][RTW89_IC][2] = 44, + [1][0][0][0][RTW89_KCC][2] = 68, [1][0][0][0][RTW89_ACMA][2] = 60, - [1][0][0][0][RTW89_FCC][3] = 42, + [1][0][0][0][RTW89_CN][2] = 58, + [1][0][0][0][RTW89_UK][2] = 60, + [1][0][0][0][RTW89_FCC][3] = 60, [1][0][0][0][RTW89_ETSI][3] = 60, [1][0][0][0][RTW89_MKK][3] = 66, - [1][0][0][0][RTW89_IC][3] = 52, + [1][0][0][0][RTW89_IC][3] = 60, + [1][0][0][0][RTW89_KCC][3] = 68, [1][0][0][0][RTW89_ACMA][3] = 60, - [1][0][0][0][RTW89_FCC][4] = 42, + [1][0][0][0][RTW89_CN][3] = 58, + [1][0][0][0][RTW89_UK][3] = 60, + [1][0][0][0][RTW89_FCC][4] = 60, [1][0][0][0][RTW89_ETSI][4] = 60, [1][0][0][0][RTW89_MKK][4] = 66, - [1][0][0][0][RTW89_IC][4] = 52, + [1][0][0][0][RTW89_IC][4] = 60, + [1][0][0][0][RTW89_KCC][4] = 68, [1][0][0][0][RTW89_ACMA][4] = 60, - [1][0][0][0][RTW89_FCC][5] = 58, + [1][0][0][0][RTW89_CN][4] = 58, + [1][0][0][0][RTW89_UK][4] = 60, + [1][0][0][0][RTW89_FCC][5] = 62, [1][0][0][0][RTW89_ETSI][5] = 60, [1][0][0][0][RTW89_MKK][5] = 66, - [1][0][0][0][RTW89_IC][5] = 68, + [1][0][0][0][RTW89_IC][5] = 62, + [1][0][0][0][RTW89_KCC][5] = 68, [1][0][0][0][RTW89_ACMA][5] = 60, - [1][0][0][0][RTW89_FCC][6] = 42, + [1][0][0][0][RTW89_CN][5] = 58, + [1][0][0][0][RTW89_UK][5] = 60, + [1][0][0][0][RTW89_FCC][6] = 46, [1][0][0][0][RTW89_ETSI][6] = 60, [1][0][0][0][RTW89_MKK][6] = 66, - [1][0][0][0][RTW89_IC][6] = 52, + [1][0][0][0][RTW89_IC][6] = 46, + [1][0][0][0][RTW89_KCC][6] = 68, [1][0][0][0][RTW89_ACMA][6] = 60, - [1][0][0][0][RTW89_FCC][7] = 42, + [1][0][0][0][RTW89_CN][6] = 58, + [1][0][0][0][RTW89_UK][6] = 60, + [1][0][0][0][RTW89_FCC][7] = 46, [1][0][0][0][RTW89_ETSI][7] = 60, [1][0][0][0][RTW89_MKK][7] = 66, - [1][0][0][0][RTW89_IC][7] = 52, + [1][0][0][0][RTW89_IC][7] = 46, + [1][0][0][0][RTW89_KCC][7] = 68, [1][0][0][0][RTW89_ACMA][7] = 60, - [1][0][0][0][RTW89_FCC][8] = 42, + [1][0][0][0][RTW89_CN][7] = 58, + [1][0][0][0][RTW89_UK][7] = 60, + [1][0][0][0][RTW89_FCC][8] = 28, [1][0][0][0][RTW89_ETSI][8] = 60, [1][0][0][0][RTW89_MKK][8] = 66, - [1][0][0][0][RTW89_IC][8] = 52, + [1][0][0][0][RTW89_IC][8] = 28, + [1][0][0][0][RTW89_KCC][8] = 70, [1][0][0][0][RTW89_ACMA][8] = 60, - [1][0][0][0][RTW89_FCC][9] = 34, + [1][0][0][0][RTW89_CN][8] = 58, + [1][0][0][0][RTW89_UK][8] = 60, + [1][0][0][0][RTW89_FCC][9] = 26, [1][0][0][0][RTW89_ETSI][9] = 60, [1][0][0][0][RTW89_MKK][9] = 66, - [1][0][0][0][RTW89_IC][9] = 44, + [1][0][0][0][RTW89_IC][9] = 26, + [1][0][0][0][RTW89_KCC][9] = 70, [1][0][0][0][RTW89_ACMA][9] = 60, - [1][0][0][0][RTW89_FCC][10] = 22, + [1][0][0][0][RTW89_CN][9] = 58, + [1][0][0][0][RTW89_UK][9] = 60, + [1][0][0][0][RTW89_FCC][10] = 26, [1][0][0][0][RTW89_ETSI][10] = 60, [1][0][0][0][RTW89_MKK][10] = 66, - [1][0][0][0][RTW89_IC][10] = 32, + [1][0][0][0][RTW89_IC][10] = 26, + [1][0][0][0][RTW89_KCC][10] = 70, [1][0][0][0][RTW89_ACMA][10] = 60, + [1][0][0][0][RTW89_CN][10] = 58, + [1][0][0][0][RTW89_UK][10] = 60, [1][0][0][0][RTW89_FCC][11] = 127, [1][0][0][0][RTW89_ETSI][11] = 127, [1][0][0][0][RTW89_MKK][11] = 127, [1][0][0][0][RTW89_IC][11] = 127, + [1][0][0][0][RTW89_KCC][11] = 127, [1][0][0][0][RTW89_ACMA][11] = 127, + [1][0][0][0][RTW89_CN][11] = 127, + [1][0][0][0][RTW89_UK][11] = 127, [1][0][0][0][RTW89_FCC][12] = 127, [1][0][0][0][RTW89_ETSI][12] = 127, [1][0][0][0][RTW89_MKK][12] = 127, [1][0][0][0][RTW89_IC][12] = 127, + [1][0][0][0][RTW89_KCC][12] = 127, [1][0][0][0][RTW89_ACMA][12] = 127, + [1][0][0][0][RTW89_CN][12] = 127, + [1][0][0][0][RTW89_UK][12] = 127, [1][0][0][0][RTW89_FCC][13] = 127, [1][0][0][0][RTW89_ETSI][13] = 127, [1][0][0][0][RTW89_MKK][13] = 127, [1][0][0][0][RTW89_IC][13] = 127, + [1][0][0][0][RTW89_KCC][13] = 127, [1][0][0][0][RTW89_ACMA][13] = 127, + [1][0][0][0][RTW89_CN][13] = 127, + [1][0][0][0][RTW89_UK][13] = 127, [1][1][0][0][RTW89_FCC][0] = 127, [1][1][0][0][RTW89_ETSI][0] = 127, [1][1][0][0][RTW89_MKK][0] = 127, [1][1][0][0][RTW89_IC][0] = 127, + [1][1][0][0][RTW89_KCC][0] = 127, [1][1][0][0][RTW89_ACMA][0] = 127, + [1][1][0][0][RTW89_CN][0] = 127, + [1][1][0][0][RTW89_UK][0] = 127, [1][1][0][0][RTW89_FCC][1] = 127, [1][1][0][0][RTW89_ETSI][1] = 127, [1][1][0][0][RTW89_MKK][1] = 127, [1][1][0][0][RTW89_IC][1] = 127, + [1][1][0][0][RTW89_KCC][1] = 127, [1][1][0][0][RTW89_ACMA][1] = 127, - [1][1][0][0][RTW89_FCC][2] = 38, + [1][1][0][0][RTW89_CN][1] = 127, + [1][1][0][0][RTW89_UK][1] = 127, + [1][1][0][0][RTW89_FCC][2] = 46, [1][1][0][0][RTW89_ETSI][2] = 48, [1][1][0][0][RTW89_MKK][2] = 58, - [1][1][0][0][RTW89_IC][2] = 48, + [1][1][0][0][RTW89_IC][2] = 46, + [1][1][0][0][RTW89_KCC][2] = 56, [1][1][0][0][RTW89_ACMA][2] = 48, - [1][1][0][0][RTW89_FCC][3] = 38, + [1][1][0][0][RTW89_CN][2] = 46, + [1][1][0][0][RTW89_UK][2] = 48, + [1][1][0][0][RTW89_FCC][3] = 46, [1][1][0][0][RTW89_ETSI][3] = 48, [1][1][0][0][RTW89_MKK][3] = 58, - [1][1][0][0][RTW89_IC][3] = 48, + [1][1][0][0][RTW89_IC][3] = 46, + [1][1][0][0][RTW89_KCC][3] = 56, [1][1][0][0][RTW89_ACMA][3] = 48, - [1][1][0][0][RTW89_FCC][4] = 38, + [1][1][0][0][RTW89_CN][3] = 46, + [1][1][0][0][RTW89_UK][3] = 48, + [1][1][0][0][RTW89_FCC][4] = 46, [1][1][0][0][RTW89_ETSI][4] = 48, [1][1][0][0][RTW89_MKK][4] = 58, - [1][1][0][0][RTW89_IC][4] = 48, + [1][1][0][0][RTW89_IC][4] = 46, + [1][1][0][0][RTW89_KCC][4] = 56, [1][1][0][0][RTW89_ACMA][4] = 48, - [1][1][0][0][RTW89_FCC][5] = 54, + [1][1][0][0][RTW89_CN][4] = 46, + [1][1][0][0][RTW89_UK][4] = 48, + [1][1][0][0][RTW89_FCC][5] = 48, [1][1][0][0][RTW89_ETSI][5] = 48, [1][1][0][0][RTW89_MKK][5] = 58, - [1][1][0][0][RTW89_IC][5] = 64, + [1][1][0][0][RTW89_IC][5] = 48, + [1][1][0][0][RTW89_KCC][5] = 56, [1][1][0][0][RTW89_ACMA][5] = 48, - [1][1][0][0][RTW89_FCC][6] = 26, + [1][1][0][0][RTW89_CN][5] = 46, + [1][1][0][0][RTW89_UK][5] = 48, + [1][1][0][0][RTW89_FCC][6] = 40, [1][1][0][0][RTW89_ETSI][6] = 48, [1][1][0][0][RTW89_MKK][6] = 58, - [1][1][0][0][RTW89_IC][6] = 36, + [1][1][0][0][RTW89_IC][6] = 40, + [1][1][0][0][RTW89_KCC][6] = 56, [1][1][0][0][RTW89_ACMA][6] = 48, - [1][1][0][0][RTW89_FCC][7] = 26, + [1][1][0][0][RTW89_CN][6] = 46, + [1][1][0][0][RTW89_UK][6] = 48, + [1][1][0][0][RTW89_FCC][7] = 40, [1][1][0][0][RTW89_ETSI][7] = 48, [1][1][0][0][RTW89_MKK][7] = 58, - [1][1][0][0][RTW89_IC][7] = 36, + [1][1][0][0][RTW89_IC][7] = 40, + [1][1][0][0][RTW89_KCC][7] = 56, [1][1][0][0][RTW89_ACMA][7] = 48, - [1][1][0][0][RTW89_FCC][8] = 26, + [1][1][0][0][RTW89_CN][7] = 46, + [1][1][0][0][RTW89_UK][7] = 48, + [1][1][0][0][RTW89_FCC][8] = 14, [1][1][0][0][RTW89_ETSI][8] = 48, [1][1][0][0][RTW89_MKK][8] = 58, - [1][1][0][0][RTW89_IC][8] = 36, + [1][1][0][0][RTW89_IC][8] = 14, + [1][1][0][0][RTW89_KCC][8] = 58, [1][1][0][0][RTW89_ACMA][8] = 48, - [1][1][0][0][RTW89_FCC][9] = 22, + [1][1][0][0][RTW89_CN][8] = 46, + [1][1][0][0][RTW89_UK][8] = 48, + [1][1][0][0][RTW89_FCC][9] = 14, [1][1][0][0][RTW89_ETSI][9] = 48, [1][1][0][0][RTW89_MKK][9] = 58, - [1][1][0][0][RTW89_IC][9] = 32, + [1][1][0][0][RTW89_IC][9] = 14, + [1][1][0][0][RTW89_KCC][9] = 58, [1][1][0][0][RTW89_ACMA][9] = 48, - [1][1][0][0][RTW89_FCC][10] = 22, + [1][1][0][0][RTW89_CN][9] = 46, + [1][1][0][0][RTW89_UK][9] = 48, + [1][1][0][0][RTW89_FCC][10] = 12, [1][1][0][0][RTW89_ETSI][10] = 48, [1][1][0][0][RTW89_MKK][10] = 56, - [1][1][0][0][RTW89_IC][10] = 32, + [1][1][0][0][RTW89_IC][10] = 12, + [1][1][0][0][RTW89_KCC][10] = 58, [1][1][0][0][RTW89_ACMA][10] = 48, + [1][1][0][0][RTW89_CN][10] = 46, + [1][1][0][0][RTW89_UK][10] = 48, [1][1][0][0][RTW89_FCC][11] = 127, [1][1][0][0][RTW89_ETSI][11] = 127, [1][1][0][0][RTW89_MKK][11] = 127, [1][1][0][0][RTW89_IC][11] = 127, + [1][1][0][0][RTW89_KCC][11] = 127, [1][1][0][0][RTW89_ACMA][11] = 127, + [1][1][0][0][RTW89_CN][11] = 127, + [1][1][0][0][RTW89_UK][11] = 127, [1][1][0][0][RTW89_FCC][12] = 127, [1][1][0][0][RTW89_ETSI][12] = 127, [1][1][0][0][RTW89_MKK][12] = 127, [1][1][0][0][RTW89_IC][12] = 127, + [1][1][0][0][RTW89_KCC][12] = 127, [1][1][0][0][RTW89_ACMA][12] = 127, + [1][1][0][0][RTW89_CN][12] = 127, + [1][1][0][0][RTW89_UK][12] = 127, [1][1][0][0][RTW89_FCC][13] = 127, [1][1][0][0][RTW89_ETSI][13] = 127, [1][1][0][0][RTW89_MKK][13] = 127, [1][1][0][0][RTW89_IC][13] = 127, + [1][1][0][0][RTW89_KCC][13] = 127, [1][1][0][0][RTW89_ACMA][13] = 127, - [0][0][1][0][RTW89_FCC][0] = 68, + [1][1][0][0][RTW89_CN][13] = 127, + [1][1][0][0][RTW89_UK][13] = 127, + [0][0][1][0][RTW89_FCC][0] = 66, [0][0][1][0][RTW89_ETSI][0] = 60, [0][0][1][0][RTW89_MKK][0] = 76, - [0][0][1][0][RTW89_IC][0] = 78, + [0][0][1][0][RTW89_IC][0] = 66, + [0][0][1][0][RTW89_KCC][0] = 68, [0][0][1][0][RTW89_ACMA][0] = 60, + [0][0][1][0][RTW89_CN][0] = 58, + [0][0][1][0][RTW89_UK][0] = 60, [0][0][1][0][RTW89_FCC][1] = 68, [0][0][1][0][RTW89_ETSI][1] = 60, [0][0][1][0][RTW89_MKK][1] = 78, - [0][0][1][0][RTW89_IC][1] = 78, + [0][0][1][0][RTW89_IC][1] = 68, + [0][0][1][0][RTW89_KCC][1] = 68, [0][0][1][0][RTW89_ACMA][1] = 60, - [0][0][1][0][RTW89_FCC][2] = 70, + [0][0][1][0][RTW89_CN][1] = 58, + [0][0][1][0][RTW89_UK][1] = 60, + [0][0][1][0][RTW89_FCC][2] = 72, [0][0][1][0][RTW89_ETSI][2] = 60, [0][0][1][0][RTW89_MKK][2] = 78, - [0][0][1][0][RTW89_IC][2] = 78, + [0][0][1][0][RTW89_IC][2] = 72, + [0][0][1][0][RTW89_KCC][2] = 68, [0][0][1][0][RTW89_ACMA][2] = 60, - [0][0][1][0][RTW89_FCC][3] = 70, + [0][0][1][0][RTW89_CN][2] = 58, + [0][0][1][0][RTW89_UK][2] = 60, + [0][0][1][0][RTW89_FCC][3] = 76, [0][0][1][0][RTW89_ETSI][3] = 60, [0][0][1][0][RTW89_MKK][3] = 78, - [0][0][1][0][RTW89_IC][3] = 78, + [0][0][1][0][RTW89_IC][3] = 76, + [0][0][1][0][RTW89_KCC][3] = 68, [0][0][1][0][RTW89_ACMA][3] = 60, - [0][0][1][0][RTW89_FCC][4] = 70, + [0][0][1][0][RTW89_CN][3] = 58, + [0][0][1][0][RTW89_UK][3] = 60, + [0][0][1][0][RTW89_FCC][4] = 80, [0][0][1][0][RTW89_ETSI][4] = 60, [0][0][1][0][RTW89_MKK][4] = 78, - [0][0][1][0][RTW89_IC][4] = 78, + [0][0][1][0][RTW89_IC][4] = 80, + [0][0][1][0][RTW89_KCC][4] = 76, [0][0][1][0][RTW89_ACMA][4] = 60, - [0][0][1][0][RTW89_FCC][5] = 70, + [0][0][1][0][RTW89_CN][4] = 58, + [0][0][1][0][RTW89_UK][4] = 60, + [0][0][1][0][RTW89_FCC][5] = 80, [0][0][1][0][RTW89_ETSI][5] = 60, [0][0][1][0][RTW89_MKK][5] = 78, - [0][0][1][0][RTW89_IC][5] = 78, + [0][0][1][0][RTW89_IC][5] = 80, + [0][0][1][0][RTW89_KCC][5] = 76, [0][0][1][0][RTW89_ACMA][5] = 60, - [0][0][1][0][RTW89_FCC][6] = 70, + [0][0][1][0][RTW89_CN][5] = 58, + [0][0][1][0][RTW89_UK][5] = 60, + [0][0][1][0][RTW89_FCC][6] = 80, [0][0][1][0][RTW89_ETSI][6] = 60, [0][0][1][0][RTW89_MKK][6] = 76, - [0][0][1][0][RTW89_IC][6] = 78, + [0][0][1][0][RTW89_IC][6] = 80, + [0][0][1][0][RTW89_KCC][6] = 76, [0][0][1][0][RTW89_ACMA][6] = 60, - [0][0][1][0][RTW89_FCC][7] = 70, + [0][0][1][0][RTW89_CN][6] = 58, + [0][0][1][0][RTW89_UK][6] = 60, + [0][0][1][0][RTW89_FCC][7] = 80, [0][0][1][0][RTW89_ETSI][7] = 60, [0][0][1][0][RTW89_MKK][7] = 78, - [0][0][1][0][RTW89_IC][7] = 78, + [0][0][1][0][RTW89_IC][7] = 80, + [0][0][1][0][RTW89_KCC][7] = 76, [0][0][1][0][RTW89_ACMA][7] = 60, - [0][0][1][0][RTW89_FCC][8] = 70, + [0][0][1][0][RTW89_CN][7] = 58, + [0][0][1][0][RTW89_UK][7] = 60, + [0][0][1][0][RTW89_FCC][8] = 80, [0][0][1][0][RTW89_ETSI][8] = 60, [0][0][1][0][RTW89_MKK][8] = 78, - [0][0][1][0][RTW89_IC][8] = 78, + [0][0][1][0][RTW89_IC][8] = 80, + [0][0][1][0][RTW89_KCC][8] = 76, [0][0][1][0][RTW89_ACMA][8] = 60, - [0][0][1][0][RTW89_FCC][9] = 66, + [0][0][1][0][RTW89_CN][8] = 58, + [0][0][1][0][RTW89_UK][8] = 60, + [0][0][1][0][RTW89_FCC][9] = 76, [0][0][1][0][RTW89_ETSI][9] = 60, [0][0][1][0][RTW89_MKK][9] = 78, [0][0][1][0][RTW89_IC][9] = 76, + [0][0][1][0][RTW89_KCC][9] = 70, [0][0][1][0][RTW89_ACMA][9] = 60, + [0][0][1][0][RTW89_CN][9] = 58, + [0][0][1][0][RTW89_UK][9] = 60, [0][0][1][0][RTW89_FCC][10] = 66, [0][0][1][0][RTW89_ETSI][10] = 60, [0][0][1][0][RTW89_MKK][10] = 78, - [0][0][1][0][RTW89_IC][10] = 76, + [0][0][1][0][RTW89_IC][10] = 66, + [0][0][1][0][RTW89_KCC][10] = 70, [0][0][1][0][RTW89_ACMA][10] = 60, - [0][0][1][0][RTW89_FCC][11] = 46, + [0][0][1][0][RTW89_CN][10] = 58, + [0][0][1][0][RTW89_UK][10] = 60, + [0][0][1][0][RTW89_FCC][11] = 62, [0][0][1][0][RTW89_ETSI][11] = 60, [0][0][1][0][RTW89_MKK][11] = 78, - [0][0][1][0][RTW89_IC][11] = 56, + [0][0][1][0][RTW89_IC][11] = 62, + [0][0][1][0][RTW89_KCC][11] = 70, [0][0][1][0][RTW89_ACMA][11] = 60, - [0][0][1][0][RTW89_FCC][12] = 42, + [0][0][1][0][RTW89_CN][11] = 58, + [0][0][1][0][RTW89_UK][11] = 60, + [0][0][1][0][RTW89_FCC][12] = 60, [0][0][1][0][RTW89_ETSI][12] = 60, [0][0][1][0][RTW89_MKK][12] = 78, - [0][0][1][0][RTW89_IC][12] = 52, + [0][0][1][0][RTW89_IC][12] = 60, + [0][0][1][0][RTW89_KCC][12] = 70, [0][0][1][0][RTW89_ACMA][12] = 60, + [0][0][1][0][RTW89_CN][12] = 58, + [0][0][1][0][RTW89_UK][12] = 60, [0][0][1][0][RTW89_FCC][13] = 127, [0][0][1][0][RTW89_ETSI][13] = 127, [0][0][1][0][RTW89_MKK][13] = 127, [0][0][1][0][RTW89_IC][13] = 127, + [0][0][1][0][RTW89_KCC][13] = 127, [0][0][1][0][RTW89_ACMA][13] = 127, - [0][1][1][0][RTW89_FCC][0] = 54, + [0][0][1][0][RTW89_CN][13] = 127, + [0][0][1][0][RTW89_UK][13] = 127, + [0][1][1][0][RTW89_FCC][0] = 66, [0][1][1][0][RTW89_ETSI][0] = 48, [0][1][1][0][RTW89_MKK][0] = 66, - [0][1][1][0][RTW89_IC][0] = 64, + [0][1][1][0][RTW89_IC][0] = 66, + [0][1][1][0][RTW89_KCC][0] = 64, [0][1][1][0][RTW89_ACMA][0] = 48, - [0][1][1][0][RTW89_FCC][1] = 54, + [0][1][1][0][RTW89_CN][0] = 46, + [0][1][1][0][RTW89_UK][0] = 48, + [0][1][1][0][RTW89_FCC][1] = 68, [0][1][1][0][RTW89_ETSI][1] = 48, [0][1][1][0][RTW89_MKK][1] = 66, - [0][1][1][0][RTW89_IC][1] = 64, + [0][1][1][0][RTW89_IC][1] = 68, + [0][1][1][0][RTW89_KCC][1] = 64, [0][1][1][0][RTW89_ACMA][1] = 48, - [0][1][1][0][RTW89_FCC][2] = 58, + [0][1][1][0][RTW89_CN][1] = 46, + [0][1][1][0][RTW89_UK][1] = 48, + [0][1][1][0][RTW89_FCC][2] = 72, [0][1][1][0][RTW89_ETSI][2] = 48, [0][1][1][0][RTW89_MKK][2] = 66, - [0][1][1][0][RTW89_IC][2] = 68, + [0][1][1][0][RTW89_IC][2] = 72, + [0][1][1][0][RTW89_KCC][2] = 64, [0][1][1][0][RTW89_ACMA][2] = 48, - [0][1][1][0][RTW89_FCC][3] = 62, + [0][1][1][0][RTW89_CN][2] = 46, + [0][1][1][0][RTW89_UK][2] = 48, + [0][1][1][0][RTW89_FCC][3] = 76, [0][1][1][0][RTW89_ETSI][3] = 48, [0][1][1][0][RTW89_MKK][3] = 66, - [0][1][1][0][RTW89_IC][3] = 72, + [0][1][1][0][RTW89_IC][3] = 76, + [0][1][1][0][RTW89_KCC][3] = 64, [0][1][1][0][RTW89_ACMA][3] = 48, - [0][1][1][0][RTW89_FCC][4] = 70, + [0][1][1][0][RTW89_CN][3] = 46, + [0][1][1][0][RTW89_UK][3] = 48, + [0][1][1][0][RTW89_FCC][4] = 80, [0][1][1][0][RTW89_ETSI][4] = 48, [0][1][1][0][RTW89_MKK][4] = 66, - [0][1][1][0][RTW89_IC][4] = 78, + [0][1][1][0][RTW89_IC][4] = 80, + [0][1][1][0][RTW89_KCC][4] = 66, [0][1][1][0][RTW89_ACMA][4] = 48, - [0][1][1][0][RTW89_FCC][5] = 70, + [0][1][1][0][RTW89_CN][4] = 46, + [0][1][1][0][RTW89_UK][4] = 48, + [0][1][1][0][RTW89_FCC][5] = 80, [0][1][1][0][RTW89_ETSI][5] = 48, [0][1][1][0][RTW89_MKK][5] = 66, - [0][1][1][0][RTW89_IC][5] = 78, + [0][1][1][0][RTW89_IC][5] = 80, + [0][1][1][0][RTW89_KCC][5] = 66, [0][1][1][0][RTW89_ACMA][5] = 48, - [0][1][1][0][RTW89_FCC][6] = 70, + [0][1][1][0][RTW89_CN][5] = 46, + [0][1][1][0][RTW89_UK][5] = 48, + [0][1][1][0][RTW89_FCC][6] = 80, [0][1][1][0][RTW89_ETSI][6] = 48, [0][1][1][0][RTW89_MKK][6] = 66, - [0][1][1][0][RTW89_IC][6] = 78, + [0][1][1][0][RTW89_IC][6] = 80, + [0][1][1][0][RTW89_KCC][6] = 66, [0][1][1][0][RTW89_ACMA][6] = 48, - [0][1][1][0][RTW89_FCC][7] = 62, + [0][1][1][0][RTW89_CN][6] = 46, + [0][1][1][0][RTW89_UK][6] = 48, + [0][1][1][0][RTW89_FCC][7] = 78, [0][1][1][0][RTW89_ETSI][7] = 48, [0][1][1][0][RTW89_MKK][7] = 66, - [0][1][1][0][RTW89_IC][7] = 72, + [0][1][1][0][RTW89_IC][7] = 78, + [0][1][1][0][RTW89_KCC][7] = 66, [0][1][1][0][RTW89_ACMA][7] = 48, - [0][1][1][0][RTW89_FCC][8] = 58, + [0][1][1][0][RTW89_CN][7] = 46, + [0][1][1][0][RTW89_UK][7] = 48, + [0][1][1][0][RTW89_FCC][8] = 74, [0][1][1][0][RTW89_ETSI][8] = 48, [0][1][1][0][RTW89_MKK][8] = 66, - [0][1][1][0][RTW89_IC][8] = 68, + [0][1][1][0][RTW89_IC][8] = 74, + [0][1][1][0][RTW89_KCC][8] = 66, [0][1][1][0][RTW89_ACMA][8] = 48, - [0][1][1][0][RTW89_FCC][9] = 54, + [0][1][1][0][RTW89_CN][8] = 46, + [0][1][1][0][RTW89_UK][8] = 48, + [0][1][1][0][RTW89_FCC][9] = 70, [0][1][1][0][RTW89_ETSI][9] = 48, [0][1][1][0][RTW89_MKK][9] = 66, - [0][1][1][0][RTW89_IC][9] = 64, + [0][1][1][0][RTW89_IC][9] = 70, + [0][1][1][0][RTW89_KCC][9] = 64, [0][1][1][0][RTW89_ACMA][9] = 48, - [0][1][1][0][RTW89_FCC][10] = 54, + [0][1][1][0][RTW89_CN][9] = 46, + [0][1][1][0][RTW89_UK][9] = 48, + [0][1][1][0][RTW89_FCC][10] = 62, [0][1][1][0][RTW89_ETSI][10] = 48, [0][1][1][0][RTW89_MKK][10] = 66, - [0][1][1][0][RTW89_IC][10] = 64, + [0][1][1][0][RTW89_IC][10] = 62, + [0][1][1][0][RTW89_KCC][10] = 64, [0][1][1][0][RTW89_ACMA][10] = 48, - [0][1][1][0][RTW89_FCC][11] = 38, + [0][1][1][0][RTW89_CN][10] = 46, + [0][1][1][0][RTW89_UK][10] = 48, + [0][1][1][0][RTW89_FCC][11] = 60, [0][1][1][0][RTW89_ETSI][11] = 48, [0][1][1][0][RTW89_MKK][11] = 66, - [0][1][1][0][RTW89_IC][11] = 48, + [0][1][1][0][RTW89_IC][11] = 60, + [0][1][1][0][RTW89_KCC][11] = 64, [0][1][1][0][RTW89_ACMA][11] = 48, - [0][1][1][0][RTW89_FCC][12] = 34, + [0][1][1][0][RTW89_CN][11] = 46, + [0][1][1][0][RTW89_UK][11] = 48, + [0][1][1][0][RTW89_FCC][12] = 36, [0][1][1][0][RTW89_ETSI][12] = 48, [0][1][1][0][RTW89_MKK][12] = 66, - [0][1][1][0][RTW89_IC][12] = 44, + [0][1][1][0][RTW89_IC][12] = 36, + [0][1][1][0][RTW89_KCC][12] = 64, [0][1][1][0][RTW89_ACMA][12] = 48, + [0][1][1][0][RTW89_CN][12] = 46, + [0][1][1][0][RTW89_UK][12] = 48, [0][1][1][0][RTW89_FCC][13] = 127, [0][1][1][0][RTW89_ETSI][13] = 127, [0][1][1][0][RTW89_MKK][13] = 127, [0][1][1][0][RTW89_IC][13] = 127, + [0][1][1][0][RTW89_KCC][13] = 127, [0][1][1][0][RTW89_ACMA][13] = 127, - [0][0][2][0][RTW89_FCC][0] = 68, + [0][1][1][0][RTW89_CN][13] = 127, + [0][1][1][0][RTW89_UK][13] = 127, + [0][0][2][0][RTW89_FCC][0] = 66, [0][0][2][0][RTW89_ETSI][0] = 60, [0][0][2][0][RTW89_MKK][0] = 78, - [0][0][2][0][RTW89_IC][0] = 78, + [0][0][2][0][RTW89_IC][0] = 66, + [0][0][2][0][RTW89_KCC][0] = 70, [0][0][2][0][RTW89_ACMA][0] = 60, - [0][0][2][0][RTW89_FCC][1] = 68, + [0][0][2][0][RTW89_CN][0] = 58, + [0][0][2][0][RTW89_UK][0] = 60, + [0][0][2][0][RTW89_FCC][1] = 70, [0][0][2][0][RTW89_ETSI][1] = 60, [0][0][2][0][RTW89_MKK][1] = 78, - [0][0][2][0][RTW89_IC][1] = 78, + [0][0][2][0][RTW89_IC][1] = 70, + [0][0][2][0][RTW89_KCC][1] = 70, [0][0][2][0][RTW89_ACMA][1] = 60, - [0][0][2][0][RTW89_FCC][2] = 70, + [0][0][2][0][RTW89_CN][1] = 58, + [0][0][2][0][RTW89_UK][1] = 60, + [0][0][2][0][RTW89_FCC][2] = 74, [0][0][2][0][RTW89_ETSI][2] = 60, [0][0][2][0][RTW89_MKK][2] = 78, - [0][0][2][0][RTW89_IC][2] = 78, + [0][0][2][0][RTW89_IC][2] = 74, + [0][0][2][0][RTW89_KCC][2] = 70, [0][0][2][0][RTW89_ACMA][2] = 60, - [0][0][2][0][RTW89_FCC][3] = 70, + [0][0][2][0][RTW89_CN][2] = 58, + [0][0][2][0][RTW89_UK][2] = 60, + [0][0][2][0][RTW89_FCC][3] = 78, [0][0][2][0][RTW89_ETSI][3] = 60, [0][0][2][0][RTW89_MKK][3] = 78, [0][0][2][0][RTW89_IC][3] = 78, + [0][0][2][0][RTW89_KCC][3] = 70, [0][0][2][0][RTW89_ACMA][3] = 60, - [0][0][2][0][RTW89_FCC][4] = 70, + [0][0][2][0][RTW89_CN][3] = 58, + [0][0][2][0][RTW89_UK][3] = 60, + [0][0][2][0][RTW89_FCC][4] = 80, [0][0][2][0][RTW89_ETSI][4] = 60, [0][0][2][0][RTW89_MKK][4] = 78, - [0][0][2][0][RTW89_IC][4] = 78, + [0][0][2][0][RTW89_IC][4] = 80, + [0][0][2][0][RTW89_KCC][4] = 78, [0][0][2][0][RTW89_ACMA][4] = 60, - [0][0][2][0][RTW89_FCC][5] = 70, + [0][0][2][0][RTW89_CN][4] = 58, + [0][0][2][0][RTW89_UK][4] = 60, + [0][0][2][0][RTW89_FCC][5] = 80, [0][0][2][0][RTW89_ETSI][5] = 60, [0][0][2][0][RTW89_MKK][5] = 78, - [0][0][2][0][RTW89_IC][5] = 78, + [0][0][2][0][RTW89_IC][5] = 80, + [0][0][2][0][RTW89_KCC][5] = 78, [0][0][2][0][RTW89_ACMA][5] = 60, - [0][0][2][0][RTW89_FCC][6] = 70, + [0][0][2][0][RTW89_CN][5] = 58, + [0][0][2][0][RTW89_UK][5] = 60, + [0][0][2][0][RTW89_FCC][6] = 80, [0][0][2][0][RTW89_ETSI][6] = 60, [0][0][2][0][RTW89_MKK][6] = 78, - [0][0][2][0][RTW89_IC][6] = 78, + [0][0][2][0][RTW89_IC][6] = 80, + [0][0][2][0][RTW89_KCC][6] = 78, [0][0][2][0][RTW89_ACMA][6] = 60, - [0][0][2][0][RTW89_FCC][7] = 70, + [0][0][2][0][RTW89_CN][6] = 58, + [0][0][2][0][RTW89_UK][6] = 60, + [0][0][2][0][RTW89_FCC][7] = 80, [0][0][2][0][RTW89_ETSI][7] = 60, [0][0][2][0][RTW89_MKK][7] = 78, - [0][0][2][0][RTW89_IC][7] = 78, + [0][0][2][0][RTW89_IC][7] = 80, + [0][0][2][0][RTW89_KCC][7] = 78, [0][0][2][0][RTW89_ACMA][7] = 60, - [0][0][2][0][RTW89_FCC][8] = 68, + [0][0][2][0][RTW89_CN][7] = 58, + [0][0][2][0][RTW89_UK][7] = 60, + [0][0][2][0][RTW89_FCC][8] = 78, [0][0][2][0][RTW89_ETSI][8] = 60, [0][0][2][0][RTW89_MKK][8] = 78, [0][0][2][0][RTW89_IC][8] = 78, + [0][0][2][0][RTW89_KCC][8] = 78, [0][0][2][0][RTW89_ACMA][8] = 60, - [0][0][2][0][RTW89_FCC][9] = 64, + [0][0][2][0][RTW89_CN][8] = 58, + [0][0][2][0][RTW89_UK][8] = 60, + [0][0][2][0][RTW89_FCC][9] = 74, [0][0][2][0][RTW89_ETSI][9] = 60, [0][0][2][0][RTW89_MKK][9] = 78, [0][0][2][0][RTW89_IC][9] = 74, + [0][0][2][0][RTW89_KCC][9] = 66, [0][0][2][0][RTW89_ACMA][9] = 60, - [0][0][2][0][RTW89_FCC][10] = 64, + [0][0][2][0][RTW89_CN][9] = 58, + [0][0][2][0][RTW89_UK][9] = 60, + [0][0][2][0][RTW89_FCC][10] = 62, [0][0][2][0][RTW89_ETSI][10] = 60, [0][0][2][0][RTW89_MKK][10] = 78, - [0][0][2][0][RTW89_IC][10] = 74, + [0][0][2][0][RTW89_IC][10] = 62, + [0][0][2][0][RTW89_KCC][10] = 66, [0][0][2][0][RTW89_ACMA][10] = 60, - [0][0][2][0][RTW89_FCC][11] = 46, + [0][0][2][0][RTW89_CN][10] = 58, + [0][0][2][0][RTW89_UK][10] = 60, + [0][0][2][0][RTW89_FCC][11] = 60, [0][0][2][0][RTW89_ETSI][11] = 60, [0][0][2][0][RTW89_MKK][11] = 78, - [0][0][2][0][RTW89_IC][11] = 56, + [0][0][2][0][RTW89_IC][11] = 60, + [0][0][2][0][RTW89_KCC][11] = 66, [0][0][2][0][RTW89_ACMA][11] = 60, - [0][0][2][0][RTW89_FCC][12] = 42, + [0][0][2][0][RTW89_CN][11] = 58, + [0][0][2][0][RTW89_UK][11] = 60, + [0][0][2][0][RTW89_FCC][12] = 38, [0][0][2][0][RTW89_ETSI][12] = 60, [0][0][2][0][RTW89_MKK][12] = 78, - [0][0][2][0][RTW89_IC][12] = 52, + [0][0][2][0][RTW89_IC][12] = 38, + [0][0][2][0][RTW89_KCC][12] = 66, [0][0][2][0][RTW89_ACMA][12] = 60, + [0][0][2][0][RTW89_CN][12] = 58, + [0][0][2][0][RTW89_UK][12] = 60, [0][0][2][0][RTW89_FCC][13] = 127, [0][0][2][0][RTW89_ETSI][13] = 127, [0][0][2][0][RTW89_MKK][13] = 127, [0][0][2][0][RTW89_IC][13] = 127, + [0][0][2][0][RTW89_KCC][13] = 127, [0][0][2][0][RTW89_ACMA][13] = 127, - [0][1][2][0][RTW89_FCC][0] = 50, + [0][0][2][0][RTW89_CN][13] = 127, + [0][0][2][0][RTW89_UK][13] = 127, + [0][1][2][0][RTW89_FCC][0] = 64, [0][1][2][0][RTW89_ETSI][0] = 48, [0][1][2][0][RTW89_MKK][0] = 68, - [0][1][2][0][RTW89_IC][0] = 60, + [0][1][2][0][RTW89_IC][0] = 64, + [0][1][2][0][RTW89_KCC][0] = 66, [0][1][2][0][RTW89_ACMA][0] = 48, - [0][1][2][0][RTW89_FCC][1] = 50, + [0][1][2][0][RTW89_CN][0] = 46, + [0][1][2][0][RTW89_UK][0] = 48, + [0][1][2][0][RTW89_FCC][1] = 70, [0][1][2][0][RTW89_ETSI][1] = 48, [0][1][2][0][RTW89_MKK][1] = 68, - [0][1][2][0][RTW89_IC][1] = 60, + [0][1][2][0][RTW89_IC][1] = 70, + [0][1][2][0][RTW89_KCC][1] = 66, [0][1][2][0][RTW89_ACMA][1] = 48, - [0][1][2][0][RTW89_FCC][2] = 54, + [0][1][2][0][RTW89_CN][1] = 46, + [0][1][2][0][RTW89_UK][1] = 48, + [0][1][2][0][RTW89_FCC][2] = 74, [0][1][2][0][RTW89_ETSI][2] = 48, [0][1][2][0][RTW89_MKK][2] = 68, - [0][1][2][0][RTW89_IC][2] = 64, + [0][1][2][0][RTW89_IC][2] = 74, + [0][1][2][0][RTW89_KCC][2] = 66, [0][1][2][0][RTW89_ACMA][2] = 48, - [0][1][2][0][RTW89_FCC][3] = 58, + [0][1][2][0][RTW89_CN][2] = 46, + [0][1][2][0][RTW89_UK][2] = 48, + [0][1][2][0][RTW89_FCC][3] = 78, [0][1][2][0][RTW89_ETSI][3] = 48, [0][1][2][0][RTW89_MKK][3] = 68, - [0][1][2][0][RTW89_IC][3] = 68, + [0][1][2][0][RTW89_IC][3] = 78, + [0][1][2][0][RTW89_KCC][3] = 66, [0][1][2][0][RTW89_ACMA][3] = 48, - [0][1][2][0][RTW89_FCC][4] = 64, + [0][1][2][0][RTW89_CN][3] = 46, + [0][1][2][0][RTW89_UK][3] = 48, + [0][1][2][0][RTW89_FCC][4] = 80, [0][1][2][0][RTW89_ETSI][4] = 48, [0][1][2][0][RTW89_MKK][4] = 68, - [0][1][2][0][RTW89_IC][4] = 74, + [0][1][2][0][RTW89_IC][4] = 80, + [0][1][2][0][RTW89_KCC][4] = 66, [0][1][2][0][RTW89_ACMA][4] = 48, - [0][1][2][0][RTW89_FCC][5] = 70, + [0][1][2][0][RTW89_CN][4] = 46, + [0][1][2][0][RTW89_UK][4] = 48, + [0][1][2][0][RTW89_FCC][5] = 80, [0][1][2][0][RTW89_ETSI][5] = 48, [0][1][2][0][RTW89_MKK][5] = 68, - [0][1][2][0][RTW89_IC][5] = 78, + [0][1][2][0][RTW89_IC][5] = 80, + [0][1][2][0][RTW89_KCC][5] = 66, [0][1][2][0][RTW89_ACMA][5] = 48, - [0][1][2][0][RTW89_FCC][6] = 66, + [0][1][2][0][RTW89_CN][5] = 46, + [0][1][2][0][RTW89_UK][5] = 48, + [0][1][2][0][RTW89_FCC][6] = 80, [0][1][2][0][RTW89_ETSI][6] = 48, [0][1][2][0][RTW89_MKK][6] = 68, - [0][1][2][0][RTW89_IC][6] = 76, + [0][1][2][0][RTW89_IC][6] = 80, + [0][1][2][0][RTW89_KCC][6] = 66, [0][1][2][0][RTW89_ACMA][6] = 48, - [0][1][2][0][RTW89_FCC][7] = 58, + [0][1][2][0][RTW89_CN][6] = 46, + [0][1][2][0][RTW89_UK][6] = 48, + [0][1][2][0][RTW89_FCC][7] = 74, [0][1][2][0][RTW89_ETSI][7] = 48, [0][1][2][0][RTW89_MKK][7] = 68, - [0][1][2][0][RTW89_IC][7] = 68, + [0][1][2][0][RTW89_IC][7] = 74, + [0][1][2][0][RTW89_KCC][7] = 66, [0][1][2][0][RTW89_ACMA][7] = 48, - [0][1][2][0][RTW89_FCC][8] = 54, + [0][1][2][0][RTW89_CN][7] = 46, + [0][1][2][0][RTW89_UK][7] = 48, + [0][1][2][0][RTW89_FCC][8] = 70, [0][1][2][0][RTW89_ETSI][8] = 48, [0][1][2][0][RTW89_MKK][8] = 68, - [0][1][2][0][RTW89_IC][8] = 64, + [0][1][2][0][RTW89_IC][8] = 70, + [0][1][2][0][RTW89_KCC][8] = 66, [0][1][2][0][RTW89_ACMA][8] = 48, - [0][1][2][0][RTW89_FCC][9] = 50, + [0][1][2][0][RTW89_CN][8] = 46, + [0][1][2][0][RTW89_UK][8] = 48, + [0][1][2][0][RTW89_FCC][9] = 66, [0][1][2][0][RTW89_ETSI][9] = 48, [0][1][2][0][RTW89_MKK][9] = 68, - [0][1][2][0][RTW89_IC][9] = 60, + [0][1][2][0][RTW89_IC][9] = 66, + [0][1][2][0][RTW89_KCC][9] = 64, [0][1][2][0][RTW89_ACMA][9] = 48, - [0][1][2][0][RTW89_FCC][10] = 50, + [0][1][2][0][RTW89_CN][9] = 46, + [0][1][2][0][RTW89_UK][9] = 48, + [0][1][2][0][RTW89_FCC][10] = 58, [0][1][2][0][RTW89_ETSI][10] = 48, [0][1][2][0][RTW89_MKK][10] = 68, - [0][1][2][0][RTW89_IC][10] = 60, + [0][1][2][0][RTW89_IC][10] = 58, + [0][1][2][0][RTW89_KCC][10] = 64, [0][1][2][0][RTW89_ACMA][10] = 48, - [0][1][2][0][RTW89_FCC][11] = 38, + [0][1][2][0][RTW89_CN][10] = 46, + [0][1][2][0][RTW89_UK][10] = 48, + [0][1][2][0][RTW89_FCC][11] = 58, [0][1][2][0][RTW89_ETSI][11] = 48, [0][1][2][0][RTW89_MKK][11] = 68, - [0][1][2][0][RTW89_IC][11] = 48, + [0][1][2][0][RTW89_IC][11] = 58, + [0][1][2][0][RTW89_KCC][11] = 64, [0][1][2][0][RTW89_ACMA][11] = 48, - [0][1][2][0][RTW89_FCC][12] = 34, + [0][1][2][0][RTW89_CN][11] = 46, + [0][1][2][0][RTW89_UK][11] = 48, + [0][1][2][0][RTW89_FCC][12] = 16, [0][1][2][0][RTW89_ETSI][12] = 48, [0][1][2][0][RTW89_MKK][12] = 68, - [0][1][2][0][RTW89_IC][12] = 44, + [0][1][2][0][RTW89_IC][12] = 16, + [0][1][2][0][RTW89_KCC][12] = 64, [0][1][2][0][RTW89_ACMA][12] = 48, + [0][1][2][0][RTW89_CN][12] = 46, + [0][1][2][0][RTW89_UK][12] = 48, [0][1][2][0][RTW89_FCC][13] = 127, [0][1][2][0][RTW89_ETSI][13] = 127, [0][1][2][0][RTW89_MKK][13] = 127, [0][1][2][0][RTW89_IC][13] = 127, + [0][1][2][0][RTW89_KCC][13] = 127, [0][1][2][0][RTW89_ACMA][13] = 127, - [0][1][2][1][RTW89_FCC][0] = 50, + [0][1][2][0][RTW89_CN][13] = 127, + [0][1][2][0][RTW89_UK][13] = 127, + [0][1][2][1][RTW89_FCC][0] = 64, [0][1][2][1][RTW89_ETSI][0] = 36, [0][1][2][1][RTW89_MKK][0] = 68, - [0][1][2][1][RTW89_IC][0] = 60, + [0][1][2][1][RTW89_IC][0] = 64, + [0][1][2][1][RTW89_KCC][0] = 66, [0][1][2][1][RTW89_ACMA][0] = 36, - [0][1][2][1][RTW89_FCC][1] = 50, + [0][1][2][1][RTW89_CN][0] = 36, + [0][1][2][1][RTW89_UK][0] = 36, + [0][1][2][1][RTW89_FCC][1] = 70, [0][1][2][1][RTW89_ETSI][1] = 36, [0][1][2][1][RTW89_MKK][1] = 68, - [0][1][2][1][RTW89_IC][1] = 60, + [0][1][2][1][RTW89_IC][1] = 70, + [0][1][2][1][RTW89_KCC][1] = 66, [0][1][2][1][RTW89_ACMA][1] = 36, - [0][1][2][1][RTW89_FCC][2] = 54, + [0][1][2][1][RTW89_CN][1] = 34, + [0][1][2][1][RTW89_UK][1] = 36, + [0][1][2][1][RTW89_FCC][2] = 74, [0][1][2][1][RTW89_ETSI][2] = 36, [0][1][2][1][RTW89_MKK][2] = 68, - [0][1][2][1][RTW89_IC][2] = 64, + [0][1][2][1][RTW89_IC][2] = 74, + [0][1][2][1][RTW89_KCC][2] = 66, [0][1][2][1][RTW89_ACMA][2] = 36, - [0][1][2][1][RTW89_FCC][3] = 58, + [0][1][2][1][RTW89_CN][2] = 34, + [0][1][2][1][RTW89_UK][2] = 36, + [0][1][2][1][RTW89_FCC][3] = 78, [0][1][2][1][RTW89_ETSI][3] = 36, [0][1][2][1][RTW89_MKK][3] = 68, - [0][1][2][1][RTW89_IC][3] = 68, + [0][1][2][1][RTW89_IC][3] = 78, + [0][1][2][1][RTW89_KCC][3] = 66, [0][1][2][1][RTW89_ACMA][3] = 36, - [0][1][2][1][RTW89_FCC][4] = 64, + [0][1][2][1][RTW89_CN][3] = 34, + [0][1][2][1][RTW89_UK][3] = 36, + [0][1][2][1][RTW89_FCC][4] = 80, [0][1][2][1][RTW89_ETSI][4] = 36, [0][1][2][1][RTW89_MKK][4] = 68, - [0][1][2][1][RTW89_IC][4] = 74, + [0][1][2][1][RTW89_IC][4] = 80, + [0][1][2][1][RTW89_KCC][4] = 66, [0][1][2][1][RTW89_ACMA][4] = 36, - [0][1][2][1][RTW89_FCC][5] = 70, + [0][1][2][1][RTW89_CN][4] = 34, + [0][1][2][1][RTW89_UK][4] = 36, + [0][1][2][1][RTW89_FCC][5] = 80, [0][1][2][1][RTW89_ETSI][5] = 36, [0][1][2][1][RTW89_MKK][5] = 68, - [0][1][2][1][RTW89_IC][5] = 78, + [0][1][2][1][RTW89_IC][5] = 80, + [0][1][2][1][RTW89_KCC][5] = 66, [0][1][2][1][RTW89_ACMA][5] = 36, - [0][1][2][1][RTW89_FCC][6] = 66, + [0][1][2][1][RTW89_CN][5] = 34, + [0][1][2][1][RTW89_UK][5] = 36, + [0][1][2][1][RTW89_FCC][6] = 80, [0][1][2][1][RTW89_ETSI][6] = 36, [0][1][2][1][RTW89_MKK][6] = 68, - [0][1][2][1][RTW89_IC][6] = 76, + [0][1][2][1][RTW89_IC][6] = 80, + [0][1][2][1][RTW89_KCC][6] = 66, [0][1][2][1][RTW89_ACMA][6] = 36, - [0][1][2][1][RTW89_FCC][7] = 58, + [0][1][2][1][RTW89_CN][6] = 34, + [0][1][2][1][RTW89_UK][6] = 36, + [0][1][2][1][RTW89_FCC][7] = 74, [0][1][2][1][RTW89_ETSI][7] = 36, [0][1][2][1][RTW89_MKK][7] = 68, - [0][1][2][1][RTW89_IC][7] = 68, + [0][1][2][1][RTW89_IC][7] = 74, + [0][1][2][1][RTW89_KCC][7] = 66, [0][1][2][1][RTW89_ACMA][7] = 36, - [0][1][2][1][RTW89_FCC][8] = 54, + [0][1][2][1][RTW89_CN][7] = 34, + [0][1][2][1][RTW89_UK][7] = 36, + [0][1][2][1][RTW89_FCC][8] = 70, [0][1][2][1][RTW89_ETSI][8] = 36, [0][1][2][1][RTW89_MKK][8] = 68, - [0][1][2][1][RTW89_IC][8] = 64, + [0][1][2][1][RTW89_IC][8] = 70, + [0][1][2][1][RTW89_KCC][8] = 66, [0][1][2][1][RTW89_ACMA][8] = 36, - [0][1][2][1][RTW89_FCC][9] = 50, + [0][1][2][1][RTW89_CN][8] = 34, + [0][1][2][1][RTW89_UK][8] = 36, + [0][1][2][1][RTW89_FCC][9] = 66, [0][1][2][1][RTW89_ETSI][9] = 36, [0][1][2][1][RTW89_MKK][9] = 68, - [0][1][2][1][RTW89_IC][9] = 60, + [0][1][2][1][RTW89_IC][9] = 66, + [0][1][2][1][RTW89_KCC][9] = 64, [0][1][2][1][RTW89_ACMA][9] = 36, - [0][1][2][1][RTW89_FCC][10] = 50, + [0][1][2][1][RTW89_CN][9] = 34, + [0][1][2][1][RTW89_UK][9] = 36, + [0][1][2][1][RTW89_FCC][10] = 58, [0][1][2][1][RTW89_ETSI][10] = 36, [0][1][2][1][RTW89_MKK][10] = 68, - [0][1][2][1][RTW89_IC][10] = 60, + [0][1][2][1][RTW89_IC][10] = 58, + [0][1][2][1][RTW89_KCC][10] = 64, [0][1][2][1][RTW89_ACMA][10] = 36, - [0][1][2][1][RTW89_FCC][11] = 38, + [0][1][2][1][RTW89_CN][10] = 34, + [0][1][2][1][RTW89_UK][10] = 36, + [0][1][2][1][RTW89_FCC][11] = 58, [0][1][2][1][RTW89_ETSI][11] = 36, [0][1][2][1][RTW89_MKK][11] = 68, - [0][1][2][1][RTW89_IC][11] = 48, + [0][1][2][1][RTW89_IC][11] = 58, + [0][1][2][1][RTW89_KCC][11] = 64, [0][1][2][1][RTW89_ACMA][11] = 36, - [0][1][2][1][RTW89_FCC][12] = 34, + [0][1][2][1][RTW89_CN][11] = 34, + [0][1][2][1][RTW89_UK][11] = 36, + [0][1][2][1][RTW89_FCC][12] = 16, [0][1][2][1][RTW89_ETSI][12] = 36, [0][1][2][1][RTW89_MKK][12] = 68, - [0][1][2][1][RTW89_IC][12] = 44, + [0][1][2][1][RTW89_IC][12] = 16, + [0][1][2][1][RTW89_KCC][12] = 64, [0][1][2][1][RTW89_ACMA][12] = 36, + [0][1][2][1][RTW89_CN][12] = 34, + [0][1][2][1][RTW89_UK][12] = 36, [0][1][2][1][RTW89_FCC][13] = 127, [0][1][2][1][RTW89_ETSI][13] = 127, [0][1][2][1][RTW89_MKK][13] = 127, [0][1][2][1][RTW89_IC][13] = 127, + [0][1][2][1][RTW89_KCC][13] = 127, [0][1][2][1][RTW89_ACMA][13] = 127, + [0][1][2][1][RTW89_CN][13] = 127, + [0][1][2][1][RTW89_UK][13] = 127, [1][0][2][0][RTW89_FCC][0] = 127, [1][0][2][0][RTW89_ETSI][0] = 127, [1][0][2][0][RTW89_MKK][0] = 127, [1][0][2][0][RTW89_IC][0] = 127, + [1][0][2][0][RTW89_KCC][0] = 127, [1][0][2][0][RTW89_ACMA][0] = 127, + [1][0][2][0][RTW89_CN][0] = 127, + [1][0][2][0][RTW89_UK][0] = 127, [1][0][2][0][RTW89_FCC][1] = 127, [1][0][2][0][RTW89_ETSI][1] = 127, [1][0][2][0][RTW89_MKK][1] = 127, [1][0][2][0][RTW89_IC][1] = 127, + [1][0][2][0][RTW89_KCC][1] = 127, [1][0][2][0][RTW89_ACMA][1] = 127, - [1][0][2][0][RTW89_FCC][2] = 62, + [1][0][2][0][RTW89_CN][1] = 127, + [1][0][2][0][RTW89_UK][1] = 127, + [1][0][2][0][RTW89_FCC][2] = 64, [1][0][2][0][RTW89_ETSI][2] = 60, [1][0][2][0][RTW89_MKK][2] = 74, - [1][0][2][0][RTW89_IC][2] = 72, + [1][0][2][0][RTW89_IC][2] = 64, + [1][0][2][0][RTW89_KCC][2] = 68, [1][0][2][0][RTW89_ACMA][2] = 60, - [1][0][2][0][RTW89_FCC][3] = 62, + [1][0][2][0][RTW89_CN][2] = 58, + [1][0][2][0][RTW89_UK][2] = 60, + [1][0][2][0][RTW89_FCC][3] = 64, [1][0][2][0][RTW89_ETSI][3] = 60, [1][0][2][0][RTW89_MKK][3] = 74, - [1][0][2][0][RTW89_IC][3] = 72, + [1][0][2][0][RTW89_IC][3] = 64, + [1][0][2][0][RTW89_KCC][3] = 68, [1][0][2][0][RTW89_ACMA][3] = 60, - [1][0][2][0][RTW89_FCC][4] = 64, + [1][0][2][0][RTW89_CN][3] = 58, + [1][0][2][0][RTW89_UK][3] = 60, + [1][0][2][0][RTW89_FCC][4] = 68, [1][0][2][0][RTW89_ETSI][4] = 60, [1][0][2][0][RTW89_MKK][4] = 74, - [1][0][2][0][RTW89_IC][4] = 74, + [1][0][2][0][RTW89_IC][4] = 68, + [1][0][2][0][RTW89_KCC][4] = 68, [1][0][2][0][RTW89_ACMA][4] = 60, - [1][0][2][0][RTW89_FCC][5] = 64, + [1][0][2][0][RTW89_CN][4] = 58, + [1][0][2][0][RTW89_UK][4] = 60, + [1][0][2][0][RTW89_FCC][5] = 68, [1][0][2][0][RTW89_ETSI][5] = 60, [1][0][2][0][RTW89_MKK][5] = 74, - [1][0][2][0][RTW89_IC][5] = 74, + [1][0][2][0][RTW89_IC][5] = 68, + [1][0][2][0][RTW89_KCC][5] = 74, [1][0][2][0][RTW89_ACMA][5] = 60, - [1][0][2][0][RTW89_FCC][6] = 64, + [1][0][2][0][RTW89_CN][5] = 58, + [1][0][2][0][RTW89_UK][5] = 60, + [1][0][2][0][RTW89_FCC][6] = 66, [1][0][2][0][RTW89_ETSI][6] = 60, [1][0][2][0][RTW89_MKK][6] = 74, - [1][0][2][0][RTW89_IC][6] = 74, + [1][0][2][0][RTW89_IC][6] = 66, + [1][0][2][0][RTW89_KCC][6] = 74, [1][0][2][0][RTW89_ACMA][6] = 60, - [1][0][2][0][RTW89_FCC][7] = 60, + [1][0][2][0][RTW89_CN][6] = 58, + [1][0][2][0][RTW89_UK][6] = 60, + [1][0][2][0][RTW89_FCC][7] = 62, [1][0][2][0][RTW89_ETSI][7] = 60, [1][0][2][0][RTW89_MKK][7] = 74, - [1][0][2][0][RTW89_IC][7] = 70, + [1][0][2][0][RTW89_IC][7] = 62, + [1][0][2][0][RTW89_KCC][7] = 74, [1][0][2][0][RTW89_ACMA][7] = 60, - [1][0][2][0][RTW89_FCC][8] = 60, + [1][0][2][0][RTW89_CN][7] = 58, + [1][0][2][0][RTW89_UK][7] = 60, + [1][0][2][0][RTW89_FCC][8] = 62, [1][0][2][0][RTW89_ETSI][8] = 60, [1][0][2][0][RTW89_MKK][8] = 74, - [1][0][2][0][RTW89_IC][8] = 70, + [1][0][2][0][RTW89_IC][8] = 62, + [1][0][2][0][RTW89_KCC][8] = 68, [1][0][2][0][RTW89_ACMA][8] = 60, + [1][0][2][0][RTW89_CN][8] = 58, + [1][0][2][0][RTW89_UK][8] = 60, [1][0][2][0][RTW89_FCC][9] = 60, [1][0][2][0][RTW89_ETSI][9] = 60, [1][0][2][0][RTW89_MKK][9] = 74, - [1][0][2][0][RTW89_IC][9] = 70, + [1][0][2][0][RTW89_IC][9] = 60, + [1][0][2][0][RTW89_KCC][9] = 68, [1][0][2][0][RTW89_ACMA][9] = 60, - [1][0][2][0][RTW89_FCC][10] = 58, + [1][0][2][0][RTW89_CN][9] = 58, + [1][0][2][0][RTW89_UK][9] = 60, + [1][0][2][0][RTW89_FCC][10] = 56, [1][0][2][0][RTW89_ETSI][10] = 60, [1][0][2][0][RTW89_MKK][10] = 74, - [1][0][2][0][RTW89_IC][10] = 68, + [1][0][2][0][RTW89_IC][10] = 56, + [1][0][2][0][RTW89_KCC][10] = 68, [1][0][2][0][RTW89_ACMA][10] = 60, + [1][0][2][0][RTW89_CN][10] = 58, + [1][0][2][0][RTW89_UK][10] = 60, [1][0][2][0][RTW89_FCC][11] = 127, [1][0][2][0][RTW89_ETSI][11] = 127, [1][0][2][0][RTW89_MKK][11] = 127, [1][0][2][0][RTW89_IC][11] = 127, + [1][0][2][0][RTW89_KCC][11] = 127, [1][0][2][0][RTW89_ACMA][11] = 127, + [1][0][2][0][RTW89_CN][11] = 127, + [1][0][2][0][RTW89_UK][11] = 127, [1][0][2][0][RTW89_FCC][12] = 127, [1][0][2][0][RTW89_ETSI][12] = 127, [1][0][2][0][RTW89_MKK][12] = 127, [1][0][2][0][RTW89_IC][12] = 127, + [1][0][2][0][RTW89_KCC][12] = 127, [1][0][2][0][RTW89_ACMA][12] = 127, + [1][0][2][0][RTW89_CN][12] = 127, + [1][0][2][0][RTW89_UK][12] = 127, [1][0][2][0][RTW89_FCC][13] = 127, [1][0][2][0][RTW89_ETSI][13] = 127, [1][0][2][0][RTW89_MKK][13] = 127, [1][0][2][0][RTW89_IC][13] = 127, + [1][0][2][0][RTW89_KCC][13] = 127, [1][0][2][0][RTW89_ACMA][13] = 127, + [1][0][2][0][RTW89_CN][13] = 127, + [1][0][2][0][RTW89_UK][13] = 127, [1][1][2][0][RTW89_FCC][0] = 127, [1][1][2][0][RTW89_ETSI][0] = 127, [1][1][2][0][RTW89_MKK][0] = 127, [1][1][2][0][RTW89_IC][0] = 127, + [1][1][2][0][RTW89_KCC][0] = 127, [1][1][2][0][RTW89_ACMA][0] = 127, + [1][1][2][0][RTW89_CN][0] = 127, + [1][1][2][0][RTW89_UK][0] = 127, [1][1][2][0][RTW89_FCC][1] = 127, [1][1][2][0][RTW89_ETSI][1] = 127, [1][1][2][0][RTW89_MKK][1] = 127, [1][1][2][0][RTW89_IC][1] = 127, + [1][1][2][0][RTW89_KCC][1] = 127, [1][1][2][0][RTW89_ACMA][1] = 127, - [1][1][2][0][RTW89_FCC][2] = 46, + [1][1][2][0][RTW89_CN][1] = 127, + [1][1][2][0][RTW89_UK][1] = 127, + [1][1][2][0][RTW89_FCC][2] = 60, [1][1][2][0][RTW89_ETSI][2] = 48, [1][1][2][0][RTW89_MKK][2] = 68, - [1][1][2][0][RTW89_IC][2] = 56, + [1][1][2][0][RTW89_IC][2] = 60, + [1][1][2][0][RTW89_KCC][2] = 64, [1][1][2][0][RTW89_ACMA][2] = 48, - [1][1][2][0][RTW89_FCC][3] = 46, + [1][1][2][0][RTW89_CN][2] = 34, + [1][1][2][0][RTW89_UK][2] = 48, + [1][1][2][0][RTW89_FCC][3] = 60, [1][1][2][0][RTW89_ETSI][3] = 48, [1][1][2][0][RTW89_MKK][3] = 68, - [1][1][2][0][RTW89_IC][3] = 56, + [1][1][2][0][RTW89_IC][3] = 60, + [1][1][2][0][RTW89_KCC][3] = 64, [1][1][2][0][RTW89_ACMA][3] = 48, - [1][1][2][0][RTW89_FCC][4] = 50, + [1][1][2][0][RTW89_CN][3] = 34, + [1][1][2][0][RTW89_UK][3] = 48, + [1][1][2][0][RTW89_FCC][4] = 60, [1][1][2][0][RTW89_ETSI][4] = 48, [1][1][2][0][RTW89_MKK][4] = 68, [1][1][2][0][RTW89_IC][4] = 60, + [1][1][2][0][RTW89_KCC][4] = 64, [1][1][2][0][RTW89_ACMA][4] = 48, - [1][1][2][0][RTW89_FCC][5] = 58, + [1][1][2][0][RTW89_CN][4] = 34, + [1][1][2][0][RTW89_UK][4] = 48, + [1][1][2][0][RTW89_FCC][5] = 60, [1][1][2][0][RTW89_ETSI][5] = 48, [1][1][2][0][RTW89_MKK][5] = 68, - [1][1][2][0][RTW89_IC][5] = 68, + [1][1][2][0][RTW89_IC][5] = 60, + [1][1][2][0][RTW89_KCC][5] = 66, [1][1][2][0][RTW89_ACMA][5] = 48, - [1][1][2][0][RTW89_FCC][6] = 50, + [1][1][2][0][RTW89_CN][5] = 34, + [1][1][2][0][RTW89_UK][5] = 48, + [1][1][2][0][RTW89_FCC][6] = 58, [1][1][2][0][RTW89_ETSI][6] = 48, [1][1][2][0][RTW89_MKK][6] = 68, - [1][1][2][0][RTW89_IC][6] = 60, + [1][1][2][0][RTW89_IC][6] = 58, + [1][1][2][0][RTW89_KCC][6] = 66, [1][1][2][0][RTW89_ACMA][6] = 48, - [1][1][2][0][RTW89_FCC][7] = 46, + [1][1][2][0][RTW89_CN][6] = 34, + [1][1][2][0][RTW89_UK][6] = 48, + [1][1][2][0][RTW89_FCC][7] = 54, [1][1][2][0][RTW89_ETSI][7] = 48, [1][1][2][0][RTW89_MKK][7] = 68, - [1][1][2][0][RTW89_IC][7] = 56, + [1][1][2][0][RTW89_IC][7] = 54, + [1][1][2][0][RTW89_KCC][7] = 66, [1][1][2][0][RTW89_ACMA][7] = 48, - [1][1][2][0][RTW89_FCC][8] = 46, + [1][1][2][0][RTW89_CN][7] = 34, + [1][1][2][0][RTW89_UK][7] = 48, + [1][1][2][0][RTW89_FCC][8] = 54, [1][1][2][0][RTW89_ETSI][8] = 48, [1][1][2][0][RTW89_MKK][8] = 68, - [1][1][2][0][RTW89_IC][8] = 56, + [1][1][2][0][RTW89_IC][8] = 54, + [1][1][2][0][RTW89_KCC][8] = 64, [1][1][2][0][RTW89_ACMA][8] = 48, - [1][1][2][0][RTW89_FCC][9] = 34, + [1][1][2][0][RTW89_CN][8] = 34, + [1][1][2][0][RTW89_UK][8] = 48, + [1][1][2][0][RTW89_FCC][9] = 54, [1][1][2][0][RTW89_ETSI][9] = 48, [1][1][2][0][RTW89_MKK][9] = 68, - [1][1][2][0][RTW89_IC][9] = 44, + [1][1][2][0][RTW89_IC][9] = 54, + [1][1][2][0][RTW89_KCC][9] = 64, [1][1][2][0][RTW89_ACMA][9] = 48, - [1][1][2][0][RTW89_FCC][10] = 30, + [1][1][2][0][RTW89_CN][9] = 34, + [1][1][2][0][RTW89_UK][9] = 48, + [1][1][2][0][RTW89_FCC][10] = 46, [1][1][2][0][RTW89_ETSI][10] = 48, [1][1][2][0][RTW89_MKK][10] = 68, - [1][1][2][0][RTW89_IC][10] = 40, + [1][1][2][0][RTW89_IC][10] = 46, + [1][1][2][0][RTW89_KCC][10] = 64, [1][1][2][0][RTW89_ACMA][10] = 48, + [1][1][2][0][RTW89_CN][10] = 34, + [1][1][2][0][RTW89_UK][10] = 48, [1][1][2][0][RTW89_FCC][11] = 127, [1][1][2][0][RTW89_ETSI][11] = 127, [1][1][2][0][RTW89_MKK][11] = 127, [1][1][2][0][RTW89_IC][11] = 127, + [1][1][2][0][RTW89_KCC][11] = 127, [1][1][2][0][RTW89_ACMA][11] = 127, + [1][1][2][0][RTW89_CN][11] = 127, + [1][1][2][0][RTW89_UK][11] = 127, [1][1][2][0][RTW89_FCC][12] = 127, [1][1][2][0][RTW89_ETSI][12] = 127, [1][1][2][0][RTW89_MKK][12] = 127, [1][1][2][0][RTW89_IC][12] = 127, + [1][1][2][0][RTW89_KCC][12] = 127, [1][1][2][0][RTW89_ACMA][12] = 127, + [1][1][2][0][RTW89_CN][12] = 127, + [1][1][2][0][RTW89_UK][12] = 127, [1][1][2][0][RTW89_FCC][13] = 127, [1][1][2][0][RTW89_ETSI][13] = 127, [1][1][2][0][RTW89_MKK][13] = 127, [1][1][2][0][RTW89_IC][13] = 127, + [1][1][2][0][RTW89_KCC][13] = 127, [1][1][2][0][RTW89_ACMA][13] = 127, + [1][1][2][0][RTW89_CN][13] = 127, + [1][1][2][0][RTW89_UK][13] = 127, [1][1][2][1][RTW89_FCC][0] = 127, [1][1][2][1][RTW89_ETSI][0] = 127, [1][1][2][1][RTW89_MKK][0] = 127, [1][1][2][1][RTW89_IC][0] = 127, + [1][1][2][1][RTW89_KCC][0] = 127, [1][1][2][1][RTW89_ACMA][0] = 127, + [1][1][2][1][RTW89_CN][0] = 127, + [1][1][2][1][RTW89_UK][0] = 127, [1][1][2][1][RTW89_FCC][1] = 127, [1][1][2][1][RTW89_ETSI][1] = 127, [1][1][2][1][RTW89_MKK][1] = 127, [1][1][2][1][RTW89_IC][1] = 127, + [1][1][2][1][RTW89_KCC][1] = 127, [1][1][2][1][RTW89_ACMA][1] = 127, - [1][1][2][1][RTW89_FCC][2] = 46, + [1][1][2][1][RTW89_CN][1] = 127, + [1][1][2][1][RTW89_UK][1] = 127, + [1][1][2][1][RTW89_FCC][2] = 60, [1][1][2][1][RTW89_ETSI][2] = 36, [1][1][2][1][RTW89_MKK][2] = 68, - [1][1][2][1][RTW89_IC][2] = 56, + [1][1][2][1][RTW89_IC][2] = 60, + [1][1][2][1][RTW89_KCC][2] = 64, [1][1][2][1][RTW89_ACMA][2] = 36, - [1][1][2][1][RTW89_FCC][3] = 46, + [1][1][2][1][RTW89_CN][2] = 34, + [1][1][2][1][RTW89_UK][2] = 36, + [1][1][2][1][RTW89_FCC][3] = 60, [1][1][2][1][RTW89_ETSI][3] = 36, [1][1][2][1][RTW89_MKK][3] = 68, - [1][1][2][1][RTW89_IC][3] = 56, + [1][1][2][1][RTW89_IC][3] = 60, + [1][1][2][1][RTW89_KCC][3] = 64, [1][1][2][1][RTW89_ACMA][3] = 36, - [1][1][2][1][RTW89_FCC][4] = 50, + [1][1][2][1][RTW89_CN][3] = 34, + [1][1][2][1][RTW89_UK][3] = 36, + [1][1][2][1][RTW89_FCC][4] = 60, [1][1][2][1][RTW89_ETSI][4] = 36, [1][1][2][1][RTW89_MKK][4] = 68, [1][1][2][1][RTW89_IC][4] = 60, + [1][1][2][1][RTW89_KCC][4] = 64, [1][1][2][1][RTW89_ACMA][4] = 36, - [1][1][2][1][RTW89_FCC][5] = 58, + [1][1][2][1][RTW89_CN][4] = 34, + [1][1][2][1][RTW89_UK][4] = 36, + [1][1][2][1][RTW89_FCC][5] = 60, [1][1][2][1][RTW89_ETSI][5] = 36, [1][1][2][1][RTW89_MKK][5] = 68, - [1][1][2][1][RTW89_IC][5] = 68, + [1][1][2][1][RTW89_IC][5] = 60, + [1][1][2][1][RTW89_KCC][5] = 66, [1][1][2][1][RTW89_ACMA][5] = 36, - [1][1][2][1][RTW89_FCC][6] = 50, + [1][1][2][1][RTW89_CN][5] = 34, + [1][1][2][1][RTW89_UK][5] = 36, + [1][1][2][1][RTW89_FCC][6] = 58, [1][1][2][1][RTW89_ETSI][6] = 36, [1][1][2][1][RTW89_MKK][6] = 68, - [1][1][2][1][RTW89_IC][6] = 60, + [1][1][2][1][RTW89_IC][6] = 58, + [1][1][2][1][RTW89_KCC][6] = 66, [1][1][2][1][RTW89_ACMA][6] = 36, - [1][1][2][1][RTW89_FCC][7] = 46, + [1][1][2][1][RTW89_CN][6] = 34, + [1][1][2][1][RTW89_UK][6] = 36, + [1][1][2][1][RTW89_FCC][7] = 54, [1][1][2][1][RTW89_ETSI][7] = 36, [1][1][2][1][RTW89_MKK][7] = 68, - [1][1][2][1][RTW89_IC][7] = 56, + [1][1][2][1][RTW89_IC][7] = 54, + [1][1][2][1][RTW89_KCC][7] = 66, [1][1][2][1][RTW89_ACMA][7] = 36, - [1][1][2][1][RTW89_FCC][8] = 46, + [1][1][2][1][RTW89_CN][7] = 34, + [1][1][2][1][RTW89_UK][7] = 36, + [1][1][2][1][RTW89_FCC][8] = 54, [1][1][2][1][RTW89_ETSI][8] = 36, [1][1][2][1][RTW89_MKK][8] = 68, - [1][1][2][1][RTW89_IC][8] = 56, + [1][1][2][1][RTW89_IC][8] = 54, + [1][1][2][1][RTW89_KCC][8] = 64, [1][1][2][1][RTW89_ACMA][8] = 36, - [1][1][2][1][RTW89_FCC][9] = 34, + [1][1][2][1][RTW89_CN][8] = 34, + [1][1][2][1][RTW89_UK][8] = 36, + [1][1][2][1][RTW89_FCC][9] = 54, [1][1][2][1][RTW89_ETSI][9] = 36, [1][1][2][1][RTW89_MKK][9] = 68, - [1][1][2][1][RTW89_IC][9] = 44, + [1][1][2][1][RTW89_IC][9] = 54, + [1][1][2][1][RTW89_KCC][9] = 64, [1][1][2][1][RTW89_ACMA][9] = 36, - [1][1][2][1][RTW89_FCC][10] = 30, + [1][1][2][1][RTW89_CN][9] = 34, + [1][1][2][1][RTW89_UK][9] = 36, + [1][1][2][1][RTW89_FCC][10] = 46, [1][1][2][1][RTW89_ETSI][10] = 36, [1][1][2][1][RTW89_MKK][10] = 68, - [1][1][2][1][RTW89_IC][10] = 40, + [1][1][2][1][RTW89_IC][10] = 46, + [1][1][2][1][RTW89_KCC][10] = 64, [1][1][2][1][RTW89_ACMA][10] = 36, + [1][1][2][1][RTW89_CN][10] = 36, + [1][1][2][1][RTW89_UK][10] = 36, [1][1][2][1][RTW89_FCC][11] = 127, [1][1][2][1][RTW89_ETSI][11] = 127, [1][1][2][1][RTW89_MKK][11] = 127, [1][1][2][1][RTW89_IC][11] = 127, + [1][1][2][1][RTW89_KCC][11] = 127, [1][1][2][1][RTW89_ACMA][11] = 127, + [1][1][2][1][RTW89_CN][11] = 127, + [1][1][2][1][RTW89_UK][11] = 127, [1][1][2][1][RTW89_FCC][12] = 127, [1][1][2][1][RTW89_ETSI][12] = 127, [1][1][2][1][RTW89_MKK][12] = 127, [1][1][2][1][RTW89_IC][12] = 127, + [1][1][2][1][RTW89_KCC][12] = 127, [1][1][2][1][RTW89_ACMA][12] = 127, + [1][1][2][1][RTW89_CN][12] = 127, + [1][1][2][1][RTW89_UK][12] = 127, [1][1][2][1][RTW89_FCC][13] = 127, [1][1][2][1][RTW89_ETSI][13] = 127, [1][1][2][1][RTW89_MKK][13] = 127, [1][1][2][1][RTW89_IC][13] = 127, + [1][1][2][1][RTW89_KCC][13] = 127, [1][1][2][1][RTW89_ACMA][13] = 127, + [1][1][2][1][RTW89_CN][13] = 127, + [1][1][2][1][RTW89_UK][13] = 127, }; const s8 rtw89_8852c_txpwr_lmt_5g[RTW89_5G_BW_NUM][RTW89_NTX_NUM] [RTW89_RS_LMT_NUM][RTW89_BF_NUM] [RTW89_REGD_NUM][RTW89_5G_CH_NUM] = { - [0][0][1][0][RTW89_WW][0] = 60, - [0][0][1][0][RTW89_WW][2] = 60, - [0][0][1][0][RTW89_WW][4] = 60, - [0][0][1][0][RTW89_WW][6] = 60, - [0][0][1][0][RTW89_WW][8] = 60, - [0][0][1][0][RTW89_WW][10] = 60, - [0][0][1][0][RTW89_WW][12] = 60, - [0][0][1][0][RTW89_WW][14] = 60, - [0][0][1][0][RTW89_WW][15] = 60, - [0][0][1][0][RTW89_WW][17] = 60, - [0][0][1][0][RTW89_WW][19] = 60, - [0][0][1][0][RTW89_WW][21] = 60, - [0][0][1][0][RTW89_WW][23] = 60, + [0][0][1][0][RTW89_WW][0] = 50, + [0][0][1][0][RTW89_WW][2] = 50, + [0][0][1][0][RTW89_WW][4] = 50, + [0][0][1][0][RTW89_WW][6] = 50, + [0][0][1][0][RTW89_WW][8] = 50, + [0][0][1][0][RTW89_WW][10] = 50, + [0][0][1][0][RTW89_WW][12] = 50, + [0][0][1][0][RTW89_WW][14] = 50, + [0][0][1][0][RTW89_WW][15] = 66, + [0][0][1][0][RTW89_WW][17] = 66, + [0][0][1][0][RTW89_WW][19] = 66, + [0][0][1][0][RTW89_WW][21] = 66, + [0][0][1][0][RTW89_WW][23] = 66, [0][0][1][0][RTW89_WW][25] = 66, [0][0][1][0][RTW89_WW][27] = 66, [0][0][1][0][RTW89_WW][29] = 66, - [0][0][1][0][RTW89_WW][31] = 60, - [0][0][1][0][RTW89_WW][33] = 60, + [0][0][1][0][RTW89_WW][31] = 66, + [0][0][1][0][RTW89_WW][33] = 66, [0][0][1][0][RTW89_WW][35] = 60, - [0][0][1][0][RTW89_WW][37] = 70, + [0][0][1][0][RTW89_WW][37] = 64, [0][0][1][0][RTW89_WW][38] = 30, [0][0][1][0][RTW89_WW][40] = 30, [0][0][1][0][RTW89_WW][42] = 30, [0][0][1][0][RTW89_WW][44] = 30, [0][0][1][0][RTW89_WW][46] = 30, - [0][0][1][0][RTW89_WW][48] = 70, - [0][0][1][0][RTW89_WW][50] = 70, - [0][0][1][0][RTW89_WW][52] = 70, - [0][1][1][0][RTW89_WW][0] = 42, - [0][1][1][0][RTW89_WW][2] = 42, - [0][1][1][0][RTW89_WW][4] = 42, - [0][1][1][0][RTW89_WW][6] = 42, - [0][1][1][0][RTW89_WW][8] = 48, - [0][1][1][0][RTW89_WW][10] = 48, - [0][1][1][0][RTW89_WW][12] = 48, - [0][1][1][0][RTW89_WW][14] = 48, - [0][1][1][0][RTW89_WW][15] = 48, - [0][1][1][0][RTW89_WW][17] = 48, - [0][1][1][0][RTW89_WW][19] = 48, - [0][1][1][0][RTW89_WW][21] = 48, - [0][1][1][0][RTW89_WW][23] = 48, + [0][0][1][0][RTW89_WW][48] = 72, + [0][0][1][0][RTW89_WW][50] = 72, + [0][0][1][0][RTW89_WW][52] = 72, + [0][1][1][0][RTW89_WW][0] = 34, + [0][1][1][0][RTW89_WW][2] = 34, + [0][1][1][0][RTW89_WW][4] = 34, + [0][1][1][0][RTW89_WW][6] = 36, + [0][1][1][0][RTW89_WW][8] = 46, + [0][1][1][0][RTW89_WW][10] = 46, + [0][1][1][0][RTW89_WW][12] = 46, + [0][1][1][0][RTW89_WW][14] = 46, + [0][1][1][0][RTW89_WW][15] = 54, + [0][1][1][0][RTW89_WW][17] = 54, + [0][1][1][0][RTW89_WW][19] = 54, + [0][1][1][0][RTW89_WW][21] = 54, + [0][1][1][0][RTW89_WW][23] = 54, [0][1][1][0][RTW89_WW][25] = 54, [0][1][1][0][RTW89_WW][27] = 54, [0][1][1][0][RTW89_WW][29] = 54, - [0][1][1][0][RTW89_WW][31] = 48, - [0][1][1][0][RTW89_WW][33] = 48, - [0][1][1][0][RTW89_WW][35] = 48, - [0][1][1][0][RTW89_WW][37] = 60, + [0][1][1][0][RTW89_WW][31] = 54, + [0][1][1][0][RTW89_WW][33] = 54, + [0][1][1][0][RTW89_WW][35] = 52, + [0][1][1][0][RTW89_WW][37] = 52, [0][1][1][0][RTW89_WW][38] = 18, - [0][1][1][0][RTW89_WW][40] = 16, + [0][1][1][0][RTW89_WW][40] = 18, [0][1][1][0][RTW89_WW][42] = 18, - [0][1][1][0][RTW89_WW][44] = 16, + [0][1][1][0][RTW89_WW][44] = 18, [0][1][1][0][RTW89_WW][46] = 18, [0][1][1][0][RTW89_WW][48] = 48, [0][1][1][0][RTW89_WW][50] = 48, [0][1][1][0][RTW89_WW][52] = 48, - [0][0][2][0][RTW89_WW][0] = 62, - [0][0][2][0][RTW89_WW][2] = 62, - [0][0][2][0][RTW89_WW][4] = 62, - [0][0][2][0][RTW89_WW][6] = 60, - [0][0][2][0][RTW89_WW][8] = 58, - [0][0][2][0][RTW89_WW][10] = 62, - [0][0][2][0][RTW89_WW][12] = 62, - [0][0][2][0][RTW89_WW][14] = 62, - [0][0][2][0][RTW89_WW][15] = 62, - [0][0][2][0][RTW89_WW][17] = 62, - [0][0][2][0][RTW89_WW][19] = 62, - [0][0][2][0][RTW89_WW][21] = 62, - [0][0][2][0][RTW89_WW][23] = 62, + [0][0][2][0][RTW89_WW][0] = 52, + [0][0][2][0][RTW89_WW][2] = 52, + [0][0][2][0][RTW89_WW][4] = 52, + [0][0][2][0][RTW89_WW][6] = 52, + [0][0][2][0][RTW89_WW][8] = 52, + [0][0][2][0][RTW89_WW][10] = 52, + [0][0][2][0][RTW89_WW][12] = 52, + [0][0][2][0][RTW89_WW][14] = 52, + [0][0][2][0][RTW89_WW][15] = 66, + [0][0][2][0][RTW89_WW][17] = 66, + [0][0][2][0][RTW89_WW][19] = 66, + [0][0][2][0][RTW89_WW][21] = 66, + [0][0][2][0][RTW89_WW][23] = 66, [0][0][2][0][RTW89_WW][25] = 66, [0][0][2][0][RTW89_WW][27] = 66, [0][0][2][0][RTW89_WW][29] = 66, - [0][0][2][0][RTW89_WW][31] = 62, - [0][0][2][0][RTW89_WW][33] = 62, - [0][0][2][0][RTW89_WW][35] = 62, - [0][0][2][0][RTW89_WW][37] = 70, + [0][0][2][0][RTW89_WW][31] = 66, + [0][0][2][0][RTW89_WW][33] = 66, + [0][0][2][0][RTW89_WW][35] = 56, + [0][0][2][0][RTW89_WW][37] = 64, [0][0][2][0][RTW89_WW][38] = 30, [0][0][2][0][RTW89_WW][40] = 30, [0][0][2][0][RTW89_WW][42] = 30, [0][0][2][0][RTW89_WW][44] = 30, [0][0][2][0][RTW89_WW][46] = 30, - [0][0][2][0][RTW89_WW][48] = 70, - [0][0][2][0][RTW89_WW][50] = 70, - [0][0][2][0][RTW89_WW][52] = 70, - [0][1][2][0][RTW89_WW][0] = 44, - [0][1][2][0][RTW89_WW][2] = 44, - [0][1][2][0][RTW89_WW][4] = 44, - [0][1][2][0][RTW89_WW][6] = 44, - [0][1][2][0][RTW89_WW][8] = 42, - [0][1][2][0][RTW89_WW][10] = 50, - [0][1][2][0][RTW89_WW][12] = 50, - [0][1][2][0][RTW89_WW][14] = 50, - [0][1][2][0][RTW89_WW][15] = 50, - [0][1][2][0][RTW89_WW][17] = 50, - [0][1][2][0][RTW89_WW][19] = 50, - [0][1][2][0][RTW89_WW][21] = 50, - [0][1][2][0][RTW89_WW][23] = 50, + [0][0][2][0][RTW89_WW][48] = 72, + [0][0][2][0][RTW89_WW][50] = 72, + [0][0][2][0][RTW89_WW][52] = 72, + [0][1][2][0][RTW89_WW][0] = 36, + [0][1][2][0][RTW89_WW][2] = 36, + [0][1][2][0][RTW89_WW][4] = 36, + [0][1][2][0][RTW89_WW][6] = 38, + [0][1][2][0][RTW89_WW][8] = 40, + [0][1][2][0][RTW89_WW][10] = 40, + [0][1][2][0][RTW89_WW][12] = 40, + [0][1][2][0][RTW89_WW][14] = 40, + [0][1][2][0][RTW89_WW][15] = 54, + [0][1][2][0][RTW89_WW][17] = 54, + [0][1][2][0][RTW89_WW][19] = 54, + [0][1][2][0][RTW89_WW][21] = 54, + [0][1][2][0][RTW89_WW][23] = 54, [0][1][2][0][RTW89_WW][25] = 54, [0][1][2][0][RTW89_WW][27] = 54, [0][1][2][0][RTW89_WW][29] = 54, - [0][1][2][0][RTW89_WW][31] = 50, - [0][1][2][0][RTW89_WW][33] = 50, - [0][1][2][0][RTW89_WW][35] = 50, - [0][1][2][0][RTW89_WW][37] = 62, + [0][1][2][0][RTW89_WW][31] = 54, + [0][1][2][0][RTW89_WW][33] = 54, + [0][1][2][0][RTW89_WW][35] = 46, + [0][1][2][0][RTW89_WW][37] = 52, [0][1][2][0][RTW89_WW][38] = 18, [0][1][2][0][RTW89_WW][40] = 18, [0][1][2][0][RTW89_WW][42] = 18, [0][1][2][0][RTW89_WW][44] = 18, [0][1][2][0][RTW89_WW][46] = 18, - [0][1][2][0][RTW89_WW][48] = 50, + [0][1][2][0][RTW89_WW][48] = 48, [0][1][2][0][RTW89_WW][50] = 50, - [0][1][2][0][RTW89_WW][52] = 50, - [0][1][2][1][RTW89_WW][0] = 38, - [0][1][2][1][RTW89_WW][2] = 38, - [0][1][2][1][RTW89_WW][4] = 38, - [0][1][2][1][RTW89_WW][6] = 38, - [0][1][2][1][RTW89_WW][8] = 38, - [0][1][2][1][RTW89_WW][10] = 38, - [0][1][2][1][RTW89_WW][12] = 38, - [0][1][2][1][RTW89_WW][14] = 38, - [0][1][2][1][RTW89_WW][15] = 38, - [0][1][2][1][RTW89_WW][17] = 38, - [0][1][2][1][RTW89_WW][19] = 38, - [0][1][2][1][RTW89_WW][21] = 38, - [0][1][2][1][RTW89_WW][23] = 38, + [0][1][2][0][RTW89_WW][52] = 48, + [0][1][2][1][RTW89_WW][0] = 36, + [0][1][2][1][RTW89_WW][2] = 36, + [0][1][2][1][RTW89_WW][4] = 36, + [0][1][2][1][RTW89_WW][6] = 36, + [0][1][2][1][RTW89_WW][8] = 36, + [0][1][2][1][RTW89_WW][10] = 36, + [0][1][2][1][RTW89_WW][12] = 36, + [0][1][2][1][RTW89_WW][14] = 36, + [0][1][2][1][RTW89_WW][15] = 40, + [0][1][2][1][RTW89_WW][17] = 40, + [0][1][2][1][RTW89_WW][19] = 40, + [0][1][2][1][RTW89_WW][21] = 40, + [0][1][2][1][RTW89_WW][23] = 40, [0][1][2][1][RTW89_WW][25] = 40, [0][1][2][1][RTW89_WW][27] = 40, [0][1][2][1][RTW89_WW][29] = 40, - [0][1][2][1][RTW89_WW][31] = 38, - [0][1][2][1][RTW89_WW][33] = 38, - [0][1][2][1][RTW89_WW][35] = 38, - [0][1][2][1][RTW89_WW][37] = 60, + [0][1][2][1][RTW89_WW][31] = 40, + [0][1][2][1][RTW89_WW][33] = 40, + [0][1][2][1][RTW89_WW][35] = 40, + [0][1][2][1][RTW89_WW][37] = 40, [0][1][2][1][RTW89_WW][38] = 6, [0][1][2][1][RTW89_WW][40] = 6, [0][1][2][1][RTW89_WW][42] = 6, [0][1][2][1][RTW89_WW][44] = 6, [0][1][2][1][RTW89_WW][46] = 6, - [0][1][2][1][RTW89_WW][48] = 50, + [0][1][2][1][RTW89_WW][48] = 48, [0][1][2][1][RTW89_WW][50] = 50, - [0][1][2][1][RTW89_WW][52] = 50, - [1][0][2][0][RTW89_WW][1] = 58, - [1][0][2][0][RTW89_WW][5] = 66, - [1][0][2][0][RTW89_WW][9] = 66, - [1][0][2][0][RTW89_WW][13] = 58, + [0][1][2][1][RTW89_WW][52] = 48, + [1][0][2][0][RTW89_WW][1] = 54, + [1][0][2][0][RTW89_WW][5] = 54, + [1][0][2][0][RTW89_WW][9] = 54, + [1][0][2][0][RTW89_WW][13] = 52, [1][0][2][0][RTW89_WW][16] = 56, - [1][0][2][0][RTW89_WW][20] = 66, - [1][0][2][0][RTW89_WW][24] = 66, + [1][0][2][0][RTW89_WW][20] = 56, + [1][0][2][0][RTW89_WW][24] = 56, [1][0][2][0][RTW89_WW][28] = 66, - [1][0][2][0][RTW89_WW][32] = 66, - [1][0][2][0][RTW89_WW][36] = 66, + [1][0][2][0][RTW89_WW][32] = 62, + [1][0][2][0][RTW89_WW][36] = 64, [1][0][2][0][RTW89_WW][39] = 30, [1][0][2][0][RTW89_WW][43] = 30, [1][0][2][0][RTW89_WW][47] = 68, [1][0][2][0][RTW89_WW][51] = 68, - [1][1][2][0][RTW89_WW][1] = 48, - [1][1][2][0][RTW89_WW][5] = 52, - [1][1][2][0][RTW89_WW][9] = 52, - [1][1][2][0][RTW89_WW][13] = 52, - [1][1][2][0][RTW89_WW][16] = 48, + [1][1][2][0][RTW89_WW][1] = 42, + [1][1][2][0][RTW89_WW][5] = 42, + [1][1][2][0][RTW89_WW][9] = 42, + [1][1][2][0][RTW89_WW][13] = 42, + [1][1][2][0][RTW89_WW][16] = 54, [1][1][2][0][RTW89_WW][20] = 54, [1][1][2][0][RTW89_WW][24] = 54, [1][1][2][0][RTW89_WW][28] = 54, [1][1][2][0][RTW89_WW][32] = 54, - [1][1][2][0][RTW89_WW][36] = 66, + [1][1][2][0][RTW89_WW][36] = 52, [1][1][2][0][RTW89_WW][39] = 18, [1][1][2][0][RTW89_WW][43] = 18, - [1][1][2][0][RTW89_WW][47] = 60, - [1][1][2][0][RTW89_WW][51] = 58, + [1][1][2][0][RTW89_WW][47] = 62, + [1][1][2][0][RTW89_WW][51] = 60, [1][1][2][1][RTW89_WW][1] = 40, [1][1][2][1][RTW89_WW][5] = 40, [1][1][2][1][RTW89_WW][9] = 40, @@ -15035,2082 +29517,3694 @@ const s8 rtw89_8852c_txpwr_lmt_5g[RTW89_5G_BW_NUM][RTW89_NTX_NUM] [1][1][2][1][RTW89_WW][24] = 40, [1][1][2][1][RTW89_WW][28] = 40, [1][1][2][1][RTW89_WW][32] = 40, - [1][1][2][1][RTW89_WW][36] = 60, + [1][1][2][1][RTW89_WW][36] = 40, [1][1][2][1][RTW89_WW][39] = 6, [1][1][2][1][RTW89_WW][43] = 6, - [1][1][2][1][RTW89_WW][47] = 60, - [1][1][2][1][RTW89_WW][51] = 58, - [2][0][2][0][RTW89_WW][3] = 56, - [2][0][2][0][RTW89_WW][11] = 58, - [2][0][2][0][RTW89_WW][18] = 54, + [1][1][2][1][RTW89_WW][47] = 62, + [1][1][2][1][RTW89_WW][51] = 60, + [2][0][2][0][RTW89_WW][3] = 54, + [2][0][2][0][RTW89_WW][11] = 50, + [2][0][2][0][RTW89_WW][18] = 56, [2][0][2][0][RTW89_WW][26] = 60, [2][0][2][0][RTW89_WW][34] = 60, [2][0][2][0][RTW89_WW][41] = 30, - [2][0][2][0][RTW89_WW][49] = 56, - [2][1][2][0][RTW89_WW][3] = 48, - [2][1][2][0][RTW89_WW][11] = 52, - [2][1][2][0][RTW89_WW][18] = 48, - [2][1][2][0][RTW89_WW][26] = 54, - [2][1][2][0][RTW89_WW][34] = 60, + [2][0][2][0][RTW89_WW][49] = 62, + [2][1][2][0][RTW89_WW][3] = 46, + [2][1][2][0][RTW89_WW][11] = 38, + [2][1][2][0][RTW89_WW][18] = 50, + [2][1][2][0][RTW89_WW][26] = 52, + [2][1][2][0][RTW89_WW][34] = 52, [2][1][2][0][RTW89_WW][41] = 18, - [2][1][2][0][RTW89_WW][49] = 50, + [2][1][2][0][RTW89_WW][49] = 62, [2][1][2][1][RTW89_WW][3] = 40, - [2][1][2][1][RTW89_WW][11] = 40, + [2][1][2][1][RTW89_WW][11] = 38, [2][1][2][1][RTW89_WW][18] = 40, [2][1][2][1][RTW89_WW][26] = 42, - [2][1][2][1][RTW89_WW][34] = 60, + [2][1][2][1][RTW89_WW][34] = 40, [2][1][2][1][RTW89_WW][41] = 6, - [2][1][2][1][RTW89_WW][49] = 50, - [3][0][2][0][RTW89_WW][7] = 38, - [3][0][2][0][RTW89_WW][22] = 50, - [3][0][2][0][RTW89_WW][45] = 0, - [3][1][2][0][RTW89_WW][7] = 26, - [3][1][2][0][RTW89_WW][22] = 42, - [3][1][2][0][RTW89_WW][45] = 0, - [3][1][2][1][RTW89_WW][7] = 14, - [3][1][2][1][RTW89_WW][22] = 30, - [3][1][2][1][RTW89_WW][45] = 0, - [0][0][1][0][RTW89_FCC][0] = 70, + [2][1][2][1][RTW89_WW][49] = 62, + [3][0][2][0][RTW89_WW][7] = 40, + [3][0][2][0][RTW89_WW][22] = 42, + [3][0][2][0][RTW89_WW][45] = 52, + [3][1][2][0][RTW89_WW][7] = 32, + [3][1][2][0][RTW89_WW][22] = 36, + [3][1][2][0][RTW89_WW][45] = 46, + [3][1][2][1][RTW89_WW][7] = 32, + [3][1][2][1][RTW89_WW][22] = 36, + [3][1][2][1][RTW89_WW][45] = 46, + [0][0][1][0][RTW89_FCC][0] = 72, [0][0][1][0][RTW89_ETSI][0] = 66, [0][0][1][0][RTW89_MKK][0] = 66, - [0][0][1][0][RTW89_IC][0] = 62, - [0][0][1][0][RTW89_ACMA][0] = 60, - [0][0][1][0][RTW89_FCC][2] = 70, + [0][0][1][0][RTW89_IC][0] = 60, + [0][0][1][0][RTW89_KCC][0] = 52, + [0][0][1][0][RTW89_ACMA][0] = 66, + [0][0][1][0][RTW89_CN][0] = 50, + [0][0][1][0][RTW89_UK][0] = 66, + [0][0][1][0][RTW89_FCC][2] = 72, [0][0][1][0][RTW89_ETSI][2] = 66, [0][0][1][0][RTW89_MKK][2] = 66, - [0][0][1][0][RTW89_IC][2] = 62, - [0][0][1][0][RTW89_ACMA][2] = 60, - [0][0][1][0][RTW89_FCC][4] = 70, + [0][0][1][0][RTW89_IC][2] = 60, + [0][0][1][0][RTW89_KCC][2] = 52, + [0][0][1][0][RTW89_ACMA][2] = 66, + [0][0][1][0][RTW89_CN][2] = 50, + [0][0][1][0][RTW89_UK][2] = 66, + [0][0][1][0][RTW89_FCC][4] = 72, [0][0][1][0][RTW89_ETSI][4] = 66, [0][0][1][0][RTW89_MKK][4] = 66, - [0][0][1][0][RTW89_IC][4] = 62, - [0][0][1][0][RTW89_ACMA][4] = 60, - [0][0][1][0][RTW89_FCC][6] = 70, + [0][0][1][0][RTW89_IC][4] = 60, + [0][0][1][0][RTW89_KCC][4] = 52, + [0][0][1][0][RTW89_ACMA][4] = 66, + [0][0][1][0][RTW89_CN][4] = 50, + [0][0][1][0][RTW89_UK][4] = 66, + [0][0][1][0][RTW89_FCC][6] = 72, [0][0][1][0][RTW89_ETSI][6] = 66, [0][0][1][0][RTW89_MKK][6] = 66, - [0][0][1][0][RTW89_IC][6] = 62, - [0][0][1][0][RTW89_ACMA][6] = 60, - [0][0][1][0][RTW89_FCC][8] = 70, + [0][0][1][0][RTW89_IC][6] = 58, + [0][0][1][0][RTW89_KCC][6] = 62, + [0][0][1][0][RTW89_ACMA][6] = 66, + [0][0][1][0][RTW89_CN][6] = 50, + [0][0][1][0][RTW89_UK][6] = 66, + [0][0][1][0][RTW89_FCC][8] = 72, [0][0][1][0][RTW89_ETSI][8] = 66, [0][0][1][0][RTW89_MKK][8] = 66, - [0][0][1][0][RTW89_IC][8] = 66, - [0][0][1][0][RTW89_ACMA][8] = 60, - [0][0][1][0][RTW89_FCC][10] = 70, + [0][0][1][0][RTW89_IC][8] = 64, + [0][0][1][0][RTW89_KCC][8] = 70, + [0][0][1][0][RTW89_ACMA][8] = 66, + [0][0][1][0][RTW89_CN][8] = 50, + [0][0][1][0][RTW89_UK][8] = 66, + [0][0][1][0][RTW89_FCC][10] = 72, [0][0][1][0][RTW89_ETSI][10] = 66, [0][0][1][0][RTW89_MKK][10] = 66, - [0][0][1][0][RTW89_IC][10] = 66, - [0][0][1][0][RTW89_ACMA][10] = 60, - [0][0][1][0][RTW89_FCC][12] = 70, + [0][0][1][0][RTW89_IC][10] = 64, + [0][0][1][0][RTW89_KCC][10] = 70, + [0][0][1][0][RTW89_ACMA][10] = 66, + [0][0][1][0][RTW89_CN][10] = 50, + [0][0][1][0][RTW89_UK][10] = 66, + [0][0][1][0][RTW89_FCC][12] = 72, [0][0][1][0][RTW89_ETSI][12] = 66, [0][0][1][0][RTW89_MKK][12] = 66, - [0][0][1][0][RTW89_IC][12] = 66, - [0][0][1][0][RTW89_ACMA][12] = 60, + [0][0][1][0][RTW89_IC][12] = 64, + [0][0][1][0][RTW89_KCC][12] = 66, + [0][0][1][0][RTW89_ACMA][12] = 66, + [0][0][1][0][RTW89_CN][12] = 50, + [0][0][1][0][RTW89_UK][12] = 66, [0][0][1][0][RTW89_FCC][14] = 70, [0][0][1][0][RTW89_ETSI][14] = 66, [0][0][1][0][RTW89_MKK][14] = 66, - [0][0][1][0][RTW89_IC][14] = 66, - [0][0][1][0][RTW89_ACMA][14] = 60, - [0][0][1][0][RTW89_FCC][15] = 68, + [0][0][1][0][RTW89_IC][14] = 64, + [0][0][1][0][RTW89_KCC][14] = 66, + [0][0][1][0][RTW89_ACMA][14] = 66, + [0][0][1][0][RTW89_CN][14] = 50, + [0][0][1][0][RTW89_UK][14] = 66, + [0][0][1][0][RTW89_FCC][15] = 72, [0][0][1][0][RTW89_ETSI][15] = 66, [0][0][1][0][RTW89_MKK][15] = 70, - [0][0][1][0][RTW89_IC][15] = 70, - [0][0][1][0][RTW89_ACMA][15] = 60, - [0][0][1][0][RTW89_FCC][17] = 70, + [0][0][1][0][RTW89_IC][15] = 72, + [0][0][1][0][RTW89_KCC][15] = 70, + [0][0][1][0][RTW89_ACMA][15] = 66, + [0][0][1][0][RTW89_CN][15] = 127, + [0][0][1][0][RTW89_UK][15] = 66, + [0][0][1][0][RTW89_FCC][17] = 72, [0][0][1][0][RTW89_ETSI][17] = 66, [0][0][1][0][RTW89_MKK][17] = 70, - [0][0][1][0][RTW89_IC][17] = 70, - [0][0][1][0][RTW89_ACMA][17] = 60, - [0][0][1][0][RTW89_FCC][19] = 70, + [0][0][1][0][RTW89_IC][17] = 72, + [0][0][1][0][RTW89_KCC][17] = 70, + [0][0][1][0][RTW89_ACMA][17] = 66, + [0][0][1][0][RTW89_CN][17] = 127, + [0][0][1][0][RTW89_UK][17] = 66, + [0][0][1][0][RTW89_FCC][19] = 72, [0][0][1][0][RTW89_ETSI][19] = 66, [0][0][1][0][RTW89_MKK][19] = 70, - [0][0][1][0][RTW89_IC][19] = 70, - [0][0][1][0][RTW89_ACMA][19] = 60, - [0][0][1][0][RTW89_FCC][21] = 70, + [0][0][1][0][RTW89_IC][19] = 72, + [0][0][1][0][RTW89_KCC][19] = 70, + [0][0][1][0][RTW89_ACMA][19] = 66, + [0][0][1][0][RTW89_CN][19] = 127, + [0][0][1][0][RTW89_UK][19] = 66, + [0][0][1][0][RTW89_FCC][21] = 72, [0][0][1][0][RTW89_ETSI][21] = 66, [0][0][1][0][RTW89_MKK][21] = 70, - [0][0][1][0][RTW89_IC][21] = 70, - [0][0][1][0][RTW89_ACMA][21] = 60, - [0][0][1][0][RTW89_FCC][23] = 70, + [0][0][1][0][RTW89_IC][21] = 72, + [0][0][1][0][RTW89_KCC][21] = 70, + [0][0][1][0][RTW89_ACMA][21] = 66, + [0][0][1][0][RTW89_CN][21] = 127, + [0][0][1][0][RTW89_UK][21] = 66, + [0][0][1][0][RTW89_FCC][23] = 72, [0][0][1][0][RTW89_ETSI][23] = 66, [0][0][1][0][RTW89_MKK][23] = 70, - [0][0][1][0][RTW89_IC][23] = 70, - [0][0][1][0][RTW89_ACMA][23] = 60, - [0][0][1][0][RTW89_FCC][25] = 70, + [0][0][1][0][RTW89_IC][23] = 72, + [0][0][1][0][RTW89_KCC][23] = 70, + [0][0][1][0][RTW89_ACMA][23] = 66, + [0][0][1][0][RTW89_CN][23] = 127, + [0][0][1][0][RTW89_UK][23] = 66, + [0][0][1][0][RTW89_FCC][25] = 72, [0][0][1][0][RTW89_ETSI][25] = 66, [0][0][1][0][RTW89_MKK][25] = 70, [0][0][1][0][RTW89_IC][25] = 127, + [0][0][1][0][RTW89_KCC][25] = 70, [0][0][1][0][RTW89_ACMA][25] = 127, - [0][0][1][0][RTW89_FCC][27] = 70, + [0][0][1][0][RTW89_CN][25] = 127, + [0][0][1][0][RTW89_UK][25] = 66, + [0][0][1][0][RTW89_FCC][27] = 72, [0][0][1][0][RTW89_ETSI][27] = 66, [0][0][1][0][RTW89_MKK][27] = 70, [0][0][1][0][RTW89_IC][27] = 127, + [0][0][1][0][RTW89_KCC][27] = 70, [0][0][1][0][RTW89_ACMA][27] = 127, - [0][0][1][0][RTW89_FCC][29] = 70, + [0][0][1][0][RTW89_CN][27] = 127, + [0][0][1][0][RTW89_UK][27] = 66, + [0][0][1][0][RTW89_FCC][29] = 72, [0][0][1][0][RTW89_ETSI][29] = 66, [0][0][1][0][RTW89_MKK][29] = 70, [0][0][1][0][RTW89_IC][29] = 127, + [0][0][1][0][RTW89_KCC][29] = 70, [0][0][1][0][RTW89_ACMA][29] = 127, - [0][0][1][0][RTW89_FCC][31] = 70, + [0][0][1][0][RTW89_CN][29] = 127, + [0][0][1][0][RTW89_UK][29] = 66, + [0][0][1][0][RTW89_FCC][31] = 72, [0][0][1][0][RTW89_ETSI][31] = 66, [0][0][1][0][RTW89_MKK][31] = 70, - [0][0][1][0][RTW89_IC][31] = 70, - [0][0][1][0][RTW89_ACMA][31] = 60, - [0][0][1][0][RTW89_FCC][33] = 70, + [0][0][1][0][RTW89_IC][31] = 72, + [0][0][1][0][RTW89_KCC][31] = 70, + [0][0][1][0][RTW89_ACMA][31] = 66, + [0][0][1][0][RTW89_CN][31] = 127, + [0][0][1][0][RTW89_UK][31] = 66, + [0][0][1][0][RTW89_FCC][33] = 72, [0][0][1][0][RTW89_ETSI][33] = 66, [0][0][1][0][RTW89_MKK][33] = 70, - [0][0][1][0][RTW89_IC][33] = 70, - [0][0][1][0][RTW89_ACMA][33] = 60, - [0][0][1][0][RTW89_FCC][35] = 62, + [0][0][1][0][RTW89_IC][33] = 72, + [0][0][1][0][RTW89_KCC][33] = 70, + [0][0][1][0][RTW89_ACMA][33] = 66, + [0][0][1][0][RTW89_CN][33] = 127, + [0][0][1][0][RTW89_UK][33] = 66, + [0][0][1][0][RTW89_FCC][35] = 60, [0][0][1][0][RTW89_ETSI][35] = 66, [0][0][1][0][RTW89_MKK][35] = 70, - [0][0][1][0][RTW89_IC][35] = 70, - [0][0][1][0][RTW89_ACMA][35] = 60, - [0][0][1][0][RTW89_FCC][37] = 70, + [0][0][1][0][RTW89_IC][35] = 60, + [0][0][1][0][RTW89_KCC][35] = 70, + [0][0][1][0][RTW89_ACMA][35] = 66, + [0][0][1][0][RTW89_CN][35] = 127, + [0][0][1][0][RTW89_UK][35] = 66, + [0][0][1][0][RTW89_FCC][37] = 72, [0][0][1][0][RTW89_ETSI][37] = 127, [0][0][1][0][RTW89_MKK][37] = 70, - [0][0][1][0][RTW89_IC][37] = 70, + [0][0][1][0][RTW89_IC][37] = 72, + [0][0][1][0][RTW89_KCC][37] = 70, [0][0][1][0][RTW89_ACMA][37] = 70, - [0][0][1][0][RTW89_FCC][38] = 70, + [0][0][1][0][RTW89_CN][37] = 127, + [0][0][1][0][RTW89_UK][37] = 64, + [0][0][1][0][RTW89_FCC][38] = 72, [0][0][1][0][RTW89_ETSI][38] = 30, [0][0][1][0][RTW89_MKK][38] = 127, - [0][0][1][0][RTW89_IC][38] = 70, + [0][0][1][0][RTW89_IC][38] = 72, + [0][0][1][0][RTW89_KCC][38] = 62, [0][0][1][0][RTW89_ACMA][38] = 70, - [0][0][1][0][RTW89_FCC][40] = 70, + [0][0][1][0][RTW89_CN][38] = 68, + [0][0][1][0][RTW89_UK][38] = 64, + [0][0][1][0][RTW89_FCC][40] = 72, [0][0][1][0][RTW89_ETSI][40] = 30, [0][0][1][0][RTW89_MKK][40] = 127, - [0][0][1][0][RTW89_IC][40] = 70, + [0][0][1][0][RTW89_IC][40] = 72, + [0][0][1][0][RTW89_KCC][40] = 62, [0][0][1][0][RTW89_ACMA][40] = 70, - [0][0][1][0][RTW89_FCC][42] = 70, + [0][0][1][0][RTW89_CN][40] = 68, + [0][0][1][0][RTW89_UK][40] = 64, + [0][0][1][0][RTW89_FCC][42] = 72, [0][0][1][0][RTW89_ETSI][42] = 30, [0][0][1][0][RTW89_MKK][42] = 127, - [0][0][1][0][RTW89_IC][42] = 70, + [0][0][1][0][RTW89_IC][42] = 72, + [0][0][1][0][RTW89_KCC][42] = 62, [0][0][1][0][RTW89_ACMA][42] = 70, - [0][0][1][0][RTW89_FCC][44] = 70, + [0][0][1][0][RTW89_CN][42] = 68, + [0][0][1][0][RTW89_UK][42] = 64, + [0][0][1][0][RTW89_FCC][44] = 72, [0][0][1][0][RTW89_ETSI][44] = 30, [0][0][1][0][RTW89_MKK][44] = 127, - [0][0][1][0][RTW89_IC][44] = 70, + [0][0][1][0][RTW89_IC][44] = 72, + [0][0][1][0][RTW89_KCC][44] = 62, [0][0][1][0][RTW89_ACMA][44] = 70, - [0][0][1][0][RTW89_FCC][46] = 70, + [0][0][1][0][RTW89_CN][44] = 68, + [0][0][1][0][RTW89_UK][44] = 64, + [0][0][1][0][RTW89_FCC][46] = 72, [0][0][1][0][RTW89_ETSI][46] = 30, [0][0][1][0][RTW89_MKK][46] = 127, - [0][0][1][0][RTW89_IC][46] = 70, + [0][0][1][0][RTW89_IC][46] = 72, + [0][0][1][0][RTW89_KCC][46] = 62, [0][0][1][0][RTW89_ACMA][46] = 70, - [0][0][1][0][RTW89_FCC][48] = 70, + [0][0][1][0][RTW89_CN][46] = 68, + [0][0][1][0][RTW89_UK][46] = 64, + [0][0][1][0][RTW89_FCC][48] = 72, [0][0][1][0][RTW89_ETSI][48] = 127, [0][0][1][0][RTW89_MKK][48] = 127, [0][0][1][0][RTW89_IC][48] = 127, + [0][0][1][0][RTW89_KCC][48] = 127, [0][0][1][0][RTW89_ACMA][48] = 127, - [0][0][1][0][RTW89_FCC][50] = 70, + [0][0][1][0][RTW89_CN][48] = 127, + [0][0][1][0][RTW89_UK][48] = 127, + [0][0][1][0][RTW89_FCC][50] = 72, [0][0][1][0][RTW89_ETSI][50] = 127, [0][0][1][0][RTW89_MKK][50] = 127, [0][0][1][0][RTW89_IC][50] = 127, + [0][0][1][0][RTW89_KCC][50] = 127, [0][0][1][0][RTW89_ACMA][50] = 127, - [0][0][1][0][RTW89_FCC][52] = 70, + [0][0][1][0][RTW89_CN][50] = 127, + [0][0][1][0][RTW89_UK][50] = 127, + [0][0][1][0][RTW89_FCC][52] = 72, [0][0][1][0][RTW89_ETSI][52] = 127, [0][0][1][0][RTW89_MKK][52] = 127, [0][0][1][0][RTW89_IC][52] = 127, + [0][0][1][0][RTW89_KCC][52] = 127, [0][0][1][0][RTW89_ACMA][52] = 127, + [0][0][1][0][RTW89_CN][52] = 127, + [0][0][1][0][RTW89_UK][52] = 127, [0][1][1][0][RTW89_FCC][0] = 60, [0][1][1][0][RTW89_ETSI][0] = 54, [0][1][1][0][RTW89_MKK][0] = 54, - [0][1][1][0][RTW89_IC][0] = 42, - [0][1][1][0][RTW89_ACMA][0] = 48, + [0][1][1][0][RTW89_IC][0] = 34, + [0][1][1][0][RTW89_KCC][0] = 40, + [0][1][1][0][RTW89_ACMA][0] = 54, + [0][1][1][0][RTW89_CN][0] = 46, + [0][1][1][0][RTW89_UK][0] = 54, [0][1][1][0][RTW89_FCC][2] = 60, [0][1][1][0][RTW89_ETSI][2] = 54, [0][1][1][0][RTW89_MKK][2] = 54, - [0][1][1][0][RTW89_IC][2] = 42, - [0][1][1][0][RTW89_ACMA][2] = 48, + [0][1][1][0][RTW89_IC][2] = 34, + [0][1][1][0][RTW89_KCC][2] = 40, + [0][1][1][0][RTW89_ACMA][2] = 54, + [0][1][1][0][RTW89_CN][2] = 46, + [0][1][1][0][RTW89_UK][2] = 54, [0][1][1][0][RTW89_FCC][4] = 60, [0][1][1][0][RTW89_ETSI][4] = 54, [0][1][1][0][RTW89_MKK][4] = 54, - [0][1][1][0][RTW89_IC][4] = 42, - [0][1][1][0][RTW89_ACMA][4] = 48, + [0][1][1][0][RTW89_IC][4] = 34, + [0][1][1][0][RTW89_KCC][4] = 40, + [0][1][1][0][RTW89_ACMA][4] = 54, + [0][1][1][0][RTW89_CN][4] = 46, + [0][1][1][0][RTW89_UK][4] = 54, [0][1][1][0][RTW89_FCC][6] = 60, [0][1][1][0][RTW89_ETSI][6] = 54, [0][1][1][0][RTW89_MKK][6] = 54, - [0][1][1][0][RTW89_IC][6] = 42, - [0][1][1][0][RTW89_ACMA][6] = 48, - [0][1][1][0][RTW89_FCC][8] = 60, + [0][1][1][0][RTW89_IC][6] = 36, + [0][1][1][0][RTW89_KCC][6] = 60, + [0][1][1][0][RTW89_ACMA][6] = 54, + [0][1][1][0][RTW89_CN][6] = 46, + [0][1][1][0][RTW89_UK][6] = 54, + [0][1][1][0][RTW89_FCC][8] = 62, [0][1][1][0][RTW89_ETSI][8] = 54, [0][1][1][0][RTW89_MKK][8] = 52, - [0][1][1][0][RTW89_IC][8] = 54, - [0][1][1][0][RTW89_ACMA][8] = 48, - [0][1][1][0][RTW89_FCC][10] = 60, + [0][1][1][0][RTW89_IC][8] = 52, + [0][1][1][0][RTW89_KCC][8] = 60, + [0][1][1][0][RTW89_ACMA][8] = 54, + [0][1][1][0][RTW89_CN][8] = 46, + [0][1][1][0][RTW89_UK][8] = 54, + [0][1][1][0][RTW89_FCC][10] = 62, [0][1][1][0][RTW89_ETSI][10] = 54, [0][1][1][0][RTW89_MKK][10] = 54, - [0][1][1][0][RTW89_IC][10] = 54, - [0][1][1][0][RTW89_ACMA][10] = 48, - [0][1][1][0][RTW89_FCC][12] = 60, + [0][1][1][0][RTW89_IC][10] = 52, + [0][1][1][0][RTW89_KCC][10] = 60, + [0][1][1][0][RTW89_ACMA][10] = 54, + [0][1][1][0][RTW89_CN][10] = 46, + [0][1][1][0][RTW89_UK][10] = 54, + [0][1][1][0][RTW89_FCC][12] = 62, [0][1][1][0][RTW89_ETSI][12] = 54, [0][1][1][0][RTW89_MKK][12] = 54, - [0][1][1][0][RTW89_IC][12] = 54, - [0][1][1][0][RTW89_ACMA][12] = 48, + [0][1][1][0][RTW89_IC][12] = 52, + [0][1][1][0][RTW89_KCC][12] = 60, + [0][1][1][0][RTW89_ACMA][12] = 54, + [0][1][1][0][RTW89_CN][12] = 46, + [0][1][1][0][RTW89_UK][12] = 54, [0][1][1][0][RTW89_FCC][14] = 60, [0][1][1][0][RTW89_ETSI][14] = 54, [0][1][1][0][RTW89_MKK][14] = 54, - [0][1][1][0][RTW89_IC][14] = 54, - [0][1][1][0][RTW89_ACMA][14] = 48, - [0][1][1][0][RTW89_FCC][15] = 58, + [0][1][1][0][RTW89_IC][14] = 52, + [0][1][1][0][RTW89_KCC][14] = 60, + [0][1][1][0][RTW89_ACMA][14] = 54, + [0][1][1][0][RTW89_CN][14] = 46, + [0][1][1][0][RTW89_UK][14] = 54, + [0][1][1][0][RTW89_FCC][15] = 60, [0][1][1][0][RTW89_ETSI][15] = 54, [0][1][1][0][RTW89_MKK][15] = 70, - [0][1][1][0][RTW89_IC][15] = 68, - [0][1][1][0][RTW89_ACMA][15] = 48, + [0][1][1][0][RTW89_IC][15] = 60, + [0][1][1][0][RTW89_KCC][15] = 60, + [0][1][1][0][RTW89_ACMA][15] = 54, + [0][1][1][0][RTW89_CN][15] = 127, + [0][1][1][0][RTW89_UK][15] = 54, [0][1][1][0][RTW89_FCC][17] = 60, [0][1][1][0][RTW89_ETSI][17] = 54, [0][1][1][0][RTW89_MKK][17] = 70, - [0][1][1][0][RTW89_IC][17] = 70, - [0][1][1][0][RTW89_ACMA][17] = 48, + [0][1][1][0][RTW89_IC][17] = 60, + [0][1][1][0][RTW89_KCC][17] = 60, + [0][1][1][0][RTW89_ACMA][17] = 54, + [0][1][1][0][RTW89_CN][17] = 127, + [0][1][1][0][RTW89_UK][17] = 54, [0][1][1][0][RTW89_FCC][19] = 60, [0][1][1][0][RTW89_ETSI][19] = 54, [0][1][1][0][RTW89_MKK][19] = 70, - [0][1][1][0][RTW89_IC][19] = 70, - [0][1][1][0][RTW89_ACMA][19] = 48, + [0][1][1][0][RTW89_IC][19] = 60, + [0][1][1][0][RTW89_KCC][19] = 60, + [0][1][1][0][RTW89_ACMA][19] = 54, + [0][1][1][0][RTW89_CN][19] = 127, + [0][1][1][0][RTW89_UK][19] = 54, [0][1][1][0][RTW89_FCC][21] = 60, [0][1][1][0][RTW89_ETSI][21] = 54, [0][1][1][0][RTW89_MKK][21] = 70, - [0][1][1][0][RTW89_IC][21] = 70, - [0][1][1][0][RTW89_ACMA][21] = 48, + [0][1][1][0][RTW89_IC][21] = 60, + [0][1][1][0][RTW89_KCC][21] = 60, + [0][1][1][0][RTW89_ACMA][21] = 54, + [0][1][1][0][RTW89_CN][21] = 127, + [0][1][1][0][RTW89_UK][21] = 54, [0][1][1][0][RTW89_FCC][23] = 60, [0][1][1][0][RTW89_ETSI][23] = 54, [0][1][1][0][RTW89_MKK][23] = 70, - [0][1][1][0][RTW89_IC][23] = 70, - [0][1][1][0][RTW89_ACMA][23] = 48, + [0][1][1][0][RTW89_IC][23] = 60, + [0][1][1][0][RTW89_KCC][23] = 60, + [0][1][1][0][RTW89_ACMA][23] = 54, + [0][1][1][0][RTW89_CN][23] = 127, + [0][1][1][0][RTW89_UK][23] = 54, [0][1][1][0][RTW89_FCC][25] = 60, [0][1][1][0][RTW89_ETSI][25] = 54, [0][1][1][0][RTW89_MKK][25] = 70, [0][1][1][0][RTW89_IC][25] = 127, + [0][1][1][0][RTW89_KCC][25] = 60, [0][1][1][0][RTW89_ACMA][25] = 127, + [0][1][1][0][RTW89_CN][25] = 127, + [0][1][1][0][RTW89_UK][25] = 54, [0][1][1][0][RTW89_FCC][27] = 60, [0][1][1][0][RTW89_ETSI][27] = 54, [0][1][1][0][RTW89_MKK][27] = 70, [0][1][1][0][RTW89_IC][27] = 127, + [0][1][1][0][RTW89_KCC][27] = 60, [0][1][1][0][RTW89_ACMA][27] = 127, + [0][1][1][0][RTW89_CN][27] = 127, + [0][1][1][0][RTW89_UK][27] = 54, [0][1][1][0][RTW89_FCC][29] = 60, [0][1][1][0][RTW89_ETSI][29] = 54, [0][1][1][0][RTW89_MKK][29] = 70, [0][1][1][0][RTW89_IC][29] = 127, + [0][1][1][0][RTW89_KCC][29] = 60, [0][1][1][0][RTW89_ACMA][29] = 127, + [0][1][1][0][RTW89_CN][29] = 127, + [0][1][1][0][RTW89_UK][29] = 54, [0][1][1][0][RTW89_FCC][31] = 60, [0][1][1][0][RTW89_ETSI][31] = 54, [0][1][1][0][RTW89_MKK][31] = 70, - [0][1][1][0][RTW89_IC][31] = 70, - [0][1][1][0][RTW89_ACMA][31] = 48, + [0][1][1][0][RTW89_IC][31] = 60, + [0][1][1][0][RTW89_KCC][31] = 58, + [0][1][1][0][RTW89_ACMA][31] = 54, + [0][1][1][0][RTW89_CN][31] = 127, + [0][1][1][0][RTW89_UK][31] = 54, [0][1][1][0][RTW89_FCC][33] = 60, [0][1][1][0][RTW89_ETSI][33] = 54, [0][1][1][0][RTW89_MKK][33] = 70, - [0][1][1][0][RTW89_IC][33] = 70, - [0][1][1][0][RTW89_ACMA][33] = 48, - [0][1][1][0][RTW89_FCC][35] = 58, + [0][1][1][0][RTW89_IC][33] = 60, + [0][1][1][0][RTW89_KCC][33] = 58, + [0][1][1][0][RTW89_ACMA][33] = 54, + [0][1][1][0][RTW89_CN][33] = 127, + [0][1][1][0][RTW89_UK][33] = 54, + [0][1][1][0][RTW89_FCC][35] = 52, [0][1][1][0][RTW89_ETSI][35] = 54, [0][1][1][0][RTW89_MKK][35] = 70, - [0][1][1][0][RTW89_IC][35] = 68, - [0][1][1][0][RTW89_ACMA][35] = 48, - [0][1][1][0][RTW89_FCC][37] = 60, + [0][1][1][0][RTW89_IC][35] = 52, + [0][1][1][0][RTW89_KCC][35] = 58, + [0][1][1][0][RTW89_ACMA][35] = 54, + [0][1][1][0][RTW89_CN][35] = 127, + [0][1][1][0][RTW89_UK][35] = 54, + [0][1][1][0][RTW89_FCC][37] = 62, [0][1][1][0][RTW89_ETSI][37] = 127, [0][1][1][0][RTW89_MKK][37] = 70, - [0][1][1][0][RTW89_IC][37] = 70, - [0][1][1][0][RTW89_ACMA][37] = 70, - [0][1][1][0][RTW89_FCC][38] = 70, + [0][1][1][0][RTW89_IC][37] = 62, + [0][1][1][0][RTW89_KCC][37] = 58, + [0][1][1][0][RTW89_ACMA][37] = 64, + [0][1][1][0][RTW89_CN][37] = 127, + [0][1][1][0][RTW89_UK][37] = 52, + [0][1][1][0][RTW89_FCC][38] = 72, [0][1][1][0][RTW89_ETSI][38] = 18, [0][1][1][0][RTW89_MKK][38] = 127, - [0][1][1][0][RTW89_IC][38] = 70, + [0][1][1][0][RTW89_IC][38] = 72, + [0][1][1][0][RTW89_KCC][38] = 60, [0][1][1][0][RTW89_ACMA][38] = 70, - [0][1][1][0][RTW89_FCC][40] = 70, + [0][1][1][0][RTW89_CN][38] = 64, + [0][1][1][0][RTW89_UK][38] = 52, + [0][1][1][0][RTW89_FCC][40] = 72, [0][1][1][0][RTW89_ETSI][40] = 18, [0][1][1][0][RTW89_MKK][40] = 127, - [0][1][1][0][RTW89_IC][40] = 70, - [0][1][1][0][RTW89_ACMA][40] = 16, - [0][1][1][0][RTW89_FCC][42] = 70, + [0][1][1][0][RTW89_IC][40] = 72, + [0][1][1][0][RTW89_KCC][40] = 60, + [0][1][1][0][RTW89_ACMA][40] = 70, + [0][1][1][0][RTW89_CN][40] = 64, + [0][1][1][0][RTW89_UK][40] = 52, + [0][1][1][0][RTW89_FCC][42] = 72, [0][1][1][0][RTW89_ETSI][42] = 18, [0][1][1][0][RTW89_MKK][42] = 127, - [0][1][1][0][RTW89_IC][42] = 70, + [0][1][1][0][RTW89_IC][42] = 72, + [0][1][1][0][RTW89_KCC][42] = 60, [0][1][1][0][RTW89_ACMA][42] = 70, - [0][1][1][0][RTW89_FCC][44] = 70, + [0][1][1][0][RTW89_CN][42] = 64, + [0][1][1][0][RTW89_UK][42] = 52, + [0][1][1][0][RTW89_FCC][44] = 72, [0][1][1][0][RTW89_ETSI][44] = 18, [0][1][1][0][RTW89_MKK][44] = 127, - [0][1][1][0][RTW89_IC][44] = 70, - [0][1][1][0][RTW89_ACMA][44] = 16, - [0][1][1][0][RTW89_FCC][46] = 70, + [0][1][1][0][RTW89_IC][44] = 72, + [0][1][1][0][RTW89_KCC][44] = 60, + [0][1][1][0][RTW89_ACMA][44] = 70, + [0][1][1][0][RTW89_CN][44] = 60, + [0][1][1][0][RTW89_UK][44] = 52, + [0][1][1][0][RTW89_FCC][46] = 72, [0][1][1][0][RTW89_ETSI][46] = 18, [0][1][1][0][RTW89_MKK][46] = 127, - [0][1][1][0][RTW89_IC][46] = 70, + [0][1][1][0][RTW89_IC][46] = 72, + [0][1][1][0][RTW89_KCC][46] = 60, [0][1][1][0][RTW89_ACMA][46] = 70, + [0][1][1][0][RTW89_CN][46] = 60, + [0][1][1][0][RTW89_UK][46] = 52, [0][1][1][0][RTW89_FCC][48] = 48, [0][1][1][0][RTW89_ETSI][48] = 127, [0][1][1][0][RTW89_MKK][48] = 127, [0][1][1][0][RTW89_IC][48] = 127, + [0][1][1][0][RTW89_KCC][48] = 127, [0][1][1][0][RTW89_ACMA][48] = 127, + [0][1][1][0][RTW89_CN][48] = 127, + [0][1][1][0][RTW89_UK][48] = 127, [0][1][1][0][RTW89_FCC][50] = 48, [0][1][1][0][RTW89_ETSI][50] = 127, [0][1][1][0][RTW89_MKK][50] = 127, [0][1][1][0][RTW89_IC][50] = 127, + [0][1][1][0][RTW89_KCC][50] = 127, [0][1][1][0][RTW89_ACMA][50] = 127, + [0][1][1][0][RTW89_CN][50] = 127, + [0][1][1][0][RTW89_UK][50] = 127, [0][1][1][0][RTW89_FCC][52] = 48, [0][1][1][0][RTW89_ETSI][52] = 127, [0][1][1][0][RTW89_MKK][52] = 127, [0][1][1][0][RTW89_IC][52] = 127, + [0][1][1][0][RTW89_KCC][52] = 127, [0][1][1][0][RTW89_ACMA][52] = 127, + [0][1][1][0][RTW89_CN][52] = 127, + [0][1][1][0][RTW89_UK][52] = 127, [0][0][2][0][RTW89_FCC][0] = 70, [0][0][2][0][RTW89_ETSI][0] = 66, [0][0][2][0][RTW89_MKK][0] = 68, - [0][0][2][0][RTW89_IC][0] = 66, - [0][0][2][0][RTW89_ACMA][0] = 62, - [0][0][2][0][RTW89_FCC][2] = 70, + [0][0][2][0][RTW89_IC][0] = 60, + [0][0][2][0][RTW89_KCC][0] = 54, + [0][0][2][0][RTW89_ACMA][0] = 66, + [0][0][2][0][RTW89_CN][0] = 52, + [0][0][2][0][RTW89_UK][0] = 66, + [0][0][2][0][RTW89_FCC][2] = 72, [0][0][2][0][RTW89_ETSI][2] = 66, [0][0][2][0][RTW89_MKK][2] = 68, - [0][0][2][0][RTW89_IC][2] = 66, - [0][0][2][0][RTW89_ACMA][2] = 62, - [0][0][2][0][RTW89_FCC][4] = 70, + [0][0][2][0][RTW89_IC][2] = 60, + [0][0][2][0][RTW89_KCC][2] = 54, + [0][0][2][0][RTW89_ACMA][2] = 66, + [0][0][2][0][RTW89_CN][2] = 52, + [0][0][2][0][RTW89_UK][2] = 66, + [0][0][2][0][RTW89_FCC][4] = 72, [0][0][2][0][RTW89_ETSI][4] = 66, [0][0][2][0][RTW89_MKK][4] = 68, - [0][0][2][0][RTW89_IC][4] = 66, - [0][0][2][0][RTW89_ACMA][4] = 62, - [0][0][2][0][RTW89_FCC][6] = 70, + [0][0][2][0][RTW89_IC][4] = 60, + [0][0][2][0][RTW89_KCC][4] = 54, + [0][0][2][0][RTW89_ACMA][4] = 66, + [0][0][2][0][RTW89_CN][4] = 52, + [0][0][2][0][RTW89_UK][4] = 66, + [0][0][2][0][RTW89_FCC][6] = 72, [0][0][2][0][RTW89_ETSI][6] = 66, [0][0][2][0][RTW89_MKK][6] = 60, - [0][0][2][0][RTW89_IC][6] = 66, - [0][0][2][0][RTW89_ACMA][6] = 62, - [0][0][2][0][RTW89_FCC][8] = 70, + [0][0][2][0][RTW89_IC][6] = 60, + [0][0][2][0][RTW89_KCC][6] = 68, + [0][0][2][0][RTW89_ACMA][6] = 66, + [0][0][2][0][RTW89_CN][6] = 52, + [0][0][2][0][RTW89_UK][6] = 66, + [0][0][2][0][RTW89_FCC][8] = 72, [0][0][2][0][RTW89_ETSI][8] = 66, [0][0][2][0][RTW89_MKK][8] = 58, - [0][0][2][0][RTW89_IC][8] = 66, - [0][0][2][0][RTW89_ACMA][8] = 62, - [0][0][2][0][RTW89_FCC][10] = 70, + [0][0][2][0][RTW89_IC][8] = 64, + [0][0][2][0][RTW89_KCC][8] = 70, + [0][0][2][0][RTW89_ACMA][8] = 66, + [0][0][2][0][RTW89_CN][8] = 52, + [0][0][2][0][RTW89_UK][8] = 66, + [0][0][2][0][RTW89_FCC][10] = 72, [0][0][2][0][RTW89_ETSI][10] = 66, [0][0][2][0][RTW89_MKK][10] = 70, - [0][0][2][0][RTW89_IC][10] = 66, - [0][0][2][0][RTW89_ACMA][10] = 62, - [0][0][2][0][RTW89_FCC][12] = 70, + [0][0][2][0][RTW89_IC][10] = 64, + [0][0][2][0][RTW89_KCC][10] = 70, + [0][0][2][0][RTW89_ACMA][10] = 66, + [0][0][2][0][RTW89_CN][10] = 52, + [0][0][2][0][RTW89_UK][10] = 66, + [0][0][2][0][RTW89_FCC][12] = 72, [0][0][2][0][RTW89_ETSI][12] = 66, [0][0][2][0][RTW89_MKK][12] = 70, - [0][0][2][0][RTW89_IC][12] = 66, - [0][0][2][0][RTW89_ACMA][12] = 62, - [0][0][2][0][RTW89_FCC][14] = 70, + [0][0][2][0][RTW89_IC][12] = 64, + [0][0][2][0][RTW89_KCC][12] = 66, + [0][0][2][0][RTW89_ACMA][12] = 66, + [0][0][2][0][RTW89_CN][12] = 52, + [0][0][2][0][RTW89_UK][12] = 66, + [0][0][2][0][RTW89_FCC][14] = 68, [0][0][2][0][RTW89_ETSI][14] = 66, [0][0][2][0][RTW89_MKK][14] = 70, - [0][0][2][0][RTW89_IC][14] = 66, - [0][0][2][0][RTW89_ACMA][14] = 62, - [0][0][2][0][RTW89_FCC][15] = 66, + [0][0][2][0][RTW89_IC][14] = 64, + [0][0][2][0][RTW89_KCC][14] = 66, + [0][0][2][0][RTW89_ACMA][14] = 66, + [0][0][2][0][RTW89_CN][14] = 52, + [0][0][2][0][RTW89_UK][14] = 66, + [0][0][2][0][RTW89_FCC][15] = 70, [0][0][2][0][RTW89_ETSI][15] = 66, [0][0][2][0][RTW89_MKK][15] = 70, [0][0][2][0][RTW89_IC][15] = 70, - [0][0][2][0][RTW89_ACMA][15] = 62, - [0][0][2][0][RTW89_FCC][17] = 70, + [0][0][2][0][RTW89_KCC][15] = 70, + [0][0][2][0][RTW89_ACMA][15] = 66, + [0][0][2][0][RTW89_CN][15] = 127, + [0][0][2][0][RTW89_UK][15] = 66, + [0][0][2][0][RTW89_FCC][17] = 72, [0][0][2][0][RTW89_ETSI][17] = 66, [0][0][2][0][RTW89_MKK][17] = 70, - [0][0][2][0][RTW89_IC][17] = 70, - [0][0][2][0][RTW89_ACMA][17] = 62, - [0][0][2][0][RTW89_FCC][19] = 70, + [0][0][2][0][RTW89_IC][17] = 72, + [0][0][2][0][RTW89_KCC][17] = 70, + [0][0][2][0][RTW89_ACMA][17] = 66, + [0][0][2][0][RTW89_CN][17] = 127, + [0][0][2][0][RTW89_UK][17] = 66, + [0][0][2][0][RTW89_FCC][19] = 72, [0][0][2][0][RTW89_ETSI][19] = 66, [0][0][2][0][RTW89_MKK][19] = 70, - [0][0][2][0][RTW89_IC][19] = 70, - [0][0][2][0][RTW89_ACMA][19] = 62, - [0][0][2][0][RTW89_FCC][21] = 70, + [0][0][2][0][RTW89_IC][19] = 72, + [0][0][2][0][RTW89_KCC][19] = 70, + [0][0][2][0][RTW89_ACMA][19] = 66, + [0][0][2][0][RTW89_CN][19] = 127, + [0][0][2][0][RTW89_UK][19] = 66, + [0][0][2][0][RTW89_FCC][21] = 72, [0][0][2][0][RTW89_ETSI][21] = 66, [0][0][2][0][RTW89_MKK][21] = 70, - [0][0][2][0][RTW89_IC][21] = 70, - [0][0][2][0][RTW89_ACMA][21] = 62, - [0][0][2][0][RTW89_FCC][23] = 70, + [0][0][2][0][RTW89_IC][21] = 72, + [0][0][2][0][RTW89_KCC][21] = 70, + [0][0][2][0][RTW89_ACMA][21] = 66, + [0][0][2][0][RTW89_CN][21] = 127, + [0][0][2][0][RTW89_UK][21] = 66, + [0][0][2][0][RTW89_FCC][23] = 72, [0][0][2][0][RTW89_ETSI][23] = 66, [0][0][2][0][RTW89_MKK][23] = 70, - [0][0][2][0][RTW89_IC][23] = 70, - [0][0][2][0][RTW89_ACMA][23] = 62, - [0][0][2][0][RTW89_FCC][25] = 70, + [0][0][2][0][RTW89_IC][23] = 72, + [0][0][2][0][RTW89_KCC][23] = 70, + [0][0][2][0][RTW89_ACMA][23] = 66, + [0][0][2][0][RTW89_CN][23] = 127, + [0][0][2][0][RTW89_UK][23] = 66, + [0][0][2][0][RTW89_FCC][25] = 72, [0][0][2][0][RTW89_ETSI][25] = 66, [0][0][2][0][RTW89_MKK][25] = 70, [0][0][2][0][RTW89_IC][25] = 127, + [0][0][2][0][RTW89_KCC][25] = 70, [0][0][2][0][RTW89_ACMA][25] = 127, - [0][0][2][0][RTW89_FCC][27] = 70, + [0][0][2][0][RTW89_CN][25] = 127, + [0][0][2][0][RTW89_UK][25] = 66, + [0][0][2][0][RTW89_FCC][27] = 72, [0][0][2][0][RTW89_ETSI][27] = 66, [0][0][2][0][RTW89_MKK][27] = 70, [0][0][2][0][RTW89_IC][27] = 127, + [0][0][2][0][RTW89_KCC][27] = 70, [0][0][2][0][RTW89_ACMA][27] = 127, - [0][0][2][0][RTW89_FCC][29] = 70, + [0][0][2][0][RTW89_CN][27] = 127, + [0][0][2][0][RTW89_UK][27] = 66, + [0][0][2][0][RTW89_FCC][29] = 72, [0][0][2][0][RTW89_ETSI][29] = 66, [0][0][2][0][RTW89_MKK][29] = 70, [0][0][2][0][RTW89_IC][29] = 127, + [0][0][2][0][RTW89_KCC][29] = 70, [0][0][2][0][RTW89_ACMA][29] = 127, - [0][0][2][0][RTW89_FCC][31] = 70, + [0][0][2][0][RTW89_CN][29] = 127, + [0][0][2][0][RTW89_UK][29] = 66, + [0][0][2][0][RTW89_FCC][31] = 72, [0][0][2][0][RTW89_ETSI][31] = 66, [0][0][2][0][RTW89_MKK][31] = 70, - [0][0][2][0][RTW89_IC][31] = 70, - [0][0][2][0][RTW89_ACMA][31] = 62, - [0][0][2][0][RTW89_FCC][33] = 70, + [0][0][2][0][RTW89_IC][31] = 72, + [0][0][2][0][RTW89_KCC][31] = 70, + [0][0][2][0][RTW89_ACMA][31] = 66, + [0][0][2][0][RTW89_CN][31] = 127, + [0][0][2][0][RTW89_UK][31] = 66, + [0][0][2][0][RTW89_FCC][33] = 72, [0][0][2][0][RTW89_ETSI][33] = 66, [0][0][2][0][RTW89_MKK][33] = 70, - [0][0][2][0][RTW89_IC][33] = 70, - [0][0][2][0][RTW89_ACMA][33] = 62, - [0][0][2][0][RTW89_FCC][35] = 62, + [0][0][2][0][RTW89_IC][33] = 72, + [0][0][2][0][RTW89_KCC][33] = 70, + [0][0][2][0][RTW89_ACMA][33] = 66, + [0][0][2][0][RTW89_CN][33] = 127, + [0][0][2][0][RTW89_UK][33] = 66, + [0][0][2][0][RTW89_FCC][35] = 56, [0][0][2][0][RTW89_ETSI][35] = 66, [0][0][2][0][RTW89_MKK][35] = 70, - [0][0][2][0][RTW89_IC][35] = 70, - [0][0][2][0][RTW89_ACMA][35] = 62, - [0][0][2][0][RTW89_FCC][37] = 70, + [0][0][2][0][RTW89_IC][35] = 56, + [0][0][2][0][RTW89_KCC][35] = 70, + [0][0][2][0][RTW89_ACMA][35] = 66, + [0][0][2][0][RTW89_CN][35] = 127, + [0][0][2][0][RTW89_UK][35] = 66, + [0][0][2][0][RTW89_FCC][37] = 72, [0][0][2][0][RTW89_ETSI][37] = 127, [0][0][2][0][RTW89_MKK][37] = 70, - [0][0][2][0][RTW89_IC][37] = 70, + [0][0][2][0][RTW89_IC][37] = 72, + [0][0][2][0][RTW89_KCC][37] = 70, [0][0][2][0][RTW89_ACMA][37] = 70, - [0][0][2][0][RTW89_FCC][38] = 70, + [0][0][2][0][RTW89_CN][37] = 127, + [0][0][2][0][RTW89_UK][37] = 64, + [0][0][2][0][RTW89_FCC][38] = 72, [0][0][2][0][RTW89_ETSI][38] = 30, [0][0][2][0][RTW89_MKK][38] = 127, - [0][0][2][0][RTW89_IC][38] = 70, + [0][0][2][0][RTW89_IC][38] = 72, + [0][0][2][0][RTW89_KCC][38] = 58, [0][0][2][0][RTW89_ACMA][38] = 70, - [0][0][2][0][RTW89_FCC][40] = 70, + [0][0][2][0][RTW89_CN][38] = 68, + [0][0][2][0][RTW89_UK][38] = 64, + [0][0][2][0][RTW89_FCC][40] = 72, [0][0][2][0][RTW89_ETSI][40] = 30, [0][0][2][0][RTW89_MKK][40] = 127, - [0][0][2][0][RTW89_IC][40] = 70, + [0][0][2][0][RTW89_IC][40] = 72, + [0][0][2][0][RTW89_KCC][40] = 58, [0][0][2][0][RTW89_ACMA][40] = 70, - [0][0][2][0][RTW89_FCC][42] = 70, + [0][0][2][0][RTW89_CN][40] = 68, + [0][0][2][0][RTW89_UK][40] = 64, + [0][0][2][0][RTW89_FCC][42] = 72, [0][0][2][0][RTW89_ETSI][42] = 30, [0][0][2][0][RTW89_MKK][42] = 127, - [0][0][2][0][RTW89_IC][42] = 70, + [0][0][2][0][RTW89_IC][42] = 72, + [0][0][2][0][RTW89_KCC][42] = 58, [0][0][2][0][RTW89_ACMA][42] = 70, - [0][0][2][0][RTW89_FCC][44] = 70, + [0][0][2][0][RTW89_CN][42] = 68, + [0][0][2][0][RTW89_UK][42] = 64, + [0][0][2][0][RTW89_FCC][44] = 72, [0][0][2][0][RTW89_ETSI][44] = 30, [0][0][2][0][RTW89_MKK][44] = 127, - [0][0][2][0][RTW89_IC][44] = 70, + [0][0][2][0][RTW89_IC][44] = 72, + [0][0][2][0][RTW89_KCC][44] = 58, [0][0][2][0][RTW89_ACMA][44] = 70, - [0][0][2][0][RTW89_FCC][46] = 70, + [0][0][2][0][RTW89_CN][44] = 68, + [0][0][2][0][RTW89_UK][44] = 64, + [0][0][2][0][RTW89_FCC][46] = 72, [0][0][2][0][RTW89_ETSI][46] = 30, [0][0][2][0][RTW89_MKK][46] = 127, - [0][0][2][0][RTW89_IC][46] = 70, + [0][0][2][0][RTW89_IC][46] = 72, + [0][0][2][0][RTW89_KCC][46] = 58, [0][0][2][0][RTW89_ACMA][46] = 70, - [0][0][2][0][RTW89_FCC][48] = 70, + [0][0][2][0][RTW89_CN][46] = 68, + [0][0][2][0][RTW89_UK][46] = 64, + [0][0][2][0][RTW89_FCC][48] = 72, [0][0][2][0][RTW89_ETSI][48] = 127, [0][0][2][0][RTW89_MKK][48] = 127, [0][0][2][0][RTW89_IC][48] = 127, + [0][0][2][0][RTW89_KCC][48] = 127, [0][0][2][0][RTW89_ACMA][48] = 127, - [0][0][2][0][RTW89_FCC][50] = 70, + [0][0][2][0][RTW89_CN][48] = 127, + [0][0][2][0][RTW89_UK][48] = 127, + [0][0][2][0][RTW89_FCC][50] = 72, [0][0][2][0][RTW89_ETSI][50] = 127, [0][0][2][0][RTW89_MKK][50] = 127, [0][0][2][0][RTW89_IC][50] = 127, + [0][0][2][0][RTW89_KCC][50] = 127, [0][0][2][0][RTW89_ACMA][50] = 127, - [0][0][2][0][RTW89_FCC][52] = 70, + [0][0][2][0][RTW89_CN][50] = 127, + [0][0][2][0][RTW89_UK][50] = 127, + [0][0][2][0][RTW89_FCC][52] = 72, [0][0][2][0][RTW89_ETSI][52] = 127, [0][0][2][0][RTW89_MKK][52] = 127, [0][0][2][0][RTW89_IC][52] = 127, + [0][0][2][0][RTW89_KCC][52] = 127, [0][0][2][0][RTW89_ACMA][52] = 127, - [0][1][2][0][RTW89_FCC][0] = 62, + [0][0][2][0][RTW89_CN][52] = 127, + [0][0][2][0][RTW89_UK][52] = 127, + [0][1][2][0][RTW89_FCC][0] = 60, [0][1][2][0][RTW89_ETSI][0] = 54, [0][1][2][0][RTW89_MKK][0] = 54, - [0][1][2][0][RTW89_IC][0] = 44, - [0][1][2][0][RTW89_ACMA][0] = 50, + [0][1][2][0][RTW89_IC][0] = 36, + [0][1][2][0][RTW89_KCC][0] = 40, + [0][1][2][0][RTW89_ACMA][0] = 54, + [0][1][2][0][RTW89_CN][0] = 40, + [0][1][2][0][RTW89_UK][0] = 54, [0][1][2][0][RTW89_FCC][2] = 62, [0][1][2][0][RTW89_ETSI][2] = 54, [0][1][2][0][RTW89_MKK][2] = 54, - [0][1][2][0][RTW89_IC][2] = 44, - [0][1][2][0][RTW89_ACMA][2] = 50, + [0][1][2][0][RTW89_IC][2] = 36, + [0][1][2][0][RTW89_KCC][2] = 40, + [0][1][2][0][RTW89_ACMA][2] = 54, + [0][1][2][0][RTW89_CN][2] = 40, + [0][1][2][0][RTW89_UK][2] = 54, [0][1][2][0][RTW89_FCC][4] = 62, [0][1][2][0][RTW89_ETSI][4] = 54, [0][1][2][0][RTW89_MKK][4] = 54, - [0][1][2][0][RTW89_IC][4] = 44, - [0][1][2][0][RTW89_ACMA][4] = 50, + [0][1][2][0][RTW89_IC][4] = 36, + [0][1][2][0][RTW89_KCC][4] = 40, + [0][1][2][0][RTW89_ACMA][4] = 54, + [0][1][2][0][RTW89_CN][4] = 40, + [0][1][2][0][RTW89_UK][4] = 54, [0][1][2][0][RTW89_FCC][6] = 62, [0][1][2][0][RTW89_ETSI][6] = 54, [0][1][2][0][RTW89_MKK][6] = 50, - [0][1][2][0][RTW89_IC][6] = 44, - [0][1][2][0][RTW89_ACMA][6] = 50, + [0][1][2][0][RTW89_IC][6] = 38, + [0][1][2][0][RTW89_KCC][6] = 64, + [0][1][2][0][RTW89_ACMA][6] = 54, + [0][1][2][0][RTW89_CN][6] = 40, + [0][1][2][0][RTW89_UK][6] = 54, [0][1][2][0][RTW89_FCC][8] = 62, [0][1][2][0][RTW89_ETSI][8] = 54, [0][1][2][0][RTW89_MKK][8] = 42, - [0][1][2][0][RTW89_IC][8] = 54, - [0][1][2][0][RTW89_ACMA][8] = 50, + [0][1][2][0][RTW89_IC][8] = 52, + [0][1][2][0][RTW89_KCC][8] = 62, + [0][1][2][0][RTW89_ACMA][8] = 54, + [0][1][2][0][RTW89_CN][8] = 40, + [0][1][2][0][RTW89_UK][8] = 54, [0][1][2][0][RTW89_FCC][10] = 62, [0][1][2][0][RTW89_ETSI][10] = 54, [0][1][2][0][RTW89_MKK][10] = 54, - [0][1][2][0][RTW89_IC][10] = 54, - [0][1][2][0][RTW89_ACMA][10] = 50, + [0][1][2][0][RTW89_IC][10] = 52, + [0][1][2][0][RTW89_KCC][10] = 62, + [0][1][2][0][RTW89_ACMA][10] = 54, + [0][1][2][0][RTW89_CN][10] = 40, + [0][1][2][0][RTW89_UK][10] = 54, [0][1][2][0][RTW89_FCC][12] = 62, [0][1][2][0][RTW89_ETSI][12] = 54, [0][1][2][0][RTW89_MKK][12] = 54, - [0][1][2][0][RTW89_IC][12] = 54, - [0][1][2][0][RTW89_ACMA][12] = 50, + [0][1][2][0][RTW89_IC][12] = 52, + [0][1][2][0][RTW89_KCC][12] = 62, + [0][1][2][0][RTW89_ACMA][12] = 54, + [0][1][2][0][RTW89_CN][12] = 40, + [0][1][2][0][RTW89_UK][12] = 54, [0][1][2][0][RTW89_FCC][14] = 62, [0][1][2][0][RTW89_ETSI][14] = 54, [0][1][2][0][RTW89_MKK][14] = 54, - [0][1][2][0][RTW89_IC][14] = 54, - [0][1][2][0][RTW89_ACMA][14] = 50, + [0][1][2][0][RTW89_IC][14] = 52, + [0][1][2][0][RTW89_KCC][14] = 62, + [0][1][2][0][RTW89_ACMA][14] = 54, + [0][1][2][0][RTW89_CN][14] = 40, + [0][1][2][0][RTW89_UK][14] = 54, [0][1][2][0][RTW89_FCC][15] = 60, [0][1][2][0][RTW89_ETSI][15] = 54, [0][1][2][0][RTW89_MKK][15] = 68, - [0][1][2][0][RTW89_IC][15] = 70, - [0][1][2][0][RTW89_ACMA][15] = 50, + [0][1][2][0][RTW89_IC][15] = 60, + [0][1][2][0][RTW89_KCC][15] = 64, + [0][1][2][0][RTW89_ACMA][15] = 54, + [0][1][2][0][RTW89_CN][15] = 127, + [0][1][2][0][RTW89_UK][15] = 54, [0][1][2][0][RTW89_FCC][17] = 62, [0][1][2][0][RTW89_ETSI][17] = 54, [0][1][2][0][RTW89_MKK][17] = 68, - [0][1][2][0][RTW89_IC][17] = 70, - [0][1][2][0][RTW89_ACMA][17] = 50, + [0][1][2][0][RTW89_IC][17] = 62, + [0][1][2][0][RTW89_KCC][17] = 64, + [0][1][2][0][RTW89_ACMA][17] = 54, + [0][1][2][0][RTW89_CN][17] = 127, + [0][1][2][0][RTW89_UK][17] = 54, [0][1][2][0][RTW89_FCC][19] = 62, [0][1][2][0][RTW89_ETSI][19] = 54, [0][1][2][0][RTW89_MKK][19] = 68, - [0][1][2][0][RTW89_IC][19] = 70, - [0][1][2][0][RTW89_ACMA][19] = 50, + [0][1][2][0][RTW89_IC][19] = 62, + [0][1][2][0][RTW89_KCC][19] = 64, + [0][1][2][0][RTW89_ACMA][19] = 54, + [0][1][2][0][RTW89_CN][19] = 127, + [0][1][2][0][RTW89_UK][19] = 54, [0][1][2][0][RTW89_FCC][21] = 62, [0][1][2][0][RTW89_ETSI][21] = 54, [0][1][2][0][RTW89_MKK][21] = 68, - [0][1][2][0][RTW89_IC][21] = 70, - [0][1][2][0][RTW89_ACMA][21] = 50, + [0][1][2][0][RTW89_IC][21] = 62, + [0][1][2][0][RTW89_KCC][21] = 64, + [0][1][2][0][RTW89_ACMA][21] = 54, + [0][1][2][0][RTW89_CN][21] = 127, + [0][1][2][0][RTW89_UK][21] = 54, [0][1][2][0][RTW89_FCC][23] = 62, [0][1][2][0][RTW89_ETSI][23] = 54, [0][1][2][0][RTW89_MKK][23] = 68, - [0][1][2][0][RTW89_IC][23] = 70, - [0][1][2][0][RTW89_ACMA][23] = 50, + [0][1][2][0][RTW89_IC][23] = 62, + [0][1][2][0][RTW89_KCC][23] = 64, + [0][1][2][0][RTW89_ACMA][23] = 54, + [0][1][2][0][RTW89_CN][23] = 127, + [0][1][2][0][RTW89_UK][23] = 54, [0][1][2][0][RTW89_FCC][25] = 62, [0][1][2][0][RTW89_ETSI][25] = 54, [0][1][2][0][RTW89_MKK][25] = 68, [0][1][2][0][RTW89_IC][25] = 127, + [0][1][2][0][RTW89_KCC][25] = 64, [0][1][2][0][RTW89_ACMA][25] = 127, + [0][1][2][0][RTW89_CN][25] = 127, + [0][1][2][0][RTW89_UK][25] = 54, [0][1][2][0][RTW89_FCC][27] = 62, [0][1][2][0][RTW89_ETSI][27] = 54, [0][1][2][0][RTW89_MKK][27] = 68, [0][1][2][0][RTW89_IC][27] = 127, + [0][1][2][0][RTW89_KCC][27] = 64, [0][1][2][0][RTW89_ACMA][27] = 127, + [0][1][2][0][RTW89_CN][27] = 127, + [0][1][2][0][RTW89_UK][27] = 54, [0][1][2][0][RTW89_FCC][29] = 62, [0][1][2][0][RTW89_ETSI][29] = 54, [0][1][2][0][RTW89_MKK][29] = 68, [0][1][2][0][RTW89_IC][29] = 127, + [0][1][2][0][RTW89_KCC][29] = 64, [0][1][2][0][RTW89_ACMA][29] = 127, + [0][1][2][0][RTW89_CN][29] = 127, + [0][1][2][0][RTW89_UK][29] = 54, [0][1][2][0][RTW89_FCC][31] = 62, [0][1][2][0][RTW89_ETSI][31] = 54, [0][1][2][0][RTW89_MKK][31] = 68, - [0][1][2][0][RTW89_IC][31] = 70, - [0][1][2][0][RTW89_ACMA][31] = 50, + [0][1][2][0][RTW89_IC][31] = 62, + [0][1][2][0][RTW89_KCC][31] = 62, + [0][1][2][0][RTW89_ACMA][31] = 54, + [0][1][2][0][RTW89_CN][31] = 127, + [0][1][2][0][RTW89_UK][31] = 54, [0][1][2][0][RTW89_FCC][33] = 62, [0][1][2][0][RTW89_ETSI][33] = 54, [0][1][2][0][RTW89_MKK][33] = 68, - [0][1][2][0][RTW89_IC][33] = 70, - [0][1][2][0][RTW89_ACMA][33] = 50, - [0][1][2][0][RTW89_FCC][35] = 58, + [0][1][2][0][RTW89_IC][33] = 62, + [0][1][2][0][RTW89_KCC][33] = 62, + [0][1][2][0][RTW89_ACMA][33] = 54, + [0][1][2][0][RTW89_CN][33] = 127, + [0][1][2][0][RTW89_UK][33] = 54, + [0][1][2][0][RTW89_FCC][35] = 46, [0][1][2][0][RTW89_ETSI][35] = 54, [0][1][2][0][RTW89_MKK][35] = 68, - [0][1][2][0][RTW89_IC][35] = 68, - [0][1][2][0][RTW89_ACMA][35] = 50, - [0][1][2][0][RTW89_FCC][37] = 62, + [0][1][2][0][RTW89_IC][35] = 46, + [0][1][2][0][RTW89_KCC][35] = 62, + [0][1][2][0][RTW89_ACMA][35] = 54, + [0][1][2][0][RTW89_CN][35] = 127, + [0][1][2][0][RTW89_UK][35] = 54, + [0][1][2][0][RTW89_FCC][37] = 64, [0][1][2][0][RTW89_ETSI][37] = 127, [0][1][2][0][RTW89_MKK][37] = 68, - [0][1][2][0][RTW89_IC][37] = 70, - [0][1][2][0][RTW89_ACMA][37] = 70, - [0][1][2][0][RTW89_FCC][38] = 70, + [0][1][2][0][RTW89_IC][37] = 64, + [0][1][2][0][RTW89_KCC][37] = 62, + [0][1][2][0][RTW89_ACMA][37] = 64, + [0][1][2][0][RTW89_CN][37] = 127, + [0][1][2][0][RTW89_UK][37] = 52, + [0][1][2][0][RTW89_FCC][38] = 72, [0][1][2][0][RTW89_ETSI][38] = 18, [0][1][2][0][RTW89_MKK][38] = 127, - [0][1][2][0][RTW89_IC][38] = 70, + [0][1][2][0][RTW89_IC][38] = 72, + [0][1][2][0][RTW89_KCC][38] = 56, [0][1][2][0][RTW89_ACMA][38] = 70, - [0][1][2][0][RTW89_FCC][40] = 70, + [0][1][2][0][RTW89_CN][38] = 68, + [0][1][2][0][RTW89_UK][38] = 52, + [0][1][2][0][RTW89_FCC][40] = 72, [0][1][2][0][RTW89_ETSI][40] = 18, [0][1][2][0][RTW89_MKK][40] = 127, - [0][1][2][0][RTW89_IC][40] = 70, + [0][1][2][0][RTW89_IC][40] = 72, + [0][1][2][0][RTW89_KCC][40] = 56, [0][1][2][0][RTW89_ACMA][40] = 70, - [0][1][2][0][RTW89_FCC][42] = 70, + [0][1][2][0][RTW89_CN][40] = 68, + [0][1][2][0][RTW89_UK][40] = 52, + [0][1][2][0][RTW89_FCC][42] = 72, [0][1][2][0][RTW89_ETSI][42] = 18, [0][1][2][0][RTW89_MKK][42] = 127, - [0][1][2][0][RTW89_IC][42] = 70, + [0][1][2][0][RTW89_IC][42] = 72, + [0][1][2][0][RTW89_KCC][42] = 56, [0][1][2][0][RTW89_ACMA][42] = 70, - [0][1][2][0][RTW89_FCC][44] = 70, + [0][1][2][0][RTW89_CN][42] = 68, + [0][1][2][0][RTW89_UK][42] = 52, + [0][1][2][0][RTW89_FCC][44] = 72, [0][1][2][0][RTW89_ETSI][44] = 18, [0][1][2][0][RTW89_MKK][44] = 127, - [0][1][2][0][RTW89_IC][44] = 70, + [0][1][2][0][RTW89_IC][44] = 72, + [0][1][2][0][RTW89_KCC][44] = 56, [0][1][2][0][RTW89_ACMA][44] = 70, - [0][1][2][0][RTW89_FCC][46] = 70, + [0][1][2][0][RTW89_CN][44] = 68, + [0][1][2][0][RTW89_UK][44] = 52, + [0][1][2][0][RTW89_FCC][46] = 72, [0][1][2][0][RTW89_ETSI][46] = 18, [0][1][2][0][RTW89_MKK][46] = 127, - [0][1][2][0][RTW89_IC][46] = 70, + [0][1][2][0][RTW89_IC][46] = 72, + [0][1][2][0][RTW89_KCC][46] = 56, [0][1][2][0][RTW89_ACMA][46] = 70, - [0][1][2][0][RTW89_FCC][48] = 50, + [0][1][2][0][RTW89_CN][46] = 68, + [0][1][2][0][RTW89_UK][46] = 52, + [0][1][2][0][RTW89_FCC][48] = 48, [0][1][2][0][RTW89_ETSI][48] = 127, [0][1][2][0][RTW89_MKK][48] = 127, [0][1][2][0][RTW89_IC][48] = 127, + [0][1][2][0][RTW89_KCC][48] = 127, [0][1][2][0][RTW89_ACMA][48] = 127, + [0][1][2][0][RTW89_CN][48] = 127, + [0][1][2][0][RTW89_UK][48] = 127, [0][1][2][0][RTW89_FCC][50] = 50, [0][1][2][0][RTW89_ETSI][50] = 127, [0][1][2][0][RTW89_MKK][50] = 127, [0][1][2][0][RTW89_IC][50] = 127, + [0][1][2][0][RTW89_KCC][50] = 127, [0][1][2][0][RTW89_ACMA][50] = 127, - [0][1][2][0][RTW89_FCC][52] = 50, + [0][1][2][0][RTW89_CN][50] = 127, + [0][1][2][0][RTW89_UK][50] = 127, + [0][1][2][0][RTW89_FCC][52] = 48, [0][1][2][0][RTW89_ETSI][52] = 127, [0][1][2][0][RTW89_MKK][52] = 127, [0][1][2][0][RTW89_IC][52] = 127, + [0][1][2][0][RTW89_KCC][52] = 127, [0][1][2][0][RTW89_ACMA][52] = 127, + [0][1][2][0][RTW89_CN][52] = 127, + [0][1][2][0][RTW89_UK][52] = 127, [0][1][2][1][RTW89_FCC][0] = 60, [0][1][2][1][RTW89_ETSI][0] = 40, [0][1][2][1][RTW89_MKK][0] = 54, - [0][1][2][1][RTW89_IC][0] = 42, - [0][1][2][1][RTW89_ACMA][0] = 38, - [0][1][2][1][RTW89_FCC][2] = 60, + [0][1][2][1][RTW89_IC][0] = 40, + [0][1][2][1][RTW89_KCC][0] = 40, + [0][1][2][1][RTW89_ACMA][0] = 40, + [0][1][2][1][RTW89_CN][0] = 36, + [0][1][2][1][RTW89_UK][0] = 40, + [0][1][2][1][RTW89_FCC][2] = 62, [0][1][2][1][RTW89_ETSI][2] = 40, [0][1][2][1][RTW89_MKK][2] = 54, - [0][1][2][1][RTW89_IC][2] = 42, - [0][1][2][1][RTW89_ACMA][2] = 38, - [0][1][2][1][RTW89_FCC][4] = 60, + [0][1][2][1][RTW89_IC][2] = 40, + [0][1][2][1][RTW89_KCC][2] = 40, + [0][1][2][1][RTW89_ACMA][2] = 40, + [0][1][2][1][RTW89_CN][2] = 36, + [0][1][2][1][RTW89_UK][2] = 40, + [0][1][2][1][RTW89_FCC][4] = 62, [0][1][2][1][RTW89_ETSI][4] = 40, [0][1][2][1][RTW89_MKK][4] = 54, - [0][1][2][1][RTW89_IC][4] = 42, - [0][1][2][1][RTW89_ACMA][4] = 38, - [0][1][2][1][RTW89_FCC][6] = 60, + [0][1][2][1][RTW89_IC][4] = 40, + [0][1][2][1][RTW89_KCC][4] = 40, + [0][1][2][1][RTW89_ACMA][4] = 40, + [0][1][2][1][RTW89_CN][4] = 36, + [0][1][2][1][RTW89_UK][4] = 40, + [0][1][2][1][RTW89_FCC][6] = 62, [0][1][2][1][RTW89_ETSI][6] = 40, [0][1][2][1][RTW89_MKK][6] = 50, - [0][1][2][1][RTW89_IC][6] = 42, - [0][1][2][1][RTW89_ACMA][6] = 38, - [0][1][2][1][RTW89_FCC][8] = 60, + [0][1][2][1][RTW89_IC][6] = 40, + [0][1][2][1][RTW89_KCC][6] = 64, + [0][1][2][1][RTW89_ACMA][6] = 40, + [0][1][2][1][RTW89_CN][6] = 36, + [0][1][2][1][RTW89_UK][6] = 40, + [0][1][2][1][RTW89_FCC][8] = 62, [0][1][2][1][RTW89_ETSI][8] = 40, [0][1][2][1][RTW89_MKK][8] = 42, - [0][1][2][1][RTW89_IC][8] = 42, - [0][1][2][1][RTW89_ACMA][8] = 38, - [0][1][2][1][RTW89_FCC][10] = 60, + [0][1][2][1][RTW89_IC][8] = 40, + [0][1][2][1][RTW89_KCC][8] = 62, + [0][1][2][1][RTW89_ACMA][8] = 40, + [0][1][2][1][RTW89_CN][8] = 36, + [0][1][2][1][RTW89_UK][8] = 40, + [0][1][2][1][RTW89_FCC][10] = 62, [0][1][2][1][RTW89_ETSI][10] = 40, - [0][1][2][1][RTW89_MKK][10] = 66, - [0][1][2][1][RTW89_IC][10] = 42, - [0][1][2][1][RTW89_ACMA][10] = 38, - [0][1][2][1][RTW89_FCC][12] = 60, + [0][1][2][1][RTW89_MKK][10] = 54, + [0][1][2][1][RTW89_IC][10] = 40, + [0][1][2][1][RTW89_KCC][10] = 62, + [0][1][2][1][RTW89_ACMA][10] = 40, + [0][1][2][1][RTW89_CN][10] = 36, + [0][1][2][1][RTW89_UK][10] = 40, + [0][1][2][1][RTW89_FCC][12] = 62, [0][1][2][1][RTW89_ETSI][12] = 40, - [0][1][2][1][RTW89_MKK][12] = 66, - [0][1][2][1][RTW89_IC][12] = 42, - [0][1][2][1][RTW89_ACMA][12] = 38, - [0][1][2][1][RTW89_FCC][14] = 60, + [0][1][2][1][RTW89_MKK][12] = 54, + [0][1][2][1][RTW89_IC][12] = 40, + [0][1][2][1][RTW89_KCC][12] = 62, + [0][1][2][1][RTW89_ACMA][12] = 40, + [0][1][2][1][RTW89_CN][12] = 36, + [0][1][2][1][RTW89_UK][12] = 40, + [0][1][2][1][RTW89_FCC][14] = 62, [0][1][2][1][RTW89_ETSI][14] = 40, - [0][1][2][1][RTW89_MKK][14] = 66, - [0][1][2][1][RTW89_IC][14] = 42, - [0][1][2][1][RTW89_ACMA][14] = 38, + [0][1][2][1][RTW89_MKK][14] = 54, + [0][1][2][1][RTW89_IC][14] = 40, + [0][1][2][1][RTW89_KCC][14] = 62, + [0][1][2][1][RTW89_ACMA][14] = 40, + [0][1][2][1][RTW89_CN][14] = 36, + [0][1][2][1][RTW89_UK][14] = 40, [0][1][2][1][RTW89_FCC][15] = 60, [0][1][2][1][RTW89_ETSI][15] = 40, [0][1][2][1][RTW89_MKK][15] = 68, - [0][1][2][1][RTW89_IC][15] = 70, - [0][1][2][1][RTW89_ACMA][15] = 38, - [0][1][2][1][RTW89_FCC][17] = 60, + [0][1][2][1][RTW89_IC][15] = 60, + [0][1][2][1][RTW89_KCC][15] = 64, + [0][1][2][1][RTW89_ACMA][15] = 40, + [0][1][2][1][RTW89_CN][15] = 127, + [0][1][2][1][RTW89_UK][15] = 40, + [0][1][2][1][RTW89_FCC][17] = 62, [0][1][2][1][RTW89_ETSI][17] = 40, [0][1][2][1][RTW89_MKK][17] = 68, - [0][1][2][1][RTW89_IC][17] = 70, - [0][1][2][1][RTW89_ACMA][17] = 38, - [0][1][2][1][RTW89_FCC][19] = 60, + [0][1][2][1][RTW89_IC][17] = 62, + [0][1][2][1][RTW89_KCC][17] = 64, + [0][1][2][1][RTW89_ACMA][17] = 40, + [0][1][2][1][RTW89_CN][17] = 127, + [0][1][2][1][RTW89_UK][17] = 40, + [0][1][2][1][RTW89_FCC][19] = 62, [0][1][2][1][RTW89_ETSI][19] = 40, [0][1][2][1][RTW89_MKK][19] = 68, - [0][1][2][1][RTW89_IC][19] = 70, - [0][1][2][1][RTW89_ACMA][19] = 38, - [0][1][2][1][RTW89_FCC][21] = 60, + [0][1][2][1][RTW89_IC][19] = 62, + [0][1][2][1][RTW89_KCC][19] = 64, + [0][1][2][1][RTW89_ACMA][19] = 40, + [0][1][2][1][RTW89_CN][19] = 127, + [0][1][2][1][RTW89_UK][19] = 40, + [0][1][2][1][RTW89_FCC][21] = 62, [0][1][2][1][RTW89_ETSI][21] = 40, [0][1][2][1][RTW89_MKK][21] = 68, - [0][1][2][1][RTW89_IC][21] = 70, - [0][1][2][1][RTW89_ACMA][21] = 38, - [0][1][2][1][RTW89_FCC][23] = 60, + [0][1][2][1][RTW89_IC][21] = 62, + [0][1][2][1][RTW89_KCC][21] = 64, + [0][1][2][1][RTW89_ACMA][21] = 40, + [0][1][2][1][RTW89_CN][21] = 127, + [0][1][2][1][RTW89_UK][21] = 40, + [0][1][2][1][RTW89_FCC][23] = 62, [0][1][2][1][RTW89_ETSI][23] = 40, [0][1][2][1][RTW89_MKK][23] = 68, - [0][1][2][1][RTW89_IC][23] = 70, - [0][1][2][1][RTW89_ACMA][23] = 38, - [0][1][2][1][RTW89_FCC][25] = 58, + [0][1][2][1][RTW89_IC][23] = 62, + [0][1][2][1][RTW89_KCC][23] = 64, + [0][1][2][1][RTW89_ACMA][23] = 40, + [0][1][2][1][RTW89_CN][23] = 127, + [0][1][2][1][RTW89_UK][23] = 40, + [0][1][2][1][RTW89_FCC][25] = 46, [0][1][2][1][RTW89_ETSI][25] = 40, [0][1][2][1][RTW89_MKK][25] = 68, [0][1][2][1][RTW89_IC][25] = 127, + [0][1][2][1][RTW89_KCC][25] = 64, [0][1][2][1][RTW89_ACMA][25] = 127, - [0][1][2][1][RTW89_FCC][27] = 58, + [0][1][2][1][RTW89_CN][25] = 127, + [0][1][2][1][RTW89_UK][25] = 40, + [0][1][2][1][RTW89_FCC][27] = 46, [0][1][2][1][RTW89_ETSI][27] = 40, [0][1][2][1][RTW89_MKK][27] = 68, [0][1][2][1][RTW89_IC][27] = 127, + [0][1][2][1][RTW89_KCC][27] = 64, [0][1][2][1][RTW89_ACMA][27] = 127, - [0][1][2][1][RTW89_FCC][29] = 58, + [0][1][2][1][RTW89_CN][27] = 127, + [0][1][2][1][RTW89_UK][27] = 40, + [0][1][2][1][RTW89_FCC][29] = 46, [0][1][2][1][RTW89_ETSI][29] = 40, [0][1][2][1][RTW89_MKK][29] = 68, [0][1][2][1][RTW89_IC][29] = 127, + [0][1][2][1][RTW89_KCC][29] = 64, [0][1][2][1][RTW89_ACMA][29] = 127, - [0][1][2][1][RTW89_FCC][31] = 58, + [0][1][2][1][RTW89_CN][29] = 127, + [0][1][2][1][RTW89_UK][29] = 40, + [0][1][2][1][RTW89_FCC][31] = 46, [0][1][2][1][RTW89_ETSI][31] = 40, [0][1][2][1][RTW89_MKK][31] = 68, - [0][1][2][1][RTW89_IC][31] = 68, - [0][1][2][1][RTW89_ACMA][31] = 38, - [0][1][2][1][RTW89_FCC][33] = 58, + [0][1][2][1][RTW89_IC][31] = 46, + [0][1][2][1][RTW89_KCC][31] = 62, + [0][1][2][1][RTW89_ACMA][31] = 40, + [0][1][2][1][RTW89_CN][31] = 127, + [0][1][2][1][RTW89_UK][31] = 40, + [0][1][2][1][RTW89_FCC][33] = 46, [0][1][2][1][RTW89_ETSI][33] = 40, [0][1][2][1][RTW89_MKK][33] = 68, - [0][1][2][1][RTW89_IC][33] = 68, - [0][1][2][1][RTW89_ACMA][33] = 38, - [0][1][2][1][RTW89_FCC][35] = 58, + [0][1][2][1][RTW89_IC][33] = 46, + [0][1][2][1][RTW89_KCC][33] = 62, + [0][1][2][1][RTW89_ACMA][33] = 40, + [0][1][2][1][RTW89_CN][33] = 127, + [0][1][2][1][RTW89_UK][33] = 40, + [0][1][2][1][RTW89_FCC][35] = 46, [0][1][2][1][RTW89_ETSI][35] = 40, [0][1][2][1][RTW89_MKK][35] = 68, - [0][1][2][1][RTW89_IC][35] = 68, - [0][1][2][1][RTW89_ACMA][35] = 38, - [0][1][2][1][RTW89_FCC][37] = 60, + [0][1][2][1][RTW89_IC][35] = 46, + [0][1][2][1][RTW89_KCC][35] = 62, + [0][1][2][1][RTW89_ACMA][35] = 40, + [0][1][2][1][RTW89_CN][35] = 127, + [0][1][2][1][RTW89_UK][35] = 40, + [0][1][2][1][RTW89_FCC][37] = 64, [0][1][2][1][RTW89_ETSI][37] = 127, [0][1][2][1][RTW89_MKK][37] = 68, - [0][1][2][1][RTW89_IC][37] = 70, - [0][1][2][1][RTW89_ACMA][37] = 70, - [0][1][2][1][RTW89_FCC][38] = 70, + [0][1][2][1][RTW89_IC][37] = 64, + [0][1][2][1][RTW89_KCC][37] = 62, + [0][1][2][1][RTW89_ACMA][37] = 64, + [0][1][2][1][RTW89_CN][37] = 127, + [0][1][2][1][RTW89_UK][37] = 40, + [0][1][2][1][RTW89_FCC][38] = 72, [0][1][2][1][RTW89_ETSI][38] = 6, [0][1][2][1][RTW89_MKK][38] = 127, - [0][1][2][1][RTW89_IC][38] = 70, + [0][1][2][1][RTW89_IC][38] = 72, + [0][1][2][1][RTW89_KCC][38] = 56, [0][1][2][1][RTW89_ACMA][38] = 70, - [0][1][2][1][RTW89_FCC][40] = 70, + [0][1][2][1][RTW89_CN][38] = 60, + [0][1][2][1][RTW89_UK][38] = 40, + [0][1][2][1][RTW89_FCC][40] = 72, [0][1][2][1][RTW89_ETSI][40] = 6, [0][1][2][1][RTW89_MKK][40] = 127, - [0][1][2][1][RTW89_IC][40] = 70, + [0][1][2][1][RTW89_IC][40] = 72, + [0][1][2][1][RTW89_KCC][40] = 56, [0][1][2][1][RTW89_ACMA][40] = 70, - [0][1][2][1][RTW89_FCC][42] = 70, + [0][1][2][1][RTW89_CN][40] = 60, + [0][1][2][1][RTW89_UK][40] = 40, + [0][1][2][1][RTW89_FCC][42] = 72, [0][1][2][1][RTW89_ETSI][42] = 6, [0][1][2][1][RTW89_MKK][42] = 127, - [0][1][2][1][RTW89_IC][42] = 70, + [0][1][2][1][RTW89_IC][42] = 72, + [0][1][2][1][RTW89_KCC][42] = 56, [0][1][2][1][RTW89_ACMA][42] = 70, - [0][1][2][1][RTW89_FCC][44] = 70, + [0][1][2][1][RTW89_CN][42] = 60, + [0][1][2][1][RTW89_UK][42] = 40, + [0][1][2][1][RTW89_FCC][44] = 72, [0][1][2][1][RTW89_ETSI][44] = 6, [0][1][2][1][RTW89_MKK][44] = 127, - [0][1][2][1][RTW89_IC][44] = 70, + [0][1][2][1][RTW89_IC][44] = 72, + [0][1][2][1][RTW89_KCC][44] = 56, [0][1][2][1][RTW89_ACMA][44] = 70, - [0][1][2][1][RTW89_FCC][46] = 70, + [0][1][2][1][RTW89_CN][44] = 54, + [0][1][2][1][RTW89_UK][44] = 40, + [0][1][2][1][RTW89_FCC][46] = 72, [0][1][2][1][RTW89_ETSI][46] = 6, [0][1][2][1][RTW89_MKK][46] = 127, - [0][1][2][1][RTW89_IC][46] = 70, + [0][1][2][1][RTW89_IC][46] = 72, + [0][1][2][1][RTW89_KCC][46] = 56, [0][1][2][1][RTW89_ACMA][46] = 70, - [0][1][2][1][RTW89_FCC][48] = 50, + [0][1][2][1][RTW89_CN][46] = 54, + [0][1][2][1][RTW89_UK][46] = 40, + [0][1][2][1][RTW89_FCC][48] = 48, [0][1][2][1][RTW89_ETSI][48] = 127, [0][1][2][1][RTW89_MKK][48] = 127, [0][1][2][1][RTW89_IC][48] = 127, + [0][1][2][1][RTW89_KCC][48] = 127, [0][1][2][1][RTW89_ACMA][48] = 127, + [0][1][2][1][RTW89_CN][48] = 127, + [0][1][2][1][RTW89_UK][48] = 127, [0][1][2][1][RTW89_FCC][50] = 50, [0][1][2][1][RTW89_ETSI][50] = 127, [0][1][2][1][RTW89_MKK][50] = 127, [0][1][2][1][RTW89_IC][50] = 127, + [0][1][2][1][RTW89_KCC][50] = 127, [0][1][2][1][RTW89_ACMA][50] = 127, - [0][1][2][1][RTW89_FCC][52] = 50, + [0][1][2][1][RTW89_CN][50] = 127, + [0][1][2][1][RTW89_UK][50] = 127, + [0][1][2][1][RTW89_FCC][52] = 48, [0][1][2][1][RTW89_ETSI][52] = 127, [0][1][2][1][RTW89_MKK][52] = 127, [0][1][2][1][RTW89_IC][52] = 127, + [0][1][2][1][RTW89_KCC][52] = 127, [0][1][2][1][RTW89_ACMA][52] = 127, - [1][0][2][0][RTW89_FCC][1] = 58, + [0][1][2][1][RTW89_CN][52] = 127, + [0][1][2][1][RTW89_UK][52] = 127, + [1][0][2][0][RTW89_FCC][1] = 64, [1][0][2][0][RTW89_ETSI][1] = 66, [1][0][2][0][RTW89_MKK][1] = 66, - [1][0][2][0][RTW89_IC][1] = 66, + [1][0][2][0][RTW89_IC][1] = 62, + [1][0][2][0][RTW89_KCC][1] = 66, [1][0][2][0][RTW89_ACMA][1] = 66, + [1][0][2][0][RTW89_CN][1] = 54, + [1][0][2][0][RTW89_UK][1] = 66, [1][0][2][0][RTW89_FCC][5] = 68, [1][0][2][0][RTW89_ETSI][5] = 66, [1][0][2][0][RTW89_MKK][5] = 66, - [1][0][2][0][RTW89_IC][5] = 66, + [1][0][2][0][RTW89_IC][5] = 64, + [1][0][2][0][RTW89_KCC][5] = 54, [1][0][2][0][RTW89_ACMA][5] = 66, + [1][0][2][0][RTW89_CN][5] = 54, + [1][0][2][0][RTW89_UK][5] = 66, [1][0][2][0][RTW89_FCC][9] = 68, [1][0][2][0][RTW89_ETSI][9] = 66, [1][0][2][0][RTW89_MKK][9] = 66, - [1][0][2][0][RTW89_IC][9] = 66, + [1][0][2][0][RTW89_IC][9] = 64, + [1][0][2][0][RTW89_KCC][9] = 66, [1][0][2][0][RTW89_ACMA][9] = 66, - [1][0][2][0][RTW89_FCC][13] = 58, + [1][0][2][0][RTW89_CN][9] = 54, + [1][0][2][0][RTW89_UK][9] = 66, + [1][0][2][0][RTW89_FCC][13] = 60, [1][0][2][0][RTW89_ETSI][13] = 66, [1][0][2][0][RTW89_MKK][13] = 66, - [1][0][2][0][RTW89_IC][13] = 66, + [1][0][2][0][RTW89_IC][13] = 60, + [1][0][2][0][RTW89_KCC][13] = 52, [1][0][2][0][RTW89_ACMA][13] = 66, - [1][0][2][0][RTW89_FCC][16] = 56, + [1][0][2][0][RTW89_CN][13] = 54, + [1][0][2][0][RTW89_UK][13] = 66, + [1][0][2][0][RTW89_FCC][16] = 64, [1][0][2][0][RTW89_ETSI][16] = 66, [1][0][2][0][RTW89_MKK][16] = 66, - [1][0][2][0][RTW89_IC][16] = 66, + [1][0][2][0][RTW89_IC][16] = 64, + [1][0][2][0][RTW89_KCC][16] = 56, [1][0][2][0][RTW89_ACMA][16] = 66, + [1][0][2][0][RTW89_CN][16] = 127, + [1][0][2][0][RTW89_UK][16] = 66, [1][0][2][0][RTW89_FCC][20] = 68, [1][0][2][0][RTW89_ETSI][20] = 66, [1][0][2][0][RTW89_MKK][20] = 66, - [1][0][2][0][RTW89_IC][20] = 66, + [1][0][2][0][RTW89_IC][20] = 68, + [1][0][2][0][RTW89_KCC][20] = 56, [1][0][2][0][RTW89_ACMA][20] = 66, + [1][0][2][0][RTW89_CN][20] = 127, + [1][0][2][0][RTW89_UK][20] = 66, [1][0][2][0][RTW89_FCC][24] = 68, [1][0][2][0][RTW89_ETSI][24] = 66, [1][0][2][0][RTW89_MKK][24] = 66, [1][0][2][0][RTW89_IC][24] = 127, + [1][0][2][0][RTW89_KCC][24] = 56, [1][0][2][0][RTW89_ACMA][24] = 127, + [1][0][2][0][RTW89_CN][24] = 127, + [1][0][2][0][RTW89_UK][24] = 66, [1][0][2][0][RTW89_FCC][28] = 68, [1][0][2][0][RTW89_ETSI][28] = 66, [1][0][2][0][RTW89_MKK][28] = 66, [1][0][2][0][RTW89_IC][28] = 127, + [1][0][2][0][RTW89_KCC][28] = 66, [1][0][2][0][RTW89_ACMA][28] = 127, - [1][0][2][0][RTW89_FCC][32] = 68, + [1][0][2][0][RTW89_CN][28] = 127, + [1][0][2][0][RTW89_UK][28] = 66, + [1][0][2][0][RTW89_FCC][32] = 62, [1][0][2][0][RTW89_ETSI][32] = 66, [1][0][2][0][RTW89_MKK][32] = 66, - [1][0][2][0][RTW89_IC][32] = 66, + [1][0][2][0][RTW89_IC][32] = 62, + [1][0][2][0][RTW89_KCC][32] = 66, [1][0][2][0][RTW89_ACMA][32] = 66, + [1][0][2][0][RTW89_CN][32] = 127, + [1][0][2][0][RTW89_UK][32] = 66, [1][0][2][0][RTW89_FCC][36] = 68, [1][0][2][0][RTW89_ETSI][36] = 127, [1][0][2][0][RTW89_MKK][36] = 66, - [1][0][2][0][RTW89_IC][36] = 66, + [1][0][2][0][RTW89_IC][36] = 68, + [1][0][2][0][RTW89_KCC][36] = 66, [1][0][2][0][RTW89_ACMA][36] = 66, + [1][0][2][0][RTW89_CN][36] = 127, + [1][0][2][0][RTW89_UK][36] = 64, [1][0][2][0][RTW89_FCC][39] = 68, [1][0][2][0][RTW89_ETSI][39] = 30, [1][0][2][0][RTW89_MKK][39] = 127, - [1][0][2][0][RTW89_IC][39] = 66, + [1][0][2][0][RTW89_IC][39] = 68, + [1][0][2][0][RTW89_KCC][39] = 66, [1][0][2][0][RTW89_ACMA][39] = 66, + [1][0][2][0][RTW89_CN][39] = 62, + [1][0][2][0][RTW89_UK][39] = 64, [1][0][2][0][RTW89_FCC][43] = 68, [1][0][2][0][RTW89_ETSI][43] = 30, [1][0][2][0][RTW89_MKK][43] = 127, - [1][0][2][0][RTW89_IC][43] = 66, + [1][0][2][0][RTW89_IC][43] = 68, + [1][0][2][0][RTW89_KCC][43] = 66, [1][0][2][0][RTW89_ACMA][43] = 66, + [1][0][2][0][RTW89_CN][43] = 66, + [1][0][2][0][RTW89_UK][43] = 64, [1][0][2][0][RTW89_FCC][47] = 68, [1][0][2][0][RTW89_ETSI][47] = 127, [1][0][2][0][RTW89_MKK][47] = 127, [1][0][2][0][RTW89_IC][47] = 127, + [1][0][2][0][RTW89_KCC][47] = 127, [1][0][2][0][RTW89_ACMA][47] = 127, + [1][0][2][0][RTW89_CN][47] = 127, + [1][0][2][0][RTW89_UK][47] = 127, [1][0][2][0][RTW89_FCC][51] = 68, [1][0][2][0][RTW89_ETSI][51] = 127, [1][0][2][0][RTW89_MKK][51] = 127, [1][0][2][0][RTW89_IC][51] = 127, + [1][0][2][0][RTW89_KCC][51] = 127, [1][0][2][0][RTW89_ACMA][51] = 127, + [1][0][2][0][RTW89_CN][51] = 127, + [1][0][2][0][RTW89_UK][51] = 127, [1][1][2][0][RTW89_FCC][1] = 54, [1][1][2][0][RTW89_ETSI][1] = 54, [1][1][2][0][RTW89_MKK][1] = 48, - [1][1][2][0][RTW89_IC][1] = 60, - [1][1][2][0][RTW89_ACMA][1] = 60, + [1][1][2][0][RTW89_IC][1] = 48, + [1][1][2][0][RTW89_KCC][1] = 54, + [1][1][2][0][RTW89_ACMA][1] = 54, + [1][1][2][0][RTW89_CN][1] = 42, + [1][1][2][0][RTW89_UK][1] = 54, [1][1][2][0][RTW89_FCC][5] = 68, [1][1][2][0][RTW89_ETSI][5] = 54, [1][1][2][0][RTW89_MKK][5] = 52, - [1][1][2][0][RTW89_IC][5] = 60, - [1][1][2][0][RTW89_ACMA][5] = 60, + [1][1][2][0][RTW89_IC][5] = 48, + [1][1][2][0][RTW89_KCC][5] = 54, + [1][1][2][0][RTW89_ACMA][5] = 54, + [1][1][2][0][RTW89_CN][5] = 42, + [1][1][2][0][RTW89_UK][5] = 54, [1][1][2][0][RTW89_FCC][9] = 68, [1][1][2][0][RTW89_ETSI][9] = 54, [1][1][2][0][RTW89_MKK][9] = 52, - [1][1][2][0][RTW89_IC][9] = 60, - [1][1][2][0][RTW89_ACMA][9] = 60, + [1][1][2][0][RTW89_IC][9] = 52, + [1][1][2][0][RTW89_KCC][9] = 64, + [1][1][2][0][RTW89_ACMA][9] = 54, + [1][1][2][0][RTW89_CN][9] = 42, + [1][1][2][0][RTW89_UK][9] = 54, [1][1][2][0][RTW89_FCC][13] = 54, [1][1][2][0][RTW89_ETSI][13] = 54, [1][1][2][0][RTW89_MKK][13] = 52, - [1][1][2][0][RTW89_IC][13] = 60, - [1][1][2][0][RTW89_ACMA][13] = 60, - [1][1][2][0][RTW89_FCC][16] = 48, + [1][1][2][0][RTW89_IC][13] = 52, + [1][1][2][0][RTW89_KCC][13] = 52, + [1][1][2][0][RTW89_ACMA][13] = 54, + [1][1][2][0][RTW89_CN][13] = 42, + [1][1][2][0][RTW89_UK][13] = 54, + [1][1][2][0][RTW89_FCC][16] = 56, [1][1][2][0][RTW89_ETSI][16] = 54, [1][1][2][0][RTW89_MKK][16] = 66, - [1][1][2][0][RTW89_IC][16] = 58, - [1][1][2][0][RTW89_ACMA][16] = 60, + [1][1][2][0][RTW89_IC][16] = 56, + [1][1][2][0][RTW89_KCC][16] = 54, + [1][1][2][0][RTW89_ACMA][16] = 54, + [1][1][2][0][RTW89_CN][16] = 127, + [1][1][2][0][RTW89_UK][16] = 54, [1][1][2][0][RTW89_FCC][20] = 68, [1][1][2][0][RTW89_ETSI][20] = 54, [1][1][2][0][RTW89_MKK][20] = 66, - [1][1][2][0][RTW89_IC][20] = 66, - [1][1][2][0][RTW89_ACMA][20] = 60, + [1][1][2][0][RTW89_IC][20] = 68, + [1][1][2][0][RTW89_KCC][20] = 54, + [1][1][2][0][RTW89_ACMA][20] = 54, + [1][1][2][0][RTW89_CN][20] = 127, + [1][1][2][0][RTW89_UK][20] = 54, [1][1][2][0][RTW89_FCC][24] = 68, [1][1][2][0][RTW89_ETSI][24] = 54, [1][1][2][0][RTW89_MKK][24] = 66, [1][1][2][0][RTW89_IC][24] = 127, + [1][1][2][0][RTW89_KCC][24] = 54, [1][1][2][0][RTW89_ACMA][24] = 127, + [1][1][2][0][RTW89_CN][24] = 127, + [1][1][2][0][RTW89_UK][24] = 54, [1][1][2][0][RTW89_FCC][28] = 68, [1][1][2][0][RTW89_ETSI][28] = 54, [1][1][2][0][RTW89_MKK][28] = 66, [1][1][2][0][RTW89_IC][28] = 127, + [1][1][2][0][RTW89_KCC][28] = 66, [1][1][2][0][RTW89_ACMA][28] = 127, - [1][1][2][0][RTW89_FCC][32] = 60, + [1][1][2][0][RTW89_CN][28] = 127, + [1][1][2][0][RTW89_UK][28] = 54, + [1][1][2][0][RTW89_FCC][32] = 56, [1][1][2][0][RTW89_ETSI][32] = 54, [1][1][2][0][RTW89_MKK][32] = 66, - [1][1][2][0][RTW89_IC][32] = 66, + [1][1][2][0][RTW89_IC][32] = 56, + [1][1][2][0][RTW89_KCC][32] = 66, [1][1][2][0][RTW89_ACMA][32] = 54, + [1][1][2][0][RTW89_CN][32] = 127, + [1][1][2][0][RTW89_UK][32] = 54, [1][1][2][0][RTW89_FCC][36] = 68, [1][1][2][0][RTW89_ETSI][36] = 127, [1][1][2][0][RTW89_MKK][36] = 66, - [1][1][2][0][RTW89_IC][36] = 66, + [1][1][2][0][RTW89_IC][36] = 68, + [1][1][2][0][RTW89_KCC][36] = 66, [1][1][2][0][RTW89_ACMA][36] = 66, + [1][1][2][0][RTW89_CN][36] = 127, + [1][1][2][0][RTW89_UK][36] = 52, [1][1][2][0][RTW89_FCC][39] = 68, [1][1][2][0][RTW89_ETSI][39] = 18, [1][1][2][0][RTW89_MKK][39] = 127, - [1][1][2][0][RTW89_IC][39] = 66, + [1][1][2][0][RTW89_IC][39] = 68, + [1][1][2][0][RTW89_KCC][39] = 56, [1][1][2][0][RTW89_ACMA][39] = 66, + [1][1][2][0][RTW89_CN][39] = 62, + [1][1][2][0][RTW89_UK][39] = 52, [1][1][2][0][RTW89_FCC][43] = 68, [1][1][2][0][RTW89_ETSI][43] = 18, [1][1][2][0][RTW89_MKK][43] = 127, - [1][1][2][0][RTW89_IC][43] = 66, + [1][1][2][0][RTW89_IC][43] = 68, + [1][1][2][0][RTW89_KCC][43] = 56, [1][1][2][0][RTW89_ACMA][43] = 66, - [1][1][2][0][RTW89_FCC][47] = 60, + [1][1][2][0][RTW89_CN][43] = 66, + [1][1][2][0][RTW89_UK][43] = 52, + [1][1][2][0][RTW89_FCC][47] = 62, [1][1][2][0][RTW89_ETSI][47] = 127, [1][1][2][0][RTW89_MKK][47] = 127, [1][1][2][0][RTW89_IC][47] = 127, + [1][1][2][0][RTW89_KCC][47] = 127, [1][1][2][0][RTW89_ACMA][47] = 127, - [1][1][2][0][RTW89_FCC][51] = 58, + [1][1][2][0][RTW89_CN][47] = 127, + [1][1][2][0][RTW89_UK][47] = 127, + [1][1][2][0][RTW89_FCC][51] = 60, [1][1][2][0][RTW89_ETSI][51] = 127, [1][1][2][0][RTW89_MKK][51] = 127, [1][1][2][0][RTW89_IC][51] = 127, + [1][1][2][0][RTW89_KCC][51] = 127, [1][1][2][0][RTW89_ACMA][51] = 127, + [1][1][2][0][RTW89_CN][51] = 127, + [1][1][2][0][RTW89_UK][51] = 127, [1][1][2][1][RTW89_FCC][1] = 54, [1][1][2][1][RTW89_ETSI][1] = 40, [1][1][2][1][RTW89_MKK][1] = 48, - [1][1][2][1][RTW89_IC][1] = 48, - [1][1][2][1][RTW89_ACMA][1] = 48, - [1][1][2][1][RTW89_FCC][5] = 60, + [1][1][2][1][RTW89_IC][1] = 40, + [1][1][2][1][RTW89_KCC][1] = 54, + [1][1][2][1][RTW89_ACMA][1] = 40, + [1][1][2][1][RTW89_CN][1] = 42, + [1][1][2][1][RTW89_UK][1] = 40, + [1][1][2][1][RTW89_FCC][5] = 68, [1][1][2][1][RTW89_ETSI][5] = 40, [1][1][2][1][RTW89_MKK][5] = 52, - [1][1][2][1][RTW89_IC][5] = 48, - [1][1][2][1][RTW89_ACMA][5] = 48, - [1][1][2][1][RTW89_FCC][9] = 60, + [1][1][2][1][RTW89_IC][5] = 40, + [1][1][2][1][RTW89_KCC][5] = 54, + [1][1][2][1][RTW89_ACMA][5] = 40, + [1][1][2][1][RTW89_CN][5] = 42, + [1][1][2][1][RTW89_UK][5] = 40, + [1][1][2][1][RTW89_FCC][9] = 68, [1][1][2][1][RTW89_ETSI][9] = 40, [1][1][2][1][RTW89_MKK][9] = 52, - [1][1][2][1][RTW89_IC][9] = 48, - [1][1][2][1][RTW89_ACMA][9] = 48, + [1][1][2][1][RTW89_IC][9] = 40, + [1][1][2][1][RTW89_KCC][9] = 64, + [1][1][2][1][RTW89_ACMA][9] = 40, + [1][1][2][1][RTW89_CN][9] = 42, + [1][1][2][1][RTW89_UK][9] = 40, [1][1][2][1][RTW89_FCC][13] = 54, [1][1][2][1][RTW89_ETSI][13] = 40, [1][1][2][1][RTW89_MKK][13] = 52, - [1][1][2][1][RTW89_IC][13] = 48, - [1][1][2][1][RTW89_ACMA][13] = 48, - [1][1][2][1][RTW89_FCC][16] = 48, + [1][1][2][1][RTW89_IC][13] = 40, + [1][1][2][1][RTW89_KCC][13] = 52, + [1][1][2][1][RTW89_ACMA][13] = 40, + [1][1][2][1][RTW89_CN][13] = 42, + [1][1][2][1][RTW89_UK][13] = 40, + [1][1][2][1][RTW89_FCC][16] = 56, [1][1][2][1][RTW89_ETSI][16] = 40, [1][1][2][1][RTW89_MKK][16] = 66, - [1][1][2][1][RTW89_IC][16] = 58, - [1][1][2][1][RTW89_ACMA][16] = 48, - [1][1][2][1][RTW89_FCC][20] = 60, + [1][1][2][1][RTW89_IC][16] = 56, + [1][1][2][1][RTW89_KCC][16] = 54, + [1][1][2][1][RTW89_ACMA][16] = 40, + [1][1][2][1][RTW89_CN][16] = 127, + [1][1][2][1][RTW89_UK][16] = 40, + [1][1][2][1][RTW89_FCC][20] = 68, [1][1][2][1][RTW89_ETSI][20] = 40, [1][1][2][1][RTW89_MKK][20] = 66, - [1][1][2][1][RTW89_IC][20] = 66, - [1][1][2][1][RTW89_ACMA][20] = 48, - [1][1][2][1][RTW89_FCC][24] = 60, + [1][1][2][1][RTW89_IC][20] = 68, + [1][1][2][1][RTW89_KCC][20] = 54, + [1][1][2][1][RTW89_ACMA][20] = 40, + [1][1][2][1][RTW89_CN][20] = 127, + [1][1][2][1][RTW89_UK][20] = 40, + [1][1][2][1][RTW89_FCC][24] = 68, [1][1][2][1][RTW89_ETSI][24] = 40, [1][1][2][1][RTW89_MKK][24] = 66, [1][1][2][1][RTW89_IC][24] = 127, + [1][1][2][1][RTW89_KCC][24] = 54, [1][1][2][1][RTW89_ACMA][24] = 127, - [1][1][2][1][RTW89_FCC][28] = 60, + [1][1][2][1][RTW89_CN][24] = 127, + [1][1][2][1][RTW89_UK][24] = 40, + [1][1][2][1][RTW89_FCC][28] = 68, [1][1][2][1][RTW89_ETSI][28] = 40, [1][1][2][1][RTW89_MKK][28] = 66, [1][1][2][1][RTW89_IC][28] = 127, + [1][1][2][1][RTW89_KCC][28] = 66, [1][1][2][1][RTW89_ACMA][28] = 127, - [1][1][2][1][RTW89_FCC][32] = 60, + [1][1][2][1][RTW89_CN][28] = 127, + [1][1][2][1][RTW89_UK][28] = 40, + [1][1][2][1][RTW89_FCC][32] = 56, [1][1][2][1][RTW89_ETSI][32] = 40, [1][1][2][1][RTW89_MKK][32] = 66, - [1][1][2][1][RTW89_IC][32] = 66, - [1][1][2][1][RTW89_ACMA][32] = 42, - [1][1][2][1][RTW89_FCC][36] = 60, + [1][1][2][1][RTW89_IC][32] = 56, + [1][1][2][1][RTW89_KCC][32] = 66, + [1][1][2][1][RTW89_ACMA][32] = 40, + [1][1][2][1][RTW89_CN][32] = 127, + [1][1][2][1][RTW89_UK][32] = 40, + [1][1][2][1][RTW89_FCC][36] = 68, [1][1][2][1][RTW89_ETSI][36] = 127, [1][1][2][1][RTW89_MKK][36] = 66, - [1][1][2][1][RTW89_IC][36] = 66, + [1][1][2][1][RTW89_IC][36] = 68, + [1][1][2][1][RTW89_KCC][36] = 66, [1][1][2][1][RTW89_ACMA][36] = 66, + [1][1][2][1][RTW89_CN][36] = 127, + [1][1][2][1][RTW89_UK][36] = 40, [1][1][2][1][RTW89_FCC][39] = 68, [1][1][2][1][RTW89_ETSI][39] = 6, [1][1][2][1][RTW89_MKK][39] = 127, - [1][1][2][1][RTW89_IC][39] = 66, + [1][1][2][1][RTW89_IC][39] = 68, + [1][1][2][1][RTW89_KCC][39] = 56, [1][1][2][1][RTW89_ACMA][39] = 66, + [1][1][2][1][RTW89_CN][39] = 60, + [1][1][2][1][RTW89_UK][39] = 40, [1][1][2][1][RTW89_FCC][43] = 68, [1][1][2][1][RTW89_ETSI][43] = 6, [1][1][2][1][RTW89_MKK][43] = 127, - [1][1][2][1][RTW89_IC][43] = 66, + [1][1][2][1][RTW89_IC][43] = 68, + [1][1][2][1][RTW89_KCC][43] = 56, [1][1][2][1][RTW89_ACMA][43] = 66, - [1][1][2][1][RTW89_FCC][47] = 60, + [1][1][2][1][RTW89_CN][43] = 52, + [1][1][2][1][RTW89_UK][43] = 40, + [1][1][2][1][RTW89_FCC][47] = 62, [1][1][2][1][RTW89_ETSI][47] = 127, [1][1][2][1][RTW89_MKK][47] = 127, [1][1][2][1][RTW89_IC][47] = 127, + [1][1][2][1][RTW89_KCC][47] = 127, [1][1][2][1][RTW89_ACMA][47] = 127, - [1][1][2][1][RTW89_FCC][51] = 58, + [1][1][2][1][RTW89_CN][47] = 127, + [1][1][2][1][RTW89_UK][47] = 127, + [1][1][2][1][RTW89_FCC][51] = 60, [1][1][2][1][RTW89_ETSI][51] = 127, [1][1][2][1][RTW89_MKK][51] = 127, [1][1][2][1][RTW89_IC][51] = 127, + [1][1][2][1][RTW89_KCC][51] = 127, [1][1][2][1][RTW89_ACMA][51] = 127, - [2][0][2][0][RTW89_FCC][3] = 56, + [1][1][2][1][RTW89_CN][51] = 127, + [1][1][2][1][RTW89_UK][51] = 127, + [2][0][2][0][RTW89_FCC][3] = 58, [2][0][2][0][RTW89_ETSI][3] = 60, [2][0][2][0][RTW89_MKK][3] = 60, - [2][0][2][0][RTW89_IC][3] = 60, + [2][0][2][0][RTW89_IC][3] = 56, + [2][0][2][0][RTW89_KCC][3] = 60, [2][0][2][0][RTW89_ACMA][3] = 60, - [2][0][2][0][RTW89_FCC][11] = 58, + [2][0][2][0][RTW89_CN][3] = 54, + [2][0][2][0][RTW89_UK][3] = 60, + [2][0][2][0][RTW89_FCC][11] = 50, [2][0][2][0][RTW89_ETSI][11] = 60, [2][0][2][0][RTW89_MKK][11] = 60, - [2][0][2][0][RTW89_IC][11] = 60, + [2][0][2][0][RTW89_IC][11] = 50, + [2][0][2][0][RTW89_KCC][11] = 58, [2][0][2][0][RTW89_ACMA][11] = 60, - [2][0][2][0][RTW89_FCC][18] = 54, + [2][0][2][0][RTW89_CN][11] = 54, + [2][0][2][0][RTW89_UK][11] = 60, + [2][0][2][0][RTW89_FCC][18] = 60, [2][0][2][0][RTW89_ETSI][18] = 60, [2][0][2][0][RTW89_MKK][18] = 60, [2][0][2][0][RTW89_IC][18] = 60, + [2][0][2][0][RTW89_KCC][18] = 56, [2][0][2][0][RTW89_ACMA][18] = 60, + [2][0][2][0][RTW89_CN][18] = 127, + [2][0][2][0][RTW89_UK][18] = 60, [2][0][2][0][RTW89_FCC][26] = 62, [2][0][2][0][RTW89_ETSI][26] = 60, [2][0][2][0][RTW89_MKK][26] = 60, [2][0][2][0][RTW89_IC][26] = 127, + [2][0][2][0][RTW89_KCC][26] = 60, [2][0][2][0][RTW89_ACMA][26] = 127, + [2][0][2][0][RTW89_CN][26] = 127, + [2][0][2][0][RTW89_UK][26] = 60, [2][0][2][0][RTW89_FCC][34] = 62, [2][0][2][0][RTW89_ETSI][34] = 127, [2][0][2][0][RTW89_MKK][34] = 60, - [2][0][2][0][RTW89_IC][34] = 60, + [2][0][2][0][RTW89_IC][34] = 62, + [2][0][2][0][RTW89_KCC][34] = 60, [2][0][2][0][RTW89_ACMA][34] = 60, + [2][0][2][0][RTW89_CN][34] = 127, + [2][0][2][0][RTW89_UK][34] = 60, [2][0][2][0][RTW89_FCC][41] = 62, [2][0][2][0][RTW89_ETSI][41] = 30, [2][0][2][0][RTW89_MKK][41] = 127, - [2][0][2][0][RTW89_IC][41] = 60, + [2][0][2][0][RTW89_IC][41] = 62, + [2][0][2][0][RTW89_KCC][41] = 58, [2][0][2][0][RTW89_ACMA][41] = 60, - [2][0][2][0][RTW89_FCC][49] = 56, + [2][0][2][0][RTW89_CN][41] = 62, + [2][0][2][0][RTW89_UK][41] = 60, + [2][0][2][0][RTW89_FCC][49] = 62, [2][0][2][0][RTW89_ETSI][49] = 127, [2][0][2][0][RTW89_MKK][49] = 127, [2][0][2][0][RTW89_IC][49] = 127, + [2][0][2][0][RTW89_KCC][49] = 127, [2][0][2][0][RTW89_ACMA][49] = 127, + [2][0][2][0][RTW89_CN][49] = 127, + [2][0][2][0][RTW89_UK][49] = 127, [2][1][2][0][RTW89_FCC][3] = 48, [2][1][2][0][RTW89_ETSI][3] = 54, [2][1][2][0][RTW89_MKK][3] = 56, - [2][1][2][0][RTW89_IC][3] = 52, - [2][1][2][0][RTW89_ACMA][3] = 52, - [2][1][2][0][RTW89_FCC][11] = 54, + [2][1][2][0][RTW89_IC][3] = 46, + [2][1][2][0][RTW89_KCC][3] = 56, + [2][1][2][0][RTW89_ACMA][3] = 54, + [2][1][2][0][RTW89_CN][3] = 52, + [2][1][2][0][RTW89_UK][3] = 54, + [2][1][2][0][RTW89_FCC][11] = 38, [2][1][2][0][RTW89_ETSI][11] = 54, [2][1][2][0][RTW89_MKK][11] = 54, - [2][1][2][0][RTW89_IC][11] = 52, - [2][1][2][0][RTW89_ACMA][11] = 52, - [2][1][2][0][RTW89_FCC][18] = 48, + [2][1][2][0][RTW89_IC][11] = 38, + [2][1][2][0][RTW89_KCC][11] = 52, + [2][1][2][0][RTW89_ACMA][11] = 54, + [2][1][2][0][RTW89_CN][11] = 52, + [2][1][2][0][RTW89_UK][11] = 54, + [2][1][2][0][RTW89_FCC][18] = 50, [2][1][2][0][RTW89_ETSI][18] = 54, [2][1][2][0][RTW89_MKK][18] = 60, - [2][1][2][0][RTW89_IC][18] = 58, - [2][1][2][0][RTW89_ACMA][18] = 52, - [2][1][2][0][RTW89_FCC][26] = 62, + [2][1][2][0][RTW89_IC][18] = 50, + [2][1][2][0][RTW89_KCC][18] = 54, + [2][1][2][0][RTW89_ACMA][18] = 54, + [2][1][2][0][RTW89_CN][18] = 127, + [2][1][2][0][RTW89_UK][18] = 54, + [2][1][2][0][RTW89_FCC][26] = 52, [2][1][2][0][RTW89_ETSI][26] = 54, [2][1][2][0][RTW89_MKK][26] = 56, [2][1][2][0][RTW89_IC][26] = 127, + [2][1][2][0][RTW89_KCC][26] = 60, [2][1][2][0][RTW89_ACMA][26] = 127, + [2][1][2][0][RTW89_CN][26] = 127, + [2][1][2][0][RTW89_UK][26] = 54, [2][1][2][0][RTW89_FCC][34] = 62, [2][1][2][0][RTW89_ETSI][34] = 127, [2][1][2][0][RTW89_MKK][34] = 60, - [2][1][2][0][RTW89_IC][34] = 60, + [2][1][2][0][RTW89_IC][34] = 62, + [2][1][2][0][RTW89_KCC][34] = 60, [2][1][2][0][RTW89_ACMA][34] = 60, - [2][1][2][0][RTW89_FCC][41] = 62, + [2][1][2][0][RTW89_CN][34] = 127, + [2][1][2][0][RTW89_UK][34] = 52, + [2][1][2][0][RTW89_FCC][41] = 60, [2][1][2][0][RTW89_ETSI][41] = 18, [2][1][2][0][RTW89_MKK][41] = 127, [2][1][2][0][RTW89_IC][41] = 60, - [2][1][2][0][RTW89_ACMA][41] = 60, - [2][1][2][0][RTW89_FCC][49] = 50, + [2][1][2][0][RTW89_KCC][41] = 50, + [2][1][2][0][RTW89_ACMA][41] = 58, + [2][1][2][0][RTW89_CN][41] = 62, + [2][1][2][0][RTW89_UK][41] = 52, + [2][1][2][0][RTW89_FCC][49] = 62, [2][1][2][0][RTW89_ETSI][49] = 127, [2][1][2][0][RTW89_MKK][49] = 127, [2][1][2][0][RTW89_IC][49] = 127, + [2][1][2][0][RTW89_KCC][49] = 127, [2][1][2][0][RTW89_ACMA][49] = 127, + [2][1][2][0][RTW89_CN][49] = 127, + [2][1][2][0][RTW89_UK][49] = 127, [2][1][2][1][RTW89_FCC][3] = 48, [2][1][2][1][RTW89_ETSI][3] = 40, [2][1][2][1][RTW89_MKK][3] = 56, [2][1][2][1][RTW89_IC][3] = 40, + [2][1][2][1][RTW89_KCC][3] = 56, [2][1][2][1][RTW89_ACMA][3] = 40, - [2][1][2][1][RTW89_FCC][11] = 54, + [2][1][2][1][RTW89_CN][3] = 42, + [2][1][2][1][RTW89_UK][3] = 40, + [2][1][2][1][RTW89_FCC][11] = 38, [2][1][2][1][RTW89_ETSI][11] = 40, [2][1][2][1][RTW89_MKK][11] = 54, - [2][1][2][1][RTW89_IC][11] = 40, + [2][1][2][1][RTW89_IC][11] = 38, + [2][1][2][1][RTW89_KCC][11] = 52, [2][1][2][1][RTW89_ACMA][11] = 40, - [2][1][2][1][RTW89_FCC][18] = 48, + [2][1][2][1][RTW89_CN][11] = 42, + [2][1][2][1][RTW89_UK][11] = 40, + [2][1][2][1][RTW89_FCC][18] = 50, [2][1][2][1][RTW89_ETSI][18] = 40, [2][1][2][1][RTW89_MKK][18] = 60, - [2][1][2][1][RTW89_IC][18] = 58, + [2][1][2][1][RTW89_IC][18] = 50, + [2][1][2][1][RTW89_KCC][18] = 54, [2][1][2][1][RTW89_ACMA][18] = 40, - [2][1][2][1][RTW89_FCC][26] = 60, + [2][1][2][1][RTW89_CN][18] = 127, + [2][1][2][1][RTW89_UK][18] = 40, + [2][1][2][1][RTW89_FCC][26] = 52, [2][1][2][1][RTW89_ETSI][26] = 42, [2][1][2][1][RTW89_MKK][26] = 56, [2][1][2][1][RTW89_IC][26] = 127, + [2][1][2][1][RTW89_KCC][26] = 60, [2][1][2][1][RTW89_ACMA][26] = 127, - [2][1][2][1][RTW89_FCC][34] = 60, + [2][1][2][1][RTW89_CN][26] = 127, + [2][1][2][1][RTW89_UK][26] = 42, + [2][1][2][1][RTW89_FCC][34] = 62, [2][1][2][1][RTW89_ETSI][34] = 127, [2][1][2][1][RTW89_MKK][34] = 60, - [2][1][2][1][RTW89_IC][34] = 60, + [2][1][2][1][RTW89_IC][34] = 62, + [2][1][2][1][RTW89_KCC][34] = 60, [2][1][2][1][RTW89_ACMA][34] = 60, - [2][1][2][1][RTW89_FCC][41] = 62, + [2][1][2][1][RTW89_CN][34] = 127, + [2][1][2][1][RTW89_UK][34] = 40, + [2][1][2][1][RTW89_FCC][41] = 60, [2][1][2][1][RTW89_ETSI][41] = 6, [2][1][2][1][RTW89_MKK][41] = 127, [2][1][2][1][RTW89_IC][41] = 60, - [2][1][2][1][RTW89_ACMA][41] = 60, - [2][1][2][1][RTW89_FCC][49] = 50, + [2][1][2][1][RTW89_KCC][41] = 50, + [2][1][2][1][RTW89_ACMA][41] = 58, + [2][1][2][1][RTW89_CN][41] = 40, + [2][1][2][1][RTW89_UK][41] = 40, + [2][1][2][1][RTW89_FCC][49] = 62, [2][1][2][1][RTW89_ETSI][49] = 127, [2][1][2][1][RTW89_MKK][49] = 127, [2][1][2][1][RTW89_IC][49] = 127, + [2][1][2][1][RTW89_KCC][49] = 127, [2][1][2][1][RTW89_ACMA][49] = 127, - [3][0][2][0][RTW89_FCC][7] = 38, + [2][1][2][1][RTW89_CN][49] = 127, + [2][1][2][1][RTW89_UK][49] = 127, + [3][0][2][0][RTW89_FCC][7] = 40, [3][0][2][0][RTW89_ETSI][7] = 50, [3][0][2][0][RTW89_MKK][7] = 50, - [3][0][2][0][RTW89_IC][7] = 50, - [3][0][2][0][RTW89_ACMA][7] = 50, - [3][0][2][0][RTW89_FCC][22] = 52, + [3][0][2][0][RTW89_IC][7] = 40, + [3][0][2][0][RTW89_KCC][7] = 44, + [3][0][2][0][RTW89_ACMA][7] = 127, + [3][0][2][0][RTW89_CN][7] = 66, + [3][0][2][0][RTW89_UK][7] = 127, + [3][0][2][0][RTW89_FCC][22] = 42, [3][0][2][0][RTW89_ETSI][22] = 50, [3][0][2][0][RTW89_MKK][22] = 50, - [3][0][2][0][RTW89_IC][22] = 50, - [3][0][2][0][RTW89_ACMA][22] = 50, - [3][0][2][0][RTW89_FCC][45] = 127, + [3][0][2][0][RTW89_IC][22] = 127, + [3][0][2][0][RTW89_KCC][22] = 50, + [3][0][2][0][RTW89_ACMA][22] = 127, + [3][0][2][0][RTW89_CN][22] = 66, + [3][0][2][0][RTW89_UK][22] = 127, + [3][0][2][0][RTW89_FCC][45] = 52, [3][0][2][0][RTW89_ETSI][45] = 127, [3][0][2][0][RTW89_MKK][45] = 127, [3][0][2][0][RTW89_IC][45] = 127, + [3][0][2][0][RTW89_KCC][45] = 127, [3][0][2][0][RTW89_ACMA][45] = 127, - [3][1][2][0][RTW89_FCC][7] = 26, + [3][0][2][0][RTW89_CN][45] = 127, + [3][0][2][0][RTW89_UK][45] = 127, + [3][1][2][0][RTW89_FCC][7] = 32, [3][1][2][0][RTW89_ETSI][7] = 50, [3][1][2][0][RTW89_MKK][7] = 36, [3][1][2][0][RTW89_IC][7] = 44, - [3][1][2][0][RTW89_ACMA][7] = 44, - [3][1][2][0][RTW89_FCC][22] = 42, + [3][1][2][0][RTW89_KCC][7] = 50, + [3][1][2][0][RTW89_ACMA][7] = 127, + [3][1][2][0][RTW89_CN][7] = 54, + [3][1][2][0][RTW89_UK][7] = 127, + [3][1][2][0][RTW89_FCC][22] = 36, [3][1][2][0][RTW89_ETSI][22] = 50, [3][1][2][0][RTW89_MKK][22] = 48, - [3][1][2][0][RTW89_IC][22] = 44, - [3][1][2][0][RTW89_ACMA][22] = 44, - [3][1][2][0][RTW89_FCC][45] = 127, + [3][1][2][0][RTW89_IC][22] = 127, + [3][1][2][0][RTW89_KCC][22] = 50, + [3][1][2][0][RTW89_ACMA][22] = 127, + [3][1][2][0][RTW89_CN][22] = 54, + [3][1][2][0][RTW89_UK][22] = 127, + [3][1][2][0][RTW89_FCC][45] = 46, [3][1][2][0][RTW89_ETSI][45] = 127, [3][1][2][0][RTW89_MKK][45] = 127, [3][1][2][0][RTW89_IC][45] = 127, + [3][1][2][0][RTW89_KCC][45] = 127, [3][1][2][0][RTW89_ACMA][45] = 127, - [3][1][2][1][RTW89_FCC][7] = 14, + [3][1][2][0][RTW89_CN][45] = 127, + [3][1][2][0][RTW89_UK][45] = 127, + [3][1][2][1][RTW89_FCC][7] = 32, [3][1][2][1][RTW89_ETSI][7] = 42, [3][1][2][1][RTW89_MKK][7] = 36, - [3][1][2][1][RTW89_IC][7] = 32, - [3][1][2][1][RTW89_ACMA][7] = 32, - [3][1][2][1][RTW89_FCC][22] = 30, + [3][1][2][1][RTW89_IC][7] = 44, + [3][1][2][1][RTW89_KCC][7] = 50, + [3][1][2][1][RTW89_ACMA][7] = 127, + [3][1][2][1][RTW89_CN][7] = 42, + [3][1][2][1][RTW89_UK][7] = 127, + [3][1][2][1][RTW89_FCC][22] = 36, [3][1][2][1][RTW89_ETSI][22] = 42, [3][1][2][1][RTW89_MKK][22] = 48, - [3][1][2][1][RTW89_IC][22] = 32, - [3][1][2][1][RTW89_ACMA][22] = 32, - [3][1][2][1][RTW89_FCC][45] = 127, + [3][1][2][1][RTW89_IC][22] = 127, + [3][1][2][1][RTW89_KCC][22] = 50, + [3][1][2][1][RTW89_ACMA][22] = 127, + [3][1][2][1][RTW89_CN][22] = 42, + [3][1][2][1][RTW89_UK][22] = 127, + [3][1][2][1][RTW89_FCC][45] = 46, [3][1][2][1][RTW89_ETSI][45] = 127, [3][1][2][1][RTW89_MKK][45] = 127, [3][1][2][1][RTW89_IC][45] = 127, + [3][1][2][1][RTW89_KCC][45] = 127, [3][1][2][1][RTW89_ACMA][45] = 127, + [3][1][2][1][RTW89_CN][45] = 127, + [3][1][2][1][RTW89_UK][45] = 127, }; const s8 rtw89_8852c_txpwr_lmt_6g[RTW89_6G_BW_NUM][RTW89_NTX_NUM] [RTW89_RS_LMT_NUM][RTW89_BF_NUM] [RTW89_REGD_NUM][RTW89_6G_CH_NUM] = { - [0][0][1][0][RTW89_WW][0] = 72, - [0][0][1][0][RTW89_WW][2] = 72, - [0][0][1][0][RTW89_WW][4] = 72, - [0][0][1][0][RTW89_WW][6] = 72, - [0][0][1][0][RTW89_WW][8] = 72, - [0][0][1][0][RTW89_WW][10] = 72, - [0][0][1][0][RTW89_WW][12] = 72, - [0][0][1][0][RTW89_WW][14] = 72, - [0][0][1][0][RTW89_WW][15] = 72, - [0][0][1][0][RTW89_WW][17] = 72, - [0][0][1][0][RTW89_WW][19] = 72, - [0][0][1][0][RTW89_WW][21] = 72, - [0][0][1][0][RTW89_WW][23] = 72, - [0][0][1][0][RTW89_WW][25] = 72, - [0][0][1][0][RTW89_WW][27] = 72, - [0][0][1][0][RTW89_WW][29] = 72, - [0][0][1][0][RTW89_WW][30] = 72, - [0][0][1][0][RTW89_WW][32] = 72, - [0][0][1][0][RTW89_WW][34] = 72, - [0][0][1][0][RTW89_WW][36] = 72, - [0][0][1][0][RTW89_WW][38] = 72, - [0][0][1][0][RTW89_WW][40] = 72, - [0][0][1][0][RTW89_WW][42] = 72, - [0][0][1][0][RTW89_WW][44] = 72, - [0][0][1][0][RTW89_WW][45] = 72, - [0][0][1][0][RTW89_WW][47] = 72, - [0][0][1][0][RTW89_WW][49] = 72, - [0][0][1][0][RTW89_WW][51] = 72, - [0][0][1][0][RTW89_WW][53] = 72, - [0][0][1][0][RTW89_WW][55] = 72, - [0][0][1][0][RTW89_WW][57] = 72, - [0][0][1][0][RTW89_WW][59] = 72, - [0][0][1][0][RTW89_WW][60] = 72, - [0][0][1][0][RTW89_WW][62] = 72, - [0][0][1][0][RTW89_WW][64] = 72, - [0][0][1][0][RTW89_WW][66] = 72, - [0][0][1][0][RTW89_WW][68] = 72, - [0][0][1][0][RTW89_WW][70] = 72, - [0][0][1][0][RTW89_WW][72] = 72, - [0][0][1][0][RTW89_WW][74] = 72, - [0][0][1][0][RTW89_WW][75] = 72, - [0][0][1][0][RTW89_WW][77] = 72, - [0][0][1][0][RTW89_WW][79] = 72, - [0][0][1][0][RTW89_WW][81] = 72, - [0][0][1][0][RTW89_WW][83] = 72, - [0][0][1][0][RTW89_WW][85] = 72, - [0][0][1][0][RTW89_WW][87] = 72, - [0][0][1][0][RTW89_WW][89] = 72, - [0][0][1][0][RTW89_WW][90] = 72, - [0][0][1][0][RTW89_WW][92] = 72, - [0][0][1][0][RTW89_WW][94] = 72, - [0][0][1][0][RTW89_WW][96] = 72, - [0][0][1][0][RTW89_WW][98] = 72, - [0][0][1][0][RTW89_WW][100] = 72, - [0][0][1][0][RTW89_WW][102] = 72, - [0][0][1][0][RTW89_WW][104] = 72, - [0][0][1][0][RTW89_WW][105] = 72, - [0][0][1][0][RTW89_WW][107] = 72, - [0][0][1][0][RTW89_WW][109] = 72, + [0][0][1][0][RTW89_WW][0] = 24, + [0][0][1][0][RTW89_WW][2] = 22, + [0][0][1][0][RTW89_WW][4] = 22, + [0][0][1][0][RTW89_WW][6] = 22, + [0][0][1][0][RTW89_WW][8] = 22, + [0][0][1][0][RTW89_WW][10] = 22, + [0][0][1][0][RTW89_WW][12] = 22, + [0][0][1][0][RTW89_WW][14] = 22, + [0][0][1][0][RTW89_WW][15] = 22, + [0][0][1][0][RTW89_WW][17] = 22, + [0][0][1][0][RTW89_WW][19] = 22, + [0][0][1][0][RTW89_WW][21] = 22, + [0][0][1][0][RTW89_WW][23] = 22, + [0][0][1][0][RTW89_WW][25] = 22, + [0][0][1][0][RTW89_WW][27] = 22, + [0][0][1][0][RTW89_WW][29] = 22, + [0][0][1][0][RTW89_WW][30] = 22, + [0][0][1][0][RTW89_WW][32] = 22, + [0][0][1][0][RTW89_WW][34] = 22, + [0][0][1][0][RTW89_WW][36] = 22, + [0][0][1][0][RTW89_WW][38] = 22, + [0][0][1][0][RTW89_WW][40] = 22, + [0][0][1][0][RTW89_WW][42] = 22, + [0][0][1][0][RTW89_WW][44] = 22, + [0][0][1][0][RTW89_WW][45] = 22, + [0][0][1][0][RTW89_WW][47] = 22, + [0][0][1][0][RTW89_WW][49] = 24, + [0][0][1][0][RTW89_WW][51] = 22, + [0][0][1][0][RTW89_WW][53] = 22, + [0][0][1][0][RTW89_WW][55] = 22, + [0][0][1][0][RTW89_WW][57] = 22, + [0][0][1][0][RTW89_WW][59] = 22, + [0][0][1][0][RTW89_WW][60] = 22, + [0][0][1][0][RTW89_WW][62] = 22, + [0][0][1][0][RTW89_WW][64] = 22, + [0][0][1][0][RTW89_WW][66] = 22, + [0][0][1][0][RTW89_WW][68] = 22, + [0][0][1][0][RTW89_WW][70] = 24, + [0][0][1][0][RTW89_WW][72] = 22, + [0][0][1][0][RTW89_WW][74] = 22, + [0][0][1][0][RTW89_WW][75] = 22, + [0][0][1][0][RTW89_WW][77] = 22, + [0][0][1][0][RTW89_WW][79] = 22, + [0][0][1][0][RTW89_WW][81] = 22, + [0][0][1][0][RTW89_WW][83] = 22, + [0][0][1][0][RTW89_WW][85] = 22, + [0][0][1][0][RTW89_WW][87] = 22, + [0][0][1][0][RTW89_WW][89] = 22, + [0][0][1][0][RTW89_WW][90] = 22, + [0][0][1][0][RTW89_WW][92] = 22, + [0][0][1][0][RTW89_WW][94] = 22, + [0][0][1][0][RTW89_WW][96] = 22, + [0][0][1][0][RTW89_WW][98] = 22, + [0][0][1][0][RTW89_WW][100] = 22, + [0][0][1][0][RTW89_WW][102] = 22, + [0][0][1][0][RTW89_WW][104] = 22, + [0][0][1][0][RTW89_WW][105] = 22, + [0][0][1][0][RTW89_WW][107] = 24, + [0][0][1][0][RTW89_WW][109] = 24, [0][0][1][0][RTW89_WW][111] = 0, [0][0][1][0][RTW89_WW][113] = 0, [0][0][1][0][RTW89_WW][115] = 0, [0][0][1][0][RTW89_WW][117] = 0, [0][0][1][0][RTW89_WW][119] = 0, - [0][1][1][0][RTW89_WW][0] = 60, - [0][1][1][0][RTW89_WW][2] = 60, - [0][1][1][0][RTW89_WW][4] = 60, - [0][1][1][0][RTW89_WW][6] = 60, - [0][1][1][0][RTW89_WW][8] = 60, - [0][1][1][0][RTW89_WW][10] = 60, - [0][1][1][0][RTW89_WW][12] = 60, - [0][1][1][0][RTW89_WW][14] = 60, - [0][1][1][0][RTW89_WW][15] = 60, - [0][1][1][0][RTW89_WW][17] = 60, - [0][1][1][0][RTW89_WW][19] = 60, - [0][1][1][0][RTW89_WW][21] = 60, - [0][1][1][0][RTW89_WW][23] = 60, - [0][1][1][0][RTW89_WW][25] = 60, - [0][1][1][0][RTW89_WW][27] = 60, - [0][1][1][0][RTW89_WW][29] = 60, - [0][1][1][0][RTW89_WW][30] = 60, - [0][1][1][0][RTW89_WW][32] = 60, - [0][1][1][0][RTW89_WW][34] = 60, - [0][1][1][0][RTW89_WW][36] = 60, - [0][1][1][0][RTW89_WW][38] = 60, - [0][1][1][0][RTW89_WW][40] = 60, - [0][1][1][0][RTW89_WW][42] = 60, - [0][1][1][0][RTW89_WW][44] = 60, - [0][1][1][0][RTW89_WW][45] = 60, - [0][1][1][0][RTW89_WW][47] = 60, - [0][1][1][0][RTW89_WW][49] = 60, - [0][1][1][0][RTW89_WW][51] = 60, - [0][1][1][0][RTW89_WW][53] = 60, - [0][1][1][0][RTW89_WW][55] = 60, - [0][1][1][0][RTW89_WW][57] = 60, - [0][1][1][0][RTW89_WW][59] = 60, - [0][1][1][0][RTW89_WW][60] = 60, - [0][1][1][0][RTW89_WW][62] = 60, - [0][1][1][0][RTW89_WW][64] = 60, - [0][1][1][0][RTW89_WW][66] = 60, - [0][1][1][0][RTW89_WW][68] = 60, - [0][1][1][0][RTW89_WW][70] = 60, - [0][1][1][0][RTW89_WW][72] = 60, - [0][1][1][0][RTW89_WW][74] = 60, - [0][1][1][0][RTW89_WW][75] = 60, - [0][1][1][0][RTW89_WW][77] = 60, - [0][1][1][0][RTW89_WW][79] = 60, - [0][1][1][0][RTW89_WW][81] = 60, - [0][1][1][0][RTW89_WW][83] = 60, - [0][1][1][0][RTW89_WW][85] = 60, - [0][1][1][0][RTW89_WW][87] = 60, - [0][1][1][0][RTW89_WW][89] = 60, - [0][1][1][0][RTW89_WW][90] = 60, - [0][1][1][0][RTW89_WW][92] = 60, - [0][1][1][0][RTW89_WW][94] = 60, - [0][1][1][0][RTW89_WW][96] = 60, - [0][1][1][0][RTW89_WW][98] = 60, - [0][1][1][0][RTW89_WW][100] = 60, - [0][1][1][0][RTW89_WW][102] = 60, - [0][1][1][0][RTW89_WW][104] = 60, - [0][1][1][0][RTW89_WW][105] = 60, - [0][1][1][0][RTW89_WW][107] = 60, - [0][1][1][0][RTW89_WW][109] = 60, + [0][1][1][0][RTW89_WW][0] = -2, + [0][1][1][0][RTW89_WW][2] = -4, + [0][1][1][0][RTW89_WW][4] = -4, + [0][1][1][0][RTW89_WW][6] = -4, + [0][1][1][0][RTW89_WW][8] = -4, + [0][1][1][0][RTW89_WW][10] = -4, + [0][1][1][0][RTW89_WW][12] = -4, + [0][1][1][0][RTW89_WW][14] = -4, + [0][1][1][0][RTW89_WW][15] = -4, + [0][1][1][0][RTW89_WW][17] = -4, + [0][1][1][0][RTW89_WW][19] = -4, + [0][1][1][0][RTW89_WW][21] = -4, + [0][1][1][0][RTW89_WW][23] = -4, + [0][1][1][0][RTW89_WW][25] = -4, + [0][1][1][0][RTW89_WW][27] = -4, + [0][1][1][0][RTW89_WW][29] = -4, + [0][1][1][0][RTW89_WW][30] = -4, + [0][1][1][0][RTW89_WW][32] = -4, + [0][1][1][0][RTW89_WW][34] = -4, + [0][1][1][0][RTW89_WW][36] = -4, + [0][1][1][0][RTW89_WW][38] = -4, + [0][1][1][0][RTW89_WW][40] = -4, + [0][1][1][0][RTW89_WW][42] = -4, + [0][1][1][0][RTW89_WW][44] = -2, + [0][1][1][0][RTW89_WW][45] = -2, + [0][1][1][0][RTW89_WW][47] = -2, + [0][1][1][0][RTW89_WW][49] = -2, + [0][1][1][0][RTW89_WW][51] = -2, + [0][1][1][0][RTW89_WW][53] = -2, + [0][1][1][0][RTW89_WW][55] = -2, + [0][1][1][0][RTW89_WW][57] = -2, + [0][1][1][0][RTW89_WW][59] = -2, + [0][1][1][0][RTW89_WW][60] = -2, + [0][1][1][0][RTW89_WW][62] = -2, + [0][1][1][0][RTW89_WW][64] = -2, + [0][1][1][0][RTW89_WW][66] = -2, + [0][1][1][0][RTW89_WW][68] = -2, + [0][1][1][0][RTW89_WW][70] = -2, + [0][1][1][0][RTW89_WW][72] = -2, + [0][1][1][0][RTW89_WW][74] = -2, + [0][1][1][0][RTW89_WW][75] = -2, + [0][1][1][0][RTW89_WW][77] = -2, + [0][1][1][0][RTW89_WW][79] = -2, + [0][1][1][0][RTW89_WW][81] = -2, + [0][1][1][0][RTW89_WW][83] = -2, + [0][1][1][0][RTW89_WW][85] = -2, + [0][1][1][0][RTW89_WW][87] = -2, + [0][1][1][0][RTW89_WW][89] = -2, + [0][1][1][0][RTW89_WW][90] = -2, + [0][1][1][0][RTW89_WW][92] = -2, + [0][1][1][0][RTW89_WW][94] = -2, + [0][1][1][0][RTW89_WW][96] = -2, + [0][1][1][0][RTW89_WW][98] = -2, + [0][1][1][0][RTW89_WW][100] = -2, + [0][1][1][0][RTW89_WW][102] = -2, + [0][1][1][0][RTW89_WW][104] = -2, + [0][1][1][0][RTW89_WW][105] = -2, + [0][1][1][0][RTW89_WW][107] = 1, + [0][1][1][0][RTW89_WW][109] = 1, [0][1][1][0][RTW89_WW][111] = 0, [0][1][1][0][RTW89_WW][113] = 0, [0][1][1][0][RTW89_WW][115] = 0, [0][1][1][0][RTW89_WW][117] = 0, [0][1][1][0][RTW89_WW][119] = 0, - [0][0][2][0][RTW89_WW][0] = 72, - [0][0][2][0][RTW89_WW][2] = 72, - [0][0][2][0][RTW89_WW][4] = 72, - [0][0][2][0][RTW89_WW][6] = 72, - [0][0][2][0][RTW89_WW][8] = 72, - [0][0][2][0][RTW89_WW][10] = 72, - [0][0][2][0][RTW89_WW][12] = 72, - [0][0][2][0][RTW89_WW][14] = 72, - [0][0][2][0][RTW89_WW][15] = 72, - [0][0][2][0][RTW89_WW][17] = 72, - [0][0][2][0][RTW89_WW][19] = 72, - [0][0][2][0][RTW89_WW][21] = 72, - [0][0][2][0][RTW89_WW][23] = 72, - [0][0][2][0][RTW89_WW][25] = 72, - [0][0][2][0][RTW89_WW][27] = 72, - [0][0][2][0][RTW89_WW][29] = 72, - [0][0][2][0][RTW89_WW][30] = 72, - [0][0][2][0][RTW89_WW][32] = 72, - [0][0][2][0][RTW89_WW][34] = 72, - [0][0][2][0][RTW89_WW][36] = 72, - [0][0][2][0][RTW89_WW][38] = 72, - [0][0][2][0][RTW89_WW][40] = 72, - [0][0][2][0][RTW89_WW][42] = 72, - [0][0][2][0][RTW89_WW][44] = 72, - [0][0][2][0][RTW89_WW][45] = 72, - [0][0][2][0][RTW89_WW][47] = 72, - [0][0][2][0][RTW89_WW][49] = 72, - [0][0][2][0][RTW89_WW][51] = 72, - [0][0][2][0][RTW89_WW][53] = 72, - [0][0][2][0][RTW89_WW][55] = 72, - [0][0][2][0][RTW89_WW][57] = 72, - [0][0][2][0][RTW89_WW][59] = 72, - [0][0][2][0][RTW89_WW][60] = 72, - [0][0][2][0][RTW89_WW][62] = 72, - [0][0][2][0][RTW89_WW][64] = 72, - [0][0][2][0][RTW89_WW][66] = 72, - [0][0][2][0][RTW89_WW][68] = 72, - [0][0][2][0][RTW89_WW][70] = 72, - [0][0][2][0][RTW89_WW][72] = 72, - [0][0][2][0][RTW89_WW][74] = 72, - [0][0][2][0][RTW89_WW][75] = 72, - [0][0][2][0][RTW89_WW][77] = 72, - [0][0][2][0][RTW89_WW][79] = 72, - [0][0][2][0][RTW89_WW][81] = 72, - [0][0][2][0][RTW89_WW][83] = 72, - [0][0][2][0][RTW89_WW][85] = 72, - [0][0][2][0][RTW89_WW][87] = 72, - [0][0][2][0][RTW89_WW][89] = 72, - [0][0][2][0][RTW89_WW][90] = 72, - [0][0][2][0][RTW89_WW][92] = 72, - [0][0][2][0][RTW89_WW][94] = 72, - [0][0][2][0][RTW89_WW][96] = 72, - [0][0][2][0][RTW89_WW][98] = 72, - [0][0][2][0][RTW89_WW][100] = 72, - [0][0][2][0][RTW89_WW][102] = 72, - [0][0][2][0][RTW89_WW][104] = 72, - [0][0][2][0][RTW89_WW][105] = 72, - [0][0][2][0][RTW89_WW][107] = 72, - [0][0][2][0][RTW89_WW][109] = 72, + [0][0][2][0][RTW89_WW][0] = 24, + [0][0][2][0][RTW89_WW][2] = 22, + [0][0][2][0][RTW89_WW][4] = 22, + [0][0][2][0][RTW89_WW][6] = 22, + [0][0][2][0][RTW89_WW][8] = 22, + [0][0][2][0][RTW89_WW][10] = 22, + [0][0][2][0][RTW89_WW][12] = 22, + [0][0][2][0][RTW89_WW][14] = 22, + [0][0][2][0][RTW89_WW][15] = 22, + [0][0][2][0][RTW89_WW][17] = 22, + [0][0][2][0][RTW89_WW][19] = 22, + [0][0][2][0][RTW89_WW][21] = 22, + [0][0][2][0][RTW89_WW][23] = 22, + [0][0][2][0][RTW89_WW][25] = 22, + [0][0][2][0][RTW89_WW][27] = 22, + [0][0][2][0][RTW89_WW][29] = 22, + [0][0][2][0][RTW89_WW][30] = 22, + [0][0][2][0][RTW89_WW][32] = 22, + [0][0][2][0][RTW89_WW][34] = 22, + [0][0][2][0][RTW89_WW][36] = 22, + [0][0][2][0][RTW89_WW][38] = 22, + [0][0][2][0][RTW89_WW][40] = 22, + [0][0][2][0][RTW89_WW][42] = 22, + [0][0][2][0][RTW89_WW][44] = 22, + [0][0][2][0][RTW89_WW][45] = 22, + [0][0][2][0][RTW89_WW][47] = 22, + [0][0][2][0][RTW89_WW][49] = 24, + [0][0][2][0][RTW89_WW][51] = 22, + [0][0][2][0][RTW89_WW][53] = 22, + [0][0][2][0][RTW89_WW][55] = 22, + [0][0][2][0][RTW89_WW][57] = 22, + [0][0][2][0][RTW89_WW][59] = 22, + [0][0][2][0][RTW89_WW][60] = 22, + [0][0][2][0][RTW89_WW][62] = 22, + [0][0][2][0][RTW89_WW][64] = 22, + [0][0][2][0][RTW89_WW][66] = 22, + [0][0][2][0][RTW89_WW][68] = 22, + [0][0][2][0][RTW89_WW][70] = 24, + [0][0][2][0][RTW89_WW][72] = 22, + [0][0][2][0][RTW89_WW][74] = 22, + [0][0][2][0][RTW89_WW][75] = 22, + [0][0][2][0][RTW89_WW][77] = 22, + [0][0][2][0][RTW89_WW][79] = 22, + [0][0][2][0][RTW89_WW][81] = 22, + [0][0][2][0][RTW89_WW][83] = 22, + [0][0][2][0][RTW89_WW][85] = 22, + [0][0][2][0][RTW89_WW][87] = 22, + [0][0][2][0][RTW89_WW][89] = 22, + [0][0][2][0][RTW89_WW][90] = 22, + [0][0][2][0][RTW89_WW][92] = 22, + [0][0][2][0][RTW89_WW][94] = 22, + [0][0][2][0][RTW89_WW][96] = 22, + [0][0][2][0][RTW89_WW][98] = 22, + [0][0][2][0][RTW89_WW][100] = 22, + [0][0][2][0][RTW89_WW][102] = 22, + [0][0][2][0][RTW89_WW][104] = 22, + [0][0][2][0][RTW89_WW][105] = 22, + [0][0][2][0][RTW89_WW][107] = 24, + [0][0][2][0][RTW89_WW][109] = 24, [0][0][2][0][RTW89_WW][111] = 0, [0][0][2][0][RTW89_WW][113] = 0, [0][0][2][0][RTW89_WW][115] = 0, [0][0][2][0][RTW89_WW][117] = 0, [0][0][2][0][RTW89_WW][119] = 0, - [0][1][2][0][RTW89_WW][0] = 60, - [0][1][2][0][RTW89_WW][2] = 60, - [0][1][2][0][RTW89_WW][4] = 60, - [0][1][2][0][RTW89_WW][6] = 60, - [0][1][2][0][RTW89_WW][8] = 60, - [0][1][2][0][RTW89_WW][10] = 60, - [0][1][2][0][RTW89_WW][12] = 60, - [0][1][2][0][RTW89_WW][14] = 60, - [0][1][2][0][RTW89_WW][15] = 60, - [0][1][2][0][RTW89_WW][17] = 60, - [0][1][2][0][RTW89_WW][19] = 60, - [0][1][2][0][RTW89_WW][21] = 60, - [0][1][2][0][RTW89_WW][23] = 60, - [0][1][2][0][RTW89_WW][25] = 60, - [0][1][2][0][RTW89_WW][27] = 60, - [0][1][2][0][RTW89_WW][29] = 60, - [0][1][2][0][RTW89_WW][30] = 60, - [0][1][2][0][RTW89_WW][32] = 60, - [0][1][2][0][RTW89_WW][34] = 60, - [0][1][2][0][RTW89_WW][36] = 60, - [0][1][2][0][RTW89_WW][38] = 60, - [0][1][2][0][RTW89_WW][40] = 60, - [0][1][2][0][RTW89_WW][42] = 60, - [0][1][2][0][RTW89_WW][44] = 60, - [0][1][2][0][RTW89_WW][45] = 60, - [0][1][2][0][RTW89_WW][47] = 60, - [0][1][2][0][RTW89_WW][49] = 60, - [0][1][2][0][RTW89_WW][51] = 60, - [0][1][2][0][RTW89_WW][53] = 60, - [0][1][2][0][RTW89_WW][55] = 60, - [0][1][2][0][RTW89_WW][57] = 60, - [0][1][2][0][RTW89_WW][59] = 60, - [0][1][2][0][RTW89_WW][60] = 60, - [0][1][2][0][RTW89_WW][62] = 60, - [0][1][2][0][RTW89_WW][64] = 60, - [0][1][2][0][RTW89_WW][66] = 60, - [0][1][2][0][RTW89_WW][68] = 60, - [0][1][2][0][RTW89_WW][70] = 60, - [0][1][2][0][RTW89_WW][72] = 60, - [0][1][2][0][RTW89_WW][74] = 60, - [0][1][2][0][RTW89_WW][75] = 60, - [0][1][2][0][RTW89_WW][77] = 60, - [0][1][2][0][RTW89_WW][79] = 60, - [0][1][2][0][RTW89_WW][81] = 60, - [0][1][2][0][RTW89_WW][83] = 60, - [0][1][2][0][RTW89_WW][85] = 60, - [0][1][2][0][RTW89_WW][87] = 60, - [0][1][2][0][RTW89_WW][89] = 60, - [0][1][2][0][RTW89_WW][90] = 60, - [0][1][2][0][RTW89_WW][92] = 60, - [0][1][2][0][RTW89_WW][94] = 60, - [0][1][2][0][RTW89_WW][96] = 60, - [0][1][2][0][RTW89_WW][98] = 60, - [0][1][2][0][RTW89_WW][100] = 60, - [0][1][2][0][RTW89_WW][102] = 60, - [0][1][2][0][RTW89_WW][104] = 60, - [0][1][2][0][RTW89_WW][105] = 60, - [0][1][2][0][RTW89_WW][107] = 60, - [0][1][2][0][RTW89_WW][109] = 60, + [0][1][2][0][RTW89_WW][0] = -2, + [0][1][2][0][RTW89_WW][2] = -4, + [0][1][2][0][RTW89_WW][4] = -4, + [0][1][2][0][RTW89_WW][6] = -4, + [0][1][2][0][RTW89_WW][8] = -4, + [0][1][2][0][RTW89_WW][10] = -4, + [0][1][2][0][RTW89_WW][12] = -4, + [0][1][2][0][RTW89_WW][14] = -4, + [0][1][2][0][RTW89_WW][15] = -4, + [0][1][2][0][RTW89_WW][17] = -4, + [0][1][2][0][RTW89_WW][19] = -4, + [0][1][2][0][RTW89_WW][21] = -4, + [0][1][2][0][RTW89_WW][23] = -4, + [0][1][2][0][RTW89_WW][25] = -4, + [0][1][2][0][RTW89_WW][27] = -4, + [0][1][2][0][RTW89_WW][29] = -4, + [0][1][2][0][RTW89_WW][30] = -4, + [0][1][2][0][RTW89_WW][32] = -4, + [0][1][2][0][RTW89_WW][34] = -4, + [0][1][2][0][RTW89_WW][36] = -4, + [0][1][2][0][RTW89_WW][38] = -4, + [0][1][2][0][RTW89_WW][40] = -4, + [0][1][2][0][RTW89_WW][42] = -4, + [0][1][2][0][RTW89_WW][44] = -2, + [0][1][2][0][RTW89_WW][45] = -2, + [0][1][2][0][RTW89_WW][47] = -2, + [0][1][2][0][RTW89_WW][49] = -2, + [0][1][2][0][RTW89_WW][51] = -2, + [0][1][2][0][RTW89_WW][53] = -2, + [0][1][2][0][RTW89_WW][55] = -2, + [0][1][2][0][RTW89_WW][57] = -2, + [0][1][2][0][RTW89_WW][59] = -2, + [0][1][2][0][RTW89_WW][60] = -2, + [0][1][2][0][RTW89_WW][62] = -2, + [0][1][2][0][RTW89_WW][64] = -2, + [0][1][2][0][RTW89_WW][66] = -2, + [0][1][2][0][RTW89_WW][68] = -2, + [0][1][2][0][RTW89_WW][70] = -2, + [0][1][2][0][RTW89_WW][72] = -2, + [0][1][2][0][RTW89_WW][74] = -2, + [0][1][2][0][RTW89_WW][75] = -2, + [0][1][2][0][RTW89_WW][77] = -2, + [0][1][2][0][RTW89_WW][79] = -2, + [0][1][2][0][RTW89_WW][81] = -2, + [0][1][2][0][RTW89_WW][83] = -2, + [0][1][2][0][RTW89_WW][85] = -2, + [0][1][2][0][RTW89_WW][87] = -2, + [0][1][2][0][RTW89_WW][89] = -2, + [0][1][2][0][RTW89_WW][90] = -2, + [0][1][2][0][RTW89_WW][92] = -2, + [0][1][2][0][RTW89_WW][94] = -2, + [0][1][2][0][RTW89_WW][96] = -2, + [0][1][2][0][RTW89_WW][98] = -2, + [0][1][2][0][RTW89_WW][100] = -2, + [0][1][2][0][RTW89_WW][102] = -2, + [0][1][2][0][RTW89_WW][104] = -2, + [0][1][2][0][RTW89_WW][105] = -2, + [0][1][2][0][RTW89_WW][107] = 1, + [0][1][2][0][RTW89_WW][109] = 1, [0][1][2][0][RTW89_WW][111] = 0, [0][1][2][0][RTW89_WW][113] = 0, [0][1][2][0][RTW89_WW][115] = 0, [0][1][2][0][RTW89_WW][117] = 0, [0][1][2][0][RTW89_WW][119] = 0, - [0][1][2][1][RTW89_WW][0] = 48, - [0][1][2][1][RTW89_WW][2] = 48, - [0][1][2][1][RTW89_WW][4] = 48, - [0][1][2][1][RTW89_WW][6] = 48, - [0][1][2][1][RTW89_WW][8] = 48, - [0][1][2][1][RTW89_WW][10] = 48, - [0][1][2][1][RTW89_WW][12] = 48, - [0][1][2][1][RTW89_WW][14] = 48, - [0][1][2][1][RTW89_WW][15] = 48, - [0][1][2][1][RTW89_WW][17] = 48, - [0][1][2][1][RTW89_WW][19] = 48, - [0][1][2][1][RTW89_WW][21] = 48, - [0][1][2][1][RTW89_WW][23] = 48, - [0][1][2][1][RTW89_WW][25] = 48, - [0][1][2][1][RTW89_WW][27] = 48, - [0][1][2][1][RTW89_WW][29] = 48, - [0][1][2][1][RTW89_WW][30] = 48, - [0][1][2][1][RTW89_WW][32] = 48, - [0][1][2][1][RTW89_WW][34] = 48, - [0][1][2][1][RTW89_WW][36] = 48, - [0][1][2][1][RTW89_WW][38] = 48, - [0][1][2][1][RTW89_WW][40] = 48, - [0][1][2][1][RTW89_WW][42] = 48, - [0][1][2][1][RTW89_WW][44] = 48, - [0][1][2][1][RTW89_WW][45] = 48, - [0][1][2][1][RTW89_WW][47] = 48, - [0][1][2][1][RTW89_WW][49] = 48, - [0][1][2][1][RTW89_WW][51] = 48, - [0][1][2][1][RTW89_WW][53] = 48, - [0][1][2][1][RTW89_WW][55] = 48, - [0][1][2][1][RTW89_WW][57] = 48, - [0][1][2][1][RTW89_WW][59] = 48, - [0][1][2][1][RTW89_WW][60] = 48, - [0][1][2][1][RTW89_WW][62] = 48, - [0][1][2][1][RTW89_WW][64] = 48, - [0][1][2][1][RTW89_WW][66] = 48, - [0][1][2][1][RTW89_WW][68] = 48, - [0][1][2][1][RTW89_WW][70] = 48, - [0][1][2][1][RTW89_WW][72] = 48, - [0][1][2][1][RTW89_WW][74] = 48, - [0][1][2][1][RTW89_WW][75] = 48, - [0][1][2][1][RTW89_WW][77] = 48, - [0][1][2][1][RTW89_WW][79] = 48, - [0][1][2][1][RTW89_WW][81] = 48, - [0][1][2][1][RTW89_WW][83] = 48, - [0][1][2][1][RTW89_WW][85] = 48, - [0][1][2][1][RTW89_WW][87] = 48, - [0][1][2][1][RTW89_WW][89] = 48, - [0][1][2][1][RTW89_WW][90] = 48, - [0][1][2][1][RTW89_WW][92] = 48, - [0][1][2][1][RTW89_WW][94] = 48, - [0][1][2][1][RTW89_WW][96] = 48, - [0][1][2][1][RTW89_WW][98] = 48, - [0][1][2][1][RTW89_WW][100] = 48, - [0][1][2][1][RTW89_WW][102] = 48, - [0][1][2][1][RTW89_WW][104] = 48, - [0][1][2][1][RTW89_WW][105] = 48, - [0][1][2][1][RTW89_WW][107] = 48, - [0][1][2][1][RTW89_WW][109] = 48, + [0][1][2][1][RTW89_WW][0] = -2, + [0][1][2][1][RTW89_WW][2] = -4, + [0][1][2][1][RTW89_WW][4] = -4, + [0][1][2][1][RTW89_WW][6] = -4, + [0][1][2][1][RTW89_WW][8] = -4, + [0][1][2][1][RTW89_WW][10] = -4, + [0][1][2][1][RTW89_WW][12] = -4, + [0][1][2][1][RTW89_WW][14] = -4, + [0][1][2][1][RTW89_WW][15] = -4, + [0][1][2][1][RTW89_WW][17] = -4, + [0][1][2][1][RTW89_WW][19] = -4, + [0][1][2][1][RTW89_WW][21] = -4, + [0][1][2][1][RTW89_WW][23] = -4, + [0][1][2][1][RTW89_WW][25] = -4, + [0][1][2][1][RTW89_WW][27] = -4, + [0][1][2][1][RTW89_WW][29] = -4, + [0][1][2][1][RTW89_WW][30] = -4, + [0][1][2][1][RTW89_WW][32] = -4, + [0][1][2][1][RTW89_WW][34] = -4, + [0][1][2][1][RTW89_WW][36] = -4, + [0][1][2][1][RTW89_WW][38] = -4, + [0][1][2][1][RTW89_WW][40] = -4, + [0][1][2][1][RTW89_WW][42] = -4, + [0][1][2][1][RTW89_WW][44] = -2, + [0][1][2][1][RTW89_WW][45] = -2, + [0][1][2][1][RTW89_WW][47] = -2, + [0][1][2][1][RTW89_WW][49] = -2, + [0][1][2][1][RTW89_WW][51] = -2, + [0][1][2][1][RTW89_WW][53] = -2, + [0][1][2][1][RTW89_WW][55] = -2, + [0][1][2][1][RTW89_WW][57] = -2, + [0][1][2][1][RTW89_WW][59] = -2, + [0][1][2][1][RTW89_WW][60] = -2, + [0][1][2][1][RTW89_WW][62] = -2, + [0][1][2][1][RTW89_WW][64] = -2, + [0][1][2][1][RTW89_WW][66] = -2, + [0][1][2][1][RTW89_WW][68] = -2, + [0][1][2][1][RTW89_WW][70] = -2, + [0][1][2][1][RTW89_WW][72] = -2, + [0][1][2][1][RTW89_WW][74] = -2, + [0][1][2][1][RTW89_WW][75] = -2, + [0][1][2][1][RTW89_WW][77] = -2, + [0][1][2][1][RTW89_WW][79] = -2, + [0][1][2][1][RTW89_WW][81] = -2, + [0][1][2][1][RTW89_WW][83] = -2, + [0][1][2][1][RTW89_WW][85] = -2, + [0][1][2][1][RTW89_WW][87] = -2, + [0][1][2][1][RTW89_WW][89] = -2, + [0][1][2][1][RTW89_WW][90] = -2, + [0][1][2][1][RTW89_WW][92] = -2, + [0][1][2][1][RTW89_WW][94] = -2, + [0][1][2][1][RTW89_WW][96] = -2, + [0][1][2][1][RTW89_WW][98] = -2, + [0][1][2][1][RTW89_WW][100] = -2, + [0][1][2][1][RTW89_WW][102] = -2, + [0][1][2][1][RTW89_WW][104] = -2, + [0][1][2][1][RTW89_WW][105] = -2, + [0][1][2][1][RTW89_WW][107] = 1, + [0][1][2][1][RTW89_WW][109] = 1, [0][1][2][1][RTW89_WW][111] = 0, [0][1][2][1][RTW89_WW][113] = 0, [0][1][2][1][RTW89_WW][115] = 0, [0][1][2][1][RTW89_WW][117] = 0, [0][1][2][1][RTW89_WW][119] = 0, - [1][0][2][0][RTW89_WW][1] = 72, - [1][0][2][0][RTW89_WW][5] = 72, - [1][0][2][0][RTW89_WW][9] = 72, - [1][0][2][0][RTW89_WW][13] = 72, - [1][0][2][0][RTW89_WW][16] = 72, - [1][0][2][0][RTW89_WW][20] = 72, - [1][0][2][0][RTW89_WW][24] = 72, - [1][0][2][0][RTW89_WW][28] = 72, - [1][0][2][0][RTW89_WW][31] = 72, - [1][0][2][0][RTW89_WW][35] = 72, - [1][0][2][0][RTW89_WW][39] = 72, - [1][0][2][0][RTW89_WW][43] = 72, - [1][0][2][0][RTW89_WW][46] = 72, - [1][0][2][0][RTW89_WW][50] = 72, - [1][0][2][0][RTW89_WW][54] = 72, - [1][0][2][0][RTW89_WW][58] = 72, - [1][0][2][0][RTW89_WW][61] = 72, - [1][0][2][0][RTW89_WW][65] = 72, - [1][0][2][0][RTW89_WW][69] = 72, - [1][0][2][0][RTW89_WW][73] = 72, - [1][0][2][0][RTW89_WW][76] = 72, - [1][0][2][0][RTW89_WW][80] = 72, - [1][0][2][0][RTW89_WW][84] = 72, - [1][0][2][0][RTW89_WW][88] = 72, - [1][0][2][0][RTW89_WW][91] = 72, - [1][0][2][0][RTW89_WW][95] = 72, - [1][0][2][0][RTW89_WW][99] = 72, - [1][0][2][0][RTW89_WW][103] = 72, - [1][0][2][0][RTW89_WW][106] = 72, + [1][0][2][0][RTW89_WW][1] = 34, + [1][0][2][0][RTW89_WW][5] = 34, + [1][0][2][0][RTW89_WW][9] = 34, + [1][0][2][0][RTW89_WW][13] = 34, + [1][0][2][0][RTW89_WW][16] = 34, + [1][0][2][0][RTW89_WW][20] = 34, + [1][0][2][0][RTW89_WW][24] = 36, + [1][0][2][0][RTW89_WW][28] = 34, + [1][0][2][0][RTW89_WW][31] = 34, + [1][0][2][0][RTW89_WW][35] = 34, + [1][0][2][0][RTW89_WW][39] = 34, + [1][0][2][0][RTW89_WW][43] = 34, + [1][0][2][0][RTW89_WW][46] = 34, + [1][0][2][0][RTW89_WW][50] = 34, + [1][0][2][0][RTW89_WW][54] = 36, + [1][0][2][0][RTW89_WW][58] = 36, + [1][0][2][0][RTW89_WW][61] = 34, + [1][0][2][0][RTW89_WW][65] = 34, + [1][0][2][0][RTW89_WW][69] = 34, + [1][0][2][0][RTW89_WW][73] = 34, + [1][0][2][0][RTW89_WW][76] = 34, + [1][0][2][0][RTW89_WW][80] = 34, + [1][0][2][0][RTW89_WW][84] = 34, + [1][0][2][0][RTW89_WW][88] = 34, + [1][0][2][0][RTW89_WW][91] = 36, + [1][0][2][0][RTW89_WW][95] = 34, + [1][0][2][0][RTW89_WW][99] = 34, + [1][0][2][0][RTW89_WW][103] = 34, + [1][0][2][0][RTW89_WW][106] = 36, [1][0][2][0][RTW89_WW][110] = 0, [1][0][2][0][RTW89_WW][114] = 0, [1][0][2][0][RTW89_WW][118] = 0, - [1][1][2][0][RTW89_WW][1] = 60, - [1][1][2][0][RTW89_WW][5] = 60, - [1][1][2][0][RTW89_WW][9] = 60, - [1][1][2][0][RTW89_WW][13] = 60, - [1][1][2][0][RTW89_WW][16] = 60, - [1][1][2][0][RTW89_WW][20] = 60, - [1][1][2][0][RTW89_WW][24] = 60, - [1][1][2][0][RTW89_WW][28] = 60, - [1][1][2][0][RTW89_WW][31] = 60, - [1][1][2][0][RTW89_WW][35] = 60, - [1][1][2][0][RTW89_WW][39] = 60, - [1][1][2][0][RTW89_WW][43] = 60, - [1][1][2][0][RTW89_WW][46] = 60, - [1][1][2][0][RTW89_WW][50] = 60, - [1][1][2][0][RTW89_WW][54] = 60, - [1][1][2][0][RTW89_WW][58] = 60, - [1][1][2][0][RTW89_WW][61] = 60, - [1][1][2][0][RTW89_WW][65] = 60, - [1][1][2][0][RTW89_WW][69] = 60, - [1][1][2][0][RTW89_WW][73] = 60, - [1][1][2][0][RTW89_WW][76] = 60, - [1][1][2][0][RTW89_WW][80] = 60, - [1][1][2][0][RTW89_WW][84] = 60, - [1][1][2][0][RTW89_WW][88] = 60, - [1][1][2][0][RTW89_WW][91] = 60, - [1][1][2][0][RTW89_WW][95] = 60, - [1][1][2][0][RTW89_WW][99] = 60, - [1][1][2][0][RTW89_WW][103] = 60, - [1][1][2][0][RTW89_WW][106] = 60, + [1][1][2][0][RTW89_WW][1] = 10, + [1][1][2][0][RTW89_WW][5] = 10, + [1][1][2][0][RTW89_WW][9] = 10, + [1][1][2][0][RTW89_WW][13] = 10, + [1][1][2][0][RTW89_WW][16] = 10, + [1][1][2][0][RTW89_WW][20] = 10, + [1][1][2][0][RTW89_WW][24] = 10, + [1][1][2][0][RTW89_WW][28] = 10, + [1][1][2][0][RTW89_WW][31] = 10, + [1][1][2][0][RTW89_WW][35] = 10, + [1][1][2][0][RTW89_WW][39] = 10, + [1][1][2][0][RTW89_WW][43] = 10, + [1][1][2][0][RTW89_WW][46] = 12, + [1][1][2][0][RTW89_WW][50] = 12, + [1][1][2][0][RTW89_WW][54] = 10, + [1][1][2][0][RTW89_WW][58] = 10, + [1][1][2][0][RTW89_WW][61] = 10, + [1][1][2][0][RTW89_WW][65] = 10, + [1][1][2][0][RTW89_WW][69] = 10, + [1][1][2][0][RTW89_WW][73] = 10, + [1][1][2][0][RTW89_WW][76] = 10, + [1][1][2][0][RTW89_WW][80] = 10, + [1][1][2][0][RTW89_WW][84] = 10, + [1][1][2][0][RTW89_WW][88] = 10, + [1][1][2][0][RTW89_WW][91] = 12, + [1][1][2][0][RTW89_WW][95] = 10, + [1][1][2][0][RTW89_WW][99] = 10, + [1][1][2][0][RTW89_WW][103] = 10, + [1][1][2][0][RTW89_WW][106] = 12, [1][1][2][0][RTW89_WW][110] = 0, [1][1][2][0][RTW89_WW][114] = 0, [1][1][2][0][RTW89_WW][118] = 0, - [1][1][2][1][RTW89_WW][1] = 48, - [1][1][2][1][RTW89_WW][5] = 48, - [1][1][2][1][RTW89_WW][9] = 48, - [1][1][2][1][RTW89_WW][13] = 48, - [1][1][2][1][RTW89_WW][16] = 48, - [1][1][2][1][RTW89_WW][20] = 48, - [1][1][2][1][RTW89_WW][24] = 48, - [1][1][2][1][RTW89_WW][28] = 48, - [1][1][2][1][RTW89_WW][31] = 48, - [1][1][2][1][RTW89_WW][35] = 48, - [1][1][2][1][RTW89_WW][39] = 48, - [1][1][2][1][RTW89_WW][43] = 48, - [1][1][2][1][RTW89_WW][46] = 48, - [1][1][2][1][RTW89_WW][50] = 48, - [1][1][2][1][RTW89_WW][54] = 48, - [1][1][2][1][RTW89_WW][58] = 48, - [1][1][2][1][RTW89_WW][61] = 48, - [1][1][2][1][RTW89_WW][65] = 48, - [1][1][2][1][RTW89_WW][69] = 48, - [1][1][2][1][RTW89_WW][73] = 48, - [1][1][2][1][RTW89_WW][76] = 48, - [1][1][2][1][RTW89_WW][80] = 48, - [1][1][2][1][RTW89_WW][84] = 48, - [1][1][2][1][RTW89_WW][88] = 48, - [1][1][2][1][RTW89_WW][91] = 48, - [1][1][2][1][RTW89_WW][95] = 48, - [1][1][2][1][RTW89_WW][99] = 48, - [1][1][2][1][RTW89_WW][103] = 48, - [1][1][2][1][RTW89_WW][106] = 48, + [1][1][2][1][RTW89_WW][1] = 10, + [1][1][2][1][RTW89_WW][5] = 10, + [1][1][2][1][RTW89_WW][9] = 10, + [1][1][2][1][RTW89_WW][13] = 10, + [1][1][2][1][RTW89_WW][16] = 10, + [1][1][2][1][RTW89_WW][20] = 10, + [1][1][2][1][RTW89_WW][24] = 10, + [1][1][2][1][RTW89_WW][28] = 10, + [1][1][2][1][RTW89_WW][31] = 10, + [1][1][2][1][RTW89_WW][35] = 10, + [1][1][2][1][RTW89_WW][39] = 10, + [1][1][2][1][RTW89_WW][43] = 10, + [1][1][2][1][RTW89_WW][46] = 12, + [1][1][2][1][RTW89_WW][50] = 12, + [1][1][2][1][RTW89_WW][54] = 10, + [1][1][2][1][RTW89_WW][58] = 10, + [1][1][2][1][RTW89_WW][61] = 10, + [1][1][2][1][RTW89_WW][65] = 10, + [1][1][2][1][RTW89_WW][69] = 10, + [1][1][2][1][RTW89_WW][73] = 10, + [1][1][2][1][RTW89_WW][76] = 10, + [1][1][2][1][RTW89_WW][80] = 10, + [1][1][2][1][RTW89_WW][84] = 10, + [1][1][2][1][RTW89_WW][88] = 10, + [1][1][2][1][RTW89_WW][91] = 12, + [1][1][2][1][RTW89_WW][95] = 10, + [1][1][2][1][RTW89_WW][99] = 10, + [1][1][2][1][RTW89_WW][103] = 10, + [1][1][2][1][RTW89_WW][106] = 12, [1][1][2][1][RTW89_WW][110] = 0, [1][1][2][1][RTW89_WW][114] = 0, [1][1][2][1][RTW89_WW][118] = 0, - [2][0][2][0][RTW89_WW][3] = 64, - [2][0][2][0][RTW89_WW][11] = 64, - [2][0][2][0][RTW89_WW][18] = 64, - [2][0][2][0][RTW89_WW][26] = 64, - [2][0][2][0][RTW89_WW][33] = 64, - [2][0][2][0][RTW89_WW][41] = 64, - [2][0][2][0][RTW89_WW][48] = 64, - [2][0][2][0][RTW89_WW][56] = 64, - [2][0][2][0][RTW89_WW][63] = 64, - [2][0][2][0][RTW89_WW][71] = 64, - [2][0][2][0][RTW89_WW][78] = 64, - [2][0][2][0][RTW89_WW][86] = 64, - [2][0][2][0][RTW89_WW][93] = 64, - [2][0][2][0][RTW89_WW][101] = 64, + [2][0][2][0][RTW89_WW][3] = 46, + [2][0][2][0][RTW89_WW][11] = 46, + [2][0][2][0][RTW89_WW][18] = 46, + [2][0][2][0][RTW89_WW][26] = 46, + [2][0][2][0][RTW89_WW][33] = 46, + [2][0][2][0][RTW89_WW][41] = 46, + [2][0][2][0][RTW89_WW][48] = 46, + [2][0][2][0][RTW89_WW][56] = 46, + [2][0][2][0][RTW89_WW][63] = 46, + [2][0][2][0][RTW89_WW][71] = 46, + [2][0][2][0][RTW89_WW][78] = 46, + [2][0][2][0][RTW89_WW][86] = 46, + [2][0][2][0][RTW89_WW][93] = 46, + [2][0][2][0][RTW89_WW][101] = 44, [2][0][2][0][RTW89_WW][108] = 0, [2][0][2][0][RTW89_WW][116] = 0, - [2][1][2][0][RTW89_WW][3] = 52, - [2][1][2][0][RTW89_WW][11] = 52, - [2][1][2][0][RTW89_WW][18] = 52, - [2][1][2][0][RTW89_WW][26] = 52, - [2][1][2][0][RTW89_WW][33] = 52, - [2][1][2][0][RTW89_WW][41] = 52, - [2][1][2][0][RTW89_WW][48] = 52, - [2][1][2][0][RTW89_WW][56] = 52, - [2][1][2][0][RTW89_WW][63] = 52, - [2][1][2][0][RTW89_WW][71] = 52, - [2][1][2][0][RTW89_WW][78] = 52, - [2][1][2][0][RTW89_WW][86] = 52, - [2][1][2][0][RTW89_WW][93] = 52, - [2][1][2][0][RTW89_WW][101] = 52, + [2][1][2][0][RTW89_WW][3] = 22, + [2][1][2][0][RTW89_WW][11] = 20, + [2][1][2][0][RTW89_WW][18] = 20, + [2][1][2][0][RTW89_WW][26] = 20, + [2][1][2][0][RTW89_WW][33] = 20, + [2][1][2][0][RTW89_WW][41] = 22, + [2][1][2][0][RTW89_WW][48] = 22, + [2][1][2][0][RTW89_WW][56] = 20, + [2][1][2][0][RTW89_WW][63] = 22, + [2][1][2][0][RTW89_WW][71] = 20, + [2][1][2][0][RTW89_WW][78] = 20, + [2][1][2][0][RTW89_WW][86] = 20, + [2][1][2][0][RTW89_WW][93] = 22, + [2][1][2][0][RTW89_WW][101] = 22, [2][1][2][0][RTW89_WW][108] = 0, [2][1][2][0][RTW89_WW][116] = 0, - [2][1][2][1][RTW89_WW][3] = 40, - [2][1][2][1][RTW89_WW][11] = 40, - [2][1][2][1][RTW89_WW][18] = 40, - [2][1][2][1][RTW89_WW][26] = 40, - [2][1][2][1][RTW89_WW][33] = 40, - [2][1][2][1][RTW89_WW][41] = 40, - [2][1][2][1][RTW89_WW][48] = 40, - [2][1][2][1][RTW89_WW][56] = 40, - [2][1][2][1][RTW89_WW][63] = 40, - [2][1][2][1][RTW89_WW][71] = 40, - [2][1][2][1][RTW89_WW][78] = 40, - [2][1][2][1][RTW89_WW][86] = 40, - [2][1][2][1][RTW89_WW][93] = 40, - [2][1][2][1][RTW89_WW][101] = 40, + [2][1][2][1][RTW89_WW][3] = 22, + [2][1][2][1][RTW89_WW][11] = 20, + [2][1][2][1][RTW89_WW][18] = 20, + [2][1][2][1][RTW89_WW][26] = 20, + [2][1][2][1][RTW89_WW][33] = 20, + [2][1][2][1][RTW89_WW][41] = 22, + [2][1][2][1][RTW89_WW][48] = 22, + [2][1][2][1][RTW89_WW][56] = 20, + [2][1][2][1][RTW89_WW][63] = 22, + [2][1][2][1][RTW89_WW][71] = 20, + [2][1][2][1][RTW89_WW][78] = 20, + [2][1][2][1][RTW89_WW][86] = 20, + [2][1][2][1][RTW89_WW][93] = 22, + [2][1][2][1][RTW89_WW][101] = 22, [2][1][2][1][RTW89_WW][108] = 0, [2][1][2][1][RTW89_WW][116] = 0, - [3][0][2][0][RTW89_WW][7] = 56, - [3][0][2][0][RTW89_WW][22] = 56, - [3][0][2][0][RTW89_WW][37] = 56, - [3][0][2][0][RTW89_WW][52] = 56, - [3][0][2][0][RTW89_WW][67] = 56, - [3][0][2][0][RTW89_WW][82] = 56, - [3][0][2][0][RTW89_WW][97] = 56, + [3][0][2][0][RTW89_WW][7] = 38, + [3][0][2][0][RTW89_WW][22] = 38, + [3][0][2][0][RTW89_WW][37] = 38, + [3][0][2][0][RTW89_WW][52] = 54, + [3][0][2][0][RTW89_WW][67] = 54, + [3][0][2][0][RTW89_WW][82] = 26, + [3][0][2][0][RTW89_WW][97] = 26, [3][0][2][0][RTW89_WW][112] = 0, - [3][1][2][0][RTW89_WW][7] = 44, - [3][1][2][0][RTW89_WW][22] = 44, - [3][1][2][0][RTW89_WW][37] = 44, - [3][1][2][0][RTW89_WW][52] = 44, - [3][1][2][0][RTW89_WW][67] = 44, - [3][1][2][0][RTW89_WW][82] = 44, - [3][1][2][0][RTW89_WW][97] = 44, + [3][1][2][0][RTW89_WW][7] = 32, + [3][1][2][0][RTW89_WW][22] = 30, + [3][1][2][0][RTW89_WW][37] = 30, + [3][1][2][0][RTW89_WW][52] = 30, + [3][1][2][0][RTW89_WW][67] = 32, + [3][1][2][0][RTW89_WW][82] = 24, + [3][1][2][0][RTW89_WW][97] = 14, [3][1][2][0][RTW89_WW][112] = 0, [3][1][2][1][RTW89_WW][7] = 32, - [3][1][2][1][RTW89_WW][22] = 32, - [3][1][2][1][RTW89_WW][37] = 32, - [3][1][2][1][RTW89_WW][52] = 32, + [3][1][2][1][RTW89_WW][22] = 30, + [3][1][2][1][RTW89_WW][37] = 30, + [3][1][2][1][RTW89_WW][52] = 30, [3][1][2][1][RTW89_WW][67] = 32, - [3][1][2][1][RTW89_WW][82] = 32, - [3][1][2][1][RTW89_WW][97] = 32, + [3][1][2][1][RTW89_WW][82] = 24, + [3][1][2][1][RTW89_WW][97] = 14, [3][1][2][1][RTW89_WW][112] = 0, - [0][0][1][0][RTW89_FCC][0] = 72, - [0][0][1][0][RTW89_FCC][2] = 72, - [0][0][1][0][RTW89_FCC][4] = 72, - [0][0][1][0][RTW89_FCC][6] = 72, - [0][0][1][0][RTW89_FCC][8] = 72, - [0][0][1][0][RTW89_FCC][10] = 72, - [0][0][1][0][RTW89_FCC][12] = 72, - [0][0][1][0][RTW89_FCC][14] = 72, - [0][0][1][0][RTW89_FCC][15] = 72, - [0][0][1][0][RTW89_FCC][17] = 72, - [0][0][1][0][RTW89_FCC][19] = 72, - [0][0][1][0][RTW89_FCC][21] = 72, - [0][0][1][0][RTW89_FCC][23] = 72, - [0][0][1][0][RTW89_FCC][25] = 72, - [0][0][1][0][RTW89_FCC][27] = 72, - [0][0][1][0][RTW89_FCC][29] = 72, - [0][0][1][0][RTW89_FCC][30] = 72, - [0][0][1][0][RTW89_FCC][32] = 72, - [0][0][1][0][RTW89_FCC][34] = 72, - [0][0][1][0][RTW89_FCC][36] = 72, - [0][0][1][0][RTW89_FCC][38] = 72, - [0][0][1][0][RTW89_FCC][40] = 72, - [0][0][1][0][RTW89_FCC][42] = 72, - [0][0][1][0][RTW89_FCC][44] = 72, - [0][0][1][0][RTW89_FCC][45] = 72, - [0][0][1][0][RTW89_FCC][47] = 72, - [0][0][1][0][RTW89_FCC][49] = 72, - [0][0][1][0][RTW89_FCC][51] = 72, - [0][0][1][0][RTW89_FCC][53] = 72, - [0][0][1][0][RTW89_FCC][55] = 72, - [0][0][1][0][RTW89_FCC][57] = 72, - [0][0][1][0][RTW89_FCC][59] = 72, - [0][0][1][0][RTW89_FCC][60] = 72, - [0][0][1][0][RTW89_FCC][62] = 72, - [0][0][1][0][RTW89_FCC][64] = 72, - [0][0][1][0][RTW89_FCC][66] = 72, - [0][0][1][0][RTW89_FCC][68] = 72, - [0][0][1][0][RTW89_FCC][70] = 72, - [0][0][1][0][RTW89_FCC][72] = 72, - [0][0][1][0][RTW89_FCC][74] = 72, - [0][0][1][0][RTW89_FCC][75] = 72, - [0][0][1][0][RTW89_FCC][77] = 72, - [0][0][1][0][RTW89_FCC][79] = 72, - [0][0][1][0][RTW89_FCC][81] = 72, - [0][0][1][0][RTW89_FCC][83] = 72, - [0][0][1][0][RTW89_FCC][85] = 72, - [0][0][1][0][RTW89_FCC][87] = 72, - [0][0][1][0][RTW89_FCC][89] = 72, - [0][0][1][0][RTW89_FCC][90] = 72, - [0][0][1][0][RTW89_FCC][92] = 72, - [0][0][1][0][RTW89_FCC][94] = 72, - [0][0][1][0][RTW89_FCC][96] = 72, - [0][0][1][0][RTW89_FCC][98] = 72, - [0][0][1][0][RTW89_FCC][100] = 72, - [0][0][1][0][RTW89_FCC][102] = 72, - [0][0][1][0][RTW89_FCC][104] = 72, - [0][0][1][0][RTW89_FCC][105] = 72, - [0][0][1][0][RTW89_FCC][107] = 72, - [0][0][1][0][RTW89_FCC][109] = 72, + [0][0][1][0][RTW89_FCC][0] = 24, + [0][0][1][0][RTW89_ETSI][0] = 66, + [0][0][1][0][RTW89_KCC][0] = 24, + [0][0][1][0][RTW89_FCC][2] = 22, + [0][0][1][0][RTW89_ETSI][2] = 66, + [0][0][1][0][RTW89_KCC][2] = 24, + [0][0][1][0][RTW89_FCC][4] = 22, + [0][0][1][0][RTW89_ETSI][4] = 66, + [0][0][1][0][RTW89_KCC][4] = 24, + [0][0][1][0][RTW89_FCC][6] = 22, + [0][0][1][0][RTW89_ETSI][6] = 66, + [0][0][1][0][RTW89_KCC][6] = 24, + [0][0][1][0][RTW89_FCC][8] = 22, + [0][0][1][0][RTW89_ETSI][8] = 66, + [0][0][1][0][RTW89_KCC][8] = 24, + [0][0][1][0][RTW89_FCC][10] = 22, + [0][0][1][0][RTW89_ETSI][10] = 66, + [0][0][1][0][RTW89_KCC][10] = 24, + [0][0][1][0][RTW89_FCC][12] = 22, + [0][0][1][0][RTW89_ETSI][12] = 66, + [0][0][1][0][RTW89_KCC][12] = 24, + [0][0][1][0][RTW89_FCC][14] = 22, + [0][0][1][0][RTW89_ETSI][14] = 66, + [0][0][1][0][RTW89_KCC][14] = 24, + [0][0][1][0][RTW89_FCC][15] = 22, + [0][0][1][0][RTW89_ETSI][15] = 66, + [0][0][1][0][RTW89_KCC][15] = 24, + [0][0][1][0][RTW89_FCC][17] = 22, + [0][0][1][0][RTW89_ETSI][17] = 66, + [0][0][1][0][RTW89_KCC][17] = 24, + [0][0][1][0][RTW89_FCC][19] = 22, + [0][0][1][0][RTW89_ETSI][19] = 66, + [0][0][1][0][RTW89_KCC][19] = 24, + [0][0][1][0][RTW89_FCC][21] = 22, + [0][0][1][0][RTW89_ETSI][21] = 66, + [0][0][1][0][RTW89_KCC][21] = 24, + [0][0][1][0][RTW89_FCC][23] = 22, + [0][0][1][0][RTW89_ETSI][23] = 66, + [0][0][1][0][RTW89_KCC][23] = 24, + [0][0][1][0][RTW89_FCC][25] = 22, + [0][0][1][0][RTW89_ETSI][25] = 66, + [0][0][1][0][RTW89_KCC][25] = 24, + [0][0][1][0][RTW89_FCC][27] = 22, + [0][0][1][0][RTW89_ETSI][27] = 66, + [0][0][1][0][RTW89_KCC][27] = 24, + [0][0][1][0][RTW89_FCC][29] = 22, + [0][0][1][0][RTW89_ETSI][29] = 66, + [0][0][1][0][RTW89_KCC][29] = 24, + [0][0][1][0][RTW89_FCC][30] = 22, + [0][0][1][0][RTW89_ETSI][30] = 66, + [0][0][1][0][RTW89_KCC][30] = 24, + [0][0][1][0][RTW89_FCC][32] = 22, + [0][0][1][0][RTW89_ETSI][32] = 66, + [0][0][1][0][RTW89_KCC][32] = 24, + [0][0][1][0][RTW89_FCC][34] = 22, + [0][0][1][0][RTW89_ETSI][34] = 66, + [0][0][1][0][RTW89_KCC][34] = 24, + [0][0][1][0][RTW89_FCC][36] = 22, + [0][0][1][0][RTW89_ETSI][36] = 66, + [0][0][1][0][RTW89_KCC][36] = 24, + [0][0][1][0][RTW89_FCC][38] = 22, + [0][0][1][0][RTW89_ETSI][38] = 66, + [0][0][1][0][RTW89_KCC][38] = 24, + [0][0][1][0][RTW89_FCC][40] = 22, + [0][0][1][0][RTW89_ETSI][40] = 66, + [0][0][1][0][RTW89_KCC][40] = 24, + [0][0][1][0][RTW89_FCC][42] = 22, + [0][0][1][0][RTW89_ETSI][42] = 66, + [0][0][1][0][RTW89_KCC][42] = 24, + [0][0][1][0][RTW89_FCC][44] = 22, + [0][0][1][0][RTW89_ETSI][44] = 66, + [0][0][1][0][RTW89_KCC][44] = 24, + [0][0][1][0][RTW89_FCC][45] = 22, + [0][0][1][0][RTW89_ETSI][45] = 127, + [0][0][1][0][RTW89_KCC][45] = 24, + [0][0][1][0][RTW89_FCC][47] = 22, + [0][0][1][0][RTW89_ETSI][47] = 127, + [0][0][1][0][RTW89_KCC][47] = 24, + [0][0][1][0][RTW89_FCC][49] = 24, + [0][0][1][0][RTW89_ETSI][49] = 127, + [0][0][1][0][RTW89_KCC][49] = 24, + [0][0][1][0][RTW89_FCC][51] = 22, + [0][0][1][0][RTW89_ETSI][51] = 127, + [0][0][1][0][RTW89_KCC][51] = 24, + [0][0][1][0][RTW89_FCC][53] = 22, + [0][0][1][0][RTW89_ETSI][53] = 127, + [0][0][1][0][RTW89_KCC][53] = 24, + [0][0][1][0][RTW89_FCC][55] = 22, + [0][0][1][0][RTW89_ETSI][55] = 127, + [0][0][1][0][RTW89_KCC][55] = 26, + [0][0][1][0][RTW89_FCC][57] = 22, + [0][0][1][0][RTW89_ETSI][57] = 127, + [0][0][1][0][RTW89_KCC][57] = 26, + [0][0][1][0][RTW89_FCC][59] = 22, + [0][0][1][0][RTW89_ETSI][59] = 127, + [0][0][1][0][RTW89_KCC][59] = 26, + [0][0][1][0][RTW89_FCC][60] = 22, + [0][0][1][0][RTW89_ETSI][60] = 127, + [0][0][1][0][RTW89_KCC][60] = 26, + [0][0][1][0][RTW89_FCC][62] = 22, + [0][0][1][0][RTW89_ETSI][62] = 127, + [0][0][1][0][RTW89_KCC][62] = 26, + [0][0][1][0][RTW89_FCC][64] = 22, + [0][0][1][0][RTW89_ETSI][64] = 127, + [0][0][1][0][RTW89_KCC][64] = 26, + [0][0][1][0][RTW89_FCC][66] = 22, + [0][0][1][0][RTW89_ETSI][66] = 127, + [0][0][1][0][RTW89_KCC][66] = 26, + [0][0][1][0][RTW89_FCC][68] = 22, + [0][0][1][0][RTW89_ETSI][68] = 127, + [0][0][1][0][RTW89_KCC][68] = 26, + [0][0][1][0][RTW89_FCC][70] = 24, + [0][0][1][0][RTW89_ETSI][70] = 127, + [0][0][1][0][RTW89_KCC][70] = 26, + [0][0][1][0][RTW89_FCC][72] = 22, + [0][0][1][0][RTW89_ETSI][72] = 127, + [0][0][1][0][RTW89_KCC][72] = 26, + [0][0][1][0][RTW89_FCC][74] = 22, + [0][0][1][0][RTW89_ETSI][74] = 127, + [0][0][1][0][RTW89_KCC][74] = 26, + [0][0][1][0][RTW89_FCC][75] = 22, + [0][0][1][0][RTW89_ETSI][75] = 127, + [0][0][1][0][RTW89_KCC][75] = 26, + [0][0][1][0][RTW89_FCC][77] = 22, + [0][0][1][0][RTW89_ETSI][77] = 127, + [0][0][1][0][RTW89_KCC][77] = 26, + [0][0][1][0][RTW89_FCC][79] = 22, + [0][0][1][0][RTW89_ETSI][79] = 127, + [0][0][1][0][RTW89_KCC][79] = 26, + [0][0][1][0][RTW89_FCC][81] = 22, + [0][0][1][0][RTW89_ETSI][81] = 127, + [0][0][1][0][RTW89_KCC][81] = 26, + [0][0][1][0][RTW89_FCC][83] = 22, + [0][0][1][0][RTW89_ETSI][83] = 127, + [0][0][1][0][RTW89_KCC][83] = 32, + [0][0][1][0][RTW89_FCC][85] = 22, + [0][0][1][0][RTW89_ETSI][85] = 127, + [0][0][1][0][RTW89_KCC][85] = 32, + [0][0][1][0][RTW89_FCC][87] = 22, + [0][0][1][0][RTW89_ETSI][87] = 127, + [0][0][1][0][RTW89_KCC][87] = 32, + [0][0][1][0][RTW89_FCC][89] = 22, + [0][0][1][0][RTW89_ETSI][89] = 127, + [0][0][1][0][RTW89_KCC][89] = 32, + [0][0][1][0][RTW89_FCC][90] = 22, + [0][0][1][0][RTW89_ETSI][90] = 127, + [0][0][1][0][RTW89_KCC][90] = 32, + [0][0][1][0][RTW89_FCC][92] = 22, + [0][0][1][0][RTW89_ETSI][92] = 127, + [0][0][1][0][RTW89_KCC][92] = 32, + [0][0][1][0][RTW89_FCC][94] = 22, + [0][0][1][0][RTW89_ETSI][94] = 127, + [0][0][1][0][RTW89_KCC][94] = 32, + [0][0][1][0][RTW89_FCC][96] = 22, + [0][0][1][0][RTW89_ETSI][96] = 127, + [0][0][1][0][RTW89_KCC][96] = 32, + [0][0][1][0][RTW89_FCC][98] = 22, + [0][0][1][0][RTW89_ETSI][98] = 127, + [0][0][1][0][RTW89_KCC][98] = 32, + [0][0][1][0][RTW89_FCC][100] = 22, + [0][0][1][0][RTW89_ETSI][100] = 127, + [0][0][1][0][RTW89_KCC][100] = 32, + [0][0][1][0][RTW89_FCC][102] = 22, + [0][0][1][0][RTW89_ETSI][102] = 127, + [0][0][1][0][RTW89_KCC][102] = 32, + [0][0][1][0][RTW89_FCC][104] = 22, + [0][0][1][0][RTW89_ETSI][104] = 127, + [0][0][1][0][RTW89_KCC][104] = 32, + [0][0][1][0][RTW89_FCC][105] = 22, + [0][0][1][0][RTW89_ETSI][105] = 127, + [0][0][1][0][RTW89_KCC][105] = 32, + [0][0][1][0][RTW89_FCC][107] = 24, + [0][0][1][0][RTW89_ETSI][107] = 127, + [0][0][1][0][RTW89_KCC][107] = 32, + [0][0][1][0][RTW89_FCC][109] = 24, + [0][0][1][0][RTW89_ETSI][109] = 127, + [0][0][1][0][RTW89_KCC][109] = 32, [0][0][1][0][RTW89_FCC][111] = 127, + [0][0][1][0][RTW89_ETSI][111] = 127, + [0][0][1][0][RTW89_KCC][111] = 127, [0][0][1][0][RTW89_FCC][113] = 127, + [0][0][1][0][RTW89_ETSI][113] = 127, + [0][0][1][0][RTW89_KCC][113] = 127, [0][0][1][0][RTW89_FCC][115] = 127, + [0][0][1][0][RTW89_ETSI][115] = 127, + [0][0][1][0][RTW89_KCC][115] = 127, [0][0][1][0][RTW89_FCC][117] = 127, + [0][0][1][0][RTW89_ETSI][117] = 127, + [0][0][1][0][RTW89_KCC][117] = 127, [0][0][1][0][RTW89_FCC][119] = 127, - [0][1][1][0][RTW89_FCC][0] = 60, - [0][1][1][0][RTW89_FCC][2] = 60, - [0][1][1][0][RTW89_FCC][4] = 60, - [0][1][1][0][RTW89_FCC][6] = 60, - [0][1][1][0][RTW89_FCC][8] = 60, - [0][1][1][0][RTW89_FCC][10] = 60, - [0][1][1][0][RTW89_FCC][12] = 60, - [0][1][1][0][RTW89_FCC][14] = 60, - [0][1][1][0][RTW89_FCC][15] = 60, - [0][1][1][0][RTW89_FCC][17] = 60, - [0][1][1][0][RTW89_FCC][19] = 60, - [0][1][1][0][RTW89_FCC][21] = 60, - [0][1][1][0][RTW89_FCC][23] = 60, - [0][1][1][0][RTW89_FCC][25] = 60, - [0][1][1][0][RTW89_FCC][27] = 60, - [0][1][1][0][RTW89_FCC][29] = 60, - [0][1][1][0][RTW89_FCC][30] = 60, - [0][1][1][0][RTW89_FCC][32] = 60, - [0][1][1][0][RTW89_FCC][34] = 60, - [0][1][1][0][RTW89_FCC][36] = 60, - [0][1][1][0][RTW89_FCC][38] = 60, - [0][1][1][0][RTW89_FCC][40] = 60, - [0][1][1][0][RTW89_FCC][42] = 60, - [0][1][1][0][RTW89_FCC][44] = 60, - [0][1][1][0][RTW89_FCC][45] = 60, - [0][1][1][0][RTW89_FCC][47] = 60, - [0][1][1][0][RTW89_FCC][49] = 60, - [0][1][1][0][RTW89_FCC][51] = 60, - [0][1][1][0][RTW89_FCC][53] = 60, - [0][1][1][0][RTW89_FCC][55] = 60, - [0][1][1][0][RTW89_FCC][57] = 60, - [0][1][1][0][RTW89_FCC][59] = 60, - [0][1][1][0][RTW89_FCC][60] = 60, - [0][1][1][0][RTW89_FCC][62] = 60, - [0][1][1][0][RTW89_FCC][64] = 60, - [0][1][1][0][RTW89_FCC][66] = 60, - [0][1][1][0][RTW89_FCC][68] = 60, - [0][1][1][0][RTW89_FCC][70] = 60, - [0][1][1][0][RTW89_FCC][72] = 60, - [0][1][1][0][RTW89_FCC][74] = 60, - [0][1][1][0][RTW89_FCC][75] = 60, - [0][1][1][0][RTW89_FCC][77] = 60, - [0][1][1][0][RTW89_FCC][79] = 60, - [0][1][1][0][RTW89_FCC][81] = 60, - [0][1][1][0][RTW89_FCC][83] = 60, - [0][1][1][0][RTW89_FCC][85] = 60, - [0][1][1][0][RTW89_FCC][87] = 60, - [0][1][1][0][RTW89_FCC][89] = 60, - [0][1][1][0][RTW89_FCC][90] = 60, - [0][1][1][0][RTW89_FCC][92] = 60, - [0][1][1][0][RTW89_FCC][94] = 60, - [0][1][1][0][RTW89_FCC][96] = 60, - [0][1][1][0][RTW89_FCC][98] = 60, - [0][1][1][0][RTW89_FCC][100] = 60, - [0][1][1][0][RTW89_FCC][102] = 60, - [0][1][1][0][RTW89_FCC][104] = 60, - [0][1][1][0][RTW89_FCC][105] = 60, - [0][1][1][0][RTW89_FCC][107] = 60, - [0][1][1][0][RTW89_FCC][109] = 60, + [0][0][1][0][RTW89_ETSI][119] = 127, + [0][0][1][0][RTW89_KCC][119] = 127, + [0][1][1][0][RTW89_FCC][0] = -2, + [0][1][1][0][RTW89_ETSI][0] = 54, + [0][1][1][0][RTW89_KCC][0] = 12, + [0][1][1][0][RTW89_FCC][2] = -4, + [0][1][1][0][RTW89_ETSI][2] = 54, + [0][1][1][0][RTW89_KCC][2] = 12, + [0][1][1][0][RTW89_FCC][4] = -4, + [0][1][1][0][RTW89_ETSI][4] = 54, + [0][1][1][0][RTW89_KCC][4] = 12, + [0][1][1][0][RTW89_FCC][6] = -4, + [0][1][1][0][RTW89_ETSI][6] = 54, + [0][1][1][0][RTW89_KCC][6] = 12, + [0][1][1][0][RTW89_FCC][8] = -4, + [0][1][1][0][RTW89_ETSI][8] = 54, + [0][1][1][0][RTW89_KCC][8] = 12, + [0][1][1][0][RTW89_FCC][10] = -4, + [0][1][1][0][RTW89_ETSI][10] = 54, + [0][1][1][0][RTW89_KCC][10] = 12, + [0][1][1][0][RTW89_FCC][12] = -4, + [0][1][1][0][RTW89_ETSI][12] = 54, + [0][1][1][0][RTW89_KCC][12] = 12, + [0][1][1][0][RTW89_FCC][14] = -4, + [0][1][1][0][RTW89_ETSI][14] = 54, + [0][1][1][0][RTW89_KCC][14] = 12, + [0][1][1][0][RTW89_FCC][15] = -4, + [0][1][1][0][RTW89_ETSI][15] = 54, + [0][1][1][0][RTW89_KCC][15] = 12, + [0][1][1][0][RTW89_FCC][17] = -4, + [0][1][1][0][RTW89_ETSI][17] = 54, + [0][1][1][0][RTW89_KCC][17] = 12, + [0][1][1][0][RTW89_FCC][19] = -4, + [0][1][1][0][RTW89_ETSI][19] = 54, + [0][1][1][0][RTW89_KCC][19] = 12, + [0][1][1][0][RTW89_FCC][21] = -4, + [0][1][1][0][RTW89_ETSI][21] = 54, + [0][1][1][0][RTW89_KCC][21] = 12, + [0][1][1][0][RTW89_FCC][23] = -4, + [0][1][1][0][RTW89_ETSI][23] = 54, + [0][1][1][0][RTW89_KCC][23] = 12, + [0][1][1][0][RTW89_FCC][25] = -4, + [0][1][1][0][RTW89_ETSI][25] = 54, + [0][1][1][0][RTW89_KCC][25] = 12, + [0][1][1][0][RTW89_FCC][27] = -4, + [0][1][1][0][RTW89_ETSI][27] = 54, + [0][1][1][0][RTW89_KCC][27] = 12, + [0][1][1][0][RTW89_FCC][29] = -4, + [0][1][1][0][RTW89_ETSI][29] = 54, + [0][1][1][0][RTW89_KCC][29] = 12, + [0][1][1][0][RTW89_FCC][30] = -4, + [0][1][1][0][RTW89_ETSI][30] = 54, + [0][1][1][0][RTW89_KCC][30] = 12, + [0][1][1][0][RTW89_FCC][32] = -4, + [0][1][1][0][RTW89_ETSI][32] = 54, + [0][1][1][0][RTW89_KCC][32] = 12, + [0][1][1][0][RTW89_FCC][34] = -4, + [0][1][1][0][RTW89_ETSI][34] = 54, + [0][1][1][0][RTW89_KCC][34] = 12, + [0][1][1][0][RTW89_FCC][36] = -4, + [0][1][1][0][RTW89_ETSI][36] = 54, + [0][1][1][0][RTW89_KCC][36] = 12, + [0][1][1][0][RTW89_FCC][38] = -4, + [0][1][1][0][RTW89_ETSI][38] = 54, + [0][1][1][0][RTW89_KCC][38] = 12, + [0][1][1][0][RTW89_FCC][40] = -4, + [0][1][1][0][RTW89_ETSI][40] = 54, + [0][1][1][0][RTW89_KCC][40] = 12, + [0][1][1][0][RTW89_FCC][42] = -4, + [0][1][1][0][RTW89_ETSI][42] = 54, + [0][1][1][0][RTW89_KCC][42] = 12, + [0][1][1][0][RTW89_FCC][44] = -2, + [0][1][1][0][RTW89_ETSI][44] = 54, + [0][1][1][0][RTW89_KCC][44] = 12, + [0][1][1][0][RTW89_FCC][45] = -2, + [0][1][1][0][RTW89_ETSI][45] = 127, + [0][1][1][0][RTW89_KCC][45] = 12, + [0][1][1][0][RTW89_FCC][47] = -2, + [0][1][1][0][RTW89_ETSI][47] = 127, + [0][1][1][0][RTW89_KCC][47] = 12, + [0][1][1][0][RTW89_FCC][49] = -2, + [0][1][1][0][RTW89_ETSI][49] = 127, + [0][1][1][0][RTW89_KCC][49] = 12, + [0][1][1][0][RTW89_FCC][51] = -2, + [0][1][1][0][RTW89_ETSI][51] = 127, + [0][1][1][0][RTW89_KCC][51] = 12, + [0][1][1][0][RTW89_FCC][53] = -2, + [0][1][1][0][RTW89_ETSI][53] = 127, + [0][1][1][0][RTW89_KCC][53] = 12, + [0][1][1][0][RTW89_FCC][55] = -2, + [0][1][1][0][RTW89_ETSI][55] = 127, + [0][1][1][0][RTW89_KCC][55] = 12, + [0][1][1][0][RTW89_FCC][57] = -2, + [0][1][1][0][RTW89_ETSI][57] = 127, + [0][1][1][0][RTW89_KCC][57] = 12, + [0][1][1][0][RTW89_FCC][59] = -2, + [0][1][1][0][RTW89_ETSI][59] = 127, + [0][1][1][0][RTW89_KCC][59] = 12, + [0][1][1][0][RTW89_FCC][60] = -2, + [0][1][1][0][RTW89_ETSI][60] = 127, + [0][1][1][0][RTW89_KCC][60] = 12, + [0][1][1][0][RTW89_FCC][62] = -2, + [0][1][1][0][RTW89_ETSI][62] = 127, + [0][1][1][0][RTW89_KCC][62] = 12, + [0][1][1][0][RTW89_FCC][64] = -2, + [0][1][1][0][RTW89_ETSI][64] = 127, + [0][1][1][0][RTW89_KCC][64] = 12, + [0][1][1][0][RTW89_FCC][66] = -2, + [0][1][1][0][RTW89_ETSI][66] = 127, + [0][1][1][0][RTW89_KCC][66] = 12, + [0][1][1][0][RTW89_FCC][68] = -2, + [0][1][1][0][RTW89_ETSI][68] = 127, + [0][1][1][0][RTW89_KCC][68] = 12, + [0][1][1][0][RTW89_FCC][70] = -2, + [0][1][1][0][RTW89_ETSI][70] = 127, + [0][1][1][0][RTW89_KCC][70] = 12, + [0][1][1][0][RTW89_FCC][72] = -2, + [0][1][1][0][RTW89_ETSI][72] = 127, + [0][1][1][0][RTW89_KCC][72] = 12, + [0][1][1][0][RTW89_FCC][74] = -2, + [0][1][1][0][RTW89_ETSI][74] = 127, + [0][1][1][0][RTW89_KCC][74] = 12, + [0][1][1][0][RTW89_FCC][75] = -2, + [0][1][1][0][RTW89_ETSI][75] = 127, + [0][1][1][0][RTW89_KCC][75] = 12, + [0][1][1][0][RTW89_FCC][77] = -2, + [0][1][1][0][RTW89_ETSI][77] = 127, + [0][1][1][0][RTW89_KCC][77] = 12, + [0][1][1][0][RTW89_FCC][79] = -2, + [0][1][1][0][RTW89_ETSI][79] = 127, + [0][1][1][0][RTW89_KCC][79] = 12, + [0][1][1][0][RTW89_FCC][81] = -2, + [0][1][1][0][RTW89_ETSI][81] = 127, + [0][1][1][0][RTW89_KCC][81] = 12, + [0][1][1][0][RTW89_FCC][83] = -2, + [0][1][1][0][RTW89_ETSI][83] = 127, + [0][1][1][0][RTW89_KCC][83] = 20, + [0][1][1][0][RTW89_FCC][85] = -2, + [0][1][1][0][RTW89_ETSI][85] = 127, + [0][1][1][0][RTW89_KCC][85] = 20, + [0][1][1][0][RTW89_FCC][87] = -2, + [0][1][1][0][RTW89_ETSI][87] = 127, + [0][1][1][0][RTW89_KCC][87] = 20, + [0][1][1][0][RTW89_FCC][89] = -2, + [0][1][1][0][RTW89_ETSI][89] = 127, + [0][1][1][0][RTW89_KCC][89] = 20, + [0][1][1][0][RTW89_FCC][90] = -2, + [0][1][1][0][RTW89_ETSI][90] = 127, + [0][1][1][0][RTW89_KCC][90] = 20, + [0][1][1][0][RTW89_FCC][92] = -2, + [0][1][1][0][RTW89_ETSI][92] = 127, + [0][1][1][0][RTW89_KCC][92] = 20, + [0][1][1][0][RTW89_FCC][94] = -2, + [0][1][1][0][RTW89_ETSI][94] = 127, + [0][1][1][0][RTW89_KCC][94] = 20, + [0][1][1][0][RTW89_FCC][96] = -2, + [0][1][1][0][RTW89_ETSI][96] = 127, + [0][1][1][0][RTW89_KCC][96] = 20, + [0][1][1][0][RTW89_FCC][98] = -2, + [0][1][1][0][RTW89_ETSI][98] = 127, + [0][1][1][0][RTW89_KCC][98] = 20, + [0][1][1][0][RTW89_FCC][100] = -2, + [0][1][1][0][RTW89_ETSI][100] = 127, + [0][1][1][0][RTW89_KCC][100] = 20, + [0][1][1][0][RTW89_FCC][102] = -2, + [0][1][1][0][RTW89_ETSI][102] = 127, + [0][1][1][0][RTW89_KCC][102] = 20, + [0][1][1][0][RTW89_FCC][104] = -2, + [0][1][1][0][RTW89_ETSI][104] = 127, + [0][1][1][0][RTW89_KCC][104] = 20, + [0][1][1][0][RTW89_FCC][105] = -2, + [0][1][1][0][RTW89_ETSI][105] = 127, + [0][1][1][0][RTW89_KCC][105] = 20, + [0][1][1][0][RTW89_FCC][107] = 0, + [0][1][1][0][RTW89_ETSI][107] = 127, + [0][1][1][0][RTW89_KCC][107] = 20, + [0][1][1][0][RTW89_FCC][109] = 0, + [0][1][1][0][RTW89_ETSI][109] = 127, + [0][1][1][0][RTW89_KCC][109] = 20, [0][1][1][0][RTW89_FCC][111] = 127, + [0][1][1][0][RTW89_ETSI][111] = 127, + [0][1][1][0][RTW89_KCC][111] = 127, [0][1][1][0][RTW89_FCC][113] = 127, + [0][1][1][0][RTW89_ETSI][113] = 127, + [0][1][1][0][RTW89_KCC][113] = 127, [0][1][1][0][RTW89_FCC][115] = 127, + [0][1][1][0][RTW89_ETSI][115] = 127, + [0][1][1][0][RTW89_KCC][115] = 127, [0][1][1][0][RTW89_FCC][117] = 127, + [0][1][1][0][RTW89_ETSI][117] = 127, + [0][1][1][0][RTW89_KCC][117] = 127, [0][1][1][0][RTW89_FCC][119] = 127, - [0][0][2][0][RTW89_FCC][0] = 72, - [0][0][2][0][RTW89_FCC][2] = 72, - [0][0][2][0][RTW89_FCC][4] = 72, - [0][0][2][0][RTW89_FCC][6] = 72, - [0][0][2][0][RTW89_FCC][8] = 72, - [0][0][2][0][RTW89_FCC][10] = 72, - [0][0][2][0][RTW89_FCC][12] = 72, - [0][0][2][0][RTW89_FCC][14] = 72, - [0][0][2][0][RTW89_FCC][15] = 72, - [0][0][2][0][RTW89_FCC][17] = 72, - [0][0][2][0][RTW89_FCC][19] = 72, - [0][0][2][0][RTW89_FCC][21] = 72, - [0][0][2][0][RTW89_FCC][23] = 72, - [0][0][2][0][RTW89_FCC][25] = 72, - [0][0][2][0][RTW89_FCC][27] = 72, - [0][0][2][0][RTW89_FCC][29] = 72, - [0][0][2][0][RTW89_FCC][30] = 72, - [0][0][2][0][RTW89_FCC][32] = 72, - [0][0][2][0][RTW89_FCC][34] = 72, - [0][0][2][0][RTW89_FCC][36] = 72, - [0][0][2][0][RTW89_FCC][38] = 72, - [0][0][2][0][RTW89_FCC][40] = 72, - [0][0][2][0][RTW89_FCC][42] = 72, - [0][0][2][0][RTW89_FCC][44] = 72, - [0][0][2][0][RTW89_FCC][45] = 72, - [0][0][2][0][RTW89_FCC][47] = 72, - [0][0][2][0][RTW89_FCC][49] = 72, - [0][0][2][0][RTW89_FCC][51] = 72, - [0][0][2][0][RTW89_FCC][53] = 72, - [0][0][2][0][RTW89_FCC][55] = 72, - [0][0][2][0][RTW89_FCC][57] = 72, - [0][0][2][0][RTW89_FCC][59] = 72, - [0][0][2][0][RTW89_FCC][60] = 72, - [0][0][2][0][RTW89_FCC][62] = 72, - [0][0][2][0][RTW89_FCC][64] = 72, - [0][0][2][0][RTW89_FCC][66] = 72, - [0][0][2][0][RTW89_FCC][68] = 72, - [0][0][2][0][RTW89_FCC][70] = 72, - [0][0][2][0][RTW89_FCC][72] = 72, - [0][0][2][0][RTW89_FCC][74] = 72, - [0][0][2][0][RTW89_FCC][75] = 72, - [0][0][2][0][RTW89_FCC][77] = 72, - [0][0][2][0][RTW89_FCC][79] = 72, - [0][0][2][0][RTW89_FCC][81] = 72, - [0][0][2][0][RTW89_FCC][83] = 72, - [0][0][2][0][RTW89_FCC][85] = 72, - [0][0][2][0][RTW89_FCC][87] = 72, - [0][0][2][0][RTW89_FCC][89] = 72, - [0][0][2][0][RTW89_FCC][90] = 72, - [0][0][2][0][RTW89_FCC][92] = 72, - [0][0][2][0][RTW89_FCC][94] = 72, - [0][0][2][0][RTW89_FCC][96] = 72, - [0][0][2][0][RTW89_FCC][98] = 72, - [0][0][2][0][RTW89_FCC][100] = 72, - [0][0][2][0][RTW89_FCC][102] = 72, - [0][0][2][0][RTW89_FCC][104] = 72, - [0][0][2][0][RTW89_FCC][105] = 72, - [0][0][2][0][RTW89_FCC][107] = 72, - [0][0][2][0][RTW89_FCC][109] = 72, + [0][1][1][0][RTW89_ETSI][119] = 127, + [0][1][1][0][RTW89_KCC][119] = 127, + [0][0][2][0][RTW89_FCC][0] = 24, + [0][0][2][0][RTW89_ETSI][0] = 66, + [0][0][2][0][RTW89_KCC][0] = 24, + [0][0][2][0][RTW89_FCC][2] = 22, + [0][0][2][0][RTW89_ETSI][2] = 66, + [0][0][2][0][RTW89_KCC][2] = 24, + [0][0][2][0][RTW89_FCC][4] = 22, + [0][0][2][0][RTW89_ETSI][4] = 66, + [0][0][2][0][RTW89_KCC][4] = 24, + [0][0][2][0][RTW89_FCC][6] = 22, + [0][0][2][0][RTW89_ETSI][6] = 66, + [0][0][2][0][RTW89_KCC][6] = 24, + [0][0][2][0][RTW89_FCC][8] = 22, + [0][0][2][0][RTW89_ETSI][8] = 66, + [0][0][2][0][RTW89_KCC][8] = 24, + [0][0][2][0][RTW89_FCC][10] = 22, + [0][0][2][0][RTW89_ETSI][10] = 66, + [0][0][2][0][RTW89_KCC][10] = 24, + [0][0][2][0][RTW89_FCC][12] = 22, + [0][0][2][0][RTW89_ETSI][12] = 66, + [0][0][2][0][RTW89_KCC][12] = 24, + [0][0][2][0][RTW89_FCC][14] = 22, + [0][0][2][0][RTW89_ETSI][14] = 66, + [0][0][2][0][RTW89_KCC][14] = 24, + [0][0][2][0][RTW89_FCC][15] = 22, + [0][0][2][0][RTW89_ETSI][15] = 66, + [0][0][2][0][RTW89_KCC][15] = 24, + [0][0][2][0][RTW89_FCC][17] = 22, + [0][0][2][0][RTW89_ETSI][17] = 66, + [0][0][2][0][RTW89_KCC][17] = 24, + [0][0][2][0][RTW89_FCC][19] = 22, + [0][0][2][0][RTW89_ETSI][19] = 66, + [0][0][2][0][RTW89_KCC][19] = 24, + [0][0][2][0][RTW89_FCC][21] = 22, + [0][0][2][0][RTW89_ETSI][21] = 66, + [0][0][2][0][RTW89_KCC][21] = 24, + [0][0][2][0][RTW89_FCC][23] = 22, + [0][0][2][0][RTW89_ETSI][23] = 66, + [0][0][2][0][RTW89_KCC][23] = 24, + [0][0][2][0][RTW89_FCC][25] = 22, + [0][0][2][0][RTW89_ETSI][25] = 66, + [0][0][2][0][RTW89_KCC][25] = 24, + [0][0][2][0][RTW89_FCC][27] = 22, + [0][0][2][0][RTW89_ETSI][27] = 66, + [0][0][2][0][RTW89_KCC][27] = 24, + [0][0][2][0][RTW89_FCC][29] = 22, + [0][0][2][0][RTW89_ETSI][29] = 66, + [0][0][2][0][RTW89_KCC][29] = 24, + [0][0][2][0][RTW89_FCC][30] = 22, + [0][0][2][0][RTW89_ETSI][30] = 66, + [0][0][2][0][RTW89_KCC][30] = 24, + [0][0][2][0][RTW89_FCC][32] = 22, + [0][0][2][0][RTW89_ETSI][32] = 66, + [0][0][2][0][RTW89_KCC][32] = 24, + [0][0][2][0][RTW89_FCC][34] = 22, + [0][0][2][0][RTW89_ETSI][34] = 66, + [0][0][2][0][RTW89_KCC][34] = 24, + [0][0][2][0][RTW89_FCC][36] = 22, + [0][0][2][0][RTW89_ETSI][36] = 66, + [0][0][2][0][RTW89_KCC][36] = 24, + [0][0][2][0][RTW89_FCC][38] = 22, + [0][0][2][0][RTW89_ETSI][38] = 66, + [0][0][2][0][RTW89_KCC][38] = 24, + [0][0][2][0][RTW89_FCC][40] = 22, + [0][0][2][0][RTW89_ETSI][40] = 66, + [0][0][2][0][RTW89_KCC][40] = 24, + [0][0][2][0][RTW89_FCC][42] = 22, + [0][0][2][0][RTW89_ETSI][42] = 66, + [0][0][2][0][RTW89_KCC][42] = 24, + [0][0][2][0][RTW89_FCC][44] = 22, + [0][0][2][0][RTW89_ETSI][44] = 66, + [0][0][2][0][RTW89_KCC][44] = 24, + [0][0][2][0][RTW89_FCC][45] = 22, + [0][0][2][0][RTW89_ETSI][45] = 127, + [0][0][2][0][RTW89_KCC][45] = 24, + [0][0][2][0][RTW89_FCC][47] = 22, + [0][0][2][0][RTW89_ETSI][47] = 127, + [0][0][2][0][RTW89_KCC][47] = 24, + [0][0][2][0][RTW89_FCC][49] = 24, + [0][0][2][0][RTW89_ETSI][49] = 127, + [0][0][2][0][RTW89_KCC][49] = 24, + [0][0][2][0][RTW89_FCC][51] = 22, + [0][0][2][0][RTW89_ETSI][51] = 127, + [0][0][2][0][RTW89_KCC][51] = 24, + [0][0][2][0][RTW89_FCC][53] = 22, + [0][0][2][0][RTW89_ETSI][53] = 127, + [0][0][2][0][RTW89_KCC][53] = 24, + [0][0][2][0][RTW89_FCC][55] = 22, + [0][0][2][0][RTW89_ETSI][55] = 127, + [0][0][2][0][RTW89_KCC][55] = 26, + [0][0][2][0][RTW89_FCC][57] = 22, + [0][0][2][0][RTW89_ETSI][57] = 127, + [0][0][2][0][RTW89_KCC][57] = 26, + [0][0][2][0][RTW89_FCC][59] = 22, + [0][0][2][0][RTW89_ETSI][59] = 127, + [0][0][2][0][RTW89_KCC][59] = 26, + [0][0][2][0][RTW89_FCC][60] = 22, + [0][0][2][0][RTW89_ETSI][60] = 127, + [0][0][2][0][RTW89_KCC][60] = 26, + [0][0][2][0][RTW89_FCC][62] = 22, + [0][0][2][0][RTW89_ETSI][62] = 127, + [0][0][2][0][RTW89_KCC][62] = 26, + [0][0][2][0][RTW89_FCC][64] = 22, + [0][0][2][0][RTW89_ETSI][64] = 127, + [0][0][2][0][RTW89_KCC][64] = 26, + [0][0][2][0][RTW89_FCC][66] = 22, + [0][0][2][0][RTW89_ETSI][66] = 127, + [0][0][2][0][RTW89_KCC][66] = 26, + [0][0][2][0][RTW89_FCC][68] = 22, + [0][0][2][0][RTW89_ETSI][68] = 127, + [0][0][2][0][RTW89_KCC][68] = 26, + [0][0][2][0][RTW89_FCC][70] = 24, + [0][0][2][0][RTW89_ETSI][70] = 127, + [0][0][2][0][RTW89_KCC][70] = 26, + [0][0][2][0][RTW89_FCC][72] = 22, + [0][0][2][0][RTW89_ETSI][72] = 127, + [0][0][2][0][RTW89_KCC][72] = 26, + [0][0][2][0][RTW89_FCC][74] = 22, + [0][0][2][0][RTW89_ETSI][74] = 127, + [0][0][2][0][RTW89_KCC][74] = 26, + [0][0][2][0][RTW89_FCC][75] = 22, + [0][0][2][0][RTW89_ETSI][75] = 127, + [0][0][2][0][RTW89_KCC][75] = 26, + [0][0][2][0][RTW89_FCC][77] = 22, + [0][0][2][0][RTW89_ETSI][77] = 127, + [0][0][2][0][RTW89_KCC][77] = 26, + [0][0][2][0][RTW89_FCC][79] = 22, + [0][0][2][0][RTW89_ETSI][79] = 127, + [0][0][2][0][RTW89_KCC][79] = 26, + [0][0][2][0][RTW89_FCC][81] = 22, + [0][0][2][0][RTW89_ETSI][81] = 127, + [0][0][2][0][RTW89_KCC][81] = 26, + [0][0][2][0][RTW89_FCC][83] = 22, + [0][0][2][0][RTW89_ETSI][83] = 127, + [0][0][2][0][RTW89_KCC][83] = 32, + [0][0][2][0][RTW89_FCC][85] = 22, + [0][0][2][0][RTW89_ETSI][85] = 127, + [0][0][2][0][RTW89_KCC][85] = 32, + [0][0][2][0][RTW89_FCC][87] = 22, + [0][0][2][0][RTW89_ETSI][87] = 127, + [0][0][2][0][RTW89_KCC][87] = 32, + [0][0][2][0][RTW89_FCC][89] = 22, + [0][0][2][0][RTW89_ETSI][89] = 127, + [0][0][2][0][RTW89_KCC][89] = 32, + [0][0][2][0][RTW89_FCC][90] = 22, + [0][0][2][0][RTW89_ETSI][90] = 127, + [0][0][2][0][RTW89_KCC][90] = 32, + [0][0][2][0][RTW89_FCC][92] = 22, + [0][0][2][0][RTW89_ETSI][92] = 127, + [0][0][2][0][RTW89_KCC][92] = 32, + [0][0][2][0][RTW89_FCC][94] = 22, + [0][0][2][0][RTW89_ETSI][94] = 127, + [0][0][2][0][RTW89_KCC][94] = 32, + [0][0][2][0][RTW89_FCC][96] = 22, + [0][0][2][0][RTW89_ETSI][96] = 127, + [0][0][2][0][RTW89_KCC][96] = 32, + [0][0][2][0][RTW89_FCC][98] = 22, + [0][0][2][0][RTW89_ETSI][98] = 127, + [0][0][2][0][RTW89_KCC][98] = 32, + [0][0][2][0][RTW89_FCC][100] = 22, + [0][0][2][0][RTW89_ETSI][100] = 127, + [0][0][2][0][RTW89_KCC][100] = 32, + [0][0][2][0][RTW89_FCC][102] = 22, + [0][0][2][0][RTW89_ETSI][102] = 127, + [0][0][2][0][RTW89_KCC][102] = 32, + [0][0][2][0][RTW89_FCC][104] = 22, + [0][0][2][0][RTW89_ETSI][104] = 127, + [0][0][2][0][RTW89_KCC][104] = 32, + [0][0][2][0][RTW89_FCC][105] = 22, + [0][0][2][0][RTW89_ETSI][105] = 127, + [0][0][2][0][RTW89_KCC][105] = 32, + [0][0][2][0][RTW89_FCC][107] = 24, + [0][0][2][0][RTW89_ETSI][107] = 127, + [0][0][2][0][RTW89_KCC][107] = 32, + [0][0][2][0][RTW89_FCC][109] = 24, + [0][0][2][0][RTW89_ETSI][109] = 127, + [0][0][2][0][RTW89_KCC][109] = 32, [0][0][2][0][RTW89_FCC][111] = 127, + [0][0][2][0][RTW89_ETSI][111] = 127, + [0][0][2][0][RTW89_KCC][111] = 127, [0][0][2][0][RTW89_FCC][113] = 127, + [0][0][2][0][RTW89_ETSI][113] = 127, + [0][0][2][0][RTW89_KCC][113] = 127, [0][0][2][0][RTW89_FCC][115] = 127, + [0][0][2][0][RTW89_ETSI][115] = 127, + [0][0][2][0][RTW89_KCC][115] = 127, [0][0][2][0][RTW89_FCC][117] = 127, + [0][0][2][0][RTW89_ETSI][117] = 127, + [0][0][2][0][RTW89_KCC][117] = 127, [0][0][2][0][RTW89_FCC][119] = 127, - [0][1][2][0][RTW89_FCC][0] = 60, - [0][1][2][0][RTW89_FCC][2] = 60, - [0][1][2][0][RTW89_FCC][4] = 60, - [0][1][2][0][RTW89_FCC][6] = 60, - [0][1][2][0][RTW89_FCC][8] = 60, - [0][1][2][0][RTW89_FCC][10] = 60, - [0][1][2][0][RTW89_FCC][12] = 60, - [0][1][2][0][RTW89_FCC][14] = 60, - [0][1][2][0][RTW89_FCC][15] = 60, - [0][1][2][0][RTW89_FCC][17] = 60, - [0][1][2][0][RTW89_FCC][19] = 60, - [0][1][2][0][RTW89_FCC][21] = 60, - [0][1][2][0][RTW89_FCC][23] = 60, - [0][1][2][0][RTW89_FCC][25] = 60, - [0][1][2][0][RTW89_FCC][27] = 60, - [0][1][2][0][RTW89_FCC][29] = 60, - [0][1][2][0][RTW89_FCC][30] = 60, - [0][1][2][0][RTW89_FCC][32] = 60, - [0][1][2][0][RTW89_FCC][34] = 60, - [0][1][2][0][RTW89_FCC][36] = 60, - [0][1][2][0][RTW89_FCC][38] = 60, - [0][1][2][0][RTW89_FCC][40] = 60, - [0][1][2][0][RTW89_FCC][42] = 60, - [0][1][2][0][RTW89_FCC][44] = 60, - [0][1][2][0][RTW89_FCC][45] = 60, - [0][1][2][0][RTW89_FCC][47] = 60, - [0][1][2][0][RTW89_FCC][49] = 60, - [0][1][2][0][RTW89_FCC][51] = 60, - [0][1][2][0][RTW89_FCC][53] = 60, - [0][1][2][0][RTW89_FCC][55] = 60, - [0][1][2][0][RTW89_FCC][57] = 60, - [0][1][2][0][RTW89_FCC][59] = 60, - [0][1][2][0][RTW89_FCC][60] = 60, - [0][1][2][0][RTW89_FCC][62] = 60, - [0][1][2][0][RTW89_FCC][64] = 60, - [0][1][2][0][RTW89_FCC][66] = 60, - [0][1][2][0][RTW89_FCC][68] = 60, - [0][1][2][0][RTW89_FCC][70] = 60, - [0][1][2][0][RTW89_FCC][72] = 60, - [0][1][2][0][RTW89_FCC][74] = 60, - [0][1][2][0][RTW89_FCC][75] = 60, - [0][1][2][0][RTW89_FCC][77] = 60, - [0][1][2][0][RTW89_FCC][79] = 60, - [0][1][2][0][RTW89_FCC][81] = 60, - [0][1][2][0][RTW89_FCC][83] = 60, - [0][1][2][0][RTW89_FCC][85] = 60, - [0][1][2][0][RTW89_FCC][87] = 60, - [0][1][2][0][RTW89_FCC][89] = 60, - [0][1][2][0][RTW89_FCC][90] = 60, - [0][1][2][0][RTW89_FCC][92] = 60, - [0][1][2][0][RTW89_FCC][94] = 60, - [0][1][2][0][RTW89_FCC][96] = 60, - [0][1][2][0][RTW89_FCC][98] = 60, - [0][1][2][0][RTW89_FCC][100] = 60, - [0][1][2][0][RTW89_FCC][102] = 60, - [0][1][2][0][RTW89_FCC][104] = 60, - [0][1][2][0][RTW89_FCC][105] = 60, - [0][1][2][0][RTW89_FCC][107] = 60, - [0][1][2][0][RTW89_FCC][109] = 60, + [0][0][2][0][RTW89_ETSI][119] = 127, + [0][0][2][0][RTW89_KCC][119] = 127, + [0][1][2][0][RTW89_FCC][0] = -2, + [0][1][2][0][RTW89_ETSI][0] = 54, + [0][1][2][0][RTW89_KCC][0] = 12, + [0][1][2][0][RTW89_FCC][2] = -4, + [0][1][2][0][RTW89_ETSI][2] = 54, + [0][1][2][0][RTW89_KCC][2] = 12, + [0][1][2][0][RTW89_FCC][4] = -4, + [0][1][2][0][RTW89_ETSI][4] = 54, + [0][1][2][0][RTW89_KCC][4] = 12, + [0][1][2][0][RTW89_FCC][6] = -4, + [0][1][2][0][RTW89_ETSI][6] = 54, + [0][1][2][0][RTW89_KCC][6] = 12, + [0][1][2][0][RTW89_FCC][8] = -4, + [0][1][2][0][RTW89_ETSI][8] = 54, + [0][1][2][0][RTW89_KCC][8] = 12, + [0][1][2][0][RTW89_FCC][10] = -4, + [0][1][2][0][RTW89_ETSI][10] = 54, + [0][1][2][0][RTW89_KCC][10] = 12, + [0][1][2][0][RTW89_FCC][12] = -4, + [0][1][2][0][RTW89_ETSI][12] = 54, + [0][1][2][0][RTW89_KCC][12] = 12, + [0][1][2][0][RTW89_FCC][14] = -4, + [0][1][2][0][RTW89_ETSI][14] = 54, + [0][1][2][0][RTW89_KCC][14] = 12, + [0][1][2][0][RTW89_FCC][15] = -4, + [0][1][2][0][RTW89_ETSI][15] = 54, + [0][1][2][0][RTW89_KCC][15] = 12, + [0][1][2][0][RTW89_FCC][17] = -4, + [0][1][2][0][RTW89_ETSI][17] = 54, + [0][1][2][0][RTW89_KCC][17] = 12, + [0][1][2][0][RTW89_FCC][19] = -4, + [0][1][2][0][RTW89_ETSI][19] = 54, + [0][1][2][0][RTW89_KCC][19] = 12, + [0][1][2][0][RTW89_FCC][21] = -4, + [0][1][2][0][RTW89_ETSI][21] = 54, + [0][1][2][0][RTW89_KCC][21] = 12, + [0][1][2][0][RTW89_FCC][23] = -4, + [0][1][2][0][RTW89_ETSI][23] = 54, + [0][1][2][0][RTW89_KCC][23] = 12, + [0][1][2][0][RTW89_FCC][25] = -4, + [0][1][2][0][RTW89_ETSI][25] = 54, + [0][1][2][0][RTW89_KCC][25] = 12, + [0][1][2][0][RTW89_FCC][27] = -4, + [0][1][2][0][RTW89_ETSI][27] = 54, + [0][1][2][0][RTW89_KCC][27] = 12, + [0][1][2][0][RTW89_FCC][29] = -4, + [0][1][2][0][RTW89_ETSI][29] = 54, + [0][1][2][0][RTW89_KCC][29] = 12, + [0][1][2][0][RTW89_FCC][30] = -4, + [0][1][2][0][RTW89_ETSI][30] = 54, + [0][1][2][0][RTW89_KCC][30] = 12, + [0][1][2][0][RTW89_FCC][32] = -4, + [0][1][2][0][RTW89_ETSI][32] = 54, + [0][1][2][0][RTW89_KCC][32] = 12, + [0][1][2][0][RTW89_FCC][34] = -4, + [0][1][2][0][RTW89_ETSI][34] = 54, + [0][1][2][0][RTW89_KCC][34] = 12, + [0][1][2][0][RTW89_FCC][36] = -4, + [0][1][2][0][RTW89_ETSI][36] = 54, + [0][1][2][0][RTW89_KCC][36] = 12, + [0][1][2][0][RTW89_FCC][38] = -4, + [0][1][2][0][RTW89_ETSI][38] = 54, + [0][1][2][0][RTW89_KCC][38] = 12, + [0][1][2][0][RTW89_FCC][40] = -4, + [0][1][2][0][RTW89_ETSI][40] = 54, + [0][1][2][0][RTW89_KCC][40] = 12, + [0][1][2][0][RTW89_FCC][42] = -4, + [0][1][2][0][RTW89_ETSI][42] = 54, + [0][1][2][0][RTW89_KCC][42] = 12, + [0][1][2][0][RTW89_FCC][44] = -2, + [0][1][2][0][RTW89_ETSI][44] = 54, + [0][1][2][0][RTW89_KCC][44] = 12, + [0][1][2][0][RTW89_FCC][45] = -2, + [0][1][2][0][RTW89_ETSI][45] = 127, + [0][1][2][0][RTW89_KCC][45] = 12, + [0][1][2][0][RTW89_FCC][47] = -2, + [0][1][2][0][RTW89_ETSI][47] = 127, + [0][1][2][0][RTW89_KCC][47] = 12, + [0][1][2][0][RTW89_FCC][49] = -2, + [0][1][2][0][RTW89_ETSI][49] = 127, + [0][1][2][0][RTW89_KCC][49] = 12, + [0][1][2][0][RTW89_FCC][51] = -2, + [0][1][2][0][RTW89_ETSI][51] = 127, + [0][1][2][0][RTW89_KCC][51] = 12, + [0][1][2][0][RTW89_FCC][53] = -2, + [0][1][2][0][RTW89_ETSI][53] = 127, + [0][1][2][0][RTW89_KCC][53] = 12, + [0][1][2][0][RTW89_FCC][55] = -2, + [0][1][2][0][RTW89_ETSI][55] = 127, + [0][1][2][0][RTW89_KCC][55] = 12, + [0][1][2][0][RTW89_FCC][57] = -2, + [0][1][2][0][RTW89_ETSI][57] = 127, + [0][1][2][0][RTW89_KCC][57] = 12, + [0][1][2][0][RTW89_FCC][59] = -2, + [0][1][2][0][RTW89_ETSI][59] = 127, + [0][1][2][0][RTW89_KCC][59] = 12, + [0][1][2][0][RTW89_FCC][60] = -2, + [0][1][2][0][RTW89_ETSI][60] = 127, + [0][1][2][0][RTW89_KCC][60] = 12, + [0][1][2][0][RTW89_FCC][62] = -2, + [0][1][2][0][RTW89_ETSI][62] = 127, + [0][1][2][0][RTW89_KCC][62] = 12, + [0][1][2][0][RTW89_FCC][64] = -2, + [0][1][2][0][RTW89_ETSI][64] = 127, + [0][1][2][0][RTW89_KCC][64] = 12, + [0][1][2][0][RTW89_FCC][66] = -2, + [0][1][2][0][RTW89_ETSI][66] = 127, + [0][1][2][0][RTW89_KCC][66] = 12, + [0][1][2][0][RTW89_FCC][68] = -2, + [0][1][2][0][RTW89_ETSI][68] = 127, + [0][1][2][0][RTW89_KCC][68] = 12, + [0][1][2][0][RTW89_FCC][70] = -2, + [0][1][2][0][RTW89_ETSI][70] = 127, + [0][1][2][0][RTW89_KCC][70] = 12, + [0][1][2][0][RTW89_FCC][72] = -2, + [0][1][2][0][RTW89_ETSI][72] = 127, + [0][1][2][0][RTW89_KCC][72] = 12, + [0][1][2][0][RTW89_FCC][74] = -2, + [0][1][2][0][RTW89_ETSI][74] = 127, + [0][1][2][0][RTW89_KCC][74] = 12, + [0][1][2][0][RTW89_FCC][75] = -2, + [0][1][2][0][RTW89_ETSI][75] = 127, + [0][1][2][0][RTW89_KCC][75] = 12, + [0][1][2][0][RTW89_FCC][77] = -2, + [0][1][2][0][RTW89_ETSI][77] = 127, + [0][1][2][0][RTW89_KCC][77] = 12, + [0][1][2][0][RTW89_FCC][79] = -2, + [0][1][2][0][RTW89_ETSI][79] = 127, + [0][1][2][0][RTW89_KCC][79] = 12, + [0][1][2][0][RTW89_FCC][81] = -2, + [0][1][2][0][RTW89_ETSI][81] = 127, + [0][1][2][0][RTW89_KCC][81] = 12, + [0][1][2][0][RTW89_FCC][83] = -2, + [0][1][2][0][RTW89_ETSI][83] = 127, + [0][1][2][0][RTW89_KCC][83] = 20, + [0][1][2][0][RTW89_FCC][85] = -2, + [0][1][2][0][RTW89_ETSI][85] = 127, + [0][1][2][0][RTW89_KCC][85] = 20, + [0][1][2][0][RTW89_FCC][87] = -2, + [0][1][2][0][RTW89_ETSI][87] = 127, + [0][1][2][0][RTW89_KCC][87] = 20, + [0][1][2][0][RTW89_FCC][89] = -2, + [0][1][2][0][RTW89_ETSI][89] = 127, + [0][1][2][0][RTW89_KCC][89] = 20, + [0][1][2][0][RTW89_FCC][90] = -2, + [0][1][2][0][RTW89_ETSI][90] = 127, + [0][1][2][0][RTW89_KCC][90] = 20, + [0][1][2][0][RTW89_FCC][92] = -2, + [0][1][2][0][RTW89_ETSI][92] = 127, + [0][1][2][0][RTW89_KCC][92] = 20, + [0][1][2][0][RTW89_FCC][94] = -2, + [0][1][2][0][RTW89_ETSI][94] = 127, + [0][1][2][0][RTW89_KCC][94] = 20, + [0][1][2][0][RTW89_FCC][96] = -2, + [0][1][2][0][RTW89_ETSI][96] = 127, + [0][1][2][0][RTW89_KCC][96] = 20, + [0][1][2][0][RTW89_FCC][98] = -2, + [0][1][2][0][RTW89_ETSI][98] = 127, + [0][1][2][0][RTW89_KCC][98] = 20, + [0][1][2][0][RTW89_FCC][100] = -2, + [0][1][2][0][RTW89_ETSI][100] = 127, + [0][1][2][0][RTW89_KCC][100] = 20, + [0][1][2][0][RTW89_FCC][102] = -2, + [0][1][2][0][RTW89_ETSI][102] = 127, + [0][1][2][0][RTW89_KCC][102] = 20, + [0][1][2][0][RTW89_FCC][104] = -2, + [0][1][2][0][RTW89_ETSI][104] = 127, + [0][1][2][0][RTW89_KCC][104] = 20, + [0][1][2][0][RTW89_FCC][105] = -2, + [0][1][2][0][RTW89_ETSI][105] = 127, + [0][1][2][0][RTW89_KCC][105] = 20, + [0][1][2][0][RTW89_FCC][107] = 0, + [0][1][2][0][RTW89_ETSI][107] = 127, + [0][1][2][0][RTW89_KCC][107] = 20, + [0][1][2][0][RTW89_FCC][109] = 0, + [0][1][2][0][RTW89_ETSI][109] = 127, + [0][1][2][0][RTW89_KCC][109] = 20, [0][1][2][0][RTW89_FCC][111] = 127, + [0][1][2][0][RTW89_ETSI][111] = 127, + [0][1][2][0][RTW89_KCC][111] = 127, [0][1][2][0][RTW89_FCC][113] = 127, + [0][1][2][0][RTW89_ETSI][113] = 127, + [0][1][2][0][RTW89_KCC][113] = 127, [0][1][2][0][RTW89_FCC][115] = 127, + [0][1][2][0][RTW89_ETSI][115] = 127, + [0][1][2][0][RTW89_KCC][115] = 127, [0][1][2][0][RTW89_FCC][117] = 127, + [0][1][2][0][RTW89_ETSI][117] = 127, + [0][1][2][0][RTW89_KCC][117] = 127, [0][1][2][0][RTW89_FCC][119] = 127, - [0][1][2][1][RTW89_FCC][0] = 48, - [0][1][2][1][RTW89_FCC][2] = 48, - [0][1][2][1][RTW89_FCC][4] = 48, - [0][1][2][1][RTW89_FCC][6] = 48, - [0][1][2][1][RTW89_FCC][8] = 48, - [0][1][2][1][RTW89_FCC][10] = 48, - [0][1][2][1][RTW89_FCC][12] = 48, - [0][1][2][1][RTW89_FCC][14] = 48, - [0][1][2][1][RTW89_FCC][15] = 48, - [0][1][2][1][RTW89_FCC][17] = 48, - [0][1][2][1][RTW89_FCC][19] = 48, - [0][1][2][1][RTW89_FCC][21] = 48, - [0][1][2][1][RTW89_FCC][23] = 48, - [0][1][2][1][RTW89_FCC][25] = 48, - [0][1][2][1][RTW89_FCC][27] = 48, - [0][1][2][1][RTW89_FCC][29] = 48, - [0][1][2][1][RTW89_FCC][30] = 48, - [0][1][2][1][RTW89_FCC][32] = 48, - [0][1][2][1][RTW89_FCC][34] = 48, - [0][1][2][1][RTW89_FCC][36] = 48, - [0][1][2][1][RTW89_FCC][38] = 48, - [0][1][2][1][RTW89_FCC][40] = 48, - [0][1][2][1][RTW89_FCC][42] = 48, - [0][1][2][1][RTW89_FCC][44] = 48, - [0][1][2][1][RTW89_FCC][45] = 48, - [0][1][2][1][RTW89_FCC][47] = 48, - [0][1][2][1][RTW89_FCC][49] = 48, - [0][1][2][1][RTW89_FCC][51] = 48, - [0][1][2][1][RTW89_FCC][53] = 48, - [0][1][2][1][RTW89_FCC][55] = 48, - [0][1][2][1][RTW89_FCC][57] = 48, - [0][1][2][1][RTW89_FCC][59] = 48, - [0][1][2][1][RTW89_FCC][60] = 48, - [0][1][2][1][RTW89_FCC][62] = 48, - [0][1][2][1][RTW89_FCC][64] = 48, - [0][1][2][1][RTW89_FCC][66] = 48, - [0][1][2][1][RTW89_FCC][68] = 48, - [0][1][2][1][RTW89_FCC][70] = 48, - [0][1][2][1][RTW89_FCC][72] = 48, - [0][1][2][1][RTW89_FCC][74] = 48, - [0][1][2][1][RTW89_FCC][75] = 48, - [0][1][2][1][RTW89_FCC][77] = 48, - [0][1][2][1][RTW89_FCC][79] = 48, - [0][1][2][1][RTW89_FCC][81] = 48, - [0][1][2][1][RTW89_FCC][83] = 48, - [0][1][2][1][RTW89_FCC][85] = 48, - [0][1][2][1][RTW89_FCC][87] = 48, - [0][1][2][1][RTW89_FCC][89] = 48, - [0][1][2][1][RTW89_FCC][90] = 48, - [0][1][2][1][RTW89_FCC][92] = 48, - [0][1][2][1][RTW89_FCC][94] = 48, - [0][1][2][1][RTW89_FCC][96] = 48, - [0][1][2][1][RTW89_FCC][98] = 48, - [0][1][2][1][RTW89_FCC][100] = 48, - [0][1][2][1][RTW89_FCC][102] = 48, - [0][1][2][1][RTW89_FCC][104] = 48, - [0][1][2][1][RTW89_FCC][105] = 48, - [0][1][2][1][RTW89_FCC][107] = 48, - [0][1][2][1][RTW89_FCC][109] = 48, + [0][1][2][0][RTW89_ETSI][119] = 127, + [0][1][2][0][RTW89_KCC][119] = 127, + [0][1][2][1][RTW89_FCC][0] = -2, + [0][1][2][1][RTW89_ETSI][0] = 42, + [0][1][2][1][RTW89_KCC][0] = 12, + [0][1][2][1][RTW89_FCC][2] = -4, + [0][1][2][1][RTW89_ETSI][2] = 42, + [0][1][2][1][RTW89_KCC][2] = 12, + [0][1][2][1][RTW89_FCC][4] = -4, + [0][1][2][1][RTW89_ETSI][4] = 42, + [0][1][2][1][RTW89_KCC][4] = 12, + [0][1][2][1][RTW89_FCC][6] = -4, + [0][1][2][1][RTW89_ETSI][6] = 42, + [0][1][2][1][RTW89_KCC][6] = 12, + [0][1][2][1][RTW89_FCC][8] = -4, + [0][1][2][1][RTW89_ETSI][8] = 42, + [0][1][2][1][RTW89_KCC][8] = 12, + [0][1][2][1][RTW89_FCC][10] = -4, + [0][1][2][1][RTW89_ETSI][10] = 42, + [0][1][2][1][RTW89_KCC][10] = 12, + [0][1][2][1][RTW89_FCC][12] = -4, + [0][1][2][1][RTW89_ETSI][12] = 42, + [0][1][2][1][RTW89_KCC][12] = 12, + [0][1][2][1][RTW89_FCC][14] = -4, + [0][1][2][1][RTW89_ETSI][14] = 42, + [0][1][2][1][RTW89_KCC][14] = 12, + [0][1][2][1][RTW89_FCC][15] = -4, + [0][1][2][1][RTW89_ETSI][15] = 42, + [0][1][2][1][RTW89_KCC][15] = 12, + [0][1][2][1][RTW89_FCC][17] = -4, + [0][1][2][1][RTW89_ETSI][17] = 42, + [0][1][2][1][RTW89_KCC][17] = 12, + [0][1][2][1][RTW89_FCC][19] = -4, + [0][1][2][1][RTW89_ETSI][19] = 42, + [0][1][2][1][RTW89_KCC][19] = 12, + [0][1][2][1][RTW89_FCC][21] = -4, + [0][1][2][1][RTW89_ETSI][21] = 42, + [0][1][2][1][RTW89_KCC][21] = 12, + [0][1][2][1][RTW89_FCC][23] = -4, + [0][1][2][1][RTW89_ETSI][23] = 42, + [0][1][2][1][RTW89_KCC][23] = 12, + [0][1][2][1][RTW89_FCC][25] = -4, + [0][1][2][1][RTW89_ETSI][25] = 42, + [0][1][2][1][RTW89_KCC][25] = 12, + [0][1][2][1][RTW89_FCC][27] = -4, + [0][1][2][1][RTW89_ETSI][27] = 42, + [0][1][2][1][RTW89_KCC][27] = 12, + [0][1][2][1][RTW89_FCC][29] = -4, + [0][1][2][1][RTW89_ETSI][29] = 42, + [0][1][2][1][RTW89_KCC][29] = 12, + [0][1][2][1][RTW89_FCC][30] = -4, + [0][1][2][1][RTW89_ETSI][30] = 42, + [0][1][2][1][RTW89_KCC][30] = 12, + [0][1][2][1][RTW89_FCC][32] = -4, + [0][1][2][1][RTW89_ETSI][32] = 42, + [0][1][2][1][RTW89_KCC][32] = 12, + [0][1][2][1][RTW89_FCC][34] = -4, + [0][1][2][1][RTW89_ETSI][34] = 42, + [0][1][2][1][RTW89_KCC][34] = 12, + [0][1][2][1][RTW89_FCC][36] = -4, + [0][1][2][1][RTW89_ETSI][36] = 42, + [0][1][2][1][RTW89_KCC][36] = 12, + [0][1][2][1][RTW89_FCC][38] = -4, + [0][1][2][1][RTW89_ETSI][38] = 42, + [0][1][2][1][RTW89_KCC][38] = 12, + [0][1][2][1][RTW89_FCC][40] = -4, + [0][1][2][1][RTW89_ETSI][40] = 42, + [0][1][2][1][RTW89_KCC][40] = 12, + [0][1][2][1][RTW89_FCC][42] = -4, + [0][1][2][1][RTW89_ETSI][42] = 42, + [0][1][2][1][RTW89_KCC][42] = 12, + [0][1][2][1][RTW89_FCC][44] = -2, + [0][1][2][1][RTW89_ETSI][44] = 42, + [0][1][2][1][RTW89_KCC][44] = 12, + [0][1][2][1][RTW89_FCC][45] = -2, + [0][1][2][1][RTW89_ETSI][45] = 127, + [0][1][2][1][RTW89_KCC][45] = 12, + [0][1][2][1][RTW89_FCC][47] = -2, + [0][1][2][1][RTW89_ETSI][47] = 127, + [0][1][2][1][RTW89_KCC][47] = 12, + [0][1][2][1][RTW89_FCC][49] = -2, + [0][1][2][1][RTW89_ETSI][49] = 127, + [0][1][2][1][RTW89_KCC][49] = 12, + [0][1][2][1][RTW89_FCC][51] = -2, + [0][1][2][1][RTW89_ETSI][51] = 127, + [0][1][2][1][RTW89_KCC][51] = 12, + [0][1][2][1][RTW89_FCC][53] = -2, + [0][1][2][1][RTW89_ETSI][53] = 127, + [0][1][2][1][RTW89_KCC][53] = 12, + [0][1][2][1][RTW89_FCC][55] = -2, + [0][1][2][1][RTW89_ETSI][55] = 127, + [0][1][2][1][RTW89_KCC][55] = 12, + [0][1][2][1][RTW89_FCC][57] = -2, + [0][1][2][1][RTW89_ETSI][57] = 127, + [0][1][2][1][RTW89_KCC][57] = 12, + [0][1][2][1][RTW89_FCC][59] = -2, + [0][1][2][1][RTW89_ETSI][59] = 127, + [0][1][2][1][RTW89_KCC][59] = 12, + [0][1][2][1][RTW89_FCC][60] = -2, + [0][1][2][1][RTW89_ETSI][60] = 127, + [0][1][2][1][RTW89_KCC][60] = 12, + [0][1][2][1][RTW89_FCC][62] = -2, + [0][1][2][1][RTW89_ETSI][62] = 127, + [0][1][2][1][RTW89_KCC][62] = 12, + [0][1][2][1][RTW89_FCC][64] = -2, + [0][1][2][1][RTW89_ETSI][64] = 127, + [0][1][2][1][RTW89_KCC][64] = 12, + [0][1][2][1][RTW89_FCC][66] = -2, + [0][1][2][1][RTW89_ETSI][66] = 127, + [0][1][2][1][RTW89_KCC][66] = 12, + [0][1][2][1][RTW89_FCC][68] = -2, + [0][1][2][1][RTW89_ETSI][68] = 127, + [0][1][2][1][RTW89_KCC][68] = 12, + [0][1][2][1][RTW89_FCC][70] = -2, + [0][1][2][1][RTW89_ETSI][70] = 127, + [0][1][2][1][RTW89_KCC][70] = 12, + [0][1][2][1][RTW89_FCC][72] = -2, + [0][1][2][1][RTW89_ETSI][72] = 127, + [0][1][2][1][RTW89_KCC][72] = 12, + [0][1][2][1][RTW89_FCC][74] = -2, + [0][1][2][1][RTW89_ETSI][74] = 127, + [0][1][2][1][RTW89_KCC][74] = 12, + [0][1][2][1][RTW89_FCC][75] = -2, + [0][1][2][1][RTW89_ETSI][75] = 127, + [0][1][2][1][RTW89_KCC][75] = 12, + [0][1][2][1][RTW89_FCC][77] = -2, + [0][1][2][1][RTW89_ETSI][77] = 127, + [0][1][2][1][RTW89_KCC][77] = 12, + [0][1][2][1][RTW89_FCC][79] = -2, + [0][1][2][1][RTW89_ETSI][79] = 127, + [0][1][2][1][RTW89_KCC][79] = 12, + [0][1][2][1][RTW89_FCC][81] = -2, + [0][1][2][1][RTW89_ETSI][81] = 127, + [0][1][2][1][RTW89_KCC][81] = 12, + [0][1][2][1][RTW89_FCC][83] = -2, + [0][1][2][1][RTW89_ETSI][83] = 127, + [0][1][2][1][RTW89_KCC][83] = 20, + [0][1][2][1][RTW89_FCC][85] = -2, + [0][1][2][1][RTW89_ETSI][85] = 127, + [0][1][2][1][RTW89_KCC][85] = 20, + [0][1][2][1][RTW89_FCC][87] = -2, + [0][1][2][1][RTW89_ETSI][87] = 127, + [0][1][2][1][RTW89_KCC][87] = 20, + [0][1][2][1][RTW89_FCC][89] = -2, + [0][1][2][1][RTW89_ETSI][89] = 127, + [0][1][2][1][RTW89_KCC][89] = 20, + [0][1][2][1][RTW89_FCC][90] = -2, + [0][1][2][1][RTW89_ETSI][90] = 127, + [0][1][2][1][RTW89_KCC][90] = 20, + [0][1][2][1][RTW89_FCC][92] = -2, + [0][1][2][1][RTW89_ETSI][92] = 127, + [0][1][2][1][RTW89_KCC][92] = 20, + [0][1][2][1][RTW89_FCC][94] = -2, + [0][1][2][1][RTW89_ETSI][94] = 127, + [0][1][2][1][RTW89_KCC][94] = 20, + [0][1][2][1][RTW89_FCC][96] = -2, + [0][1][2][1][RTW89_ETSI][96] = 127, + [0][1][2][1][RTW89_KCC][96] = 20, + [0][1][2][1][RTW89_FCC][98] = -2, + [0][1][2][1][RTW89_ETSI][98] = 127, + [0][1][2][1][RTW89_KCC][98] = 20, + [0][1][2][1][RTW89_FCC][100] = -2, + [0][1][2][1][RTW89_ETSI][100] = 127, + [0][1][2][1][RTW89_KCC][100] = 20, + [0][1][2][1][RTW89_FCC][102] = -2, + [0][1][2][1][RTW89_ETSI][102] = 127, + [0][1][2][1][RTW89_KCC][102] = 20, + [0][1][2][1][RTW89_FCC][104] = -2, + [0][1][2][1][RTW89_ETSI][104] = 127, + [0][1][2][1][RTW89_KCC][104] = 20, + [0][1][2][1][RTW89_FCC][105] = -2, + [0][1][2][1][RTW89_ETSI][105] = 127, + [0][1][2][1][RTW89_KCC][105] = 20, + [0][1][2][1][RTW89_FCC][107] = 0, + [0][1][2][1][RTW89_ETSI][107] = 127, + [0][1][2][1][RTW89_KCC][107] = 20, + [0][1][2][1][RTW89_FCC][109] = 0, + [0][1][2][1][RTW89_ETSI][109] = 127, + [0][1][2][1][RTW89_KCC][109] = 20, [0][1][2][1][RTW89_FCC][111] = 127, + [0][1][2][1][RTW89_ETSI][111] = 127, + [0][1][2][1][RTW89_KCC][111] = 127, [0][1][2][1][RTW89_FCC][113] = 127, + [0][1][2][1][RTW89_ETSI][113] = 127, + [0][1][2][1][RTW89_KCC][113] = 127, [0][1][2][1][RTW89_FCC][115] = 127, + [0][1][2][1][RTW89_ETSI][115] = 127, + [0][1][2][1][RTW89_KCC][115] = 127, [0][1][2][1][RTW89_FCC][117] = 127, + [0][1][2][1][RTW89_ETSI][117] = 127, + [0][1][2][1][RTW89_KCC][117] = 127, [0][1][2][1][RTW89_FCC][119] = 127, - [1][0][2][0][RTW89_FCC][1] = 72, - [1][0][2][0][RTW89_FCC][5] = 72, - [1][0][2][0][RTW89_FCC][9] = 72, - [1][0][2][0][RTW89_FCC][13] = 72, - [1][0][2][0][RTW89_FCC][16] = 72, - [1][0][2][0][RTW89_FCC][20] = 72, - [1][0][2][0][RTW89_FCC][24] = 72, - [1][0][2][0][RTW89_FCC][28] = 72, - [1][0][2][0][RTW89_FCC][31] = 72, - [1][0][2][0][RTW89_FCC][35] = 72, - [1][0][2][0][RTW89_FCC][39] = 72, - [1][0][2][0][RTW89_FCC][43] = 72, - [1][0][2][0][RTW89_FCC][46] = 72, - [1][0][2][0][RTW89_FCC][50] = 72, - [1][0][2][0][RTW89_FCC][54] = 72, - [1][0][2][0][RTW89_FCC][58] = 72, - [1][0][2][0][RTW89_FCC][61] = 72, - [1][0][2][0][RTW89_FCC][65] = 72, - [1][0][2][0][RTW89_FCC][69] = 72, - [1][0][2][0][RTW89_FCC][73] = 72, - [1][0][2][0][RTW89_FCC][76] = 72, - [1][0][2][0][RTW89_FCC][80] = 72, - [1][0][2][0][RTW89_FCC][84] = 72, - [1][0][2][0][RTW89_FCC][88] = 72, - [1][0][2][0][RTW89_FCC][91] = 72, - [1][0][2][0][RTW89_FCC][95] = 72, - [1][0][2][0][RTW89_FCC][99] = 72, - [1][0][2][0][RTW89_FCC][103] = 72, - [1][0][2][0][RTW89_FCC][106] = 72, + [0][1][2][1][RTW89_ETSI][119] = 127, + [0][1][2][1][RTW89_KCC][119] = 127, + [1][0][2][0][RTW89_FCC][1] = 34, + [1][0][2][0][RTW89_ETSI][1] = 66, + [1][0][2][0][RTW89_KCC][1] = 40, + [1][0][2][0][RTW89_FCC][5] = 34, + [1][0][2][0][RTW89_ETSI][5] = 66, + [1][0][2][0][RTW89_KCC][5] = 40, + [1][0][2][0][RTW89_FCC][9] = 34, + [1][0][2][0][RTW89_ETSI][9] = 66, + [1][0][2][0][RTW89_KCC][9] = 40, + [1][0][2][0][RTW89_FCC][13] = 34, + [1][0][2][0][RTW89_ETSI][13] = 66, + [1][0][2][0][RTW89_KCC][13] = 40, + [1][0][2][0][RTW89_FCC][16] = 34, + [1][0][2][0][RTW89_ETSI][16] = 66, + [1][0][2][0][RTW89_KCC][16] = 40, + [1][0][2][0][RTW89_FCC][20] = 34, + [1][0][2][0][RTW89_ETSI][20] = 66, + [1][0][2][0][RTW89_KCC][20] = 40, + [1][0][2][0][RTW89_FCC][24] = 36, + [1][0][2][0][RTW89_ETSI][24] = 66, + [1][0][2][0][RTW89_KCC][24] = 40, + [1][0][2][0][RTW89_FCC][28] = 34, + [1][0][2][0][RTW89_ETSI][28] = 66, + [1][0][2][0][RTW89_KCC][28] = 40, + [1][0][2][0][RTW89_FCC][31] = 34, + [1][0][2][0][RTW89_ETSI][31] = 66, + [1][0][2][0][RTW89_KCC][31] = 40, + [1][0][2][0][RTW89_FCC][35] = 34, + [1][0][2][0][RTW89_ETSI][35] = 66, + [1][0][2][0][RTW89_KCC][35] = 40, + [1][0][2][0][RTW89_FCC][39] = 34, + [1][0][2][0][RTW89_ETSI][39] = 66, + [1][0][2][0][RTW89_KCC][39] = 40, + [1][0][2][0][RTW89_FCC][43] = 34, + [1][0][2][0][RTW89_ETSI][43] = 66, + [1][0][2][0][RTW89_KCC][43] = 40, + [1][0][2][0][RTW89_FCC][46] = 34, + [1][0][2][0][RTW89_ETSI][46] = 127, + [1][0][2][0][RTW89_KCC][46] = 40, + [1][0][2][0][RTW89_FCC][50] = 34, + [1][0][2][0][RTW89_ETSI][50] = 127, + [1][0][2][0][RTW89_KCC][50] = 40, + [1][0][2][0][RTW89_FCC][54] = 36, + [1][0][2][0][RTW89_ETSI][54] = 127, + [1][0][2][0][RTW89_KCC][54] = 40, + [1][0][2][0][RTW89_FCC][58] = 36, + [1][0][2][0][RTW89_ETSI][58] = 127, + [1][0][2][0][RTW89_KCC][58] = 40, + [1][0][2][0][RTW89_FCC][61] = 34, + [1][0][2][0][RTW89_ETSI][61] = 127, + [1][0][2][0][RTW89_KCC][61] = 40, + [1][0][2][0][RTW89_FCC][65] = 34, + [1][0][2][0][RTW89_ETSI][65] = 127, + [1][0][2][0][RTW89_KCC][65] = 40, + [1][0][2][0][RTW89_FCC][69] = 34, + [1][0][2][0][RTW89_ETSI][69] = 127, + [1][0][2][0][RTW89_KCC][69] = 40, + [1][0][2][0][RTW89_FCC][73] = 34, + [1][0][2][0][RTW89_ETSI][73] = 127, + [1][0][2][0][RTW89_KCC][73] = 40, + [1][0][2][0][RTW89_FCC][76] = 34, + [1][0][2][0][RTW89_ETSI][76] = 127, + [1][0][2][0][RTW89_KCC][76] = 40, + [1][0][2][0][RTW89_FCC][80] = 34, + [1][0][2][0][RTW89_ETSI][80] = 127, + [1][0][2][0][RTW89_KCC][80] = 42, + [1][0][2][0][RTW89_FCC][84] = 34, + [1][0][2][0][RTW89_ETSI][84] = 127, + [1][0][2][0][RTW89_KCC][84] = 42, + [1][0][2][0][RTW89_FCC][88] = 34, + [1][0][2][0][RTW89_ETSI][88] = 127, + [1][0][2][0][RTW89_KCC][88] = 42, + [1][0][2][0][RTW89_FCC][91] = 36, + [1][0][2][0][RTW89_ETSI][91] = 127, + [1][0][2][0][RTW89_KCC][91] = 42, + [1][0][2][0][RTW89_FCC][95] = 34, + [1][0][2][0][RTW89_ETSI][95] = 127, + [1][0][2][0][RTW89_KCC][95] = 42, + [1][0][2][0][RTW89_FCC][99] = 34, + [1][0][2][0][RTW89_ETSI][99] = 127, + [1][0][2][0][RTW89_KCC][99] = 42, + [1][0][2][0][RTW89_FCC][103] = 34, + [1][0][2][0][RTW89_ETSI][103] = 127, + [1][0][2][0][RTW89_KCC][103] = 42, + [1][0][2][0][RTW89_FCC][106] = 36, + [1][0][2][0][RTW89_ETSI][106] = 127, + [1][0][2][0][RTW89_KCC][106] = 42, [1][0][2][0][RTW89_FCC][110] = 127, + [1][0][2][0][RTW89_ETSI][110] = 127, + [1][0][2][0][RTW89_KCC][110] = 127, [1][0][2][0][RTW89_FCC][114] = 127, + [1][0][2][0][RTW89_ETSI][114] = 127, + [1][0][2][0][RTW89_KCC][114] = 127, [1][0][2][0][RTW89_FCC][118] = 127, - [1][1][2][0][RTW89_FCC][1] = 60, - [1][1][2][0][RTW89_FCC][5] = 60, - [1][1][2][0][RTW89_FCC][9] = 60, - [1][1][2][0][RTW89_FCC][13] = 60, - [1][1][2][0][RTW89_FCC][16] = 60, - [1][1][2][0][RTW89_FCC][20] = 60, - [1][1][2][0][RTW89_FCC][24] = 60, - [1][1][2][0][RTW89_FCC][28] = 60, - [1][1][2][0][RTW89_FCC][31] = 60, - [1][1][2][0][RTW89_FCC][35] = 60, - [1][1][2][0][RTW89_FCC][39] = 60, - [1][1][2][0][RTW89_FCC][43] = 60, - [1][1][2][0][RTW89_FCC][46] = 60, - [1][1][2][0][RTW89_FCC][50] = 60, - [1][1][2][0][RTW89_FCC][54] = 60, - [1][1][2][0][RTW89_FCC][58] = 60, - [1][1][2][0][RTW89_FCC][61] = 60, - [1][1][2][0][RTW89_FCC][65] = 60, - [1][1][2][0][RTW89_FCC][69] = 60, - [1][1][2][0][RTW89_FCC][73] = 60, - [1][1][2][0][RTW89_FCC][76] = 60, - [1][1][2][0][RTW89_FCC][80] = 60, - [1][1][2][0][RTW89_FCC][84] = 60, - [1][1][2][0][RTW89_FCC][88] = 60, - [1][1][2][0][RTW89_FCC][91] = 60, - [1][1][2][0][RTW89_FCC][95] = 60, - [1][1][2][0][RTW89_FCC][99] = 60, - [1][1][2][0][RTW89_FCC][103] = 60, - [1][1][2][0][RTW89_FCC][106] = 60, + [1][0][2][0][RTW89_ETSI][118] = 127, + [1][0][2][0][RTW89_KCC][118] = 127, + [1][1][2][0][RTW89_FCC][1] = 10, + [1][1][2][0][RTW89_ETSI][1] = 54, + [1][1][2][0][RTW89_KCC][1] = 28, + [1][1][2][0][RTW89_FCC][5] = 10, + [1][1][2][0][RTW89_ETSI][5] = 54, + [1][1][2][0][RTW89_KCC][5] = 28, + [1][1][2][0][RTW89_FCC][9] = 10, + [1][1][2][0][RTW89_ETSI][9] = 54, + [1][1][2][0][RTW89_KCC][9] = 28, + [1][1][2][0][RTW89_FCC][13] = 10, + [1][1][2][0][RTW89_ETSI][13] = 54, + [1][1][2][0][RTW89_KCC][13] = 28, + [1][1][2][0][RTW89_FCC][16] = 10, + [1][1][2][0][RTW89_ETSI][16] = 54, + [1][1][2][0][RTW89_KCC][16] = 28, + [1][1][2][0][RTW89_FCC][20] = 10, + [1][1][2][0][RTW89_ETSI][20] = 54, + [1][1][2][0][RTW89_KCC][20] = 28, + [1][1][2][0][RTW89_FCC][24] = 10, + [1][1][2][0][RTW89_ETSI][24] = 54, + [1][1][2][0][RTW89_KCC][24] = 28, + [1][1][2][0][RTW89_FCC][28] = 10, + [1][1][2][0][RTW89_ETSI][28] = 54, + [1][1][2][0][RTW89_KCC][28] = 28, + [1][1][2][0][RTW89_FCC][31] = 10, + [1][1][2][0][RTW89_ETSI][31] = 54, + [1][1][2][0][RTW89_KCC][31] = 28, + [1][1][2][0][RTW89_FCC][35] = 10, + [1][1][2][0][RTW89_ETSI][35] = 54, + [1][1][2][0][RTW89_KCC][35] = 28, + [1][1][2][0][RTW89_FCC][39] = 10, + [1][1][2][0][RTW89_ETSI][39] = 54, + [1][1][2][0][RTW89_KCC][39] = 28, + [1][1][2][0][RTW89_FCC][43] = 10, + [1][1][2][0][RTW89_ETSI][43] = 54, + [1][1][2][0][RTW89_KCC][43] = 28, + [1][1][2][0][RTW89_FCC][46] = 12, + [1][1][2][0][RTW89_ETSI][46] = 127, + [1][1][2][0][RTW89_KCC][46] = 28, + [1][1][2][0][RTW89_FCC][50] = 12, + [1][1][2][0][RTW89_ETSI][50] = 127, + [1][1][2][0][RTW89_KCC][50] = 28, + [1][1][2][0][RTW89_FCC][54] = 10, + [1][1][2][0][RTW89_ETSI][54] = 127, + [1][1][2][0][RTW89_KCC][54] = 28, + [1][1][2][0][RTW89_FCC][58] = 10, + [1][1][2][0][RTW89_ETSI][58] = 127, + [1][1][2][0][RTW89_KCC][58] = 28, + [1][1][2][0][RTW89_FCC][61] = 10, + [1][1][2][0][RTW89_ETSI][61] = 127, + [1][1][2][0][RTW89_KCC][61] = 28, + [1][1][2][0][RTW89_FCC][65] = 10, + [1][1][2][0][RTW89_ETSI][65] = 127, + [1][1][2][0][RTW89_KCC][65] = 28, + [1][1][2][0][RTW89_FCC][69] = 10, + [1][1][2][0][RTW89_ETSI][69] = 127, + [1][1][2][0][RTW89_KCC][69] = 28, + [1][1][2][0][RTW89_FCC][73] = 10, + [1][1][2][0][RTW89_ETSI][73] = 127, + [1][1][2][0][RTW89_KCC][73] = 28, + [1][1][2][0][RTW89_FCC][76] = 10, + [1][1][2][0][RTW89_ETSI][76] = 127, + [1][1][2][0][RTW89_KCC][76] = 28, + [1][1][2][0][RTW89_FCC][80] = 10, + [1][1][2][0][RTW89_ETSI][80] = 127, + [1][1][2][0][RTW89_KCC][80] = 32, + [1][1][2][0][RTW89_FCC][84] = 10, + [1][1][2][0][RTW89_ETSI][84] = 127, + [1][1][2][0][RTW89_KCC][84] = 32, + [1][1][2][0][RTW89_FCC][88] = 10, + [1][1][2][0][RTW89_ETSI][88] = 127, + [1][1][2][0][RTW89_KCC][88] = 32, + [1][1][2][0][RTW89_FCC][91] = 12, + [1][1][2][0][RTW89_ETSI][91] = 127, + [1][1][2][0][RTW89_KCC][91] = 32, + [1][1][2][0][RTW89_FCC][95] = 10, + [1][1][2][0][RTW89_ETSI][95] = 127, + [1][1][2][0][RTW89_KCC][95] = 32, + [1][1][2][0][RTW89_FCC][99] = 10, + [1][1][2][0][RTW89_ETSI][99] = 127, + [1][1][2][0][RTW89_KCC][99] = 32, + [1][1][2][0][RTW89_FCC][103] = 10, + [1][1][2][0][RTW89_ETSI][103] = 127, + [1][1][2][0][RTW89_KCC][103] = 32, + [1][1][2][0][RTW89_FCC][106] = 12, + [1][1][2][0][RTW89_ETSI][106] = 127, + [1][1][2][0][RTW89_KCC][106] = 32, [1][1][2][0][RTW89_FCC][110] = 127, + [1][1][2][0][RTW89_ETSI][110] = 127, + [1][1][2][0][RTW89_KCC][110] = 127, [1][1][2][0][RTW89_FCC][114] = 127, + [1][1][2][0][RTW89_ETSI][114] = 127, + [1][1][2][0][RTW89_KCC][114] = 127, [1][1][2][0][RTW89_FCC][118] = 127, - [1][1][2][1][RTW89_FCC][1] = 48, - [1][1][2][1][RTW89_FCC][5] = 48, - [1][1][2][1][RTW89_FCC][9] = 48, - [1][1][2][1][RTW89_FCC][13] = 48, - [1][1][2][1][RTW89_FCC][16] = 48, - [1][1][2][1][RTW89_FCC][20] = 48, - [1][1][2][1][RTW89_FCC][24] = 48, - [1][1][2][1][RTW89_FCC][28] = 48, - [1][1][2][1][RTW89_FCC][31] = 48, - [1][1][2][1][RTW89_FCC][35] = 48, - [1][1][2][1][RTW89_FCC][39] = 48, - [1][1][2][1][RTW89_FCC][43] = 48, - [1][1][2][1][RTW89_FCC][46] = 48, - [1][1][2][1][RTW89_FCC][50] = 48, - [1][1][2][1][RTW89_FCC][54] = 48, - [1][1][2][1][RTW89_FCC][58] = 48, - [1][1][2][1][RTW89_FCC][61] = 48, - [1][1][2][1][RTW89_FCC][65] = 48, - [1][1][2][1][RTW89_FCC][69] = 48, - [1][1][2][1][RTW89_FCC][73] = 48, - [1][1][2][1][RTW89_FCC][76] = 48, - [1][1][2][1][RTW89_FCC][80] = 48, - [1][1][2][1][RTW89_FCC][84] = 48, - [1][1][2][1][RTW89_FCC][88] = 48, - [1][1][2][1][RTW89_FCC][91] = 48, - [1][1][2][1][RTW89_FCC][95] = 48, - [1][1][2][1][RTW89_FCC][99] = 48, - [1][1][2][1][RTW89_FCC][103] = 48, - [1][1][2][1][RTW89_FCC][106] = 48, + [1][1][2][0][RTW89_ETSI][118] = 127, + [1][1][2][0][RTW89_KCC][118] = 127, + [1][1][2][1][RTW89_FCC][1] = 10, + [1][1][2][1][RTW89_ETSI][1] = 42, + [1][1][2][1][RTW89_KCC][1] = 28, + [1][1][2][1][RTW89_FCC][5] = 10, + [1][1][2][1][RTW89_ETSI][5] = 42, + [1][1][2][1][RTW89_KCC][5] = 28, + [1][1][2][1][RTW89_FCC][9] = 10, + [1][1][2][1][RTW89_ETSI][9] = 42, + [1][1][2][1][RTW89_KCC][9] = 28, + [1][1][2][1][RTW89_FCC][13] = 10, + [1][1][2][1][RTW89_ETSI][13] = 42, + [1][1][2][1][RTW89_KCC][13] = 28, + [1][1][2][1][RTW89_FCC][16] = 10, + [1][1][2][1][RTW89_ETSI][16] = 42, + [1][1][2][1][RTW89_KCC][16] = 28, + [1][1][2][1][RTW89_FCC][20] = 10, + [1][1][2][1][RTW89_ETSI][20] = 42, + [1][1][2][1][RTW89_KCC][20] = 28, + [1][1][2][1][RTW89_FCC][24] = 10, + [1][1][2][1][RTW89_ETSI][24] = 42, + [1][1][2][1][RTW89_KCC][24] = 28, + [1][1][2][1][RTW89_FCC][28] = 10, + [1][1][2][1][RTW89_ETSI][28] = 42, + [1][1][2][1][RTW89_KCC][28] = 28, + [1][1][2][1][RTW89_FCC][31] = 10, + [1][1][2][1][RTW89_ETSI][31] = 42, + [1][1][2][1][RTW89_KCC][31] = 28, + [1][1][2][1][RTW89_FCC][35] = 10, + [1][1][2][1][RTW89_ETSI][35] = 42, + [1][1][2][1][RTW89_KCC][35] = 28, + [1][1][2][1][RTW89_FCC][39] = 10, + [1][1][2][1][RTW89_ETSI][39] = 42, + [1][1][2][1][RTW89_KCC][39] = 28, + [1][1][2][1][RTW89_FCC][43] = 10, + [1][1][2][1][RTW89_ETSI][43] = 42, + [1][1][2][1][RTW89_KCC][43] = 28, + [1][1][2][1][RTW89_FCC][46] = 12, + [1][1][2][1][RTW89_ETSI][46] = 127, + [1][1][2][1][RTW89_KCC][46] = 28, + [1][1][2][1][RTW89_FCC][50] = 12, + [1][1][2][1][RTW89_ETSI][50] = 127, + [1][1][2][1][RTW89_KCC][50] = 28, + [1][1][2][1][RTW89_FCC][54] = 10, + [1][1][2][1][RTW89_ETSI][54] = 127, + [1][1][2][1][RTW89_KCC][54] = 28, + [1][1][2][1][RTW89_FCC][58] = 10, + [1][1][2][1][RTW89_ETSI][58] = 127, + [1][1][2][1][RTW89_KCC][58] = 28, + [1][1][2][1][RTW89_FCC][61] = 10, + [1][1][2][1][RTW89_ETSI][61] = 127, + [1][1][2][1][RTW89_KCC][61] = 28, + [1][1][2][1][RTW89_FCC][65] = 10, + [1][1][2][1][RTW89_ETSI][65] = 127, + [1][1][2][1][RTW89_KCC][65] = 28, + [1][1][2][1][RTW89_FCC][69] = 10, + [1][1][2][1][RTW89_ETSI][69] = 127, + [1][1][2][1][RTW89_KCC][69] = 28, + [1][1][2][1][RTW89_FCC][73] = 10, + [1][1][2][1][RTW89_ETSI][73] = 127, + [1][1][2][1][RTW89_KCC][73] = 28, + [1][1][2][1][RTW89_FCC][76] = 10, + [1][1][2][1][RTW89_ETSI][76] = 127, + [1][1][2][1][RTW89_KCC][76] = 28, + [1][1][2][1][RTW89_FCC][80] = 10, + [1][1][2][1][RTW89_ETSI][80] = 127, + [1][1][2][1][RTW89_KCC][80] = 32, + [1][1][2][1][RTW89_FCC][84] = 10, + [1][1][2][1][RTW89_ETSI][84] = 127, + [1][1][2][1][RTW89_KCC][84] = 32, + [1][1][2][1][RTW89_FCC][88] = 10, + [1][1][2][1][RTW89_ETSI][88] = 127, + [1][1][2][1][RTW89_KCC][88] = 32, + [1][1][2][1][RTW89_FCC][91] = 12, + [1][1][2][1][RTW89_ETSI][91] = 127, + [1][1][2][1][RTW89_KCC][91] = 32, + [1][1][2][1][RTW89_FCC][95] = 10, + [1][1][2][1][RTW89_ETSI][95] = 127, + [1][1][2][1][RTW89_KCC][95] = 32, + [1][1][2][1][RTW89_FCC][99] = 10, + [1][1][2][1][RTW89_ETSI][99] = 127, + [1][1][2][1][RTW89_KCC][99] = 32, + [1][1][2][1][RTW89_FCC][103] = 10, + [1][1][2][1][RTW89_ETSI][103] = 127, + [1][1][2][1][RTW89_KCC][103] = 32, + [1][1][2][1][RTW89_FCC][106] = 12, + [1][1][2][1][RTW89_ETSI][106] = 127, + [1][1][2][1][RTW89_KCC][106] = 32, [1][1][2][1][RTW89_FCC][110] = 127, + [1][1][2][1][RTW89_ETSI][110] = 127, + [1][1][2][1][RTW89_KCC][110] = 127, [1][1][2][1][RTW89_FCC][114] = 127, + [1][1][2][1][RTW89_ETSI][114] = 127, + [1][1][2][1][RTW89_KCC][114] = 127, [1][1][2][1][RTW89_FCC][118] = 127, - [2][0][2][0][RTW89_FCC][3] = 64, - [2][0][2][0][RTW89_FCC][11] = 64, - [2][0][2][0][RTW89_FCC][18] = 64, - [2][0][2][0][RTW89_FCC][26] = 64, - [2][0][2][0][RTW89_FCC][33] = 64, - [2][0][2][0][RTW89_FCC][41] = 64, - [2][0][2][0][RTW89_FCC][48] = 64, - [2][0][2][0][RTW89_FCC][56] = 64, - [2][0][2][0][RTW89_FCC][63] = 64, - [2][0][2][0][RTW89_FCC][71] = 64, - [2][0][2][0][RTW89_FCC][78] = 64, - [2][0][2][0][RTW89_FCC][86] = 64, - [2][0][2][0][RTW89_FCC][93] = 64, - [2][0][2][0][RTW89_FCC][101] = 64, + [1][1][2][1][RTW89_ETSI][118] = 127, + [1][1][2][1][RTW89_KCC][118] = 127, + [2][0][2][0][RTW89_FCC][3] = 46, + [2][0][2][0][RTW89_ETSI][3] = 48, + [2][0][2][0][RTW89_KCC][3] = 50, + [2][0][2][0][RTW89_FCC][11] = 46, + [2][0][2][0][RTW89_ETSI][11] = 48, + [2][0][2][0][RTW89_KCC][11] = 50, + [2][0][2][0][RTW89_FCC][18] = 46, + [2][0][2][0][RTW89_ETSI][18] = 48, + [2][0][2][0][RTW89_KCC][18] = 50, + [2][0][2][0][RTW89_FCC][26] = 46, + [2][0][2][0][RTW89_ETSI][26] = 48, + [2][0][2][0][RTW89_KCC][26] = 50, + [2][0][2][0][RTW89_FCC][33] = 46, + [2][0][2][0][RTW89_ETSI][33] = 48, + [2][0][2][0][RTW89_KCC][33] = 50, + [2][0][2][0][RTW89_FCC][41] = 46, + [2][0][2][0][RTW89_ETSI][41] = 48, + [2][0][2][0][RTW89_KCC][41] = 50, + [2][0][2][0][RTW89_FCC][48] = 46, + [2][0][2][0][RTW89_ETSI][48] = 127, + [2][0][2][0][RTW89_KCC][48] = 48, + [2][0][2][0][RTW89_FCC][56] = 46, + [2][0][2][0][RTW89_ETSI][56] = 127, + [2][0][2][0][RTW89_KCC][56] = 48, + [2][0][2][0][RTW89_FCC][63] = 46, + [2][0][2][0][RTW89_ETSI][63] = 127, + [2][0][2][0][RTW89_KCC][63] = 48, + [2][0][2][0][RTW89_FCC][71] = 46, + [2][0][2][0][RTW89_ETSI][71] = 127, + [2][0][2][0][RTW89_KCC][71] = 48, + [2][0][2][0][RTW89_FCC][78] = 46, + [2][0][2][0][RTW89_ETSI][78] = 127, + [2][0][2][0][RTW89_KCC][78] = 52, + [2][0][2][0][RTW89_FCC][86] = 46, + [2][0][2][0][RTW89_ETSI][86] = 127, + [2][0][2][0][RTW89_KCC][86] = 52, + [2][0][2][0][RTW89_FCC][93] = 46, + [2][0][2][0][RTW89_ETSI][93] = 127, + [2][0][2][0][RTW89_KCC][93] = 50, + [2][0][2][0][RTW89_FCC][101] = 44, + [2][0][2][0][RTW89_ETSI][101] = 127, + [2][0][2][0][RTW89_KCC][101] = 50, [2][0][2][0][RTW89_FCC][108] = 127, + [2][0][2][0][RTW89_ETSI][108] = 127, + [2][0][2][0][RTW89_KCC][108] = 127, [2][0][2][0][RTW89_FCC][116] = 127, - [2][1][2][0][RTW89_FCC][3] = 52, - [2][1][2][0][RTW89_FCC][11] = 52, - [2][1][2][0][RTW89_FCC][18] = 52, - [2][1][2][0][RTW89_FCC][26] = 52, - [2][1][2][0][RTW89_FCC][33] = 52, - [2][1][2][0][RTW89_FCC][41] = 52, - [2][1][2][0][RTW89_FCC][48] = 52, - [2][1][2][0][RTW89_FCC][56] = 52, - [2][1][2][0][RTW89_FCC][63] = 52, - [2][1][2][0][RTW89_FCC][71] = 52, - [2][1][2][0][RTW89_FCC][78] = 52, - [2][1][2][0][RTW89_FCC][86] = 52, - [2][1][2][0][RTW89_FCC][93] = 52, - [2][1][2][0][RTW89_FCC][101] = 52, + [2][0][2][0][RTW89_ETSI][116] = 127, + [2][0][2][0][RTW89_KCC][116] = 127, + [2][1][2][0][RTW89_FCC][3] = 22, + [2][1][2][0][RTW89_ETSI][3] = 48, + [2][1][2][0][RTW89_KCC][3] = 38, + [2][1][2][0][RTW89_FCC][11] = 20, + [2][1][2][0][RTW89_ETSI][11] = 48, + [2][1][2][0][RTW89_KCC][11] = 38, + [2][1][2][0][RTW89_FCC][18] = 20, + [2][1][2][0][RTW89_ETSI][18] = 48, + [2][1][2][0][RTW89_KCC][18] = 38, + [2][1][2][0][RTW89_FCC][26] = 20, + [2][1][2][0][RTW89_ETSI][26] = 48, + [2][1][2][0][RTW89_KCC][26] = 38, + [2][1][2][0][RTW89_FCC][33] = 20, + [2][1][2][0][RTW89_ETSI][33] = 48, + [2][1][2][0][RTW89_KCC][33] = 38, + [2][1][2][0][RTW89_FCC][41] = 22, + [2][1][2][0][RTW89_ETSI][41] = 48, + [2][1][2][0][RTW89_KCC][41] = 38, + [2][1][2][0][RTW89_FCC][48] = 22, + [2][1][2][0][RTW89_ETSI][48] = 127, + [2][1][2][0][RTW89_KCC][48] = 38, + [2][1][2][0][RTW89_FCC][56] = 20, + [2][1][2][0][RTW89_ETSI][56] = 127, + [2][1][2][0][RTW89_KCC][56] = 38, + [2][1][2][0][RTW89_FCC][63] = 22, + [2][1][2][0][RTW89_ETSI][63] = 127, + [2][1][2][0][RTW89_KCC][63] = 38, + [2][1][2][0][RTW89_FCC][71] = 20, + [2][1][2][0][RTW89_ETSI][71] = 127, + [2][1][2][0][RTW89_KCC][71] = 38, + [2][1][2][0][RTW89_FCC][78] = 20, + [2][1][2][0][RTW89_ETSI][78] = 127, + [2][1][2][0][RTW89_KCC][78] = 38, + [2][1][2][0][RTW89_FCC][86] = 20, + [2][1][2][0][RTW89_ETSI][86] = 127, + [2][1][2][0][RTW89_KCC][86] = 38, + [2][1][2][0][RTW89_FCC][93] = 22, + [2][1][2][0][RTW89_ETSI][93] = 127, + [2][1][2][0][RTW89_KCC][93] = 38, + [2][1][2][0][RTW89_FCC][101] = 22, + [2][1][2][0][RTW89_ETSI][101] = 127, + [2][1][2][0][RTW89_KCC][101] = 38, [2][1][2][0][RTW89_FCC][108] = 127, + [2][1][2][0][RTW89_ETSI][108] = 127, + [2][1][2][0][RTW89_KCC][108] = 127, [2][1][2][0][RTW89_FCC][116] = 127, - [2][1][2][1][RTW89_FCC][3] = 40, - [2][1][2][1][RTW89_FCC][11] = 40, - [2][1][2][1][RTW89_FCC][18] = 40, - [2][1][2][1][RTW89_FCC][26] = 40, - [2][1][2][1][RTW89_FCC][33] = 40, - [2][1][2][1][RTW89_FCC][41] = 40, - [2][1][2][1][RTW89_FCC][48] = 40, - [2][1][2][1][RTW89_FCC][56] = 40, - [2][1][2][1][RTW89_FCC][63] = 40, - [2][1][2][1][RTW89_FCC][71] = 40, - [2][1][2][1][RTW89_FCC][78] = 40, - [2][1][2][1][RTW89_FCC][86] = 40, - [2][1][2][1][RTW89_FCC][93] = 40, - [2][1][2][1][RTW89_FCC][101] = 40, + [2][1][2][0][RTW89_ETSI][116] = 127, + [2][1][2][0][RTW89_KCC][116] = 127, + [2][1][2][1][RTW89_FCC][3] = 22, + [2][1][2][1][RTW89_ETSI][3] = 42, + [2][1][2][1][RTW89_KCC][3] = 38, + [2][1][2][1][RTW89_FCC][11] = 20, + [2][1][2][1][RTW89_ETSI][11] = 42, + [2][1][2][1][RTW89_KCC][11] = 38, + [2][1][2][1][RTW89_FCC][18] = 20, + [2][1][2][1][RTW89_ETSI][18] = 42, + [2][1][2][1][RTW89_KCC][18] = 38, + [2][1][2][1][RTW89_FCC][26] = 20, + [2][1][2][1][RTW89_ETSI][26] = 42, + [2][1][2][1][RTW89_KCC][26] = 38, + [2][1][2][1][RTW89_FCC][33] = 20, + [2][1][2][1][RTW89_ETSI][33] = 42, + [2][1][2][1][RTW89_KCC][33] = 38, + [2][1][2][1][RTW89_FCC][41] = 22, + [2][1][2][1][RTW89_ETSI][41] = 42, + [2][1][2][1][RTW89_KCC][41] = 38, + [2][1][2][1][RTW89_FCC][48] = 22, + [2][1][2][1][RTW89_ETSI][48] = 127, + [2][1][2][1][RTW89_KCC][48] = 38, + [2][1][2][1][RTW89_FCC][56] = 20, + [2][1][2][1][RTW89_ETSI][56] = 127, + [2][1][2][1][RTW89_KCC][56] = 38, + [2][1][2][1][RTW89_FCC][63] = 22, + [2][1][2][1][RTW89_ETSI][63] = 127, + [2][1][2][1][RTW89_KCC][63] = 38, + [2][1][2][1][RTW89_FCC][71] = 20, + [2][1][2][1][RTW89_ETSI][71] = 127, + [2][1][2][1][RTW89_KCC][71] = 38, + [2][1][2][1][RTW89_FCC][78] = 20, + [2][1][2][1][RTW89_ETSI][78] = 127, + [2][1][2][1][RTW89_KCC][78] = 38, + [2][1][2][1][RTW89_FCC][86] = 20, + [2][1][2][1][RTW89_ETSI][86] = 127, + [2][1][2][1][RTW89_KCC][86] = 38, + [2][1][2][1][RTW89_FCC][93] = 22, + [2][1][2][1][RTW89_ETSI][93] = 127, + [2][1][2][1][RTW89_KCC][93] = 38, + [2][1][2][1][RTW89_FCC][101] = 22, + [2][1][2][1][RTW89_ETSI][101] = 127, + [2][1][2][1][RTW89_KCC][101] = 38, [2][1][2][1][RTW89_FCC][108] = 127, + [2][1][2][1][RTW89_ETSI][108] = 127, + [2][1][2][1][RTW89_KCC][108] = 127, [2][1][2][1][RTW89_FCC][116] = 127, - [3][0][2][0][RTW89_FCC][7] = 56, - [3][0][2][0][RTW89_FCC][22] = 56, - [3][0][2][0][RTW89_FCC][37] = 56, - [3][0][2][0][RTW89_FCC][52] = 56, - [3][0][2][0][RTW89_FCC][67] = 56, - [3][0][2][0][RTW89_FCC][82] = 56, - [3][0][2][0][RTW89_FCC][97] = 56, + [2][1][2][1][RTW89_ETSI][116] = 127, + [2][1][2][1][RTW89_KCC][116] = 127, + [3][0][2][0][RTW89_FCC][7] = 52, + [3][0][2][0][RTW89_ETSI][7] = 38, + [3][0][2][0][RTW89_KCC][7] = 42, + [3][0][2][0][RTW89_FCC][22] = 52, + [3][0][2][0][RTW89_ETSI][22] = 38, + [3][0][2][0][RTW89_KCC][22] = 42, + [3][0][2][0][RTW89_FCC][37] = 52, + [3][0][2][0][RTW89_ETSI][37] = 38, + [3][0][2][0][RTW89_KCC][37] = 42, + [3][0][2][0][RTW89_FCC][52] = 54, + [3][0][2][0][RTW89_ETSI][52] = 127, + [3][0][2][0][RTW89_KCC][52] = 56, + [3][0][2][0][RTW89_FCC][67] = 54, + [3][0][2][0][RTW89_ETSI][67] = 127, + [3][0][2][0][RTW89_KCC][67] = 54, + [3][0][2][0][RTW89_FCC][82] = 54, + [3][0][2][0][RTW89_ETSI][82] = 127, + [3][0][2][0][RTW89_KCC][82] = 26, + [3][0][2][0][RTW89_FCC][97] = 40, + [3][0][2][0][RTW89_ETSI][97] = 127, + [3][0][2][0][RTW89_KCC][97] = 26, [3][0][2][0][RTW89_FCC][112] = 127, - [3][1][2][0][RTW89_FCC][7] = 44, - [3][1][2][0][RTW89_FCC][22] = 44, - [3][1][2][0][RTW89_FCC][37] = 44, - [3][1][2][0][RTW89_FCC][52] = 44, - [3][1][2][0][RTW89_FCC][67] = 44, - [3][1][2][0][RTW89_FCC][82] = 44, - [3][1][2][0][RTW89_FCC][97] = 44, + [3][0][2][0][RTW89_ETSI][112] = 127, + [3][0][2][0][RTW89_KCC][112] = 127, + [3][1][2][0][RTW89_FCC][7] = 32, + [3][1][2][0][RTW89_ETSI][7] = 38, + [3][1][2][0][RTW89_KCC][7] = 40, + [3][1][2][0][RTW89_FCC][22] = 30, + [3][1][2][0][RTW89_ETSI][22] = 38, + [3][1][2][0][RTW89_KCC][22] = 40, + [3][1][2][0][RTW89_FCC][37] = 30, + [3][1][2][0][RTW89_ETSI][37] = 38, + [3][1][2][0][RTW89_KCC][37] = 40, + [3][1][2][0][RTW89_FCC][52] = 30, + [3][1][2][0][RTW89_ETSI][52] = 127, + [3][1][2][0][RTW89_KCC][52] = 48, + [3][1][2][0][RTW89_FCC][67] = 32, + [3][1][2][0][RTW89_ETSI][67] = 127, + [3][1][2][0][RTW89_KCC][67] = 48, + [3][1][2][0][RTW89_FCC][82] = 32, + [3][1][2][0][RTW89_ETSI][82] = 127, + [3][1][2][0][RTW89_KCC][82] = 24, + [3][1][2][0][RTW89_FCC][97] = 14, + [3][1][2][0][RTW89_ETSI][97] = 127, + [3][1][2][0][RTW89_KCC][97] = 24, [3][1][2][0][RTW89_FCC][112] = 127, + [3][1][2][0][RTW89_ETSI][112] = 127, + [3][1][2][0][RTW89_KCC][112] = 127, [3][1][2][1][RTW89_FCC][7] = 32, - [3][1][2][1][RTW89_FCC][22] = 32, - [3][1][2][1][RTW89_FCC][37] = 32, - [3][1][2][1][RTW89_FCC][52] = 32, + [3][1][2][1][RTW89_ETSI][7] = 38, + [3][1][2][1][RTW89_KCC][7] = 40, + [3][1][2][1][RTW89_FCC][22] = 30, + [3][1][2][1][RTW89_ETSI][22] = 38, + [3][1][2][1][RTW89_KCC][22] = 40, + [3][1][2][1][RTW89_FCC][37] = 30, + [3][1][2][1][RTW89_ETSI][37] = 38, + [3][1][2][1][RTW89_KCC][37] = 40, + [3][1][2][1][RTW89_FCC][52] = 30, + [3][1][2][1][RTW89_ETSI][52] = 127, + [3][1][2][1][RTW89_KCC][52] = 48, [3][1][2][1][RTW89_FCC][67] = 32, + [3][1][2][1][RTW89_ETSI][67] = 127, + [3][1][2][1][RTW89_KCC][67] = 48, [3][1][2][1][RTW89_FCC][82] = 32, - [3][1][2][1][RTW89_FCC][97] = 32, + [3][1][2][1][RTW89_ETSI][82] = 127, + [3][1][2][1][RTW89_KCC][82] = 24, + [3][1][2][1][RTW89_FCC][97] = 14, + [3][1][2][1][RTW89_ETSI][97] = 127, + [3][1][2][1][RTW89_KCC][97] = 24, [3][1][2][1][RTW89_FCC][112] = 127, + [3][1][2][1][RTW89_ETSI][112] = 127, + [3][1][2][1][RTW89_KCC][112] = 127, }; const s8 rtw89_8852c_txpwr_lmt_ru_2g[RTW89_RU_NUM][RTW89_NTX_NUM] @@ -17126,8 +33220,8 @@ const s8 rtw89_8852c_txpwr_lmt_ru_2g[RTW89_RU_NUM][RTW89_NTX_NUM] [0][0][RTW89_WW][8] = 32, [0][0][RTW89_WW][9] = 32, [0][0][RTW89_WW][10] = 32, - [0][0][RTW89_WW][11] = 32, - [0][0][RTW89_WW][12] = 24, + [0][0][RTW89_WW][11] = 26, + [0][0][RTW89_WW][12] = -20, [0][0][RTW89_WW][13] = 0, [0][1][RTW89_WW][0] = 20, [0][1][RTW89_WW][1] = 22, @@ -17141,7 +33235,7 @@ const s8 rtw89_8852c_txpwr_lmt_ru_2g[RTW89_RU_NUM][RTW89_NTX_NUM] [0][1][RTW89_WW][9] = 22, [0][1][RTW89_WW][10] = 22, [0][1][RTW89_WW][11] = 22, - [0][1][RTW89_WW][12] = 20, + [0][1][RTW89_WW][12] = -30, [0][1][RTW89_WW][13] = 0, [1][0][RTW89_WW][0] = 42, [1][0][RTW89_WW][1] = 44, @@ -17154,8 +33248,8 @@ const s8 rtw89_8852c_txpwr_lmt_ru_2g[RTW89_RU_NUM][RTW89_NTX_NUM] [1][0][RTW89_WW][8] = 44, [1][0][RTW89_WW][9] = 44, [1][0][RTW89_WW][10] = 44, - [1][0][RTW89_WW][11] = 42, - [1][0][RTW89_WW][12] = 30, + [1][0][RTW89_WW][11] = 36, + [1][0][RTW89_WW][12] = 4, [1][0][RTW89_WW][13] = 0, [1][1][RTW89_WW][0] = 32, [1][1][RTW89_WW][1] = 32, @@ -17169,7 +33263,7 @@ const s8 rtw89_8852c_txpwr_lmt_ru_2g[RTW89_RU_NUM][RTW89_NTX_NUM] [1][1][RTW89_WW][9] = 32, [1][1][RTW89_WW][10] = 32, [1][1][RTW89_WW][11] = 30, - [1][1][RTW89_WW][12] = 24, + [1][1][RTW89_WW][12] = -6, [1][1][RTW89_WW][13] = 0, [2][0][RTW89_WW][0] = 56, [2][0][RTW89_WW][1] = 56, @@ -17182,8 +33276,8 @@ const s8 rtw89_8852c_txpwr_lmt_ru_2g[RTW89_RU_NUM][RTW89_NTX_NUM] [2][0][RTW89_WW][8] = 56, [2][0][RTW89_WW][9] = 56, [2][0][RTW89_WW][10] = 56, - [2][0][RTW89_WW][11] = 42, - [2][0][RTW89_WW][12] = 38, + [2][0][RTW89_WW][11] = 48, + [2][0][RTW89_WW][12] = 16, [2][0][RTW89_WW][13] = 0, [2][1][RTW89_WW][0] = 44, [2][1][RTW89_WW][1] = 44, @@ -17196,2213 +33290,3353 @@ const s8 rtw89_8852c_txpwr_lmt_ru_2g[RTW89_RU_NUM][RTW89_NTX_NUM] [2][1][RTW89_WW][8] = 44, [2][1][RTW89_WW][9] = 44, [2][1][RTW89_WW][10] = 44, - [2][1][RTW89_WW][11] = 30, - [2][1][RTW89_WW][12] = 26, + [2][1][RTW89_WW][11] = 44, + [2][1][RTW89_WW][12] = 6, [2][1][RTW89_WW][13] = 0, [0][0][RTW89_FCC][0] = 60, [0][0][RTW89_ETSI][0] = 34, [0][0][RTW89_MKK][0] = 36, - [0][0][RTW89_IC][0] = 68, - [0][0][RTW89_ACMA][0] = 32, + [0][0][RTW89_IC][0] = 60, + [0][0][RTW89_KCC][0] = 42, + [0][0][RTW89_ACMA][0] = 34, + [0][0][RTW89_CN][0] = 32, + [0][0][RTW89_UK][0] = 34, [0][0][RTW89_FCC][1] = 60, [0][0][RTW89_ETSI][1] = 38, [0][0][RTW89_MKK][1] = 40, - [0][0][RTW89_IC][1] = 68, - [0][0][RTW89_ACMA][1] = 32, + [0][0][RTW89_IC][1] = 60, + [0][0][RTW89_KCC][1] = 42, + [0][0][RTW89_ACMA][1] = 38, + [0][0][RTW89_CN][1] = 32, + [0][0][RTW89_UK][1] = 38, [0][0][RTW89_FCC][2] = 64, [0][0][RTW89_ETSI][2] = 38, [0][0][RTW89_MKK][2] = 40, - [0][0][RTW89_IC][2] = 72, - [0][0][RTW89_ACMA][2] = 32, + [0][0][RTW89_IC][2] = 64, + [0][0][RTW89_KCC][2] = 42, + [0][0][RTW89_ACMA][2] = 38, + [0][0][RTW89_CN][2] = 32, + [0][0][RTW89_UK][2] = 38, [0][0][RTW89_FCC][3] = 68, [0][0][RTW89_ETSI][3] = 38, [0][0][RTW89_MKK][3] = 40, - [0][0][RTW89_IC][3] = 76, - [0][0][RTW89_ACMA][3] = 32, + [0][0][RTW89_IC][3] = 68, + [0][0][RTW89_KCC][3] = 42, + [0][0][RTW89_ACMA][3] = 38, + [0][0][RTW89_CN][3] = 32, + [0][0][RTW89_UK][3] = 38, [0][0][RTW89_FCC][4] = 68, [0][0][RTW89_ETSI][4] = 38, [0][0][RTW89_MKK][4] = 40, - [0][0][RTW89_IC][4] = 76, - [0][0][RTW89_ACMA][4] = 32, - [0][0][RTW89_FCC][5] = 76, + [0][0][RTW89_IC][4] = 68, + [0][0][RTW89_KCC][4] = 42, + [0][0][RTW89_ACMA][4] = 38, + [0][0][RTW89_CN][4] = 32, + [0][0][RTW89_UK][4] = 38, + [0][0][RTW89_FCC][5] = 78, [0][0][RTW89_ETSI][5] = 38, [0][0][RTW89_MKK][5] = 40, - [0][0][RTW89_IC][5] = 84, - [0][0][RTW89_ACMA][5] = 32, - [0][0][RTW89_FCC][6] = 66, + [0][0][RTW89_IC][5] = 78, + [0][0][RTW89_KCC][5] = 42, + [0][0][RTW89_ACMA][5] = 38, + [0][0][RTW89_CN][5] = 32, + [0][0][RTW89_UK][5] = 38, + [0][0][RTW89_FCC][6] = 54, [0][0][RTW89_ETSI][6] = 38, [0][0][RTW89_MKK][6] = 40, - [0][0][RTW89_IC][6] = 74, - [0][0][RTW89_ACMA][6] = 32, - [0][0][RTW89_FCC][7] = 66, + [0][0][RTW89_IC][6] = 54, + [0][0][RTW89_KCC][6] = 42, + [0][0][RTW89_ACMA][6] = 38, + [0][0][RTW89_CN][6] = 32, + [0][0][RTW89_UK][6] = 38, + [0][0][RTW89_FCC][7] = 54, [0][0][RTW89_ETSI][7] = 38, [0][0][RTW89_MKK][7] = 40, - [0][0][RTW89_IC][7] = 74, - [0][0][RTW89_ACMA][7] = 32, - [0][0][RTW89_FCC][8] = 62, + [0][0][RTW89_IC][7] = 54, + [0][0][RTW89_KCC][7] = 42, + [0][0][RTW89_ACMA][7] = 38, + [0][0][RTW89_CN][7] = 32, + [0][0][RTW89_UK][7] = 38, + [0][0][RTW89_FCC][8] = 50, [0][0][RTW89_ETSI][8] = 38, [0][0][RTW89_MKK][8] = 40, - [0][0][RTW89_IC][8] = 70, - [0][0][RTW89_ACMA][8] = 32, - [0][0][RTW89_FCC][9] = 58, + [0][0][RTW89_IC][8] = 50, + [0][0][RTW89_KCC][8] = 42, + [0][0][RTW89_ACMA][8] = 38, + [0][0][RTW89_CN][8] = 32, + [0][0][RTW89_UK][8] = 38, + [0][0][RTW89_FCC][9] = 46, [0][0][RTW89_ETSI][9] = 38, [0][0][RTW89_MKK][9] = 40, - [0][0][RTW89_IC][9] = 66, - [0][0][RTW89_ACMA][9] = 32, - [0][0][RTW89_FCC][10] = 58, + [0][0][RTW89_IC][9] = 46, + [0][0][RTW89_KCC][9] = 40, + [0][0][RTW89_ACMA][9] = 38, + [0][0][RTW89_CN][9] = 32, + [0][0][RTW89_UK][9] = 38, + [0][0][RTW89_FCC][10] = 46, [0][0][RTW89_ETSI][10] = 38, [0][0][RTW89_MKK][10] = 40, - [0][0][RTW89_IC][10] = 66, - [0][0][RTW89_ACMA][10] = 32, - [0][0][RTW89_FCC][11] = 42, + [0][0][RTW89_IC][10] = 46, + [0][0][RTW89_KCC][10] = 40, + [0][0][RTW89_ACMA][10] = 38, + [0][0][RTW89_CN][10] = 32, + [0][0][RTW89_UK][10] = 38, + [0][0][RTW89_FCC][11] = 26, [0][0][RTW89_ETSI][11] = 38, [0][0][RTW89_MKK][11] = 40, - [0][0][RTW89_IC][11] = 56, - [0][0][RTW89_ACMA][11] = 32, - [0][0][RTW89_FCC][12] = 24, + [0][0][RTW89_IC][11] = 26, + [0][0][RTW89_KCC][11] = 40, + [0][0][RTW89_ACMA][11] = 38, + [0][0][RTW89_CN][11] = 32, + [0][0][RTW89_UK][11] = 38, + [0][0][RTW89_FCC][12] = -20, [0][0][RTW89_ETSI][12] = 34, [0][0][RTW89_MKK][12] = 36, - [0][0][RTW89_IC][12] = 32, - [0][0][RTW89_ACMA][12] = 32, + [0][0][RTW89_IC][12] = -20, + [0][0][RTW89_KCC][12] = 40, + [0][0][RTW89_ACMA][12] = 34, + [0][0][RTW89_CN][12] = 32, + [0][0][RTW89_UK][12] = 34, [0][0][RTW89_FCC][13] = 127, [0][0][RTW89_ETSI][13] = 127, [0][0][RTW89_MKK][13] = 127, [0][0][RTW89_IC][13] = 127, + [0][0][RTW89_KCC][13] = 127, [0][0][RTW89_ACMA][13] = 127, - [0][1][RTW89_FCC][0] = 46, + [0][0][RTW89_CN][13] = 127, + [0][0][RTW89_UK][13] = 127, + [0][1][RTW89_FCC][0] = 56, [0][1][RTW89_ETSI][0] = 22, [0][1][RTW89_MKK][0] = 24, - [0][1][RTW89_IC][0] = 62, - [0][1][RTW89_ACMA][0] = 20, - [0][1][RTW89_FCC][1] = 46, + [0][1][RTW89_IC][0] = 56, + [0][1][RTW89_KCC][0] = 30, + [0][1][RTW89_ACMA][0] = 22, + [0][1][RTW89_CN][0] = 20, + [0][1][RTW89_UK][0] = 22, + [0][1][RTW89_FCC][1] = 56, [0][1][RTW89_ETSI][1] = 24, [0][1][RTW89_MKK][1] = 30, - [0][1][RTW89_IC][1] = 62, - [0][1][RTW89_ACMA][1] = 22, - [0][1][RTW89_FCC][2] = 50, + [0][1][RTW89_IC][1] = 56, + [0][1][RTW89_KCC][1] = 30, + [0][1][RTW89_ACMA][1] = 24, + [0][1][RTW89_CN][1] = 22, + [0][1][RTW89_UK][1] = 24, + [0][1][RTW89_FCC][2] = 60, [0][1][RTW89_ETSI][2] = 24, [0][1][RTW89_MKK][2] = 30, - [0][1][RTW89_IC][2] = 66, - [0][1][RTW89_ACMA][2] = 22, - [0][1][RTW89_FCC][3] = 54, + [0][1][RTW89_IC][2] = 60, + [0][1][RTW89_KCC][2] = 30, + [0][1][RTW89_ACMA][2] = 24, + [0][1][RTW89_CN][2] = 22, + [0][1][RTW89_UK][2] = 24, + [0][1][RTW89_FCC][3] = 64, [0][1][RTW89_ETSI][3] = 24, [0][1][RTW89_MKK][3] = 30, - [0][1][RTW89_IC][3] = 70, - [0][1][RTW89_ACMA][3] = 22, - [0][1][RTW89_FCC][4] = 58, + [0][1][RTW89_IC][3] = 64, + [0][1][RTW89_KCC][3] = 30, + [0][1][RTW89_ACMA][3] = 24, + [0][1][RTW89_CN][3] = 22, + [0][1][RTW89_UK][3] = 24, + [0][1][RTW89_FCC][4] = 68, [0][1][RTW89_ETSI][4] = 24, [0][1][RTW89_MKK][4] = 30, - [0][1][RTW89_IC][4] = 74, - [0][1][RTW89_ACMA][4] = 22, - [0][1][RTW89_FCC][5] = 66, + [0][1][RTW89_IC][4] = 68, + [0][1][RTW89_KCC][4] = 28, + [0][1][RTW89_ACMA][4] = 24, + [0][1][RTW89_CN][4] = 22, + [0][1][RTW89_UK][4] = 24, + [0][1][RTW89_FCC][5] = 76, [0][1][RTW89_ETSI][5] = 24, [0][1][RTW89_MKK][5] = 30, - [0][1][RTW89_IC][5] = 74, - [0][1][RTW89_ACMA][5] = 22, - [0][1][RTW89_FCC][6] = 58, + [0][1][RTW89_IC][5] = 76, + [0][1][RTW89_KCC][5] = 28, + [0][1][RTW89_ACMA][5] = 24, + [0][1][RTW89_CN][5] = 22, + [0][1][RTW89_UK][5] = 24, + [0][1][RTW89_FCC][6] = 54, [0][1][RTW89_ETSI][6] = 24, [0][1][RTW89_MKK][6] = 30, - [0][1][RTW89_IC][6] = 72, - [0][1][RTW89_ACMA][6] = 22, - [0][1][RTW89_FCC][7] = 54, + [0][1][RTW89_IC][6] = 54, + [0][1][RTW89_KCC][6] = 28, + [0][1][RTW89_ACMA][6] = 24, + [0][1][RTW89_CN][6] = 22, + [0][1][RTW89_UK][6] = 24, + [0][1][RTW89_FCC][7] = 50, [0][1][RTW89_ETSI][7] = 24, [0][1][RTW89_MKK][7] = 30, - [0][1][RTW89_IC][7] = 68, - [0][1][RTW89_ACMA][7] = 22, - [0][1][RTW89_FCC][8] = 50, + [0][1][RTW89_IC][7] = 50, + [0][1][RTW89_KCC][7] = 28, + [0][1][RTW89_ACMA][7] = 24, + [0][1][RTW89_CN][7] = 22, + [0][1][RTW89_UK][7] = 24, + [0][1][RTW89_FCC][8] = 46, [0][1][RTW89_ETSI][8] = 24, [0][1][RTW89_MKK][8] = 30, - [0][1][RTW89_IC][8] = 64, - [0][1][RTW89_ACMA][8] = 22, - [0][1][RTW89_FCC][9] = 46, + [0][1][RTW89_IC][8] = 46, + [0][1][RTW89_KCC][8] = 28, + [0][1][RTW89_ACMA][8] = 24, + [0][1][RTW89_CN][8] = 22, + [0][1][RTW89_UK][8] = 24, + [0][1][RTW89_FCC][9] = 42, [0][1][RTW89_ETSI][9] = 24, [0][1][RTW89_MKK][9] = 30, - [0][1][RTW89_IC][9] = 60, - [0][1][RTW89_ACMA][9] = 22, - [0][1][RTW89_FCC][10] = 46, + [0][1][RTW89_IC][9] = 42, + [0][1][RTW89_KCC][9] = 28, + [0][1][RTW89_ACMA][9] = 24, + [0][1][RTW89_CN][9] = 22, + [0][1][RTW89_UK][9] = 24, + [0][1][RTW89_FCC][10] = 42, [0][1][RTW89_ETSI][10] = 24, [0][1][RTW89_MKK][10] = 30, - [0][1][RTW89_IC][10] = 60, - [0][1][RTW89_ACMA][10] = 22, - [0][1][RTW89_FCC][11] = 30, + [0][1][RTW89_IC][10] = 42, + [0][1][RTW89_KCC][10] = 28, + [0][1][RTW89_ACMA][10] = 24, + [0][1][RTW89_CN][10] = 22, + [0][1][RTW89_UK][10] = 24, + [0][1][RTW89_FCC][11] = 22, [0][1][RTW89_ETSI][11] = 24, [0][1][RTW89_MKK][11] = 30, - [0][1][RTW89_IC][11] = 52, - [0][1][RTW89_ACMA][11] = 22, - [0][1][RTW89_FCC][12] = 22, + [0][1][RTW89_IC][11] = 22, + [0][1][RTW89_KCC][11] = 28, + [0][1][RTW89_ACMA][11] = 24, + [0][1][RTW89_CN][11] = 22, + [0][1][RTW89_UK][11] = 24, + [0][1][RTW89_FCC][12] = -30, [0][1][RTW89_ETSI][12] = 20, [0][1][RTW89_MKK][12] = 24, - [0][1][RTW89_IC][12] = 30, + [0][1][RTW89_IC][12] = -30, + [0][1][RTW89_KCC][12] = 28, [0][1][RTW89_ACMA][12] = 20, + [0][1][RTW89_CN][12] = 20, + [0][1][RTW89_UK][12] = 20, [0][1][RTW89_FCC][13] = 127, [0][1][RTW89_ETSI][13] = 127, [0][1][RTW89_MKK][13] = 127, [0][1][RTW89_IC][13] = 127, + [0][1][RTW89_KCC][13] = 127, [0][1][RTW89_ACMA][13] = 127, - [1][0][RTW89_FCC][0] = 64, + [0][1][RTW89_CN][13] = 127, + [0][1][RTW89_UK][13] = 127, + [1][0][RTW89_FCC][0] = 66, [1][0][RTW89_ETSI][0] = 46, [1][0][RTW89_MKK][0] = 48, - [1][0][RTW89_IC][0] = 78, - [1][0][RTW89_ACMA][0] = 42, - [1][0][RTW89_FCC][1] = 64, + [1][0][RTW89_IC][0] = 66, + [1][0][RTW89_KCC][0] = 50, + [1][0][RTW89_ACMA][0] = 46, + [1][0][RTW89_CN][0] = 42, + [1][0][RTW89_UK][0] = 46, + [1][0][RTW89_FCC][1] = 66, [1][0][RTW89_ETSI][1] = 46, [1][0][RTW89_MKK][1] = 48, - [1][0][RTW89_IC][1] = 78, - [1][0][RTW89_ACMA][1] = 44, - [1][0][RTW89_FCC][2] = 68, + [1][0][RTW89_IC][1] = 66, + [1][0][RTW89_KCC][1] = 50, + [1][0][RTW89_ACMA][1] = 46, + [1][0][RTW89_CN][1] = 44, + [1][0][RTW89_UK][1] = 46, + [1][0][RTW89_FCC][2] = 70, [1][0][RTW89_ETSI][2] = 46, [1][0][RTW89_MKK][2] = 48, - [1][0][RTW89_IC][2] = 82, - [1][0][RTW89_ACMA][2] = 44, - [1][0][RTW89_FCC][3] = 70, + [1][0][RTW89_IC][2] = 70, + [1][0][RTW89_KCC][2] = 50, + [1][0][RTW89_ACMA][2] = 46, + [1][0][RTW89_CN][2] = 44, + [1][0][RTW89_UK][2] = 46, + [1][0][RTW89_FCC][3] = 72, [1][0][RTW89_ETSI][3] = 46, [1][0][RTW89_MKK][3] = 48, - [1][0][RTW89_IC][3] = 84, - [1][0][RTW89_ACMA][3] = 44, - [1][0][RTW89_FCC][4] = 70, + [1][0][RTW89_IC][3] = 72, + [1][0][RTW89_KCC][3] = 50, + [1][0][RTW89_ACMA][3] = 46, + [1][0][RTW89_CN][3] = 44, + [1][0][RTW89_UK][3] = 46, + [1][0][RTW89_FCC][4] = 72, [1][0][RTW89_ETSI][4] = 46, [1][0][RTW89_MKK][4] = 48, - [1][0][RTW89_IC][4] = 84, - [1][0][RTW89_ACMA][4] = 44, - [1][0][RTW89_FCC][5] = 76, + [1][0][RTW89_IC][4] = 72, + [1][0][RTW89_KCC][4] = 50, + [1][0][RTW89_ACMA][4] = 46, + [1][0][RTW89_CN][4] = 44, + [1][0][RTW89_UK][4] = 46, + [1][0][RTW89_FCC][5] = 82, [1][0][RTW89_ETSI][5] = 46, [1][0][RTW89_MKK][5] = 48, - [1][0][RTW89_IC][5] = 84, - [1][0][RTW89_ACMA][5] = 44, - [1][0][RTW89_FCC][6] = 64, + [1][0][RTW89_IC][5] = 82, + [1][0][RTW89_KCC][5] = 50, + [1][0][RTW89_ACMA][5] = 46, + [1][0][RTW89_CN][5] = 44, + [1][0][RTW89_UK][5] = 46, + [1][0][RTW89_FCC][6] = 58, [1][0][RTW89_ETSI][6] = 44, [1][0][RTW89_MKK][6] = 48, - [1][0][RTW89_IC][6] = 78, + [1][0][RTW89_IC][6] = 58, + [1][0][RTW89_KCC][6] = 50, [1][0][RTW89_ACMA][6] = 44, - [1][0][RTW89_FCC][7] = 64, + [1][0][RTW89_CN][6] = 44, + [1][0][RTW89_UK][6] = 44, + [1][0][RTW89_FCC][7] = 58, [1][0][RTW89_ETSI][7] = 46, [1][0][RTW89_MKK][7] = 48, - [1][0][RTW89_IC][7] = 78, - [1][0][RTW89_ACMA][7] = 44, - [1][0][RTW89_FCC][8] = 64, + [1][0][RTW89_IC][7] = 58, + [1][0][RTW89_KCC][7] = 50, + [1][0][RTW89_ACMA][7] = 46, + [1][0][RTW89_CN][7] = 44, + [1][0][RTW89_UK][7] = 46, + [1][0][RTW89_FCC][8] = 58, [1][0][RTW89_ETSI][8] = 46, [1][0][RTW89_MKK][8] = 48, - [1][0][RTW89_IC][8] = 78, - [1][0][RTW89_ACMA][8] = 44, - [1][0][RTW89_FCC][9] = 60, + [1][0][RTW89_IC][8] = 58, + [1][0][RTW89_KCC][8] = 50, + [1][0][RTW89_ACMA][8] = 46, + [1][0][RTW89_CN][8] = 44, + [1][0][RTW89_UK][8] = 46, + [1][0][RTW89_FCC][9] = 54, [1][0][RTW89_ETSI][9] = 46, [1][0][RTW89_MKK][9] = 48, - [1][0][RTW89_IC][9] = 74, - [1][0][RTW89_ACMA][9] = 44, - [1][0][RTW89_FCC][10] = 60, + [1][0][RTW89_IC][9] = 54, + [1][0][RTW89_KCC][9] = 50, + [1][0][RTW89_ACMA][9] = 46, + [1][0][RTW89_CN][9] = 44, + [1][0][RTW89_UK][9] = 46, + [1][0][RTW89_FCC][10] = 54, [1][0][RTW89_ETSI][10] = 46, [1][0][RTW89_MKK][10] = 48, - [1][0][RTW89_IC][10] = 74, - [1][0][RTW89_ACMA][10] = 44, - [1][0][RTW89_FCC][11] = 42, + [1][0][RTW89_IC][10] = 54, + [1][0][RTW89_KCC][10] = 50, + [1][0][RTW89_ACMA][10] = 46, + [1][0][RTW89_CN][10] = 44, + [1][0][RTW89_UK][10] = 46, + [1][0][RTW89_FCC][11] = 36, [1][0][RTW89_ETSI][11] = 46, [1][0][RTW89_MKK][11] = 48, - [1][0][RTW89_IC][11] = 72, - [1][0][RTW89_ACMA][11] = 44, - [1][0][RTW89_FCC][12] = 30, + [1][0][RTW89_IC][11] = 36, + [1][0][RTW89_KCC][11] = 50, + [1][0][RTW89_ACMA][11] = 46, + [1][0][RTW89_CN][11] = 44, + [1][0][RTW89_UK][11] = 46, + [1][0][RTW89_FCC][12] = 4, [1][0][RTW89_ETSI][12] = 46, [1][0][RTW89_MKK][12] = 46, - [1][0][RTW89_IC][12] = 38, - [1][0][RTW89_ACMA][12] = 42, + [1][0][RTW89_IC][12] = 4, + [1][0][RTW89_KCC][12] = 50, + [1][0][RTW89_ACMA][12] = 46, + [1][0][RTW89_CN][12] = 42, + [1][0][RTW89_UK][12] = 46, [1][0][RTW89_FCC][13] = 127, [1][0][RTW89_ETSI][13] = 127, [1][0][RTW89_MKK][13] = 127, [1][0][RTW89_IC][13] = 127, + [1][0][RTW89_KCC][13] = 127, [1][0][RTW89_ACMA][13] = 127, - [1][1][RTW89_FCC][0] = 46, + [1][0][RTW89_CN][13] = 127, + [1][0][RTW89_UK][13] = 127, + [1][1][RTW89_FCC][0] = 58, [1][1][RTW89_ETSI][0] = 32, [1][1][RTW89_MKK][0] = 34, - [1][1][RTW89_IC][0] = 66, + [1][1][RTW89_IC][0] = 58, + [1][1][RTW89_KCC][0] = 38, [1][1][RTW89_ACMA][0] = 32, - [1][1][RTW89_FCC][1] = 46, + [1][1][RTW89_CN][0] = 32, + [1][1][RTW89_UK][0] = 32, + [1][1][RTW89_FCC][1] = 58, [1][1][RTW89_ETSI][1] = 34, [1][1][RTW89_MKK][1] = 34, - [1][1][RTW89_IC][1] = 66, - [1][1][RTW89_ACMA][1] = 32, - [1][1][RTW89_FCC][2] = 50, + [1][1][RTW89_IC][1] = 58, + [1][1][RTW89_KCC][1] = 38, + [1][1][RTW89_ACMA][1] = 34, + [1][1][RTW89_CN][1] = 32, + [1][1][RTW89_UK][1] = 34, + [1][1][RTW89_FCC][2] = 62, [1][1][RTW89_ETSI][2] = 34, [1][1][RTW89_MKK][2] = 34, - [1][1][RTW89_IC][2] = 70, - [1][1][RTW89_ACMA][2] = 32, - [1][1][RTW89_FCC][3] = 54, + [1][1][RTW89_IC][2] = 62, + [1][1][RTW89_KCC][2] = 38, + [1][1][RTW89_ACMA][2] = 34, + [1][1][RTW89_CN][2] = 32, + [1][1][RTW89_UK][2] = 34, + [1][1][RTW89_FCC][3] = 66, [1][1][RTW89_ETSI][3] = 34, [1][1][RTW89_MKK][3] = 34, - [1][1][RTW89_IC][3] = 74, - [1][1][RTW89_ACMA][3] = 32, - [1][1][RTW89_FCC][4] = 58, + [1][1][RTW89_IC][3] = 66, + [1][1][RTW89_KCC][3] = 38, + [1][1][RTW89_ACMA][3] = 34, + [1][1][RTW89_CN][3] = 32, + [1][1][RTW89_UK][3] = 34, + [1][1][RTW89_FCC][4] = 70, [1][1][RTW89_ETSI][4] = 34, [1][1][RTW89_MKK][4] = 34, - [1][1][RTW89_IC][4] = 74, - [1][1][RTW89_ACMA][4] = 32, - [1][1][RTW89_FCC][5] = 66, + [1][1][RTW89_IC][4] = 70, + [1][1][RTW89_KCC][4] = 38, + [1][1][RTW89_ACMA][4] = 34, + [1][1][RTW89_CN][4] = 32, + [1][1][RTW89_UK][4] = 34, + [1][1][RTW89_FCC][5] = 82, [1][1][RTW89_ETSI][5] = 34, [1][1][RTW89_MKK][5] = 34, - [1][1][RTW89_IC][5] = 74, - [1][1][RTW89_ACMA][5] = 32, - [1][1][RTW89_FCC][6] = 58, + [1][1][RTW89_IC][5] = 82, + [1][1][RTW89_KCC][5] = 38, + [1][1][RTW89_ACMA][5] = 34, + [1][1][RTW89_CN][5] = 32, + [1][1][RTW89_UK][5] = 34, + [1][1][RTW89_FCC][6] = 60, [1][1][RTW89_ETSI][6] = 34, [1][1][RTW89_MKK][6] = 34, - [1][1][RTW89_IC][6] = 74, - [1][1][RTW89_ACMA][6] = 32, - [1][1][RTW89_FCC][7] = 54, + [1][1][RTW89_IC][6] = 60, + [1][1][RTW89_KCC][6] = 38, + [1][1][RTW89_ACMA][6] = 34, + [1][1][RTW89_CN][6] = 32, + [1][1][RTW89_UK][6] = 34, + [1][1][RTW89_FCC][7] = 56, [1][1][RTW89_ETSI][7] = 34, [1][1][RTW89_MKK][7] = 34, - [1][1][RTW89_IC][7] = 74, - [1][1][RTW89_ACMA][7] = 32, - [1][1][RTW89_FCC][8] = 50, + [1][1][RTW89_IC][7] = 56, + [1][1][RTW89_KCC][7] = 38, + [1][1][RTW89_ACMA][7] = 34, + [1][1][RTW89_CN][7] = 32, + [1][1][RTW89_UK][7] = 34, + [1][1][RTW89_FCC][8] = 52, [1][1][RTW89_ETSI][8] = 34, [1][1][RTW89_MKK][8] = 34, - [1][1][RTW89_IC][8] = 70, - [1][1][RTW89_ACMA][8] = 32, - [1][1][RTW89_FCC][9] = 46, + [1][1][RTW89_IC][8] = 52, + [1][1][RTW89_KCC][8] = 38, + [1][1][RTW89_ACMA][8] = 34, + [1][1][RTW89_CN][8] = 32, + [1][1][RTW89_UK][8] = 34, + [1][1][RTW89_FCC][9] = 48, [1][1][RTW89_ETSI][9] = 34, [1][1][RTW89_MKK][9] = 34, - [1][1][RTW89_IC][9] = 66, - [1][1][RTW89_ACMA][9] = 32, - [1][1][RTW89_FCC][10] = 46, + [1][1][RTW89_IC][9] = 48, + [1][1][RTW89_KCC][9] = 38, + [1][1][RTW89_ACMA][9] = 34, + [1][1][RTW89_CN][9] = 32, + [1][1][RTW89_UK][9] = 34, + [1][1][RTW89_FCC][10] = 48, [1][1][RTW89_ETSI][10] = 34, [1][1][RTW89_MKK][10] = 34, - [1][1][RTW89_IC][10] = 66, - [1][1][RTW89_ACMA][10] = 32, + [1][1][RTW89_IC][10] = 48, + [1][1][RTW89_KCC][10] = 38, + [1][1][RTW89_ACMA][10] = 34, + [1][1][RTW89_CN][10] = 32, + [1][1][RTW89_UK][10] = 34, [1][1][RTW89_FCC][11] = 30, [1][1][RTW89_ETSI][11] = 34, [1][1][RTW89_MKK][11] = 34, - [1][1][RTW89_IC][11] = 48, - [1][1][RTW89_ACMA][11] = 32, - [1][1][RTW89_FCC][12] = 24, + [1][1][RTW89_IC][11] = 30, + [1][1][RTW89_KCC][11] = 38, + [1][1][RTW89_ACMA][11] = 34, + [1][1][RTW89_CN][11] = 32, + [1][1][RTW89_UK][11] = 34, + [1][1][RTW89_FCC][12] = -6, [1][1][RTW89_ETSI][12] = 34, [1][1][RTW89_MKK][12] = 34, - [1][1][RTW89_IC][12] = 32, - [1][1][RTW89_ACMA][12] = 32, + [1][1][RTW89_IC][12] = -6, + [1][1][RTW89_KCC][12] = 38, + [1][1][RTW89_ACMA][12] = 34, + [1][1][RTW89_CN][12] = 32, + [1][1][RTW89_UK][12] = 34, [1][1][RTW89_FCC][13] = 127, [1][1][RTW89_ETSI][13] = 127, [1][1][RTW89_MKK][13] = 127, [1][1][RTW89_IC][13] = 127, + [1][1][RTW89_KCC][13] = 127, [1][1][RTW89_ACMA][13] = 127, - [2][0][RTW89_FCC][0] = 64, + [1][1][RTW89_CN][13] = 127, + [1][1][RTW89_UK][13] = 127, + [2][0][RTW89_FCC][0] = 70, [2][0][RTW89_ETSI][0] = 58, [2][0][RTW89_MKK][0] = 58, - [2][0][RTW89_IC][0] = 78, - [2][0][RTW89_ACMA][0] = 56, - [2][0][RTW89_FCC][1] = 64, + [2][0][RTW89_IC][0] = 70, + [2][0][RTW89_KCC][0] = 64, + [2][0][RTW89_ACMA][0] = 58, + [2][0][RTW89_CN][0] = 56, + [2][0][RTW89_UK][0] = 58, + [2][0][RTW89_FCC][1] = 70, [2][0][RTW89_ETSI][1] = 58, [2][0][RTW89_MKK][1] = 58, - [2][0][RTW89_IC][1] = 78, - [2][0][RTW89_ACMA][1] = 56, - [2][0][RTW89_FCC][2] = 66, + [2][0][RTW89_IC][1] = 70, + [2][0][RTW89_KCC][1] = 64, + [2][0][RTW89_ACMA][1] = 58, + [2][0][RTW89_CN][1] = 56, + [2][0][RTW89_UK][1] = 58, + [2][0][RTW89_FCC][2] = 72, [2][0][RTW89_ETSI][2] = 58, [2][0][RTW89_MKK][2] = 58, - [2][0][RTW89_IC][2] = 80, - [2][0][RTW89_ACMA][2] = 56, - [2][0][RTW89_FCC][3] = 66, + [2][0][RTW89_IC][2] = 72, + [2][0][RTW89_KCC][2] = 64, + [2][0][RTW89_ACMA][2] = 58, + [2][0][RTW89_CN][2] = 56, + [2][0][RTW89_UK][2] = 58, + [2][0][RTW89_FCC][3] = 72, [2][0][RTW89_ETSI][3] = 58, [2][0][RTW89_MKK][3] = 58, - [2][0][RTW89_IC][3] = 80, - [2][0][RTW89_ACMA][3] = 56, - [2][0][RTW89_FCC][4] = 66, + [2][0][RTW89_IC][3] = 72, + [2][0][RTW89_KCC][3] = 64, + [2][0][RTW89_ACMA][3] = 58, + [2][0][RTW89_CN][3] = 56, + [2][0][RTW89_UK][3] = 58, + [2][0][RTW89_FCC][4] = 72, [2][0][RTW89_ETSI][4] = 58, [2][0][RTW89_MKK][4] = 58, - [2][0][RTW89_IC][4] = 80, - [2][0][RTW89_ACMA][4] = 56, - [2][0][RTW89_FCC][5] = 76, + [2][0][RTW89_IC][4] = 72, + [2][0][RTW89_KCC][4] = 64, + [2][0][RTW89_ACMA][4] = 58, + [2][0][RTW89_CN][4] = 56, + [2][0][RTW89_UK][4] = 58, + [2][0][RTW89_FCC][5] = 82, [2][0][RTW89_ETSI][5] = 58, [2][0][RTW89_MKK][5] = 58, - [2][0][RTW89_IC][5] = 84, - [2][0][RTW89_ACMA][5] = 56, - [2][0][RTW89_FCC][6] = 62, + [2][0][RTW89_IC][5] = 82, + [2][0][RTW89_KCC][5] = 64, + [2][0][RTW89_ACMA][5] = 58, + [2][0][RTW89_CN][5] = 56, + [2][0][RTW89_UK][5] = 58, + [2][0][RTW89_FCC][6] = 66, [2][0][RTW89_ETSI][6] = 56, [2][0][RTW89_MKK][6] = 58, - [2][0][RTW89_IC][6] = 76, + [2][0][RTW89_IC][6] = 66, + [2][0][RTW89_KCC][6] = 64, [2][0][RTW89_ACMA][6] = 56, - [2][0][RTW89_FCC][7] = 62, + [2][0][RTW89_CN][6] = 56, + [2][0][RTW89_UK][6] = 56, + [2][0][RTW89_FCC][7] = 66, [2][0][RTW89_ETSI][7] = 58, [2][0][RTW89_MKK][7] = 58, - [2][0][RTW89_IC][7] = 76, - [2][0][RTW89_ACMA][7] = 56, - [2][0][RTW89_FCC][8] = 62, + [2][0][RTW89_IC][7] = 66, + [2][0][RTW89_KCC][7] = 64, + [2][0][RTW89_ACMA][7] = 58, + [2][0][RTW89_CN][7] = 56, + [2][0][RTW89_UK][7] = 58, + [2][0][RTW89_FCC][8] = 66, [2][0][RTW89_ETSI][8] = 58, [2][0][RTW89_MKK][8] = 58, - [2][0][RTW89_IC][8] = 76, - [2][0][RTW89_ACMA][8] = 56, - [2][0][RTW89_FCC][9] = 60, + [2][0][RTW89_IC][8] = 66, + [2][0][RTW89_KCC][8] = 64, + [2][0][RTW89_ACMA][8] = 58, + [2][0][RTW89_CN][8] = 56, + [2][0][RTW89_UK][8] = 58, + [2][0][RTW89_FCC][9] = 64, [2][0][RTW89_ETSI][9] = 58, [2][0][RTW89_MKK][9] = 58, - [2][0][RTW89_IC][9] = 74, - [2][0][RTW89_ACMA][9] = 56, - [2][0][RTW89_FCC][10] = 60, + [2][0][RTW89_IC][9] = 64, + [2][0][RTW89_KCC][9] = 64, + [2][0][RTW89_ACMA][9] = 58, + [2][0][RTW89_CN][9] = 56, + [2][0][RTW89_UK][9] = 58, + [2][0][RTW89_FCC][10] = 64, [2][0][RTW89_ETSI][10] = 58, [2][0][RTW89_MKK][10] = 58, - [2][0][RTW89_IC][10] = 74, - [2][0][RTW89_ACMA][10] = 56, - [2][0][RTW89_FCC][11] = 42, + [2][0][RTW89_IC][10] = 64, + [2][0][RTW89_KCC][10] = 64, + [2][0][RTW89_ACMA][10] = 58, + [2][0][RTW89_CN][10] = 56, + [2][0][RTW89_UK][10] = 58, + [2][0][RTW89_FCC][11] = 48, [2][0][RTW89_ETSI][11] = 58, [2][0][RTW89_MKK][11] = 58, - [2][0][RTW89_IC][11] = 66, - [2][0][RTW89_ACMA][11] = 56, - [2][0][RTW89_FCC][12] = 38, + [2][0][RTW89_IC][11] = 48, + [2][0][RTW89_KCC][11] = 64, + [2][0][RTW89_ACMA][11] = 58, + [2][0][RTW89_CN][11] = 56, + [2][0][RTW89_UK][11] = 58, + [2][0][RTW89_FCC][12] = 16, [2][0][RTW89_ETSI][12] = 58, [2][0][RTW89_MKK][12] = 58, - [2][0][RTW89_IC][12] = 56, - [2][0][RTW89_ACMA][12] = 56, + [2][0][RTW89_IC][12] = 16, + [2][0][RTW89_KCC][12] = 64, + [2][0][RTW89_ACMA][12] = 58, + [2][0][RTW89_CN][12] = 56, + [2][0][RTW89_UK][12] = 58, [2][0][RTW89_FCC][13] = 127, [2][0][RTW89_ETSI][13] = 127, [2][0][RTW89_MKK][13] = 127, [2][0][RTW89_IC][13] = 127, + [2][0][RTW89_KCC][13] = 127, [2][0][RTW89_ACMA][13] = 127, - [2][1][RTW89_FCC][0] = 46, + [2][0][RTW89_CN][13] = 127, + [2][0][RTW89_UK][13] = 127, + [2][1][RTW89_FCC][0] = 64, [2][1][RTW89_ETSI][0] = 46, [2][1][RTW89_MKK][0] = 46, - [2][1][RTW89_IC][0] = 70, - [2][1][RTW89_ACMA][0] = 44, - [2][1][RTW89_FCC][1] = 46, + [2][1][RTW89_IC][0] = 64, + [2][1][RTW89_KCC][0] = 52, + [2][1][RTW89_ACMA][0] = 46, + [2][1][RTW89_CN][0] = 44, + [2][1][RTW89_UK][0] = 46, + [2][1][RTW89_FCC][1] = 64, [2][1][RTW89_ETSI][1] = 46, [2][1][RTW89_MKK][1] = 46, - [2][1][RTW89_IC][1] = 70, - [2][1][RTW89_ACMA][1] = 44, - [2][1][RTW89_FCC][2] = 50, + [2][1][RTW89_IC][1] = 64, + [2][1][RTW89_KCC][1] = 52, + [2][1][RTW89_ACMA][1] = 46, + [2][1][RTW89_CN][1] = 44, + [2][1][RTW89_UK][1] = 46, + [2][1][RTW89_FCC][2] = 68, [2][1][RTW89_ETSI][2] = 46, [2][1][RTW89_MKK][2] = 46, - [2][1][RTW89_IC][2] = 74, - [2][1][RTW89_ACMA][2] = 44, - [2][1][RTW89_FCC][3] = 54, + [2][1][RTW89_IC][2] = 68, + [2][1][RTW89_KCC][2] = 52, + [2][1][RTW89_ACMA][2] = 46, + [2][1][RTW89_CN][2] = 44, + [2][1][RTW89_UK][2] = 46, + [2][1][RTW89_FCC][3] = 72, [2][1][RTW89_ETSI][3] = 46, [2][1][RTW89_MKK][3] = 46, - [2][1][RTW89_IC][3] = 78, - [2][1][RTW89_ACMA][3] = 44, - [2][1][RTW89_FCC][4] = 56, + [2][1][RTW89_IC][3] = 72, + [2][1][RTW89_KCC][3] = 52, + [2][1][RTW89_ACMA][3] = 46, + [2][1][RTW89_CN][3] = 44, + [2][1][RTW89_UK][3] = 46, + [2][1][RTW89_FCC][4] = 74, [2][1][RTW89_ETSI][4] = 46, [2][1][RTW89_MKK][4] = 46, - [2][1][RTW89_IC][4] = 80, - [2][1][RTW89_ACMA][4] = 44, - [2][1][RTW89_FCC][5] = 72, + [2][1][RTW89_IC][4] = 74, + [2][1][RTW89_KCC][4] = 50, + [2][1][RTW89_ACMA][4] = 46, + [2][1][RTW89_CN][4] = 44, + [2][1][RTW89_UK][4] = 46, + [2][1][RTW89_FCC][5] = 82, [2][1][RTW89_ETSI][5] = 46, [2][1][RTW89_MKK][5] = 46, - [2][1][RTW89_IC][5] = 80, - [2][1][RTW89_ACMA][5] = 44, - [2][1][RTW89_FCC][6] = 54, + [2][1][RTW89_IC][5] = 82, + [2][1][RTW89_KCC][5] = 50, + [2][1][RTW89_ACMA][5] = 46, + [2][1][RTW89_CN][5] = 44, + [2][1][RTW89_UK][5] = 46, + [2][1][RTW89_FCC][6] = 72, [2][1][RTW89_ETSI][6] = 44, [2][1][RTW89_MKK][6] = 46, - [2][1][RTW89_IC][6] = 78, + [2][1][RTW89_IC][6] = 72, + [2][1][RTW89_KCC][6] = 50, [2][1][RTW89_ACMA][6] = 44, - [2][1][RTW89_FCC][7] = 54, + [2][1][RTW89_CN][6] = 44, + [2][1][RTW89_UK][6] = 44, + [2][1][RTW89_FCC][7] = 72, [2][1][RTW89_ETSI][7] = 46, [2][1][RTW89_MKK][7] = 46, - [2][1][RTW89_IC][7] = 78, - [2][1][RTW89_ACMA][7] = 44, - [2][1][RTW89_FCC][8] = 50, + [2][1][RTW89_IC][7] = 72, + [2][1][RTW89_KCC][7] = 50, + [2][1][RTW89_ACMA][7] = 46, + [2][1][RTW89_CN][7] = 44, + [2][1][RTW89_UK][7] = 46, + [2][1][RTW89_FCC][8] = 68, [2][1][RTW89_ETSI][8] = 46, [2][1][RTW89_MKK][8] = 46, - [2][1][RTW89_IC][8] = 74, - [2][1][RTW89_ACMA][8] = 44, - [2][1][RTW89_FCC][9] = 46, + [2][1][RTW89_IC][8] = 68, + [2][1][RTW89_KCC][8] = 50, + [2][1][RTW89_ACMA][8] = 46, + [2][1][RTW89_CN][8] = 44, + [2][1][RTW89_UK][8] = 46, + [2][1][RTW89_FCC][9] = 64, [2][1][RTW89_ETSI][9] = 46, [2][1][RTW89_MKK][9] = 46, - [2][1][RTW89_IC][9] = 70, - [2][1][RTW89_ACMA][9] = 44, - [2][1][RTW89_FCC][10] = 46, + [2][1][RTW89_IC][9] = 64, + [2][1][RTW89_KCC][9] = 52, + [2][1][RTW89_ACMA][9] = 46, + [2][1][RTW89_CN][9] = 44, + [2][1][RTW89_UK][9] = 46, + [2][1][RTW89_FCC][10] = 64, [2][1][RTW89_ETSI][10] = 46, [2][1][RTW89_MKK][10] = 46, - [2][1][RTW89_IC][10] = 70, - [2][1][RTW89_ACMA][10] = 44, - [2][1][RTW89_FCC][11] = 30, + [2][1][RTW89_IC][10] = 64, + [2][1][RTW89_KCC][10] = 52, + [2][1][RTW89_ACMA][10] = 46, + [2][1][RTW89_CN][10] = 44, + [2][1][RTW89_UK][10] = 46, + [2][1][RTW89_FCC][11] = 46, [2][1][RTW89_ETSI][11] = 46, [2][1][RTW89_MKK][11] = 46, - [2][1][RTW89_IC][11] = 60, - [2][1][RTW89_ACMA][11] = 44, - [2][1][RTW89_FCC][12] = 26, + [2][1][RTW89_IC][11] = 46, + [2][1][RTW89_KCC][11] = 52, + [2][1][RTW89_ACMA][11] = 46, + [2][1][RTW89_CN][11] = 44, + [2][1][RTW89_UK][11] = 46, + [2][1][RTW89_FCC][12] = 6, [2][1][RTW89_ETSI][12] = 44, [2][1][RTW89_MKK][12] = 46, - [2][1][RTW89_IC][12] = 44, - [2][1][RTW89_ACMA][12] = 42, + [2][1][RTW89_IC][12] = 6, + [2][1][RTW89_KCC][12] = 52, + [2][1][RTW89_ACMA][12] = 44, + [2][1][RTW89_CN][12] = 42, + [2][1][RTW89_UK][12] = 44, [2][1][RTW89_FCC][13] = 127, [2][1][RTW89_ETSI][13] = 127, [2][1][RTW89_MKK][13] = 127, [2][1][RTW89_IC][13] = 127, + [2][1][RTW89_KCC][13] = 127, [2][1][RTW89_ACMA][13] = 127, + [2][1][RTW89_CN][13] = 127, + [2][1][RTW89_UK][13] = 127, }; const s8 rtw89_8852c_txpwr_lmt_ru_5g[RTW89_RU_NUM][RTW89_NTX_NUM] [RTW89_REGD_NUM][RTW89_5G_CH_NUM] = { - [0][0][RTW89_WW][0] = 24, - [0][0][RTW89_WW][2] = 24, - [0][0][RTW89_WW][4] = 22, - [0][0][RTW89_WW][6] = 22, - [0][0][RTW89_WW][8] = 18, - [0][0][RTW89_WW][10] = 18, - [0][0][RTW89_WW][12] = 24, - [0][0][RTW89_WW][14] = 24, - [0][0][RTW89_WW][15] = 24, - [0][0][RTW89_WW][17] = 24, - [0][0][RTW89_WW][19] = 24, - [0][0][RTW89_WW][21] = 24, - [0][0][RTW89_WW][23] = 24, + [0][0][RTW89_WW][0] = 16, + [0][0][RTW89_WW][2] = 16, + [0][0][RTW89_WW][4] = 16, + [0][0][RTW89_WW][6] = 10, + [0][0][RTW89_WW][8] = 16, + [0][0][RTW89_WW][10] = 16, + [0][0][RTW89_WW][12] = 16, + [0][0][RTW89_WW][14] = 16, + [0][0][RTW89_WW][15] = 30, + [0][0][RTW89_WW][17] = 30, + [0][0][RTW89_WW][19] = 30, + [0][0][RTW89_WW][21] = 30, + [0][0][RTW89_WW][23] = 30, [0][0][RTW89_WW][25] = 30, [0][0][RTW89_WW][27] = 30, [0][0][RTW89_WW][29] = 30, - [0][0][RTW89_WW][31] = 24, - [0][0][RTW89_WW][33] = 24, - [0][0][RTW89_WW][35] = 24, - [0][0][RTW89_WW][37] = 44, + [0][0][RTW89_WW][31] = 30, + [0][0][RTW89_WW][33] = 30, + [0][0][RTW89_WW][35] = 30, + [0][0][RTW89_WW][37] = 30, [0][0][RTW89_WW][38] = 28, [0][0][RTW89_WW][40] = 28, [0][0][RTW89_WW][42] = 28, [0][0][RTW89_WW][44] = 28, [0][0][RTW89_WW][46] = 28, - [0][0][RTW89_WW][48] = 24, - [0][0][RTW89_WW][50] = 24, - [0][0][RTW89_WW][52] = 24, - [0][1][RTW89_WW][0] = 0, + [0][0][RTW89_WW][48] = 46, + [0][0][RTW89_WW][50] = 44, + [0][0][RTW89_WW][52] = 34, + [0][1][RTW89_WW][0] = 4, [0][1][RTW89_WW][2] = 4, - [0][1][RTW89_WW][4] = 0, - [0][1][RTW89_WW][6] = 0, - [0][1][RTW89_WW][8] = 12, - [0][1][RTW89_WW][10] = 12, - [0][1][RTW89_WW][12] = 12, - [0][1][RTW89_WW][14] = 12, - [0][1][RTW89_WW][15] = 12, - [0][1][RTW89_WW][17] = 12, - [0][1][RTW89_WW][19] = 12, - [0][1][RTW89_WW][21] = 12, - [0][1][RTW89_WW][23] = 12, + [0][1][RTW89_WW][4] = 4, + [0][1][RTW89_WW][6] = 1, + [0][1][RTW89_WW][8] = 4, + [0][1][RTW89_WW][10] = 4, + [0][1][RTW89_WW][12] = 4, + [0][1][RTW89_WW][14] = 4, + [0][1][RTW89_WW][15] = 18, + [0][1][RTW89_WW][17] = 18, + [0][1][RTW89_WW][19] = 18, + [0][1][RTW89_WW][21] = 18, + [0][1][RTW89_WW][23] = 18, [0][1][RTW89_WW][25] = 18, [0][1][RTW89_WW][27] = 16, [0][1][RTW89_WW][29] = 16, - [0][1][RTW89_WW][31] = 12, - [0][1][RTW89_WW][33] = 12, - [0][1][RTW89_WW][35] = 12, - [0][1][RTW89_WW][37] = 30, + [0][1][RTW89_WW][31] = 16, + [0][1][RTW89_WW][33] = 16, + [0][1][RTW89_WW][35] = 16, + [0][1][RTW89_WW][37] = 18, [0][1][RTW89_WW][38] = 16, [0][1][RTW89_WW][40] = 16, [0][1][RTW89_WW][42] = 16, [0][1][RTW89_WW][44] = 16, [0][1][RTW89_WW][46] = 16, - [0][1][RTW89_WW][48] = 12, - [0][1][RTW89_WW][50] = 12, - [0][1][RTW89_WW][52] = 12, - [1][0][RTW89_WW][0] = 34, - [1][0][RTW89_WW][2] = 34, - [1][0][RTW89_WW][4] = 34, - [1][0][RTW89_WW][6] = 34, - [1][0][RTW89_WW][8] = 34, - [1][0][RTW89_WW][10] = 34, - [1][0][RTW89_WW][12] = 34, - [1][0][RTW89_WW][14] = 34, - [1][0][RTW89_WW][15] = 34, - [1][0][RTW89_WW][17] = 34, - [1][0][RTW89_WW][19] = 34, - [1][0][RTW89_WW][21] = 34, - [1][0][RTW89_WW][23] = 34, + [0][1][RTW89_WW][48] = 20, + [0][1][RTW89_WW][50] = 20, + [0][1][RTW89_WW][52] = 8, + [1][0][RTW89_WW][0] = 26, + [1][0][RTW89_WW][2] = 26, + [1][0][RTW89_WW][4] = 26, + [1][0][RTW89_WW][6] = 24, + [1][0][RTW89_WW][8] = 26, + [1][0][RTW89_WW][10] = 26, + [1][0][RTW89_WW][12] = 26, + [1][0][RTW89_WW][14] = 26, + [1][0][RTW89_WW][15] = 40, + [1][0][RTW89_WW][17] = 40, + [1][0][RTW89_WW][19] = 40, + [1][0][RTW89_WW][21] = 40, + [1][0][RTW89_WW][23] = 40, [1][0][RTW89_WW][25] = 40, [1][0][RTW89_WW][27] = 42, [1][0][RTW89_WW][29] = 42, - [1][0][RTW89_WW][31] = 34, - [1][0][RTW89_WW][33] = 34, - [1][0][RTW89_WW][35] = 34, - [1][0][RTW89_WW][37] = 56, + [1][0][RTW89_WW][31] = 42, + [1][0][RTW89_WW][33] = 42, + [1][0][RTW89_WW][35] = 42, + [1][0][RTW89_WW][37] = 42, [1][0][RTW89_WW][38] = 28, [1][0][RTW89_WW][40] = 28, [1][0][RTW89_WW][42] = 28, [1][0][RTW89_WW][44] = 28, [1][0][RTW89_WW][46] = 28, - [1][0][RTW89_WW][48] = 36, - [1][0][RTW89_WW][50] = 36, - [1][0][RTW89_WW][52] = 36, - [1][1][RTW89_WW][0] = 10, + [1][0][RTW89_WW][48] = 56, + [1][0][RTW89_WW][50] = 58, + [1][0][RTW89_WW][52] = 56, + [1][1][RTW89_WW][0] = 14, [1][1][RTW89_WW][2] = 14, - [1][1][RTW89_WW][4] = 10, - [1][1][RTW89_WW][6] = 10, - [1][1][RTW89_WW][8] = 20, - [1][1][RTW89_WW][10] = 20, - [1][1][RTW89_WW][12] = 22, - [1][1][RTW89_WW][14] = 22, - [1][1][RTW89_WW][15] = 22, - [1][1][RTW89_WW][17] = 22, - [1][1][RTW89_WW][19] = 22, - [1][1][RTW89_WW][21] = 22, - [1][1][RTW89_WW][23] = 22, + [1][1][RTW89_WW][4] = 14, + [1][1][RTW89_WW][6] = 8, + [1][1][RTW89_WW][8] = 14, + [1][1][RTW89_WW][10] = 14, + [1][1][RTW89_WW][12] = 14, + [1][1][RTW89_WW][14] = 14, + [1][1][RTW89_WW][15] = 28, + [1][1][RTW89_WW][17] = 28, + [1][1][RTW89_WW][19] = 28, + [1][1][RTW89_WW][21] = 28, + [1][1][RTW89_WW][23] = 28, [1][1][RTW89_WW][25] = 28, [1][1][RTW89_WW][27] = 30, [1][1][RTW89_WW][29] = 30, - [1][1][RTW89_WW][31] = 22, - [1][1][RTW89_WW][33] = 22, - [1][1][RTW89_WW][35] = 22, - [1][1][RTW89_WW][37] = 40, + [1][1][RTW89_WW][31] = 30, + [1][1][RTW89_WW][33] = 30, + [1][1][RTW89_WW][35] = 30, + [1][1][RTW89_WW][37] = 32, [1][1][RTW89_WW][38] = 16, [1][1][RTW89_WW][40] = 16, [1][1][RTW89_WW][42] = 16, [1][1][RTW89_WW][44] = 16, [1][1][RTW89_WW][46] = 16, - [1][1][RTW89_WW][48] = 24, - [1][1][RTW89_WW][50] = 24, - [1][1][RTW89_WW][52] = 24, - [2][0][RTW89_WW][0] = 46, - [2][0][RTW89_WW][2] = 46, - [2][0][RTW89_WW][4] = 46, - [2][0][RTW89_WW][6] = 46, - [2][0][RTW89_WW][8] = 44, - [2][0][RTW89_WW][10] = 44, - [2][0][RTW89_WW][12] = 48, - [2][0][RTW89_WW][14] = 48, - [2][0][RTW89_WW][15] = 48, - [2][0][RTW89_WW][17] = 48, - [2][0][RTW89_WW][19] = 48, - [2][0][RTW89_WW][21] = 48, - [2][0][RTW89_WW][23] = 48, + [1][1][RTW89_WW][48] = 34, + [1][1][RTW89_WW][50] = 34, + [1][1][RTW89_WW][52] = 30, + [2][0][RTW89_WW][0] = 40, + [2][0][RTW89_WW][2] = 40, + [2][0][RTW89_WW][4] = 40, + [2][0][RTW89_WW][6] = 36, + [2][0][RTW89_WW][8] = 40, + [2][0][RTW89_WW][10] = 40, + [2][0][RTW89_WW][12] = 40, + [2][0][RTW89_WW][14] = 40, + [2][0][RTW89_WW][15] = 52, + [2][0][RTW89_WW][17] = 52, + [2][0][RTW89_WW][19] = 52, + [2][0][RTW89_WW][21] = 52, + [2][0][RTW89_WW][23] = 52, [2][0][RTW89_WW][25] = 52, [2][0][RTW89_WW][27] = 52, [2][0][RTW89_WW][29] = 52, - [2][0][RTW89_WW][31] = 48, - [2][0][RTW89_WW][33] = 48, - [2][0][RTW89_WW][35] = 48, - [2][0][RTW89_WW][37] = 62, + [2][0][RTW89_WW][31] = 52, + [2][0][RTW89_WW][33] = 52, + [2][0][RTW89_WW][35] = 52, + [2][0][RTW89_WW][37] = 52, [2][0][RTW89_WW][38] = 28, [2][0][RTW89_WW][40] = 28, [2][0][RTW89_WW][42] = 28, [2][0][RTW89_WW][44] = 28, [2][0][RTW89_WW][46] = 28, - [2][0][RTW89_WW][48] = 48, - [2][0][RTW89_WW][50] = 48, - [2][0][RTW89_WW][52] = 48, - [2][1][RTW89_WW][0] = 20, - [2][1][RTW89_WW][2] = 18, - [2][1][RTW89_WW][4] = 22, - [2][1][RTW89_WW][6] = 22, - [2][1][RTW89_WW][8] = 32, - [2][1][RTW89_WW][10] = 32, - [2][1][RTW89_WW][12] = 36, - [2][1][RTW89_WW][14] = 36, - [2][1][RTW89_WW][15] = 36, - [2][1][RTW89_WW][17] = 36, - [2][1][RTW89_WW][19] = 36, - [2][1][RTW89_WW][21] = 36, - [2][1][RTW89_WW][23] = 36, + [2][0][RTW89_WW][48] = 64, + [2][0][RTW89_WW][50] = 64, + [2][0][RTW89_WW][52] = 64, + [2][1][RTW89_WW][0] = 26, + [2][1][RTW89_WW][2] = 26, + [2][1][RTW89_WW][4] = 26, + [2][1][RTW89_WW][6] = 20, + [2][1][RTW89_WW][8] = 28, + [2][1][RTW89_WW][10] = 28, + [2][1][RTW89_WW][12] = 28, + [2][1][RTW89_WW][14] = 28, + [2][1][RTW89_WW][15] = 40, + [2][1][RTW89_WW][17] = 40, + [2][1][RTW89_WW][19] = 40, + [2][1][RTW89_WW][21] = 40, + [2][1][RTW89_WW][23] = 40, [2][1][RTW89_WW][25] = 40, [2][1][RTW89_WW][27] = 40, [2][1][RTW89_WW][29] = 40, - [2][1][RTW89_WW][31] = 36, - [2][1][RTW89_WW][33] = 36, - [2][1][RTW89_WW][35] = 36, + [2][1][RTW89_WW][31] = 40, + [2][1][RTW89_WW][33] = 40, + [2][1][RTW89_WW][35] = 40, [2][1][RTW89_WW][37] = 42, [2][1][RTW89_WW][38] = 16, [2][1][RTW89_WW][40] = 16, [2][1][RTW89_WW][42] = 16, [2][1][RTW89_WW][44] = 16, [2][1][RTW89_WW][46] = 16, - [2][1][RTW89_WW][48] = 36, - [2][1][RTW89_WW][50] = 36, - [2][1][RTW89_WW][52] = 36, - [0][0][RTW89_FCC][0] = 44, + [2][1][RTW89_WW][48] = 40, + [2][1][RTW89_WW][50] = 40, + [2][1][RTW89_WW][52] = 40, + [0][0][RTW89_FCC][0] = 50, [0][0][RTW89_ETSI][0] = 30, [0][0][RTW89_MKK][0] = 36, - [0][0][RTW89_IC][0] = 24, - [0][0][RTW89_ACMA][0] = 24, - [0][0][RTW89_FCC][2] = 44, + [0][0][RTW89_IC][0] = 32, + [0][0][RTW89_KCC][0] = 42, + [0][0][RTW89_ACMA][0] = 30, + [0][0][RTW89_CN][0] = 16, + [0][0][RTW89_UK][0] = 30, + [0][0][RTW89_FCC][2] = 50, [0][0][RTW89_ETSI][2] = 30, [0][0][RTW89_MKK][2] = 36, - [0][0][RTW89_IC][2] = 24, - [0][0][RTW89_ACMA][2] = 24, - [0][0][RTW89_FCC][4] = 44, + [0][0][RTW89_IC][2] = 32, + [0][0][RTW89_KCC][2] = 42, + [0][0][RTW89_ACMA][2] = 30, + [0][0][RTW89_CN][2] = 16, + [0][0][RTW89_UK][2] = 30, + [0][0][RTW89_FCC][4] = 50, [0][0][RTW89_ETSI][4] = 30, [0][0][RTW89_MKK][4] = 22, - [0][0][RTW89_IC][4] = 24, - [0][0][RTW89_ACMA][4] = 24, - [0][0][RTW89_FCC][6] = 44, + [0][0][RTW89_IC][4] = 32, + [0][0][RTW89_KCC][4] = 42, + [0][0][RTW89_ACMA][4] = 30, + [0][0][RTW89_CN][4] = 16, + [0][0][RTW89_UK][4] = 30, + [0][0][RTW89_FCC][6] = 50, [0][0][RTW89_ETSI][6] = 30, [0][0][RTW89_MKK][6] = 22, - [0][0][RTW89_IC][6] = 24, - [0][0][RTW89_ACMA][6] = 24, - [0][0][RTW89_FCC][8] = 44, + [0][0][RTW89_IC][6] = 32, + [0][0][RTW89_KCC][6] = 10, + [0][0][RTW89_ACMA][6] = 30, + [0][0][RTW89_CN][6] = 16, + [0][0][RTW89_UK][6] = 30, + [0][0][RTW89_FCC][8] = 52, [0][0][RTW89_ETSI][8] = 28, [0][0][RTW89_MKK][8] = 18, [0][0][RTW89_IC][8] = 52, - [0][0][RTW89_ACMA][8] = 24, - [0][0][RTW89_FCC][10] = 44, + [0][0][RTW89_KCC][8] = 44, + [0][0][RTW89_ACMA][8] = 28, + [0][0][RTW89_CN][8] = 16, + [0][0][RTW89_UK][8] = 28, + [0][0][RTW89_FCC][10] = 52, [0][0][RTW89_ETSI][10] = 28, [0][0][RTW89_MKK][10] = 18, [0][0][RTW89_IC][10] = 52, - [0][0][RTW89_ACMA][10] = 24, - [0][0][RTW89_FCC][12] = 44, + [0][0][RTW89_KCC][10] = 44, + [0][0][RTW89_ACMA][10] = 28, + [0][0][RTW89_CN][10] = 16, + [0][0][RTW89_UK][10] = 28, + [0][0][RTW89_FCC][12] = 52, [0][0][RTW89_ETSI][12] = 28, [0][0][RTW89_MKK][12] = 34, [0][0][RTW89_IC][12] = 52, - [0][0][RTW89_ACMA][12] = 24, - [0][0][RTW89_FCC][14] = 44, + [0][0][RTW89_KCC][12] = 40, + [0][0][RTW89_ACMA][12] = 28, + [0][0][RTW89_CN][12] = 16, + [0][0][RTW89_UK][12] = 28, + [0][0][RTW89_FCC][14] = 52, [0][0][RTW89_ETSI][14] = 28, [0][0][RTW89_MKK][14] = 34, [0][0][RTW89_IC][14] = 52, - [0][0][RTW89_ACMA][14] = 24, - [0][0][RTW89_FCC][15] = 44, + [0][0][RTW89_KCC][14] = 40, + [0][0][RTW89_ACMA][14] = 28, + [0][0][RTW89_CN][14] = 16, + [0][0][RTW89_UK][14] = 28, + [0][0][RTW89_FCC][15] = 52, [0][0][RTW89_ETSI][15] = 30, [0][0][RTW89_MKK][15] = 56, [0][0][RTW89_IC][15] = 52, - [0][0][RTW89_ACMA][15] = 24, - [0][0][RTW89_FCC][17] = 44, + [0][0][RTW89_KCC][15] = 42, + [0][0][RTW89_ACMA][15] = 30, + [0][0][RTW89_CN][15] = 127, + [0][0][RTW89_UK][15] = 30, + [0][0][RTW89_FCC][17] = 52, [0][0][RTW89_ETSI][17] = 30, [0][0][RTW89_MKK][17] = 58, [0][0][RTW89_IC][17] = 52, - [0][0][RTW89_ACMA][17] = 24, - [0][0][RTW89_FCC][19] = 44, + [0][0][RTW89_KCC][17] = 42, + [0][0][RTW89_ACMA][17] = 30, + [0][0][RTW89_CN][17] = 127, + [0][0][RTW89_UK][17] = 30, + [0][0][RTW89_FCC][19] = 52, [0][0][RTW89_ETSI][19] = 30, [0][0][RTW89_MKK][19] = 58, [0][0][RTW89_IC][19] = 52, - [0][0][RTW89_ACMA][19] = 24, - [0][0][RTW89_FCC][21] = 44, + [0][0][RTW89_KCC][19] = 42, + [0][0][RTW89_ACMA][19] = 30, + [0][0][RTW89_CN][19] = 127, + [0][0][RTW89_UK][19] = 30, + [0][0][RTW89_FCC][21] = 52, [0][0][RTW89_ETSI][21] = 30, [0][0][RTW89_MKK][21] = 58, [0][0][RTW89_IC][21] = 52, - [0][0][RTW89_ACMA][21] = 24, - [0][0][RTW89_FCC][23] = 44, + [0][0][RTW89_KCC][21] = 42, + [0][0][RTW89_ACMA][21] = 30, + [0][0][RTW89_CN][21] = 127, + [0][0][RTW89_UK][21] = 30, + [0][0][RTW89_FCC][23] = 52, [0][0][RTW89_ETSI][23] = 30, [0][0][RTW89_MKK][23] = 58, [0][0][RTW89_IC][23] = 52, - [0][0][RTW89_ACMA][23] = 24, - [0][0][RTW89_FCC][25] = 44, + [0][0][RTW89_KCC][23] = 42, + [0][0][RTW89_ACMA][23] = 30, + [0][0][RTW89_CN][23] = 127, + [0][0][RTW89_UK][23] = 30, + [0][0][RTW89_FCC][25] = 52, [0][0][RTW89_ETSI][25] = 30, [0][0][RTW89_MKK][25] = 58, [0][0][RTW89_IC][25] = 127, + [0][0][RTW89_KCC][25] = 42, [0][0][RTW89_ACMA][25] = 127, - [0][0][RTW89_FCC][27] = 44, + [0][0][RTW89_CN][25] = 127, + [0][0][RTW89_UK][25] = 30, + [0][0][RTW89_FCC][27] = 52, [0][0][RTW89_ETSI][27] = 30, [0][0][RTW89_MKK][27] = 58, [0][0][RTW89_IC][27] = 127, + [0][0][RTW89_KCC][27] = 42, [0][0][RTW89_ACMA][27] = 127, - [0][0][RTW89_FCC][29] = 44, + [0][0][RTW89_CN][27] = 127, + [0][0][RTW89_UK][27] = 30, + [0][0][RTW89_FCC][29] = 52, [0][0][RTW89_ETSI][29] = 30, [0][0][RTW89_MKK][29] = 58, [0][0][RTW89_IC][29] = 127, + [0][0][RTW89_KCC][29] = 42, [0][0][RTW89_ACMA][29] = 127, - [0][0][RTW89_FCC][31] = 44, + [0][0][RTW89_CN][29] = 127, + [0][0][RTW89_UK][29] = 30, + [0][0][RTW89_FCC][31] = 52, [0][0][RTW89_ETSI][31] = 30, [0][0][RTW89_MKK][31] = 58, - [0][0][RTW89_IC][31] = 52, - [0][0][RTW89_ACMA][31] = 24, + [0][0][RTW89_IC][31] = 44, + [0][0][RTW89_KCC][31] = 42, + [0][0][RTW89_ACMA][31] = 30, + [0][0][RTW89_CN][31] = 127, + [0][0][RTW89_UK][31] = 30, [0][0][RTW89_FCC][33] = 44, [0][0][RTW89_ETSI][33] = 30, [0][0][RTW89_MKK][33] = 58, - [0][0][RTW89_IC][33] = 52, - [0][0][RTW89_ACMA][33] = 24, + [0][0][RTW89_IC][33] = 44, + [0][0][RTW89_KCC][33] = 42, + [0][0][RTW89_ACMA][33] = 30, + [0][0][RTW89_CN][33] = 127, + [0][0][RTW89_UK][33] = 30, [0][0][RTW89_FCC][35] = 44, [0][0][RTW89_ETSI][35] = 30, [0][0][RTW89_MKK][35] = 58, - [0][0][RTW89_IC][35] = 52, - [0][0][RTW89_ACMA][35] = 24, - [0][0][RTW89_FCC][37] = 44, + [0][0][RTW89_IC][35] = 44, + [0][0][RTW89_KCC][35] = 42, + [0][0][RTW89_ACMA][35] = 30, + [0][0][RTW89_CN][35] = 127, + [0][0][RTW89_UK][35] = 30, + [0][0][RTW89_FCC][37] = 52, [0][0][RTW89_ETSI][37] = 127, [0][0][RTW89_MKK][37] = 58, [0][0][RTW89_IC][37] = 52, + [0][0][RTW89_KCC][37] = 42, [0][0][RTW89_ACMA][37] = 52, - [0][0][RTW89_FCC][38] = 76, + [0][0][RTW89_CN][37] = 127, + [0][0][RTW89_UK][37] = 30, + [0][0][RTW89_FCC][38] = 64, [0][0][RTW89_ETSI][38] = 28, [0][0][RTW89_MKK][38] = 127, - [0][0][RTW89_IC][38] = 84, - [0][0][RTW89_ACMA][38] = 84, - [0][0][RTW89_FCC][40] = 76, + [0][0][RTW89_IC][38] = 64, + [0][0][RTW89_KCC][38] = 42, + [0][0][RTW89_ACMA][38] = 64, + [0][0][RTW89_CN][38] = 54, + [0][0][RTW89_UK][38] = 30, + [0][0][RTW89_FCC][40] = 64, [0][0][RTW89_ETSI][40] = 28, [0][0][RTW89_MKK][40] = 127, - [0][0][RTW89_IC][40] = 84, - [0][0][RTW89_ACMA][40] = 84, - [0][0][RTW89_FCC][42] = 76, + [0][0][RTW89_IC][40] = 64, + [0][0][RTW89_KCC][40] = 42, + [0][0][RTW89_ACMA][40] = 64, + [0][0][RTW89_CN][40] = 54, + [0][0][RTW89_UK][40] = 30, + [0][0][RTW89_FCC][42] = 60, [0][0][RTW89_ETSI][42] = 28, [0][0][RTW89_MKK][42] = 127, - [0][0][RTW89_IC][42] = 84, - [0][0][RTW89_ACMA][42] = 84, - [0][0][RTW89_FCC][44] = 76, + [0][0][RTW89_IC][42] = 60, + [0][0][RTW89_KCC][42] = 42, + [0][0][RTW89_ACMA][42] = 60, + [0][0][RTW89_CN][42] = 54, + [0][0][RTW89_UK][42] = 30, + [0][0][RTW89_FCC][44] = 60, [0][0][RTW89_ETSI][44] = 28, [0][0][RTW89_MKK][44] = 127, - [0][0][RTW89_IC][44] = 84, - [0][0][RTW89_ACMA][44] = 84, - [0][0][RTW89_FCC][46] = 76, + [0][0][RTW89_IC][44] = 60, + [0][0][RTW89_KCC][44] = 42, + [0][0][RTW89_ACMA][44] = 60, + [0][0][RTW89_CN][44] = 54, + [0][0][RTW89_UK][44] = 30, + [0][0][RTW89_FCC][46] = 60, [0][0][RTW89_ETSI][46] = 28, [0][0][RTW89_MKK][46] = 127, - [0][0][RTW89_IC][46] = 84, - [0][0][RTW89_ACMA][46] = 84, - [0][0][RTW89_FCC][48] = 24, + [0][0][RTW89_IC][46] = 60, + [0][0][RTW89_KCC][46] = 42, + [0][0][RTW89_ACMA][46] = 60, + [0][0][RTW89_CN][46] = 54, + [0][0][RTW89_UK][46] = 30, + [0][0][RTW89_FCC][48] = 46, [0][0][RTW89_ETSI][48] = 127, [0][0][RTW89_MKK][48] = 127, [0][0][RTW89_IC][48] = 127, + [0][0][RTW89_KCC][48] = 127, [0][0][RTW89_ACMA][48] = 127, - [0][0][RTW89_FCC][50] = 24, + [0][0][RTW89_CN][48] = 127, + [0][0][RTW89_UK][48] = 127, + [0][0][RTW89_FCC][50] = 44, [0][0][RTW89_ETSI][50] = 127, [0][0][RTW89_MKK][50] = 127, [0][0][RTW89_IC][50] = 127, + [0][0][RTW89_KCC][50] = 127, [0][0][RTW89_ACMA][50] = 127, - [0][0][RTW89_FCC][52] = 24, + [0][0][RTW89_CN][50] = 127, + [0][0][RTW89_UK][50] = 127, + [0][0][RTW89_FCC][52] = 34, [0][0][RTW89_ETSI][52] = 127, [0][0][RTW89_MKK][52] = 127, [0][0][RTW89_IC][52] = 127, + [0][0][RTW89_KCC][52] = 127, [0][0][RTW89_ACMA][52] = 127, - [0][1][RTW89_FCC][0] = 26, + [0][0][RTW89_CN][52] = 127, + [0][0][RTW89_UK][52] = 127, + [0][1][RTW89_FCC][0] = 30, [0][1][RTW89_ETSI][0] = 18, [0][1][RTW89_MKK][0] = 20, - [0][1][RTW89_IC][0] = 0, - [0][1][RTW89_ACMA][0] = 12, - [0][1][RTW89_FCC][2] = 30, + [0][1][RTW89_IC][0] = 8, + [0][1][RTW89_KCC][0] = 26, + [0][1][RTW89_ACMA][0] = 18, + [0][1][RTW89_CN][0] = 4, + [0][1][RTW89_UK][0] = 18, + [0][1][RTW89_FCC][2] = 32, [0][1][RTW89_ETSI][2] = 18, [0][1][RTW89_MKK][2] = 20, - [0][1][RTW89_IC][2] = 4, - [0][1][RTW89_ACMA][2] = 12, - [0][1][RTW89_FCC][4] = 26, + [0][1][RTW89_IC][2] = 8, + [0][1][RTW89_KCC][2] = 26, + [0][1][RTW89_ACMA][2] = 18, + [0][1][RTW89_CN][2] = 4, + [0][1][RTW89_UK][2] = 18, + [0][1][RTW89_FCC][4] = 30, [0][1][RTW89_ETSI][4] = 18, [0][1][RTW89_MKK][4] = 8, - [0][1][RTW89_IC][4] = 0, - [0][1][RTW89_ACMA][4] = 12, - [0][1][RTW89_FCC][6] = 26, + [0][1][RTW89_IC][4] = 8, + [0][1][RTW89_KCC][4] = 26, + [0][1][RTW89_ACMA][4] = 18, + [0][1][RTW89_CN][4] = 4, + [0][1][RTW89_UK][4] = 18, + [0][1][RTW89_FCC][6] = 30, [0][1][RTW89_ETSI][6] = 18, [0][1][RTW89_MKK][6] = 8, - [0][1][RTW89_IC][6] = 0, - [0][1][RTW89_ACMA][6] = 12, - [0][1][RTW89_FCC][8] = 26, + [0][1][RTW89_IC][6] = 8, + [0][1][RTW89_KCC][6] = 0, + [0][1][RTW89_ACMA][6] = 18, + [0][1][RTW89_CN][6] = 4, + [0][1][RTW89_UK][6] = 18, + [0][1][RTW89_FCC][8] = 30, [0][1][RTW89_ETSI][8] = 16, [0][1][RTW89_MKK][8] = 20, - [0][1][RTW89_IC][8] = 34, - [0][1][RTW89_ACMA][8] = 12, - [0][1][RTW89_FCC][10] = 26, + [0][1][RTW89_IC][8] = 30, + [0][1][RTW89_KCC][8] = 28, + [0][1][RTW89_ACMA][8] = 16, + [0][1][RTW89_CN][8] = 4, + [0][1][RTW89_UK][8] = 16, + [0][1][RTW89_FCC][10] = 30, [0][1][RTW89_ETSI][10] = 16, [0][1][RTW89_MKK][10] = 20, - [0][1][RTW89_IC][10] = 34, - [0][1][RTW89_ACMA][10] = 12, + [0][1][RTW89_IC][10] = 30, + [0][1][RTW89_KCC][10] = 28, + [0][1][RTW89_ACMA][10] = 16, + [0][1][RTW89_CN][10] = 4, + [0][1][RTW89_UK][10] = 16, [0][1][RTW89_FCC][12] = 30, [0][1][RTW89_ETSI][12] = 16, [0][1][RTW89_MKK][12] = 34, - [0][1][RTW89_IC][12] = 38, - [0][1][RTW89_ACMA][12] = 12, - [0][1][RTW89_FCC][14] = 26, + [0][1][RTW89_IC][12] = 30, + [0][1][RTW89_KCC][12] = 28, + [0][1][RTW89_ACMA][12] = 16, + [0][1][RTW89_CN][12] = 4, + [0][1][RTW89_UK][12] = 16, + [0][1][RTW89_FCC][14] = 30, [0][1][RTW89_ETSI][14] = 16, [0][1][RTW89_MKK][14] = 34, - [0][1][RTW89_IC][14] = 34, - [0][1][RTW89_ACMA][14] = 12, - [0][1][RTW89_FCC][15] = 26, + [0][1][RTW89_IC][14] = 30, + [0][1][RTW89_KCC][14] = 28, + [0][1][RTW89_ACMA][14] = 16, + [0][1][RTW89_CN][14] = 4, + [0][1][RTW89_UK][14] = 16, + [0][1][RTW89_FCC][15] = 32, [0][1][RTW89_ETSI][15] = 18, [0][1][RTW89_MKK][15] = 44, - [0][1][RTW89_IC][15] = 34, - [0][1][RTW89_ACMA][15] = 12, - [0][1][RTW89_FCC][17] = 26, + [0][1][RTW89_IC][15] = 32, + [0][1][RTW89_KCC][15] = 28, + [0][1][RTW89_ACMA][15] = 18, + [0][1][RTW89_CN][15] = 127, + [0][1][RTW89_UK][15] = 18, + [0][1][RTW89_FCC][17] = 32, [0][1][RTW89_ETSI][17] = 18, [0][1][RTW89_MKK][17] = 44, - [0][1][RTW89_IC][17] = 34, - [0][1][RTW89_ACMA][17] = 12, - [0][1][RTW89_FCC][19] = 30, + [0][1][RTW89_IC][17] = 32, + [0][1][RTW89_KCC][17] = 28, + [0][1][RTW89_ACMA][17] = 18, + [0][1][RTW89_CN][17] = 127, + [0][1][RTW89_UK][17] = 18, + [0][1][RTW89_FCC][19] = 32, [0][1][RTW89_ETSI][19] = 18, [0][1][RTW89_MKK][19] = 44, - [0][1][RTW89_IC][19] = 38, - [0][1][RTW89_ACMA][19] = 12, - [0][1][RTW89_FCC][21] = 30, + [0][1][RTW89_IC][19] = 32, + [0][1][RTW89_KCC][19] = 28, + [0][1][RTW89_ACMA][19] = 18, + [0][1][RTW89_CN][19] = 127, + [0][1][RTW89_UK][19] = 18, + [0][1][RTW89_FCC][21] = 32, [0][1][RTW89_ETSI][21] = 18, [0][1][RTW89_MKK][21] = 44, - [0][1][RTW89_IC][21] = 38, - [0][1][RTW89_ACMA][21] = 12, - [0][1][RTW89_FCC][23] = 30, + [0][1][RTW89_IC][21] = 32, + [0][1][RTW89_KCC][21] = 28, + [0][1][RTW89_ACMA][21] = 18, + [0][1][RTW89_CN][21] = 127, + [0][1][RTW89_UK][21] = 18, + [0][1][RTW89_FCC][23] = 32, [0][1][RTW89_ETSI][23] = 18, [0][1][RTW89_MKK][23] = 44, - [0][1][RTW89_IC][23] = 38, - [0][1][RTW89_ACMA][23] = 12, - [0][1][RTW89_FCC][25] = 30, + [0][1][RTW89_IC][23] = 32, + [0][1][RTW89_KCC][23] = 28, + [0][1][RTW89_ACMA][23] = 18, + [0][1][RTW89_CN][23] = 127, + [0][1][RTW89_UK][23] = 18, + [0][1][RTW89_FCC][25] = 32, [0][1][RTW89_ETSI][25] = 18, [0][1][RTW89_MKK][25] = 44, [0][1][RTW89_IC][25] = 127, + [0][1][RTW89_KCC][25] = 28, [0][1][RTW89_ACMA][25] = 127, - [0][1][RTW89_FCC][27] = 30, + [0][1][RTW89_CN][25] = 127, + [0][1][RTW89_UK][25] = 18, + [0][1][RTW89_FCC][27] = 32, [0][1][RTW89_ETSI][27] = 16, [0][1][RTW89_MKK][27] = 44, [0][1][RTW89_IC][27] = 127, + [0][1][RTW89_KCC][27] = 28, [0][1][RTW89_ACMA][27] = 127, - [0][1][RTW89_FCC][29] = 30, + [0][1][RTW89_CN][27] = 127, + [0][1][RTW89_UK][27] = 16, + [0][1][RTW89_FCC][29] = 32, [0][1][RTW89_ETSI][29] = 16, [0][1][RTW89_MKK][29] = 44, [0][1][RTW89_IC][29] = 127, + [0][1][RTW89_KCC][29] = 28, [0][1][RTW89_ACMA][29] = 127, - [0][1][RTW89_FCC][31] = 30, + [0][1][RTW89_CN][29] = 127, + [0][1][RTW89_UK][29] = 16, + [0][1][RTW89_FCC][31] = 32, [0][1][RTW89_ETSI][31] = 16, [0][1][RTW89_MKK][31] = 44, - [0][1][RTW89_IC][31] = 34, - [0][1][RTW89_ACMA][31] = 12, - [0][1][RTW89_FCC][33] = 26, + [0][1][RTW89_IC][31] = 30, + [0][1][RTW89_KCC][31] = 28, + [0][1][RTW89_ACMA][31] = 16, + [0][1][RTW89_CN][31] = 127, + [0][1][RTW89_UK][31] = 16, + [0][1][RTW89_FCC][33] = 30, [0][1][RTW89_ETSI][33] = 16, [0][1][RTW89_MKK][33] = 44, - [0][1][RTW89_IC][33] = 34, - [0][1][RTW89_ACMA][33] = 12, - [0][1][RTW89_FCC][35] = 26, + [0][1][RTW89_IC][33] = 30, + [0][1][RTW89_KCC][33] = 28, + [0][1][RTW89_ACMA][33] = 16, + [0][1][RTW89_CN][33] = 127, + [0][1][RTW89_UK][33] = 16, + [0][1][RTW89_FCC][35] = 30, [0][1][RTW89_ETSI][35] = 16, [0][1][RTW89_MKK][35] = 44, - [0][1][RTW89_IC][35] = 34, - [0][1][RTW89_ACMA][35] = 12, - [0][1][RTW89_FCC][37] = 30, + [0][1][RTW89_IC][35] = 30, + [0][1][RTW89_KCC][35] = 28, + [0][1][RTW89_ACMA][35] = 16, + [0][1][RTW89_CN][35] = 127, + [0][1][RTW89_UK][35] = 16, + [0][1][RTW89_FCC][37] = 34, [0][1][RTW89_ETSI][37] = 127, [0][1][RTW89_MKK][37] = 44, - [0][1][RTW89_IC][37] = 38, - [0][1][RTW89_ACMA][37] = 38, - [0][1][RTW89_FCC][38] = 74, + [0][1][RTW89_IC][37] = 34, + [0][1][RTW89_KCC][37] = 28, + [0][1][RTW89_ACMA][37] = 34, + [0][1][RTW89_CN][37] = 127, + [0][1][RTW89_UK][37] = 18, + [0][1][RTW89_FCC][38] = 62, [0][1][RTW89_ETSI][38] = 16, [0][1][RTW89_MKK][38] = 127, - [0][1][RTW89_IC][38] = 82, - [0][1][RTW89_ACMA][38] = 84, - [0][1][RTW89_FCC][40] = 74, + [0][1][RTW89_IC][38] = 62, + [0][1][RTW89_KCC][38] = 28, + [0][1][RTW89_ACMA][38] = 62, + [0][1][RTW89_CN][38] = 42, + [0][1][RTW89_UK][38] = 18, + [0][1][RTW89_FCC][40] = 62, [0][1][RTW89_ETSI][40] = 16, [0][1][RTW89_MKK][40] = 127, - [0][1][RTW89_IC][40] = 82, - [0][1][RTW89_ACMA][40] = 84, - [0][1][RTW89_FCC][42] = 74, + [0][1][RTW89_IC][40] = 62, + [0][1][RTW89_KCC][40] = 28, + [0][1][RTW89_ACMA][40] = 62, + [0][1][RTW89_CN][40] = 42, + [0][1][RTW89_UK][40] = 18, + [0][1][RTW89_FCC][42] = 58, [0][1][RTW89_ETSI][42] = 16, [0][1][RTW89_MKK][42] = 127, - [0][1][RTW89_IC][42] = 82, - [0][1][RTW89_ACMA][42] = 84, - [0][1][RTW89_FCC][44] = 74, + [0][1][RTW89_IC][42] = 58, + [0][1][RTW89_KCC][42] = 28, + [0][1][RTW89_ACMA][42] = 58, + [0][1][RTW89_CN][42] = 42, + [0][1][RTW89_UK][42] = 18, + [0][1][RTW89_FCC][44] = 56, [0][1][RTW89_ETSI][44] = 16, [0][1][RTW89_MKK][44] = 127, - [0][1][RTW89_IC][44] = 82, - [0][1][RTW89_ACMA][44] = 84, - [0][1][RTW89_FCC][46] = 74, + [0][1][RTW89_IC][44] = 56, + [0][1][RTW89_KCC][44] = 28, + [0][1][RTW89_ACMA][44] = 56, + [0][1][RTW89_CN][44] = 42, + [0][1][RTW89_UK][44] = 18, + [0][1][RTW89_FCC][46] = 56, [0][1][RTW89_ETSI][46] = 16, [0][1][RTW89_MKK][46] = 127, - [0][1][RTW89_IC][46] = 82, - [0][1][RTW89_ACMA][46] = 84, - [0][1][RTW89_FCC][48] = 12, + [0][1][RTW89_IC][46] = 56, + [0][1][RTW89_KCC][46] = 28, + [0][1][RTW89_ACMA][46] = 56, + [0][1][RTW89_CN][46] = 42, + [0][1][RTW89_UK][46] = 18, + [0][1][RTW89_FCC][48] = 20, [0][1][RTW89_ETSI][48] = 127, [0][1][RTW89_MKK][48] = 127, [0][1][RTW89_IC][48] = 127, + [0][1][RTW89_KCC][48] = 127, [0][1][RTW89_ACMA][48] = 127, - [0][1][RTW89_FCC][50] = 12, + [0][1][RTW89_CN][48] = 127, + [0][1][RTW89_UK][48] = 127, + [0][1][RTW89_FCC][50] = 20, [0][1][RTW89_ETSI][50] = 127, [0][1][RTW89_MKK][50] = 127, [0][1][RTW89_IC][50] = 127, + [0][1][RTW89_KCC][50] = 127, [0][1][RTW89_ACMA][50] = 127, - [0][1][RTW89_FCC][52] = 12, + [0][1][RTW89_CN][50] = 127, + [0][1][RTW89_UK][50] = 127, + [0][1][RTW89_FCC][52] = 8, [0][1][RTW89_ETSI][52] = 127, [0][1][RTW89_MKK][52] = 127, [0][1][RTW89_IC][52] = 127, + [0][1][RTW89_KCC][52] = 127, [0][1][RTW89_ACMA][52] = 127, - [1][0][RTW89_FCC][0] = 54, + [0][1][RTW89_CN][52] = 127, + [0][1][RTW89_UK][52] = 127, + [1][0][RTW89_FCC][0] = 62, [1][0][RTW89_ETSI][0] = 40, [1][0][RTW89_MKK][0] = 48, - [1][0][RTW89_IC][0] = 36, - [1][0][RTW89_ACMA][0] = 34, - [1][0][RTW89_FCC][2] = 54, + [1][0][RTW89_IC][0] = 42, + [1][0][RTW89_KCC][0] = 50, + [1][0][RTW89_ACMA][0] = 40, + [1][0][RTW89_CN][0] = 26, + [1][0][RTW89_UK][0] = 40, + [1][0][RTW89_FCC][2] = 62, [1][0][RTW89_ETSI][2] = 40, [1][0][RTW89_MKK][2] = 48, - [1][0][RTW89_IC][2] = 36, - [1][0][RTW89_ACMA][2] = 34, - [1][0][RTW89_FCC][4] = 54, + [1][0][RTW89_IC][2] = 42, + [1][0][RTW89_KCC][2] = 50, + [1][0][RTW89_ACMA][2] = 40, + [1][0][RTW89_CN][2] = 26, + [1][0][RTW89_UK][2] = 40, + [1][0][RTW89_FCC][4] = 64, [1][0][RTW89_ETSI][4] = 40, [1][0][RTW89_MKK][4] = 40, - [1][0][RTW89_IC][4] = 36, - [1][0][RTW89_ACMA][4] = 34, - [1][0][RTW89_FCC][6] = 54, + [1][0][RTW89_IC][4] = 42, + [1][0][RTW89_KCC][4] = 50, + [1][0][RTW89_ACMA][4] = 40, + [1][0][RTW89_CN][4] = 26, + [1][0][RTW89_UK][4] = 40, + [1][0][RTW89_FCC][6] = 64, [1][0][RTW89_ETSI][6] = 40, [1][0][RTW89_MKK][6] = 40, - [1][0][RTW89_IC][6] = 36, - [1][0][RTW89_ACMA][6] = 34, - [1][0][RTW89_FCC][8] = 54, + [1][0][RTW89_IC][6] = 42, + [1][0][RTW89_KCC][6] = 24, + [1][0][RTW89_ACMA][6] = 40, + [1][0][RTW89_CN][6] = 26, + [1][0][RTW89_UK][6] = 40, + [1][0][RTW89_FCC][8] = 62, [1][0][RTW89_ETSI][8] = 40, [1][0][RTW89_MKK][8] = 34, [1][0][RTW89_IC][8] = 62, - [1][0][RTW89_ACMA][8] = 34, - [1][0][RTW89_FCC][10] = 54, + [1][0][RTW89_KCC][8] = 52, + [1][0][RTW89_ACMA][8] = 40, + [1][0][RTW89_CN][8] = 26, + [1][0][RTW89_UK][8] = 40, + [1][0][RTW89_FCC][10] = 62, [1][0][RTW89_ETSI][10] = 40, [1][0][RTW89_MKK][10] = 34, [1][0][RTW89_IC][10] = 62, - [1][0][RTW89_ACMA][10] = 34, - [1][0][RTW89_FCC][12] = 56, + [1][0][RTW89_KCC][10] = 52, + [1][0][RTW89_ACMA][10] = 40, + [1][0][RTW89_CN][10] = 26, + [1][0][RTW89_UK][10] = 40, + [1][0][RTW89_FCC][12] = 62, [1][0][RTW89_ETSI][12] = 40, [1][0][RTW89_MKK][12] = 46, - [1][0][RTW89_IC][12] = 64, - [1][0][RTW89_ACMA][12] = 34, - [1][0][RTW89_FCC][14] = 54, + [1][0][RTW89_IC][12] = 62, + [1][0][RTW89_KCC][12] = 52, + [1][0][RTW89_ACMA][12] = 40, + [1][0][RTW89_CN][12] = 26, + [1][0][RTW89_UK][12] = 40, + [1][0][RTW89_FCC][14] = 62, [1][0][RTW89_ETSI][14] = 40, [1][0][RTW89_MKK][14] = 46, [1][0][RTW89_IC][14] = 62, - [1][0][RTW89_ACMA][14] = 34, - [1][0][RTW89_FCC][15] = 54, + [1][0][RTW89_KCC][14] = 52, + [1][0][RTW89_ACMA][14] = 40, + [1][0][RTW89_CN][14] = 26, + [1][0][RTW89_UK][14] = 40, + [1][0][RTW89_FCC][15] = 62, [1][0][RTW89_ETSI][15] = 40, [1][0][RTW89_MKK][15] = 62, [1][0][RTW89_IC][15] = 62, - [1][0][RTW89_ACMA][15] = 34, - [1][0][RTW89_FCC][17] = 54, + [1][0][RTW89_KCC][15] = 52, + [1][0][RTW89_ACMA][15] = 40, + [1][0][RTW89_CN][15] = 127, + [1][0][RTW89_UK][15] = 40, + [1][0][RTW89_FCC][17] = 62, [1][0][RTW89_ETSI][17] = 40, [1][0][RTW89_MKK][17] = 68, [1][0][RTW89_IC][17] = 62, - [1][0][RTW89_ACMA][17] = 34, - [1][0][RTW89_FCC][19] = 54, + [1][0][RTW89_KCC][17] = 52, + [1][0][RTW89_ACMA][17] = 40, + [1][0][RTW89_CN][17] = 127, + [1][0][RTW89_UK][17] = 40, + [1][0][RTW89_FCC][19] = 64, [1][0][RTW89_ETSI][19] = 40, [1][0][RTW89_MKK][19] = 68, - [1][0][RTW89_IC][19] = 62, - [1][0][RTW89_ACMA][19] = 34, - [1][0][RTW89_FCC][21] = 54, + [1][0][RTW89_IC][19] = 64, + [1][0][RTW89_KCC][19] = 52, + [1][0][RTW89_ACMA][19] = 40, + [1][0][RTW89_CN][19] = 127, + [1][0][RTW89_UK][19] = 40, + [1][0][RTW89_FCC][21] = 64, [1][0][RTW89_ETSI][21] = 40, [1][0][RTW89_MKK][21] = 68, - [1][0][RTW89_IC][21] = 62, - [1][0][RTW89_ACMA][21] = 34, - [1][0][RTW89_FCC][23] = 54, + [1][0][RTW89_IC][21] = 64, + [1][0][RTW89_KCC][21] = 52, + [1][0][RTW89_ACMA][21] = 40, + [1][0][RTW89_CN][21] = 127, + [1][0][RTW89_UK][21] = 40, + [1][0][RTW89_FCC][23] = 64, [1][0][RTW89_ETSI][23] = 40, [1][0][RTW89_MKK][23] = 68, - [1][0][RTW89_IC][23] = 62, - [1][0][RTW89_ACMA][23] = 34, - [1][0][RTW89_FCC][25] = 54, + [1][0][RTW89_IC][23] = 64, + [1][0][RTW89_KCC][23] = 52, + [1][0][RTW89_ACMA][23] = 40, + [1][0][RTW89_CN][23] = 127, + [1][0][RTW89_UK][23] = 40, + [1][0][RTW89_FCC][25] = 64, [1][0][RTW89_ETSI][25] = 40, [1][0][RTW89_MKK][25] = 68, [1][0][RTW89_IC][25] = 127, + [1][0][RTW89_KCC][25] = 52, [1][0][RTW89_ACMA][25] = 127, - [1][0][RTW89_FCC][27] = 54, + [1][0][RTW89_CN][25] = 127, + [1][0][RTW89_UK][25] = 40, + [1][0][RTW89_FCC][27] = 64, [1][0][RTW89_ETSI][27] = 42, [1][0][RTW89_MKK][27] = 68, [1][0][RTW89_IC][27] = 127, + [1][0][RTW89_KCC][27] = 52, [1][0][RTW89_ACMA][27] = 127, - [1][0][RTW89_FCC][29] = 54, + [1][0][RTW89_CN][27] = 127, + [1][0][RTW89_UK][27] = 42, + [1][0][RTW89_FCC][29] = 64, [1][0][RTW89_ETSI][29] = 42, [1][0][RTW89_MKK][29] = 68, [1][0][RTW89_IC][29] = 127, + [1][0][RTW89_KCC][29] = 52, [1][0][RTW89_ACMA][29] = 127, - [1][0][RTW89_FCC][31] = 54, + [1][0][RTW89_CN][29] = 127, + [1][0][RTW89_UK][29] = 42, + [1][0][RTW89_FCC][31] = 64, [1][0][RTW89_ETSI][31] = 42, [1][0][RTW89_MKK][31] = 68, - [1][0][RTW89_IC][31] = 62, - [1][0][RTW89_ACMA][31] = 34, - [1][0][RTW89_FCC][33] = 54, + [1][0][RTW89_IC][31] = 56, + [1][0][RTW89_KCC][31] = 52, + [1][0][RTW89_ACMA][31] = 42, + [1][0][RTW89_CN][31] = 127, + [1][0][RTW89_UK][31] = 42, + [1][0][RTW89_FCC][33] = 56, [1][0][RTW89_ETSI][33] = 42, [1][0][RTW89_MKK][33] = 68, - [1][0][RTW89_IC][33] = 62, - [1][0][RTW89_ACMA][33] = 34, - [1][0][RTW89_FCC][35] = 54, + [1][0][RTW89_IC][33] = 56, + [1][0][RTW89_KCC][33] = 52, + [1][0][RTW89_ACMA][33] = 42, + [1][0][RTW89_CN][33] = 127, + [1][0][RTW89_UK][33] = 42, + [1][0][RTW89_FCC][35] = 56, [1][0][RTW89_ETSI][35] = 42, [1][0][RTW89_MKK][35] = 68, - [1][0][RTW89_IC][35] = 62, - [1][0][RTW89_ACMA][35] = 34, - [1][0][RTW89_FCC][37] = 56, + [1][0][RTW89_IC][35] = 56, + [1][0][RTW89_KCC][35] = 52, + [1][0][RTW89_ACMA][35] = 42, + [1][0][RTW89_CN][35] = 127, + [1][0][RTW89_UK][35] = 42, + [1][0][RTW89_FCC][37] = 66, [1][0][RTW89_ETSI][37] = 127, [1][0][RTW89_MKK][37] = 68, - [1][0][RTW89_IC][37] = 64, - [1][0][RTW89_ACMA][37] = 64, + [1][0][RTW89_IC][37] = 66, + [1][0][RTW89_KCC][37] = 52, + [1][0][RTW89_ACMA][37] = 66, + [1][0][RTW89_CN][37] = 127, + [1][0][RTW89_UK][37] = 42, [1][0][RTW89_FCC][38] = 76, [1][0][RTW89_ETSI][38] = 28, [1][0][RTW89_MKK][38] = 127, - [1][0][RTW89_IC][38] = 84, - [1][0][RTW89_ACMA][38] = 84, + [1][0][RTW89_IC][38] = 76, + [1][0][RTW89_KCC][38] = 54, + [1][0][RTW89_ACMA][38] = 76, + [1][0][RTW89_CN][38] = 66, + [1][0][RTW89_UK][38] = 44, [1][0][RTW89_FCC][40] = 76, [1][0][RTW89_ETSI][40] = 28, [1][0][RTW89_MKK][40] = 127, - [1][0][RTW89_IC][40] = 84, - [1][0][RTW89_ACMA][40] = 84, - [1][0][RTW89_FCC][42] = 76, + [1][0][RTW89_IC][40] = 76, + [1][0][RTW89_KCC][40] = 54, + [1][0][RTW89_ACMA][40] = 76, + [1][0][RTW89_CN][40] = 66, + [1][0][RTW89_UK][40] = 44, + [1][0][RTW89_FCC][42] = 68, [1][0][RTW89_ETSI][42] = 28, [1][0][RTW89_MKK][42] = 127, - [1][0][RTW89_IC][42] = 84, - [1][0][RTW89_ACMA][42] = 84, - [1][0][RTW89_FCC][44] = 76, + [1][0][RTW89_IC][42] = 68, + [1][0][RTW89_KCC][42] = 54, + [1][0][RTW89_ACMA][42] = 68, + [1][0][RTW89_CN][42] = 66, + [1][0][RTW89_UK][42] = 44, + [1][0][RTW89_FCC][44] = 70, [1][0][RTW89_ETSI][44] = 28, [1][0][RTW89_MKK][44] = 127, - [1][0][RTW89_IC][44] = 84, - [1][0][RTW89_ACMA][44] = 84, - [1][0][RTW89_FCC][46] = 76, + [1][0][RTW89_IC][44] = 70, + [1][0][RTW89_KCC][44] = 54, + [1][0][RTW89_ACMA][44] = 70, + [1][0][RTW89_CN][44] = 66, + [1][0][RTW89_UK][44] = 42, + [1][0][RTW89_FCC][46] = 70, [1][0][RTW89_ETSI][46] = 28, [1][0][RTW89_MKK][46] = 127, - [1][0][RTW89_IC][46] = 84, - [1][0][RTW89_ACMA][46] = 84, - [1][0][RTW89_FCC][48] = 36, + [1][0][RTW89_IC][46] = 70, + [1][0][RTW89_KCC][46] = 54, + [1][0][RTW89_ACMA][46] = 70, + [1][0][RTW89_CN][46] = 66, + [1][0][RTW89_UK][46] = 42, + [1][0][RTW89_FCC][48] = 56, [1][0][RTW89_ETSI][48] = 127, [1][0][RTW89_MKK][48] = 127, [1][0][RTW89_IC][48] = 127, + [1][0][RTW89_KCC][48] = 127, [1][0][RTW89_ACMA][48] = 127, - [1][0][RTW89_FCC][50] = 36, + [1][0][RTW89_CN][48] = 127, + [1][0][RTW89_UK][48] = 127, + [1][0][RTW89_FCC][50] = 58, [1][0][RTW89_ETSI][50] = 127, [1][0][RTW89_MKK][50] = 127, [1][0][RTW89_IC][50] = 127, + [1][0][RTW89_KCC][50] = 127, [1][0][RTW89_ACMA][50] = 127, - [1][0][RTW89_FCC][52] = 36, + [1][0][RTW89_CN][50] = 127, + [1][0][RTW89_UK][50] = 127, + [1][0][RTW89_FCC][52] = 56, [1][0][RTW89_ETSI][52] = 127, [1][0][RTW89_MKK][52] = 127, [1][0][RTW89_IC][52] = 127, + [1][0][RTW89_KCC][52] = 127, [1][0][RTW89_ACMA][52] = 127, - [1][1][RTW89_FCC][0] = 34, + [1][0][RTW89_CN][52] = 127, + [1][0][RTW89_UK][52] = 127, + [1][1][RTW89_FCC][0] = 44, [1][1][RTW89_ETSI][0] = 30, [1][1][RTW89_MKK][0] = 34, - [1][1][RTW89_IC][0] = 10, - [1][1][RTW89_ACMA][0] = 22, - [1][1][RTW89_FCC][2] = 36, + [1][1][RTW89_IC][0] = 20, + [1][1][RTW89_KCC][0] = 34, + [1][1][RTW89_ACMA][0] = 30, + [1][1][RTW89_CN][0] = 14, + [1][1][RTW89_UK][0] = 30, + [1][1][RTW89_FCC][2] = 44, [1][1][RTW89_ETSI][2] = 30, [1][1][RTW89_MKK][2] = 34, - [1][1][RTW89_IC][2] = 14, - [1][1][RTW89_ACMA][2] = 22, - [1][1][RTW89_FCC][4] = 34, + [1][1][RTW89_IC][2] = 18, + [1][1][RTW89_KCC][2] = 34, + [1][1][RTW89_ACMA][2] = 30, + [1][1][RTW89_CN][2] = 14, + [1][1][RTW89_UK][2] = 30, + [1][1][RTW89_FCC][4] = 46, [1][1][RTW89_ETSI][4] = 30, [1][1][RTW89_MKK][4] = 26, - [1][1][RTW89_IC][4] = 10, - [1][1][RTW89_ACMA][4] = 22, - [1][1][RTW89_FCC][6] = 34, + [1][1][RTW89_IC][4] = 20, + [1][1][RTW89_KCC][4] = 34, + [1][1][RTW89_ACMA][4] = 30, + [1][1][RTW89_CN][4] = 14, + [1][1][RTW89_UK][4] = 30, + [1][1][RTW89_FCC][6] = 46, [1][1][RTW89_ETSI][6] = 30, [1][1][RTW89_MKK][6] = 26, - [1][1][RTW89_IC][6] = 10, - [1][1][RTW89_ACMA][6] = 22, - [1][1][RTW89_FCC][8] = 36, + [1][1][RTW89_IC][6] = 20, + [1][1][RTW89_KCC][6] = 8, + [1][1][RTW89_ACMA][6] = 30, + [1][1][RTW89_CN][6] = 14, + [1][1][RTW89_UK][6] = 30, + [1][1][RTW89_FCC][8] = 44, [1][1][RTW89_ETSI][8] = 30, [1][1][RTW89_MKK][8] = 20, [1][1][RTW89_IC][8] = 44, - [1][1][RTW89_ACMA][8] = 22, - [1][1][RTW89_FCC][10] = 36, + [1][1][RTW89_KCC][8] = 34, + [1][1][RTW89_ACMA][8] = 30, + [1][1][RTW89_CN][8] = 14, + [1][1][RTW89_UK][8] = 30, + [1][1][RTW89_FCC][10] = 44, [1][1][RTW89_ETSI][10] = 30, [1][1][RTW89_MKK][10] = 20, [1][1][RTW89_IC][10] = 44, - [1][1][RTW89_ACMA][10] = 22, - [1][1][RTW89_FCC][12] = 38, + [1][1][RTW89_KCC][10] = 34, + [1][1][RTW89_ACMA][10] = 30, + [1][1][RTW89_CN][10] = 14, + [1][1][RTW89_UK][10] = 30, + [1][1][RTW89_FCC][12] = 44, [1][1][RTW89_ETSI][12] = 30, [1][1][RTW89_MKK][12] = 34, - [1][1][RTW89_IC][12] = 46, - [1][1][RTW89_ACMA][12] = 22, - [1][1][RTW89_FCC][14] = 34, + [1][1][RTW89_IC][12] = 44, + [1][1][RTW89_KCC][12] = 38, + [1][1][RTW89_ACMA][12] = 30, + [1][1][RTW89_CN][12] = 14, + [1][1][RTW89_UK][12] = 30, + [1][1][RTW89_FCC][14] = 44, [1][1][RTW89_ETSI][14] = 30, [1][1][RTW89_MKK][14] = 34, - [1][1][RTW89_IC][14] = 40, - [1][1][RTW89_ACMA][14] = 22, - [1][1][RTW89_FCC][15] = 34, + [1][1][RTW89_IC][14] = 44, + [1][1][RTW89_KCC][14] = 38, + [1][1][RTW89_ACMA][14] = 30, + [1][1][RTW89_CN][14] = 14, + [1][1][RTW89_UK][14] = 30, + [1][1][RTW89_FCC][15] = 44, [1][1][RTW89_ETSI][15] = 28, [1][1][RTW89_MKK][15] = 56, - [1][1][RTW89_IC][15] = 42, - [1][1][RTW89_ACMA][15] = 22, - [1][1][RTW89_FCC][17] = 34, + [1][1][RTW89_IC][15] = 44, + [1][1][RTW89_KCC][15] = 36, + [1][1][RTW89_ACMA][15] = 28, + [1][1][RTW89_CN][15] = 127, + [1][1][RTW89_UK][15] = 28, + [1][1][RTW89_FCC][17] = 44, [1][1][RTW89_ETSI][17] = 28, [1][1][RTW89_MKK][17] = 58, - [1][1][RTW89_IC][17] = 42, - [1][1][RTW89_ACMA][17] = 22, - [1][1][RTW89_FCC][19] = 34, + [1][1][RTW89_IC][17] = 44, + [1][1][RTW89_KCC][17] = 36, + [1][1][RTW89_ACMA][17] = 28, + [1][1][RTW89_CN][17] = 127, + [1][1][RTW89_UK][17] = 28, + [1][1][RTW89_FCC][19] = 44, [1][1][RTW89_ETSI][19] = 28, [1][1][RTW89_MKK][19] = 58, - [1][1][RTW89_IC][19] = 42, - [1][1][RTW89_ACMA][19] = 22, - [1][1][RTW89_FCC][21] = 34, + [1][1][RTW89_IC][19] = 44, + [1][1][RTW89_KCC][19] = 36, + [1][1][RTW89_ACMA][19] = 28, + [1][1][RTW89_CN][19] = 127, + [1][1][RTW89_UK][19] = 28, + [1][1][RTW89_FCC][21] = 44, [1][1][RTW89_ETSI][21] = 28, [1][1][RTW89_MKK][21] = 58, - [1][1][RTW89_IC][21] = 42, - [1][1][RTW89_ACMA][21] = 22, - [1][1][RTW89_FCC][23] = 34, + [1][1][RTW89_IC][21] = 44, + [1][1][RTW89_KCC][21] = 36, + [1][1][RTW89_ACMA][21] = 28, + [1][1][RTW89_CN][21] = 127, + [1][1][RTW89_UK][21] = 28, + [1][1][RTW89_FCC][23] = 44, [1][1][RTW89_ETSI][23] = 28, [1][1][RTW89_MKK][23] = 58, - [1][1][RTW89_IC][23] = 42, - [1][1][RTW89_ACMA][23] = 22, - [1][1][RTW89_FCC][25] = 34, + [1][1][RTW89_IC][23] = 44, + [1][1][RTW89_KCC][23] = 36, + [1][1][RTW89_ACMA][23] = 28, + [1][1][RTW89_CN][23] = 127, + [1][1][RTW89_UK][23] = 28, + [1][1][RTW89_FCC][25] = 44, [1][1][RTW89_ETSI][25] = 28, [1][1][RTW89_MKK][25] = 58, [1][1][RTW89_IC][25] = 127, + [1][1][RTW89_KCC][25] = 36, [1][1][RTW89_ACMA][25] = 127, - [1][1][RTW89_FCC][27] = 34, + [1][1][RTW89_CN][25] = 127, + [1][1][RTW89_UK][25] = 28, + [1][1][RTW89_FCC][27] = 44, [1][1][RTW89_ETSI][27] = 30, [1][1][RTW89_MKK][27] = 58, [1][1][RTW89_IC][27] = 127, + [1][1][RTW89_KCC][27] = 36, [1][1][RTW89_ACMA][27] = 127, - [1][1][RTW89_FCC][29] = 34, + [1][1][RTW89_CN][27] = 127, + [1][1][RTW89_UK][27] = 30, + [1][1][RTW89_FCC][29] = 44, [1][1][RTW89_ETSI][29] = 30, [1][1][RTW89_MKK][29] = 58, [1][1][RTW89_IC][29] = 127, + [1][1][RTW89_KCC][29] = 36, [1][1][RTW89_ACMA][29] = 127, - [1][1][RTW89_FCC][31] = 34, + [1][1][RTW89_CN][29] = 127, + [1][1][RTW89_UK][29] = 30, + [1][1][RTW89_FCC][31] = 44, [1][1][RTW89_ETSI][31] = 30, [1][1][RTW89_MKK][31] = 58, [1][1][RTW89_IC][31] = 38, - [1][1][RTW89_ACMA][31] = 22, - [1][1][RTW89_FCC][33] = 32, + [1][1][RTW89_KCC][31] = 36, + [1][1][RTW89_ACMA][31] = 30, + [1][1][RTW89_CN][31] = 127, + [1][1][RTW89_UK][31] = 30, + [1][1][RTW89_FCC][33] = 38, [1][1][RTW89_ETSI][33] = 30, [1][1][RTW89_MKK][33] = 58, [1][1][RTW89_IC][33] = 38, - [1][1][RTW89_ACMA][33] = 22, - [1][1][RTW89_FCC][35] = 32, + [1][1][RTW89_KCC][33] = 36, + [1][1][RTW89_ACMA][33] = 30, + [1][1][RTW89_CN][33] = 127, + [1][1][RTW89_UK][33] = 30, + [1][1][RTW89_FCC][35] = 38, [1][1][RTW89_ETSI][35] = 30, [1][1][RTW89_MKK][35] = 58, [1][1][RTW89_IC][35] = 38, - [1][1][RTW89_ACMA][35] = 22, - [1][1][RTW89_FCC][37] = 40, + [1][1][RTW89_KCC][35] = 36, + [1][1][RTW89_ACMA][35] = 30, + [1][1][RTW89_CN][35] = 127, + [1][1][RTW89_UK][35] = 30, + [1][1][RTW89_FCC][37] = 46, [1][1][RTW89_ETSI][37] = 127, [1][1][RTW89_MKK][37] = 58, - [1][1][RTW89_IC][37] = 48, - [1][1][RTW89_ACMA][37] = 48, - [1][1][RTW89_FCC][38] = 76, + [1][1][RTW89_IC][37] = 46, + [1][1][RTW89_KCC][37] = 36, + [1][1][RTW89_ACMA][37] = 46, + [1][1][RTW89_CN][37] = 127, + [1][1][RTW89_UK][37] = 32, + [1][1][RTW89_FCC][38] = 74, [1][1][RTW89_ETSI][38] = 16, [1][1][RTW89_MKK][38] = 127, - [1][1][RTW89_IC][38] = 84, - [1][1][RTW89_ACMA][38] = 82, - [1][1][RTW89_FCC][40] = 76, + [1][1][RTW89_IC][38] = 74, + [1][1][RTW89_KCC][38] = 36, + [1][1][RTW89_ACMA][38] = 74, + [1][1][RTW89_CN][38] = 54, + [1][1][RTW89_UK][38] = 30, + [1][1][RTW89_FCC][40] = 74, [1][1][RTW89_ETSI][40] = 16, [1][1][RTW89_MKK][40] = 127, - [1][1][RTW89_IC][40] = 84, - [1][1][RTW89_ACMA][40] = 82, - [1][1][RTW89_FCC][42] = 76, + [1][1][RTW89_IC][40] = 74, + [1][1][RTW89_KCC][40] = 36, + [1][1][RTW89_ACMA][40] = 74, + [1][1][RTW89_CN][40] = 54, + [1][1][RTW89_UK][40] = 30, + [1][1][RTW89_FCC][42] = 74, [1][1][RTW89_ETSI][42] = 16, [1][1][RTW89_MKK][42] = 127, - [1][1][RTW89_IC][42] = 84, - [1][1][RTW89_ACMA][42] = 84, - [1][1][RTW89_FCC][44] = 76, + [1][1][RTW89_IC][42] = 74, + [1][1][RTW89_KCC][42] = 36, + [1][1][RTW89_ACMA][42] = 74, + [1][1][RTW89_CN][42] = 54, + [1][1][RTW89_UK][42] = 30, + [1][1][RTW89_FCC][44] = 74, [1][1][RTW89_ETSI][44] = 16, [1][1][RTW89_MKK][44] = 127, - [1][1][RTW89_IC][44] = 84, - [1][1][RTW89_ACMA][44] = 84, - [1][1][RTW89_FCC][46] = 76, + [1][1][RTW89_IC][44] = 74, + [1][1][RTW89_KCC][44] = 36, + [1][1][RTW89_ACMA][44] = 74, + [1][1][RTW89_CN][44] = 54, + [1][1][RTW89_UK][44] = 30, + [1][1][RTW89_FCC][46] = 74, [1][1][RTW89_ETSI][46] = 16, [1][1][RTW89_MKK][46] = 127, - [1][1][RTW89_IC][46] = 84, - [1][1][RTW89_ACMA][46] = 84, - [1][1][RTW89_FCC][48] = 24, + [1][1][RTW89_IC][46] = 74, + [1][1][RTW89_KCC][46] = 36, + [1][1][RTW89_ACMA][46] = 74, + [1][1][RTW89_CN][46] = 54, + [1][1][RTW89_UK][46] = 30, + [1][1][RTW89_FCC][48] = 34, [1][1][RTW89_ETSI][48] = 127, [1][1][RTW89_MKK][48] = 127, [1][1][RTW89_IC][48] = 127, + [1][1][RTW89_KCC][48] = 127, [1][1][RTW89_ACMA][48] = 127, - [1][1][RTW89_FCC][50] = 24, + [1][1][RTW89_CN][48] = 127, + [1][1][RTW89_UK][48] = 127, + [1][1][RTW89_FCC][50] = 34, [1][1][RTW89_ETSI][50] = 127, [1][1][RTW89_MKK][50] = 127, [1][1][RTW89_IC][50] = 127, + [1][1][RTW89_KCC][50] = 127, [1][1][RTW89_ACMA][50] = 127, - [1][1][RTW89_FCC][52] = 24, + [1][1][RTW89_CN][50] = 127, + [1][1][RTW89_UK][50] = 127, + [1][1][RTW89_FCC][52] = 30, [1][1][RTW89_ETSI][52] = 127, [1][1][RTW89_MKK][52] = 127, [1][1][RTW89_IC][52] = 127, + [1][1][RTW89_KCC][52] = 127, [1][1][RTW89_ACMA][52] = 127, - [2][0][RTW89_FCC][0] = 62, + [1][1][RTW89_CN][52] = 127, + [1][1][RTW89_UK][52] = 127, + [2][0][RTW89_FCC][0] = 68, [2][0][RTW89_ETSI][0] = 52, [2][0][RTW89_MKK][0] = 60, - [2][0][RTW89_IC][0] = 46, - [2][0][RTW89_ACMA][0] = 48, - [2][0][RTW89_FCC][2] = 62, + [2][0][RTW89_IC][0] = 52, + [2][0][RTW89_KCC][0] = 64, + [2][0][RTW89_ACMA][0] = 52, + [2][0][RTW89_CN][0] = 40, + [2][0][RTW89_UK][0] = 52, + [2][0][RTW89_FCC][2] = 64, [2][0][RTW89_ETSI][2] = 52, [2][0][RTW89_MKK][2] = 60, - [2][0][RTW89_IC][2] = 46, - [2][0][RTW89_ACMA][2] = 48, - [2][0][RTW89_FCC][4] = 62, + [2][0][RTW89_IC][2] = 50, + [2][0][RTW89_KCC][2] = 64, + [2][0][RTW89_ACMA][2] = 52, + [2][0][RTW89_CN][2] = 40, + [2][0][RTW89_UK][2] = 52, + [2][0][RTW89_FCC][4] = 68, [2][0][RTW89_ETSI][4] = 52, [2][0][RTW89_MKK][4] = 50, - [2][0][RTW89_IC][4] = 46, - [2][0][RTW89_ACMA][4] = 48, - [2][0][RTW89_FCC][6] = 62, + [2][0][RTW89_IC][4] = 50, + [2][0][RTW89_KCC][4] = 64, + [2][0][RTW89_ACMA][4] = 52, + [2][0][RTW89_CN][4] = 40, + [2][0][RTW89_UK][4] = 52, + [2][0][RTW89_FCC][6] = 68, [2][0][RTW89_ETSI][6] = 52, [2][0][RTW89_MKK][6] = 50, - [2][0][RTW89_IC][6] = 46, - [2][0][RTW89_ACMA][6] = 48, - [2][0][RTW89_FCC][8] = 62, + [2][0][RTW89_IC][6] = 50, + [2][0][RTW89_KCC][6] = 36, + [2][0][RTW89_ACMA][6] = 52, + [2][0][RTW89_CN][6] = 40, + [2][0][RTW89_UK][6] = 52, + [2][0][RTW89_FCC][8] = 68, [2][0][RTW89_ETSI][8] = 52, [2][0][RTW89_MKK][8] = 44, - [2][0][RTW89_IC][8] = 66, - [2][0][RTW89_ACMA][8] = 48, - [2][0][RTW89_FCC][10] = 62, + [2][0][RTW89_IC][8] = 64, + [2][0][RTW89_KCC][8] = 62, + [2][0][RTW89_ACMA][8] = 52, + [2][0][RTW89_CN][8] = 40, + [2][0][RTW89_UK][8] = 52, + [2][0][RTW89_FCC][10] = 68, [2][0][RTW89_ETSI][10] = 52, [2][0][RTW89_MKK][10] = 44, - [2][0][RTW89_IC][10] = 66, - [2][0][RTW89_ACMA][10] = 48, - [2][0][RTW89_FCC][12] = 62, + [2][0][RTW89_IC][10] = 64, + [2][0][RTW89_KCC][10] = 62, + [2][0][RTW89_ACMA][10] = 52, + [2][0][RTW89_CN][10] = 40, + [2][0][RTW89_UK][10] = 52, + [2][0][RTW89_FCC][12] = 68, [2][0][RTW89_ETSI][12] = 52, [2][0][RTW89_MKK][12] = 58, - [2][0][RTW89_IC][12] = 66, - [2][0][RTW89_ACMA][12] = 48, - [2][0][RTW89_FCC][14] = 62, + [2][0][RTW89_IC][12] = 64, + [2][0][RTW89_KCC][12] = 62, + [2][0][RTW89_ACMA][12] = 52, + [2][0][RTW89_CN][12] = 40, + [2][0][RTW89_UK][12] = 52, + [2][0][RTW89_FCC][14] = 68, [2][0][RTW89_ETSI][14] = 52, [2][0][RTW89_MKK][14] = 58, - [2][0][RTW89_IC][14] = 66, - [2][0][RTW89_ACMA][14] = 48, - [2][0][RTW89_FCC][15] = 62, + [2][0][RTW89_IC][14] = 64, + [2][0][RTW89_KCC][14] = 62, + [2][0][RTW89_ACMA][14] = 52, + [2][0][RTW89_CN][14] = 40, + [2][0][RTW89_UK][14] = 52, + [2][0][RTW89_FCC][15] = 68, [2][0][RTW89_ETSI][15] = 52, [2][0][RTW89_MKK][15] = 68, - [2][0][RTW89_IC][15] = 70, - [2][0][RTW89_ACMA][15] = 48, - [2][0][RTW89_FCC][17] = 62, + [2][0][RTW89_IC][15] = 68, + [2][0][RTW89_KCC][15] = 62, + [2][0][RTW89_ACMA][15] = 52, + [2][0][RTW89_CN][15] = 127, + [2][0][RTW89_UK][15] = 52, + [2][0][RTW89_FCC][17] = 68, [2][0][RTW89_ETSI][17] = 52, [2][0][RTW89_MKK][17] = 74, - [2][0][RTW89_IC][17] = 70, - [2][0][RTW89_ACMA][17] = 48, - [2][0][RTW89_FCC][19] = 62, + [2][0][RTW89_IC][17] = 68, + [2][0][RTW89_KCC][17] = 62, + [2][0][RTW89_ACMA][17] = 52, + [2][0][RTW89_CN][17] = 127, + [2][0][RTW89_UK][17] = 52, + [2][0][RTW89_FCC][19] = 70, [2][0][RTW89_ETSI][19] = 52, [2][0][RTW89_MKK][19] = 74, [2][0][RTW89_IC][19] = 70, - [2][0][RTW89_ACMA][19] = 48, - [2][0][RTW89_FCC][21] = 62, + [2][0][RTW89_KCC][19] = 62, + [2][0][RTW89_ACMA][19] = 52, + [2][0][RTW89_CN][19] = 127, + [2][0][RTW89_UK][19] = 52, + [2][0][RTW89_FCC][21] = 70, [2][0][RTW89_ETSI][21] = 52, [2][0][RTW89_MKK][21] = 74, [2][0][RTW89_IC][21] = 70, - [2][0][RTW89_ACMA][21] = 48, - [2][0][RTW89_FCC][23] = 62, + [2][0][RTW89_KCC][21] = 62, + [2][0][RTW89_ACMA][21] = 52, + [2][0][RTW89_CN][21] = 127, + [2][0][RTW89_UK][21] = 52, + [2][0][RTW89_FCC][23] = 70, [2][0][RTW89_ETSI][23] = 52, [2][0][RTW89_MKK][23] = 74, [2][0][RTW89_IC][23] = 70, - [2][0][RTW89_ACMA][23] = 48, - [2][0][RTW89_FCC][25] = 62, + [2][0][RTW89_KCC][23] = 62, + [2][0][RTW89_ACMA][23] = 52, + [2][0][RTW89_CN][23] = 127, + [2][0][RTW89_UK][23] = 52, + [2][0][RTW89_FCC][25] = 70, [2][0][RTW89_ETSI][25] = 52, [2][0][RTW89_MKK][25] = 74, [2][0][RTW89_IC][25] = 127, + [2][0][RTW89_KCC][25] = 62, [2][0][RTW89_ACMA][25] = 127, - [2][0][RTW89_FCC][27] = 62, + [2][0][RTW89_CN][25] = 127, + [2][0][RTW89_UK][25] = 52, + [2][0][RTW89_FCC][27] = 70, [2][0][RTW89_ETSI][27] = 52, [2][0][RTW89_MKK][27] = 74, [2][0][RTW89_IC][27] = 127, + [2][0][RTW89_KCC][27] = 62, [2][0][RTW89_ACMA][27] = 127, - [2][0][RTW89_FCC][29] = 62, + [2][0][RTW89_CN][27] = 127, + [2][0][RTW89_UK][27] = 52, + [2][0][RTW89_FCC][29] = 70, [2][0][RTW89_ETSI][29] = 52, [2][0][RTW89_MKK][29] = 74, [2][0][RTW89_IC][29] = 127, + [2][0][RTW89_KCC][29] = 62, [2][0][RTW89_ACMA][29] = 127, - [2][0][RTW89_FCC][31] = 62, + [2][0][RTW89_CN][29] = 127, + [2][0][RTW89_UK][29] = 52, + [2][0][RTW89_FCC][31] = 70, [2][0][RTW89_ETSI][31] = 52, [2][0][RTW89_MKK][31] = 74, - [2][0][RTW89_IC][31] = 72, - [2][0][RTW89_ACMA][31] = 48, - [2][0][RTW89_FCC][33] = 64, + [2][0][RTW89_IC][31] = 62, + [2][0][RTW89_KCC][31] = 62, + [2][0][RTW89_ACMA][31] = 52, + [2][0][RTW89_CN][31] = 127, + [2][0][RTW89_UK][31] = 52, + [2][0][RTW89_FCC][33] = 62, [2][0][RTW89_ETSI][33] = 52, [2][0][RTW89_MKK][33] = 74, - [2][0][RTW89_IC][33] = 72, - [2][0][RTW89_ACMA][33] = 48, - [2][0][RTW89_FCC][35] = 64, + [2][0][RTW89_IC][33] = 62, + [2][0][RTW89_KCC][33] = 62, + [2][0][RTW89_ACMA][33] = 52, + [2][0][RTW89_CN][33] = 127, + [2][0][RTW89_UK][33] = 52, + [2][0][RTW89_FCC][35] = 62, [2][0][RTW89_ETSI][35] = 52, [2][0][RTW89_MKK][35] = 74, - [2][0][RTW89_IC][35] = 72, - [2][0][RTW89_ACMA][35] = 48, - [2][0][RTW89_FCC][37] = 62, + [2][0][RTW89_IC][35] = 62, + [2][0][RTW89_KCC][35] = 62, + [2][0][RTW89_ACMA][35] = 52, + [2][0][RTW89_CN][35] = 127, + [2][0][RTW89_UK][35] = 52, + [2][0][RTW89_FCC][37] = 70, [2][0][RTW89_ETSI][37] = 127, [2][0][RTW89_MKK][37] = 74, [2][0][RTW89_IC][37] = 70, - [2][0][RTW89_ACMA][37] = 76, - [2][0][RTW89_FCC][38] = 76, + [2][0][RTW89_KCC][37] = 62, + [2][0][RTW89_ACMA][37] = 70, + [2][0][RTW89_CN][37] = 127, + [2][0][RTW89_UK][37] = 52, + [2][0][RTW89_FCC][38] = 82, [2][0][RTW89_ETSI][38] = 28, [2][0][RTW89_MKK][38] = 127, - [2][0][RTW89_IC][38] = 84, - [2][0][RTW89_ACMA][38] = 84, - [2][0][RTW89_FCC][40] = 76, + [2][0][RTW89_IC][38] = 82, + [2][0][RTW89_KCC][38] = 64, + [2][0][RTW89_ACMA][38] = 82, + [2][0][RTW89_CN][38] = 68, + [2][0][RTW89_UK][38] = 54, + [2][0][RTW89_FCC][40] = 82, [2][0][RTW89_ETSI][40] = 28, [2][0][RTW89_MKK][40] = 127, - [2][0][RTW89_IC][40] = 84, - [2][0][RTW89_ACMA][40] = 84, + [2][0][RTW89_IC][40] = 82, + [2][0][RTW89_KCC][40] = 64, + [2][0][RTW89_ACMA][40] = 82, + [2][0][RTW89_CN][40] = 68, + [2][0][RTW89_UK][40] = 54, [2][0][RTW89_FCC][42] = 76, [2][0][RTW89_ETSI][42] = 28, [2][0][RTW89_MKK][42] = 127, - [2][0][RTW89_IC][42] = 84, - [2][0][RTW89_ACMA][42] = 84, - [2][0][RTW89_FCC][44] = 76, + [2][0][RTW89_IC][42] = 76, + [2][0][RTW89_KCC][42] = 64, + [2][0][RTW89_ACMA][42] = 76, + [2][0][RTW89_CN][42] = 68, + [2][0][RTW89_UK][42] = 54, + [2][0][RTW89_FCC][44] = 80, [2][0][RTW89_ETSI][44] = 28, [2][0][RTW89_MKK][44] = 127, - [2][0][RTW89_IC][44] = 84, - [2][0][RTW89_ACMA][44] = 84, - [2][0][RTW89_FCC][46] = 76, + [2][0][RTW89_IC][44] = 80, + [2][0][RTW89_KCC][44] = 64, + [2][0][RTW89_ACMA][44] = 80, + [2][0][RTW89_CN][44] = 68, + [2][0][RTW89_UK][44] = 54, + [2][0][RTW89_FCC][46] = 80, [2][0][RTW89_ETSI][46] = 28, [2][0][RTW89_MKK][46] = 127, - [2][0][RTW89_IC][46] = 84, - [2][0][RTW89_ACMA][46] = 84, - [2][0][RTW89_FCC][48] = 48, + [2][0][RTW89_IC][46] = 80, + [2][0][RTW89_KCC][46] = 64, + [2][0][RTW89_ACMA][46] = 80, + [2][0][RTW89_CN][46] = 68, + [2][0][RTW89_UK][46] = 54, + [2][0][RTW89_FCC][48] = 64, [2][0][RTW89_ETSI][48] = 127, [2][0][RTW89_MKK][48] = 127, [2][0][RTW89_IC][48] = 127, + [2][0][RTW89_KCC][48] = 127, [2][0][RTW89_ACMA][48] = 127, - [2][0][RTW89_FCC][50] = 48, + [2][0][RTW89_CN][48] = 127, + [2][0][RTW89_UK][48] = 127, + [2][0][RTW89_FCC][50] = 64, [2][0][RTW89_ETSI][50] = 127, [2][0][RTW89_MKK][50] = 127, [2][0][RTW89_IC][50] = 127, + [2][0][RTW89_KCC][50] = 127, [2][0][RTW89_ACMA][50] = 127, - [2][0][RTW89_FCC][52] = 48, + [2][0][RTW89_CN][50] = 127, + [2][0][RTW89_UK][50] = 127, + [2][0][RTW89_FCC][52] = 64, [2][0][RTW89_ETSI][52] = 127, [2][0][RTW89_MKK][52] = 127, [2][0][RTW89_IC][52] = 127, + [2][0][RTW89_KCC][52] = 127, [2][0][RTW89_ACMA][52] = 127, - [2][1][RTW89_FCC][0] = 42, + [2][0][RTW89_CN][52] = 127, + [2][0][RTW89_UK][52] = 127, + [2][1][RTW89_FCC][0] = 50, [2][1][RTW89_ETSI][0] = 40, [2][1][RTW89_MKK][0] = 44, - [2][1][RTW89_IC][0] = 20, - [2][1][RTW89_ACMA][0] = 36, - [2][1][RTW89_FCC][2] = 42, + [2][1][RTW89_IC][0] = 26, + [2][1][RTW89_KCC][0] = 44, + [2][1][RTW89_ACMA][0] = 40, + [2][1][RTW89_CN][0] = 28, + [2][1][RTW89_UK][0] = 40, + [2][1][RTW89_FCC][2] = 50, [2][1][RTW89_ETSI][2] = 40, [2][1][RTW89_MKK][2] = 44, - [2][1][RTW89_IC][2] = 18, - [2][1][RTW89_ACMA][2] = 36, - [2][1][RTW89_FCC][4] = 42, + [2][1][RTW89_IC][2] = 26, + [2][1][RTW89_KCC][2] = 44, + [2][1][RTW89_ACMA][2] = 40, + [2][1][RTW89_CN][2] = 28, + [2][1][RTW89_UK][2] = 40, + [2][1][RTW89_FCC][4] = 50, [2][1][RTW89_ETSI][4] = 40, [2][1][RTW89_MKK][4] = 36, - [2][1][RTW89_IC][4] = 22, - [2][1][RTW89_ACMA][4] = 36, - [2][1][RTW89_FCC][6] = 42, + [2][1][RTW89_IC][4] = 26, + [2][1][RTW89_KCC][4] = 44, + [2][1][RTW89_ACMA][4] = 40, + [2][1][RTW89_CN][4] = 28, + [2][1][RTW89_UK][4] = 40, + [2][1][RTW89_FCC][6] = 50, [2][1][RTW89_ETSI][6] = 40, [2][1][RTW89_MKK][6] = 36, - [2][1][RTW89_IC][6] = 22, - [2][1][RTW89_ACMA][6] = 36, - [2][1][RTW89_FCC][8] = 42, + [2][1][RTW89_IC][6] = 26, + [2][1][RTW89_KCC][6] = 20, + [2][1][RTW89_ACMA][6] = 40, + [2][1][RTW89_CN][6] = 28, + [2][1][RTW89_UK][6] = 40, + [2][1][RTW89_FCC][8] = 50, [2][1][RTW89_ETSI][8] = 40, [2][1][RTW89_MKK][8] = 32, [2][1][RTW89_IC][8] = 50, - [2][1][RTW89_ACMA][8] = 36, - [2][1][RTW89_FCC][10] = 42, + [2][1][RTW89_KCC][8] = 46, + [2][1][RTW89_ACMA][8] = 40, + [2][1][RTW89_CN][8] = 28, + [2][1][RTW89_UK][8] = 40, + [2][1][RTW89_FCC][10] = 50, [2][1][RTW89_ETSI][10] = 40, [2][1][RTW89_MKK][10] = 32, [2][1][RTW89_IC][10] = 50, - [2][1][RTW89_ACMA][10] = 36, - [2][1][RTW89_FCC][12] = 44, + [2][1][RTW89_KCC][10] = 46, + [2][1][RTW89_ACMA][10] = 40, + [2][1][RTW89_CN][10] = 28, + [2][1][RTW89_UK][10] = 40, + [2][1][RTW89_FCC][12] = 48, [2][1][RTW89_ETSI][12] = 40, [2][1][RTW89_MKK][12] = 44, - [2][1][RTW89_IC][12] = 52, - [2][1][RTW89_ACMA][12] = 36, - [2][1][RTW89_FCC][14] = 44, + [2][1][RTW89_IC][12] = 48, + [2][1][RTW89_KCC][12] = 46, + [2][1][RTW89_ACMA][12] = 40, + [2][1][RTW89_CN][12] = 28, + [2][1][RTW89_UK][12] = 40, + [2][1][RTW89_FCC][14] = 48, [2][1][RTW89_ETSI][14] = 40, [2][1][RTW89_MKK][14] = 44, - [2][1][RTW89_IC][14] = 52, - [2][1][RTW89_ACMA][14] = 36, - [2][1][RTW89_FCC][15] = 42, + [2][1][RTW89_IC][14] = 48, + [2][1][RTW89_KCC][14] = 46, + [2][1][RTW89_ACMA][14] = 40, + [2][1][RTW89_CN][14] = 28, + [2][1][RTW89_UK][14] = 40, + [2][1][RTW89_FCC][15] = 50, [2][1][RTW89_ETSI][15] = 40, [2][1][RTW89_MKK][15] = 66, [2][1][RTW89_IC][15] = 50, - [2][1][RTW89_ACMA][15] = 36, - [2][1][RTW89_FCC][17] = 42, + [2][1][RTW89_KCC][15] = 46, + [2][1][RTW89_ACMA][15] = 40, + [2][1][RTW89_CN][15] = 127, + [2][1][RTW89_UK][15] = 40, + [2][1][RTW89_FCC][17] = 50, [2][1][RTW89_ETSI][17] = 40, [2][1][RTW89_MKK][17] = 66, [2][1][RTW89_IC][17] = 50, - [2][1][RTW89_ACMA][17] = 36, - [2][1][RTW89_FCC][19] = 42, + [2][1][RTW89_KCC][17] = 46, + [2][1][RTW89_ACMA][17] = 40, + [2][1][RTW89_CN][17] = 127, + [2][1][RTW89_UK][17] = 40, + [2][1][RTW89_FCC][19] = 50, [2][1][RTW89_ETSI][19] = 40, [2][1][RTW89_MKK][19] = 66, [2][1][RTW89_IC][19] = 50, - [2][1][RTW89_ACMA][19] = 36, - [2][1][RTW89_FCC][21] = 42, + [2][1][RTW89_KCC][19] = 46, + [2][1][RTW89_ACMA][19] = 40, + [2][1][RTW89_CN][19] = 127, + [2][1][RTW89_UK][19] = 40, + [2][1][RTW89_FCC][21] = 50, [2][1][RTW89_ETSI][21] = 40, [2][1][RTW89_MKK][21] = 66, [2][1][RTW89_IC][21] = 50, - [2][1][RTW89_ACMA][21] = 36, - [2][1][RTW89_FCC][23] = 42, + [2][1][RTW89_KCC][21] = 46, + [2][1][RTW89_ACMA][21] = 40, + [2][1][RTW89_CN][21] = 127, + [2][1][RTW89_UK][21] = 40, + [2][1][RTW89_FCC][23] = 50, [2][1][RTW89_ETSI][23] = 40, [2][1][RTW89_MKK][23] = 66, [2][1][RTW89_IC][23] = 50, - [2][1][RTW89_ACMA][23] = 36, - [2][1][RTW89_FCC][25] = 42, + [2][1][RTW89_KCC][23] = 46, + [2][1][RTW89_ACMA][23] = 40, + [2][1][RTW89_CN][23] = 127, + [2][1][RTW89_UK][23] = 40, + [2][1][RTW89_FCC][25] = 50, [2][1][RTW89_ETSI][25] = 40, [2][1][RTW89_MKK][25] = 66, [2][1][RTW89_IC][25] = 127, + [2][1][RTW89_KCC][25] = 46, [2][1][RTW89_ACMA][25] = 127, - [2][1][RTW89_FCC][27] = 42, + [2][1][RTW89_CN][25] = 127, + [2][1][RTW89_UK][25] = 40, + [2][1][RTW89_FCC][27] = 50, [2][1][RTW89_ETSI][27] = 40, [2][1][RTW89_MKK][27] = 66, [2][1][RTW89_IC][27] = 127, + [2][1][RTW89_KCC][27] = 46, [2][1][RTW89_ACMA][27] = 127, - [2][1][RTW89_FCC][29] = 42, + [2][1][RTW89_CN][27] = 127, + [2][1][RTW89_UK][27] = 40, + [2][1][RTW89_FCC][29] = 50, [2][1][RTW89_ETSI][29] = 40, [2][1][RTW89_MKK][29] = 66, [2][1][RTW89_IC][29] = 127, + [2][1][RTW89_KCC][29] = 46, [2][1][RTW89_ACMA][29] = 127, - [2][1][RTW89_FCC][31] = 42, + [2][1][RTW89_CN][29] = 127, + [2][1][RTW89_UK][29] = 40, + [2][1][RTW89_FCC][31] = 50, [2][1][RTW89_ETSI][31] = 40, [2][1][RTW89_MKK][31] = 66, - [2][1][RTW89_IC][31] = 50, - [2][1][RTW89_ACMA][31] = 36, - [2][1][RTW89_FCC][33] = 42, + [2][1][RTW89_IC][31] = 48, + [2][1][RTW89_KCC][31] = 46, + [2][1][RTW89_ACMA][31] = 40, + [2][1][RTW89_CN][31] = 127, + [2][1][RTW89_UK][31] = 40, + [2][1][RTW89_FCC][33] = 48, [2][1][RTW89_ETSI][33] = 40, [2][1][RTW89_MKK][33] = 66, - [2][1][RTW89_IC][33] = 50, - [2][1][RTW89_ACMA][33] = 36, - [2][1][RTW89_FCC][35] = 42, + [2][1][RTW89_IC][33] = 48, + [2][1][RTW89_KCC][33] = 46, + [2][1][RTW89_ACMA][33] = 40, + [2][1][RTW89_CN][33] = 127, + [2][1][RTW89_UK][33] = 40, + [2][1][RTW89_FCC][35] = 48, [2][1][RTW89_ETSI][35] = 40, [2][1][RTW89_MKK][35] = 66, - [2][1][RTW89_IC][35] = 50, - [2][1][RTW89_ACMA][35] = 36, - [2][1][RTW89_FCC][37] = 42, + [2][1][RTW89_IC][35] = 48, + [2][1][RTW89_KCC][35] = 46, + [2][1][RTW89_ACMA][35] = 40, + [2][1][RTW89_CN][35] = 127, + [2][1][RTW89_UK][35] = 40, + [2][1][RTW89_FCC][37] = 52, [2][1][RTW89_ETSI][37] = 127, [2][1][RTW89_MKK][37] = 66, - [2][1][RTW89_IC][37] = 50, - [2][1][RTW89_ACMA][37] = 60, - [2][1][RTW89_FCC][38] = 76, + [2][1][RTW89_IC][37] = 52, + [2][1][RTW89_KCC][37] = 46, + [2][1][RTW89_ACMA][37] = 52, + [2][1][RTW89_CN][37] = 127, + [2][1][RTW89_UK][37] = 42, + [2][1][RTW89_FCC][38] = 78, [2][1][RTW89_ETSI][38] = 16, [2][1][RTW89_MKK][38] = 127, - [2][1][RTW89_IC][38] = 84, - [2][1][RTW89_ACMA][38] = 84, - [2][1][RTW89_FCC][40] = 76, + [2][1][RTW89_IC][38] = 78, + [2][1][RTW89_KCC][38] = 46, + [2][1][RTW89_ACMA][38] = 78, + [2][1][RTW89_CN][38] = 56, + [2][1][RTW89_UK][38] = 42, + [2][1][RTW89_FCC][40] = 78, [2][1][RTW89_ETSI][40] = 16, [2][1][RTW89_MKK][40] = 127, - [2][1][RTW89_IC][40] = 84, - [2][1][RTW89_ACMA][40] = 84, - [2][1][RTW89_FCC][42] = 76, + [2][1][RTW89_IC][40] = 78, + [2][1][RTW89_KCC][40] = 46, + [2][1][RTW89_ACMA][40] = 78, + [2][1][RTW89_CN][40] = 56, + [2][1][RTW89_UK][40] = 42, + [2][1][RTW89_FCC][42] = 78, [2][1][RTW89_ETSI][42] = 16, [2][1][RTW89_MKK][42] = 127, - [2][1][RTW89_IC][42] = 84, - [2][1][RTW89_ACMA][42] = 84, - [2][1][RTW89_FCC][44] = 76, + [2][1][RTW89_IC][42] = 78, + [2][1][RTW89_KCC][42] = 46, + [2][1][RTW89_ACMA][42] = 78, + [2][1][RTW89_CN][42] = 56, + [2][1][RTW89_UK][42] = 42, + [2][1][RTW89_FCC][44] = 74, [2][1][RTW89_ETSI][44] = 16, [2][1][RTW89_MKK][44] = 127, - [2][1][RTW89_IC][44] = 84, - [2][1][RTW89_ACMA][44] = 84, - [2][1][RTW89_FCC][46] = 76, + [2][1][RTW89_IC][44] = 74, + [2][1][RTW89_KCC][44] = 46, + [2][1][RTW89_ACMA][44] = 74, + [2][1][RTW89_CN][44] = 56, + [2][1][RTW89_UK][44] = 42, + [2][1][RTW89_FCC][46] = 74, [2][1][RTW89_ETSI][46] = 16, [2][1][RTW89_MKK][46] = 127, - [2][1][RTW89_IC][46] = 84, - [2][1][RTW89_ACMA][46] = 84, - [2][1][RTW89_FCC][48] = 36, + [2][1][RTW89_IC][46] = 74, + [2][1][RTW89_KCC][46] = 46, + [2][1][RTW89_ACMA][46] = 74, + [2][1][RTW89_CN][46] = 56, + [2][1][RTW89_UK][46] = 42, + [2][1][RTW89_FCC][48] = 40, [2][1][RTW89_ETSI][48] = 127, [2][1][RTW89_MKK][48] = 127, [2][1][RTW89_IC][48] = 127, + [2][1][RTW89_KCC][48] = 127, [2][1][RTW89_ACMA][48] = 127, - [2][1][RTW89_FCC][50] = 36, + [2][1][RTW89_CN][48] = 127, + [2][1][RTW89_UK][48] = 127, + [2][1][RTW89_FCC][50] = 40, [2][1][RTW89_ETSI][50] = 127, [2][1][RTW89_MKK][50] = 127, [2][1][RTW89_IC][50] = 127, + [2][1][RTW89_KCC][50] = 127, [2][1][RTW89_ACMA][50] = 127, - [2][1][RTW89_FCC][52] = 36, + [2][1][RTW89_CN][50] = 127, + [2][1][RTW89_UK][50] = 127, + [2][1][RTW89_FCC][52] = 40, [2][1][RTW89_ETSI][52] = 127, [2][1][RTW89_MKK][52] = 127, [2][1][RTW89_IC][52] = 127, + [2][1][RTW89_KCC][52] = 127, [2][1][RTW89_ACMA][52] = 127, + [2][1][RTW89_CN][52] = 127, + [2][1][RTW89_UK][52] = 127, }; const s8 rtw89_8852c_txpwr_lmt_ru_6g[RTW89_RU_NUM][RTW89_NTX_NUM] [RTW89_REGD_NUM][RTW89_6G_CH_NUM] = { - [0][0][RTW89_WW][0] = 76, - [0][0][RTW89_WW][2] = 76, - [0][0][RTW89_WW][4] = 76, - [0][0][RTW89_WW][6] = 76, - [0][0][RTW89_WW][8] = 76, - [0][0][RTW89_WW][10] = 76, - [0][0][RTW89_WW][12] = 76, - [0][0][RTW89_WW][14] = 76, - [0][0][RTW89_WW][15] = 76, - [0][0][RTW89_WW][17] = 76, - [0][0][RTW89_WW][19] = 76, - [0][0][RTW89_WW][21] = 76, - [0][0][RTW89_WW][23] = 76, - [0][0][RTW89_WW][25] = 76, - [0][0][RTW89_WW][27] = 76, - [0][0][RTW89_WW][29] = 76, - [0][0][RTW89_WW][30] = 76, - [0][0][RTW89_WW][32] = 76, - [0][0][RTW89_WW][34] = 76, - [0][0][RTW89_WW][36] = 76, - [0][0][RTW89_WW][38] = 76, - [0][0][RTW89_WW][40] = 76, - [0][0][RTW89_WW][42] = 76, - [0][0][RTW89_WW][44] = 76, - [0][0][RTW89_WW][45] = 76, - [0][0][RTW89_WW][47] = 76, - [0][0][RTW89_WW][49] = 76, - [0][0][RTW89_WW][51] = 76, - [0][0][RTW89_WW][53] = 76, - [0][0][RTW89_WW][55] = 76, - [0][0][RTW89_WW][57] = 76, - [0][0][RTW89_WW][59] = 76, - [0][0][RTW89_WW][60] = 76, - [0][0][RTW89_WW][62] = 76, - [0][0][RTW89_WW][64] = 76, - [0][0][RTW89_WW][66] = 76, - [0][0][RTW89_WW][68] = 76, - [0][0][RTW89_WW][70] = 76, - [0][0][RTW89_WW][72] = 76, - [0][0][RTW89_WW][74] = 76, - [0][0][RTW89_WW][75] = 76, - [0][0][RTW89_WW][77] = 76, - [0][0][RTW89_WW][79] = 76, - [0][0][RTW89_WW][81] = 76, - [0][0][RTW89_WW][83] = 76, - [0][0][RTW89_WW][85] = 76, - [0][0][RTW89_WW][87] = 76, - [0][0][RTW89_WW][89] = 76, - [0][0][RTW89_WW][90] = 76, - [0][0][RTW89_WW][92] = 76, - [0][0][RTW89_WW][94] = 76, - [0][0][RTW89_WW][96] = 76, - [0][0][RTW89_WW][98] = 76, - [0][0][RTW89_WW][100] = 76, - [0][0][RTW89_WW][102] = 76, - [0][0][RTW89_WW][104] = 76, - [0][0][RTW89_WW][105] = 76, - [0][0][RTW89_WW][107] = 76, - [0][0][RTW89_WW][109] = 76, + [0][0][RTW89_WW][0] = -16, + [0][0][RTW89_WW][2] = -18, + [0][0][RTW89_WW][4] = -18, + [0][0][RTW89_WW][6] = -18, + [0][0][RTW89_WW][8] = -18, + [0][0][RTW89_WW][10] = -18, + [0][0][RTW89_WW][12] = -18, + [0][0][RTW89_WW][14] = -18, + [0][0][RTW89_WW][15] = -18, + [0][0][RTW89_WW][17] = -18, + [0][0][RTW89_WW][19] = -18, + [0][0][RTW89_WW][21] = -18, + [0][0][RTW89_WW][23] = -18, + [0][0][RTW89_WW][25] = -18, + [0][0][RTW89_WW][27] = -18, + [0][0][RTW89_WW][29] = -18, + [0][0][RTW89_WW][30] = -18, + [0][0][RTW89_WW][32] = -18, + [0][0][RTW89_WW][34] = -18, + [0][0][RTW89_WW][36] = -18, + [0][0][RTW89_WW][38] = -18, + [0][0][RTW89_WW][40] = -18, + [0][0][RTW89_WW][42] = -18, + [0][0][RTW89_WW][44] = -16, + [0][0][RTW89_WW][45] = -16, + [0][0][RTW89_WW][47] = -18, + [0][0][RTW89_WW][49] = -18, + [0][0][RTW89_WW][51] = -18, + [0][0][RTW89_WW][53] = -16, + [0][0][RTW89_WW][55] = -18, + [0][0][RTW89_WW][57] = -18, + [0][0][RTW89_WW][59] = -18, + [0][0][RTW89_WW][60] = -18, + [0][0][RTW89_WW][62] = -18, + [0][0][RTW89_WW][64] = -18, + [0][0][RTW89_WW][66] = -18, + [0][0][RTW89_WW][68] = -18, + [0][0][RTW89_WW][70] = -16, + [0][0][RTW89_WW][72] = -18, + [0][0][RTW89_WW][74] = -18, + [0][0][RTW89_WW][75] = -18, + [0][0][RTW89_WW][77] = -18, + [0][0][RTW89_WW][79] = -18, + [0][0][RTW89_WW][81] = -18, + [0][0][RTW89_WW][83] = -18, + [0][0][RTW89_WW][85] = -18, + [0][0][RTW89_WW][87] = -16, + [0][0][RTW89_WW][89] = -16, + [0][0][RTW89_WW][90] = -16, + [0][0][RTW89_WW][92] = -16, + [0][0][RTW89_WW][94] = -16, + [0][0][RTW89_WW][96] = -16, + [0][0][RTW89_WW][98] = -16, + [0][0][RTW89_WW][100] = -16, + [0][0][RTW89_WW][102] = -16, + [0][0][RTW89_WW][104] = -16, + [0][0][RTW89_WW][105] = -16, + [0][0][RTW89_WW][107] = -12, + [0][0][RTW89_WW][109] = -12, [0][0][RTW89_WW][111] = 0, [0][0][RTW89_WW][113] = 0, [0][0][RTW89_WW][115] = 0, [0][0][RTW89_WW][117] = 0, [0][0][RTW89_WW][119] = 0, - [0][1][RTW89_WW][0] = 76, - [0][1][RTW89_WW][2] = 76, - [0][1][RTW89_WW][4] = 76, - [0][1][RTW89_WW][6] = 76, - [0][1][RTW89_WW][8] = 76, - [0][1][RTW89_WW][10] = 76, - [0][1][RTW89_WW][12] = 76, - [0][1][RTW89_WW][14] = 76, - [0][1][RTW89_WW][15] = 76, - [0][1][RTW89_WW][17] = 76, - [0][1][RTW89_WW][19] = 76, - [0][1][RTW89_WW][21] = 76, - [0][1][RTW89_WW][23] = 76, - [0][1][RTW89_WW][25] = 76, - [0][1][RTW89_WW][27] = 76, - [0][1][RTW89_WW][29] = 76, - [0][1][RTW89_WW][30] = 76, - [0][1][RTW89_WW][32] = 76, - [0][1][RTW89_WW][34] = 76, - [0][1][RTW89_WW][36] = 76, - [0][1][RTW89_WW][38] = 76, - [0][1][RTW89_WW][40] = 76, - [0][1][RTW89_WW][42] = 76, - [0][1][RTW89_WW][44] = 76, - [0][1][RTW89_WW][45] = 76, - [0][1][RTW89_WW][47] = 76, - [0][1][RTW89_WW][49] = 76, - [0][1][RTW89_WW][51] = 76, - [0][1][RTW89_WW][53] = 76, - [0][1][RTW89_WW][55] = 76, - [0][1][RTW89_WW][57] = 76, - [0][1][RTW89_WW][59] = 76, - [0][1][RTW89_WW][60] = 76, - [0][1][RTW89_WW][62] = 76, - [0][1][RTW89_WW][64] = 76, - [0][1][RTW89_WW][66] = 76, - [0][1][RTW89_WW][68] = 76, - [0][1][RTW89_WW][70] = 76, - [0][1][RTW89_WW][72] = 76, - [0][1][RTW89_WW][74] = 76, - [0][1][RTW89_WW][75] = 76, - [0][1][RTW89_WW][77] = 76, - [0][1][RTW89_WW][79] = 76, - [0][1][RTW89_WW][81] = 76, - [0][1][RTW89_WW][83] = 76, - [0][1][RTW89_WW][85] = 76, - [0][1][RTW89_WW][87] = 76, - [0][1][RTW89_WW][89] = 76, - [0][1][RTW89_WW][90] = 76, - [0][1][RTW89_WW][92] = 76, - [0][1][RTW89_WW][94] = 76, - [0][1][RTW89_WW][96] = 76, - [0][1][RTW89_WW][98] = 76, - [0][1][RTW89_WW][100] = 76, - [0][1][RTW89_WW][102] = 76, - [0][1][RTW89_WW][104] = 76, - [0][1][RTW89_WW][105] = 76, - [0][1][RTW89_WW][107] = 76, - [0][1][RTW89_WW][109] = 76, + [0][1][RTW89_WW][0] = -40, + [0][1][RTW89_WW][2] = -40, + [0][1][RTW89_WW][4] = -40, + [0][1][RTW89_WW][6] = -40, + [0][1][RTW89_WW][8] = -40, + [0][1][RTW89_WW][10] = -40, + [0][1][RTW89_WW][12] = -40, + [0][1][RTW89_WW][14] = -40, + [0][1][RTW89_WW][15] = -40, + [0][1][RTW89_WW][17] = -40, + [0][1][RTW89_WW][19] = -40, + [0][1][RTW89_WW][21] = -40, + [0][1][RTW89_WW][23] = -40, + [0][1][RTW89_WW][25] = -40, + [0][1][RTW89_WW][27] = -40, + [0][1][RTW89_WW][29] = -40, + [0][1][RTW89_WW][30] = -40, + [0][1][RTW89_WW][32] = -40, + [0][1][RTW89_WW][34] = -40, + [0][1][RTW89_WW][36] = -40, + [0][1][RTW89_WW][38] = -40, + [0][1][RTW89_WW][40] = -40, + [0][1][RTW89_WW][42] = -40, + [0][1][RTW89_WW][44] = -40, + [0][1][RTW89_WW][45] = -40, + [0][1][RTW89_WW][47] = -40, + [0][1][RTW89_WW][49] = -40, + [0][1][RTW89_WW][51] = -40, + [0][1][RTW89_WW][53] = -40, + [0][1][RTW89_WW][55] = -40, + [0][1][RTW89_WW][57] = -40, + [0][1][RTW89_WW][59] = -40, + [0][1][RTW89_WW][60] = -40, + [0][1][RTW89_WW][62] = -40, + [0][1][RTW89_WW][64] = -40, + [0][1][RTW89_WW][66] = -40, + [0][1][RTW89_WW][68] = -40, + [0][1][RTW89_WW][70] = -38, + [0][1][RTW89_WW][72] = -38, + [0][1][RTW89_WW][74] = -38, + [0][1][RTW89_WW][75] = -38, + [0][1][RTW89_WW][77] = -38, + [0][1][RTW89_WW][79] = -38, + [0][1][RTW89_WW][81] = -38, + [0][1][RTW89_WW][83] = -38, + [0][1][RTW89_WW][85] = -38, + [0][1][RTW89_WW][87] = -40, + [0][1][RTW89_WW][89] = -38, + [0][1][RTW89_WW][90] = -38, + [0][1][RTW89_WW][92] = -38, + [0][1][RTW89_WW][94] = -38, + [0][1][RTW89_WW][96] = -38, + [0][1][RTW89_WW][98] = -38, + [0][1][RTW89_WW][100] = -38, + [0][1][RTW89_WW][102] = -38, + [0][1][RTW89_WW][104] = -38, + [0][1][RTW89_WW][105] = -38, + [0][1][RTW89_WW][107] = -34, + [0][1][RTW89_WW][109] = -34, [0][1][RTW89_WW][111] = 0, [0][1][RTW89_WW][113] = 0, [0][1][RTW89_WW][115] = 0, [0][1][RTW89_WW][117] = 0, [0][1][RTW89_WW][119] = 0, - [1][0][RTW89_WW][0] = 76, - [1][0][RTW89_WW][2] = 76, - [1][0][RTW89_WW][4] = 76, - [1][0][RTW89_WW][6] = 76, - [1][0][RTW89_WW][8] = 76, - [1][0][RTW89_WW][10] = 76, - [1][0][RTW89_WW][12] = 76, - [1][0][RTW89_WW][14] = 76, - [1][0][RTW89_WW][15] = 76, - [1][0][RTW89_WW][17] = 76, - [1][0][RTW89_WW][19] = 76, - [1][0][RTW89_WW][21] = 76, - [1][0][RTW89_WW][23] = 76, - [1][0][RTW89_WW][25] = 76, - [1][0][RTW89_WW][27] = 76, - [1][0][RTW89_WW][29] = 76, - [1][0][RTW89_WW][30] = 76, - [1][0][RTW89_WW][32] = 76, - [1][0][RTW89_WW][34] = 76, - [1][0][RTW89_WW][36] = 76, - [1][0][RTW89_WW][38] = 76, - [1][0][RTW89_WW][40] = 76, - [1][0][RTW89_WW][42] = 76, - [1][0][RTW89_WW][44] = 76, - [1][0][RTW89_WW][45] = 76, - [1][0][RTW89_WW][47] = 76, - [1][0][RTW89_WW][49] = 76, - [1][0][RTW89_WW][51] = 76, - [1][0][RTW89_WW][53] = 76, - [1][0][RTW89_WW][55] = 76, - [1][0][RTW89_WW][57] = 76, - [1][0][RTW89_WW][59] = 76, - [1][0][RTW89_WW][60] = 76, - [1][0][RTW89_WW][62] = 76, - [1][0][RTW89_WW][64] = 76, - [1][0][RTW89_WW][66] = 76, - [1][0][RTW89_WW][68] = 76, - [1][0][RTW89_WW][70] = 76, - [1][0][RTW89_WW][72] = 76, - [1][0][RTW89_WW][74] = 76, - [1][0][RTW89_WW][75] = 76, - [1][0][RTW89_WW][77] = 76, - [1][0][RTW89_WW][79] = 76, - [1][0][RTW89_WW][81] = 76, - [1][0][RTW89_WW][83] = 76, - [1][0][RTW89_WW][85] = 76, - [1][0][RTW89_WW][87] = 76, - [1][0][RTW89_WW][89] = 76, - [1][0][RTW89_WW][90] = 76, - [1][0][RTW89_WW][92] = 76, - [1][0][RTW89_WW][94] = 76, - [1][0][RTW89_WW][96] = 76, - [1][0][RTW89_WW][98] = 76, - [1][0][RTW89_WW][100] = 76, - [1][0][RTW89_WW][102] = 76, - [1][0][RTW89_WW][104] = 76, - [1][0][RTW89_WW][105] = 76, - [1][0][RTW89_WW][107] = 76, - [1][0][RTW89_WW][109] = 76, + [1][0][RTW89_WW][0] = -4, + [1][0][RTW89_WW][2] = -4, + [1][0][RTW89_WW][4] = -4, + [1][0][RTW89_WW][6] = -4, + [1][0][RTW89_WW][8] = -4, + [1][0][RTW89_WW][10] = -4, + [1][0][RTW89_WW][12] = -4, + [1][0][RTW89_WW][14] = -4, + [1][0][RTW89_WW][15] = -4, + [1][0][RTW89_WW][17] = -4, + [1][0][RTW89_WW][19] = -4, + [1][0][RTW89_WW][21] = -4, + [1][0][RTW89_WW][23] = -4, + [1][0][RTW89_WW][25] = -4, + [1][0][RTW89_WW][27] = -4, + [1][0][RTW89_WW][29] = -4, + [1][0][RTW89_WW][30] = -4, + [1][0][RTW89_WW][32] = -4, + [1][0][RTW89_WW][34] = -4, + [1][0][RTW89_WW][36] = -4, + [1][0][RTW89_WW][38] = -4, + [1][0][RTW89_WW][40] = -4, + [1][0][RTW89_WW][42] = -4, + [1][0][RTW89_WW][44] = -4, + [1][0][RTW89_WW][45] = -4, + [1][0][RTW89_WW][47] = -4, + [1][0][RTW89_WW][49] = -4, + [1][0][RTW89_WW][51] = -4, + [1][0][RTW89_WW][53] = -4, + [1][0][RTW89_WW][55] = -4, + [1][0][RTW89_WW][57] = -4, + [1][0][RTW89_WW][59] = -4, + [1][0][RTW89_WW][60] = -4, + [1][0][RTW89_WW][62] = -4, + [1][0][RTW89_WW][64] = -4, + [1][0][RTW89_WW][66] = -4, + [1][0][RTW89_WW][68] = -4, + [1][0][RTW89_WW][70] = -4, + [1][0][RTW89_WW][72] = -4, + [1][0][RTW89_WW][74] = -4, + [1][0][RTW89_WW][75] = -4, + [1][0][RTW89_WW][77] = -4, + [1][0][RTW89_WW][79] = -4, + [1][0][RTW89_WW][81] = -4, + [1][0][RTW89_WW][83] = -4, + [1][0][RTW89_WW][85] = -4, + [1][0][RTW89_WW][87] = -4, + [1][0][RTW89_WW][89] = -4, + [1][0][RTW89_WW][90] = -4, + [1][0][RTW89_WW][92] = -4, + [1][0][RTW89_WW][94] = -4, + [1][0][RTW89_WW][96] = -4, + [1][0][RTW89_WW][98] = -4, + [1][0][RTW89_WW][100] = -4, + [1][0][RTW89_WW][102] = -4, + [1][0][RTW89_WW][104] = -4, + [1][0][RTW89_WW][105] = -4, + [1][0][RTW89_WW][107] = 1, + [1][0][RTW89_WW][109] = 2, [1][0][RTW89_WW][111] = 0, [1][0][RTW89_WW][113] = 0, [1][0][RTW89_WW][115] = 0, [1][0][RTW89_WW][117] = 0, [1][0][RTW89_WW][119] = 0, - [1][1][RTW89_WW][0] = 76, - [1][1][RTW89_WW][2] = 76, - [1][1][RTW89_WW][4] = 76, - [1][1][RTW89_WW][6] = 76, - [1][1][RTW89_WW][8] = 76, - [1][1][RTW89_WW][10] = 76, - [1][1][RTW89_WW][12] = 76, - [1][1][RTW89_WW][14] = 76, - [1][1][RTW89_WW][15] = 76, - [1][1][RTW89_WW][17] = 76, - [1][1][RTW89_WW][19] = 76, - [1][1][RTW89_WW][21] = 76, - [1][1][RTW89_WW][23] = 76, - [1][1][RTW89_WW][25] = 76, - [1][1][RTW89_WW][27] = 76, - [1][1][RTW89_WW][29] = 76, - [1][1][RTW89_WW][30] = 76, - [1][1][RTW89_WW][32] = 76, - [1][1][RTW89_WW][34] = 76, - [1][1][RTW89_WW][36] = 76, - [1][1][RTW89_WW][38] = 76, - [1][1][RTW89_WW][40] = 76, - [1][1][RTW89_WW][42] = 76, - [1][1][RTW89_WW][44] = 76, - [1][1][RTW89_WW][45] = 76, - [1][1][RTW89_WW][47] = 76, - [1][1][RTW89_WW][49] = 76, - [1][1][RTW89_WW][51] = 76, - [1][1][RTW89_WW][53] = 76, - [1][1][RTW89_WW][55] = 76, - [1][1][RTW89_WW][57] = 76, - [1][1][RTW89_WW][59] = 76, - [1][1][RTW89_WW][60] = 76, - [1][1][RTW89_WW][62] = 76, - [1][1][RTW89_WW][64] = 76, - [1][1][RTW89_WW][66] = 76, - [1][1][RTW89_WW][68] = 76, - [1][1][RTW89_WW][70] = 76, - [1][1][RTW89_WW][72] = 76, - [1][1][RTW89_WW][74] = 76, - [1][1][RTW89_WW][75] = 76, - [1][1][RTW89_WW][77] = 76, - [1][1][RTW89_WW][79] = 76, - [1][1][RTW89_WW][81] = 76, - [1][1][RTW89_WW][83] = 76, - [1][1][RTW89_WW][85] = 76, - [1][1][RTW89_WW][87] = 76, - [1][1][RTW89_WW][89] = 76, - [1][1][RTW89_WW][90] = 76, - [1][1][RTW89_WW][92] = 76, - [1][1][RTW89_WW][94] = 76, - [1][1][RTW89_WW][96] = 76, - [1][1][RTW89_WW][98] = 76, - [1][1][RTW89_WW][100] = 76, - [1][1][RTW89_WW][102] = 76, - [1][1][RTW89_WW][104] = 76, - [1][1][RTW89_WW][105] = 76, - [1][1][RTW89_WW][107] = 76, - [1][1][RTW89_WW][109] = 76, + [1][1][RTW89_WW][0] = -26, + [1][1][RTW89_WW][2] = -28, + [1][1][RTW89_WW][4] = -28, + [1][1][RTW89_WW][6] = -28, + [1][1][RTW89_WW][8] = -28, + [1][1][RTW89_WW][10] = -28, + [1][1][RTW89_WW][12] = -28, + [1][1][RTW89_WW][14] = -28, + [1][1][RTW89_WW][15] = -28, + [1][1][RTW89_WW][17] = -28, + [1][1][RTW89_WW][19] = -28, + [1][1][RTW89_WW][21] = -28, + [1][1][RTW89_WW][23] = -28, + [1][1][RTW89_WW][25] = -28, + [1][1][RTW89_WW][27] = -28, + [1][1][RTW89_WW][29] = -28, + [1][1][RTW89_WW][30] = -28, + [1][1][RTW89_WW][32] = -28, + [1][1][RTW89_WW][34] = -28, + [1][1][RTW89_WW][36] = -28, + [1][1][RTW89_WW][38] = -28, + [1][1][RTW89_WW][40] = -28, + [1][1][RTW89_WW][42] = -28, + [1][1][RTW89_WW][44] = -28, + [1][1][RTW89_WW][45] = -26, + [1][1][RTW89_WW][47] = -28, + [1][1][RTW89_WW][49] = -28, + [1][1][RTW89_WW][51] = -28, + [1][1][RTW89_WW][53] = -26, + [1][1][RTW89_WW][55] = -28, + [1][1][RTW89_WW][57] = -28, + [1][1][RTW89_WW][59] = -28, + [1][1][RTW89_WW][60] = -28, + [1][1][RTW89_WW][62] = -28, + [1][1][RTW89_WW][64] = -28, + [1][1][RTW89_WW][66] = -28, + [1][1][RTW89_WW][68] = -28, + [1][1][RTW89_WW][70] = -26, + [1][1][RTW89_WW][72] = -28, + [1][1][RTW89_WW][74] = -28, + [1][1][RTW89_WW][75] = -28, + [1][1][RTW89_WW][77] = -28, + [1][1][RTW89_WW][79] = -28, + [1][1][RTW89_WW][81] = -28, + [1][1][RTW89_WW][83] = -28, + [1][1][RTW89_WW][85] = -28, + [1][1][RTW89_WW][87] = -28, + [1][1][RTW89_WW][89] = -26, + [1][1][RTW89_WW][90] = -26, + [1][1][RTW89_WW][92] = -26, + [1][1][RTW89_WW][94] = -26, + [1][1][RTW89_WW][96] = -26, + [1][1][RTW89_WW][98] = -26, + [1][1][RTW89_WW][100] = -26, + [1][1][RTW89_WW][102] = -26, + [1][1][RTW89_WW][104] = -26, + [1][1][RTW89_WW][105] = -26, + [1][1][RTW89_WW][107] = -22, + [1][1][RTW89_WW][109] = -22, [1][1][RTW89_WW][111] = 0, [1][1][RTW89_WW][113] = 0, [1][1][RTW89_WW][115] = 0, [1][1][RTW89_WW][117] = 0, [1][1][RTW89_WW][119] = 0, - [2][0][RTW89_WW][0] = 76, - [2][0][RTW89_WW][2] = 76, - [2][0][RTW89_WW][4] = 76, - [2][0][RTW89_WW][6] = 76, - [2][0][RTW89_WW][8] = 76, - [2][0][RTW89_WW][10] = 76, - [2][0][RTW89_WW][12] = 76, - [2][0][RTW89_WW][14] = 76, - [2][0][RTW89_WW][15] = 76, - [2][0][RTW89_WW][17] = 76, - [2][0][RTW89_WW][19] = 76, - [2][0][RTW89_WW][21] = 76, - [2][0][RTW89_WW][23] = 76, - [2][0][RTW89_WW][25] = 76, - [2][0][RTW89_WW][27] = 76, - [2][0][RTW89_WW][29] = 76, - [2][0][RTW89_WW][30] = 76, - [2][0][RTW89_WW][32] = 76, - [2][0][RTW89_WW][34] = 76, - [2][0][RTW89_WW][36] = 76, - [2][0][RTW89_WW][38] = 76, - [2][0][RTW89_WW][40] = 76, - [2][0][RTW89_WW][42] = 76, - [2][0][RTW89_WW][44] = 76, - [2][0][RTW89_WW][45] = 76, - [2][0][RTW89_WW][47] = 76, - [2][0][RTW89_WW][49] = 76, - [2][0][RTW89_WW][51] = 76, - [2][0][RTW89_WW][53] = 76, - [2][0][RTW89_WW][55] = 76, - [2][0][RTW89_WW][57] = 76, - [2][0][RTW89_WW][59] = 76, - [2][0][RTW89_WW][60] = 76, - [2][0][RTW89_WW][62] = 76, - [2][0][RTW89_WW][64] = 76, - [2][0][RTW89_WW][66] = 76, - [2][0][RTW89_WW][68] = 76, - [2][0][RTW89_WW][70] = 76, - [2][0][RTW89_WW][72] = 76, - [2][0][RTW89_WW][74] = 76, - [2][0][RTW89_WW][75] = 76, - [2][0][RTW89_WW][77] = 76, - [2][0][RTW89_WW][79] = 76, - [2][0][RTW89_WW][81] = 76, - [2][0][RTW89_WW][83] = 76, - [2][0][RTW89_WW][85] = 76, - [2][0][RTW89_WW][87] = 76, - [2][0][RTW89_WW][89] = 76, - [2][0][RTW89_WW][90] = 76, - [2][0][RTW89_WW][92] = 76, - [2][0][RTW89_WW][94] = 76, - [2][0][RTW89_WW][96] = 76, - [2][0][RTW89_WW][98] = 76, - [2][0][RTW89_WW][100] = 76, - [2][0][RTW89_WW][102] = 76, - [2][0][RTW89_WW][104] = 76, - [2][0][RTW89_WW][105] = 76, - [2][0][RTW89_WW][107] = 76, - [2][0][RTW89_WW][109] = 76, + [2][0][RTW89_WW][0] = 8, + [2][0][RTW89_WW][2] = 8, + [2][0][RTW89_WW][4] = 8, + [2][0][RTW89_WW][6] = 8, + [2][0][RTW89_WW][8] = 8, + [2][0][RTW89_WW][10] = 8, + [2][0][RTW89_WW][12] = 8, + [2][0][RTW89_WW][14] = 8, + [2][0][RTW89_WW][15] = 8, + [2][0][RTW89_WW][17] = 8, + [2][0][RTW89_WW][19] = 8, + [2][0][RTW89_WW][21] = 8, + [2][0][RTW89_WW][23] = 8, + [2][0][RTW89_WW][25] = 8, + [2][0][RTW89_WW][27] = 8, + [2][0][RTW89_WW][29] = 8, + [2][0][RTW89_WW][30] = 8, + [2][0][RTW89_WW][32] = 8, + [2][0][RTW89_WW][34] = 8, + [2][0][RTW89_WW][36] = 8, + [2][0][RTW89_WW][38] = 8, + [2][0][RTW89_WW][40] = 8, + [2][0][RTW89_WW][42] = 8, + [2][0][RTW89_WW][44] = 8, + [2][0][RTW89_WW][45] = 8, + [2][0][RTW89_WW][47] = 8, + [2][0][RTW89_WW][49] = 8, + [2][0][RTW89_WW][51] = 8, + [2][0][RTW89_WW][53] = 8, + [2][0][RTW89_WW][55] = 8, + [2][0][RTW89_WW][57] = 8, + [2][0][RTW89_WW][59] = 8, + [2][0][RTW89_WW][60] = 8, + [2][0][RTW89_WW][62] = 8, + [2][0][RTW89_WW][64] = 8, + [2][0][RTW89_WW][66] = 8, + [2][0][RTW89_WW][68] = 8, + [2][0][RTW89_WW][70] = 8, + [2][0][RTW89_WW][72] = 8, + [2][0][RTW89_WW][74] = 8, + [2][0][RTW89_WW][75] = 8, + [2][0][RTW89_WW][77] = 8, + [2][0][RTW89_WW][79] = 8, + [2][0][RTW89_WW][81] = 8, + [2][0][RTW89_WW][83] = 8, + [2][0][RTW89_WW][85] = 8, + [2][0][RTW89_WW][87] = 8, + [2][0][RTW89_WW][89] = 8, + [2][0][RTW89_WW][90] = 8, + [2][0][RTW89_WW][92] = 8, + [2][0][RTW89_WW][94] = 8, + [2][0][RTW89_WW][96] = 8, + [2][0][RTW89_WW][98] = 8, + [2][0][RTW89_WW][100] = 8, + [2][0][RTW89_WW][102] = 8, + [2][0][RTW89_WW][104] = 8, + [2][0][RTW89_WW][105] = 8, + [2][0][RTW89_WW][107] = 10, + [2][0][RTW89_WW][109] = 12, [2][0][RTW89_WW][111] = 0, [2][0][RTW89_WW][113] = 0, [2][0][RTW89_WW][115] = 0, [2][0][RTW89_WW][117] = 0, [2][0][RTW89_WW][119] = 0, - [2][1][RTW89_WW][0] = 76, - [2][1][RTW89_WW][2] = 76, - [2][1][RTW89_WW][4] = 76, - [2][1][RTW89_WW][6] = 76, - [2][1][RTW89_WW][8] = 76, - [2][1][RTW89_WW][10] = 76, - [2][1][RTW89_WW][12] = 76, - [2][1][RTW89_WW][14] = 76, - [2][1][RTW89_WW][15] = 76, - [2][1][RTW89_WW][17] = 76, - [2][1][RTW89_WW][19] = 76, - [2][1][RTW89_WW][21] = 76, - [2][1][RTW89_WW][23] = 76, - [2][1][RTW89_WW][25] = 76, - [2][1][RTW89_WW][27] = 76, - [2][1][RTW89_WW][29] = 76, - [2][1][RTW89_WW][30] = 76, - [2][1][RTW89_WW][32] = 76, - [2][1][RTW89_WW][34] = 76, - [2][1][RTW89_WW][36] = 76, - [2][1][RTW89_WW][38] = 76, - [2][1][RTW89_WW][40] = 76, - [2][1][RTW89_WW][42] = 76, - [2][1][RTW89_WW][44] = 76, - [2][1][RTW89_WW][45] = 76, - [2][1][RTW89_WW][47] = 76, - [2][1][RTW89_WW][49] = 76, - [2][1][RTW89_WW][51] = 76, - [2][1][RTW89_WW][53] = 76, - [2][1][RTW89_WW][55] = 76, - [2][1][RTW89_WW][57] = 76, - [2][1][RTW89_WW][59] = 76, - [2][1][RTW89_WW][60] = 76, - [2][1][RTW89_WW][62] = 76, - [2][1][RTW89_WW][64] = 76, - [2][1][RTW89_WW][66] = 76, - [2][1][RTW89_WW][68] = 76, - [2][1][RTW89_WW][70] = 76, - [2][1][RTW89_WW][72] = 76, - [2][1][RTW89_WW][74] = 76, - [2][1][RTW89_WW][75] = 76, - [2][1][RTW89_WW][77] = 76, - [2][1][RTW89_WW][79] = 76, - [2][1][RTW89_WW][81] = 76, - [2][1][RTW89_WW][83] = 76, - [2][1][RTW89_WW][85] = 76, - [2][1][RTW89_WW][87] = 76, - [2][1][RTW89_WW][89] = 76, - [2][1][RTW89_WW][90] = 76, - [2][1][RTW89_WW][92] = 76, - [2][1][RTW89_WW][94] = 76, - [2][1][RTW89_WW][96] = 76, - [2][1][RTW89_WW][98] = 76, - [2][1][RTW89_WW][100] = 76, - [2][1][RTW89_WW][102] = 76, - [2][1][RTW89_WW][104] = 76, - [2][1][RTW89_WW][105] = 76, - [2][1][RTW89_WW][107] = 76, - [2][1][RTW89_WW][109] = 76, + [2][1][RTW89_WW][0] = -16, + [2][1][RTW89_WW][2] = -16, + [2][1][RTW89_WW][4] = -16, + [2][1][RTW89_WW][6] = -16, + [2][1][RTW89_WW][8] = -16, + [2][1][RTW89_WW][10] = -16, + [2][1][RTW89_WW][12] = -16, + [2][1][RTW89_WW][14] = -16, + [2][1][RTW89_WW][15] = -16, + [2][1][RTW89_WW][17] = -16, + [2][1][RTW89_WW][19] = -16, + [2][1][RTW89_WW][21] = -16, + [2][1][RTW89_WW][23] = -16, + [2][1][RTW89_WW][25] = -16, + [2][1][RTW89_WW][27] = -16, + [2][1][RTW89_WW][29] = -16, + [2][1][RTW89_WW][30] = -16, + [2][1][RTW89_WW][32] = -16, + [2][1][RTW89_WW][34] = -16, + [2][1][RTW89_WW][36] = -16, + [2][1][RTW89_WW][38] = -16, + [2][1][RTW89_WW][40] = -16, + [2][1][RTW89_WW][42] = -16, + [2][1][RTW89_WW][44] = -16, + [2][1][RTW89_WW][45] = -16, + [2][1][RTW89_WW][47] = -16, + [2][1][RTW89_WW][49] = -16, + [2][1][RTW89_WW][51] = -16, + [2][1][RTW89_WW][53] = -16, + [2][1][RTW89_WW][55] = -16, + [2][1][RTW89_WW][57] = -16, + [2][1][RTW89_WW][59] = -16, + [2][1][RTW89_WW][60] = -16, + [2][1][RTW89_WW][62] = -16, + [2][1][RTW89_WW][64] = -16, + [2][1][RTW89_WW][66] = -16, + [2][1][RTW89_WW][68] = -16, + [2][1][RTW89_WW][70] = -16, + [2][1][RTW89_WW][72] = -16, + [2][1][RTW89_WW][74] = -16, + [2][1][RTW89_WW][75] = -16, + [2][1][RTW89_WW][77] = -16, + [2][1][RTW89_WW][79] = -16, + [2][1][RTW89_WW][81] = -16, + [2][1][RTW89_WW][83] = -16, + [2][1][RTW89_WW][85] = -18, + [2][1][RTW89_WW][87] = -16, + [2][1][RTW89_WW][89] = -16, + [2][1][RTW89_WW][90] = -16, + [2][1][RTW89_WW][92] = -16, + [2][1][RTW89_WW][94] = -16, + [2][1][RTW89_WW][96] = -16, + [2][1][RTW89_WW][98] = -16, + [2][1][RTW89_WW][100] = -16, + [2][1][RTW89_WW][102] = -16, + [2][1][RTW89_WW][104] = -16, + [2][1][RTW89_WW][105] = -16, + [2][1][RTW89_WW][107] = -12, + [2][1][RTW89_WW][109] = -10, [2][1][RTW89_WW][111] = 0, [2][1][RTW89_WW][113] = 0, [2][1][RTW89_WW][115] = 0, [2][1][RTW89_WW][117] = 0, [2][1][RTW89_WW][119] = 0, - [0][0][RTW89_FCC][0] = 76, - [0][0][RTW89_FCC][2] = 76, - [0][0][RTW89_FCC][4] = 76, - [0][0][RTW89_FCC][6] = 76, - [0][0][RTW89_FCC][8] = 76, - [0][0][RTW89_FCC][10] = 76, - [0][0][RTW89_FCC][12] = 76, - [0][0][RTW89_FCC][14] = 76, - [0][0][RTW89_FCC][15] = 76, - [0][0][RTW89_FCC][17] = 76, - [0][0][RTW89_FCC][19] = 76, - [0][0][RTW89_FCC][21] = 76, - [0][0][RTW89_FCC][23] = 76, - [0][0][RTW89_FCC][25] = 76, - [0][0][RTW89_FCC][27] = 76, - [0][0][RTW89_FCC][29] = 76, - [0][0][RTW89_FCC][30] = 76, - [0][0][RTW89_FCC][32] = 76, - [0][0][RTW89_FCC][34] = 76, - [0][0][RTW89_FCC][36] = 76, - [0][0][RTW89_FCC][38] = 76, - [0][0][RTW89_FCC][40] = 76, - [0][0][RTW89_FCC][42] = 76, - [0][0][RTW89_FCC][44] = 76, - [0][0][RTW89_FCC][45] = 76, - [0][0][RTW89_FCC][47] = 76, - [0][0][RTW89_FCC][49] = 76, - [0][0][RTW89_FCC][51] = 76, - [0][0][RTW89_FCC][53] = 76, - [0][0][RTW89_FCC][55] = 76, - [0][0][RTW89_FCC][57] = 76, - [0][0][RTW89_FCC][59] = 76, - [0][0][RTW89_FCC][60] = 76, - [0][0][RTW89_FCC][62] = 76, - [0][0][RTW89_FCC][64] = 76, - [0][0][RTW89_FCC][66] = 76, - [0][0][RTW89_FCC][68] = 76, - [0][0][RTW89_FCC][70] = 76, - [0][0][RTW89_FCC][72] = 76, - [0][0][RTW89_FCC][74] = 76, - [0][0][RTW89_FCC][75] = 76, - [0][0][RTW89_FCC][77] = 76, - [0][0][RTW89_FCC][79] = 76, - [0][0][RTW89_FCC][81] = 76, - [0][0][RTW89_FCC][83] = 76, - [0][0][RTW89_FCC][85] = 76, - [0][0][RTW89_FCC][87] = 76, - [0][0][RTW89_FCC][89] = 76, - [0][0][RTW89_FCC][90] = 76, - [0][0][RTW89_FCC][92] = 76, - [0][0][RTW89_FCC][94] = 76, - [0][0][RTW89_FCC][96] = 76, - [0][0][RTW89_FCC][98] = 76, - [0][0][RTW89_FCC][100] = 76, - [0][0][RTW89_FCC][102] = 76, - [0][0][RTW89_FCC][104] = 76, - [0][0][RTW89_FCC][105] = 76, - [0][0][RTW89_FCC][107] = 76, - [0][0][RTW89_FCC][109] = 76, + [0][0][RTW89_FCC][0] = -16, + [0][0][RTW89_ETSI][0] = 32, + [0][0][RTW89_FCC][2] = -18, + [0][0][RTW89_ETSI][2] = 32, + [0][0][RTW89_FCC][4] = -18, + [0][0][RTW89_ETSI][4] = 32, + [0][0][RTW89_FCC][6] = -18, + [0][0][RTW89_ETSI][6] = 32, + [0][0][RTW89_FCC][8] = -18, + [0][0][RTW89_ETSI][8] = 32, + [0][0][RTW89_FCC][10] = -18, + [0][0][RTW89_ETSI][10] = 32, + [0][0][RTW89_FCC][12] = -18, + [0][0][RTW89_ETSI][12] = 32, + [0][0][RTW89_FCC][14] = -18, + [0][0][RTW89_ETSI][14] = 32, + [0][0][RTW89_FCC][15] = -18, + [0][0][RTW89_ETSI][15] = 32, + [0][0][RTW89_FCC][17] = -18, + [0][0][RTW89_ETSI][17] = 32, + [0][0][RTW89_FCC][19] = -18, + [0][0][RTW89_ETSI][19] = 32, + [0][0][RTW89_FCC][21] = -18, + [0][0][RTW89_ETSI][21] = 32, + [0][0][RTW89_FCC][23] = -18, + [0][0][RTW89_ETSI][23] = 32, + [0][0][RTW89_FCC][25] = -18, + [0][0][RTW89_ETSI][25] = 32, + [0][0][RTW89_FCC][27] = -18, + [0][0][RTW89_ETSI][27] = 32, + [0][0][RTW89_FCC][29] = -18, + [0][0][RTW89_ETSI][29] = 32, + [0][0][RTW89_FCC][30] = -18, + [0][0][RTW89_ETSI][30] = 32, + [0][0][RTW89_FCC][32] = -18, + [0][0][RTW89_ETSI][32] = 32, + [0][0][RTW89_FCC][34] = -18, + [0][0][RTW89_ETSI][34] = 32, + [0][0][RTW89_FCC][36] = -18, + [0][0][RTW89_ETSI][36] = 32, + [0][0][RTW89_FCC][38] = -18, + [0][0][RTW89_ETSI][38] = 32, + [0][0][RTW89_FCC][40] = -18, + [0][0][RTW89_ETSI][40] = 32, + [0][0][RTW89_FCC][42] = -18, + [0][0][RTW89_ETSI][42] = 32, + [0][0][RTW89_FCC][44] = -16, + [0][0][RTW89_ETSI][44] = 32, + [0][0][RTW89_FCC][45] = -16, + [0][0][RTW89_ETSI][45] = 127, + [0][0][RTW89_FCC][47] = -18, + [0][0][RTW89_ETSI][47] = 127, + [0][0][RTW89_FCC][49] = -18, + [0][0][RTW89_ETSI][49] = 127, + [0][0][RTW89_FCC][51] = -18, + [0][0][RTW89_ETSI][51] = 127, + [0][0][RTW89_FCC][53] = -16, + [0][0][RTW89_ETSI][53] = 127, + [0][0][RTW89_FCC][55] = -18, + [0][0][RTW89_ETSI][55] = 127, + [0][0][RTW89_FCC][57] = -18, + [0][0][RTW89_ETSI][57] = 127, + [0][0][RTW89_FCC][59] = -18, + [0][0][RTW89_ETSI][59] = 127, + [0][0][RTW89_FCC][60] = -18, + [0][0][RTW89_ETSI][60] = 127, + [0][0][RTW89_FCC][62] = -18, + [0][0][RTW89_ETSI][62] = 127, + [0][0][RTW89_FCC][64] = -18, + [0][0][RTW89_ETSI][64] = 127, + [0][0][RTW89_FCC][66] = -18, + [0][0][RTW89_ETSI][66] = 127, + [0][0][RTW89_FCC][68] = -18, + [0][0][RTW89_ETSI][68] = 127, + [0][0][RTW89_FCC][70] = -16, + [0][0][RTW89_ETSI][70] = 127, + [0][0][RTW89_FCC][72] = -18, + [0][0][RTW89_ETSI][72] = 127, + [0][0][RTW89_FCC][74] = -18, + [0][0][RTW89_ETSI][74] = 127, + [0][0][RTW89_FCC][75] = -18, + [0][0][RTW89_ETSI][75] = 127, + [0][0][RTW89_FCC][77] = -18, + [0][0][RTW89_ETSI][77] = 127, + [0][0][RTW89_FCC][79] = -18, + [0][0][RTW89_ETSI][79] = 127, + [0][0][RTW89_FCC][81] = -18, + [0][0][RTW89_ETSI][81] = 127, + [0][0][RTW89_FCC][83] = -18, + [0][0][RTW89_ETSI][83] = 127, + [0][0][RTW89_FCC][85] = -18, + [0][0][RTW89_ETSI][85] = 127, + [0][0][RTW89_FCC][87] = -16, + [0][0][RTW89_ETSI][87] = 127, + [0][0][RTW89_FCC][89] = -16, + [0][0][RTW89_ETSI][89] = 127, + [0][0][RTW89_FCC][90] = -16, + [0][0][RTW89_ETSI][90] = 127, + [0][0][RTW89_FCC][92] = -16, + [0][0][RTW89_ETSI][92] = 127, + [0][0][RTW89_FCC][94] = -16, + [0][0][RTW89_ETSI][94] = 127, + [0][0][RTW89_FCC][96] = -16, + [0][0][RTW89_ETSI][96] = 127, + [0][0][RTW89_FCC][98] = -16, + [0][0][RTW89_ETSI][98] = 127, + [0][0][RTW89_FCC][100] = -16, + [0][0][RTW89_ETSI][100] = 127, + [0][0][RTW89_FCC][102] = -16, + [0][0][RTW89_ETSI][102] = 127, + [0][0][RTW89_FCC][104] = -16, + [0][0][RTW89_ETSI][104] = 127, + [0][0][RTW89_FCC][105] = -16, + [0][0][RTW89_ETSI][105] = 127, + [0][0][RTW89_FCC][107] = -12, + [0][0][RTW89_ETSI][107] = 127, + [0][0][RTW89_FCC][109] = -12, + [0][0][RTW89_ETSI][109] = 127, [0][0][RTW89_FCC][111] = 127, + [0][0][RTW89_ETSI][111] = 127, [0][0][RTW89_FCC][113] = 127, + [0][0][RTW89_ETSI][113] = 127, [0][0][RTW89_FCC][115] = 127, + [0][0][RTW89_ETSI][115] = 127, [0][0][RTW89_FCC][117] = 127, + [0][0][RTW89_ETSI][117] = 127, [0][0][RTW89_FCC][119] = 127, - [0][1][RTW89_FCC][0] = 76, - [0][1][RTW89_FCC][2] = 76, - [0][1][RTW89_FCC][4] = 76, - [0][1][RTW89_FCC][6] = 76, - [0][1][RTW89_FCC][8] = 76, - [0][1][RTW89_FCC][10] = 76, - [0][1][RTW89_FCC][12] = 76, - [0][1][RTW89_FCC][14] = 76, - [0][1][RTW89_FCC][15] = 76, - [0][1][RTW89_FCC][17] = 76, - [0][1][RTW89_FCC][19] = 76, - [0][1][RTW89_FCC][21] = 76, - [0][1][RTW89_FCC][23] = 76, - [0][1][RTW89_FCC][25] = 76, - [0][1][RTW89_FCC][27] = 76, - [0][1][RTW89_FCC][29] = 76, - [0][1][RTW89_FCC][30] = 76, - [0][1][RTW89_FCC][32] = 76, - [0][1][RTW89_FCC][34] = 76, - [0][1][RTW89_FCC][36] = 76, - [0][1][RTW89_FCC][38] = 76, - [0][1][RTW89_FCC][40] = 76, - [0][1][RTW89_FCC][42] = 76, - [0][1][RTW89_FCC][44] = 76, - [0][1][RTW89_FCC][45] = 76, - [0][1][RTW89_FCC][47] = 76, - [0][1][RTW89_FCC][49] = 76, - [0][1][RTW89_FCC][51] = 76, - [0][1][RTW89_FCC][53] = 76, - [0][1][RTW89_FCC][55] = 76, - [0][1][RTW89_FCC][57] = 76, - [0][1][RTW89_FCC][59] = 76, - [0][1][RTW89_FCC][60] = 76, - [0][1][RTW89_FCC][62] = 76, - [0][1][RTW89_FCC][64] = 76, - [0][1][RTW89_FCC][66] = 76, - [0][1][RTW89_FCC][68] = 76, - [0][1][RTW89_FCC][70] = 76, - [0][1][RTW89_FCC][72] = 76, - [0][1][RTW89_FCC][74] = 76, - [0][1][RTW89_FCC][75] = 76, - [0][1][RTW89_FCC][77] = 76, - [0][1][RTW89_FCC][79] = 76, - [0][1][RTW89_FCC][81] = 76, - [0][1][RTW89_FCC][83] = 76, - [0][1][RTW89_FCC][85] = 76, - [0][1][RTW89_FCC][87] = 76, - [0][1][RTW89_FCC][89] = 76, - [0][1][RTW89_FCC][90] = 76, - [0][1][RTW89_FCC][92] = 76, - [0][1][RTW89_FCC][94] = 76, - [0][1][RTW89_FCC][96] = 76, - [0][1][RTW89_FCC][98] = 76, - [0][1][RTW89_FCC][100] = 76, - [0][1][RTW89_FCC][102] = 76, - [0][1][RTW89_FCC][104] = 76, - [0][1][RTW89_FCC][105] = 76, - [0][1][RTW89_FCC][107] = 76, - [0][1][RTW89_FCC][109] = 76, + [0][0][RTW89_ETSI][119] = 127, + [0][1][RTW89_FCC][0] = -40, + [0][1][RTW89_ETSI][0] = 20, + [0][1][RTW89_FCC][2] = -40, + [0][1][RTW89_ETSI][2] = 20, + [0][1][RTW89_FCC][4] = -40, + [0][1][RTW89_ETSI][4] = 20, + [0][1][RTW89_FCC][6] = -40, + [0][1][RTW89_ETSI][6] = 20, + [0][1][RTW89_FCC][8] = -40, + [0][1][RTW89_ETSI][8] = 20, + [0][1][RTW89_FCC][10] = -40, + [0][1][RTW89_ETSI][10] = 20, + [0][1][RTW89_FCC][12] = -40, + [0][1][RTW89_ETSI][12] = 20, + [0][1][RTW89_FCC][14] = -40, + [0][1][RTW89_ETSI][14] = 20, + [0][1][RTW89_FCC][15] = -40, + [0][1][RTW89_ETSI][15] = 20, + [0][1][RTW89_FCC][17] = -40, + [0][1][RTW89_ETSI][17] = 20, + [0][1][RTW89_FCC][19] = -40, + [0][1][RTW89_ETSI][19] = 20, + [0][1][RTW89_FCC][21] = -40, + [0][1][RTW89_ETSI][21] = 20, + [0][1][RTW89_FCC][23] = -40, + [0][1][RTW89_ETSI][23] = 20, + [0][1][RTW89_FCC][25] = -40, + [0][1][RTW89_ETSI][25] = 20, + [0][1][RTW89_FCC][27] = -40, + [0][1][RTW89_ETSI][27] = 20, + [0][1][RTW89_FCC][29] = -40, + [0][1][RTW89_ETSI][29] = 20, + [0][1][RTW89_FCC][30] = -40, + [0][1][RTW89_ETSI][30] = 20, + [0][1][RTW89_FCC][32] = -40, + [0][1][RTW89_ETSI][32] = 20, + [0][1][RTW89_FCC][34] = -40, + [0][1][RTW89_ETSI][34] = 20, + [0][1][RTW89_FCC][36] = -40, + [0][1][RTW89_ETSI][36] = 20, + [0][1][RTW89_FCC][38] = -40, + [0][1][RTW89_ETSI][38] = 20, + [0][1][RTW89_FCC][40] = -40, + [0][1][RTW89_ETSI][40] = 20, + [0][1][RTW89_FCC][42] = -40, + [0][1][RTW89_ETSI][42] = 20, + [0][1][RTW89_FCC][44] = -40, + [0][1][RTW89_ETSI][44] = 20, + [0][1][RTW89_FCC][45] = -40, + [0][1][RTW89_ETSI][45] = 127, + [0][1][RTW89_FCC][47] = -40, + [0][1][RTW89_ETSI][47] = 127, + [0][1][RTW89_FCC][49] = -40, + [0][1][RTW89_ETSI][49] = 127, + [0][1][RTW89_FCC][51] = -40, + [0][1][RTW89_ETSI][51] = 127, + [0][1][RTW89_FCC][53] = -40, + [0][1][RTW89_ETSI][53] = 127, + [0][1][RTW89_FCC][55] = -40, + [0][1][RTW89_ETSI][55] = 127, + [0][1][RTW89_FCC][57] = -40, + [0][1][RTW89_ETSI][57] = 127, + [0][1][RTW89_FCC][59] = -40, + [0][1][RTW89_ETSI][59] = 127, + [0][1][RTW89_FCC][60] = -40, + [0][1][RTW89_ETSI][60] = 127, + [0][1][RTW89_FCC][62] = -40, + [0][1][RTW89_ETSI][62] = 127, + [0][1][RTW89_FCC][64] = -40, + [0][1][RTW89_ETSI][64] = 127, + [0][1][RTW89_FCC][66] = -40, + [0][1][RTW89_ETSI][66] = 127, + [0][1][RTW89_FCC][68] = -40, + [0][1][RTW89_ETSI][68] = 127, + [0][1][RTW89_FCC][70] = -38, + [0][1][RTW89_ETSI][70] = 127, + [0][1][RTW89_FCC][72] = -38, + [0][1][RTW89_ETSI][72] = 127, + [0][1][RTW89_FCC][74] = -38, + [0][1][RTW89_ETSI][74] = 127, + [0][1][RTW89_FCC][75] = -38, + [0][1][RTW89_ETSI][75] = 127, + [0][1][RTW89_FCC][77] = -38, + [0][1][RTW89_ETSI][77] = 127, + [0][1][RTW89_FCC][79] = -38, + [0][1][RTW89_ETSI][79] = 127, + [0][1][RTW89_FCC][81] = -38, + [0][1][RTW89_ETSI][81] = 127, + [0][1][RTW89_FCC][83] = -38, + [0][1][RTW89_ETSI][83] = 127, + [0][1][RTW89_FCC][85] = -38, + [0][1][RTW89_ETSI][85] = 127, + [0][1][RTW89_FCC][87] = -40, + [0][1][RTW89_ETSI][87] = 127, + [0][1][RTW89_FCC][89] = -38, + [0][1][RTW89_ETSI][89] = 127, + [0][1][RTW89_FCC][90] = -38, + [0][1][RTW89_ETSI][90] = 127, + [0][1][RTW89_FCC][92] = -38, + [0][1][RTW89_ETSI][92] = 127, + [0][1][RTW89_FCC][94] = -38, + [0][1][RTW89_ETSI][94] = 127, + [0][1][RTW89_FCC][96] = -38, + [0][1][RTW89_ETSI][96] = 127, + [0][1][RTW89_FCC][98] = -38, + [0][1][RTW89_ETSI][98] = 127, + [0][1][RTW89_FCC][100] = -38, + [0][1][RTW89_ETSI][100] = 127, + [0][1][RTW89_FCC][102] = -38, + [0][1][RTW89_ETSI][102] = 127, + [0][1][RTW89_FCC][104] = -38, + [0][1][RTW89_ETSI][104] = 127, + [0][1][RTW89_FCC][105] = -38, + [0][1][RTW89_ETSI][105] = 127, + [0][1][RTW89_FCC][107] = -34, + [0][1][RTW89_ETSI][107] = 127, + [0][1][RTW89_FCC][109] = -34, + [0][1][RTW89_ETSI][109] = 127, [0][1][RTW89_FCC][111] = 127, + [0][1][RTW89_ETSI][111] = 127, [0][1][RTW89_FCC][113] = 127, + [0][1][RTW89_ETSI][113] = 127, [0][1][RTW89_FCC][115] = 127, + [0][1][RTW89_ETSI][115] = 127, [0][1][RTW89_FCC][117] = 127, + [0][1][RTW89_ETSI][117] = 127, [0][1][RTW89_FCC][119] = 127, - [1][0][RTW89_FCC][0] = 76, - [1][0][RTW89_FCC][2] = 76, - [1][0][RTW89_FCC][4] = 76, - [1][0][RTW89_FCC][6] = 76, - [1][0][RTW89_FCC][8] = 76, - [1][0][RTW89_FCC][10] = 76, - [1][0][RTW89_FCC][12] = 76, - [1][0][RTW89_FCC][14] = 76, - [1][0][RTW89_FCC][15] = 76, - [1][0][RTW89_FCC][17] = 76, - [1][0][RTW89_FCC][19] = 76, - [1][0][RTW89_FCC][21] = 76, - [1][0][RTW89_FCC][23] = 76, - [1][0][RTW89_FCC][25] = 76, - [1][0][RTW89_FCC][27] = 76, - [1][0][RTW89_FCC][29] = 76, - [1][0][RTW89_FCC][30] = 76, - [1][0][RTW89_FCC][32] = 76, - [1][0][RTW89_FCC][34] = 76, - [1][0][RTW89_FCC][36] = 76, - [1][0][RTW89_FCC][38] = 76, - [1][0][RTW89_FCC][40] = 76, - [1][0][RTW89_FCC][42] = 76, - [1][0][RTW89_FCC][44] = 76, - [1][0][RTW89_FCC][45] = 76, - [1][0][RTW89_FCC][47] = 76, - [1][0][RTW89_FCC][49] = 76, - [1][0][RTW89_FCC][51] = 76, - [1][0][RTW89_FCC][53] = 76, - [1][0][RTW89_FCC][55] = 76, - [1][0][RTW89_FCC][57] = 76, - [1][0][RTW89_FCC][59] = 76, - [1][0][RTW89_FCC][60] = 76, - [1][0][RTW89_FCC][62] = 76, - [1][0][RTW89_FCC][64] = 76, - [1][0][RTW89_FCC][66] = 76, - [1][0][RTW89_FCC][68] = 76, - [1][0][RTW89_FCC][70] = 76, - [1][0][RTW89_FCC][72] = 76, - [1][0][RTW89_FCC][74] = 76, - [1][0][RTW89_FCC][75] = 76, - [1][0][RTW89_FCC][77] = 76, - [1][0][RTW89_FCC][79] = 76, - [1][0][RTW89_FCC][81] = 76, - [1][0][RTW89_FCC][83] = 76, - [1][0][RTW89_FCC][85] = 76, - [1][0][RTW89_FCC][87] = 76, - [1][0][RTW89_FCC][89] = 76, - [1][0][RTW89_FCC][90] = 76, - [1][0][RTW89_FCC][92] = 76, - [1][0][RTW89_FCC][94] = 76, - [1][0][RTW89_FCC][96] = 76, - [1][0][RTW89_FCC][98] = 76, - [1][0][RTW89_FCC][100] = 76, - [1][0][RTW89_FCC][102] = 76, - [1][0][RTW89_FCC][104] = 76, - [1][0][RTW89_FCC][105] = 76, - [1][0][RTW89_FCC][107] = 76, - [1][0][RTW89_FCC][109] = 76, + [0][1][RTW89_ETSI][119] = 127, + [1][0][RTW89_FCC][0] = -4, + [1][0][RTW89_ETSI][0] = 46, + [1][0][RTW89_FCC][2] = -4, + [1][0][RTW89_ETSI][2] = 46, + [1][0][RTW89_FCC][4] = -4, + [1][0][RTW89_ETSI][4] = 46, + [1][0][RTW89_FCC][6] = -4, + [1][0][RTW89_ETSI][6] = 46, + [1][0][RTW89_FCC][8] = -4, + [1][0][RTW89_ETSI][8] = 46, + [1][0][RTW89_FCC][10] = -4, + [1][0][RTW89_ETSI][10] = 46, + [1][0][RTW89_FCC][12] = -4, + [1][0][RTW89_ETSI][12] = 46, + [1][0][RTW89_FCC][14] = -4, + [1][0][RTW89_ETSI][14] = 46, + [1][0][RTW89_FCC][15] = -4, + [1][0][RTW89_ETSI][15] = 46, + [1][0][RTW89_FCC][17] = -4, + [1][0][RTW89_ETSI][17] = 46, + [1][0][RTW89_FCC][19] = -4, + [1][0][RTW89_ETSI][19] = 46, + [1][0][RTW89_FCC][21] = -4, + [1][0][RTW89_ETSI][21] = 46, + [1][0][RTW89_FCC][23] = -4, + [1][0][RTW89_ETSI][23] = 46, + [1][0][RTW89_FCC][25] = -4, + [1][0][RTW89_ETSI][25] = 46, + [1][0][RTW89_FCC][27] = -4, + [1][0][RTW89_ETSI][27] = 46, + [1][0][RTW89_FCC][29] = -4, + [1][0][RTW89_ETSI][29] = 46, + [1][0][RTW89_FCC][30] = -4, + [1][0][RTW89_ETSI][30] = 46, + [1][0][RTW89_FCC][32] = -4, + [1][0][RTW89_ETSI][32] = 46, + [1][0][RTW89_FCC][34] = -4, + [1][0][RTW89_ETSI][34] = 46, + [1][0][RTW89_FCC][36] = -4, + [1][0][RTW89_ETSI][36] = 46, + [1][0][RTW89_FCC][38] = -4, + [1][0][RTW89_ETSI][38] = 46, + [1][0][RTW89_FCC][40] = -4, + [1][0][RTW89_ETSI][40] = 46, + [1][0][RTW89_FCC][42] = -4, + [1][0][RTW89_ETSI][42] = 46, + [1][0][RTW89_FCC][44] = -4, + [1][0][RTW89_ETSI][44] = 46, + [1][0][RTW89_FCC][45] = -4, + [1][0][RTW89_ETSI][45] = 127, + [1][0][RTW89_FCC][47] = -4, + [1][0][RTW89_ETSI][47] = 127, + [1][0][RTW89_FCC][49] = -4, + [1][0][RTW89_ETSI][49] = 127, + [1][0][RTW89_FCC][51] = -4, + [1][0][RTW89_ETSI][51] = 127, + [1][0][RTW89_FCC][53] = -4, + [1][0][RTW89_ETSI][53] = 127, + [1][0][RTW89_FCC][55] = -4, + [1][0][RTW89_ETSI][55] = 127, + [1][0][RTW89_FCC][57] = -4, + [1][0][RTW89_ETSI][57] = 127, + [1][0][RTW89_FCC][59] = -4, + [1][0][RTW89_ETSI][59] = 127, + [1][0][RTW89_FCC][60] = -4, + [1][0][RTW89_ETSI][60] = 127, + [1][0][RTW89_FCC][62] = -4, + [1][0][RTW89_ETSI][62] = 127, + [1][0][RTW89_FCC][64] = -4, + [1][0][RTW89_ETSI][64] = 127, + [1][0][RTW89_FCC][66] = -4, + [1][0][RTW89_ETSI][66] = 127, + [1][0][RTW89_FCC][68] = -4, + [1][0][RTW89_ETSI][68] = 127, + [1][0][RTW89_FCC][70] = -4, + [1][0][RTW89_ETSI][70] = 127, + [1][0][RTW89_FCC][72] = -4, + [1][0][RTW89_ETSI][72] = 127, + [1][0][RTW89_FCC][74] = -4, + [1][0][RTW89_ETSI][74] = 127, + [1][0][RTW89_FCC][75] = -4, + [1][0][RTW89_ETSI][75] = 127, + [1][0][RTW89_FCC][77] = -4, + [1][0][RTW89_ETSI][77] = 127, + [1][0][RTW89_FCC][79] = -4, + [1][0][RTW89_ETSI][79] = 127, + [1][0][RTW89_FCC][81] = -4, + [1][0][RTW89_ETSI][81] = 127, + [1][0][RTW89_FCC][83] = -4, + [1][0][RTW89_ETSI][83] = 127, + [1][0][RTW89_FCC][85] = -4, + [1][0][RTW89_ETSI][85] = 127, + [1][0][RTW89_FCC][87] = -4, + [1][0][RTW89_ETSI][87] = 127, + [1][0][RTW89_FCC][89] = -4, + [1][0][RTW89_ETSI][89] = 127, + [1][0][RTW89_FCC][90] = -4, + [1][0][RTW89_ETSI][90] = 127, + [1][0][RTW89_FCC][92] = -4, + [1][0][RTW89_ETSI][92] = 127, + [1][0][RTW89_FCC][94] = -4, + [1][0][RTW89_ETSI][94] = 127, + [1][0][RTW89_FCC][96] = -4, + [1][0][RTW89_ETSI][96] = 127, + [1][0][RTW89_FCC][98] = -4, + [1][0][RTW89_ETSI][98] = 127, + [1][0][RTW89_FCC][100] = -4, + [1][0][RTW89_ETSI][100] = 127, + [1][0][RTW89_FCC][102] = -4, + [1][0][RTW89_ETSI][102] = 127, + [1][0][RTW89_FCC][104] = -4, + [1][0][RTW89_ETSI][104] = 127, + [1][0][RTW89_FCC][105] = -4, + [1][0][RTW89_ETSI][105] = 127, + [1][0][RTW89_FCC][107] = 0, + [1][0][RTW89_ETSI][107] = 127, + [1][0][RTW89_FCC][109] = 2, + [1][0][RTW89_ETSI][109] = 127, [1][0][RTW89_FCC][111] = 127, + [1][0][RTW89_ETSI][111] = 127, [1][0][RTW89_FCC][113] = 127, + [1][0][RTW89_ETSI][113] = 127, [1][0][RTW89_FCC][115] = 127, + [1][0][RTW89_ETSI][115] = 127, [1][0][RTW89_FCC][117] = 127, + [1][0][RTW89_ETSI][117] = 127, [1][0][RTW89_FCC][119] = 127, - [1][1][RTW89_FCC][0] = 76, - [1][1][RTW89_FCC][2] = 76, - [1][1][RTW89_FCC][4] = 76, - [1][1][RTW89_FCC][6] = 76, - [1][1][RTW89_FCC][8] = 76, - [1][1][RTW89_FCC][10] = 76, - [1][1][RTW89_FCC][12] = 76, - [1][1][RTW89_FCC][14] = 76, - [1][1][RTW89_FCC][15] = 76, - [1][1][RTW89_FCC][17] = 76, - [1][1][RTW89_FCC][19] = 76, - [1][1][RTW89_FCC][21] = 76, - [1][1][RTW89_FCC][23] = 76, - [1][1][RTW89_FCC][25] = 76, - [1][1][RTW89_FCC][27] = 76, - [1][1][RTW89_FCC][29] = 76, - [1][1][RTW89_FCC][30] = 76, - [1][1][RTW89_FCC][32] = 76, - [1][1][RTW89_FCC][34] = 76, - [1][1][RTW89_FCC][36] = 76, - [1][1][RTW89_FCC][38] = 76, - [1][1][RTW89_FCC][40] = 76, - [1][1][RTW89_FCC][42] = 76, - [1][1][RTW89_FCC][44] = 76, - [1][1][RTW89_FCC][45] = 76, - [1][1][RTW89_FCC][47] = 76, - [1][1][RTW89_FCC][49] = 76, - [1][1][RTW89_FCC][51] = 76, - [1][1][RTW89_FCC][53] = 76, - [1][1][RTW89_FCC][55] = 76, - [1][1][RTW89_FCC][57] = 76, - [1][1][RTW89_FCC][59] = 76, - [1][1][RTW89_FCC][60] = 76, - [1][1][RTW89_FCC][62] = 76, - [1][1][RTW89_FCC][64] = 76, - [1][1][RTW89_FCC][66] = 76, - [1][1][RTW89_FCC][68] = 76, - [1][1][RTW89_FCC][70] = 76, - [1][1][RTW89_FCC][72] = 76, - [1][1][RTW89_FCC][74] = 76, - [1][1][RTW89_FCC][75] = 76, - [1][1][RTW89_FCC][77] = 76, - [1][1][RTW89_FCC][79] = 76, - [1][1][RTW89_FCC][81] = 76, - [1][1][RTW89_FCC][83] = 76, - [1][1][RTW89_FCC][85] = 76, - [1][1][RTW89_FCC][87] = 76, - [1][1][RTW89_FCC][89] = 76, - [1][1][RTW89_FCC][90] = 76, - [1][1][RTW89_FCC][92] = 76, - [1][1][RTW89_FCC][94] = 76, - [1][1][RTW89_FCC][96] = 76, - [1][1][RTW89_FCC][98] = 76, - [1][1][RTW89_FCC][100] = 76, - [1][1][RTW89_FCC][102] = 76, - [1][1][RTW89_FCC][104] = 76, - [1][1][RTW89_FCC][105] = 76, - [1][1][RTW89_FCC][107] = 76, - [1][1][RTW89_FCC][109] = 76, + [1][0][RTW89_ETSI][119] = 127, + [1][1][RTW89_FCC][0] = -26, + [1][1][RTW89_ETSI][0] = 32, + [1][1][RTW89_FCC][2] = -28, + [1][1][RTW89_ETSI][2] = 32, + [1][1][RTW89_FCC][4] = -28, + [1][1][RTW89_ETSI][4] = 32, + [1][1][RTW89_FCC][6] = -28, + [1][1][RTW89_ETSI][6] = 32, + [1][1][RTW89_FCC][8] = -28, + [1][1][RTW89_ETSI][8] = 32, + [1][1][RTW89_FCC][10] = -28, + [1][1][RTW89_ETSI][10] = 32, + [1][1][RTW89_FCC][12] = -28, + [1][1][RTW89_ETSI][12] = 32, + [1][1][RTW89_FCC][14] = -28, + [1][1][RTW89_ETSI][14] = 32, + [1][1][RTW89_FCC][15] = -28, + [1][1][RTW89_ETSI][15] = 32, + [1][1][RTW89_FCC][17] = -28, + [1][1][RTW89_ETSI][17] = 32, + [1][1][RTW89_FCC][19] = -28, + [1][1][RTW89_ETSI][19] = 32, + [1][1][RTW89_FCC][21] = -28, + [1][1][RTW89_ETSI][21] = 32, + [1][1][RTW89_FCC][23] = -28, + [1][1][RTW89_ETSI][23] = 32, + [1][1][RTW89_FCC][25] = -28, + [1][1][RTW89_ETSI][25] = 32, + [1][1][RTW89_FCC][27] = -28, + [1][1][RTW89_ETSI][27] = 32, + [1][1][RTW89_FCC][29] = -28, + [1][1][RTW89_ETSI][29] = 32, + [1][1][RTW89_FCC][30] = -28, + [1][1][RTW89_ETSI][30] = 32, + [1][1][RTW89_FCC][32] = -28, + [1][1][RTW89_ETSI][32] = 32, + [1][1][RTW89_FCC][34] = -28, + [1][1][RTW89_ETSI][34] = 32, + [1][1][RTW89_FCC][36] = -28, + [1][1][RTW89_ETSI][36] = 32, + [1][1][RTW89_FCC][38] = -28, + [1][1][RTW89_ETSI][38] = 32, + [1][1][RTW89_FCC][40] = -28, + [1][1][RTW89_ETSI][40] = 32, + [1][1][RTW89_FCC][42] = -28, + [1][1][RTW89_ETSI][42] = 32, + [1][1][RTW89_FCC][44] = -28, + [1][1][RTW89_ETSI][44] = 34, + [1][1][RTW89_FCC][45] = -26, + [1][1][RTW89_ETSI][45] = 127, + [1][1][RTW89_FCC][47] = -28, + [1][1][RTW89_ETSI][47] = 127, + [1][1][RTW89_FCC][49] = -28, + [1][1][RTW89_ETSI][49] = 127, + [1][1][RTW89_FCC][51] = -28, + [1][1][RTW89_ETSI][51] = 127, + [1][1][RTW89_FCC][53] = -26, + [1][1][RTW89_ETSI][53] = 127, + [1][1][RTW89_FCC][55] = -28, + [1][1][RTW89_ETSI][55] = 127, + [1][1][RTW89_FCC][57] = -28, + [1][1][RTW89_ETSI][57] = 127, + [1][1][RTW89_FCC][59] = -28, + [1][1][RTW89_ETSI][59] = 127, + [1][1][RTW89_FCC][60] = -28, + [1][1][RTW89_ETSI][60] = 127, + [1][1][RTW89_FCC][62] = -28, + [1][1][RTW89_ETSI][62] = 127, + [1][1][RTW89_FCC][64] = -28, + [1][1][RTW89_ETSI][64] = 127, + [1][1][RTW89_FCC][66] = -28, + [1][1][RTW89_ETSI][66] = 127, + [1][1][RTW89_FCC][68] = -28, + [1][1][RTW89_ETSI][68] = 127, + [1][1][RTW89_FCC][70] = -26, + [1][1][RTW89_ETSI][70] = 127, + [1][1][RTW89_FCC][72] = -28, + [1][1][RTW89_ETSI][72] = 127, + [1][1][RTW89_FCC][74] = -28, + [1][1][RTW89_ETSI][74] = 127, + [1][1][RTW89_FCC][75] = -28, + [1][1][RTW89_ETSI][75] = 127, + [1][1][RTW89_FCC][77] = -28, + [1][1][RTW89_ETSI][77] = 127, + [1][1][RTW89_FCC][79] = -28, + [1][1][RTW89_ETSI][79] = 127, + [1][1][RTW89_FCC][81] = -28, + [1][1][RTW89_ETSI][81] = 127, + [1][1][RTW89_FCC][83] = -28, + [1][1][RTW89_ETSI][83] = 127, + [1][1][RTW89_FCC][85] = -28, + [1][1][RTW89_ETSI][85] = 127, + [1][1][RTW89_FCC][87] = -28, + [1][1][RTW89_ETSI][87] = 127, + [1][1][RTW89_FCC][89] = -26, + [1][1][RTW89_ETSI][89] = 127, + [1][1][RTW89_FCC][90] = -26, + [1][1][RTW89_ETSI][90] = 127, + [1][1][RTW89_FCC][92] = -26, + [1][1][RTW89_ETSI][92] = 127, + [1][1][RTW89_FCC][94] = -26, + [1][1][RTW89_ETSI][94] = 127, + [1][1][RTW89_FCC][96] = -26, + [1][1][RTW89_ETSI][96] = 127, + [1][1][RTW89_FCC][98] = -26, + [1][1][RTW89_ETSI][98] = 127, + [1][1][RTW89_FCC][100] = -26, + [1][1][RTW89_ETSI][100] = 127, + [1][1][RTW89_FCC][102] = -26, + [1][1][RTW89_ETSI][102] = 127, + [1][1][RTW89_FCC][104] = -26, + [1][1][RTW89_ETSI][104] = 127, + [1][1][RTW89_FCC][105] = -26, + [1][1][RTW89_ETSI][105] = 127, + [1][1][RTW89_FCC][107] = -22, + [1][1][RTW89_ETSI][107] = 127, + [1][1][RTW89_FCC][109] = -22, + [1][1][RTW89_ETSI][109] = 127, [1][1][RTW89_FCC][111] = 127, + [1][1][RTW89_ETSI][111] = 127, [1][1][RTW89_FCC][113] = 127, + [1][1][RTW89_ETSI][113] = 127, [1][1][RTW89_FCC][115] = 127, + [1][1][RTW89_ETSI][115] = 127, [1][1][RTW89_FCC][117] = 127, + [1][1][RTW89_ETSI][117] = 127, [1][1][RTW89_FCC][119] = 127, - [2][0][RTW89_FCC][0] = 76, - [2][0][RTW89_FCC][2] = 76, - [2][0][RTW89_FCC][4] = 76, - [2][0][RTW89_FCC][6] = 76, - [2][0][RTW89_FCC][8] = 76, - [2][0][RTW89_FCC][10] = 76, - [2][0][RTW89_FCC][12] = 76, - [2][0][RTW89_FCC][14] = 76, - [2][0][RTW89_FCC][15] = 76, - [2][0][RTW89_FCC][17] = 76, - [2][0][RTW89_FCC][19] = 76, - [2][0][RTW89_FCC][21] = 76, - [2][0][RTW89_FCC][23] = 76, - [2][0][RTW89_FCC][25] = 76, - [2][0][RTW89_FCC][27] = 76, - [2][0][RTW89_FCC][29] = 76, - [2][0][RTW89_FCC][30] = 76, - [2][0][RTW89_FCC][32] = 76, - [2][0][RTW89_FCC][34] = 76, - [2][0][RTW89_FCC][36] = 76, - [2][0][RTW89_FCC][38] = 76, - [2][0][RTW89_FCC][40] = 76, - [2][0][RTW89_FCC][42] = 76, - [2][0][RTW89_FCC][44] = 76, - [2][0][RTW89_FCC][45] = 76, - [2][0][RTW89_FCC][47] = 76, - [2][0][RTW89_FCC][49] = 76, - [2][0][RTW89_FCC][51] = 76, - [2][0][RTW89_FCC][53] = 76, - [2][0][RTW89_FCC][55] = 76, - [2][0][RTW89_FCC][57] = 76, - [2][0][RTW89_FCC][59] = 76, - [2][0][RTW89_FCC][60] = 76, - [2][0][RTW89_FCC][62] = 76, - [2][0][RTW89_FCC][64] = 76, - [2][0][RTW89_FCC][66] = 76, - [2][0][RTW89_FCC][68] = 76, - [2][0][RTW89_FCC][70] = 76, - [2][0][RTW89_FCC][72] = 76, - [2][0][RTW89_FCC][74] = 76, - [2][0][RTW89_FCC][75] = 76, - [2][0][RTW89_FCC][77] = 76, - [2][0][RTW89_FCC][79] = 76, - [2][0][RTW89_FCC][81] = 76, - [2][0][RTW89_FCC][83] = 76, - [2][0][RTW89_FCC][85] = 76, - [2][0][RTW89_FCC][87] = 76, - [2][0][RTW89_FCC][89] = 76, - [2][0][RTW89_FCC][90] = 76, - [2][0][RTW89_FCC][92] = 76, - [2][0][RTW89_FCC][94] = 76, - [2][0][RTW89_FCC][96] = 76, - [2][0][RTW89_FCC][98] = 76, - [2][0][RTW89_FCC][100] = 76, - [2][0][RTW89_FCC][102] = 76, - [2][0][RTW89_FCC][104] = 76, - [2][0][RTW89_FCC][105] = 76, - [2][0][RTW89_FCC][107] = 76, - [2][0][RTW89_FCC][109] = 76, + [1][1][RTW89_ETSI][119] = 127, + [2][0][RTW89_FCC][0] = 8, + [2][0][RTW89_ETSI][0] = 56, + [2][0][RTW89_FCC][2] = 8, + [2][0][RTW89_ETSI][2] = 56, + [2][0][RTW89_FCC][4] = 8, + [2][0][RTW89_ETSI][4] = 56, + [2][0][RTW89_FCC][6] = 8, + [2][0][RTW89_ETSI][6] = 56, + [2][0][RTW89_FCC][8] = 8, + [2][0][RTW89_ETSI][8] = 56, + [2][0][RTW89_FCC][10] = 8, + [2][0][RTW89_ETSI][10] = 56, + [2][0][RTW89_FCC][12] = 8, + [2][0][RTW89_ETSI][12] = 56, + [2][0][RTW89_FCC][14] = 8, + [2][0][RTW89_ETSI][14] = 56, + [2][0][RTW89_FCC][15] = 8, + [2][0][RTW89_ETSI][15] = 56, + [2][0][RTW89_FCC][17] = 8, + [2][0][RTW89_ETSI][17] = 56, + [2][0][RTW89_FCC][19] = 8, + [2][0][RTW89_ETSI][19] = 56, + [2][0][RTW89_FCC][21] = 8, + [2][0][RTW89_ETSI][21] = 56, + [2][0][RTW89_FCC][23] = 8, + [2][0][RTW89_ETSI][23] = 56, + [2][0][RTW89_FCC][25] = 8, + [2][0][RTW89_ETSI][25] = 56, + [2][0][RTW89_FCC][27] = 8, + [2][0][RTW89_ETSI][27] = 56, + [2][0][RTW89_FCC][29] = 8, + [2][0][RTW89_ETSI][29] = 56, + [2][0][RTW89_FCC][30] = 8, + [2][0][RTW89_ETSI][30] = 56, + [2][0][RTW89_FCC][32] = 8, + [2][0][RTW89_ETSI][32] = 56, + [2][0][RTW89_FCC][34] = 8, + [2][0][RTW89_ETSI][34] = 56, + [2][0][RTW89_FCC][36] = 8, + [2][0][RTW89_ETSI][36] = 56, + [2][0][RTW89_FCC][38] = 8, + [2][0][RTW89_ETSI][38] = 56, + [2][0][RTW89_FCC][40] = 8, + [2][0][RTW89_ETSI][40] = 56, + [2][0][RTW89_FCC][42] = 8, + [2][0][RTW89_ETSI][42] = 56, + [2][0][RTW89_FCC][44] = 8, + [2][0][RTW89_ETSI][44] = 56, + [2][0][RTW89_FCC][45] = 8, + [2][0][RTW89_ETSI][45] = 127, + [2][0][RTW89_FCC][47] = 8, + [2][0][RTW89_ETSI][47] = 127, + [2][0][RTW89_FCC][49] = 8, + [2][0][RTW89_ETSI][49] = 127, + [2][0][RTW89_FCC][51] = 8, + [2][0][RTW89_ETSI][51] = 127, + [2][0][RTW89_FCC][53] = 8, + [2][0][RTW89_ETSI][53] = 127, + [2][0][RTW89_FCC][55] = 8, + [2][0][RTW89_ETSI][55] = 127, + [2][0][RTW89_FCC][57] = 8, + [2][0][RTW89_ETSI][57] = 127, + [2][0][RTW89_FCC][59] = 8, + [2][0][RTW89_ETSI][59] = 127, + [2][0][RTW89_FCC][60] = 8, + [2][0][RTW89_ETSI][60] = 127, + [2][0][RTW89_FCC][62] = 8, + [2][0][RTW89_ETSI][62] = 127, + [2][0][RTW89_FCC][64] = 8, + [2][0][RTW89_ETSI][64] = 127, + [2][0][RTW89_FCC][66] = 8, + [2][0][RTW89_ETSI][66] = 127, + [2][0][RTW89_FCC][68] = 8, + [2][0][RTW89_ETSI][68] = 127, + [2][0][RTW89_FCC][70] = 8, + [2][0][RTW89_ETSI][70] = 127, + [2][0][RTW89_FCC][72] = 8, + [2][0][RTW89_ETSI][72] = 127, + [2][0][RTW89_FCC][74] = 8, + [2][0][RTW89_ETSI][74] = 127, + [2][0][RTW89_FCC][75] = 8, + [2][0][RTW89_ETSI][75] = 127, + [2][0][RTW89_FCC][77] = 8, + [2][0][RTW89_ETSI][77] = 127, + [2][0][RTW89_FCC][79] = 8, + [2][0][RTW89_ETSI][79] = 127, + [2][0][RTW89_FCC][81] = 8, + [2][0][RTW89_ETSI][81] = 127, + [2][0][RTW89_FCC][83] = 8, + [2][0][RTW89_ETSI][83] = 127, + [2][0][RTW89_FCC][85] = 8, + [2][0][RTW89_ETSI][85] = 127, + [2][0][RTW89_FCC][87] = 8, + [2][0][RTW89_ETSI][87] = 127, + [2][0][RTW89_FCC][89] = 8, + [2][0][RTW89_ETSI][89] = 127, + [2][0][RTW89_FCC][90] = 8, + [2][0][RTW89_ETSI][90] = 127, + [2][0][RTW89_FCC][92] = 8, + [2][0][RTW89_ETSI][92] = 127, + [2][0][RTW89_FCC][94] = 8, + [2][0][RTW89_ETSI][94] = 127, + [2][0][RTW89_FCC][96] = 8, + [2][0][RTW89_ETSI][96] = 127, + [2][0][RTW89_FCC][98] = 8, + [2][0][RTW89_ETSI][98] = 127, + [2][0][RTW89_FCC][100] = 8, + [2][0][RTW89_ETSI][100] = 127, + [2][0][RTW89_FCC][102] = 8, + [2][0][RTW89_ETSI][102] = 127, + [2][0][RTW89_FCC][104] = 8, + [2][0][RTW89_ETSI][104] = 127, + [2][0][RTW89_FCC][105] = 8, + [2][0][RTW89_ETSI][105] = 127, + [2][0][RTW89_FCC][107] = 10, + [2][0][RTW89_ETSI][107] = 127, + [2][0][RTW89_FCC][109] = 12, + [2][0][RTW89_ETSI][109] = 127, [2][0][RTW89_FCC][111] = 127, + [2][0][RTW89_ETSI][111] = 127, [2][0][RTW89_FCC][113] = 127, + [2][0][RTW89_ETSI][113] = 127, [2][0][RTW89_FCC][115] = 127, + [2][0][RTW89_ETSI][115] = 127, [2][0][RTW89_FCC][117] = 127, + [2][0][RTW89_ETSI][117] = 127, [2][0][RTW89_FCC][119] = 127, - [2][1][RTW89_FCC][0] = 76, - [2][1][RTW89_FCC][2] = 76, - [2][1][RTW89_FCC][4] = 76, - [2][1][RTW89_FCC][6] = 76, - [2][1][RTW89_FCC][8] = 76, - [2][1][RTW89_FCC][10] = 76, - [2][1][RTW89_FCC][12] = 76, - [2][1][RTW89_FCC][14] = 76, - [2][1][RTW89_FCC][15] = 76, - [2][1][RTW89_FCC][17] = 76, - [2][1][RTW89_FCC][19] = 76, - [2][1][RTW89_FCC][21] = 76, - [2][1][RTW89_FCC][23] = 76, - [2][1][RTW89_FCC][25] = 76, - [2][1][RTW89_FCC][27] = 76, - [2][1][RTW89_FCC][29] = 76, - [2][1][RTW89_FCC][30] = 76, - [2][1][RTW89_FCC][32] = 76, - [2][1][RTW89_FCC][34] = 76, - [2][1][RTW89_FCC][36] = 76, - [2][1][RTW89_FCC][38] = 76, - [2][1][RTW89_FCC][40] = 76, - [2][1][RTW89_FCC][42] = 76, - [2][1][RTW89_FCC][44] = 76, - [2][1][RTW89_FCC][45] = 76, - [2][1][RTW89_FCC][47] = 76, - [2][1][RTW89_FCC][49] = 76, - [2][1][RTW89_FCC][51] = 76, - [2][1][RTW89_FCC][53] = 76, - [2][1][RTW89_FCC][55] = 76, - [2][1][RTW89_FCC][57] = 76, - [2][1][RTW89_FCC][59] = 76, - [2][1][RTW89_FCC][60] = 76, - [2][1][RTW89_FCC][62] = 76, - [2][1][RTW89_FCC][64] = 76, - [2][1][RTW89_FCC][66] = 76, - [2][1][RTW89_FCC][68] = 76, - [2][1][RTW89_FCC][70] = 76, - [2][1][RTW89_FCC][72] = 76, - [2][1][RTW89_FCC][74] = 76, - [2][1][RTW89_FCC][75] = 76, - [2][1][RTW89_FCC][77] = 76, - [2][1][RTW89_FCC][79] = 76, - [2][1][RTW89_FCC][81] = 76, - [2][1][RTW89_FCC][83] = 76, - [2][1][RTW89_FCC][85] = 76, - [2][1][RTW89_FCC][87] = 76, - [2][1][RTW89_FCC][89] = 76, - [2][1][RTW89_FCC][90] = 76, - [2][1][RTW89_FCC][92] = 76, - [2][1][RTW89_FCC][94] = 76, - [2][1][RTW89_FCC][96] = 76, - [2][1][RTW89_FCC][98] = 76, - [2][1][RTW89_FCC][100] = 76, - [2][1][RTW89_FCC][102] = 76, - [2][1][RTW89_FCC][104] = 76, - [2][1][RTW89_FCC][105] = 76, - [2][1][RTW89_FCC][107] = 76, - [2][1][RTW89_FCC][109] = 76, + [2][0][RTW89_ETSI][119] = 127, + [2][1][RTW89_FCC][0] = -16, + [2][1][RTW89_ETSI][0] = 44, + [2][1][RTW89_FCC][2] = -16, + [2][1][RTW89_ETSI][2] = 44, + [2][1][RTW89_FCC][4] = -16, + [2][1][RTW89_ETSI][4] = 44, + [2][1][RTW89_FCC][6] = -16, + [2][1][RTW89_ETSI][6] = 44, + [2][1][RTW89_FCC][8] = -16, + [2][1][RTW89_ETSI][8] = 44, + [2][1][RTW89_FCC][10] = -16, + [2][1][RTW89_ETSI][10] = 44, + [2][1][RTW89_FCC][12] = -16, + [2][1][RTW89_ETSI][12] = 44, + [2][1][RTW89_FCC][14] = -16, + [2][1][RTW89_ETSI][14] = 44, + [2][1][RTW89_FCC][15] = -16, + [2][1][RTW89_ETSI][15] = 44, + [2][1][RTW89_FCC][17] = -16, + [2][1][RTW89_ETSI][17] = 44, + [2][1][RTW89_FCC][19] = -16, + [2][1][RTW89_ETSI][19] = 44, + [2][1][RTW89_FCC][21] = -16, + [2][1][RTW89_ETSI][21] = 44, + [2][1][RTW89_FCC][23] = -16, + [2][1][RTW89_ETSI][23] = 44, + [2][1][RTW89_FCC][25] = -16, + [2][1][RTW89_ETSI][25] = 44, + [2][1][RTW89_FCC][27] = -16, + [2][1][RTW89_ETSI][27] = 44, + [2][1][RTW89_FCC][29] = -16, + [2][1][RTW89_ETSI][29] = 44, + [2][1][RTW89_FCC][30] = -16, + [2][1][RTW89_ETSI][30] = 44, + [2][1][RTW89_FCC][32] = -16, + [2][1][RTW89_ETSI][32] = 44, + [2][1][RTW89_FCC][34] = -16, + [2][1][RTW89_ETSI][34] = 44, + [2][1][RTW89_FCC][36] = -16, + [2][1][RTW89_ETSI][36] = 44, + [2][1][RTW89_FCC][38] = -16, + [2][1][RTW89_ETSI][38] = 44, + [2][1][RTW89_FCC][40] = -16, + [2][1][RTW89_ETSI][40] = 44, + [2][1][RTW89_FCC][42] = -16, + [2][1][RTW89_ETSI][42] = 44, + [2][1][RTW89_FCC][44] = -16, + [2][1][RTW89_ETSI][44] = 44, + [2][1][RTW89_FCC][45] = -16, + [2][1][RTW89_ETSI][45] = 127, + [2][1][RTW89_FCC][47] = -16, + [2][1][RTW89_ETSI][47] = 127, + [2][1][RTW89_FCC][49] = -16, + [2][1][RTW89_ETSI][49] = 127, + [2][1][RTW89_FCC][51] = -16, + [2][1][RTW89_ETSI][51] = 127, + [2][1][RTW89_FCC][53] = -16, + [2][1][RTW89_ETSI][53] = 127, + [2][1][RTW89_FCC][55] = -16, + [2][1][RTW89_ETSI][55] = 127, + [2][1][RTW89_FCC][57] = -16, + [2][1][RTW89_ETSI][57] = 127, + [2][1][RTW89_FCC][59] = -16, + [2][1][RTW89_ETSI][59] = 127, + [2][1][RTW89_FCC][60] = -16, + [2][1][RTW89_ETSI][60] = 127, + [2][1][RTW89_FCC][62] = -16, + [2][1][RTW89_ETSI][62] = 127, + [2][1][RTW89_FCC][64] = -16, + [2][1][RTW89_ETSI][64] = 127, + [2][1][RTW89_FCC][66] = -16, + [2][1][RTW89_ETSI][66] = 127, + [2][1][RTW89_FCC][68] = -16, + [2][1][RTW89_ETSI][68] = 127, + [2][1][RTW89_FCC][70] = -16, + [2][1][RTW89_ETSI][70] = 127, + [2][1][RTW89_FCC][72] = -16, + [2][1][RTW89_ETSI][72] = 127, + [2][1][RTW89_FCC][74] = -16, + [2][1][RTW89_ETSI][74] = 127, + [2][1][RTW89_FCC][75] = -16, + [2][1][RTW89_ETSI][75] = 127, + [2][1][RTW89_FCC][77] = -16, + [2][1][RTW89_ETSI][77] = 127, + [2][1][RTW89_FCC][79] = -16, + [2][1][RTW89_ETSI][79] = 127, + [2][1][RTW89_FCC][81] = -16, + [2][1][RTW89_ETSI][81] = 127, + [2][1][RTW89_FCC][83] = -16, + [2][1][RTW89_ETSI][83] = 127, + [2][1][RTW89_FCC][85] = -18, + [2][1][RTW89_ETSI][85] = 127, + [2][1][RTW89_FCC][87] = -16, + [2][1][RTW89_ETSI][87] = 127, + [2][1][RTW89_FCC][89] = -16, + [2][1][RTW89_ETSI][89] = 127, + [2][1][RTW89_FCC][90] = -16, + [2][1][RTW89_ETSI][90] = 127, + [2][1][RTW89_FCC][92] = -16, + [2][1][RTW89_ETSI][92] = 127, + [2][1][RTW89_FCC][94] = -16, + [2][1][RTW89_ETSI][94] = 127, + [2][1][RTW89_FCC][96] = -16, + [2][1][RTW89_ETSI][96] = 127, + [2][1][RTW89_FCC][98] = -16, + [2][1][RTW89_ETSI][98] = 127, + [2][1][RTW89_FCC][100] = -16, + [2][1][RTW89_ETSI][100] = 127, + [2][1][RTW89_FCC][102] = -16, + [2][1][RTW89_ETSI][102] = 127, + [2][1][RTW89_FCC][104] = -16, + [2][1][RTW89_ETSI][104] = 127, + [2][1][RTW89_FCC][105] = -16, + [2][1][RTW89_ETSI][105] = 127, + [2][1][RTW89_FCC][107] = -12, + [2][1][RTW89_ETSI][107] = 127, + [2][1][RTW89_FCC][109] = -10, + [2][1][RTW89_ETSI][109] = 127, [2][1][RTW89_FCC][111] = 127, + [2][1][RTW89_ETSI][111] = 127, [2][1][RTW89_FCC][113] = 127, + [2][1][RTW89_ETSI][113] = 127, [2][1][RTW89_FCC][115] = 127, + [2][1][RTW89_ETSI][115] = 127, [2][1][RTW89_FCC][117] = 127, + [2][1][RTW89_ETSI][117] = 127, [2][1][RTW89_FCC][119] = 127, + [2][1][RTW89_ETSI][119] = 127, }; const struct rtw89_phy_table rtw89_8852c_phy_bb_table = { diff --git a/drivers/net/wireless/realtek/rtw89/rtw8852ce.c b/drivers/net/wireless/realtek/rtw89/rtw8852ce.c index fc0394494013..35901f64d17d 100644 --- a/drivers/net/wireless/realtek/rtw89/rtw8852ce.c +++ b/drivers/net/wireless/realtek/rtw89/rtw8852ce.c @@ -42,14 +42,15 @@ static const struct rtw89_pci_info rtw8852c_pci_info = { .max_tag_num_mask = B_AX_MAX_TAG_NUM_V1_MASK, .rxbd_rwptr_clr_reg = R_AX_RXBD_RWPTR_CLR_V1, .txbd_rwptr_clr2_reg = R_AX_TXBD_RWPTR_CLR2_V1, - .dma_stop1_reg = R_AX_HAXI_DMA_STOP1, - .dma_stop2_reg = R_AX_HAXI_DMA_STOP2, - .dma_busy1_reg = R_AX_HAXI_DMA_BUSY1, + .dma_stop1 = {R_AX_HAXI_DMA_STOP1, B_AX_TX_STOP1_MASK}, + .dma_stop2 = {R_AX_HAXI_DMA_STOP2, B_AX_TX_STOP2_ALL}, + .dma_busy1 = {R_AX_HAXI_DMA_BUSY1, DMA_BUSY1_CHECK}, .dma_busy2_reg = R_AX_HAXI_DMA_BUSY2, .dma_busy3_reg = R_AX_HAXI_DMA_BUSY3, .rpwm_addr = R_AX_PCIE_HRPWM_V1, .cpwm_addr = R_AX_PCIE_CRPWM, + .tx_dma_ch_mask = 0, .bd_idx_addr_low_power = &rtw8852c_bd_idx_addr_low_power, .dma_addr_set = &rtw89_pci_ch_dma_addr_set_v1, diff --git a/drivers/net/wireless/realtek/rtw89/sar.c b/drivers/net/wireless/realtek/rtw89/sar.c index eb2d3ec28775..dfccae81c380 100644 --- a/drivers/net/wireless/realtek/rtw89/sar.c +++ b/drivers/net/wireless/realtek/rtw89/sar.c @@ -81,9 +81,9 @@ static const struct rtw89_sar_span rtw89_sar_overlapping_6ghz[] = { static int rtw89_query_sar_config_common(struct rtw89_dev *rtwdev, s32 *cfg) { struct rtw89_sar_cfg_common *rtwsar = &rtwdev->sar.cfg_common; - struct rtw89_hal *hal = &rtwdev->hal; - enum rtw89_band band = hal->current_band_type; - u32 center_freq = hal->current_freq; + const struct rtw89_chan *chan = rtw89_chan_get(rtwdev, RTW89_SUB_ENTITY_0); + enum rtw89_band band = chan->band_type; + u32 center_freq = chan->freq; const struct rtw89_sar_span *span = NULL; enum rtw89_sar_subband subband_l, subband_h; int idx; @@ -228,7 +228,7 @@ static int rtw89_apply_sar_common(struct rtw89_dev *rtwdev, } rtw89_sar_set_src(rtwdev, RTW89_SAR_SOURCE_COMMON, cfg_common, sar); - rtw89_chip_set_txpwr(rtwdev); + rtw89_core_set_chip_txpwr(rtwdev); exit: mutex_unlock(&rtwdev->mutex); diff --git a/drivers/net/wireless/realtek/rtw89/ser.c b/drivers/net/wireless/realtek/rtw89/ser.c index 726223f25dc6..c1a4bc1c64d1 100644 --- a/drivers/net/wireless/realtek/rtw89/ser.c +++ b/drivers/net/wireless/realtek/rtw89/ser.c @@ -5,6 +5,7 @@ #include <linux/devcoredump.h> #include "cam.h" +#include "chan.h" #include "debug.h" #include "fw.h" #include "mac.h" @@ -152,7 +153,10 @@ static void ser_state_run(struct rtw89_ser *ser, u8 evt) rtw89_debug(rtwdev, RTW89_DBG_SER, "ser: %s receive %s\n", ser_st_name(ser), ser_ev_name(ser, evt)); + mutex_lock(&rtwdev->mutex); rtw89_leave_lps(rtwdev); + mutex_unlock(&rtwdev->mutex); + ser->st_tbl[ser->state].st_func(ser, evt); } @@ -298,7 +302,7 @@ static void ser_reset_vif(struct rtw89_dev *rtwdev, struct rtw89_vif *rtwvif) rtwvif->trigger = false; } -static void ser_sta_deinit_addr_cam_iter(void *data, struct ieee80211_sta *sta) +static void ser_sta_deinit_cam_iter(void *data, struct ieee80211_sta *sta) { struct rtw89_vif *rtwvif = (struct rtw89_vif *)data; struct rtw89_dev *rtwdev = rtwvif->rtwdev; @@ -308,15 +312,19 @@ static void ser_sta_deinit_addr_cam_iter(void *data, struct ieee80211_sta *sta) rtw89_cam_deinit_addr_cam(rtwdev, &rtwsta->addr_cam); if (sta->tdls) rtw89_cam_deinit_bssid_cam(rtwdev, &rtwsta->bssid_cam); + + INIT_LIST_HEAD(&rtwsta->ba_cam_list); } static void ser_deinit_cam(struct rtw89_dev *rtwdev, struct rtw89_vif *rtwvif) { ieee80211_iterate_stations_atomic(rtwdev->hw, - ser_sta_deinit_addr_cam_iter, + ser_sta_deinit_cam_iter, rtwvif); rtw89_cam_deinit(rtwdev, rtwvif); + + bitmap_zero(rtwdev->cam_info.ba_cam_map, RTW89_MAX_BA_CAM_NUM); } static void ser_reset_mac_binding(struct rtw89_dev *rtwdev) @@ -388,6 +396,7 @@ static void ser_idle_st_hdl(struct rtw89_ser *ser, u8 evt) switch (evt) { case SER_EV_STATE_IN: rtw89_hci_recovery_complete(rtwdev); + clear_bit(RTW89_FLAG_CRASH_SIMULATING, rtwdev->flags); break; case SER_EV_L1_RESET: ser_state_goto(ser, SER_RESET_TRX_ST); @@ -531,7 +540,7 @@ static int rtw89_ser_fw_backtrace_dump(struct rtw89_dev *rtwdev, u8 *buf, const struct __fw_backtrace_entry *ent) { struct __fw_backtrace_info *ptr = (struct __fw_backtrace_info *)buf; - u32 fwbt_addr = ent->wcpu_addr - RTW89_WCPU_BASE_ADDR; + u32 fwbt_addr = ent->wcpu_addr & RTW89_WCPU_BASE_MASK; u32 fwbt_size = ent->size; u32 fwbt_key = ent->key; u32 i; @@ -601,6 +610,7 @@ bottom: ser_reset_mac_binding(rtwdev); rtw89_core_stop(rtwdev); + rtw89_entity_init(rtwdev); INIT_LIST_HEAD(&rtwdev->rtwvifs_list); } @@ -623,7 +633,6 @@ static void ser_l2_reset_st_hdl(struct rtw89_ser *ser, u8 evt) fallthrough; case SER_EV_L2_RECFG_DONE: ser_state_goto(ser, SER_IDLE_ST); - clear_bit(RTW89_FLAG_RESTART_TRIGGER, rtwdev->flags); break; case SER_EV_STATE_OUT: diff --git a/drivers/net/wireless/rndis_wlan.c b/drivers/net/wireless/rndis_wlan.c index 05524291d60c..82a7458e01ae 100644 --- a/drivers/net/wireless/rndis_wlan.c +++ b/drivers/net/wireless/rndis_wlan.c @@ -251,7 +251,7 @@ struct ndis_80211_bssid_ex { struct ndis_80211_bssid_list_ex { __le32 num_items; - struct ndis_80211_bssid_ex bssid[]; + u8 bssid_data[]; } __packed; struct ndis_80211_fixed_ies { @@ -489,14 +489,16 @@ static int rndis_join_ibss(struct wiphy *wiphy, struct net_device *dev, static int rndis_leave_ibss(struct wiphy *wiphy, struct net_device *dev); static int rndis_add_key(struct wiphy *wiphy, struct net_device *netdev, - u8 key_index, bool pairwise, const u8 *mac_addr, - struct key_params *params); + int link_id, u8 key_index, bool pairwise, + const u8 *mac_addr, struct key_params *params); static int rndis_del_key(struct wiphy *wiphy, struct net_device *netdev, - u8 key_index, bool pairwise, const u8 *mac_addr); + int link_id, u8 key_index, bool pairwise, + const u8 *mac_addr); static int rndis_set_default_key(struct wiphy *wiphy, struct net_device *netdev, - u8 key_index, bool unicast, bool multicast); + int link_id, u8 key_index, bool unicast, + bool multicast); static int rndis_get_station(struct wiphy *wiphy, struct net_device *dev, const u8 *mac, struct station_info *sinfo); @@ -2082,7 +2084,8 @@ resize_buf: netdev_dbg(usbdev->net, "%s(): buflen: %d\n", __func__, len); bssid_len = 0; - bssid = next_bssid_list_item(bssid_list->bssid, &bssid_len, buf, len); + bssid = next_bssid_list_item((void *)bssid_list->bssid_data, + &bssid_len, buf, len); /* Device returns incorrect 'num_items'. Workaround by ignoring the * received 'num_items' and walking through full bssid buffer instead. @@ -2377,8 +2380,8 @@ static int rndis_leave_ibss(struct wiphy *wiphy, struct net_device *dev) } static int rndis_add_key(struct wiphy *wiphy, struct net_device *netdev, - u8 key_index, bool pairwise, const u8 *mac_addr, - struct key_params *params) + int link_id, u8 key_index, bool pairwise, + const u8 *mac_addr, struct key_params *params) { struct rndis_wlan_private *priv = wiphy_priv(wiphy); struct usbnet *usbdev = priv->usbdev; @@ -2413,7 +2416,8 @@ static int rndis_add_key(struct wiphy *wiphy, struct net_device *netdev, } static int rndis_del_key(struct wiphy *wiphy, struct net_device *netdev, - u8 key_index, bool pairwise, const u8 *mac_addr) + int link_id, u8 key_index, bool pairwise, + const u8 *mac_addr) { struct rndis_wlan_private *priv = wiphy_priv(wiphy); struct usbnet *usbdev = priv->usbdev; @@ -2424,7 +2428,8 @@ static int rndis_del_key(struct wiphy *wiphy, struct net_device *netdev, } static int rndis_set_default_key(struct wiphy *wiphy, struct net_device *netdev, - u8 key_index, bool unicast, bool multicast) + int link_id, u8 key_index, bool unicast, + bool multicast) { struct rndis_wlan_private *priv = wiphy_priv(wiphy); struct usbnet *usbdev = priv->usbdev; diff --git a/drivers/net/wireless/rsi/rsi_91x_mac80211.c b/drivers/net/wireless/rsi/rsi_91x_mac80211.c index bf39c4bda26f..2fbec51c8f94 100644 --- a/drivers/net/wireless/rsi/rsi_91x_mac80211.c +++ b/drivers/net/wireless/rsi/rsi_91x_mac80211.c @@ -889,6 +889,7 @@ static void rsi_mac80211_conf_filter(struct ieee80211_hw *hw, * for a hardware TX queue. * @hw: Pointer to the ieee80211_hw structure * @vif: Pointer to the ieee80211_vif structure. + * @link_id: the link ID if MLO is used, otherwise 0 * @queue: Queue number. * @params: Pointer to ieee80211_tx_queue_params structure. * diff --git a/drivers/net/wireless/silabs/wfx/main.c b/drivers/net/wireless/silabs/wfx/main.c index e015bfb8d221..84d82ddded56 100644 --- a/drivers/net/wireless/silabs/wfx/main.c +++ b/drivers/net/wireless/silabs/wfx/main.c @@ -181,7 +181,7 @@ int wfx_send_pds(struct wfx_dev *wdev, u8 *buf, size_t len) while (len > 0) { chunk_type = get_unaligned_le16(buf + 0); chunk_len = get_unaligned_le16(buf + 2); - if (chunk_len > len) { + if (chunk_len < 4 || chunk_len > len) { dev_err(wdev->dev, "PDS:%d: corrupted file\n", chunk_num); return -EINVAL; } diff --git a/drivers/net/wireless/st/cw1200/queue.c b/drivers/net/wireless/st/cw1200/queue.c index e06da4b3b0d4..805a3c1bf8fe 100644 --- a/drivers/net/wireless/st/cw1200/queue.c +++ b/drivers/net/wireless/st/cw1200/queue.c @@ -91,23 +91,25 @@ static void __cw1200_queue_gc(struct cw1200_queue *queue, bool unlock) { struct cw1200_queue_stats *stats = queue->stats; - struct cw1200_queue_item *item = NULL, *tmp; + struct cw1200_queue_item *item = NULL, *iter, *tmp; bool wakeup_stats = false; - list_for_each_entry_safe(item, tmp, &queue->queue, head) { - if (time_is_after_jiffies(item->queue_timestamp + queue->ttl)) + list_for_each_entry_safe(iter, tmp, &queue->queue, head) { + if (time_is_after_jiffies(iter->queue_timestamp + queue->ttl)) { + item = iter; break; + } --queue->num_queued; - --queue->link_map_cache[item->txpriv.link_id]; + --queue->link_map_cache[iter->txpriv.link_id]; spin_lock_bh(&stats->lock); --stats->num_queued; - if (!--stats->link_map_cache[item->txpriv.link_id]) + if (!--stats->link_map_cache[iter->txpriv.link_id]) wakeup_stats = true; spin_unlock_bh(&stats->lock); cw1200_debug_tx_ttl(stats->priv); - cw1200_queue_register_post_gc(head, item); - item->skb = NULL; - list_move_tail(&item->head, &queue->free_pool); + cw1200_queue_register_post_gc(head, iter); + iter->skb = NULL; + list_move_tail(&iter->head, &queue->free_pool); } if (wakeup_stats) diff --git a/drivers/net/wireless/st/cw1200/sta.c b/drivers/net/wireless/st/cw1200/sta.c index 26d3614519b1..8ef1d06b9bbd 100644 --- a/drivers/net/wireless/st/cw1200/sta.c +++ b/drivers/net/wireless/st/cw1200/sta.c @@ -195,7 +195,7 @@ void __cw1200_cqm_bssloss_sm(struct cw1200_common *priv, priv->bss_loss_state++; - skb = ieee80211_nullfunc_get(priv->hw, priv->vif, false); + skb = ieee80211_nullfunc_get(priv->hw, priv->vif, -1, false); WARN_ON(!skb); if (skb) cw1200_tx(priv->hw, NULL, skb); @@ -2263,7 +2263,7 @@ static int cw1200_upload_null(struct cw1200_common *priv) .rate = 0xFF, }; - frame.skb = ieee80211_nullfunc_get(priv->hw, priv->vif, false); + frame.skb = ieee80211_nullfunc_get(priv->hw, priv->vif,-1, false); if (!frame.skb) return -ENOMEM; diff --git a/drivers/net/wireless/st/cw1200/txrx.c b/drivers/net/wireless/st/cw1200/txrx.c index fde21fca6c5e..6894b919ff94 100644 --- a/drivers/net/wireless/st/cw1200/txrx.c +++ b/drivers/net/wireless/st/cw1200/txrx.c @@ -762,8 +762,7 @@ void cw1200_tx(struct ieee80211_hw *dev, if (ret) goto drop; - rcu_read_lock(); - sta = rcu_dereference(t.sta); + sta = t.sta; spin_lock_bh(&priv->ps_state_lock); { @@ -776,8 +775,6 @@ void cw1200_tx(struct ieee80211_hw *dev, if (tid_update && sta) ieee80211_sta_set_buffered(sta, t.txpriv.tid, true); - rcu_read_unlock(); - cw1200_bh_wakeup(priv); return; @@ -1145,8 +1142,7 @@ void cw1200_rx_cb(struct cw1200_common *priv, /* Remove TSF from the end of frame */ if (arg->flags & WSM_RX_STATUS_TSF_INCLUDED) { - memcpy(&hdr->mactime, skb->data + skb->len - 8, 8); - hdr->mactime = le64_to_cpu(hdr->mactime); + hdr->mactime = get_unaligned_le64(skb->data + skb->len - 8); if (skb->len >= 8) skb_trim(skb, skb->len - 8); } else { diff --git a/drivers/net/wireless/ti/wl1251/main.c b/drivers/net/wireless/ti/wl1251/main.c index 9144ef5538a8..289371689a8d 100644 --- a/drivers/net/wireless/ti/wl1251/main.c +++ b/drivers/net/wireless/ti/wl1251/main.c @@ -546,7 +546,7 @@ static int wl1251_build_null_data(struct wl1251 *wl) size = sizeof(struct wl12xx_null_data_template); ptr = NULL; } else { - skb = ieee80211_nullfunc_get(wl->hw, wl->vif, false); + skb = ieee80211_nullfunc_get(wl->hw, wl->vif, -1, false); if (!skb) goto out; size = skb->len; diff --git a/drivers/net/wireless/ti/wl18xx/event.c b/drivers/net/wireless/ti/wl18xx/event.c index 13d78ada4bb6..34d95f458e1a 100644 --- a/drivers/net/wireless/ti/wl18xx/event.c +++ b/drivers/net/wireless/ti/wl18xx/event.c @@ -131,10 +131,10 @@ int wl18xx_process_mailbox_events(struct wl1271 *wl) if (vector & TIME_SYNC_EVENT_ID) wlcore_event_time_sync(wl, - mbox->time_sync_tsf_high_msb, - mbox->time_sync_tsf_high_lsb, - mbox->time_sync_tsf_low_msb, - mbox->time_sync_tsf_low_lsb); + le16_to_cpu(mbox->time_sync_tsf_high_msb), + le16_to_cpu(mbox->time_sync_tsf_high_lsb), + le16_to_cpu(mbox->time_sync_tsf_low_msb), + le16_to_cpu(mbox->time_sync_tsf_low_lsb)); if (vector & RADAR_DETECTED_EVENT_ID) { wl1271_info("radar event: channel %d type %s", diff --git a/drivers/net/wireless/ti/wlcore/cmd.c b/drivers/net/wireless/ti/wlcore/cmd.c index 138edd28b0de..a939fd89a7f5 100644 --- a/drivers/net/wireless/ti/wlcore/cmd.c +++ b/drivers/net/wireless/ti/wlcore/cmd.c @@ -1065,7 +1065,7 @@ int wl12xx_cmd_build_null_data(struct wl1271 *wl, struct wl12xx_vif *wlvif) } else { skb = ieee80211_nullfunc_get(wl->hw, wl12xx_wlvif_to_vif(wlvif), - false); + -1, false); if (!skb) goto out; size = skb->len; @@ -1092,7 +1092,7 @@ int wl12xx_cmd_build_klv_null_data(struct wl1271 *wl, struct sk_buff *skb = NULL; int ret = -ENOMEM; - skb = ieee80211_nullfunc_get(wl->hw, vif, false); + skb = ieee80211_nullfunc_get(wl->hw, vif,-1, false); if (!skb) goto out; diff --git a/drivers/net/wireless/wl3501_cs.c b/drivers/net/wireless/wl3501_cs.c index dad38fc04243..1b532e00a56f 100644 --- a/drivers/net/wireless/wl3501_cs.c +++ b/drivers/net/wireless/wl3501_cs.c @@ -1441,7 +1441,7 @@ static void wl3501_detach(struct pcmcia_device *link) static int wl3501_get_name(struct net_device *dev, struct iw_request_info *info, union iwreq_data *wrqu, char *extra) { - strlcpy(wrqu->name, "IEEE 802.11-DS", sizeof(wrqu->name)); + strscpy(wrqu->name, "IEEE 802.11-DS", sizeof(wrqu->name)); return 0; } @@ -1652,7 +1652,7 @@ static int wl3501_set_nick(struct net_device *dev, struct iw_request_info *info, if (wrqu->data.length > sizeof(this->nick)) return -E2BIG; - strlcpy(this->nick, extra, wrqu->data.length); + strscpy(this->nick, extra, wrqu->data.length); return 0; } @@ -1661,7 +1661,7 @@ static int wl3501_get_nick(struct net_device *dev, struct iw_request_info *info, { struct wl3501_card *this = netdev_priv(dev); - strlcpy(extra, this->nick, 32); + strscpy(extra, this->nick, 32); wrqu->data.length = strlen(extra); return 0; } @@ -1965,7 +1965,7 @@ static int wl3501_config(struct pcmcia_device *link) this->firmware_date[0] = '\0'; this->rssi = 255; this->chan = iw_default_channel(this->reg_domain); - strlcpy(this->nick, "Planet WL3501", sizeof(this->nick)); + strscpy(this->nick, "Planet WL3501", sizeof(this->nick)); spin_lock_init(&this->lock); init_waitqueue_head(&this->wait); netif_start_queue(dev); diff --git a/drivers/net/wwan/iosm/iosm_ipc_imem_ops.c b/drivers/net/wwan/iosm/iosm_ipc_imem_ops.c index 57304a5adf68..b7f9237dedf7 100644 --- a/drivers/net/wwan/iosm/iosm_ipc_imem_ops.c +++ b/drivers/net/wwan/iosm/iosm_ipc_imem_ops.c @@ -590,7 +590,7 @@ int ipc_imem_sys_devlink_write(struct iosm_devlink *ipc_devlink, goto out; } - memcpy(skb_put(skb, count), buf, count); + skb_put_data(skb, buf, count); IPC_CB(skb)->op_type = UL_USR_OP_BLOCKED; diff --git a/drivers/net/wwan/iosm/iosm_ipc_wwan.c b/drivers/net/wwan/iosm/iosm_ipc_wwan.c index 27151148c782..2f1f8b5d5b59 100644 --- a/drivers/net/wwan/iosm/iosm_ipc_wwan.c +++ b/drivers/net/wwan/iosm/iosm_ipc_wwan.c @@ -103,8 +103,8 @@ static int ipc_wwan_link_stop(struct net_device *netdev) } /* Transmit a packet */ -static int ipc_wwan_link_transmit(struct sk_buff *skb, - struct net_device *netdev) +static netdev_tx_t ipc_wwan_link_transmit(struct sk_buff *skb, + struct net_device *netdev) { struct iosm_netdev_priv *priv = wwan_netdev_drvpriv(netdev); struct iosm_wwan *ipc_wwan = priv->ipc_wwan; @@ -323,15 +323,16 @@ struct iosm_wwan *ipc_wwan_init(struct iosm_imem *ipc_imem, struct device *dev) ipc_wwan->dev = dev; ipc_wwan->ipc_imem = ipc_imem; + mutex_init(&ipc_wwan->if_mutex); + /* WWAN core will create a netdev for the default IP MUX channel */ if (wwan_register_ops(ipc_wwan->dev, &iosm_wwan_ops, ipc_wwan, IP_MUX_SESSION_DEFAULT)) { + mutex_destroy(&ipc_wwan->if_mutex); kfree(ipc_wwan); return NULL; } - mutex_init(&ipc_wwan->if_mutex); - return ipc_wwan; } diff --git a/drivers/net/wwan/mhi_wwan_ctrl.c b/drivers/net/wwan/mhi_wwan_ctrl.c index e4d0f696687f..f7ca52353f40 100644 --- a/drivers/net/wwan/mhi_wwan_ctrl.c +++ b/drivers/net/wwan/mhi_wwan_ctrl.c @@ -258,6 +258,7 @@ static void mhi_wwan_ctrl_remove(struct mhi_device *mhi_dev) static const struct mhi_device_id mhi_wwan_ctrl_match_table[] = { { .chan = "DUN", .driver_data = WWAN_PORT_AT }, + { .chan = "DUN2", .driver_data = WWAN_PORT_AT }, { .chan = "MBIM", .driver_data = WWAN_PORT_MBIM }, { .chan = "QMI", .driver_data = WWAN_PORT_QMI }, { .chan = "DIAG", .driver_data = WWAN_PORT_QCDM }, diff --git a/drivers/net/wwan/t7xx/t7xx_netdev.c b/drivers/net/wwan/t7xx/t7xx_netdev.c index c6b6547f2c6f..f71d3bc3b237 100644 --- a/drivers/net/wwan/t7xx/t7xx_netdev.c +++ b/drivers/net/wwan/t7xx/t7xx_netdev.c @@ -74,7 +74,7 @@ static int t7xx_ccmni_send_packet(struct t7xx_ccmni *ccmni, struct sk_buff *skb, return 0; } -static int t7xx_ccmni_start_xmit(struct sk_buff *skb, struct net_device *dev) +static netdev_tx_t t7xx_ccmni_start_xmit(struct sk_buff *skb, struct net_device *dev) { struct t7xx_ccmni *ccmni = wwan_netdev_drvpriv(dev); int skb_len = skb->len; diff --git a/drivers/net/wwan/wwan_hwsim.c b/drivers/net/wwan/wwan_hwsim.c index fad642f9ffd8..ff09a8cedf93 100644 --- a/drivers/net/wwan/wwan_hwsim.c +++ b/drivers/net/wwan/wwan_hwsim.c @@ -157,8 +157,8 @@ static int wwan_hwsim_port_tx(struct wwan_port *wport, struct sk_buff *in) if ((i + 1) < in->len && in->data[i + 1] == '\n') i++; n = i - s + 1; - memcpy(skb_put(out, n), &in->data[s], n);/* Echo */ - memcpy(skb_put(out, 6), "\r\nOK\r\n", 6); + skb_put_data(out, &in->data[s], n);/* Echo */ + skb_put_data(out, "\r\nOK\r\n", 6); s = i + 1; port->pstate = AT_PARSER_WAIT_A; } else if (port->pstate == AT_PARSER_SKIP_LINE) { @@ -171,7 +171,7 @@ static int wwan_hwsim_port_tx(struct wwan_port *wport, struct sk_buff *in) if (i > s) { /* Echo the processed portion of a not yet completed command */ n = i - s; - memcpy(skb_put(out, n), &in->data[s], n); + skb_put_data(out, &in->data[s], n); } consume_skb(in); diff --git a/drivers/net/xen-netback/common.h b/drivers/net/xen-netback/common.h index 8174d7b2966c..1545cbee77a4 100644 --- a/drivers/net/xen-netback/common.h +++ b/drivers/net/xen-netback/common.h @@ -62,7 +62,7 @@ struct pending_tx_info { * ubuf_to_vif is a helper which finds the struct xenvif from a pointer * to this field. */ - struct ubuf_info callback_struct; + struct ubuf_info_msgzc callback_struct; }; #define XEN_NETIF_TX_RING_SIZE __CONST_RING_SIZE(xen_netif_tx, XEN_PAGE_SIZE) diff --git a/drivers/net/xen-netback/interface.c b/drivers/net/xen-netback/interface.c index fb32ae82d9b0..650fa180220f 100644 --- a/drivers/net/xen-netback/interface.c +++ b/drivers/net/xen-netback/interface.c @@ -591,8 +591,8 @@ int xenvif_init_queue(struct xenvif_queue *queue) } for (i = 0; i < MAX_PENDING_REQS; i++) { - queue->pending_tx_info[i].callback_struct = (struct ubuf_info) - { .callback = xenvif_zerocopy_callback, + queue->pending_tx_info[i].callback_struct = (struct ubuf_info_msgzc) + { { .callback = xenvif_zerocopy_callback }, { { .ctx = NULL, .desc = i } } }; queue->grant_tx_handle[i] = NETBACK_INVALID_HANDLE; @@ -723,8 +723,7 @@ int xenvif_connect_data(struct xenvif_queue *queue, init_waitqueue_head(&queue->dealloc_wq); atomic_set(&queue->inflight_packets, 0); - netif_napi_add(queue->vif->dev, &queue->napi, xenvif_poll, - NAPI_POLL_WEIGHT); + netif_napi_add(queue->vif->dev, &queue->napi, xenvif_poll); queue->stalled = true; diff --git a/drivers/net/xen-netback/netback.c b/drivers/net/xen-netback/netback.c index a256695fc89e..3d2081bbbc86 100644 --- a/drivers/net/xen-netback/netback.c +++ b/drivers/net/xen-netback/netback.c @@ -133,7 +133,7 @@ static inline unsigned long idx_to_kaddr(struct xenvif_queue *queue, /* Find the containing VIF's structure from a pointer in pending_tx_info array */ -static inline struct xenvif_queue *ubuf_to_queue(const struct ubuf_info *ubuf) +static inline struct xenvif_queue *ubuf_to_queue(const struct ubuf_info_msgzc *ubuf) { u16 pending_idx = ubuf->desc; struct pending_tx_info *temp = @@ -1228,11 +1228,12 @@ static int xenvif_tx_submit(struct xenvif_queue *queue) return work_done; } -void xenvif_zerocopy_callback(struct sk_buff *skb, struct ubuf_info *ubuf, +void xenvif_zerocopy_callback(struct sk_buff *skb, struct ubuf_info *ubuf_base, bool zerocopy_success) { unsigned long flags; pending_ring_idx_t index; + struct ubuf_info_msgzc *ubuf = uarg_to_msgzc(ubuf_base); struct xenvif_queue *queue = ubuf_to_queue(ubuf); /* This is the only place where we grab this lock, to protect callbacks @@ -1241,7 +1242,7 @@ void xenvif_zerocopy_callback(struct sk_buff *skb, struct ubuf_info *ubuf, spin_lock_irqsave(&queue->callback_lock, flags); do { u16 pending_idx = ubuf->desc; - ubuf = (struct ubuf_info *) ubuf->ctx; + ubuf = (struct ubuf_info_msgzc *) ubuf->ctx; BUG_ON(queue->dealloc_prod - queue->dealloc_cons >= MAX_PENDING_REQS); index = pending_index(queue->dealloc_prod); diff --git a/drivers/net/xen-netback/xenbus.c b/drivers/net/xen-netback/xenbus.c index e85b3c5d4acc..c1ba4294f364 100644 --- a/drivers/net/xen-netback/xenbus.c +++ b/drivers/net/xen-netback/xenbus.c @@ -865,13 +865,12 @@ static int connect_data_rings(struct backend_info *be, * queue-N. */ if (num_queues == 1) { - xspath = kzalloc(strlen(dev->otherend) + 1, GFP_KERNEL); + xspath = kstrdup(dev->otherend, GFP_KERNEL); if (!xspath) { xenbus_dev_fatal(dev, -ENOMEM, "reading ring references"); return -ENOMEM; } - strcpy(xspath, dev->otherend); } else { xspathsize = strlen(dev->otherend) + xenstore_path_ext_size; xspath = kzalloc(xspathsize, GFP_KERNEL); diff --git a/drivers/net/xen-netfront.c b/drivers/net/xen-netfront.c index 27a11cc08c61..9af2b027c19c 100644 --- a/drivers/net/xen-netfront.c +++ b/drivers/net/xen-netfront.c @@ -673,7 +673,7 @@ static int xennet_xdp_xmit(struct net_device *dev, int n, return nxmit; } -struct sk_buff *bounce_skb(const struct sk_buff *skb) +static struct sk_buff *bounce_skb(const struct sk_buff *skb) { unsigned int headerlen = skb_headroom(skb); /* Align size to allocate full pages and avoid contiguous data leaks */ @@ -2224,8 +2224,7 @@ static int xennet_create_queues(struct netfront_info *info, return ret; } - netif_napi_add(queue->info->netdev, &queue->napi, - xennet_poll, 64); + netif_napi_add(queue->info->netdev, &queue->napi, xennet_poll); if (netif_running(info->netdev)) napi_enable(&queue->napi); } diff --git a/drivers/of/base.c b/drivers/of/base.c index 7fa960bd3df1..42da760e0f45 100644 --- a/drivers/of/base.c +++ b/drivers/of/base.c @@ -578,6 +578,7 @@ int of_device_compatible_match(struct device_node *device, return score; } +EXPORT_SYMBOL_GPL(of_device_compatible_match); /** * of_machine_is_compatible - Test root of device tree for a given compatible value diff --git a/drivers/phy/microchip/lan966x_serdes.c b/drivers/phy/microchip/lan966x_serdes.c index e86a879b92b5..d1a50fa81130 100644 --- a/drivers/phy/microchip/lan966x_serdes.c +++ b/drivers/phy/microchip/lan966x_serdes.c @@ -401,6 +401,9 @@ static int serdes_set_mode(struct phy *phy, enum phy_mode mode, int submode) submode == PHY_INTERFACE_MODE_2500BASEX) submode = PHY_INTERFACE_MODE_SGMII; + if (submode == PHY_INTERFACE_MODE_QUSGMII) + submode = PHY_INTERFACE_MODE_QSGMII; + for (i = 0; i < ARRAY_SIZE(lan966x_serdes_muxes); i++) { if (macro->idx != lan966x_serdes_muxes[i].idx || mode != lan966x_serdes_muxes[i].mode || diff --git a/drivers/pinctrl/Kconfig b/drivers/pinctrl/Kconfig index 1cf74b0c42e5..d768dcf75cf1 100644 --- a/drivers/pinctrl/Kconfig +++ b/drivers/pinctrl/Kconfig @@ -292,7 +292,7 @@ config PINCTRL_MCP23S08 corresponding interrupt-controller. config PINCTRL_MICROCHIP_SGPIO - bool "Pinctrl driver for Microsemi/Microchip Serial GPIO" + tristate "Pinctrl driver for Microsemi/Microchip Serial GPIO" depends on OF depends on HAS_IOMEM select GPIOLIB @@ -310,6 +310,9 @@ config PINCTRL_MICROCHIP_SGPIO connect control signals from SFP modules and to act as an LED controller. + If compiled as a module, the module name will be + pinctrl-microchip-sgpio. + config PINCTRL_OCELOT tristate "Pinctrl driver for the Microsemi Ocelot and Jaguar2 SoCs" depends on OF diff --git a/drivers/pinctrl/pinctrl-microchip-sgpio.c b/drivers/pinctrl/pinctrl-microchip-sgpio.c index 6f55bf7d5e05..2b4167a09b3b 100644 --- a/drivers/pinctrl/pinctrl-microchip-sgpio.c +++ b/drivers/pinctrl/pinctrl-microchip-sgpio.c @@ -12,6 +12,7 @@ #include <linux/clk.h> #include <linux/gpio/driver.h> #include <linux/io.h> +#include <linux/mfd/ocelot.h> #include <linux/mod_devicetable.h> #include <linux/module.h> #include <linux/pinctrl/pinmux.h> @@ -904,7 +905,6 @@ static int microchip_sgpio_probe(struct platform_device *pdev) struct reset_control *reset; struct sgpio_priv *priv; struct clk *clk; - u32 __iomem *regs; u32 val; struct regmap_config regmap_config = { .reg_bits = 32, @@ -937,11 +937,7 @@ static int microchip_sgpio_probe(struct platform_device *pdev) return -EINVAL; } - regs = devm_platform_ioremap_resource(pdev, 0); - if (IS_ERR(regs)) - return PTR_ERR(regs); - - priv->regs = devm_regmap_init_mmio(dev, regs, ®map_config); + priv->regs = ocelot_regmap_from_resource(pdev, 0, ®map_config); if (IS_ERR(priv->regs)) return PTR_ERR(priv->regs); @@ -999,6 +995,7 @@ static const struct of_device_id microchip_sgpio_gpio_of_match[] = { /* sentinel */ } }; +MODULE_DEVICE_TABLE(of, microchip_sgpio_gpio_of_match); static struct platform_driver microchip_sgpio_pinctrl_driver = { .driver = { @@ -1008,4 +1005,7 @@ static struct platform_driver microchip_sgpio_pinctrl_driver = { }, .probe = microchip_sgpio_probe, }; -builtin_platform_driver(microchip_sgpio_pinctrl_driver); +module_platform_driver(microchip_sgpio_pinctrl_driver); + +MODULE_DESCRIPTION("Microchip SGPIO Pinctrl Driver"); +MODULE_LICENSE("GPL"); diff --git a/drivers/pinctrl/pinctrl-ocelot.c b/drivers/pinctrl/pinctrl-ocelot.c index c7df8c5fe585..83464e0bf4e6 100644 --- a/drivers/pinctrl/pinctrl-ocelot.c +++ b/drivers/pinctrl/pinctrl-ocelot.c @@ -10,6 +10,7 @@ #include <linux/gpio/driver.h> #include <linux/interrupt.h> #include <linux/io.h> +#include <linux/mfd/ocelot.h> #include <linux/of_device.h> #include <linux/of_irq.h> #include <linux/of_platform.h> @@ -2044,7 +2045,6 @@ static int ocelot_pinctrl_probe(struct platform_device *pdev) struct ocelot_pinctrl *info; struct reset_control *reset; struct regmap *pincfg; - void __iomem *base; int ret; struct regmap_config regmap_config = { .reg_bits = 32, @@ -2077,20 +2077,14 @@ static int ocelot_pinctrl_probe(struct platform_device *pdev) "Failed to get reset\n"); reset_control_reset(reset); - base = devm_ioremap_resource(dev, - platform_get_resource(pdev, IORESOURCE_MEM, 0)); - if (IS_ERR(base)) - return PTR_ERR(base); - info->stride = 1 + (info->desc->npins - 1) / 32; regmap_config.max_register = OCELOT_GPIO_SD_MAP * info->stride + 15 * 4; - info->map = devm_regmap_init_mmio(dev, base, ®map_config); - if (IS_ERR(info->map)) { - dev_err(dev, "Failed to create regmap\n"); - return PTR_ERR(info->map); - } + info->map = ocelot_regmap_from_resource(pdev, 0, ®map_config); + if (IS_ERR(info->map)) + return dev_err_probe(dev, PTR_ERR(info->map), + "Failed to create regmap\n"); dev_set_drvdata(dev, info); info->dev = dev; diff --git a/drivers/ptp/ptp_clock.c b/drivers/ptp/ptp_clock.c index 688cde320bb0..51cae72bb6db 100644 --- a/drivers/ptp/ptp_clock.c +++ b/drivers/ptp/ptp_clock.c @@ -174,7 +174,7 @@ static void ptp_clock_release(struct device *dev) mutex_destroy(&ptp->tsevq_mux); mutex_destroy(&ptp->pincfg_mux); mutex_destroy(&ptp->n_vclocks_mux); - ida_simple_remove(&ptp_clocks_map, ptp->index); + ida_free(&ptp_clocks_map, ptp->index); kfree(ptp); } @@ -217,7 +217,7 @@ struct ptp_clock *ptp_clock_register(struct ptp_clock_info *info, if (ptp == NULL) goto no_memory; - index = ida_simple_get(&ptp_clocks_map, 0, MINORMASK + 1, GFP_KERNEL); + index = ida_alloc_max(&ptp_clocks_map, MINORMASK, GFP_KERNEL); if (index < 0) { err = index; goto no_slot; @@ -332,7 +332,7 @@ kworker_err: mutex_destroy(&ptp->tsevq_mux); mutex_destroy(&ptp->pincfg_mux); mutex_destroy(&ptp->n_vclocks_mux); - ida_simple_remove(&ptp_clocks_map, index); + ida_free(&ptp_clocks_map, index); no_slot: kfree(ptp); no_memory: diff --git a/drivers/ptp/ptp_ocp.c b/drivers/ptp/ptp_ocp.c index e59ea2173aac..d36c3f597f77 100644 --- a/drivers/ptp/ptp_ocp.c +++ b/drivers/ptp/ptp_ocp.c @@ -1311,12 +1311,6 @@ fail: goto out; } -static int -ptp_ocp_firstchild(struct device *dev, void *data) -{ - return 1; -} - static struct device * ptp_ocp_find_flash(struct ptp_ocp *bp) { @@ -1325,7 +1319,7 @@ ptp_ocp_find_flash(struct ptp_ocp *bp) last = NULL; dev = &bp->spi_flash->dev; - while ((dev = device_find_child(dev, NULL, ptp_ocp_firstchild))) { + while ((dev = device_find_any_child(dev))) { if (!strcmp("mtd", dev_bus_name(dev))) break; put_device(last); diff --git a/drivers/s390/net/qeth_l2_main.c b/drivers/s390/net/qeth_l2_main.c index 2d4436cbcb47..9dc935886e9f 100644 --- a/drivers/s390/net/qeth_l2_main.c +++ b/drivers/s390/net/qeth_l2_main.c @@ -1133,7 +1133,7 @@ static int qeth_l2_setup_netdev(struct qeth_card *card) PAGE_SIZE * (QDIO_MAX_ELEMENTS_PER_BUFFER - 1)); } - netif_napi_add(card->dev, &card->napi, qeth_poll, NAPI_POLL_WEIGHT); + netif_napi_add(card->dev, &card->napi, qeth_poll); return register_netdev(card->dev); } @@ -1530,8 +1530,8 @@ static void qeth_addr_change_event(struct qeth_card *card, else INIT_DELAYED_WORK(&data->dwork, qeth_l2_dev2br_worker); data->card = card; - memcpy(&data->ac_event, hostevs, - sizeof(struct qeth_ipacmd_addr_change) + extrasize); + data->ac_event = *hostevs; + memcpy(data->ac_event.entry, hostevs->entry, extrasize); queue_delayed_work(card->event_wq, &data->dwork, 0); } diff --git a/drivers/s390/net/qeth_l3_main.c b/drivers/s390/net/qeth_l3_main.c index 8d44bce0477a..d8487a10cd55 100644 --- a/drivers/s390/net/qeth_l3_main.c +++ b/drivers/s390/net/qeth_l3_main.c @@ -1910,7 +1910,7 @@ static int qeth_l3_setup_netdev(struct qeth_card *card) netif_set_tso_max_size(card->dev, PAGE_SIZE * (QETH_MAX_BUFFER_ELEMENTS(card) - 1)); - netif_napi_add(card->dev, &card->napi, qeth_poll, NAPI_POLL_WEIGHT); + netif_napi_add(card->dev, &card->napi, qeth_poll); return register_netdev(card->dev); } diff --git a/drivers/soc/fsl/qbman/qman.c b/drivers/soc/fsl/qbman/qman.c index fde4edd83c14..739e4eee6b75 100644 --- a/drivers/soc/fsl/qbman/qman.c +++ b/drivers/soc/fsl/qbman/qman.c @@ -2483,13 +2483,8 @@ out: } EXPORT_SYMBOL(qman_create_cgr); -int qman_delete_cgr(struct qman_cgr *cgr) +static struct qman_portal *qman_cgr_get_affine_portal(struct qman_cgr *cgr) { - unsigned long irqflags; - struct qm_mcr_querycgr cgr_state; - struct qm_mcc_initcgr local_opts; - int ret = 0; - struct qman_cgr *i; struct qman_portal *p = get_affine_portal(); if (cgr->chan != p->config->channel) { @@ -2497,10 +2492,25 @@ int qman_delete_cgr(struct qman_cgr *cgr) dev_err(p->config->dev, "CGR not owned by current portal"); dev_dbg(p->config->dev, " create 0x%x, delete 0x%x\n", cgr->chan, p->config->channel); - - ret = -EINVAL; - goto put_portal; + put_affine_portal(); + return NULL; } + + return p; +} + +int qman_delete_cgr(struct qman_cgr *cgr) +{ + unsigned long irqflags; + struct qm_mcr_querycgr cgr_state; + struct qm_mcc_initcgr local_opts; + int ret = 0; + struct qman_cgr *i; + struct qman_portal *p = qman_cgr_get_affine_portal(cgr); + + if (!p) + return -EINVAL; + memset(&local_opts, 0, sizeof(struct qm_mcc_initcgr)); spin_lock_irqsave(&p->cgr_lock, irqflags); list_del(&cgr->node); @@ -2528,7 +2538,6 @@ int qman_delete_cgr(struct qman_cgr *cgr) list_add(&cgr->node, &p->cgr_cbs); release_lock: spin_unlock_irqrestore(&p->cgr_lock, irqflags); -put_portal: put_affine_portal(); return ret; } @@ -2559,6 +2568,54 @@ void qman_delete_cgr_safe(struct qman_cgr *cgr) } EXPORT_SYMBOL(qman_delete_cgr_safe); +static int qman_update_cgr(struct qman_cgr *cgr, struct qm_mcc_initcgr *opts) +{ + int ret; + unsigned long irqflags; + struct qman_portal *p = qman_cgr_get_affine_portal(cgr); + + if (!p) + return -EINVAL; + + spin_lock_irqsave(&p->cgr_lock, irqflags); + ret = qm_modify_cgr(cgr, 0, opts); + spin_unlock_irqrestore(&p->cgr_lock, irqflags); + put_affine_portal(); + return ret; +} + +struct update_cgr_params { + struct qman_cgr *cgr; + struct qm_mcc_initcgr *opts; + int ret; +}; + +static void qman_update_cgr_smp_call(void *p) +{ + struct update_cgr_params *params = p; + + params->ret = qman_update_cgr(params->cgr, params->opts); +} + +int qman_update_cgr_safe(struct qman_cgr *cgr, struct qm_mcc_initcgr *opts) +{ + struct update_cgr_params params = { + .cgr = cgr, + .opts = opts, + }; + + preempt_disable(); + if (qman_cgr_cpus[cgr->cgrid] != smp_processor_id()) + smp_call_function_single(qman_cgr_cpus[cgr->cgrid], + qman_update_cgr_smp_call, ¶ms, + true); + else + params.ret = qman_update_cgr(cgr, opts); + preempt_enable(); + return params.ret; +} +EXPORT_SYMBOL(qman_update_cgr_safe); + /* Cleanup FQs */ static int _qm_mr_consume_and_match_verb(struct qm_portal *p, int v) diff --git a/drivers/staging/qlge/qlge_main.c b/drivers/staging/qlge/qlge_main.c index ca6b966f5dd3..1ead7793062a 100644 --- a/drivers/staging/qlge/qlge_main.c +++ b/drivers/staging/qlge/qlge_main.c @@ -3041,8 +3041,8 @@ static int qlge_start_rx_ring(struct qlge_adapter *qdev, struct rx_ring *rx_ring /* Inbound completion handling rx_rings run in * separate NAPI contexts. */ - netif_napi_add_weight(qdev->ndev, &rx_ring->napi, - qlge_napi_poll_msix, 64); + netif_napi_add(qdev->ndev, &rx_ring->napi, + qlge_napi_poll_msix); cqicb->irq_delay = cpu_to_le16(qdev->rx_coalesce_usecs); cqicb->pkt_delay = cpu_to_le16(qdev->rx_max_coalesced_frames); } else { diff --git a/drivers/staging/rtl8723bs/os_dep/ioctl_cfg80211.c b/drivers/staging/rtl8723bs/os_dep/ioctl_cfg80211.c index cb6d287f580d..2d09be6425d5 100644 --- a/drivers/staging/rtl8723bs/os_dep/ioctl_cfg80211.c +++ b/drivers/staging/rtl8723bs/os_dep/ioctl_cfg80211.c @@ -850,8 +850,8 @@ exit: } static int cfg80211_rtw_add_key(struct wiphy *wiphy, struct net_device *ndev, - u8 key_index, bool pairwise, const u8 *mac_addr, - struct key_params *params) + int link_id, u8 key_index, bool pairwise, + const u8 *mac_addr, struct key_params *params) { char *alg_name; u32 param_len; @@ -932,8 +932,8 @@ addkey_end: } static int cfg80211_rtw_get_key(struct wiphy *wiphy, struct net_device *ndev, - u8 key_index, bool pairwise, const u8 *mac_addr, - void *cookie, + int link_id, u8 key_index, bool pairwise, + const u8 *mac_addr, void *cookie, void (*callback)(void *cookie, struct key_params*)) { @@ -941,7 +941,8 @@ static int cfg80211_rtw_get_key(struct wiphy *wiphy, struct net_device *ndev, } static int cfg80211_rtw_del_key(struct wiphy *wiphy, struct net_device *ndev, - u8 key_index, bool pairwise, const u8 *mac_addr) + int link_id, u8 key_index, bool pairwise, + const u8 *mac_addr) { struct adapter *padapter = rtw_netdev_priv(ndev); struct security_priv *psecuritypriv = &padapter->securitypriv; @@ -955,7 +956,7 @@ static int cfg80211_rtw_del_key(struct wiphy *wiphy, struct net_device *ndev, } static int cfg80211_rtw_set_default_key(struct wiphy *wiphy, - struct net_device *ndev, u8 key_index + struct net_device *ndev, int link_id, u8 key_index , bool unicast, bool multicast ) { diff --git a/drivers/staging/wlan-ng/cfg80211.c b/drivers/staging/wlan-ng/cfg80211.c index b7b56d8406d1..471bb310176f 100644 --- a/drivers/staging/wlan-ng/cfg80211.c +++ b/drivers/staging/wlan-ng/cfg80211.c @@ -143,8 +143,8 @@ exit: } static int prism2_add_key(struct wiphy *wiphy, struct net_device *dev, - u8 key_index, bool pairwise, const u8 *mac_addr, - struct key_params *params) + int link_id, u8 key_index, bool pairwise, + const u8 *mac_addr, struct key_params *params) { struct wlandevice *wlandev = dev->ml_priv; u32 did; @@ -172,7 +172,7 @@ static int prism2_add_key(struct wiphy *wiphy, struct net_device *dev, } static int prism2_get_key(struct wiphy *wiphy, struct net_device *dev, - u8 key_index, bool pairwise, + int link_id, u8 key_index, bool pairwise, const u8 *mac_addr, void *cookie, void (*callback)(void *cookie, struct key_params*)) { @@ -202,7 +202,8 @@ static int prism2_get_key(struct wiphy *wiphy, struct net_device *dev, } static int prism2_del_key(struct wiphy *wiphy, struct net_device *dev, - u8 key_index, bool pairwise, const u8 *mac_addr) + int link_id, u8 key_index, bool pairwise, + const u8 *mac_addr) { struct wlandevice *wlandev = dev->ml_priv; u32 did; @@ -227,7 +228,8 @@ static int prism2_del_key(struct wiphy *wiphy, struct net_device *dev, } static int prism2_set_default_key(struct wiphy *wiphy, struct net_device *dev, - u8 key_index, bool unicast, bool multicast) + int link_id, u8 key_index, bool unicast, + bool multicast) { struct wlandevice *wlandev = dev->ml_priv; diff --git a/drivers/target/target_core_user.c b/drivers/target/target_core_user.c index 3deaeecb712e..2940559c3086 100644 --- a/drivers/target/target_core_user.c +++ b/drivers/target/target_core_user.c @@ -486,6 +486,7 @@ static struct genl_family tcmu_genl_family __ro_after_init = { .netnsok = true, .small_ops = tcmu_genl_ops, .n_small_ops = ARRAY_SIZE(tcmu_genl_ops), + .resv_start_op = TCMU_CMD_SET_FEATURES + 1, }; #define tcmu_cmd_set_dbi_cur(cmd, index) ((cmd)->dbi_cur = (index)) diff --git a/drivers/thermal/thermal_netlink.c b/drivers/thermal/thermal_netlink.c index 050d243a5fa1..e2d78a996b5f 100644 --- a/drivers/thermal/thermal_netlink.c +++ b/drivers/thermal/thermal_netlink.c @@ -693,6 +693,7 @@ static struct genl_family thermal_gnl_family __ro_after_init = { .policy = thermal_genl_policy, .small_ops = thermal_genl_ops, .n_small_ops = ARRAY_SIZE(thermal_genl_ops), + .resv_start_op = THERMAL_GENL_CMD_CDEV_GET + 1, .mcgrps = thermal_genl_mcgrps, .n_mcgrps = ARRAY_SIZE(thermal_genl_mcgrps), }; diff --git a/drivers/thunderbolt/nhi.c b/drivers/thunderbolt/nhi.c index cb8c9c4ae93a..b5cd9673e15d 100644 --- a/drivers/thunderbolt/nhi.c +++ b/drivers/thunderbolt/nhi.c @@ -28,7 +28,11 @@ #define RING_TYPE(ring) ((ring)->is_tx ? "TX ring" : "RX ring") #define RING_FIRST_USABLE_HOPID 1 - +/* + * Used with QUIRK_E2E to specify an unused HopID the Rx credits are + * transferred. + */ +#define RING_E2E_RESERVED_HOPID RING_FIRST_USABLE_HOPID /* * Minimal number of vectors when we use MSI-X. Two for control channel * Rx/Tx and the rest four are for cross domain DMA paths. @@ -38,7 +42,9 @@ #define NHI_MAILBOX_TIMEOUT 500 /* ms */ +/* Host interface quirks */ #define QUIRK_AUTO_CLEAR_INT BIT(0) +#define QUIRK_E2E BIT(1) static int ring_interrupt_index(struct tb_ring *ring) { @@ -458,8 +464,18 @@ static void ring_release_msix(struct tb_ring *ring) static int nhi_alloc_hop(struct tb_nhi *nhi, struct tb_ring *ring) { + unsigned int start_hop = RING_FIRST_USABLE_HOPID; int ret = 0; + if (nhi->quirks & QUIRK_E2E) { + start_hop = RING_FIRST_USABLE_HOPID + 1; + if (ring->flags & RING_FLAG_E2E && !ring->is_tx) { + dev_dbg(&nhi->pdev->dev, "quirking E2E TX HopID %u -> %u\n", + ring->e2e_tx_hop, RING_E2E_RESERVED_HOPID); + ring->e2e_tx_hop = RING_E2E_RESERVED_HOPID; + } + } + spin_lock_irq(&nhi->lock); if (ring->hop < 0) { @@ -469,7 +485,7 @@ static int nhi_alloc_hop(struct tb_nhi *nhi, struct tb_ring *ring) * Automatically allocate HopID from the non-reserved * range 1 .. hop_count - 1. */ - for (i = RING_FIRST_USABLE_HOPID; i < nhi->hop_count; i++) { + for (i = start_hop; i < nhi->hop_count; i++) { if (ring->is_tx) { if (!nhi->tx_rings[i]) { ring->hop = i; @@ -484,6 +500,11 @@ static int nhi_alloc_hop(struct tb_nhi *nhi, struct tb_ring *ring) } } + if (ring->hop > 0 && ring->hop < start_hop) { + dev_warn(&nhi->pdev->dev, "invalid hop: %d\n", ring->hop); + ret = -EINVAL; + goto err_unlock; + } if (ring->hop < 0 || ring->hop >= nhi->hop_count) { dev_warn(&nhi->pdev->dev, "invalid hop: %d\n", ring->hop); ret = -EINVAL; @@ -1097,12 +1118,26 @@ static void nhi_shutdown(struct tb_nhi *nhi) static void nhi_check_quirks(struct tb_nhi *nhi) { - /* - * Intel hardware supports auto clear of the interrupt status - * reqister right after interrupt is being issued. - */ - if (nhi->pdev->vendor == PCI_VENDOR_ID_INTEL) + if (nhi->pdev->vendor == PCI_VENDOR_ID_INTEL) { + /* + * Intel hardware supports auto clear of the interrupt + * status register right after interrupt is being + * issued. + */ nhi->quirks |= QUIRK_AUTO_CLEAR_INT; + + switch (nhi->pdev->device) { + case PCI_DEVICE_ID_INTEL_FALCON_RIDGE_2C_NHI: + case PCI_DEVICE_ID_INTEL_FALCON_RIDGE_4C_NHI: + /* + * Falcon Ridge controller needs the end-to-end + * flow control workaround to avoid losing Rx + * packets when RING_FLAG_E2E is set. + */ + nhi->quirks |= QUIRK_E2E; + break; + } + } } static int nhi_check_iommu_pdev(struct pci_dev *pdev, void *data) diff --git a/drivers/thunderbolt/tb.c b/drivers/thunderbolt/tb.c index 9853f6c7e81d..9a277078338c 100644 --- a/drivers/thunderbolt/tb.c +++ b/drivers/thunderbolt/tb.c @@ -174,10 +174,10 @@ static void tb_discover_tunnels(struct tb *tb) } } -static int tb_port_configure_xdomain(struct tb_port *port) +static int tb_port_configure_xdomain(struct tb_port *port, struct tb_xdomain *xd) { if (tb_switch_is_usb4(port->sw)) - return usb4_port_configure_xdomain(port); + return usb4_port_configure_xdomain(port, xd); return tb_lc_configure_xdomain(port); } @@ -212,7 +212,7 @@ static void tb_scan_xdomain(struct tb_port *port) NULL); if (xd) { tb_port_at(route, sw)->xdomain = xd; - tb_port_configure_xdomain(port); + tb_port_configure_xdomain(port, xd); tb_xdomain_add(xd); } } @@ -1516,7 +1516,7 @@ static void tb_restore_children(struct tb_switch *sw) tb_restore_children(port->remote->sw); } else if (port->xdomain) { - tb_port_configure_xdomain(port); + tb_port_configure_xdomain(port, port->xdomain); } } } diff --git a/drivers/thunderbolt/tb.h b/drivers/thunderbolt/tb.h index 5db76de40cc1..0f067c06cba6 100644 --- a/drivers/thunderbolt/tb.h +++ b/drivers/thunderbolt/tb.h @@ -1176,7 +1176,7 @@ void usb4_switch_remove_ports(struct tb_switch *sw); int usb4_port_unlock(struct tb_port *port); int usb4_port_configure(struct tb_port *port); void usb4_port_unconfigure(struct tb_port *port); -int usb4_port_configure_xdomain(struct tb_port *port); +int usb4_port_configure_xdomain(struct tb_port *port, struct tb_xdomain *xd); void usb4_port_unconfigure_xdomain(struct tb_port *port); int usb4_port_router_offline(struct tb_port *port); int usb4_port_router_online(struct tb_port *port); diff --git a/drivers/thunderbolt/usb4.c b/drivers/thunderbolt/usb4.c index 3a2e7126db9d..a386228a44ee 100644 --- a/drivers/thunderbolt/usb4.c +++ b/drivers/thunderbolt/usb4.c @@ -1115,12 +1115,14 @@ static int usb4_set_xdomain_configured(struct tb_port *port, bool configured) /** * usb4_port_configure_xdomain() - Configure port for XDomain * @port: USB4 port connected to another host + * @xd: XDomain that is connected to the port * - * Marks the USB4 port as being connected to another host. Returns %0 in - * success and negative errno in failure. + * Marks the USB4 port as being connected to another host and updates + * the link type. Returns %0 in success and negative errno in failure. */ -int usb4_port_configure_xdomain(struct tb_port *port) +int usb4_port_configure_xdomain(struct tb_port *port, struct tb_xdomain *xd) { + xd->link_usb4 = link_is_usb4(port); return usb4_set_xdomain_configured(port, true); } diff --git a/drivers/thunderbolt/usb4_port.c b/drivers/thunderbolt/usb4_port.c index 6b02945624ee..1a30c0a23286 100644 --- a/drivers/thunderbolt/usb4_port.c +++ b/drivers/thunderbolt/usb4_port.c @@ -53,6 +53,8 @@ static ssize_t link_show(struct device *dev, struct device_attribute *attr, link = port->sw->link_usb4 ? "usb4" : "tbt"; else if (tb_port_has_remote(port)) link = port->remote->sw->link_usb4 ? "usb4" : "tbt"; + else if (port->xdomain) + link = port->xdomain->link_usb4 ? "usb4" : "tbt"; else link = "none"; diff --git a/drivers/vdpa/vdpa.c b/drivers/vdpa/vdpa.c index c06c02704461..7badf5777597 100644 --- a/drivers/vdpa/vdpa.c +++ b/drivers/vdpa/vdpa.c @@ -1183,6 +1183,7 @@ static struct genl_family vdpa_nl_family __ro_after_init = { .module = THIS_MODULE, .ops = vdpa_nl_ops, .n_ops = ARRAY_SIZE(vdpa_nl_ops), + .resv_start_op = VDPA_CMD_DEV_VSTATS_GET + 1, }; static int vdpa_init(void) diff --git a/drivers/vhost/net.c b/drivers/vhost/net.c index 68e4ecd1cc0e..d7a04d573988 100644 --- a/drivers/vhost/net.c +++ b/drivers/vhost/net.c @@ -118,7 +118,7 @@ struct vhost_net_virtqueue { /* Number of XDP frames batched */ int batched_xdp; /* an array of userspace buffers info */ - struct ubuf_info *ubuf_info; + struct ubuf_info_msgzc *ubuf_info; /* Reference counting for outstanding ubufs. * Protected by vq mutex. Writers must also take device mutex. */ struct vhost_net_ubuf_ref *ubufs; @@ -382,8 +382,9 @@ static void vhost_zerocopy_signal_used(struct vhost_net *net, } static void vhost_zerocopy_callback(struct sk_buff *skb, - struct ubuf_info *ubuf, bool success) + struct ubuf_info *ubuf_base, bool success) { + struct ubuf_info_msgzc *ubuf = uarg_to_msgzc(ubuf_base); struct vhost_net_ubuf_ref *ubufs = ubuf->ctx; struct vhost_virtqueue *vq = ubufs->vq; int cnt; @@ -871,7 +872,7 @@ static void handle_tx_zerocopy(struct vhost_net *net, struct socket *sock) size_t len, total_len = 0; int err; struct vhost_net_ubuf_ref *ubufs; - struct ubuf_info *ubuf; + struct ubuf_info_msgzc *ubuf; bool zcopy_used; int sent_pkts = 0; @@ -907,14 +908,14 @@ static void handle_tx_zerocopy(struct vhost_net *net, struct socket *sock) ubuf = nvq->ubuf_info + nvq->upend_idx; vq->heads[nvq->upend_idx].id = cpu_to_vhost32(vq, head); vq->heads[nvq->upend_idx].len = VHOST_DMA_IN_PROGRESS; - ubuf->callback = vhost_zerocopy_callback; ubuf->ctx = nvq->ubufs; ubuf->desc = nvq->upend_idx; - ubuf->flags = SKBFL_ZEROCOPY_FRAG; - refcount_set(&ubuf->refcnt, 1); + ubuf->ubuf.callback = vhost_zerocopy_callback; + ubuf->ubuf.flags = SKBFL_ZEROCOPY_FRAG; + refcount_set(&ubuf->ubuf.refcnt, 1); msg.msg_control = &ctl; ctl.type = TUN_MSG_UBUF; - ctl.ptr = ubuf; + ctl.ptr = &ubuf->ubuf; msg.msg_controllen = sizeof(ctl); ubufs = nvq->ubufs; atomic_inc(&ubufs->refcount); diff --git a/drivers/vhost/vsock.c b/drivers/vhost/vsock.c index 368330417bde..5703775af129 100644 --- a/drivers/vhost/vsock.c +++ b/drivers/vhost/vsock.c @@ -393,7 +393,7 @@ vhost_vsock_alloc_pkt(struct vhost_virtqueue *vq, return NULL; } - pkt->buf = kmalloc(pkt->len, GFP_KERNEL); + pkt->buf = kvmalloc(pkt->len, GFP_KERNEL); if (!pkt->buf) { kfree(pkt); return NULL; |