Enhance code standards, update contributing guidelines, and add tests for SYNC plugin functionality

Author: jokob-sk

.github/skills/code-standards/SKILL.md

@@ -5,12 +5,12 @@ description: NetAlertX coding standards and conventions. Use this when writing c
 
 # Code Standards
 
-- ask me to review before going to each next step (mention n step out of x)
-- before starting, prepare implementation plan
+- ask me to review before going to each next step (mention n step out of x)  (AI only)
+- before starting, prepare implementation plan  (AI only)
 - ask me to review it and ask any clarifying questions first
 - add test creation as last step - follow repo architecture patterns - do not place in the root of /test
 - code has to be maintainable, no duplicate code
-- follow DRY principle
+- follow DRY principle - maintainability of code is more important than speed of implementation
 - code files should be less than 500 LOC for better maintainability
 
 ## File Length

.github/workflows/run-all-tests.yml

@@ -3,6 +3,10 @@ name: 🧪 Manual Test Suite Selector
 on:
   workflow_dispatch:
     inputs:
+      run_all:
+        description: '✅ Run ALL tests (overrides individual selectors)'
+        type: boolean
+        default: false
       run_scan:
         description: '📂 scan/ (Scan, Logic, Locks, IPs)'
         type: boolean
@@ -23,6 +27,10 @@ on:
         description: '📂 ui/ (Selenium & Dashboard)'
         type: boolean
         default: false
+      run_plugins:
+        description: '📂 plugins/ (Sync insert schema-aware logic)'
+        type: boolean
+        default: false
       run_root_files:
         description: '📄 Root Test Files (WOL, Atomicity, etc.)'
         type: boolean
@@ -42,12 +50,20 @@ jobs:
         id: builder
         run: |
           PATHS=""
+
+          # run_all overrides everything
+          if [ "${{ github.event.inputs.run_all }}" == "true" ]; then
+            echo "final_paths=test/" >> $GITHUB_OUTPUT
+            exit 0
+          fi
+
           # Folder Mapping with 'test/' prefix
           if [ "${{ github.event.inputs.run_scan }}" == "true" ]; then PATHS="$PATHS test/scan/"; fi
           if [ "${{ github.event.inputs.run_api }}" == "true" ]; then PATHS="$PATHS test/api_endpoints/ test/server/"; fi
           if [ "${{ github.event.inputs.run_backend }}" == "true" ]; then PATHS="$PATHS test/backend/ test/db/"; fi
           if [ "${{ github.event.inputs.run_docker_env }}" == "true" ]; then PATHS="$PATHS test/docker_tests/"; fi
           if [ "${{ github.event.inputs.run_ui }}" == "true" ]; then PATHS="$PATHS test/ui/"; fi
+          if [ "${{ github.event.inputs.run_plugins }}" == "true" ]; then PATHS="$PATHS test/plugins/"; fi
 
           # Root Files Mapping (files sitting directly in /test/)
           if [ "${{ github.event.inputs.run_root_files }}" == "true" ]; then

CONTRIBUTING.md

@@ -1,23 +1,23 @@
-# 🤝 Contributing to NetAlertX
+# Contributing to NetAlertX
 
 First off, **thank you** for taking the time to contribute! NetAlertX is built and improved with the help of passionate people like you.
 
 ---
 
-## 📂 Issues, Bugs, and Feature Requests
+## Issues, Bugs, and Feature Requests
 
 Please use the [GitHub Issue Tracker](https://github.com/netalertx/NetAlertX/issues) for:
-- Bug reports 🐞
-- Feature requests 💡
-- Documentation feedback 📖
+- Bug reports
+- Feature requests
+- Documentation feedback
 
 Before opening a new issue:
-- 🛑 [Check Common Issues & Debug Tips](https://docs.netalertx.com/DEBUG_TIPS#common-issues)
-- 🔍 [Search Closed Issues](https://github.com/netalertx/NetAlertX/issues?q=is%3Aissue+is%3Aclosed)
+- [Check Common Issues & Debug Tips](https://docs.netalertx.com/DEBUG_TIPS#common-issues)
+- [Search Closed Issues](https://github.com/netalertx/NetAlertX/issues?q=is%3Aissue+is%3Aclosed)
 
 ---
 
-## 🚀 Submitting Pull Requests (PRs)
+## Submitting Pull Requests (PRs)
 
 We welcome PRs to improve the code, docs, or UI!
 
@@ -29,25 +29,30 @@ Please:
 - If relevant, add or update tests and documentation
 - For plugins, refer to the [Plugin Dev Guide](https://docs.netalertx.com/PLUGINS_DEV)
 
+
+## Code quality
+
+- read and follow the [code-standards](/.github/skills/code-standards/SKILL.md)
+
 ---
 
-## 🌟 First-Time Contributors
+## First-Time Contributors
 
 New to open source? Check out these resources:
 - [How to Fork and Submit a PR](https://opensource.guide/how-to-contribute/)
 - Ask questions or get support in our [Discord](https://discord.gg/NczTUTWyRr)
 
 ---
 
-## 🔐 Code of Conduct
+## Code of Conduct
 
 By participating, you agree to follow our [Code of Conduct](./CODE_OF_CONDUCT.md), which ensures a respectful and welcoming community.
 
 ---
 
-## 📬 Contact
+## Contact
 
 If you have more in-depth questions or want to discuss contributing in other ways, feel free to reach out at:
-📧 [jokob@duck.com](mailto:jokob@duck.com?subject=NetAlertX%20Contribution)
+[jokob.sk@gmail.com](mailto:jokob.sk@gmail.com?subject=NetAlertX%20Contribution)
 
 We appreciate every contribution, big or small! 💙

front/plugins/sync/sync.py

@@ -222,27 +222,30 @@ def main():
                         extra       = '',
                         foreignKey  = device['devGUID'])
 
+            # Resolve the actual columns that exist in the Devices table once.
+            # This automatically excludes computed/virtual fields (e.g. devStatus,
+            # devIsSleeping) and 'rowid' without needing a maintained exclusion list.
+            cursor.execute("PRAGMA table_info(Devices)")
+            db_columns = {row[1] for row in cursor.fetchall()}
+
             # Filter out existing devices
             new_devices = [device for device in device_data if device['devMac'] not in existing_mac_addresses]
 
-            #  Remove 'rowid' key if it exists
-            for device in new_devices:
-                device.pop('rowid', None)
-                device.pop('devStatus', None)
-
             mylog('verbose', [f'[{pluginName}] All devices: "{len(device_data)}"'])
             mylog('verbose', [f'[{pluginName}] New devices: "{len(new_devices)}"'])
 
             # Prepare the insert statement
             if new_devices:
 
-                # creating insert statement, removing 'rowid', 'devStatus' as handled on the target and devStatus is resolved on the fly
-                columns = ', '.join(k for k in new_devices[0].keys() if k not in ['rowid', 'devStatus'])
-                placeholders = ', '.join('?' for k in new_devices[0] if k not in ['rowid', 'devStatus'])
+                # Only keep keys that are real columns in the target DB; computed
+                # or unknown fields are silently dropped regardless of source schema.
+                insert_cols = [k for k in new_devices[0].keys() if k in db_columns]
+                columns = ', '.join(insert_cols)
+                placeholders = ', '.join('?' for _ in insert_cols)
                 sql = f'INSERT INTO Devices ({columns}) VALUES ({placeholders})'
 
-                # Extract values for the new devices
-                values = [tuple(device.values()) for device in new_devices]
+                # Extract only the whitelisted column values for each device
+                values = [tuple(device.get(col) for col in insert_cols) for device in new_devices]
 
                 mylog('verbose', [f'[{pluginName}] Inserting Devices SQL   : "{sql}"'])
                 mylog('verbose', [f'[{pluginName}] Inserting Devices VALUES: "{values}"'])

test/db_test_helpers.py

@@ -5,7 +5,7 @@
 
     import sys, os
     sys.path.insert(0, os.path.join(os.path.dirname(__file__), ".."))
-    from db_test_helpers import make_db, insert_device, minutes_ago, DummyDB, down_event_macs
+    from db_test_helpers import make_db, insert_device, minutes_ago, DummyDB, down_event_macs, make_device_dict, sync_insert_devices
 """
 
 import sqlite3
@@ -202,6 +202,125 @@ def insert_device(
     )
 
 
+def make_device_dict(mac: str = "aa:bb:cc:dd:ee:ff", **overrides) -> dict:
+    """
+    Return a fully-populated Devices row dict with safe defaults.
+
+    Mirrors every column in CREATE_DEVICES so callers can be inserted
+    directly via sync_insert_devices() or similar helpers.  Pass keyword
+    arguments to override any individual field.
+
+    Computed/view-only columns (devStatus, devIsSleeping, devFlapping,
+    rowid, …) are intentionally absent — tests that need to verify they are
+    dropped should add them after calling this function.
+    """
+    base = {
+        "devMac":                 mac,
+        "devName":                "Test Device",
+        "devOwner":               "",
+        "devType":                "",
+        "devVendor":              "Acme",
+        "devFavorite":            0,
+        "devGroup":               "",
+        "devComments":            "",
+        "devFirstConnection":     "2024-01-01 00:00:00",
+        "devLastConnection":      "2024-01-02 00:00:00",
+        "devLastIP":              "192.168.1.10",
+        "devPrimaryIPv4":         "192.168.1.10",
+        "devPrimaryIPv6":         "",
+        "devVlan":                "",
+        "devForceStatus":         "",
+        "devStaticIP":            "",
+        "devScan":                1,
+        "devLogEvents":           1,
+        "devAlertEvents":         1,
+        "devAlertDown":           1,
+        "devCanSleep":            0,
+        "devSkipRepeated":        0,
+        "devLastNotification":    "",
+        "devPresentLastScan":     1,
+        "devIsNew":               0,
+        "devLocation":            "",
+        "devIsArchived":          0,
+        "devParentMAC":           "",
+        "devParentPort":          "",
+        "devIcon":                "",
+        "devGUID":                "test-guid-1",
+        "devSite":                "",
+        "devSSID":                "",
+        "devSyncHubNode":         "node1",
+        "devSourcePlugin":        "",
+        "devCustomProps":         "",
+        "devFQDN":                "",
+        "devParentRelType":       "",
+        "devReqNicsOnline":       0,
+        "devMacSource":           "",
+        "devNameSource":          "",
+        "devFQDNSource":          "",
+        "devLastIPSource":        "",
+        "devVendorSource":        "",
+        "devSSIDSource":          "",
+        "devParentMACSource":     "",
+        "devParentPortSource":    "",
+        "devParentRelTypeSource": "",
+        "devVlanSource":          "",
+    }
+    base.update(overrides)
+    return base
+
+
+# ---------------------------------------------------------------------------
+# Sync insert helper (shared by test/plugins/test_sync_insert.py and
+# test/plugins/test_sync_protocol.py — mirrors sync.py's insert block)
+# ---------------------------------------------------------------------------
+
+def sync_insert_devices(
+    conn: sqlite3.Connection,
+    device_data: list,
+    existing_macs: set | None = None,
+) -> int:
+    """
+    Schema-aware device INSERT mirroring sync.py's Mode-3 insert block.
+
+    Parameters
+    ----------
+    conn:
+        In-memory (or real) SQLite connection with a Devices table.
+    device_data:
+        List of device dicts as received from table_devices.json or a node log.
+    existing_macs:
+        Set of MAC addresses already present in Devices.  Rows whose devMac is
+        in this set are skipped.  Pass ``None`` (default) to insert everything.
+
+    Returns the number of rows actually inserted.
+    """
+    if not device_data:
+        return 0
+
+    cursor = conn.cursor()
+
+    candidates = (
+        [d for d in device_data if d["devMac"] not in existing_macs]
+        if existing_macs is not None
+        else list(device_data)
+    )
+
+    if not candidates:
+        return 0
+
+    cursor.execute("PRAGMA table_info(Devices)")
+    db_columns = {row[1] for row in cursor.fetchall()}
+
+    insert_cols = [k for k in candidates[0].keys() if k in db_columns]
+    columns = ", ".join(insert_cols)
+    placeholders = ", ".join("?" for _ in insert_cols)
+    sql = f"INSERT INTO Devices ({columns}) VALUES ({placeholders})"
+    values = [tuple(d.get(col) for col in insert_cols) for d in candidates]
+    cursor.executemany(sql, values)
+    conn.commit()
+    return len(values)
+
+
 # ---------------------------------------------------------------------------
 # Assertion helpers
 # ---------------------------------------------------------------------------