router: add conditional support for Logs #11055
Conversation
Signed-off-by: Eduardo Silva <eduardo@chronosphere.io>
Signed-off-by: Eduardo Silva <eduardo@chronosphere.io>
Signed-off-by: Eduardo Silva <eduardo@chronosphere.io>
Signed-off-by: Eduardo Silva <eduardo@chronosphere.io>
Signed-off-by: Eduardo Silva <eduardo@chronosphere.io>
|
Note Other AI code review bot(s) detectedCodeRabbit has detected other AI code review bot(s) in this pull request and will avoid duplicating their findings in the review comments. This may lead to a less comprehensive review. WalkthroughAdds a per-chunk router context and threads it through condition compilation/evaluation and routing, migrates list APIs from mk_list to cfl_list, implements per-route payload splitting/encoding for logs, exposes a chunk real-size accessor, and updates tests for the new behavior. Changes
Sequence Diagram(s)sequenceDiagram
autonumber
participant Task as Task/Create
participant Router as Router
participant Context as RouterChunkContext
participant Cond as ConditionCompiler
participant Eval as ConditionEvaluator
participant Input as InputLog/Encoder
Task->>Router: create task (direct routes)
Router->>Context: flb_router_chunk_context_init()
Context-->>Router: context ready
loop each path
Router->>Cond: flb_router_route_get_condition(route)
alt compiled exists
Cond-->>Router: compiled condition
else
Cond->>Cond: route_condition_get_compiled()
Cond-->>Router: compiled or failure
end
Router->>Eval: flb_router_path_should_route(chunk, Context, path)
Eval->>Context: flb_router_chunk_context_prepare_logs() -- (if logs)
Context->>Input: encode/read records
Input-->>Eval: records iterator
Eval->>Cond: evaluate per-record against compiled condition
alt match
Eval-->>Router: accept path
else
Eval-->>Router: reject path
end
end
alt any accepted
Router->>Task: create task with filtered routes
else none accepted
Router->>Context: flb_router_chunk_context_destroy()
Router->>Task: drop task
end
Router->>Context: flb_router_chunk_context_destroy()
Context-->>Router: cleaned
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Files/areas needing extra attention:
Possibly related PRs
Suggested reviewers
Poem
Pre-merge checks and finishing touches❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✨ Finishing touches
🧪 Generate unit tests (beta)
📜 Recent review detailsConfiguration used: CodeRabbit UI Review profile: CHILL Plan: Pro 📒 Files selected for processing (2)
🚧 Files skipped from review as they are similar to previous changes (1)
🧰 Additional context used🧠 Learnings (8)📚 Learning: 2025-08-31T12:46:11.940ZApplied to files:
📚 Learning: 2025-08-29T06:24:55.855ZApplied to files:
📚 Learning: 2025-08-29T06:25:27.250ZApplied to files:
📚 Learning: 2025-08-29T06:25:02.561ZApplied to files:
📚 Learning: 2025-08-29T06:24:26.170ZApplied to files:
📚 Learning: 2025-08-29T06:25:27.250ZApplied to files:
📚 Learning: 2025-08-29T06:24:44.797ZApplied to files:
📚 Learning: 2025-08-31T12:46:11.940ZApplied to files:
🧬 Code graph analysis (1)src/flb_router_config.c (5)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (30)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| } | ||
|
|
||
| if (flb_router_connect_direct(input_ins, output_ins) == 0) { | ||
| struct flb_router_path *path; | ||
|
|
||
| path = mk_list_entry_last(&input_ins->routes_direct, | ||
| struct flb_router_path, _head); | ||
| path->route = route; |
There was a problem hiding this comment.
The new conditional router connects direct paths in a loop, but the early input_has_direct_route() check drops every route after the first one that targets the same output instance. With conditional routing it is valid to define multiple routes (e.g. an error rule and a default rule) that all forward to the same output but with different conditions or processors. Because only the first route is connected, later routes are silently ignored and chunks that don’t satisfy the first route’s condition are dropped in flb_task_create (no direct routes match and the task is destroyed). This makes common configurations unusable whenever two routes share the same output alias. The router should allow multiple direct routes per output or include the route identity in the deduplication.
Useful? React with 👍 / 👎.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 5
🧹 Nitpick comments (7)
src/flb_router.c (1)
157-159: Route pointer init: good; consider zero-initializing the struct.Setting p->route = NULL prevents dangling reads. Optionally switch to flb_calloc and drop the explicit assignment for future fields.
Apply (optional):
- p = flb_malloc(sizeof(struct flb_router_path)); + p = flb_calloc(1, sizeof(struct flb_router_path)); @@ - p->route = NULL;Also applies to: 175-177
tests/internal/router_config.c (3)
32-86: Helper looks correct; minor DRY improvement.build_log_chunk is fine. You can reduce repeated error cleanup with a single exit path to improve readability.
88-129: Destructor mirrors production; avoid duplication long‑term.free_route_condition fully cleans nested allocations (incl. compiled). Consider exposing a small public/test helper to avoid divergence from route_condition_destroy.
Would you prefer a tiny test-only header that reuses the production destroy routine?
1127-1203: path_should_route test: LGTM; consider also default-route case.Add a small case asserting flb_router_path_should_route(..., path with route==NULL) returns FLB_TRUE.
include/fluent-bit/flb_router.h (1)
29-30: New dependency include: OK; consider forward-decl.If feasible, forward-declare struct flb_condition and enum to avoid tight header coupling to flb_conditionals.h.
src/flb_router_config.c (2)
548-550: Rule parsing for array values: LGTM; add early operator validation.Array handling and cleanup paths are correct. Consider validating rule->op against the supported set (eq, neq, gt, lt, gte, lte, regex, not_regex, in, not_in) here to fail fast and give clearer config errors.
Example:
@@ - rule->op = copy_from_cfl_sds(op_var->data.as_string); + rule->op = copy_from_cfl_sds(op_var->data.as_string); if (!rule->op) { ... } + /* optional: early operator validation */ + if (!is_valid_rule_op(rule->op)) { + flb_sds_destroy(rule->op); + flb_sds_destroy(rule->field); + flb_free(rule); + return NULL; + }Implement is_valid_rule_op() locally or reuse existing helpers if available.
Also applies to: 564-569, 571-610
1310-1315: Assigning path->route: verify lifetime; guard access.Linking path->route to a route is correct, but it assumes these route objects live at least as long as input.routes_direct. If routes are later destroyed (e.g., via flb_router_routes_destroy) while paths persist, this becomes a dangling pointer.
- Confirm lifecycle: when and where are input_routes freed relative to path usage?
- If not guaranteed, either:
- Retain routes until paths are torn down, or
- Store only condition/compiled state on the path, or
- Null path->route on teardown to make flb_router_path_should_route() return FLB_TRUE (safe default).
Additionally, consider a defensive NULL-check on path retrieved via mk_list_entry_last to avoid undefined behavior if list ops change in the future.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (6)
include/fluent-bit/flb_router.h(3 hunks)src/flb_router.c(2 hunks)src/flb_router_condition.c(4 hunks)src/flb_router_config.c(7 hunks)src/flb_task.c(3 hunks)tests/internal/router_config.c(4 hunks)
🧰 Additional context used
🧬 Code graph analysis (5)
tests/internal/router_config.c (4)
src/flb_log_event_encoder.c (5)
flb_log_event_encoder_init(42-74)flb_log_event_encoder_begin_record(246-254)flb_log_event_encoder_destroy(99-116)flb_log_event_encoder_set_current_timestamp(289-292)flb_log_event_encoder_commit_record(256-274)src/flb_sds.c (2)
flb_sds_destroy(389-399)flb_sds_create(78-90)include/fluent-bit/flb_mem.h (2)
flb_free(126-128)flb_calloc(84-96)src/flb_router_condition.c (2)
flb_condition_eval_logs(54-119)flb_router_path_should_route(179-191)
src/flb_task.c (3)
src/flb_router_condition.c (1)
flb_router_path_should_route(179-191)include/fluent-bit/flb_mem.h (1)
flb_calloc(84-96)src/flb_routes_mask.c (1)
flb_routes_mask_get_bit(125-140)
src/flb_router_condition.c (4)
src/flb_log_event_decoder.c (4)
flb_log_event_decoder_init(99-116)flb_log_event_decoder_read_groups(85-97)flb_log_event_decoder_next(310-406)flb_log_event_decoder_destroy(147-179)src/flb_mp.c (1)
flb_mp_object_to_cfl(899-934)src/flb_conditionals.c (4)
flb_condition_evaluate(359-406)flb_condition_create(219-233)flb_condition_destroy(257-274)flb_condition_add_rule(235-255)lib/cfl/src/cfl_object.c (1)
cfl_object_destroy(95-106)
include/fluent-bit/flb_router.h (1)
src/flb_router_condition.c (1)
flb_router_path_should_route(179-191)
src/flb_router_config.c (5)
src/flb_sds.c (1)
flb_sds_destroy(389-399)include/fluent-bit/flb_mem.h (2)
flb_free(126-128)flb_calloc(84-96)src/flb_conditionals.c (1)
flb_condition_destroy(257-274)lib/cfl/include/cfl/cfl_array.h (2)
cfl_array_size(46-49)cfl_array_fetch_by_index(36-44)lib/cfl/src/cfl_kvlist.c (1)
cfl_kvlist_fetch(418-421)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (30)
- GitHub Check: pr-windows-build / call-build-windows-package (Windows 64bit, x64, x64-windows-static, 3.31.6)
- GitHub Check: pr-windows-build / call-build-windows-package (Windows 32bit, x86, x86-windows-static, 3.31.6)
- GitHub Check: pr-windows-build / call-build-windows-package (Windows 64bit (Arm64), amd64_arm64, -DCMAKE_SYSTEM_NAME=Windows -DCMA...
- GitHub Check: run-ubuntu-unit-tests (-DFLB_ARROW=On, 3.31.6, gcc, g++)
- GitHub Check: run-ubuntu-unit-tests (-DFLB_SIMD=On, 3.31.6, clang, clang++)
- GitHub Check: run-ubuntu-unit-tests (-DSANITIZE_ADDRESS=On, 3.31.6, clang, clang++)
- GitHub Check: run-ubuntu-unit-tests (-DFLB_SIMD=Off, 3.31.6, gcc, g++)
- GitHub Check: run-ubuntu-unit-tests (-DFLB_SIMD=On, 3.31.6, gcc, g++)
- GitHub Check: run-ubuntu-unit-tests (-DFLB_SANITIZE_THREAD=On, 3.31.6, gcc, g++)
- GitHub Check: run-ubuntu-unit-tests (-DFLB_SANITIZE_THREAD=On, 3.31.6, clang, clang++)
- GitHub Check: run-ubuntu-unit-tests (-DFLB_SIMD=Off, 3.31.6, clang, clang++)
- GitHub Check: run-ubuntu-unit-tests (-DSANITIZE_ADDRESS=On, 3.31.6, gcc, g++)
- GitHub Check: run-ubuntu-unit-tests (-DFLB_COVERAGE=On, 3.31.6, gcc, g++)
- GitHub Check: run-ubuntu-unit-tests (-DSANITIZE_UNDEFINED=On, 3.31.6, gcc, g++)
- GitHub Check: run-ubuntu-unit-tests (-DFLB_SANITIZE_MEMORY=On, 3.31.6, gcc, g++)
- GitHub Check: run-ubuntu-unit-tests (-DFLB_JEMALLOC=On, 3.31.6, clang, clang++)
- GitHub Check: run-ubuntu-unit-tests (-DFLB_SANITIZE_MEMORY=On, 3.31.6, clang, clang++)
- GitHub Check: run-ubuntu-unit-tests (-DFLB_JEMALLOC=Off, 3.31.6, clang, clang++)
- GitHub Check: run-ubuntu-unit-tests (-DSANITIZE_UNDEFINED=On, 3.31.6, clang, clang++)
- GitHub Check: run-ubuntu-unit-tests (-DFLB_JEMALLOC=On, 3.31.6, gcc, g++)
- GitHub Check: run-ubuntu-unit-tests (-DFLB_SMALL=On, 3.31.6, gcc, g++)
- GitHub Check: run-ubuntu-unit-tests (-DFLB_JEMALLOC=Off, 3.31.6, gcc, g++)
- GitHub Check: run-ubuntu-unit-tests (-DFLB_SMALL=On, 3.31.6, clang, clang++)
- GitHub Check: PR - fuzzing test
- GitHub Check: pr-compile-system-libs (-DFLB_PREFER_SYSTEM_LIBS=On, 3.31.6, clang, clang++, ubuntu-24.04, clang-14)
- GitHub Check: pr-compile-system-libs (-DFLB_PREFER_SYSTEM_LIBS=On, 3.31.6, gcc, g++, ubuntu-24.04, clang-14)
- GitHub Check: pr-compile-system-libs (-DFLB_PREFER_SYSTEM_LIBS=On, 3.31.6, gcc, g++, ubuntu-22.04, clang-12)
- GitHub Check: pr-compile-system-libs (-DFLB_PREFER_SYSTEM_LIBS=On, 3.31.6, clang, clang++, ubuntu-22.04, clang-12)
- GitHub Check: pr-compile-without-cxx (3.31.6)
- GitHub Check: pr-compile-centos-7
🔇 Additional comments (20)
tests/internal/router_config.c (3)
954-1026: Logs condition “eq” test: LGTM.Covers positive/negative paths with proper encoder lifecycle.
1028-1125: “in” operator test: LGTM.Exercises multi-value path correctly.
1214-1217: Test registrations: LGTM.src/flb_task.c (3)
419-454: Direct-route gating: LGTM.Correctly filters direct paths and initializes routes; drops empty-direct tasks to save work.
Please confirm that flb_router_path->route remains valid for the engine lifetime (or until the path is removed). Otherwise flb_router_path_should_route() could dereference a stale pointer if routes are freed (e.g., via flb_router_routes_destroy) while paths persist.
444-450: Early drop path: OK.Cleanup is consistent (NULLing event_chunk->data before destroy). No further action.
467-472: Minor consistency nit.Both code paths now use flb_calloc for struct flb_task_route; good consistency.
include/fluent-bit/flb_router.h (4)
34-37: route back-reference on path: LGTM.Enables content-aware routing decisions at task time.
75-82: Multi-value rule support: LGTM.values/values_count fields are appropriate for in/not_in.
84-90: Condition fields for op/compiled state: LGTM.Matches compiled-eval flow.
146-148: New API declaration: LGTM.src/flb_router_config.c (3)
32-32: Conditionals include: LGTM.
294-297: Compiled-condition cleanup: LGTM.Ensures compiled objects are destroyed with the route condition.
647-671: Condition op parsing: LGTM.Default AND, accepts “and”/“or”, rejects others.
src/flb_router_condition.c (7)
152-164: Default-route semantics vs signals mask: confirm orderCurrently, signals filtering runs before default-route short‑circuiting. If the intent is “default routes bypass rule evaluation but still honor the route’s signals,” this is correct. If default routes should catch any signal, move the is_default check before the signals mask.
Option if default should ignore signals:
- signal = flb_router_signal_from_chunk(chunk); - if (signal == 0) { - return FLB_FALSE; - } - - if ((route->signals != 0) && (route->signals != FLB_ROUTER_SIGNAL_ANY) && ((route->signals & signal) == 0)) { - return FLB_FALSE; - } - - if (route->condition->is_default) { + if (route->condition->is_default) { return FLB_TRUE; } + + signal = flb_router_signal_from_chunk(chunk); + if (signal == 0) { + return FLB_FALSE; + } + if ((route->signals != 0) && (route->signals != FLB_ROUTER_SIGNAL_ANY) && ((route->signals & signal) == 0)) { + return FLB_FALSE; + }Also applies to: 157-159
179-191: Path-level routing gate looks goodNull path → false, path without route → true, otherwise delegate to route evaluation. Matches the intended behavior.
193-235: Operator parsing is fine; ensure portability is covered by headersThe mapping is clear and case‑insensitive. The earlier header fix covers strcasecmp. No further changes required.
237-254: Numeric parsing is strict; OK once headers are addedThe ERANGE and trailing‑garbage checks are solid. After adding stdlib.h/errno.h, this is good.
303-309: Numeric value lifetime: confirm add_rule copies the valueYou pass &numeric_value (stack address) to flb_condition_add_rule(). If add_rule/rule_create does not deep‑copy the double, this becomes a dangling pointer.
If it does not copy, apply:
- ret = flb_condition_add_rule(compiled, rule->field, op, - &numeric_value, 1, RECORD_CONTEXT_BODY); + double *nv = flb_malloc(sizeof(*nv)); + if (nv == NULL) { + cfl_errno(); + flb_condition_destroy(compiled); + return NULL; + } + *nv = numeric_value; + ret = flb_condition_add_rule(compiled, rule->field, op, + nv, 1, RECORD_CONTEXT_BODY);
316-321: IN/NOT_IN values ownership: confirm deep copyrule->values is passed directly. Please confirm flb_condition_add_rule/rule_create duplicates the array and its elements. Otherwise, the route’s config buffers must outlive condition use.
360-375: Record cleanup helper looks correctProperly destroys CFL objects and nulls pointers. LGTM.
| #include <fluent-bit/flb_mem.h> | ||
| #include <fluent-bit/flb_log.h> | ||
| #include <fluent-bit/flb_router.h> | ||
| #include <fluent-bit/flb_conditionals.h> | ||
| #include <fluent-bit/flb_log_event_decoder.h> | ||
| #include <fluent-bit/flb_mp.h> | ||
| #include <fluent-bit/flb_mp_chunk.h> | ||
|
|
There was a problem hiding this comment.
Add missing standard headers for strcasecmp/strtod/errno
This file uses strcasecmp, strtod and errno but doesn’t include their headers. Some platforms won’t get these via transitive includes, causing build failures.
Apply:
#include <fluent-bit/flb_mp_chunk.h>
+#include <strings.h> /* strcasecmp */
+#include <stdlib.h> /* strtod */
+#include <errno.h> /* errno, ERANGE */📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| #include <fluent-bit/flb_mem.h> | |
| #include <fluent-bit/flb_log.h> | |
| #include <fluent-bit/flb_router.h> | |
| #include <fluent-bit/flb_conditionals.h> | |
| #include <fluent-bit/flb_log_event_decoder.h> | |
| #include <fluent-bit/flb_mp.h> | |
| #include <fluent-bit/flb_mp_chunk.h> | |
| #include <fluent-bit/flb_mem.h> | |
| #include <fluent-bit/flb_log.h> | |
| #include <fluent-bit/flb_router.h> | |
| #include <fluent-bit/flb_conditionals.h> | |
| #include <fluent-bit/flb_log_event_decoder.h> | |
| #include <fluent-bit/flb_mp.h> | |
| #include <fluent-bit/flb_mp_chunk.h> | |
| #include <strings.h> /* strcasecmp */ | |
| #include <stdlib.h> /* strtod */ | |
| #include <errno.h> /* errno, ERANGE */ |
🤖 Prompt for AI Agents
In src/flb_router_condition.c around lines 20 to 27, the file uses strcasecmp,
strtod and errno but lacks their standard headers; add the missing includes at
the top with the other headers: include <string.h> for strcasecmp, <stdlib.h>
for strtod, and <errno.h> for errno so builds on platforms without transitive
includes succeed.
src/flb_router_condition.c
Outdated
| flb_log_event_decoder_read_groups(&decoder, FLB_TRUE); | ||
|
|
There was a problem hiding this comment.
Avoid reading group meta records; skip them and check the return
Setting read_groups = TRUE surfaces group open/close meta records which can be (incorrectly) evaluated by conditions. Decoder already propagates group context to normal records without needing read_groups = TRUE.
- flb_log_event_decoder_read_groups(&decoder, FLB_TRUE);
+ if (flb_log_event_decoder_read_groups(&decoder, FLB_FALSE) != 0) {
+ flb_log_event_decoder_destroy(&decoder);
+ return FLB_FALSE;
+ }📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| flb_log_event_decoder_read_groups(&decoder, FLB_TRUE); | |
| if (flb_log_event_decoder_read_groups(&decoder, FLB_FALSE) != 0) { | |
| flb_log_event_decoder_destroy(&decoder); | |
| return FLB_FALSE; | |
| } |
🤖 Prompt for AI Agents
In src/flb_router_condition.c around lines 85-86, the call to
flb_log_event_decoder_read_groups(&decoder, FLB_TRUE) causes group open/close
meta records to be surfaced and wrongly evaluated; change this to disable
reading group meta records (use FLB_FALSE / remove the read_groups enable) so
the decoder only propagates group context to normal records, and check the
function's return value for errors—handle a non-success return by logging an
error and returning/propagating failure appropriately.
There was a problem hiding this comment.
Actionable comments posted: 3
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (3)
src/flb_router_config.c (2)
503-510: Fix processor cleanup: wrong destructor used on properties list (type confusion/UAF).On add_processor_properties() failure you call route_processors_destroy(&processor->properties), but that function expects a list of struct flb_route_processor, not struct flb_route_processor_property. This can corrupt memory. Clean up the properties list explicitly instead.
@@ - if (parse_processors(processors_var, &route->processors, config) != 0) { - if (route->condition) { - route_condition_destroy(route->condition); - } - flb_sds_destroy(route->name); - flb_free(route); - return -1; - } + if (parse_processors(processors_var, &route->processors, config) != 0) { + if (route->condition) { + route_condition_destroy(route->condition); + } + flb_sds_destroy(route->name); + flb_free(route); + return -1; + } @@ - if (add_processor_properties(processor, kvlist) != 0) { - route_processors_destroy(&processor->properties); - if (processor->name) { - flb_sds_destroy(processor->name); - } - flb_free(processor); - return -1; - } + if (add_processor_properties(processor, kvlist) != 0) { + /* Clean up properties list correctly (list of properties, not processors) */ + struct cfl_list *p_head; + struct cfl_list *p_tmp; + struct flb_route_processor_property *prop; + cfl_list_foreach_safe(p_head, p_tmp, &processor->properties) { + prop = cfl_list_entry(p_head, struct flb_route_processor_property, _head); + cfl_list_del(&prop->_head); + if (prop->key) { flb_sds_destroy(prop->key); } + if (prop->value) { flb_sds_destroy(prop->value); } + flb_free(prop); + } + if (processor->name) { + flb_sds_destroy(processor->name); + } + flb_free(processor); + return -1; + }Optionally, introduce a static helper route_processor_destroy() to reuse in other paths.
1231-1248: Signal mask bug: ANY check matches every non-zero mask.(signals & FLB_ROUTER_SIGNAL_ANY) is true for any non-zero signals since ANY is a bitwise union. Outputs will be accepted even when they don’t support requested signals.
- /* Handle ANY signal - if ANY is present in the bitmask, allow all signals */ - if (signals == 0 || (signals & FLB_ROUTER_SIGNAL_ANY)) { + /* Handle ANY signal: no specific signals or explicit 'any' */ + if (signals == 0 || signals == FLB_ROUTER_SIGNAL_ANY) { return FLB_TRUE; }src/flb_mp.c (1)
1556-1560: Avoid double list removal in record destroy.flb_router_condition.c already cfl_list_del() before calling flb_mp_chunk_cobj_record_destroy(NULL, rec). Your destructor always calls cfl_list_del(), causing double-del and list corruption.
- cfl_list_del(&record->_head); + if (chunk_cobj) { + cfl_list_del(&record->_head); + } flb_free(record);Alternatively, ensure callers never cfl_list_del() and let destroy handle it, but the guarded approach is safer across call sites.
♻️ Duplicate comments (4)
src/flb_router_config.c (1)
1205-1223: Do not dedup direct routes by output only; allow multiple routes per output.Current input_has_direct_route() drops later routes to the same output, breaking common configs (e.g., error and default routes pointing to one output). This issue was already raised earlier and still applies. Include route identity in deduplication.
@@ -static int input_has_direct_route(struct flb_input_instance *in, - struct flb_output_instance *out) +static int input_has_direct_route(struct flb_input_instance *in, + struct flb_output_instance *out, + struct flb_route *route) { @@ - if (path->ins == out) { - return FLB_TRUE; - } + if (path->ins == out) { + /* Allow multiple paths to the same output as long as they are for different routes */ + if (path->route == NULL || path->route == route) { + return FLB_TRUE; + } + } @@ - if (input_has_direct_route(input_ins, output_ins)) { + if (input_has_direct_route(input_ins, output_ins, route)) { continue; } @@ if (flb_router_connect_direct(input_ins, output_ins) == 0) { struct flb_router_path *path; path = mk_list_entry_last(&input_ins->routes_direct, struct flb_router_path, _head); path->route = route;This preserves dedup for exact same route/out pair while enabling multiple conditional/default routes to share the output.
Also applies to: 1314-1324
src/flb_router_condition.c (3)
20-27: Add missing standard headers (build break across platforms).This file uses strcasecmp, strtod, errno, and strlen but doesn’t include their headers. Add them to avoid implicit declarations and portability issues.
#include <fluent-bit/flb_mp_chunk.h> +/* std headers required by this file */ +#include <string.h> /* strlen */ +#include <strings.h> /* strcasecmp */ +#include <stdlib.h> /* strtod */ +#include <errno.h> /* errno, ERANGE */
102-109: Don’t surface group meta records; disable read_groups and check return.Evaluating group open/close meta records can cause false matches. Disable them and handle errors.
- flb_log_event_decoder_read_groups(context->log_decoder, FLB_TRUE); + if (flb_log_event_decoder_read_groups(context->log_decoder, FLB_FALSE) != 0) { + flb_router_chunk_context_reset(context); + return -1; + }
630-653: Racy lazy compilation; synchronize or precompile to avoid data races.route_condition_get_compiled() is unsynchronized; multiple workers can compile and assign concurrently, leading to leaks or corruption. Precompile during config load (preferred), or guard lazy init with a lock.
Minimal lock around lazy compile:
+#include <pthread.h> +static pthread_mutex_t g_route_cond_compile_lock = PTHREAD_MUTEX_INITIALIZER; ... static struct flb_condition *route_condition_get_compiled(struct flb_route_condition *condition) { if (!condition) { return NULL; } @@ - condition->compiled = route_condition_compile(condition); - if (!condition->compiled) { - condition->compiled_status = FLB_ROUTE_CONDITION_COMPILED_FAILURE; - return NULL; - } - - condition->compiled_status = FLB_ROUTE_CONDITION_COMPILED_SUCCESS; - return condition->compiled; + pthread_mutex_lock(&g_route_cond_compile_lock); + if (condition->compiled_status == FLB_ROUTE_CONDITION_COMPILED_SUCCESS && condition->compiled) { + pthread_mutex_unlock(&g_route_cond_compile_lock); + return condition->compiled; + } + condition->compiled = route_condition_compile(condition); + if (!condition->compiled) { + condition->compiled_status = FLB_ROUTE_CONDITION_COMPILED_FAILURE; + pthread_mutex_unlock(&g_route_cond_compile_lock); + return NULL; + } + condition->compiled_status = FLB_ROUTE_CONDITION_COMPILED_SUCCESS; + pthread_mutex_unlock(&g_route_cond_compile_lock); + return condition->compiled; }Alternative: compile eagerly when parsing the route and drop lazy compile here.
🧹 Nitpick comments (6)
src/flb_task.c (1)
398-406: Initialize router context lazily only when needed (direct routes path).Small perf cleanup: avoid creating/destroying router_context when there are no direct routes or for non-log signals.
@@ - if (flb_router_chunk_context_init(&router_context) != 0) { - flb_error("[task] failed to initialize router chunk context"); - flb_event_chunk_destroy(evc); - flb_free(task); - *err = FLB_TRUE; - return NULL; - } - router_context_initialized = FLB_TRUE; @@ - if (mk_list_size(&i_ins->routes_direct) > 0) { + if (mk_list_size(&i_ins->routes_direct) > 0) { + if (flb_router_chunk_context_init(&router_context) != 0) { + flb_error("[task] failed to initialize router chunk context"); + flb_event_chunk_destroy(evc); + flb_free(task); + *err = FLB_TRUE; + return NULL; + } + router_context_initialized = FLB_TRUE; @@ - if (router_context_initialized) { + if (router_context_initialized) { flb_router_chunk_context_destroy(&router_context); router_context_initialized = FLB_FALSE; }Also applies to: 429-479, 512-524
src/flb_router_condition.c (5)
356-384: Prefer skipping only the bad record instead of aborting the whole route chunk.On a single record’s encode/convert failure you return -1, dropping all prior matches. Safer UX is to log and continue with remaining records.
If a conversion/encode fails:
- Reset the in-progress record (e.g., flb_log_event_encoder_reset_record if available) and continue; or
- Guard converts with “if (…) { … } else { continue; }” and only commit successful ones.
Let me know if you want a concrete patch once you confirm the preferred reset API.
291-294: Use flb_sds_len for tag length (avoid strlen on SDS).Minor perf/clarity win and avoids surprises if tags ever contain embedded NULs.
- new_chunk = flb_event_chunk_create(chunk->type, match_count, - chunk->tag, strlen(chunk->tag), + new_chunk = flb_event_chunk_create(chunk->type, match_count, + chunk->tag, flb_sds_len(chunk->tag), out_buf, out_size);Add include:
#include <fluent-bit/flb_mp_chunk.h> +#include <fluent-bit/flb_sds.h>
91-94: Treat empty log chunks as a no-op instead of an error.Returning -1 for empty data short-circuits routing; consider returning 0 so evaluation proceeds gracefully.
- if (!context || !chunk) { + if (!context || !chunk) { return -1; } ... - if (!chunk->data || chunk->size == 0) { - return -1; - } + if (!chunk->data || chunk->size == 0) { + return 0; /* nothing to prepare */ + }
471-486: Avoid unnecessary decode when path has no route.If path->route is NULL you return TRUE; move that check before preparing the logs context to save work.
- if (chunk && chunk->type == FLB_EVENT_TYPE_LOGS) { - if (!context) { - return FLB_FALSE; - } - if (flb_router_chunk_context_prepare_logs(context, chunk) != 0) { - return FLB_FALSE; - } - } - - if (!path->route) { - return FLB_TRUE; - } + if (!path->route) { + return FLB_TRUE; + } + + if (chunk && chunk->type == FLB_EVENT_TYPE_LOGS) { + if (!context) { + return FLB_FALSE; + } + if (flb_router_chunk_context_prepare_logs(context, chunk) != 0) { + return FLB_FALSE; + } + }
317-323: Unused parameters in flb_router_create_chunk_from_records.tag and event_type aren’t used in this function. Either remove them or document why they’re needed.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (8)
include/fluent-bit/flb_mp_chunk.h(1 hunks)include/fluent-bit/flb_router.h(4 hunks)src/flb_event.c(0 hunks)src/flb_mp.c(1 hunks)src/flb_router_condition.c(3 hunks)src/flb_router_config.c(9 hunks)src/flb_task.c(6 hunks)tests/internal/router_config.c(5 hunks)
💤 Files with no reviewable changes (1)
- src/flb_event.c
🚧 Files skipped from review as they are similar to previous changes (1)
- tests/internal/router_config.c
🧰 Additional context used
🧬 Code graph analysis (6)
src/flb_task.c (4)
src/flb_router_condition.c (3)
flb_router_chunk_context_init(33-44)flb_router_path_should_route(463-486)flb_router_chunk_context_destroy(68-71)src/flb_event.c (1)
flb_event_chunk_destroy(69-79)include/fluent-bit/flb_mem.h (2)
flb_free(126-128)flb_calloc(84-96)src/flb_routes_mask.c (1)
flb_routes_mask_get_bit(125-140)
include/fluent-bit/flb_mp_chunk.h (1)
src/flb_mp.c (1)
flb_mp_chunk_record_dup(1438-1511)
src/flb_router_config.c (5)
src/flb_sds.c (1)
flb_sds_destroy(389-399)include/fluent-bit/flb_mem.h (2)
flb_free(126-128)flb_calloc(84-96)src/flb_conditionals.c (1)
flb_condition_destroy(257-274)lib/cfl/include/cfl/cfl_array.h (2)
cfl_array_size(46-49)cfl_array_fetch_by_index(36-44)lib/cfl/src/cfl_kvlist.c (1)
cfl_kvlist_fetch(418-421)
src/flb_router_condition.c (6)
src/flb_mp.c (5)
flb_mp_chunk_cobj_create(1065-1087)flb_mp_chunk_cobj_record_next(1243-1435)flb_mp_chunk_record_dup(1438-1511)flb_mp_chunk_cobj_record_destroy(1513-1560)flb_mp_cfl_to_msgpack(1017-1045)src/flb_log_event_decoder.c (4)
flb_log_event_decoder_destroy(147-179)flb_log_event_decoder_create(118-145)flb_log_event_decoder_read_groups(85-97)flb_log_event_decoder_reset(67-83)src/flb_log_event_encoder.c (5)
flb_log_event_encoder_destroy(99-116)flb_log_event_encoder_create(76-97)flb_log_event_encoder_begin_record(246-254)flb_log_event_encoder_set_timestamp(276-287)flb_log_event_encoder_commit_record(256-274)src/flb_conditionals.c (4)
flb_condition_evaluate(359-406)flb_condition_create(219-233)flb_condition_destroy(257-274)flb_condition_add_rule(235-255)src/flb_event.c (1)
flb_event_chunk_create(26-57)include/fluent-bit/flb_mem.h (1)
flb_free(126-128)
include/fluent-bit/flb_router.h (1)
src/flb_router_condition.c (12)
flb_router_chunk_context_init(33-44)flb_router_chunk_context_reset(46-66)flb_router_chunk_context_destroy(68-71)flb_router_chunk_context_prepare_logs(73-134)flb_route_condition_eval(422-461)flb_condition_eval_logs(156-195)flb_condition_eval_logs_per_record(198-244)flb_router_route_per_record(247-315)flb_router_create_chunk_from_records(318-398)flb_condition_eval_metrics(400-409)flb_condition_eval_traces(411-420)flb_router_path_should_route(463-486)
src/flb_mp.c (2)
include/fluent-bit/flb_mem.h (2)
flb_calloc(84-96)flb_free(126-128)lib/cfl/src/cfl_object.c (1)
cfl_object_destroy(95-106)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (27)
- GitHub Check: pr-windows-build / call-build-windows-package (Windows 64bit (Arm64), amd64_arm64, -DCMAKE_SYSTEM_NAME=Windows -DCMA...
- GitHub Check: pr-windows-build / call-build-windows-package (Windows 32bit, x86, x86-windows-static, 3.31.6)
- GitHub Check: pr-windows-build / call-build-windows-package (Windows 64bit, x64, x64-windows-static, 3.31.6)
- GitHub Check: run-ubuntu-unit-tests (-DFLB_SANITIZE_THREAD=On, 3.31.6, gcc, g++)
- GitHub Check: run-ubuntu-unit-tests (-DFLB_SANITIZE_THREAD=On, 3.31.6, clang, clang++)
- GitHub Check: run-ubuntu-unit-tests (-DFLB_SIMD=Off, 3.31.6, gcc, g++)
- GitHub Check: run-ubuntu-unit-tests (-DFLB_SIMD=On, 3.31.6, gcc, g++)
- GitHub Check: run-ubuntu-unit-tests (-DFLB_SIMD=Off, 3.31.6, clang, clang++)
- GitHub Check: run-ubuntu-unit-tests (-DFLB_SIMD=On, 3.31.6, clang, clang++)
- GitHub Check: run-ubuntu-unit-tests (-DFLB_ARROW=On, 3.31.6, gcc, g++)
- GitHub Check: run-ubuntu-unit-tests (-DFLB_SMALL=On, 3.31.6, gcc, g++)
- GitHub Check: run-ubuntu-unit-tests (-DFLB_JEMALLOC=Off, 3.31.6, clang, clang++)
- GitHub Check: run-ubuntu-unit-tests (-DFLB_SANITIZE_MEMORY=On, 3.31.6, gcc, g++)
- GitHub Check: run-ubuntu-unit-tests (-DSANITIZE_UNDEFINED=On, 3.31.6, gcc, g++)
- GitHub Check: run-ubuntu-unit-tests (-DFLB_SANITIZE_MEMORY=On, 3.31.6, clang, clang++)
- GitHub Check: run-ubuntu-unit-tests (-DSANITIZE_ADDRESS=On, 3.31.6, clang, clang++)
- GitHub Check: run-ubuntu-unit-tests (-DFLB_JEMALLOC=On, 3.31.6, gcc, g++)
- GitHub Check: run-ubuntu-unit-tests (-DSANITIZE_UNDEFINED=On, 3.31.6, clang, clang++)
- GitHub Check: run-ubuntu-unit-tests (-DFLB_SMALL=On, 3.31.6, clang, clang++)
- GitHub Check: run-ubuntu-unit-tests (-DFLB_JEMALLOC=On, 3.31.6, clang, clang++)
- GitHub Check: run-ubuntu-unit-tests (-DFLB_JEMALLOC=Off, 3.31.6, gcc, g++)
- GitHub Check: PR - fuzzing test
- GitHub Check: pr-compile-system-libs (-DFLB_PREFER_SYSTEM_LIBS=On, 3.31.6, clang, clang++, ubuntu-22.04, clang-12)
- GitHub Check: pr-compile-without-cxx (3.31.6)
- GitHub Check: pr-compile-system-libs (-DFLB_PREFER_SYSTEM_LIBS=On, 3.31.6, clang, clang++, ubuntu-24.04, clang-14)
- GitHub Check: pr-compile-system-libs (-DFLB_PREFER_SYSTEM_LIBS=On, 3.31.6, gcc, g++, ubuntu-24.04, clang-14)
- GitHub Check: pr-compile-centos-7
🔇 Additional comments (5)
src/flb_router_config.c (1)
280-289: LGTM on condition memory management and path->route association.
- Properly free rule->values and compiled condition in route_condition_destroy.
- Initialize rule->values/values_count.
- Validate/marshal array values safely.
- Default op=AND and op parsing are correct.
- Associate created path with route via path->route.
Ensure route_condition_get_compiled() destroys/refreshes compiled on config reloads to avoid stale pointers.
Also applies to: 294-297, 548-550, 564-611, 613-618, 647-650, 653-672, 1319-1324
include/fluent-bit/flb_router.h (1)
45-46: Header additions look consistent.
- route pointer on flb_router_path
- condition values/op/compiled fields
- per_record_routing and context APIs
Also applies to: 85-101, 120-128, 149-174, 180-183
include/fluent-bit/flb_mp_chunk.h (1)
63-64: New duplication API declaration is fine.Matches implementation intent for per-record routing. Ensure ownership semantics are documented (caller frees via flb_mp_chunk_cobj_record_destroy).
Document whether the duplicate's group objects are owned by the record (they should be).
src/flb_router_condition.c (2)
281-286: The NULL parameter is intentionally supported for standalone duplicated records.The implementation of
flb_mp_chunk_cobj_record_destroy()properly handles NULL as the first parameter. All dereferences ofchunk_cobjare guarded by conditional checks (lines 1519, 1523–1525). The actual record cleanup (lines 1537–1546) destroys the record's own object copies (cobj_metadata,cobj_record,cobj_group_metadata) independently ofchunk_cobj.Since
flb_mp_chunk_record_dup()creates independent record copies with duplicated object data (viacfl_object_dup()), these duplicates own their resources and do not depend onchunk_cobjfor cleanup. Passing NULL is correct for these standalone records and matches the existing usage pattern inplugins/processor_opentelemetry_envelope/otel_envelope.c:137.
598-604: Review comment is incorrect—no dangling pointer issue exists.The concern about
numeric_valuebeing a stack variable with a dangling pointer is unfounded. Inrule_create(src/flb_conditionals.c:128-131), the numeric value is handled via:rule->value.num_val = *(double *)value;This dereferences the pointer and copies the value into the
num_valunion member. The stack variable's contents are copied into the allocated rule structure—the pointer is never stored. Once the copy completes, the stack variable going out of scope causes no issue.This is safe memory management and requires no allocation or deallocation changes.
src/flb_mp.c
Outdated
| /* Duplicate a chunk record for per-record routing */ | ||
| struct flb_mp_chunk_record *flb_mp_chunk_record_dup(struct flb_mp_chunk_record *src) | ||
| { | ||
| struct flb_mp_chunk_record *dup; | ||
|
|
||
| if (!src) { | ||
| return NULL; | ||
| } | ||
|
|
||
| dup = flb_calloc(1, sizeof(struct flb_mp_chunk_record)); | ||
| if (!dup) { | ||
| return NULL; | ||
| } | ||
|
|
||
| /* Copy basic fields */ | ||
| dup->event = src->event; | ||
| dup->owns_group_metadata = FLB_FALSE; | ||
| dup->owns_group_attributes = FLB_FALSE; | ||
|
|
||
| /* Duplicate metadata if present */ | ||
| if (src->cobj_metadata) { | ||
| dup->cobj_metadata = cfl_object_dup(src->cobj_metadata); | ||
| if (!dup->cobj_metadata) { | ||
| flb_free(dup); | ||
| return NULL; | ||
| } | ||
| } | ||
|
|
||
| /* Duplicate record body if present */ | ||
| if (src->cobj_record) { | ||
| dup->cobj_record = cfl_object_dup(src->cobj_record); | ||
| if (!dup->cobj_record) { | ||
| if (dup->cobj_metadata) { | ||
| cfl_object_destroy(dup->cobj_metadata); | ||
| } | ||
| flb_free(dup); | ||
| return NULL; | ||
| } | ||
| } | ||
|
|
||
| /* Duplicate group metadata if present */ | ||
| if (src->cobj_group_metadata) { | ||
| dup->cobj_group_metadata = cfl_object_dup(src->cobj_group_metadata); | ||
| if (!dup->cobj_group_metadata) { | ||
| if (dup->cobj_metadata) { | ||
| cfl_object_destroy(dup->cobj_metadata); | ||
| } | ||
| if (dup->cobj_record) { | ||
| cfl_object_destroy(dup->cobj_record); | ||
| } | ||
| flb_free(dup); | ||
| return NULL; | ||
| } | ||
| } | ||
|
|
||
| /* Duplicate group attributes if present */ | ||
| if (src->cobj_group_attributes) { | ||
| dup->cobj_group_attributes = cfl_object_dup(src->cobj_group_attributes); | ||
| if (!dup->cobj_group_attributes) { | ||
| if (dup->cobj_metadata) { | ||
| cfl_object_destroy(dup->cobj_metadata); | ||
| } | ||
| if (dup->cobj_record) { | ||
| cfl_object_destroy(dup->cobj_record); | ||
| } | ||
| if (dup->cobj_group_metadata) { | ||
| cfl_object_destroy(dup->cobj_group_metadata); | ||
| } | ||
| flb_free(dup); | ||
| return NULL; | ||
| } | ||
| } | ||
|
|
||
| return dup; | ||
| } |
There was a problem hiding this comment.
Leak: duplicated group objects not freed (owns_group_ not set).*
flb_mp_chunk_record_dup() deep-copies cobj_group_metadata/attributes but leaves owns_group_metadata/owns_group_attributes = FLB_FALSE, so the destructor won’t free them.
@@
- dup->owns_group_metadata = FLB_FALSE;
- dup->owns_group_attributes = FLB_FALSE;
+ dup->owns_group_metadata = FLB_FALSE;
+ dup->owns_group_attributes = FLB_FALSE;
@@
if (src->cobj_group_metadata) {
dup->cobj_group_metadata = cfl_object_dup(src->cobj_group_metadata);
if (!dup->cobj_group_metadata) {
@@
}
+ dup->owns_group_metadata = FLB_TRUE;
}
@@
if (src->cobj_group_attributes) {
dup->cobj_group_attributes = cfl_object_dup(src->cobj_group_attributes);
if (!dup->cobj_group_attributes) {
@@
}
+ dup->owns_group_attributes = FLB_TRUE;
}📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| /* Duplicate a chunk record for per-record routing */ | |
| struct flb_mp_chunk_record *flb_mp_chunk_record_dup(struct flb_mp_chunk_record *src) | |
| { | |
| struct flb_mp_chunk_record *dup; | |
| if (!src) { | |
| return NULL; | |
| } | |
| dup = flb_calloc(1, sizeof(struct flb_mp_chunk_record)); | |
| if (!dup) { | |
| return NULL; | |
| } | |
| /* Copy basic fields */ | |
| dup->event = src->event; | |
| dup->owns_group_metadata = FLB_FALSE; | |
| dup->owns_group_attributes = FLB_FALSE; | |
| /* Duplicate metadata if present */ | |
| if (src->cobj_metadata) { | |
| dup->cobj_metadata = cfl_object_dup(src->cobj_metadata); | |
| if (!dup->cobj_metadata) { | |
| flb_free(dup); | |
| return NULL; | |
| } | |
| } | |
| /* Duplicate record body if present */ | |
| if (src->cobj_record) { | |
| dup->cobj_record = cfl_object_dup(src->cobj_record); | |
| if (!dup->cobj_record) { | |
| if (dup->cobj_metadata) { | |
| cfl_object_destroy(dup->cobj_metadata); | |
| } | |
| flb_free(dup); | |
| return NULL; | |
| } | |
| } | |
| /* Duplicate group metadata if present */ | |
| if (src->cobj_group_metadata) { | |
| dup->cobj_group_metadata = cfl_object_dup(src->cobj_group_metadata); | |
| if (!dup->cobj_group_metadata) { | |
| if (dup->cobj_metadata) { | |
| cfl_object_destroy(dup->cobj_metadata); | |
| } | |
| if (dup->cobj_record) { | |
| cfl_object_destroy(dup->cobj_record); | |
| } | |
| flb_free(dup); | |
| return NULL; | |
| } | |
| } | |
| /* Duplicate group attributes if present */ | |
| if (src->cobj_group_attributes) { | |
| dup->cobj_group_attributes = cfl_object_dup(src->cobj_group_attributes); | |
| if (!dup->cobj_group_attributes) { | |
| if (dup->cobj_metadata) { | |
| cfl_object_destroy(dup->cobj_metadata); | |
| } | |
| if (dup->cobj_record) { | |
| cfl_object_destroy(dup->cobj_record); | |
| } | |
| if (dup->cobj_group_metadata) { | |
| cfl_object_destroy(dup->cobj_group_metadata); | |
| } | |
| flb_free(dup); | |
| return NULL; | |
| } | |
| } | |
| return dup; | |
| } | |
| /* Duplicate a chunk record for per-record routing */ | |
| struct flb_mp_chunk_record *flb_mp_chunk_record_dup(struct flb_mp_chunk_record *src) | |
| { | |
| struct flb_mp_chunk_record *dup; | |
| if (!src) { | |
| return NULL; | |
| } | |
| dup = flb_calloc(1, sizeof(struct flb_mp_chunk_record)); | |
| if (!dup) { | |
| return NULL; | |
| } | |
| /* Copy basic fields */ | |
| dup->event = src->event; | |
| dup->owns_group_metadata = FLB_FALSE; | |
| dup->owns_group_attributes = FLB_FALSE; | |
| /* Duplicate metadata if present */ | |
| if (src->cobj_metadata) { | |
| dup->cobj_metadata = cfl_object_dup(src->cobj_metadata); | |
| if (!dup->cobj_metadata) { | |
| flb_free(dup); | |
| return NULL; | |
| } | |
| } | |
| /* Duplicate record body if present */ | |
| if (src->cobj_record) { | |
| dup->cobj_record = cfl_object_dup(src->cobj_record); | |
| if (!dup->cobj_record) { | |
| if (dup->cobj_metadata) { | |
| cfl_object_destroy(dup->cobj_metadata); | |
| } | |
| flb_free(dup); | |
| return NULL; | |
| } | |
| } | |
| /* Duplicate group metadata if present */ | |
| if (src->cobj_group_metadata) { | |
| dup->cobj_group_metadata = cfl_object_dup(src->cobj_group_metadata); | |
| if (!dup->cobj_group_metadata) { | |
| if (dup->cobj_metadata) { | |
| cfl_object_destroy(dup->cobj_metadata); | |
| } | |
| if (dup->cobj_record) { | |
| cfl_object_destroy(dup->cobj_record); | |
| } | |
| flb_free(dup); | |
| return NULL; | |
| } | |
| dup->owns_group_metadata = FLB_TRUE; | |
| } | |
| /* Duplicate group attributes if present */ | |
| if (src->cobj_group_attributes) { | |
| dup->cobj_group_attributes = cfl_object_dup(src->cobj_group_attributes); | |
| if (!dup->cobj_group_attributes) { | |
| if (dup->cobj_metadata) { | |
| cfl_object_destroy(dup->cobj_metadata); | |
| } | |
| if (dup->cobj_record) { | |
| cfl_object_destroy(dup->cobj_record); | |
| } | |
| if (dup->cobj_group_metadata) { | |
| cfl_object_destroy(dup->cobj_group_metadata); | |
| } | |
| flb_free(dup); | |
| return NULL; | |
| } | |
| dup->owns_group_attributes = FLB_TRUE; | |
| } | |
| return dup; | |
| } |
src/flb_router_condition.c
Outdated
| ret = flb_log_event_encoder_begin_record(encoder); | ||
| if (ret == -1) { | ||
| flb_log_event_encoder_destroy(encoder); | ||
| return -1; | ||
| } | ||
|
|
||
| ret = flb_log_event_encoder_set_timestamp(encoder, &record->event.timestamp); | ||
| if (ret == -1) { | ||
| flb_log_event_encoder_destroy(encoder); | ||
| return -1; | ||
| } | ||
|
|
||
| /* Encode metadata */ | ||
| if (record->cobj_metadata) { | ||
| ret = flb_mp_cfl_to_msgpack(record->cobj_metadata, &mp_buf, &mp_size); | ||
| if (ret == -1) { | ||
| flb_log_event_encoder_destroy(encoder); | ||
| return -1; | ||
| } | ||
| ret = flb_log_event_encoder_set_metadata_from_raw_msgpack(encoder, mp_buf, mp_size); | ||
| flb_free(mp_buf); | ||
| if (ret != FLB_EVENT_ENCODER_SUCCESS) { | ||
| flb_log_event_encoder_destroy(encoder); | ||
| return -1; | ||
| } | ||
| } | ||
|
|
||
| /* Encode record body */ | ||
| if (record->cobj_record) { | ||
| ret = flb_mp_cfl_to_msgpack(record->cobj_record, &mp_buf, &mp_size); | ||
| if (ret == -1) { | ||
| flb_log_event_encoder_destroy(encoder); | ||
| return -1; | ||
| } | ||
| ret = flb_log_event_encoder_set_body_from_raw_msgpack(encoder, mp_buf, mp_size); | ||
| flb_free(mp_buf); | ||
| if (ret != FLB_EVENT_ENCODER_SUCCESS) { | ||
| flb_log_event_encoder_destroy(encoder); | ||
| return -1; | ||
| } | ||
| } | ||
|
|
||
| ret = flb_log_event_encoder_commit_record(encoder); | ||
| if (ret == -1) { | ||
| flb_log_event_encoder_destroy(encoder); | ||
| return -1; | ||
| } | ||
| } |
There was a problem hiding this comment.
Encoder API checks are wrong; compare against FLB_EVENT_ENCODER_SUCCESS.
flb_log_event_encoder_* functions don’t return -1 reliably. Use the FLB_EVENT_ENCODER_SUCCESS constant to detect errors.
- ret = flb_log_event_encoder_begin_record(encoder);
- if (ret == -1) {
+ ret = flb_log_event_encoder_begin_record(encoder);
+ if (ret != FLB_EVENT_ENCODER_SUCCESS) {
flb_log_event_encoder_destroy(encoder);
return -1;
}
- ret = flb_log_event_encoder_set_timestamp(encoder, &record->event.timestamp);
- if (ret == -1) {
+ ret = flb_log_event_encoder_set_timestamp(encoder, &record->event.timestamp);
+ if (ret != FLB_EVENT_ENCODER_SUCCESS) {
flb_log_event_encoder_destroy(encoder);
return -1;
}
...
- ret = flb_log_event_encoder_commit_record(encoder);
- if (ret == -1) {
+ ret = flb_log_event_encoder_commit_record(encoder);
+ if (ret != FLB_EVENT_ENCODER_SUCCESS) {
flb_log_event_encoder_destroy(encoder);
return -1;
}🤖 Prompt for AI Agents
In src/flb_router_condition.c around lines 344 to 391, the code is checking
flb_log_event_encoder_* return values against -1 but the encoder API signals
success via FLB_EVENT_ENCODER_SUCCESS; update all checks that compare return
values to -1 to instead compare against FLB_EVENT_ENCODER_SUCCESS (i.e. treat
any return != FLB_EVENT_ENCODER_SUCCESS as an error), and keep the existing
cleanup (flb_log_event_encoder_destroy and flb_free) and early returns intact.
edsiper
force-pushed
the
router-conditionals
branch
from
7be7826 to
947c874
Compare
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
include/fluent-bit/flb_input.h (1)
23-36: Minor: duplicated include of flb_coro.flb_coro.h is included twice (Lines 25 and 35). Remove the duplicate to reduce churn.
-#include <fluent-bit/flb_coro.h> ... -#include <fluent-bit/flb_coro.h>
🧹 Nitpick comments (7)
src/flb_conditionals.c (3)
309-327: Consider error handling for non-numeric field values in numeric comparisons.The
atof()function returns0.0for non-numeric strings, making it impossible to distinguish between actual zero values and conversion failures. When a user attempts numeric comparisons (GT, LT, GTE, LTE) on non-numeric fields (e.g.,"status" > 5where status is"production"), the field silently converts to0.0and the comparison proceeds with potentially unexpected results.Consider adding validation to detect non-numeric values and either:
- Log a warning when attempting numeric comparison on non-numeric fields
- Check if the variant type is actually numeric before string conversion
- Use
strtod()with error checking instead ofatof()</review_comment_end>
387-387: Consider moving flag assignment for better semantics.Setting
any_rule_evaluated = FLB_TRUEimmediately afterget_record_variant()but before the NULL check marks a rule as "evaluated" even when it's skipped due to a missing field (OR case). While this doesn't cause incorrect behavior, it's semantically confusing—the flag name suggests a rule was evaluated, but skipped rules aren't passed toevaluate_rule().For clarity, consider moving this assignment to line 399 (after the NULL check, right before calling
evaluate_rule()), so the flag only tracks rules that were actually evaluated./* Get the variant for this rule's context */ record_variant = get_record_variant(record, rule->context); - any_rule_evaluated = FLB_TRUE; if (!record_variant) { flb_trace("[condition] no record variant found for context %d", rule->context); if (cond->op == FLB_COND_OP_AND) { flb_trace("[condition] AND condition missing field, returning FALSE"); return FLB_FALSE; } else { continue; } } flb_trace("[condition] evaluating rule against record"); + any_rule_evaluated = FLB_TRUE; result = evaluate_rule(rule, record_variant);</review_comment_end>
422-425: Defensive code block appears unreachable in practice.This check defaults to FALSE when no rules were evaluated in an AND condition. However, this code appears unreachable:
- Empty rule lists are handled earlier (lines 376-378)
- AND conditions return FALSE immediately when a variant is missing (line 392)
- AND conditions return FALSE immediately when any rule fails (line 405)
- Therefore, reaching line 422 in AND mode implies all rules had valid variants and evaluated to TRUE, making
any_rule_evaluatedalways TRUE at this pointThe defensive check is harmless and may catch unexpected edge cases, but it's worth noting for maintenance.
</review_comment_end>
include/fluent-bit/flb_input.h (1)
55-56: Public API: cfl_list adoption looks good; confirm external impact.Switching struct flb_input_instance.routes_direct/routes to struct cfl_list changes the public struct layout. Please confirm:
- Out-of-tree plugins don’t rely on these fields, or
- We document the ABI break and bump/release-notes accordingly.
If external, add a migration note and guard in headers if needed.
src/flb_sosreport.c (1)
170-177: Migration to cfl_list is correct.Iteration and emptiness checks use CFL API appropriately. Consider adding an explicit
#include <cfl/cfl.h>to avoid relying on transitive headers.Also applies to: 269-273
tests/internal/conditionals.c (1)
1-1: Minor: BOM at file start.The file starts with a UTF‑8 BOM (before
#include). Consider removing it to prevent toolchain warnings.src/flb_task.c (1)
432-441: Minor perf: avoid repeated prepare per route.flb_router_path_should_route() prepares the logs context on each call. Consider preparing once before the loop (or making prepare idempotent/no-op when already prepared) to reduce overhead on many routes.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (14)
include/fluent-bit/flb_input.h(2 hunks)include/fluent-bit/flb_input_chunk.h(1 hunks)include/fluent-bit/flb_router.h(4 hunks)src/flb_conditionals.c(3 hunks)src/flb_input.c(1 hunks)src/flb_input_chunk.c(1 hunks)src/flb_input_log.c(3 hunks)src/flb_router.c(4 hunks)src/flb_router_config.c(12 hunks)src/flb_sosreport.c(2 hunks)src/flb_task.c(5 hunks)tests/internal/conditionals.c(2 hunks)tests/internal/input_chunk.c(0 hunks)tests/internal/router_config.c(8 hunks)
💤 Files with no reviewable changes (1)
- tests/internal/input_chunk.c
🚧 Files skipped from review as they are similar to previous changes (1)
- src/flb_router.c
🧰 Additional context used
🧠 Learnings (8)
📚 Learning: 2025-08-29T06:25:02.561Z
Learnt from: shadowshot-x
PR: fluent/fluent-bit#10794
File: tests/internal/aws_compress.c:7-7
Timestamp: 2025-08-29T06:25:02.561Z
Learning: In Fluent Bit, ZSTD (zstandard) compression library is bundled directly in the source tree at `lib/zstd-1.5.7` and is built unconditionally as a static library. Unlike optional external dependencies, ZSTD does not use conditional compilation guards like `FLB_HAVE_ZSTD` and is always available. Headers like `<fluent-bit/flb_zstd.h>` can be included directly without guards.
Applied to files:
tests/internal/router_config.csrc/flb_router_config.c
📚 Learning: 2025-08-29T06:25:27.250Z
Learnt from: shadowshot-x
PR: fluent/fluent-bit#10794
File: tests/internal/aws_compress.c:93-107
Timestamp: 2025-08-29T06:25:27.250Z
Learning: In Fluent Bit, ZSTD compression is enabled by default and is treated as a core dependency, not requiring conditional compilation guards like `#ifdef FLB_HAVE_ZSTD`. Unlike some other optional components such as ARROW/PARQUET (which use `#ifdef FLB_HAVE_ARROW` guards), ZSTD support is always available and doesn't need build-time conditionals. ZSTD headers are included directly without guards across multiple plugins and core components.
Applied to files:
tests/internal/router_config.csrc/flb_router_config.c
📚 Learning: 2025-08-31T12:46:11.940Z
Learnt from: ThomasDevoogdt
PR: fluent/fluent-bit#9277
File: .github/workflows/pr-compile-check.yaml:147-151
Timestamp: 2025-08-31T12:46:11.940Z
Learning: In fluent-bit CMakeLists.txt, the system library preference flags are defined as FLB_PREFER_SYSTEM_LIB_ZSTD and FLB_PREFER_SYSTEM_LIB_KAFKA with the FLB_ prefix.
Applied to files:
tests/internal/router_config.csrc/flb_router_config.c
📚 Learning: 2025-08-29T06:24:26.170Z
Learnt from: shadowshot-x
PR: fluent/fluent-bit#10794
File: tests/internal/aws_compress.c:39-42
Timestamp: 2025-08-29T06:24:26.170Z
Learning: In Fluent Bit, ZSTD compression support is enabled by default and does not require conditional compilation guards (like #ifdef FLB_HAVE_ZSTD) around ZSTD-related code declarations and implementations.
Applied to files:
tests/internal/router_config.csrc/flb_router_config.c
📚 Learning: 2025-08-29T06:24:55.855Z
Learnt from: shadowshot-x
PR: fluent/fluent-bit#10794
File: src/aws/flb_aws_compress.c:52-56
Timestamp: 2025-08-29T06:24:55.855Z
Learning: ZSTD compression is always available in Fluent Bit and does not require conditional compilation guards. Unlike Arrow/Parquet which use #ifdef FLB_HAVE_ARROW guards, ZSTD is built unconditionally with flb_zstd.c included directly in src/CMakeLists.txt and a bundled ZSTD library at lib/zstd-1.5.7/.
Applied to files:
src/flb_router_config.c
📚 Learning: 2025-08-29T06:25:27.250Z
Learnt from: shadowshot-x
PR: fluent/fluent-bit#10794
File: tests/internal/aws_compress.c:93-107
Timestamp: 2025-08-29T06:25:27.250Z
Learning: In Fluent Bit, ZSTD compression is enabled by default and is treated as a core dependency, not requiring conditional compilation guards like `#ifdef FLB_HAVE_ZSTD`. Unlike some other optional components, ZSTD support is always available and doesn't need build-time conditionals.
Applied to files:
src/flb_router_config.c
📚 Learning: 2025-08-29T06:24:44.797Z
Learnt from: shadowshot-x
PR: fluent/fluent-bit#10794
File: src/aws/flb_aws_compress.c:26-26
Timestamp: 2025-08-29T06:24:44.797Z
Learning: In Fluent Bit, ZSTD support is always available and enabled by default. The build system automatically detects and uses either the system libzstd library or builds the bundled ZSTD version. Unlike other optional dependencies like Arrow which use conditional compilation guards (e.g., FLB_HAVE_ARROW), ZSTD does not require conditional includes or build flags.
Applied to files:
src/flb_router_config.c
📚 Learning: 2025-08-31T12:46:11.940Z
Learnt from: ThomasDevoogdt
PR: fluent/fluent-bit#9277
File: .github/workflows/pr-compile-check.yaml:147-151
Timestamp: 2025-08-31T12:46:11.940Z
Learning: In fluent-bit, the correct CMake flag for using system librdkafka is `FLB_PREFER_SYSTEM_LIB_KAFKA=ON`.
Applied to files:
src/flb_router_config.c
🧬 Code graph analysis (7)
include/fluent-bit/flb_input_chunk.h (1)
src/flb_input_chunk.c (1)
flb_input_chunk_get_real_size(285-313)
src/flb_task.c (5)
src/flb_router_condition.c (3)
flb_router_chunk_context_init(33-44)flb_router_path_should_route(260-283)flb_router_chunk_context_destroy(68-71)src/flb_event.c (1)
flb_event_chunk_destroy(69-79)include/fluent-bit/flb_mem.h (2)
flb_free(126-128)flb_calloc(84-96)src/flb_routes_mask.c (1)
flb_routes_mask_get_bit(125-140)src/flb_output.c (1)
flb_output_name(1087-1094)
tests/internal/router_config.c (4)
src/flb_log_event_encoder.c (5)
flb_log_event_encoder_init(42-74)flb_log_event_encoder_begin_record(246-254)flb_log_event_encoder_destroy(99-116)flb_log_event_encoder_set_current_timestamp(289-292)flb_log_event_encoder_commit_record(256-274)src/flb_sds.c (2)
flb_sds_destroy(389-399)flb_sds_create(78-90)include/fluent-bit/flb_mem.h (2)
flb_free(126-128)flb_calloc(84-96)src/flb_router_condition.c (6)
flb_route_condition_eval(219-258)flb_router_chunk_context_init(33-44)flb_condition_eval_logs(156-195)flb_router_chunk_context_reset(46-66)flb_router_chunk_context_destroy(68-71)flb_router_path_should_route(260-283)
tests/internal/conditionals.c (1)
src/flb_conditionals.c (4)
flb_condition_create(219-233)flb_condition_add_rule(235-255)flb_condition_evaluate(359-429)flb_condition_destroy(257-274)
src/flb_input_log.c (10)
src/flb_sds.c (2)
flb_sds_destroy(389-399)flb_sds_create(78-90)include/fluent-bit/flb_mem.h (2)
flb_free(126-128)flb_calloc(84-96)src/flb_hash_table.c (2)
flb_hash_table_get(496-522)flb_hash_table_del_ptr(154-194)src/flb_input_chunk.c (4)
flb_input_chunk_get_real_size(285-313)flb_input_chunk_update_output_instances(2205-2234)flb_input_chunk_append_raw(2017-2040)flb_input_chunk_destroy(1066-1168)src/flb_routes_mask.c (4)
flb_routes_mask_get_bit(125-140)flb_routes_mask_clear_bit(100-115)flb_routes_mask_set_bit(76-91)flb_routes_mask_is_empty(142-148)src/flb_mp.c (1)
flb_mp_cfl_to_msgpack(1017-1045)src/flb_log_event_encoder.c (5)
flb_log_event_encoder_begin_record(246-254)flb_log_event_encoder_set_timestamp(276-287)flb_log_event_encoder_commit_record(256-274)flb_log_event_encoder_create(76-97)flb_log_event_encoder_destroy(99-116)src/flb_router_condition.c (4)
flb_router_route_get_condition(285-292)flb_router_chunk_context_init(33-44)flb_router_chunk_context_prepare_logs(73-134)flb_router_chunk_context_destroy(68-71)src/flb_conditionals.c (1)
flb_condition_evaluate(359-429)src/flb_event.c (2)
flb_event_chunk_create(26-57)flb_event_chunk_destroy(69-79)
include/fluent-bit/flb_router.h (1)
src/flb_router_condition.c (10)
flb_router_chunk_context_init(33-44)flb_router_chunk_context_reset(46-66)flb_router_chunk_context_destroy(68-71)flb_router_chunk_context_prepare_logs(73-134)flb_route_condition_eval(219-258)flb_condition_eval_logs(156-195)flb_condition_eval_metrics(197-206)flb_condition_eval_traces(208-217)flb_router_path_should_route(260-283)flb_router_route_get_condition(285-292)
src/flb_router_config.c (5)
src/flb_sds.c (1)
flb_sds_destroy(389-399)include/fluent-bit/flb_mem.h (2)
flb_free(126-128)flb_calloc(84-96)src/flb_conditionals.c (1)
flb_condition_destroy(257-274)lib/cfl/include/cfl/cfl_array.h (2)
cfl_array_size(46-49)cfl_array_fetch_by_index(36-44)lib/cfl/src/cfl_kvlist.c (1)
cfl_kvlist_fetch(418-421)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (24)
- GitHub Check: pr-windows-build / call-build-windows-package (Windows 64bit, x64, x64-windows-static, 3.31.6)
- GitHub Check: pr-windows-build / call-build-windows-package (Windows 64bit (Arm64), amd64_arm64, -DCMAKE_SYSTEM_NAME=Windows -DCMA...
- GitHub Check: pr-windows-build / call-build-windows-package (Windows 32bit, x86, x86-windows-static, 3.31.6)
- GitHub Check: run-ubuntu-unit-tests (-DFLB_SIMD=Off, 3.31.6, clang, clang++)
- GitHub Check: run-ubuntu-unit-tests (-DFLB_SANITIZE_THREAD=On, 3.31.6, gcc, g++)
- GitHub Check: run-ubuntu-unit-tests (-DFLB_ARROW=On, 3.31.6, gcc, g++)
- GitHub Check: run-ubuntu-unit-tests (-DFLB_SANITIZE_THREAD=On, 3.31.6, clang, clang++)
- GitHub Check: run-ubuntu-unit-tests (-DFLB_SIMD=Off, 3.31.6, gcc, g++)
- GitHub Check: run-ubuntu-unit-tests (-DFLB_SIMD=On, 3.31.6, clang, clang++)
- GitHub Check: run-ubuntu-unit-tests (-DFLB_SIMD=On, 3.31.6, gcc, g++)
- GitHub Check: run-ubuntu-unit-tests (-DFLB_COVERAGE=On, 3.31.6, gcc, g++)
- GitHub Check: run-ubuntu-unit-tests (-DFLB_SANITIZE_MEMORY=On, 3.31.6, gcc, g++)
- GitHub Check: run-ubuntu-unit-tests (-DFLB_SANITIZE_MEMORY=On, 3.31.6, clang, clang++)
- GitHub Check: run-ubuntu-unit-tests (-DSANITIZE_UNDEFINED=On, 3.31.6, clang, clang++)
- GitHub Check: run-ubuntu-unit-tests (-DFLB_JEMALLOC=On, 3.31.6, gcc, g++)
- GitHub Check: run-ubuntu-unit-tests (-DFLB_JEMALLOC=Off, 3.31.6, clang, clang++)
- GitHub Check: run-ubuntu-unit-tests (-DSANITIZE_ADDRESS=On, 3.31.6, clang, clang++)
- GitHub Check: run-ubuntu-unit-tests (-DFLB_SMALL=On, 3.31.6, gcc, g++)
- GitHub Check: run-ubuntu-unit-tests (-DSANITIZE_UNDEFINED=On, 3.31.6, gcc, g++)
- GitHub Check: run-ubuntu-unit-tests (-DSANITIZE_ADDRESS=On, 3.31.6, gcc, g++)
- GitHub Check: run-ubuntu-unit-tests (-DFLB_JEMALLOC=Off, 3.31.6, gcc, g++)
- GitHub Check: run-ubuntu-unit-tests (-DFLB_SMALL=On, 3.31.6, clang, clang++)
- GitHub Check: run-ubuntu-unit-tests (-DFLB_JEMALLOC=On, 3.31.6, clang, clang++)
- GitHub Check: PR - fuzzing test
🔇 Additional comments (10)
src/flb_conditionals.c (3)
366-367: Tracking variables implementation looks correct.The introduction of
any_rule_evaluatedandany_rule_matchedflags correctly implements the tracking needed for proper OR/AND evaluation semantics. The logic ensures:
- OR returns TRUE if any rule matches, FALSE if all fail or are skipped
- AND returns FALSE if any rule fails or is missing, TRUE if all succeed
</review_comment_end>
Also applies to: 387-387, 411-415
390-396: Missing variant handling is correct.The different handling for AND vs OR conditions when a field is missing makes semantic sense:
- AND: Returns FALSE immediately (all conditions must be met, missing field cannot match)
- OR: Continues to next rule (at least one condition must match, skip unavailable fields)
</review_comment_end>
417-429: Final evaluation logic is correct.The final evaluation correctly implements:
- OR: Returns TRUE if any rule matched (short-circuit case handled earlier), FALSE otherwise
- AND: Returns TRUE if all rules passed (short-circuit on first FALSE handled earlier)
</review_comment_end>
src/flb_input.c (1)
355-357: LGTM: initialize CFL lists.cfl_list_init for routes_direct/routes matches the header changes.
tests/internal/conditionals.c (1)
879-894: Good addition: missing metadata behavior verified.Covers absent metadata path; matches AND semantics.
src/flb_input_chunk.c (1)
285-313: Publicizing get_real_size is fine; logic unchanged.Implementation mirrors prior static version. Ensure export lists are updated (see header comment).
src/flb_task.c (2)
398-406: Router chunk context lifecycle is handled correctly.Init and destroy paths cover all returns; good hygiene. Optionally add a small comment noting the invariant to aid future edits.
532-535: Cleanup symmetry retained.Destroying router_context before returning (non-direct branch) prevents leaks. LGTM.
include/fluent-bit/flb_input_chunk.h (1)
116-117: Symbol exports already handled via CMake configuration—no action needed.The repository uses CMake's
ENABLE_EXPORTS ONsetting (src/CMakeLists.txt:585) rather than explicit version scripts (.map/.def files). Sinceflb_input_chunk_get_real_sizeis already declared in the public header file (include/fluent-bit/flb_input_chunk.h), it will be automatically exported in shared builds through the existing CMake configuration. No version script or .def file updates are required.include/fluent-bit/flb_input.h (1)
340-344: I need to verify whether thetask->routesusage is part of this migration or a separate struct, and confirm the field types in flb_input.h:Based on the comprehensive verification through script execution and web search, the struct field migration from
mk_listtocfl_listfor theroutes_directandroutesfields inflb_input.his complete:Migration status verified:
The header currently shows
struct mk_list routes_directandstruct mk_list routesin the current master, but the PR changes these tocfl_list(as shown in the review snippet).All operations on
input->routes,in->routes,in->routes_direct, andins->routes_directconsistently usecfl_list_*functions (initialization, add, foreach, del, size checks).Initialization correctly uses
cfl_list_init()in bothsrc/flb_router_config.cand test files.All traversals and modifications across
src/flb_router.c,src/flb_router_config.c, andsrc/flb_input_log.cuse thecfl_list_*API.Note: The
task->routesfield (inflb_taskstruct) remains onmk_listand is a separate struct—not part of this migration scope.
Signed-off-by: Eduardo Silva <eduardo@chronosphere.io>
Signed-off-by: Eduardo Silva <eduardo@chronosphere.io>
Signed-off-by: Eduardo Silva <eduardo@chronosphere.io>
|
The PR #11055 removed the routes_mask guard, which exposed a design flaw in the previous logic. Unconditional routes were being forced through prepare_logs() first. The proper fix is to allow unconditional routes to short-circuit early, so that chunk_trace routes are wired correctly. With this change, the core_chunk_trace unit test will no longer fail. Signed-off-by: Hiroshi Hatake <hiroshi@chronosphere.io>
The PR #11055 removed the routes_mask guard, which exposed a design flaw in the previous logic. Unconditional routes were being forced through prepare_logs() first. The proper fix is to allow unconditional routes to short-circuit early, so that chunk_trace routes are wired correctly. With this change, the core_chunk_trace unit test will no longer fail. Signed-off-by: Hiroshi Hatake <hiroshi@chronosphere.io>
Add New Routing System with Conditional Logic (Logs)
Overview
This PR introduces a new routing system for Fluent Bit that provides content-aware routing based on actual data content rather than simple tags. The new system supports sophisticated conditional logic, multiple output support, and advanced telemetry handling.
Key Features
🆕 New Capabilities
eq,gt,regex,in, etc.).confformatlogssignal typeConfiguration Example
Available Operators
eq$status eq 200neq$level neq "debug"gt$metric.value gt 0.8lt$response_time lt 100gte$cpu_usage gte 90lte$memory lte 1024regex$message regex "^ERROR"not_regex$path not_regex "\.tmp$"in$level in ["error", "warn"]not_in$env not_in ["dev", "test"]Signal Types
logs- for log data ✅ Conditionals supportedmetrics- for metric data ❌ Conditionals not yet supportedtraces- for trace data ❌ Conditionals not yet supportedany- for all telemetry types ❌ Conditionals not yet supportedProcessing Order
Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.
Summary by CodeRabbit
New Features
Bug Fixes
Tests