Resolve merge conflicts to keep both SearXNG and GLM Search
providers. Updated search priority order to:
Perplexity > Brave > SearXNG > Tavily > DuckDuckGo > GLM Search
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The TestConvertProvidersToModelList_AllProviders test expected 19
providers but adding Avian brings the total to 20.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
addMsg now calls f.Sync() before f.Close(), matching the durability
guarantee of writeMeta and rewriteJSONL (both use WriteFileAtomic
with fsync). Without this, a power loss could leave the appended
line in the kernel page cache only — lost on reboot.
When the LLM returns multiple tool calls, they are now executed
concurrently using goroutines + sync.WaitGroup instead of sequentially.
Results are collected in an indexed slice and processed in original order
to preserve message ordering. MessageTool.sentInRound is changed to
atomic.Bool for thread safety.
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Address Copilot review feedback:
- Move resolveDiscordRefs(content) before the referenced message
concatenation to prevent message links in quoted replies from being
expanded twice.
- Add unit tests for channelRefRe and msgLinkRe regex patterns,
covering valid/invalid inputs and the 3-link cap.
* feat(config): add GLMSearchConfig for GLM Search provider
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* test(tools): add failing tests for GLM Search provider
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat(tools): add GLMSearchProvider for web search
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat(agent): wire GLM Search config into web search tool registration
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Security fix: resolveDiscordRefs now takes a guildID parameter and
skips message links pointing to a different guild, preventing the bot
from leaking content across guilds.
Also uses s.State.Channel() cache before falling back to API calls
to reduce Discord API usage and rate limit risk.
- Guard against nil ReferencedMessage.Author to prevent panic
- Hoist regexp.MustCompile to package-level vars to avoid
re-compilation on every handleMessage call
- Both are defensive programming improvements
Add resolveDiscordRefs method that:
1. Resolves <#id> channel mentions to #channel-name by calling
the Discord API to fetch channel info
2. Expands Discord message links (up to 3) by fetching the linked
message content and appending it as '[linked message from User]: content'
Applied to both quoted/referenced messages and the main message
content for full context resolution.
When a user replies to a message in Discord, the bot now reads
m.ReferencedMessage and prepends its content to the incoming
message as '[quoted message from Username]: content'.
This gives the LLM full context of what message the user is
replying to, enabling meaningful follow-up conversations.
Add Avian (https://avian.io) as an OpenAI-compatible provider with
API base https://api.avian.io/v1 and AVIAN_API_KEY env var support.
Models: deepseek/deepseek-v3.2, moonshotai/kimi-k2.5, z-ai/glm-5,
minimax/minimax-m2.5. Supports chat completions, streaming, and
function calling.
Changes:
- Add Avian to ProvidersConfig struct, IsEmpty(), HasProvidersConfig()
- Add avian protocol to factory provider and default API base
- Add avian case to legacy provider selection (factory.go)
- Add avian migration rule for old config format
- Add default model entries to ModelList (deepseek-v3.2, kimi-k2.5)
- Add avian to example config
- Update AllProviders test count from 18 to 19
* feat(telegram): add base_url support for custom Telegram Bot API server
Allow users to specify a custom Telegram Bot API server URL via
config field `base_url` or env var `PICOCLAW_CHANNELS_TELEGRAM_BASE_URL`.
Defaults to the official https://api.telegram.org when left empty.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(telegram): trim whitespace and trailing slash from base_url
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
- Separate third-party imports from local module imports (gci)
- Fix byte slice literal formatting (gofumpt)
- Rename shadowed err variable to ftErr (govet)
- Remove trailing blank lines in test files
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Consolidate extractImageKey/extractFileKey/extractFileName into shared
extractJSONStringField helper to reduce code duplication
- Move mentionPlaceholderRegex to package-level position after imports
- Rename feishuCfg field to config for clarity within FeishuChannel
- Replace @_user_1 heuristic with GET /open-apis/bot/v3/info API call
at startup for reliable bot @mention detection
- Fix double close on file handle in downloadResource by removing defer
and using explicit close in both success and error paths
- Add unit tests for common.go and feishu_64.go helpers (53 test cases)
- serializeMessages: preserve ToolCallID/ToolCalls when Media is present
- resolveMediaRefs: add 20MB file size limit to prevent OOM
- mimeFromExtension: return empty string for unknown extensions
- Add 11 unit tests for serializeMessages, resolveMediaRefs, mimeFromExtension
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Remove stale "falls back to plain text" comment on Send
- Add empty ChatID validation in SendMedia to match Send
- Use messageID+fileKey as local filename to avoid write collisions
- Check allowlist before downloading inbound media to avoid wasted I/O
- Return errUnsupported consistently from all 32-bit stub methods
Upgrade the Feishu channel from basic text-only to full feature parity with
Telegram/Discord: interactive card messages with markdown rendering, message
editing (MessageEditor), placeholder messages (PlaceholderCapable), emoji
reactions (ReactionCapable), and inbound/outbound media support (MediaSender).
Also add @mention detection with lazy bot open_id discovery, group trigger
filtering with mention awareness, and multi-type inbound message parsing
(text, post, image, file, audio, video).
- Use atomic.Bool for closed flag to prevent TOCTOU race between
CallTool and Close operations
- Add double-check pattern in CallTool for thread-safe closed state
- Use atomic Swap in Close to ensure no new calls can start after
closed flag is set
- Move MCP manager cleanup defer before initialization to handle
partial initialization failures
- Update tests to use atomic.Bool operations
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The previous dedupe map rotation logic completely cleared the map when it reached max size, causing an 'amnesia cliff' where immediately arriving duplicates of just-forgotten messages would be processed.
This change replaces that with a MessageDeduplicator struct that uses a circular queue (ring buffer) to track insertions. When the limit is reached, it only evicts the absolute oldest message from the map, completely resolving the cliff issue.
This also cleans up the WeCom Bot and App webhook handlers by encapsulating the mutex and map state.
When the dedupe map rotates, the previous logic entirely cleared the map, meaning the message that triggered the rotation was immediately forgotten and could be duplicated immediately.
This change seeds the new map with the current message to prevent that. Also adds a defensive nil check.
Match rotation semantics to prior behavior by fully resetting the dedupe map
once the size limit is exceeded, and add focused tests for duplicate detection
and boundary rotation behavior.
Centralize dedupe map access behind a mutex-safe helper and use it in both
WeCom bot and WeCom app channels to eliminate concurrent map access races while
preserving current dedupe behavior.
- classifier.go: s/honour/honor/ (American English per misspell)
- router.go: break SelectModel signature across lines (golines)
- router_test.go: break long Message literal (golines)
- router_test.go: replace CJK string literal with rune slice so
gosmopolitan does not flag the source file; behaviour is identical
instance.go:
- Add Router *routing.Router and LightCandidates []FallbackCandidate
to AgentInstance.
- At agent creation, when routing.enabled and light_model resolves
successfully in model_list, pre-build the Router and resolve the
light model candidates once. If the light model isn't in model_list,
log a warning and disable routing for that agent gracefully.
loop.go:
- Add selectCandidates(agent, userMsg, history) helper.
It calls Router.SelectModel and returns either agent.Candidates /
agent.Model (primary tier) or agent.LightCandidates / light_model
(light tier). Returns primary unchanged when routing is disabled.
- In runLLMIteration, resolve (activeCandidates, activeModel) once
before entering the tool-iteration loop. The model tier is sticky
for the entire turn so a multi-step tool chain doesn't switch
models mid-way.
- Replace hard-coded agent.Candidates / agent.Model references in
callLLM and the debug log with the resolved active values.
The fallback chain and retry logic are untouched. When light_model
returns an error the fallback chain handles escalation normally.
Add three new files to pkg/routing/:
features.go — ExtractFeatures(msg, history) → Features
Computes five structural dimensions with zero keyword matching:
- TokenEstimate: rune_count/3 (CJK-safe token proxy)
- CodeBlockCount: ``` pairs in the message
- RecentToolCalls: tool call count in the last 6 history entries
- ConversationDepth: total messages in session
- HasAttachments: data URIs or media file extensions
classifier.go — Classifier interface + RuleClassifier
RuleClassifier uses a weighted sum that is capped at 1.0:
code block → +0.40 (triggers heavy model alone at 0.35 threshold)
token > 200 → +0.35 (triggers heavy model alone)
tool calls > 3 → +0.25
token 50-200 → +0.15
conversation depth > 10 → +0.10
attachment → 1.00 (hard gate, always heavy)
router.go — Router wraps config + Classifier
Router.SelectModel(msg, history, primaryModel) returns either the
configured light_model or the primary model depending on whether
the complexity score clears the threshold. Threshold defaults to
0.35 when zero/negative to prevent misconfiguration.
router_test.go — 34 tests covering all branches and edge cases
Introduce RoutingConfig with three fields:
- enabled: activates per-turn model routing
- light_model: references a model_name in model_list
- threshold: complexity score cutoff in [0,1]
When routing.enabled is true and the incoming message scores below
threshold, the agent switches to light_model for that turn. Absent or
disabled config leaves existing behaviour completely unchanged.
Example:
"agents": {
"defaults": {
"model": "claude-sonnet-4-6",
"routing": {
"enabled": true,
"light_model": "gemini-flash",
"threshold": 0.35
}
}
}
Without this function, media:// refs stored by MediaStore are passed
directly to the LLM API, which rejects them as invalid URLs.
resolveMediaRefs() runs after BuildMessages() and before the LLM call,
converting each media:// ref to a data:image/...;base64,... URL that
vision-capable models can process.
Also adds mimeFromExtension() helper for MIME type inference from
file extensions when ContentType metadata is not available.
- Introduced WeCom AIBot channel configuration in config.go with relevant fields.
- Implemented WeCom AIBot channel factory registration in init.go.
- Created unit tests for WeCom AIBot channel functionalities including initialization, start/stop behavior, webhook path handling, message encryption/decryption, and signature generation.
- Set default values for WeCom AIBot configuration in defaults.go.
The serializeMessages() function was not preserving the reasoning_content
field when serializing messages for vision API calls. This caused the
TestProviderChat_PreservesReasoningContentInHistory test to fail.
This fix ensures reasoning_content is included in both text-only messages
and vision messages with media attachments.
Co-authored-by: Zachary Guerrero <zack.grrr@gmail.com>
Add Media field to processOptions, pass msg.Media from inbound
messages through to BuildMessages and serializeMessages so
vision-capable LLMs receive image_url content parts.
Based on work by @as3k in sipeed/picoclaw#555.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add Media []string field to Message struct for image/media URLs
- Implement serializeMessages() to format messages with image_url content parts
- Enables OpenAI-compatible vision APIs to receive image attachments
* fix(tools): allow /dev/null redirection and add read/write sandbox split
- Remove deny pattern that incorrectly blocked redirects to /dev/null
- Expand block device write pattern to cover nvme, mmcblk, vd, xvd,
hd, loop, dm-, md, sr and nbd in addition to sd
- Add safe path whitelist for kernel pseudo-devices so workspace path
check does not reject /dev/null, /dev/zero, /dev/random, /dev/urandom,
/dev/stdin, /dev/stdout and /dev/stderr
- Add allow_read_outside_workspace config option (default true) so file
read and list tools are unrestricted while write tools stay sandboxed
Closes https://github.com/sipeed/picoclaw/issues/964
Closes https://github.com/sipeed/picoclaw/issues/965
Signed-off-by: Huang Rui <vowstar@gmail.com>
* feat(tools): add configurable allow patterns and path whitelists
- Add custom_allow_patterns to exec config so users can exempt specific
commands from deny pattern checks
- Add allow_read_paths and allow_write_paths regex lists to tools config
for whitelisting specific paths outside the workspace
- Introduce whitelistFs that wraps sandboxFs and falls through to hostFs
for paths matching whitelist patterns
- Use variadic constructor signatures to keep backward compatibility
Suggested-by: lxowalle
Signed-off-by: Huang Rui <vowstar@gmail.com>
---------
Signed-off-by: Huang Rui <vowstar@gmail.com>
* fix: return fetched content to LLM in web_fetch tool
WebFetchTool.Execute was setting ForLLM to a summary string
("Fetched N bytes from URL ...") instead of the actual extracted
text. This meant the LLM never saw the page content and could not
answer questions based on fetched web pages.
Return the extracted text in ForLLM so the model can use it.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: put full JSON result in ForLLM, summary in ForUser
Accept suggestion from afjcjsbx: the LLM should receive the full JSON
result (including extracted text) while the user sees a short summary.
Update tests to match the new field assignment.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* feat(config): Add support for env var configuration
This commit introduces support for two environment variables,
allowing users to override the default paths for picoclaw's home
directory and configuration file.
- `PICOCLAW_CONFIG`: Directly specifies the path to the `config.json` file.
This is initialised first, takes precedence over the hardcoded path, and is ideal
for containerized deployments or custom config management.
- `PICOCLAW_HOME`: Overrides the root directory for all picoclaw data, (except the config)
(e.g., `~/.picoclaw`). This is useful for portable installations or placing
data in non-standard locations.
This change provides greater flexibility for running picoclaw in various environments without
being tied to the default home directory structure.
* `README.md` updated explain PICOCLAW_CONFIG and PICOCLAW_HOME
* docs: translate environment variables section to multiple languages
---------
Co-authored-by: picoclaw <picoclaw@sipeed.com>
- Remove Feishu from webhook channel list in README.md and README.zh.md;
add clarifying note that Feishu uses WebSocket/SDK mode instead
- Replace Chinese text in README.vi.md header with Vietnamese equivalent
- Translate mixed-language WeCom note in README.vi.md to full Vietnamese
- Mark webhook_path as optional (否) in docs/channels/line/README.zh.md
- Remove incorrect yaml struct tags from new-channel example in
pkg/channels/README.md and README.zh.md (config uses json tags only)
- Fix multi-mode initChannel example to use whatsapp/whatsapp_native
(matching the "WhatsApp Bridge vs Native" comment) instead of matrix
- Correct ReasoningChannelID description: list the 12 channels that
have the field and note that PicoConfig does not expose it
* fix(pkg/providers):do regex precompile insteadd on the fly
* fix(providers): replace HTTP-specific regex with standalone status code matcher
The precompiled HTTP regex used uppercase "HTTP" which never matched
because ClassifyError lowercases the input. Replace it with a
case-insensitive word-boundary pattern that matches any standalone
3-digit status code (300-599), which also subsumes the HTTP/x.x case.
Add test case for standalone status code extraction.
* fix(providers): restore http regex and add standalone status code matcher
Restore the http-prefixed regex (without unnecessary (?i) flag since
input is already lowercased by ClassifyError) as a mid-priority pattern
to reduce false positives. Add a standalone word-boundary matcher as a
fallback for bare status codes like "429". Fix test to use lowercased
input matching the actual calling convention.
* perf(tools): move path regex compilation from per-call to package init
The path regex in guardCommand was compiled on every call. Hoist it
to a package-level var (absolutePathPattern) alongside defaultDenyPatterns
in a single var block, so it is compiled once at init time.
* style(tools): move inline comment to fix golines formatting error
- Fix ignored error from SendAndWait call
- Improve error message for unimplemented stdio mode with helpful guidance
- Add TODO comment with reference link for future stdio implementation
The openaiMessage struct and stripSystemParts() were not carrying over
the ReasoningContent field when serializing conversation history for
API requests. This caused thinking models (e.g. kimi-k2.5) to receive
incomplete assistant messages on subsequent turns, resulting in 400
errors from the Moonshot API.
Add the ReasoningContent field to openaiMessage and copy it in
stripSystemParts(). Also add a test to verify reasoning_content is
preserved when sending conversation history.
Fixes#588
Related: #876
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
