Open
Add FieldShardPolicy: business-field-based table sharding strategy#85
Conversation
Open
Agent-Logs-Url: https://github.com/NewLifeX/NewLife.XCode/sessions/9ea0dca5-2f74-4a06-88fa-ccadd9f32bfd Co-authored-by: Soar360 <15421284+Soar360@users.noreply.github.com>
Copilot
AI
changed the title
[WIP] Add field-based sharding strategy for logs
Add FieldShardPolicy: business-field-based table sharding strategy
May 22, 2026
Agent-Logs-Url: https://github.com/NewLifeX/NewLife.XCode/sessions/d5d17c0c-309f-44f4-acb5-ec2c14a914a5 Co-authored-by: Soar360 <15421284+Soar360@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
本 PR 为 XCode 增加“按业务字段值分表”的新策略 FieldShardPolicy，并补齐批量 Save 在自动分表场景下的路由分组逻辑，同时新增/扩展单元测试与文档，帮助多租户等场景按字段值稳定落表。
Changes:
- 新增
XCode.Shards.FieldShardPolicy：按指定字段值进行分库/分表路由，并支持从等值查询表达式中提取分片。 - 修复
EntityExtension.Save批量保存时的自动分表路由：新增GroupByShardSession，按分片会话分组后分别执行BatchSave。 - 新增/增强测试与文档：覆盖 FieldShardPolicy 的行为、SQL 路由以及列表
Save的分表写入验证。
影响（按仓库模板补充）：
- 公共 API：是（新增
FieldShardPolicy公共类型；EntityExtension增加 internal 辅助方法对外无破坏，但会影响行为/测试）。 - 性能影响：低（批量
Save增加一次分组遍历；换来正确路由）。
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 10 comments.
Show a summary per file
| File | Description |
|---|---|
| XCode/Shards/FieldShardPolicy.cs | 新增业务字段分片策略实现（按字段值路由、表达式提取等值条件） |
| XCode/Entity/EntityExtension.cs | 批量 Save 自动分表时按分片会话分组后分别 BatchSave，并新增 GroupByShardSession 复用 |
| XUnitTest.XCode/Shards/FieldShardPolicyTests.cs | 新增 FieldShardPolicy 单测与 SQL 路由集成测试 |
| XUnitTest.XCode/EntityTests/SqlTests.cs | 增加列表 Save 在 TimeShardPolicy/FieldShardPolicy 下的分表路由测试 |
| Doc/Shards分配策略与路由.md | 文档补充 FieldShardPolicy 的定位、能力与示例 |
Comment on lines
245
to
+249
| if (valid) others = others.Valid(true); | ||
| rs += BatchSave(session, others, null); | ||
| if (useShard) | ||
| { | ||
| var groups = GroupByShardSession(others, fact, session); | ||
| foreach (var item in groups) |
Comment on lines
+86
to
+96
| if (ConnPolicy.IsNullOrEmpty() && TablePolicy.IsNullOrEmpty()) return null; | ||
|
|
||
| if (Factory == null) throw new XCodeException("字段分表策略要求指定实体工厂！"); | ||
| var table = Factory.Table; | ||
|
|
||
| var connName = table.ConnName; | ||
| var tableName = table.TableName; | ||
| if (!ConnPolicy.IsNullOrEmpty()) connName = String.Format(ConnPolicy, connName, fieldValue); | ||
| if (!TablePolicy.IsNullOrEmpty()) tableName = String.Format(TablePolicy, tableName, fieldValue); | ||
|
|
||
| return new(connName, tableName); |
Comment on lines
+15
to
+16
| public class FieldShardPolicyTests | ||
| { |
Comment on lines
+239
to
+245
| // 字段异步加载，等待短暂时间 | ||
| System.Threading.Thread.Sleep(200); | ||
|
|
||
| var model = policy.Shard(3000); | ||
|
|
||
| Assert.NotNull(model); | ||
| Assert.Equal("Log2_3000", model.TableName); |
Comment on lines
+278
to
+296
| try | ||
| { | ||
| var log = new Log2 | ||
| { | ||
| Action = "字段分表", | ||
| Category = "Test", | ||
| CreateUserID = 1000, | ||
| }; | ||
| log.Insert(); | ||
|
|
||
| Assert.Contains("Log2_1000", sql); | ||
| Assert.StartsWith("[test] Insert Into Log2_1000(", sql); | ||
| } | ||
| catch { } | ||
| finally | ||
| { | ||
| Log2.Meta.ShardPolicy = null; | ||
| DAL.LocalFilter = null; | ||
| } |
Comment on lines
+311
to
+331
| try | ||
| { | ||
| var log = new Log2 | ||
| { | ||
| Action = "字段分表", | ||
| Category = "Test", | ||
| CreateUserID = 2000, | ||
| }; | ||
| log.Insert(); | ||
|
|
||
| log.Category = "Updated"; | ||
| log.Update(); | ||
|
|
||
| Assert.Contains("Log2_2000", sqls[^1]); | ||
| } | ||
| catch { } | ||
| finally | ||
| { | ||
| Log2.Meta.ShardPolicy = null; | ||
| DAL.LocalFilter = null; | ||
| } |
Comment on lines
+346
to
+367
| try | ||
| { | ||
| var log = new Log2 | ||
| { | ||
| Action = "字段分表", | ||
| Category = "Test", | ||
| CreateUserID = 3000, | ||
| }; | ||
| log.Insert(); | ||
|
|
||
| log.Delete(); | ||
|
|
||
| var deleteSql = sqls.LastOrDefault(s => s.Contains("Delete")); | ||
| Assert.NotNull(deleteSql); | ||
| Assert.Contains("Log2_3000", deleteSql); | ||
| } | ||
| catch { } | ||
| finally | ||
| { | ||
| Log2.Meta.ShardPolicy = null; | ||
| DAL.LocalFilter = null; | ||
| } |
Comment on lines
+382
to
+395
| try | ||
| { | ||
| Log2.FindAll(Log2._.CreateUserID == 9000); | ||
|
|
||
| sqls.RemoveAll(e => e.EndsWith("sqlite_master")); | ||
| Assert.NotEmpty(sqls); | ||
| Assert.Contains("Log2_9000", sqls[^1]); | ||
| } | ||
| catch { } | ||
| finally | ||
| { | ||
| Log2.Meta.ShardPolicy = null; | ||
| DAL.LocalFilter = null; | ||
| } |
Comment on lines
+421
to
+429
| if (sqls.Count > 0) | ||
| { | ||
| var inserts = sqls.Where(e => e.Contains("Insert Into", StringComparison.OrdinalIgnoreCase)).ToList(); | ||
| if (inserts.Count > 0) | ||
| { | ||
| Assert.All(inserts, s => Assert.Contains(tableName, s, StringComparison.OrdinalIgnoreCase)); | ||
| Assert.DoesNotContain(inserts, s => s.Contains("Insert Into Log2(", StringComparison.OrdinalIgnoreCase)); | ||
| } | ||
| } |
Comment on lines
+482
to
+490
| if (sqls.Count > 0) | ||
| { | ||
| var inserts = sqls.Where(e => e.Contains("Insert Into", StringComparison.OrdinalIgnoreCase)).ToList(); | ||
| if (inserts.Count > 0) | ||
| { | ||
| Assert.All(inserts, s => Assert.Contains(tableName, s, StringComparison.OrdinalIgnoreCase)); | ||
| Assert.DoesNotContain(inserts, s => s.Contains("Insert Into Log2(", StringComparison.OrdinalIgnoreCase)); | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
XUnitTest.XCode/EntityTests/SqlTests.cs增加列表Save分表测试（TimeShardPolicy）XUnitTest.XCode/EntityTests/SqlTests.cs增加列表Save分表测试（FieldShardPolicy）EntityExtension.Save批量场景下的分表路由：按分片会话分组后分别BatchSaveEntityExtension.GroupByShardSession（internal）供Save复用和测试校验SqlTests.ListSave_TimeShardPolicySqlTests.ListSave_FieldShardPolicyFieldShardPolicyTestscodeql_checker安全检查（工具超时，按提示未重试）