WIP: Feature Quantization Scaffolding#682
Conversation
| # node ids in the produced batch when reading serialized tfrecords. | ||
| if entity_key not in feature_spec_dict: | ||
| logger.info( | ||
| f"Injecting entity key {entity_key} into feature spec dictionary with value `tf.io.FixedLenFeature(shape=[], dtype=tf.int64)`" |
There was a problem hiding this comment.
Double check why we inject tf.int64
There was a problem hiding this comment.
|
/unit_test |
GiGL Automation@ 20:23:16UTC : 🔄 @ 20:25:02UTC : ✅ Workflow completed successfully. |
GiGL Automation@ 20:23:18UTC : 🔄 @ 20:28:19UTC : ❌ Workflow failed. |
GiGL Automation@ 20:23:19UTC : 🔄 @ 20:33:56UTC : ✅ Workflow completed successfully. |
| "Cannot materialize quantized features with " | ||
| f"{dequantized.size(0)} rows into existing x with {x.size(0)} rows." | ||
| ) | ||
| node_store.x = torch.cat([x, dequantized], dim=1) |
There was a problem hiding this comment.
Here is where we'd need to scatter write raw and quantized node features if preserving original feature order is a requirement.
|
/unit_test |
GiGL Automation@ 22:21:08UTC : 🔄 @ 23:37:31UTC : ✅ Workflow completed successfully. |
GiGL Automation@ 22:21:08UTC : 🔄 @ 22:29:50UTC : ✅ Workflow completed successfully. |
GiGL Automation@ 22:21:08UTC : 🔄 @ 22:22:54UTC : ✅ Workflow completed successfully. |
| "Computed 1-bit quantization stats: " | ||
| f"neg_mean={q.neg_mean}, pos_mean={q.pos_mean}" | ||
| ) | ||
| else: |
There was a problem hiding this comment.
Alternatively we can simplify proto model and compute all stats regardless of bits, but I suspect avoiding tft.quantiles in the 1-bit case is worth doing.
1 participant
TODO