Add metric scrape module

README-zh.md

@@ -61,7 +61,9 @@ IoTDB (物联网数据库) 是一个专为物联网 (IoT) 场景设计的时序
 
 5. **MyBatis 生成器**：数据库访问代码生成工具
 
-6. **示例**：展示 IoTDB 与各种技术结合使用的示例应用程序和代码示例
+6. **Metric Scrape**：用于 IoTDB 表模型的独立 Prometheus 文本格式指标抓取工具
+
+7. **示例**：展示 IoTDB 与各种技术结合使用的示例应用程序和代码示例
 
 ## 环境要求
 
@@ -83,7 +85,7 @@ IoTDB (物联网数据库) 是一个专为物联网 (IoT) 场景设计的时序
 2. 使用 Maven 构建项目：
 
    ```bash
-   # 构建整个项目（包括 distributions、iotdb-collector、mybatis-generator）
+   # 构建整个项目（包括 distributions、iotdb-collector、metric-scrape、mybatis-generator）
    mvn clean package -DskipTests
 
    # 或者构建所有组件
@@ -310,6 +312,7 @@ mvn clean package -Pwith-all-connectors,with-examples,with-springboot -DskipTest
 - [Flink IoTDB 连接器](/connectors/flink-iotdb-connector/README.md)
 - [Grafana 插件](/connectors/grafana-plugin/README.md)
 - [IoTDB Spring Boot Starter](/iotdb-spring-boot-starter/README.md)
+- [Metric Scrape](/metric-scrape/README.md)
 - [Kubernetes Helm Charts](/helm/README.md)
 - [IoTDB Operator](/iotdb-operator/README.md)
 

README.md

@@ -61,7 +61,9 @@ This repository includes:
 
 5. **MyBatis Generator**: Code generation tools for database access
 
-6. **Examples**: Sample applications and code examples demonstrating the use of IoTDB with various technologies
+6. **Metric Scrape**: Standalone Prometheus text exposition scraper for the IoTDB table model
+
+7. **Examples**: Sample applications and code examples demonstrating the use of IoTDB with various technologies
 
 ## Prerequisites
 
@@ -83,7 +85,7 @@ To build the project from source, follow these steps:
 2. Build the project with Maven:
 
    ```bash
-   # Build the entire project (includes distributions,iotdb-collector,mybatis-generator)
+   # Build the entire project (includes distributions,iotdb-collector,metric-scrape,mybatis-generator)
    mvn clean package -DskipTests
 
    # Or build all
@@ -312,6 +314,7 @@ You can also refer to module-specific documentation:
 - [Flink IoTDB Connector](/connectors/flink-iotdb-connector/README.md)
 - [Grafana Plugin](/connectors/grafana-plugin/README.md)
 - [IoTDB Spring Boot Starter](/iotdb-spring-boot-starter/README.md)
+- [Metric Scrape](/metric-scrape/README.md)
 - [Kubernetes Helm Charts](/helm/README.md)
 - [IoTDB Operator](/iotdb-operator/README.md)
 

metric-scrape/Dockerfile

@@ -0,0 +1,26 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+FROM eclipse-temurin:17-jre-jammy
+
+WORKDIR /app
+
+COPY target/metric-scrape-2.0.4-SNAPSHOT.jar /app/metric-scrape.jar
+
+ENTRYPOINT ["java", "-jar", "/app/metric-scrape.jar", "-c", "/app/conf/metric-scrape.yml"]

metric-scrape/README.md

@@ -0,0 +1,146 @@
+<!--
+
+    Licensed to the Apache Software Foundation (ASF) under one
+    or more contributor license agreements.  See the NOTICE file
+    distributed with this work for additional information
+    regarding copyright ownership.  The ASF licenses this file
+    to you under the Apache License, Version 2.0 (the
+    "License"); you may not use this file except in compliance
+    with the License.  You may obtain a copy of the License at
+
+        http://www.apache.org/licenses/LICENSE-2.0
+
+    Unless required by applicable law or agreed to in writing,
+    software distributed under the License is distributed on an
+    "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+    KIND, either express or implied.  See the License for the
+    specific language governing permissions and limitations
+    under the License.
+
+-->
+
+# Metric Scrape
+
+Metric Scrape is a standalone service that periodically scrapes Prometheus text exposition
+endpoints and writes metrics into the Apache IoTDB table model through the
+`org.apache.iotdb:iotdb-session` client.
+
+It is a pull scraper, not a Prometheus remote write receiver.
+
+## Build
+
+```bash
+mvn -pl metric-scrape -am clean package -DskipTests
+```
+
+The runnable jar is generated at:
+
+```text
+metric-scrape/target/metric-scrape-2.0.4-SNAPSHOT.jar
+```
+
+## Run
+
+```bash
+java -jar metric-scrape/target/metric-scrape-2.0.4-SNAPSHOT.jar \
+  -c metric-scrape/conf/metric-scrape.yml
+```
+
+If `-c` is omitted, the service reads `conf/metric-scrape.yml` from the current working
+directory.
+
+## Configuration
+
+```yaml
+global:
+  scrape_interval: 15s
+  scrape_timeout: 10s
+  iotdb_database: 'metrics'
+  iotdb_username: 'root'
+  iotdb_password: 'root'
+  iotdb_urls: ['127.0.0.1:6667']
+
+scrape_configs:
+  - job_name: 'iotdbModel'
+    static_configs:
+      - targets: ['localhost:9090']
+    metrics_path: /metrics
+    relabel_configs:
+      - target_label: k8s_cluster
+        replacement: example-cluster
+      - target_label: k8s_namespace
+        replacement: default
+      - target_label: instance_id
+        replacement: model-service-local
+```
+
+Supported fields:
+
+| Field | Description |
+| --- | --- |
+| `global.scrape_interval` | Default scrape interval. Supports `ms`, `s`, `m`, `h`; no suffix means seconds. |
+| `global.scrape_timeout` | Default HTTP timeout. Supports the same duration format. |
+| `global.iotdb_database` | Target IoTDB table-model database. |
+| `global.iotdb_username` | Session username. |
+| `global.iotdb_password` | Session password. |
+| `global.iotdb_urls` | IoTDB node URLs, for example `127.0.0.1:6667`. |
+| `global.write_batch_size` | Optional tablet batch size. Default is `1000`. |
+| `scrape_configs[].job_name` | Scrape job name. It is written as tag column `job_name`. |
+| `scrape_configs[].scheme` | Optional target scheme. Default is `http`. |
+| `scrape_configs[].metrics_path` | Metrics path. Default is `/metrics`. |
+| `scrape_configs[].scrape_interval` | Optional interval override for this job. |
+| `scrape_configs[].scrape_timeout` | Optional timeout override for this job. |
+| `scrape_configs[].static_configs[].targets` | Target host list. |
+| `scrape_configs[].static_configs[].labels` | Extra labels for targets. They are written as tag columns. |
+| `scrape_configs[].relabel_configs[].target_label` | Constant target label name. |
+| `scrape_configs[].relabel_configs[].replacement` | Constant target label value. |
+
+Only constant relabeling with `target_label` and `replacement` is supported for now.
+
+## Data Model
+
+Metric Scrape writes into the IoTDB table model:
+
+| Prometheus content | Table model mapping |
+| --- | --- |
+| `global.iotdb_database` | Database name. |
+| `# HELP` metric family name | Table name. |
+| Sample metric name | Field column name. |
+| Sample value | Field column value, `DOUBLE`. |
+| `job_name` | Tag column. |
+| Target address | Tag column `instance`. |
+| Sample labels, static labels, relabel replacement labels | Tag columns, `STRING`. |
+| Sample timestamp | Row time. If missing, the current scrape time in milliseconds is used. |
+
+For histogram and summary samples, the longest matching `# HELP` metric family is used as the
+table name. For example, `request_duration_seconds_sum` and
+`request_duration_seconds_count` are written into table `request_duration_seconds` if the text
+contains `# HELP request_duration_seconds ...`.
+
+Metric names and label names are normalized before being used as table or column identifiers:
+characters other than letters, digits, and `_` are replaced by `_`, and identifiers starting with
+a digit get a leading `_`.
+
+## Docker And Kubernetes
+
+The module includes a `Dockerfile` and example Kubernetes manifests under `k8s/`.
+
+```bash
+docker build -t metric-scrape:latest metric-scrape
+kubectl apply -f metric-scrape/k8s/configmap.yaml
+kubectl apply -f metric-scrape/k8s/deployment.yaml
+```
+
+Update the image name, IoTDB URLs, credentials, and scrape targets before deploying to a real
+cluster.
+
+## Example Query
+
+```sql
+USE metrics;
+
+SELECT time, job_name, instance, env, http_server_requests_seconds_count
+FROM http_server_requests_seconds
+ORDER BY time DESC
+LIMIT 10;
+```

metric-scrape/conf/metric-scrape.yml

@@ -0,0 +1,61 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+global:
+  scrape_interval: 15s
+  scrape_timeout: 10s
+  iotdb_database: 'metrics'
+  iotdb_username: 'root'
+  iotdb_password: 'root'
+  iotdb_urls: ['127.0.0.1:6667']
+
+scrape_configs:
+  - job_name: 'iotdbModel'
+    static_configs:
+      - targets: ['localhost:9090']
+    metrics_path: /metrics
+    relabel_configs:
+      - target_label: k8s_cluster
+        replacement: example-cluster
+      - target_label: k8s_namespace
+        replacement: default
+      - target_label: instance_id
+        replacement: model-service-local
+
+  - job_name: 'iotdbWeb'
+    metrics_path: /actuator/prometheus
+    static_configs:
+      - targets: ['localhost:8080']
+        labels: {env: local, instance_id: web-service-local}
+    relabel_configs:
+      - target_label: k8s_cluster
+        replacement: example-cluster
+      - target_label: k8s_namespace
+        replacement: default
+
+  - job_name: 'iotdbAdmin'
+    metrics_path: /actuator/prometheus
+    static_configs:
+      - targets: ['localhost:8081']
+        labels: {env: local, instance_id: admin-service-local}
+    relabel_configs:
+      - target_label: k8s_cluster
+        replacement: example-cluster
+      - target_label: k8s_namespace
+        replacement: default

metric-scrape/k8s/configmap.yaml

@@ -0,0 +1,60 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: metric-scrape-config
+  namespace: default
+  labels:
+    app.kubernetes.io/name: metric-scrape
+    app.kubernetes.io/part-of: iotdb-monitoring
+data:
+  metric-scrape.yml: |
+    global:
+      scrape_interval: 30s
+      scrape_timeout: 10s
+      iotdb_database: 'metrics'
+      iotdb_username: 'root'
+      iotdb_password: 'root'
+      iotdb_urls: ['iotdb-service.default.svc.cluster.local:6667']
+
+    scrape_configs:
+      - job_name: 'iotdbModel'
+        metrics_path: /metrics
+        static_configs:
+          - targets: ['model-service.default.svc.cluster.local:9090']
+        relabel_configs:
+          - target_label: k8s_cluster
+            replacement: example-cluster
+          - target_label: k8s_namespace
+            replacement: default
+          - target_label: instance_id
+            replacement: model-service
+
+      - job_name: 'iotdbWeb'
+        metrics_path: /actuator/prometheus
+        static_configs:
+          - targets: ['web-service.default.svc.cluster.local:8080']
+            labels: {env: example, instance_id: web-service}
+        relabel_configs:
+          - target_label: k8s_cluster
+            replacement: example-cluster
+          - target_label: k8s_namespace
+            replacement: default