Инструменты пользователя

Инструменты сайта


how-to:loki

loki

Проверить loki: http://lokiserver:3100/ready
Информация о сборке: http://lokiserver:3100/loki/api/v1/status/buildinfo

Параметры запуска:

docker run --rm grafana/loki:latest -help

Docker

docker-compose.yaml

docker-compose.yaml

version: "3"

volumes:
  data:
    driver_opts:
      type: none
      device: ${VOLPATH-/data/docker/loki}/data
      o: bind

services:
  loki:
    image: grafana/loki:latest
    ports:
      - "3100:3100"
    command: |
      -config.file=/etc/loki/local-config.yaml
      -validation.reject-old-samples=false
      -distributor.ingestion-burst-size-mb=16
    volumes:
      - data:/loki


Для тестов loki и grafana с анонимным входом:

docker-compose.yaml

docker-compose.yaml

version: "3.7"

services:
  loki:
    image: grafana/loki:latest
    command: |
      -config.file=/etc/loki/local-config.yaml
      -distributor.ingestion-burst-size-mb=32
      -distributor.ingestion-rate-limit-mb=16
      -server.grpc-max-recv-msg-size-bytes=16777216
      -ingester.per-stream-rate-limit=16MB
      -ingester.max-chunk-age=24h
      -validation.reject-old-samples.max-age=24h
      -validation.reject-old-samples=false
    ports:
      - "3100:3100"
  grafana:
    image: grafana/grafana-oss:latest
    environment:
      - GF_PATHS_PROVISIONING=/etc/grafana/provisioning
      - GF_AUTH_ANONYMOUS_ENABLED=true
      - GF_AUTH_ANONYMOUS_ORG_ROLE=Admin
      - GF_AUTH_BASIC_ENABLED=false
    entrypoint:
      - sh
      - -euc
      - |
        mkdir -p /etc/grafana/provisioning/datasources
        cat <<EOF > /etc/grafana/provisioning/datasources/ds.yaml
        apiVersion: 1
        datasources:
        - name: Loki
          type: loki
          access: proxy
          orgId: 1
          url: http://loki:3100
          basicAuth: false
          isDefault: true
          version: 1
          editable: false
        EOF
        /run.sh
    ports:
      - "3000:3000"


https://grafana.com/docs/loki/latest/installation/docker/

Driver for Docker

Установить плагин:

docker plugin install  grafana/loki-docker-driver:latest --alias loki --grant-all-permissions

Запустить контейнер с log-driver loki:

docker run --log-driver=loki \
    --log-opt loki-url="http://lokiserver:3100/api/prom/push" \
    --log-opt loki-retries=5 \
    --log-opt loki-batch-size=400 \
    grafana/grafana

или для всего сервиса docker в /etc/docker/daemon.json:

{
    "log-driver": "loki",
    "log-opts": {
        "loki-url": "http://lokiserver:3100/api/prom/push",
        "loki-batch-size": "400"
    }
}

https://github.com/grafana/loki/tree/main/clients/cmd/docker-driver
https://github.com/grafana/loki/blob/master/docs/sources/clients/docker-driver/configuration.md
https://github.com/grafana/loki/issues/2361

Удаление данных

Удалить данные с диска:

kubectl exec loki-0 -- rm -r /data/loki/chunks /data/loki/index

K8s

helm repo add grafana https://grafana.github.io/helm-charts
helm repo update grafana

SingleBinary

helm

helm

Пример, для storage=filesystem, с доменом loki.dev.domain.com и с ограничением по ip 1.1.1.1 и 2.2.2.2, с установкой на ноду с Taints: sku=monitoring:NoSchedule, Labels: sku=monitoring и харением логов 30 дней

helm -n monitoring upgrade --create-namespace --install loki grafana/loki \
 --set global.extraArgs='["-log.format=json"]' \
 --set loki.commonConfig.replication_factor=1 \
 --set-json loki.schemaConfig='{"configs":[{"from":"2025-01-01","index":{"period":"24h","prefix":"loki_index_"},"object_store":"filesystem","schema":"v13","store":"tsdb"}]}' \
 --set loki.compactor.retention_enabled=true \
 --set loki.compactor.delete_request_store=filesystem \
 --set loki.pattern_ingester.enabled=true \
 --set loki.limits_config.retention_period=740h \
 --set loki.limits_config.allow_structured_metadata=true \
 --set loki.limits_config.volume_enabled=true \
 --set loki.ruler.enable_api=true \
 --set loki.auth_enabled=false \
 --set loki.storage.type=filesystem \
 --set gateway.ingress.enabled=true \
 --set gateway.ingress.ingressClassName=nginx \
 --set-json gateway.ingress.annotations='{"cert-manager.io/cluster-issuer":"letsencrypt","nginx.ingress.kubernetes.io/whitelist-source-range":"1.1.1.1,8.8.8.8"}' \
 --set-json gateway.ingress.hosts='[{"host":"loki.dev.domain.com","paths":[{"path":"/","pathType":"Prefix"}]}]' \
 --set-json gateway.ingress.tls='[{"hosts":["loki.dev.domain.com"],"secretName":"loki-gateway-tls"}]' \
 --set deploymentMode=SingleBinary --set singleBinary.replicas=1 \
 --set singleBinary.resources.requests.memory=500Mi --set singleBinary.resources.requests.cpu=50m --set singleBinary.resources.limits.memory=2Gi --set singleBinary.resources.limits.cpu=1 \
 --set-json singleBinary.tolerations='[{ "key": "sku", "operator": "Equal", "value": "monitoring", "effect": "NoSchedule"}]' \
 --set-json singleBinary.nodeSelector='{ "sku":"monitoring" }' \
 --set-json lokiCanary.tolerations='[{ "key": "sku", "operator": "Equal", "value": "monitoring", "effect": "NoSchedule"}]' \
 --set-json lokiCanary.nodeSelector='{ "sku":"monitoring" }' \
 --set-json gateway.tolerations='[{ "key": "sku", "operator": "Equal", "value": "monitoring", "effect": "NoSchedule"}]' \
 --set-json gateway.nodeSelector='{ "sku":"monitoring" }' \
 --set-json resultsCache.tolerations='[{ "key": "sku", "operator": "Equal", "value": "monitoring", "effect": "NoSchedule"}]' \
 --set-json resultsCache.affinity='{ "nodeAffinity": { "requiredDuringSchedulingIgnoredDuringExecution": { "nodeSelectorTerms": [{ "matchExpressions": [{ "key": "sku", "operator": "In", "values": ["monitoring"] }]}]}}}' \
 --set-json chunksCache.tolerations='[{ "key": "sku", "operator": "Equal", "value": "monitoring", "effect": "NoSchedule"}]' \
 --set-json chunksCache.affinity='{ "nodeAffinity": { "requiredDuringSchedulingIgnoredDuringExecution": { "nodeSelectorTerms": [{ "matchExpressions": [{ "key": "sku", "operator": "In", "values": ["monitoring"] }]}]}}}' \
 --set backend.replicas=0  --set read.replicas=0 --set write.replicas=0 --set ingester.replicas=0 --set querier.replicas=0 --set queryFrontend.replicas=0 --set queryScheduler.replicas=0 --set distributor.replicas=0 --set compactor.replicas=0 --set indexGateway.replicas=0 --set bloomCompactor.replicas=0 --set bloomGateway.replicas=0 \
 --atomic

для storage=s3 (minio), заменить:

 --set-json loki.schemaConfig='{"configs":[{"from":"2025-01-01","index":{"period":"24h","prefix":"loki_index_"},"object_store":"filesystem","schema":"v13","store":"tsdb"}]}' \
 --set loki.compactor.delete_request_store=filesystem \
 --set loki.storage.type=filesystem \

на:

 --set-json loki.schemaConfig='{"configs":[{"from":"2025-01-01","index":{"period":"24h","prefix":"loki_index_"},"object_store":"s3","schema":"v13","store":"tsdb"}]}' \
 --set minio.enabled=true \

https://github.com/grafana/loki/blob/main/production/helm/loki/values.yaml
https://grafana.com/docs/loki/next/setup/install/helm/install-monolithic/

loki-stack depricated

loki-stack.values.yaml

loki-stack.values.yaml

loki:
  user: loki
  pass: lokiPass
  config:
    compactor:
      shared_store: filesystem
      working_directory: /data/loki/boltdb-shipper-compactor
      retention_enabled: true
    limits_config:
      retention_period: 740h
  persistence:
    enabled: true
    accessModes:
    - ReadWriteOnce
    size: 8Gi
  ingress:
    enabled: true
    annotations:
      nginx.ingress.kubernetes.io/whitelist-source-range: "1.1.1.1,8.8.8.8"
    tls:
    - hosts:
      - loki.domain.com
      secretName: loki-tls-secret
    hosts:
    - host: loki.domain.com
      paths:
        path: /
promtail:
  tolerations:
  - effect: NoSchedule
    key: sku
    operator: Equal
    value: hcpu
  - effect: NoSchedule
    key: sku
    operator: Equal
    value: gpu
helm -n monitoring upgrade --install loki grafana/loki-stack -f loki-stack.values.yaml


Ссылки

how-to/loki.txt · Последнее изменение: 127.0.0.1