Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,13 @@

## Unreleased

## [1.0.0b11](https://github.com/microsoft/ApplicationInsights-Python/releases/tag/v1.0.0b11) - 2023-04-12

- Remove most configuration for Public Preview
([#277](https://github.com/microsoft/ApplicationInsights-Python/pull/277))
- Infer telemetry category disablement from exporter environment variables
([#278](https://github.com/microsoft/ApplicationInsights-Python/pull/278))

## [1.0.0b11](https://github.com/microsoft/ApplicationInsights-Python/releases/tag/v1.0.0b11) - 2023-04-12

- Reverse default behavior of instrumentations and implement configuration for exclusion
([#253](https://github.com/microsoft/ApplicationInsights-Python/pull/253))
- Use entrypoints instead of importlib to load instrumentations
Expand Down
46 changes: 14 additions & 32 deletions azure-monitor-opentelemetry/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,38 +55,20 @@ You can use `configure_azure_monitor` to set up instrumentation for your app to
| Parameter | Description | Environment Variable |
|-------------------|----------------------------------------------------|----------------------|
| `connection_string` | The [connection string][connection_string_doc] for your Application Insights resource. The connection string will be automatically populated from the `APPLICATIONINSIGHTS_CONNECTION_STRING` environment variable if not explicitly passed in. | `APPLICATIONINSIGHTS_CONNECTION_STRING` |
| `exclude_instrumentations` | By default, all supported [instrumentations](#officially-supported-instrumentations) are enabled to collect telemetry. Specify instrumentations you do not want to enable to collect telemetry by passing in a comma separated list of instrumented library names. e.g. `["requests", "flask"]` | |
| `resource` | Specifies the OpenTelemetry [resource][opentelemetry_spec_resource] associated with your application. See [this][ot_sdk_python_resource] for default behavior. | [OTEL_SERVICE_NAME][opentelemetry_spec_service_name], [OTEL_RESOURCE_ATTRIBUTES][opentelemetry_spec_resource_attributes] |
| `disable_logging` | If set to `True`, disables collection and export of logging telemetry. Defaults to `False`. | |
| `disable_metrics` | If set to `True`, disables collection and export of metric telemetry. Defaults to `False`. | |
| `disable_tracing` | If set to `True`, disables collection and export of distributed tracing telemetry. Defaults to `False`. | |
| `logging_level` | Specifies the [logging level][logging_level] of the logs you would like to collect for your logging pipeline. Defaults to 0 which is `logging.NOTSET`. | |
| `logger_name` | Specifies the [logger name][logger_name_hierarchy_doc] under which logging will be instrumented. Defaults to "" which corresponds to the root logger. | |
| `logging_export_interval_ms`| Specifies the logging export interval in milliseconds. Defaults to 5000. | `OTEL_BLRP_SCHEDULE_DELAY` |
| `metric_readers` | Specifies the [metric readers][ot_metric_reader] that you would like to use for your metric pipeline. Accepts a list of [metric readers][ot_sdk_python_metric_reader]. | |
| `views` | Specifies the list of [views][opentelemetry_spec_view] to configure for the metric pipeline. See [here][ot_sdk_python_view_examples] for example usage. | |
| `sampling_ratio` | Specifies the ratio of distributed tracing telemetry to be [sampled][application_insights_sampling]. Accepted values are in the range [0,1]. Defaults to 1.0, meaning no telemetry is sampled out. | `OTEL_TRACES_SAMPLER_ARG` |
| `tracing_export_interval_ms`| Specifies the distributed tracing export interval in milliseconds. Defaults to 5000. | `OTEL_BSP_SCHEDULE_DELAY` |
| `instrumentation_config` | Specifies a dictionary of kwargs that will be applied to instrumentation configuration. You can specify which instrumentation you want to configure by name in the key field and value as a dictionary representing `kwargs` for the corresponding instrumentation. Refer to the `Supported Library` section [above](#officially-supported-instrumentations) for the list of supported library names. | |

Example for use of `instrumentation_config`:
```python
...
configure_azure_monitor(
connection_string="<your-connection-string>",
instrumentation_config={
"flask": {
"excluded_urls": "http://localhost:8080/ignore",
},
"requests": {
"excluded_urls": "http://example.com"
}
}
)
...
```

Take a look at the specific [instrumenation][ot_instrumentations] documentation for available configurations.

You can configure further with [OpenTelemetry environment variables][ot_env_vars] such as:
| Environment Variable | Description |
|-------------|----------------------|
| [OTEL_SERVICE_NAME][opentelemetry_spec_service_name], [OTEL_RESOURCE_ATTRIBUTES][opentelemetry_spec_resource_attributes] | Specifies the OpenTelemetry [resource][opentelemetry_spec_resource] associated with your application. |
| `OTEL_LOGS_EXPORTER` | If set to `None`, disables collection and export of logging telemetry. |
| `OTEL_METRICS_EXPORTER` | If set to `None`, disables collection and export of metric telemetry. |
| `OTEL_TRACES_EXPORTER` | If set to `None`, disables collection and export of distributed tracing telemetry. |
| `OTEL_BLRP_SCHEDULE_DELAY` | Specifies the logging export interval in milliseconds. Defaults to 5000. |
| `OTEL_BSP_SCHEDULE_DELAY` | Specifies the distributed tracing export interval in milliseconds. Defaults to 5000. |
| `OTEL_TRACES_SAMPLER_ARG` | Specifies the ratio of distributed tracing telemetry to be [sampled][application_insights_sampling]. Accepted values are in the range [0,1]. Defaults to 1.0, meaning no telemetry is sampled out. |
<!-- TODO: add once OTEL_LOG_LEVEL is supported in sdk -->
<!-- Specifies the [logging level][logging_level] of the logs you would like to collect for your logging pipeline. Defaults to 0 which is `logging.NOTSET`. | OTEL_LOG_LEVEL | -->

#### Azure monitor OpenTelemetry Exporter configurations

Expand Down Expand Up @@ -123,12 +105,12 @@ Samples are available [here][samples] to demonstrate how to utilize the above co
[exporter_configuration_docs]: https://github.com/Azure/azure-sdk-for-python/tree/main/sdk/monitor/azure-monitor-opentelemetry-exporter#configuration
[logging_level]: https://docs.python.org/3/library/logging.html#levels
[logger_name_hierarchy_doc]: https://docs.python.org/3/library/logging.html#logger-objects
[ot_env_vars]: https://opentelemetry.io/docs/reference/specification/sdk-environment-variables/
[ot_instrumentations]: https://github.com/open-telemetry/opentelemetry-python-contrib/tree/main/instrumentation
[ot_metric_reader]: https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/metrics/sdk.md#metricreader
[ot_python_docs]: https://opentelemetry.io/docs/instrumentation/python/
[ot_sdk_python]: https://github.com/open-telemetry/opentelemetry-python
[ot_sdk_python_metric_reader]: https://opentelemetry-python.readthedocs.io/en/stable/sdk/metrics.export.html#opentelemetry.sdk.metrics.export.MetricReader
[ot_sdk_python_resource]: https://github.com/open-telemetry/opentelemetry-python/blob/main/opentelemetry-sdk/src/opentelemetry/sdk/resources/__init__.py#L153
[ot_sdk_python_view_examples]: https://github.com/open-telemetry/opentelemetry-python/tree/main/docs/examples/metrics/views
[ot_instrumentation_django]: https://github.com/open-telemetry/opentelemetry-python-contrib/tree/main/instrumentation/opentelemetry-instrumentation-django
[ot_instrumentation_django_version]: https://github.com/open-telemetry/opentelemetry-python-contrib/blob/main/instrumentation/opentelemetry-instrumentation-django/src/opentelemetry/instrumentation/django/package.py#L16
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,22 +4,14 @@
# license information.
# --------------------------------------------------------------------------
from logging import getLogger
from typing import Any, Dict
from typing import Dict

from azure.monitor.opentelemetry._constants import (
DISABLE_LOGGING_ARG,
DISABLE_METRICS_ARG,
DISABLE_TRACING_ARG,
EXCLUDE_INSTRUMENTATIONS_ARG,
INSTRUMENTATION_CONFIG_ARG,
LOGGER_NAME_ARG,
LOGGING_EXPORT_INTERVAL_MS_ARG,
LOGGING_LEVEL_ARG,
METRIC_READERS_ARG,
RESOURCE_ARG,
SAMPLING_RATIO_ARG,
TRACING_EXPORT_INTERVAL_MS_ARG,
VIEWS_ARG,
)
from azure.monitor.opentelemetry._types import ConfigurationValue
from azure.monitor.opentelemetry.exporter import (
Expand All @@ -39,7 +31,6 @@
from opentelemetry.sdk._logs.export import BatchLogRecordProcessor
from opentelemetry.sdk.metrics import MeterProvider
from opentelemetry.sdk.metrics.export import PeriodicExportingMetricReader
from opentelemetry.sdk.resources import Resource
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.trace import get_tracer_provider, set_tracer_provider
Expand All @@ -59,30 +50,13 @@
)


InstrumentationConfig = Dict[str, Dict[str, Any]]


def configure_azure_monitor(**kwargs) -> None:
"""
This function works as a configuration layer that allows the
end user to configure OpenTelemetry and Azure monitor components. The
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we still need InstrumentationConfig type above?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point. Will remove.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

configuration can be done via arguments passed to this function.
:keyword str connection_string: Connection string for your Application Insights resource.
:keyword Sequence[str] exclude_instrumentations: Specifies instrumentations you want to disable.
:keyword Resource resource: Specified the OpenTelemetry [resource][opentelemetry_spec_resource] associated with your application.
:keyword bool disable_logging: If set to `True`, disables collection and export of logging telemetry. Defaults to `False`.
:keyword bool disable_metrics: If set to `True`, disables collection and export of metric telemetry. Defaults to `False`.
:keyword bool disable_tracing: If set to `True`, disables collection and export of distributed tracing telemetry. Defaults to `False`.
:keyword int logging_level: Specifies the logging of the logs you would like to collect for your logging pipeline.
:keyword str logger_name: Specifies the logger name under which logging will be instrumented. Defaults to "" which corresponds to the root logger.
:keyword int logging_export_interval_ms: Specifies the logging export interval in milliseconds. Defaults to 5000.
:keyword Sequence[MetricReader] metric_readers: Specifies the metric readers that you would like to use for your metric pipeline.
:keyword Sequence[View] views: Specifies the list of views to configure for the metric pipeline.
:keyword float sampling_ratio: Specifies the ratio of distributed tracing telemetry to be sampled. Accepted values are in the range [0,1]. Defaults to 1.0, meaning no telemetry is sampled out.
:keyword int tracing_export_interval_ms: Specifies the distributed tracing export interval in milliseconds. Defaults to 5000.
:keyword InstrumentationConfig instrumentation_config: Specifies a dictionary of kwargs that will be applied to instrumentation configuration. You can specify which instrumentation you want to
configure by name in the key field and value as a dictionary representing `kwargs` for the corresponding instrumentation.
Refer to the `Supported Library` section of https://github.com/microsoft/ApplicationInsights-Python/tree/main/azure-monitor-opentelemetry#officially-supported-instrumentations for the list of suppoprted library names.
:keyword ManagedIdentityCredential/ClientSecretCredential credential: Token credential, such as ManagedIdentityCredential or ClientSecretCredential, used for Azure Active Directory (AAD) authentication. Defaults to None.
:keyword bool disable_offline_storage: Boolean value to determine whether to disable storing failed telemetry records for retry. Defaults to `False`.
:keyword str storage_directory: Storage directory in which to store retry files. Defaults to `<tempfile.gettempdir()>/Microsoft/AzureMonitor/opentelemetry-python-<your-instrumentation-key>`.
:rtype: None
Expand All @@ -94,97 +68,67 @@ def configure_azure_monitor(**kwargs) -> None:
disable_logging = configurations[DISABLE_LOGGING_ARG]
disable_metrics = configurations[DISABLE_METRICS_ARG]

resource = None
if not disable_logging or not disable_tracing or not disable_metrics:
resource = _get_resource(configurations)

# Setup tracing pipeline
if not disable_tracing:
_setup_tracing(resource, configurations)
_setup_tracing(configurations)

# Setup logging pipeline
if not disable_logging:
_setup_logging(resource, configurations)
_setup_logging(configurations)

# Setup metrics pipeline
if not disable_metrics:
_setup_metrics(resource, configurations)
_setup_metrics(configurations)

# Setup instrumentations
# Instrumentations need to be setup last so to use the global providers
# instanstiated in the other setup steps
_setup_instrumentations(configurations)

_setup_instrumentations()

def _get_resource(configurations: Dict[str, ConfigurationValue]) -> Resource:
return configurations.get(RESOURCE_ARG, Resource.create())


def _setup_tracing(
resource: Resource, configurations: Dict[str, ConfigurationValue]
):
def _setup_tracing(configurations: Dict[str, ConfigurationValue]):
sampling_ratio = configurations[SAMPLING_RATIO_ARG]
tracing_export_interval_ms = configurations[TRACING_EXPORT_INTERVAL_MS_ARG]
tracer_provider = TracerProvider(
sampler=ApplicationInsightsSampler(sampling_ratio=sampling_ratio),
resource=resource,
)
set_tracer_provider(tracer_provider)
trace_exporter = AzureMonitorTraceExporter(**configurations)
span_processor = BatchSpanProcessor(
trace_exporter,
schedule_delay_millis=tracing_export_interval_ms,
)
get_tracer_provider().add_span_processor(span_processor)


def _setup_logging(
resource: Resource, configurations: Dict[str, ConfigurationValue]
):
logger_name = configurations[LOGGER_NAME_ARG]
logging_level = configurations[LOGGING_LEVEL_ARG]
def _setup_logging(configurations: Dict[str, ConfigurationValue]):
# TODO: Remove after upgrading to OTel SDK 1.18
logging_export_interval_ms = configurations[LOGGING_EXPORT_INTERVAL_MS_ARG]
logger_provider = LoggerProvider(resource=resource)
logger_provider = LoggerProvider()
set_logger_provider(logger_provider)
log_exporter = AzureMonitorLogExporter(**configurations)
log_record_processor = BatchLogRecordProcessor(
log_exporter,
schedule_delay_millis=logging_export_interval_ms,
)
get_logger_provider().add_log_record_processor(log_record_processor)
handler = LoggingHandler(
level=logging_level, logger_provider=get_logger_provider()
)
getLogger(logger_name).addHandler(handler)
handler = LoggingHandler(logger_provider=get_logger_provider())
getLogger().addHandler(handler)


def _setup_metrics(
resource: Resource, configurations: Dict[str, ConfigurationValue]
):
views = configurations[VIEWS_ARG]
metric_readers = configurations[METRIC_READERS_ARG]
def _setup_metrics(configurations: Dict[str, ConfigurationValue]):
metric_exporter = AzureMonitorMetricExporter(**configurations)
reader = PeriodicExportingMetricReader(metric_exporter)
meter_provider = MeterProvider(
metric_readers=[reader] + metric_readers,
resource=resource,
views=views,
metric_readers=[reader],
)
set_meter_provider(meter_provider)


def _setup_instrumentations(configurations: Dict[str, ConfigurationValue]):
exclude_instrumentations = configurations[EXCLUDE_INSTRUMENTATIONS_ARG]
instrumentation_configs = configurations[INSTRUMENTATION_CONFIG_ARG]

def _setup_instrumentations():
# use pkg_resources for now until https://github.com/open-telemetry/opentelemetry-python/pull/3168 is merged
for entry_point in iter_entry_points("opentelemetry_instrumentor"):
lib_name = entry_point.name
if lib_name not in _SUPPORTED_INSTRUMENTED_LIBRARIES:
continue
if lib_name in exclude_instrumentations:
_logger.debug("Instrumentation excluded for library %s", lib_name)
continue
try:
# Check if dependent libraries/version are installed
conflict = get_dist_dependency_conflicts(entry_point.dist)
Expand All @@ -197,11 +141,8 @@ def _setup_instrumentations(configurations: Dict[str, ConfigurationValue]):
continue
# Load the instrumentor via entrypoint
instrumentor: BaseInstrumentor = entry_point.load()
# Call instrument() with configuration
config = instrumentation_configs.get(lib_name, {})
# tell instrumentation to not run dep checks again as we already did it above
config["skip_dep_check"] = True
instrumentor().instrument(**config)
instrumentor().instrument(skip_dep_check=True)
except Exception as ex:
_logger.warning(
"Exception occured when instrumenting: %s.",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,20 +16,11 @@
# --------------------Configuration------------------------------------------

CONNECTION_STRING_ARG = "connection_string"
EXCLUDE_INSTRUMENTATIONS_ARG = "exclude_instrumentations"
RESOURCE_ARG = "resource"
DISABLE_LOGGING_ARG = "disable_logging"
DISABLE_METRICS_ARG = "disable_metrics"
DISABLE_TRACING_ARG = "disable_tracing"
# TODO: Consider Log Level to match env var
LOGGING_LEVEL_ARG = "logging_level"
LOGGER_NAME_ARG = "logger_name"
LOGGING_EXPORT_INTERVAL_MS_ARG = "logging_export_interval_ms"
METRIC_READERS_ARG = "metric_readers"
VIEWS_ARG = "views"
SAMPLING_RATIO_ARG = "sampling_ratio"
TRACING_EXPORT_INTERVAL_MS_ARG = "tracing_export_interval_ms"
INSTRUMENTATION_CONFIG_ARG = "instrumentation_config"


# --------------------Diagnostic/status logging------------------------------
Expand Down
Loading