Files
Stirling-PDF/src/main/java/stirling/software/SPDF/service/MetricsAggregatorService.java
T
d8cca66560 Add default languages to OCR, fix compression for QPDF and embedded images (#3202)
# Description of Changes

This pull request includes several changes to the codebase, focusing on
enhancing OCR support, improving endpoint management, and adding new
functionality for PDF compression. The most important changes are
detailed below.

### Enhancements to OCR support:

* `Dockerfile` and `Dockerfile.fat`: Added support for multiple new OCR
languages including Chinese (Simplified), German, French, and
Portuguese. (Our top 5 languages including English)
[[1]](diffhunk://#diff-dd2c0eb6ea5cfc6c4bd4eac30934e2d5746747af48fef6da689e85b752f39557R69-R72)
[[2]](diffhunk://#diff-571631582b988e88c52c86960cc083b0b8fa63cf88f056f26e9e684195221c27L78-R81)

### Improvements to endpoint management:

*
[`src/main/java/stirling/software/SPDF/config/EndpointConfiguration.java`](diffhunk://#diff-750f31f6ecbd64b025567108a33775cad339e835a04360affff82a09410b697dR51-R66):
Added a new method `isGroupEnabled` to check if a group of endpoints is
enabled.
*
[`src/main/java/stirling/software/SPDF/config/EndpointConfiguration.java`](diffhunk://#diff-750f31f6ecbd64b025567108a33775cad339e835a04360affff82a09410b697dL179-L193):
Updated endpoint groups and removed redundant qpdf endpoints.
[[1]](diffhunk://#diff-750f31f6ecbd64b025567108a33775cad339e835a04360affff82a09410b697dL179-L193)
[[2]](diffhunk://#diff-750f31f6ecbd64b025567108a33775cad339e835a04360affff82a09410b697dL243-L244)
*
[`src/main/java/stirling/software/SPDF/config/EndpointInspector.java`](diffhunk://#diff-845de13e140bb1264014539714860f044405274ad2a9481f38befdd1c1333818R1-R291):
Introduced a new `EndpointInspector` class to discover and validate GET
endpoints dynamically.

### New functionality for PDF compression:

*
[`src/main/java/stirling/software/SPDF/controller/api/misc/CompressController.java`](diffhunk://#diff-c307589e9f958f2593c9567c5ad9d63cd03788aa4803b3017b1c13b0d0485805R10):
Enhanced the `CompressController` to handle nested images within form
XObjects, improving the accuracy of image compression in PDFs.
Remove Compresses Dependency on QPDF
[[1]](diffhunk://#diff-c307589e9f958f2593c9567c5ad9d63cd03788aa4803b3017b1c13b0d0485805R10)
[[2]](diffhunk://#diff-c307589e9f958f2593c9567c5ad9d63cd03788aa4803b3017b1c13b0d0485805R28-R44)
[[3]](diffhunk://#diff-c307589e9f958f2593c9567c5ad9d63cd03788aa4803b3017b1c13b0d0485805L49-R61)
[[4]](diffhunk://#diff-c307589e9f958f2593c9567c5ad9d63cd03788aa4803b3017b1c13b0d0485805R77-R99)
[[5]](diff
hunk://#diff-c307589e9f958f2593c9567c5ad9d63cd03788aa4803b3017b1c13b0d0485805L92-R191)
Closes #(issue_number)

---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#6-testing)
for more details.

---------

Co-authored-by: a <a>
2025-03-20 09:39:57 +00:00

99 lines
4.1 KiB
Java

package stirling.software.SPDF.service;
import java.util.HashMap;
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.scheduling.annotation.Scheduled;
import org.springframework.stereotype.Service;
import io.micrometer.core.instrument.MeterRegistry;
import io.micrometer.core.instrument.search.Search;
import stirling.software.SPDF.config.EndpointInspector;
@Service
public class MetricsAggregatorService {
private static final Logger logger = LoggerFactory.getLogger(MetricsAggregatorService.class);
private final MeterRegistry meterRegistry;
private final PostHogService postHogService;
private final EndpointInspector endpointInspector;
private final Map<String, Double> lastSentMetrics = new ConcurrentHashMap<>();
@Autowired
public MetricsAggregatorService(
MeterRegistry meterRegistry,
PostHogService postHogService,
EndpointInspector endpointInspector) {
this.meterRegistry = meterRegistry;
this.postHogService = postHogService;
this.endpointInspector = endpointInspector;
}
@Scheduled(fixedRate = 7200000) // Run every 2 hours
public void aggregateAndSendMetrics() {
Map<String, Object> metrics = new HashMap<>();
final boolean validateGetEndpoints = endpointInspector.getValidGetEndpoints().size() != 0;
Search.in(meterRegistry)
.name("http.requests")
.counters()
.forEach(
counter -> {
String method = counter.getId().getTag("method");
String uri = counter.getId().getTag("uri");
// Skip if either method or uri is null
if (method == null || uri == null) {
return;
}
// Skip URIs that are 2 characters or shorter
if (uri.length() <= 2) {
return;
}
// Skip non-GET and non-POST requests
if (!"GET".equals(method) && !"POST".equals(method)) {
return;
}
// For POST requests, only include if they start with /api/v1
if ("POST".equals(method) && !uri.contains("api/v1")) {
return;
}
if (uri.contains(".txt")) {
return;
}
// For GET requests, validate if we have a list of valid endpoints
if ("GET".equals(method)
&& validateGetEndpoints
&& !endpointInspector.isValidGetEndpoint(uri)) {
logger.debug("Skipping invalid GET endpoint: {}", uri);
return;
}
String key =
String.format(
"http_requests_%s_%s", method, uri.replace("/", "_"));
double currentCount = counter.count();
double lastCount = lastSentMetrics.getOrDefault(key, 0.0);
double difference = currentCount - lastCount;
if (difference > 0) {
logger.info("{}, {}", key, difference);
metrics.put(key, difference);
lastSentMetrics.put(key, currentCount);
}
});
// Send aggregated metrics to PostHog
if (!metrics.isEmpty()) {
postHogService.captureEvent("aggregated_metrics", metrics);
}
}
}