Feature Request | JSON | Ability to configure maximum sessions

[omit9154]

My preferred JSON provider initially allows queries, but then starts blocking with a ‘too many requests’ message.

Is it possible to add a global configuration option for JSON queries to prevent this blocking - e.g. allow a maximum of x queries every y seconds?

Thanks.

[Sn1kk3r5]

Technically of course, if someone is willing to code it.

[ProgFriese]

@omit9154

My preferred JSON provider

I hope it is not yahoo.

[omit9154]

No it isn’t Yahoo. I use them for a few holdings that are problematic in Yahoo - e.g. LGUG.UK.

It seems that they’re happy to provide a few quotes, but I tried to use them for all my daily quotes as a workaround to yesterday’s Yahoo outage. After the first few quotes completed, they started blocking further requests. I’ve switched back to Yahoo for the majority of my holdings.

I think it would probably work reliably if we can slow things down a bit. I currently have 28 active holdings, so I’d be happy to submit a request every two seconds to update the portfolio in just under a minute.

It isn’t clear how long Yahoo is going to keep working, so I think we’ll need to work with the remaining providers and not upset them with aggressive data harvesting.

[omit9154]

CoPilot has suggested the following code, but I have no idea if it is correct or how it could be added to the GUI.

AI generated

import java.util.concurrent.Executors;
import java.util.concurrent.ScheduledExecutorService;
import java.util.concurrent.TimeUnit;
import java.util.Queue;
import java.util.LinkedList;

public class QuoteRateLimiter {

    private final int maxRequests;
    private final int timeInterval;
    private final ScheduledExecutorService scheduler;
    private final Queue<String> requestQueue;

    public QuoteRateLimiter(int maxRequests, int timeInterval) {
        this.maxRequests = maxRequests;
        this.timeInterval = timeInterval;
        this.scheduler = Executors.newScheduledThreadPool(1);
        this.requestQueue = new LinkedList<>();
    }

    public void addRequest(String apiUrl) {
        requestQueue.add(apiUrl);
    }

    public void startProcessing() {
        scheduler.scheduleAtFixedRate(() -> {
            for (int i = 0; i < maxRequests && !requestQueue.isEmpty(); i++) {
                String requestUrl = requestQueue.poll();
                if (requestUrl != null) {
                    fetchQuote(requestUrl);
                }
            }
        }, 0, timeInterval, TimeUnit.SECONDS);
    }

    private void fetchQuote(String url) {
        // Implement actual API call logic here
        System.out.println("Fetching: " + url);
    }

    public void stopProcessing() {
        scheduler.shutdown();
    }

    public static void main(String[] args) {
        QuoteRateLimiter limiter = new QuoteRateLimiter(5, 10); // Max 5 requests every 10 seconds
        limiter.addRequest("https://api.example.com/quote1");
        limiter.addRequest("https://api.example.com/quote2");
        limiter.startProcessing();
    }
}

Alright! To integrate this rate-limiting mechanism into Portfolio Performance, you’ll need to modify the JSON quote retrieval system. Here’s how you can do it:

Steps to Hook It In:

1. Locate the JSON Quote Fetching Code

• The relevant code for fetching JSON quotes is likely in the net.tod.watchex.json or similar package.
• Look for a method handling HTTP API requests—it may use Java’s HttpClient or an existing URLConnection.

2. Modify the Request Handling Logic

• Replace the direct API calls with queued requests controlled by your new QuoteRateLimiter class.
• Introduce a global configuration setting for users to define max requests per X seconds.

3. Store the User Settings

• Portfolio Performance uses an XML-based settings system. You’ll need to update preferences.xml to store the rate-limit parameters.

4. Integrate Rate-Limiting in the Fetching Process

• Instead of fetching all quotes simultaneously, enqueue them and process them at intervals.
• Modify the class responsible for API requests to interact with your QuoteRateLimiter.

5. Update the UI for Configuration

• Add a new setting in the Preferences Panel (PortfolioPreferences.java).
• Let users define X requests per Y seconds via a slider or input box.

Where to Make These Changes?

• API Quote Retrieval: Look for classes like JsonQuoteSource.java or similar.
• Settings Storage: Modify PortfolioPreferences.java to store rate-limit settings.
• UI Modifications: Likely in QuoteFeedSettingsPanel.java or equivalent.
• Queue Implementation: Your QuoteRateLimiter class will manage API call scheduling.

Once you’ve adjusted these areas, your modified Portfolio Performance will prevent excessive API calls and avoid being blocked by data providers.

To make the rate-limiting feature configurable via the Portfolio Performance UI, you’ll need to modify the settings panel and store user preferences. Here’s how you can do it:

Steps to Add UI Settings

1. Modify the Preferences Panel

• Add input fields for users to specify:

• Maximum JSON requests per cycle (X)

• Time interval between request batches (Y seconds)

2. Store User Settings in Preferences

• Update the PortfolioPreferences.java (or similar settings manager) to store these values persistently.

3. Modify UI Components

• Update QuoteFeedSettingsPanel.java (or equivalent UI file) to display sliders or input boxes for users to set values.
• Add labels explaining the rate-limiting feature.

Example UI Implementation in Java (Swing)

Here’s a sample UI modification that adds a simple panel for rate-limiting settings

import javax.swing.*;
import java.awt.*;
import java.util.prefs.Preferences;

public class QuoteRateLimiterSettingsPanel extends JPanel {
    
    private final Preferences prefs;
    private final JSpinner maxRequestsSpinner;
    private final JSpinner timeIntervalSpinner;

    public QuoteRateLimiterSettingsPanel() {
        prefs = Preferences.userNodeForPackage(QuoteRateLimiterSettingsPanel.class);

        setLayout(new GridLayout(3, 2));

        JLabel maxRequestsLabel = new JLabel("Max requests per cycle:");
        maxRequestsSpinner = new JSpinner(new SpinnerNumberModel(
                prefs.getInt("maxRequests", 5), 1, 100, 1));

        JLabel timeIntervalLabel = new JLabel("Time interval (seconds):");
        timeIntervalSpinner = new JSpinner(new SpinnerNumberModel(
                prefs.getInt("timeInterval", 10), 1, 3600, 1));

        add(maxRequestsLabel);
        add(maxRequestsSpinner);
        add(timeIntervalLabel);
        add(timeIntervalSpinner);

        JButton saveButton = new JButton("Save Settings");
        saveButton.addActionListener(e -> saveSettings());
        add(saveButton);
    }

    private void saveSettings() {
        prefs.putInt("maxRequests", (int) maxRequestsSpinner.getValue());
        prefs.putInt("timeInterval", (int) timeIntervalSpinner.getValue());
        JOptionPane.showMessageDialog(this, "Settings saved successfully!");
    }

    public static void main(String[] args) {
        JFrame frame = new JFrame("Quote Rate Limiter Settings");
        frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
        frame.add(new QuoteRateLimiterSettingsPanel());
        frame.pack();
        frame.setVisible(true);
    }
}

To integrate the Rate-Limiting Settings Panel into the Portfolio Performance UI, follow these steps:

1. Locate the Portfolio Preferences Panel

The preferences UI is likely managed within QuoteFeedSettingsPanel.java or a similar class that handles JSON data feed settings.

2. Modify the Settings Panel

• Add a section for Rate-Limiting Configuration.
• Include two input fields: one for max requests per cycle (X) and one for time interval (Y seconds).
• Ensure values persist using Portfolio Performance’s preference storage.

3. Hook the Settings into the JSON Fetching Logic

• Use your QuoteRateLimiter implementation to read the stored settings.
• Adjust the fetching process to queue requests instead of running them simultaneously.

Example Integration:

Modify QuoteFeedSettingsPanel.java to insert your Quote Rate-Limiting Settings Panel:

JPanel rateLimitingPanel = new QuoteRateLimiterSettingsPanel();
mainPanel.add(rateLimitingPanel, BorderLayout.SOUTH);

If QuoteFeedSettingsPanel.java is part of a larger Tabbed UI, ensure the Rate-Limiting Configuration is added under the JSON Data Feed Settings tab.

4. Apply the Settings During API Calls

Inside JsonQuoteSource.java (or the class responsible for retrieving JSON quotes), modify the method that fetches quotes:

QuoteRateLimiter limiter = new QuoteRateLimiter(
    PortfolioPreferences.getInt("maxRequests", 5), 
    PortfolioPreferences.getInt("timeInterval", 10)
);

limiter.startProcessing();

To ensure the rate-limiting settings persist correctly in Portfolio Performance, you’ll need to integrate them into the application’s preference storage mechanism. The software typically stores user settings in an XML-based system, or within Java Preferences.

Steps for Preference Storage Integration

1. Modify the Preferences Manager (PortfolioPreferences.java or equivalent)

• Add new preference fields for max requests per cycle (X) and time interval (Y seconds).
• Use Java’s Preferences API or the existing settings structure.

2. Store and Retrieve User Settings

• Ensure settings persist across application restarts.
• Retrieve stored preferences when initializing the QuoteRateLimiter.

3. Apply Preferences to API Calls

• Modify the JSON fetching system to read these values dynamically.
• Adjust rate-limiting logic accordingly.

Implementation: Modify Portfolio Preferences

Inside PortfolioPreferences.java, add the new preference keys:

import java.util.prefs.Preferences;

public class PortfolioPreferences {

    private static final Preferences prefs = Preferences.userNodeForPackage(PortfolioPreferences.class);

    // Define preference keys
    private static final String MAX_REQUESTS_KEY = "maxRequests";
    private static final String TIME_INTERVAL_KEY = "timeInterval";

    public static int getMaxRequests() {
        return prefs.getInt(MAX_REQUESTS_KEY, 5);  // Default to 5 requests
    }

    public static void setMaxRequests(int maxRequests) {
        prefs.putInt(MAX_REQUESTS_KEY, maxRequests);
    }

    public static int getTimeInterval() {
        return prefs.getInt(TIME_INTERVAL_KEY, 10);  // Default to 10 seconds
    }

    public static void setTimeInterval(int timeInterval) {
        prefs.putInt(TIME_INTERVAL_KEY, timeInterval);
    }
}

Integrate Preferences into Quote Fetching System

Modify JsonQuoteSource.java (or the JSON fetching class) to use stored preferences:

QuoteRateLimiter limiter = new QuoteRateLimiter(
    PortfolioPreferences.getMaxRequests(), 
    PortfolioPreferences.getTimeInterval()
);

limiter.startProcessing();