Automatic import of classifications

Hi,
thanks a lot for this great piece of software and all the efforts you put into it!

Are there any plans on supporting dynamic import/fetching of classifications?
I’d like to add all ETF holdings to an ETF via classifications. Doing this manually is not an option (e.g. MSCI World contains around 1500 holdings).
So far I didn’t find any free/cheap API which provides this information in an uniform way, but the emmitents provide this information for free.
However, all emmitents are using different formats (e.g. JSON, CSV, XSL, HTML table) and therefore I think its not viable to directly implement this in PP.
Nevertheless, there are a lot of utilities already in place to parse the different formats (JSON, CSV & HTML table), but they don’t work for importing classifications.
Is there a technical reason why this isn’t supported, or was it just not in scope yet?

I know that there are some further obstacles to overcome to get this working:

  1. Different exchanges are using different tickers (e.g. ‘Electronic Arts Inc’ is using ticker ‘EA’ for ‘NASDAQ’, but ‘ERT’ for ‘STU’ [the german exchange in Stuttgart]). I live in Germany so I’d probably assign the ‘ERT’ ticker in PP. If one of my ETFs was also holding this stock, it would probably be mentioned with ticker ‘EA’ instead in the emmitents holdings list, and therefore they couldn’t be matched to calculate the overall portfolio exposure of this stock. To overcome this its either necessary to maintain a list of multiple tickers per stock, or use a different identifier (e.g. ISIN), but this information isn’t provided in each of the emmitents holdings lists and therefore it needs to be gathered elsewhere.
  2. Most emmitents use custom IDs in their URLs. Therefore it would be necessary to utilize the emmitents search function to the the internal ID (e.g. by ticker or ISIN). I think something like this currently isn’t possible in PP.

How can this be integrated in PP?
As I have some experience in web scraping, so I’d love to have a feature which enables creation of custom scrapers.
Something that enables the following:

  • Perform multiple fetches, parse and store the relevant data in variables (e.g. use emmitents search to get internal ID by ticker/ISIN).
  • Use these variables in subsequent fetches (e.g. fetch holdings by emmitents internal ID), parse/map data (e.g. JSON, CSV or HTML table), convert to classification and store it in the security details.
    I don’t know where this could be integrated in PP, but one way would be to introduce an additional tab in the securitiy details (maybe similar to the ‘Current prices’ tab) or integrating it into the ‘Classifications’ tab, by defining scraping profiles elsewhere and selecting them as data source for the classification.

Edit:
I was also thinking about using GoogleSheets as workaround for the fetching part, and providing holdings data in a uniform way in that spreadsheet, but I think that there currently isn’t a way to import classifications from spreadsheet either.

2 Likes

…off classifications AND regions, please! :smiley:

1 Like

I currently achieve this by using a tool created by bosix from this on going github issue.

its a golang application you run against main.xml and saves the results directly, I mainly use it for region and industry classification but it also has an option for top 10 holdings.

Only thing I had to change to get it working right was updating maps.go to map to English names as it uses a german API to get the names. I have uploaded my English copy of maps.go to save you the time of doing it yourself, Here.

Hope this helps, examples below:

Countries

Industry