Hey @Alfons1Qvor12 , you were right. Actually your description (“(the id of length 10 after the “?id=”, something like 0P00012345)”) led me to the assumption that we need the id attribute…
Now I tested with “normal” xml and it worked!
What I also noticed is that if I rename a classification it get’s re-created freshly when I run the script again. It would be nice to make this configurable as I would like to give them special names so that I see that they are the automatic classifications from your script/morningstar (I did it with a prefix “ac_”).
But so far - nice work! Thanks a ton!
@AndreasB , this test brought up another feature request towards PP itself: It would be nice if the classifications can be sorted or ordered via drag’n’drop. Don’t know if this is already possible somehow (like in some config files), but at least I don’t see the option in the GUI.
PS: An I don’t know where to put my feature requests… Is this the right way?
That is implemented. Have you tried? There is no “visible” option for drag’n’dop other than start dragging a classification or instrument in the tree views.
BTW: Is there a tool/way to fill the Additional Attributes in PP (Logo, Gesamtkostenquote (TER), Fondsgröße, Anbieter, …)? In principle, some of them could also be retrieved from Morningstar.
I’m not familiar with any. That would be a nice one ;).
Hi @Alfons1Qvor12 . I just tried the import (based on the new-api branch) , looking great so far! Thanks a bunch for all the work you put in this great tool!
For my two mixed funds ARERO (ISIN LU0360863863) and Xtrackers Portfolio (LU0397221945) I don’t quite understand some of the classifications compared to the values shown on the (German) Morningstar homepage.
For example ARERO:
After the import Portfolio Performance shows a region distribution of 38,75% for Europe Developed while the Morningstar homepage (under Portfolio/Engagement/Region) states 17,75% for “Europa, Industrieländer”.
Country according to PP consists of 25,05% for Germany, compared to 3,31% from Morningstar.
The asset allocation is also a strange beast. PP shows an Asset Type distribution of 59,94% Stocks and 54,79% Bonds. While this reasonates with the Morningstar Classic Asset Allocation the total sum is obviously more than 100%.
Switching from Classic to “Wirtschaftliches Engagement” or “Marktwert” on Morningstar, the distribution values seem to make more sense.
Actually, it is not a bug, it is a feature ;-). I haven’t documented it yet properly, but the new-api branch now also considers bonds in the classification. On top, I am still struggling with normalising data with respect to asset allocation (long/short/net and negative numbers from Morningstar site).
The Morningstar homepage tells you that you have 3,31% in Germany, they actually mean that 3,31% of your shares are assigned to Germany. They don’t talk about your bonds. Also the script on the main branch only considers equity and then PP shows the percentages within the classified data (like Morningstar) or the percentage of the portofilo (which then shows that you have less than 2% of equity in Germany).
If you want to have that behaviour with the new-api branch, please use the command line option -equity_only.
Without that option, the script now also considers your German bonds (which alone are already contributing 20% of the total).
If you want to distinguish German equity and German bonds, then please use the command line option -bond_segregation, then you will get two classification entries in the Region and Country taxonomies. One for equity (“Germany”) and one for bonds (“Germany (Bonds)”).
I am using “net” numbers from Morningstar and the script ignores negative values. If this adds up to values higher than 100%, the script is still struggling. This didn’t show so clearly in the old script without considering bonds and the old script also normalised asset allocation differently (based on “long” allocation), but also there, country and region allocation sometimes exceed share of equity. Thanks for the ISINs. I will use them to think about how to address the >100% issue.
(P.S.: I thought about it and will make -equity_only the default behaviour in order to avoid confusion. I will replace it with something like -bonds to allow dedicated activativation of classification of bonds).
@Alfons1Qvor12 Thanks again for your detailed answer! That makes totally sense given the distribution between bonds and stocks.
If I’m using the -equity_only option, the distribution values for the provided funds are displayed according to the stock distribution. Very nice!
Now, on to my next problem… While this solution works for the ISINs provided, for example my iShares € High Yield Corp Bond (ISIN IE00B66F4759), which focusses on bonds, won’t get any data like Bond Style or Bond Sector. These values are only filled in if I omit the -equity_only option (which again makes sense).
Any chance to somehow combine these those options so that I would get all the classifications for both bonds and stocks including mixed funds ?
If I understood you correctly I would have to call your script twice - initial run with -equity_only , and then another run based on the previous outcome with the (not yet existing) -bonds option.
I also tried the -bond_seg option but I’m honestly that much of a fan of it. The values provided are correct, yet I want to see the whole regional distribution of all my assets no matter the asset type (meaning no differentiation between “Germany” and “Germany (Bonds)”).
Not sure, if I understand your question. If you run the new script in its current form without the -equity_only option, it fills both equity-related and bond-related taxonomies. So you get Bond Style and Bond Sector from the bonds of your mixed funds as well as Stock Style and Stock Sector from the equity of your mixed fund. Also in future, I plan to keep that behaviour with the -bonds option. So -bonds will just mean that in addition to equity also values for bonds will be retrieved (which will be similar behaviour as for the -stocks option).
And Country and Region will either be for equity only, or for the sum of equity and bonds (or if wanted, separated in different entries via -bond_seg option).
Hope that helps.
(I don’t plan to introduce an option where equity is not updated but bonds are. And I don’t have clean-up functions yet to remove already added taxonomies or classifications again.)
BTW: LU0360863863 (ARE) and LU0397221945 (XTP) are really tough in combination.
At first glance, everything is perfect for LU0397221945 (XTP). 47.17% Stocks, 50.58% Bonds. Total of 100% together with 1.07% Cash and 1.18% Other.
Then I look at LU0360863863 (ARE) „Classic“ aka „MorningStarDefault“ Asset Allocation Breakdown: 59.94% Stocks. 54.79% Bonds. -14,79% Cash (and 0.07% Others). I ask myself how to translate this so that Stocks plus Bonds is smaller or equal 100%? Alternative views based on „Economic Exposure“ and „Market Value“ look promising. They have a much smaller value for Bonds. So e.g. 59.64% for Stocks plus 39.02% for Bonds plus 0.74% for Cash in „Economic Exposure“ look quite ok.
Then I go back to LU0397221945 (XTP). „Economic Exposure“ there is: 82.73% for Stocks and 16.73% for Bonds. Compared to 47.17% Stocks and 50.58% Bonds in „Classic“ view, this is shockingly different. I must say that I am lost, but I think I should stay away from those alternative views, especially as I believe that the other breakdown values are all relative to the MorningStarDefault view.
So how to translate LU0360863863 (ARE) Breakdown of 59.94% Stocks. 54.79% Bonds. -14,79% Cash so that Stocks plus Bonds is smaller or equal 100% remains open.
And there is another issue with LU0360863863 (ARE). The Bond Sector breakdown does not only identify 52.96% of Government Bonds (which is almost the same value as the overall share of bonds), but in addition indicates 22.1% Derivatives. Not sure, if they belong to Bond Sector at all and in any case, they are possibly the root cause of this whole confusion.
Please comment, if you have an idea how to address the issues with LU0360863863 (ARE) asset allocation breakdown.
So while the stocks value is correct, bonds are somewhat off. The numbers, even if you substract one from another, don’t add up.
I also can’t see any indication of derivates.
Honestly, I wouldn’t put too much effort into this. If you can’t rely on the figures from Morningstar I don’t see added value in a specific calculation for such rare edge cases.
There is one thing which I need to correct in the new script after looking at these issues. I normalized the values for PP according to net value of asset alloaction. This was very convenient, because it often avoids problems with sums larger than 100%. However, the Morningstar site explicitly says that their values (which add up to 100%) are the share of the long positions of equity or bonds, not the share of the net positions. On top, there is something wrong with the scaling for bond sector. Government Bonds plus Derivatives for LU0360863863 (ARE) should sum up to the share of bonds, not to much higher value as it currently does***. So in the end, I will have to address both together: correct scaling for both equity and bonds in the presence of long values which sum up to values larger than 100% (accompanied by negative cash values).
Thanks a lot @tbx for checking and your reply. I fear if I switch back to long values it will not be a corner case anymore. But you are right, I will not spend too much time on this ARE case. I will address the issue of handling long values correctly, and if this solved the ARE issue as well, it is fine. Otherwise, it is as it is in the numbers provided by Morningstar.
***) UPDATE: This is caused by the negative cash. Here some debugging info: calculate_grouping ['Government', 'Cash', 'Derivative'] [96.657589, -36.989978, 40.332388] Bond Sector 0.5478721
UPDATE2: I have to correct myself. There is no need to change from net values to long values. The current version of the script is consistently based on net values. It can easily be changed to use of long values, but there is nothing wrong with using net values as far as I can see. It is just different from what MorningStar provides on their snapshot page. I asked earlier in this thread, if anybody has a strong opinion on that, but up to now, I think that both approaches are in principle ok and in most cases, they are anyway not so different from each other. On the other hand, I have to admit that at least for this ARE case and Bond Sector, the long values are much more convenient: calculate_grouping ['Government', 'Cash', 'Derivative'] [69.566479, 1.405462, 29.028059] Bond Sector 0.5478721 . (Still there is no solution to having Asset Allocation >> 100% which is much more frequent for long).
Currently, the script only modifies taxonomies. Is does not touch any other parts of the data. As said, in principle it would be feasible to also fill/modify other parts of the xml with data than can be retreived from Morningstar (e.g. name of the security or maybe TER). However, this is not on top of my to-do list currently and when it comes to logo information, I would not be aware that it is available from Morningstar API.
Technically, I see the following challenges/tasks:
Find a site which provides the all the logos based on ISIN and which allows download for personal use (this might be a difficult legal issue). One could e.g. ask the guys of Divvydiary, if it is ok to use https://divvydiary.com/logos/{ISIN} as a source.
Use python to convert from the source format to png. I am not at all an expert on this and I don’t know, if e.g. svg to png conversion is easily available across all platforms.
Create data:image/png;base64 string from png and insert it into PP xml as logo. (However, I have no idea, what PP expects here in terms of e.g. size).
Frankly, I am not very interested in doing this, because I don’t use logos in PP myself. And manually adding them should also be a quite fast as easy one time job.
I see that they are used in lists in combination with the name. But is there any other use for them?
Hi @Alfons1Qvor12 . I don’t have any strong opinion (yet) on the usage of net vs long values. In my case the values provided are correct for most of my assets, for those with inconsistencies I can easily adjust the classifications by myself.
My question is more related to the handling of updates once initial classifications are provided. Let’s assume I run your script, take the classifications as provided and adjust e.g. the values for ARE manually. The next time I use your script I would assume these manual changes will be overriden by the values from Morningstar.
So I think for me it all boils down to - is there a way (e.g. through a command line option in the script or an attribute in PP) to exlude specific assets from being updated so I don’t have to adjust the values every time I ran the script?
Yes, I was thinking along the same lines. Those products are just too complex to fit into a simple mapping scheme. I found an old paper from Morningstar (Methodology_ShortsDerivativesPortStats.pdf) which gives receommendations on how to display such information and which explains that there is good reason to show long and short and not just one of them (or just net) for complex financial products.
Based on this, I plan to proceed as follows: I will stick to net values which in my opinion are appropriate for many funds and etfs. For those which have negative values in net values, I will add an additional warning and a recommendation to set them manually. And then I will add support for a token in the PP notes field (similar to “#PPC:[ISIN2=XY0011223344]”) which will take the whole security out of the automatic updating (maybe “#PPC:skip”).
While this solution works for the ISINs provided, for example my iShares € High Yield Corp Bond (ISIN IE00B66F4759), which focusses on bonds, won’t get any data like Bond Style or Bond Sector. These values are only filled in if I omit the -equity_only option (which again makes sense).