My bash scripts to generate HTML tables (onvista.de, cash.ch, finanzen.ch)

Hi everyone

Just want to share my script to generate a HTML-Table which PP can read. It’s mostly for Swiss ETFs since I couldn’t find them anywhere else. I’m not a programmer so be gentle with my programming skills :upside_down_face:

To help the search-engine here are the ETFs I use:

  • SGKB
  • Swisscanto
  • CSIF (VIAC)

I publish them via a Webserver so I can use the “Table on Website” Option and I let the Scripts run via a Cron-Job, so PP can just poll the data when I open it.

For www.onvista.de

Get the URL:

  1. Search for the ETF
  2. Click on Alle Kurse & Handelsplätze
  3. Select Currency and Timerange
  4. Open Developer Settings via F12 and Switch to the Network Tab
  5. Clear the current Log and click on “Download als CSV” on the Website
  6. Now in the Developer Settings you have the Link directly to the Historic Values

Use this Link in the following Script:
Variables to change are:

  • all_url_cash: Array for the URLs
  • arr_isin: ISIN of the ETF
  • path_html: Destination where the HTML-File will be stored
#!/bin/bash

arr_url_cash=(\
"https://api.onvista.de/api/v1/instruments/FUND/18490292/eod_history?idNotation=206747561&range=Y5&startDate=2016-09-03" \
"https://api.onvista.de/api/v1/instruments/FUND/19908300/eod_history?idNotation=22128766&range=Y5&startDate=2016-09-03" \
"https://api.onvista.de/api/v1/instruments/FUND/17827095/eod_history?idNotation=281108121&range=Y5&startDate=2016-09-03" \
"https://api.onvista.de/api/v1/instruments/FUND/17827091/eod_history?idNotation=206747547&range=Y5&startDate=2016-09-03" \
"https://api.onvista.de/api/v1/instruments/FUND/11473305/eod_history?idNotation=10668488&range=Y5&startDate=2016-09-03" \
"https://api.onvista.de/api/v1/instruments/FUND/17827093/eod_history?idNotation=281108113&range=Y5&startDate=2016-09-03" \
)
arr_isin=(\
"CH0033782431" \
"CH0037606552" \
"CH0030849613" \
"CH0030849654" \
"CH0017844686" \
"CH0030849712" \
)

path_html="/home/pi/docker/ha/www/"

for (( y=0;y<${#arr_url_cash[@]};y++ ))
do
	rm "${path_html}${arr_isin[$y]}.html"
	
	historic_full=$(wget -O - "${arr_url_cash[$y]}")
	
	IFS='['
	xcounter=0
	for x in $historic_full
	do
		IFS=','
		for z in $x
		do
			case $xcounter in
			1)
			arr_date+=("$z")
			;;
			3)
			arr_price+=("$z")
			;;
			esac
		done
	((xcounter++))
	done
	
	printf "<table>\n<tr><th>Date.*</th><th>Schluss.*</th></tr>" >> "${path_html}${arr_isin[$y]}.html"
	for (( i=0;i<${#arr_date[@]}-1;i++ ))
	do
		printf "\n<tr><td>$(date -d @${arr_date[$i]//]} +"%Y-%m-%d")</td><td>${arr_price[$i]//]}</td></tr>" >> "${path_html}${arr_isin[$y]}.html"
	done
	printf "\n</table>" >> "${path_html}${arr_isin[$y]}.html"
	
	unset IFS
	unset arr_date
	unset arr_price

done

For https://fonds.cash.ch/

Get the URL:

  1. Search for the ETF
  2. Right Click on Download historische Werte to Copy URL

Use this Link in the following Script:
Variables to change are:

  • all_url_cash: Array for the URLs
  • arr_isin: ISIN of the ETF
  • path_html: Destination where the HTML-File will be stored
#!/bin/bash

arr_url_cash=("https://fonds.cash.ch/funds/excelData/40239" "https://fonds.cash.ch/funds/excelData/87121")
arr_isin=("CH0238046459" "CH0373465365")

path_html="/home/pi/docker/ha/www/"

skip_lines=2
for (( y=0;y<${#arr_url_cash[@]};y++ ))
do
	rm "${arr_isin[$y]}.csv"
	rm "${path_html}${arr_isin[$y]}.html"
	
	wget -O "${arr_isin[$y]}.csv" "${arr_url_cash[$y]}"
	
	print_header=true
	x=0
	
	echo "<table>" >> "${path_html}${arr_isin[$y]}.html";
	
	while read INPUT ; do	
			if (($x >= $skip_lines)); then 
				if $print_header;then
					echo "<tr><th>$INPUT" | sed -e 's/Date/Date.*/g' -e 's/;Chart[[:space:]]Price;/;Schluss.*;/g' -e 's/Price//g' -e 's/;/<\/th><th>/g' -e 's/$/<\/th><\/tr>/' >> "${path_html}${arr_isin[$y]}.html"
					print_header=false
				else
				echo "<tr><td>${INPUT//;/</td><td>}</td></tr>" >> "${path_html}${arr_isin[$y]}.html" ;
				fi
			fi
	((x++))
	done < "${arr_isin[$y]}.csv" ;
	
	echo "</table>" >> "${path_html}${arr_isin[$y]}.html"
	
	rm "${arr_isin[$y]}.csv"
done

For https://www.finanzen.ch

WARNING: This one is really ugly but so far it works :slight_smile:

Get the URL:

Use this Link in the following Script:
Variables to change are:

  • all_url_cash: Array for the URLs
  • arr_isin: ISIN of the ETF
  • path_html: Destination where the HTML-File will be stored
#!/bin/bash

arr_url_cash=(\
"https://www.finanzen.ch/fonds/credit-suisse-index-fund-ch-csif-ch-equity-switzerland-small-mid-zb-ch0110869143" \
"https://www.finanzen.ch/fonds/csif-ch-i-equity-japan-blue-pension-fund-zb-ch0357515474" \
)
arr_isin=(\
"CH0110869143" \
"CH0357515474" \
)

path_html="/home/pi/docker/ha/www/"

for (( y=0;y<${#arr_url_cash[@]};y++ ))
do

	if test -f "${path_html}${arr_isin[$y]}.html"; then	
		sed -i '$d' "${path_html}${arr_isin[$y]}.html"		
	else
		printf "<table>\n<tr><th>Date.*</th><th>Price</th></tr>\n" >> "${path_html}${arr_isin[$y]}.html"	
	fi
	
	historic_full=$(wget -O - "${arr_url_cash[$y]}")
	IFS='/'
	printnextline=false
	
	for x in $historic_full
	do
		
		if $printnextline;then
			case $nextvariable in
				closeprice)
					closeprice=$( printf "${x}" | sed 's/[^0-9.]*//g' | sed -z 's/\n//g')
					;;
				closepricedate)
					closepricedate=$( printf "${x}" | sed 's/[^0-9.]*//g' | sed -z 's/\n//g')
					;;
				currentprice)
					currentprice=$( printf "${x}" | sed 's/[^0-9.]*//g' | sed '$!d' )
					;;
				currentpricedate)
					currentpricedate=$( printf "${x}" | sed 's/[^0-9.]*//g' | sed -z 's/\n//g')
					;;
			esac
			printnextline=false
		fi
		case $x in
			*Vortag*)
			nextvariable="closeprice"
			printnextline=true
			;;
			*Datum*)
			nextvariable="closepricedate"
			printnextline=true
			;;
			*Nettoinventarwert*)
			nextvariable="currentprice"
			printnextline=true
			;;
			*Berichtsstand*)
			nextvariable="currentpricedate"
			printnextline=true
			;;
		esac	
	done
	sed -i "/${closepricedate}/d" "${path_html}${arr_isin[$y]}.html"
	sed -i "/${currentpricedate}/d" "${path_html}${arr_isin[$y]}.html"
	printf "<tr><td>${closepricedate}</td><td>${closeprice}</td></tr>\n" >> "${path_html}${arr_isin[$y]}.html"
	printf "<tr><td>${currentpricedate}</td><td>${currentprice}</td></tr>\n" >> "${path_html}${arr_isin[$y]}.html"
	printf "</table>" >> "${path_html}${arr_isin[$y]}.html"
	unset IFS
done
5 Likes

For Finanzen.ch I take now the current date since Berichtsstand seems not to be the date it last updated the price

#!/bin/bash

arr_url_cash=(\
"https://www.finanzen.ch/fonds/credit-suisse-index-fund-ch-csif-ch-equity-switzerland-small-mid-zb-ch0110869143" \
"https://www.finanzen.ch/fonds/csif-ch-i-equity-japan-blue-pension-fund-zb-ch0357515474" \
)
arr_isin=(\
"CH0110869143" \
"CH0357515474" \
)

path_html="/home/pi/docker/ha/www/"

for (( y=0;y<${#arr_url_cash[@]};y++ ))
do

	if test -f "${path_html}${arr_isin[$y]}.html"; then	
		sed -i '$d' "${path_html}${arr_isin[$y]}.html"		
	else
		printf "<table>\n<tr><th>Date.*</th><th>Price</th></tr>\n" >> "${path_html}${arr_isin[$y]}.html"	
	fi
	
	historic_full=$(wget -O - "${arr_url_cash[$y]}")
	IFS='/'
	printnextline=false
	
	for x in $historic_full
	do
		
		if $printnextline;then
			case $nextvariable in
				closeprice)
					closeprice=$( printf "${x}" | sed 's/[^0-9.]*//g' | sed -z 's/\n//g')
					;;
				closepricedate)
					closepricedate=$( printf "${x}" | sed 's/[^0-9.]*//g' | sed -z 's/\n//g')
					;;
				currentprice)
					currentprice=$( printf "${x}" | sed 's/[^0-9.]*//g' | sed '$!d' )
					currentpricedate=$(date +"%d.%m.%Y")
					;;
			esac
			printnextline=false
		fi
		case $x in
			*Vortag*)
			nextvariable="closeprice"
			printnextline=true
			;;
			*Datum*)
			nextvariable="closepricedate"
			printnextline=true
			;;
			*Nettoinventarwert*)
			nextvariable="currentprice"
			printnextline=true
			;;
		esac	
	done
	sed -i "/${closepricedate}/d" "${path_html}${arr_isin[$y]}.html"
	sed -i "/${currentpricedate}/d" "${path_html}${arr_isin[$y]}.html"
	printf "<tr><td>${closepricedate}</td><td>${closeprice}</td></tr>\n" >> "${path_html}${arr_isin[$y]}.html"
	printf "<tr><td>${currentpricedate}</td><td>${currentprice}</td></tr>\n" >> "${path_html}${arr_isin[$y]}.html"
	printf "</table>" >> "${path_html}${arr_isin[$y]}.html"
	unset IFS
done
1 Like

For Onvista the URL has to be dynamic since it contains the start date and then outputs 5 years from that, so heres how I generate it:

arr_url_cash=(\
"https://api.onvista.de/api/v1/instruments/FUND/18490292/eod_history?idNotation=206747561&range=Y5&startDate=$(date +"%Y-%m-%d" -d "5 years ago")" \
"https://api.onvista.de/api/v1/instruments/FUND/19908300/eod_history?idNotation=22128766&range=Y5&startDate=$(date +"%Y-%m-%d" -d "5 years ago")" \
"https://api.onvista.de/api/v1/instruments/FUND/17827095/eod_history?idNotation=281108121&range=Y5&startDate=$(date +"%Y-%m-%d" -d "5 years ago")" \
"https://api.onvista.de/api/v1/instruments/FUND/17827091/eod_history?idNotation=206747547&range=Y5&startDate=$(date +"%Y-%m-%d" -d "5 years ago")" \
"https://api.onvista.de/api/v1/instruments/FUND/11473305/eod_history?idNotation=10668488&range=Y5&startDate=$(date +"%Y-%m-%d" -d "5 years ago")" \
"https://api.onvista.de/api/v1/instruments/FUND/17827093/eod_history?idNotation=281108113&range=Y5&startDate=$(date +"%Y-%m-%d" -d "5 years ago")" \
)

Thank you!!

I’ve done the same with Python. Also runs as a Cronjob. :slightly_smiling_face:

is the cron-job and webserver on the same local machine as PP? Or do we need a server to do this?

no hardware is required, a webservice on localhost is sufficiant

1 Like

For Onvista and cash.ch the Webserver and Script only need to run when you have PP open, so they can be on the same machine. This is because they offer historical data.
For Finanzen.ch the script needs to run at least once a day if you want to have historical data since the website doesn’'t offer historical data. But here as well the Webserver doesn’t need to run when PP is not open so it can be on the same machine.

1 Like

A bit more detail on what I’ve done:

  • A small Python script that essentially does the same as the OP’s bash scripts: Read data from a CSV, convert to HTML, then save as a HTML file.
  • My script runs on a web server and is executed every night by a cronjob.
  • I don’t open PP everyday, but the workload on the server is so minimal that I just let it run every night anyway.
  • PP then uses the “table on a website” option to read the HTML table from the web server.

What I had briefly considered was a Python script that runs locally, before I start PP, that writes the data directly into the XML file that PP uses, where historical data is stored by PP. Maybe that’s a project for another day.