As the previous speakers have already said, the JSON structure simply doesn’t fit.
As a workaround, you could use a proxy script that delivers the historical CSV data as JSON. As an example, I have created a Golang script here (the use of “curl --compressed” works for me on Linux, no idea how it is on other OSs).
tsp_share_price_history.go
package main
import (
"log"
"net/http"
"fmt"
"os/exec"
"encoding/json"
"strings"
"regexp"
)
var nlines int = 0
type Record struct {
Date string `json:"date"`
LIncome string `json:"LIncome"`
L2025 string `json:"L2025"`
L2030 string `json:"L2030"`
L2035 string `json:"L2035"`
L2040 string `json:"L2040"`
L2045 string `json:"L2045"`
L2050 string `json:"L2050"`
L2055 string `json:"L2055"`
L2060 string `json:"L2060"`
L2065 string `json:"L2065"`
GFund string `json:"GFund"`
FFund string `json:"FFund"`
CFund string `json:"CFund"`
SFund string `json:"SFund"`
IFund string `json:"IFund"`
}
func gethistory(w http.ResponseWriter, req *http.Request) {
getjson := "curl --compressed -H \"User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:120.0) Gecko/20100101 Firefox/125.0\" -H \"Accept: */*\" -H \"Accept-Language: de-DE,en-US;q=0.7,en;q=0.3\" -H \"Accept-Encoding: gzip, deflate, br\" -H \"Referer: https://www.tsp.gov/share-price-history/\" -H \"X-Requested-With: XMLHttpRequest\" -H \"DNT: 1\" -H \"Sec-Fetch-Dest: empty\" -H \"Sec-Fetch-Mode: cors\" -H \"Sec-Fetch-Site: same-origin\" -H \"TE: trailers\" \"https://www.tsp.gov/data/fund-price-history.csv\""
body, err := exec.Command("bash", "-c", getjson).Output()
if err != nil {
log.Fatalln(err)
}
csvstring := string(body)
records := []Record{}
lines := strings.Split(csvstring, "\n")
for key, _ := range lines[0:] {
nlines = key
}
// L Income,L 2025,L 2030,L 2035,L 2040,L 2045,L 2050,L 2055,L 2060,L 2065,G Fund,F Fund,C Fund,S Fund,I Fund
for _, line := range lines[1:nlines] {
fields := strings.Split(line, ",")
if fields[0] == "" {
continue
}
date := fields[0]
LIncome := fields[1]
L2025 := fields[2]
L2030 := fields[3]
L2035 := fields[4]
L2040 := fields[5]
L2045 := fields[6]
L2050 := fields[7]
L2055 := fields[8]
L2060 := fields[9]
L2065 := fields[10]
GFund := fields[11]
FFund := fields[12]
CFund := fields[13]
SFund := fields[14]
IFund := fields[15]
records = append(records, Record{
Date: date,
LIncome: LIncome,
L2025: L2025,
L2030: L2030,
L2035: L2035,
L2040: L2040,
L2045: L2045,
L2050: L2050,
L2055: L2055,
L2060: L2060,
L2065: L2065,
GFund: GFund,
FFund: FFund,
CFund: CFund,
SFund: SFund,
IFund: IFund,
})
}
jsonData, err := json.Marshal(records)
if err != nil {
panic(err)
}
nonullspleaseRegexp := regexp.MustCompile(`:\"\"`)
newbody := nonullspleaseRegexp.ReplaceAllString(string(jsonData), ":\"0\"")
fmt.Fprintf(w, string(newbody))
}
func headers(w http.ResponseWriter, req *http.Request) {
for name, headers := range req.Header {
for _, h := range headers {
fmt.Fprintf(w, "%v: %v\n", name, h)
}
}
}
func main() {
http.HandleFunc("/", gethistory)
http.HandleFunc("/headers", headers)
http.ListenAndServe(":8083", nil)
}
Start the script with
go run tsp_share_price_history.go
and let it run in the background while PP is running.
You can access the script via a browser, or also via PP
ă…¤
ă…¤
ă…¤
TSP: F Fund for example
http://localhost:8083/gethistory
$[*].date
$[*].FFund
ă…¤
ă…¤
ă…¤
TSP: L Income for example
http://localhost:8083/gethistory
$[*].date
$[*].LIncome
I just cobbled the script together quickly. If you want to query more than three or four funds in PP, you should consider whether it would be better to prevent the csv from being fetched multiple times (buffer CSV data). I didn’t feel like doing that now.
If you also want to have the data in the mobile app, the script must of course run on an accessible server, otherwise the machine where PP is also running, or a NAS, or something like that, is sufficient.
