Re: need help: have issues with gpsd-client under python

[Beta Tester]

no, the email button isn't working for me

> Gesendet: Donnerstag, 30. Juni 2022 um 21:48 Uhr
> Von: "Beta Tester" <alpha-beta-release@gmx.net>
> An: 20220630102045.0dcde6b0@spidey.rellim.com
> Betreff: Re: need help: have issues with gpsd-client under python
>
> hi Gary,
>
> for some reason some emails arrivey me with a huge delay.
> this email i still not got yet - not in inbox nor in spam folder.
> i try to use copy-n-paste from the thread index of the mailing list.
>
>
> On Thu, 30 Jun 2022 06:52:56 +0200
> Beta Tester <alpha-beta-release@gmx.net> wrote:
>
> > > hi, i have several issues with gpsd under python.
> > > i have two Raspberry Pi Zero W (rev.1.3) acting as gpsd-server (via
> > > WiFi, HOST_1:192.168.1.7 & HOST_2:192.168.1.8) with an GPS device on
> > > each connected via serial port on GPIO. and i have a Raspberry Pi 4
> > > acting as gpsd-client (via LAN-cable, IP:192.168.1.2) to collect and
> > > store gps data from both gpsd-servers.
>
> > You neglect to mention your distribution, your gps decive,
> > versions of gpsd and python.
>
> distro is RasPi OS Bullseye (newest version and updated).
> Linux zero1 5.15.32+ #1538 Thu Mar 31 19:37:58 BST 2022 armv6l GNU/Linux
> Linux zero2 5.15.32+ #1538 Thu Mar 31 19:37:58 BST 2022 armv6l GNU/Linux
> Linux rpi4 5.15.32-v7l+ #1538 SMP Thu Mar 31 19:39:41 BST 2022 armv7l 
> GNU/Linux
>
> gpsd versions tried:
> gpsd 3.22 from distro
> gpsd release-3.22, release-3.23, release-3.24, master (3.24.1-dev) compiled 
> from source (on each gpsd-server/-client the same version)
>
> Python 3.9.2
>
>
> > > the gpsd-client is made in python:
> > > two threads are running `gps.gpd(host=HOST_1)`,
> > > `gps.gpd(host=HOST_2)` to collect gps data of interest (`TPV`,`SKY`)
>
> > Python sucks at threading, so I hope you dont really mean threading.
> > The python gpsd module is not tested for threading either.
>
> unfortunately i am using threadding.Thread(). i also tried 
> multiprocessing.Process(), but that was even more a fail.
> is there something else i can use from the console that let me run multiple 
> gps.gps() sessions "parallel" ?
>
>
> > > and make a copy of it (with transforming nested dictwrapper to dict)
> > > and put the data to a queue. another thread is taking the data from
> > > the queue and store them to a in-memory database first and every 10
> > > minutes it store the data from the in-memory database to a file and
> > > clean the in-memory database and it starts a script that uses the
> > > data from the database file to publish the data for a local webpage.
>
> > Weird.
>
> that ma way i go.
>
>
> > > everything works well except:
> > >
> > > 1. `gps.next()` seems to get crazy from time to time in two ways.
> > >    case 1: `gps.next()` blocks forever and never returns.
> > >    case 2: `gpd.next()` returns the same gps record repeatly over and
> > > over.
>
> > I think you said later that you had a bad conenction, causing that?
>
> gotcha, i have a poor WiFi connection.
>
>
> > > 2. because `gps.next()` or `gps.read()` blocks the thread, is there a
> > > way to set a timeout to get back control to the loop?
>
> > Yeah, don't do that!  That is whay gps_waiting() is for.
>
> yes, i am using it now.
>
>
> > > 3. when starting with `gps.gps(reconnect=True)`, when does a
> > > reconnect happen? i cannot see any difference between reconnect
> > > enabled or disabled. e.g., the gpsd-servers powers off and powers on
> > > again, gps.gps(reconnect=True) does not come back receiving gps data.
>
> > Souds like a missing "-n", but later you says you use "-n"
>
> i the tests i used the -n but in the normal use i forgot that option.
> maybe it was going under may other silly sleep() issues.
> i will add -n to the normal start as well.
>
>
> > > 4. not every time i get a `StopIteration` exception, then the
> > > gpsd-server goes down while `gps.next()`.
>
> > gps.next() is not from us.
>
> shall i use gps.read() instead?
>
>
> > on the gpsd-server on the RPi ZeroW i run `gpsd -D 5 -G -N -n -s
> > 115200 /dev/ttyAMA0 2>&1 | tee /tmp/test-gpsd.txt` as root
>
> The hard way....
>
> > do i something wrong in the two gpsd-client threads?
>
> You are on your own with Python threads.
>
> > maybe the RPi Zero W with its single core CPU is too weak to handle a
> > gpsd-client?
>
> Nope.
>
> > > the link to the python source code i use, in case someone will watch
> > > the code to point me, what i am doing wrong there:
> > > https://github.com/beta-tester/python-gpsd-client/blob/main/gps_issue.py
>
> > what is IFNULL()???
>
> if there is a row already existing with that key, i solve the conflict with 
> looking if there is a NULL in the column, if NULL, then take the new value 
> from the gps record, otherwise keep what is in the column in the database.
>
>
> > You are not always checking that items are set before trying to save
> > them to your DB.
>
> i use the conflict mechanism of the database - add new row is not already 
> exist, if already exist, fill the missing columns.
>
>
> > You are not collecting data per epoch, instead you are over writing data.
>
> is there a timestamp from which epoch the data is?
>
>
> > Here you are forgetting the PRN:
> >    INSERT INTO data_gps_sky_map (host, el, az, ss)
>
> no i don't. it is the heatmap. there is no interest to know which PRN is how 
> strong. i just want to know how is the sky view.
> if i neef PRN, then i can reconstruct it from the data_gps_sky table.
>
>
> > Never, ever, sleep, Use gps_waiting():
> >                 sleep(SLEEP_LOOP)
>
> > No one ever, ever, said the timestamps would be unique!
> >    # DEBUG: detect running crazy (then receiving same timestamp over and 
> > over)
>
> when the timestamp is exact the same for a very long time, then there is 
> something wrong... that was my thought.
> anf in gps.gps(), received gives time.time().
>
>
> > They are no epmpty, just not what you expected:
> >    # gpsd 3.24+ send lots of empty SKY records
>
> ya, i am interested in satellites. that list is empty.
>
>
> > > PS.: i know `gpsplot` exists. i just want to know what is going wrong
> > > in my application.
>
> > A lot is wrong.
>
> ya i know now... before i was guessing...
>
>
> PS.: i hope this email will go the right way. the email address is not the 
> usual gpsd-users one.