Skip to content

Changes to Data Warehouse Upload

by on December 27, 2013

Hi,

We are extremely grateful to all the groups and individuals who have uploaded data to the warehouse and the totals on the upload ranking page are quite amazing.

Since launch we have been monitoring the service which receives, processes and stores that data and now are about to make some changes to it to improve it performance and reliablity.

The first change is to introduce an acceptance time window for packets uploaded from the Dashboard. This restricts packets to those whose sequence numbers are within +/- 48 hours of the latest sequence number stored in the warehouse. We are doing this for two reasons:

  1. We have received erroneous packets whose sequence numbers are significantly different to the latest value.
  2. To reduce the chance of double scoring of points when we release stored data back to the user community

The second change, which will not affect normal operation but will improve scoring:

  • We currently process packets immediaely they arrive at the server. Under load, this can cause us to lose a few packet scores because of contention in the database. To alleviate this problem, we are going to change the processing slightly to process packets through a buffer. This buffer will be processed every 5 seconds, so there will be a slight delay
    before packets are acknowledged on the ranking.

73

– Dave, G4DPZ

Advertisements

From → News

10 Comments
  1. Marsh Parker permalink

    This is Marsh, NC7V in DM13. Since loading the new version of the Dashboard, I have not been able to capture any telemetry. I have observed the pass on 12/27 at 19:35:12 start at my location and I obtained a lock on and off during the pass. I had lock a couple of times of at least one minute. No different than when using the older version. I have not decoded anything yet with the new version. I am stuck at 513 frames uploaded. Can I try to install the earlier version? I can not find a copy on my system now.

    73, Marsh, NC7V

  2. Alex Deligiannis permalink

    Hi,

    The best is to have a list with the real time uploaded data to the warehouse.

    Happy New Year

    Alex

    • Please give more detail about your thoughts. We upload RealTime data to the warehouse with every packet.

      73

      – Dave

  3. Alex Deligiannis permalink

    My comment was for the Upload Ranking, at least the upper part of the scoring list is fake.
    A list of call sings with the real time uploaded packets will be better for us.
    73s
    Alex SV8QG

    • Mark permalink

      Alex, why do you think the upper part of the Upload Ranking list is fake? I can assure you mine are real packets from my station at N8MH!

      73,
      Mark N8MH

  4. Alex Deligiannis permalink

    Hi Mike

    Your rate is about 280 packets per day till now and the top scorer has a rate of 630 that means more than double of yours. If the top scorer has 100 % decoded packets (unreasonable) you had lost the half available satellite time.

    Can you please tell me the average number of packets you receive during the high power (day passes) and how many during the eclipse? Mine are 80 – 120 (morning) and 20 -25 (eclipse), receiving conditions here, Turnstyle, 15m RG214, dual cavity filter and Funcube pro.

    73s
    Alex

  5. Mark permalink

    Alex, remember the latitude of the station matters also. For Funcube-1, the closer to the pole (north or south) the more passes and time above the horizon. I suspect I am getting about 50-75% of the packets I could get every day because of trees, the house, and the very high noise near the 2M frequency downlink. So, I can easily see how somebody could get 2-3X as many packets as me–better station, better view to horizon, and more northern/southern latitude so more passes each day.

    You can test the latitude out with tracking programs. You will see what I mean. So I think it’s real data they are collecting.

    I have a 7 element directional array (automated azimuth/elevation control) and very good preamp for receiving. But I could easily do better with less noise, less trees, etc.

    73!

    Mark N8MH

  6. g4dpz permalink

    Hi

    For me at lat 52N the best case would be about 33k realtime packets since launch.

    So the values are reasonable for a well equipped oscar class station.

    73

    Dave, g4dpz
    FUNcube Data Warehouse Dev.

  7. Yes they are reasonable. I have not collected since I went away for Christmas so am slipping down the table rather quickly. I used to get 100 or so packets on a good high power pass, fewer in the evenings. I have a lot of noise in some directions, hills and trees and as I live in the UK* I only have a 7 element crossed yagi with pre-amp.

    Mike G0MJW

    * In the UK towers and yagi antennas are not considered beautiful. Strange but true.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: