Well, it felt a bit sluggish to me at various points throughout the day when I would be opening various pages, like refreshing the schedule or opening a game's submission page. That might just be a problem on my end though.
There are three big issues I have with your approach that you don't seem to be grasping. A) Web/browser apps are going to have more latency than something running locally. Period. There is no way around this. Having to tweak a million settings will be a lot more responsive with a local app especially since, again, the server only needs to be kept relatively in sync. B) You're creating an earth mover when all we need is a hatchet and a shovel. That's an invitation for security holes and instability with something we're going to use *maybe* twice a year. There is NO REASON for our donation tracking app to need to handle USER REGISTRATIONS when there are going to be a handful of machine that even need anything other than read-only access, and that can be handled by MySQL or whatever we end up using for the database. Which leads me into C) I don't trust anything you write.
There are three big issues I have with your approach that you don't seem to be grasping. A) Web/browser apps are going to have more latency than something running locally. Period. There is no way around this. Having to tweak a million settings will be a lot more responsive with a local app especially since, again, the server only needs to be kept relatively in sync.
If you are so kind to bring this forward, then make the damn app yourself. Make it happen.
Quote:
B) You're creating an earth mover when all we need is a hatchet and a shovel. That's an invitation for security holes and instability with something we're going to use *maybe* twice a year. There is NO REASON for our donation tracking app to need to handle USER REGISTRATIONS when there are going to be a handful of machine that even need anything other than read-only access, and that can be handled by MySQL or whatever we end up using for the database.
WE, as it stands now, myself and SMK, are planning on making a donation system that will be automatic that save time for those involved. This apparently has been requested. Even by mike. We need more than a hatchet and shovel. Get that into your head. No one has SAID that it will handle user registrations because that detail hasn't been finalized. I will not write this myself. I will be collaborating with SMK. That is why I put up suggestions, which can then be discussed and finalized. I am not even sure we need user registrations.
Quote:
Which leads me into C) I don't trust anything you write.
Er... why does the web app have to have administrative features? Can't you split it up? The web app accepts donations, and a local administrative app can interface with the database in any way required.
The real mystery behind Mystery is why he ever thought he had my support in the first place. Or maybe it's how he manages to be more stubborn and pigheaded with every successive post. There's just so much to the guy there's an enigma wrapped around the mystery!
Serious note: I'm watching you Mystery. Consider this your warning. And if you're wondering what the warning is about, let's just say it's a mystery.
I have to agree; there's no need to be anything but rational and pragmatic about what is being brought up here. We all (hopefully) have the same goal, which is basically do a better job than last year.
Anyways, I just finished moving into a new place for, like, 4th time in less that 6 months, so I don't really feel like addressing anything in-depth right now. All I can say is:
- Trying to pull information from chip-in was getting sluggish towards the end as well. No matter how we get donations, they're going to have to come from the internet somehow. - It is possible to run web-based applications locally or over a LAN with no external access. - It is also possible to hook up a local application to a remote database. - WE, as it stands now, is everyone replying to this thread, because WE all have a stake in how any system WE create will perform.
Er... why does the web app have to have administrative features? Can't you split it up? The web app accepts donations, and a local administrative app can interface with the database in any way required.
No one has decided it should yet. It's a possibility. I'm just giving suggestions.
Quote from mikwuyma:
The real mystery behind Mystery is why he ever thought he had my support in the first place. Or maybe it's how he manages to be more stubborn and pigheaded with every successive post. There's just so much to the guy there's an enigma wrapped around the mystery!
Serious note: I'm watching you Mystery. Consider this your warning. And if you're wondering what the warning is about, let's just say it's a mystery.
I never said support. I meant you wanted automation. No manual emails. That seems to have led to a disaster this year. And uranium anchor is basically just suggesting to dumb it down and do it all manually. Plus the guy is just extremely rude and unfriendly, so I just wish to return that in kind.
I am open to suggestions. I have been pondering over the whole web problem. But suffice to say, there have been but a very few suggestions about what to do about it now. Before we do anything, we need a concrete plan. A local app will create extra complexity. Running it on the server might cause problems if the connections fail. Really, I an open to both ideas so long as we can discuss it friendly.
EDIT: So let me sum this up. Lag.com first comes in with a very real concern. A valid one. I take note. Then Uranium comes in and smears it in my face. Nice of you. I respond in turn by being rude. People then agree that it is a concern. I suggest one option to solve this. Then Uranium comes in again and smears it in my face by stating I am reinventing the wheel and stating of not trusting me. I reply in turn rudely saying that we need to reinvent the wheel to make it automatic. I even add that mike wants automation (which is how I understood it, apologies if wrong) since he vented on emails missing. Those emails were either forgotten or skipped in the high volume period of donations. So anyhow, now we get to where we are. It is not unreasonable to see why I get upset with Uranium.
Anyhow... Can we just stop this bickering and get back to discussing the marathon app? It's not fun to wake up every morning to respond to insults and defend oneself.
We have a problem. Lag.com brought up a troubling problem. We need to solve it. There are multiple ways. One way is to run the script locally on a server on the marathon. If anyone is for this idea and think it is plausible, would anyone care to test it? We'd basically just need a computer with Apache or IIS and MySql/PHP. WAMP is one good alternative, at least on Windows. Another alternative is to try to isolate a local app from the web by performing all the work in the app and the app will sync with the server every X minutes. This brings the problem of the actual synchronization. Is it plausible? Is it simple? Can it be done without too much work? Will it be better than the first alternative? Those are the things we need to test. Any other ideas? Please do mention them.
Oh I think we know exactly what happened, you don't need to spell it out for anybody. I wish this forum had a fucking ignore button so I wouldn't feel compelled to respond to your idiocy.
But to respond to your post in a more reasonable way...
Quote from Mystery:
I am open to suggestions. I have been pondering over the whole web problem. But suffice to say, there have been but a very few suggestions about what to do about it now. Before we do anything, we need a concrete plan. A local app will create extra complexity. Running it on the server might cause problems if the connections fail. Really, I an open to both ideas so long as we can discuss it friendly.
I'll repeat myself again since you seem to think this doesn't qualify as a suggestion: Put up a web page that reads from the database and displays current bid wars. This is incredibly trivial if you know the slightest thing about SQL syntax. If the database can only be written to from a machine at the marathon, this makes security also fairly trivial and completely eliminates the need for any kind of user registration. The only remaining thing it needs to do from a user perspective is handle donations, which only requires sanitizing user input in case anybody gets any bright DROP TABLE ideas.
Quote from Mystery:
We have a problem. Lag.com brought up a troubling problem. We need to solve it. There are multiple ways. One way is to run the script locally on a server on the marathon. If anyone is for this idea and think it is plausible, would anyone care to test it? We'd basically just need a computer with Apache or IIS and MySql/PHP. WAMP is one good alternative, at least on Windows. Another alternative is to try to isolate a local app from the web by performing all the work in the app and the app will sync with the server every X minutes. This brings the problem of the actual synchronization. Is it plausible? Is it simple? Can it be done without too much work? Will it be better than the first alternative? Those are the things we need to test. Any other ideas? Please do mention them.
The only thing that needs to be a script is, again, the user/donor side of it. Everything else can and should be written as a local app if only for speed. Web apps are for things that expect dozens or hundreds of people to need access to, and most of the donation administration does not fall into that category in the slightest. Especially since we already have a fairly complete Java app already. Java app writes to database, web app reads from database, sync problem solved.
Any more complexity is just needless bells and whistles on a system that MIGHT get used twice a year.
The only thing that needs to be a script is, again, the user/donor side of it. Everything else can and should be written as a local app if only for speed. Web apps are for things that expect dozens or hundreds of people to need access to, and most of the donation administration does not fall into that category in the slightest. Especially since we already have a fairly complete Java app already. Java app writes to database, web app reads from database, sync problem solved.
Any more complexity is just needless bells and whistles on a system that MIGHT get used twice a year.
You realize that the web system needs to write to a database too, for all the incoming donations, right? And if we do make it automatic (not just a comment field), it probably needs to be of roughly the same database structure as the local app. So I think it is fairly (don't take my word for it, though) safe to say that the local app needs to talk to the web server. Problem not solved. More details are required to fix this.
If we write our own donation interface, then actually, yes, it would need to write new entries to *A* database. It doesn't necessarily have to be the same database, if we wanted to be extra paranoid. Or even a flat file somewhere. Or if we wanted to get super crazy it could send emails.
I feel somewhat hesitant reviving this topic, but I feel like its been left somewhat up in the air.
I guess this is specifically directed to people who were on the job during JRDQ: do you think this is worth investing more time into, given that you did okay without it?
I don't mind spending time working on this in the least; for me, its a chance to improve my own programming skill.
An app would've made things much easier, and in the case of both on-site and online marathons, it would be much easier for donation trackers to keep up with everything. It's definitely something you can work on expanding, and perhaps set up to be used online in case we ever have a similar marathon in the future (I think we were looking at like, 1 per year like this now that this one went well).
While I don't have experience with the app to compare working on the spreadsheet to, to say that we handled donation tracking OK without it is kinda dodging the issue. There were definitely varying levels of quality control going on: at one stage I sorted the prize spreadsheets alphabetically to pick out duplicates (and insert missed donations) in order to ensure the draws were fair. In a few cases I spotted the same name being entered five or more times into the prize spreadsheets. That sort of checking shouldn't need to be done (a simple Ctrl+F would have solved this problem), but we're all human here, we get lazy and make mistakes. (As an aside, I am convinced that all the game-specific prize draws were fair and accurate, though.)
I'm not entirely sure how the app itself works but if it only requires one data entry instead of three or four, that in itself saves a significant amount of time, time the donation tracker can spend, you know, actually watching the run. The unfortunate thing is that we are reliant on ChipIn for that data in the first place, which a) takes considerable time to load once the donations start coming in, which is a real problem for time-specific stuff and b) often inserts donations out of order so that people get missed. Any way in which b) in particular can be bypassed to eliminate the possibility for human error would be fantastic, and something that would be very valuable for people tracking donations.
After looking at the html source for the chipin page (if we use it again), it's very well suited to parsing by a program like this, so I think it'll be a good idea to make it do that, at least. Also having it interface with something on the SDA server to keep people up to date on bid wars would be good. This time we had something running on the commentary stream that people could watch but a web page that people could look at would be even better.
The problem with the chipin load times is because it dumps all the data at you and then re-renders the page with a giant clusterfuck of javascript. Using Chrome or FF4 alleviates this somewhat, but we don't even have to go that far if we just feed it right into the donation tracker. Doing it this way also prevents stuff from being 'out of order' or missed because we can easily identify new entries and flag them.
I guess when you put it that way, quality control would probably be the biggest motivating factor, with tracker convenience being a close second.
I've been doing a little tinkering over the last 2 weeks (in between exams, the tourney and portal 2 mind), mostly playing around with some php to make viewers able to see how bids are going. This in and of itself implies online distribution in some form regardless, and it shouldn't be at all difficult to do, it just requires knowing that it needs to be done.
@UA: does chip-in use https or just http? My understanding was that it used https, which would make pulling data directly somewhat difficult, not to mention logging in. In any case, I think that going forward having a chip-in based system to fall back on would still be beneficial regardless, so its worth looking at in any case.
Well, I'll see what I can get done. If you feel like you have anything to contribute, please don't hesitate (I can give read/write access to the repository I'm using if anyone likes).
EDIT: About donation tracking during JRDQ, it would be a lot more convenient if we had a donation tracker program for it. Going through donations manually by using ctrl+f is not only very tedious, it opens up more room for error. I didn't do a whole lot of donation tracking myself, but I did enough to know that it's a pain.
If you do plan on working on it, I'd say the two(three?) major features it needs are:
1) Remote database, probably MySQL since you can set that up just about anywhere, but I'm not married to any particular DB type. 1b) Exporting some kind of page for bid data, but this could be handled on the SDA server if we settle on a table layout. 2) Parsing the chipin html. Even if it means just saving the file locally and importing it instead of trying to fetch it from the server. If I recall correctly each donation even has its own unique id (long hexit string).
Edit: I just checked and it looks like this: <tr> <td>Some Dude</td> <td>some@email.com</td> <td></td> <td>romscout is so sexy!</td> <td>1301796394000</td> <--- this is a timestamp that seems to be second-accurate <td>1.00</td> <td>/profileimage/qCc97rP%2FfLAjS%2FyGHoDMgaDSk6qCreB9</td> <td>donations@speeddemosarchive.com</td> </tr>
Should be trivial to parse once we get it into the app itself.
#1: I'm planning on setting the system up so that you can connect to any type of database. This helps with testing since I can use an in-memory db (like SQLite or HSQL) for unit tests. The main DB will be MySQL, hands down.
#2: I was kind of hoping we could get nate on board with loading two or three php documents that fetch the bid data during the marathon. I know nothing about how SDA is hosted, so I'm not sure how feasible that is, or whether they are comfortable putting other stuff up on the site.
As for #3, could you please confirm the unique ID thing (or perhaps can I get a password to a chip-in account so I can see myself); if this is the case, then the idea I have for tracking donations should work out fairly well. Last year I used timestamps, which ended up being quite a bad idea; I've since learned not to use weird primary keys.
EDIT: Also, I'm perfectly willing to admit that the UI for the tracker client program needs work, I have some ideas that I'll post laster on.
If chip-in requires you to use an e-mail address (since it requires Paypal, I think it does now), then you could try using it as a primary key. It's at least mostly unique, and if two people are somehow using the same address, you'd have to send any prize notifications via that same e-mail anyway. Just a thought if there's no unique ID.
If chip-in requires you to use an e-mail address (since it requires Paypal, I think it does now), then you could try using it as a primary key. It's at least mostly unique, and if two people are somehow using the same address, you'd have to send any prize notifications via that same e-mail anyway. Just a thought if there's no unique ID.
Paypal uses e-mail as primary identifier, so e-mails should be unique across all donations. That being said, although I will be making e-mail a required and unique field for each donor (and hence e-mail will act as a primary key in some instances), I've learned my lesson and will just be using opaque integer IDs for primary keys on all entities.
does no one else have webspace that supports php? i find that hard to believe. or is there something special about sda's servers that requires it to be there?
It doesn't have to be, but I think it makes sense to keep it there. I can host it if you'd rather not have to go through the trouble.
As far as 'unique id', that long number in the fifth field is a second-accurate timestamp (in milliseconds for some reason), so between that and an email should be enough to uniquely identify a particular donation, since I don't think anybody is going to donate twice in the same second.