Author Topic: Hey, you software and website designer guys here  (Read 13115 times)

0 Members and 1 Guest are viewing this topic.

Offline Xpiatio

  • Posts: 409
  • First Name (Displayed): Benjamin
Re: Hey, you software and website designer guys here
« Reply #20 on: March 27, 2014, 11:11:01 AM »
Makes me wonder if they are locking the entire table when they do their inserts and never unlocking it.

Offline jgillmanjr

  • MOC Member
  • *
  • Posts: 654
    • Freedom Forged Security Consulting
  • First Name (Displayed): Jason
Re: Hey, you software and website designer guys here
« Reply #21 on: March 27, 2014, 11:27:44 AM »
Makes me wonder if they are locking the entire table when they do their inserts and never unlocking it.

That would be... quite awful.
IT Director
Deputy Treasurer
Legislative Aide

Offline gryphon

  • Administrator
  • *****
  • Posts: 4037
  • First Name (Displayed): Dan
Re: Hey, you software and website designer guys here
« Reply #22 on: March 27, 2014, 02:48:33 PM »
Idiots.  These people are incompetent idiots.

http://www.thetruthaboutguns.com/2014/03/foghorn/atf-issue-cease-desist-batch-nfa-paperwork-uploading/

The ATF is in the process of drafting a “cease and desist” letter to all users of the eForms system demanding that they immediately stop any batch uploading procedures. The reported punishment for anyone using such a process would be an immediate revocation of their ability to use the system.

The ATF has reportedly secured new funding to scrap their (brand new, very expensive) eForms system and replace it with another brand new one.

Offline TheQ

  • Website Content Manager
  • MOC Lifetime Member
  • *
  • Posts: 4263
    • Michigan Open Carry, Inc.
  • First Name (Displayed): Phillip
Hey, you software and website designer guys here
« Reply #23 on: March 27, 2014, 02:53:19 PM »

Idiots.  These people are incompetent idiots.

http://www.thetruthaboutguns.com/2014/03/foghorn/atf-issue-cease-desist-batch-nfa-paperwork-uploading/

The ATF is in the process of drafting a “cease and desist” letter to all users of the eForms system demanding that they immediately stop any batch uploading procedures. The reported punishment for anyone using such a process would be an immediate revocation of their ability to use the system.

The ATF has reportedly secured new funding to scrap their (brand new, very expensive) eForms system and replace it with another brand new one.

Wouldn't it be easier to disable that portion of the system than to issue C/D "orders"?
I Am Not A Lawyer (nor a gunsmith).

Offline SD40VE

  • SE Region Posse
  • Posts: 996
  • New Haven, Macomb County
  • First Name (Displayed): Bradley
Re: Hey, you software and website designer guys here
« Reply #24 on: March 27, 2014, 03:52:50 PM »
Wouldn't it be easier to disable that portion of the system than to issue C/D "orders"?

nah its our government, we are just going to develop a new system from the ground up and go further into debt funding it! Murica!

Offline part deux

  • MOC Member
  • *
  • Posts: 683
Re: Hey, you software and website designer guys here
« Reply #25 on: March 27, 2014, 07:13:42 PM »
Idiots.  These people are incompetent idiots.

The ATF has reportedly secured new funding to scrap their (brand new, very expensive) eForms system and replace it with another brand new one.

ACA Web Designers are available, and are willing to work for government wages.

Offline jgillmanjr

  • MOC Member
  • *
  • Posts: 654
    • Freedom Forged Security Consulting
  • First Name (Displayed): Jason
Re: Hey, you software and website designer guys here
« Reply #26 on: March 27, 2014, 09:15:41 PM »
Actually, who wants to do a fun proof of concept project to show how bloody inept these ass hats are?

Some one want to send me the "form" that needs to be processed?

I guarantee that we could hack something together in a damn scripting language that can handle more transactions per second than these clowns.
IT Director
Deputy Treasurer
Legislative Aide

Offline SD40VE

  • SE Region Posse
  • Posts: 996
  • New Haven, Macomb County
  • First Name (Displayed): Bradley
Re: Hey, you software and website designer guys here
« Reply #27 on: March 28, 2014, 08:23:20 AM »
Actually, who wants to do a fun proof of concept project to show how bloody inept these ass hats are?

Some one want to send me the "form" that needs to be processed?

I guarantee that we could hack something together in a damn scripting language that can handle more transactions per second than these clowns.

then sell it to the government for 100 Billion dollars!!


Offline jgillmanjr

  • MOC Member
  • *
  • Posts: 654
    • Freedom Forged Security Consulting
  • First Name (Displayed): Jason
Re: Hey, you software and website designer guys here
« Reply #28 on: March 28, 2014, 09:40:10 AM »
I'm curious though - how exactly are they submitting the data?

Are they OCR'ing the form and sending it in that way, or how is the actual data getting entered?
IT Director
Deputy Treasurer
Legislative Aide

Offline SD40VE

  • SE Region Posse
  • Posts: 996
  • New Haven, Macomb County
  • First Name (Displayed): Bradley
Re: Hey, you software and website designer guys here
« Reply #29 on: March 28, 2014, 09:43:11 AM »
never used the system so i have n o clue how the forms are submitted and received and databased

Offline bigt8261

  • MOC President
  • MOC Board Member
  • ******
  • Posts: 1482
    • Michigan Open Carry, Inc
  • First Name (Displayed): Tom
Re: Hey, you software and website designer guys here
« Reply #30 on: March 28, 2014, 02:15:25 PM »
The local shop responded:

Quote
There’s one other small piece of evidence that makes me sympathetic to Mark’s claim. “The [batch uploading] system has been offline since Saturday, it’s being upgraded” he says. “You haven’t heard any reports that things are getting better, have you?”

http://www.thetruthaboutguns.com/2014/03/foghorn/silencer-shop-the-atf-is-blaming-us-for-their-crappy-website/#more-307241

Offline gryphon

  • Administrator
  • *****
  • Posts: 4037
  • First Name (Displayed): Dan
Re: Hey, you software and website designer guys here
« Reply #31 on: March 28, 2014, 02:59:04 PM »
I'm curious though - how exactly are they submitting the data?

Are they OCR'ing the form and sending it in that way, or how is the actual data getting entered?

Info is entered digitally.  Don't know the exact details of how, but I would not assume OCR as that is subject to error!

Offline jgillmanjr

  • MOC Member
  • *
  • Posts: 654
    • Freedom Forged Security Consulting
  • First Name (Displayed): Jason
Re: Hey, you software and website designer guys here
« Reply #32 on: March 28, 2014, 03:20:12 PM »
Info is entered digitally.  Don't know the exact details of how, but I would not assume OCR as that is subject to error!

Fair enough.

If it's the case that the shop is punching in the data from the form itself, well, all I have to say is:


JSON
IT Director
Deputy Treasurer
Legislative Aide

Offline gryphon

  • Administrator
  • *****
  • Posts: 4037
  • First Name (Displayed): Dan
Re: Hey, you software and website designer guys here
« Reply #33 on: March 29, 2014, 10:53:46 AM »
Here's a letter the ATF put out about this back in early February claiming all was more-or-less normal before they took the expensive, new system down and decided to scrap it for another expensive, new system. Here's a choice excerpt of the letter.

In spite of the performance issues, eForms is doing what it was designed to do.  It may be slower and it may be temperamental at times, but we believe it to still be a worthwhile investment.

While we are trying to resolve these issues it may be necessary for us to perform system restarts throughout the day where we may bring eforms down for an hour or so.


Yeah, just randomly taking your system down for an hour at a time while you periodically reboot it throughout the day is just dandy.  Never mind the business people on the other end who are spending hours trying to enter their customers' info in and losing all that work when your system crashes or is rebooted yet once again.  Unfortunately it's just sad but not surprising that we expect this level of incompetence and inefficiency out of our government.

http://www.thetruthaboutguns.com/wp-content/uploads/2014/02/ATF-email.pdf

Offline SD40VE

  • SE Region Posse
  • Posts: 996
  • New Haven, Macomb County
  • First Name (Displayed): Bradley
Re: Hey, you software and website designer guys here
« Reply #34 on: March 31, 2014, 08:32:00 AM »
Here's a letter the ATF put out about this back in early February claiming all was more-or-less normal before they took the expensive, new system down and decided to scrap it for another expensive, new system. Here's a choice excerpt of the letter.

In spite of the performance issues, eForms is doing what it was designed to do.  It may be slower and it may be temperamental at times, but we believe it to still be a worthwhile investment.

While we are trying to resolve these issues it may be necessary for us to perform system restarts throughout the day where we may bring eforms down for an hour or so.


Yeah, just randomly taking your system down for an hour at a time while you periodically reboot it throughout the day is just dandy.  Never mind the business people on the other end who are spending hours trying to enter their customers' info in and losing all that work when your system crashes or is rebooted yet once again.  Unfortunately it's just sad but not surprising that we expect this level of incompetence and inefficiency out of our government.

http://www.thetruthaboutguns.com/wp-content/uploads/2014/02/ATF-email.pdf

thats unacceptable down time. in any other business setting it would more or less be a breach of contract due to RTO times. but hey its uncle sam. he does what he wants

Offline jgillmanjr

  • MOC Member
  • *
  • Posts: 654
    • Freedom Forged Security Consulting
  • First Name (Displayed): Jason
Re: Hey, you software and website designer guys here
« Reply #35 on: March 31, 2014, 08:36:13 AM »
thats unacceptable down time. in any other business setting it would more or less be a breach of contract due to RTO times. but hey its uncle sam. he does what he wants

Probably all the rejects that couldn't hack it in the private sector.
IT Director
Deputy Treasurer
Legislative Aide

Offline gryphon

  • Administrator
  • *****
  • Posts: 4037
  • First Name (Displayed): Dan
Re: Hey, you software and website designer guys here
« Reply #36 on: April 02, 2014, 06:01:37 PM »
Why the ATF’s eForms System Sucks
By Christopher Cicchitelli



As a fervent Second Amendment supporter and the founder of a technology company, I’ve followed the ATF eForms story with great personal interest. It showed great promise at first, cutting wait times down to under 60 days…and then it imploded. Exactly what happened was a question we’ve all asked, and until reading the ATFs announcement yesterday I didn’t really have a good idea. However, in that announcement I believe the ATF tipped us off to the key that unravels the mystery . . .

First, some full disclosure: prior to starting my company, CastleOS, I worked as a contractor to a government R&D lab where, among other responsibilities, I designed integrated systems and managed servers for multiple civilian and military agencies.

Now for some key facts we know: despite the ATFs initial claims otherwise, the eForms system was unusable even when batch processing was not taking place. Even if we presume Silencer Shop initiated batch processes outside the 4-5 AM window they claim, I find it hard to believe they did it all day every day.

In addition, in today’s ATF announcement, they claimed there are “memory allocation errors” within the system. In order to keep it usable for all, it needs to be rebooted multiple times a day, taking up to an hour for each reboot. That’s key, and I’ll come back to it later.

In addition to these facts, there are several questions we are all asking. How was batch processing ongoing if the system wasn’t built for it? Why does the system need to be rebooted every few hours (this is 2014 after all!)? Whose fault is it really?

After doing some research, I think I have narrowed down the realm of possibilities for the answers to those questions, so here goes. One of the first things I noticed when visiting the eForms website (well, after I got to the website to load – it took a few minutes) was the technology it’s built upon: JavaServer Pages. To say JSP isn’t exactly the most popular platform these days would be an understatement. While still around for legacy reasons, you don’t often seen it used for new projects without a specific reason – a reason I don’t see the ATF having. From my own experience as a contractor, I wonder if this was a matter of the contracting company having available labor in this speciality, and thus pushing the platform on the ATF. If so, the ATF contracting officer would certainly deserve some blame for this as well.

The other immediate observation was the speed of the website loading up. It repeatedly took about five minutes to reach the login screen at around 7pm EST. What this shows is a faulty system architecture design. A proper design would separate all tiers, including at the servers themselves, into the following three categories: user interface, data broker, and data warehouse.

There’s no doubt that there is a heavy amount of processing of data going on in this system, and it’s slowing the whole system down – so there isn’t a proper separation of tiers in this design. The fact that the data warehousing can slow down the most basic parts of the user interface – the website itself and login page – is a major failure. So much so that for the millions of dollars spent on this website, I’d argue the government would have recourse to recoup (at least some of) the money it laid out.

Next is the question of how batch processing was happening if it wasn’t designed for it. The ATF has continually sounded the horn about the batch processing, but the truth is it’s misdirection. Whether the eForms system has a batch option or whether the Silencer Shop is using a custom method — possibly something as simple as a macro — to automate the process is 100% irrelevant. The reason is for all intents and purposes, a batch process just simulates the effect of multiple stores logging in at the same time. What we are seeing is a system that goes down with very little load, possible just a few dozen simultaneous users, or one user uploading a few dozen forms a night, and I think the reason lies in the ATF’s “memory allocation error”.

Traditionally, a memory allocation error is when data is corrupted or misplaced on actual memory. I don’t think the ATF has an actual error with the server memory itself – using JavaServer Pages should prevent that – but rather is PR spin for a more complicated resource allocation error.

What I think is happening is as forms are entered into the system, they enter a bureaucratically-inflated workflow. In that workflow, the system is probably generating loads of actual documents in addition to lots of database entries and so forth. (After all, do you really expect the ATF not to keep a backup paper trail? Pfff.) In other words, each application isn’t as simple as the typical web form we are used to using, and it does require some real horsepower to process it from start to finish.

Now normally that’s not an issue, each server has its maximum number of users, and you add servers as needed. The government even has a gov-only cloud it can use to deploy servers on demand – so keeping up with demand shouldn’t require more than a few minutes to boot up a new instance. However with the eForms system, the opposite seems to be true, and they can never bring enough servers online to keep up with demand (they tried at the beginning). I believe the reason is because load isn’t the issue, batch or otherwise, but a fundamental flaw in the design of the system: applications are gobbling up resources as they are processed, and then when complete, not releasing those resources.

The proof of that is the reboots every few hours to clear the “memory allocation errors” – if the code isn’t releasing the resource, you need to reboot. (Why it takes an hour isn’t clear – but that may have more to do with their attempts jury rig fixes into the system, and/or having to bring systems online in a certain order.)

I also believe that in the beginning, before too many people were using the eForms system, this same flaw was present, but was being covered up by nightly reboots. Once the load increased, the problem literally spread exponentially, and now the system can only run for roughly 4 hours until all the servers run out of resources.

The saddest thing about this for me, is not that the system failed, I’ve seen that far too many times before in government. It’s that so many millions were spent to bring a system online with such a fundamental flaw. I’d give the Obamacare website team a break long before I’d cut ATF any slack for this fiasco.

Offline jgillmanjr

  • MOC Member
  • *
  • Posts: 654
    • Freedom Forged Security Consulting
  • First Name (Displayed): Jason
Re: Hey, you software and website designer guys here
« Reply #37 on: April 03, 2014, 07:39:00 PM »
You mean it's not cool to stack everything in a single tier?

Sonuvabitch....
IT Director
Deputy Treasurer
Legislative Aide

Offline gryphon

  • Administrator
  • *****
  • Posts: 4037
  • First Name (Displayed): Dan
Re: Hey, you software and website designer guys here
« Reply #38 on: April 04, 2014, 08:56:59 PM »
CRASH

If you were hoping to submit a form electronically in order to cut your wait time, your window of opportunity may have just closed. ATF eForms users are being greeted with this message:

The eForms software is not performing to our expectations.  As a result, we are taking the eForms system down until further notice.  We apologize for any inconvenience and appreciate your patience as we work with our industry partners to deliver a quality product.  Any eForm submitted will continue to be processed. The finalized forms will be sent to the user via email.

Until the eForms system is returned to service for the industry, all imports forms (Forms 6 Part I and 6A), NFA forms (1, 2, 3, 4, 5, 9 and 10), and AFMER reports (Form 5300.11) must be submitted via paper, including any eForms in draft status.

* Tax refund for draft eForms
* Direct a request for refund to eForms.Refunds@atf.gov
* Include the email that was generated for the payment
* Copy of submitted or finalized eForm
* Direct a request to eForms.Request@atf.gov
* Provide specific information regarding what is being requested
* Status queries
* Contact the NFA Branch at (304) 616-4550
* Contact the Firearms and Explosives Imports Branch at (304) 616-4550
* Other eForms questions
* Direct the question to eForms.admin@atf.gov

http://silencernews.com/atf-takes-down-eforms-until-further-notice/