Skip to content
This repository has been archived by the owner on Jul 24, 2020. It is now read-only.

Intermittently failing test due to another spec not cleaning up after itself #1514

Closed
orenyk opened this issue Mar 6, 2016 · 9 comments
Closed

Comments

@orenyk
Copy link
Contributor

orenyk commented Mar 6, 2016

1) Reservations can be created by checkout persons with override permissions behaves like can create failing reservation successfully

     Failure/Error:

       within(:css, "#add_to_cart_#{eq_model.id}") do

         click_link 'Add to Cart'



     Capybara::ElementNotFound:

       Unable to find css "#add_to_cart_474"

     Shared Example Group: "can create failing reservation" called from ./spec/features/reservations_spec.rb:141

     # ./spec/support/feature_helpers.rb:56:in `add_item_to_cart'

     # ./spec/features/reservations_spec.rb:87:in `block (4 levels) in <top (required)>'

In the test environment we default to having a single equipment model per page displayed on the catalog, so if there are any other categories / equipment models in the database, depending on the sort order, the expected equipment model's add_to_cart button won't be present when the catalog loads (e.g. on the first page). Since this test only fails intermittently, it appears as though at least one of our specs isn't properly cleaning up after itself and is leaving other equipment in the database, so if this spec runs after the problem spec, it fails, but otherwise it passes.

We could simply leave the tests and just increase the default pagination value in test, but that's more treating the symptom, not the cause, so I'd like to figure out which test is causing the problem and to fix it.

@esoterik
Copy link
Collaborator

Also, there are intermittently failing tests that have to do with TOS acceptance.

@esoterik
Copy link
Collaborator

If you have the seeds that result in failing tests, you can use rspec bisect to further analyze the intermittent failures!

@orenyk
Copy link
Contributor Author

orenyk commented Apr 28, 2016

Holy cow that's awesome!

@esoterik
Copy link
Collaborator

Do we have any of the seeds for this kind of intermittent failure?

@orenyk
Copy link
Contributor Author

orenyk commented Apr 28, 2016

Not that I know of, unfortunately. Based on my comment / the blog post you found let's just double check our DatabaseCleaner config and make sure we manually clean up all before(:all) blocks with an after(:all) block (or get rid of them if it makes sense), after which we can probably close this. Thoughts?

@orenyk
Copy link
Contributor Author

orenyk commented May 15, 2016

Another seed: 18208, the test that failed was /spec/features/guest_spec.rb:84

@esoterik
Copy link
Collaborator

Note that rspec bisect should NOT be run on that seed until the before(:all)s are cleaned up; I had it running for nearly six hours and it was not even half way done, likely because nearly every spec causes the failure

@esoterik
Copy link
Collaborator

Currently working on getting rid of before(:all)s, which are exclusively used to set up the AppConfig. I'm replacing them with appropriate mocks

@orenyk
Copy link
Contributor Author

orenyk commented May 16, 2016

Sounds great 😄

esoterik pushed a commit that referenced this issue May 21, 2016
Resolves #1514
  - Only create a second equipment model and item when necessary in
    feature specs
  - Replace nearly all AppConfig creations in tests with a mocked
    AppConfig
  - Get rid of most before(:all) callbacks
  - Minor refactor of a reservations controller spec
esoterik pushed a commit that referenced this issue May 25, 2016
Resolves #1514
  - Only create a second equipment model and item when necessary in
    feature specs
  - Replace nearly all AppConfig creations in tests with a mocked
    AppConfig
  - Get rid of most before(:all) callbacks
  - Minor refactor of a reservations controller spec
esoterik pushed a commit that referenced this issue May 25, 2016
Resolves #1514
  - Only create a second equipment model and item when necessary in
    feature specs
  - Replace nearly all AppConfig creations in tests with a mocked
    AppConfig
  - Get rid of most before(:all) callbacks
  - Minor refactor of a reservations controller spec
esoterik pushed a commit that referenced this issue May 31, 2016
Resolves #1514
  - Only create a second equipment model and item when necessary in
    feature specs
  - Replace nearly all AppConfig creations in tests with a mocked
    AppConfig
  - Get rid of most before(:all) callbacks
  - Minor refactor of a reservations controller spec
esoterik pushed a commit that referenced this issue May 31, 2016
Resolves #1514
  - Only create a second equipment model and item when necessary in
    feature specs
  - Replace nearly all AppConfig creations in tests with a mocked
    AppConfig
  - Get rid of most before(:all) callbacks
  - Minor refactor of a reservations controller spec
esoterik pushed a commit that referenced this issue May 31, 2016
Resolves #1514
  - Only create a second equipment model and item when necessary in
    feature specs
  - Replace nearly all AppConfig creations in tests with a mocked
    AppConfig
  - Get rid of most before(:all) callbacks
  - Minor refactor of a reservations controller spec
esoterik pushed a commit that referenced this issue Jun 2, 2016
zeffman pushed a commit that referenced this issue Sep 18, 2016
Resolves #1360
- Add Calendarable controller concern to generate HTML, JSON, and
  iCalendar representations of all the reservations for a given
  resource
- Add calendarable routing concern and clean up config/routes.rb
- Add Reservation#end_date to find the last day for a reservation,
  status-dependent (with model specs)
- Add the associated controller specs as a shared example as well
  as feature specs for equipment models
- Add calendars for categories, equipment models, equipment items,
  and users
- Fix routing issue with jQuery-UI-Bootstrap assets

Add updated schema for Rails 4.2

Resolves #1517
- schema now has field limits, see issue for details

Clarify availability error messages

Closes #1242
   - change app/models/cart_validations.rb to tell
     maximum number of available items, pluralizing properly

Fix UI issues

Closes #1492
  - Change big-numbers mixin so font does not overflow table
  - Adjust equipment_model show table columns to align better
  - Add table_woo to CSS formatting to center rows vertically

Resolve issues with contact form

Resolves #1524
- add AppConfig.contact_email to correctly default to admin e-mail when
  no contact e-mail is set
- add validations for AppConfig.contact_link_location to ensure it is a
  valid e-mail if it's not blank
- update seed and application setup script to assign valid e-mails to
  contact_link_location by default, as well as update factories
- add specs

Add optional archival deactivation of equipment

Resolves #1384
- add autodeactivate_on_archive parameter to AppConfig
- move equipment item deactivation to a model method
- tweak ReservationsController#archive method to implement
  auto-deactivation
- add feature specs for reservation archiving
- add model specs for EquipmentItem#deactivate
- add controller specs for ReservationsController#archive with
  auto-deactivation
- FeatureHelpers#app_setup now deletes any pre-existing AppConfigs

Prevents duplicate items in search results

Resolves #933
- Flattens search results array before applying uniq
- Wrote test to check for duplicate results

Fix / replace availability calendar

Resolves #1498
- add FullCalendar-based availability calendar
- move availability calculations to private controller method
- add controller specs for availability
- remove legacy calendar code

Fix whenever / Capistrano integration

Resolves #1373
- use built-in whenever tasks

Fix view alignment of dates in reservation show

closes #1532
- minor edit in reservations show.html to remove wrapper around h4 headings
- p tag used to cause alignment problem

Remove Rails dependency from whenever.rb

Resolves #1542
- whoops

Fix rubocop error

Introduced in #1542, whoops again!

Request text is now shown after invalid cart edit

Resolves #1544
- moves request text assignment code to ReservationsHelper
- manually include ReservationsHelper in CatalogController

Update CHANGELOG for v6.0.0

Resolves #1362
- include changes in the v5.5.x releases

Update Travis status image in README

This was previously reflecting the status of the build from the most recently
pushed commit / PR, which is often red. To accurately reflect the health of the
project the status image will now only refer to builds from `master`.

Change default Heroku e-mail addon to Sendgrid

Resolves #1515
- update `devise_cas_authenticatable` to v1.7.1 to resolve logger issue

Clean up #1337
Resolves #1499
  - Include CsvExport in all relevant controllers
  - Add single category export

Overhaul availability

Resolves #1501
  - Availability methods all work as expected
  - Adds counter cache on equipment models for current overdue reservations
  - Reservation factory trait :past allows for creation of past
    reservations without skipping validations
  - New reservation model methods for counting reservations, attribute
    checking, and checking for overlapping

Seed script no longer generates reservations with impossible dates

Resolves #1229 and #1263
 - Seed script uses TimeHelpers to save reservations in the past,
   allowing validations to be run on them

Fix ElementNotFound feature test intermittent failure

Resolves #1514
  - Only create a second equipment model and item when necessary in
    feature specs
  - Replace nearly all AppConfig creations in tests with a mocked
    AppConfig
  - Get rid of most before(:all) callbacks
  - Minor refactor of a reservations controller spec

Update dependencies (May 2016)

Resolves #1553
- Highlights:
  - Update Ruby to 2.2.5
  - Update Devise to v4.1
  - Update Capybara to v2.7
- Email validators are now more permissive due to Devise update
- Capybara matchers no longer return hidden inputs by default
- Several new rubocop cops were implemented and dealt with

Update CHANGELOG for v6.1.0

Resolves #1548

Fix intermittent asset failure in specs

Resolves #1528
  - Mock relative root in application helper specs

Configure Hound CI

Resolves #1577

Add option for requests to block equipment availability

Resolves #1521

Refactor EquipmentModel Model spec

Resolves #1600

Add ESlint to Travis; only style check the diff

Resolves #1581
  - Adds 'rake check_style'
  - Fixes rubocop's TargetRubyVersion
  - Removes HoundCI config

Refactor EquipmentModels controller spec

Resolves #1587
 - Adds Mocker classes
 - Adds shared controller examples for successful and failed requests

Refactor periodic rake tasks to ActiveJobs

Resolves #1275
  - Add buttons to manually run periodic tasks
  - Moves all periodic tasks from rake to ActiveJob
  - Heavy refactoring of former rake tasks and their specs

Update ruby version in readme

Updated ruby version in readme (literally changed one character)

Refactor Reservation Model spec

Resolves #1554

Refactor EquipmentItems Controller spec

Resolves #1588

Refactor Reservations Controller Spec

Resolves #1586
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

2 participants