Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pandaproxy: Invert error_code dependencies #4722

Merged
merged 5 commits into from
May 16, 2022

Conversation

BenPope
Copy link
Member

@BenPope BenPope commented May 12, 2022

Cover letter

Refactor pandaproxy errors, so that the conversion of error_codes to reply_error_condition becomes the responsibility of pandaproxy, not each submodule. This makes the dependencies the right way round.

There are two ways to map to an error_condition

  1. Override error_category::default_error_condition
  2. Override error_category::equivalent

However:

  1. Is what's current, but it requires a global
    default_error_condition, but reply_error_code is specific
    to pandaproxy, so the dependency works the wrong way.
  2. Could work, but only for checking equivalence of specific
    std::error_code with a specific reply_error_code, not
    for directly converting it.

Solution

To break the dependency, overload make_error_condition that
takes a std::error_code, and maps from kafka::error_code to
reply_error_condition as before. Further commits will move code
to break the dependencies.

Implement reply_error_category::equivalent in terns of
the new make_error_condition.

  • Move the error_category and make_error_code for kafka into kafka
  • Move the error_code to error_condition code from pandaproxy::parse to pandaproxy
  • Move the error_code to error_condition code from pandaproxy::json to pandaproxy

Useful resource: https://akrzemi1.wordpress.com/2017/09/04/using-error-codes-effectively/

Closes #4307 - this is a prerequisite of updating libfmt #4260

In a future PR, it will be possible to split the error_codes for Schema Registry and REST API.

Alternative

If reply_error_category::equivalent were to be implemented instead, it would something like:

    bool
    equivalent(const std::error_code& ec, int cond) const noexcept override {
        using rec = reply_error_code;
        using kec = kafka::error_code;
        switch (static_cast<rec>(cond)) {
        case rec::kafka_bad_request:
            return ec == kec::offset_out_of_range || ec ==
            kec::corrupt_message
                   || ec == kec::invalid_fetch_size
                   || ec == kec::not_leader_for_partition
                   || ec == kec::message_too_large
                   || ec == kec::stale_controller_epoch
                   || ec == kec::offset_metadata_too_large
                   || ec == kec::invalid_topic_exception
                   || ec == kec::record_list_too_large
                   || ec == kec::illegal_generation
                   || ec == kec::inconsistent_group_protocol
                   || ec == kec::invalid_group_id
                   || ec == kec::invalid_session_timeout
                   || ec == kec::invalid_commit_offset_size
                   || ec == kec::invalid_timestamp
                   || ec == kec::unsupported_sasl_mechanism
                   || ec == kec::illegal_sasl_state
                   || ec == kec::unsupported_version
                   || ec == kec::topic_already_exists
                   || ec == kec::invalid_partitions
                   || ec == kec::invalid_replication_factor
                   || ec == kec::invalid_replica_assignment
                   || ec == kec::invalid_config || ec == kec::not_controller
                   || ec == kec::invalid_request
                   || ec == kec::unsupported_for_message_format
                   || ec == kec::policy_violation
                   || ec == kec::invalid_required_acks
                   || ec == kec::invalid_producer_epoch
                   || ec == kec::invalid_txn_state
                   || ec == kec::invalid_producer_id_mapping
                   || ec == kec::invalid_transaction_timeout
                   || ec == kec::concurrent_transactions
                   || ec == kec::transaction_coordinator_fenced
                   || ec == kec::security_disabled
                   || ec == kec::log_dir_not_found
                   || ec == kec::unknown_producer_id
                   || ec == kec::delegation_token_not_found
                   || ec == kec::delegation_token_owner_mismatch
                   || ec == kec::delegation_token_request_not_allowed
                   || ec == kec::delegation_token_expired
                   || ec == kec::invalid_principal_type
                   || ec == kec::non_empty_group
                   || ec == kec::group_id_not_found
                   || ec == kec::fetch_session_id_not_found
                   || ec == kec::invalid_fetch_session_epoch
                   || ec == kec::listener_not_found
                   || ec == kec::topic_deletion_disabled
                   || ec == kec::fenced_leader_epoch
                   || ec == kec::unknown_leader_epoch
                   || ec == kec::unsupported_compression_type
                   || ec == kec::stale_broker_epoch
                   || ec == kec::offset_not_available
                   || ec == kec::member_id_required
                   || ec == kec::group_max_size_reached
                   || ec == kec::fenced_instance_id
                   || ec == kec::invalid_record;
        case rec::kafka_error:
            return ec == kec::not_enough_replicas
                   || ec == kec::coordinator_not_available
                   || ec == kec::not_coordinator
                   || ec == kec::not_enough_replicas_after_append
                   || ec == kec::out_of_order_sequence_number
                   || ec == kec::duplicate_sequence_number
                   || ec == kec::operation_not_attempted
                   || ec == kec::kafka_storage_error
                   || ec == kec::unknown_server_error;
        case rec::kafka_retriable_error:
            return ec == kec::network_exception
                   || ec == kec::coordinator_load_in_progress
                   || ec == kec::request_timed_out
                   || ec == kec::rebalance_in_progress
                   || ec == kec::reassignment_in_progress;
        case rec::partition_not_found:
            return ec == kec::unknown_topic_or_partition;
        case rec::consumer_instance_not_found:
            return ec == kec::unknown_member_id;
        case rec::broker_not_available:
            return ec == kec::leader_not_available
                   || ec == kec::broker_not_available
                   || ec == kec::replica_not_available
                   || ec == kec::preferred_leader_not_available;
        case rec::kafka_authorization_error:
            return ec == kec::topic_authorization_failed
                   || ec == kec::group_authorization_failed
                   || ec == kec::transactional_id_authorization_failed
                   || ec == kec::delegation_token_authorization_failed
                   || ec == kec::cluster_authorization_failed;
        case rec::kafka_authentication_error:
            return ec == kec::sasl_authentication_failed
                   || ec == kec::delegation_token_auth_disabled;
        }
        vassert(false, "unhandled {}", ec);
    }

And then the other ~18 reply_error codes would have to be implemented for all the error_codes in other namespaces such as pandaproxy::parse, pandaproxy::json, etc., and there would be no way to have the compiler statically check for incomplete handling of enum cases in the distant modules. In addition, conversion to an http status code would have to exhaustive switch over each reply_error_code., requiring two conversions.

Signed-off-by: Ben Pope ben@redpanda.com

Release notes

  • none

@BenPope BenPope requested review from dotnwat and NyaliaLui May 13, 2022 05:01
Signed-off-by: Ben Pope <ben@redpanda.com>
There are currently several projects that depend on
`v::pandaproxy_common` due to the conversion from
`error_code` to `error_condition`, `reply_error_code`

There are two ways to map to an `error_condition`
1) Override `error_category::default_error_condition`
2) Override `error_category::equivalent`

1) Is what's current, but it requires a global
   `default_error_condition`, but `reply_error_code` is specific
   to pandaproxy, so the dependency works the wrong way.
2) Could work, but only for checking equivalence of specific
   `std::error_code` with a specific `reply_error_code`, not
   for directly converting it.

To break the dependency, overload `make_error_condition` that
takes a `std::error_code`, and maps from `kafka::error_code` to
`reply_error_condition` as before. Further commits will move code
to break the dependencies.

Implement replay_error_category::equivalent in terns of
the new `make_error_condition`.

Signed-off-by: Ben Pope <ben@redpanda.com>
Move `make_error_code`  and `error_category` from pandaproxy to kafka.

Signed-off-by: Ben Pope <ben@redpanda.com>
Move it from the pandaproxy::parse to pandaproxy.

Change `exception_base` to use an `error_code`, which is the
correct tool for passing around error information.

Signed-off-by: Ben Pope <ben@redpanda.com>
Move it from the pandaproxy::json to pandaproxy.

Signed-off-by: Ben Pope <ben@redpanda.com>
@BenPope BenPope force-pushed the move-kafka-error-code-to-kafka branch from 3c044bc to fd0eeb4 Compare May 13, 2022 16:50
@BenPope BenPope changed the title kafka: Move make_error_code(kafka::error_code) pandaproxy: Invert error_code dependencies May 13, 2022
@BenPope BenPope marked this pull request as ready for review May 13, 2022 16:54
@BenPope BenPope mentioned this pull request May 14, 2022
Copy link
Member

@dotnwat dotnwat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. Needs:

  • CI failures linked to issues

@@ -190,6 +191,17 @@ std::error_condition make_error_condition(std::error_code ec) {
return rec::kafka_authentication_error;
}
return {}; // keep gcc happy
} else if (ec.category() == make_error_code(pec::empty_param).category()) {
switch (static_cast<pec>(ec.value())) {
case pec::empty_param:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: [[fallthrough]];

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tend to do that only for cases where there is a statement. I don't think the compiler will warn unless there is a statement, anyhow.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm, i thought clang-tidy warned in this case, but indeed im not see it.

@BenPope BenPope merged commit 68935f7 into redpanda-data:dev May 16, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants