Auto merge of rust-lang#35605 - eddyb:substs, r=nikomatsakis

Remove the ParamSpace separation from formal and actual generics in rustc.

This is the first step towards enabling the typesystem implemented by `rustc` to be extended
(with generic modules, HKT associated types, generics over constants, etc.).

The current implementation splits all formal (`ty::Generics`) and actual (`Substs`) lifetime and type parameters (and even `where` clauses) into 3 "parameter spaces":
* `TypeSpace` for `enum`, `struct`, `trait` and `impl`
* `SelfSpace` for `Self` in a `trait`
* `FnSpace` for functions and methods

For example, in `<X as Trait<A, B>>::method::<T, U>`, the `Substs` are `[[A, B], [X], [T, U]]`.
The representation uses a single `Vec` with 2 indices where it's split into the 3 "parameter spaces".
Such a simplistic approach doesn't scale beyond the Rust 1.0 typesystem, and its existence was mainly motivated by keeping code manipulating generic parameters correct, across all possible situations.

Summary of changes:
* `ty::Generics` are uniformly stored and can be queried with `tcx.lookup_generics(def_id)`
 * the `typeck::collect` changes for this resulted in a function to lazily compute the `ty::Generics` for a local node, given only its `DefId` - this can be further generalized to other kinds of type information
* `ty::Generics` and `ty::GenericPredicates` now contain only their own parameters (or `where` clauses, respectively), and refer to their "parent", forming a linked list
 * right now most items have one level of nesting, only associated items and variants having two
 * in the future, if `<X as mod1<A>::mod2<B>::mod3::Trait<C>>::Assoc<Y>` is supported, it would be represented by item with the path `mod1::mod2::mod3::Trait::Assoc`, and 4 levels of generics: `mod1` with `[A]`, `mod2` with `[B]`, `Trait` with `[X, C]` and `Assoc` with `[Y]`
* `Substs` gets two new APIs for working with arbitrary items:
 * `Substs::for_item(def_id, mk_region, mk_type)` will construct `Substs` expected by the definition `def_id`, calling `mk_region` for lifetime parameters and `mk_type` for type parameters, and it's guaranteed to *always* return `Substs` compatible with `def_id`
 * `substs.rebase_onto(from_base_def_id, to_base_substs)` can be used if `substs` is for an item nested within `from_base_def_id` (e.g. an associated item), to replace the "outer parameters" with `to_base_substs` - for example, you can translate a method's `Substs` between a `trait` and an `impl` (in both directions) if you have the `DefId` of one and `Substs` for the other
* trait objects, without a `Self` in their `Substs`, use *solely* `ExistentialTraitRef` now, letting `TraitRef` assume it *always* has a `Self` present
* both `TraitRef` and `ExistentialTraitRef` get methods which do operations on their `Substs` which are valid only for traits (or trait objects, respectively)
* `Substs` loses its "parameter spaces" distinction, with effectively no code creating `Substs` in an ad-hoc manner, or inspecting them, without knowing what shape they have already

Future plans:
* combine both lifetimes and types in a single `Vec<Kind<'tcx>>` where `Kind` would be a tagged pointer that can be `Ty<'tcx>`, `&'tcx ty::Region` or, in the future, potentially-polymorphic constants
 * this would require some performance investigation, if it implies a lot of dynamic checks
* introduce an abstraction for `(T, Substs)`, where the `Substs` are even more hidden away from code
manipulating it; a precedent for this is `Instance` in trans, which has `T = DefId`; @nikomatsakis also referred to this, as "lazy substitution", when `T = Ty`
* rewrite type pretty-printing to fully take advantage of this to inject actual in the exact places of formal generic parameters in any paths
* extend the set of type-level information (e.g. beyond `ty::Generics`) that can be lazily queried during `typeck` and introduce a way to do those queries from code that can't refer to `typeck` directly
 * this is almost unrelated but is necessary for DAG-shaped recursion between constant evaluation and type-level information, i.e. for implementing generics over constants

r? @nikomatsakis
cc @rust-lang/compiler

cc @nrc Could get any perf numbers ahead of merging this?

src/librustc/hir/def.rs

@@ -9,7 +9,6 @@
 // except according to those terms.
 
 use hir::def_id::DefId;
-use ty::subst::ParamSpace;
 use util::nodemap::NodeMap;
 use syntax::ast;
 use hir;
@@ -31,7 +30,7 @@ pub enum Def {
     AssociatedTy(DefId /* trait */, DefId),
     Trait(DefId),
     PrimTy(hir::PrimTy),
-    TyParam(ParamSpace, u32, DefId, ast::Name),
+    TyParam(DefId),
     Upvar(DefId,        // def id of closed over local
              ast::NodeId,  // node id of closed over local
              usize,        // index in the freevars list of the closure
@@ -122,7 +121,7 @@ impl Def {
         match *self {
             Def::Fn(id) | Def::Mod(id) | Def::ForeignMod(id) | Def::Static(id, _) |
             Def::Variant(_, id) | Def::Enum(id) | Def::TyAlias(id) | Def::AssociatedTy(_, id) |
-            Def::TyParam(_, _, id, _) | Def::Struct(id) | Def::Trait(id) |
+            Def::TyParam(id) | Def::Struct(id) | Def::Trait(id) |
             Def::Method(id) | Def::Const(id) | Def::AssociatedConst(id) |
             Def::Local(id, _) | Def::Upvar(id, _, _, _) => {
                 id

src/librustc/infer/error_reporting.rs

@@ -82,7 +82,6 @@ use hir::def::Def;
 use hir::def_id::DefId;
 use infer::{self, TypeOrigin};
 use middle::region;
-use ty::subst;
 use ty::{self, TyCtxt, TypeFoldable};
 use ty::{Region, ReFree};
 use ty::error::TypeError;
@@ -1366,10 +1365,10 @@ impl<'a, 'gcx, 'tcx> Rebuilder<'a, 'gcx, 'tcx> {
                 hir::TyPath(ref maybe_qself, ref path) => {
                     match self.tcx.expect_def(cur_ty.id) {
                         Def::Enum(did) | Def::TyAlias(did) | Def::Struct(did) => {
-                            let generics = self.tcx.lookup_item_type(did).generics;
+                            let generics = self.tcx.lookup_generics(did);
 
                             let expected =
-                                generics.regions.len(subst::TypeSpace) as u32;
+                                generics.regions.len() as u32;
                             let lifetimes =
                                 path.segments.last().unwrap().parameters.lifetimes();
                             let mut insert = Vec::new();

src/librustc/infer/mod.rs

@@ -25,9 +25,7 @@ use middle::mem_categorization as mc;
 use middle::mem_categorization::McResult;
 use middle::region::CodeExtent;
 use mir::tcx::LvalueTy;
-use ty::subst;
-use ty::subst::Substs;
-use ty::subst::Subst;
+use ty::subst::{Subst, Substs};
 use ty::adjustment;
 use ty::{TyVid, IntVid, FloatVid};
 use ty::{self, Ty, TyCtxt};
@@ -1172,15 +1170,6 @@ impl<'a, 'gcx, 'tcx> InferCtxt<'a, 'gcx, 'tcx> {
         self.tcx.mk_var(self.next_ty_var_id(false))
     }
 
-    pub fn next_ty_var_with_default(&self,
-                                    default: Option<type_variable::Default<'tcx>>) -> Ty<'tcx> {
-        let ty_var_id = self.type_variables
-                            .borrow_mut()
-                            .new_var(false, default);
-
-        self.tcx.mk_var(ty_var_id)
-    }
-
     pub fn next_diverging_ty_var(&self) -> Ty<'tcx> {
         self.tcx.mk_var(self.next_ty_var_id(true))
     }
@@ -1205,89 +1194,55 @@ impl<'a, 'gcx, 'tcx> InferCtxt<'a, 'gcx, 'tcx> {
         ty::ReVar(self.region_vars.new_region_var(origin))
     }
 
-    pub fn region_vars_for_defs(&self,
-                                span: Span,
-                                defs: &[ty::RegionParameterDef])
-                                -> Vec<ty::Region> {
-        defs.iter()
-            .map(|d| self.next_region_var(EarlyBoundRegion(span, d.name)))
-            .collect()
-    }
-
-    // We have to take `&mut Substs` in order to provide the correct substitutions for defaults
-    // along the way, for this reason we don't return them.
-    pub fn type_vars_for_defs(&self,
+    /// Create a region inference variable for the given
+    /// region parameter definition.
+    pub fn region_var_for_def(&self,
                               span: Span,
-                              space: subst::ParamSpace,
-                              substs: &mut Substs<'tcx>,
-                              defs: &[ty::TypeParameterDef<'tcx>]) {
-
-        for def in defs.iter() {
-            let default = def.default.map(|default| {
-                type_variable::Default {
-                    ty: default.subst_spanned(self.tcx, substs, Some(span)),
-                    origin_span: span,
-                    def_id: def.default_def_id
-                }
-            });
-
-            let ty_var = self.next_ty_var_with_default(default);
-            substs.types.push(space, ty_var);
-        }
-    }
-
-    /// Given a set of generics defined on a type or impl, returns a substitution mapping each
-    /// type/region parameter to a fresh inference variable.
-    pub fn fresh_substs_for_generics(&self,
-                                     span: Span,
-                                     generics: &ty::Generics<'tcx>)
-                                     -> &'tcx subst::Substs<'tcx>
-    {
-        let type_params = subst::VecPerParamSpace::empty();
-
-        let region_params =
-            generics.regions.map(
-                |d| self.next_region_var(EarlyBoundRegion(span, d.name)));
+                              def: &ty::RegionParameterDef)
+                              -> ty::Region {
+        self.next_region_var(EarlyBoundRegion(span, def.name))
+    }
+
+    /// Create a type inference variable for the given
+    /// type parameter definition. The substitutions are
+    /// for actual parameters that may be referred to by
+    /// the default of this type parameter, if it exists.
+    /// E.g. `struct Foo<A, B, C = (A, B)>(...);` when
+    /// used in a path such as `Foo::<T, U>::new()` will
+    /// use an inference variable for `C` with `[T, U]`
+    /// as the substitutions for the default, `(T, U)`.
+    pub fn type_var_for_def(&self,
+                            span: Span,
+                            def: &ty::TypeParameterDef<'tcx>,
+                            substs: &Substs<'tcx>)
+                            -> Ty<'tcx> {
+        let default = def.default.map(|default| {
+            type_variable::Default {
+                ty: default.subst_spanned(self.tcx, substs, Some(span)),
+                origin_span: span,
+                def_id: def.default_def_id
+            }
+        });
 
-        let mut substs = subst::Substs::new(type_params, region_params);
 
-        for space in subst::ParamSpace::all().iter() {
-            self.type_vars_for_defs(
-                span,
-                *space,
-                &mut substs,
-                generics.types.get_slice(*space));
-        }
+        let ty_var_id = self.type_variables
+                            .borrow_mut()
+                            .new_var(false, default);
 
-        self.tcx.mk_substs(substs)
+        self.tcx.mk_var(ty_var_id)
     }
 
-    /// Given a set of generics defined on a trait, returns a substitution mapping each output
-    /// type/region parameter to a fresh inference variable, and mapping the self type to
-    /// `self_ty`.
-    pub fn fresh_substs_for_trait(&self,
-                                  span: Span,
-                                  generics: &ty::Generics<'tcx>,
-                                  self_ty: Ty<'tcx>)
-                                  -> subst::Substs<'tcx>
-    {
-
-        assert!(generics.types.len(subst::SelfSpace) == 1);
-        assert!(generics.types.len(subst::FnSpace) == 0);
-        assert!(generics.regions.len(subst::SelfSpace) == 0);
-        assert!(generics.regions.len(subst::FnSpace) == 0);
-
-        let type_params = Vec::new();
-
-        let region_param_defs = generics.regions.get_slice(subst::TypeSpace);
-        let regions = self.region_vars_for_defs(span, region_param_defs);
-
-        let mut substs = subst::Substs::new_trait(type_params, regions, self_ty);
-
-        let type_parameter_defs = generics.types.get_slice(subst::TypeSpace);
-        self.type_vars_for_defs(span, subst::TypeSpace, &mut substs, type_parameter_defs);
-
-        return substs;
+    /// Given a set of generics defined on a type or impl, returns a substitution mapping each
+    /// type/region parameter to a fresh inference variable.
+    pub fn fresh_substs_for_item(&self,
+                                 span: Span,
+                                 def_id: DefId)
+                                 -> &'tcx Substs<'tcx> {
+        Substs::for_item(self.tcx, def_id, |def, _| {
+            self.region_var_for_def(span, def)
+        }, |def, substs| {
+            self.type_var_for_def(span, def, substs)
+        })
     }
 
     pub fn fresh_bound_region(&self, debruijn: ty::DebruijnIndex) -> ty::Region {

src/librustc/middle/cstore.rs

@@ -154,14 +154,16 @@ pub trait CrateStore<'tcx> {
     fn item_variances(&self, def: DefId) -> ty::ItemVariances;
     fn repr_attrs(&self, def: DefId) -> Vec<attr::ReprAttr>;
     fn item_type<'a>(&self, tcx: TyCtxt<'a, 'tcx, 'tcx>, def: DefId)
-                     -> ty::TypeScheme<'tcx>;
+                     -> Ty<'tcx>;
     fn visible_parent_map<'a>(&'a self) -> ::std::cell::RefMut<'a, DefIdMap<DefId>>;
     fn item_name(&self, def: DefId) -> ast::Name;
     fn opt_item_name(&self, def: DefId) -> Option<ast::Name>;
     fn item_predicates<'a>(&self, tcx: TyCtxt<'a, 'tcx, 'tcx>, def: DefId)
                            -> ty::GenericPredicates<'tcx>;
     fn item_super_predicates<'a>(&self, tcx: TyCtxt<'a, 'tcx, 'tcx>, def: DefId)
                                  -> ty::GenericPredicates<'tcx>;
+    fn item_generics<'a>(&self, tcx: TyCtxt<'a, 'tcx, 'tcx>, def: DefId)
+                         -> &'tcx ty::Generics<'tcx>;
     fn item_attrs(&self, def_id: DefId) -> Vec<ast::Attribute>;
     fn trait_def<'a>(&self, tcx: TyCtxt<'a, 'tcx, 'tcx>, def: DefId)-> ty::TraitDef<'tcx>;
     fn adt_def<'a>(&self, tcx: TyCtxt<'a, 'tcx, 'tcx>, def: DefId) -> ty::AdtDefMaster<'tcx>;
@@ -187,8 +189,7 @@ pub trait CrateStore<'tcx> {
     fn impl_parent(&self, impl_def_id: DefId) -> Option<DefId>;
 
     // trait/impl-item info
-    fn trait_of_item<'a>(&self, tcx: TyCtxt<'a, 'tcx, 'tcx>, def_id: DefId)
-                         -> Option<DefId>;
+    fn trait_of_item(&self, def_id: DefId) -> Option<DefId>;
     fn impl_or_trait_item<'a>(&self, tcx: TyCtxt<'a, 'tcx, 'tcx>, def: DefId)
                               -> Option<ty::ImplOrTraitItem<'tcx>>;
 
@@ -334,7 +335,7 @@ impl<'tcx> CrateStore<'tcx> for DummyCrateStore {
     fn item_variances(&self, def: DefId) -> ty::ItemVariances { bug!("item_variances") }
     fn repr_attrs(&self, def: DefId) -> Vec<attr::ReprAttr> { bug!("repr_attrs") }
     fn item_type<'a>(&self, tcx: TyCtxt<'a, 'tcx, 'tcx>, def: DefId)
-                     -> ty::TypeScheme<'tcx> { bug!("item_type") }
+                     -> Ty<'tcx> { bug!("item_type") }
     fn visible_parent_map<'a>(&'a self) -> ::std::cell::RefMut<'a, DefIdMap<DefId>> {
         bug!("visible_parent_map")
     }
@@ -344,6 +345,8 @@ impl<'tcx> CrateStore<'tcx> for DummyCrateStore {
                            -> ty::GenericPredicates<'tcx> { bug!("item_predicates") }
     fn item_super_predicates<'a>(&self, tcx: TyCtxt<'a, 'tcx, 'tcx>, def: DefId)
                                  -> ty::GenericPredicates<'tcx> { bug!("item_super_predicates") }
+    fn item_generics<'a>(&self, tcx: TyCtxt<'a, 'tcx, 'tcx>, def: DefId)
+                         -> &'tcx ty::Generics<'tcx> { bug!("item_generics") }
     fn item_attrs(&self, def_id: DefId) -> Vec<ast::Attribute> { bug!("item_attrs") }
     fn trait_def<'a>(&self, tcx: TyCtxt<'a, 'tcx, 'tcx>, def: DefId)-> ty::TraitDef<'tcx>
         { bug!("trait_def") }
@@ -379,8 +382,7 @@ impl<'tcx> CrateStore<'tcx> for DummyCrateStore {
     fn impl_parent(&self, def: DefId) -> Option<DefId> { bug!("impl_parent") }
 
     // trait/impl-item info
-    fn trait_of_item<'a>(&self, tcx: TyCtxt<'a, 'tcx, 'tcx>, def_id: DefId)
-                         -> Option<DefId> { bug!("trait_of_item") }
+    fn trait_of_item(&self, def_id: DefId) -> Option<DefId> { bug!("trait_of_item") }
     fn impl_or_trait_item<'a>(&self, tcx: TyCtxt<'a, 'tcx, 'tcx>, def: DefId)
                               -> Option<ty::ImplOrTraitItem<'tcx>> { bug!("impl_or_trait_item") }
 
@@ -583,7 +585,7 @@ pub mod tls {
     pub trait DecodingContext<'tcx> {
         fn tcx<'a>(&'a self) -> TyCtxt<'a, 'tcx, 'tcx>;
         fn decode_ty(&self, decoder: &mut OpaqueDecoder) -> ty::Ty<'tcx>;
-        fn decode_substs(&self, decoder: &mut OpaqueDecoder) -> Substs<'tcx>;
+        fn decode_substs(&self, decoder: &mut OpaqueDecoder) -> &'tcx Substs<'tcx>;
         fn translate_def_id(&self, def_id: DefId) -> DefId;
     }
 

src/librustc/middle/dead.rs

@@ -88,15 +88,24 @@ impl<'a, 'tcx> MarkSymbolVisitor<'a, 'tcx> {
     fn lookup_and_handle_definition(&mut self, id: ast::NodeId) {
         use ty::TypeVariants::{TyEnum, TyStruct};
 
+        let def = self.tcx.expect_def(id);
+
         // If `bar` is a trait item, make sure to mark Foo as alive in `Foo::bar`
-        self.tcx.tables.borrow().item_substs.get(&id)
-            .and_then(|substs| substs.substs.self_ty())
-            .map(|ty| match ty.sty {
-                TyEnum(tyid, _) | TyStruct(tyid, _) => self.check_def_id(tyid.did),
-                _ => (),
-            });
+        match def {
+            Def::AssociatedTy(..) | Def::Method(_) | Def::AssociatedConst(_)
+            if self.tcx.trait_of_item(def.def_id()).is_some() => {
+                if let Some(substs) = self.tcx.tables.borrow().item_substs.get(&id) {
+                    match substs.substs.types[0].sty {
+                        TyEnum(tyid, _) | TyStruct(tyid, _) => {
+                            self.check_def_id(tyid.did)
+                        }
+                        _ => {}
+                    }
+                }
+            }
+            _ => {}
+        }
 
-        let def = self.tcx.expect_def(id);
         match def {
             Def::Const(_) | Def::AssociatedConst(..) => {
                 self.check_def_id(def.def_id());