Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add File Service Discovery #492

Closed
bwplotka opened this issue Aug 28, 2018 · 10 comments
Closed

Add File Service Discovery #492

bwplotka opened this issue Aug 28, 2018 · 10 comments

Comments

@bwplotka
Copy link
Member

bwplotka commented Aug 28, 2018

Rationales
Similarly to Prometheus File SD our idea for File SD will allow to update hosts for various purposes (StoreAPIs for Querier, Queriers for Ruler) to be discovered on runtime. Additionally it will allow to build custom integrations that will fill those in run time.

Iinitially I thought we could just use Prometheus package but it is quite tied to targets, so no longer sure. Sticking to same library will really help us to not maintain it and have proven solution that works with ANY integration that is already build into that. We should consider this.

Acceptance Criteria

@bwplotka
Copy link
Member Author

This is related to #484

(in fact we want to combine DNS SD as well here)

@bwplotka
Copy link
Member Author

We need to also make sure this is #452 within the scope.

@yuecong
Copy link
Contributor

yuecong commented Aug 28, 2018

could we also support https as prometheus's tls_config and basic_auth? With these, I think we can even config thanos cluster discovery over the internet.

@xjewer
Copy link
Contributor

xjewer commented Aug 28, 2018

We need to also make sure this is #452 within the scope.

removing gossip/memberlist solves this problem, as it's all about memberlist's resolver

@bwplotka
Copy link
Member Author

@xjewer Ah good point. But we will introduce another place for DNS resolution, so still it would be nice to remember about /etc/hosts (:

@yuecong I don't get this. How this is connected to File SD?

@xjewer
Copy link
Contributor

xjewer commented Aug 29, 2018

so still it would be nice to remember about /etc/hosts (:

@Bplotka afaik, the golang unix resolver lookup Resolver.LookupHost respects a /etc/nsswitch.conf file and an order of lookup accordingly

@Ehekatl
Copy link

Ehekatl commented Sep 21, 2018

DNS based sd would be more than enough, I'm using headless service to publish thanos peer endpoint within k8s, even without k8s, manage several dns records by hand is not much of work.

It would be better if you make --cluster.peers= compatible with dns sd,
so my configuration--cluster.peers=peers.thanos.svc.cluster.local works the same after remove gossip.

@xjewer
Copy link
Contributor

xjewer commented Sep 21, 2018

Is anyone who works on this issue or has plan doing it?

@ivan-valkov
Copy link

Hey, I am working on file sd right now. I am working on getting a basic version of file sd running without dns resolution. When this is done, I plan on implementing a dns resolver that can be used in the file service discovery.

@bwplotka
Copy link
Member Author

This was being merged! 🎉

Thanks @ivan-valkov for this!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants